Uqbar 2.0.0

About Uqbar

Uqbar is a hyper-converged time-series database built on the openGauss open-source database, designed specifically for IoT (Internet of Things) scenarios. As a professional time-series database solution, Uqbar integrates several advantages, including high performance, low cost, stability, and ease of use, aiming to meet the demands of large-scale time-series data management and efficient querying in IoT environments.

Building upon the robust OLTP (Online Transaction Processing) capabilities of openGauss, Uqbar further strengthens its ability to handle time series data and conducts in-depth optimization tailored to the characteristics of time-series data. Time-series data is at the core of IoT, typically requiring continuous high-concurrency writes and the ability to perform high-performance queries and analyses on massive datasets. Uqbar achieves this by deeply optimizing its underlying storage and query engine, ensuring exceptional performance and stability to handle massive streams of time-series data.

Uqbar's strength not only lies in its powerful time-series data management capabilities but also in its flexibility to handle traditional OLTP scenarios. It inherits the relational database features of openGauss, enabling users to manage both time-series and relational data on the same platform. This integrated design makes Uqbar a one-stop IoT data solution, providing users with comprehensive data management and query support.

Version Description

Uqbar 2.0.0 is the first Long-Term Support (LTS) version of the Uqbar time-series database. This version builds upon the Online Transaction Processing (OLTP) capabilities of openGauss while extending functionality and providing targeted optimizations for time-series data processing scenarios.

Basic Functions

Time-series Lifecycle Policies

Time-series lifecycle policies are used for managing the lifecycle of time-series data. By defining time-series lifecycle policies, you can specify data retention periods and time partition sizes. Time-series lifecycle policy management functions include: creating time-series policies, modifying time-series policies, deleting time-series policies, and setting the default time-series policy for the database.

Time-Series Table Management

The fundamental object for organizing and storing time-series data, similar to the definition of tables in a relational database. When creating a time-series table, it is necessary to specify the time series policy to be used. The data in the time-series table is automatically partitioned based on the time partitions defined in the time series policy, and the data is automatically aligned by time. Time-series table management functions include: creating time-series tables, modifying time-series tables, and deleting time-series tables.

Data Compression

Supports automatic and manual compression of stored time-series data. Uqbar uses a hybrid row-column storage engine to store time-series data, automatically converting row-stored data to column-stored data in the background while compressing the data. Data compression features include: automatic background compression and manual execution of compression commands.

Expired Data Deletion

Based on the time-series lifecycle policy, data that exceeds the retention period on the time-series table is automatically deleted according to the partition granularity. The expired data deletion feature includes: automatic background deletion and manual execution of deletion commands.

Continuous Aggregation Management

The continuous aggregation feature involves periodic queries on time-series data in the background, saving the query results for use in query optimization and downsampling of time-series data, among other scenarios. Continuous aggregation management includes: creating continuous aggregations, modifying continuous aggregations, and deleting continuous aggregations.

High Availability (HA)

This version adopts a primary-secondary architecture, supporting a maximum of one primary and seven standby instances. The primary instance provides read-write capabilities, while the standby instances provide query capabilities. The system supports automatic failover between the primary and standby instances in case of failure.

Backup and Recovery

Supports data backup and recovery. You can use backup and recovery tools to perform data backup and restoration.

Operations and Maintenance Tools

PTK Installation Tool and Graphical Management Tool. The PTK installation tool supports automated deployment and installation of the database. The graphical management tool, Manager, provides database monitoring, backup and recovery management, and alarm management.

Third-Party Tool Support

Kafka: Supports integration with the Kafka messaging component, actively consuming time-series data from Kafka.
Grafana: Supports integration with the visualization tool Grafana, allowing for the visualization of time-series data through Grafana.
Prometheus: Supports integration with Prometheus, serving as an external time-series data store for Prometheus, providing improved write and query performance.

Time-Series Scenario Optimization

Write Performance Optimization

Optimized time-series data write performance to accommodate the unique characteristics of time-series data. Supports continuous, high-pressure writing 24/7 and handles out-of-order time-series data writes. For performance data in different scenarios, please refer to the Performance Whitepaper.

Query Optimization

Through a vectorized query engine and parallel query capabilities, it provides high-performance query and analysis capabilities for massive data.

Supports inverted indexes and allows querying of time-series data using arbitrary label combinations.

Compression Rate Optimization

Separates labels from data storage to avoid redundant storage of label data.

Uses dedicated time-series compression algorithms based on data types and characteristics, achieving compression rates of over 10 times in typical scenarios.

Time-Series Query Operators

In addition to the aggregation operators already supported by relational tables, such as sum, avg, count, etc., it now includes support for commonly used aggregation operators in time-series scenarios:

First: The data with the smallest timestamp that meets the criteria in the dataset.
Last: The data with the largest timestamp that meets the criteria in the dataset.
Time_bucket: Aggregates data into groups based on a specified time window.
Time_bucket_gapfill: Aggregates data into groups based on a specified time window and fills in missing data according to specified rules.
Histogram: Generates histogram data for the data according to specified rules.

Time-series Views

timeseries_views.bgw_job: Background job view, used to query which background jobs are executing on time-series tables.
timeseries_views.compression_chunkgroup: Compression rate view, aggregated by chunk group, used to display the compression rates of time-series tables.
timeseries_views.compression_table: Compression rate view, aggregated by time-series table, used to display the compression rates of time-series tables.
timeseries_views.continuous_aggregation: Continuous aggregation view, used to view continuous aggregations on time-series tables.
timeseries_views.policies: Time-series policy view, used to view the time-series policies in the current database.
timeseries_views.tschunkgroup: Time-series table view, aggregated by chunk group, used to view partitions of time-series tables.
timeseries_views.tstable: Time-series table view, aggregated by time-series table, used to view the time-series tables in the current database.

Defect Fixes

Intermittent coredump issue during data expiration deletion.
High memory usage during data copying process, suspected memory leak issue.
Coredump issue during database reload after resetting guc parameter 'timeseries_compression_delay'.
Intermittent process termination when performing value queries on compressed data, such as querying the 'field' or 'time' columns.

Supported Platforms

Uqbar 2.0.0 supports the following combinations of operating systems and CPU architectures:

OS	CPU Architecture	Note
CentOS 7.x	X86_64 (Intel, AMD, Hygon, ZHAOXIN)
Redhat 7.x	X86_64 (Intel, AMD, Hygon, ZHAOXIN)
openEuler 20.03LTS	ARM (Kunpeng)、X86_64 (Intel, AMD, Hygon, ZHAOXIN)
Kylin V10	ARM (Kunpeng)、X86_64 (Intel, AMD, Hygon, ZHAOXIN)	OM is not compatible with openssh-8.2p1-9.p06 or above (including p06, p07, p08, p09) in the Kylin V10 sp2 upgrade patch for now.
UOS V20-D / V20-E	ARM (Kunpeng)、X86_64 (Intel, AMD, Hygon, ZHAOXIN)
UOS V20-A	X86_64 (Intel, AMD, Hygon, ZHAOXIN)