Enterprise level and domestic ARM adaptation

Introduction: PolarDB-X officially released version 2.2.0, which is an important milestone version. It focuses on the introduction of enterprise level and domestic ARM adaptation under the financial standard of distributed database, including eight core features, and comprehensively improves the universality of PolarDB-X distributed database in the financial, communications, government and other industries.

Introduction to architecture

PolarDB-X is designed with a shared nothing and storage separation computing architecture. The system consists of four core components.

• Compute node (CN)

The computing node is the entrance of the system, which adopts stateless design, including SQL parser, optimizer, executor and other modules. Be responsible for data distributed routing, computing and dynamic scheduling, 2PC coordination of distributed transactions, global secondary index maintenance, etc., and provide enterprise level features such as SQL flow restriction and separation of powers.

• Storage node (DN, Data Node)

The storage node is responsible for data persistence, providing high reliability and strong consistency of data based on Paxos protocol of majority, and maintaining the visibility of distributed transactions through MVCC.

• Metadata service (GMS, Global Meta Service)

The metadata service is responsible for maintaining globally strong consistent table/schema, statistics and other system meta information, maintaining security information such as accounts and permissions, and providing global time service (i.e. TSO).

• Log node (CDC, Change Data Capture)

The log node provides incremental subscription capability that is fully compatible with MySQL Binlog format and protocol, and master slave replication capability that is compatible with MySQL Replication protocol.

Version Description

To sort out PolarDB-X's open source context:

• In October 2021, at the Yunqi Conference, Alibaba Cloud officially opened the cloud native distributed database PolarDB-X, which uses a full core open source model. The open source content includes computing engine, storage engine, log engine, Kube, etc.

• In January 2022, PolarDB-X officially released version 2.0.0, the first version update since the cloud habitat conference was officially opened on October 20, 2021. The updated content includes new cluster expansion, binlog ecological compatibility and other features, compatibility with maxwell and debezium incremental log subscriptions, as well as adding many other new features and fixing several problems.

• In March 2022, PolarDB-X officially released version 2.1.0, which contains four core features, comprehensively improving the stability and ecological compatibility of PolarDB-X, including the Paxos based three copy consensus agreement.

• In May 2022, PolarDB-X officially released version 2.1.1, focusing on new features of hot and cold data, which can support the storage of business table data on different storage media according to data characteristics, such as the storage of cold data on Aliyun OSS object storage.

In September 2022, PolarDB-X database passed the distributed database financial standard verification with high scores. A total of 337 inspection items were verified, including architecture, operation and maintenance, security, disaster recovery, performance, etc. After the expert review, PolarDB-X determined 323 qualified test items, and the overall test results were excellent.

In October 2022, PolarDB-X officially released version 2.2.0, which is an important milestone version. It focuses on the introduction of enterprise level and domestic ARM adaptation under the financial standard of distributed database, including eight core features, and comprehensively improves the universality of PolarDB-X distributed database in the financial, communications, government and other industries.

01 Domestic ARM adaptation

At present, there is a strong demand for the adaptation of domestic servers in the market. The common demand is to be compatible with the CPU ARM architecture. In addition to the normal operation of the database on the ARM architecture, it is also necessary to optimize the performance of the database in combination with the domestic ARM architecture.

PolarDB-X V2.2.0 starts. At the same time, the binary version compatible with the X86/ARM architecture is released. In addition, the supporting database deployment tool will also support the numa core binding capability under the ARM architecture to improve the performance of the database.


02 New read write separation architecture



In MySQL ecosystem, read/write separation is a common technology. In addition to solving the optimization of multiple scenarios of less write and less read, it is also often used in typical scenarios of OLTP/OLAP business isolation, such as considering the stability of online databases, sending individual complex queries to MySQL standby databases through read/write separation.

Existing problems:

1. Additional Proxy components or business code routing are required to realize controllable read/write separation routing

2. There is a data consistency problem in the standby database reading. For example, request A to complete writing in the primary database, and then immediately go to the standby database to read the data. Because of the replication delay, the data just written may not be read

3. It requires additional OLAP column data+external replication synchronization (such as canal like view) to support more complex report queries

The read/write separation architecture of PolarDB-X:

1: The computing node (CN) acts as a routing component with separate read and write, without introducing additional components. Multiple CN nodes need to be configured with a load balancing device (such as LVS)

2: It provides global consistent read. It routes requests to the standby database in the read/write separation, ensures the consistency of data read after business write, and provides easier read/write separation capabilities

3: Provide HTAP row and column mixed storage architecture, multiple data+one set of SQL engine, and provide HTAP mixed load capacity

Strong consistent read/write separation

The distributed database naturally has the ability of multiple copies. PolarDB-X adopts Paxos consensus protocol. With the help of the Paxos protocol's log stream LogIndex (a globally increasing unique sequence that records Paxos log subscripts), PolarDB-X can achieve consistent read across multiple copies based on the LogIndex, achieving the effect of separating read from write.

On the basis of the traditional MySQL read/write separation architecture, PolarDB-X introduces the Paxos Learner node as a read-only RO node (it does not participate in the Paxos three replica voting, and only replicates data asynchronously. Running full of the RO node CPU will not affect the writing of the three replica majority). In the read/write separation mode, PolarDB-X will give priority to obtaining the LogIndex of the master database for the traffic routed to the read-only RO node to ensure that the LogIndex of the RO replica exceeds this value. At the same time, the TSO timestamp version of the distributed transaction MVCC can be used to achieve strong consistent read/write separation under the RC/RR isolation level.

From the test results:

1. In the case of strong consistent reading, the throughput performance of the OLTP reading scenario is reduced by 20~30% when the traffic is switched from the primary instance to the read-only instance. However, by adding the number of read-only instances, the performance can be increased linearly;

2. In the case of weakly consistent reads, the throughput performance on the TP read scenario is not degraded when the traffic is switched from the primary instance to the read-only instance, and the performance can be increased linearly by adding the number of read-only instances;

HTAP mixed load

The core goal of HTAP architecture is to help users reduce costs: operation and maintenance costs, and use costs. For example, the biggest challenge of the traditional OLTP+ETL+OLAP solution is the complexity and stability of the operation and maintenance costs. The integrated architecture of the HTAP database can effectively reduce the external operation and maintenance costs.

At present, HTAP architectures on the market have various forms. Here are three key elements of core technology:

• MPP parallel computing improves the linear scalability of data analysis scenarios based on the parallel capability, which is the basic architecture requirement

• Resource isolation to ensure that data analysis and query do not affect online business. Common single process logical isolation has limitations, and physical multi copy isolation is recommended

• Column storage of replicas reduces the cost of data analysis and query resources. The compact coding of column storage is very suitable for AP queries, with better cost performance

PolarDB-X provides the following capabilities in open-source V2.2.0:

1. Resource isolation, based on Paxos multi copy+read/write separation, realizes the physical isolation of online business and data analysis business

2. MPP parallel computing, combined with the read-write separation architecture, adopts the MPP parallel query plan by default for requests routed to read-only nodes to improve query performance

3. Column storage capacity. At present, PolarDB-X has introduced chunk-at-a-time memory column structure in the SQL engine, which effectively improves the efficiency of single core queries. However, there is the overhead of dynamically converting the row structure of physical data to column storage, which has some room for optimization. In the future, PolarDB-X is developing a column storage engine, which is expected to be officially open source in the middle of 2023, forming a unified architecture of mixed column storage.

Even if there is resource isolation in HTAP architecture, if data analysis and query go to the main database during business use, the performance of online business will be affected. Therefore, it is also very important for business requests to correctly use multiple replicas.

From the perspective of ease of use and stability, PolarDB-X provides intelligent read/write separation based on optimizer cost. The PolarDB-X optimizer will analyze the query physical scan lines, CPU, memory, IO, network and other core resource consumption based on the cost, and divide the request into TP and AP loads. If intelligent routing is enabled on the cluster address, it will actively identify the SQL workload type for routing, such as routing the traffic identified as AP load to the read-only RO replica. You can view the recognition of SQL workload types through the explain cost command. For example, the following query involves a very small rowcount of physical scan lines and a low consumption of CPU&Memory. Therefore, this query is identified as a TP load.

Take the TPCH Q13 as an example to demonstrate the acceleration effect of the actuator in different scenarios. To facilitate screenshots, limits are added after Q13

Corresponding CN/DN node specification: 2 * 16C64G

Running under single machine and single thread, taking 3min31s

03 MySQL Ecological Compatibility

In addition to the basic SQL DML/DDL/DAL, MySQL also has many advanced features, such as lock system, binlog protocol, active/standby replication, stored procedures, triggers, foreign keys, views, etc.

Traditional distributed middleware or distributed databases boast high MySQL compatibility, mostly in SQL functional syntax, such as DML/DDL/DAL. However, there are still many differences in performance compatibility and ecological compatibility, resulting in complexity in use.

Here are some examples:

1. The transaction isolation level of MySQL is generally RC/RR, which has its unique features in the transaction model. For example, MySQL update v=v+1 is a typical pessimistic transaction model, which is different from the optimistic transaction model of Google Percolator. In addition, MySQL's common use of select for update is a typical GAP lock scenario under B+Ttree. Currently, NewSQL based on LSM Tree architecture is almost incompatible with this feature.

2. As a typical design of the database CDC (change data capture) architecture, MySQL binlog component is also an important capability to solve the compatibility of downstream ecosystems. Many NewSQL still provide some CDC capabilities from external data synchronization components, but it is not compatible with the binlog protocol.

PolarDB-X has well supported MySQL isolation level and MySQL binlog ecological compatibility in the open source version in March 22. This open source version of V2.2.0 focuses on strengthening several new compatibility capabilities:

stored procedure

Stored Procedure is a set of SQL statements to complete specific functions. When businesses need to write a set of complex SQL statements to meet certain requirements, many senior database users are used to using stored procedures.

Using stored procedures can bring the following benefits:

• High reusability. Stored procedures can be reused to reduce the workload of database developers and reduce the probability of business errors.

• High efficiency. After the stored procedure is compiled once, it will be saved to the database and executed directly each time it is called.

• Reduce network traffic. The compiled stored procedure will be placed in the database. When we call it remotely, we will not transmit a large number of SQL statements of string type.

• High security. Stored procedures that perform a specific function are generally only available to specific users, with identity restrictions, and are more secure.

Of course, stored procedures also have some disadvantages:

• Poor cross platform portability

• Complex ETL brings more resource overhead, such as memory/CPU, etc

• Easy to generate large and long transactions

PolarDB-X is based on GMS storage, fully compatible with MySQL stored procedure syntax, and supports management operations such as creation, modification, and deletion of stored procedures. At the same time, the PL Engine engine is introduced to support the running of stored procedures and the running memory management. It supports GB level large transactions and 2-hour long transactions.


04 Database security

In IT circles, "deleting databases and running away" has become a joke frequently mentioned by programmers. Although it is a joke, it reflects the importance of the database to the enterprise. In order to improve the database security, in addition to network isolation and permission management, it is also necessary to do a good job of auditing in the whole link to ensure that there is a good comparison. At the same time, in the case of data loss, the ability of emergency recovery should be improved.

Full SQL audit

The traditional database audit generally adopts the network bypass mode. All network traffic is collected through external components, from which the corresponding SQL audit is collected. In addition, an audit query entry is provided for fast retrieval.

PolarDB-X V2.2.0 can quickly and completely record all SQL logs through the built-in SQL audit capability of the database. PolarDB-X uses the Logstash component as the log parsing and reporting component. By default, it collects full SQL text in quasi real time. Through the extension of the logstash delivery component, for example, it can interface with the ElasticSearch component or user-defined extension component to achieve SQL audit persistent storage and white screen SQL audit query.