PolarDB-X and X-DB, PolarDB

PolarDB-X, X-DB and PolarDB are Alibaba's database products. What kind of relationship do they have?

To answer this question, we must first understand what X-DB is.

What is X-DB?

In short, X-DB mainly refers to a distributed cross AZ highly available database built on the basis of MySQL and the X Engine engine.

One of the core capabilities of X-DB is that the RPO across AZ is 0 based on Paxos. Initially, the MySQL used by Alibaba is a traditional master/slave architecture. Each MySQL instance consists of two nodes.

This architecture has an important defect. If the active/standby switch occurs, the replica data may be inconsistent. In order to solve the problem of the active/standby architecture, Alibaba initially used a lot of operation and maintenance methods to solve this problem, such as:

A system called ADHA is specially used for MySQL service activation, active/standby switching, etc

Before the active/standby switchover, perform some complex data verification to ensure that the active/standby data are consistent before the switchover (so how to switch in case of inconsistency?)

In the cross city scenario, the self-developed data synchronization service is used to replace the internal synchronization mechanism of MySQL

The rollback and supplement of business data, using a very complex data correction program (related to the business, and there are certain requirements for the business table structure)

Although the architecture works, it is not elegant. Its complexity and use cost are very high, and it is difficult to keep up with the development of the business.

In 2016, Alibaba began to develop X-DB internally. X-DB is first to overcome the problem of replica data consistency. The X-DB team chose to develop the Paxos protocol to replace MySQL's built-in active/standby synchronization logic. There are many articles about Paxos, so I won't repeat them here. Interested students can search them by themselves.

Here are two core advantages of MySQL after using the Paxos protocol: 1. The master switch operation can be performed at any time without any data consistency problem, RPO=0. 2. Leader election, exploration, etc. are completely completed in the X-DB kernel (MySQL process), without relying on any external system

After several years of development in Alibaba, X-DB's Paxos protocol library, combined with MySQL, has completely replaced the traditional active and standby MySQL clusters with high reliability.

PolarDB-X and X-DB

It can be seen from the above that X-DB has solved problems such as disaster recovery and fault recovery well, but PolarDB-X's strengths include the adaptability of service scenarios and horizontal expansion.

PolarDB-X 2.0's data node (DN) integrates X-DB's cluster disaster recovery technology and provides horizontal expansion capability.

In fact, the relationship between X-DB and PolarDB-X is far beyond Paxos protocol. A lot of development work has been done in distributed transactions, scalability, computing power push down, HTAP and other places. Please pay attention to our subsequent articles.

PolarDB-X and PolarDB

PolarDB (here PolarDB For MySQL) is a cloud native database based on shared storage technology. PolarDB can achieve strong flexibility in storage space, but in general use, its computing capacity and write capacity still have the upper limit of a single machine.

PolarDB-X 2.0, a share nothing architecture, enables all resources, including computing, writing, reading, and storage, to be horizontally scalable, so there is no bottleneck limit for a single machine.

However, the share nothing architecture is not as flexible as PolarDB's shared storage architecture in terms of simple data capacity.

For example, no matter what kind of share nothing database is used, such as PolarDB-X 2.0 and TiDB, if there are 10 machines and 10T of data, and you want to expand the storage space by adding machines, half of the data will always need to be moved to a new machine, It will be positively related to the data volume (for example, in general, it will take at least several hours to move dozens of terabytes of data).

For PolarDB, the above scenario can be expanded in minutes.

So, is it possible to combine the advantages of share nothing with the advantages of shared storage/share everything?

The answer is yes.

At present, the next generation of distributed databases will introduce PolarDB's core technology based on RDMA hardware+shared storage architecture in DN, so that all resources can be expanded horizontally and the cost of capacity elasticity can be greatly reduced.

PolarDB-X local disk version and shared storage version will coexist for a long time in the future, and there will be no problem of who completely replaces who. Because the local disk version does not rely on any special hardware, it is mainly more friendly to the lightweight output of the private cloud. The goal is to complete the delivery of PolarDB-X based on the user's three machines; The capacity of the shared storage version is highly elastic, but this elasticity can only be well reflected in the public cloud of a certain scale (only on the public cloud can there be a large enough machine pool for business de elasticity).