PolarDB:Three data centers across two regions

Weighted election mechanism

PolarDB-X introduces the weighted election mechanism into the three data centers across two region architecture:

• Optimistic weighted election mechanism: In the Paxos protocol-based election, nodes that are more likely to initiate elections tend to become leaders. To optimize the leader election process, the optimistic weighted election mechanism assigns weights to nodes to influence the likelihood of nodes to be elected as the leader. The mechanism imposes a calculated delay for initiating a leader election on the nodes, which is inversely proportional to the assigned weights of the nodes. As a result, nodes that have higher weights can first initiate a leader election.

• Mandatory weighted election mechanism: When a new leader node finds that it is not the node that has the highest weight among all nodes (not applicable to the scenario in which the weights of all nodes are the same), it does not immediately allow write operations. Instead, it enters an election timeout period known as the abdication phase. During the abdication phase, the new leader node periodically (for example, every 1 to 2 seconds) sends heartbeat signals to the other nodes. If the new leader node receives a response from another node that has a higher weight before the abdication phase ends, it triggers a leadership transfer towards the node that has the highest weight.

Election weights of replicas in the PolarDB-X three data centers across two regions architecture:

If the leader replica in Primary Data Center 1 fails, a leader election process is triggered. A follower replica in the same data center that has an election weight of 7 is prioritized for selection as the new leader. This arrangement is consistent with the policy that favors the selection of a node in the same data center.

Dynamic adjustment of replica quantities

In the three data centers across two regions architecture that uses a five-replica majority consensus replication group, a synchronized response from at least three replicas is required to achieve data consistency. The four replicas in the primary data center can quickly reach a majority synchronization, often within approximately 1 millisecond, because of low network latency due to the geographical proximity. However, if a failure occurs in a primary data center and only three replicas remain, the system must wait for the replica in the secondary data center to respond. As a result, the network latency for majority-based synchronization increases by 30 milliseconds. In industries in which the three data centers across two regions architecture is commonly used, such as the financial industry, the network latency between the primary centers and the secondary data center is approximately 30 milliseconds.

Common replica quantity adjustment scenarios:

• Five replicas decreased to three replicas. Run the downgrade_follower command to dynamically change the roles of two replicas from the follower role to the learner role.

• Three replicas increased to five replicas. Run the upgrade_learner command to upgrade the roles of two replicas from the learner role to the follower role. Before you upgrade the roles of the two replicas, make sure that the asynchronous replication logs of the replicas are up-to-date.

• One replica increased to three replicas. Run the add_follower command to dynamically add new replicas to the system. The new replicas initially assume the learner role. After the logs of the new replicas are updated with the current data, the replicas are automatically upgraded to the follower role.

Forced start of a single replica

When the primary data centers fail in the three data centers across two regions architecture, the remaining replica in the secondary data center cannot provide data services because the replica does not meet the requirements of the majority consensus protocol.

PolarDB-X addresses this issue by providing a capability known as "forced start of a single replica." By running the force_single_mode command, the system can forcibly transition into a mode in which the system operates by using a single replica, which effectively sidelines all follower replicas. Once the primary data centers recover from failure and restore normal conditions, PolarDB-X uses a dynamic replica quality adjustment capability to gradually rebuild the distributed system. It incrementally increases the number of replicas from 1 back to 3 and then to 5. As a result, full data services are restored and system redundancy is ensured.

Remote secondary instance

The finance industry has specific requirements for geo-discovery metrics: recovery time objective (RTO) and RPO. The following table describes the requirements.

To meet the RTO requirements of geo-disaster recovery, PolarDB-X uses the three data centers across two regions architecture that uses five replicas based on the Paxos majority consensus protocol and incorporates primary and secondary instances. In the event of a complete failure in the primary data centers, business operations can be swiftly redirected to the remote secondary instance for recovery.

Key design points of the three data centers across two regions architecture that incorporates primary and secondary instances:

• The primary instance of PolarDB-X uses the cross-region Paxos replication protocol that requires synchronized responses from at least three replicas. By default, the four replicas of the primary data center are able to quickly complete the majority of synchronized responses due to low network latency. In most cases, the remote replica of the primary instance respond asynchronously. As a result, the synchronization process of the primary instance remains unaffected by the network latency of the remote replica.

• In most cases, the secondary instance of PolarDB-X is deployed in a remote location. PolarDB-X uses the Change Data Capture (CDC) log node to enable quasi-real-time replication between primary and secondary instances across different regions. Data replication latency may occur in the remote secondary instance. You must ensure atomic replication in distributed transaction scenarios to prevent transaction inconsistency.

• In scenarios that involve the three data centers across two regions architecture, the replication progress to the remote replicas that belong to different Paxos groups may be different in a distributed transaction. In this case, to replicate data from the primary instance to the secondary instance, you must deploy CDC log nodes in the remote region to sort and reorganize distributed transactions and ensure the atomic replication of the transactions across instances. This method ensures that no transactions are partially committed during the data replication from the primary instance to the secondary instance. By using atomic transaction replication, you can ensure the integrity of the transaction data during routine disaster recovery drills and real-world failure scenarios when the business is transferred to the remote secondary instance.

Create a PolarDB-X instance

When you purchase a PolarDB-X instance, you can set the Topology parameter to Three Data Centers Across Two Regions.

View the topology of a PolarDB-X instance

Go to the Basic Information page of the instance. View the zone information of resources in the Topology Information section.

Perform a failover

1. Log on to the PolarDB for Xscale console.

2. In the top navigation bar, select the region where the target instance is located.

3. On the Instances page, click the PolarDB-X 2.0 tab.

4. Find the target instance and click its ID.

5. Go to the Basic Information page of the instance. In the upper-right corner of the Topology Information section, click Specify Primary Zone

6. In the Specify Primary Zone dialog box, configure the Data Center, Primary Zone, and Switch Mode parameters.

7. Click OK.