On June 29, Alibaba Cloud officially announced the open source MongoShake, a MongoDB-based disaster tolerance and multi-active IDCs/regions solution. MongoShake can synchronize data between data centers with a maximum QPS of 500,000. It solves the inflexible disaster tolerance and single-point write-in problems, implements disaster tolerance across data centers and remote multi-active IDCs/regions, and significantly reduces the O&M workload.
Mongo-Shake is a universal platform for services based on MongoDB's oplog. It fetches oplog from source mongo database, and replays in the target mongo database or sends to other ends in different tunnels. If the target side is mongo database, which means replay oplog directly, it's like a syncing tool that is used to copy data from source MongoDB to another MongoDB to build redundant replication or active-active replication. In addition to this direct way, there are others tunnel types such like rpc, file, tcp, and kafka. Receivers written by users must define their own interfaces to connect to these tunnels respectively. Users can also define their own tunnel type, which is pluggable. When connecting to a third-party message middleware like kafka, the consumer can get the subscriber data in an asynchronous way in pub/sub module flexibly.
MongoShake has shined in various major sales promotions. Take the sales promotion held by Amap on China's National Day as an example. MongoShake has met Amap's requirements for disaster tolerance and high availability of multiple data centers and store all data on each of the three data centers. This relieves the pressure of testing and O&M personnel, meets the flexible service switching requirements, and ensures stable service operations under the peak traffic on the National Day.
Meanwhile, as a general platform service, MongoShake also flexibly interconnects closed-loop nodes to adapt to different scenarios, such as log subscription, data center synchronization, and monitoring and auditing. By making MongoShake open source, we hope to enable MongoDB to play a bigger role in the database field.
In the current database ecosystem, most systems support data synchronization between multiple node instances, such as MySQL master/slave synchronization and Redis AOF master/slave synchronization. MongoDB even supports replica set synchronization of three nodes or more.
Therefore, data synchronization across units and data centers is especially important at the business layer. To eliminate the current constraints imposed by MongoDB on data synchronization across data centers, Alibaba Cloud develops the MongoShake solution that can be used for inter-instance synchronization or data synchronization across data centers, meeting the disaster tolerance and multi-active IDCs/regions requirements.
To further improve the transmission performance, Alibaba Cloud adopts multiple ideas of design, such as multi-end pulling, parallel execution, conflict detection, and asynchronous optimization, to meet developers' performance requirements. MongoShake also supports DML and DDL synchronization of the database, which ensures data consistency at the bottom layer.
MongoShake's open source code provides an excellent solution for developers who currently use the MongoDB database. For more information about the details and design of MongoShake as well as the official open source website, visit the following websites:
Interactive Viewing Experience for the World Cup Using Redis
How ApsaraDB for Redis Supports for the November 11 shopping spree
Alibaba Cloud Storage - April 3, 2019
Michelle - July 10, 2018
Alibaba Cloud Community - October 9, 2022
Alibaba Cloud Native Community - June 29, 2022
ApsaraDB - November 12, 2024
Alibaba Cloud Community - December 23, 2021
Backup and Archive Solution
Alibaba Cloud provides products and services to help you properly plan and execute data backup, massive data archiving, and storage-level disaster recovery.
Learn More
Database Backup
A reliable, cost-efficient backup service for continuous data protection.
Learn More
ApsaraDB for MongoDB
A secure, reliable, and elastically scalable cloud database service for automatic monitoring, backup, and recovery by time point
Learn More
Cloud Backup
Cloud Backup is an easy-to-use and cost-effective online data management service.
Learn MoreMore Posts by ApsaraDB
