MySQL binary log (binlog) files record all operations that result in data changes. These operations include INSERT, UPDATE, DELETE, and DDL. MySQL implements key features such as fault recovery, primary/secondary replication, and backup and restoration based on binlogs. In addition, MySQL provides the open binlog dump protocol. Binlogs and this protocol provide an ideal tool to implement change data capture (CDC). Third-party applications can subscribe to and consume binlogs to implement various scenarios. Examples:
You can subscribe to and send binlogs to real-time computing platforms to support scenarios such as risk control and recommendation.
You can subscribe to and send binlogs to offline big data platforms to support scenarios such data warehousing and auditing.
You can subscribe to and send binlogs to search engines to support scenarios such as search and analysis.
You can subscribe to and send binlogs to function compute platforms to construct an event-driven application architecture.
You can subscribe to and send binlogs to cache platforms to build and refresh caches.
PolarDB-X is a distributed database service that is compatible with the MySQL ecosystem. PolarDB-X provides change logs that are fully compatible with the MySQL binlog format. PolarDB-X also provides the binlog dump protocol that is fully compatible with MySQL. Such binlogs are named global binlogs in PolarDB-X. Global binlogs are named relative to local binlogs on each data node (DN) in PolarDB-X. For example, a distributed transaction generates change logs in binlog files on multiple DN nodes. These logs are local and distributed, whereas global binlogs sort, merge, and shape such local and distributed data to generate binlog data in the form of a standalone transaction.
The CDC log engine of PolarDB-X generates global binlogs and provides the binlog dump service. The CDC subsystem shields the internal complexity of a distributed database. For example, the CDC subsystem shields the details of distributed transactions. All binlog data is in the form of standalone transactions. The CDC subsystem also shields the scaling of internal nodes. In this case, additional adaptation is not required. In short, the binlog service provided by CDC of PolarDB-X allows you to obtain user experience the same as that from a standalone MySQL database.