ApsaraDB for OceanBase:Overview

Background information

MySQL binary logging (Binlog) is the foundation for master-slave replication and change tracking. You can enable binary logging with low performance overhead for scenarios such as real-time change tracking. This feature enables you to reuse your existing MySQL Binlog incremental parsing system to sync incremental data from the MySQL-compatible mode of OceanBase Database. You do not need to perform custom development or set up a new environment. You can seamlessly switch your existing MySQL-based incremental change tracking solution to ApsaraDB for OceanBase.

Performance limitations

The performance of the Binlog conversion service is limited by resource configuration and Clog parsing capabilities. Currently, the Binlog service supports a maximum Clog parsing speed of 25 MB/s. The upper limit for the Binlog conversion rate is about 50,000 records per second (RPS). If the source service traffic exceeds these limits, the latency of the Binlog conversion service may increase. The Binlog service also supports a maximum of 50 subscription connections. If the number of concurrent subscription connections exceeds this limit, the service rejects the request. In scenarios with high traffic and large transactions, the Binlog service must process Clogs incrementally to maintain real-time performance.

When service traffic is manageable, the latency of the Binlog conversion service is typically less than 1 second. However, in special cases, such as frequent data definition language (DDL) operations or large transactions in the database, a sub-second conversion latency cannot be guaranteed.

Features of the Binlog service

• The Binlog service will be automatically disabled if no subscription connections are active for more than 7 days.

• The OceanBase Binlog Service is managed at the tenant level. You cannot enable binary logging for specific databases or tables.

• The OceanBase Binlog Service is compatible with the Row format of MySQL Binlog. It records changes at the row level.

• The MySQL-compatible mode of OceanBase Database is compatible with most MySQL data types. For more information, see Overview of MySQL data types in OceanBase. The DDL syntax of the MySQL-compatible mode of OceanBase Database differs from MySQL DDL syntax, and the MySQL-compatible mode has some unique extended syntax. Therefore, these DDL statements may not be parsed correctly.

  Note

  • The extended semantics for ENUM and SET types within a MySQL-compatible tenant of OceanBase Database are not supported. For example, the service does not support SET type definitions with more than 64 members, duplicate values in SET types, or ENUM types that allow inserting undefined data such as ''.

  • The VARCHAR type can store a maximum of 65,535 bytes of data. If the length of a VARCHAR type exceeds this value, this may cause parsing errors for the corresponding table data.

    Note: The character set configuration affects the VARCHAR length configuration. For example, if the character set is utf8mb4, one character occupies 4 bytes. In this case, the maximum length that you can configure for VARCHAR is 16,383.

  • In a MySQL-compatible tenant of OceanBase Database, the LONGBLOB and LONGTEXT types support a maximum size of 48 MB. This differs from the MySQL implementation.

• If you use OceanBase Database V4.x, configure the STORED attribute for generated columns. Otherwise, the Binlog service will not convert data for these columns. This may cause downstream subscription parsing to fail or result in data loss.

• The Binlog protocol for the MySQL-compatible mode of OceanBase Database is mostly consistent with the MySQL 5.7 Binlog protocol and shares similar basic features:

  • It supports two extraction modes: filename + offset and GTID. The GTID is generated by the OceanBase Binlog service and is also globally unique.

  • It supports reading Binlog files for all databases and tables. The OceanBase Binlog service dumps incremental data from all databases and tables at the tenant level and stores it in Binlog files for persistent storage.

  • It supports multiple Binlog Dump subscription connections. You can subscribe from any offset within the range of [earliest binary log file position, latest position].

• It supports periodic cleanup of Binlog files. After you enable the OceanBase Binlog service on Alibaba Cloud, logs are retained for 18 hours or up to 50 GB by default. Log cleanup is triggered when either condition is met.

• It supports integration with mainstream MySQL Binlog parsing tools, such as FlinkCDC, Canal, and MaxWell.

• The Binlog service provides high availability (HA) and can automatically recover from failures within minutes. However, downstream subscribers must have a reconnection mechanism to prevent subscription interruptions during a failure.

How to get Binlog files

Connection method: You can connect through the primary endpoint of the tenant.

Version requirement: The OceanBase database instance must be V3.2.4 or later.

Procedure:

1. The OceanBase Binlog Service is disabled by default. You must first enable the service. For more information, see Enable the Binlog service.

2. Connect to the OceanBase database using a MySQL client. For more information, see Connect to an OceanBase database instance using a MySQL client.

3. After you connect to the OceanBase database, you can run the relevant MySQL Binlog commands in the connected client.

   1. You can run the show binary logs statement to view the list of Binlog files.

   2. You can run the show master status statement to view the Binlog file that is currently being written to.

   3. You can run the show binlog events statement to view the specific events in a Binlog file.

4. Subscribe to changes using Canal. For more information, see QuickStart.