PolarDB:Release notes for PolarDB-X CDC nodes

Category

Description

Feature optimization

The default size of the binary log file is updated to 1 GB. When the binary log file number exceeds 999999, the binary log file can be automatically rotated.

The download of the transparent consumption file for the binary logs is optimized.

Category

Description

Feature optimization

The replication logic of the primary and secondary nodes is optimized.

The heartbeat timeout detection logic of the CDC high-availability module is optimized to avoid unnecessary process restarts.

Fixed issue

The following issue is fixed: The update of an earlier version of CDC is interrupted due to the lack of the RuntimeVersion field in the system table.

The following issue is fixed: The WITH PARSER NGRAM attribute is not included in the results of SHOW CREATE TABLE statement.

Category

Description

Fixed issue

The following issue is fixed: CDC links are interrupted due to compatibility differences when REAL and DOUBLE data types in CDC metadata are processed.

Category

Description

Fixed issue

The following issue is fixed: When a single SQL statement is executed to update multiple tables, consecutive TableMapEvents may occur, resulting in disordered trace IDs.

Category

Description

Fixed issue

The following issue is fixed: The CDC process cannot be started because a single compute node is unavailable.

The following issue is fixed: Dump operations fails on binlogs because the offset values in the memory are inconsistent with those in binlogs.

Category

Description

Fixed issue

The following issue is fixed: Information returned from the SHOW FULL BINARY LOGS statement is incomplete.

The following issue is fixed: A binlog file is lost due to concurrent uploads of the file during the CDC topology reconstruction process.

The following issues are fixed: The ALTER statements that contain implicit table groups fail to be parsed by the SQL parser or fail to be correctly processed.

The following issue is fixed: The system automatically removes local files on the primary node during the primary-secondary synchronization process by using a CDC dumper. As a result, the replication links cannot be automatically restored.

Category

Description

New feature

Sync points are periodically created on CDC nodes.

Fixed issue

The following issue is fixed: The binlog cursor fails to advance through binlogs as expected, which leads to hindered binlog consumption.

Category

Description

New feature and feature optimization

Sync points are created on CDC nodes at specific intervals.

The full data verification capability used to evaluate the upgrade of a PolarDB-X 1.0 instance to a PolarDB-X 2.0 instance is optimized.

Fixed issue

The following issue is fixed: Global binlogs fail to be generated due to metadata changes.

The following issue is fixed: The binlog dump service cannot be provided because the destination binlog file is empty.

Category

Description

New feature and feature optimization

DDL operations on functions can be replicated from the primary instance to the secondary instance.

DDL operations on sequences can be replicated from the primary instance to the secondary instance.

DDL operations on views can be replicated from the primary instance to the secondary instance.

DDL operations on procedures can be replicated from the primary instance to the secondary instance.

DDL operations on accounts, roles, and permissions can be replicated from the primary instance to the secondary instance.

ALTER TABLEGROUP operations can be replicated from the primary instance to the secondary instance.

ALTER INDEX operations can be replicated from the primary instance to the secondary instance.

DDL operations are applied in a consistent and coordinated manner across binlog streams in a multi-stream binlog environment.

Replication links can be created based on timestamps for replications from the primary instance to the secondary instance.

Consistency verification of upstream and downstream data by using SQL statements is supported in replications from the primary instance to the secondary instance.

The secondary instance role is added. An instance to which the role is assigned can be read-only.

WHERE clauses in DML statements can be fully replicated from the primary instance to the secondary instance.

Multiple binlog streams can be associated with specific account names. When you use the associated account to execute binlog-related SQL statements, you do not need to add the with option.

Server_id-based two-way replication between the primary instance and the secondary instance is optimized. Filter rules can be specified by using the CLI and anomalies with server_id can be automatically detected and reported.

Fixed issue

The following issue is fixed: The column order of metadata is incorrect after multiple ALTER ADD COLUMN or ALTER DROP COLUMN statements are executed, which leads to interrupted binlog links.

The following issue is fixed: Metadata indexes are lost after multiple ALTER ADD INDEX, ALTER DROP INDEX, and ALTER RENAME INDEX statements are executed, which causes incorrect DDL statements to be recorded in binlogs.

The following issue is fixed: DDL is filtered when the ALTER DDL SQL statement contains the 'user' keyword due to a configuration defect in the meta_build_physical_ddl_sql_blacklist_regex parameter.

The following issue is fixed: DROP INDEX DDL statements cannot be recorded in binlogs.

Category

Description

New feature

Binlog-related statements such as SHOW FULL BINARY LOGS and SHOW FULL MASTER STATUS are added.

Fixed issue

The following issue is fixed: SQL flashback exact match fails.

Category

Description

New feature

The SHOW FULL MASTER STATUS statement is added.

Feature optimization

The MySQL binlog dump process can use parameters such as checksum and heartbeat to optimize data integrity and replication monitoring.

Category

Description

New feature

SQL flashback is supported and can roll back a transaction.

Fixed issue

The method used to record transaction sequence object (TSO) heartbeat entries in binlogs is modified. This resolves the associated downstream crash issue.

Category

Description

Feature optimization

The gRPC backpressure sleep duration is optimized. This increases the binlog dump speed to 130 MB/s.

Fixed issue

The following issue is fixed: The SQL flashback process fails to complete due to the absence of DML or DDL operations within the specified time range.

Category

Description

Feature optimization

The query efficiency of the SHOW BINARY LOGS statement in scenarios that involve large numbers of binlogs is optimized. The slow query issue is also fixed.

Fixed issue

The following issue is fixed: A buffer overflow occurs when you perform data shaping on JSON data.

The following issue is fixed: During the upgrade of the change data capture (CDC) feature, the metaDataSource scan thread exits due to high-availability (HA) switchover in the metadatabase.

The following issue is fixed: Deadlocks occur when binlog_dumper_info is updated.

Category

Description

New feature and feature optimization

The flush logs statement is supported.

The data dump efficiency of the transparent consumption feature is optimized.

CDC nodes do not need to obtain the connection string of the data node from the is_vip parameter.

CDC nodes do not need to obtain the connection string of the metadata database from the control module.

Fixed issue

The following issue is fixed: Errors occur if the table creation statement and parsing syntax contain the BIGINT SIGNED column definition.

The following issue is fixed: Holes exist in the binlog list when the syntax for generating transaction sequence objects (TSOs) is restored.

The following issue is fixed: After the read-only instance is scaled, task dump cannot locate the replication start point on the data node of the primary instance.

Category

Description

New feature

Transaction objects can be saved to persistent medium to reduce memory usage.

Fixed issue

The following issue is fixed: The number of TSOs to be restored based on previous TSOs must be greater than or equal to the number of TSOs of transactions that are most recently tagged with scaling.

The following issue is fixed: When data is synchronized from the primary instance to a secondary instance, the gRPC thread cannot exit because local files generated by Dumper Master are deleted.

Category

Description

Fixed issue

The following issue is fixed: When you consume binlogs from Object Storage Service (OSS), a conflict in the timing of sending heartbeat packets may occur and lead to a link failure.

The following issue is fixed: Errors in data type conversion may occur when you reformat TableMapEvent.

The following issue is fixed: The Sql_Mode value recorded in a DDL event may differ from the original Sql_Mode value.

The following issue is fixed: Inconsistencies in metadata exist between logical and physical tables when NCHAR and NVARCHAR data types are used.

The following issue is fixed: When a DDL SQL statement contains line breaks, the statement may fail to be converted into a single-line comment.

Category

Description

New feature and feature optimization

The statistics on the CPU utilization of worker processes can be monitored.

Fine-grained monitoring of EPS (INSERT, UPDATE, and DELETE) is supported.

The upload speed of binlog files to OSS is increased to 700 MB/s.

The CDC O&M statement (START MASTER, STOP MASTER, RESTART MASTER, and RESET MASTER) are added.

Fixed issue

The following issue is fixed: Blocks occur after a long period of time when offline binlogs are transparently consumed.

The following issue is fixed: The dump thread cannot exit when binlogs are deleted during data reads.

The following issue is fixed: The RESET MASTER statement fails when a large amount of metadata is involved in DDL physical duplication.

The following issue is fixed: The sql_mode value in the DDL event is inconsistent with the original sql_mode value.

The following issue is fixed: When an SQL statement contains the add primary key and drop primary key, an error occurs in the MySQL database.

Category

Description

Fixed issue

The following issue is fixed: The column order is incorrect after an SQL statement is executed in the SchemaRepository.

The following issue is fixed: A null pointer exception (NPE) is triggered when an empty string is reshaped.

Category

Description

Fixed issue

The following issue is fixed: Data is truncated because the length of metadata information is greater than the length of the data of the mediumtext data type.

Category

Description

Fixed issue

The following issue is fixed: Data fails to be parsed because the character sets of MySQL are not converted to the character sets of Java.

Category

Description

Feature optimization

The recovery time objective (RTO) for binlog synchronization is reduced to 15 seconds.

The speed for restoring CDC metadata when hundreds of thousands of tables are involved is optimized.

Fixed issue

The following issue is fixed: An error may occur in the calculation of the binlog file position when binlogs are directly consumed.

The following issue is fixed: Compatibility errors occur when metadata consistency is checked in scenarios in which text(0) is used.

Category

Description

Fixed issue

The following issue is fixed: The CDC link is interrupted because the primary keys are continuously created and deleted.

The following issue is fixed: CDC closes the connection because a consumer program does not send the COM_REGISTER_SLAVE command to CDC before the program dumps the binlog files.

The following issue is fixed: The data link is interrupted when the hash value is Integer.MIN_VALUE in a multi-stream routing scenario.

Category

Description

New feature and feature optimization

The multi-level merging feature is supported. Data nodes can linearly increase without significant performance degradation.

Transparent consumption is supported. The downstream consumer program can use the binlog dump protocol to directly consume binlog files that are archived in OSS.

Multi-stream binlogs are supported. You can specify a custom number of streams. Each stream of binlogs is independent.

Scheduled snapshots are supported for the full metadata. This helps resolve the excessive expansion of labeling records for historical DDL operations.

Binlogs can be concurrently downloaded from OSS, and the amount of time required to restore the PolarDB-X instance is decreased.

Fixed issue

The following issue is fixed: Compatibility errors and incorrect results may be returned when the event data in binlogs is reshaped. This improves the stability of CDC links during DDL operations.

The following issue is fixed: A timeout may occur when the downstream consumer program dumps binlogs after a CDC cluster is restarted.

The following issue is fixed: The estimated latency is incorrect the first time the Dumper process is started.