Overview of migration solutions for PolarDB-X 2.0 as the source
Review the notes and limits for your migration task based on the following solutions:
Migrate from PolarDB-X 2.0 to ApsaraDB RDS for MySQL or a self-managed MySQL database
The following notes and limits apply:
Type | Description |
Source database limits | Bandwidth requirements: The server that hosts the source database must have sufficient outbound bandwidth. Otherwise, the data migration speed is affected. Read-only instances of PolarDB-X 2.0 Enterprise Edition are not supported as the source. The tables to be migrated must have primary keys or UNIQUE constraints, and the fields must be unique. Otherwise, duplicate data may appear in the destination database. If you migrate data at the table level and need to edit the tables, such as mapping column names, a single data migration task supports a maximum of 1,000 tables. If you exceed this limit, an error is reported when you submit the task. In this case, split the tables into multiple batches and configure a separate task for each batch, or configure a task to migrate the entire database. For incremental migration, binary logs: Binary logging must be enabled, and the binlog_row_image parameter must be set to full. Otherwise, an error is reported during the precheck, and the data migration task cannot start. For an incremental migration task, Data Transmission Service (DTS) requires that the binary logs on the source database are retained for at least 24 hours. For a task that includes both full migration and incremental migration, DTS requires that the binary logs on the source database are retained for at least 7 days. After the full migration is complete, you can change the retention period to 24 hours or more. If the retention period is shorter than required, the DTS task may fail because it cannot obtain the binary logs. In extreme cases, data inconsistency or data loss may occur. Issues caused by a binary log retention period that is shorter than the required period are not covered by the Service-Level Agreement (SLA).
If the names of tables to be migrated in the PolarDB-X 2.0 instance contain uppercase letters, only schema migration is supported. Source database operation limits: During schema migration and full migration, do not perform DDL operations to change the schema of databases or tables. Otherwise, the data migration task fails.
Note During full migration, DTS queries the source database. This creates a metadata lock, which may block DDL operations on the source database. To switch the network type of the PolarDB-X 2.0 instance during migration, update the network connection information for the migration link after the switch is successful. If you perform only full data migration, do not write new data to the source instance. Otherwise, data inconsistency occurs between the source and destination. To maintain real-time data consistency, select Schema Migration, Full Data Migration, and Incremental Data Migration.
Migration of table groups and databases or tables that have the Locality property is not supported. Migration of tables whose names are reserved words, such as select, is not supported. In a PolarDB-X 2.0 instance, synchronization of database partitions in DRDS mode is not supported.
|
Other limits | Evaluate the performance of the source and destination databases before migration. Migrate data during off-peak hours. During full data migration, DTS consumes read and write resources on the source and destination databases. This may increase the database load. Full data migration involves concurrent INSERT operations. This causes fragmentation in the tables of the destination database. As a result, the storage space used by the tables in the destination database is larger than that in the source instance after full migration is complete. DTS attempts to resume failed migration tasks that are less than seven days old. Before you switch your business to the destination instance, you must end or release the task. You can also revoke the write permissions of the account that DTS uses to access the destination instance using the revoke command. This prevents the source data from overwriting the data in the destination instance if the task is automatically resumed. If the data to be migrated contains content that requires four-byte storage, such as uncommon characters or emojis, the destination database and tables that receive the data must use the utf8mb4 charset.
Note If you use DTS for schema migration, you must set the instance-level parameter character_set_server of the destination database to utf8mb4. If a DDL statement fails to be written to the destination database, the DTS task continues to run. You need to check the task logs for the failed DDL statement. For more information about how to view task logs, see View task logs. If you write fields whose column names differ only in case to the same table in the destination MySQL database, the migration result may not be as expected. This is because column names in MySQL databases are case-insensitive. After the data migration is complete (the Status of the instance is Completed), we recommend that you run the analyze table <table_name> command to confirm that all data has been written to the destination table. For example, if an HA switchover is triggered in the destination MySQL database, data may be written only to the memory, causing data loss. If an instance fails, DTS helpdesk will try to recover the instance within 8 hours. During the recovery process, operations such as restarting the instance or adjusting its parameters may be performed.
Note When parameters are adjusted, only the parameters of the DTS instance are modified. The parameters in the database are not modified. The parameters that may be modified include but are not limited to those described in Modify instance parameters.
|
Other notes | DTS periodically updates the `dts_health_check`.`ha_health_check` table in the source database to advance the binary log offset. If the destination database is ApsaraDB RDS for MySQL: DTS automatically creates a database in ApsaraDB RDS for MySQL. If the name of the database to be migrated does not comply with the naming conventions of ApsaraDB RDS for MySQL, you must create the database in ApsaraDB RDS for MySQL before you configure the migration task. For more information, see Manage databases.
|
Migrate from PolarDB-X 2.0 to PolarDB for MySQL
The following notes and limits apply:
Type | Description |
Source database limits | Bandwidth requirements: The server that hosts the source database must have sufficient outbound bandwidth. Otherwise, the data migration speed is affected. Read-only instances of PolarDB-X 2.0 Enterprise Edition are not supported as the source. The tables to be migrated must have primary keys or UNIQUE constraints, and the fields must be unique. Otherwise, duplicate data may appear in the destination database. If you migrate data at the table level and need to edit the tables, such as mapping column names, a single data migration task supports a maximum of 1,000 tables. If you exceed this limit, an error is reported when you submit the task. In this case, split the tables into multiple batches and configure a separate task for each batch, or configure a task to migrate the entire database. For incremental migration, binary logs: Binary logging must be enabled, and the binlog_row_image parameter must be set to full. Otherwise, an error is reported during the precheck, and the data migration task cannot start. For an incremental migration task, Data Transmission Service (DTS) requires that the binary logs on the source database are retained for at least 24 hours. For a task that includes both full migration and incremental migration, DTS requires that the binary logs on the source database are retained for at least 7 days. After the full migration is complete, you can change the retention period to 24 hours or more. If the retention period is shorter than required, the DTS task may fail because it cannot obtain the binary logs. In extreme cases, data inconsistency or data loss may occur. Issues caused by a binary log retention period that is shorter than the required period are not covered by the Service-Level Agreement (SLA).
If the names of tables to be migrated in the PolarDB-X 2.0 instance contain uppercase letters, only schema migration is supported. Source database operation limits: During schema migration and full migration, do not perform DDL operations to change the schema of databases or tables. Otherwise, the data migration task fails.
Note During full migration, DTS queries the source database. This creates a metadata lock, which may block DDL operations on the source database. To switch the network type of the PolarDB-X 2.0 instance during migration, update the network connection information for the migration link after the switch is successful. If you perform only full data migration, do not write new data to the source instance. Otherwise, data inconsistency occurs between the source and destination. To maintain real-time data consistency, select Schema Migration, Full Data Migration, and Incremental Data Migration.
Migration of table groups and databases or tables that have the Locality property is not supported. Migration of tables whose names are reserved words, such as select, is not supported. In a PolarDB-X 2.0 instance, synchronization of database partitions in DRDS mode is not supported.
|
Other limits | Evaluate the performance of the source and destination databases before migration. Migrate data during off-peak hours. During full data migration, DTS consumes read and write resources on the source and destination databases. This may increase the database load. Full data migration involves concurrent INSERT operations. This causes fragmentation in the tables of the destination database. As a result, the storage space used by the tables in the destination database is larger than that in the source instance after full migration is complete. DTS attempts to resume failed migration tasks that are less than seven days old. Before you switch your business to the destination instance, you must end or release the task. You can also revoke the write permissions of the account that DTS uses to access the destination instance using the revoke command. This prevents the source data from overwriting the data in the destination instance if the task is automatically resumed. If the data to be migrated contains content that requires four-byte storage, such as uncommon characters or emojis, the destination database and tables that receive the data must use the utf8mb4 charset.
Note If you use DTS for schema migration, you must set the instance-level parameter character_set_server of the destination database to utf8mb4. If a DDL statement fails to be written to the destination database, the DTS task continues to run. You need to check the task logs for the failed DDL statement. For more information about how to view task logs, see View task logs. If an instance fails, DTS helpdesk will try to recover the instance within 8 hours. During the recovery process, operations such as restarting the instance or adjusting its parameters may be performed.
Note When parameters are adjusted, only the parameters of the DTS instance are modified. The parameters in the database are not modified. The parameters that may be modified include but are not limited to those described in Modify instance parameters.
|
Other notes | DTS periodically updates the `dts_health_check`.`ha_health_check` table in the source database to advance the binary log offset. If the destination database is PolarDB for MySQL: DTS automatically creates a database in PolarDB for MySQL. If the name of the database to be migrated does not comply with the naming conventions of PolarDB for MySQL, you must create the database in PolarDB for MySQL before you configure the migration task. For more information, see Manage databases.
|
Migrate between PolarDB-X 2.0 instances
The following notes and limits apply:
Type | Description |
Source database limits | Bandwidth requirements: The server that hosts the source database must have sufficient outbound bandwidth. Otherwise, the data migration speed is affected. Read-only instances of PolarDB-X 2.0 Enterprise Edition are not supported as the source. The tables to be migrated must have primary keys or UNIQUE constraints, and the fields must be unique. Otherwise, duplicate data may appear in the destination database. If you migrate data at the table level and need to edit the tables, such as mapping column names, a single data migration task supports a maximum of 1,000 tables. If you exceed this limit, an error is reported when you submit the task. In this case, split the tables into multiple batches and configure a separate task for each batch, or configure a task to migrate the entire database. For incremental migration, binary logs: Binary logging must be enabled, and the binlog_row_image parameter must be set to full. Otherwise, an error is reported during the precheck, and the data migration task cannot start. For an incremental migration task, Data Transmission Service (DTS) requires that the binary logs on the source database are retained for at least 24 hours. For a task that includes both full migration and incremental migration, DTS requires that the binary logs on the source database are retained for at least 7 days. After the full migration is complete, you can change the retention period to 24 hours or more. If the retention period is shorter than required, the DTS task may fail because it cannot obtain the binary logs. In extreme cases, data inconsistency or data loss may occur. Issues caused by a binary log retention period that is shorter than the required period are not covered by the Service-Level Agreement (SLA).
If the names of tables to be migrated in the PolarDB-X 2.0 instance contain uppercase letters, only schema migration is supported. Source database operation limits: During schema migration and full migration, do not perform DDL operations to change the schema of databases or tables. Otherwise, the data migration task fails.
Note During full migration, DTS queries the source database. This creates a metadata lock, which may block DDL operations on the source database. To switch the network type of the PolarDB-X 2.0 instance during migration, update the network connection information for the migration link after the switch is successful. If you perform only full data migration, do not write new data to the source instance. Otherwise, data inconsistency occurs between the source and destination. To maintain real-time data consistency, select Schema Migration, Full Data Migration, and Incremental Data Migration.
Migration of table groups and databases or tables that have the Locality property is not supported. Migration of tables whose names are reserved words, such as select, is not supported. In a PolarDB-X 2.0 instance, synchronization of database partitions in DRDS mode is not supported.
|
Other limits | Evaluate the performance of the source and destination databases before migration. Migrate data during off-peak hours. During full data migration, DTS consumes read and write resources on the source and destination databases. This may increase the database load. Full data migration involves concurrent INSERT operations. This causes fragmentation in the tables of the destination database. As a result, the storage space used by the tables in the destination database is larger than that in the source instance after full migration is complete. DTS attempts to resume failed migration tasks that are less than seven days old. Before you switch your business to the destination instance, you must end or release the task. You can also revoke the write permissions of the account that DTS uses to access the destination instance using the revoke command. This prevents the source data from overwriting the data in the destination instance if the task is automatically resumed. If an instance fails, DTS helpdesk will try to recover the instance within 8 hours. During the recovery process, operations such as restarting the instance or adjusting its parameters may be performed.
Note When parameters are adjusted, only the parameters of the DTS instance are modified. The parameters in the database are not modified. The parameters that may be modified include but are not limited to those described in Modify instance parameters.
|
Other notes | DTS periodically updates the `dts_health_check`.`ha_health_check` table in the source database to advance the binary log offset. |
Migrate from PolarDB-X 2.0 to AnalyticDB for MySQL 3.0
The following notes and limits apply:
Type | Description |
Source database limits | Bandwidth requirements: The server that hosts the source database must have sufficient outbound bandwidth. Otherwise, the data migration speed is affected. Read-only instances of PolarDB-X 2.0 Enterprise Edition are not supported as the source. The tables to be migrated must have primary keys or UNIQUE constraints, and the fields must be unique. Otherwise, duplicate data may appear in the destination database. If you migrate data at the table level and need to edit the tables, such as mapping column names, a single data migration task supports a maximum of 1,000 tables. If you exceed this limit, an error is reported when you submit the task. In this case, split the tables into multiple batches and configure a separate task for each batch, or configure a task to migrate the entire database. For incremental migration, binary logs: Binary logging must be enabled, and the binlog_row_image parameter must be set to full. Otherwise, an error is reported during the precheck, and the data migration task cannot start. For an incremental migration task, Data Transmission Service (DTS) requires that the binary logs on the source database are retained for at least 24 hours. For a task that includes both full migration and incremental migration, DTS requires that the binary logs on the source database are retained for at least 7 days. After the full migration is complete, you can change the retention period to 24 hours or more. If the retention period is shorter than required, the DTS task may fail because it cannot obtain the binary logs. In extreme cases, data inconsistency or data loss may occur. Issues caused by a binary log retention period that is shorter than the required period are not covered by the Service-Level Agreement (SLA).
If the names of tables to be migrated in the PolarDB-X 2.0 instance contain uppercase letters, only schema migration is supported. Source database operation limits: During schema migration and full migration, do not perform DDL operations to change the schema of databases or tables. Otherwise, the data migration task fails.
Note During full migration, DTS queries the source database. This creates a metadata lock, which may block DDL operations on the source database. To switch the network type of the PolarDB-X 2.0 instance during migration, update the network connection information for the migration link after the switch is successful. If you perform only full data migration, do not write new data to the source instance. Otherwise, data inconsistency occurs between the source and destination. To maintain real-time data consistency, select Schema Migration, Full Data Migration, and Incremental Data Migration.
Migration of table groups and databases or tables that have the Locality property is not supported. Migration of tables whose names are reserved words, such as select, is not supported. In a PolarDB-X 2.0 instance, synchronization of database partitions in DRDS mode is not supported.
|
Other limits | Migration of prefix indexes is not supported. If the source database contains prefix indexes, the data migration task may fail. The destination database must have a custom primary key, or you must configure the Primary Key Column in the Configurations for Databases, Tables, and Columns step. Otherwise, the data migration may fail. Due to the limits of AnalyticDB for MySQL 3.0, if the disk space usage of a node in the AnalyticDB for MySQL 3.0 cluster exceeds 80%, the DTS task becomes abnormal and latency occurs. Estimate the required space based on the objects to be migrated and ensure that the destination cluster has sufficient storage space. If the destination AnalyticDB for MySQL 3.0 cluster is being backed up while the DTS task is running, the task fails. Evaluate the performance of the source and destination databases before migration. Migrate data during off-peak hours. During full data migration, DTS consumes read and write resources on the source and destination databases. This may increase the database load. Full data migration involves concurrent INSERT operations. This causes fragmentation in the tables of the destination database. As a result, the storage space used by the tables in the destination database is larger than that in the source instance after full migration is complete. DTS attempts to resume failed migration tasks that are less than seven days old. Before you switch your business to the destination instance, you must end or release the task. You can also revoke the write permissions of the account that DTS uses to access the destination instance using the revoke command. This prevents the source data from overwriting the data in the destination instance if the task is automatically resumed. If a DDL statement fails to be written to the destination database, the DTS task continues to run. You need to check the task logs for the failed DDL statement. For more information about how to view task logs, see View task logs. If an instance fails, DTS helpdesk will try to recover the instance within 8 hours. During the recovery process, operations such as restarting the instance or adjusting its parameters may be performed.
Note When parameters are adjusted, only the parameters of the DTS instance are modified. The parameters in the database are not modified. The parameters that may be modified include but are not limited to those described in Modify instance parameters.
|
Other notes | DTS periodically updates the `dts_health_check`.`ha_health_check` table in the source database to advance the binary log offset. |
Migrate from PolarDB-X 2.0 to Alibaba Cloud Kafka
Type | Description |
Source database limits | Bandwidth requirements: The server that hosts the source database must have sufficient outbound bandwidth. Otherwise, the data migration speed is affected. Read-only instances of PolarDB-X 2.0 Enterprise Edition are not supported as the source. The tables to be migrated must have primary keys or UNIQUE constraints, and the fields must be unique. Otherwise, duplicate data may appear in the destination database. If you migrate data at the table level and need to edit the tables, such as mapping column names, a single data migration task supports a maximum of 1,000 tables. If you exceed this limit, an error is reported when you submit the task. In this case, split the tables into multiple batches and configure a separate task for each batch, or configure a task to migrate the entire database. For incremental migration, binary logs: Binary logging must be enabled, and the binlog_row_image parameter must be set to full. Otherwise, an error is reported during the precheck, and the data migration task cannot start. For an incremental migration task, Data Transmission Service (DTS) requires that the binary logs on the source database are retained for at least 24 hours. For a task that includes both full migration and incremental migration, DTS requires that the binary logs on the source database are retained for at least 7 days. After the full migration is complete, you can change the retention period to 24 hours or more. If the retention period is shorter than required, the DTS task may fail because it cannot obtain the binary logs. In extreme cases, data inconsistency or data loss may occur. Issues caused by a binary log retention period that is shorter than the required period are not covered by the Service-Level Agreement (SLA).
If the names of tables to be migrated in the PolarDB-X 2.0 instance contain uppercase letters, only schema migration is supported. Source database operation limits: During schema migration and full migration, do not perform DDL operations to change the schema of databases or tables. Otherwise, the data migration task fails.
Note During full migration, DTS queries the source database. This creates a metadata lock, which may block DDL operations on the source database. To switch the network type of the PolarDB-X 2.0 instance during migration, update the network connection information for the migration link after the switch is successful. If you perform only full data migration, do not write new data to the source instance. Otherwise, data inconsistency occurs between the source and destination. To maintain real-time data consistency, select Schema Migration, Full Data Migration, and Incremental Data Migration.
Migration of table groups and databases or tables that have the Locality property is not supported. Migration of tables whose names are reserved words, such as select, is not supported. In a PolarDB-X 2.0 instance, synchronization of database partitions in DRDS mode is not supported.
|
Other limits | Migration of INDEX, PARTITION, VIEW, PROCEDURE, FUNCTION, TRIGGER, and FK objects is not supported. Evaluate the performance of the source and destination databases before migration. Migrate data during off-peak hours. During full data migration, DTS consumes read and write resources on the source and destination databases. This may increase the database load. Full data migration involves concurrent INSERT operations. This causes fragmentation in the tables of the destination database. As a result, the storage space used by the tables in the destination database is larger than that in the source instance after full migration is complete. DTS attempts to resume failed migration tasks that are less than seven days old. Before you switch your business to the destination instance, you must end or release the task. You can also revoke the write permissions of the account that DTS uses to access the destination instance using the revoke command. This prevents the source data from overwriting the data in the destination instance if the task is automatically resumed. If the destination Kafka instance is scaled out or scaled in during migration, you need to restart the instance. If an instance fails, DTS helpdesk will try to recover the instance within 8 hours. During the recovery process, operations such as restarting the instance or adjusting its parameters may be performed.
Note When parameters are adjusted, only the parameters of the DTS instance are modified. The parameters in the database are not modified. The parameters that may be modified include but are not limited to those described in Modify instance parameters.
|
Other notes | DTS periodically updates the `dts_health_check`.`ha_health_check` table in the source database to advance the binary log offset. |
Migrate from PolarDB-X 2.0 to Elasticsearch
Type | Description |
Source database limits | Bandwidth requirements: The server that hosts the source database must have sufficient outbound bandwidth. Otherwise, the data migration speed is affected. Read-only instances of PolarDB-X 2.0 Enterprise Edition are not supported as the source. The tables to be migrated must have primary keys or UNIQUE constraints, and the fields must be unique. Otherwise, duplicate data may appear in the destination database. If you migrate data at the table level and need to edit the tables, such as mapping column names, a single data migration task supports a maximum of 1,000 tables. If you exceed this limit, an error is reported when you submit the task. In this case, split the tables into multiple batches and configure a separate task for each batch, or configure a task to migrate the entire database. For incremental migration, binary logs: Binary logging must be enabled, and the binlog_row_image parameter must be set to full. Otherwise, an error is reported during the precheck, and the data migration task cannot start. For an incremental migration task, Data Transmission Service (DTS) requires that the binary logs on the source database are retained for at least 24 hours. For a task that includes both full migration and incremental migration, DTS requires that the binary logs on the source database are retained for at least 7 days. After the full migration is complete, you can change the retention period to 24 hours or more. If the retention period is shorter than required, the DTS task may fail because it cannot obtain the binary logs. In extreme cases, data inconsistency or data loss may occur. Issues caused by a binary log retention period that is shorter than the required period are not covered by the Service-Level Agreement (SLA).
If the names of tables to be migrated in the PolarDB-X 2.0 instance contain uppercase letters, only schema migration is supported. Source database operation limits: During schema migration and full migration, do not perform DDL operations to change the schema of databases or tables. Otherwise, the data migration task fails.
Note During full migration, DTS queries the source database. This creates a metadata lock, which may block DDL operations on the source database. To switch the network type of the PolarDB-X 2.0 instance during migration, update the network connection information for the migration link after the switch is successful. If you perform only full data migration, do not write new data to the source instance. Otherwise, data inconsistency occurs between the source and destination. To maintain real-time data consistency, select Schema Migration, Full Data Migration, and Incremental Data Migration.
Migration of table groups and databases or tables that have the Locality property is not supported. Migration of tables whose names are reserved words, such as select, is not supported. In a PolarDB-X 2.0 instance, synchronization of database partitions in DRDS mode is not supported.
|
Other limits | To add a column to a table to be migrated from the source database, first modify the mapping of the corresponding table in the Elasticsearch instance. Then, perform the corresponding DDL operation in the source database. Finally, pause and restart the migration task. You cannot migrate data to an index in the destination that has a parent-child relationship or a Join field type mapping. Otherwise, task exceptions or query failures in the destination may occur. Elasticsearch instances of the development and test specifications are not supported. Evaluate the performance of the source and destination databases before migration. Migrate data during off-peak hours. During full data migration, DTS consumes read and write resources on the source and destination databases. This may increase the database load. Full data migration involves concurrent INSERT operations. This causes fragmentation in the tables of the destination database. As a result, the storage space used by the tables in the destination database is larger than that in the source instance after full migration is complete. DTS attempts to resume failed migration tasks that are less than seven days old. Before you switch your business to the destination instance, you must end or release the task. You can also revoke the write permissions of the account that DTS uses to access the destination instance using the revoke command. This prevents the source data from overwriting the data in the destination instance if the task is automatically resumed. If an instance fails, DTS helpdesk will try to recover the instance within 8 hours. During the recovery process, operations such as restarting the instance or adjusting its parameters may be performed.
Note When parameters are adjusted, only the parameters of the DTS instance are modified. The parameters in the database are not modified. The parameters that may be modified include but are not limited to those described in Modify instance parameters.
|
Other notes | DTS periodically updates the `dts_health_check`.`ha_health_check` table in the source database to advance the binary log offset. |