All Products
Search
Document Center

ApsaraDB for OceanBase:View the details of a data migration task

Last Updated:Feb 26, 2025

After a data migration task starts, you can view the task information on the details page of the task, such as the basic information, and the progress and status of the task.

Access the details page

On the details page of a data migration task, you can view the current phase and the progress and status of each phase to learn about the overall progress of the data migration task, as well as the migration status of each table and each object.

  1. Log on to the ApsaraDB for OceanBase console.

  2. In the left-side navigation pane, choose Data Transmission > Data Migration.

  3. On the Data Migration page, click the name of the target task in the Migration Tasks section. On the details page that appears, view its Basic Information and Migration Details.

    image.png

    On the Data Migration page, you can filter data migration tasks by status and keyword. A data migration task can be in one of the following states:

    • Not Configured: The data migration task has not been configured. You can click Configure in the Actions column to configure the task.

    • Configuration changing: The instance specifications are being upgraded for the data migration task.

      If the current instance specifications cannot meet the business needs, you can upgrade the instance specifications for the data migration task. For more information, see Data transmission specifications.

    • Not Started: The data migration task has not been started. You can click Start in the Actions column to start the task.

    • Running: The data migration task is in progress. You can view the data migration plan and current progress on the right and click buttons in the Actions column to perform related operations.

    • Modifying: Migration objects are being modified for the data migration task.

    • Integrating: The data migration task is being integrated with the corresponding migration object modification task.

    • Paused: The data migration task is manually paused. You can click Restore in the Actions column to restore the task.

      Note

      A data migration task remaining in the Paused state for a long time may not be able to be restored due to the retention period of incremental logs.

    • Failed: The data migration task failed. You can locate the phase where the failure occurred on the right. To view the error message, click the task name to go to the task details page.

    • Completed: The data migration task is completed and the data transmission service has migrated the specified data to the target database in the configured migration mode.

    • Stopping: The data migration task is being stopped. You cannot edit a data migration task in this state.

    • Stopped: The data migration task is stopped.

    You can change the name of a task in the Not Started, Running, Paused, or Failed state. For more information, see Change the name of a data migration task.

View basic information

The Basic Information section displays the basic information of the current data migration task.

image.png

Parameter

Description

ID

The unique ID of the data migration task.

Migration Type

The migration type specified when the data migration task was configured.

Alert Status

The alert status of the data migration task. We recommend that you enable alert monitoring to keep track of the task status. For more information, see Configure alert monitoring settings.

Important

You cannot configure or modify alert monitoring for a data migration task in the Not Configured, Completed, Stopping, or Stopped state.

Created At

The time when the data migration task was created.

Connection Details

Click Connection Details to view the information about the source and target databases of the data migration task. You can also copy the data source identifiers.

Billing method

The billing method specified when the data migration task was purchased.

Instance Specification

The instance specifications specified when the data migration task was purchased. For more information, see Data transmission specifications.

You can perform the following operations on the current data migration task:

  • View migration objects

    Click View Objects in the upper-right corner. The migration objects of the current data migration task are displayed. You can modify the migration objects when the task is running. For more information, see View and modify migration objects and their filter conditions.

  • Upgrade the instance specifications

    If the current instance specifications cannot meet the business needs, you can upgrade the instance specifications for the data migration task. For more information, see Upgrade the instance specifications.

  • View or modify parameter configurations

    You can view the parameter configurations of a data migration task in the Configuration changing, Running, Modifying, Integrating, Completed, Stopping, or Stopped state. You can modify the parameter configurations for a data migration task in the Not Started, Paused, or Failed state. For more information, see View and modify the parameter configurations of a data migration task.

    The parameters that can be modified vary with the type of the data migration task and the phase of the task.

  • Download object configurations

    The data transmission service allows you to download the object settings of data migration tasks and import the settings in batches. For more information, see Download and import the settings of migration objects.

  • Download conflicting data

    You can download the conflicting data, if any, of a two-way migration task for which both the forward and reverse tasks have been configured. For more information, see Download conflicting data.

View migration details

The Migration Details section displays the status, progress, start time, completion time, and total duration of all subtasks.

Schema migration

The definitions of data objects, such as tables, indexes, constraints, comments, and views, are migrated from the source database to the target database. Temporary tables are automatically filtered out. If the source database is not an OceanBase database, the data transmission service automatically performs data type conversion and SQL syntax concatenation based on the syntax definition of the mode of the target OceanBase Database tenant, and then replicates the data to the target database. If a migration object with the same name already exists in the target, the data transmission service will skip the migration object. You must ensure the consistency of table schemas between the source and the target.

If you have not manually dropped the hidden columns and unique indexes, the data transmission service will automatically drop the hidden columns and unique indexes based on the type of the data migration task in the forward switchover phase. For more information, see Hidden column mechanism of the data transmission service.

You can view the status, start time, completion time, total amount of time consumed, and table and view migration progress for a schema migration task on the Schema Migration page.

image.png

You can also perform the following operations on an object:

  • View Creation Syntax: On the Database or table tab, click View in the Actions column of an object to view the database or table creation syntax.

    Compatible DDL syntax executed on the OBServer node is displayed. Incompatible syntax is converted before it is displayed.

  • Modify Creation Syntax and Try Again: View the error information, check and modify the definition of the conversion result of a failed DDL statement, and then migrate the data to the target again.

  • Retry/Retry All Failed Objects: You can retry failed schema migration tasks one by one or retry all failed tasks at a time.

  • Skip/Batch Skip: You can skip failed schema migration tasks one by one or skip multiple failed tasks at a time. To skip multiple objects at a time, click Batch Skip in the upper-right corner.

  • Remove/Batch Remove: You can remove failed schema migration tasks one by one or remove multiple failed tasks at a time. To remove multiple failed tasks at a time, click Batch Remove in the upper-right corner.

  • View Database Return Code: You can view the DDL statements executed on the OBServer node and the execution error information of a failed schema migration task.

Full migration

The existing data is migrated from tables in the source database to the corresponding tables in the target database. On the Full Migration page, you can filter objects by source and target databases, or select View Objects with Errors to filter out objects that hinder the overall migration progress. You can also view related information on the Table Objects, Table Indexes, and Full Migration Performance tabs. The status of a full migration task changes to Completed only after both the table objects and table indexes are migrated.

image.png

  • On the Table Objects tab, you can view the names, source and target databases, estimated data volume, migrated data volume, and status of tables.

  • On the Table Indexes tab, you can view the objects, source and target databases, creation time, end time, amount of time consumed, and status of tables. You can also view the index creation syntax, retry a single failed object or all failed indexes, and skip or remove one or more failed indexes.

  • On the Full Migration Performance tab, you can view the graphs of performance metrics such as the requests per second (RPS) and migration traffic of the source and target, average read time and slicing time at the source, average write time at the target, and performance benchmark, to identify performance issues in a timely manner.

    image.png

    You can combine full migration with incremental synchronization to ensure data consistency between the source and target databases. If any objects fail to be migrated during full migration, the causes of the failure are displayed.

    Important

    If you did not select Schema Migration for Migration Type, the fields in the source database that match those in the target database are migrated during full migration. The system does not check whether the table structures are consistent.

Incremental synchronization

After incremental synchronization starts, the data transmission service synchronizes the data that has been changed (added, modified, or deleted) in the source database to the corresponding tables in the target database. When services continuously write data to the source database, the data transmission service starts the incremental data pull module to pull incremental data from the source instance, parses and encapsulates the incremental data, and then stores the data. After that, the data transmission service starts the full migration.

After the full migration task is completed, the data transmission service starts the incremental data replay module to pull incremental data from the incremental data pull module. The incremental data is synchronized to the target database after being filtered, mapped, and converted. If you execute a DDL statement in the source database and cause a JDBCWriter exception, which then causes the data migration task to fail, the DDL statement and the Skip button are displayed. You can click Skip and confirm your operation.

Important

This operation may lead to data structure inconsistency between the source and target databases. Proceed with caution.

When you create a data migration task, we recommend that you enable alert monitoring to monitor the task status in real time. For more information, see Configure alert monitoring settings. If alert monitoring is disabled, when the incremental synchronization latency is less than or equal to 1 minute, the incremental synchronization status changes from Running to Monitoring by default.

image.png

Take note of the following items if alert monitoring is enabled:

  • When the incremental synchronization latency exceeds the specified alert threshold, the incremental synchronization status stays at Running and the system does not trigger any alerts.

  • When the incremental synchronization latency is less than or equal to the specified alert threshold, the incremental synchronization status changes from Running to Monitoring. After the incremental synchronization status changes to Monitoring, it will not change back to Running when the latency exceeds the specified alert threshold.

For a data migration task in the Running state, you can view its latency, current timestamp, and incremental synchronization performance in the incremental synchronization section. The latency is displayed in the following format: X seconds (updated Y seconds ago). Normally, Y is less than 20.

Note

When incremental synchronization is in the Failed or Paused state, you can click the Edit icon next to the current timestamp to modify the parameter configurations.

For a data migration task in the Paused or Failed state, you can enable the DDL/DML statistics feature to collect statistics on the database operations performed after this feature is enabled. You can also view the specific information about incremental synchronization objects and the incremental synchronization performance.

  • The Synchronization Object Statistics tab displays the statistics about table-level DML statements executed for each incremental synchronization object in the current task. The numbers displayed in the Change Sum, Delete, Insert, and Update fields in the section above the Incremental Synchronization Objects tab are the sums of the corresponding columns on this tab.

    统计数量

    If you enable the DDL/DML statistic feature and then disable it, the data transmission service will record the historical statistics.

  • The Incremental Synchronization Performance tab displays the following content:

    • Latency: the latency in synchronizing incremental data from the source database to the target database, in seconds.

    • Migration traffic: the traffic throughput of incremental data synchronization from the source database to the target database, in Kbit/s.

    • Average execution time: the average execution time of an SQL statement, in milliseconds.

    • Average commit time: the average commit time of a transaction, in milliseconds.

    • RPS: the number of records processed per second.

Full verification

After full migration and incremental synchronization are completed, the data transmission service automatically initiates a full verification task to verify the data tables in the source and target data sources.

The data transmission service also provides the corresponding APIs for you to initiate custom data verification tasks during incremental data synchronization.

On the Full Verification page, you can view the overall status, start time, end time, total amount of time consumed, estimated total number of rows, number of verified rows, RPS, and real-time traffic of the full verification task.

image.png

The Full Verification page contains the Verified Objects and Verification Performance tabs.

  • On the Verified Objects tab, you can view the verification progress and verification object list.

    image

    • You can view the names, source and target databases, full verification progress and results, and result summary of all migration objects.

    • You can filter migration objects by source database, target database, or verification result.

    • You can select View Completed Objects Only to view the basic information such as the names of objects for which schema migration has been completed, such as the object names.

    • You can choose Re-verify > Restart Full Verification to run a full verification again for all migration objects.

    • Observe the following notes for tables with inconsistent verification results:

      If you need to obtain the original inconsistent data to check or analyze the verification result, click Download inconsistent results to download the inconsistent data to your computer.

      Important

      You can download inconsistent data only when full verification is in the Completed state and data whose verification result is Inconsistent exists.

      If you need to reverify all data in the tables, choose Re-verify > Reverify Abnormal Table.

      If you only need to reverify inconsistent data, choose Re-verify > Verify Inconsistent Objects.

      Important

      Correction operations are not supported if the source database has no corresponding data.

  • On the Full Verification Performance tab, you can view the graphs of RPS and migration traffic of the source and target databases, performance benchmark, and other performance statistics to identify performance issues in a timely manner.

The data transmission service allows you to skip full verification for a task that is being verified or has failed verification. On the Full Verification page, click Skip Full Verification. In the dialog box that appears, click OK.

Important

If you skip full verification, you cannot restore the verification task for data comparison and correction. You can initiate full verification again only by cloning the current task. Proceed with caution.

After full verification is completed, you can click Go To Next Stage to start a forward switchover. After you enter the switchover process, you cannot rerun the current verification task to compare or correct data.

image.png

Forward switchover

Forward switchover is an abstract and standard process of traditional system cutover and does not involve the switchover of application connections. This process includes a series of tasks that are performed for application switchover in a data migration task. You must make sure that the entire forward switchover process is completed before the application connections are switched over to the target database.

Forward switchover is required for data migration. The data transmission service can ensure the completion of forward data migration in this process, and you can start the reverse incremental synchronization component based on your business needs. The forward switchover process involves the following operations:

  1. You must make sure that data migration is completed and wait until forward synchronization is completed.

  2. The data transmission service automatically supplements the check constraints and FOREIGN KEY constraints that are ignored in schema migration when you migrate data to an Oracle database or an Oracle tenant of OceanBase Database.

  3. The data transmission service automatically deletes the hidden columns and unique indexes on which the migration depends.

    This operation is performed only for data migration between an Oracle database and an OceanBase database or between OceanBase databases. For more information, see Hidden column mechanisms.

  4. You must migrate triggers, functions, and stored procedures in the source database that are not supported by the data transmission service to the target database.

  5. You must disable triggers and FOREIGN KEY constraints in the source database. This operation is required only when the data migration task involves reverse incremental synchronization.

The forward switchover process contains the following steps:

image.png

  1. Start forward switchover.

    In this step, you can start forward switchover, but no operation is performed in the background. If you make sure that data migration is completed, you can click Start Forward Switchover to start the process.

    Important

    Before you start forward switchover, make sure that data writing has stopped in the source database.

  2. Perform a switchover precheck.

    In this step, the data transmission service checks the following items:

    • Synchronization latency between the source and target databases. If the synchronization latency is within 15 seconds, this check item is passed.

    • Write privilege of the account in the source database. If the data migration task involves reverse incremental synchronization, the data transmission service additionally checks whether the account configured in the source database has the privilege to write data, to ensure that data can be properly written during reverse incremental synchronization.

    • Privilege of the account in the target database for incremental data read. If the data migration task involves reverse incremental synchronization, the data transmission service additionally checks whether the account configured in the target database has the privilege to read data. This ensures that data can be properly written to the target database during reverse incremental synchronization.

    • Incremental logs in the target database. If the data migration task involves reverse incremental synchronization, the data transmission service additionally checks whether the incremental logging configuration in the target database meets the log extraction requirements of reverse incremental synchronization.

    If the switchover precheck is passed, the data transmission service automatically performs the next step. If the precheck fails, you can click Retry or Skip.

    Important

    If you click Skip, data loss may occur in the target database, or the reverse incremental synchronization process may fail. Proceed with caution.

  3. Start the target store.

    Note

    This step is available only when the data migration task involves reverse incremental synchronization.

    If the precheck for forward switchover is passed, the data transmission service automatically starts incremental log pulling for the target database. This way, the data transmission service obtains DML and DDL operations in the target database and parse and save related log data to prepare for reverse incremental synchronization. This step takes 3 to 5 minutes.

  4. Confirm that data writing has stopped in the source database.

    In this step, the data transmission service checks whether business data is still being written to the source database. After you confirm that no new data is written to the source database, click OK to go to the next step.

  5. Confirm the data writing stop timestamp upon synchronization completion.

    In this step, the data transmission service checks whether the target database is synchronized to the data writing stop timestamp in the source database. If this step is in progress or fails and the synchronization is not completed after a long period, you can click Skip.

    Important

    If you choose to skip this step, data inconsistency may occur between the source and target. Proceed with caution.

  6. Stop forward synchronization.

    In this step, you can stop forward synchronization. After forward synchronization is stopped, any data changes in the source database will no longer be synchronized to the target database. If forward synchronization fails to be stopped, you can click Retry or Skip.

    Important

    You can click Skip only after you confirm that forward synchronization is completed in the background. Otherwise, data in the source database may be unexpectedly written to the target database. Proceed with caution.

  7. Process database objects.

    In this step, you can process the objects that are ignored in data migration or not supported by the data transmission service, to ensure normal operation of your business after the switchover to the target database. The following operations are supported:

    • Migrate database objects to the target database: You must migrate triggers, functions, and stored procedures in the source database that are not supported by the data transmission service to the target database. After you complete the migration, click Mark as Complete.

    • Disable triggers and FOREIGN KEY constraints in the source database: This operation is required only when the data migration task involves reverse incremental synchronization. It prevents data from being affected by triggers or FOREIGN KEY constraints, to avoid failures of reverse incremental synchronization. After you complete this operation, click Mark as Complete.

    • Supplement the objects ignored in schema migration to the target database: If the target is an Oracle database or an Oracle tenant of OceanBase Database, the data transmission service automatically supplements the objects that are ignored in schema migration, such as CHECK constraints and FOREIGN KEY constraints, to the target. In other cases, the preceding objects are automatically migrated during schema migration, and no extra operation is required.

    • Drop hidden columns and unique indexes added by the data transmission service: This operation is required only when you migrate data between an Oracle database and an OceanBase database or between OceanBase databases. The data transmission service automatically drops the hidden columns and unique indexes that are added to the target database to ensure data consistency. This operation runs automatically, and the amount of time required depends on the amount of data in the target database. You can click Skip to skip this operation, but you must manually perform it later. Proceed with caution. For more information, see Mechanisms for handling hidden columns.

  8. Start reverse incremental synchronization.

    Note

    This step is available only when the data migration task involves reverse incremental synchronization.

    In this step, you can start incremental synchronization for the target database to synchronize incremental DML or DDL operations from the target database to the source database in real time. The configuration of incremental synchronization is the same as that specified when the task was created. For more information, see Incremental synchronization of DDL operations.

Reverse incremental synchronization

Note

Reverse incremental synchronization is not supported when you migrate data from an Oracle database to an Oracle tenant of OceanBase Database.

For a Running data migration task, you can view its latency, current timestamp, and performance of reverse incremental synchronization in the Reverse Incremental Migration section. The latency is displayed in the following format: X seconds (updated Y seconds ago). Normally, Y is less than 20.

Note

When reverse incremental synchronization is in the Failed or Paused state, you can click the Edit icon next to the current timestamp to modify the parameter configurations.

For a data migration task in the Paused or Failed state, you can enable the DDL/DML statistics feature to collect statistics on the database operations performed after this feature is enabled. You can also view the specific information about the objects and performance of reverse incremental synchronization.

image.png

  • The Synchronization Object Statistics tab displays the statistics on table-level DML statements executed for each incremental synchronization object in the current task. The numbers displayed in the Change Sum, Delete, Insert, and Update fields in the section above the Synchronization Object Statistics tab are the sums of the statistics in the corresponding columns on this tab.

  • The Reverse Incremental Migration Performance tab displays the following content:

    • Latency: the latency in synchronizing incremental data from the target database to the source database, in seconds.

    • Migration traffic: the traffic throughput of incremental data synchronization from the target database to the source database, in Kbit/s.

    • Average execution time: the average execution time of an SQL statement, in milliseconds.

    • Average commit time: the average commit time of a transaction, in milliseconds.

    • RPS: the number of records processed per second.