OceanBase provides the data transmission service to support data exchange between a homogeneous or heterogeneous relational database management system (RDBMS) and OceanBase Database. The data transmission service provides the capabilities for online migration of existing data and real-time synchronization of incremental data.
Features
The data transmission service provides a visualized and centralized management platform. You can migrate data in real time with simple configurations. The data transmission service aims to help you achieve real-time data migration and synchronization from homogeneous or heterogeneous databases to OceanBase Database at a low cost and low risk.
Data migration: A data migration project is a one-time task. After a data migration project is completed, resources occupied by the project are released. You can create a data migration project to migrate data between homogeneous or heterogeneous data sources in business scenarios such as database upgrades, cross-instance data migration, database splitting, and database scaling.
Data migration projects are the basic unit of the data migration feature. When you create a data migration project, you can specify the migration level, which ranges from table to database. For more information, see Data migration.
NoteData migration projects support only the pay-as-you-go billing method. You can use data migration projects for free until further notice. For more information, see Billing for data transmission.
Data synchronization: Data synchronization is a continuous operation. After a data synchronization project is created, data will be synchronized continuously to ensure consistency between the source and destination databases as well as the real-time flow of data of key business systems. You can create a data synchronization project to synchronize data between data sources in real time in business scenarios, such as multi-site high availability, remote disaster recovery, data aggregation, and real-time data warehousing. For more information, see Data synchronization.
NoteData synchronization projects support only the subscription billing method. For more information, see Billing for data transmission.
Supported migration types
Abbreviations
Instance type | Abbreviation |
ApsaraDB RDS instance | RDS |
PolarDB for MySQL instance | PolarDB |
Self-managed database in a virtual private cloud (VPC) | VPC |
Database gateway | DG |
Self-managed database with a public IP address | Public network |
MySQL tenant of OceanBase Database | OB_MySQL |
Oracle tenant of OceanBase Database | OB_Oracle |
Data migration
At present, migration from an Oracle database to a MySQL tenant of OceanBase Database and from a MySQL database to an Oracle tenant of OceanBase Database is not supported.
For more information about DDL operations supported in incremental synchronization, see topics in the Supported DDL operations for synchronization and limitations chapter.
Data migration project | Schema migration | Full migration | Incremental synchronization of DML operations | Incremental synchronization of DDL operations | Full verification | Reverse incremental migration | Table without a primary key |
MySQL (RDS/PolarDB/VPC/public network/DG) -> OB_MySQL (cluster instance) | Supported | Supported | Supported | Supported | Supported | Supported | Supported |
MySQL (RDS/PolarDB/VPC/public network/DG) -> OB_MySQL (tenant instance) | Supported | Supported | Supported | Supported | Supported | Supported | Supported |
OB_MySQL (cluster instance) -> MySQL (RDS/PolarDB/VPC/public network/DG) | Supported | Supported | Supported | Supported | Supported | Supported | Not supported |
Oracle (public network/VPC/DG) -> OB_Oracle (cluster instance) | Supported | Supported | Supported | Supported | Supported | Supported | Supported |
OB_MySQL (cluster instance/VPC) -> OB_MySQL (cluster instance) | Supported | Supported | Supported | Supported | Supported | Supported | Supported |
OB_Oracle (cluster instance/VPC) -> OB_Oracle (cluster instance) | Supported | Supported | Supported | Supported | Supported | Supported | Supported |
OB_MySQL (cluster instance) -> OB_MySQL (VPC) | Supported | Supported | Supported | Supported | Supported | Supported | Supported |
OB_Oracle (cluster instance) -> OB_Oracle (VPC) | Supported | Supported | Supported | Supported | Supported | Supported | Supported |
PolarDB-X 1.0 -> OB_MySQL (cluster instance) | Not supported | Supported | Supported | Not supported | Supported | Not supported | Supported |
TiDB -> OB_MySQL (cluster instance) | Supported | Supported | Supported | Not supported | Supported | Supported | Not supported |
OB_Oracle (cluster instance) -> MySQL (RDS/PolarDB/VPC/public network/DG) | Not supported | Not supported | Supported | Not supported | Not supported | Not supported | Not supported |
PolarDB-X 2.0 -> OB_MySQL (cluster instance) | Supported | Supported | Supported | Not supported | Supported | Supported | Supported |
PostgreSQL (RDS instance) -> OB_Oracle (cluster instance) | Supported | Supported | Supported | Not supported | Supported | Supported | Supported |
Migration types
Migration type | Description |
Schema migration | The definitions of data objects, such as tables, indexes, constraints, comments, and views, are migrated from the source database to the destination database. Temporary tables are automatically filtered out. If the source database is not an OceanBase database, the data transmission service performs format conversion and encapsulation based on the syntax definition and standard of the type of the destination tenant of OceanBase Database and then replicates the data to the destination database. |
Full migration | The existing data is migrated from tables in the source database to the corresponding tables in the destination database. On the Full Migration page, you can filter objects by source and destination databases, or select View Objects with Errors to filter out objects that hinder the overall migration progress. You can view Tables, Table Indexes, and Full Load Performance. The status of a full migration task changes to Completed only after the table objects and table indexes are migrated. |
Incremental synchronization | After incremental synchronization starts, the data transmission service synchronizes the data that has been changed (added, modified, or deleted) in the source database to the corresponding tables in the destination database. When services continuously write data to the source database, the data transmission service starts the incremental data pull module to pull incremental data from the source instance, parses and encapsulates the incremental data, and then stores the data. After that, the data transmission service starts the full data migration. After the full data migration task is completed, the data transmission service starts the incremental data replay module to pull incremental data from the incremental data pull module. The incremental data is synchronized to the destination instance after being filtered, mapped, and converted. |
Full verification | After the full data migration and incremental data migration are completed, the data transmission service automatically initiates a full data verification task to verify the data tables in the source and destination data sources. You can also initiate custom data verification tasks in the incremental data synchronization process. On the Full Verification page, you can view the overall status, start time, end time, total consumed time, estimated total number of rows, number of migrated rows, real-time traffic, and RPS of the full verification task. |
Forward switchover | Forward switchover is an abstract and standard process of traditional system cutover and does not involve the switchover of application connections. This process includes a series of tasks that are performed for application switchover in a data migration project. You must make sure that the entire forward switchover process is completed before the application connections are switched over to the destination database. Forward switchover is required for data migration. The data transmission service can ensure the completion of forward data migration in this process, and you can start the reverse incremental migration component based on your business needs. The forward switchover process involves the following operations:
|
Reverse incremental migration | In business cutover scenarios, after the migration is completed, you can start an incremental synchronization project in a reverse direction before the business database switchover. A data synchronization project synchronizes data from the destination database to the source database. In this way, data changes made in the destination database after the switchover can be applied to the source business database in real time. |
Data synchronization
For more information about DDL operations supported in incremental synchronization, see corresponding topics about data synchronization projects.
Data synchronization project | Schema synchronization | Full synchronization | Incremental synchronization of DML operations | Incremental synchronization of DDL operations | Table without a primary key |
OB_MySQL (cluster instance) -> OB_MySQL (cluster instance/VPC) | Supported | Supported | Supported | Supported | Not supported |
OB_MySQL (VPC) -> OB_MySQL (cluster instance) | Supported | Supported | Supported | Supported | Not supported |
OB_MySQL (cluster instance) -> ADB (Alibaba Cloud instance) | Supported | Supported | Supported | Supported | Not supported |
OB_MySQL (cluster instance) -> DataHub (public network/VPC/Alibaba Cloud instance) | Supported | Supported | Supported | Supported | Not supported |
OB_Oracle (cluster instance) -> DataHub (public network/VPC/Alibaba Cloud instance) | Supported | Supported | Supported | Supported | Not supported |
OceanBase (cluster instance) -> Kafka (public network/VPC/Alibaba Cloud instance) | Not supported | Supported | Supported | Supported | Not supported |
OceanBase (cluster instance) -> RocketMQ (public network/VPC/Alibaba Cloud instance) | Not supported | Supported | Supported | Not supported | Not supported |
PostgreSQL (RDS instance) -> OB_Oracle (cluster instance) | Supported | Supported | Supported | Not supported | Supported |
Supported database versions
Feature | OceanBase Database version | Other database version |
Data migration | V1.4.79, V2.2.30, V2.2.52, V2.2.76, V2.2.77, V3.1.x, V3.2.x, V4.0.0, V4.1.0, V4.2.0, V4.2.1, V4.2.2, V4.2.3, and V4.3.0 |
|
Data synchronization | V2.2.30, V2.2.52, V2.2.76, V2.2.77, V3.1.x, V3.2.x, V4.0.0, V4.1.0, V4.2.0, V4.2.1, V4.2.2, V4.2.3, and V4.3.0 |
|