By the Database Business Unit
More and more enterprise customers are choosing to migrate to the cloud to fully benefit from its convenience and high reliability. However, for many customers, the first issue encountered when migrating to the cloud is to smoothly complete the cloud migration without affecting the business. Many traditional migration solutions require businesses to stop services during migration, which affects business continuity.
Alibaba Cloud Data Transmission Service (DTS) has the ability to perform data migration without stopping services, meaning the cloud migration is completed while the source database provides the normal services of the business. In this article, we will discuss the principles and process of cloud migration, as well as explain the benefits of using Alibaba Cloud's DTS.
A variety of database engines have emerged in the market to meet different user scenarios. To meet user requirements for importing data, database vendors typically have their own data migration tools. For example: mysqldump for MySQL and exp/imp for Oracle.
However, traditional migration tools realize the migration of existing data through data export and import. To ensure the integrity of the migrated data, the source database service is usually stopped and the amount of time for which the service remains stopped depends on how long it takes to migrate the existing data. This can be several hours, and results in an extremely high migration cost for users.
Alibaba Cloud provides a migration service called Data Transmission Service (DTS), to reduce the migration cost to users. DTS supports data transmission between data sources such as relational databases, NoSQL, and Big Data (OLAP). It is a data transmission service that integrates data migration, data log real-time subscription, and real-time data synchronization.
DTS provides data migration without stopping services, enabling the smooth completion of the migration of database data without stopping the services related to the source database. Users can apply data migration in various scenarios:
In addition to data migration, DTS helps complete various business scenarios such as data disaster backup, multi-idc-live, real-time synchronization of business data with the data warehouse, asynchronous message notification, and cache update.
DTS supports dozens of data sources including Oracle, MySQL, SQLServer, PostgreSQL, DB2, Redis, and MongoDB. Data transmission services have been provided to tens of thousands of enterprise customers.
The general process of using DTS to perform database migration without stopping services is divided into the following phases:
Throughout the entire data migration period, the source database normally provides services, but points the business to the target database during the business switching phase.
In this section, we introduce the basic technical principles of each migration phase.
During the schema migration phase, DTS generates a schema on the target instance based on the schema definition (e.g., tables and views) of the source instance. If the source instance and the target instance are heterogeneous data sources, DTS needs to convert data types and SQL syntax according to the schema syntax of the target instance. For objects that are incompatible with the target data sources, you can manually correct them in DTS Console – Migration Details before synchronizing them with the target instance.
During the full migration phase, DTS migrates historical data from the source database to the target instance. The table fragmentation technology allows simultaneous execution of fragmentation in the table, maximizing the migration speed.
DTS also supports resumable upload. When the migration is interrupted due to a network issue – source instance or target instance – DTS can resume data transmission from the breakpoint after the issue has been resolved, which greatly reduces the migration workload.
During the incremental migration phase, DTS captures data changes on the source database in real time and synchronizes it with the target database in real time.
With the deadlock algorithm, DTS can implement a transaction level-based concurrent write strategy to effectively improve the synchronization performance while ensuring the transaction sequentiality.
With DTS, users can completely configure the entire task in just 3 steps. We demonstrate below the use of DTS with a case involving migrating data from another cloud vendor's RDS to Alibaba Cloud RDS.
Before configuring DTS tasks, you need to enable the external network address of the other cloud vendor's RDS instances.
DTS supports access to another cloud vendor's RDS instances through hardware devices such as private lines and VPN gateways, in addition to access through public networks.
Open the Alibaba Cloud DTS console and click Create Migration Task in the upper-right corner to start task configuration.
Step 1: Configure the connection information for the source MySQL instance and the target RDS for the MySQL instance.
This step involves setting the migration task name and configuring the connection information for the source and target instances involved in the migration. Details are as follows:
You will need to provide source instance connection information, including:
You will also need to configure target RDS for MySQL instance connection information. The target instance configures the instance connection information for Alibaba Cloud RDS. The specific configuration information is as follows:
Step 2: Configure the migration object and migration type
DTS supports schema migration, full migration, and incremental migration. To migrate without stopping services, select: Schema Migration + Full Migration + Incremental Migration.
Select the object to be migrated. The migration object can be a database, a table, or a column. After the migration object and migration type are configured, the pre-check for the migration task is started.
Step 3: Migration pre-check
A pre-check is required before migration tasks can start. The pre-check needs to be passed successfully in order for the migration to start.
After passing the pre-check, you can start the migration task. After the task starts, you can view the specific migration status and progress in the task list.
This completes the configuration of the task for migrating data from a MySQL database instance on another cloud vendor to an Alibaba Cloud RDS for MySQL instance.
We introduced the basic principles of Alibaba Cloud Data Transmission Service (DTS) and the database migration process. You have learned that DTS is a convenient, safe, and efficient tool for database migration, and you can easily get started with Migration to Cloud based on the ability of DTS to perform migration without stopping services.
In addition to the function of data migration, DTS also supports real-time subscription of database logs and real-time synchronization of data. With these functions, users can realize business scenarios such as data disaster backup, multi-idc-live, real-time synchronization of business data to the data warehouse, asynchronous message notification, and cache update. Due to space limitations, no detailed introduction is given here.
10 posts | 2 followers
FollowAlibaba Clouder - February 22, 2021
Alibaba Clouder - July 9, 2020
Alibaba Clouder - February 26, 2019
5544031433091282 - July 1, 2022
Alibaba Clouder - November 6, 2020
ApsaraDB - October 29, 2024
10 posts | 2 followers
Follow
Data Transmission Service
Supports data migration and data synchronization between data engines, such as relational database, NoSQL and OLAP
Learn More
Cloud Migration Solution
Secure and easy solutions for moving you workloads to the cloud
Learn More
Database Migration Solution
Migrating to fully managed cloud databases brings a host of benefits including scalability, reliability, and cost efficiency.
Learn More
Tair
Tair is a Redis-compatible in-memory database service that provides a variety of data structures and enterprise-level capabilities.
Learn MoreMore Posts by Alibaba Cloud Product Launch
Raja_KT March 9, 2019 at 3:43 am
Good one. Hope you can explain more on " deadlock algorithm"