×
Community Blog DTS Data Synchronization

DTS Data Synchronization

Data Transmission Service supports data migration and data synchronization between data engines, such as relational database, NoSQL, data warehouses, and OLAP.

By Jessie Angelica,Solution Architect Intern

Hands-On Tutorial : DTS for Real-Time Data Synchronization between RDS MySQL

Data Transmission Service supports data migration and data synchronization between data engines, such as relational database, NoSQL, data warehouses, and OLAP. With Data Transmission Service (DTS), you can implement the following business scenarios: data migration, remote real time data backup, real time data integration and cache refresh. The Data Transmission Service (DTS) supports incremental data transfers from on-premises to the cloud and from cloud to cloud, and helps you migrate data with virtually no downtime. 

This blog focused on Data Synchronization which used to synchronize data in real-time between two data stores. Usually, this is used with Online Transaction Processing to Online Analytical Processing (OLTP to OLAP.) Typical scenarios include active geo-redundancy, geo-disaster recovery, zone-disaster recovery, cross-border data synchronization, cloud business intelligence (BI) systems, and real-time data warehousing. It can be Schema synchronization, Full data synchronization, and Incremental data synchronization.

STEP 1 Prepare the RDS data

1). Go to the ApsaraDB RDS console
2). Create and differentiate the two instances by renaming them to RDS_A and RDS_B.
3). Click Configure Whitelist in Basic Information side and configure the whitelist to 0.0.0.0/0
4). Click Apply for Internet Public Endpoint in Database Connection
5). Create Account in RDS_A by filling the Database Account name, password, and choose Privileged Account and Click Create.
6). Then, create the same things on RDS_B.
7). Create Database in RDS_A by filling the database name and choose utf8

STEP 2 Prepare the ECS

1). Log on to your ECS. Enter the following command and view the internal network address of ECS
2). Enter command "vim /etc/hosts" and Replace YOUR-PRIVATE-IP and YOUR-HOST-NAME with your own
YOUR-PRIVATE-IP YOUR-HOST-NAME
download

3). Update the apt installation source:
download

4). Install the dependency package of the MySQL client and Python dependency package of MySQL
download

download

5). Run the following command to connect to the database of the RDSA instance. Remember to replace YOUR-RDS-INTERNET-ADDRESS with the public IP address of your RDS_A instance.
mysql -uYOUR-DATABASE -hYOUR-RDS-INTERNET-ADDRESS -pYOUR-PASSWORD

6). Switch to your database and create a table
download

7). Run the "vim data.py" and copy the following content to the file used to add data to your database.
download

8). Execute the data.py script:
download

9). Run the "vim select.py" command, and copy the following content to the file used to query the data volume of the info table in the database of the RDS_A instance.
download

10). Execute the following command, then the info table in the RDS_A currently has a total of 1,000 data records.
download

PART 3 Create a DTS data synchronization instance

1). Go to Data Transmission Service.
2). Create task in the data synchronization task. Configure the source database and target database settings according this following figure.
download

3). Select your database and click Next
download

4). Click Next: Save Task Settings and Precheck.
download

5). Click Purchase, then Click Buy and Start
download

After initialization is complete, the synchronization link enters the Synchronizing state, and the synchronization link between the RDS_A and the RDS_B is fully established.
download

PART 4 Verify the data synchronization results

1). Return to the CLI terminal and enters this following command.
mysql -uYOUR-DATABASE-NAME -hYOUR-RDS_B-INTERNET-ADDRESS -pYOUR-PASSWORD
download

The number of data records is 1,000, which indicates that data is successfully synchronized from RDS_A.

2). Run the following command to execute the data.py script, which inserts 1,000 data records to RDS_A:
download

After the data is successfully inserted, RDS_A has a total of 2,000 data records.

3). Run the following command to query the data volume again:
download
The number of data records in RDS_B now changes to 2,000, which indicates that your real-time synchronization task is successfully implemented.

4). Return to the DTS console and release the real-time synchronization task by click Delete Task
download

5). Return to the CLI terminal and run the following command to execute the data.py script, which inserts 1,000 more data records to RDS_A:
download

download
After the data is successfully inserted, RDS_A has a total of 3,000 data records, but the number of data records in RDS_B is still 2,000. It means the synchronization task is already deleted and data is no longer synchronized between two RDS instances.

0 1 0
Share on

Alibaba Cloud Indonesia

99 posts | 15 followers

You may also like

Comments