This topic describes how to use Data Transmission Service (DTS) to migrate data from Amazon RDS for SQL Server to RDS for SQL Server.
Background information
This topic provides two methods for migrating Amazon RDS for SQL Server to the cloud. You can select a method based on your actual needs.
-
Use the RDS one-stop cloud migration feature
The RDS console provides a one-stop cloud migration feature to help you migrate data from Amazon RDS for SQL Server to Alibaba Cloud RDS for SQL Server quickly and easily.
NoteBy default, this method includes schema migration, full data migration, and incremental migration tasks.
-
Use DTS to migrate to the cloud
DTS supports cloud migration through its data migration feature. Compared with the RDS one-stop cloud migration feature, DTS lets you select specific migration task types—such as schema migration, full data migration, and incremental migration—and configure more advanced parameters.
Prerequisites
-
You have connected the Amazon RDS for SQL Server database to Alibaba Cloud so that DTS can connect to the source database. For more information, see Connect databases from the AWS platform to Alibaba Cloud.
-
You have created a target RDS SQL Server instance with storage capacity greater than that of the source database. For more information, see Quickly create and use an RDS SQL Server instance.
Important notes
DTS does not migrate foreign keys from the source database to the destination database. Therefore, cascade and delete operations in the source database are not migrated to the destination database.
Type | Description |
Source database limits |
|
Other limits |
|
Billing
|
Migration type |
Instance configuration fee |
Internet traffic fee |
|
Schema migration and full data migration |
Free of charge. |
When the Access Method parameter of the destination database is set to Public IP Address, you are charged for Internet traffic. For more information, see Billing overview. |
|
Incremental data migration |
Charged. For more information, see Billing overview. |
SQL operations supported for incremental migration
|
Operation type |
SQL statement |
|
DML |
INSERT, UPDATE, DELETE |
|
DDL |
Note
|
Database account permissions
|
Database |
Schema migration |
Full migration |
Incremental migration |
How to create and grant permissions |
|
Amazon RDS for SQL Server |
SELECT permission |
SELECT permission |
db_owner permission Note
RDS for SQL Server's privileged account meets the requirements and supports enabling database-level CDC for stored procedures. |
For specific steps, contact AWS. |
|
RDS for SQL Server instance |
Read and write permission |
Create standard, privileged, and global read-only accounts and Modify account permissions. |
||
Procedure
Use the RDS one-stop cloud migration feature
-
Go to the Data Migration tab of the target RDS for SQL Server instance.
-
Go to the RDS Instance List.
-
In the upper part of the page, select the region where the target RDS for SQL Server instance resides.
-
Click the ID of the target RDS for SQL Server instance.
-
In the navigation pane on the left, click Data Migration and Synchronization.
-
Click the Data Migration tab.
-
-
Click One-Stop Cloud Migration.
-
Configure source and destination database information.
Category
Configuration
Description
None
Task Name
The system generates a task name by default. We recommend specifying a meaningful name (not required to be unique) for easier identification later.
Source Database
Database Type
By default, this is set to SQL Server. You do not need to select it.
Type
Select AWS.
Access Method
Select Public IP Address or Express Connect, VPN Gateway, or Smart Access Gateway based on how Amazon RDS for SQL Server connects to Alibaba Cloud. This example selects Express Connect, VPN Gateway, or Smart Access Gateway.
Instance Region
-
When the Access Method is set to Public IP Address, select the region where the RDS for SQL Server instance resides.
NoteIf the region where Amazon RDS for SQL Server is located does not appear in the options, you can select a region closest to that database.
-
When the Access Method is set to Express Connect, VPN Gateway, or Smart Access Gateway, select the region where the Alibaba Cloud Virtual Private Cloud (VPC) for the Amazon RDS for SQL Server connection is located.
Connected VPC
Select the ID of the Alibaba Cloud Virtual Private Cloud (VPC) connected to Amazon RDS for SQL Server.
NoteThis option appears only when you select Access Method as Express Connect, VPN Gateway, or Smart Access Gateway.
Domain Name or IP
Enter the domain name or IP address of Amazon RDS for SQL Server.
NoteWe recommend entering the domain name.
Port Number
Enter the service port of Amazon RDS for SQL Server.
Database Account
Enter the database account for Amazon RDS for SQL Server. For more information, see Permission requirements for database accounts.
Database Password
Enter the password for the database account.
Encryption
Specifies whether to encrypt the connection to the source database. Select Non-encrypted or SSL-encrypted based on your business requirements.
If SSL encryption is disabled for the source database, select Non-encrypted.
If SSL encryption is enabled for the source database, select SSL-encrypted. By default, DTS trusts the server certificate.
Destination Database
Database Type
By default, this is set to SQL Server. You do not need to select it.
Access Method
By default, this is set to Alibaba Cloud Instance. You do not need to select it.
Instance Region
This is fixed to the region of the current RDS for SQL Server instance and cannot be changed.
Instance ID
This is fixed to the ID of the current RDS for SQL Server instance and cannot be changed.
Database Account
Enter the database account of the current RDS for SQL Server instance. For permission requirements, see Database account permissions.
Database Password
Enter the password for the database account.
Encryption
Specifies whether to encrypt the connection to the destination database. Select Non-encrypted or SSL-encrypted based on your business requirements.
If SSL encryption is disabled for the destination database, select Non-encrypted.
If SSL encryption is enabled for the destination database, select SSL-encrypted. By default, DTS trusts the server certificate.
-
-
After configuration, click Test Connectivity and Proceed at the bottom of the page.
Ensure that the DTS server IP address shown in the pop-up dialog box is added to the whitelist security settings of your Amazon RDS for SQL Server. Then click Test Connectivity.
ImportantAdding the public IP address range of DTS servers may pose security risks. Using this product means you understand and accept these risks. You must implement basic security measures, such as strengthening account password strength, limiting open ports for each CIDR block, using authentication for internal API communication, and regularly reviewing and restricting unnecessary CIDR blocks. For more information, see Add DTS server IP addresses to the whitelist.
-
Select a migration plan.
The system evaluates your source database type and presents two cloud migration plans.
-
To select the Full and Incremental Data Migration plan, click Configure Objects at the bottom of the page and continue to the next step.
-
To select the Full and Incremental Backup-Based Data Migration plan, no further action is needed.
NoteYou can click View Migration Documentation at the bottom of the page to view the corresponding cloud migration steps.
-
-
On the Configure Objects page, configure the objects to migrate.
Configuration
Description
Method to Migrate Triggers in Source Database
During incremental data migration, triggers may migrate to the destination database too early, causing data inconsistency between the source and destination databases. Choose a trigger migration method based on your needs. We recommend selecting Manual Migration. For more information, see Configure trigger migration or synchronization.
NoteIf your objects do not include triggers, skip this step (keep the default setting).
Source Objects
In the Source Objects box, click the objects to migrate, and then click
to move them to the Selected Objects box.NoteYou can select databases, tables, or columns as migration objects. If you select tables or columns, other objects (such as views, triggers, and stored procedures) are not migrated to the destination database.
Selected Objects
-
To change the name of a migration object in the destination instance, right-click the object in the Selected Objects box and edit it. For instructions, see Map database, table, and column names.
-
To remove a selected migration object, click the object in the Selected Objects box, then click
to move it back to the Source Objects box.
Note-
Using the object name mapping feature may cause migration failures for objects that depend on the mapped object.
-
To filter data using a WHERE clause, right-click the table to migrate in the Selected Objects box, then set the filter condition in the dialog box. For instructions, see Set filter conditions.
-
To select SQL operations to migrate at the database or table level, right-click the object to migrate in the Selected Objects box and select the required SQL operations in the dialog box.
-
-
Optional: Configure advanced settings.
You can expand advanced settings to configure advanced parameters.
Configuration
Description
Enable Throttling for Full Data Migration
During full migration, DTS consumes read and write resources on the source and destination databases, which may increase the database load. If required, you can enable throttling for the full migration task. You can set Queries per second (QPS) to the source database, RPS of Full Data Migration, and Data migration speed for full migration (MB/s) to reduce the load on the destination database.
Enable Throttling for Incremental Data Migration
If required, you can also choose to set speed limits for the incremental migration task. You can set RPS of Incremental Data Migration and Data migration speed for incremental migration (MB/s) to reduce the load on the destination database.
Data Verification Mode
Full data validation consumes some read resources from the database. If you select Full Data Verification, you must also set Maximum number of rows of data read per second by full verification (RPS) and Maximum amount of data read per second by full verification (MBps) to limit the full validation rate (rows and data volume per second) and reduce database load.
NoteA value of 0 means no limit. If both Maximum number of rows of data read per second by full verification (RPS) and Maximum amount of data read per second by full verification (MBps) are 0, there is no rate limit.
-
Save the task and run a precheck.
-
To view the parameters for configuring this instance when you call the API operation, move the pointer over the Next: Save Task Settings and Precheck button and click Preview OpenAPI parameters in the bubble that appears.
-
If you do not need to view or have finished viewing the API parameters, click Next: Save Task Settings and Precheck at the bottom of the page.
Note-
Before the migration task starts, DTS performs a precheck. The task starts only after it passes the precheck.
-
If the precheck fails, click View Details next to the failed check item, fix the issue based on the prompt, and then run the precheck again.
-
If a warning is reported during the precheck:
-
For check items that cannot be ignored, click View Details next to the failed item, fix the issue based on the prompt, and then run the precheck again.
-
For check items that can be ignored, you can click Confirm Alert Details, Ignore, OK, and Precheck Again to skip the alert item and run the precheck again. If you choose to ignore a warning, it may cause issues such as data inconsistency and pose risks to your business.
-
-
-
When the Success Rate is 100%, click Next: Purchase Instance.
-
Purchase the instance.
-
On the Purchase page, select the link specification for the data migration instance. For more information, see the following table.
Category
Parameter
Description
New Instance Class
Resource Group Settings
Select the resource group to which the instance belongs. The default value is default resource group. For more information, see What is Resource Management?
Instance Class
DTS provides migration specifications with different performance levels. The link specification affects the migration speed. You can select a specification based on your business scenario. For more information, see Data migration link specifications.
-
After the configuration is complete, read and select Data Transmission Service (Pay-as-you-go) Service Terms.
-
Click Buy and Start. In the OK dialog box that appears, click OK.
You can view the progress of the migration task on the Data Migration Tasks list page.
Note-
If the migration task does not include incremental migration, it stops automatically after the full migration is complete. After the task stops, its Status changes to Completed.
-
If the migration task includes incremental migration, it does not stop automatically. The incremental migration task continues to run. While the incremental migration task is running, the Status of the task is Running.
-
-
Use DTS to migrate to the cloud
-
Go to the migration task list page for the destination region.
-
Log on to the Data Transmission Service (DTS) console.
-
In the navigation pane on the left, click Data Migration.
-
In the upper-left corner of the page, select the region where the instance resides.
-
-
Click Create Task to navigate to the task configuration page.
-
Configure the source and destination databases.
Category
Configuration
Description
None
Task Name
DTS automatically generates a task name. We recommend that you specify a descriptive name for easy identification. The name does not need to be unique.
Source Database
Select Existing Connection
You can choose whether to use an existing instance based on your needs.
-
If you use an existing instance, the database information below is filled in automatically. You do not need to enter it again.
-
If you do not use an existing instance, you must enter the database information below.
NoteYou can register databases in DTS on the Database Connections page or the new configuration page. For more information, see Data Connection Management.
Database Type
Select SQL Server.
Access Method
Select Public IP Address or Express Connect, VPN Gateway, or Smart Access Gateway based on how Amazon RDS for SQL Server connects to Alibaba Cloud. This example selects Express Connect, VPN Gateway, or Smart Access Gateway.
Instance Region
-
When the Access Method is set to Public IP Address, select the region where the RDS for SQL Server instance resides.
NoteIf the region where Amazon RDS for SQL Server is located does not appear in the options, you can select a region closest to that database.
-
When the Access Method is set to Express Connect, VPN Gateway, or Smart Access Gateway, select the region where the Alibaba Cloud Virtual Private Cloud (VPC) for the Amazon RDS for SQL Server connection is located.
Replicate Data Across Alibaba Cloud Accounts
In this example, a database instance under the current Alibaba Cloud account is used. Select No.
NoteThis option appears only when you select Access Method as Express Connect, VPN Gateway, or Smart Access Gateway.
Connected VPC
Select the ID of the Alibaba Cloud Virtual Private Cloud (VPC) connected to Amazon RDS for SQL Server.
NoteThis option appears only when you select Access Method as Express Connect, VPN Gateway, or Smart Access Gateway.
Domain Name or IP
Enter the domain name or IP address of Amazon RDS for SQL Server.
NoteWe recommend entering the domain name.
Port Number
Enter the service port for Amazon RDS for SQL Server.
Database Account
Enter the database account for Amazon RDS for SQL Server. For permission requirements, see Permission requirements for the database account.
Database Password
Enter the password for the database account.
Encryption
Specifies whether to encrypt the connection to the source database. Select Non-encrypted or SSL-encrypted based on your business requirements.
If SSL encryption is disabled for the source database, select Non-encrypted.
If SSL encryption is enabled for the source database, select SSL-encrypted. By default, DTS trusts the server certificate.
Destination Database
Select Existing Connection
You can choose whether to use an existing instance based on your needs.
-
If you use an existing instance, the database information below is filled in automatically. You do not need to enter it again.
-
If you do not use an existing instance, you must enter the database information below.
NoteYou can register databases in DTS on the Database Connections page or the new configuration page. For more information, see Data Connection Management.
Database Type
Select SQL Server.
Access Method
Select Alibaba Cloud Instance.
Instance Region
Select the region where the destination RDS for SQL Server instance resides.
Instance ID
Select the ID of the destination RDS for SQL Server instance.
Database Account
Enter the database account of the destination RDS for SQL Server instance. For permission requirements, see Database account permissions.
Database Password
Enter the password for the database account.
Encryption
Specifies whether to encrypt the connection to the destination database. Select Non-encrypted or SSL-encrypted based on your business requirements.
If SSL encryption is disabled for the destination database, select Non-encrypted.
If SSL encryption is enabled for the destination database, select SSL-encrypted. By default, DTS trusts the server certificate.
-
-
After configuration, click Test Connectivity and Proceed at the bottom of the page.
Ensure that the DTS server IP address shown in the pop-up dialog box is added to the whitelist security settings of your Amazon RDS for SQL Server. Then click Test Connectivity.
ImportantAdding the public IP address range of DTS servers may pose security risks. Using this product means you understand and accept these risks. You must implement basic security measures, such as strengthening account password strength, limiting open ports for each CIDR block, using authentication for internal API communication, and regularly reviewing and restricting unnecessary CIDR blocks. For more information, see Add DTS server IP addresses to the whitelist.
-
Configure the task objects.
-
On the Configure Objects page, configure the objects that you want to migrate.
Configuration
Description
Migration Types
-
If you only need to perform a full migration, select both Schema Migration and Full Data Migration.
-
To perform a migration with no downtime, select Schema Migration, Full Data Migration, and Incremental Data Migration.
Note-
If you do not select Schema Migration, you must ensure that a database and tables to receive the data exist in the destination database. You can also use the object name mapping feature in the Selected Objects box as needed.
-
If you do not select Incremental Data Migration, do not write new data to the source instance during data migration to ensure data consistency.
Method to Migrate Triggers in Source Database
During incremental data migration, triggers may migrate to the destination database too early, causing data inconsistency between the source and destination databases. Choose a trigger migration method based on your needs. We recommend selecting Manual Migration. For more information, see Configure trigger migration or synchronization.
Note-
You can configure this only when you select both Migration Types options: Schema Migration and Incremental Data Migration.
-
If your objects do not include triggers, skip this step (keep the default setting).
SQL Server Incremental Synchronization Mode
Select Polling and querying CDC instances for incremental synchronization.
NoteThis option appears only when Migration Types includes Incremental Data Migration.
The maximum number of tables for which CDC is enabled that DTS supports.
In this example, keep the default value.
Processing Mode of Conflicting Tables
-
Precheck and Report Errors: Checks whether tables with the same names exist in the destination database. If no tables with the same names exist, the precheck is passed. If tables with the same names exist, an error is reported during the precheck, and the data migration task does not start.
NoteIf a table in the destination database has the same name but cannot be easily deleted or renamed, you can change the name of the table in the destination database. For more information, see Object name mapping.
-
Ignore Errors and Proceed: Skips the check for tables with the same names.
WarningSelecting Ignore Errors and Proceed may cause data inconsistency and business risks. For example:
-
If the table schemas are consistent and a record in the destination database has the same primary key value as a record in the source database:
-
During full migration, DTS keeps the record in the destination database. The record from the source database is not migrated.
-
During incremental migration, DTS does not keep the record in the destination database. The record from the source database overwrites the record in the destination database.
-
-
If the table schemas are inconsistent, only some columns of data may be migrated, or the migration may fail. Proceed with caution.
-
Capitalization of Object Names in Destination Instance
You can configure the case sensitivity policy for the names of migrated objects, such as databases, tables, and columns, in the destination instance. By default, DTS default policy is selected. You can also choose to keep the case sensitivity consistent with the default policy of the source or destination database. For more information, see Case sensitivity of object names in the destination database.
Source Objects
In the Source Objects box, click the objects to migrate, and then click
to move them to the Selected Objects box.NoteYou can select databases, tables, or columns as migration objects. If you select tables or columns, other objects (such as views, triggers, and stored procedures) are not migrated to the destination database.
Selected Objects
-
To change the name of a migration object in the destination instance, right-click the object in the Selected Objects box and edit it. For instructions, see Map database, table, and column names.
-
To remove a selected migration object, click the object in the Selected Objects box, then click
to move it back to the Source Objects box.
Note-
Using the object name mapping feature may cause migration failures for objects that depend on the mapped object.
-
To filter data using a WHERE clause, right-click the table to migrate in the Selected Objects box, then set the filter condition in the dialog box. For instructions, see Set filter conditions.
-
To select SQL operations to migrate at the database or table level, right-click the object to migrate in the Selected Objects box and select the required SQL operations in the dialog box.
-
-
Click Next: Advanced Settings to configure advanced parameters.
Configuration
Description
Dedicated Cluster for Task Scheduling
By default, DTS schedules tasks on a shared cluster. You do not need to select one. If you want more stable tasks, you can purchase a dedicated cluster to run DTS migration tasks.
Retry Time for Failed Connections
After the migration task starts, if the connection to the source or destination database fails, DTS reports an error and immediately begins to retry the connection. The default retry duration is 720 minutes. You can customize the retry time to a value from 10 to 1440 minutes. We recommend that you set the duration to more than 30 minutes. If DTS reconnects to the source and destination databases within the specified duration, the migration task automatically resumes. Otherwise, the task fails.
Note-
For multiple DTS instances that share the same source or destination, the network retry time is determined by the setting of the last created task.
-
Because you are charged for the task during the connection retry period, we recommend that you customize the retry time based on your business needs, or release the DTS instance as soon as possible after the source and destination database instances are released.
Retry Time for Other Issues
After the migration task starts, if a non-connectivity issue, such as a DDL or DML execution exception, occurs in the source or destination database, DTS reports an error and immediately begins to retry the operation. The default retry duration is 10 minutes. You can customize the retry time to a value from 1 to 1440 minutes. We recommend that you set the duration to more than 10 minutes. If the related operations succeed within the specified retry duration, the migration task automatically resumes. Otherwise, the task fails.
ImportantThe value of Retry Time for Other Issues must be less than the value of Retry Time for Failed Connections.
Enable Throttling for Full Data Migration
During full migration, DTS consumes read and write resources on the source and destination databases, which may increase the database load. If required, you can enable throttling for the full migration task. You can set Queries per second (QPS) to the source database, RPS of Full Data Migration, and Data migration speed for full migration (MB/s) to reduce the load on the destination database.
Note-
This configuration item is available only if you select Full Data Migration for Migration Types.
-
You can also adjust the full migration speed after the migration instance is running.
Enable Throttling for Incremental Data Migration
If required, you can also choose to set speed limits for the incremental migration task. You can set RPS of Incremental Data Migration and Data migration speed for incremental migration (MB/s) to reduce the load on the destination database.
Note-
This configuration item is available only if you select Incremental Data Migration for Migration Types.
-
You can also adjust the incremental migration speed after the migration instance is running.
Environment Tag
You can select an environment label to identify the instance. In this example, no selection is needed.
Configure ETL
Choose whether to enable the extract, transform, and load (ETL) feature. For more information, see What is ETL? Valid values:
-
Yes: Enables the ETL feature. Enter data processing statements in the code editor. For more information, see Configure ETL in a data migration or data synchronization task.
-
No: Disables the ETL feature.
Monitoring and Alerting
Select whether to set alerts and receive alert notifications based on your business needs.
-
No: Does not set an alert.
-
Yes: Configure alerts by setting an alert threshold and an alert notifications. If a migration fails or the latency exceeds the threshold, the system sends an alert notification.
-
-
Click Next: Data Validation to configure the data validation task.
If you want to use data validation, see Configure data validation for instructions.
-
-
Save the task and run a precheck.
-
To view the parameters for configuring this instance when you call the API operation, move the pointer over the Next: Save Task Settings and Precheck button and click Preview OpenAPI parameters in the bubble that appears.
-
If you do not need to view or have finished viewing the API parameters, click Next: Save Task Settings and Precheck at the bottom of the page.
Note-
Before the migration task starts, DTS performs a precheck. The task starts only after it passes the precheck.
-
If the precheck fails, click View Details next to the failed check item, fix the issue based on the prompt, and then run the precheck again.
-
If a warning is reported during the precheck:
-
For check items that cannot be ignored, click View Details next to the failed item, fix the issue based on the prompt, and then run the precheck again.
-
For check items that can be ignored, you can click Confirm Alert Details, Ignore, OK, and Precheck Again to skip the alert item and run the precheck again. If you choose to ignore a warning, it may cause issues such as data inconsistency and pose risks to your business.
-
-
-
Purchase the instance.
-
When the Success Rate is 100%, click Next: Purchase Instance.
-
On the Purchase page, select the link specification for the data migration instance. For more information, see the following table.
Category
Parameter
Description
New Instance Class
Resource Group Settings
Select the resource group to which the instance belongs. The default value is default resource group. For more information, see What is Resource Management?
Instance Class
DTS provides migration specifications with different performance levels. The link specification affects the migration speed. You can select a specification based on your business scenario. For more information, see Data migration link specifications.
-
After the configuration is complete, read and select Data Transmission Service (Pay-as-you-go) Service Terms.
-
Click Buy and Start. In the OK dialog box that appears, click OK.
You can view the progress of the migration task on the Data Migration Tasks list page.
Note-
If the migration task does not include incremental migration, it stops automatically after the full migration is complete. After the task stops, its Status changes to Completed.
-
If the migration task includes incremental migration, it does not stop automatically. The incremental migration task continues to run. While the incremental migration task is running, the Status of the task is Running.
-
-
FAQ
Does the one-stop cloud migration feature of RDS support selecting objects for validation?
No, it does not. By default, the objects for validation are the same as the objects for migration.
What is the mode for full validation in the one-stop cloud migration feature of RDS?
The mode is HASH validation with a 100% sampling rate.
What is the baseline for full validation in the one-stop cloud migration feature of RDS?
The union of the source and destination databases is used as the baseline to validate data consistency between them.