ApsaraDB RDS for SQL Server provides the cloud migration of incremental backup data. You can store full backup files of the source database on a self-managed SQL Server instance to an Object Storage Service (OSS) bucket, and restore the full backup data of the source database to the destination database on your ApsaraDB RDS for SQL Server instance in the ApsaraDB RDS console. Then, you can import the differential backup files or log backup files to the destination database on your RDS instance in the ApsaraDB RDS console to implement the cloud migration of incremental backup data. This migration method reduces downtime to minutes.
Scenarios
The migration method in this topic is suitable in the following scenarios:
Migrate data to the destination database on your RDS instance in physical mode rather than in logical mode.
NotePhysical migration allows you to migrate data by using physical backup files. Logical migration allows you to write the executed DML statements to the destination database on your RDS instance.
Physical migration ensures 100% data consistency between the source database and the destination database. Logical migration cannot ensure 100% data consistency. For example, index fragmentation and statistical information may change after the migration.
Migrate data with minute-level downtime.
NoteIf your self-managed SQL Server instance has a data volume of less than 100 GB and does not provide time-sensitive services, we recommend that you migrate the data to your RDS instance by using full backup files. The migration may cause a downtime of 2 hours. For more information, see Migrate the full backup data of a self-managed SQL Server instance to an ApsaraDB RDS instance that runs SQL Server 2008 R2 with cloud disks or runs SQL Server 2012 or later.
Prerequisites
The RDS instance runs SQL Server 2008 R2 with cloud disks or runs SQL Server 2012 or later. The names of existing databases on your RDS instance are different from the name of the source database on the self-managed SQL Server instance. For more information about how to create an RDS instance, see Create an ApsaraDB RDS for SQL Server instance.
NoteRDS instances that run SQL Server 2008 R2 with cloud disks are no longer available for purchase. For more information, see [EOS/Discontinuation] ApsaraDB RDS instances running SQL Server 2008 R2 with cloud disks are no longer available for purchase from July 14, 2023.
The available storage of the RDS instance is sufficient. If the available storage is insufficient, you must expand the storage capacity of the RDS instance before you start the migration. For more information, see Change the specifications of an ApsaraDB RDS for SQL Server instance.
A privileged account is created for the RDS instance. For more information, see Create accounts and databases.
The self-managed SQL Server instance uses the
FULL
recovery model.NoteTransaction log backups are required when you migrate the incremental backup data of a self-managed SQL Server instance to an RDS instance. If the self-managed SQL Server instance uses the SIMPLE recovery model, transaction logs cannot be backed up.
If the size of differential backup files is large, the time that is required to migrate the incremental backup data of a self-managed SQL Server instance to an RDS instance may increase.
The output of the
DBCC CHECKDB
statement that is executed in the self-managed SQL Server instance indicates that noallocation errors
orconsistency errors
occur. If no allocation errors or consistency errors occur, the following execution result is returned:... CHECKDB found 0 allocation errors and 0 consistency errors in database 'xxx'. DBCC execution completed. If DBCC printed error messages, contact your system administrator.
OSS is activated. For more information, see Activate OSS.
The OSS bucket and the RDS instance reside in the same region. For more information, see Step 2: Upload the generated full backup file to the OSS bucket.
If you use a RAM user, make sure that the following requirements are met:
The AliyunOSSFullAccess and AliyunRDSFullAccess policies are attached to the RAM user. For more information about how to grant permissions to RAM users, see Use RAM to manage OSS permissions and Use RAM to manage ApsaraDB RDS permissions.
The service account of ApsaraDB RDS is authorized by using your Alibaba Cloud account to access the OSS bucket.
A custom policy is manually created by using your Alibaba Cloud account and is attached to the RAM user. For more information about how to create a custom policy, see the Create a custom policy on the JSON tab section in Create custom policies.
You must use the following content for the custom policy:
{ "Version": "1", "Statement": [ { "Action": [ "ram:GetRole" ], "Resource": "acs:ram:*:*:role/AliyunRDSImportRole", "Effect": "Allow" } ] }
Usage notes
The migration method that is described in this topic is at the database level. You can migrate the backup data only of one database on a self-managed SQL Server instance to your RDS instance at a time. If you want to migrate the backup data of multiple or all databases on a self-managed SQL Server instance at a time, we recommend that you use an instance-level migration method. For more information, see Migrate data from a self-managed SQL Server instance to an ApsaraDB RDS for SQL Server instance.
The migration from a later SQL Server version to an earlier SQL Server version is not supported. For example, if a self-managed SQL Server instance runs SQL Server 2016 and your RDS instance runs SQL Server 2012, you cannot migrate the backup data of the self-managed SQL Server instance to the RDS instance.
The names of the backup files cannot contain special characters, such as at signs (@) and vertical bars (|). If the names of the backup files contain special characters, the migration fails.
After you authorize the service account of ApsaraDB RDS to access the OSS bucket, a role named AliyunRDSImportRole is created in Resource Access Management (RAM). Do not modify or delete this role. If you modify or delete this role, the backup files cannot be downloaded from the OSS bucket. If you modify or delete this role, you must re-authorize the service account by using the migration wizard.
The RDS instance does not carry over the accounts of the self-managed SQL Server instance. After the migration is complete, you must create accounts for the RDS instance in the ApsaraDB RDS console.
Before the migration is complete, do not delete the backup files from the OSS bucket. If you delete the backup files before the migration is complete, the migration fails.
The names of backup files must be suffixed with
bak
(full backup files),diff
(differential backup files),trn
, orlog
(log backup files of transactions).NoteIn actual business scenarios, a backup file may be stored in a different format. For example, a file suffixed with
.bak
can be a full backup file, a differential backup file, or a log backup file of transactions.If the backup files do not use the preceding suffixes, the system may fail to identify the types of the backup files. This affects subsequent operations.
If the backup file is a log backup file of the self-managed SQL Server instance and downloaded in the ApsaraDB RDS console, the default format of the downloaded backup file is
.zip.log
instead of.bak
. The downloaded backup file can be used for cloud migration after you convert the file format. The .bak file refers to the backup script file generated in Step 1. For more information, see Download data backup files and log backup files.The following operations describe how to convert the file format: Change the extension of the file to
.zip
for decompression, rename the decompresseddatabase_name.lbak
file to a file suffixed with.bak
, and then upload the.bak
file to an OSS bucket as an incremental log backup file for cloud migration.
Migration process
Migration phase | Step | Description |
Full backup and restoration | Step1. Before 00:00 | Complete the following preparations:
|
Step2. 00:01 | Perform a full backup on the source database. Time required: about 1 hour. | |
Step3. 02:00 | Upload the full backup file to the OSS bucket. Time required: about 1 hour. | |
Step4. 03:00 | Restore data from the full backup file to your RDS instance in the ApsaraDB RDS console. Time required: about 19 hours. | |
Incremental backup and restoration | Step5. 22:00 | Perform a log backup on the source database. Time required: about 20 minutes. |
Step6. 22:20 | Upload the log backup file to the OSS bucket. Time required: about 10 minutes. | |
Step6. 22:30 |
| |
Database opening | Step8. 22:34 | Restore data from the last log backup file to your RDS instance. Time required: about 4 minutes. |
Step9. 22:35 | Open the destination database on your RDS instance. If you execute the DBCC statement in asynchronous mode, the destination database can be opened in 1 minute. |
The preceding migration provides an example of how to minimize downtime. Your application can continue to run, and you do not need to stop your application until the last log backup. In this example, the downtime of your application does not exceed 5 minutes.
Step 1: Back up the source database
Download the backup script file. Then, open the file by using SQL Server Management Studio (SSMS).
Configure the following parameters.
Parameter
Description
@backup_databases_list
The name of the source database that you want to back up. If you specify multiple databases, separate the names of these databases with semicolons (;) or commas (,).
@backup_type
The backup type. Valid values:
FULL: full backup
DIFF: differential backup
LOG: log backup
@backup_folder
The directory that is used to store the backup files. If the specified directory does not exist, the system automatically creates one.
@is_run
Specifies whether to perform a backup or a check. Valid values:
1: performs a backup.
0: performs a check only.
Execute the backup script.
After the execution, a
.bak
file is automatically generated regardless of the backup type that you specified.
Step 2: Upload backup file to the OSS bucket
If an OSS bucket is created, check whether the bucket meets the following requirements:
The storage class of the OSS bucket is Standard. The storage class cannot be Standard, Infrequent Access (IA), Archive, Cold Archive, or Deep Cold Archive. For more information, see Overview.
Data encryption is not enabled for the OSS bucket. For more information, see Data encryption.
Create an OSS bucket.
Log on to the OSS console.
In the left-side navigation pane, click Buckets. On the Buckets page, click Create Bucket.
Configure the following parameters. Retain the default values for other parameters.
ImportantThe created OSS bucket is used only for the data migration and is no longer used after the data migration is complete. You need to only configure key parameters. To prevent data leaks and excessive costs, we recommend that you delete the OSS bucket after the data migration is complete at the earliest opportunity.
Do not enable data encryption when you create an OSS bucket. For more information, see Data encryption.
Parameter
Description
Example
Bucket Name
The name of the OSS bucket. The name is globally unique and cannot be modified after it is configured.
Naming conventions:
The name can contain only lowercase letters, digits, and hyphens (-).
It must start and end with a lowercase letter or a digit.
The name must be 3 to 63 characters in length.
migratetest
Region
The region of the OSS bucket. If you want to upload data to the OSS bucket from an Elastic Compute Service (ECS) instance over an internal network and then restore the data to the RDS instance over the internal network, make sure that the OSS bucket, ECS instance, and RDS instance reside in the same region.
China (Hangzhou)
Storage Class
The storage class of the bucket. Select Standard. The cloud migration operations described in this topic cannot be performed in buckets of other storage classes.
Standard
Upload backup files to the OSS bucket.
NoteIf the RDS instance and the OSS bucket reside in the same region, they can communicate with each other over an internal network. You can use the internal network to upload the backup data. The method is faster, and no fees are generated for Internet traffic. We recommend that you upload the backup file to an OSS bucket that is in the same region as the destination RDS instance.
After the full backup on the self-managed SQL Server instance is complete, you must use one of the following methods to upload the generated full backup file to the OSS bucket:
Step 3: Create a cloud migration task
Go to the Instances page. In the top navigation bar, select the region in which the RDS instance resides. Then, find the RDS instance and click the ID of the instance.
In the left-side navigation pane, click Backup and Restoration.
In the upper-right corner of the page, click Migrate OSS Backup Data to RDS.
In the Import Guide wizard, click Next twice to import data.
NoteIf you use the OSS-based migration wizard for the first time, you must authorize the service account of ApsaraDB RDS to access the OSS bucket. In this case, you must click Authorize and complete the authorization. Otherwise, the OSS Bucket drop-down list in the Import Data step is empty.
Configure the following parameters and click OK.
Wait until the migration task is complete. You can click Refresh to view the most recent status of the migration task.
Parameter
Description
Database Name
Enter the name of the destination database on your RDS instance. The destination database is used to store the data that is migrated from the source database on the self-managed SQL Server instance. The name of the destination database must meet the requirements of open source SQL Server.
ImportantBefore migration, you must make sure that the names of the databases on the destination RDS instance are different from the name of the database that you want to restore by using the specified backup file. In addition, make sure that database files with the same name as the database that you want to restore by using the specified backup file are not added to the databases on the destination RDS instance. If both preceding requirements are met, you can restore the database by using a database file in the backup set. Note that the database file must have the same name as the database that you want to restore.
If one of the preceding requirements is not met, the migration fails.
OSS Bucket
Select the OSS bucket that stores the backup file.
OSS Subfolder Name
Enter the name of the OSS bucket that stores the backup file.
OSS File
Specify the backup file that you want to import. You can enter a prefix in the search box and click the search icon to search for the backup file by using fuzzy match. The name, size, and update time of each backup file whose name contains the prefix are displayed. Select the backup file that you want to migrate to the RDS instance.
Cloud Migration Method
Select Access Pending (Incremental Backup). Valid values:
Immediate Access (Full Backup): If you want to migrate only a full backup file, select this migration method. In this case, the following parameter settings take effect in the CreateMigrateTask operation:
BackupMode = FULL
andIsOnlineDB = True
.Access Pending (Incremental Backup): If you want to migrate a full backup file and a log or differential backup file, select this migration method. In this case, the following parameter setting takes effect in the CreateMigrateTask operation:
BackupMode = UPDF
andIsOnlineDB = False
.
Step 4: Import the log or differential backup file
After the full backup file of the source database on the self-managed SQL Server instance is imported into the destination database on your RDS instance, you must import the log or differential backup file.
Go to the Instances page. In the top navigation bar, select the region in which the RDS instance resides. Then, find the RDS instance and click the ID of the instance.
In the left-side navigation pane, click Backup and Restoration. On the page that appears, click the Cloud Migration Records of Backup Data tab.
Find the destination database and click Upload Incremental Files in the Task Actions column. Select the log or differential backup file and click OK.
NoteIf you have multiple log or differential backup files, you must use the same method to upload the log backup files one by one.
Make sure that the size of the last log or differential backup file does not exceed 500 MB. This minimizes the time that is required to complete the migration.
Before the last log or differential backup file is generated, you must stop data writes to the source database. This ensures data consistency between the source database and the destination database on your RDS instance.
Step 5: Open the database
After you import all the backup files into the destination database on your RDS instance, the destination database is in the In Recovery or Restoring state. If your RDS instance runs RDS High-availability Edition, the destination database is in the In Recovery state. If your RDS instance runs RDS Basic Edition, the destination database is in the Restoring state. In these cases, you cannot perform read or write operations on the destination database. Before you can perform read and write operations, you must open the destination database.
Go to the Instances page. In the top navigation bar, select the region in which the RDS instance resides. Then, find the RDS instance and click the ID of the instance.
In the left-side navigation pane, click Backup and Restoration. On the page that appears, click the Cloud Migration Records of Backup Data tab.
Find the destination database and click Open Database in the Task Actions column.
Select a consistency check mode and click OK.
NoteApsaraDB RDS provides the following consistency check modes:
Asynchronous DBCC: The DBCC CHECKDB statement is executed after the destination database is opened. This reduces the time that is required to open the destination database and minimizes the downtime of your application. If the destination database is large, a long period of time is required to execute the DBCC CHECKDB statement. If your application is sensitive to downtime but insensitive to the result of the DBCC CHECKDB statement, we recommend that you select this consistency check mode. In this case, the following parameter setting takes effect in the CreateMigrateTask operation:
CheckDBMode = AsyncExecuteDBCheck
.Synchronous DBCC: The DBCC CHECKDB statement is executed at the same time when the destination database is opened. If you want to identify consistency errors between the self-managed database and the destination database based on the result of the DBCC CHECKDB statement, we recommend that you select this consistency check mode. However, the time that is required to open the destination database increases. In this case, the following parameter setting takes effect in the CreateMigrateTask operation:
CheckDBMode = SyncExecuteDBCheck
.
Step 6: View details of the imported backup files
If you want to view details of the backup files that are imported by using a migration task, perform the following operations: Go to the Backup and Restoration page. Click the Cloud Migration Records of Backup Data tab. Find the required migration task and click View File Details in the Task Actions column. Then, view the details of the imported backup files.
Common errors
For more information about the common errors that may occur during the migration of full backup data, see Common errors.
During the migration of incremental backup data, you may encounter the following errors:
The destination database cannot be opened.
Error message: Failed to open database xxx.
Cause: Some advanced features are enabled for the self-managed SQL Server instance. However, these advanced features are not supported by your RDS instance. For example, the self-managed SQL Server instance runs an Enterprise Edition of SQL Server and your RDS instance runs a Web edition of SQL Server. If the data compression and partition features are enabled for the self-managed SQL Server instance, this error is reported when you open the destination database on the RDS instance.
Solution:
Disable the advanced features for the self-managed SQL Server instance, back up data again, and then migrate the data by using OSS.
Purchase an RDS instance that runs the same SQL Server edition as the self-managed SQL Server instance. Then, migrate the data of the source database on the self-managed SQL Server instance to the new RDS instance. For more information about how to purchase, see Create and use an ApsaraDB RDS for SQL Server instance.
NoteFor more information about the SQL Server editions, see Features of ApsaraDB RDS instances that run different SQL Server versions and RDS editions.
The log sequence numbers (LSNs) in the backup chain are not consecutive.
Error message: The log in this backup set begins at LSN XXX, which is too recent to apply to the database.RESTORE LOG is terminating abnormally.
Cause: The LSNs in the log or differential backup file are different from the LSNs in the previous backup file that is used for the restoration.
Solution: Select the log or differential backup file whose LSNs are the same as the LSNs of the previous backup file that is used for the restoration.
The DBCC CHECKDB statement cannot be executed in asynchronous mode.
Error message: asynchronously DBCC checkdb failed: CHECKDB found 0 allocation errors and 2 consistency errors in table 'XXX' (object ID XXX).
Cause: After data is restored to your RDS instance with the Asynchronous DBCC consistency check mode selected, ApsaraDB RDS executes the DBCC CHECKDB statement. If the destination database fails the consistency check, consistency errors occur in the source database.
Solution:
Execute the following statement on the destination database:
DBCC CHECKDB (DBName,REPAIR_ALLOW_DATA_LOSS)
ImportantIf you use this solution to fix the issue, your data may be lost.
Execute the following statement on the source database to fix the error and then migrate data again:
DBCC CHECKDB (DBName,REPAIR_ALLOW_DATA_LOSS)
The selected backup file is a full backup file.
Error message: Backup set (xxx) is a Database FULL backup, we only accept transaction log or differential backup.
Cause: After the data is restored to your RDS instance by using a full backup file, you can select only a log or differential backup file. If you select a full backup file again, this error is reported.
Solution: Select a log or differential backup file.
The number of specified source databases exceeds the upper limit.
Error message: The database (xxx) migration failed due to databases count limitation.
Cause: If the number of specified source databases exceeds the upper limit, this error is reported.
Solution: Migrate the data of source databases to another RDS instance. Otherwise, delete unnecessary databases.
The RAM user does not have the required permissions.
Error 1
Error: The parameters that are described in the substep 5 of Step 3: Create a cloud migration task are correctly configured, but the OK button is dimmed.
Cause: You are using a RAM user that does not have the required permissions.
Solution: Refer to the "Prerequisites" section of this topic to grant required permissions to the RAM user.
Error 2
Error: A message appears to indicate that the RAM user does not have permissions to grant the
AliyunRDSImportRole
permission.Cause: The RAM user that you are using does not have the required permissions.
Solution: Use your Alibaba Cloud account to grant the
AliyunRAMFullAccess
permission to your RAM user. For more information, see Use RAM to manage ApsaraDB RDS permissions.
Related operations
Operation | Description |
Creates a data migration task. | |
Opens the database to which backup data is migrated on an ApsaraDB RDS for SQL Server instance. | |
Queries the tasks that are created to migrate the backup data of an ApsaraDB RDS for SQL Server instance. | |
Queries the backup file details of a backup data migration task for an ApsaraDB RDS for SQL Server instance. |