Data Transmission Service:Use DTS to migrate data from Azure to ApsaraDB RDS over a VPN gateway

Scenario

The preceding scenario is used as an example in this topic. an enterprise creates a virtual network in the Germany West Central region of Azure and deploys an Azure Database for MySQL flexible server in the virtual network. The enterprise also creates a VPC in the Germany (Frankfurt) region of Alibaba Cloud and deploys an ApsaraDB RDS for MySQL instance in the VPC. The enterprise wants to migrate data from the Azure Database for MySQL flexible server to the ApsaraDB RDS for MySQL instance.

The enterprise can establish an IPsec-VPN connection associated with a VPN gateway between the Azure virtual network and the Alibaba Cloud VPC. After the connection is established, the enterprise can use DTS to migrate data from the Azure Database for MySQL flexible server to the Alibaba Cloud ApsaraDB RDS for MySQL instance.

Prerequisites

Before you start, make sure that the following conditions are met:

• A virtual network is created in the Germany West Central region of Azure. An Azure Database for MySQL flexible server (hereinafter referred to as Azure MySQL server) is deployed in the virtual network. For more information, visit Azure.

• A VPC is created in the Germany (Frankfurt) region of Alibaba Cloud. An ApsaraDB RDS for MySQL database is deployed in the VPC. For more information, see Create an ApsaraDB RDS for MySQL instance.

• The CIDR blocks of the Azure virtual network and the Alibaba Cloud VPC that need to communicate with each other as well as the account information of the two databases are obtained.

  Important

  • You can plan the CIDR blocks by yourself. Make sure that the CIDR blocks of the Azure virtual network and the Alibaba Cloud VPC do not overlap each other.

  • Make sure that you understand the limits on data migration and the database accounts have relevant permissions. For more information, see Limits and Permissions required for database accounts.

  Resource	CIDR block	IP address	Database account
  Alibaba Cloud VPC	10.0.0.0/16	Internal endpoint of the ApsaraDB RDS for MySQL instance: rm-gw8x4h4tg****.mysql.germany.rds.aliyuncs.com For more information about how to obtain the internal endpoint of an ApsaraDB RDS for MySQL instance, see View and manage instance endpoints and ports.	Username: AliyunUser Password: Hello1234****
  Azure virtual network	192.168.0.0/16	Azure MySQL server name: zho****-azure.mysql.database.azure.com	Username: AzureUser Password: Hello5678**** Service port of the database: 3306

• View the information about the database to be migrated to Alibaba Cloud

  

Step 1: Create a VPN gateway on Alibaba Cloud

You must first create a VPN gateway on Alibaba Cloud. After the VPN gateway is created, the system assigns two IP addresses to the VPN gateway. The IP addresses are used to establish an IPsec-VPN connection to the Azure virtual network.

1. Log on to the VPN Gateway console.

2. In the top navigation bar, select the region where you want to create the VPN gateway.

   The region of the VPN gateway must be the same as that of the VPC to be associated.

3. On the VPN Gateway page, click Create VPN Gateway.

4. On the buy page, configure the following parameters, click Buy Now, and then complete the payment.

   The following table describes only the key parameters that you must configure. For other parameters, use the default values or leave them empty. For more information, see Create and manage a VPN gateway.

   Parameter	Description	Example
   Name	The name of the VPN gateway.	Enter VPN Gateway.
   Region	The region in which you want to create the VPN gateway.	Select Germany (Frankfurt).
   Gateway Type	The type of the VPN gateway.	Select Standard.
   Network Type	The network type of the VPN gateway.	Select Public.
   Tunnels	The tunnel mode of the VPN gateway. The system displays the tunnel modes that are supported in this region. Valid values: Dual-tunnel Single-tunnel For more information about the single-tunnel mode and dual-tunnel mode, see [Upgrade notice] IPsec-VPN connections support the dual-tunnel mode.	Use the default value Dual-tunnel.
   VPC	The VPC with which you want to associate the VPN gateway.	Select a VPC in the Germany (Frankfurt) region.
   vSwitch 1	The vSwitch with which you want to associate the VPN gateway in the associated VPC. If you select Single-tunnel, you need to specify only one vSwitch. If you select Dual-tunnel, you need to specify two vSwitches. After the IPsec-VPN feature is enabled, the system creates an elastic network interface (ENI) for each of the two vSwitches as an interface to communicate with the VPC over an IPsec-VPN connection. Each ENI occupies one IP address in the vSwitch. Note The system selects a vSwitch by default. You can change or use the default vSwitch. After a VPN gateway is created, you cannot modify the vSwitch associated with the VPN gateway. You can view the vSwitch associated with the VPN gateway, the zone to which the vSwitch belongs, and the ENI in the vSwitch on the details page of the VPN gateway.	Select a vSwitch in the associated VPC.
   vSwitch 2	The other vSwitch with which you want to associate the VPN gateway in the associated VPC. Specify two vSwitches in different zones in the associated VPC to implement disaster recovery across zones for IPsec-VPN connections. For a region that supports only one zone, disaster recovery across zones is not supported. We recommend that you specify two vSwitches in the zone to implement high availability of IPsec-VPN connections. You can also select the same vSwitch as the first one. Note If only one vSwitch is deployed in the VPC, create a vSwitch. For more information, see Create and manage a vSwitch.	Select another vSwitch in the associated VPC.
   IPsec-VPN	Specifies whether to enable IPsec-VPN for the VPN gateway. Default value: Enable.	Select Enable.
   SSL-VPN	Specifies whether to enable SSL-VPN for the VPN gateway. Default value: Disable.	Select Disable.

5. After the VPN gateway is created, view the VPN gateway on the VPN Gateway page.

   The newly created VPN gateway is in the Preparing state and changes to the Normal state after about 1 to 5 minutes. After the status changes to Normal, the VPN gateway is ready for use.

   The following table describes the two IP addresses assigned by the system to the VPN gateway.

   VPN gateway name	VPN gateway ID	IP address
   VPN Gateway	vpn-gw8dickm386d2qi2g****	IPsec address 1: 8.XX.XX.130, which is the IP address of the active tunnel by default.
   		IPsec address 2: 47.XX.XX.27, which is the IP address of the standby tunnel by default.

Step 2: Deploy VPN resources on Azure

To establish an IPsec-VPN connection between the Azure virtual network and the Alibaba Cloud VPC, you must deploy VPN resources on Azure based on the following information. Consult Azure for specific commands or operations.

1. Create a gateway subnet in the virtual network. The subnet is required when you create a virtual network gateway.

   

2. Create a virtual network gateway.

   You must associate the virtual network gateway with the virtual network that needs to communicate with Alibaba Cloud. In this example, you must enable the active-active mode for the virtual network gateway and create two public IP addresses. Use the default values for other parameters.

   After the virtual network gateway is created, you can view the public IP addresses assigned by the system to the virtual network gateway on the Public IP addresses page. In this example, the public IP addresses assigned by the system are 4.XX.XX.224 and 4.XX.XX.166.

3. Create local network gateways.

   You must create two local network gateways on Azure. Each local network gateway must be configured with one IP address of the Alibaba Cloud VPN gateway. In addition, you must specify the CIDR block of the Alibaba Cloud VPC and the CIDR block 100.104.0.0/16 on each local network gateway.

   Note

   When you create a local network gateway, you must specify the CIDR block 100.104.0.0/16. DTS uses IP addresses in this CIDR block to migrate data.

   

4. Create a Site-to-Site VPN connection.

   Important

   Both Alibaba Cloud and Azure IPsec-VPN connections support the dual-tunnel mode. By default, the two tunnels of an Azure IPsec-VPN connection are associated with the same local network gateway, and the two tunnels of an Alibaba Cloud IPsec-VPN connection have different IP addresses. Therefore, the two tunnels of Azure are connected to only one tunnel of Alibaba Cloud. To ensure that the two tunnels of the Alibaba Cloud IPsec-VPN connection are enabled at the same time, you must create two Site-to-Site VPN connections on Azure and associate the Site-to-Site VPN connections with different local network gateways.

   The following figure shows the configurations of one of the Site-to-Site VPN connections. When you create the VPN connection, select Site-to-site (IPsec) for the Connection type parameter and associate the VPN connection with the virtual network gateway that needs to establish a VPN connection with Alibaba Cloud. Then, select a local network gateway and specify the shared key. Use the default values for other parameters. Specify a different local network gateway for the other Site-to-Site VPN connection. Use the same values for other parameters.

   

Step 3: Deploy the VPN gateway on Alibaba Cloud

After you configure VPN resources on Azure, deploy a VPN gateway on Alibaba Cloud based on the following information to establish an IPsec-VPN connection between the Azure virtual network and the Alibaba Cloud VPC.

1. Create customer gateways.

   1. Log on to the VPN Gateway console.

   2. In the left-side navigation pane, choose Interconnections > VPN > Customer Gateways.

   3. In the top navigation bar, select the region in which you want to create the customer gateway.

      Make sure that the customer gateway and the VPN gateway to be connected are deployed in the same region.

   4. On the Customer Gateways page, click Create Customer Gateway.

   5. In the Create Customer Gateway panel, configure the following parameters and click OK.

      You must create two customer gateways and use the two public IP addresses of the Azure virtual network gateway as customer gateway IP addresses to establish two encrypted tunnels. The following table describes only the parameters that are relevant to this topic. You can use the default values of other parameters or leave them empty. For more information, see Create and manage a customer gateway.

      Parameter	Description	Customer Gateway 1	Customer Gateway 2
      Name	The name of the customer gateway.	Enter Customer Gateway 1.	Enter Customer Gateway 2.
      IP Address	The public IP address of the Azure virtual network gateway.	Enter 4.XX.XX.224.	Enter 4.XX.XX.166.

2. Create an IPsec-VPN connection.

   1. In the left-side navigation pane, choose Interconnections > VPN > IPsec Connections.

   2. In the top navigation bar, select the region in which you want to create the IPsec-VPN connection.

      Make sure that the IPsec-VPN connection and the VPN gateway are in the same region.

   3. On the IPsec Connections page, click Create IPsec-VPN Connection.

   4. On the Create IPsec-VPN Connection page, configure the IPsec-VPN connection based on the following information and click OK.

      Parameter	Description	Example
      Name	The name of the IPsec-VPN connection.	Enter IPsec-VPN Connection.
      Resource Group	The resource group to which the VPN gateway belongs.	Select the default resource group.
      Associate Resource	Select the type of network resource to be associated with the IPsec-VPN connection.	In this example, VPN Gateway is selected.
      VPN Gateway	The VPN gateway that you want to associate with the IPsec-VPN connection.	Select VPN Gateway.
      Routing Mode	Select a routing mode. Destination Routing Mode: Traffic is forwarded based on the destination IP address. Protected Data Flows: forwards traffic based on the source and destination IP addresses.	Select Protected Data Flows.
      Local Network	The CIDR block of the VPC with which the VPN gateway is associated.	In this example, enter the following two CIDR blocks: CIDR block of the VPC: 10.0.0.0/16 CIDR block of DTS servers: 100.104.0.0/16 Important You must add the CIDR block used by DTS to the local CIDR block. This way, DTS can access the peer database by using the VPN gateway. For more information about the CIDR blocks used by DTS, see Add the CIDR blocks of DTS servers.
      Remote Network	The peer CIDR block that the VPC associated with the VPN gateway wants to access.	Enter 192.168.0.0/16.
      Effective Immediately	Specify whether to immediately start negotiations for the connection. Valid value: Yes: starts negotiations after the configuration is complete. No: starts negotiations when inbound traffic is detected.	In this example, Yes is selected.
      Enable BGP	Specifies whether to enable Border Gateway Protocol (BGP). If you want to use BGP routing for the IPsec-VPN connection, turn on Enable BGP. By default, Enable BGP is turned off.	In this example, Enable BGP is turned off.
      Tunnel 1	Configure VPN parameters for the active tunnel. By default, Tunnel 1 serves as the active tunnel and Tunnel 2 serves as the standby tunnel. You cannot modify this configuration.
      Customer Gateway	The customer gateway that you want to associate with the active tunnel.	Select Customer Gateway 1.
      Pre-Shared Key	The pre-shared key of the active tunnel that is used to verify identities. The pre-shared key must be 1 to 100 characters in length, and can contain digits, letters, and the following characters: ~ ` ! @ # $ % ^ & * ( ) _ - + = { } [ ] \ | ; : ' , . < > / ? If you do not specify a pre-shared key, the system randomly generates a 16-character string as the pre-shared key. Important The tunnel and the peer gateway device must use the same pre-shared key. Otherwise, the system cannot establish an IPsec-VPN connection.	Use the same pre-shared key as the key of the Azure tunnel to be connected.
      Encryption Configuration	Configure the parameters for the IKE, IPsec, dead peer detection (DPD), and NAT traversal features.	Use the default encryption settings. For more information, see Create and manage IPsec-VPN connections in dual-tunnel mode.
      Tunnel 2	Configure VPN parameters for the standby tunnel.
      Customer Gateway	The customer gateway that you want to associate with the standby tunnel.	Select Customer Gateway 2.
      Pre-Shared Key	The pre-shared key of the standby tunnel that is used to verify identities.	Use the same pre-shared key as the key of the Azure tunnel to be connected.
      Encryption Configuration	Configure the parameters for the IKE, IPsec, dead peer detection (DPD), and NAT traversal features.	Use the default encryption settings. For more information, see Create and manage IPsec-VPN connections in dual-tunnel mode.
      Tag	The tags of the IPsec-VPN connection.	Leave this parameter empty.

   5. In the Created message, click OK.

3. Advertise the route of the VPN gateway.

   After you create the IPsec-VPN connection, you must advertise the route of the VPN gateway. If you select Protected Data Flows as Routing Mode, the system creates a policy-based route for the VPN gateway after the IPsec-VPN connection is created. The route is in the Unpublished state. You must advertise the policy-based route of the VPN gateway to the VPC.

   1. In the left-side navigation pane, choose Interconnections > VPN > VPN Gateways.

   2. In the top navigation bar, select the region in which the VPN gateway resides.

   3. On the VPN Gateways page, find the VPN gateway that you want to manage and click its ID.

   4. On the details page of the VPN gateway, click the Policy-based Route Table tab, find the route that you want to manage, and then click Advertise in the Actions column.

   5. In the Advertise Route message, click OK.

Step 4: Test network connectivity

After you complete the preceding steps, the resources in the Alibaba Cloud VPC and the Azure virtual network can communicate with each other. The following section describes how to verify the network connectivity between the Alibaba Cloud VPC and the Azure virtual network.

1. Create an Azure VM in the Azure virtual network and log on to the Azure VM. Consult Azure for specific commands or operations.

   Important

   Before you test the connectivity, make sure that you understand the security group rules applied to the Alibaba Cloud VPC and the Azure virtual network. Make sure that the security group rules allow resources in the Alibaba Cloud VPC and the Azure virtual network to communicate with each other.

   • For more information about security group rules on Alibaba Cloud, see View security group rules and Add a security group rule.

   • For more information about Azure security group rules, consult Azure.

2. Run the ping command on the Azure VM to access the internal endpoint of the ApsaraDB RDS for MySQL instance.

   ping <Internal endpoint of the ApsaraDB RDS for MySQL instance>

   If the Azure VM receives response packets as shown in the following figure, the resources in the Alibaba Cloud VPC and the Azure virtual network can communicate with each other.

3. Obtain the IP address of the Azure MySQL server.

   On the Azure VM, access the name of the Azure MySQL server by using the ping command. The system automatically returns the IP address of the Azure MySQL server, which will be used in data migration in DTS.

Step 5: Create a DTS data migration task

After resources in the Alibaba Cloud VPC and the Azure virtual network can communicate with each other, you can create a DTS data migration task. Then, you can use the DTS data migration task to migrate data from the Azure MySQL server to the ApsaraDB RDS for MySQL instance.

1. Use one of the following methods to go to the Data Migration page and select the region in which the data migration instance resides.

   DTS console

   1. Log on to the DTS console.

   2. In the left-side navigation pane, click Data Migration.

   3. In the upper-left corner of the page, select the region in which the data migration instance resides.

   DMS console

   Note

   The actual operation may vary based on the mode and layout of the DMS console. For more information, see Simple mode and Customize the layout and style of the DMS console.

   1. Log on to the DMS console.

   2. In the top navigation bar, move the pointer over Data Development > DTS (DTS) > Data Migration .

   3. From the drop-down list to the right of Data Migration Tasks, select the region in which the data synchronization instance resides.

2. Click Create Task to go to the task configuration page.

3. Optional. Click New Configuration Page in the upper-right corner of the page.

   Note

   1. Skip this step if the Back to Previous Version button is displayed in the upper-right corner of the page.

   2. Specific parameters in the new and previous versions of the configuration page may be different. We recommend that you use the new version of the configuration page.

4. Configure the source and destination databases. The following table describes the parameters.

   Section	Parameter	Description
   N/A	Task Name	The name of the DTS task. DTS automatically generates a task name. We recommend that you specify a descriptive name that makes it easy to identify the task. You do not need to specify a unique task name.
   Source Database	Database Type	The type of the destination instance. Select MySQL.
   	Access Method	The access method of the source database. Select Express Connect, VPN Gateway, or Smart Access Gateway.
   	Instance Region	The region in which the source database resides. Select Germany (Frankfurt).
   	Replicate Data Across Alibaba Cloud Accounts	Specifies whether to migrate data across Alibaba Cloud accounts. Select No.
   	Connected VPC	The VPC with which the VPN gateway is associated. DTS accesses the database on the Azure MySQL server by using an IPsec-VPN connection.
   	Domain Name or IP	The IP address of the source MySQL database. Enter the private IP address 192.168.0.4 of the Azure MySQL server.
   	Port Number	The service port number of the source MySQL database. Enter 3306.
   	Database Account	The username that is used to access the source database.
   	Database Password	The password that is used to access the destination database.
   	Encryption	Specifies whether to encrypt the connection to the destination database. Select Non-encrypted or SSL-encrypted based on your business requirements. If SSL encryption is disabled for the Azure MySQL server, select Non-encrypted. If SSL encryption is enabled for the Azure MySQL server, select SSL-encrypted. In this case, you must upload a CA Certificate and configure the CA Key parameter. In this example, SSL encryption is disabled for the Azure MySQL server. Select Non-encrypted. Note By default, an Azure MySQL server uses encrypted connections. You can disable SSL encryption. For more information, see Disable SSL enforcement on your Azure Database for MySQL flexible server instance.
   Destination Database	Database Type	The type of the destination instance. Select MySQL.
   	Access Method	The access method of the destination database. Select Alibaba Cloud Instance.
   	Instance Region	The region in which the ApsaraDB RDS for MySQL instance resides. Select Germany (Frankfurt).
   	Replicate Data Across Alibaba Cloud Accounts	Specifies whether to migrate data across Alibaba Cloud accounts. Select No.
   	RDS Instance ID	The ID of the ApsaraDB RDS for MySQL instance.
   	Database Account	The username that is used to access the destination database. Enter the database account of the ApsaraDB RDS for MySQL instance.
   	Database Password	The password that is used to access the destination database.
   	Encryption	Specifies whether to encrypt the connection to the destination database. Select Non-encrypted.

5. In the lower part of the page, click Test Connectivity and Proceed. In the CIDR Blocks of DTS Servers dialog box, click Test Connectivity.

   Make sure that the security group rules applied to the Alibaba Cloud VPC and the Azure virtual network allow access from DTS. For example, a security group rule is configured to allow access from resources in the CIDR block 100.104.0.0/16. For more information about the CIDR blocks used by DTS, see Add the CIDR blocks of DTS servers.

   • If the databases in the Alibaba Cloud VPC and the Azure virtual network can be connected, the network connectivity between the databases is normal. Then, you are navigated to the Configure Task Objects page.

   • If the databases in the Alibaba Cloud VPC and the Azure virtual network cannot be properly connected, you are not navigated to the next page. To troubleshoot the issue, follow the instructions on the page. For more information, see What do I do if an error is reported when I connect a database instance to DTS over VPN?

6. In the Configure Task Objects step, select a migration type and the database that you want to migrate from the Azure MySQL server to Alibaba Cloud. Use the default settings for other parameters in this and subsequent steps. For more information, see Migrate data from a self-managed MySQL database to an ApsaraDB RDS for MySQL instance.

   

   After the configuration is complete, DTS automatically starts the data migration task. You can log on to the ApsaraDB RDS for MySQL instance to view the data migration results.

References

• For more information about the issues that you may encounter during data migration, see FAQ about data migration.

• For more information about other data migration scenarios, see Overview of data migration scenarios.