Data Transmission Service:Use DTS to synchronize data between ApsaraDB RDS and Amazon EC2 over a VPN gateway

Scenario

In this example, an enterprise has deployed a VPC in the Europe (Frankfurt) region of AWS. A MySQL database is deployed on an Amazon EC2 instance in the VPC. The enterprise has also deployed a VPC in the Germany (Frankfurt) region of Alibaba Cloud. An ApsaraDB RDS for MySQL instance is created in the VPC. The enterprise wants to synchronize data from the Amazon EC2 instance to the ApsaraDB for RDS MySQL instance in real time to meet business requirements.

The enterprise can establish an IPsec-VPN connection associated with a VPN gateway between the Amazon VPC and the Alibaba Cloud VPC. If resources in the two VPCs can communicate with each other, the enterprise can use DTS to synchronize data from the Amazon EC2 instance to the ApsaraDB RDS for MySQL instance.

Prerequisites

Before you start, make sure that the following conditions are met:

• A VPC is created in the Europe (Frankfurt) region of AWS. A MySQL database is deployed on an Amazon EC2 instance the VPC. For more information, visit AWS.

• A VPC is created in the Germany (Frankfurt) region of Alibaba Cloud. An ApsaraDB RDS for MySQL instance is created in the VPC. For more information, see Create an ApsaraDB RDS for MySQL instance.

• The CIDR blocks of the two VPCs that need to communicate with each other as well as the account information of the two databases are obtained.

  Important

  • You can plan the CIDR blocks by yourself. Make sure that the CIDR blocks of the two VPCs do not overlap each other.

  • Make sure that you understand the limits on data synchronization and the database accounts have relevant permissions. For more information, see Limits, SQL operations that can be synchronized, and Permissions required for database accounts.

  Resource	CIDR block	IP address	Database account
  Alibaba Cloud VPC	10.0.0.0/16	Internal endpoint of the ApsaraDB RDS for MySQL instance: rm-gw8x4h4tg****.mysql.germany.rds.aliyuncs.com For more information about how to obtain the internal endpoint of an ApsaraDB RDS for MySQL instance, see View and manage instance endpoints and ports.	Username: AliyunUser Password: Hello1234****
  Amazon VPC	192.168.0.0/16	IP address of the Amazon EC2 instance to which the database belongs: 192.168.30.158	Username: AWSUser Password: Hello5678**** Service port of the database: 3306

• View the information about the database from which data is to be synchronized

  

Step 1: Create a VPN gateway on Alibaba Cloud

You must first create a VPN gateway on Alibaba Cloud. After the VPN gateway is created, the system assigns two IP addresses to the VPN gateway. The IP addresses are used to establish an IPsec-VPN connection to AWS.

1. Log on to the VPN Gateway console.

2. In the top navigation bar, select the region where you want to create the VPN gateway.

   The region of the VPN gateway must be the same as that of the VPC to be associated.

3. On the VPN Gateway page, click Create VPN Gateway.

4. On the buy page, configure the following parameters, click Buy Now, and then complete the payment.

   The following table describes only the key parameters that you must configure. For other parameters, use the default values or leave them empty. For more information, see Create and manage a VPN gateway.

   Parameter	Description	Example
   Name	The name of the VPN gateway.	Enter VPN Gateway.
   Region	The region in which you want to create the VPN gateway.	Select Germany (Frankfurt).
   Gateway Type	The type of the VPN gateway.	Select Standard.
   Network Type	The network type of the VPN gateway.	Select Public.
   Tunnels	The tunnel mode of the VPN gateway. The system displays the tunnel modes that are supported in this region. Valid values: Dual-tunnel Single-tunnel For more information about the single-tunnel mode and dual-tunnel mode, see [Upgrade notice] IPsec-VPN connections support the dual-tunnel mode.	Use the default value Dual-tunnel.
   VPC	The VPC with which you want to associate the VPN gateway.	Select a VPC in the Germany (Frankfurt) region.
   VSwitch	The vSwitch with which you want to associate the VPN gateway in the associated VPC. If you select Single-tunnel, you need to specify only one vSwitch. If you select Dual-tunnel, you need to specify two vSwitches. After the IPsec-VPN feature is enabled, the system creates an elastic network interface (ENI) for each of the two vSwitches as an interface to communicate with the VPC over an IPsec-VPN connection. Each ENI occupies one IP address in the vSwitch. Note By default, the system selects a vSwitch. You can change or use the default vSwitch. After a VPN gateway is created, you cannot modify the vSwitch associated with the VPN gateway. You can view the vSwitch associated with the VPN gateway, the zone to which the vSwitch belongs, and the ENI in the vSwitch on the details page of the VPN gateway.	Select a vSwitch in the associated VPC.
   vSwitch 2	The other vSwitch with which you want to associate the VPN gateway in the associated VPC. Specify two vSwitches in different zones in the associated VPC to implement disaster recovery across zones for IPsec-VPN connections. For a region that supports only one zone, disaster recovery across zones is not supported. We recommend that you specify two vSwitches in the zone to implement high availability of IPsec-VPN connections. You can also select the same vSwitch as the first one. Note If only one vSwitch is deployed in the VPC, create a vSwitch. For more information, see Create and manage a vSwitch.	Select another vSwitch in the associated VPC.
   IPsec-VPN	Specifies whether to enable IPsec-VPN for the VPN gateway. Default value: Enable.	Select Enable.
   SSL-VPN	Specifies whether to enable SSL-VPN for the VPN gateway. Default value: Disable.	Select Disable.

5. After the VPN gateway is created, view the VPN gateway on the VPN Gateway page.

   The newly created VPN gateway is in the Preparing state and changes to the Normal state after about 1 to 5 minutes. After the status changes to Normal, the VPN gateway is ready for use.

   The following table describes the two IP addresses assigned by the system to the VPN gateway.

   VPN gateway name	VPN gateway ID	IP address
   VPN Gateway	vpn-gw8dickm386d2qi2g****	IPsec address 1: 8.XX.XX.130, which is the IP address of the active tunnel by default.
   		IPsec address 2: 47.XX.XX.27, which is the IP address of the standby tunnel by default.

Step 2: Deploy VPN resources on AWS

To establish an IPsec-VPN connection between the AWS VPC and the Alibaba Cloud VPC, you must deploy VPN resources on AWS based on the following information. Consult AWS for specific commands or operations.

1. Create customer gateways.

   You must create two customer gateways on AWS and use the IP addresses of the Alibaba Cloud VPN gateway as the IP addresses of the customer gateways.

2. Create a virtual private gateway.

   You must create a virtual private gateway on AWS and associate the virtual private gateway with the Amazon VPC that needs to communicate with Alibaba Cloud.

3. Create Site-to-Site VPN connections.

   Important

   • Both Alibaba Cloud and AWS IPsec-VPN connections support the dual-tunnel mode. By default, the two tunnels of an AWS IPsec-VPN connection are associated with the same gateway, and the two tunnels of an Alibaba Cloud IPsec-VPN connection have different IP addresses. Therefore, the two tunnels of AWS are connected to only one tunnel of Alibaba Cloud. To ensure that the two tunnels of the Alibaba Cloud IPsec-VPN connection are enabled at the same time, you must create two Site-to-Site VPN connections on AWS and associate the Site-to-Site VPN connections with different customer gateways.

   • When you configure routes for the Site-to-Site VPN connections, you must also specify the CIDR block 100.104.0.0/16 apart from the CIDR block of the Alibaba Cloud VPC. DTS uses the CIDR block 100.104.0.0/16 to synchronize data.

   The following figure shows the configurations of one of the Site-to-Site VPN connections. We recommend that you use the default values for the tunnel configurations. Specify a different customer gateway when you configure the other Site-to-Site VPN connection. Use the same values for other parameters.

   After the Site-to-Site VPN connections are created, you can view the tunnel addresses of the connections, which are used to create an IPsec-VPN connection on Alibaba Cloud.

   The following table describes the external IP addresses of Tunnel 1 of each Site-to-Site VPN connection and the IP addresses of the associated customer gateways.

   Site-to-Site VPN connection	Tunnel	External IP address	Associated customer gateway IP address
   Site-to-Site VPN Connection 1	Tunnel 1	3.XX.XX.5	8.XX.XX.130
   Site-to-Site VPN Connection 2	Tunnel 1	3.XX.XX.239	47.XX.XX.27

4. Configure route advertising.

   You must enable route advertising for the route table of the Amazon VPC that is associated with the virtual private gateway to ensure that the routes of the Site-to-Site VPN connections are automatically advertised to the route table of the Amazon VPC.

   The route table of the Amazon VPC contains the CIDR block of the Alibaba Cloud VPC and the CIDR block used by DTS.

Step 3: Deploy the VPN gateway on Alibaba Cloud

After you configure VPN resources on AWS, deploy a VPN gateway on Alibaba Cloud based on the following information to establish an IPsec-VPN connection between the Amazon VPC and Alibaba Cloud VPC.

1. Creates customer gateways.

   1. Log on to the VPN Gateway console.

   2. In the left-side navigation pane, choose Interconnections > VPN > Customer Gateways.

   3. In the top navigation bar, select the region in which you want to create the customer gateway.

      Make sure that the customer gateway and the VPN gateway to be connected are deployed in the same region.

   4. On the Customer Gateways page, click Create Customer Gateway.

   5. In the Create Customer Gateway panel, configure the following parameters and click OK.

      You must create two customer gateways on Alibaba Cloud and use the external IP addresses of the tunnels of the AWS Site-to-Site VPN connections as the IP addresses of the customer gateways. The following table describes only the parameters that are relevant to this topic. You can use the default values for other parameters or leave them empty. For more information, see Create and manage a customer gateway.

      Important

      Use only the external IP address of Tunnel 1 of each Site-to-Site VPN connection as the customer gateway IP address. By default, the external IP address of Tunnel 2 of each Site-to-Site VPN connection is not used. After the IPsec-VPN connections are created, Tunnel 2 of each Site-to-Site VPN connection is unavailable.

      Parameter	Description	Customer Gateway 1	Customer Gateway 2
      Name	The name of the customer gateway.	Enter Customer Gateway 1.	Enter Customer Gateway 2.
      IP Address	The external IP address of the AWS tunnel.	Enter 3.XX.XX.5.	Enter 3.XX.XX.239.

2. Create an IPsec-VPN connection.

   1. In the left-side navigation pane, choose Interconnections > VPN > IPsec Connections.

   2. In the top navigation bar, select the region in which you want to create the IPsec-VPN connection.

      Make sure that the IPsec-VPN connection and the VPN gateway are in the same region.

   3. On the IPsec Connections page, click Create IPsec-VPN Connection.

   4. On the Create IPsec-VPN Connection page, configure the IPsec-VPN connection based on the following information and click OK.

      Parameter	Description	Example
      Name	The name of the IPsec-VPN connection.	Enter IPsec-VPN Connection.
      Resource Group	The resource group to which the VPN gateway belongs.	Select the default resource group.
      Associate Resource	The type of the network resource that you want to associate with the IPsec-VPN connection.	Select VPN Gateway.
      VPN Gateway	The VPN gateway that you want to associate with the IPsec-VPN connection.	Select the VPN gateway that you created.
      Routing Mode	The routing mode of the traffic. Valid values: Destination Routing Mode: forwards traffic based on the destination IP address. Protected Data Flows: forwards traffic based on the source and destination IP addresses.	Select Protected Data Flows.
      Local Network	The CIDR block of the VPC with which the VPN gateway is associated.	In this example, enter the following two CIDR blocks: CIDR block of the VPC: 10.0.0.0/16 CIDR block of DTS servers: 100.104.0.0/16 Important You must add the CIDR block used by DTS to the local network. This way, DTS can access the peer database by using the VPN gateway. For more information about the CIDR blocks used by DTS, see Add the CIDR blocks of DTS servers.
      Remote Network	The peer CIDR block that the VPC associated with the VPN gateway wants to access.	Enter 192.168.0.0/16.
      Effective Immediately	Specifies whether to immediately start negotiations for the connection. Valid values: Yes: starts negotiations after the configuration is complete. No: starts negotiations when inbound traffic is detected.	Select Yes.
      Enable BGP	Specifies whether to enable BGP. If you want to use BGP routing for the IPsec-VPN connection, turn on Enable BGP. By default, Enable BGP is turned off.	Turn off Enable BGP.
      Tunnel 1	Configure VPN parameters for the active tunnel. By default, Tunnel 1 serves as the active tunnel and Tunnel 2 serves as the standby tunnel. You cannot modify this configuration.
      Customer Gateway	The customer gateway that you want to associate with the active tunnel.	Select Customer Gateway 1.
      Pre-Shared Key	The pre-shared key of the active tunnel that is used to verify identities. The pre-shared key must be 1 to 100 characters in length, and can contain digits, letters, and the following characters: ~ ` ! @ # $ % ^ & * ( ) _ - + = { } [ ] \ | ; : ' , . < > / ? If you do not specify a pre-shared key, the system generates a random 16-character string as the pre-shared key. Important The tunnel and the peer gateway device must use the same pre-shared key. Otherwise, the system cannot establish an IPsec-VPN connection.	Use the same pre-shared key as the key of the AWS tunnel to be connected.
      Encryption Configuration	Configure the parameters for the Internet Key Exchange (IKE), IPsec, dead peer detection (DPD), and NAT traversal features.	The value of the SA Life Cycle (seconds) parameter in the IKE Configurations section must be the same as the value specified on AWS. In this example, the value is set to 28800. The value of the SA Life Cycle (seconds) parameter in the IPsec Configurations section must be the same as the value specified on AWS. In this example, the value is set to 3600. Use the default values for other parameters. For more information, see Create and manage IPsec-VPN connections in dual-tunnel mode.
      Tunnel 2	Configure VPN parameters for the standby tunnel.
      Customer Gateway	The customer gateway that you want to associate with the standby tunnel.	Select Customer Gateway 2.
      Pre-Shared Key	The pre-shared key of the standby tunnel that is used to verify identities.	Use the same pre-shared key as the key of the AWS tunnel to be connected.
      Encryption Configuration	Configure the parameters for the IKE, IPsec, DPD, and NAT traversal features.	The value of the SA Life Cycle (seconds) parameter in the IKE Configurations section must be the same as the value specified on AWS. In this example, the value is set to 28800. The value of the SA Life Cycle (seconds) parameter in the IPsec Configurations section must be the same as the value specified on AWS. In this example, the value is set to 3600. Use the default values for other parameters. For more information, see Create and manage IPsec-VPN connections in dual-tunnel mode.
      Tags	The tags of the IPsec-VPN connection.	Leave this parameter empty.

   5. In the Created message, click OK.

3. Advertise the route of the VPN gateway.

   After you create the IPsec-VPN connection, you must advertise the route of the VPN gateway. If you select Protected Data Flows as Routing Mode, the system creates a policy-based route for the VPN gateway after the IPsec-VPN connection is created. The route is in the Unpublished state. You must advertise the policy-based route of the VPN gateway to the VPC.

   1. In the left-side navigation pane, choose Interconnections > VPN > VPN Gateways.

   2. In the top navigation bar, select the region in which the VPN gateway resides.

   3. On the VPN Gateways page, find the VPN gateway that you want to manage and click its ID.

   4. On the details page of the VPN gateway, click the Policy-based Route Table tab, find the route that you want to manage, and then click Advertise in the Actions column.

   5. In the Advertise Route message, click OK.

   The route table of the Alibaba Cloud VPC contains the CIDR block of the Amazon VPC.

Step 4: Test network connectivity

After you complete the preceding steps, the resources in the Alibaba Cloud VPC and the Amazon VPC can communicate with each other. The following section describes how to verify the network connectivity between the Alibaba Cloud VPC and the Amazon VPC.

1. Log on to the Amazon EC2 instance. Consult AWS for specific commands or operations.

2. Run the ping command on the Amazon EC2 instance to access the internal endpoint of the ApsaraDB RDS for MySQL instance.

   ping <Internal endpoint of the ApsaraDB RDS for MySQL instance>

   If the Amazon EC2 instance receives response packets as shown in the following figure, the resources in the Alibaba Cloud VPC and the Amazon VPC can communicate with each other.

Step 5: Create a DTS data synchronization task

After resources in the Alibaba Cloud VPC and the Amazon VPC can communicate with each other, you can create a DTS data synchronization task. Then, you can use the DTS data synchronization task to migrate data from the Amazon EC2 instance to the ApsaraDB RDS for MySQL instance.

1. Go to the Data Synchronization Tasks page.

   1. Log on to the Data Management (DMS) console.

   2. In the top navigation bar, click DTS.

   3. In the left-side navigation pane, choose DTS (DTS) > Data Synchronization.

   Note

   • Operations may vary based on the mode and layout of the DMS console. For more information, see Simple mode and Customize the layout and style of the DMS console.

   • You can also go to the Data Synchronization Tasks page of the new DTS console.

2. On the right side of Data Synchronization Tasks, select the region in which the data synchronization instance resides.

   Note

   If you use the new DTS console, you must select the region in which the data synchronization instance resides in the top navigation bar.

3. Click Create Task. On the Create Data Synchronization Task page, configure the source and destination databases. The following table describes the parameters.

   Warning

   After you configure the source and destination databases, we recommend that you read the Limits that are displayed in the upper part of the page. Otherwise, the task may fail or data inconsistency may occur.

   Section	Parameter	Description
   N/A	Task Name	The name of the task. DTS automatically assigns a name to the task. We recommend that you specify a descriptive name that makes it easy to identify the task. You do not need to specify a unique task name.
   Source Database	Database Type	The type of the source instance. Select MySQL.
   	Access Method	The access method of the source database. Select Express Connect, VPN Gateway, or Smart Access Gateway.
   	Instance Region	The region in which the source database resides. Select Germany (Frankfurt).
   	Replicate Data Across Alibaba Cloud Accounts	Specifies whether to synchronize data across Alibaba Cloud accounts. Select No.
   	Connected VPC	The VPC with which the VPN gateway is associated. DTS accesses the database on the Amazon EC2 instance by using an IPsec-VPN connection.
   	IP Address or Domain Name	The host IP address of the source MySQL database. Enter the private IP address 192.168.30.158 of the Amazon EC2 instance.
   	Port Number	The service port number of the source MySQL database. Enter 3306.
   	Database Account	The username that is used to access the source database.
   	Database Password	The password that is used to access the source database.
   	Encryption	Specifies whether to encrypt the connection to the source database. Use the default value Non-encrypted.
   Destination Database	Database Type	The type of the destination database. Select MySQL.
   	Access Method	The access method of the destination database. Select Alibaba Cloud Instance.
   	Instance Region	The region in which the ApsaraDB RDS for MySQL instance resides. Select Germany (Frankfurt).
   	Replicate Data Across Alibaba Cloud Accounts	Specifies whether to synchronize data across Alibaba Cloud accounts. Select No.
   	RDS Instance ID	The ID of the ApsaraDB RDS for MySQL instance.
   	Database Account	The username that is used to access the destination database. Enter the database account of the ApsaraDB RDS for MySQL instance.
   	Database Password	The password that is used to access the destination database.
   	Encryption	Specifies whether to encrypt the connection to the destination database. Use the default value Non-encrypted.

4. In the lower part of the page, click Test Connectivity and Proceed. In the CIDR Blocks of DTS Servers dialog box, click Test Connectivity.

   Make sure that the security group rules applied to the Alibaba Cloud VPC and the Amazon VPC allow access from DTS. For example, a security group rule is configured to allow access from resources in the CIDR block 100.104.0.0/16. For more information about the CIDR blocks used by DTS, see Add the CIDR blocks of DTS servers.

   • If the databases in the Alibaba Cloud VPC and the Amazon VPC can be connected, the network connectivity between the databases is normal. Then, you are navigated to the Configure Task Object page.

   • If the databases in the Alibaba Cloud VPC and the Amazon VPC cannot be properly connected, you are not navigated to the next page. To troubleshoot the issue, follow the instructions on the page. For more information, see What do I do if an error is reported when I connect a database instance to DTS over VPN?

5. In the Configure Task Objects step, select the database on the Amazon EC2 instance from which you want to synchronize data to Alibaba Cloud. Use the default settings for other parameters in this and subsequent steps. For more information, see Synchronize data from a self-managed MySQL database connected over Express Connect, VPN Gateway, or Smart Access Gateway to an ApsaraDB RDS for MySQL instance.

   

   After the configuration is complete, DTS automatically starts the data synchronization task. You can log on to the ApsaraDB RDS for MySQL instance to view the data synchronization results.

References

• For more information about the issues that you may encounter during data synchronization, see FAQ about data synchronization.

• For more information about other data synchronization scenarios, see Overview of data synchronization scenarios.