All Products
Search
Document Center

Data Transmission Service:Use DTS to synchronize data between ApsaraDB RDS and Amazon EC2 over a VPN gateway

Last Updated:Aug 07, 2024

If you want to synchronize data from Amazon Web Services (AWS) to Alibaba Cloud, you can establish a network connection between an Amazon virtual private cloud (VPC) in AWS and an Alibaba Cloud VPC. Then, you can use Data Transmission Service (DTS) to synchronize data from AWS to Alibaba Cloud. This topic describes how to establish an IPsec-VPN connection associated with a VPN gateway to enable communication between an Amazon VPC and an Alibaba Cloud VPC. This topic also describes how to use DTS to synchronize data from an Amazon Elastic Compute Cloud (Amazon EC2) instance to an ApsaraDB RDS instance.

Scenario

image

In this example, an enterprise has deployed a VPC in the Europe (Frankfurt) region of AWS. A MySQL database is deployed on an Amazon EC2 instance in the VPC. The enterprise has also deployed a VPC in the Germany (Frankfurt) region of Alibaba Cloud. An ApsaraDB RDS for MySQL instance is created in the VPC. The enterprise wants to synchronize data from the Amazon EC2 instance to the ApsaraDB for RDS MySQL instance in real time to meet business requirements.

The enterprise can establish an IPsec-VPN connection associated with a VPN gateway between the Amazon VPC and the Alibaba Cloud VPC. If resources in the two VPCs can communicate with each other, the enterprise can use DTS to synchronize data from the Amazon EC2 instance to the ApsaraDB RDS for MySQL instance.

Prerequisites

Before you start, make sure that the following conditions are met:

  • A VPC is created in the Europe (Frankfurt) region of AWS. A MySQL database is deployed on an Amazon EC2 instance the VPC. For more information, visit AWS.

  • A VPC is created in the Germany (Frankfurt) region of Alibaba Cloud. An ApsaraDB RDS for MySQL instance is created in the VPC. For more information, see Create an ApsaraDB RDS for MySQL instance.

  • The CIDR blocks of the two VPCs that need to communicate with each other as well as the account information of the two databases are obtained.

    Important

    Resource

    CIDR block

    IP address

    Database account

    Alibaba Cloud VPC

    10.0.0.0/16

    Internal endpoint of the ApsaraDB RDS for MySQL instance: rm-gw8x4h4tg****.mysql.germany.rds.aliyuncs.com

    For more information about how to obtain the internal endpoint of an ApsaraDB RDS for MySQL instance, see View and manage instance endpoints and ports.

    • Username: AliyunUser

    • Password: Hello1234****

    Amazon VPC

    192.168.0.0/16

    IP address of the Amazon EC2 instance to which the database belongs: 192.168.30.158

    • Username: AWSUser

    • Password: Hello5678****

    • Service port of the database: 3306

  • View the information about the database from which data is to be synchronized

    AWS表

Procedure

image

Step 1: Create a VPN gateway on Alibaba Cloud

You must first create a VPN gateway on Alibaba Cloud. After the VPN gateway is created, the system assigns two IP addresses to the VPN gateway. The IP addresses are used to establish an IPsec-VPN connection to AWS.

  1. Log on to the VPN Gateway console.

  2. In the top navigation bar, select the region where you want to create the VPN gateway.

    The region of the VPN gateway must be the same as that of the VPC to be associated.

  3. On the VPN Gateway page, click Create VPN Gateway.

  4. On the buy page, configure the following parameters, click Buy Now, and then complete the payment.

    The following table describes only the key parameters that you must configure. For other parameters, use the default values or leave them empty. For more information, see Create and manage a VPN gateway.

    Parameter

    Description

    Example

    Name

    The name of the VPN gateway.

    Enter VPN Gateway.

    Region

    The region in which you want to create the VPN gateway.

    Select Germany (Frankfurt).

    Gateway Type

    The type of the VPN gateway.

    Select Standard.

    Network Type

    The network type of the VPN gateway.

    Select Public.

    Tunnels

    The tunnel mode of the VPN gateway. The system displays the tunnel modes that are supported in this region. Valid values:

    • Dual-tunnel

    • Single-tunnel

    For more information about the single-tunnel mode and dual-tunnel mode, see [Upgrade notice] IPsec-VPN connections support the dual-tunnel mode.

    Use the default value Dual-tunnel.

    VPC

    The VPC with which you want to associate the VPN gateway.

    Select a VPC in the Germany (Frankfurt) region.

    VSwitch

    The vSwitch with which you want to associate the VPN gateway in the associated VPC.

    • If you select Single-tunnel, you need to specify only one vSwitch.

    • If you select Dual-tunnel, you need to specify two vSwitches.

      After the IPsec-VPN feature is enabled, the system creates an elastic network interface (ENI) for each of the two vSwitches as an interface to communicate with the VPC over an IPsec-VPN connection. Each ENI occupies one IP address in the vSwitch.

    Note
    • By default, the system selects a vSwitch. You can change or use the default vSwitch.

    • After a VPN gateway is created, you cannot modify the vSwitch associated with the VPN gateway. You can view the vSwitch associated with the VPN gateway, the zone to which the vSwitch belongs, and the ENI in the vSwitch on the details page of the VPN gateway.

    Select a vSwitch in the associated VPC.

    vSwitch 2

    The other vSwitch with which you want to associate the VPN gateway in the associated VPC.

    • Specify two vSwitches in different zones in the associated VPC to implement disaster recovery across zones for IPsec-VPN connections.

    • For a region that supports only one zone, disaster recovery across zones is not supported. We recommend that you specify two vSwitches in the zone to implement high availability of IPsec-VPN connections. You can also select the same vSwitch as the first one.

    Note

    If only one vSwitch is deployed in the VPC, create a vSwitch. For more information, see Create and manage a vSwitch.

    Select another vSwitch in the associated VPC.

    IPsec-VPN

    Specifies whether to enable IPsec-VPN for the VPN gateway. Default value: Enable.

    Select Enable.

    SSL-VPN

    Specifies whether to enable SSL-VPN for the VPN gateway. Default value: Disable.

    Select Disable.

  5. After the VPN gateway is created, view the VPN gateway on the VPN Gateway page.

    The newly created VPN gateway is in the Preparing state and changes to the Normal state after about 1 to 5 minutes. After the status changes to Normal, the VPN gateway is ready for use.

    The following table describes the two IP addresses assigned by the system to the VPN gateway.

    VPN gateway name

    VPN gateway ID

    IP address

    VPN Gateway

    vpn-gw8dickm386d2qi2g****

    IPsec address 1: 8.XX.XX.130, which is the IP address of the active tunnel by default.

    IPsec address 2: 47.XX.XX.27, which is the IP address of the standby tunnel by default.

Step 2: Deploy VPN resources on AWS

To establish an IPsec-VPN connection between the AWS VPC and the Alibaba Cloud VPC, you must deploy VPN resources on AWS based on the following information. Consult AWS for specific commands or operations.

Note

In this example, the IPsec-VPN connection established between the AWS VPC and the Alibaba Cloud VPC uses static routing. You can also use Border Gateway Protocol (BGP) dynamic routing. For more information, see Use IPsec-VPN to connect Alibaba Cloud VPCs to Amazon VPCs.

  1. Create customer gateways.

    You must create two customer gateways on AWS and use the IP addresses of the Alibaba Cloud VPN gateway as the IP addresses of the customer gateways.客户网关

  2. Create a virtual private gateway.

    You must create a virtual private gateway on AWS and associate the virtual private gateway with the Amazon VPC that needs to communicate with Alibaba Cloud.虚拟私有网关

  3. Create Site-to-Site VPN connections.

    Important
    • Both Alibaba Cloud and AWS IPsec-VPN connections support the dual-tunnel mode. By default, the two tunnels of an AWS IPsec-VPN connection are associated with the same gateway, and the two tunnels of an Alibaba Cloud IPsec-VPN connection have different IP addresses. Therefore, the two tunnels of AWS are connected to only one tunnel of Alibaba Cloud. To ensure that the two tunnels of the Alibaba Cloud IPsec-VPN connection are enabled at the same time, you must create two Site-to-Site VPN connections on AWS and associate the Site-to-Site VPN connections with different customer gateways.

    • When you configure routes for the Site-to-Site VPN connections, you must also specify the CIDR block 100.104.0.0/16 apart from the CIDR block of the Alibaba Cloud VPC. DTS uses the CIDR block 100.104.0.0/16 to synchronize data.

    The following figure shows the configurations of one of the Site-to-Site VPN connections. We recommend that you use the default values for the tunnel configurations. Specify a different customer gateway when you configure the other Site-to-Site VPN connection. Use the same values for other parameters.VPN连接

    After the Site-to-Site VPN connections are created, you can view the tunnel addresses of the connections, which are used to create an IPsec-VPN connection on Alibaba Cloud.隧道地址

    The following table describes the external IP addresses of Tunnel 1 of each Site-to-Site VPN connection and the IP addresses of the associated customer gateways.

    Site-to-Site VPN connection

    Tunnel

    External IP address

    Associated customer gateway IP address

    Site-to-Site VPN Connection 1

    Tunnel 1

    3.XX.XX.5

    8.XX.XX.130

    Site-to-Site VPN Connection 2

    Tunnel 1

    3.XX.XX.239

    47.XX.XX.27

  4. Configure route advertising.

    You must enable route advertising for the route table of the Amazon VPC that is associated with the virtual private gateway to ensure that the routes of the Site-to-Site VPN connections are automatically advertised to the route table of the Amazon VPC.AWS路由传播.png

    The route table of the Amazon VPC contains the CIDR block of the Alibaba Cloud VPC and the CIDR block used by DTS.VPC路由表

Step 3: Deploy the VPN gateway on Alibaba Cloud

After you configure VPN resources on AWS, deploy a VPN gateway on Alibaba Cloud based on the following information to establish an IPsec-VPN connection between the Amazon VPC and Alibaba Cloud VPC.

  1. Creates customer gateways.

    1. Log on to the VPN Gateway console.

    2. In the left-side navigation pane, choose Interconnections > VPN > Customer Gateways.

    3. In the top navigation bar, select the region in which you want to create the customer gateway.

      Make sure that the customer gateway and the VPN gateway to be connected are deployed in the same region.

    4. On the Customer Gateways page, click Create Customer Gateway.

    5. In the Create Customer Gateway panel, configure the following parameters and click OK.

      You must create two customer gateways on Alibaba Cloud and use the external IP addresses of the tunnels of the AWS Site-to-Site VPN connections as the IP addresses of the customer gateways. The following table describes only the parameters that are relevant to this topic. You can use the default values for other parameters or leave them empty. For more information, see Create and manage a customer gateway.

      Important

      Use only the external IP address of Tunnel 1 of each Site-to-Site VPN connection as the customer gateway IP address. By default, the external IP address of Tunnel 2 of each Site-to-Site VPN connection is not used. After the IPsec-VPN connections are created, Tunnel 2 of each Site-to-Site VPN connection is unavailable.

      Parameter

      Description

      Customer Gateway 1

      Customer Gateway 2

      Name

      The name of the customer gateway.

      Enter Customer Gateway 1.

      Enter Customer Gateway 2.

      IP Address

      The external IP address of the AWS tunnel.

      Enter 3.XX.XX.5.

      Enter 3.XX.XX.239.

  2. Create an IPsec-VPN connection.

    1. In the left-side navigation pane, choose Interconnections > VPN > IPsec Connections.

    2. In the top navigation bar, select the region in which you want to create the IPsec-VPN connection.

      Make sure that the IPsec-VPN connection and the VPN gateway are in the same region.

    3. On the IPsec Connections page, click Create IPsec-VPN Connection.

    4. On the Create IPsec-VPN Connection page, configure the IPsec-VPN connection based on the following information and click OK.

      Parameter

      Description

      Example

      Name

      The name of the IPsec-VPN connection.

      Enter IPsec-VPN Connection.

      Resource Group

      The resource group to which the VPN gateway belongs.

      Select the default resource group.

      Associate Resource

      The type of the network resource that you want to associate with the IPsec-VPN connection.

      Select VPN Gateway.

      VPN Gateway

      The VPN gateway that you want to associate with the IPsec-VPN connection.

      Select the VPN gateway that you created.

      Routing Mode

      The routing mode of the traffic. Valid values:

      • Destination Routing Mode: forwards traffic based on the destination IP address.

      • Protected Data Flows: forwards traffic based on the source and destination IP addresses.

      Select Protected Data Flows.

      Local Network

      The CIDR block of the VPC with which the VPN gateway is associated.

      In this example, enter the following two CIDR blocks:

      • CIDR block of the VPC: 10.0.0.0/16

      • CIDR block of DTS servers: 100.104.0.0/16

      Important

      You must add the CIDR block used by DTS to the local network. This way, DTS can access the peer database by using the VPN gateway.

      For more information about the CIDR blocks used by DTS, see Add the CIDR blocks of DTS servers.

      Remote Network

      The peer CIDR block that the VPC associated with the VPN gateway wants to access.

      Enter 192.168.0.0/16.

      Effective Immediately

      Specifies whether to immediately start negotiations for the connection. Valid values:

      • Yes: starts negotiations after the configuration is complete.

      • No: starts negotiations when inbound traffic is detected.

      Select Yes.

      Enable BGP

      Specifies whether to enable BGP. If you want to use BGP routing for the IPsec-VPN connection, turn on Enable BGP. By default, Enable BGP is turned off.

      Turn off Enable BGP.

      Tunnel 1

      Configure VPN parameters for the active tunnel.

      By default, Tunnel 1 serves as the active tunnel and Tunnel 2 serves as the standby tunnel. You cannot modify this configuration.

      Customer Gateway

      The customer gateway that you want to associate with the active tunnel.

      Select Customer Gateway 1.

      Pre-Shared Key

      The pre-shared key of the active tunnel that is used to verify identities.

      • The pre-shared key must be 1 to 100 characters in length, and can contain digits, letters, and the following characters: ~ ` ! @ # $ % ^ & * ( ) _ - + = { } [ ] \ | ; : ' , . < > / ?

      • If you do not specify a pre-shared key, the system generates a random 16-character string as the pre-shared key.

      Important

      The tunnel and the peer gateway device must use the same pre-shared key. Otherwise, the system cannot establish an IPsec-VPN connection.

      Use the same pre-shared key as the key of the AWS tunnel to be connected.

      Encryption Configuration

      Configure the parameters for the Internet Key Exchange (IKE), IPsec, dead peer detection (DPD), and NAT traversal features.

      • The value of the SA Life Cycle (seconds) parameter in the IKE Configurations section must be the same as the value specified on AWS. In this example, the value is set to 28800.

      • The value of the SA Life Cycle (seconds) parameter in the IPsec Configurations section must be the same as the value specified on AWS. In this example, the value is set to 3600.

      Use the default values for other parameters. For more information, see Create and manage IPsec-VPN connections in dual-tunnel mode.

      Tunnel 2

      Configure VPN parameters for the standby tunnel.

      Customer Gateway

      The customer gateway that you want to associate with the standby tunnel.

      Select Customer Gateway 2.

      Pre-Shared Key

      The pre-shared key of the standby tunnel that is used to verify identities.

      Use the same pre-shared key as the key of the AWS tunnel to be connected.

      Encryption Configuration

      Configure the parameters for the IKE, IPsec, DPD, and NAT traversal features.

      • The value of the SA Life Cycle (seconds) parameter in the IKE Configurations section must be the same as the value specified on AWS. In this example, the value is set to 28800.

      • The value of the SA Life Cycle (seconds) parameter in the IPsec Configurations section must be the same as the value specified on AWS. In this example, the value is set to 3600.

      Use the default values for other parameters. For more information, see Create and manage IPsec-VPN connections in dual-tunnel mode.

      Tags

      The tags of the IPsec-VPN connection.

      Leave this parameter empty.

    5. In the Created message, click OK.

  3. Advertise the route of the VPN gateway.

    After you create the IPsec-VPN connection, you must advertise the route of the VPN gateway. If you select Protected Data Flows as Routing Mode, the system creates a policy-based route for the VPN gateway after the IPsec-VPN connection is created. The route is in the Unpublished state. You must advertise the policy-based route of the VPN gateway to the VPC.

    1. In the left-side navigation pane, choose Interconnections > VPN > VPN Gateways.

    2. In the top navigation bar, select the region in which the VPN gateway resides.

    3. On the VPN Gateways page, find the VPN gateway that you want to manage and click its ID.

    4. On the details page of the VPN gateway, click the Policy-based Route Table tab, find the route that you want to manage, and then click Advertise in the Actions column.

    5. In the Advertise Route message, click OK.

    The route table of the Alibaba Cloud VPC contains the CIDR block of the Amazon VPC.VPC路由表1

Step 4: Test network connectivity

After you complete the preceding steps, the resources in the Alibaba Cloud VPC and the Amazon VPC can communicate with each other. The following section describes how to verify the network connectivity between the Alibaba Cloud VPC and the Amazon VPC.

Important

Before you test the connectivity, make sure that you understand the security group rules applied to the Alibaba Cloud VPC and the Amazon VPC. Make sure that the security group rules allow resources in the two VPCs to communicate with each other.

  1. Log on to the Amazon EC2 instance. Consult AWS for specific commands or operations.

  2. Run the ping command on the Amazon EC2 instance to access the internal endpoint of the ApsaraDB RDS for MySQL instance.

    ping <Internal endpoint of the ApsaraDB RDS for MySQL instance>

    If the Amazon EC2 instance receives response packets as shown in the following figure, the resources in the Alibaba Cloud VPC and the Amazon VPC can communicate with each other.测试连通性

Step 5: Create a DTS data synchronization task

After resources in the Alibaba Cloud VPC and the Amazon VPC can communicate with each other, you can create a DTS data synchronization task. Then, you can use the DTS data synchronization task to migrate data from the Amazon EC2 instance to the ApsaraDB RDS for MySQL instance.

  1. Go to the Data Synchronization Tasks page.

    1. Log on to the Data Management (DMS) console.

    2. In the top navigation bar, click DTS.

    3. In the left-side navigation pane, choose DTS (DTS) > Data Synchronization.

    Note
  2. On the right side of Data Synchronization Tasks, select the region in which the data synchronization instance resides.

    Note

    If you use the new DTS console, you must select the region in which the data synchronization instance resides in the top navigation bar.

  3. Click Create Task. On the Create Data Synchronization Task page, configure the source and destination databases. The following table describes the parameters.

    Warning

    After you configure the source and destination databases, we recommend that you read the Limits that are displayed in the upper part of the page. Otherwise, the task may fail or data inconsistency may occur.

    Section

    Parameter

    Description

    N/A

    Task Name

    The name of the task. DTS automatically assigns a name to the task. We recommend that you specify a descriptive name that makes it easy to identify the task. You do not need to specify a unique task name.

    Source Database

    Database Type

    The type of the source instance. Select MySQL.

    Access Method

    The access method of the source database. Select Express Connect, VPN Gateway, or Smart Access Gateway.

    Instance Region

    The region in which the source database resides.

    Select Germany (Frankfurt).

    Replicate Data Across Alibaba Cloud Accounts

    Specifies whether to synchronize data across Alibaba Cloud accounts. Select No.

    Connected VPC

    The VPC with which the VPN gateway is associated.

    DTS accesses the database on the Amazon EC2 instance by using an IPsec-VPN connection.

    IP Address or Domain Name

    The host IP address of the source MySQL database.

    Enter the private IP address 192.168.30.158 of the Amazon EC2 instance.

    Port Number

    The service port number of the source MySQL database. Enter 3306.

    Database Account

    The username that is used to access the source database.

    Database Password

    The password that is used to access the source database.

    Encryption

    Specifies whether to encrypt the connection to the source database. Use the default value Non-encrypted.

    Destination Database

    Database Type

    The type of the destination database. Select MySQL.

    Access Method

    The access method of the destination database. Select Alibaba Cloud Instance.

    Instance Region

    The region in which the ApsaraDB RDS for MySQL instance resides.

    Select Germany (Frankfurt).

    Replicate Data Across Alibaba Cloud Accounts

    Specifies whether to synchronize data across Alibaba Cloud accounts. Select No.

    RDS Instance ID

    The ID of the ApsaraDB RDS for MySQL instance.

    Database Account

    The username that is used to access the destination database. Enter the database account of the ApsaraDB RDS for MySQL instance.

    Database Password

    The password that is used to access the destination database.

    Encryption

    Specifies whether to encrypt the connection to the destination database. Use the default value Non-encrypted.

  4. In the lower part of the page, click Test Connectivity and Proceed. In the CIDR Blocks of DTS Servers dialog box, click Test Connectivity.

    Make sure that the security group rules applied to the Alibaba Cloud VPC and the Amazon VPC allow access from DTS. For example, a security group rule is configured to allow access from resources in the CIDR block 100.104.0.0/16. For more information about the CIDR blocks used by DTS, see Add the CIDR blocks of DTS servers.

    • If the databases in the Alibaba Cloud VPC and the Amazon VPC can be connected, the network connectivity between the databases is normal. Then, you are navigated to the Configure Task Object page.

    • If the databases in the Alibaba Cloud VPC and the Amazon VPC cannot be properly connected, you are not navigated to the next page. To troubleshoot the issue, follow the instructions on the page. For more information, see What do I do if an error is reported when I connect a database instance to DTS over VPN?

  5. In the Configure Task Objects step, select the database on the Amazon EC2 instance from which you want to synchronize data to Alibaba Cloud. Use the default settings for other parameters in this and subsequent steps. For more information, see Synchronize data from a self-managed MySQL database connected over Express Connect, VPN Gateway, or Smart Access Gateway to an ApsaraDB RDS for MySQL instance.

    同步的表

    After the configuration is complete, DTS automatically starts the data synchronization task. You can log on to the ApsaraDB RDS for MySQL instance to view the data synchronization results.数据同步结果

References