AnalyticDB:Use zero-ETL to synchronize data

Create a service-linked role for AnalyticDB for MySQL

1. Log on to the Resource Access Management (RAM) console. In the left-side navigation pane, choose Identities > Roles.

2. Check whether a service-linked role named AliyunServiceRoleForAnalyticDBForMySQL exists in the role list. If the service-linked role does not exist, you must create the role.

3. In the upper-left corner of the Roles page, click Create Role.

4. In the Select Role Type step, select Alibaba Cloud Service and click Next.

5. In the Configure Role step, select Service Linked Role for the Role Type parameter and select AnalyticDB for MySQL from the Select Service drop-down list.

6. Click OK. Return to the Roles page and check whether the service-linked role is created.

Grant management permissions to a RAM user

To allow a RAM user to create and manage zero-ETL tasks, you must perform the following operations:

Grant the RAM user the permissions to manage AnalyticDB for MySQL

Before a RAM user can create and manage zero-ETL tasks of AnalyticDB for MySQL, you must attach the AliyunADBFullAccess policy of AnalyticDB for MySQL to the RAM user. For more information, see Grant permissions to a RAM user.

Grant the RAM user the permissions to manage the zero-ETL feature

Before a RAM user can create and manage zero-ETL tasks, the RAM user must have the permissions to create data synchronization tasks between the data source and AnalyticDB for MySQL. You can configure a custom policy that allows RAM users to create data synchronization tasks for all instances or clusters of the data source and AnalyticDB for MySQL. You can also configure a custom policy that allows RAM users to create data synchronization tasks for specific instances or clusters. Then, you can attach the custom policy to the RAM user. For more information, see Create custom policies.

The following section describes the scripts of custom policies:

{
    "Version": "1",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "dts:*",
            "Resource": [
                "acs:adb:*:*:*",
                "acs:rds:*:*:*",
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "dts:DescribeRegions",
                "dts:DescribeConfigRelations",
                "dts:DescribeSrcLinkConfig",
                "dts:DescribeDestLinkConfig",
                "dts:DescribeLinkConfig"
            ],
            "Resource": [
                "acs:dts:*:*:*"
            ]
        }
    ]
}

{
    "Version": "1",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "dts:*",
            "Resource": [
                "acs:adb:*:*:dbcluster/am-2zeod8ax4b9a****",
                "acs:rds:*:*:dbinstance/rm-2ze6fs8ouh43****",
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "dts:DescribeRegions",
                "dts:DescribeConfigRelations",
                "dts:DescribeSrcLinkConfig",
                "dts:DescribeDestLinkConfig",
                "dts:DescribeLinkConfig"
            ],
            "Resource": "acs:dts:*:*:*"
        }
    ]
}

Procedure

1. Log on to the AnalyticDB for MySQL console. In the upper-left corner of the console, select a region. In the left-side navigation pane, click Clusters. On the Clusters page, click an edition tab. Find the cluster that you want to manage and click the cluster ID.

2. Go to the zero-ETL configuration page based on the cluster edition.

   • For a Data Lakehouse Edition cluster: In the left-side navigation pane, choose Data Ingestion > Database Data Synchronization.

   • For a Data Warehouse Edition cluster: In the left-side navigation pane, choose Data Transmission Service > Zero-ETL.

3. Click Create Zero-ETL Task. In the Configure Source and Destination Databases step of the Create Zero-ETL Task page, configure the parameters of the source and destination databases.

   • The following table describes the parameters of the source database.

     Parameter	Description
     Task Name	The name of the zero-ETL task.
     Database Type	The database engine of the source database. Select RDS for MySQL.
     Access Method	The access method of the source database. The value is automatically set to Alibaba Cloud Instance.
     Instance Region	The region in which the ApsaraDB RDS for MySQL instance resides.
     RDS Instance ID	The ID of the ApsaraDB RDS for MySQL instance.
     Database Account	The name of the database account of the ApsaraDB RDS for MySQL instance.
     Database Password	The password of the database account of the ApsaraDB RDS for MySQL instance.
     Encryption	The encryption method that is used to connect to the ApsaraDB RDS for MySQL instance. Valid values: Non-encrypted and SSL-encrypted. Note If you select SSL-encrypted, you must first enable the SSL encryption feature for the ApsaraDB RDS for MySQL instance. For more information, see Use a cloud certificate to enable SSL encryption.

   • The following table describes the parameters of the destination database.

     Parameter	Description
     Database Type	The database engine of the destination database. The value is automatically set to AnalyticDB for MySQL 3.0.
     Access Method	The access method of the destination database. The value is automatically set to Alibaba Cloud Instance.
     Instance Region	The region in which the AnalyticDB for MySQL cluster resides.
     Instance ID	The ID of the AnalyticDB for MySQL cluster.
     Database Account	The name of the database account of the AnalyticDB for MySQL cluster.
     Database Password	The password of the database account of the AnalyticDB for MySQL cluster.

4. After you configure the preceding parameters, click Test Connectivity and Proceed. In the Configure Zero-ETL step, configure the parameters that are described in the following table.

   Parameter	Description
   Synchronization Types	The type of the data synchronization between the source and destination databases. Valid values: Schema Synchronization, Full Data Synchronization, and Incremental Data Synchronization. Note For synchronization tasks, Incremental Data Synchronization is automatically selected and cannot be cleared. Incremental data of the source database is synchronized to the destination database in real time. If you do not want to synchronize incremental data, use the data migration feature.
   DDL and DML Operations to Be Synchronized	The DML operations (INSERT, UPDATE, and DELETE) and DDL operations (CREATE, ALTER, DROP, and TRUNCATE) that you want to synchronize. By default, all operations are selected.
   Source Objects and Selected Objects	The objects in the source database and the objects that you want to synchronize.
   Advanced Settings (Optional)	The retry time for failed connections between the source and destination databases and the retry time for other issues that occur on the source and destination databases.

5. After you configure the preceding parameters, click Next: Configure Database and Table Fields. In the Configure Database and Table Fields step, configure the parameters that are described in the following table.

   Parameter	Description
   Database Name	The name of the selected database.
   Table Name	The name of the selected table.
   Primary Key Column	The primary key column of the selected table.
   Distribution Key	The distribution key column of the selected table.
   Type	The type of the table, which can be a partitioned table or a replicated table.
   Definition Status	The status of the selected table. After you configure the table fields, the status of the table changes from Undefined to Defined.

6. After you configure the preceding parameters, click Next: Save Task Settings and Precheck.

7. If the precheck is successful, click Start to start the zero-ETL task.

   You can view the Name, Source/Destination, and Status parameters of the zero-ETL task on the zero-ETL configuration page.