AnalyticDB:Use zero-ETL to synchronize data

Create a service-linked role for AnalyticDB for MySQL

1. Log on to the Resource Access Management (RAM) console. In the left-side navigation pane, choose Identities > Roles.

2. Check whether a service-linked role named AliyunServiceRoleForAnalyticDBForMySQL exists in the role list. If the service-linked role does not exist, create the role.

3. In the upper-left corner of the Roles page, click Create Role.

4. In the Select Role Type step, select Alibaba Cloud Service and click Next.

5. In the Configure Role step, select Service Linked Role for the Role Type parameter and select AnalyticDB for MySQL from the Select Service drop-down list.

6. Click OK. Return to the Roles page and check whether the service-linked role is created.

Grant management permissions to a RAM user

To allow a RAM user to create and manage zero-ETL tasks, you must perform the following operations:

Grant the RAM user the permissions to manage AnalyticDB for MySQL

Before a RAM user can create and manage zero-ETL tasks of AnalyticDB for MySQL, you must attach the AliyunADBFullAccess policy of AnalyticDB for MySQL to the RAM user. For more information, see Grant permissions to a RAM user.

Grant the RAM user the permissions to manage the zero-ETL feature

Before a RAM user can create and manage zero-ETL tasks, the RAM user must have the permissions to create data synchronization tasks between the data source and AnalyticDB for MySQL. You can configure a custom policy that allows RAM users to create data synchronization tasks for all instances or clusters of the data source and AnalyticDB for MySQL. You can also configure a custom policy that allows RAM users to create data synchronization tasks for specific instances or clusters. Then, you can attach the custom policy to the RAM user. For more information, see Create custom policies.

The following section describes the scripts of custom policies:

{
    "Version": "1",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "dts:*",
            "Resource": [
                "acs:adb:*:*:*",
                "acs:rds:*:*:*",
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "dts:DescribeRegions",
                "dts:DescribeConfigRelations",
                "dts:DescribeSrcLinkConfig",
                "dts:DescribeDestLinkConfig",
                "dts:DescribeLinkConfig"
            ],
            "Resource": [
                "acs:dts:*:*:*"
            ]
        }
    ]
}

{
    "Version": "1",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "dts:*",
            "Resource": [
                "acs:adb:*:*:dbcluster/am-2zeod8ax4b9a****",
                "acs:rds:*:*:dbinstance/rm-2ze6fs8ouh43****",
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "dts:DescribeRegions",
                "dts:DescribeConfigRelations",
                "dts:DescribeSrcLinkConfig",
                "dts:DescribeDestLinkConfig",
                "dts:DescribeLinkConfig"
            ],
            "Resource": "acs:dts:*:*:*"
        }
    ]
}

Procedure

1. Log on to the AnalyticDB for MySQL console. In the upper-left corner of the console, select a region. In the left-side navigation pane, click Clusters. On the Clusters page, click an edition tab. Find the cluster that you want to manage and click the cluster ID.

2. Go to the zero-ETL configuration page based on the cluster edition.

   • For a Data Lakehouse Edition cluster: In the left-side navigation pane, choose Data Ingestion > Database Data Synchronization.

   • For a Data Warehouse Edition cluster: In the left-side navigation pane, choose Data Transmission Service > Zero-ETL.

3. Click Create Zero-ETL Task. In the Configure Source and Destination Databases step of the Create Zero-ETL Task page, configure the parameters of the source instance and the destination cluster.

   • The following table describes the parameters of the source instance.

     Parameter	Description
     Task Name	The name of the zero-ETL task.
     Database Type	The database engine of the source instance. Select RDS for MySQL.
     Access Method	The access method of the source instance. The value is automatically set to Alibaba Cloud Instance.
     Instance Region	The region in which the ApsaraDB RDS for MySQL instance resides.
     RDS Instance ID	The ID of the ApsaraDB RDS for MySQL instance.
     Database Account	The name of the database account of the ApsaraDB RDS for MySQL instance.
     Database Password	The password of the database account of the ApsaraDB RDS for MySQL instance.
     Encryption	The encryption method that is used to connect to the ApsaraDB RDS for MySQL instance. Select Non-encrypted or SSL-encrypted. Note If you select SSL-encrypted, you must first enable the SSL encryption feature for the ApsaraDB RDS for MySQL instance. For more information, see Use a cloud certificate to enable SSL encryption.

   • The following table describes the parameters of the destination cluster.

     Parameter	Description
     Database Type	The database engine of the destination cluster. The value is automatically set to AnalyticDB for MySQL 3.0.
     Access Method	The access method of the destination cluster. The value is automatically set to Alibaba Cloud Instance.
     Instance Region	The region in which the AnalyticDB for MySQL cluster resides.
     Instance ID	The ID of the AnalyticDB for MySQL cluster.
     Database Account	The name of the database account of the AnalyticDB for MySQL cluster.
     Database Password	The password of the database account of the AnalyticDB for MySQL cluster.

4. After you configure the preceding parameters, click Test Connectivity and Proceed. In the Configure Zero-ETL step, configure the parameters that are described in the following table.

   Parameter	Description
   DDL and DML Operations to Be Synchronized	The DML operations (INSERT, UPDATE, and DELETE) and DDL operations (CREATE, ALTER, DROP, and TRUNCATE) that you want to synchronize. By default, all operations are selected.
   Source Objects and Selected Objects	The source objects and the objects that you want to synchronize.
   Advanced Settings (Optional)	The retry time for failed connections between the source and destination instances and clusters and the retry time for other issues that occur on the source and destination instances and clusters.

5. After you configure the preceding parameters, click Next: Configure Database and Table Fields. In the Configure Database and Table Fields step, configure the parameters that are described in the following table.

   Parameter	Description
   Database Name	The name of the selected database.
   Table Name	The name of the selected table.
   Primary Key Column	The primary key column of the selected table.
   Distribution Key	The distribution key column of the selected table.
   Type	The type of the selected table, which can be a partitioned table or a replicated table.
   Definition Status	The status of the selected table. After you configure the table fields, the status of the table changes from Undefined to Defined.

6. After you configure the preceding parameters, click Next: Save Task Settings and Precheck.

7. If the precheck is successful, click Start to start the zero-ETL task.

   You can view the Name, Source/Destination, and Status parameters of the zero-ETL task on the zero-ETL configuration page.

1. Log on to the CloudMonitor console.

2. View the monitoring information.

   1. In the left-side navigation pane, choose Cloud Service Monitoring > Cloud Service Monitoring.

   2. Move the pointer over the AnalyticDB for MySQL card and click AnalyticDB for mysql 3.0 - ZeroETL Latency.

   3. Find the cluster that you want to manage and click the cluster ID to view the monitoring information about zero-ETL tasks.

3. Create an alert rule.

   1. In the left-side navigation pane, choose Alerts > Alert Rules.

   2. Click Create Alert Rule. In the Create Alert Rule panel, configure alert rule parameters for zero-ETL tasks. For more information, see Create an alert rule.

      Note

      You must select AnalyticDB for mysql 3.0 - ZeroETL Latency for the Product parameter.

4. Create a subscription policy.

   1. In the left-side navigation pane, choose Event Center > Event Subscription.

   2. Click Create Subscription Policy. On the Create Subscription Policy page, configure subscription policy parameters for zero-ETL tasks. For more information, see the "Create a subscription policy" section of the Manage event subscription policies (recommended) topic.

      Note

      • Products: Select AnalyticDB for MySQL V3.0.

      • Event Type: Select Abnormal or Restore.

      • Event Name: Select zero_etl_job_abnormal or zero_etl_job_restore.