AnalyticDB:Synchronize data using seamless integration (Zero-ETL)

Create a service-linked role for AnalyticDB for MySQL

1. Navigate to the Roles list in the Resource Access Management (RAM) console.

2. Check whether a service-linked role named AliyunServiceRoleForAnalyticDBForMySQL exists in the role list. If not, create the role.

3. In the navigation pane on the left, click Create Role.

4. In the Create Role dialog box, select Alibaba Cloud Service, and click Next.

5. Set Role Type to Service-Linked Role, and select AnalyticDB for MySQL.

6. Click OK. Return to the role list and verify that the service-linked role is created.

Grant management permissions to a RAM user

A Resource Access Management (RAM) user needs the following two permissions to create and manage Zero-ETL pipelines.

{
    "Version": "1",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "dts:*",
            "Resource": [
                "acs:adb:*:*:*",
                "acs:rds:*:*:*",
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "dts:DescribeRegions",
                "dts:DescribeConfigRelations",
                "dts:DescribeSrcLinkConfig",
                "dts:DescribeDestLinkConfig",
                "dts:DescribeLinkConfig"
            ],
            "Resource": [
                "acs:dts:*:*:*"
            ]
        }
    ]
}

{
    "Version": "1",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "dts:*",
            "Resource": [
                "acs:adb:*:*:dbcluster/am-2zeod8ax4b9a****",
                "acs:rds:*:*:dbinstance/rm-2ze6fs8ouh43****",
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "dts:DescribeRegions",
                "dts:DescribeConfigRelations",
                "dts:DescribeSrcLinkConfig",
                "dts:DescribeDestLinkConfig",
                "dts:DescribeLinkConfig"
            ],
            "Resource": "acs:dts:*:*:*"
        }
    ]
}

Procedure

1. Log on to the AnalyticDB for MySQL console. In the upper-left corner of the console, select a region. In the left-side navigation pane, click Clusters. Find the cluster that you want to manage and click the cluster ID.

2. Navigate to the seamless integration (Zero-ETL) page. The navigation path varies depending on the product edition.

   • For Enterprise Edition, Basic Edition, or Data Lakehouse Edition clusters: In the navigation pane on the left, choose Data Ingestion > Database Data Synchronization.

   • For Data Warehouse Edition clusters: In the navigation pane on the left, choose Data Transmission Service > Zero-ETL.

3. Click Create Zero-ETL Task. On the Create Zero-ETL Task page, configure the source and destination database information.

   • Configure the source database information:

     Source Database Information	Description
     Task Name	The name of the Zero-ETL task.
     Database Type	Select RDS for MySQL.
     Access Method	Only Alibaba Cloud Instance is supported.
     Instance Region	The region where the RDS for MySQL instance resides.
     RDS Instance ID	The ID of the RDS for MySQL instance.
     Database Account	The database account of the RDS for MySQL instance.
     Database Password	The password for the database account of the RDS for MySQL instance.
     Encryption	The method to connect to the RDS for MySQL instance. Options include Unencrypted Connection and SSL Secure Connection. Note If you set this parameter to SSL-encrypted, you must enable SSL encryption for the RDS for MySQL instance in advance. For more information, see Use a cloud certificate to quickly enable SSL encryption.

   • Configure the destination database information:

     Destination Database Information	Description
     Database Type	Currently, only AnalyticDB for MySQL 3.0 is supported as the destination database.
     Access Method	Only Alibaba Cloud Instance is supported.
     Instance Region	The region where the AnalyticDB for MySQL cluster resides.
     Instance ID	The ID of the AnalyticDB for MySQL cluster.
     Database Account	The database account of the AnalyticDB for MySQL cluster.
     Database Password	The password for the database account of the AnalyticDB for MySQL cluster.

4. After you configure the preceding parameters, click Test Connectivity and Proceed. On the Zero-ETL configuration page, configure the following parameters:

   Configuration Item	Description
   DDL and DML Operations to Be Synchronized	Select the DML and DDL operations to synchronize. By default, all operations are selected.
   Source Objects and Selected Objects	The source objects and the objects to be synchronized in the database.
   Advanced Settings (Optional)	Set the retry time for when the source or destination database cannot be connected and for when other issues occur with the source or destination database.

5. After you configure the preceding parameters, click Next: Configure Database and Table Fields, and configure the following information:

   Database, Table, and Field Configuration	Description
   Database Name	Select an existing database.
   Table Name	Select an existing data table.
   Primary Key Column	The field that serves as the primary key in the selected data table.
   Distribution Key	The field that serves as the distribution key in the selected data table.
   Type	The type of the data table. Options include Partitioned Table and Replicated Table.
   Definition Status	After the database, table, and field configurations are defined, the status changes from Undefined to Defined.

6. After you configure all the preceding parameters, click Next: Save Task Settings and Precheck.

7. After the precheck passes, click Start to start the Zero-ETL task.

   On the Seamless Data Integration page, you can view information about the Zero-ETL task, such as its ID/Name, Source/Destination, and Status.

1. Log on to the Cloud Monitor console.

2. View monitoring information.

   1. In the navigation pane on the left, choose Resource Monitoring > Cloud Service Monitoring.

   2. Hover over the AnalyticDB for MySQL card and click AnalyticDB for mysql 3.0 - ZeroETL Latency.

   3. Click a cluster ID to view the monitoring information for the Zero-ETL tasks in that cluster.

3. Create an alert rule.

   1. In the navigation pane on the left, click Alerts > Alert Rules.

   2. Click Create Alert Rule. In the Create Alert Rule panel, configure an alert rule for the Zero-ETL task. For more information, see Create an alert rule.

      Note

      Set Product to AnalyticDB for MySQL 3.0 - ZeroETL Latency.

4. Create a subscription policy.

   1. In the navigation pane on the left, click Event Hub > Event Subscriptions.

   2. Click Create Subscription Policy. On the Create Subscription Policy page, configure a subscription policy for the Zero-ETL task. For more information, see Create an event subscription.

      Note

      • Product: Select AnalyticDB for MySQL 3.0.

      • Event Type: Select Abnormal or Restore.

      • Event Name: Select ZeroETL Task Abnormal or ZeroETL Task Recovered.