DataWorks:Add an AnalyticDB for MySQL V3.0 data source

Prerequisites

• An AnalyticDB for MySQL V3.0 cluster is purchased, and an AnalyticDB for MySQL V3.0 database is created in the cluster. For more information, see Create a cluster and CREATE DATABASE.

  Note

  We recommend that you create an AnalyticDB for MySQL V3.0 cluster in the same region as the workspace to which you want to add an AnalyticDB for MySQL V3.0 data source. If the regions are different, you can add only a cross-region data source to the workspace. The data source cannot be associated with DataStudio for data development or periodic task scheduling. The data source can be used only for data synchronization.

• The required resource group is purchased and configured.

  An AnalyticDB for MySQL V3.0 data source supports serverless resource groups and old-version exclusive resource groups. Old-version exclusive resource groups include exclusive resource groups for Data Integration, exclusive resource groups for scheduling, and exclusive resource groups for DataService Studio. We recommend that you use serverless resource groups. After the AnalyticDB for MySQL V3.0 data source is added, you can use the data source in scenarios such as data synchronization, development and scheduling of computing tasks, and generation of DataService Studio APIs.

  Note

  You must purchase and configure the required resource group based on the use scenario of the AnalyticDB for MySQL V3.0 data source and establish a network connection between the data source and the resource group in advance. For information about resource groups provided by DataWorks and how to configure a resource group, see Overview.

• A DataWorks workspace is created, or the account that you use is added to the desired workspace as a member.

  You must add the desired AnalyticDB for MySQL V3.0 cluster to the workspace as a data source. This way, you can use the data source to perform data development operations in the workspace. In addition, you must associate the purchased resource group with the workspace and establish a network connection between the resource group and the data source. For information about how to create a workspace, see Create and manage workspaces.

  Note

  You can add the same AnalyticDB for MySQL V3.0 cluster to multiple DataWorks workspaces as a data source.

Limits

• An AnalyticDB for MySQL V3.0 data source can be associated with DataStudio only if the data source meets the following conditions: The AnalyticDB for MySQL V3.0 cluster based on which the data source is added resides in the same region and belongs to the same Alibaba Cloud account as the workspace. This way, the AnalyticDB for MySQL V3.0 data source can be used for data development and periodic task scheduling.

• You can add an AnalyticDB for MySQL V3.0 cluster that does not belong to the current Alibaba Cloud account to a workspace within the current Alibaba Cloud account as a data source. After the data source is added, you can use only a RAM role to access the related AnalyticDB for MySQL V3.0 cluster. AnalyticDB for MySQL V3.0 data sources that are added across accounts cannot be used for data development or periodic task scheduling. These data sources can be used only for data synchronization.

• AnalyticDB for MySQL V3.0 data sources that are added in connection string mode cannot be used for data development or periodic task scheduling. These data sources can be used only for data synchronization. For more information, see Add a data source.

• You can use only a serverless resource group or an exclusive resource group for Data Integration or exclusive resource group for scheduling to run an AnalyticDB for MySQL V3.0 task. We recommend that you use a serverless resource group.

• If you want to create APIs in DataService Studio based on an AnalyticDB for MySQL V3.0 data source, we recommend that you use a serverless resource group.

Preparations: Permission description and configuration

• If you want to add a data source as a RAM user or by using a RAM role, make sure that the RAM user or RAM role meets one of the following requirements:

  • The RAM user or RAM role is added to the desired workspace as a member and is assigned the Workspace Owner, Workspace Administrator, or O&M role. For more information, see Add a RAM user to a workspace as a member and assign roles to the member.

  • The RAM user or RAM role is attached the AliyunDataWorksFullAccess or AdministratorAccess policy. For more information, see Grant permissions to a RAM user or Grant permissions to a RAM role.

• If you want to add a data source across accounts, you must also make the following permission configurations:

  If you add an AnalyticDB for MySQL V3.0 data source across Alibaba Cloud accounts, you can use only a RAM role that is granted the required permissions to access the related AnalyticDB for MySQL V3.0 cluster.

  Sample scenario

  In this sample scenario, an AnalyticDB for MySQL V3.0 cluster that belongs to Alibaba Cloud Account B is added to a DataWorks workspace that belongs to Alibaba Cloud Account A as a data source, and Alibaba Cloud Account A needs to access the AnalyticDB for MySQL V3.0 cluster.

  • Alibaba Cloud Account A: A DataWorks workspace is created within Alibaba Cloud Account A, and Alibaba Cloud Account A needs to access an AnalyticDB for MySQL V3.0 cluster that is created within Alibaba Cloud Account B.

  • Alibaba Cloud Account B: An AnalyticDB for MySQL V3.0 cluster is created within Alibaba Cloud Account B, and an AnalyticDB for MySQL V3.0 database is created in the cluster. In addition, a RAM role is created within Alibaba Cloud Account B and is assigned to Alibaba Cloud Account A. This way, Alibaba Cloud Account A can access the AnalyticDB for MySQL V3.0 cluster by assuming the RAM role.

  After the preceding operations are performed, you can use Alibaba Cloud Account A to add the AnalyticDB for MySQL V3.0 cluster to DataWorks as a data source and then access the AnalyticDB for MySQL V3.0 cluster across Alibaba Cloud accounts.

  Create and grant permissions to a RAM role

  Requirements on a RAM role of Alibaba Cloud Account B and permission configuration for the RAM role:

  1. You must create a RAM role within Alibaba Cloud Account B and grant the RAM role the permissions to access a specified AnalyticDB for MySQL V3.0 cluster. You must add Alibaba Cloud Account A as the trusted cloud account of the RAM role to allow Alibaba Cloud Account A to assume the RAM role. For more information, see Create a RAM role for a trusted Alibaba Cloud account and Create custom policies.

  2. You must modify the trust policy of the RAM role to allow Alibaba Cloud Account A to assume the RAM role to access the AnalyticDB for MySQL V3.0 cluster. For more information, see Edit the trust policy of a RAM role. The following code shows the document of the trust policy:

     {
         "Version": "1",
         "Statement": [
             {
                 "Action": [
                     "sts:AssumeRole",
                     "adb:DescribeDBClusters",
                     "adb:DescribeDBClusterAttribute",
                     "adb:DescribeDBClusterAccessWhiteList",
                     "adb:ModifyDBClusterAccessWhiteList",
                     "adb:DescribeDBClusterNetInfo"
                 ],
                 "Effect": "Allow",
                 "Principal": {
                     "Service": [
                         "ID of Alibaba Cloud Account A@engine.dataworks.aliyuncs.com"
                     ]
                 }
             }
         ]
     }

Add a data source

DataWorks allows you to add a data source in one of the following modes.

What to do next

After the data source is added, you can perform the following operations based on your business requirements:

• Develop and schedule computing tasks:

  DataWorks DataStudio and Operation Center provide the capabilities of developing and scheduling AnalyticDB for MySQL V3.0 tasks. If you want to develop AnalyticDB for MySQL V3.0 tasks based on the AnalyticDB for MySQL V3.0 data source or periodically schedule AnalyticDB for MySQL V3.0 tasks, you must go to the DataStudio page in the DataWorks console and associate the AnalyticDB for MySQL V3.0 data source with DataStudio.

  Note

  You can associate an AnalyticDB for MySQL V3.0 data source with DataStudio only if the AnalyticDB for MySQL V3.0 cluster based on which the data source is added resides in the same region and belongs to the same Alibaba Cloud account as the workspace to which the data source is added.

• Perform data synchronization:

  DataWorks Data Integration provides AnalyticDB for MySQL V3.0 Reader and AnalyticDB for MySQL V3.0 Writer for you to read data from and write data to the AnalyticDB for MySQL V3.0 data source. You can configure a batch or real-time synchronization task for the AnalyticDB for MySQL V3.0 data source in DataStudio or configure a synchronization task for the AnalyticDB for MySQL V3.0 data source in Data Integration based on your business requirements to perform data synchronization.

• Manage the data source: You can go to the Data Sources page in Management Center to perform management operations on the data source. For example, you can edit or delete the data source.