DataWorks:Use the RAM role-based authorization mode to add a data source

Process

Procedure

1. Create a role to be assumed and a policy to be attached and attach the policy to the role.

   You can create different custom roles for different data sources based on your security requirements. The following scenario is used as an example on how to create a role to be assumed:

   Note

   Only an Alibaba Cloud account or a RAM user to which the AliyunRAMFullAccess policy is attached can be used to perform the operations in this step.

   An enterprise uses 100 OSS buckets to store all data, and the big data team needs to use data that is stored in a specific OSS bucket. If the preset role AliyunDataWorksAccessingOSSRole is used, data in the other 99 OSS buckets may be accessed by the big data team. This may cause data leakage in the buckets.

   In this case, the owner of an Alibaba Cloud account can create a custom role named BigDataOSSRole for the big data team and allows only the members of the big data team to use the role. This helps isolate permissions across teams.

   1. Create a custom role.

      In this example, a custom role whose trusted entity is an Alibaba Cloud account and whose name is BigDataOssRole is created. For more information about how to create a custom role, see Create a RAM role for a trusted Alibaba Cloud account.

   2. Create a custom policy.

      In this example, a policy that allows users to read data from and write data to specified buckets is created. For more information about how to create a custom policy, see Create custom policies. The following code shows the document of the policy:

      {
          "Version": "1",
          "Statement": [
              {
                  "Effect": "Allow",
                  "Action": [
                      "oss:GetObject",
                      "oss:ListObjects",
                      "oss:GetObjectMetadata",
                      "oss:GetObjectMeta",
                      "oss:GetBucketAcl",
                      "oss:GetBucketInfo",
                      "oss:PutObject",
                      "oss:DeleteObject",
                      "oss:PutBucket"
                  ],
                  "Resource": [
                      "acs:oss:*:*:bucket_name_1",
                      "acs:oss:*:*:bucket_name_1/*"
                  ]
              }
          ]
      }

   3. Attach the policy to the BigDataOSSRole role.

      Modify the trust policy of the BigDataOSSRole role. Attach the created policy to the BigDataOSSRole role. This way, the user that is assigned the BigDataOSSRole role can read data from and write data to the two specified buckets.

      Important

      To use the role, you must perform the operations in this step.

      For more information about how to modify the trust policy of a role, see Edit the trust policy of a RAM role. The following code shows the document of the policy:

      {
          "Statement": [
              {
                  "Action": "sts:AssumeRole",
                  "Effect": "Allow",
                  "Principal": {
                      "Service": [
                          "di.dataworks.aliyuncs.com"
                      ]
                  }
              }
          ],
          "Version": "1"
      }

2. Allow specific users to assume the role.

   After you determine the role to be assumed, you must attach a policy that contains the PassRole permission to specific users. This way, the users can assume the role to add a data source and run a synchronization task for the data source. You can also establish mappings between users and roles based on your business requirements.

   • Policy template 1: You can create a policy based on the following template. The policy allows authorized users to assume all roles that are related to DataWorks Data Integration. Create a policy by using the template only if it is necessary for your business.

     {
         "Version": "1",
         "Statement": [
             {
                 "Action": "ram:PassRole",
                 "Resource": "*",
                 "Effect": "Allow",
                 "Condition": {
                     "StringEquals": {
                         "acs:Service": "di.dataworks.aliyuncs.com"
                     }
                 }
             }
         ]
     }

   • Policy template 2: You can also create a custom policy that contains the PassRole permission. Then, you can establish mappings between users and roles based on your business requirements.

     Note

     Only an Alibaba Cloud account or a RAM user to which the AliyunRAMFullAccess policy is attached can be used to perform the operations in this step.

     In this example, after you create the BigDataOSSRole role for the big data team, you must use the following policy to allow specific users to assume the role based on your business requirements. You can create a custom policy named BigDataOSSRoleAllowUse and attach the policy to specific users. This way, the users can assume the BigDataOSSRole role.

     Create a policy named BigDataOssRoleAllowUse. For more information, see Create custom policies. The following code shows the document of the policy:

     {
         "Version": "1",
         "Statement": [
             {
                 "Action": "ram:PassRole",
                 "Resource": "acs:ram::19122324****:role/BigDataOssRole",
                 "Effect": "Allow",
                 "Condition": {
                     "StringEquals": {
                         "acs:Service": [
                             "oss.aliyuncs.com",
                             "di.dataworks.aliyuncs.com"
                         ]
                     }
                 }
             }
         ]
     }

     Note

     Replace the UID 19122324**** in the preceding code with the UID of your Alibaba Cloud account.

     After you create the BigDataOssRoleAllowUse policy, you can attach the policy to the RAM users who want to assume the BigDataOssRole role. This way, the RAM users can assume the BigDataOssRole role as the access identity to add data sources and run synchronization tasks for the data sources.

3. Add a data source.

   After you are granted the required permissions by the owner of an Alibaba Cloud account, you can add a data source.

   1. Use your Alibaba Cloud account or a RAM user to which the DataWorksFullAccess policy is attached to add an OSS data source.

      In the Add OSS Data Source dialog box, select RAM Role Authorization Mode for Access Mode and configure other parameters based on your business requirements. The following table describes the parameters. If you use a workspace in standard mode, you can determine whether to add the data source in the development or production environment.

      Note

      In this example, an OSS data source is added. The parameters that you need to configure vary based on the data source type. For more information about how to add an OSS data source, see Add an OSS data source.

      

      Parameter	Description
      Data Source Name	The name of the data source. The name can contain only letters, digits, and underscores (_), and must start with a letter.
      Data Source Description	The description of the data source. The description cannot exceed 80 characters in length.
      Region	Select a region from the drop-down list. Note
      Endpoint	The endpoint of OSS. Endpoint format: http://oss.aliyuncs.com. The endpoint of OSS varies based on the region. Note If you add a bucket name before the endpoint of OSS and a period (.) after the bucket name, the data source can pass the connectivity test, but data synchronization will fail. For example, you cannot set this parameter to http://xxx.oss.aliyuncs.com.
      Bucket	The name of the OSS bucket. A bucket is a container that is used to store objects in OSS. You can create one or more buckets and add one or more objects to each bucket. During data synchronization, DataWorks can search for objects only in the bucket that is specified by this parameter.
      Access Mode	The mode that is used to access the data source. In this example, RAM Role Authorization Mode is used. Then, the DataWorks service account can assume the related role to access the data source by using an STS token. This ensures higher security.
      Role	The role to be assumed. Select a RAM role from the drop-down list.

   2. Test the network connectivity.

      In the Connection Configuration section, click the Data Integration tab. Then, find the required resource group and click Test Network Connectivity in the desired column.

      A synchronization task can use only one type of resource group. To ensure that your synchronization tasks can be run as expected, you must test the connectivity between the resource groups for Data Integration on which your synchronization tasks are run and the data sources. If you want to test the connectivity of multiple resource groups for Data Integration at a time, select the resource groups and click Batch Test Network Connectivity. For more information, see Network connectivity solutions.

   3. If the connectivity test is successful, click Complete Creation.

4. Create a data synchronization task.

   After you add a data source, you can go to the DataStudio page and create a data synchronization task for the data source. For more information, see Configure a batch synchronization task by using the codeless UI.

5. Run the data synchronization task.

   On the DataStudio or Operation Center page, run the created data synchronization task.

   Note

   Make sure that you are granted the required permissions in Step 2 before you run tasks on the DataStudio page. Otherwise, the tasks fail to run.