E-MapReduce:Use a custom ECS application role to access other cloud resources in your Alibaba Cloud account

Prerequisites

• Create a cluster

• Assign roles to an Alibaba Cloud account

• An OSS bucket is created in the OSS console. Make sure that the bucket resides in the region where you want to create an EMR cluster. For more information about how to create a bucket, see Create buckets.

Procedure

1. Step 1: Create a policy

2. Step 2: Create a RAM role

3. Step 3: Create a cluster and access external resources

Step 1: Create a policy

1. Go to the Create Policy page.

   1. Log on to the RAM console by using an Alibaba Cloud account.

   2. In the left-side navigation pane, choose Permissions > Policies.

   3. On the Policies page, click Create Policy.

2. On the Create Policy page, click the JSON tab.

3. Enter the policy document and click OK.

   Enter the following policy document in the script editor:

   {
       "Version": "1",
       "Statement": [
           {
               "Action": [
                   "oss:GetObject",
                   "oss:ListObjects"
               ],
               "Resource": [
                   "acs:oss:*:*:emr-logs2",
                   "acs:oss:*:*:emr-logs2/*"
               ],
               "Effect": "Allow"
           }
       ]
   }

   Note

   Elements in the policy:

   • Action: the actions on resources. In this example, the permissions to read and query OSS data are granted.

   • Resource: the resources on which the permissions are granted. In this example, the OSS bucket emr-logs2 and the objects in this OSS bucket are used as the resources.

   For more information about the basic elements of a policy, see Policy elements.

4. Configure the Name parameter in the Basic information section to specify the policy name.

   In this example, the name is test-emr.

5. Click OK.

Step 2: Create a RAM role

1. In the left-side navigation pane of the RAM console, choose Identities > Roles.

2. On the Roles page, click Create Role.

3. Create a RAM role.

   1. In the Select Role Type step, select Alibaba Cloud Service for Select Trusted Entity.

   2. Click Next.

   3. In the Configure Role step, configure the parameters.

      Parameter	Description
      RAM Role Name	In this example, the name is test-emr.
      Select Trusted Service	Select Elastic Compute Service from the drop-down list.

   4. Click OK.

4. Optional. Change the trusted service.

   Important

   If you want to create a cluster of EMR V5.X or later, a minor version later than EMR V4.5, or a minor version later than EMR V3.32, skip this step.

   1. On the Roles page, click the name of the RAM role that you created.

   2. Click the Trust Policy tab.

   3. Click Edit Trust Policy.

   4. Change ecs.aliyuncs.com to emr.aliyuncs.com.

      

   5. Click Save trust policy document.

5. Add the required permissions.

   1. On the Roles page, find the RAM role that you created and click Grant Permission in the Actions column.

   2. In the Grant Permission panel, select Custom Policy from the drop-down list in the Policy section and select the policy that you created in Step 1.

   3. Click Grant permissions.

   4. Click Close.

Step 3: Create a cluster and access external resources

1. Log on to the EMR console. In the left-side navigation pane, click EMR on ECS.

2. In the top navigation bar, select the region in which your cluster resides and select a resource group based on your business requirements.

3. On the EMR on ECS page, click Create Cluster. In the Advanced Settings section of the Basic Configuration step, enter the name of the RAM role that you created in Step 2: Create a RAM role. For more information, see Create a cluster.

   

4. After the cluster is created, log on to the cluster by using SSH. For more information, see Log on to a cluster.

   Run the following command to check whether you can access the OSS bucket that is specified in the policy:

   hdfs dfs -ls oss://<yourBucketName>/

   Note

   Replace <yourBucketName> in the command with the name of the OSS bucket.

   • If you do not have permissions to access the bucket, access to the OSS bucket is denied.

   • If you have permissions to access the bucket, access to the OSS bucket is allowed.

FAQ

• Q: What do I do if the NoPermission message appears when I create a cluster?
  A: Perform the following operations in sequence to resolve the issue:

  1. Check whether the RAM user that you used to create the cluster is authorized to create clusters and change ECS application roles. If the RAM user is granted the AliyunEMRDevelopAccess permission, you must change the permission to AliyunEMRFullAccess.
  2. Check whether the name of the ECS application role that you specified when you create the cluster is valid.
  3. Check whether the trust policy is changed to emr.aliyuncs.com.

• Q: What do I do if I cannot access an OSS bucket from Hadoop Distributed File System (HDFS)?

  A: Perform the following operations in sequence to resolve the issue:

  1. Check whether the OSS bucket that you want to access resides in the same region as your cluster. If the OSS bucket and your cluster do not reside in the same region, you must add the endpoint of the OSS bucket to the access links.

  2. Check whether the OSS bucket that you want to access is one of the resources specified in the created policy. If the OSS bucket is not one of the resources, modify the policy.

  3. Check whether the related permissions on the OSS bucket are configured in the OSS console. If the permissions are configured, you must cancel the settings in the OSS console and specify Action in the policy to configure the permissions.