You can use the RAM Roles for Service Accounts (RRSA) feature to enforce access control on different pods that are deployed in a Container Service for Kubernetes (ACK) cluster. This achieves fine-grained API permission control on pods and reduces security risks. This topic describes how to use RRSA in ACK clusters.
Background information
Elastic container instances run on Elastic Compute Service (ECS) instances. ECS instance metadata contains information about ECS instances on Alibaba Cloud. You can log on to a running ECS instance and view its metadata, and then configure or manage the ECS instance based on the metadata. Applications deployed in an ACK cluster can use ECS instance metadata to obtain Security Token Service (STS) tokens that are used to assume different Resource Access Management (RAM) roles. This way, the applications can call the APIs of different cloud services. For more information, see Obtain instance metadata.

If you want to control the RAM permissions of different applications in a cluster, you can prohibit these applications from using the metadata of ECS instances or elastic container instances to obtain the STS tokens that are used to assume the RAM roles assigned to the instances. You can also choose not to attach RAM policies to the RAM roles that are assigned to the ECS instances or elastic container instances. However, these applications still need to obtain STS tokens for accessing cloud resources by using a secure method. To address these issues, ACK releases the RRSA feature which is developed based on RAM.
You can use the RRSA feature to allow different applications in an ACK cluster to assume different RAM roles. Applications can obtain STS tokens, use the tokens to assume specific RAM roles, and then access relevant cloud services. This enforces the principle of least privilege and allows applications to call API operations without the need to use AccessKey pairs, which prevents AccessKey pair leaks.

The following steps show how an application accesses a cloud resource when RRSA is used to enforce access control:
The tenant deploys a pod for which the feature of service account token volume projection is enabled.
The ACK cluster creates a service account OpenID Connect (OIDC) token file and mounts the token file to the pod.
The application in the pod uses the OIDC token file to call the AssumeRoleWithOIDC API operation of STS and obtain the STS token that is used to assume a RAM role.
Note
To enable the application to perform these operations, you must first create an OIDC identity provider and allow the service account used by the pod to assume the specified RAM role. For more information, see AssumeRoleWithOIDC.
The OIDC token in the OIDC token file is a temporary token. We recommend that you configure the application to read the latest token from the OIDC token file. The cluster renews the token in the OIDC token file before the token expires.
The application in the pod uses the STS token to assume the specified RAM role and then calls the API of the relevant cloud service.
Limits
The RRSA feature supports only ACK clusters that run Kubernetes 1.22 and later. ACK clusters that support the RRSA feature include ACK Basic clusters, ACK Pro clusters, ACK Serverless Basic clusters, ACK Serverless Pro clusters, and ACK Edge Pro clusters.
Enable RRSA
If you do not have an existing cluster, enable RRSA when you create an ACK managed cluster or ACK Edge cluster. Otherwise, enable it in the Security and Auditing section of the cluster details page.
Note
For ACK Serverless clusters, you can only enable RRSA in the Security and Auditing section of the cluster details page after the cluster is created.
Enable during cluster creation
Enable after cluster creation
When you create an ACK managed cluster or ACK Edge cluster, click Enable next to RRSA OIDC in the Advanced Options (Optional) section of the Cluster Configurations step.

Log on to the ACK console. In the left-side navigation pane, click Clusters.
On the Clusters page, find the cluster that you want to manage and click its name. In the left-side pane, click Cluster Information.
In the Security and Auditing section of the Basic Information tab, click Enable next to RRSA OIDC.
In the Enable RRSA dialog box, click Confirm.
In the Basic Information section, if the cluster status changes from Updating to Running, the RRSA feature is enabled for the cluster.
Obtain the URL and ARN information
After the RRSA feature is enabled for the cluster, navigate to the Security and Auditing section under the Basic Information tab. If you hover your mouse over the Enabled label next to RRSA OIDC, the URL and Alibaba Cloud Resource Name (ARN) of the OIDC provider are displayed. 
After RRSA is enabled, ACK performs the following operations in the background:
Automatically creates an OIDC issuer that is dedicated to the cluster. The OIDC issuer is managed by ACK. For more information, see OIDC Issuer.
Enables service account token volume projection for the cluster and merges configurations of the OIDC issuer with the value of the existing service-account-issuer parameter of the cluster. For more information, see Use ServiceAccount token volume projection.
Creates a RAM role identity provider within your account. The identity provider uses the OIDC issuer for single sign-on (SSO). The identity provider is named ack-rrsa-<cluster_id>. <cluster_id> indicates the ID of your cluster. For more information, see Manage an OIDC IdP.
Work with RRSA
After you enable RRSA for your cluster, perform the following steps to enable the applications in the cluster to obtain STS tokens through RRSA. The STS tokens are used to call the APIs of specific cloud services.
Example
In this example, an application is created and RRSA is enabled for the cluster to allow the application to perform the following operations: assuming a specified RAM role and then calling an API operation to query clusters that belong to the current Alibaba Cloud account.
Sample configurations
Procedure
Install the ack-pod-identity-webhook component.
On the Clusters page, click the name of your cluster. In the left-side navigation pane, choose Operations > Add-ons.
On the Add-ons page, click the Security tab, find ack-pod-identity-webhook, and then click Install in the lower-right part of the card.
In the message that appears, confirm the information and click OK.
Create a RAM role named demo-role-for-rrsa.
Log on to the RAM console with your Alibaba Cloud account.
In the left-side navigation pane, choose . On the Roles page, click Create Role.
In the Create Role panel, select IdP for Select Trusted Entity and click Next.
On the Configure Role wizard page, set the following parameters and click OK.
The following table describes the parameters.
Parameter | Description |
RAM Role Name | Set the value to demo-role-for-rrsa. |
Note | Enter the description of the RAM role. This parameter is optional. |
IdP Type | Select OIDC. |
Select IdP | Select an IdP. The IdP is named in the ack-rrsa-<cluster_id> format. <cluster_id> indicates the ID of your cluster. |
Conditions | oidc:iss: Use the default value. oidc:aud: Select sts.aliyuncs.com. oidc:sub: Set the condition operator to StringEquals and the value to system:serviceaccount:<namespace>:<serviceAccountName>. In this example, enter system:serviceaccount:rrsa-demo:demo-sa.
|
Attach the AliyunCSReadOnlyAccess policy to the RAM role created in Step 2 to grant the required permissions to the application. For more information, see Grant permissions to a RAM role.
Deploy an application. For more information about the SDK demos, see Demos of Alibaba Cloud SDKs that support OIDC token authentication of RRSA.
Create a file named demo.yaml based on the following requirements:
In the following sample YAML template, the pod-identity.alibabacloud.com/injection: 'on'
label is added to the namespace, the pod-identity.alibabacloud.com/role-name: demo-role-for-rrsa
annotation is added to the service account, and the auto injection feature of ack-pod-identity-webhook is enabled. For more information about how to configure ack-pod-identity-webhook, see ack-pod-identity-webhook.
---
apiVersion: v1
kind: Namespace
metadata:
name: rrsa-demo
labels:
pod-identity.alibabacloud.com/injection: 'on'
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: demo-sa
namespace: rrsa-demo
annotations:
pod-identity.alibabacloud.com/role-name: demo-role-for-rrsa
---
apiVersion: v1
kind: Pod
metadata:
name: demo
namespace: rrsa-demo
spec:
serviceAccountName: demo-sa
containers:
- image: registry.cn-hangzhou.aliyuncs.com/acs/ack-ram-tool:1.0.0
imagePullPolicy: "Always"
args:
- rrsa
- demo
name: demo
restartPolicy: OnFailure
Run the following command to deploy the application:
kubectl apply -f demo.yaml
Run the following command to check whether ack-pod-identity-webhook has injected the required configurations into the pod that is created for the application:
kubectl -n rrsa-demo get pod demo -o yaml
apiVersion: v1
kind: Pod
metadata:
name: demo
namespace: rrsa-demo
spec:
containers:
- args:
- rrsa
- demo
env:
- name: ALIBABA_CLOUD_ROLE_ARN
value: acs:ram::1***:role/demo-role-for-rrsa
- name: ALIBABA_CLOUD_OIDC_PROVIDER_ARN
value: acs:ram::1***:oidc-provider/ack-rrsa-c***
- name: ALIBABA_CLOUD_OIDC_TOKEN_FILE
value: /var/run/secrets/ack.alibabacloud.com/rrsa-tokens/token
image: registry.cn-hangzhou.aliyuncs.com/acs/ack-ram-tool:1.0.0
imagePullPolicy: Always
name: demo
volumeMounts:
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: kube-api-access-4bwdg
readOnly: true
- mountPath: /var/run/secrets/ack.alibabacloud.com/rrsa-tokens
name: rrsa-oidc-token
readOnly: true
restartPolicy: OnFailure
serviceAccount: demo-sa
serviceAccountName: demo-sa
volumes:
- name: kube-api-access-4bwdg
projected:
defaultMode: 420
sources:
- serviceAccountToken:
expirationSeconds: 3607
path: token
- configMap:
items:
- key: ca.crt
path: ca.crt
name: kube-root-ca.crt
- downwardAPI:
items:
- fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
path: namespace
- name: rrsa-oidc-token
projected:
defaultMode: 420
sources:
- serviceAccountToken:
audience: sts.aliyuncs.com
expirationSeconds: 3600
path: token
The output indicates that ack-pod-identity-webhook has injected the following configurations into the pod.
Category | Configuration item | Description |
Category | Configuration item | Description |
The environment variables. | ALIBABA_CLOUD_ROLE_ARN | The ARN of the RAM role to be assumed. |
ALIBABA_CLOUD_OIDC_PROVIDER_ARN | The ARN of the OIDC IdP. |
ALIBABA_CLOUD_OIDC_TOKEN_FILE | The path of the OIDC token file. |
VolumeMount | rrsa-oidc-token | The configuration for mounting the OIDC token. |
Volume | rrsa-oidc-token | The configuration for mounting the OIDC token. |
Run the following command to print the log of the application:
kubectl -n rrsa-demo logs demo
A list of clusters are displayed in the output:
cluster id: cf***, cluster name: foo*
cluster id: c8***, cluster name: bar*
cluster id: c4***, cluster name: foob*
Optional: Detach the AliyunCSReadOnlyAccess system policy from the RAM role. For more information, see Remove permissions from a RAM role.
Wait 30 seconds and run the following command to print the log of the application again:
kubectl -n rrsa-demo logs demo
The following error message indicates that the application does not have the required permissions:
StatusCode: 403
Code: StatusForbidden
Message: code: 403, STSToken policy Forbidden for action cs:DescribeClusters request id: E78A2E2D-***
Data: {"accessDeniedDetail":{"AuthAction":"cs:DescribeClusters","AuthPrincipalDisplayName":"demo-role-for-rrsa:ack-ram-tool","AuthPrincipalOwnerId":"11***","AuthPrincipalType":"AssumedRoleUser","NoPermissionType":"ImplicitDeny","PolicyType":"ResourceGroupLevelIdentityBasedPolicy"},"code":"StatusForbidden","message":"STSToken policy Forbidden for action cs:DescribeClusters","requestId":"E78A2E2D-***","status":403,"statusCode":403}
Manually modify the application template to enable RRSA
You can manually specify the required OIDC token file as environment variables in the application template. This way, you can enable RRSA without installing ack-pod-identity-webhook.
The following template provides an example.
apiVersion: v1
kind: Pod
metadata:
name: demo
namespace: rrsa-demo
spec:
containers:
- args:
- rrsa
- demo
env:
- name: ALIBABA_CLOUD_ROLE_ARN
value: <role_arn>
- name: ALIBABA_CLOUD_OIDC_PROVIDER_ARN
value: <oid_provider_arn>
- name: ALIBABA_CLOUD_OIDC_TOKEN_FILE
value: /var/run/secrets/ack.alibabacloud.com/rrsa-tokens/token
image: registry.cn-hangzhou.aliyuncs.com/acs/ack-ram-tool:1.0.0
imagePullPolicy: Always
name: demo
volumeMounts:
- mountPath: /var/run/secrets/ack.alibabacloud.com/rrsa-tokens
name: rrsa-oidc-token
readOnly: true
restartPolicy: OnFailure
serviceAccount: demo-sa
serviceAccountName: demo-sa
volumes:
- name: rrsa-oidc-token
projected:
defaultMode: 420
sources:
- serviceAccountToken:
audience: sts.aliyuncs.com
expirationSeconds: 3600
path: token
Important
Modify the following fields in the preceding template:
Replace <oid_provider_arn>
with the ARN of the OIDC provider used by the cluster. For more information about how to obtain the ARN, see Obtain the URL and ARN information.
Replace <role_arn>
with the ARN of the RAM role that is used by the application. You can obtain the ARN on the Roles page in the RAM console.
Set the audience
parameter to sts.aliyuncs.com
. This value is the ID of the client configured in the OIDC IdP that is automatically created when the RRSA feature is enabled. The value is not the domain name that is used by the SDK to call the AssumeRoleWithOIDC operation of STS. You can specify an appropriate STS domain name when you use the SDK.
Set expirationSeconds
to a value from 600 to 43200. Unit: seconds. If you specify a value larger than 43200
, the validity period of the OIDC token is still 43,200 seconds (12 hours).
After you redeploy the application based on the modified template, the application can use the OIDC token file, the RAM role ARN, and the OIDC provider ARN that you specified in the template to call the AssumeRoleWithOIDC operation of STS and obtain the STS token used to assume a specific RAM role. This way, the application can call the API operations of different cloud services. The OIDC token file is specified in the ALIBABA_CLOUD_OIDC_TOKEN_FILE environment variable, the RAM role ARN is specified in the ALIBABA_CLOUD_ROLE_ARN environment variable, and the OIDC provider ARN is specified in the ALIBABA_CLOUD_OIDC_PROVIDER_ARN environment variable. The application reads the latest OIDC token from the OIDC token file when the application uses the file. For more information about the SDK demos, see Demos of Alibaba Cloud SDKs that support OIDC token authentication of RRSA. For more information, see AssumeRoleWithOIDC.
Use an existing RAM role and grant the required permissions to the RAM role
If you want the application to use an existing RAM role, you must modify the trust policy of the RAM role based on the following template. This way, the application can use the service account to assume the RAM role and obtain an STS token. For more information, see Edit the trust policy of a RAM role.
Example of the Statement
configurations:
{
"Action": "sts:AssumeRole",
"Condition": {
"StringEquals": {
"oidc:aud": "sts.aliyuncs.com",
"oidc:iss": "<oidc_issuer_url>",
"oidc:sub": "system:serviceaccount:<namespace>:<service_account>"
}
},
"Effect": "Allow",
"Principal": {
"Federated": [
"<oidc_provider_arn>"
]
}
}
Important
Modify the following fields in the Statement
configurations:
Replace <oidc_issuer_url>
with the URL of the OIDC provider used by the cluster. For more information about how to obtain the URL, see Obtain the URL and ARN information.
Replace <oidc_provider_arn>
with the ARN of the OIDC provider used by the cluster. For more information about how to obtain the ARN, see Obtain the URL and ARN information.
Replace <namespace>
with the namespace of the application.
Replace <service_account>
with the service account used by the application.
You can also use the ack-ram-tool tool to automate the modification of the trust policy. Sample commands:
ack-ram-tool rrsa associate-role --cluster-id <cluster_id> \
--namespace <namespace> --service-account <service_account> \
--role-name <role_name> --create-role-if-not-exist
Demos of Alibaba Cloud SDKs that support OIDC token authentication of RRSA
SDK demos
Alibaba Cloud SDK V2.0 supports OIDC token authentication of RRSA. By default, all cloud service SDKs that support STS token authentication and are developed based on Alibaba Cloud SDK V2.0 support RRSA OIDC token authentication. The following table describes the supported SDK versions and demos.
Programming language | Supported SDK version | Demo |
For some cloud service SDKs, you can refer to the preceding methods to use OIDC tokens of RRSA for authentication when you call API operations. The following table describes the SDK demos.
Solutions to SDK errors
The following table describes the solutions to different errors.
Error message | Possible cause | Solution |
Error message | Possible cause | Solution |
{
"Code": "AuthenticationFail.OIDCToken.Expired",
"Message": "This JsonWebToken is expired."
}
| The OIDC token used by your application has expired. | The OIDC token file is specified in the ALIBABA_CLOUD_OIDC_TOKEN_FILE environment variable. You need to read the latest OIDC token each time when you use the file. We recommend that you use Alibaba Cloud SDKs to obtain the OIDC token instead of manually configuring your application to obtain the token. For more information, see Demos of Alibaba Cloud SDKs that support OIDC token authentication of RRSA. |
{
"Code": "Throttling.User",
"Message": "Request was denied due to user flow control."
}
| Your application frequently obtains OIDC tokens. As a result, the operation is throttled. | Do not frequently call the API operation to obtain OIDC tokens. You do not need to obtain a new OIDC token before the current OIDC token expires. We recommend that you use Alibaba Cloud SDKs to obtain the OIDC token instead of manually configuring your application to obtain the token. For more information, see Demos of Alibaba Cloud SDKs that support OIDC token authentication of RRSA. |
{
"Code": "AuthenticationFail.OIDCToken.AudienceNotMatch",
"Message": "Invalid audience."
}
| The audience parameter in your application template is not set to sts.aliyuncs.com . | You need to modify the application template to ensure that the audience parameter is set to sts.aliyuncs.com . |
{
"Code": "AuthenticationFail.OIDCToken.IssuerConfigurationBroken",
"Message": "Get public keys from OIDC Provider failed, the issuer is https://kubernetes.default.svc."
}
{
"Code": "AuthenticationFail.OIDCToken.IssuerNotMatch",
"Message": "The issuer in the OIDC Token doesn't match the OIDC Provider registered."
}
{
"Code": "AuthenticationFail.NoPermission",
"Message": "No such OIDC Provider registered."
}
| The RRSA feature is disabled for your cluster. | You need to enable the RRSA feature for the cluster in which the application resides. For more information, see Enable the RRSA feature. After you enable the RRSA feature, you must recreate the pods that use the RRSA feature. |
{
"Code": "EntityNotExist.Role",
"Message": "The role not exists: acs:ram::19981***:role/***. "
}
| The RAM role assumed by your application does not exist. | You must create the required RAM role. For more information, see Create a RAM role for an OIDC IdP and Example. |
{
"Code": "AuthenticationFail.NoPermission",
"Message": "There is no permission"
}
| The required trust policy is not configured for the RAM role assumed by your application. | You need to modify the trust policy of the RAM role to allow your application to assume the RAM role. For more information, see Use an existing RAM role and grant the required permissions to the RAM role. |
Enable commonly used CLIs to support OIDC token authentication of RRSA
You can use ack-ram-tool to enable commonly used CLIs to support OIDC token authentication of RRSA in pods. The following table describes the configurations and examples.
CLI | Configuration method | Example |
CLI | Configuration method | Example |
Alibaba Cloud CLI | Set the mode parameter in the ~/.aliyun/config.json file to OIDC to support OIDC token authentication of RRSA.
{
"current": "rrsa",
"profiles": [
{
"name": "rrsa",
"mode": "OIDC",
"region_id": "cn-hangzhou",
"ram_session_name": "test-rrsa"
}
],
"meta_path": ""
}
|
$ aliyun sts GetCallerIdentity
{
"AccountId": "11380***",
"Arn": "acs:ram::1138***:assumed-role/test-rrsa-***/test-rrsa",
"IdentityType": "AssumedRoleUser",
"PrincipalId": "33300***:test-rrsa",
"RequestId": "20F78881-F47E-5771-90D6-***",
"RoleId": "33300***"
}
|
You can also directly run the relevant commands in Alibaba Cloud CLI without creating a configuration file. Note Only Alibaba Cloud CLI v3.0.206 and later support this feature. |
$ aliyun sts GetCallerIdentity --region cn-hangzhou --role-session-name=test-rrsa
{
"AccountId": "11380***",
"Arn": "acs:ram::1138***:assumed-role/test-rrsa-***/test-rrsa",
"IdentityType": "AssumedRoleUser",
"PrincipalId": "33300***:test-rrsa",
"RequestId": "20F78881-F47E-5771-90D6-***",
"RoleId": "33300***"
}
|
Simple Log Service CLI | You cannot modify the configuration file of Simple Log Service CLI to support OIDC token authentication of RRSA. You must run the ack-ram-tool export-credentials -f environment-variables -- aliyunlog command in Simple Log Service CLI. |
$ ack-ram-tool export-credentials -f environment-variables -- aliyunlog log list_project --region-endpoint=cn-hangzhou.log.aliyuncs.com
{"count": 1, "projects": [
{"createTime": "1676282996", "description": "k8s log project, ***",
"lastModifyTime": "1676282996", "owner": "", "projectName": "k8s-log-c0edc***",
"region": "cn-hangzhou", "resourceGroupId": "rg-***", "status": "Normal"}],
"total": 24}
|
Terraform | You can configure the assume_role_with_oidc parameter in the configuration file to support OIDC token authentication of RRSA.
provider "alicloud" {
assume_role_with_oidc {
role_session_name = "terraform-with-rrsa-auth-example"
}
region = "cn-hangzhou"
}
| Examples of Terraform |
\