You can configure the aliyun-acr-credential-helper component for Container Service for Kubernetes (ACK) clusters to automatically obtain the secret when you pull private images from Container Registry Enterprise Edition instances for the ACK clusters. This eliminates the need to manually provide usernames and secrets.
Prerequisites
An ACK managed cluster is created. For more information, see Create an ACK managed cluster.
The kubeconfig file of the cluster is obtained, and the cluster is connected by using kubectl. For more information, see Obtain the kubeconfig file of a cluster and use kubectl to connect to the cluster.
Usage notes
The aliyun-acr-credential-helper component reads the configuration from the acr-configuration ConfigMap in the kube-system namespace of an ACK cluster and then pulls private images. After you configure the aliyun-acr-credential-helper component, the component generates a secret in your cluster and associates the secret with the service account that you specified in the acr-configuration ConfigMap. By default, all pods that use this service account use the secret that is generated by the component to pull images. This eliminates the need to use other secrets when you pull images.
If you create a service account to deploy applications, such as Helm charts, the service account requires a period of time to be associated with the secret that is generated by aliyun-acr-credential-helper. We recommend that you use the webhook feature to inject the secret into the service account before you pull images. For more information, see the "FAQ" section of this topic.
The following table describes the limits on using the aliyun-acr-credential-helper component to pull private images without using a secret.
Item | Usage note |
Mirroring |
|
Cluster type and version |
|
Precautions
Item | Precaution |
imagePullSecrets |
|
ServiceAccount |
|
Region | Check whether the private images that you want to pull reside in the region of your ACK cluster. By default, you can pull private images only from Container Registry instances that reside in the region of your ACK cluster. If you want to pull images across regions, see the "Scenario 2: Pull images across regions" section of this topic. |
YAML format | When you modify the acr-configuration ConfigMap in the kube-system namespace, make sure that you use the same indentation as the example in this topic. We recommend that you paste the YAML content provided in this topic to the editor, replace the corresponding values, and then apply the configuration. This ensures that the format of the ConfigMap is valid. |
Step 1: Install the aliyun-acr-credential-helper component
If you installed the aliyun-acr-credential-helper component, make sure that the component is of the latest version. Otherwise, update the component. Updating the component does not affect your business. For more information, see Manage components. For more information about the aliyun-acr-credential-helper component, see aliyun-acr-credential-helper.
After the aliyun-acr-credential-helper component is updated to the latest version, the RAM role on which the component relies is changed. The component provides the tokenMode parameter to allow you to specify the RAM role on which the component relies. For more information about the impacts of changing the RAM role on which the component relies, see [Product Changes] Revoke the permissions on which aliyun-acr-credential-helper relies.
Log on to the ACK console. In the left-side navigation pane, click Clusters.
On the Clusters page, find the cluster that you want to manage and click its name. In the left-side pane, choose .
On the Add-ons page, click the Security tab, find aliyun-acr-credential-helper, and then click Install.
On the Parameters page, use the default value auto for the tokenMode parameter and click OK.
After you update an ACK managed cluster to the latest version, the aliyun-acr-credential-helper component provides the tokenMode parameter. You can modify the parameter after the component is installed. After the component is modified, the system recreates the pod for the component. Old clusters described in the table refer to clusters that are created before April 3, 2023, and new clusters refer to clusters that are created on or after April 3, 2023. The following table describes the valid values of the tokenMode parameter.
tokenMode
Description
auto
Automatically determines whether to use the managedRole mode, which is the default value. The system uses the managedRole mode for new clusters and the workerRole mode for old clusters.
managedRole
Uses the managedRole mode.
workerRole
Uses the workerRole mode.
Step 2: Modify configurations
Perform the following operations to modify configurations for a Container Registry Enterprise Edition instance. For information about how to modify configurations for a Container Registry Personal Edition instance, see the "How do I pull private images from a Container Registry Personal Edition instance?" section of this topic.
After the aliyun-acr-credential-helper component is installed, you must configure the acr-configuration ConfigMap before you can pull images.
Use the ACK console
Log on to the ACK console. In the left-side navigation pane, click Clusters.
On the Clusters page, click the name of the cluster that you want to manage. In the left-side navigation pane of the page that appears, choose Configurations > ConfigMaps.
In the upper part of the ConfigMap page, select kube-system from the Namespace drop-down list. Then, find acr-configuration and configure values.
If the acr-configuration ConfigMap does not exist, create a ConfigMap named acr-configuration. For more information, see the "Create a ConfigMap" section of the Manage ConfigMaps topic. For information about how to update ConfigMaps, see Manage ConfigMaps.
Click Edit in the Actions column to configure keys and values for the ConfigMap.
Click Edit YAML in the Actions column to configure keys and values for the ConfigMap.
Use kubectl
Run the following command to go to the editing page of the acr-configuration ConfigMap:
kubectl edit cm acr-configuration -n kube-systemConfigure the parameters of the acr-configuration ConfigMap based on your requirements.
The following table describes the keys and values of the acr-configuration ConfigMap.
Key
Description
Value
service-account
The service accounts that are used by the aliyun-acr-credential-helper component to pull images.
Default value: default.
NoteSeparate multiple service accounts with commas (,). An asterisk (
*) specifies all service accounts in all namespaces.acr-registry-info
The information about Container Registry instances. Each instance can be specified by three parameters of the string type in a YAML file.
NoteParameters for specifying an instance:
instanceId: the ID of the Container Registry instance. This field is required for Container Registry Enterprise Edition instances.
regionId: the ID of the region where the Container Registry instance resides. This parameter is optional. The default value is the region where your ACK cluster resides.
domains: the domain names of the Container Registry instance. This parameter is optional. By default, all domain names of the instance are specified. Separate multiple domain names with commas (,).
Sample configuration for a Container Registry Enterprise Edition instance:
- instanceId: <cri-instanceId> regionId: "cn-hangzhou" domains: "xxx.com,yyy.com"watch-namespace
The namespaces from which you want to pull images without using a secret.
Default value: default. A value of all specifies all namespaces in the Container Registry instance. Separate multiple namespaces with commas (,).
NoteWe recommend that you set the values to your production namespaces. If you set the value to all or namespaces of the system components of the cluster, images in the namespaces may fail to be pulled.
expiring-threshold
The period of time after which the cached Secret expires.
Default value: 15m.
NoteWe recommended that you use the default value. The default value specifies that the Secret is updated 15 minutes before the cached Secret expires.
Step 3: Pull images in different scenarios
The permissions that are required to pull private images vary based on the scenario.
Scenario | Description | |
You want to pull images within the same account. By default, the aliyun-acr-credential-helper component pulls private images within the same account. | ||
You want to pull images across regions. | ||
Configure RAM Roles for Service Accounts (RRSA) to pull images across accounts | You want to pull images across accounts. | |
Assume a RAM role to pull images across accounts |
| |
Use the AccessKey ID and AccessKey secret of a RAM user to pull images across accounts | You want to pull images across accounts. If you pull images by using the AccessKey ID and AccessKey secret of a RAM user, you must specify the AccessKey pair in the acr-configuration ConfigMap. In this case, the AccessKey pair may be leaked. To ensure data security, make sure that the RAM role to which the AccessKey pair belongs is granted only the permissions to pull images. We recommend that you use the other two methods to pull images across accounts. | |
Configure the required permissions based on the following scenarios.
Scenario 1: Pull images within the same account
Create a custom policy and add the following content to the policy. For more information, see the "Step 1: Create a custom policy" section of the [Product Changes] Permissions of the worker RAM role of ACK managed clusters are revoked topic.
{ "Version": "1", "Statement": [ { "Action": [ "cr:GetAuthorizationToken", "cr:ListInstanceEndpoint", "cr:PullRepository" ], "Resource": "*", "Effect": "Allow" } ] }Attach the custom policy to the worker RAM role of the cluster. For more information, see the "Step 2: Attach the custom policy to the worker RAM role" section of the [Product Changes] Permissions of the worker RAM role of ACK managed clusters are revoked topic.
Modify the acr-configuration ConfigMap based on the following sample code to pull private images from a Container Registry Enterprise Edition instance.
data: service-account: "default" watch-namespace: "all" expiring-threshold: "15m" notify-email:"c*@aliyuncs.com" acr-registry-info:| - instanceId: "cri-instanceId" regionId: "cn-hangzhou" domains: "instance.default.domain.com,instance.custom.domain.com"
Scenario 2: Pull images across regions
If you want to pull images from Container Registry instances that reside in different regions from your ACK cluster, you must modify the acr-configuration ConfigMap.
After you enable cross-region image pulling, the aliyun-acr-credential-helper component sends API requests to the destination region over the Internet. You must enable Internet access for the cluster.
For example, if you want to pull images from Container Registry instances that reside in the China (Beijing) and China (Hangzhou) regions, modify the acr-configuration ConfigMap based on the following sample code.
data:
service-account: "default"
watch-namespace: "all"
expiring-threshold: "15m"
notify-email:"c*@aliyuncs.com"
acr-registry-info: |
- instanceId: "cri-instanceId"
regionId: "cn-beijing"
- instanceId: "cri-instanceId"
regionId: "cn-hangzhou" Scenario 3: Pull images across accounts
You can use one of the following methods to pull images across accounts.
Pull images across accounts by configuring RRSA: Account A assumes the RAM role of Account B to pull private images from the Container Registry instances of Account B.
Pull images across accounts by assuming a RAM role: Account A assumes the RAM role of Account B to pull private images from the Container Registry instances of Account B.
Pull images across accounts by using the AccessKey pair of a RAM user: Account A uses the AccessKey pair of Account B to pull private images from the Container Registry instances of Account B.
Pull images across accounts by configuring RRSA
The RRSA feature can be used to perform access control on pods in a cluster.
You can configure RRSA to pull images from Container Registry Enterprise Edition instances (including Basic Edition and Advanced Edition instances) only in clusters of v1.22 and later. Clusters of v1.22 and later include ACK Basic clusters, ACK Pro clusters, ACK Serverless Basic clusters, and ACK Serverless Pro clusters.
To enable the RRSA feature, you must update the aliyun-acr-credential-helper component to v23.02.06.1-74e2172-aliyun or later.
To enable the RRSA feature for aliyun-acr-credential-helper, you must enable RRSA on the Basic Information tab of the Cluster Information page in the ACK console and then configure RRSA for aliyun-acr-credential-helper. If you configure RRSA for aliyun-acr-credential-helper and then enable RRSA on the Basic Information tab of the Cluster Information page in the ACK console, you must delete the pod corresponding to aliyun-acr-credential-helper after you configure RRSA. This allows RRSA to take effect.
Enable the RRSA feature for the cluster. For more information, see the "Enable RRSA" section of the Use RRSA to authorize different pods to access different cloud services topic.
Grant the required permissions to access Container Registry resources across accounts.
After the RRSA feature is enabled for the cluster, you must perform the following steps to enable RRSA for aliyun-acr-credential-helper. For example, the current cluster is in Account A and the Container Registry instance is in Account B. If you want to pull images from the Container Registry instance, you must grant the cluster in Account A the permissions to access Container Registry resources of Account B.
Configure a RAM role for Account A
Create a RAM role in Account A and associate the AliyunSTSAssumeRoleAccess policy with the RAM role. This policy grants the RAM role the permission to assume the role of Account A. Modify the trust policy of the RAM role.
Replace <oidc_issuer_url> in the example with the URL of the OpenID Connect (OIDC) provider of the cluster. You can obtain the URL on the Basic Information tab of the Cluster Information page in the ACK console.
Replace <oidc_provider_arn> in the example with the Alibaba Cloud Resource Name (ARN) of the OIDC provider of the cluster. You can obtain the ARN on the Basic Information tab of the Cluster Information page in the ACK console.
{ "Statement": [ { "Action": "sts:AssumeRole", "Condition": { "StringEquals": { "oidc:aud": "sts.aliyuncs.com", "oidc:iss": "<oidc_issuer_url>", "oidc:sub": "system:serviceaccount:kube-system:aliyun-acr-credential-helper" } }, "Effect": "Allow", "Principal": { "Federated": [ "<oidc_provider_arn>" ] } } ], "Version": "1" }Configure a RAM role for Account B
Create a RAM role that has Container Registry-related permissions in Account B. On the Trust Policy Management tab, enter the ARN of the RAM role of Account A in the Principal field of the trust policy.
{ "Statement": [ { "Action": "sts:AssumeRole", "Effect": "Allow", "Principal": { "RAM": [ <ARN of the RAM role that you created for Account A> ] } } ], "Version": "1" }Attach a policy that allows users to obtain information of Container Registry instances and pull images from the instances to the RAM role in Account B.
{ "Version": "1", "Statement": [ { "Action": [ "cr:GetAuthorizationToken", "cr:ListInstanceEndpoint", "cr:PullRepository" ], "Resource": "*", "Effect": "Allow" } ] }You can configure the MaxSessionDuration parameter for the RAM role. Valid values of this parameter range from 3600 to 43200 seconds. You must configure the expireDuration parameter when you configure the acr-configuration ConfigMap in Step 3. We recommend that you specify the same value for MaxSessionDuration and expireDuration. The value of expireDuration can be less than or equal to the value of MaxSessionDuration.
Configure the acr-configuration ConfigMap
Modify the acr-configuration ConfigMap based on the following information:
data: rrsa: | enable: true acr-registry-info: | - instanceId: cri-xxx regionId: cn-hangzhou rrsaRoleARN: aUserRoleARN rrsaOIDCProviderRoleARN: aUserClusterOIDCProviderRoleARN assumeRoleARN: bUserRoleARN expireDuration: 3600Key
Description
Value
rrsaRoleARN
The ARN of the RAM role that you created for Account A.
acs:ram::aaa
rrsaOIDCProviderRoleARN
The ARN of the OIDC provider that you obtained on the Basic Information tab of the Cluster Information page in the ACK console.
acs:ram::bbb
assumeRoleARN
The ARN of the RAM role that you created for Account B.
acs:ram::ccc
expireDuration
The session period of the RAM role that you created for Account B. The session period is equal to the validity period of the secret that is generated by aliyun-acr-credential-helper.
ImportantThe expireDuration value cannot be greater than the MaxSessionDuration value of the RAM role of Account B.
Default value: 3600. Valid values: 3600 to 43200. Unit: seconds.
Set MaxSessionDuration of the RAM role of Account B to 43200.
Pull images across accounts by assuming a RAM role
Configuration rules:
The RAM role of Account B is authorized to pull private images from a specified private repository. This rule requires you to grant the cr.* permissions to the RAM role of Account B.
Account B allows the worker role of the ACK cluster of Account A to assume the RAM role of Account B. This rule requires you to modify the trust policy of the RAM role of Account B.
The worker role of the ACK cluster created by Account A has the permissions to assume the RAM role of Account B. This rule requires you to associate the AliyunAssumeRoleAccess policy with the worker role of Account A.
Set the worker role of Account A to assume the RAM role of Account B. In this case, you must specify the assumeRoleARN parameter in the acr-configuration ConfigMap.
Create a RAM role for Account B. Specify Alibaba Cloud Account as the trusted entity of the RAM role. Make sure that the RAM role is granted the permissions that are required to pull private images from Container Registry instances of Account B.
Create a RAM role for Account B. For more information, see Create a RAM role for a trusted Alibaba Cloud account.
Create a custom policy on the JSON tab. For more information, see Create custom policies.
ImportantMake sure that the RAM role is granted the permissions of cr.*.
``` { "Action": [ "cr:GetAuthorizationToken", "cr:ListInstanceEndpoint", "cr:PullRepository" ], "Resource": [ "*" ], "Effect": "Allow" } ```Attach the custom policy to the RAM role. For more information, see Grant permissions to a RAM role.
Modify the trust policy of the RAM role of Account B. This way, the RAM role of Account B allows the worker role of the ACK cluster in Account A to assume the RAM role of Account B.
Obtain the ARN of the worker role of the ACK cluster created by Account A.
For information about how to obtain the ARN of a RAM role, see the "How do I view the ARN of a RAM role?" section of the FAQ about RAM roles and STS tokens topic.
Modify the trust policy of the RAM role of Account B.
Log on to the RAM console. In the left-side navigation pane, click Roles. On the Roles page, find the RAM role of Account B and click the role name.
On the details page of the RAM role, click the Trust Policy tab, and add the ARN of the worker role of Account A to the Principal field of the trust policy.
Check whether the worker role of the ACK cluster in Account A is granted the AssumeRole permission. For more information, see View the basic information about a policy.
Add the assumeRoleARN parameter to the acr-configuration ConfigMap.
Set the value of the assumeRoleARN parameter to the ARN of the RAM role of Account B. For information about how to obtain the ARN of a RAM role, see the "How do I view the ARN of a RAM role?" section of the FAQ about RAM roles and STS tokens topic. The following YAML file shows the sample configuration:
data: service-account: "default" watch-namespace: "all" expiring-threshold: "15m" notify-email:"c*@aliyuncs.com" acr-registry-info: | - instanceId: "" regionId: "cn-beijing" domains: "registry.cn-beijing.aliyuncs.com" assumeRoleARN: "acs:ram::.*:role/kubernetesworkerrole-test" expireDuration: 3600
Pull images across accounts by using the AccessKey pair of a RAM user
FAQ
How do I pull private images from a Container Registry Personal Edition instance?
When you migrate images from a Container Registry Personal Edition instance to an Enterprise Edition instance, you may want to pull private images from the Enterprise Edition instance and the Personal Edition instance. In this case, you must modify the acr-configuration ConfigMap based on the following sample configuration. For information about how to push images to and pull images from a Container Registry Personal instance, see Push images to and pull images from a Container Registry Personal instance.
Before you modify the acr-configuration ConfigMap, remove # from #-instanceId.
data:
service-account: "default"
watch-namespace: "all"
expiring-threshold: "15m"
notify-email:"cs@aliyuncs.com"
acr-registry-info: |
- instanceId: ""
regionId: cn-xxxxHow do I enable the webhook feature?
After you enable the webhook feature, Secrets are immediately injected into newly created service accounts. The feature applies to scenarios in which you want to inject Secrets to service accounts without a delay, such as the scenario in which you create a service account together with a Deployment by using Helm charts. We recommend that you disable the feature in other scenarios.
To enable the webhook feature, you must update the aliyun-acr-credential-helper component to v23.02.06.1-74e2172-aliyun or later.
After you update the component to the latest version, you must modify the acr-configuration ConfigMap based on the following information:
data:
webhook-configuration: |
enable: true
failure-policy: Ignore
timeout-seconds: 10Table 1. Keys and values
Key | Description | Value |
enable | Specifies whether to enable the webhook feature. | Valid values: true and false. Default value: false. |
failure-policy | The failure handling policy when the webhook feature does not work as expected. | Valid values: Ignore and Fail.
Due to the limits of cluster API servers, when timeout-seconds is set to 15, failure-policy is set to Fail, and 10 service accounts are created per second for a period of time, service accounts may fail to be created due to timeout errors. |
timeout-seconds | The timeout period of requests to create service accounts. The failure-policy is applied when a request to create a service account times out. | Default value: 10. Unit: seconds. You do not need to specify the unit. |