By default, pods in Container Service for Kubernetes (ACK) clusters and ACK Serverless clusters are scheduled to run on x86-based virtual nodes when you schedule the pods to run as elastic container instances on virtual nodes. If your applications are deployed on an ARM architecture, you must add the nodeSelector field to specify the virtual nodes that you want to use when you schedule pods to run on virtual nodes. This topic describes how to schedule pods to an ARM-based virtual node.
Prerequisites
The cluster meets the following requirements:
ARM-based elastic container instances are supported in the region in which the cluster resides.
The g8y, c8y, and r8y ARM-based instance families of Elastic Compute Service (ECS) can be used to create elastic container instances. For information about the regions in which ARM-based elastic container instances can be created, see ECS instance types available for each region.
The cluster runs Kubernetes 1.20 or later.
The version of the ACK Virtual Node component in the cluster supports ARM-based nodes.
The version of the kube-scheduler component in the cluster supports ARM-based nodes.
Precautions
When you schedule pods to run on an ARM-based virtual node, you are charged for the corresponding elastic container instance based on the actual ARM specifications, instead of the used vCPU and memory specifications.
After you create the Elastic Container Instance-based pod, you can run the kubectl describe pod <pod name>
command to view the details of the pod and check the ECS instance type of the pod in the k8s.aliyun.com/eci-instance-spec
field.
Add an ARM-based virtual node
You can perform the following operations to modify the eci-profile and add an ARM-based virtual node to the cluster:
Modify the eci-profile.
kubectl edit configmap eci-profile -n kube-system
Set the enableLinuxArm64Node parameter to true in the data field of the eci-profile. Make sure that at least one of the configured zones supports ARM-based elastic container instances. Example:
data: enableClusterIp: "true" enableHybridMode: "false" enableLinuxArm64Node: "true" # Enables the ARM-based node feature. enableLogController: "false" enablePVCController: "false" enablePrivateZone: "false" enableReuseSSLKey: "false" featureGates: MetricsVpcNet=true,ProtectionFinalizers=false,WaitForFirstConsumer=false resourceGroupId: "" securityGroupId: sg-2zeg1fci0oq1hljo6h0a selectors: "" slsMachineGroup: "" vSwitchIds: vsw-2zewa5mb19wr45g63****,vsw-2zevanrscmoiaxryx****,vsw-2ze94pjtfuj9vaymf**** # The vSwitch IDs. At least one vSwitch is deployed in a zone in which ARM-based elastic container instances can be created. vpcId: vpc-2zeghwzptn5zii0w7****
View the ARM-based virtual node that is generated in the cluster.
Run the
kubectl get node
command to view the information about the node. An ARM-based virtual node is automatically generated in the cluster.
Schedule pods to the ARM-based virtual node
Configure the nodeSelector field
In ACK clusters and ACK Serverless clusters, all ARM-based virtual nodes have the kubernetes.io/arch: arm64
label. When you create an Elastic Container Instance-based pod, you can use the nodeSelector field to specify an ARM-based virtual node and schedule the pod to run on the node.
nodeSelector:
kubernetes.io/arch: arm64
Sample YAML file of a pod:
No specifications are specified for the pod.
If you do not specify specifications for the pod, the system automatically uses the ARM-based ECS instance type (ecs.c8y.large) that has 2 vCPUs and 4 GiB of memory.
apiVersion: apps/v1 kind: Deployment metadata: name: test labels: app: test spec: replicas: 1 selector: matchLabels: app: nginx template: metadata: name: nginx-test labels: app: nginx alibabacloud.com/eci: "true" # Schedules the pod to a virtual node. spec: containers: - name: nginx image: arm64v8/centos:7.9.2009 # Uses an ARM-based image. command: ["sleep"] args: ["999999"] nodeSelector: kubernetes.io/arch: arm64 # Specifies an ARM-based virtual node.
vCPU and memory specifications are specified.
If you specify vCPU and memory specifications, the system automatically selects an ARM-based ECS instance type that meets the vCPU and memory specifications from the ECS instance types supported by Elastic Container Instance.
NoteYou can use the limits or requests parameters of containers to specify the vCPU and memory specifications. We recommend that you use the limits parameter. You can also add the
k8s.aliyun.com/eci-use-specs
annotation to specify the vCPU and memory specifications.apiVersion: apps/v1 kind: Deployment metadata: name: test labels: app: test spec: replicas: 1 selector: matchLabels: app: nginx template: metadata: name: nginx-test labels: app: nginx alibabacloud.com/eci: "true" # Schedules the pod to a virtual node. spec: containers: - name: nginx image: arm64v8/centos:7.9.2009 # Uses an ARM-based image. command: ["sleep"] args: ["999999"] resources: limits: cpu: "1000m" # Specifies 1 vCPU. memory: "4096Mi" # Specifies that the memory size of the NGINX container is 4 GiB. nodeSelector: kubernetes.io/arch: arm64 # Specifies an ARM-based virtual node.
An ARM-based ECS instance type is specified.
If you have specific requirements for ECS instance types, you can add the
k8s.aliyun.com/eci-use-specs
annotation to specify an ARM-based ECS instance type. For information about the ARM-based ECS instance types that can be used to create elastic container instances, see Specify an ARM-based ECS instance type to create a pod.apiVersion: apps/v1 kind: Deployment metadata: name: test labels: app: test spec: replicas: 1 selector: matchLabels: app: nginx template: metadata: name: nginx-test labels: app: nginx alibabacloud.com/eci: "true" annotations: k8s.aliyun.com/eci-use-specs: "ecs.c8y.large,ecs.g8y.large" # Specifies ARM-based ECS instance types. You can specify up to five ECS instance types. spec: containers: - name: nginx image: arm64v8/centos:7.9.2009 # Uses an ARM-based image. command: ["sleep"] args: ["999999"] nodeSelector: kubernetes.io/arch: arm64 # Schedules the pod to the ARM-based node.
Configure tolerations and nodeaffinity to schedule multi-architecture images
If your application image is a multi-architecture image and you want to configure pod scheduling across the x86 and ARM architectures, you can configure tolerations and nodeAffinity to preferentially schedule pods to ARM-based or x86-based virtual nodes.
Before you use nodeAffinity to configure node affinity based on the following YAML file, make sure that the virtual node-based pod scheduling feature is enabled for the cluster. For more information, see Enable the virtual node-based pod scheduling policy for an ACK cluster.
Sample YAML file of a pod:
Preferentially schedule the pod to an ARM-based virtual node
apiVersion: apps/v1 kind: Deployment metadata: name: test labels: app: test spec: replicas: 1 selector: matchLabels: app: nginx template: metadata: name: nginx-test labels: app: nginx alibabacloud.com/eci: "true" # Schedules the pod to a virtual node. spec: tolerations: # Taints of nodes that tolerate the ARM architecture. - key: kubernetes.io/arch operator: Equal value: arm64 effect: NoSchedule affinity: nodeAffinity: preferredDuringSchedulingIgnoredDuringExecution: - weight: 1 preference: matchExpressions: - key: kubernetes.io/arch operator: In values: - arm64 # Preferentially schedules the pod to the ARM-based virtual node. containers: - name: nginx image: arm64v8/centos:7.9.2009 # Uses multi-architecture images. command: ["sleep"] args: ["999999"]
Preferentially schedule the pod to an x86-based virtual node
apiVersion: apps/v1 kind: Deployment metadata: name: test labels: app: test spec: replicas: 1 selector: matchLabels: app: nginx template: metadata: name: nginx-test labels: app: nginx alibabacloud.com/eci: "true" # Schedules the pod to a virtual node. spec: tolerations: # Taints of nodes that tolerate the ARM architecture. - key: kubernetes.io/arch operator: Equal value: arm64 effect: NoSchedule affinity: nodeAffinity: preferredDuringSchedulingIgnoredDuringExecution: - weight: 1 preference: matchExpressions: - key: kubernetes.io/arch operator: In values: - amd64 # Preferentially schedules the pod to the x86-based virtual node. containers: - name: nginx image: arm64v8/centos:7.9.2009 # Uses multi-architecture images. command: ["sleep"] args: ["999999"]
References
ACK clusters: Schedule workloads to ARM-based nodes
ACK Serverless clusters: Schedule workloads to ARM-based virtual nodes