Container Service for Kubernetes:Schedule workloads to ARM-based nodes

Prerequisites

• An ACK cluster with ARM-based nodes or an ARM-based node pool is created and the Kubernetes version of the cluster meets the requirement. For more information, see Configure an ARM-based node pool.

  ARM-based instances are available only in specific regions and zones. Make sure that the ACK cluster is deployed in a region that supports ARM-based instances. For more information about the regions and zones where ARM-based instances are available, see Instance Types Available for Each Region.

• The kube-scheduler component is installed. For more information, see kube-scheduler.

Usage notes

If both ARM-based nodes and x86-based nodes exist in your cluster, we recommend that you add the kubernetes.io/arch=arm64:NoSchedule taint to ARM-based nodes. This avoids accidentally scheduling applications or components that do not support the ARM architecture to ARM-based nodes. When you configure nodeSelector or nodeAffinity to schedule applications to ARM-based nodes in an ACK cluster that runs a Kubernetes version earlier than 1.24, you need to add a toleration to tolerate the kubernetes.io/arch=arm64:NoSchedule taint. However, if the cluster runs Kubernetes 1.24 or later, the scheduler automatically tolerates the kubernetes.io/arch=arm64:NoSchedule taint. In this case, you do not need to add a toleration to tolerate the taint.

Billing

Pods deployed on elastic container instances that adopt the ARM architecture are billed based on the ECS instance type used to create the elastic container instances. The pods are not billed based on the usage of vCPUs and memory.

For more information about the ECS instance types that use the ARM architecture, refer to the following topics:

• Overview of instance families

• Pricing of ECS instance types

• Instance Types Available for Each Region

Schedule ARM workloads to ARM-based nodes

If your cluster contains ARM-based nodes and other nodes and all workloads use the ARM architecture, you need to schedule the workloads only to ARM-based nodes. If pods are scheduled to other nodes, the pods cannot launch. By default, all ARM-based nodes have the kubernetes.io/arch=arm64 label. You can use the nodeSelector or nodeAffinity setting to schedule workloads to ARM-based nodes.

nodeSelector:
  kubernetes.io/arch: arm64 # Specify the label that is used to select an ARM-based node.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: only-arm
spec:
  selector:
    matchLabels:
      app: nginx
  replicas: 1
  template:
    metadata:
      labels:
        app: nginx
    spec:
      nodeSelector:
        kubernetes.io/arch: arm64 # Specify the label that is used to select an ARM-based node. 
      containers:
      - name: nginx
        image: nginx

affinity:
  nodeAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/arch
          operator: In
          values:
          - arm64

apiVersion: apps/v1
kind: Deployment
metadata:
  name: only-arm
spec:
  selector:
    matchLabels:
      app: nginx
  replicas: 1
  template:
    metadata:
      labels:
        app: nginx
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: kubernetes.io/arch
                operator: In
                values:
                - arm64
      containers:
      - name: nginx
        image: nginx

Schedule multi-arch workloads to ARM-based nodes

By default, ACK schedules all workloads to x86-based nodes. Pods become pending when x86 nodes are insufficient. If your application image is a multi-arch image, such as an image that supports the x86 and ARM architectures, you need to configure cross-architecture node scheduling.

For example, you can configure node affinity to preferably schedule workloads to ARM-based or x86-based nodes. Then, schedule workloads to other types of nodes when ARM-based or x86-based nodes are insufficient.

      affinity:
        nodeAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 1
            preference:
              matchExpressions:
              - key: kubernetes.io/arch
                operator: In
                values:
                - arm64

apiVersion: apps/v1
kind: Deployment
metadata:
  name: arm-prefer
spec:
  replicas: 1
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
# Preferably schedule the workload to an ARM-based node. 
      affinity:
        nodeAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 1
            preference:
              matchExpressions:
              - key: kubernetes.io/arch
                operator: In
                values:
                - arm64
      containers:
      - name: my-container
        image: nginx

apiVersion: apps/v1
kind: Deployment
metadata:
  name: amd-prefer
spec:
  replicas: 1
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      tolerations:
# Preferably schedule the workload to a x86-based node. 
      affinity:
        nodeAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 1
            preference:
              matchExpressions:
              - key: kubernetes.io/arch
                operator: In
                values:
                - amd64
      containers:
      - name: my-container
        image: nginx

FAQ

Can I use ARM-based preemptible instances?

ARM-based preemptible instances are available. For more information, see Use preemptible instances.

What are the limits on using ARM-based nodes in ACK clusters?

References

• You can create ARM-based virtual nodes and preferably schedule workloads to ARM-based virtual nodes. For more information, see Schedule workloads to ARM-based virtual nodes.

• You can use Container Registry Enterprise Edition to build multi-arch container images. For more information, see Build a multi-arch container image.

• If you want to run big data jobs without the need to worry about cluster resource O&M, see Run Spark jobs on ARM-based virtual nodes.