All Products
Search
Document Center

Container Service for Kubernetes:Introduction and comparison of virtual node-based scheduling solutions

Last Updated:Aug 19, 2024

The virtual node-based scheduling solutions supported by ACK managed clusters, including Pro Edition and Basic Edition clusters, are different from those supported by ACK dedicated clusters. Different solutions are suitable for different scenarios and requirements. For example, the solutions suitable for scheduling pods only to virtual nodes are different from those suitable for spreading pods across zones. This topic describes how to select virtual node-based scheduling solutions based on the cluster type and scenario.

Common virtual node-based scheduling scenarios

  1. You want to schedule pods only to virtual nodes.

  2. You want to prioritize Elastic Compute Service (ECS) instances for pod scheduling and use virtual nodes only when ECS instances are insufficient.

Assume that you use an ACK Basic cluster. In Scenario 1, we recommend that you add the alibabacloud.com/eci=true label to pod configurations. In Scenario 2, we recommend that you upgrade the cluster to an ACK Pro cluster. Compared with ACK Basic clusters, ACK Pro clusters provide more features, enhance cluster reliability, and service-level agreement (SLA) guarantee, and allow you to deploy more pods. ACK allows you to seamlessly migrate from ACK Basic clusters to ACK Pro clusters. For more information, see Hot migration from ACK Basic clusters to ACK Pro clusters.

Usage notes

  • We recommend that you do not use the ack-kubernetes-elastic-workload component to create elastic workloads or use the alibabacloud.com/burst-resource annotation to configure elastic resources. The ack-kubernetes-elastic-workload component and the alibabacloud.com/burst-resource annotation are in an inactive development state.

  • Alibaba Cloud has discontinued the maintenance of virtual-kubelet-autoscaler. If you use ACK Basic clusters, we recommend that you upgrade the clusters to ACK Pro clusters and uninstall virtual-kubelet-autoscaler to save node resources. You can use Kubernetes-native scheduling semantics to spread Elastic Container Instance-based pods across zones or deploy Elastic Container Instance-based pods in specific zones based on affinity settings. For more information, see Spread Elastic Container Instance-based pods across zones and configure affinities.

Solution comparison and suggestions on how to select solutions

The following table contains the following terms:

  • Priority-based scheduling: If your cluster contains different types of nodes, you can configure the priorities of different node types for pod scheduling. For example, you can prioritize ECS instances over virtual nodes for pod scheduling. This way, ACK preferentially schedules pods to ECS instances, and schedules pods to virtual nodes only when ECS instances are insufficient.

    If a scheduling method does not support the priority-based scheduling policy, it means that the scheduling method does not allow you to specify priorities for different types of nodes. For example, if you add the alibabacloud.com/eci=true label to the pod configurations, the pod can be scheduled only to virtual nodes, and ECS instances are not used for pod scheduling. In this case, priority-based scheduling cannot be enabled.

  • Strict scheduling policy: This policy allows you to specify preferred rules and required rules for scheduling pods to different types of node pools.

    • Preferred rules. For example, if you add the preferredDuringSchedulingIgnoredDuringExecution node affinity rule to the pod configurations, the pod is preferentially scheduled to ECS nodes. However, due to the node scoring policy, the pod may still be scheduled to virtual nodes when ECS nodes are sufficient.

    • Required rules. For example, you can create a ResourcePolicy for a pod to ensure that the pod is scheduled only to ECS nodes when ECS nodes are sufficient.

ACK Pro clusters

Scheduling method

Typical scenario

Priority-based scheduling

Preferentially scale in Elastic Container Instance-based pods

Recommended

References

labels: alibabacloud.com/eci=true

You can use this method to schedule pods only to virtual nodes. However, you cannot specify to which virtual node a pod is scheduled.

Not supported

N/A

Yes

Schedule pods to elastic container instances that are deployed as virtual nodes

Use Kubernetes-native scheduling semantics.

nodeSelector

You can add toleration rules to the pod configurations to schedule the pod only to virtual nodes. In addition, you can specify to which virtual node a pod is scheduled.

Not supported

N/A

Yes

nodeSelector

Affinity and anti-affinity

You can configure taints, toleration rules, and node affinity rules to schedule pods only to elastic container instances or only to ECS instances. You can also configure the preceding settings to preferentially schedule pods to ECS instances and schedule pods to virtual nodes only when ECS instances are insufficient. This improves the elasticity of pod scheduling.

This scheduling method preferentially scales in elastic container instances.

Supports preferred rules

Supported

Yes

Configure resource allocation based on ECS instances and elastic container instances

Pod topology spread constraint

You can spread pods across zones to implement high-availability and high-performance pod scheduling.

Not supported

Supported

Yes

Spread Elastic Container Instance-based pods across zones and configure affinities

ResourcePolicy

  • You can configure node pool priorities for pod scheduling. For example, you can configure the system to preferentially schedule pods to Node Pool A and schedule pods to Node Pool B when nodes in Node Pool A are insufficient.

  • When you use both ECS instances and elastic container instances for pod scheduling, you can also configure preferred rules based on the billing method. For example, you can prioritize subscription ECS instances over pay-as-you-go ECS instances and prioritize ECS instances over elastic container instances for pod scheduling.

    In this case, pods are scaled down in the reverse order of priority: Elastic Container Instance-based pods, pay-as-you-go ECS instance-based pods, and subscription ECS instance-based pods.

Supports required rules

Supported

Yes

Configure priority-based resource scheduling

UnitedDeployment

You can schedule pods to ECS instances and virtual nodes based on the number of replicated pods created for a Deployment. For example, you can configure the system to use subscription ECS instances when the number of replicas is lower than 10, use preemptible instances when the number of replicas exceeds 10 but does not exceed 20, and use elastic container instances when the number of replicas exceeds 20.

Supported

Supported

Yes

ElasticWorkload

(This method is in an inactive development state. We recommend that you use the UnitedDeployment controller)

You can schedule the replicated pods of a Deployment to ECS instances or virtual nodes in groups.

Supported

Supported

No

Install ack-kubernetes-elastic-workload (discontinued maintenance)

ElasticResource (Annotation:alibabacloud.com/burst-resource)

(in an inactive development state)

Only the following elastic scheduling policies are supported:

  • The eci policy: schedules pods to virtual nodes only when ECS instances are insufficient.

  • The eci_only policy: schedules pods only to virtual nodes.

Supported

Supported

No

Use Elastic Container Instance-based scheduling

virtual-kubelet-autoscaler

(discontinued maintenance)

You can only configure the system to preferentially schedule pods to ECS instances and schedule pods to virtual nodes only when ECS instances are insufficient.

Supported

Supported

No

None

ACK Basic clusters and ACK dedicated clusters

Scheduling method

Typical scenario

Priority-based scheduling

Preferentially scale in Elastic Container Instance-based pods

Recommended

References

labels: alibabacloud.com/eci=true

You can use this method to schedule pods only to virtual nodes. However, you cannot specify to which virtual node a pod is scheduled.

Not supported

N/A

Yes

Schedule pods to elastic container instances that are deployed as virtual nodes

UnitedDeployment

You can schedule pods to ECS instances and virtual nodes based on the number of replicated pods created for a Deployment. For example, you can configure the system to use subscription ECS instances when the number of replicas is lower than 10, use preemptible instances when the number of replicas exceeds 10 but does not exceed 20, and use elastic container instances when the number of replicas exceeds 20.

Supported

Supported

Yes

UnitedDeployment

Use Kubernetes-native scheduling semantics.

nodeSelector

You can add toleration rules to the pod configurations to schedule the pod only to virtual nodes. In addition, you can specify to which virtual node a pod is scheduled.

Not supported

Supported

No.

Compared with the kube-scheduler components in ACK Pro clusters, the kube-scheduler components in ACK Basic clusters and ACK dedicated clusters are unaware of the inventory of underlying resources during pod scheduling. As a result, the probability of successful pod creation is lower in ACK Pro clusters.

nodeSelector

Affinity and anti-affinity

You can configure taints, toleration rules, and node affinity rules to schedule pods only to elastic container instances or only to ECS instances. You can also configure the preceding settings to preferentially schedule pods to ECS instances and schedule pods to virtual nodes only when ECS instances are insufficient. This improves the elasticity of pod scheduling.

Supports preferred rules

Supported

Affinity and anti-affinity

Pod topology spread constraint

You can spread pods across zones to implement high-availability and high-performance pod scheduling.

Not supported

Supported

Pod topology spread constraints

ElasticWorkload

(This method is in an inactive development state. We recommend that you use the UnitedDeployment controller)

You can schedule the replicated pods of a Deployment to ECS instances or virtual nodes in groups.

Supported

Supported

No

Install ack-kubernetes-elastic-workload (discontinued)

ElasticResource (Annotation:alibabacloud.com/burst-resource)

(in an inactive development state)

Only the following elastic scheduling policies are supported:

  • The eci policy: schedules pods to virtual nodes only when ECS instances are insufficient.

  • The eci_only policy: schedules pods only to virtual nodes.

Supported

Supported

No

Use Elastic Container Instance-based scheduling

virtual-kubelet-autoscaler

(discontinued maintenance)

You can only configure the system to preferentially schedule pods to ECS instances and schedule pods to virtual nodes only when ECS instances are insufficient.

Supported

Supported

No

None