Select virtual node-based scheduling solutions for different types of ACK clusters and scenarios - Container Service for Kubernetes

The virtual node-based scheduling solutions supported by ACK managed clusters, including Pro Edition and Basic Edition clusters, are different from those supported by ACK dedicated clusters. Different solutions are suitable for different scenarios and requirements. For example, the solutions suitable for scheduling pods only to virtual nodes are different from those suitable for spreading pods across zones. This topic describes how to select virtual node-based scheduling solutions based on the cluster type and scenario.

Common virtual node-based scheduling scenarios

You want to schedule pods only to virtual nodes.
You want to prioritize Elastic Compute Service (ECS) instances for pod scheduling and use virtual nodes only when ECS instances are insufficient.

Assume that you use an ACK Basic cluster. In Scenario 1, we recommend that you add the alibabacloud.com/eci=true label to pod configurations. In Scenario 2, we recommend that you upgrade the cluster to an ACK Pro cluster. Compared with ACK Basic clusters, ACK Pro clusters provide more features, enhance cluster reliability, and service-level agreement (SLA) guarantee, and allow you to deploy more pods. ACK allows you to seamlessly migrate from ACK Basic clusters to ACK Pro clusters. For more information, see Hot migration from ACK Basic clusters to ACK Pro clusters.

Usage notes

The label alibabacloud.com/eci=true has a high priority. If this label is used with the following scheduling methods, those scheduling methods will not apply:
- Kubernetes native scheduling semantics, such as nodeSelector, affinity and anti-affinity, and pod topology spread constraints
- ResourcePolicy
- ElasticResource (annotation:alibabacloud.com/burst-resource)
We recommend that you do not use the ack-kubernetes-elastic-workload component to create elastic workloads or use the alibabacloud.com/burst-resource annotation to configure elastic resources. The ack-kubernetes-elastic-workload component and the alibabacloud.com/burst-resource annotation are in an inactive development state.
Alibaba Cloud has discontinued the maintenance of virtual-kubelet-autoscaler. If you use ACK Basic clusters, we recommend that you upgrade the clusters to ACK Pro clusters and uninstall virtual-kubelet-autoscaler to save node resources. You can use Kubernetes-native scheduling semantics to spread Elastic Container Instance-based pods across zones or deploy Elastic Container Instance-based pods in specific zones based on affinity settings. For more information, see Spread Elastic Container Instance-based pods across zones and configure affinities.

Solution comparison and suggestions on how to select solutions

The following table contains the following terms:

Priority-based scheduling: If your cluster contains different types of nodes, you can configure the priorities of different node types for pod scheduling. For example, you can prioritize ECS instances over virtual nodes for pod scheduling. This way, ACK preferentially schedules pods to ECS instances, and schedules pods to virtual nodes only when ECS instances are insufficient.
If a scheduling method does not support the priority-based scheduling policy, it means that the scheduling method does not allow you to specify priorities for different types of nodes. For example, if you add the alibabacloud.com/eci=true label to the pod configurations, the pod can be scheduled only to virtual nodes, and ECS instances are not used for pod scheduling. In this case, priority-based scheduling cannot be enabled.
Strict scheduling policy: This policy allows you to specify preferred rules and required rules for scheduling pods to different types of node pools.
- Preferred rules. For example, if you add the preferredDuringSchedulingIgnoredDuringExecution node affinity rule to the pod configurations, the pod is preferentially scheduled to ECS nodes. However, due to the node scoring policy, the pod may still be scheduled to virtual nodes when ECS nodes are sufficient.
- Required rules. For example, you can create a ResourcePolicy for a pod to ensure that the pod is scheduled only to ECS nodes when ECS nodes are sufficient.

ACK Pro clusters

Scheduling method		Typical scenario	Priority-based scheduling	Preferentially scale in Elastic Container Instance-based pods	Recommended	References
labels: `alibabacloud.com/eci=true`		You can use this method to schedule pods only to virtual nodes. However, you cannot specify to which virtual node a pod is scheduled.	Not supported	N/A	Yes	Schedule pods to elastic container instances that are deployed as virtual nodes
Use Kubernetes-native scheduling semantics.	nodeSelector	You can add toleration rules to the pod configurations to schedule the pod only to virtual nodes. In addition, you can specify to which virtual node a pod is scheduled.	Not supported	N/A	Yes	nodeSelector
	Affinity and anti-affinity	You can configure taints, toleration rules, and node affinity rules to schedule pods only to elastic container instances or only to ECS instances. You can also configure the preceding settings to preferentially schedule pods to ECS instances and schedule pods to virtual nodes only when ECS instances are insufficient. This improves the elasticity of pod scheduling. This scheduling method preferentially scales in elastic container instances.	Supports preferred rules	Supported	Yes	Configure resource allocation based on ECS instances and elastic container instances
	Pod topology spread constraint	You can spread pods across zones to implement high-availability and high-performance pod scheduling.	Not supported	Supported	Yes	Spread Elastic Container Instance-based pods across zones and configure affinities
ResourcePolicy		You can configure node pool priorities for pod scheduling. For example, you can configure the system to preferentially schedule pods to Node Pool A and schedule pods to Node Pool B when nodes in Node Pool A are insufficient. When you use both ECS instances and elastic container instances for pod scheduling, you can also configure preferred rules based on the billing method. For example, you can prioritize subscription ECS instances over pay-as-you-go ECS instances and prioritize ECS instances over elastic container instances for pod scheduling. In this case, pods are scaled down in the reverse order of priority: Elastic Container Instance-based pods, pay-as-you-go ECS instance-based pods, and subscription ECS instance-based pods.	Supports required rules	Supported	Yes	Configure priority-based resource scheduling
UnitedDeployment		You can schedule pods to ECS instances and virtual nodes based on the number of replicated pods created for a Deployment. For example, you can configure the system to use subscription ECS instances when the number of replicas is lower than 10, use preemptible instances when the number of replicas exceeds 10 but does not exceed 20, and use elastic container instances when the number of replicas exceeds 20.	Supported	Supported	Yes	UnitedDeployment Scenario 3: Use the UnitedDeployment controller to colocate applications on ECS instances and elastic container instances
ElasticWorkload (This method is in an inactive development state. We recommend that you use the UnitedDeployment controller)		You can schedule the replicated pods of a Deployment to ECS instances or virtual nodes in groups.	Supported	Supported	No	Install ack-kubernetes-elastic-workload (discontinued maintenance)
ElasticResource (annotation:`alibabacloud.com/burst-resource`) (in an inactive development state)		Only the following elastic scheduling policies are supported: The `eci` policy: schedules pods to virtual nodes only when ECS instances are insufficient. The `eci_only` policy: schedules pods only to virtual nodes.	Supported	Supported	No	Use elastic resources to implement Elastic Container Instance-based scheduling (discontinued)
virtual-kubelet-autoscaler (discontinued maintenance)		You can only configure the system to preferentially schedule pods to ECS instances and schedule pods to virtual nodes only when ECS instances are insufficient.	Supported	Supported	No	None

ACK Basic clusters and ACK dedicated clusters

Scheduling method		Typical scenario	Priority-based scheduling	Preferentially scale in Elastic Container Instance-based pods	Recommended	References
labels: `alibabacloud.com/eci=true`		You can use this method to schedule pods only to virtual nodes. However, you cannot specify to which virtual node a pod is scheduled.	Not supported	N/A	Yes	Schedule pods to elastic container instances that are deployed as virtual nodes
UnitedDeployment		You can schedule pods to ECS instances and virtual nodes based on the number of replicated pods created for a Deployment. For example, you can configure the system to use subscription ECS instances when the number of replicas is lower than 10, use preemptible instances when the number of replicas exceeds 10 but does not exceed 20, and use elastic container instances when the number of replicas exceeds 20.	Supported	Supported	Yes	UnitedDeployment
Use Kubernetes-native scheduling semantics.	nodeSelector	You can add toleration rules to the pod configurations to schedule the pod only to virtual nodes. In addition, you can specify to which virtual node a pod is scheduled.	Not supported	Supported	No. Compared with the kube-scheduler components in ACK Pro clusters, the kube-scheduler components in ACK Basic clusters and ACK dedicated clusters are unaware of the inventory of underlying resources during pod scheduling. As a result, the probability of successful pod creation is lower in ACK Pro clusters.	nodeSelector
	Affinity and anti-affinity	You can configure taints, toleration rules, and node affinity rules to schedule pods only to elastic container instances or only to ECS instances. You can also configure the preceding settings to preferentially schedule pods to ECS instances and schedule pods to virtual nodes only when ECS instances are insufficient. This improves the elasticity of pod scheduling.	Supports preferred rules	Supported		Affinity and anti-affinity
	Pod topology spread constraint	You can spread pods across zones to implement high-availability and high-performance pod scheduling.	Not supported	Supported		Pod topology spread constraints
ElasticWorkload (This method is in an inactive development state. We recommend that you use the UnitedDeployment controller)		You can schedule the replicated pods of a Deployment to ECS instances or virtual nodes in groups.	Supported	Supported	No	Install ack-kubernetes-elastic-workload (discontinued)
ElasticResource (annotation:`alibabacloud.com/burst-resource`) (in an inactive development state)		Only the following elastic scheduling policies are supported: The `eci` policy: schedules pods to virtual nodes only when ECS instances are insufficient. The `eci_only` policy: schedules pods only to virtual nodes.	Supported	Supported	No	Use elastic resources to implement Elastic Container Instance-based scheduling (discontinued)
virtual-kubelet-autoscaler (discontinued maintenance)		You can only configure the system to preferentially schedule pods to ECS instances and schedule pods to virtual nodes only when ECS instances are insufficient.	Supported	Supported	No	None