The virtual node-based scheduling solutions supported by ACK managed clusters, including Pro Edition and Basic Edition clusters, are different from those supported by ACK dedicated clusters. Different solutions are suitable for different scenarios and requirements. For example, the solutions suitable for scheduling pods only to virtual nodes are different from those suitable for spreading pods across zones. This topic describes how to select virtual node-based scheduling solutions based on the cluster type and scenario.
Common virtual node-based scheduling scenarios
You want to schedule pods only to virtual nodes.
You want to prioritize Elastic Compute Service (ECS) instances for pod scheduling and use virtual nodes only when ECS instances are insufficient.
Assume that you use an ACK Basic cluster. In Scenario 1, we recommend that you add the alibabacloud.com/eci=true
label to pod configurations. In Scenario 2, we recommend that you upgrade the cluster to an ACK Pro cluster. Compared with ACK Basic clusters, ACK Pro clusters provide more features, enhance cluster reliability, and service-level agreement (SLA) guarantee, and allow you to deploy more pods. ACK allows you to seamlessly migrate from ACK Basic clusters to ACK Pro clusters. For more information, see Hot migration from ACK Basic clusters to ACK Pro clusters.
Usage notes
We recommend that you do not use the ack-kubernetes-elastic-workload component to create elastic workloads or use the alibabacloud.com/burst-resource annotation to configure elastic resources. The ack-kubernetes-elastic-workload component and the alibabacloud.com/burst-resource annotation are in an inactive development state.
Alibaba Cloud has discontinued the maintenance of virtual-kubelet-autoscaler. If you use ACK Basic clusters, we recommend that you upgrade the clusters to ACK Pro clusters and uninstall virtual-kubelet-autoscaler to save node resources. You can use Kubernetes-native scheduling semantics to spread Elastic Container Instance-based pods across zones or deploy Elastic Container Instance-based pods in specific zones based on affinity settings. For more information, see Spread Elastic Container Instance-based pods across zones and configure affinities.
Solution comparison and suggestions on how to select solutions
The following table contains the following terms:
Priority-based scheduling: If your cluster contains different types of nodes, you can configure the priorities of different node types for pod scheduling. For example, you can prioritize ECS instances over virtual nodes for pod scheduling. This way, ACK preferentially schedules pods to ECS instances, and schedules pods to virtual nodes only when ECS instances are insufficient.
If a scheduling method does not support the priority-based scheduling policy, it means that the scheduling method does not allow you to specify priorities for different types of nodes. For example, if you add the
alibabacloud.com/eci=true
label to the pod configurations, the pod can be scheduled only to virtual nodes, and ECS instances are not used for pod scheduling. In this case, priority-based scheduling cannot be enabled.Strict scheduling policy: This policy allows you to specify preferred rules and required rules for scheduling pods to different types of node pools.
Preferred rules. For example, if you add the preferredDuringSchedulingIgnoredDuringExecution node affinity rule to the pod configurations, the pod is preferentially scheduled to ECS nodes. However, due to the node scoring policy, the pod may still be scheduled to virtual nodes when ECS nodes are sufficient.
Required rules. For example, you can create a ResourcePolicy for a pod to ensure that the pod is scheduled only to ECS nodes when ECS nodes are sufficient.
ACK Pro clusters
Scheduling method | Typical scenario | Priority-based scheduling | Preferentially scale in Elastic Container Instance-based pods | Recommended | References | |
labels: | You can use this method to schedule pods only to virtual nodes. However, you cannot specify to which virtual node a pod is scheduled. | Not supported | N/A | Yes | Schedule pods to elastic container instances that are deployed as virtual nodes | |
Use Kubernetes-native scheduling semantics. | nodeSelector | You can add toleration rules to the pod configurations to schedule the pod only to virtual nodes. In addition, you can specify to which virtual node a pod is scheduled. | Not supported | N/A | Yes | |
Affinity and anti-affinity | You can configure taints, toleration rules, and node affinity rules to schedule pods only to elastic container instances or only to ECS instances. You can also configure the preceding settings to preferentially schedule pods to ECS instances and schedule pods to virtual nodes only when ECS instances are insufficient. This improves the elasticity of pod scheduling. This scheduling method preferentially scales in elastic container instances. | Supports preferred rules | Supported | Yes | Configure resource allocation based on ECS instances and elastic container instances | |
Pod topology spread constraint | You can spread pods across zones to implement high-availability and high-performance pod scheduling. | Not supported | Supported | Yes | Spread Elastic Container Instance-based pods across zones and configure affinities | |
ResourcePolicy |
| Supports required rules | Supported | Yes | ||
UnitedDeployment | You can schedule pods to ECS instances and virtual nodes based on the number of replicated pods created for a Deployment. For example, you can configure the system to use subscription ECS instances when the number of replicas is lower than 10, use preemptible instances when the number of replicas exceeds 10 but does not exceed 20, and use elastic container instances when the number of replicas exceeds 20. | Supported | Supported | Yes | ||
ElasticWorkload (This method is in an inactive development state. We recommend that you use the UnitedDeployment controller) | You can schedule the replicated pods of a Deployment to ECS instances or virtual nodes in groups. | Supported | Supported | No | Install ack-kubernetes-elastic-workload (discontinued maintenance) | |
ElasticResource (Annotation: (in an inactive development state) | Only the following elastic scheduling policies are supported:
| Supported | Supported | No | ||
virtual-kubelet-autoscaler (discontinued maintenance) | You can only configure the system to preferentially schedule pods to ECS instances and schedule pods to virtual nodes only when ECS instances are insufficient. | Supported | Supported | No | None |
ACK Basic clusters and ACK dedicated clusters
Scheduling method | Typical scenario | Priority-based scheduling | Preferentially scale in Elastic Container Instance-based pods | Recommended | References | |
labels: | You can use this method to schedule pods only to virtual nodes. However, you cannot specify to which virtual node a pod is scheduled. | Not supported | N/A | Yes | Schedule pods to elastic container instances that are deployed as virtual nodes | |
UnitedDeployment | You can schedule pods to ECS instances and virtual nodes based on the number of replicated pods created for a Deployment. For example, you can configure the system to use subscription ECS instances when the number of replicas is lower than 10, use preemptible instances when the number of replicas exceeds 10 but does not exceed 20, and use elastic container instances when the number of replicas exceeds 20. | Supported | Supported | Yes | ||
Use Kubernetes-native scheduling semantics. | nodeSelector | You can add toleration rules to the pod configurations to schedule the pod only to virtual nodes. In addition, you can specify to which virtual node a pod is scheduled. | Not supported | Supported | No. Compared with the kube-scheduler components in ACK Pro clusters, the kube-scheduler components in ACK Basic clusters and ACK dedicated clusters are unaware of the inventory of underlying resources during pod scheduling. As a result, the probability of successful pod creation is lower in ACK Pro clusters. | |
Affinity and anti-affinity | You can configure taints, toleration rules, and node affinity rules to schedule pods only to elastic container instances or only to ECS instances. You can also configure the preceding settings to preferentially schedule pods to ECS instances and schedule pods to virtual nodes only when ECS instances are insufficient. This improves the elasticity of pod scheduling. | Supports preferred rules | Supported | |||
Pod topology spread constraint | You can spread pods across zones to implement high-availability and high-performance pod scheduling. | Not supported | Supported | |||
ElasticWorkload (This method is in an inactive development state. We recommend that you use the UnitedDeployment controller) | You can schedule the replicated pods of a Deployment to ECS instances or virtual nodes in groups. | Supported | Supported | No | ||
ElasticResource (Annotation: (in an inactive development state) | Only the following elastic scheduling policies are supported:
| Supported | Supported | No | ||
virtual-kubelet-autoscaler (discontinued maintenance) | You can only configure the system to preferentially schedule pods to ECS instances and schedule pods to virtual nodes only when ECS instances are insufficient. | Supported | Supported | No | None |