If your business does not need to run for a long time, such as for web service requests, is highly tolerant to interruptions, and can be released when no longer needed, you can configure a Knative Service to use preemptible instances, which are more cost-effective than pay-as-you-go Elastic Compute Service (ECS) instances. Using Knative with preemptible instances ensures the elasticity and response speed of your service, improves the cost-effectiveness of cloud computing resources to the maximum extent, and develops applications on top of a more efficient and cost-effective serverless framework.
Prerequisites
Knative has been deployed in your cluster. For more information, see Deploy Knative.
Terms
Implementation, features, and benefits
In Knative, serverless workloads are managed through Knative Services. Knative Services can automatically scale pods based on requests. To use preemptible instances, you need only to add the corresponding pod annotation. Virtual nodes then automatically apply for elastic container instances based on the specification specified in the pod annotation. Virtual nodes also support auto replacement of preemptible instances to automate the use of preemptible instances.
Benefits of using Knative with preemptible instances:
Serverless scenarios: Resources can be created and released on demand for short-term web services.
Adaptive to graceful shutdown: During the auto replacement of preemptible instances, the workload controller needs to delete the current pods and then creates a new preemptible instance. This requires the application containers to support graceful shutdown. In Knative, each pod contains a queue-proxy sidecar container. When the workload controller deletes a pod, it must wait until the queue-proxy sidecar container finishes processing all requests before it can delete all application containers.
Cost-sensitive users: Preemptible instances are more suitable for Knative users that are sensitive to costs.
Examples of using Knative with preemptible instances
Configure preemptible instances
To use elastic container instances, you must add the following annotations to Elastic Container Instance-based pods when you create the pods. If you add the annotations to existing Elastic Container Instance-based pods or you directly modify the annotations of Elastic Container Instance-based pods, the annotations do not take effect.
You can add the annotations in the following table to a Service to configure preemptible instances.
Annotation | Example | Required | Description |
k8s.aliyun.com/eci-spot-strategy | SpotAsPriceGo | Yes | Specifies the bidding policy for preemptible instances. Valid values:
|
k8s.aliyun.com/eci-spot-price-limit | "0.5" | No | The maximum hourly price of preemptible instances. This value can be accurate to three decimal places. This annotation takes effect only when |
k8s.aliyun.com/eci-spot-duration | "1" | No | Specifies the protection period of preemptible instances. Default value: 1. You can set the value to 0. A value of 0 indicates no protection period is specified. |
k8s.aliyun.com/eci-spot-fallback | "true" | No | Specifies whether to automatically create a pay-as-you-go instance when preemptible instances of the specified type are out of stock. Default value: |
Example 1: Specify an ECS instance type and use the SpotWithPriceLimit policy.
The following YAML template creates a preemptible instance of the ecs.c6 type.
If instances that meet the instance type and maximum hourly price requirements are out of stock, no preemptible instance is created.
After a preemptible instance is created, you are guaranteed to use the instance within the protection period (1 hour). After the protection period ends, if the market price exceeds your bid price or instances of the specified type become insufficient, the preemptible instance is released.
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
name: helloworld-go
spec:
template:
metadata:
labels:
alibabacloud.com/eci: "true"
annotations:
k8s.aliyun.com/eci-use-specs : "ecs.c6.large" # Specify an Elastic Compute Service (ECS) instance type.
k8s.aliyun.com/eci-spot-strategy: "SpotWithPriceLimit" # Use the SpotWithPriceLimit policy to set the maximum hourly price.
k8s.aliyun.com/eci-spot-price-limit: "0.250" # Specify the maximum hourly price.
spec:
containers:
- env:
- name: TARGET
value: "Knative"
image: registry.cn-hangzhou.aliyuncs.com/knative-sample/helloworld-go:73fbdd56
Example 2: Automatically create pay-as-you-go instances when the inventory is insufficient
The following YAML template creates a preemptible instance of the ecs.c6 type.
A preemptible instance is created if the instance type and maximum hourly price requirements are met. After a preemptible instance is created, you are guaranteed to use the instance within the protection period (1 hour). After the protection period ends, if the market price exceeds your bid price or instances of the specified type become insufficient, the preemptible instance is released.
A pay-as-you-go instance is created if instances that meet the instance type and maximum hourly price requirements are out of stock. The system does not automatically release the pay-as-you-go instance. After the pay-as-you-go instance is created, you can run the
kubectl describe pod
command to view the pod events to check whether a pay-as-you-go instance is created. If the SpotDegraded event is generated, a pay-as-you-go instance is created.
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
name: helloworld-go
spec:
template:
metadata:
labels:
alibabacloud.com/eci: "true"
annotations:
k8s.aliyun.com/eci-use-specs : "ecs.c6.large" # Specify an ECS instance type.
k8s.aliyun.com/eci-spot-strategy: "SpotWithPriceLimit" # Use the SpotWithPriceLimit policy to set the maximum hourly price.
k8s.aliyun.com/eci-spot-price-limit: "0.05" # Set the maximum hourly price.
k8s.aliyun.com/eci-spot-fallback: "true" # Create a pay-as-you-go instance when preemptible instances are out of stock.
spec:
containers:
- env:
- name: TARGET
value: "Knative"
image: registry.cn-hangzhou.aliyuncs.com/knative-sample/helloworld-go:73fbdd56
Step 1: Configure preemptible instances for a Knative Service
To minimize the computing cost, you can add the k8s.aliyun.com/eci-spot-strategy
annotation to your Knative Service. This way, the Service prioritizes preemptible elastic container instances. Preemptible instances are more cost-effective than pay-as-you-go instances.
However, preemptible instances are reclaimed when the market price exceeds the bid price or instances of the specified type are out of stock. This causes service interruptions. To ensure business continuity and stability, set the k8s.aliyun.com/eci-spot-fallback: "true"
annotation to allow the Knative Service to create pay-as-you-go instances when preemptible instances are insufficient. This helps you balance costs and service stability.
Log on to the ACK console. In the left-side navigation pane, click Clusters.
On the Clusters page, find the cluster that you want to manage and click its name. In the left-side navigation pane, choose .
On the Services tab of the Knative page, set Namespace to default and click Create from Template. Select Custom from the Sample Template drop-down list, copy the following content to the template, and click Create to create a Knative Service named helloworld-go.
apiVersion: serving.knative.dev/v1 kind: Service metadata: name: helloworld-go spec: template: metadata: labels: alibabacloud.com/eci: "true" annotations: k8s.aliyun.com/eci-use-specs : "ecs.c6.large" # Specify an ECS instance type. k8s.aliyun.com/eci-spot-strategy: "SpotAsPriceGo" # Use the SpotAsPriceGo policy. k8s.aliyun.com/eci-spot-duration: "1" # Set the protection period. Default value: 1. Unit: hours. You can specify a protection period on demand. k8s.aliyun.com/eci-spot-fallback: "true" # Create a pay-as-you-go instance when preemptible instances are out of stock. spec: containers: - env: - name: TARGET value: "Knative" image: registry-vpc.{REGION-ID}.aliyuncs.com/knative-samples/helloworld-go:160e4dc8 # {REGION-ID} indicates the region of the cluster, such as cn-hangzhou.
The preceding YAML file creates a preemptible instance of the ecs.c6 type.
When instances of the specified type are out of stock, no preemptible instance is created.
After a preemptible instance is created, you are guaranteed to use the instance for 1 hour. The instance is not released within the protection period even if the market price exceeds the bid price. After the protection period ends, if the market price exceeds the bid price or instances of the specified type are out of stock, Alibaba Cloud may release the preemptible instance. To avoid this issue, set the
k8s.aliyun.com/eci-spot-fallback: "true"
annotation to allow the system to create pay-as-you-go instances to ensure business continuity when preemptible instances are insufficient or released.
(Optional) Step 2: Configure graceful shutdown for preemptible instances
Limits
To enable notifications on preemptible instance interruptions by using pod conditions and evict preemptible instances by using the Eviction API, you must update ack-virtual-node to 2.11.0 or later. For more information about ack-virtual-node, see ack-virtual-node.
Notifications on preemptible instance interruptions
Alibaba Cloud notifies you 3 minutes before a preemptible instance interruption by sending a SpotToBeReleased event and changing the value of ContainerInstanceExpired
in the pod condition to true
.
The following figure is an example of the notification.
Configure graceful shutdown for expired preemptible instances
To avoid business interruptions caused by the reclaiming of preemptible elastic container instances, virtual nodes allow you to configure graceful shutdown for preemptible elastic container instances. To do this, add the k8s.aliyun.com/eci-spot-release-strategy: api-evict
annotation to pods that are scheduled to preemptible instances. When a virtual node receives a SpotToBeReleased event
, it calls the Eviction API to evict the preemptible instance. The eviction complies with the PodDisruptionBudgets
and terminationGracePeriodSeconds
settings. Creating an Eviction object by calling the API is similar to the DELETE
action in pod control policies.
Make an API call: When a virtual node receives a SpotToBeReleased event, the node calls the Eviction API.
Check the PDB: The API server checks the PDB configured for the pod to be evicted.
Perform eviction: After the API server validates the eviction request, the pod is evicted through the following steps:
The pod updates its deletion timestamp so that the API server can recognize the pod to be terminated. The pod is also stamped with the grace period that you configured.
The kubelet of the virtual node on which the pod runs recognizes the pod and initiates graceful shutdown for the pod.
When the kubelet terminates the pod, the control plane of the cluster disassociates the pod from relevant Endpoints and Endpointslices. In this case, the pod controller no longer recognizes the pod as valid.
When the grace period ends, the kubelet forcefully terminates the pod.
The kubelet notifies the system to delete the pod.
The API server deletes the pod.
In Knative scenarios, each Service pod contains a queue-proxy sidecar container. To ensure business continuity and data integrity, when the system deletes a pod, it waits until the queue-proxy sidecar container finishes processing all ongoing requests before it can terminate all application containers. This enables seamless pod scaling and updates.
Release of preemptible instances
After a preemptible instance is created, it can run normally during the protection period. After the protection period expires, the preemptible instance is released if the market price is higher than your bid price or the inventory resources are insufficient. You can understand the release status of preemptible instances by performing the following operations:
SpotToBeReleased event
A SpotToBeReleased event is generated about 5 minutes before a preemptible instance is released.
ImportantElastic Container Instance sends the event to the list of Kubernetes events. During the five-minute period, you can take measures to ensure that your business is not affected by the release of the preemptible instance.
Run the
kubectl describe
command to view the details of the preemptible instance. The SpotToBeReleased event is displayed in the Events section of the command output. Example:Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning SpotToBeReleased 3m32s kubelet, eci Spot ECI will be released in 3 minutes
Run the
kukubectl get events
command to view the event information. The SpotToBeReleased event is displayed in the command output. Example:LAST SEEN TYPE REASON OBJECT MESSAGE 3m39s Warning SpotToBeReleased pod/pi-frmr8 Spot ECI will be released in 3 minutes
Status of the preemptible instance after it is released
After a preemptible instance is released, the instance information is still retained. The status of the instance is changed to Failed and the cause of the failure is BidFailed.
Run the
kubectl get pod
command to view the instance information. The instance status that is displayed in the command output changes to BidFailed. Example:NAME READY STATUS RESTARTS AGE pi-frmr8 1/1 BidFailed 0 3h5m
Run the
kubectl describe
command to view the details of the preemptible instance. Then, view the details about the instance status in the command output. Example:Status: Failed Reason: BidFailed Message: The pod is spot instance, and have been released at 2020-04-08T12:36Z
Reference
To avoid the cold start latency, we recommend that you use the reserved instance feature to reserve a low-specification burstable instance. This helps you balance the costs and efficiency. For more information, see Configure a reserved instance.