Container Service for Kubernetes:Use Knative with preemptible instances

Prerequisites

Knative is deployed in a Container Service for Kubernetes (ACK) cluster or an ACK Serverless cluster. For more information, see Deploy Knative in an ACK cluster and Deploy Knative in an ACK Serverless cluster.

Terms

• Preemptible instance

  Preemptible instances are cost-effective spot instances. You can bid for idle computing instances on Alibaba Cloud and then run containers on these instances if your bid price is higher than the market price. When the market price exceeds your bid price or the instances are out of stock, preemptible instances are reclaimed. For more information, see What are preemptible instances?.

• Knative

  Knative is an open source serverless application orchestration framework developed based on Kubernetes. Knative is intended for cloud-native and cross-platform serverless application orchestration. Knative provides the following features: request-based auto scaling, scaling to zero, version management, traffic-based canary release, function deployment, and eventing. For more information about Knative, see ACK Knative overview and ACK Serverless Knative overview.

Implementation, features, and benefits

In Knative, serverless workloads are managed through Knative Services. Knative Services can automatically scale pods based on requests. To use preemptible instances, you need only to add the corresponding pod annotation. Virtual nodes then automatically apply for elastic container instances based on the specification specified in the pod annotation. Virtual nodes also support auto replacement of preemptible instances to automate the use of preemptible instances.

Benefits of using Knative with preemptible instances:

• Serverless scenarios: Resources can be created and released on demand for short-term web services.

• Adaptive to graceful shutdown: During the auto replacement of preemptible instances, the workload controller needs to delete the current pods and then creates a new preemptible instance. This requires the application containers to support graceful shutdown. In Knative, each pod contains a queue-proxy sidecar container. When the workload controller deletes a pod, it must wait until the queue-proxy sidecar container finishes processing all requests before it can delete all application containers.

• Cost-sensitive users: Preemptible instances are more suitable for Knative users that are sensitive to costs.

Examples of using Knative with preemptible instances

Configure preemptible instances

You can add the annotations in the following table to a Service to configure preemptible instances.

Example 1: Specify an ECS instance type and use the SpotWithPriceLimit policy.

The following YAML template creates a preemptible instance of the ecs.c6 type.

• If instances that meet the instance type and maximum hourly price requirements are out of stock, no preemptible instance is created.

• After a preemptible instance is created, you are guaranteed to use the instance within the protection period (1 hour). After the protection period ends, if the market price exceeds your bid price or instances of the specified type become insufficient, the preemptible instance is released.

apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: helloworld-go
spec:
  template:
    metadata:
      labels:
        alibabacloud.com/eci: "true"
      annotations:
        k8s.aliyun.com/eci-use-specs : "ecs.c6.large"           # Specify an Elastic Compute Service (ECS) instance type. 
        k8s.aliyun.com/eci-spot-strategy: "SpotWithPriceLimit"  # Use the SpotWithPriceLimit policy to set the maximum hourly price. 
        k8s.aliyun.com/eci-spot-price-limit: "0.250"   # Specify the maximum hourly price. 
    spec:         
      containers:
      - env:
        - name: TARGET
          value: "Knative"
        image: registry.cn-hangzhou.aliyuncs.com/knative-sample/helloworld-go:73fbdd56

Example 2: Automatically create pay-as-you-go instances when the inventory is insufficient

The following YAML template creates a preemptible instance of the ecs.c6 type.

• A preemptible instance is created if the instance type and maximum hourly price requirements are met. After a preemptible instance is created, you are guaranteed to use the instance within the protection period (1 hour). After the protection period ends, if the market price exceeds your bid price or instances of the specified type become insufficient, the preemptible instance is released.

• A pay-as-you-go instance is created if instances that meet the instance type and maximum hourly price requirements are out of stock. The system does not automatically release the pay-as-you-go instance. After the pay-as-you-go instance is created, you can run the kubectl describe pod command to view the pod events to check whether a pay-as-you-go instance is created. If the SpotDegraded event is generated, a pay-as-you-go instance is created.

apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: helloworld-go
spec:
  template:
    metadata:
      labels:
        alibabacloud.com/eci: "true" 
      annotations:
        k8s.aliyun.com/eci-use-specs : "ecs.c6.large"           # Specify an ECS instance type. 
        k8s.aliyun.com/eci-spot-strategy: "SpotWithPriceLimit"  # Use the SpotWithPriceLimit policy to set the maximum hourly price. 
        k8s.aliyun.com/eci-spot-price-limit: "0.05"             # Set the maximum hourly price. 
        k8s.aliyun.com/eci-spot-fallback: "true"                # Create a pay-as-you-go instance when preemptible instances are out of stock. 
    spec:         
      containers:
      - env:
        - name: TARGET
          value: "Knative"
        image: registry.cn-hangzhou.aliyuncs.com/knative-sample/helloworld-go:73fbdd56

Step 1: Configure preemptible instances for a Knative Service

To minimize the computing cost, you can add the k8s.aliyun.com/eci-spot-strategy annotation to your Knative Service. This way, the Service prioritizes preemptible elastic container instances. Preemptible instances are more cost-effective than pay-as-you-go instances.

However, preemptible instances are reclaimed when the market price exceeds the bid price or instances of the specified type are out of stock. This causes service interruptions. To ensure business continuity and stability, set the k8s.aliyun.com/eci-spot-fallback: "true" annotation to allow the Knative Service to create pay-as-you-go instances when preemptible instances are insufficient. This helps you balance costs and service stability.

1. Log on to the ACK console. In the left-side navigation pane, click Clusters.

2. On the Clusters page, find the cluster that you want to manage and click its name. In the left-side navigation pane, choose Applications > Knative.

3. On the Services tab of the Knative page, set Namespace to default and click Create from Template. Select Custom from the Sample Template drop-down list, copy the following content to the template, and click Create to create a Knative Service named helloworld-go.

   apiVersion: serving.knative.dev/v1
   kind: Service
   metadata:
     name: helloworld-go
   spec:
     template:
       metadata:
         labels:
           alibabacloud.com/eci: "true" 
         annotations:
           k8s.aliyun.com/eci-use-specs : "ecs.c6.large"           # Specify an ECS instance type. 
           k8s.aliyun.com/eci-spot-strategy: "SpotAsPriceGo"       # Use the SpotAsPriceGo policy. 
           k8s.aliyun.com/eci-spot-duration: "1"                   # Set the protection period. Default value: 1. Unit: hours. You can specify a protection period on demand. 
           k8s.aliyun.com/eci-spot-fallback: "true"                # Create a pay-as-you-go instance when preemptible instances are out of stock. 
       spec:         
         containers:
         - env:
           - name: TARGET
             value: "Knative"
           image: registry-vpc.{REGION-ID}.aliyuncs.com/knative-samples/helloworld-go:160e4dc8 # {REGION-ID} indicates the region of the cluster, such as cn-hangzhou.

   The preceding YAML file creates a preemptible instance of the ecs.c6 type.

   • When instances of the specified type are out of stock, no preemptible instance is created.

   • After a preemptible instance is created, you are guaranteed to use the instance for 1 hour. The instance is not released within the protection period even if the market price exceeds the bid price. After the protection period ends, if the market price exceeds the bid price or instances of the specified type are out of stock, Alibaba Cloud may release the preemptible instance. To avoid this issue, set the k8s.aliyun.com/eci-spot-fallback: "true" annotation to allow the system to create pay-as-you-go instances to ensure business continuity when preemptible instances are insufficient or released.

(Optional) Step 2: Configure graceful shutdown for preemptible instances

Configure graceful shutdown for expired preemptible instances

To avoid business interruptions caused by the reclaiming of preemptible elastic container instances, virtual nodes allow you to configure graceful shutdown for preemptible elastic container instances. To do this, add the k8s.aliyun.com/eci-spot-release-strategy: api-evict annotation to pods that are scheduled to preemptible instances. When a virtual node receives a SpotToBeReleased event, it calls the Eviction API to evict the preemptible instance. The eviction complies with the PodDisruptionBudgets and terminationGracePeriodSeconds settings. Creating an Eviction object by calling the API is similar to the DELETE action in pod control policies.

1. Make an API call: When a virtual node receives a SpotToBeReleased event, the node calls the Eviction API.

2. Check the PDB: The API server checks the PDB configured for the pod to be evicted.

3. Perform eviction: After the API server validates the eviction request, the pod is evicted through the following steps:

   1. The pod updates its deletion timestamp so that the API server can recognize the pod to be terminated. The pod is also stamped with the grace period that you configured.

   2. The kubelet of the virtual node on which the pod runs recognizes the pod and initiates graceful shutdown for the pod.

   3. When the kubelet terminates the pod, the control plane of the cluster disassociates the pod from relevant Endpoints and Endpointslices. In this case, the pod controller no longer recognizes the pod as valid.

   4. When the grace period ends, the kubelet forcefully terminates the pod.

   5. The kubelet notifies the system to delete the pod.

   6. The API server deletes the pod.

4. In Knative scenarios, each Service pod contains a queue-proxy sidecar container. To ensure business continuity and data integrity, when the system deletes a pod, it waits until the queue-proxy sidecar container finishes processing all ongoing requests before it can terminate all application containers. This enables seamless pod scaling and updates.

Reference

To avoid the cold start latency, we recommend that you use the reserved instance feature to reserve a low-specification burstable instance. This helps you balance the costs and efficiency. For more information, see Configure a reserved instance.