All Products
Search
Document Center

Container Service for Kubernetes:Use Knative with preemptible instances

Last Updated:Nov 06, 2024

If your business does not need to run for a long time, such as for web service requests, is highly tolerant to interruptions, and can be released when no longer needed, you can configure a Knative Service to use preemptible instances, which are more cost-effective than pay-as-you-go Elastic Compute Service (ECS) instances. Using Knative with preemptible instances ensures the elasticity and response speed of your service, improves the cost-effectiveness of cloud computing resources to the maximum extent, and develops applications on top of a more efficient and cost-effective serverless framework.

Prerequisites

Knative has been deployed in your cluster. For more information, see Deploy Knative.

Terms

  • Preemptible instance

    Preemptible instances are cost-effective spot instances. You can bid for idle computing instances on Alibaba Cloud and then run containers on these instances if your bid price is higher than the market price. When the market price exceeds your bid price or the instances are out of stock, preemptible instances are reclaimed. For more information, see What are preemptible instances?.

  • Knative

    Knative is an open source serverless application orchestration framework developed based on Kubernetes. Knative is intended for cloud-native and cross-platform serverless application orchestration. Knative provides the following features: request-based auto scaling, scaling to zero, version management, traffic-based canary release, function deployment, and eventing. For more information about Knative, see ACK Knative overview and ACK Serverless Knative overview.

Implementation, features, and benefits

In Knative, serverless workloads are managed through Knative Services. Knative Services can automatically scale pods based on requests. To use preemptible instances, you need only to add the corresponding pod annotation. Virtual nodes then automatically apply for elastic container instances based on the specification specified in the pod annotation. Virtual nodes also support auto replacement of preemptible instances to automate the use of preemptible instances.

image

Benefits of using Knative with preemptible instances:

  • Serverless scenarios: Resources can be created and released on demand for short-term web services.

  • Adaptive to graceful shutdown: During the auto replacement of preemptible instances, the workload controller needs to delete the current pods and then creates a new preemptible instance. This requires the application containers to support graceful shutdown. In Knative, each pod contains a queue-proxy sidecar container. When the workload controller deletes a pod, it must wait until the queue-proxy sidecar container finishes processing all requests before it can delete all application containers.

  • Cost-sensitive users: Preemptible instances are more suitable for Knative users that are sensitive to costs.

Examples of using Knative with preemptible instances

Configure preemptible instances

Important

To use elastic container instances, you must add the following annotations to Elastic Container Instance-based pods when you create the pods. If you add the annotations to existing Elastic Container Instance-based pods or you directly modify the annotations of Elastic Container Instance-based pods, the annotations do not take effect.

You can add the annotations in the following table to a Service to configure preemptible instances.

Annotation

Example

Required

Description

k8s.aliyun.com/eci-spot-strategy

SpotAsPriceGo

Yes

Specifies the bidding policy for preemptible instances. Valid values:

  • SpotWithPriceLimit: Create a preemptible instance with a maximum hourly price. To set the maximum hourly price, set the k8s.aliyun.com/eci-spot-price-limit annotation.

  • SpotAsPriceGo: Create a preemptible instance for which the system automatically bids based on the current market price.

    Important

    If you choose SpotAsPriceGo and instances of the specified type in the specified zone are insufficient, the maximum hourly price may reach the pay-as-you-go price of the instance.

k8s.aliyun.com/eci-spot-price-limit

"0.5"

No

The maximum hourly price of preemptible instances. This value can be accurate to three decimal places.

This annotation takes effect only when k8s.aliyun.com/eci-spot-strategy is set to SpotWithPriceLimit.

k8s.aliyun.com/eci-spot-duration

"1"

No

Specifies the protection period of preemptible instances. Default value: 1. You can set the value to 0. A value of 0 indicates no protection period is specified.

k8s.aliyun.com/eci-spot-fallback

"true"

No

Specifies whether to automatically create a pay-as-you-go instance when preemptible instances of the specified type are out of stock. Default value: false.

Example 1: Specify an ECS instance type and use the SpotWithPriceLimit policy.

The following YAML template creates a preemptible instance of the ecs.c6 type.

  • If instances that meet the instance type and maximum hourly price requirements are out of stock, no preemptible instance is created.

  • After a preemptible instance is created, you are guaranteed to use the instance within the protection period (1 hour). After the protection period ends, if the market price exceeds your bid price or instances of the specified type become insufficient, the preemptible instance is released.

apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: helloworld-go
spec:
  template:
    metadata:
      labels:
        alibabacloud.com/eci: "true"
      annotations:
        k8s.aliyun.com/eci-use-specs : "ecs.c6.large"           # Specify an Elastic Compute Service (ECS) instance type. 
        k8s.aliyun.com/eci-spot-strategy: "SpotWithPriceLimit"  # Use the SpotWithPriceLimit policy to set the maximum hourly price. 
        k8s.aliyun.com/eci-spot-price-limit: "0.250"   # Specify the maximum hourly price. 
    spec:         
      containers:
      - env:
        - name: TARGET
          value: "Knative"
        image: registry.cn-hangzhou.aliyuncs.com/knative-sample/helloworld-go:73fbdd56

Example 2: Automatically create pay-as-you-go instances when the inventory is insufficient

The following YAML template creates a preemptible instance of the ecs.c6 type.

  • A preemptible instance is created if the instance type and maximum hourly price requirements are met. After a preemptible instance is created, you are guaranteed to use the instance within the protection period (1 hour). After the protection period ends, if the market price exceeds your bid price or instances of the specified type become insufficient, the preemptible instance is released.

  • A pay-as-you-go instance is created if instances that meet the instance type and maximum hourly price requirements are out of stock. The system does not automatically release the pay-as-you-go instance. After the pay-as-you-go instance is created, you can run the kubectl describe pod command to view the pod events to check whether a pay-as-you-go instance is created. If the SpotDegraded event is generated, a pay-as-you-go instance is created.

apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: helloworld-go
spec:
  template:
    metadata:
      labels:
        alibabacloud.com/eci: "true" 
      annotations:
        k8s.aliyun.com/eci-use-specs : "ecs.c6.large"           # Specify an ECS instance type. 
        k8s.aliyun.com/eci-spot-strategy: "SpotWithPriceLimit"  # Use the SpotWithPriceLimit policy to set the maximum hourly price. 
        k8s.aliyun.com/eci-spot-price-limit: "0.05"             # Set the maximum hourly price. 
        k8s.aliyun.com/eci-spot-fallback: "true"                # Create a pay-as-you-go instance when preemptible instances are out of stock. 
    spec:         
      containers:
      - env:
        - name: TARGET
          value: "Knative"
        image: registry.cn-hangzhou.aliyuncs.com/knative-sample/helloworld-go:73fbdd56

Step 1: Configure preemptible instances for a Knative Service

To minimize the computing cost, you can add the k8s.aliyun.com/eci-spot-strategy annotation to your Knative Service. This way, the Service prioritizes preemptible elastic container instances. Preemptible instances are more cost-effective than pay-as-you-go instances.

However, preemptible instances are reclaimed when the market price exceeds the bid price or instances of the specified type are out of stock. This causes service interruptions. To ensure business continuity and stability, set the k8s.aliyun.com/eci-spot-fallback: "true" annotation to allow the Knative Service to create pay-as-you-go instances when preemptible instances are insufficient. This helps you balance costs and service stability.

  1. Log on to the ACK console. In the left-side navigation pane, click Clusters.

  2. On the Clusters page, find the cluster that you want to manage and click its name. In the left-side navigation pane, choose Applications > Knative.

  3. On the Services tab of the Knative page, set Namespace to default and click Create from Template. Select Custom from the Sample Template drop-down list, copy the following content to the template, and click Create to create a Knative Service named helloworld-go.

    apiVersion: serving.knative.dev/v1
    kind: Service
    metadata:
      name: helloworld-go
    spec:
      template:
        metadata:
          labels:
            alibabacloud.com/eci: "true" 
          annotations:
            k8s.aliyun.com/eci-use-specs : "ecs.c6.large"           # Specify an ECS instance type. 
            k8s.aliyun.com/eci-spot-strategy: "SpotAsPriceGo"       # Use the SpotAsPriceGo policy. 
            k8s.aliyun.com/eci-spot-duration: "1"                   # Set the protection period. Default value: 1. Unit: hours. You can specify a protection period on demand. 
            k8s.aliyun.com/eci-spot-fallback: "true"                # Create a pay-as-you-go instance when preemptible instances are out of stock. 
        spec:         
          containers:
          - env:
            - name: TARGET
              value: "Knative"
            image: registry-vpc.{REGION-ID}.aliyuncs.com/knative-samples/helloworld-go:160e4dc8 # {REGION-ID} indicates the region of the cluster, such as cn-hangzhou.

    The preceding YAML file creates a preemptible instance of the ecs.c6 type.

    • When instances of the specified type are out of stock, no preemptible instance is created.

    • After a preemptible instance is created, you are guaranteed to use the instance for 1 hour. The instance is not released within the protection period even if the market price exceeds the bid price. After the protection period ends, if the market price exceeds the bid price or instances of the specified type are out of stock, Alibaba Cloud may release the preemptible instance. To avoid this issue, set the k8s.aliyun.com/eci-spot-fallback: "true" annotation to allow the system to create pay-as-you-go instances to ensure business continuity when preemptible instances are insufficient or released.

(Optional) Step 2: Configure graceful shutdown for preemptible instances

Limits

To enable notifications on preemptible instance interruptions by using pod conditions and evict preemptible instances by using the Eviction API, you must update ack-virtual-node to 2.11.0 or later. For more information about ack-virtual-node, see ack-virtual-node.

Notifications on preemptible instance interruptions

Alibaba Cloud notifies you 3 minutes before a preemptible instance interruption by sending a SpotToBeReleased event and changing the value of ContainerInstanceExpired in the pod condition to true.

The following figure is an example of the notification.图片 1.png

Configure graceful shutdown for expired preemptible instances

To avoid business interruptions caused by the reclaiming of preemptible elastic container instances, virtual nodes allow you to configure graceful shutdown for preemptible elastic container instances. To do this, add the k8s.aliyun.com/eci-spot-release-strategy: api-evict annotation to pods that are scheduled to preemptible instances. When a virtual node receives a SpotToBeReleased event, it calls the Eviction API to evict the preemptible instance. The eviction complies with the PodDisruptionBudgets and terminationGracePeriodSeconds settings. Creating an Eviction object by calling the API is similar to the DELETE action in pod control policies.

  1. Make an API call: When a virtual node receives a SpotToBeReleased event, the node calls the Eviction API.

  2. Check the PDB: The API server checks the PDB configured for the pod to be evicted.

  3. Perform eviction: After the API server validates the eviction request, the pod is evicted through the following steps:

    1. The pod updates its deletion timestamp so that the API server can recognize the pod to be terminated. The pod is also stamped with the grace period that you configured.

    2. The kubelet of the virtual node on which the pod runs recognizes the pod and initiates graceful shutdown for the pod.

    3. When the kubelet terminates the pod, the control plane of the cluster disassociates the pod from relevant Endpoints and Endpointslices. In this case, the pod controller no longer recognizes the pod as valid.

    4. When the grace period ends, the kubelet forcefully terminates the pod.

    5. The kubelet notifies the system to delete the pod.

    6. The API server deletes the pod.

  4. In Knative scenarios, each Service pod contains a queue-proxy sidecar container. To ensure business continuity and data integrity, when the system deletes a pod, it waits until the queue-proxy sidecar container finishes processing all ongoing requests before it can terminate all application containers. This enables seamless pod scaling and updates.

Release of preemptible instances

After a preemptible instance is created, it can run normally during the protection period. After the protection period expires, the preemptible instance is released if the market price is higher than your bid price or the inventory resources are insufficient. You can understand the release status of preemptible instances by performing the following operations:

  • SpotToBeReleased event

    A SpotToBeReleased event is generated about 5 minutes before a preemptible instance is released.

    Important

    Elastic Container Instance sends the event to the list of Kubernetes events. During the five-minute period, you can take measures to ensure that your business is not affected by the release of the preemptible instance.

    • Run the kubectl describe command to view the details of the preemptible instance. The SpotToBeReleased event is displayed in the Events section of the command output. Example:

      Events:
        Type     Reason            Age    From          Message
        ----     ------            ----   ----          -------
        Warning  SpotToBeReleased  3m32s  kubelet, eci  Spot ECI will be released in 3 minutes
    • Run the kukubectl get events command to view the event information. The SpotToBeReleased event is displayed in the command output. Example:

      LAST SEEN   TYPE      REASON             OBJECT         MESSAGE
      3m39s       Warning   SpotToBeReleased   pod/pi-frmr8   Spot ECI will be released in 3 minutes
  • Status of the preemptible instance after it is released

    After a preemptible instance is released, the instance information is still retained. The status of the instance is changed to Failed and the cause of the failure is BidFailed.

    • Run the kubectl get pod command to view the instance information. The instance status that is displayed in the command output changes to BidFailed. Example:

      NAME       READY   STATUS      RESTARTS   AGE
      pi-frmr8   1/1     BidFailed   0          3h5m
    • Run the kubectl describe command to view the details of the preemptible instance. Then, view the details about the instance status in the command output. Example:

      Status:             Failed
      Reason:             BidFailed
      Message:            The pod is spot instance, and have been released at 2020-04-08T12:36Z

Reference

To avoid the cold start latency, we recommend that you use the reserved instance feature to reserve a low-specification burstable instance. This helps you balance the costs and efficiency. For more information, see Configure a reserved instance.