Community

Blog Events Webinars Tutorials Forum
  1. Blog
  2. Events
  3. Webinars
  4. Tutorials
  5. Forum
Create Account
×
Community Blog Serverless Cost Optimization: Knative Supports Preemptible Instances

Serverless Cost Optimization: Knative Supports Preemptible Instances

This article describes how to use preemptible instances in Knative.

By Yuanyi and Xiangqian

Knative is a cloud-native, cross-platform, and open-source serverless application orchestration framework. Preemptible instances are cost-effective resources in the public cloud. The combination of Knative and preemptible instances can further reduce user resource usage costs. This article describes how to use preemptible instances in Knative.

Background

Preemptible elastic container instances are cost-effective. You can bid for unused Alibaba Cloud resources to create preemptible instances. When the market price exceeds your bid price for a preemptible instance or when the available resources are insufficient, the resources consumed by the instance are reclaimed and the instance is released.

Knative is an open-source serverless application orchestration framework based on Kubernetes. Its goal is to formulate cloud-native and cross-platform serverless application orchestration standards. The main features of Knative include request-based auto scaling, scale-in to 0, multi-version management, traffic-based canary release, function deployment, and event-driven architecture.

1

Combination of Knative and Preemptible Instances

Knative provides serverless workloads:

Knative Service, which can automatically scale out pods based on requests. If you use preemptible instances, you only need to configure the corresponding pod annotations. Virtual nodes automatically apply for the corresponding ECI resource specifications based on the annotations of the pod. Virtual nodes provide preemptible instance replacement capability to help you use preemptible instances in a more automated way.

2

Knative combines the advantages of preemptible instances:

• Serverless scenarios: When a web service is requested for a short period of time, resources are used at any time. Resources are released after they are used, so they will not be occupied for a long time.

• Graceful disconnection: When virtual nodes are doing the automatic replacement, pods must be deleted before the workload controller creates preemptible instances. This requires graceful disconnection of business containers. In Knative, a queue-proxy sidecar container is configured for each pod. When a pod is deleted, the queue-proxy container is triggered to wait for the request to be processed before the service container is deleted.

• Cost-sensitive: Users who use Knative are more concerned about costs. Therefore, support for preemptible instances is attractive.

How to Achieve Automatic Replacement of Preemptible Instances

A preemptible instance is reclaimed when the market price of an ECI is higher than the bid price, or when the available instance type is insufficient. Therefore, the use of preemptible instances not only brings economic benefits, but challenges to application stability. To minimize service interruptions caused by the reclaiming of preemptible ECIs, Container Service virtual nodes send SpotToBeReleased events and pod conditions to notify users before preemptible ECIs are reclaimed. Based on these notifications, you can gracefully exit preemptible instances and rotate pods.

Use Preemptible Instances First

You can set the k8s.aliyun.com/eci-spot-strategy annotation, such as the SpotAsPriceGo strategy, to indicate that the system automatically bids to follow the current market price. After a preemptible instance is created, it can be used for one hour. If the market price is higher than the bid price or if the instance type is insufficient, the preemptible instance is released. Before the protection period is reached, the virtual node sends an expiration notification, and automatically evicts and deletes pods. Then, Knative automatically creates new preemptible instance pods. If the preemptible instance is preempted, Knative continues to provide services through the preemptible instance.

3

Use Standard Instances When There Is No Preemptible Instance

If you want to ensure the stability of the service as much as possible, and avoid service damage caused by the failure of an preemptible instance application, you can set the k8s.aliyun.com/eci-spot-fallback parameter to true to automatically switch the billing method to pay-as-you-go.

4

Preemptible Instance Interruption

The preemptible instance sends a SpotToBeReleased event 3 minutes before the interruption and updates the Pod Conditions ContainerInstanceExpired field to true.

The following table lists the conditions and events fields of the pod.

5

An Elegant Method to Deal with Preemptible Instance Expiration

To minimize service interruptions caused by the reclaiming of preemptible ECIs, virtual nodes provide a configurable feature that realizes the elegant ending of preemptible ECI instances. You can configure the annotation k8s.aliyun.com/eci-spot-release-strategy: api-evict for preemptible pods.

Then when the virtual node receives SpotToBeReleased event, it will call the Eviction API to evict the preemptible instance. API-initiated evictions are subject to your PodDisruptionBudgets and terminationGracePeriodSeconds configuration. Use the API to create an Eviction object, which is similar to performing a policy-controlled DELETE operation on a pod.

1.  Call API request: The virtual node receives the SpotToBeReleased event and calls the Eviction API.

2.  PDB check: The API server verifies the PodDisruptionBudget associated with the target pod.

3.  Eviction execution: If the API server allows eviction, the pod will be deleted as follows.

  1. The pod resource in the API server updates the deletion timestamp, after which the API server considers this pod resource to be terminated. This pod resource also marks the grace period configured.
  2. The kubelet on the node of the locally running pod notices that the pod resource is marked as terminated, and it starts to gracefully stop the local pod.
  3. When the kubelet stops a pod, the control panel removes the pod from the Endpoint and EndpointSlice objects. Therefore, the controller no longer considers this pod to be a useful object.
  4. After a pod's grace period expires, the kubelet forcibly terminates the local pod.
  5. The kubelet tells the API server to delete the pod resource.
  6. The API server deletes pod resources.

4.  Knative Service: In Knative, a queue-proxy sidecar container is configured for each pod. When a pod is deleted, the queue-proxy container is triggered to wait for the request to be processed before the service container is deleted.

Release Description

After a preemptible instance is created, it runs as expected during the protection period. After the protection period expires, the preemptible instance is released if the market price is higher than your bid price or the available resources are insufficient. You can view the following information about the release of preemptible instances.

  • Pre-release event: A SpotToBeReleased event is generated about 5 minutes before a preemptible instance is released. Note: The ECI notifies you that preemptible instances will be released through Kubernetes Events. During this period, you can handle the issue to ensure that your business is not affected by the release of the preemptible instances.

    • Run the kubectl describe command to view the details of the pod. You can view the pre-release event in the Events section. Sample statement:
    • Run the kukubectl get events command to view the event information. You can view the pre-release event in the response. Sample statement:

  • Status of pods after release: After a preemptible instance is released, the instance information is still retained. The status of the preemptible instance changes to Failed. The cause of the failure is BidFailed.

    • Run the kubectl get pod command to view the pod information. In the returned information, you can see that the status of the pod has changed. Sample statement:
    • Run the kubectl describe command to view the details of the pod. You can view the status of the pod in the response. Sample statement:
Events:
  Type     Reason            Age    From          Message
  ----     ------            ----   ----          -------
  Warning  SpotToBeReleased  3m32s  kubelet, eci  Spot ECI will be released in 3 minutes
LAST SEEN   TYPE      REASON             OBJECT         MESSAGE
3m39s       Warning   SpotToBeReleased   pod/pi-frmr8   Spot ECI will be released in 3 minutes
NAME       READY   STATUS      RESTARTS   AGE
pi-frmr8   1/1     BidFailed   0          3h5m
Status:             Failed
Reason:             BidFailed
Message:            The pod is spot instance, and have been released at 2020-04-08T12:36Z

Configuration Mode

Add an annotation to a Knative Service to create a preemptible instance. The following table describes the annotations:

Annotation Example Required or not Description
k8s.aliyun.com/eci-spot-strategy SpotAsPriceGo Yes Specifies the bid policy for the preemptible instance. Valid values:
• SpotWithPriceLimit: The instance is created as a preemptible instance for which you specify the maximum hourly price If you use this bid policy, you must specify the k8s.aliyun.com/eci-spot-price-limit annotation.
• SpotAsPriceGo: The instance is created as a preemptible instance for which the market price at the time of purchase is automatically used as the bid price.
k8s.aliyun.com/eci-spot-price-limit "0.5" No Specifies the maximum hourly price of the preemptible instance. This value can be accurate to up to three decimal places.
This annotation is valid only when k8s.aliyun.com/eci-spot-strategy is set to SpotWithPriceLimit.
k8s.aliyun.com/eci-spot-duration "0" No Specifies the protection period of the preemptible instance. Unit: hour. Default value: 1. A value of 0 indicates no protection period.
k8s.aliyun.com/eci-spot-fallback "true" No Specifies whether to automatically create a pay-as-you-go instance if available resources that meet the requirements for the preemptible instance specification are insufficient. Default value: false.

Note: Add ECI-related annotations when you create an ECI-related pod. If you add or modify annotations when you update an ECI-related pod, the annotations do not take effect.

Example 1: Specify the ECS instance type and use the SpotWithPriceLimit policy

apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: helloworld-go
spec:
  template:
    metadata:
      labels:
        alibabacloud.com/eci: "true"
      annotations:
        k8s.aliyun.com/eci-use-specs : "ecs.c6.large"           #Specify an ECS instance type.
        k8s.aliyun.com/eci-spot-strategy: "SpotWithPriceLimit"  #Use custom price cap strategy
        k8s.aliyun.com/eci-spot-price-limit: "0.25"            #Specify a maximum hourly price for the preemptible instance.
    spec:         
      containers:
      - env:
        - name: TARGET
          value: "Knative"
        image: registry.cn-hangzhou.aliyuncs.com/knative-sample/helloworld-go:73fbdd56

The preceding YAML code creates an ecs.c6 preemptible instance.

• If available resources that meet the requirements for the instance type and maximum hourly price are insufficient, the instance cannot be created.

• After a preemptible instance is created, it can be used for one hour. If the market price is higher than the bid price or if the instance type is insufficient, the preemptible instance is released.

Example 2: Configure automatic conversion to pay-as-you-go when no resource is available

apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: helloworld-go
spec:
  template:
    metadata:
      labels:
        alibabacloud.com/eci: "true" 
      annotations:
        k8s.aliyun.com/eci-use-specs : "ecs.c6.large"           #Specify an ECS instance type.
        k8s.aliyun.com/eci-spot-strategy: "SpotWithPriceLimit"  #Use custom price cap strategy
        k8s.aliyun.com/eci-spot-price-limit: "0.05"            #Specify a maximum hourly price for the preemptible instance.
        k8s.aliyun.com/eci-spot-fallback: "true"                #Automatically create a pay-as-you-go instance if available resources that meet the requirements for the preemptible instance specification are insufficient.
    spec:         
      containers:
      - env:
        - name: TARGET
          value: "Knative"
        image: registry.cn-hangzhou.aliyuncs.com/knative-sample/helloworld-go:73fbdd56

The preceding YAML code creates an ecs.c6 preemptible instance.

• If available resources that meet the requirements for the instance type and maximum hourly price are sufficient, a preemptible instance is created. After a preemptible instance is created, it can be used for one hour. If the market price is higher than the bid price or if the instance type is insufficient, the preemptible instance is released.

• If available resources that meet the requirements for the instance type and maximum hourly price are insufficient, a pay-as-you-go instance is created. After the instance is created, the system does not automatically release the instance. After the instance is created, you can run the kubectl describe pod command to view the events of the corresponding pods to check whether the billing method is changed to pay-as-you-go. If the SpotDegraded event is displayed, the billing method is changed to pay-as-you-go.

Best Practices

Since preemptible instances cannot always guarantee availability, and we aim for services to remain uninterrupted in most scenarios, we can set up the following configurations:

• Allow the system to bid automatically, adjusting to the current market price.

• Designate a protection period for the preemptible instance.

• If a preemptible instance is unavailable, the billing method automatically switches to pay-as-you-go to ensure the instance is successfully created.

The specific Knative Service configurations are as follows:

apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: helloworld-go
spec:
  template:
    metadata:
      labels:
        alibabacloud.com/eci: "true" 
      annotations:
        k8s.aliyun.com/eci-use-specs : "ecs.c6.large"           #Specify an ECS instance type.
        k8s.aliyun.com/eci-spot-strategy: "SpotAsPriceGo"  #Use a custom policy to set a price cap.
        k8s.aliyun.com/eci-spot-duration: "1"
        k8s.aliyun.com/eci-spot-fallback: "true"                #Automatically create a pay-as-you-go instance if available resources that meet the requirements for the preemptible instance specification are insufficient.
    spec:         
      containers:
      - env:
        - name: TARGET
          value: "Knative"
        image: registry.cn-hangzhou.aliyuncs.com/knative-sample/helloworld-go:73fbdd56

Summary

The Knative Service, integrated with Virtual Node, now facilitates the use of preemptible instances through an automatic replacement feature. Feel free to give it a try!

Cloud Computing Container Service Serverless cloud native Elastic Container Instance ECI Virtual Node Knative ACK Preemptible Instance
0 1 0
Share on

Read previous post:

Access Redis by Using Higress Based on Custom Plug-ins

Read next post:

SLS New Query Paradigm: Interactive Exploration of Logs Using SPL

Alibaba Cloud Native Community

488 posts | 48 followers

You may also like

Comments

Alibaba Cloud Native Community

488 posts | 48 followers

Related Products

  • Function Compute

    Alibaba Cloud Function Compute is a fully-managed event-driven compute service. It allows you to focus on writing and uploading code without the need to manage infrastructure such as servers.

    Learn More

  • ACK One

    Provides a control plane to allow users to manage Kubernetes clusters that run based on different infrastructure resources

    Learn More

  • Container Service for Kubernetes

    Alibaba Cloud Container Service for Kubernetes is a fully managed cloud container management service that supports native Kubernetes and integrates with other Alibaba Cloud products.

    Learn More

  • NAT(NAT Gateway)

    A public Internet gateway for flexible usage of network resources and access to VPC.

    Learn More

More Posts by Alibaba Cloud Native Community

See All