Introduction to the orchestration standard for serverless applications, the Knative features, and how to use ACK Knative - Container Service for Kubernetes

Knative is an open source Kubernetes-based serverless framework. Knative supports pod auto scaling based on resource requests, version management, and canary releases for applications. When no traffic is processed, the number of pods is scaled to zero. Container Service for Kubernetes (ACK) Knative is fully compatible with the open source Knative and Kubernetes APIs. The capabilities of ACK Knative are enhanced in multiple dimensions. For example, Knative retains instances to reduce the cold start time and supports the predictive scaling feature based on the Advanced Horizontal Pod Autoscaler (AHPA) component.

Why use Knative in a Kubernetes cluster?

Introduction to Knative

Knative is an open source Kubernetes-based serverless framework that provides a cloud-native and cross-platform orchestration standard for serverless applications. To implement the standard, Knative integrates container creation, workload management, and event models. The following section describes the advantages of Knative.

Greater focus on business logic: Knative provides simple application configuration and auto scaling. This way, developers can focus on business logic. This reduces the O&M burden and decreases the focus on underlying resources.
Standardization: When you deploy business codes to a serverless platform, the compilation and deployment of source code and the management of events must be considered. Currently, serverless solutions and FPGA as a Service (FaaS) solutions provided by communities and cloud vendors have different standards. Knative provides a standard and general-purpose serverless framework.
For example, if you want to use the event-driven mode in Knative, you can modify the corresponding YAML file and deploy the file in the cluster without the need to bind the file to cloud services. This facilitates cross-platform migration.
Ease of use: Knative can automatically package code into container images and publish the images as Services. Knative can also quickly deploy functions to Kubernetes clusters and run them as containers.
Automated application management: Knative can automatically reduce the number of pods to zero when no traffic is processed. This saves resources. Knative also provides various features, such as version management and canary releases.
Knative Eventing: Knative provides an event model to iinterface with external event systems and route the events to Services or functions for processing.

Key components

Knative consists the following core components, which provide different features:

Knative Serving: provides serverless workload management capabilities. Knative Serving enables serverless deployment, version management, and canary releases for your applications. Knative Serving also supports pod auto scaling based on resource requests. If no traffic is processed, the number of pods is scaled to zero.
Knative Eventing: provides event management capabilities, which allow you to interface with external event sources, register and subscribe to events, and filter events. The event system decouples event producers and event consumers.
Knative Functions: allows you to create, build, and deploy Knative Services in an efficient manner. You can deploy stateless, event-driven functions as Knative Services to a Kubernetes cluster by using Knative Functions without the need to have a deep understanding of the underlying technology stack, such as Kubernetes, containers, and Knative.

Features

You can use Knative in Kubernetes clusters to easily implement the following features:

Request-based auto scaling

Auto scaling based on CPU and memory requests may not reflect the actual resource usage of your business. For web services, auto scaling based on the queries per second (QPS) or requests per second (RPS) can directly reflect the performance of a Service. Knative Serving injects a queue-proxy container into each pod to collect the container concurrency or RPS metrics. After the autoscaler collects metrics as scheduled, the autoscaler automatically adjusts the number of pods of the Deployment based on the corresponding algorithm. This enables auto scaling based on requests.

To meet the same goal in an ACK cluster without using Knative, you must create a Deployment and a Service, configure an Ingress gateway, and then configure Horizontal Pod Autoscaler (HPA) parameters. When a Knative Service is used, you need only to deploy Knative and configure the YAML file of the Knative Service.

Scale the number of pods to zero when no traffic is processed

Knative can automatically reduce the number of pods to zero when no requests are received from applications, and can automatically scale out pods when requests are received. Knative defines two access modes: Proxy (proxy mode) and Serve (direct mode). The autoscaler is responsible for switching the mode. If the number of requests is zero, the autoscaler switches from the Serve mode to the Proxy mode. When a request is received, the autoscaler receives a notification to scale out the number of pods. After the status of the new pods changes to Ready, the autoscaler forwards the request. Then, the autoscaler switches from the Proxy mode to the Serve mode.

Version management and canary releases

When you create a Knative Service, a Configuration object and a Route object are automatically created at the underlying layer.

Configuration: the configuration of the current desired state. The Configuration object is updated each time the Service is updated. After the Configuration object is updated, a unique revision is created. Revisions are a version management mechanism for Configuration objects.
Route: routes requests to a revision and forwards different proportions of traffic to different revisions.

With the help of the preceding features, you can use revisions to manage versions, such as rolling back an application to a previous version. You can also perform canary releases for traffic management. For example, after you create Revision V1, you can update the Configuration object of the Service when the version needs to be updated. Then, you can create Revision V2 and specify different traffic proportions for Revision V1 and Revision V2 by using Routes. The traffic is distributed based on the predefined proportions. For example, you can route 70% traffic to Revision V1 and route 30% traffic to Revision V2.

Knative Eventing

Knative Eventing provides an event model, which can be used to interface with external event systems, such as GitHub and Message Queue (MQ), and route events to Knative Services or functions for processing.

Why use ACK Knative?

ACK Knative is fully compatible with open source Knative and Kubernetes APIs. ACK Knative also enhances Capability as a Service and provides more comprehensive solutions.

Capability as a Service: allows you to deploy applications with a few clicks. You do not need to purchase resources to build a system. A console is provided and visualized operations are supported to simplify the use of Kubernetes clusters and Knative.
Simplified O&M:
- Key component hosting: In ACK clusters, key components, Knative Serving and Knative Eventing, are created and hosted by ACK. This ensures high availability, and no fees are charged for resource usage.
- Gateway hosting: ACK Knative provides four types of gateways: Application Load Balancer (ALB), Microservices Engine (MSE), Service Mesh (ASM), and Kourier. The controllers of the cloud service gateways are created by ACK to provide fully hosted and O&M-free gateway services, except for Kourier. Kourier is compatible with the open source version.
Ecosystem integration: seamlessly integrates Alibaba Cloud computing services (Elastic Container Instance and ECS ), observability (Simple Log Service (SLS) and Managed Service for Prometheus), and application integration (EventBridge). You can use Knative Services to implement capabilities such as logging, monitoring, and alerts, continuous delivery, and Eventing, without the need to purchase servers or build Services.
More features: Based on open source Knative, ACK Knative provides out-of-the-box and more extensive solutions based on actual business scenarios. The following section describes the solutions.
- Reserved instances: By default, open source Knative scales the number of pods to zero during off-peak hours to reduce costs. However, the next time you start the application, the application will experience a time-consuming cold start. To reduce the cold start time, we recommend that you use the reserved instance feature to reserve a low-specification burstable instance. This helps you balance the costs and startup duration.
- Knative auto-scaling: In addition to the out-of-the-box features provided by HPA and Knative Pod Autoscaler (KPA), you can enable AHPA for a Knative Service. If the resource demand of your application periodically fluctuates, we recommend that you use AHPA to predict the resource demand. This way, you can preload resources that are required by Knative to reduce the cold start time.

For more information about the comparison between ACK Knative and open source Knative, see Comparison between Alibaba Cloud Knative and open-source Knative.

Use scenarios

The following table describes the use scenarios of ACK Knative.

Scenario	Description
Hosting of web services	Simplified deployment: ACK Knative encapsulates underlying details of Kubernetes and greatly simplifies the deployment and management of workloads by using Knative Services. Simplified multi-version management: The revision mechanism ensures that each revision has a unique identifier, which facilitates the management of different versions, such as version rollback. Simplified canary releases for traffic management: ACK Knative provides the traffic management feature. You can set different traffic proportions for Services of different revisions to quickly perform canary releases and A/B tests.
Serverless applications	Focus on business logic: Developers can focus on the development of business logic rather than infrastructure as a service (IaaS) resources. Application configurations are also simplified. This reduces the O&M costs on the underlying infrastructure. On-demand resource usage and auto scaling: ACK Knative can automatically scale out resources based on requests and the concurrency, and reduce the number of pods to zero when no business traffic is processed. This saves resources and costs.
AI scenarios	Focus on business logic: In heterogeneous computing scenarios such as GPU acceleration, developers can focus on the development and deployment of AI jobs rather than the maintenance of the underlying infrastructure. On-demand resource usage and auto scaling: ACK Knative can automatically scale out resources based on the actual workloads. The resource usage costs for inference services whose workloads fluctuate are greatly reduced. Portability: ACK Knative can run in Kubernetes-compatible environments. Knative Services can be deployed on on-cloud nodes, in data centers, or even on edge devices.
Knative Eventing scenarios	Knative Eventing provides an event model to simplify the process of ingesting events from external event systems. For example, when an Internet of Things (IoT) device sends sensor data to a Knative Service, ACK Knative can configure a corresponding event source to receive the data and trigger the corresponding processing logic, such as data storage, real-time analysis, monitoring, and alerting.

How to use ACK Knative

The following figure shows how to use ACK Knative.

Step	Description
Prerequisites	An ACK managed cluster is created. For more information, see Create an ACK managed cluster. The Kubernetes version of the cluster must be 1.22 or later. For more information about how to update a cluster, see Manually upgrade ACK clusters.
	Deploy ACK Knative in the console and install the Knative Serving component. For more information, see Deploy Knative.
	Select the Knative gateway that you want to install and deploy the gateway. ACK Knative supports ALB, MSE, ASM, and Kourier gateways. For more information, see Recommendations for selecting an Ingress for Knative. ALB Ingresses: ALB Ingresses are fully-managed O&M-free Ingresses empowered by the ALB service of Alibaba Cloud for traffic management. ALB Ingresses also provide auto scaling capabilities. MSE Ingresses: MSE Ingresses are next-generation Ingresses that comply with Kubernetes Ingress standards. MSE Ingresses support all features of traditional traffic gateways. ASM Ingresses: ASM is an Istio-compatible platform that allows you to centrally manage the traffic of microservices applications. ASM Ingresses provide features such as traffic control, mesh observability, and secure inter-service communication to simplify service governance and help you manage services that run on top of heterogeneous computing infrastructure. Kourier Ingresses: Kourier Ingresses are open-source lightweight Ingresses developed based on the Envoy architecture.
Service deployment and management	Specify the resource type in use: By default, ECS resources are used to run Knative Services. Use pod resources provided by elastic container instances to handle traffic bursts. For more information, see Use elastic container instances in Knative. Use GPU resources in scenarios such as AI inference. For more information, see Use GPUs in Knative. If ECS instances and elastic container instances are colocated in a cluster, you can specify the priorities of instances for scale-out activities by using a ResourcePolicy. For more information, see Colocate ECS instances and elastic container instances in Knative. You can configure a Knative Service to use preemptible instances, which reduces the usage of cloud computing resources. For more information, see Use preemptible instances. Configure a reserved instance to reserve a low-specification burstable instance to balance the cost and startup duration. For more information, see Configure a reserved instance.
	Auto scaling: For more information about how to implement KPA for a Service based on the QPS, see Implement KPA for a service based on the number of traffic requests. Configure AHPA to predict workloads based on historical metrics and perform predictive scaling, and to conduct scheduled scaling by using CRON expressions. For more information, see Use AHPA to implement scheduled auto scaling. For more information about how to configure CPU metrics and thresholds for HPA, see Use HPA in Knative.
	Version management and canary releases: Manage application versions based on revisions. For example, you can roll back an application from a new version to an earlier version. For more information, see Create a revision. For more information about how to perform a canary release for a Knative Service based on a traffic proportion and a revision, see Perform a canary release based on traffic splitting.
	Access to Knative Services: The default domain name of a Knative Service is in the `{route}.{namespace}.{default-example.com}` format. `{default-example.com}` is the default domain name suffix. You can specify a custom domain name suffix. For more information, see Configure a custom domain name. If you want to use a custom domain name to expose a Knative Service, we recommend that you configure a certificate for the domain name to secure data transmission. For more information, see Configure a certificate to access Services over HTTPS. You can configure liveness and readiness probes to monitor and manage the health status and availability of Services. For more information, see Configure port probing in Knative.
Advanced features	Knative Eventing: Knative Eventing meets common requirements in cloud-native development. Knative Eventing also provides an architecture for the serverless event-driven mode. The architecture consists of event sources, event ingesting and subscription, and event filtering. ACK Knative supports various event sources, such as GitHub and EventBridge. For more information, see Overview of Knative Eventing.
	Knative Functions: Knative Functions provides a simple method to create, build, and deploy Knative Services. For more information, see Deploy Knative Functions.
	AI inference services: KServe is Kubernetes-based machine learning model serving framework. KServe provides simple Kubernetes CustomResourceDefinitions (CRDs) that you can use to deploy one or more trained models, such as TFServing, TorchServe, and Triton inference servers, to a model serving runtime. You can deploy KServe and quickly deploy an inference Service based on KServe. ACK also provides best practices for deploying AI inference services in Knative, such as how to accelerate model deployment and how to configure GPU sharing. For more information, see Best practices for deploying AI inference services in Knative.
	ASM: If you want to integrate service meshes into Knative Services to implement complex traffic management and improve service security, we recommend that you use ASM. For more information, see What is ASM?
Observability and cost management	Log collection: You can use SLS in Knative to collect, consume, ship, query, and analyze log data without developing these features. For more information, see Enable Simple Log Service on Knative.
	Knative dashboard: You can connect Knative to Managed Service for Prometheus to view statistics such as the response latency and request concurrency. For more information, see View the Knative monitoring dashboard.
	Monitoring and alerting: You can use SLS to configure alert rules based on the collected log data. For more information, see Configure alerting for Knative Services.
	Cost insights: As a financial administrator in an IT enterprise, you can enable the cost insights feature in Knative to analyze resource usage and cost trends in ACK clusters. For more information, see Enable the cost insights feature in Knative Service.

Billing

When you use ACK Knative in an ACK cluster, ACK Knative is free of charge. However, you are charged for the cloud resources that are created when you use Knative. For example, if you create and use ECS instances, SLB instances, and NAT gateways, you are charged for the resources based on the billing rules of the resources. For more information about the billing rules of ACK clusters, see Billing rules and Cloud service billing.

FAQ

If you encounter problems when you use ACK Knative, you can first refer to Knative FAQ and troubleshoot these problems on your own.

Contact us

If you have any questions or suggestions about Knative, join the DingTalk group 23302777.

References

Update Knative Serving at the earliest opportunity to obtain the latest features and bug fixes. We recommend that you perform updates during off-peak hours. For more information, see Knative release notes and Upgrade Knative Serving.
The official documentation of Knative provides a tutorial that describes how to build, deploy, and monitor an online store application by using Knative. For more information, see Knative Bookstore Tutorial.