By Hang Yin and Yuanyi
Knative is a Kubernetes-based serverless framework that provides a cloud-native and cross-platform orchestration standard for serverless applications. Knative implements this serverless standard by integrating functions, workload management, and event driving, with the advantages of standardization, low threshold, automated management, and Knative driving.
Istio extends Kubernetes to build programmable and application-aware service mesh. Istio, combined with Knative, can bring standard and common traffic management, observability, and security capabilities to serverless application workloads.
Knative decouples the service layer from the network layer through the KIngress resource. When a Knative Service is created, the Knative Serving Controller generates the corresponding KIngress resource and converts the KIngress resource into a VirtualService by using the net-istio controller. The KIngress resource is a CRD created by Knative for the network layer, which contains all the information required for the service to be exposed to external access. The following example shows the KIngress resource:
apiVersion: networking.internal.knative.dev/v1alpha1
kind: Ingress
metadata:
annotations:
networking.internal.knative.dev/rollout: '{"configurations":[{"configurationName":"httpbin","percent":100,"revisions":[{"revisionName":"httpbin-00001","percent":100}],"stepParams":{}}]}'
networking.knative.dev/ingress.class: istio.ingress.networking.knative.dev
serving.knative.dev/creator: 1281429699509011-1724722123
serving.knative.dev/lastModifier: 1281429699509011-1724722123
labels:
serving.knative.dev/route: httpbin
serving.knative.dev/routeNamespace: default
serving.knative.dev/service: httpbin
name: httpbin
namespace: default
spec:
httpOption: Enabled
rules:
- hosts:
- httpbin.default
- httpbin.default.svc
- httpbin.default.svc.cluster.local
http:
paths:
- appendHeaders:
Knative-Serving-Default-Route: "true"
splits:
- appendHeaders:
Knative-Serving-Namespace: default
Knative-Serving-Revision: httpbin-00001
percent: 100
serviceName: httpbin-00001
serviceNamespace: default
servicePort: 80
visibility: ClusterLocal
- hosts:
- httpbin.default.example.com
http:
paths:
- appendHeaders:
Knative-Serving-Default-Route: "true"
splits:
- appendHeaders:
Knative-Serving-Namespace: default
Knative-Serving-Revision: httpbin-00001
percent: 100
serviceName: httpbin-00001
serviceNamespace: default
servicePort: 80
visibility: ExternalIPContainer Service for Kubernetes (ACK) Knative is fully compatible with Knative and provides standard Kubernetes API operations. It further enhances its capability as a service and provides more abundant solutions.
Capability as a Service: allows you to deploy applications with one click. You do not need to purchase resources to build a system. A console is provided and visualized operations are supported to simplify the use of Kubernetes clusters and Knative.
Simplified O&M:
• Key component hosting: In ACK clusters, key components, Knative Serving and Knative Eventing, are created and hosted by ACK. This ensures high availability and no fees are charged for resource usage.
• Gateway hosting: ACK Knative provides four types of gateways: Application Load Balancer (ALB), Microservices Engine (MSE), Service Mesh (ASM), and Kourier. The controllers of the cloud service gateways are created by ACK to provide fully hosted and O&M-free gateway services, except for Kourier. Kourier is compatible with the open-source version.
Ecosystem Integration: seamlessly integrates Alibaba Cloud computing services (ECI and ECS), observability (SLS and Managed Service for Prometheus), CI/CD (Apsara Devops), application integration (EventBridge), and message products (MNS). You can use Knative Services to implement capabilities such as logging, monitoring and alerts, continuous delivery, and Knative driving, without the need to purchase servers or build Services.
More Features: Based on open-source Knative, ACK Knative provides out-of-the-box and more extensive solutions based on actual business scenarios. The following section describes the solutions.
• Reserved instances: By default, open-source Knative scales the number of pods to zero during off-peak hours to reduce costs. However, the next time you start the application, the application will experience a time-consuming cold start. To reduce the cold start time, we recommend that you use the reserved instance feature to reserve a low-specification burstable instance. This helps you balance the costs and startup duration.
• Knative auto-scaling: In addition to the out-of-the-box features provided by HPA and KPA (Knative Pod Autoscaler), you can enable AHPA (Advanced Horizontal Pod Autoscaler) for a Knative Service. If the resource demand of your application periodically fluctuates, we recommend that you use AHPA to predict the resource demand. This way, you can preload resources that are required by Knative to reduce the cold start time.
ASM is an enterprise-class managed service mesh platform that is fully compatible with the open-source service mesh standards of Istio. ASM provides the following advantages in terms of capability as a service at the network layer:
• provides various routing capabilities and advanced traffic management capabilities such as throttling and circuit breaking, and traffic swimlanes. You can configure these capabilities in the console.
• supports two-way TLS security and authentication capabilities, such as JWT authentication, OPA policies, and OIDC SSO, and supports connecting to a WAF instance for security protection.
• provides a wide range of out-of-the-box observability capabilities, including logging, monitoring metrics dashboards, the topology of service calls, and tracing analysis.
• provides a plug-in marketplace and the ability to customize the plug-in.
• fully supports the Knative net-istio controller and builds based on the open-source ecosystem.
• supports the rich capabilities of service mesh. Compared with simple gateway implementation, service mesh, as a cloud-native application network infrastructure, can non-intrusively add various network layer capabilities to Knative, including rich high-availability capabilities.
After Knative is installed, you can use ASM in Knative.
By default, when Knative is integrated with ASM, it only enables gateway capabilities, but does not inject the mesh proxy into data plane components. In this case, ASM can implement capabilities that are aligned with other gateway products, such as throttling, security protection, and observability. By injecting the mesh proxy into data plane traffic components, you can further take advantage of service mesh capabilities, including circuit breaking, intra-zone provider first, and priority-based request scheduling.
Pods of activator and Knative Service are important components in Knative that carry traffic. By injecting mesh proxies into them, the service mesh can take over the traffic of Knative Service in a non-intrusive manner.
Enable automatic sidecar injection for the knative-serving namespace and the default namespace. For more information, please refer to Manage global namespaces.
In the ACK cluster, redeploy the activator deployment in the knative-serving namespace. The newly enabled activator pod will be injected into the mesh proxy.
Enable the reverse dns plug-in (the plug-in that supports Spring Cloud service) in ASM. It mainly helps the service mesh to perform secondary resolution on the requests sent by the activator to achieve rich traffic management capabilities. For more information about how to use the ASM plug-in market, see Enable plug-ins to extend ASM capabilities.
This section uses a helloworld-go as an example to demonstrate how to deploy a Knative Service.
1. Log on to the ACK console. In the left-side navigation pane, click Clusters.
2. On the Clusters page, click the name of the cluster that you want to manage and choose Applications > Knative in the left-side navigation pane.
On the Services tab of the Knative page, select default from the Namespace drop-down list, click Create from Template, copy the following YAML content to the code editor, and then click Create to create a Service named helloworld-go.
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
name: helloworld-go
spec:
template:
spec:
containers:
- image: registry.{REGION-ID}.aliyuncs.com/knative-sample/helloworld-go:73fbdd56 # Replace {REGION-ID} with the region where your cluster resides.
env:
- name: TARGET
value: "Knative"3. On the Services page, obtain the gateway address of the helloworld-go service in the Gateway column.
4. Run the following command to access the helloworld-go service:
curl -H "host: helloworld-go.default.example.com" http://39.XX.XX.XX # Replace the IP address and domain name with the actual values. Expected output:
Hello Knative!The output indicates that the service access is successful.
Service throttling is a commonly used method to build high availability for services. You can configure throttling to implement precise control over traffic to cope with issues such as traffic bursts, service overload, resource exhaustion, and malicious attacks. This protects the stability of backend services, reduces costs, and improves user experience. The throttling capability does not include the participation of the mesh proxy. It only needs to be configured on the gateway to limit the rate of external traffic entering the overall system, ensuring the overall stability of the system.
ASM allows you to use ASMLocalRateLimiter to declaratively configure throttling protection measures for any Knative Service. Taking the helloworld-go service deployed in the prerequisites as an example, you can create the following throttling rules:
apiVersion: istio.alibabacloud.com/v1
kind: ASMLocalRateLimiter
metadata:
name: helloworld
namespace: istio-system
spec:
configs:
- limit:
fill_interval:
seconds: 2
quota: 60
match:
vhost:
name: helloworld-go.default.svc.cluster.local
port: 80
isGateway: true
workloadSelector:
labels:
istio: ingressgatewayThe preceding configuration represents configuring throttling for the helloworld-go service to allow two requests to pass every 60 seconds. In addition to the basic throttling configuration in the example, ASMLocalRateLimiter also supports advanced features such as fine-grained request-matching throttling and custom response bodies. For more information, please refer to Description of ASMLocalRateLimiter CRD.
You can use kubeconfig to connect to an ACK cluster or ASM instance to deploy the preceding throttling rule:
kubectl apply -f ratelimit.yamlVerify the throttling rules through three consecutive requests:
curl -H "host: helloworld-go.default.example.com" http://{IP address of the ingress gateway}
Hello Knative-v2!
curl -H "host: helloworld-go.default.example.com" http://{IP address of the ingress gateway}
Hello Knative-v2!
curl -H "host: helloworld-go.default.example.com" http://{IP address of the ingress gateway}
local_rate_limitedYou can view consecutive requests within one minute. The third request triggers a 429 throttling response to protect internal services.
You can configure adaptive request error detection and slow request detection for a revision of a Knative service. When a service continuously returns a 5xx error or responds slowly to requests, the service-based circuit breaking automatically cuts off access to the service and returns an error code in advance to prevent further requests from causing cascading errors in the entire system.
In the example, we deploy an httpbin service that is more suitable for testing than the helloworld-go service. This service can dynamically adjust the response code and response time based on the request content.
Deploy the following KService in Knative:
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
name: httpbin
namespace: default
spec:
template:
spec:
containers:
- image: 'registry.cn-hangzhou.aliyuncs.com/acs/httpbin:latest'
imagePullPolicy: IfNotPresent
name: httpbin
ports:
- containerPort: 80
protocol: TCPService-based circuit breaking can also be declaratively configured in the service mesh. Based on the deployed httpbin service, you can configure the following circuit-breaking rule:
apiVersion: istio.alibabacloud.com/v1
kind: ASMCircuitBreaker
metadata:
name: httpbin-delay
namespace: knative-serving
spec:
configs:
- breaker_config:
break_duration: 60s
custom_response:
body: delay break!
header_to_add:
x-envoy-overload: 'true'
status_code: 498
max_slow_requests: 5
min_request_amount: 2
slow_request_rt: 0.5s
window_size: 10s
match:
vhost:
name: httpbin-00001-private.default.svc.cluster.local
port: 8012
workloadSelector:
labels:
app: activatorThe preceding rule adds the service-based circuit-breaking capability to the activator component, indicating a slow request circuit-breaking policy for the httpbin-00001 revision. Slow requests are defined as requests with a response time exceeding 0.5s. When five or more slow requests are detected within the time window of 10s, subsequent requests will be disconnected for 60s. During this period, all requests sent to the revision will directly return a 498 response code with the response delay break!.
You can use kubeconfig to connect to an ACK cluster or ASM instance to deploy the preceding throttling rule:
kubectl apply -f asmcircuitbreak.yamlVerify whether the circuit breaking is enabled by consecutively requesting the /delay/1 path of the service several times:
curl -H "host: httpbin.default.example.com" http://116.62.100.25/delay/1
{
"args": {},
"data": "",
"files": {},
"form": {},
"headers": {
"Accept": "*/*",
"Forwarded": "for=59.82.59.83;proto=http, for=192.168.0.18",
"Host": "httpbin.default.example.com",
"K-Proxy-Request": "activator",
"Knative-Serving-Default-Route": "true",
"User-Agent": "curl/8.7.1",
"X-Client-Address": "59.82.59.83:58368",
"X-Envoy-Attempt-Count": "1",
"X-Envoy-External-Address": "59.82.59.83",
"X-Envoy-Peer-Metadata": "Ch0KDkFQUF9DT05UQUlORVJTEgsaCWFjdGl2YXRvcgoxCgpDTFVTVEVSX0lEEiMaIWM1NjdhMTQ0MGU4YTk0NDQwODI2MDlkOWY1ZTVlMmI2NAoeCgxJTlNUQU5DRV9JUFMSDhoMMTkyLjE2OC4wLjEwChcKDUlTVElPX1ZFUlNJT04SBhoEMS4yMgrEAgoGTEFCRUxTErkCKrYCChIKA2FwcBILGglhY3RpdmF0b3IKKgobYXBwLmt1YmVybmV0ZXMuaW8vY29tcG9uZW50EgsaCWFjdGl2YXRvcgorChZhcHAua3ViZXJuZXRlcy5pby9uYW1lEhEaD2tuYXRpdmUtc2VydmluZwolChlhcHAua3ViZXJuZXRlcy5pby92ZXJzaW9uEggaBjEuMTIuNAoTCgRyb2xlEgsaCWFjdGl2YXRvcgokChlzZWN1cml0eS5pc3Rpby5pby90bHNNb2RlEgcaBWlzdGlvCjQKH3NlcnZpY2UuaXN0aW8uaW8vY2Fub25pY2FsLW5hbWUSERoPa25hdGl2ZS1zZXJ2aW5nCi8KI3NlcnZpY2UuaXN0aW8uaW8vY2Fub25pY2FsLXJldmlzaW9uEggaBjEuMTIuNAouCgdNRVNIX0lEEiMaIWM1NjdhMTQ0MGU4YTk0NDQwODI2MDlkOWY1ZTVlMmI2NAokCgROQU1FEhwaGmFjdGl2YXRvci01NWRiNmQ4NzdiLXZyeGJtCh4KCU5BTUVTUEFDRRIRGg9rbmF0aXZlLXNlcnZpbmcKVQoFT1dORVISTBpKa3ViZXJuZXRlczovL2FwaXMvYXBwcy92MS9uYW1lc3BhY2VzL2tuYXRpdmUtc2VydmluZy9kZXBsb3ltZW50cy9hY3RpdmF0b3IKHAoNV09SS0xPQURfTkFNRRILGglhY3RpdmF0b3I=",
"X-Envoy-Peer-Metadata-Id": "sidecar~192.168.0.10~activator-55db6d877b-vrxbm.knative-serving~knative-serving.svc.cluster.local"
},
"origin": "59.82.59.83, 192.168.0.18, 192.168.0.10",
"url": "http://httpbin.default.example.com/delay/1"
}
curl -H "host: httpbin.default.example.com" http://116.62.100.25/delay/1
{
"args": {},
"data": "",
"files": {},
"form": {},
"headers": {
"Accept": "*/*",
"Forwarded": "for=59.82.59.83;proto=http, for=192.168.0.18",
"Host": "httpbin.default.example.com",
"K-Proxy-Request": "activator",
"Knative-Serving-Default-Route": "true",
"User-Agent": "curl/8.7.1",
"X-Client-Address": "59.82.59.83:58369",
"X-Envoy-Attempt-Count": "1",
"X-Envoy-External-Address": "59.82.59.83",
"X-Envoy-Peer-Metadata": "Ch0KDkFQUF9DT05UQUlORVJTEgsaCWFjdGl2YXRvcgoxCgpDTFVTVEVSX0lEEiMaIWM1NjdhMTQ0MGU4YTk0NDQwODI2MDlkOWY1ZTVlMmI2NAoeCgxJTlNUQU5DRV9JUFMSDhoMMTkyLjE2OC4wLjEwChcKDUlTVElPX1ZFUlNJT04SBhoEMS4yMgrEAgoGTEFCRUxTErkCKrYCChIKA2FwcBILGglhY3RpdmF0b3IKKgobYXBwLmt1YmVybmV0ZXMuaW8vY29tcG9uZW50EgsaCWFjdGl2YXRvcgorChZhcHAua3ViZXJuZXRlcy5pby9uYW1lEhEaD2tuYXRpdmUtc2VydmluZwolChlhcHAua3ViZXJuZXRlcy5pby92ZXJzaW9uEggaBjEuMTIuNAoTCgRyb2xlEgsaCWFjdGl2YXRvcgokChlzZWN1cml0eS5pc3Rpby5pby90bHNNb2RlEgcaBWlzdGlvCjQKH3NlcnZpY2UuaXN0aW8uaW8vY2Fub25pY2FsLW5hbWUSERoPa25hdGl2ZS1zZXJ2aW5nCi8KI3NlcnZpY2UuaXN0aW8uaW8vY2Fub25pY2FsLXJldmlzaW9uEggaBjEuMTIuNAouCgdNRVNIX0lEEiMaIWM1NjdhMTQ0MGU4YTk0NDQwODI2MDlkOWY1ZTVlMmI2NAokCgROQU1FEhwaGmFjdGl2YXRvci01NWRiNmQ4NzdiLXZyeGJtCh4KCU5BTUVTUEFDRRIRGg9rbmF0aXZlLXNlcnZpbmcKVQoFT1dORVISTBpKa3ViZXJuZXRlczovL2FwaXMvYXBwcy92MS9uYW1lc3BhY2VzL2tuYXRpdmUtc2VydmluZy9kZXBsb3ltZW50cy9hY3RpdmF0b3IKHAoNV09SS0xPQURfTkFNRRILGglhY3RpdmF0b3I=",
"X-Envoy-Peer-Metadata-Id": "sidecar~192.168.0.10~activator-55db6d877b-vrxbm.knative-serving~knative-serving.svc.cluster.local"
},
"origin": "59.82.59.83, 192.168.0.18, 192.168.0.10",
"url": "http://httpbin.default.example.com/delay/1"
}
curl -H "host: httpbin.default.example.com" http://116.62.100.25/delay/1
{
"args": {},
"data": "",
"files": {},
"form": {},
"headers": {
"Accept": "*/*",
"Forwarded": "for=59.82.59.83;proto=http, for=192.168.0.18",
"Host": "httpbin.default.example.com",
"K-Proxy-Request": "activator",
"Knative-Serving-Default-Route": "true",
"User-Agent": "curl/8.7.1",
"X-Client-Address": "59.82.59.83:58370",
"X-Envoy-Attempt-Count": "1",
"X-Envoy-External-Address": "59.82.59.83",
"X-Envoy-Peer-Metadata": "Ch0KDkFQUF9DT05UQUlORVJTEgsaCWFjdGl2YXRvcgoxCgpDTFVTVEVSX0lEEiMaIWM1NjdhMTQ0MGU4YTk0NDQwODI2MDlkOWY1ZTVlMmI2NAoeCgxJTlNUQU5DRV9JUFMSDhoMMTkyLjE2OC4wLjEwChcKDUlTVElPX1ZFUlNJT04SBhoEMS4yMgrEAgoGTEFCRUxTErkCKrYCChIKA2FwcBILGglhY3RpdmF0b3IKKgobYXBwLmt1YmVybmV0ZXMuaW8vY29tcG9uZW50EgsaCWFjdGl2YXRvcgorChZhcHAua3ViZXJuZXRlcy5pby9uYW1lEhEaD2tuYXRpdmUtc2VydmluZwolChlhcHAua3ViZXJuZXRlcy5pby92ZXJzaW9uEggaBjEuMTIuNAoTCgRyb2xlEgsaCWFjdGl2YXRvcgokChlzZWN1cml0eS5pc3Rpby5pby90bHNNb2RlEgcaBWlzdGlvCjQKH3NlcnZpY2UuaXN0aW8uaW8vY2Fub25pY2FsLW5hbWUSERoPa25hdGl2ZS1zZXJ2aW5nCi8KI3NlcnZpY2UuaXN0aW8uaW8vY2Fub25pY2FsLXJldmlzaW9uEggaBjEuMTIuNAouCgdNRVNIX0lEEiMaIWM1NjdhMTQ0MGU4YTk0NDQwODI2MDlkOWY1ZTVlMmI2NAokCgROQU1FEhwaGmFjdGl2YXRvci01NWRiNmQ4NzdiLXZyeGJtCh4KCU5BTUVTUEFDRRIRGg9rbmF0aXZlLXNlcnZpbmcKVQoFT1dORVISTBpKa3ViZXJuZXRlczovL2FwaXMvYXBwcy92MS9uYW1lc3BhY2VzL2tuYXRpdmUtc2VydmluZy9kZXBsb3ltZW50cy9hY3RpdmF0b3IKHAoNV09SS0xPQURfTkFNRRILGglhY3RpdmF0b3I=",
"X-Envoy-Peer-Metadata-Id": "sidecar~192.168.0.10~activator-55db6d877b-vrxbm.knative-serving~knative-serving.svc.cluster.local"
},
"origin": "59.82.59.83, 192.168.0.18, 192.168.0.10",
"url": "http://httpbin.default.example.com/delay/1"
}
curl -H "host: httpbin.default.example.com" http://116.62.100.25/delay/1
{
"args": {},
"data": "",
"files": {},
"form": {},
"headers": {
"Accept": "*/*",
"Forwarded": "for=59.82.59.83;proto=http, for=192.168.0.18",
"Host": "httpbin.default.example.com",
"K-Proxy-Request": "activator",
"Knative-Serving-Default-Route": "true",
"User-Agent": "curl/8.7.1",
"X-Client-Address": "59.82.59.83:58373",
"X-Envoy-Attempt-Count": "1",
"X-Envoy-External-Address": "59.82.59.83",
"X-Envoy-Peer-Metadata": "Ch0KDkFQUF9DT05UQUlORVJTEgsaCWFjdGl2YXRvcgoxCgpDTFVTVEVSX0lEEiMaIWM1NjdhMTQ0MGU4YTk0NDQwODI2MDlkOWY1ZTVlMmI2NAoeCgxJTlNUQU5DRV9JUFMSDhoMMTkyLjE2OC4wLjEwChcKDUlTVElPX1ZFUlNJT04SBhoEMS4yMgrEAgoGTEFCRUxTErkCKrYCChIKA2FwcBILGglhY3RpdmF0b3IKKgobYXBwLmt1YmVybmV0ZXMuaW8vY29tcG9uZW50EgsaCWFjdGl2YXRvcgorChZhcHAua3ViZXJuZXRlcy5pby9uYW1lEhEaD2tuYXRpdmUtc2VydmluZwolChlhcHAua3ViZXJuZXRlcy5pby92ZXJzaW9uEggaBjEuMTIuNAoTCgRyb2xlEgsaCWFjdGl2YXRvcgokChlzZWN1cml0eS5pc3Rpby5pby90bHNNb2RlEgcaBWlzdGlvCjQKH3NlcnZpY2UuaXN0aW8uaW8vY2Fub25pY2FsLW5hbWUSERoPa25hdGl2ZS1zZXJ2aW5nCi8KI3NlcnZpY2UuaXN0aW8uaW8vY2Fub25pY2FsLXJldmlzaW9uEggaBjEuMTIuNAouCgdNRVNIX0lEEiMaIWM1NjdhMTQ0MGU4YTk0NDQwODI2MDlkOWY1ZTVlMmI2NAokCgROQU1FEhwaGmFjdGl2YXRvci01NWRiNmQ4NzdiLXZyeGJtCh4KCU5BTUVTUEFDRRIRGg9rbmF0aXZlLXNlcnZpbmcKVQoFT1dORVISTBpKa3ViZXJuZXRlczovL2FwaXMvYXBwcy92MS9uYW1lc3BhY2VzL2tuYXRpdmUtc2VydmluZy9kZXBsb3ltZW50cy9hY3RpdmF0b3IKHAoNV09SS0xPQURfTkFNRRILGglhY3RpdmF0b3I=",
"X-Envoy-Peer-Metadata-Id": "sidecar~192.168.0.10~activator-55db6d877b-vrxbm.knative-serving~knative-serving.svc.cluster.local"
},
"origin": "59.82.59.83, 192.168.0.18, 192.168.0.10",
"url": "http://httpbin.default.example.com/delay/1"
}
curl -H "host: httpbin.default.example.com" http://116.62.100.25/delay/1
{
"args": {},
"data": "",
"files": {},
"form": {},
"headers": {
"Accept": "*/*",
"Forwarded": "for=59.82.59.83;proto=http, for=192.168.0.18",
"Host": "httpbin.default.example.com",
"K-Proxy-Request": "activator",
"Knative-Serving-Default-Route": "true",
"User-Agent": "curl/8.7.1",
"X-Client-Address": "59.82.59.83:58377",
"X-Envoy-Attempt-Count": "1",
"X-Envoy-External-Address": "59.82.59.83",
"X-Envoy-Peer-Metadata": "Ch0KDkFQUF9DT05UQUlORVJTEgsaCWFjdGl2YXRvcgoxCgpDTFVTVEVSX0lEEiMaIWM1NjdhMTQ0MGU4YTk0NDQwODI2MDlkOWY1ZTVlMmI2NAoeCgxJTlNUQU5DRV9JUFMSDhoMMTkyLjE2OC4wLjEwChcKDUlTVElPX1ZFUlNJT04SBhoEMS4yMgrEAgoGTEFCRUxTErkCKrYCChIKA2FwcBILGglhY3RpdmF0b3IKKgobYXBwLmt1YmVybmV0ZXMuaW8vY29tcG9uZW50EgsaCWFjdGl2YXRvcgorChZhcHAua3ViZXJuZXRlcy5pby9uYW1lEhEaD2tuYXRpdmUtc2VydmluZwolChlhcHAua3ViZXJuZXRlcy5pby92ZXJzaW9uEggaBjEuMTIuNAoTCgRyb2xlEgsaCWFjdGl2YXRvcgokChlzZWN1cml0eS5pc3Rpby5pby90bHNNb2RlEgcaBWlzdGlvCjQKH3NlcnZpY2UuaXN0aW8uaW8vY2Fub25pY2FsLW5hbWUSERoPa25hdGl2ZS1zZXJ2aW5nCi8KI3NlcnZpY2UuaXN0aW8uaW8vY2Fub25pY2FsLXJldmlzaW9uEggaBjEuMTIuNAouCgdNRVNIX0lEEiMaIWM1NjdhMTQ0MGU4YTk0NDQwODI2MDlkOWY1ZTVlMmI2NAokCgROQU1FEhwaGmFjdGl2YXRvci01NWRiNmQ4NzdiLXZyeGJtCh4KCU5BTUVTUEFDRRIRGg9rbmF0aXZlLXNlcnZpbmcKVQoFT1dORVISTBpKa3ViZXJuZXRlczovL2FwaXMvYXBwcy92MS9uYW1lc3BhY2VzL2tuYXRpdmUtc2VydmluZy9kZXBsb3ltZW50cy9hY3RpdmF0b3IKHAoNV09SS0xPQURfTkFNRRILGglhY3RpdmF0b3I=",
"X-Envoy-Peer-Metadata-Id": "sidecar~192.168.0.10~activator-55db6d877b-vrxbm.knative-serving~knative-serving.svc.cluster.local"
},
"origin": "59.82.59.83, 192.168.0.18, 192.168.0.10",
"url": "http://httpbin.default.example.com/delay/1"
}
curl -H "host: httpbin.default.example.com" http://116.62.100.25/delay/1
delay break!%The /delay/1 path causes the httpbin service to respond to requests with a delay of 1 second, which meets the definition of slow requests. You can see that after five consecutive slow requests, the subsequent requests are disconnected and delay break! is returned.
In addition to service-based circuit breaking, ASM also supports native host-level circuit breaking of the open-source Istio. This circuit breaking-policy is performed on multiple endpoint hosts of a service. The service mesh can modify the routing decision made by the fine-grained activator. For multiple available endpoints of the same Knative Service revision, the service mesh will continuously track the response of each endpoint host to the request. When an endpoint host continuously returns a 5xx status code, the host will be temporarily removed from the available endpoint pool to maintain the overall service availability as much as possible.
To implement host-level circuit breaking, you can declare it directly through the Istio-native DestinationRule CR:
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
name: httpbin
namespace: default
spec:
host: httpbin-00001-private
trafficPolicy:
outlierDetection:
baseEjectionTime: 20s
consecutiveErrors: 3
interval: 5s
maxEjectionPercent: 100The preceding rule means that when an endpoint host of the httpbin-00001 revision has a 5xx error for three consecutive times within 5 seconds, the host is removed from the load balancing pool.
Most ACK clusters have multiple zones and the service mesh also supports disaster recovery for workloads and reduction of service response time based on the multiple zones. This feature is also available for host-level circuit breaking, which can be enabled by configuring host-level circuit breaking for the service: when there are multiple available endpoint hosts for the service, the mesh proxy will first send requests to the host in the same availability zone to reduce the request delay; if all endpoints in the same zone are unavailable due to faults, the mesh proxy can also forward requests to endpoint hosts in other zones to implement disaster recovery capabilities.
Based on the non-intrusive feature of the service mesh, ASM also supports more advanced traffic scheduling suites, including priority-based request scheduling, request ramp, and traffic rate parallelism limitation under various conditions. For more information about the traffic scheduling suite and how to enable the traffic scheduling suite in a cluster, please refer to Use the ASM traffic scheduling suite to control traffic for a distributed system. Priority-based request scheduling can be used when the ASM traffic scheduling suite is enabled.
This example demonstrates the QuotaSchedulingPolicy of the traffic scheduling suite. This policy keeps the traffic rate for the Knative Service within a certain limit and prioritizes requests that exceed the limit. Requests with higher priorities are scheduled to the service more frequently.
For the httpbin Knative service deployed above, we can declare the following policy in the traffic scheduling suite:
apiVersion: istio.alibabacloud.com/v1
kind: QuotaSchedulingPolicy
metadata:
name: quotascheduling
namespace: istio-system
spec:
quota_scheduler:
bucket_capacity: 10
fill_amount: 10
rate_limiter:
interval: 1s
scheduler:
workloads:
- label_matcher:
match_labels:
http.request.header.user_type: guest
parameters:
priority: 50.0
name: guest
- label_matcher:
match_labels:
http.request.header.user_type: subscriber
parameters:
priority: 200.0
name: subscriber
selectors:
- service: httpbin-00001-private.default.svc.cluster.localThis policy means that the rate of requests to the httpbin-00001 revision is always controlled within the range of 10 requests per second. Queue the requests that exceed the request rate limit. Requests with a user_type: guest header have a lower queue priority than requests with a user_type: subscriber header, which have four times the priority of the former.
Use kubeconfig to connect to the ASM instance to deploy this declarative policy:
kubectl apply -f policy.yamlYou can use the stress testing tool fortio to check the effect of this policy. For more information about installation instructions, see Install fortio.
Open two terminals at the same time and run the following two stress test commands respectively (start the two tests at the same time as much as possible):
Test 1:
fortio load -c 10 -qps 10000 -H "user_type:guest" -H "host: httpbin.default.example.com" -t 30s -timeout 60s -a http://${IP address of the ingress gateway} /status/201Test 2:
fortio load -c 10 -qps 10000 -H "user_type:subscriber" -H "host: httpbin.default.example.com" -t 30s -timeout 60s -a http://${IP address of the ingress gateway} /status/202Expected output from test 1:
...
# target 50% 4.83333
# target 75% 5.20763
# target 90% 5.38203
# target 99% 5.48668
# target 99.9% 5.49714
Sockets used: 10 (for perfect keepalive, would be 10)
Uniform: false, Jitter: false
Code 201 : 70 (100.0 %)
Response Header Sizes : count 70 avg 249.94286 +/- 0.2871 min 248 max 250 sum 17496
Response Body/Total Sizes : count 70 avg 249.94286 +/- 0.2871 min 248 max 250 sum 17496
All done 70 calls (plus 10 warmup) 4566.839 ms avg, 2.1 qpsExpected output from test 2:
...
# target 50% 0.253333
# target 75% 1.875
# target 90% 4.26635
# target 99% 4.47301
# target 99.9% 4.49367
Sockets used: 10 (for perfect keepalive, would be 10)
Uniform: false, Jitter: false
Code 202 : 250 (100.0 %)
Response Header Sizes : count 250 avg 250.264 +/- 0.4408 min 250 max 251 sum 62566
Response Body/Total Sizes : count 250 avg 250.264 +/- 0.4408 min 250 max 251 sum 62566
All done 250 calls (plus 10 warmup) 1226.657 ms avg, 8.0 qpsThe preceding outputs show that the average request latency of test 2 is about 1/4 times that of test 1, and the QPS is about four times that of test 1. This is because, in the previously defined policy, the priority of the requests of the subscriber type is four times that of requests of the guest type. A total of 320 requests are processed within 30 seconds in two tests. Excluding the 20 requests used for warm-up, the request rate received by the service is exactly 10 requests per second. This proves that the requests received by the service are always within the given limit.
After injecting the mesh proxy into the Knative component, based on the observability of the service mesh, you can directly obtain the traffic topology of the service traffic forwarded to the backend service through the gateway. Then, you can view the traffic status of the service. For more information about how to use the mesh topology capabilities of ASM, please refer to Enable Mesh Topology to improve observability.
The example in this article shows the service status of the mesh topology when the service-based circuit breaking occurs. The service-based circuit breaking configuration used is the example mentioned above. It can be seen that for the httpbin-00001 revision, some requests are not sent to the destination, but are disconnected (marked with the red line). For all requests accessed from the gateway, some requests fail and are in a warning state that requires attention (marked with the yellow line).
This article describes the benefits and features of combining Alibaba Cloud Knative with Alibaba Cloud Service Mesh (ASM) to implement serverless services. It also provides scenario-based best practices for leveraging service mesh capabilities and building adaptive and high-availability traffic management capabilities for Knative Service. For enterprise-class application services, Knative provides serverless O&M-free, elastic, and fast iteration capabilities. In combination with various high-availability traffic management capabilities provided by the service mesh, you can develop and iterate cloud-native services with a low threshold while maintaining business robustness and stability.
Fleet Management Feature of ACK One: Enterprise-Class Multi-Cluster Management Solution
Batch System: Batch Compute and Cloud-native Serverless Argo Workflows
171 posts | 31 followers
FollowAlibaba Cloud Native - November 3, 2022
Alibaba Cloud Native Community - November 15, 2023
Alibaba Cloud Native Community - April 9, 2024
Xi Ning Wang(王夕宁) - August 7, 2023
Alibaba Container Service - September 14, 2022
Alibaba Cloud Serverless - June 23, 2021
171 posts | 31 followers
Follow
Best Practices
Follow our step-by-step best practices guides to build your own business case.
Learn More
Container Service for Kubernetes
Alibaba Cloud Container Service for Kubernetes is a fully managed cloud container management service that supports native Kubernetes and integrates with other Alibaba Cloud products.
Learn More
Serverless Workflow
Visualization, O&M-free orchestration, and Coordination of Stateful Application Scenarios
Learn More
Serverless Application Engine
Serverless Application Engine (SAE) is the world's first application-oriented serverless PaaS, providing a cost-effective and highly efficient one-stop application hosting solution.
Learn MoreMore Posts by Alibaba Container Service