Alibaba Cloud Service Mesh:Configure global rate limiting for sidecar inbound traffic

Step 1: Create the ASMGlobalRateLimiter CR

1. Create a file named global-ratelimit-svc.yaml:

   global-ratelimit-svc.yaml

   apiVersion: istio.alibabacloud.com/v1beta1
      kind: ASMGlobalRateLimiter
      metadata:
        name: global-svc-test
        namespace: default
      spec:
        workloadSelector:
          labels:
            app: httpbin
        rateLimitService:
          host: ratelimit.default.svc.cluster.local
          port: 8081
          timeout:
            seconds: 5
        isGateway: false
        configs:
        - name: httpbin
          limit:
            unit: MINUTE
            quota: 1
          match:
            vhost:
              name: '*'
              port: 8000

   The following table explains the key fields. For a full field reference, see ASMGlobalRateLimiter field descriptions.

   Field	Description
   workloadSelector	Selects the target workload. Set app: httpbin to apply rate limiting to the HTTPBin service.
   isGateway	Set to false because the rule targets a sidecar workload, not an ingress gateway.
   rateLimitService	Connection settings for the rate limit service: hostname, gRPC port (8081), and a 5-second timeout.
   limit	Rate limiting threshold. unit: MINUTE and quota: 1 means 1 request per minute on the matched route.
   vhost	Matches the virtual host. name: '*' with port: 8000 applies the rule to all requests on HTTPBin port 8000.

2. Run the following command in the ASM instance to apply the CR:

   kubectl apply -f global-ratelimit-svc.yaml

Step 2: Sync the rate limit config to the data plane

After ASM processes the CR, it generates rate limit service configuration in the status.config.yaml field. You must copy this config into the rate limit service's ConfigMap in the ACK cluster. This manual sync is required because the ASM control plane and ACK data plane run in separate clusters.

1. Retrieve the generated config from the ASM instance:

   kubectl get asmglobalratelimiter global-svc-test -o yaml

   In the output, locate the status section:

   Expected output

   status:
        config.yaml: |
          descriptors:
          - key: generic_key
            rate_limit:
              requests_per_unit: 1
              unit: MINUTE
            value: RateLimit[global-svc-test.default]-Id[3833670472]
          domain: ratelimit.default.svc.cluster.local
        message: ok
        status: successful

2. Create a ratelimit-config.yaml file. Copy the config.yaml content from the status section into the data.config.yaml field of the ConfigMap exactly as shown:

   Important

   Copy the config.yaml value from the status section without modification. Any changes cause the rate limit service to reject the configuration.

   ratelimit-config.yaml

   apiVersion: v1
      kind: ConfigMap
      metadata:
        name: ratelimit-config
      data:
        config.yaml: |
          descriptors:
          - key: header_match
            rate_limit:
              requests_per_unit: 1
              unit: MINUTE
            value: RateLimit[global-svc-test.default]-Id[1492204717]
          domain: ratelimit.default.svc.cluster.local

3. Apply the ConfigMap to the ACK cluster:

   kubectl apply -f ratelimit-config.yaml

Verify

Send two requests to HTTPBin port 8000 from the sleep pod:

kubectl exec -it deploy/sleep -- sh

Then run:

curl httpbin:8000/get -v
curl httpbin:8000/get -v

Expected output for the second request:

< HTTP/1.1 429
< x-envoy-ratelimited: true
< x-ratelimit-limit: 1, 1;w=60
< x-ratelimit-remaining: 0
< x-ratelimit-reset: 5
< date: Thu, 26 Oct 2023 04:23:54 GMT
< server: envoy
< content-length: 0
< x-envoy-upstream-service-time: 2
<
* Connection #0 to host httpbin left intact

The second request returns 429, confirming that global rate limiting is active. Only one request is allowed to access the HTTPBin service within one minute. When you send a second request, throttling is triggered, which indicates that global rate limiting takes effect on inbound traffic of the service into which a sidecar proxy is injected.

Step 1: Create the ASMGlobalRateLimiter CR

Choose the YAML that matches your ASM version:

apiVersion: istio.alibabacloud.com/v1beta1
kind: ASMGlobalRateLimiter
metadata:
  name: global-svc-test
  namespace: default
spec:
  workloadSelector:
    labels:
      app: httpbin
  rateLimitService:
    host: ratelimit.default.svc.cluster.local
    port: 8081
    timeout:
      seconds: 5
  isGateway: false
  configs:
  - name: httpbin
    limit:
      unit: MINUTE
      quota: 1
    match:
      vhost:
        name: '*'
        port: 8000
        route:
          header_match:
          - name: ":path"
            prefix_match: "/headers"

apiVersion: istio.alibabacloud.com/v1beta1
kind: ASMGlobalRateLimiter
metadata:
  name: global-svc-test
  namespace: default
spec:
  workloadSelector:
    labels:
      app: httpbin
  rateLimitService:
    host: ratelimit.default.svc.cluster.local
    port: 8081
    timeout:
      seconds: 5
  isGateway: false
  configs:
  - name: httpbin
    limit:
      unit: SECOND
      quota: 100000
    match:
      vhost:
        name: '*'
        port: 8000
    limit_overrides:
    - request_match:
        header_match:
        - name: ":path"
          prefix_match: "/headers"
      limit:
        unit: MINUTE
        quota: 1

The following table explains the additional fields used in this scenario:

Step 2: Sync the rate limit config to the data plane

Follow the same process as Scenario 1:

1. Retrieve the generated config from the ASM instance:

   kubectl get asmglobalratelimiter global-svc-test -o yaml

   Expected status output (V1.19.0+)

   status:
        config.yaml: |
          descriptors:
          - descriptors:
            - key: header_match
              rate_limit:
                requests_per_unit: 1
                unit: MINUTE
              value: RateLimit[global-svc-test.default]-Id[2613586978]
            key: generic_key
            rate_limit:
              requests_per_unit: 100000
              unit: SECOND
            value: RateLimit[global-svc-test.default]-Id[2613586978]
          domain: ratelimit.default.svc.cluster.local
        message: ok
        status: successful

2. Copy the config.yaml from the status section into a ConfigMap and apply it to the ACK cluster:

   ratelimit-config.yaml

   apiVersion: v1
      kind: ConfigMap
      metadata:
        name: ratelimit-config
      data:
        config.yaml: |
          descriptors:
          - descriptors:
            - key: header_match
              rate_limit:
                requests_per_unit: 1
                unit: MINUTE
              value: RateLimit[global-svc-test.default]-Id[2613586978]
            key: generic_key
            rate_limit:
              requests_per_unit: 100000
              unit: SECOND
            value: RateLimit[global-svc-test.default]-Id[2613586978]
          domain: ratelimit.default.svc.cluster.local

   kubectl apply -f ratelimit-config.yaml

Verify

1. Send two requests to the /headers path from the sleep pod:

   kubectl exec -it deploy/sleep -- sh

   Then run:

   curl httpbin:8000/headers -v
   curl httpbin:8000/headers -v

   Expected output for the second request:

   < HTTP/1.1 429 Too Many Requests
      < x-envoy-ratelimited: true
      < x-ratelimit-limit: 1, 1;w=60
      < x-ratelimit-remaining: 0
      < x-ratelimit-reset: 5
      < date: Thu, 26 Oct 2023 04:23:54 GMT
      < server: envoy
      < content-length: 0
      < x-envoy-upstream-service-time: 2
      <
      * Connection #0 to host httpbin left intact

   The second request to /headers is rate limited. Only one request is allowed to access the /headers path of the HTTPBin service within one minute.

2. Confirm that other paths are unaffected:

   curl httpbin:8000/get -v

   Expected output

   *   Trying 192.168.243.21:8000...
      * Connected to httpbin (192.168.243.21) port 8000 (#0)
      > GET /get HTTP/1.1
      > Host: httpbin:8000
      > User-Agent: curl/8.1.2
      > Accept: */*
      >
      < HTTP/1.1 200 OK
      < server: envoy
      < date: Thu, 11 Jan 2024 06:25:09 GMT
      < content-type: application/json
      < content-length: 431
      < access-control-allow-origin: *
      < access-control-allow-credentials: true
      < x-envoy-upstream-service-time: 7
      <
      {
        "args": {},
        "headers": {
          "Accept": "*/*",
          "Host": "httpbin:8000",
          "User-Agent": "curl/8.1.2",
          "X-Envoy-Attempt-Count": "1",
          "X-Forwarded-Client-Cert": "By=spiffe://cluster.local/ns/default/sa/httpbin;Hash=be10819991ba1a354a89e68b3bed1553c12a4fba8b65fbe0f16299d552680b29;Subject=\"\";URI=spiffe://cluster.local/ns/default/sa/sleep"
        },
        "origin": "127.0.0.6",
        "url": "http://httpbin:8000/get"
      }
      * Connection #0 to host httpbin left intact

   Requests to /get succeed because only the /headers path is subject to rate limiting.

Enable metric reporting

1. Configure proxyStatsMatcher for the sidecar proxy. Select Regular Expression Match and set the value to .*ratelimit.*. For more information, see the "proxyStatsMatcher" section of Configure sidecar proxies.

2. Redeploy the HTTPBin service to pick up the new proxy configuration. For more information, see the "(Optional) Redeploy workloads" section of Configure sidecar proxies.

3. Configure global throttling and perform request tests. For more information, see Scenario 1 or Configure local rate limiting in Traffic Management Center.

4. Run the following command to view the global throttling metrics of the HTTPBin service:

   kubectl exec -it deploy/httpbin -c istio-proxy -- curl localhost:15090/stats/prometheus|grep envoy_cluster_ratelimit

   Example output:

   # TYPE envoy_cluster_ratelimit_ok counter
   envoy_cluster_ratelimit_ok{cluster_name="inbound|80||"} 904
   # TYPE envoy_cluster_ratelimit_over_limit counter
   envoy_cluster_ratelimit_over_limit{cluster_name="inbound|80||"} 3223

Set up Prometheus alerts

Use Managed Service for Prometheus to collect rate limiting metrics and trigger alerts when rate limiting occurs.

1. Connect the ACK cluster to the Alibaba Cloud ASM component in Managed Service for Prometheus, or upgrade the component to the latest version. For more information, see Manage components.

   Note

   If you already use a self-managed Prometheus instance to collect ASM metrics, skip this step. For more information, see Monitor ASM instances by using a self-managed Prometheus instance.

2. Create a custom alert rule with the following PromQL and message template. For detailed instructions, see Create an alert rule with a custom PromQL statement. This rule fires when any service has at least one rate-limited request in a rolling one-minute window, grouped by namespace and service name.

   Parameter	Example value
   PromQL statement	(sum by(namespace, service_istio_io_canonical_name) (increase(envoy_cluster_ratelimit_over_limit[1m]))) > 0
   Alert message	Global rate limiting triggered. Namespace: {{$labels.namespace}}. Service: {{$labels.service_istio_io_canonical_name}}. Throttled requests in the last minute: {{ $value }}