使用ConcurrencyLimitingPolicy實現請求並發數控制 - Alibaba Cloud Service Mesh

您可以使用ASM流量調度套件的ConcurrencyLimitingPolicy功能來控制發往服務的請求並發度（即正在處理的請求數量），防止服務過載。該策略會追蹤正在處理的請求數量，並在超過設定閾值時拒絕新請求。本文將詳細介紹如何使用ConcurrencyLimitingPolicy實現請求並發度控制。

前提條件

已添加Kubernetes託管版叢集到ASM執行個體，且ASM執行個體為v1.21.6.97及以上。具體操作，請參見添加叢集到ASM執行個體。
已為Kubernetes叢集中的default命名空間開啟自動注入。具體操作，請參見管理全域命名空間。
已通過kubectl串連至ACK叢集。具體操作，請參見擷取叢集KubeConfig並通過kubectl工具串連叢集。
已開啟ASM流量調度套件。具體操作，請參見開啟ASM流量調度套件。
已經部署httpbin應用，並且可以通過網關和sleep服務訪問。具體操作，請參見部署httpbin應用。

步驟一：建立ConcurrencyLimitingPolicy並發數限制規則

使用kubectl串連到ASM執行個體，具體操作，請參見通過控制面kubectl訪問Istio資源。

使用以下內容，建立ConcurrencyLimitPolicy.yaml檔案。

apiVersion: istio.alibabacloud.com/v1
kind: ConcurrencyLimitingPolicy
metadata:
  name: concurrencylimit
  namespace: istio-system
spec:
  concurrency_limiter:
    max_concurrency: 1
    parameters:
      max_inflight_duration: 60s
    selectors:
    - service: httpbin.default.svc.cluster.local

部分欄位說明如下。關於欄位的更多資訊，請參見ConcurrencyLimitingPolicy CRD說明。

欄位	說明
max_concurrency	最大請求並發數。樣本中指定為1，即只允許服務同時處理1個請求。
max_inflight_duration	請求處理逾時時間。由於叢集中可能發生Pod重啟等突發情況，導致ASM流量調度套件可能無法記錄到請求結束事件，為防止此類請求影響並發數限制演算法的判斷，需要指定請求處理逾時時間，超過此時間還未響應的請求將視作處理結束。您可以通過評估請求的期望最大回應時間來設定此值，樣本中設定為60s。
selectors	指定應用限流策略的多個服務。樣本中使用service: httpbin.default.svc.cluster.local 表示對httpbin.default.svc.cluster.local 服務進行並發數限制。

執行以下命令，開啟ConcurrencyLimitingPolicy並發數限制規則。

kubectl apply -f ConcurrencyLimitingPolicy.yaml

預期輸出：

concurrencylimitingpolicy.istio.alibabacloud.com/concurrencylimit created

步驟二：驗證並發數限制效果

執行以下命令，進入sleep應用開啟sh命令列。
```
kubectl exec -it deploy/sleep -- sh
```

執行以下命令，首先通過後台發送一個需要處理30秒的請求，然後在30秒之內緊接著發送第二個請求。

curl httpbin:8000/delay/30 -I &
curl httpbin:8000 -I

預期輸出：

HTTP/1.1 429 Too Many Requests
date: Fri, 26 Jul 2024 13:50:55 GMT
server: envoy
x-envoy-upstream-service-time: 1
transfer-encoding: chunked

~ $ HTTP/1.1 200 OK
server: envoy
date: Fri, 26 Jul 2024 13:51:05 GMT
content-type: application/json
content-length: 269
access-control-allow-origin: *
access-control-allow-credentials: true
x-envoy-upstream-service-time: 10006

[1]+  Done                       curl httpbin:8000/delay/30 -I

可以看到第二次發送請求收到了429響應碼，證明並發數限制生效。

相關操作

您可以通過Grafana大盤來觀測ConcurrencyLimitingPolicy策略的執行效果。請確保Grafana使用的資料來源Prometheus執行個體已經完成配置採集ASM流量調度套件相關指標。

將以下內容匯入到Grafana，建立ConcurrencyLimitingPolicy策略的大盤。

展開查看JSON內容

{
  "annotations": {
    "list": [
      {
        "builtIn": 1,
        "datasource": {
          "type": "grafana",
          "uid": "-- Grafana --"
        },
        "enable": true,
        "hide": true,
        "iconColor": "rgba(0, 211, 255, 1)",
        "name": "Annotations & Alerts",
        "type": "dashboard"
      }
    ]
  },
  "editable": true,
  "fiscalYearStartMonth": 0,
  "graphTooltip": 0,
  "id": 41,
  "links": [],
  "liveNow": false,
  "panels": [
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${datasource}"
      },
      "description": "Signal derived from periodic execution of query: (sum(rate(concurrency_limiter_counter_total{policy_name=\"concurrencylimit\",component_id=\"root.0\",decision_type=\"DECISION_TYPE_ACCEPTED\"}[30s])) / sum(rate(concurrency_limiter_counter_total{policy_name=\"concurrencylimit\",component_id=\"root.0\"}[30s]))) * 100",
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "palette-classic"
          },
          "custom": {
            "axisCenteredZero": false,
            "axisColorMode": "text",
            "axisLabel": "",
            "axisPlacement": "auto",
            "barAlignment": 0,
            "drawStyle": "line",
            "fillOpacity": 10,
            "gradientMode": "none",
            "hideFrom": {
              "legend": false,
              "tooltip": false,
              "viz": false
            },
            "lineInterpolation": "linear",
            "lineWidth": 1,
            "pointSize": 5,
            "scaleDistribution": {
              "type": "linear"
            },
            "showPoints": "auto",
            "spanNulls": false,
            "stacking": {
              "group": "A",
              "mode": "none"
            },
            "thresholdsStyle": {
              "mode": "off"
            }
          },
          "mappings": [],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "green",
                "value": null
              },
              {
                "color": "red",
                "value": 80
              }
            ]
          },
          "unit": ""
        },
        "overrides": []
      },
      "gridPos": {
        "h": 10,
        "w": 24,
        "x": 0,
        "y": 0
      },
      "id": 1,
      "interval": "10s",
      "options": {
        "legend": {
          "calcs": [],
          "displayMode": "list",
          "placement": "bottom",
          "showLegend": true
        },
        "tooltip": {
          "mode": "single",
          "sort": "none"
        }
      },
      "pluginVersion": "v10.1.0",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${datasource}"
          },
          "editorMode": "code",
          "expr": "(sum by (policy_name) (rate(concurrency_limiter_counter_total{component_id=\"root.0\",decision_type=\"DECISION_TYPE_ACCEPTED\"}[30s])) / sum by (policy_name) (rate(concurrency_limiter_counter_total{component_id=\"root.0\"}[30s]))) * 10",
          "intervalFactor": 1,
          "legendFormat": "accepted percentage",
          "range": true,
          "refId": "A"
        }
      ],
      "title": "Accept Percentage",
      "type": "timeseries"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${datasource}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "thresholds"
          },
          "mappings": [],
          "noValue": "No data",
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "blue",
                "value": null
              }
            ]
          },
          "unit": "short"
        },
        "overrides": []
      },
      "gridPos": {
        "h": 10,
        "w": 8,
        "x": 0,
        "y": 10
      },
      "id": 2,
      "options": {
        "colorMode": "value",
        "graphMode": "area",
        "justifyMode": "center",
        "orientation": "horizontal",
        "reduceOptions": {
          "calcs": [
            "lastNotNull"
          ],
          "fields": "",
          "values": false
        },
        "textMode": "auto"
      },
      "pluginVersion": "10.0.9",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${datasource}"
          },
          "editorMode": "code",
          "expr": "sum by (policy_name) (increase(concurrency_limiter_counter_total{component_id=\"root.0\"}[$__range]))",
          "instant": false,
          "intervalFactor": 1,
          "legendFormat": "{{ policy_name }}",
          "range": true,
          "refId": "A"
        }
      ],
      "title": "Total Requests",
      "type": "stat"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${datasource}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "thresholds"
          },
          "mappings": [],
          "noValue": "No data",
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "green",
                "value": null
              }
            ]
          },
          "unit": "short"
        },
        "overrides": []
      },
      "gridPos": {
        "h": 10,
        "w": 8,
        "x": 8,
        "y": 10
      },
      "id": 3,
      "options": {
        "colorMode": "value",
        "graphMode": "area",
        "justifyMode": "center",
        "orientation": "horizontal",
        "reduceOptions": {
          "calcs": [
            "lastNotNull"
          ],
          "fields": "",
          "values": false
        },
        "textMode": "auto"
      },
      "pluginVersion": "10.0.9",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${datasource}"
          },
          "editorMode": "code",
          "expr": "sum by (policy_name)(increase(concurrency_limiter_counter_total{component_id=\"root.0\", decision_type=\"DECISION_TYPE_ACCEPTED\"}[$__range]))",
          "instant": false,
          "intervalFactor": 1,
          "legendFormat": "{{ policy_name }}",
          "range": true,
          "refId": "A"
        }
      ],
      "title": "Total Accepted Requests",
      "type": "stat"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${datasource}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "thresholds"
          },
          "mappings": [],
          "noValue": "No rejected requests",
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "red",
                "value": null
              }
            ]
          },
          "unit": "short"
        },
        "overrides": []
      },
      "gridPos": {
        "h": 10,
        "w": 8,
        "x": 16,
        "y": 10
      },
      "id": 4,
      "options": {
        "colorMode": "value",
        "graphMode": "area",
        "justifyMode": "center",
        "orientation": "horizontal",
        "reduceOptions": {
          "calcs": [
            "lastNotNull"
          ],
          "fields": "",
          "values": false
        },
        "textMode": "auto"
      },
      "pluginVersion": "10.0.9",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${datasource}"
          },
          "editorMode": "code",
          "expr": "sum by (policy_name)(increase(concurrency_limiter_counter_total{component_id=\"root.0\", decision_type=\"DECISION_TYPE_REJECTED\"}[$__range]))",
          "instant": false,
          "intervalFactor": 1,
          "legendFormat": "{{ instance }} - {{ policy_name }}",
          "range": true,
          "refId": "A"
        }
      ],
      "title": "Total Rejected Requests",
      "type": "stat"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${datasource}"
      },
      "description": "",
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "palette-classic"
          },
          "custom": {
            "axisCenteredZero": false,
            "axisColorMode": "text",
            "axisLabel": "Decisions",
            "axisPlacement": "auto",
            "barAlignment": 0,
            "drawStyle": "line",
            "fillOpacity": 10,
            "gradientMode": "none",
            "hideFrom": {
              "legend": false,
              "tooltip": false,
              "viz": false
            },
            "lineInterpolation": "linear",
            "lineWidth": 1,
            "pointSize": 5,
            "scaleDistribution": {
              "type": "linear"
            },
            "showPoints": "auto",
            "spanNulls": false,
            "stacking": {
              "group": "A",
              "mode": "none"
            },
            "thresholdsStyle": {
              "mode": "off"
            }
          },
          "mappings": [],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "green",
                "value": null
              },
              {
                "color": "red",
                "value": 80
              }
            ]
          },
          "unit": "reqps"
        },
        "overrides": []
      },
      "gridPos": {
        "h": 10,
        "w": 24,
        "x": 0,
        "y": 20
      },
      "id": 5,
      "interval": "10s",
      "options": {
        "legend": {
          "calcs": [],
          "displayMode": "list",
          "placement": "bottom",
          "showLegend": true
        },
        "tooltip": {
          "mode": "single",
          "sort": "none"
        }
      },
      "pluginVersion": "v10.1.0",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${datasource}"
          },
          "editorMode": "code",
          "expr": "sum by(decision_type, policy_name) (rate(concurrency_limiter_counter_total{component_id=\"root.0\"}[$__rate_interval]))",
          "intervalFactor": 1,
          "range": true,
          "refId": "A"
        }
      ],
      "title": "Concurrency Limiter Decision Overview",
      "type": "timeseries"
    }
  ],
  "refresh": "",
  "schemaVersion": 38,
  "style": "dark",
  "tags": [],
  "templating": {
    "list": [
      {
        "hide": 0,
        "includeAll": false,
        "label": "Data Source",
        "multi": false,
        "name": "datasource",
        "options": [],
        "query": "prometheus",
        "refresh": 1,
        "regex": "",
        "skipUrlSync": false,
        "type": "datasource"
      }
    ]
  },
  "time": {
    "from": "now-5m",
    "to": "now"
  },
  "timepicker": {},
  "timezone": "browser",
  "title": "Policy Summary - concurrencylimit",
  "version": 1,
  "weekStart": ""
}

大盤效果如下。