本文介紹如何使用流量調度套件提供的ConcurrencySchedulingPolicy來實現可控並發下的請求優先順序調度。
背景資訊
ConcurrencySchedulingPolicy通過指定的並發數限制來判斷流量是否過載，當請求並發數超過指定上限後，後續請求將被排隊並根據請求優先順序進行調度。大致工作流程如下：
記錄請求並發數：該策略會使用並發數限制器記錄同時正在處理過程中的請求並發數量，判斷請求並發數是否已經達到上限。
請求調度：當請求並發數達到上限時，後續請求將會被排隊，等待先前請求處理完成後再發送給服務，以保證請求並發數始終保持在給定數值。同時，高優先順序的請求有更大的機會被從隊列中取出發送給服務。
當請求並發數超過系統承載上限時，使用此策略可以對請求進行排隊。與並發數限制不同，此時請求不會被直接拒絕，而是進入一個優先順序隊列，在保證請求並發數始終在限制內的同時對請求進行優先順序調度。
前提條件
已添加Kubernetes託管版叢集到ASM執行個體，且ASM執行個體為v1.21.6.97及以上。具體操作，請參見添加叢集到ASM執行個體。
已為Kubernetes叢集中的default命名空間開啟自動注入。具體操作，請參見管理全域命名空間。
已通過kubectl串連至ACK叢集。 具體操作，請參見擷取叢集KubeConfig並通過kubectl工具串連叢集。
已開啟ASM流量調度套件。具體操作，請參見開啟ASM流量調度套件。
已經部署httpbin應用，並且可以通過網關訪問，具體操作，請參見部署httpbin應用
步驟一：建立ConcurrencySchedulingPolicy限流規則
使用kubectl串連到ASM執行個體，具體操作，請參見通過控制面kubectl訪問Istio資源。
使用以下內容，建立concurrencyschedulingpoilcy.yaml檔案。
apiVersion: istio.alibabacloud.com/v1 kind: ConcurrencySchedulingPolicy metadata: name: concurrencyscheduling namespace: istio-system spec: concurrency_scheduler: max_concurrency: 10 concurrency_limiter: max_inflight_duration: 60s scheduler: workloads: - label_matcher: match_labels: http.request.header.user_type: guest parameters: priority: 50.0 name: guest - label_matcher: match_labels: http.request.header.user_type: subscriber parameters: priority: 200.0 name: subscriber selectors: - service: httpbin.default.svc.cluster.local部分配置項說明如下。關於配置項的更多說明，請參見ConcurrencySchedulingPolicy CRD說明。
配置項
說明
max_concurrency
最大請求並發數。樣本中指定為1，即只允許服務同時處理1個請求。
max_inflight_duration
請求處理逾時時間。由於叢集中可能發生Pod重啟等突發情況，導致ASM流量調度套件可能無法記錄到請求結束事件，為防止此類請求影響並發數限制演算法的判斷，需要指定請求處理逾時時間，超過此時間還未響應的請求將視作處理結束。可以通過評估請求的期望最大回應時間來設定此值，樣本中設定為60s。
workloads
根據請求header中的user_type定義了兩類請求，分別是guest和subscriber。guest類型的請求優先順序是50，subscriber類型的請求優先順序是200。
selectors
指定應用限流策略的多個服務。樣本中使用service: httpbin.default.svc.cluster.local 表示對httpbin.default.svc.cluster.local 服務進行並發數限制。
執行以下指令，建立並發數調度策略。
kubectl apply -f concurrencyschedulingpoilcy.yaml
步驟二：驗證控制並發數情境下的請求優先順序調度效果
本步驟使用壓測工具fortio進行測試，安裝方式請參見安裝fortio。
同時開啟兩個終端，並分別運行下面兩個壓測命令（儘可能同時開始這兩個測試），整個測試期間，請確保不要關閉對應的終端。兩個測試都使用10並發、10000預期qps的設定對服務發起調用，遠遠超過了服務的預期並發數限制。
fortio load -c 10 -qps 10000 -H "user_type:guest" -t 30s -timeout 60s -a http://${ASM網關IP}/status/201fortio load -c 10 -qps 10000 -H "user_type:subscriber" -t 30s -timeout 60s -a http://${ASM網關IP}/status/202說明請將上述指令中的
${ASM網關IP}替換為ASM網關的IP地址。有關擷取ASM網關IP地址的具體操作，請參見使用Istio資源實現版本流量路由。測試1的預期輸出：
... # target 50% 4.35294 # target 75% 5.39689 # target 90% 5.89697 # target 99% 6.19701 # target 99.9% 6.22702 Sockets used: 10 (for perfect keepalive, would be 10) Uniform: false, Jitter: false Code 201 : 84 (100.0 %) Response Header Sizes : count 84 avg 249.88095 +/- 0.3587 min 248 max 250 sum 20990 Response Body/Total Sizes : count 84 avg 249.88095 +/- 0.3587 min 248 max 250 sum 20990 All done 84 calls (plus 10 warmup) 3802.559 ms avg, 2.6 qps Successfully wrote 5186 bytes of Json data to xxxxxx.json記錄下測試1輸出的json檔案名稱，例如xxxxxx.json。
測試2的預期輸出：
... # target 50% 1.18121 # target 75% 1.63423 # target 90% 1.90604 # target 99% 2.22941 # target 99.9% 2.28353 Sockets used: 10 (for perfect keepalive, would be 10) Uniform: false, Jitter: false Code 202 : 270 (100.0 %) Response Header Sizes : count 270 avg 250.52963 +/- 0.5418 min 249 max 251 sum 67643 Response Body/Total Sizes : count 270 avg 250.52963 +/- 0.5418 min 249 max 251 sum 67643 All done 270 calls (plus 10 warmup) 1117.614 ms avg, 8.8 qps Successfully wrote 5305 bytes of Json data to yyyyyy.json記錄下測試2輸出的json檔案名稱，例如yyyyyy.json。
由測試1和測試2的預期輸出可以看到，測試2的平均請求延遲約為測試1的四分之一、qps約為測試1的四倍。這是由於先前定義的策略中，subscriber類型的請求優先順序是guest類型的請求的四倍。
（可選）可視化查看結果。
在上一步執行兩個測試命令的目錄中執行以下指令，開啟fortio本機伺服器。
fortio server使用瀏覽器訪問
http://localhost:8080/fortio/browse，根據上一步記錄的兩個測試輸出的json檔案名稱，點擊檔案查看測試的可視化結果。測試1的可視化結果樣本：
測試2的可視化結果樣本：
從可視化結果可以看到，除少數未被限制的請求外、guest類型的請求大多延遲在4000-6000ms內，而subscriber類型的請求大多延遲落在1000-2000ms內。在服務要求負載超限的情況下、subscriber類型的請求優先得到了響應，同時服務接受到的請求並發數始終被限制在給定數值。
相關操作
您可以通過Grafana大盤來觀測ConcurrencySchedulingPolicy策略的執行效果。請確保Grafana使用的資料來源Prometheus執行個體已經完成配置採集ASM流量調度套件相關指標。
將以下內容匯入到Grafana，建立ConcurrencySchedulingPolicy策略的大盤。
展開查看JSON內容
{
"annotations": {
"list": [
{
"builtIn": 1,
"datasource": {
"type": "grafana",
"uid": "-- Grafana --"
},
"enable": true,
"hide": true,
"iconColor": "rgba(0, 211, 255, 1)",
"name": "Annotations & Alerts",
"type": "dashboard"
}
]
},
"editable": true,
"fiscalYearStartMonth": 0,
"graphTooltip": 0,
"id": 42,
"links": [],
"liveNow": false,
"panels": [
{
"datasource": {
"type": "prometheus",
"uid": "${datasource}"
},
"description": "Signal derived from periodic execution of query: (sum(rate(workload_requests_total{policy_name=\"concurrencyscheduling\",component_id=\"root.0\",decision_type=\"DECISION_TYPE_ACCEPTED\"}[30s])) / sum(rate(workload_requests_total{policy_name=\"concurrencyscheduling\",component_id=\"root.0\"}[30s]))) * 100",
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"drawStyle": "line",
"fillOpacity": 10,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "auto",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
},
{
"color": "red",
"value": 80
}
]
},
"unit": ""
},
"overrides": []
},
"gridPos": {
"h": 10,
"w": 24,
"x": 0,
"y": 0
},
"id": 1,
"interval": "10s",
"options": {
"legend": {
"calcs": [],
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"pluginVersion": "v10.1.0",
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "${datasource}"
},
"editorMode": "code",
"expr": "(sum by (policy_name)(rate(workload_requests_total{component_id=\"root.0\",decision_type=\"DECISION_TYPE_ACCEPTED\"}[30s])) / sum by (policy_name)(rate(workload_requests_total{component_id=\"root.0\"}[30s]))) * 100",
"intervalFactor": 1,
"legendFormat": "policy={{policy_name}}",
"range": true,
"refId": "A"
}
],
"title": "Accept Percentage",
"type": "timeseries"
},
{
"datasource": {
"type": "prometheus",
"uid": "${datasource}"
},
"description": "",
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "Decisions",
"axisPlacement": "auto",
"barAlignment": 0,
"drawStyle": "line",
"fillOpacity": 10,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "auto",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
},
{
"color": "red",
"value": 80
}
]
},
"unit": "reqps"
},
"overrides": []
},
"gridPos": {
"h": 10,
"w": 24,
"x": 0,
"y": 10
},
"id": 2,
"interval": "10s",
"options": {
"legend": {
"calcs": [],
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"pluginVersion": "v10.1.0",
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "${datasource}"
},
"editorMode": "code",
"expr": "sum by(decision_type, policy_name) (rate(workload_requests_total{component_id=\"root.0\"}[$__rate_interval]))",
"intervalFactor": 1,
"range": true,
"refId": "A"
}
],
"title": "Workload Decisions",
"type": "timeseries"
},
{
"datasource": {
"type": "prometheus",
"uid": "${datasource}"
},
"description": "",
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "Decisions",
"axisPlacement": "auto",
"barAlignment": 0,
"drawStyle": "line",
"fillOpacity": 10,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "auto",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
},
{
"color": "red",
"value": 80
}
]
},
"unit": "reqps"
},
"overrides": []
},
"gridPos": {
"h": 10,
"w": 24,
"x": 0,
"y": 20
},
"id": 3,
"interval": "10s",
"options": {
"legend": {
"calcs": [],
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"pluginVersion": "v10.1.0",
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "${datasource}"
},
"editorMode": "code",
"expr": "sum by(workload_index, decision_type, policy_name) (rate(workload_requests_total{component_id=\"root.0\",decision_type=\"DECISION_TYPE_ACCEPTED\"}[$__rate_interval]))",
"intervalFactor": 1,
"range": true,
"refId": "A"
}
],
"title": "Workload Decisions (accepted)",
"type": "timeseries"
},
{
"datasource": {
"type": "prometheus",
"uid": "${datasource}"
},
"description": "",
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "Decisions",
"axisPlacement": "auto",
"barAlignment": 0,
"drawStyle": "line",
"fillOpacity": 10,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "auto",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
},
{
"color": "red",
"value": 80
}
]
},
"unit": "reqps"
},
"overrides": []
},
"gridPos": {
"h": 10,
"w": 24,
"x": 0,
"y": 30
},
"id": 4,
"interval": "10s",
"options": {
"legend": {
"calcs": [],
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"pluginVersion": "v10.1.0",
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "${datasource}"
},
"editorMode": "code",
"expr": "sum by(workload_index, decision_type, policy_name) (rate(workload_requests_total{component_id=\"root.0\",decision_type=\"DECISION_TYPE_REJECTED\"}[$__rate_interval]))",
"intervalFactor": 1,
"range": true,
"refId": "A"
}
],
"title": "Workload Decisions (rejected)",
"type": "timeseries"
},
{
"datasource": {
"type": "prometheus",
"uid": "${datasource}"
},
"fieldConfig": {
"defaults": {
"color": {
"mode": "thresholds"
},
"mappings": [],
"noValue": "No data",
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "blue",
"value": null
}
]
},
"unit": "short"
},
"overrides": []
},
"gridPos": {
"h": 10,
"w": 8,
"x": 0,
"y": 40
},
"id": 5,
"options": {
"colorMode": "value",
"graphMode": "area",
"justifyMode": "center",
"orientation": "horizontal",
"reduceOptions": {
"calcs": [
"lastNotNull"
],
"fields": "",
"values": false
},
"textMode": "auto"
},
"pluginVersion": "10.0.9",
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "${datasource}"
},
"editorMode": "code",
"expr": "sum by (policy_name)(increase(workload_requests_total{component_id=\"root.0\"}[$__range]))",
"instant": false,
"intervalFactor": 1,
"legendFormat": "{{ policy_name }}",
"range": true,
"refId": "A"
}
],
"title": "Total Requests",
"type": "stat"
},
{
"datasource": {
"type": "prometheus",
"uid": "${datasource}"
},
"fieldConfig": {
"defaults": {
"color": {
"mode": "thresholds"
},
"mappings": [],
"noValue": "No data",
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
}
]
},
"unit": "short"
},
"overrides": []
},
"gridPos": {
"h": 10,
"w": 8,
"x": 8,
"y": 40
},
"id": 6,
"options": {
"colorMode": "value",
"graphMode": "area",
"justifyMode": "center",
"orientation": "horizontal",
"reduceOptions": {
"calcs": [
"lastNotNull"
],
"fields": "",
"values": false
},
"textMode": "auto"
},
"pluginVersion": "10.0.9",
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "${datasource}"
},
"editorMode": "code",
"expr": "sum by (policy_name)(increase(workload_requests_total{component_id=\"root.0\",decision_type=\"DECISION_TYPE_ACCEPTED\"}[$__range]))",
"instant": false,
"intervalFactor": 1,
"legendFormat": "{{ policy_name }}",
"range": true,
"refId": "A"
}
],
"title": "Total Accepted Requests",
"type": "stat"
},
{
"datasource": {
"type": "prometheus",
"uid": "${datasource}"
},
"fieldConfig": {
"defaults": {
"color": {
"mode": "thresholds"
},
"mappings": [],
"noValue": "No rejected requests",
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "red",
"value": null
}
]
},
"unit": "short"
},
"overrides": []
},
"gridPos": {
"h": 10,
"w": 8,
"x": 16,
"y": 40
},
"id": 7,
"options": {
"colorMode": "value",
"graphMode": "area",
"justifyMode": "center",
"orientation": "horizontal",
"reduceOptions": {
"calcs": [
"lastNotNull"
],
"fields": "",
"values": false
},
"textMode": "auto"
},
"pluginVersion": "10.0.9",
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "${datasource}"
},
"editorMode": "code",
"expr": "sum by (policy_name)(increase(workload_requests_total{component_id=\"root.0\",decision_type=\"DECISION_TYPE_REJECTED\"}[$__range]))",
"instant": false,
"intervalFactor": 1,
"legendFormat": "{{ policy_name }}",
"range": true,
"refId": "A"
}
],
"title": "Total Rejected Requests",
"type": "stat"
},
{
"datasource": {
"type": "prometheus",
"uid": "${datasource}"
},
"description": "",
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "Latency",
"axisPlacement": "auto",
"barAlignment": 0,
"drawStyle": "line",
"fillOpacity": 10,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "auto",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
},
{
"color": "red",
"value": 80
}
]
},
"unit": "ms"
},
"overrides": []
},
"gridPos": {
"h": 10,
"w": 24,
"x": 0,
"y": 50
},
"id": 8,
"interval": "10s",
"options": {
"legend": {
"calcs": [],
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"pluginVersion": "v10.1.0",
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "${datasource}"
},
"editorMode": "code",
"expr": "(sum by (workload_index, policy_name) (increase(workload_latency_ms_sum{component_id=\"root.0\"}[$__rate_interval])))/(sum by (workload_index, policy_name) (increase(workload_latency_ms_count{component_id=\"root.0\"}[$__rate_interval])))",
"intervalFactor": 1,
"range": true,
"refId": "A"
}
],
"title": "Workload Latency",
"type": "timeseries"
},
{
"datasource": {
"type": "prometheus",
"uid": "${datasource}"
},
"description": "",
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "Wait Time",
"axisPlacement": "auto",
"barAlignment": 0,
"drawStyle": "line",
"fillOpacity": 10,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "auto",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
},
{
"color": "red",
"value": 80
}
]
},
"unit": "ms"
},
"overrides": []
},
"gridPos": {
"h": 10,
"w": 12,
"x": 0,
"y": 60
},
"id": 9,
"interval": "10s",
"options": {
"legend": {
"calcs": [],
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"pluginVersion": "v10.1.0",
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "${datasource}"
},
"editorMode": "code",
"expr": "(sum by (workload_index, policy_name) (increase(request_in_queue_duration_ms_sum{component_id=\"root.0\"}[$__rate_interval])))/ ((sum by (workload_index, policy_name) (increase(request_in_queue_duration_ms_count{component_id=\"root.0\"}[$__rate_interval]))) != 0)",
"intervalFactor": 1,
"range": true,
"refId": "A"
}
],
"title": "Request in Queue Duration",
"type": "timeseries"
},
{
"datasource": {
"type": "prometheus",
"uid": "${datasource}"
},
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisCenteredZero": false,
"axisColorMode": "text",
"axisGridShow": true,
"axisLabel": "",
"axisPlacement": "hidden",
"fillOpacity": 80,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"lineWidth": 1,
"scaleDistribution": {
"type": "linear"
},
"thresholdsStyle": {
"mode": "off"
}
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
},
{
"color": "red",
"value": 80
}
]
},
"unit": "ms"
},
"overrides": []
},
"gridPos": {
"h": 10,
"w": 12,
"x": 12,
"y": 60
},
"id": 10,
"options": {
"barRadius": 0,
"barWidth": 0.97,
"colorByField": "",
"fullHighlight": false,
"groupWidth": 0.7,
"legend": {
"calcs": [],
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"orientation": "horizontal",
"showValue": "auto",
"stacking": "none",
"tooltip": {
"mode": "single",
"sort": "sort"
},
"xTickLabelRotation": 0,
"xTickLabelSpacing": 0
},
"pluginVersion": "v10.1.0",
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "${datasource}"
},
"editorMode": "code",
"expr": "topk(10, (sum by(workload_index, policy_name) (increase(request_in_queue_duration_ms_sum{component_id=\"root.0\"}[$__range])) ) / ((sum by(workload_index, policy_name) (increase(request_in_queue_duration_ms_count{component_id=\"root.0\"}[$__range])) )) != 0)",
"format": "time_series",
"instant": true,
"intervalFactor": 1,
"legendFormat": "",
"range": false,
"refId": "A"
}
],
"title": "Request in Queue Duration",
"type": "barchart"
},
{
"datasource": {
"type": "prometheus",
"uid": "${datasource}"
},
"description": "",
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "Token Rate",
"axisPlacement": "auto",
"barAlignment": 0,
"drawStyle": "line",
"fillOpacity": 10,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "auto",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
},
{
"color": "red",
"value": 80
}
]
},
"unit": ""
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 12,
"x": 0,
"y": 70
},
"id": 13,
"interval": "10s",
"options": {
"legend": {
"calcs": [],
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"pluginVersion": "v10.1.0",
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "${datasource}"
},
"editorMode": "code",
"expr": "sum by (policy_name)(rate(incoming_tokens_total{component_id=\"root.0\"}[$__rate_interval]))",
"intervalFactor": 1,
"range": true,
"refId": "A"
}
],
"title": "Incoming Token Rate",
"type": "timeseries"
},
{
"datasource": {
"type": "prometheus",
"uid": "${datasource}"
},
"description": "",
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "Token Rate",
"axisPlacement": "auto",
"barAlignment": 0,
"drawStyle": "line",
"fillOpacity": 10,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "auto",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
},
{
"color": "red",
"value": 80
}
]
},
"unit": ""
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 12,
"x": 12,
"y": 70
},
"id": 14,
"interval": "10s",
"options": {
"legend": {
"calcs": [],
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"pluginVersion": "v10.1.0",
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "${datasource}"
},
"editorMode": "code",
"expr": "sum by (policy_name)(rate(accepted_tokens_total{component_id=\"root.0\"}[$__rate_interval]))",
"intervalFactor": 1,
"legendFormat": "Accepted Token Rate",
"range": true,
"refId": "A"
},
{
"datasource": {
"type": "prometheus",
"uid": "${datasource}"
},
"editorMode": "code",
"expr": "sum by (policy_name)(rate(rejected_tokens_total{component_id=\"root.0\"}[$__rate_interval]))",
"intervalFactor": 1,
"legendFormat": "Rejected Token Rate",
"range": true,
"refId": "B"
}
],
"title": "Accepted Token Rate vs Rejected Token Rate",
"type": "timeseries"
},
{
"datasource": {
"type": "prometheus",
"uid": "${datasource}"
},
"fieldConfig": {
"defaults": {
"color": {
"mode": "thresholds"
},
"mappings": [],
"noValue": "No data",
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "blue",
"value": null
}
]
},
"unit": "short"
},
"overrides": []
},
"gridPos": {
"h": 10,
"w": 8,
"x": 0,
"y": 78
},
"id": 15,
"options": {
"colorMode": "value",
"graphMode": "area",
"justifyMode": "center",
"orientation": "horizontal",
"reduceOptions": {
"calcs": [
"lastNotNull"
],
"fields": "",
"values": false
},
"textMode": "auto"
},
"pluginVersion": "10.0.9",
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "${datasource}"
},
"editorMode": "code",
"expr": "sum by (policy_name)(increase(incoming_tokens_total{component_id=\"root.0\"}[$__range]))",
"instant": false,
"intervalFactor": 1,
"legendFormat": "{{ policy_name }}",
"range": true,
"refId": "A"
}
],
"title": "Total Incoming Tokens",
"type": "stat"
},
{
"datasource": {
"type": "prometheus",
"uid": "${datasource}"
},
"fieldConfig": {
"defaults": {
"color": {
"mode": "thresholds"
},
"mappings": [],
"noValue": "No data",
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
}
]
},
"unit": "short"
},
"overrides": []
},
"gridPos": {
"h": 10,
"w": 8,
"x": 8,
"y": 78
},
"id": 16,
"options": {
"colorMode": "value",
"graphMode": "area",
"justifyMode": "center",
"orientation": "horizontal",
"reduceOptions": {
"calcs": [
"lastNotNull"
],
"fields": "",
"values": false
},
"textMode": "auto"
},
"pluginVersion": "10.0.9",
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "${datasource}"
},
"editorMode": "code",
"expr": "sum by (policy_name)(increase(accepted_tokens_total{component_id=\"root.0\"}[$__range]))",
"instant": false,
"intervalFactor": 1,
"legendFormat": "{{ instance }} - {{ policy_name }}",
"range": true,
"refId": "A"
}
],
"title": "Total Accepted Tokens",
"type": "stat"
},
{
"datasource": {
"type": "prometheus",
"uid": "${datasource}"
},
"fieldConfig": {
"defaults": {
"color": {
"mode": "thresholds"
},
"mappings": [],
"noValue": "No data",
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "red",
"value": null
}
]
},
"unit": "short"
},
"overrides": []
},
"gridPos": {
"h": 10,
"w": 8,
"x": 16,
"y": 78
},
"id": 17,
"options": {
"colorMode": "value",
"graphMode": "area",
"justifyMode": "center",
"orientation": "horizontal",
"reduceOptions": {
"calcs": [
"lastNotNull"
],
"fields": "",
"values": false
},
"textMode": "auto"
},
"pluginVersion": "10.0.9",
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "${datasource}"
},
"editorMode": "code",
"expr": "sum by (policy_name)(increase(rejected_tokens_total{component_id=\"root.0\"}[$__range]))",
"instant": false,
"intervalFactor": 1,
"legendFormat": "{{ policy_name }}",
"range": true,
"refId": "A"
}
],
"title": "Total Rejected Tokens",
"type": "stat"
}
],
"refresh": "",
"schemaVersion": 38,
"style": "dark",
"tags": [],
"templating": {
"list": [
{
"hide": 0,
"includeAll": false,
"label": "Data Source",
"multi": false,
"name": "datasource",
"options": [],
"query": "prometheus",
"refresh": 1,
"regex": "",
"skipUrlSync": false,
"type": "datasource"
}
]
},
"time": {
"from": "now-5m",
"to": "now"
},
"timepicker": {},
"timezone": "browser",
"title": "Policy Summary - concurrencyscheduling",
"version": 2,
"weekStart": ""
}大盤效果如下。
