When backend services receive more requests than they can handle, response times increase and cascading failures can spread across your microservices. Throttling policies on MSE cloud-native gateways cap request rates at the route level, so excess traffic is rejected before it reaches your services.
MSE supports three types of route-level throttling rules. These rules are independent and can be combined on the same route:
| Rule type | What it controls | When to use |
|---|---|---|
| Throttling rule | Queries per second (QPS) on a route | Protect against sudden traffic spikes |
| Concurrency rule | Number of requests processed simultaneously | Prevent resource exhaustion from long-running requests |
| Circuit breaking rule | Error rate or response time thresholds | Isolate unhealthy backends and allow recovery |
How throttling works
Per-node threshold distribution
The threshold you configure applies at the gateway level. For multi-node gateways, MSE divides the threshold evenly across nodes, rounding up:
Per-node threshold = ceil(Gateway-level threshold / Number of nodes)For example, if you set a QPS threshold of 1001 on a gateway with 2 nodes, each node enforces a limit of 501 QPS.
How the three rule types interact
Throttling, concurrency, and circuit breaking rules evaluate independently. A request must pass all active rules on a route to reach the backend:
Throttling rules check the request rate (QPS). If the rate exceeds the threshold, the request is rejected immediately.
Concurrency rules check how many requests are currently in flight. If the count exceeds the threshold, the request is rejected immediately.
Circuit breaking rules monitor response times or error rates over a sliding window. When the threshold is breached, the circuit opens and all requests to the route are rejected for the configured duration.
You can enable any combination of these rules on a single route to address different failure modes.
Configure throttling rules
Throttling rules reject requests when the QPS on a route exceeds the specified threshold.
The configuration interface varies by gateway version. Gateways v1.2.25 or later use inline configuration on the page. Gateways v1.1.0 to v1.2.24 use dialog boxes. Both interfaces are covered below. Gateways v1.0.0 support only basic time-window throttling. See Configure throttling for gateways v1.0.0.
Open the throttling settings
Log on to the MSE console. In the top navigation bar, select a region.
In the left-side navigation pane, choose Cloud-native Gateway > Gateways. On the Gateways page, click the name of the gateway.
In the left-side navigation pane, click Routes, and click the Routes tab.
Find the target route and click Policies in the Actions column.
In the Policies section, click the Throttling tab.
Gateways v1.2.25 or later
On the Throttling tab, click the Throttling Rules subtab.
Configure the following parameters:
Parameter Description Gateway QPS Threshold Maximum QPS allowed across the entire gateway for this route Web Fallback Behavior Action taken when the threshold is exceeded: Return Specified Content or Go to Specified Page Redirect URL (If Go to Specified Page is selected) Target URL for the redirect HTTP Status Code (If Return Specified Content is selected) HTTP status code returned to the client. Default: 429 Returned Content Type (If Return Specified Content is selected) Response format: Plain Text or JSON Returned HTTP Text (If Return Specified Content is selected) Response body content Enable Toggle the rule on or off Click New or Save. In the confirmation message, click OK.
Gateways v1.1.0 to v1.2.24
On the Throttling tab, click the Throttling Rules subtab, and click Add Throttling Rule.
In the Add Throttling Protection Rule dialog box:
In the Configure a Throttling Rule step, enter a value in the Overall QPS Threshold field and turn on the Whether to open switch. Click Next.
In the Configure Throttling Behavior step, click New behavior to define the HTTP response returned when throttling is triggered.
Click New.
Configure concurrency rules
Concurrency rules reject requests when the total number of in-flight requests on a route exceeds the specified threshold. Use concurrency rules to cap resource consumption from slow or long-running requests.
Open the throttling settings
Follow the same navigation steps as Configure throttling rules > Open the throttling settings.
Gateways v1.2.25 or later
On the Throttling tab, click the Concurrency Rules subtab.
Configure the following parameters:
Parameter Description Gateway Parallelism Threshold Maximum number of concurrent requests allowed across the entire gateway for this route Web Fallback Behavior Action taken when the threshold is exceeded: Return Specified Content or Go to Specified Page Redirect URL (If Go to Specified Page is selected) Target URL for the redirect HTTP Status Code (If Return Specified Content is selected) HTTP status code returned to the client. Default: 429 Returned Content Type (If Return Specified Content is selected) Response format: Plain Text or JSON Returned HTTP Text (If Return Specified Content is selected) Response body content Enable Toggle the rule on or off Click New or Save. In the confirmation message, click OK.
Gateways v1.1.0 to v1.2.24
On the Throttling tab, click the Concurrency Rules subtab, and click Add Concurrency Rules.
In the Add Concurrency Protection Rule dialog box:
In the Configure a Protection Rule step, enter a value in the Overall Request Concurrency Threshold field and turn on the Whether to open switch. Click Next.
In the Configure Throttling Behavior step, click Add Behavior to define the HTTP response returned when throttling is triggered.
Click New.
Configure circuit breaking rules
Circuit breaking rules temporarily stop routing traffic when the response time or error rate on a route exceeds a threshold. During the circuit breaking period, all requests to the route are rejected with the configured fallback response. After the period elapses, the gateway sends a probe request to decide whether to resume normal traffic or retrigger circuit breaking.
Open the throttling settings
Follow the same navigation steps as Configure throttling rules > Open the throttling settings.
Gateways v1.2.25 or later
On the Throttling tab, click the Fuse rule subtab.
Configure the following parameters:
Parameter Description Statistical window duration Length of the evaluation window. Valid range: 1 second to 120 minutes Minimum number of requests Minimum number of requests required within the window before circuit breaking can trigger. If the request count is below this value, circuit breaking does not activate even if the threshold is exceeded Threshold Type Metric used to evaluate circuit breaking conditions. See Threshold type options Fusing time (s) Duration of the circuit breaking period. All requests to the route are rejected during this time Web Fallback Behavior Action taken during circuit breaking: Return Specified Content or Go to Specified Page Redirect URL (If Go to Specified Page is selected) Target URL for the redirect HTTP Status Code (If Return Specified Content is selected) HTTP status code returned to the client. Default: 429 Returned Content Type (If Return Specified Content is selected) Response format: Plain Text or JSON Returned HTTP Text (If Return Specified Content is selected) Response body content Enable Toggle the rule on or off Threshold type options
Slow call ratio (%) -- Triggers circuit breaking when the percentage of slow requests exceeds the threshold. Set the Slow call RT parameter to define the maximum acceptable response time (RT). Any request with an RT above this value counts as a slow call. After circuit breaking ends, the gateway tests the next request. If its RT is within the limit, traffic resumes. Otherwise, circuit breaking retriggers.
Abnormal proportion (%) -- Triggers circuit breaking when the percentage of failed requests exceeds the threshold. After circuit breaking ends, the gateway resumes traffic automatically.
Click Create or Save. In the confirmation message, click OK.
Gateways v1.1.0 to v1.2.24
On the Throttling tab, click the Fuse rule subtab, and click Added fusing rule.
In the Add Circuit Breaking Protection Rule dialog box:
In the Configure a Protection Rule step, configure the following parameters and turn on the Whether to open switch. Click Next.
Parameter Description Statistical window duration Length of the evaluation window. Valid range: 1 second to 120 minutes Minimum number of requests Minimum request count required before circuit breaking can trigger Threshold Type Slow call ratio (%) or Abnormal proportion (%). For slow call ratio, also configure the Slow call RT parameter to set the response time threshold Fusing time (s) Duration of the circuit breaking period Fuse recovery strategy Recovery behavior after the circuit breaking period ends. Select Single probe recovery to test the next request before resuming traffic In the Configure Throttling Behavior step, click Add Behavior to define the HTTP response returned during circuit breaking.
Click New.
Configure throttling for gateways v1.0.0
Gateways running version 1.0.0 support basic time-window throttling only. To use throttling, concurrency, and circuit breaking rules, upgrade to version 1.1.0 or later.
Log on to the MSE console.
In the left-side navigation pane, choose Cloud-native Gateway > Gateways. In the top navigation bar, select a region.
On the Gateways page, click the name of the gateway.
In the left-side navigation pane, choose Routes > Route Settings. Find the target route and click Policies in the Actions column.
In the Policies section, click the Throttling tab.
If no throttling policy exists, click Configure policy.
If a throttling policy already exists, click the edit icon next to Throttling Threshold.
In the Throttling Threshold section, configure the following parameters and click OK:
Parameter Description Time Window Period over which requests are counted. Default: 1 second Maximum Number of Requests (Single-machine Request Threshold) Maximum number of single-server requests allowed per gateway node within the time window Turn on Enabled to activate the throttling policy.
When enabled, the gateway limits requests on this route based on the configured time window and threshold.
When disabled, requests are not limited.
Verify the configuration
Send a test request to confirm the throttling policy works as expected:
curl -I http://<gateway-ip>/demo/item/listReplace <gateway-ip> with the IP address of your Ingress gateway.
Normal response (throttling not triggered):
HTTP/1.1 200 OK
x-content-type-options: nosniff
x-xss-protection: 1; mode=block
cache-control: no-cache, no-store, max-age=0, must-revalidate
pragma: no-cache
expires: 0
x-frame-options: DENY
content-type: application/json
content-length: 86
date: Mon, 29 Nov 2021 08:28:00 GMT
x-envoy-upstream-service-time: 4
server: istio-envoyThrottled response (HTTP 429):
HTTP/1.1 429 Too Many Requests
x-local-rate-limit: true
content-length: 18
content-type: text/plain
date: Mon, 29 Nov 2021 08:28:01 GMT
server: istio-envoyThe x-local-rate-limit: true header confirms that the gateway's throttling policy rejected the request.
FAQ
Can I use multiple rule types on the same route?
Yes. Throttling, concurrency, and circuit breaking rules are independent. You can enable any combination on a single route. A request must pass all active rules to reach the backend.
How is the per-node threshold calculated for multi-node gateways?
MSE divides the gateway-level threshold by the number of nodes and rounds up. For example, a QPS threshold of 1001 on a 2-node gateway results in a per-node limit of 501 QPS.
What happens when circuit breaking ends?
The behavior depends on the threshold type:
Slow call ratio: The gateway sends a probe request. If the response time is within the limit, traffic resumes. Otherwise, circuit breaking retriggers.
Abnormal proportion: Traffic resumes automatically after the circuit breaking period elapses.
Which gateway version do I need for each rule type?
| Rule type | Minimum gateway version |
|---|---|
| Basic time-window throttling | v1.0.0 |
| Throttling, concurrency, and circuit breaking rules | v1.1.0 |
| Inline configuration (no dialog boxes) | v1.2.25 |