Function Compute:Limits and rules of auto instance scaling

Scaling of on-demand instances

When the total number of instances or the scaling-out speed of instances reaches the limit, Function Compute reports a throttling error, for which the HTTP status code is 429. The following figure shows how Function Compute performs throttling in a scenario in which the number of invocations rapidly increases.

• ①: Before the upper limit for burst instances is reached, Function Compute immediately creates instances when the number of requests increases. During this process, a cold start occurs but no throttling error is reported.

• ②: When the limit for burst instances is reached, the increase of instances is restricted by the growth rate. Throttling errors are reported for some requests.

• ③: When the upper limit of instances is reached, some requests are throttled.

Scaling of provisioned instances

If the number of sudden invocations is large, the creation of a large number of instances is throttled, which results in request failures. Cold starts of instances also increase request latency. To prevent such issues, you can use provisioned instances in Function Compute. Provisioned instances are reserved in advance of invocations.

The following figure shows the throttling behavior of provisioned instances in a scenario that has the same amount of traffic as the preceding figure.

• ①: Before the provisioned instances are fully loaded, requests are immediately processed. During this process, no cold start occurs and no throttling error is reported.

• ②: When the provisioned instances are fully loaded, Function Compute immediately creates instances before the upper limit for burst instances is reached. During this process, a cold start occurs but no throttling error is reported.

Scheduled scaling

Scenarios

Scheduled instance scaling applies to functions that have noticeable cyclical patterns or predictable traffic peaks. If the number of concurrent invocations is greater than the concurrency capacity of the scheduled scaling policy, excess requests are sent to on-demand instances for processing.

Sample configuration

You can configure two scheduled scaling policies. The first one increases the number of provisioned instances before traffic surges. The second policy decreases the number of provisioned instances when the traffic declines. The following figure shows the details.

The following code snippet provides an example of the request parameters of PutProvisionConfig that is called to create a scheduled scaling policy. In this example, a scheduled scaling policy is configured for function_1. The time zone is set to Asia/Shanghai (UTC+8). The policy takes effect from 10:00:00 on August 1, 2024 to 10:00:00 on August 30, 2024. The policy increases the number of instances to 50 at 20:00 every day and reduces the number of instances to 10 at 22:00 every day.

"scheduledActions": [
    {
      "name": "scale_up_action",
      "startTime": "2024-08-01T10:00:00",
      "endTime": "2024-08-30T10:00:00",
      "target": 50,
      "scheduleExpression": "cron(0 0 20 * * *)",
      "timeZone": "Asia/Shanghai"
    },
    {
      "name": "scale_down_action",
      "startTime": "2024-08-01T10:00:00",
      "endTime": "2024-08-30T10:00:00",
      "target": 10,
      "scheduleExpression": "cron(0 0 22 * * *)",
      "timeZone": "Asia/Shanghai"
    }
  ]

Cron expressions

Water-level scaling

Scenarios

Function Compute periodically collects values of metrics, such as the concurrency utilization of provisioned instances or resource utilization of provisioned instances. Provisioned instances are scaled based on values of the metrics and the minimum and maximum numbers of provisioned instances you specify.

Sample configuration

Assume you configure a water-level scaling policy in which you specify concurrency utilization threshold of provisioned instances. When the traffic volume increases, the scale-out threshold is triggered and the system starts to scale out provisioned instances. The scale-out stops if the specified maximum value is reached. Excess requests are sent to on-demand instances. When the traffic volume decreases, the scale-in threshold is triggered and the system starts to scale in provisioned instances. The following figure shows the details.

The following code snippet provides an example of the request parameters of PutProvisionConfig that is called to create a water-level scaling policy. In this example, a water-level scaling policy is configured for function_1. The time zone is set to Asia/Shanghai (UTC+8). The policy takes effect from 10:00:00 on August 1, 2024 to 10:00:00 on August 30, 2024. The policy tracks ProvisionedConcurrencyUtilization and starts to perform scale-out when the concurrency utilization exceeds 60% and scale-in when the concurrency utilization is lower than 60%. The upper limit is 100 and the lower limit is 10.

"targetTrackingPolicies": [
    {
      "name": "action_1",
      "startTime": "2024-08-01T10:00:00",
      "endTime": "2024-08-30T10:00:00",
      "metricType": "ProvisionedConcurrencyUtilization",
      "metricTarget": 0.6,
      "minCapacity": 10,
      "maxCapacity": 100,
      "timeZone": "Asia/Shanghai"
    }
  ]

Scaling principles

A relatively conservative scale-in process is achieved by using a scale-in coefficient, whose value falls in the range of (0,1]. The scale-in coefficient is a system parameter that is used to slow down the scale-in speed. You do not need to set the scale-in coefficient. The target values for scaling operations are the smallest integers that are greater than or equal to the following calculation results:

• Scale-out target = Number of current provisioned instances × (Current metric value/Specified utilization threshold)

• Scale-in target = Number of current provisioned instances × Scaling coefficient × (1 – Current metric value/Specified utilization threshold)

Example:

If the current metric value is 80%, the specified utilization threshold is 40%, and the number of current provisioned instances is 100, the target number of instances is: 100 × (80%/40%) = 200. The number of provisioned instances is increased to 200, if the specified maximum number of provisioned instances is equal to or greater than 200. This ensures that the utilization threshold remains close to 40%.