Limits and rules of instance scaling

If the instance number or scaling speed goes beyond the limit, Function Compute returns an HTTP 429 status code, indicating that a throttling error has occurred. The following figure shows how Function Compute applies throttling when invocations surge.

• ①: Function Compute immediately creates instances to handle the surge in requests. Cold starts occur during this process. No throttling errors are reported because the number of burstable instances has not reached the upper limit.

• ②: The increase in the number of instances is now limited by the instance growth rate, as the upper limit for burstable instances has been reached. Throttling errors are reported for some requests.

• ③: The number of instances reaches the upper limit, resulting in throttling errors for some requests.

When the number of sudden invocations is too large, throttling errors become inevitable. In addition, the creation of new instances introduces cold starts. Both increase the request handling latency. To mitigate latency, you can reserve instances in advance in Function Compute. These reserved instances are called provisioned instances.

The following figure shows how Function Compute applies throttling when provisioned instances are configured and invocations surge in the same manner as in the on-demand mode case.

• ①: All incoming requests are immediately processed, until the provisioned instances reach their full capacity. During this process, no cold starts occur, and no throttling errors are reported.

• ②: The provisioned instances are now fully utilized. Function Compute starts to create on-demand instances to handle subsequent requests until the number of burstable instances reaches the upper limit. During this process, cold starts occur, but no throttling errors are reported.

Scenarios

Choose scheduled scaling when your function experiences distinct periodic patterns or predictable traffic peaks. When the number of concurrent invocations exceeds the capacity defined by the scheduled scaling policy, all excess requests will be directed to on-demand instances for processing.

Sample configuration

The following figure shows two scheduled actions for instance scaling: the first action scales out the provisioned instances before the traffic peak, while the second scales in the instances afterward.

The following code snippet shows how to call the PutProvisionConfig operation to configure scheduled scaling policies. In this example, a function named function_1 is configured to automatically scale in and out, with the time zone set to Asia/Shanghai (UTC+8). The configurations take effect from 10:00:00 on August 1, 2024, to 10:00:00 on August 30, 2024 (UTC+8). During this period, the number of provisioned instances is increased to 50 at 20:00 (UTC+8) and reduced to 10 at 22:00 (UTC+8) each day.

"scheduledActions": [
    {
      "name": "scale_up_action",
      "startTime": "2024-08-01T10:00:00",
      "endTime": "2024-08-30T10:00:00",
      "target": 50,
      "scheduleExpression": "cron(0 0 20 * * *)",
      "timeZone": "Asia/Shanghai"
    },
    {
      "name": "scale_down_action",
      "startTime": "2024-08-01T10:00:00",
      "endTime": "2024-08-30T10:00:00",
      "target": 10,
      "scheduleExpression": "cron(0 0 22 * * *)",
      "timeZone": "Asia/Shanghai"
    }
  ]

Cron expressions

Scenarios

After you configure a threshold-based scaling policy, Function Compute periodically collects the concurrency or resource utilization metrics for the provisioned instances. It uses these metrics, along with the minimum and maximum numbers of provisioned instances you specify, to control instance scaling, ensuring the number of instances aligns more closely with actual resource usage.

Sample configuration

The following figure shows an example of auto scaling based on the utilization of instance concurrency. When the traffic volume increases, the scale-out threshold is triggered and Function Compute starts to increase the number of provisioned instances. The scale-out stops when the number reaches the upper limit you set. Excess requests are sent to on-demand instances for processing.

The following code snippet shows how to call the PutProvisionConfig operation to configure threshold-based scaling policies. In this example, a function named function_1 is configured to automatically scale in and out based on the ProvisionedConcurrencyUtilization metric, which tracks the concurrency utilization of provisioned instances. The time zone is set to Asia/Shanghai (UTC+8). The configurations take effect from 10:00:00 on August 1, 2024, to 10:00:00 on August 30, 2024 (UTC+8). During this period, when concurrency utilization exceeds 60%, the number of provisioned instances is increased, up to a maximum of 100. Conversely, when concurrency utilization falls below 60%, the number of provisioned instances is reduced, down to a minimum of 10.

"targetTrackingPolicies": [
    {
      "name": "action_1",
      "startTime": "2024-08-01T10:00:00",
      "endTime": "2024-08-30T10:00:00",
      "metricType": "ProvisionedConcurrencyUtilization",
      "metricTarget": 0.6,
      "minCapacity": 10,
      "maxCapacity": 100,
      "timeZone": "Asia/Shanghai"
    }
  ]

Scaling principles

When instance scale-in is triggered, Function Compute gradually reduces the number of provisioned instances based on a scale-in coefficient that ranges from 0 (excluded) to 1. The scale-in coefficient is a system parameter used to slow down the scale-in speed. It does not require manual configuration. The target values for scaling tasks are the smallest integers that are greater than or equal to the following calculation results:

• Scale-out target value = Current provisioned instances × (Current metric value/Specified utilization threshold)

• Scale-in target value = Current provisioned instances × Scale-in coefficient × (1 - Current metric value/Specified utilization threshold)

The following example demonstrates how to calculate the scale-out target. Similarly, the scale-in target can be determined using the previously mentioned principle and formula.

If the current metric value is 80%, the specified utilization threshold is 40%, and the current number of provisioned instances is 100, then the target number of instances is calculated as follows: 100 × (80%/40%) = 200. The number of provisioned instances is increased to 200 (as long as this does not exceed the maximum allowed) to ensure that utilization stays around 40%.