Configure scheduling policies for level-2 subscription quotas - MaxCompute

MaxCompute provides limited subscription computing resources. During actual data development, you may need to run multiple jobs at the same time in MaxCompute. In this case, computing resources must be properly allocated to the concurrent jobs. MaxCompute supports the following types of policies to schedule level-2 subscription quotas: first-in-first-out (FIFO) scheduling and fair scheduling. You can configure a scheduling policy for level-2 subscription quotas to properly allocate resources.

Usage notes

In actual resource scheduling, scheduling based on job priorities takes precedence over scheduling based on the quota policy. Resources are preferentially allocated to jobs with higher priorities. If jobs have the same priority, the system allocates resources based on the scheduling policy configured for level-2 quotas.
You cannot configure scheduling policies for level-2 quotas of the Interactive type.
From May 24, 2023, FIFO scheduling is used as the default scheduling policy for level-2 quotas of new projects in all regions. Fair scheduling is used as the default scheduling policy for level-2 quotas of projects created before May 24, 2023. For more information, see Service notices.

Overview

Item	Fair scheduling	FIFO scheduling
Scheduling characteristics	In scenarios where jobs have the same priority, resources are evenly allocated to all jobs that are submitted at the same time. In scenarios where jobs have different priorities, resources are evenly allocated to jobs with higher priorities. If remaining resources exist, the resources are then evenly allocated to jobs with lower priorities.	In scenarios where jobs have the same priority, resources are preferentially allocated to the jobs that are submitted earlier. In scenarios where jobs have different priorities, resources are preferentially allocated to jobs with higher priorities even if jobs with lower priorities are submitted earlier.
Use scenarios	Human-computer interaction scenario: Multiple data engineers submit jobs at the same time and they all want their jobs to start to run as early as possible. Multi-baseline dependency scenario: Multiple baselines depend on a quota group. Tasks for each baseline must be run at the same time. Otherwise, the output of the baseline job is affected.	One-task scenario: Multiple submitted jobs are considered as one task. You focus on only the throughput and final completion time of all jobs, rather than the completion time of a single job.
Advantages	You can run multiple jobs at the same time.	The average operation time of a single job is short.
Risks	The average operation time of a single job is long.	A large job may occupy most resources. As a result, other jobs in the same quota group are always queued for compiling. Note You can use Logview to check the status of a job. If the sub-status of the job is Waiting for cluster resource, the job is queued for compiling. To eliminate the risk, you can configure the maximum number of compute units (CUs) that can be consumed by a single job. For more information, see the "Configure quotas" section in Manage quotas in the new MaxCompute console.

策略特征

Configure a quota scheduling policy

Log on to the MaxCompute console, and select a region in the top navigation bar.
In the left-side navigation pane, click Quotas.
On the Quotas page, find the level-1 quota that you want to configure and click Quota Configuration in the Actions column.

On the Quota Plans tab of the Quota Configuration page, find the quota plan that you want to configure, and click Edit in the Actions column to configure the following parameter. You can also configure the following parameter in the Create Quota Plan dialog box.

Parameter	Description
Scheduling Policy	In MaxCompute, level-2 quotas can be scheduled by using the fair or FIFO scheduling policy. You must select a scheduling policy based on your business requirements.

Click OK. After the quota plan is configured, click Apply Immediately in the Actions column to enable the latest configuration.

References

For more information about quota settings, see Quota Management for computing resources (New version).