All Products
Search
Document Center

MaxCompute:Manage quotas for computing resources in the MaxCompute console

Last Updated:Dec 02, 2024

You can use the quota management feature to perform routine resource management and maintenance in the MaxCompute console. For example, you can view computing resource consumption, adjust computing resources, and perform create, read, update, and delete (CRUD) operations on quota configurations. This topic describes various capabilities that are provided by the quota management feature.

Background information

A quota is a unit of computing resources in MaxCompute. Quotas provide the computing resources (vCPUs and memory resources) that are required for computing jobs such as MaxCompute SQL jobs, MapReduce jobs, Spark jobs, Mars jobs, and Platform for AI (PAI) jobs. MaxCompute quotas are classified into level-1 quotas and level-2 quotas. Level-1 quotas are independent of each other. Level-2 quotas are subordinate to level-1 quotas. You can share resources among level-2 quotas. You cannot directly use resources in level-1 quotas because level-1 quotas are used only to manage level-2 quotas. You must use level-2 quotas to consume compute units (CUs) in level-1 quotas.

CUs are objects in a quota and are used as MaxCompute computing resources. CUs in quotas that you can purchase are classified into subscription CUs and pay-as-you-go CUs. Subscription CUs correspond to the subscription resource package and pay-as-you-go CUs correspond to the pay-as-you-go resource package.

  • A subscription quota is an exclusive quota that is created and allocated by the system after you purchase reserved CUs based on the monthly subscription period. When you use a subscription quota, you can pre-purchase elastically reserved CUs on a regular basis to meet your requirements. Elastically reserved CUs are pay-as-you-go resources and are charged based on the number of CUs that you purchased and the usage duration. If you run a job in a subscription quota, the job can use the maximum number of CUs specified in the quota. You can configure level-2 quotas for the level-1 quota to allocate CUs.

  • Pay-as-you-go quotas are common level-2 quotas that are allocated by the system when you activate the pay-as-you-go MaxCompute service. You cannot change the number of CUs or reallocate CUs in pay-as-you-go quotas.

You can choose to use pay-as-you-go quotas or subscription quotas based on your business requirements. Subscription quotas allow you to change the number of CUs to manage budgets. Pay-as-you-go quotas do not allow you to change the number of CUs.

View quotas

  1. Log on to the MaxCompute console. In the top navigation bar, select a region.

  2. In the left-side navigation pane, choose Workspace > Quotas.

  3. On the Quotas page, view the list of available quotas.

    The list displays the level-1 and level-2 subscription quotas and the default pay-as-you-go quota in the tree structure. By default, you can view the quota list by using an Alibaba Cloud account or as a RAM user.

    • Subscription quotas: Click the 列表 icon on the left of a level-1 quota to display its level-2 quotas and the effective configurations of the level-2 quotas. After you purchase subscription resources, the system automatically creates and allocates a level-1 quota and a level-2 quota named Default Subscription Quota.

    • Pay-as-you-go quotas: Click the 列表.png icon on the left of a level-1 quota to display its level-2 quotas. After you activate the Pay-as-you-go Standard Edition, the system automatically creates and allocates a level-1 quota named Default Pay-as-you-go Quota_p and a level-2 quota named Default Pay-as-you-go Quota.

      Note

      For some pay-as-you-go MaxCompute services, the default name of a level-2 quota is aliyun_nick. In regions outside the Chinese mainland, the default level-1 quota name of the pay-as-you-go service may not be properly displayed. This issue does not affect the use of level-2 quotas that belong to the level-1 quota.

Configure quotas

After you purchase subscription quotas, you can further configure subscription quotas. By default, an Alibaba Cloud account is authorized to configure level-2 subscription quotas. You can also configure level-2 subscription quotas as a RAM user after the RAM user is granted related permissions. For more information about permissions, see RAM permissions. You can configure quotas on the following tabs of the Quotas page.

  • Basic Configurations tab: On this tab, you can add or delete level-2 quotas and configure basic parameters, reserved CUs, and elastically reserved CUs. The basic parameters include Exclusive or Not, Maximum Number of CUs for a Single Job, Enable Priority, Non-reserved CUs, and Scheduling Policy. The configurations of reserved CUs and elastically reserved CUs apply only to the current effective configuration plan. If time-based scaling is not required, all quota management operations can be performed on this tab.

    Note

    Each level-1 quota supports up to 100 level-2 quotas. You need to plan and configure them reasonably based on your business scenario.

  • Scaling Configuration tab: On this tab, you can configure time-based scaling to specify the minimum and maximum numbers of reserved CUs or the number of elastically reserved CUs that can be scheduled in quotas during different periods.

Procedure:

  1. Log on to the MaxCompute console. In the top navigation bar, select a region.

  2. In the left-side navigation pane, choose Workspace > Quotas.

  3. On the Quotas page, find the level-1 quota that you want to configure and click Quota Configuration in the Actions column.

  4. Configure settings for a new level-2 quota on the Basic Configurations tab.

    1. On the Basic Configurations tab of the Quota Configuration page, click Edit Basic Configurations to add a level-2 quota and configure the basic parameters, including Exclusive or Not, Maximum Number of CUs for a Single Job, Enable Priority, Non-reserved CUs, and Scheduling Policy.

      On the Basic Configurations tab, you can also click Delete in the Actions column of a level-2 quota to delete the quota.

      Note

      After you delete a level-2 quota, the quota is deleted from all the quota resource configuration plans. The numbers of CUs specified by minCU and Elastically Reserved CUs in the deleted quota are automatically added to the default level-2 quota. To modify CU configurations, you must modify and update each quota resource configuration plan on the Scaling Configuration tab.

    2. Click Edit Basic Configurations and configure parameters for the level-2 quota. The following table describes the parameters.

      Parameter

      Description

      Quota Name

      The name of the level-2 quota that you want to add.

      Click Add Level-2 Quota, enter a quota name, and select a type. The quota name is a custom name that must start with a letter and can contain letters, digits, and underscores (_).

      Type

      Select a quota type based on your business requirements. Valid values:

      • Batch: Quotas of this type are used to run batch jobs.

      • Interactive: Quotas of this type are used to run MaxCompute Query Acceleration (MCQA) jobs. You do not need to configure this type of quota as the default quota for computing resources. If the default quota for computing resources of your project is the subscription quota for batch jobs, MCQA jobs are automatically scheduled to a quota of the Interactive type. If multiple quotas of the Interactive type are available, MCQA jobs are automatically scheduled to the first quota of this type that you create. In this case, you can configure quota rules to schedule MCQA jobs initiated by different projects to different quotas of the Interactive type. For more information about quota rules, see the "Quota rules" section in Use of computing resources.

      Reserved CUs [minCU,maxCU]

      The number of reserved CUs in a subscription quota. The number of reserved CUs in a level-1 quota is equal to the number of CUs in the subscription quota that you purchased. In a subscription quota, the value of minCU is the same as the value of maxCU. To increase or decrease the number of reserved CUs, you must use the upgrade or downgrade method. This parameter cannot be modified. Descriptions of minCU and maxCU in a level-2 quota:

      • minCU specifies the lower limit, and maxCU specifies the upper limit. If the value of maxCU is greater than the value of minCU, CUs in level-2 quotas are shared among the jobs that use the level-2 quotas. If the number of CUs that are required is greater than the value of minCU in a level-2 quota and non-exclusive CUs in other quotas are idle, the non-exclusive CUs can be preempted.

      • The total value of minCU in all the level-2 quotas is equal to the value of minCU in the level-1 quota.

      • For a level-2 quota, the value of minCU must be less than or equal to the value of maxCU in the level-2 quota and less than or equal to the value of minCU in the level-1 quota that you purchased.

        Note

        You cannot set minCU to 0.

      • The values of minCU and maxCU of the default level-2 quota cannot be changed. The value of minCU of the default level-2 quota is equal to the value of minCU of the level-1 quota subtracted by the total value of minCU of all the custom level-2 quotas. The value of maxCU of the default level-2 quota is the same as the value of maxCU of the level-1 quota.

      • If multiple scaling configuration plans exist for a new quota, the quota is added to all quota resource configuration plans. By default, the value of minCU in Reserved CUs [minCU,maxCU] is 1 for all quota resource configuration plans, except for the current effective configuration plan. The number of CUs specified by minCU is subtracted from the default subscription level-2 quota. Therefore, if you need to modify CU configurations, modify and update each quota resource configuration plan on the Scaling Configuration tab.

        Important

        For a new level-2 quota, the parameter setting applies only to the current effective configuration plan.

      Elastically Reserved CUs

      The number of elastically reserved CUs that you pre-purchase on a regular basis. For more information, see Computing pricing (pay-by-the-hour).

      • The number of elastically reserved CUs in a level-1 quota is equal to the number of CUs that you pre-purchase. The default value is 0. The maximum value of this parameter must be equal to the number of subscription-based reserved CUs and cannot exceed 10,000 CUs. You must check inventory data when you configure elastically reserved CUs. The minimum number of elastically reserved CUs is 50 and the step size is 50.

      • Descriptions of the number of elastically reserved CUs in a level-2 quota

        • The total number of elastically reserved CUs in all the level-2 quotas is equal to the number of elastically reserved CUs in the level-1 quota.

        • Elastically reserved CUs in the default level-2 quota cannot be changed. The number of elastically reserved CUs in the default level-2 quota is equal to the number of elastically reserved CUs in the level-1 quota subtracted by the total number of elastically reserved CUs in all the custom level-2 quotas.

      • If multiple scaling configuration plans exist for a new quota, the quota is added to all quota resource configuration plans. By default, the value of Elastically Reserved CUs is 0 for all quota resource configuration plans, except for the current effective configuration plan. Therefore, if you need to modify CU configurations, modify and update each quota resource configuration plan on the Scaling Configuration tab.

        Important

        For a new level-2 quota, the parameter setting applies only to the current effective configuration plan.

      Exclusive or Not

      Specifies whether the quota is strongly exclusive. Even if CUs in an exclusive quota are idle, they are not used by other quota groups. This parameter is valid only for level-2 quotas. If the type of a level-2 quota is Interactive, the quota is automatically exclusive and cannot be changed.

      You can configure an exclusive quota for services such as business intelligence (BI) and ALGO because these services may always use CUs. If some CUs in a quota are idle, the idle CUs may be occupied by jobs that use another quota in which the value of maxCU is greater than the value of minCU. To prevent CUs in a quota from being occupied by jobs that use other quotas for a long period of time, you can configure an exclusive quota.

      Maximum Number of CUs for a Single Job

      The upper limit for CUs that can be concurrently used by a job scheduled to the quota. If this parameter is left empty, no upper limit is imposed. If this parameter is set to 0, concurrent use of CUs is not allowed. You can enter a positive integer as the upper limit. If the type of a level-2 quota is Interactive, this parameter is not available by default.

      You can configure this parameter to prevent jobs from waiting for resource allocation if a single job occupies a large number of CUs for a long period of time. You can also specify the upper limit at the job level by adding the set odps.task.max.concurrent.cu=<Number of CUs>; command before the code that you want to execute. The job-level configuration takes precedence over the quota-level configuration.

      Enable Priority

      Specifies whether to enable the priority feature. For more information, see Job priority. If the priority feature is enabled for a quota, the system preferentially runs the jobs that use the quota. You can also enable the priority feature at the project level. If the type of a level-2 quota is Interactive, this parameter is unavailable.

      Non-reserved CUs

      The number of non-reserved CUs in a subscription quota.

      • The number of non-reserved CUs in a level-1 quota is equal to the number of non-reserved CUs in the subscription quota. To increase or decrease the number of non-reserved CUs, you must use the upgrade or downgrade method. You cannot increase the number of non-reserved CUs when resources are insufficient. This parameter cannot be modified.

      • Descriptions of the number of non-reserved CUs in a level-2 quota

        • The total number of non-reserved CUs in all the level-2 quotas is equal to the number of non-reserved CUs in the level-1 quota.

        • Non-reserved CUs in the default level-2 quota cannot be modified. The number of non-reserved CUs in the default level-2 quota is equal to the number of non-reserved CUs in the level-1 quota subtracted by the total number of non-reserved CUs in all the custom level-2 quotas.

      Scheduling Policy

      In MaxCompute, level-2 quotas can be scheduled by using the fair or FIFO scheduling policy.

      • FAIR

        • In scenarios where jobs have the same priority, resources are evenly allocated to all jobs that are submitted at the same time.

        • In scenarios where jobs have different priorities, resources are evenly allocated to jobs with higher priorities. If remaining resources still exist after the allocation, the remaining resources are evenly allocated to jobs with lower priorities.

      • FIFO

        • In scenarios where jobs have the same priority, resources are preferentially allocated to the jobs that are submitted earlier.

        • In scenarios where jobs have different priorities, resources are preferentially allocated to jobs with higher priorities even if jobs with lower priorities are submitted earlier.

      You can select an scheduling policy based on your business requirements. You cannot configure this parameter for level-2 quotas whose Type is Interactive. For more information, see Quota scheduling policies for computing resources.

      Important

      From May 24, 2023, FIFO is used as the default scheduling policy for new level-2 quotas of projects in the following regions: China (Hangzhou), China (Shanghai), China (Beijing), China (Zhangjiakou), China (Ulanqab), China (Shenzhen), and China (Chengdu). For the level-2 quotas that are created in the preceding regions before May 24, 2023, FAIR is used as the default scheduling policy for level-2 quotas.

    3. Click OK.

  5. Configure scaling configurations for the quota.

    After you purchase a subscription MaxCompute project, each level-1 quota has a default resource configuration plan named Default. Each resource configuration plan lists the numbers of reserved CUs and elastically reserved CUs in a level-1 quota and the numbers of reserved CUs and elastically reserved CUs in all the level-2 quotas of the level-1 quota. If you want to use different CU configurations during different periods in a day, you can use multiple resource configuration plans together with the time-based management feature. For example, the requirements for CU configurations in a level-2 quota are different during the two periods every day: 00:00 to 08:00 and 08:00 to 24:00. In this case, you can configure two resource configuration plans for the quota to schedule the configuration plans based on the time-based management feature.

    Note

    You can add up to 48 resource configuration plans for a level-2 quota. Each plan has different configurations. This allows you to enable different plans during different periods every day.

    1. On the Scaling Configuration tab of the Quota Configuration page, click Add Configuration Plan.

    2. On the Add Configuration Plan page, configure parameters for the quota. The following table describes the parameters.

      Parameter

      Description

      Reserved CUs [minCU,maxCU]

      The number of reserved CUs in a subscription quota. The number of reserved CUs in a level-1 quota is equal to the number of CUs in the subscription quota that you purchased. In a subscription quota, the value of minCU is the same as the value of maxCU. To increase or decrease the number of reserved CUs, you must use the upgrade or downgrade method. This parameter cannot be modified. Descriptions of minCU and maxCU in a level-2 quota:

      • minCU specifies the lower limit, and maxCU specifies the upper limit. If the value of maxCU is greater than the value of minCU, CUs in level-2 quotas are shared among the jobs that use the level-2 quotas. If the number of CUs that are required is greater than the value of minCU in a level-2 quota and non-exclusive CUs in other quotas are idle, the non-exclusive CUs can be preempted.

      • The total value of minCU in all the level-2 quotas is equal to the value of minCU in the level-1 quota.

      • For a level-2 quota, the value of minCU must be less than or equal to the value of maxCU in the level-2 quota and less than or equal to the value of minCU in the level-1 quota that you purchased.

        Note

        You cannot set minCU to 0.

      • The values of minCU and maxCU of the default level-2 quota cannot be changed. The value of minCU of the default level-2 quota is equal to the value of minCU of the level-1 quota subtracted by the total value of minCU of all the custom level-2 quotas. The value of maxCU of the default level-2 quota is the same as the value of maxCU of the level-1 quota.

      Elastically Reserved CUs

      The number of elastically reserved CUs that you pre-purchase on a regular basis.

      • The number of elastically reserved CUs in a level-1 quota is equal to the number of CUs that you pre-purchase. The default value is 0. The maximum value of this parameter must be equal to the number of subscription-based reserved CUs and cannot exceed 10,000 CUs. You must check inventory data when you configure elastically reserved CUs. The minimum number of elastically reserved CUs is 50 and the step size is 50.

      • Descriptions of the number of elastically reserved CUs in a level-2 quota

        • The total number of elastically reserved CUs in all the level-2 quotas is equal to the number of elastically reserved CUs in the level-1 quota.

        • Elastically reserved CUs in the default level-2 quota cannot be changed. The number of elastically reserved CUs in the default level-2 quota is equal to the number of elastically reserved CUs in the level-1 quota subtracted by the total number of elastically reserved CUs in all the custom level-2 quotas.

    3. Click OK.

    4. On the Scaling Configuration tab, click Apply Immediately in the Actions column of the new configuration plan to make the plan take effect. You can also configure to use the new configuration plan in the Scheduled Scaling Management section.

    5. Optional. Perform operations related to the resource configuration plan.

      • View the configuration plan: Click the image icon on the left of the configuration plan name to show the plan details and view the specific information of each plan.

      • Modify the configuration plan: Click Edit in the Actions column of the plan to update the plan. If the plan is in effect, you must click Apply Immediately in the Actions column to enable the latest plan after the modification.

        Note

        You can click Apply Immediately for any configuration plan. If you configure the time-based management feature, the configuration plan may be overwritten by the configuration plan that is scheduled in the next time plan. Therefore, if you want to use a configuration plan during all periods every day, you must delete other time-based settings and retain the configuration plan. This ensures that the configuration plan immediately takes effect and is not overwritten by other configuration plans.

      • Delete the configuration plan: On the Scaling Configuration tab, click Delete in the Actions column of the configuration plan.

        Note

        The Default configuration plan and the current effective configuration plan cannot be deleted.

      • Clone the configuration plan: On the Scaling Configuration tab, click image in the Actions column of the configuration plan. In the list that appears, select Clone. For the Default configuration plan, you can directly click Clone in the Actions column.

    6. Configure settings related to time-based management.

      You can configure to enable different quota resource configuration plans during different periods every day. This way, you can implement time-based management of quota resource configuration plans.

      Note

      Time-based scheduling has a specific latency, which is expected to be less than 5 minutes.

      1. On the Scaling Configuration tab of the Quota Configuration page, click Edit Time Plan in the Scheduled Scaling Management section.

      2. Click Add Effective Period, and specify Effective Start Time and Configuration Plan.

        • Time plans are considered as a whole. The default start time of a time plan is 00:00, which cannot be changed or deleted. The quota resource configuration plan that corresponds to the time plan is the Default plan and can be modified. If no periods are added, a quota resource configuration plan is enabled for an entire day.

        • You can enable different configuration plans on the hour or the half hour. A configuration plan can be repeatedly used.

    7. Preview time-based scaling: You can select the level-2 quota and view the trend of CU allocation in all periods in a chart.

    8. Display a global view for scaling configurations: You can view the details of all quota resource configuration plans in a table. This allows you to view the minimum and maximum numbers of CUs of all quotas and the number of elastically reserved CUs scheduled in different time periods.

View quota consumption

You can view the consumption of CU resources in a level-1 quota.

Note

By default, you can perform this operation with your Alibaba Cloud account or as a RAM user.

  1. Log on to the MaxCompute console. In the top navigation bar, select a region.

  2. In the left-side navigation pane, choose Workspace > Quotas.

  3. On the Quotas page, find the destination level-1 quota, and click Configure Resources in the Actions column.

  4. On the page that appears, you can view the CPU resource consumption trend chart, quotas, and associated projects.

    • CPU resource consumption trend chart: allows you to select level-2 quotas and time ranges to view the trend of CU usage. Click a time point to view the job snapshot list that corresponds to the time point.

    • Quotas and associated projects: allow you to identify the projects that define a level-2 quota as the default quota.

Configure quota rules

You can configure quota rules in MaxCompute. After you configure a quota rule, jobs that meet the rule can be scheduled to consume the related quota. For more information about quota rules, see the "Quota rules" section in Use of computing resources.

Manage quota tags

You can add tags to or remove tags from level-1 subscription quotas. For more information about how to use a tag and the limits on using a tag, see What is Tag?.

  1. Log on to the MaxCompute console. In the top navigation bar, select a region.

  2. In the left-side navigation pane, choose Workspace > Quotas.

  3. Add a tag.

    • Add a tag for a level-1 quota.

      1. Move the pointer over the 编辑 icon in the Tag column of the level-1 quota, and click Edit.

      2. In the Configure Tags dialog box, configure the Tag Key and Tag Value parameters.

      3. Click OK. In the Configure Tags successfully message, click Close.

    • Add tags for multiple level-1 quotas at a time.

      1. Select the level-1 quotas to which you want to add tags and click Batch Add Tag in the lower part of the page.

      2. In the Configure Tags dialog box, configure the Tag Key and Tag Value parameters.

      3. Click OK. In the Configure Tags successfully message, click Close.

  4. Filter quotas by tag.

    After you add tags for level-1 quotas, click Filter by Tag and select a tag key and a tag value to filter quotas.

  5. Optional. Remove tags from level-1 quotas.

    • Remove a tag from a level-1 quota.

      1. Move the pointer over the 编辑 icon in the Tag column of the level-1 quota, and click Edit.

      2. In the Configure Tags dialog box, click the 删除 icon next to the tag that you want to remove.

      3. Click OK. In the Configure Tags successfully message, click Close.

    • Remove tags from multiple level-1 quotas at a time.

      1. Select the level-1 quotas from which you want to remove tags and click Batch Remove Tag in the lower part of the page.

      2. In the Delete Tags for Multiple Resources dialog box, select the tags that you want to remove.

      3. Click Unbind x tags. x specifies the number of tags that you want to remove. In the Configure Tags successfully message, click Close.