You can use resource queues to manage and schedule cluster resources. This helps you allocate and optimize resources in a flexible manner. In the Alibaba Cloud E-MapReduce (EMR) console, you can manage resource queues and associate resource queues with different partitions.

Prerequisites

A cluster is created and is in the Running state. For more information, see Create a cluster.

Limits

You must use Capacity Scheduler as the YARN scheduler.
Custom partition-relevant features are available for EMR V5.11.1, EMR V3.45.1, or a minor version later than EMR V5.11.1 or EMR V3.45.1.

Associate a resource queue with a YARN partition

Important

This feature is available for EMR V5.12.1, EMR V3.46.1, or a minor version later than EMR V5.12.1 or EMR V3.46.1.
If partition-queue association management is disabled, all queues are disassociated from the current partition.

Go to the Configure tab of the YARN service page.
1. Log on to the EMR console. In the left-side navigation pane, click EMR on ECS.
2. In the top navigation bar, select the region in which your cluster resides and select a resource group based on your business requirements.
3. On the EMR on ECS page, find the cluster that you want to manage and click Services in the Actions column.
4. On the Services tab, click Configure in the YARN section.
Click the Edit Resource Queue tab.
Select an existing partition from the Select Partition drop-down list.
By default, the DEFAULT_PARTITION partition is displayed. All resource queues can access this partition. For information about how to add a partition, see Manage YARN partitions in the EMR console.
Turn on Enable Partition-queue Association Management.
Resource queues that can access the current partition or that have capacity in the current partition are displayed.
In the Enable Partition-queue Association Management dialog box, configure the Queue Capacity parameter and click OK.
Make the configuration take effect.
1. Click the icon in the upper part of the Edit Resource Queue tab.
2. In the dialog box that appears, click Deliver.
3. Click the icon in the upper part of the Edit Resource Queue tab.
4. In the dialog box that appears, find the desired component and click the refresh_queues operation in the Actions column for the configuration to take effect.

Add a resource queue

Important

Before you add a child queue for a leaf queue, you must stop the leaf queue first. Otherwise, the child queue fails to be added.

Optional. On the Edit Resource Queue tab of the YARN service page, select an existing partition from the Select Partition drop-down list.
By default, the DEFAULT_PARTITION partition is displayed.
Click Manage Queues. Alternatively, click Manage Child Queues in the Actions column of the queue that you want to manage.
In the Manage Child Queues dialog box, select a parent queue from the Select Parent Queue drop-down list.
By default, the root queue is displayed.
Click Add Queue.
Configure the Queue Name, Queue Capacity, and Status parameters and click OK.
The sum of capacities of all child queues in a parent queue must be 100%.
Make the configuration take effect.
1. Click the icon in the upper part of the Edit Resource Queue tab.
2. In the dialog box that appears, click Deliver.
3. Click the icon in the upper part of the Edit Resource Queue tab.
4. In the dialog box that appears, find the desired component and click the refresh_queues operation in the Actions column for the configuration to take effect.

Modify a resource queue

Optional. On the Edit Resource Queue tab of the YARN service page, select an existing partition from the Select Partition drop-down list.
By default, the DEFAULT_PARTITION partition is displayed.
Click the icon on the left side of the name of the root queue.
Find the resource queue that you want to manage and click Edit in the Actions column.

In the Edit dialog box, modify the parameters and click OK. The following table describes the parameters.

Parameter	Description
Current Queue Name	The name of the resource queue. You can specify a custom name. You can modify this parameter only in the DEFAULT_PARTITION partition.
Minimum User Capacity	The proportion of resources that can be used by a single user in the cluster. This parameter corresponds to the `Configured Minimum User Limit Percent` parameter on the YARN web UI. You can modify this parameter only in the DEFAULT_PARTITION partition.
User Limit Factor	The ratio of the maximum amount of resources that can be used by a single user to the capacity of the resource queue. This parameter corresponds to the `Configured user limit factor` parameter on the YARN web UI. You can modify this parameter only in the DEFAULT_PARTITION partition.
Resource Queue Status	The status of the resource queue. Valid values: Started and Stopped. You can modify this parameter only in the DEFAULT_PARTITION partition. Note Before you can set the status of a parent queue to Stopped, you must stop all its child queues.
Accessible Partitions	The partitions that can be accessed by the resource queue. Note By default, all resource queues can access the DEFAULT_PARTITION partition in EMR V5.12.1, EMR V3.46.1, or a minor version later than EMR V5.12.1 or EMR V3.46.1. A child queue inherits the accessible partitions of its parent queue. You can also separately configure accessible partitions for a child queue.
Default Partition	The default partition for the resource queue. If you do not configure this parameter, the DEFAULT_PARTITION partition is used as the default partition.

Make the configuration take effect.
1. Click the icon in the upper part of the Edit Resource Queue tab.
2. In the dialog box that appears, click Deliver.
3. Click the icon in the upper part of the Edit Resource Queue tab.
4. In the dialog box that appears, find the desired component and click the refresh_queues operation in the Actions column for the configuration to take effect.

Delete a resource queue

Important

Only resource queues that are in the Stopped state can be deleted.
Before you delete a resource queue, make sure that the capacity of the resource queue in all partitions is 0.
The sum of capacities of all child queues in a parent queue must be 100%.
If you use YARN 2.8.5, the refresh_queues operation does not take effect when you delete a resource queue. After you delete a resource queue, you must restart ResourceManager on the Status tab of the YARN service page for the configuration to take effect.

Optional. On the Edit Resource Queue tab of the YARN service page, select an existing partition from the Select Partition drop-down list.
By default, the DEFAULT_PARTITION partition is displayed.
Click Manage Queues.
In the Manage Child Queues dialog box, change the status of the resource queue that you want to delete to Stopped, change the value of the Queue Capacity parameter to 0 for the resource queue, and then click OK.
Make the configuration take effect.
1. Click the icon in the upper part of the Edit Resource Queue tab.
2. In the dialog box that appears, click Deliver.
3. Click the icon in the upper part of the Edit Resource Queue tab.
4. In the dialog box that appears, find the desired component and click the refresh_queues operation in the Actions column for the configuration to take effect.
Click Manage Queues again.
In the Manage Child Queues dialog box, click Delete in the Actions column of the resource queue that you want to delete. Then, click OK.
Make the configuration take effect.
1. Click the icon in the upper part of the Edit Resource Queue tab.
2. In the dialog box that appears, click Deliver.
3. Click the icon in the upper part of the Edit Resource Queue tab.
4. In the dialog box that appears, find the desired component and click the refresh_queues operation in the Actions column for the configuration to take effect.