All Products
Search

This Product

    Realtime Compute for Apache Flink:Manage queues

    Document Center

    Realtime Compute for Apache Flink:Manage queues

    Last Updated:Oct 25, 2024

    Each Realtime Compute for ApacheFlink namespace has a default queue named default-queue to run deployments. If you want to isolate and manage resources, you can add queues. This topic describes how to add, edit, and delete a queue.

    Scenarios and methods

    You can configure queues to isolate and manage resources for batch deployments. Queues help prevent a deployment or a user from occupying excessive resources, which may result in execution failures due to insufficient resources for other deployments or users. You can deploy batch drafts in a specific queue and use the Workflows feature to specify scheduling priorities of the batch drafts to ensure that critical or important deployments have higher priorities and more resources. This improves system performance and resource utilization.

    You can use a queue by using the following methods:

    • Set the Deployment Target parameter to the queue that you want to use in the Deploy draft dialog box when you deploy a draft. For more information, see Create a deployment.

      Note

      If you do not add a queue, the default queue named default-queue is used to run deployments.

    • Set the Deployment Target parameter to the queue that you want to use in the Standard section of the Create Session Cluster page when you create a session cluster. For more information, see Step 1: Create a session cluster.

    • Set the Resource Queue parameter to the queue that you want to use in the Create Workflow panel when you create a workflow. For more information, see Manage workflows.

    Limits

    Queues can run only in exclusive mode. Queues cannot run in preemption mode. If a queue runs in preemption mode, the resources in the queue cannot be preempted. After specific resources in the queue are released, the resources can be used by a new deployment.

    Add a queue

    Note

    You can add queues to isolate and manage resources. By default, the default queue named default-queue is used to run deployments if you do not add a queue.

    1. Log on to the management console of Realtime Compute for Apache Flink.

    2. Find the desired workspace and click Console in the Actions column.

    3. In the left-side navigation pane, choose O&M > Queue Management.

    4. In the upper-left corner of the Queue Management page, click Add queue.

      image.png

    5. In the Add Queue dialog box, configure the Queue Name and Resources (CU) parameters. The following table describes the parameters.

      6. Parameter

      Description

      Queue Name

      The name of the queue. The queue name must be unique in a namespace. The name must be 1 to 32 characters in length and can contain only lowercase letters, digits, and hyphens (-).

      Note

      After a queue is added, the queue name cannot be modified.

      Resources (CU)

      The number of compute units (CUs) that can be used in the queue. The maximum value of this parameter is determined based on the remaining available resources in the namespace. If the available resources in the namespace are insufficient, you can use one of the following methods to adjust the number of resources:

    6. Click OK.

    Reconfigure the number of CUs that can be used in a queue

    1. Log on to the management console of Realtime Compute for Apache Flink.

    2. Find the desired workspace and click Console in the Actions column.

    3. In the left-side navigation pane, choose O&M > Queue Management.

    4. Find the desired queue and click Edit queue in the Actions column. In the Edit Queue dialog box, change the value of the Resources (CU) parameter.

      Note

      • For more information about the Resources (CU) parameter, see Add a queue.

      • You cannot set the Resources (CU) parameter to a value that is less than the number of CUs that have been used in the queue.

    5. Click OK.

    Delete a queue

    Important

    • You cannot delete the default queue named default-queue.

    • A queue in which a deployment runs cannot be deleted.

    • If you delete a queue in which a deployment runs, an error is reported when you restart the deployment.

    1. Log on to the management console of Realtime Compute for Apache Flink.

    2. Find the desired workspace and click Console in the Actions column.

    3. In the left-side navigation pane, choose O&M > Queue Management.

    4. Find the desired queue and click Delete in the Actions column.

    5. In the message that appears, click OK.

    References

    • For more information about how to reconfigure the resources of a namespace or change the upper limit of the quota for a namespace, see Reconfigure resources.

    • For more information about how to create a deployment of the SQL, JAR, or Python type, see Create a deployment.

    • For more information about how to create a session cluster, see Step 1: Create a session cluster.

    • For more information about the Workflows feature that is used to specify the scheduling priorities of batch deployments, see Workflows (public preview).