All Products
Search

This Product

    E-MapReduce:Node compensation

    Document Center

    E-MapReduce:Node compensation

    Last Updated:Feb 24, 2025

    After you enable the automatic compensation feature for existing or newly added node groups in a cluster, if E-MapReduce (EMR) detects that an Elastic Compute Service (ECS) instance in the cluster cannot support the normal operation of the engine services, EMR will automatically attempt to purchase a new ECS instance to replace the faulty one, ensuring service stability.

    Prerequisites

    A DataLake cluster, a Dataflow cluster, or a custom cluster is created, and the YARN service is deployed in the cluster. For more information, see Create a cluster.

    Limits

    Only pay-as-you-go task node groups support the automatic compensation feature.

    Precautions

    • Automatic compensation may fail in the following scenarios:

      • A system issue occurs on the ECS instances to be replaced.

      • Release protection is enabled for the ECS instances to be replaced.

      • The number of nodes in the node group to which the ECS instances to be replaced belong is less than or equal to the minimum number of nodes that must be reserved for the node group.

    • The bootstrap action script is automatically run on new ECS instances during the scale-out process. If the ECS instances to be replaced contain manually deployed components or services, you must redeploy the components or services on the new instances.

    Enable or disable automatic compensation

    1. Go to the Nodes tab.

      1. Log on to the EMR console. In the left-side navigation pane, click EMR on ECS.

      2. In the top navigation bar, select the region in which your cluster resides and select a resource group based on your business requirements.

      3. On the EMR on ECS page, find the cluster that you want to manage and click Nodes in the Actions column.

    2. On the Nodes tab, click the name of the node group that you want to manage in the Node Group Name / ID column.

    3. In the Node Group Attributes panel, turn on or turn off Automatic Compensation.

      Note

      From 18:00 (UTC+8) on July 10, 2023, Automatic Compensation is turned on for new pay-as-you-go task node groups by default.

    4. In the message that appears, click OK.

    5. Click Save.

    View the automatic compensation status

    On the Nodes tab, you can view the automatic compensation status in the Automatic Supplementation column.

    If you enable the automatic compensation feature and automatic compensation is performed on a node group within a week, the number of times automatic compensation is performed is displayed in the Automatic Supplementation column. You can move the pointer over the number and click View event details to view the details.