If the vCPUs or memory of Elastic Compute Service (ECS) instances in a node group cannot meet your business requirements, you can upgrade the instance configurations of the node group. This topic describes how to upgrade node configurations.

Prerequisites

An E-MapReduce (EMR) cluster is created. For more information, see Create a cluster.

Limits

  • You can upgrade the node configurations of only subscription clusters.
  • You cannot upgrade the configurations of ECS instances that belong to the big data instance families or instance families that use local SSDs.
  • You can only upgrade the configurations of ECS instances in a node group. You cannot downgrade the configurations of ECS instances in a node group.

Procedure

  1. Go to the Nodes tab.
    1. Log on to the EMR on ECS console.
    2. In the top navigation bar, select the region where your cluster resides and select a resource group based on your business requirements.
    3. On the EMR on ECS page, find the desired cluster and click Nodes in the Actions column.
  2. On the Nodes tab, find the desired node group, move the pointer over the more icon in the Actions column, and then select Upgrade Configuration.
  3. Upgrade node configurations.
    1. In the Upgrade Configuration dialog box, modify the parameters based on your business requirements. The following table describes the parameters.
      ParameterDescription
      Instance TypeSelect the instance specifications and instance type. You can select only an instance type that has higher specifications.
      Note The instance types that you can select are displayed in the EMR console.
      Rolling Restart
      • By default, Rolling Restart is turned off. This indicates that the system restarts all ECS instances in the desired node group at the same time.
      • If you turn on Rolling Restart, the system restarts an ECS instance only after the previous ECS instance is restarted and all big data services that are deployed on the previous instance are restored.
    2. Click OK.
      An order is generated after a short period of time.
    3. Complete the payment.
      The new instance type is displayed for the corresponding instance in the ECS console after you complete the payment but does not take effect until the upgrade process is complete.
      Important After you complete the payment, the system restarts the nodes in the current node group. This operation may affect online business. Proceed with caution.
  4. Optional:Modify the configurations of YARN.
    If YARN is deployed in your cluster, perform the following steps to modify the configurations of YARN to allow YARN to apply the new cluster resources:
    1. On the Services tab, find YARN and click Configure in the YARN section.
    2. Select Node Group Configuration from the Default Cluster Configuration drop-down list.
      Node Group Configuration
    3. Select the node group whose node configurations are upgraded from the drop-down list.
      Node group
    4. Modify vCPU configurations.
      Search for the yarn.nodemanager.resource.cpu-vcores parameter and change the value of the parameter based on your business requirements.

      In compute-intensive scenarios, we recommend that you set this parameter to the number of vCPUs of each ECS instance. In scenarios that involve compute-intensive and I/O intensive tasks, we recommend that you set this parameter to a value that ranges from the number of vCPUs of each ECS instance to twice this number.

      For example, if you use 32-vCPU ECS instances in compute-intensive scenarios, set the yarn.nodemanager.resource.cpu-vcores parameter to 32. If you use 32-vCPU ECS instances in scenarios that involve compute-intensive and I/O intensive tasks, set the yarn.nodemanager.resource.cpu-vcores parameter to a value that ranges from 32 to 64.

    5. Modify memory configurations.
      Search for the yarn.nodemanager.resource.memory-mb parameter, and change the value of the parameter to 80% of the memory size of each ECS instance. Unit: MiB.

      For example, if the memory size of each ECS instance is 32 GiB, set the yarn.nodemanager.resource.memory-mb parameter to 26214.

    6. Save the configurations.
      1. Click Save.
      2. In the dialog box that appears, configure the Execution Reason parameter and click Save.
    7. Issue the configurations.
      1. Choose More > Configure.
      2. In the dialog box that appears, configure the Execution Reason parameter and click OK.
      3. In the Confirm message, click OK.

        You can click Operation History in the upper-right corner to view the status of the task. After the task enters the Complete state, restart the YARN service.

    8. Restart the YARN service.
      1. Choose More > Restart.
      2. In the dialog box that appears, configure the Execution Reason parameter and click OK.
      3. In the Confirm message, click OK.

        You can click Operation History in the upper-right corner to view the status of the task. After the task enters the Complete state, the YARN service is restarted.

      Important A restart of the ResourceManager component may cause jobs to fail. We recommend that you restart the component during off-peak hours.