How to configure an ECS remote commands node - Data Management

An Elastic Compute Service (ECS) remote command node in a task flow allows you to run shell, PowerShell, or batch scripts on a remote ECS instance based on Cloud Assistant. This topic describes how to configure an ECS remote commands node.

Scenarios

Process data by using advanced tools, including the NumPy and scikit-learn libraries for Python, and the MLlib library of Apache Spark. The models generated after data processing can be applied to the fine sort and recommendation features of a search system.
Consume data. For example, you can generate an Excel script when you read data or generate a script that is used to automatically send emails that contain the data you read.
Call a self-built Hadoop MapReduce or Apache Spark program.

Prerequisites

An ECS instance is purchased. For more information, see Create an instance on the Custom Launch tab.
A tag is added to the ECS instance. The key of the tag is dms, and the value of the tag is script-for-dms. For more information, see Modify the tags of an instance.
The Cloud Assistant client is installed on the ECS instance. For more information, see Install the Cloud Assistant Agent.
Note
By default, ECS instances created from public images after December 1, 2017 are pre-installed with the Cloud Assistant client.
The InvokeCommand permission on ECS instances is granted to your RAM user. To grant the permission, perform the following operations:
1. Create a policy. You can create the policy by using the visual editor or a JSON script. For more information, see Create a custom policy. To grant the InvokeCommand permission, use the following script:
```
{
  "Version": "1",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "ecs:InvokeCommand"
      ],
      "Resource": [
        "*"
      ],
      "Condition": {}
    }
  ]
}
```
2. Attach the policy to your RAM user. For more information, see Grant permissions to the RAM user.

Procedure

Log on to the DMS console V5.0.
In the top navigation bar, choose DTS > Data Development > Task Orchestration.
Note
If the console interface is in simple mode, click in the upper-left corner and choose All functions. In the navigation pane that appears, choose DTS > Data Development > Task Orchestration.
Click the name of a task flow that you want to manage to go to the details page of the task flow.
Note
For more information about how to create a task flow, see Overview.
In the Task Type list on the left side of the canvas, drag the ECS remote commands node type to a blank area of the canvas.
Double-click the ECS remote commands node on the canvas.

On the configuration page of the ECS remote commands node, set parameters for the node.

Category	Parameter	Required	Description
Basic configuration	Region	Yes	The region in which the ECS instance resides.
	ECS Instance ID	Yes	The ID of the ECS instance. In the ECS Instance ID drop-down list, select the ECS instance on which you want to run commands. Note Only the ECS instances with a tag key of `dms` and a tag value of `script-for-dms` are displayed in the ECS Instance ID drop-down list. You can select multiple ECS instances that run the same operating system.
	Command Type	Yes	The type of the commands to run. If the ECS instance runs Linux, select Shell (Linux). If the ECS instance runs Windows, select Bat (Windows) or PowerShell (Windows).
	Timeout	Yes	The timeout period for running the specified commands. Unit: seconds.
	Working Directory	Yes	The working directory of the commands on the ECS instance. If you do not set this parameter, the default working directory is automatically selected. The default working directory is `/root` for an ECS instance that runs Linux and `C:\Windows\System32` for an ECS instance that runs Windows.
Advanced settings	User	Yes	The name of the user to run the commands on the ECS instance. Note If you want to specify a regular user rather than a root or system user, the specified user must be granted the required permissions. For more information, see Run Cloud Assistant commands as a regular user.
Advanced settings	Password Name for Windows	Yes	The password of the user to run the commands on the ECS instance that runs Windows. Note For more information, see Run Cloud Assistant commands as a regular user.
Command settings	Command Settings	Yes	The commands to be executed on the ECS instance. For example, `python /home/admin/hello.py`.

Configure output variables. This step is optional. In the downstream nodes, you can view and reference the output variables.
1. Click the Variable Setting tab in the right-side navigation pane.
2. Click the Output Variables tab.
3. Click Increase Variable.
4. In the Variable Name field, enter the name of the variable. For more information, see Variables.
Note
For each ECS instance that you select, if the returned value in the last row is in the JSON format, the variable in the value can be used as an output variable. For example, if {"a":"hello"} is returned for Instance A and {"b":"world"} is returned for Instance B, both the a and b variables can be used as output variables.
In the top toolbar of the configuration page, click Try Run.
- If status SUCCEEDED appears in the last line of the logs, the task flow is successfully run.
- If status FAILED appears in the last line of the logs, the task flow fails to be run. Click Log in the top toolbar to view the cause of the failure, modify the configurations accordingly, and run the task flow again.