Create and deploy an agent - Data Online Migration - Alibaba Cloud Documentation Center

This topic describes how to create and deploy an agent.

Create an agent

An agent supports two network types: the Internet and virtual private clouds (VPCs), such as leased lines or VPN gateways. You can specify an appropriate network type for an agent based on your business requirements. When you create an agent, you must associate the agent with an existing tunnel. Therefore, you must create a tunnel before you create the agent. For more information about how to create a tunnel, see Manage tunnels.

Important

You must create a tunnel and an agent if you want to migrate data to Alibaba Cloud over a leased line or VPN, or to migrate data from a self-managed storage system to Alibaba Cloud.
An agent must be deployed on a server that runs the 64-bit Linux operating system with a kernel version of 2.6 or later. Prepare a server that meets this requirement before you deploy an agent.
You can create up to 100 agents in each region. We recommend that you properly plan and use your agents.
We recommend that you purchase Elastic Compute Service (ECS) instances of the 7th or a next generation.

Log on to the Data Migration console.
In the left-side navigation pane, choose Data Online Migration > Agent Management.
On the Agent Management page, click New Agent.

In the New Agent dialog box, configure the parameters and click OK. The following table describes the parameters.

Parameter	Required	Description
Name	Yes	The name of the agent. The name must be 3 to 63 characters in length and cannot be empty. The name is case-sensitive and can contain lowercase letters, digits, hyphens (-), and underscores (_). The name must be encoded in UTF-8 and cannot start with a hyphen (-) or an underscore (_).
Network Type	Yes	The type of the network that is used to connect to the agent. Valid values: Internet: The agent migrates data over the Internet. If this network type is used, the agent must be able to access the public endpoint of Data Online Migration in the region in which you access the Data Migration console. For example, if you access the Data Migration console in the China (Beijing) region, the agent must be able to access the public endpoint cn-beijing.mgw-tc.aliyuncs.com of Data Online Migration in the China (Beijing) region. Leased Line/VPN (VPC): The agent migrates data over a VPC. If this network type is used, the agent must be able to access the internal endpoint of Data Online Migration in the region in which you access the Data Migration console. For example, if you access the Data Migration console in the China (Beijing) region, the agent must be able to access the internal endpoint cn-beijing.mgw-tc-internal.aliyuncs.com of Data Online Migration in the China (Beijing) region. We recommend that you use an ECS instance that resides in the same region in which you access the Data Migration console.
Deployment Method	Yes	The mode in which the agent is deployed. You can deploy an agent only in independent process mode.
Tunnel	Yes	The tunnel with which the agent is associated. You can associate an agent with only one tunnel. The bandwidth of the agent is affected by the total bandwidth of the tunnel. For example, if you associate a tunnel named tunnel-1 whose maximum bandwidth is 10 Gbit/s with three agents named agent-1, agent-2, and agent-3, the total bandwidth that is available for the three agents does not exceed 10 Gbit/s. If you specify a bandwidth of 3 Gbit/s for agent-1, only 7 Gbit/s of bandwidth is available for agent-2 and agent-3. You must plan and allocate bandwidth in advance with caution.

Generate the command to deploy an agent

Prerequisites

The server on which the agent is deployed has at least 4 CPU cores, 8 GB of memory, and 4 GB of disk space for logs.
We recommend that you create a Resource Access Management (RAM) user to deploy the agent. You must attach the AliyunOSSImportReadOnlyAccess policy to the RAM user, and set the authorization scope to Alibaba Cloud Account.
Note
An AccessKey pair of a RAM user can be used to deploy multiple agents without conflicts.

Procedure

On the Agent Management page, click the name of the agent that you want to deploy.

On the agent details page, click Deploy and Run. In the Deploy and Run dialog box, configure the parameters in the AccessKey Pairs section. The following table describes the parameters.

Parameter	Required	Description
AccessKeyId	Yes	The AccessKey pair of the RAM user that has the mgw:VerifyAgentTunnel permission. Note We recommend that you grant only the mgw:VerifyAgentTunnel permission to the RAM user. Alternatively, you can attach either the AliyunOSSImportReadOnlyAccess or AliyunOSSImportFullAccess policy to the RAM user.
SecretAccessKey	Yes

Click the icon next to Advanced Settings and configure the parameters. The following table describes the parameters.

Parameter	Required	Description
CPU Cores	No	The number of CPU cores that can be used by the agent. If you do not configure this parameter, all CPU cores of the server are used by the agent by default. When you configure this parameter, read the tips in the console and enter a value based on the actual situation.
Maximum Memory Size (GB)	No	The maximum size of memory that can be used by the agent. By default, if you do not configure this parameter, all the memory of the server is used by the agent. When you configure this parameter, read the tips in the console and enter a value based on the actual situation.
Space Used by Logs (GB)	No	The maximum disk space that can be used to store the logs generated by the agent. If the size of log files reaches the value that you specify for this parameter, the earliest logs are dropped. If you do not configure this parameter, 4 GB of disk space is used to store logs by default. When you configure this parameter, read the tips in the console and enter a value based on the number of files to be migrated. Approximately 4 GB of logs are generated if you migrate one million files.
Maximum Bandwidth	No	The maximum bandwidth that is available for the agent. The maximum bandwidth that is available for the agent is affected by the total bandwidth of the tunnel with which the agent is associated, and cannot exceed the maximum bandwidth of the tunnel. Example: If the maximum bandwidth of the tunnel is 5 Gbit/s, the maximum bandwidth that is available for the agent cannot exceed 5 Gbit/s. If the maximum bandwidth is not specified for the tunnel, the maximum bandwidth that is available for the agent can be up to 100 Gbit/s. Note You can associate an agent with only one tunnel. For example, if you associate a tunnel named tunnel-1 whose maximum bandwidth is 10 Gbit/s with three agents named agent-1, agent-2, and agent-3, the total bandwidth that is available for the three agents does not exceed 10 Gbit/s. If you specify a bandwidth of 3 Gbit/s for agent-1, only 7 Gbit/s of bandwidth is available for agent-2 and agent-3. You must plan and allocate bandwidth in advance with caution.

Click Use commands to deploy and run the agent.

Deploy an agent

Important

The deployment command that is automatically generated can be used to deploy only one agent. If you want to deploy multiple agents, you must generate a deployment command for each agent in turn and deploy the agents on different servers. Otherwise, the deployment fails or the agents fail to work as expected.
If the data is migrated between a local file system and Object Storage Service (OSS) or between local file systems, you must deploy the agent as the root user. Otherwise, data migration may fail because system commands such as chown and chmod cannot be run.

Deploy an agent online

If you set the Network Type parameter to Internet for an agent, deploy the agent online.

Log on to the server on which you want to deploy the agent. You can use a connection tool provided by Alibaba Cloud such as Workbench, Virtual Network Computing (VNC), or a third-party client to connect to the server.
Verify that the curl command can be run on the server. Run the curl -I https://www.aliyun.com command on the server to verify the connectivity to the Internet.

Copy the command that is displayed in the Use commands to deploy and run the agent panel and run the copied command on the server. After the command is successfully run, the agent is deployed. The following syntax and table describe the parameters in the command.

wget https://gosspublic.alicdn.com/data_online_migration/agent/aliyun_import_agent_deploy.sh -O aliyun_import_agent_deploy.sh;chmod 755 aliyun_import_agent_deploy.sh;./aliyun_import_agent_deploy.sh -t <TunnelID> -u <UID> -i <AK> -k <SK> -n <AgentName> -v <AgentID> -e <Endpoint> -f <AgentDeployMethod> -c <CpuUsage> -m <MemoryUsage> -d <LogMemoryUsage> -b <MaxBandwidth>

Parameter	Description
TunnelID	The ID of the tunnel.
UID	The ID of the Alibaba Cloud account.
AK	The AccessKey ID.
SK	The AccessKey secret.
AgentName	The name of the agent.
AgentID	The ID of the agent.
Endpoint	The endpoint. A value of public indicates that the network type is Internet.
AgentDeployMethod	The mode in which the agent is deployed. A value of default indicates the independent process mode.
CpuUsage	The number of CPU cores that can be used by the agent. This parameter is displayed only if the CPU Cores parameter is configured.
MemoryUsage	The size of memory that can be used by the agent. Unit: GB. This parameter is displayed only if the Maximum Memory Size (GB) parameter is configured.
LogMemoryUsage	The disk space that can be used to store the logs generated by the agent. Unit: GB. This parameter is displayed only if the Space Used by Logs (GB) parameter is configured.
MaxBandwidth	The maximum bandwidth that is available for the agent. Unit: MB. This parameter is displayed only if the Maximum Bandwidth parameter is configured.

After the agent is deployed, create a data address and associate the data address with the agent. The migration task that you create by using the data address migrates data on the specified agent.

Deploy an agent offline

If you set the Network Type parameter to Leased Line/VPN (VPC) for an agent, deploy the agent offline.

Log on to the server on which you want to deploy the agent. You can use a connection tool provided by Alibaba Cloud such as Workbench, VNC, or a third-party client to connect to the server.
Download the agent installation package aliyun_import_agent_offline_bundle.tar.gz and store the agent installation package on the server.
Find the agent installation package on the server and run the tar -zxvf aliyun_import_agent_offline_bundle.tar.gz command to decompress the agent installation package.

Go to the aliyun_import_agent_offline_bundle directory. Copy the command that is displayed in the Use commands to deploy and run the agent panel and run the copied command on the server. After the command is successfully run, the agent is deployed. The following syntax and table describe the parameters in the command.

chmod 755 aliyun_import_agent_deploy.sh;./aliyun_import_agent_deploy.sh -t <TunnelID> -u <UID> -i <AK> -k <SK> -n <AgentName> -v <AgentID> -e <Endpoint> -f <AgentDeployMethod> -c <CpuUsage> -m <MemoryUsage> -d <LogMemoryUsage> -b <MaxBandwidth> -l <VPCType>

Parameter	Description
TunnelID	The ID of the tunnel.
UID	The ID of the Alibaba Cloud account.
AK	The AccessKey ID.
SK	The AccessKey secret.
AgentName	The name of the agent.
AgentID	The ID of the agent.
Endpoint	The endpoint. A value of vpc indicates that the network type is Leased Line/VPN (VPC).
AgentDeployMethod	The mode in which the agent is deployed. A value of default indicates the independent process mode.
CpuUsage	The number of CPU cores that can be used by the agent. This parameter is displayed only if the CPU Cores parameter is configured.
MemoryUsage	The size of memory that can be used by the agent. Unit: GB. This parameter is displayed only if the Maximum Memory Size (GB) parameter is configured.
LogMemoryUsage	The disk space that can be used to store the logs generated by the agent. Unit: GB. This parameter is displayed only if the Space Used by Logs (GB) parameter is configured.
MaxBandwidth	The maximum bandwidth that is available for the agent. Unit: MB. This parameter is displayed only if the Maximum Bandwidth parameter is configured.
VPCType	The type of the VPC. This parameter is displayed only if the Network Type parameter is set to Leased Line/VPN (VPC).

After the agent is deployed, create a data address and associate the data address with the agent. The migration task that you create by using the data address migrates data on the specified agent.