To add purchased ECS instances to an ACK cluster as worker nodes or to re-add removed worker nodes to a node pool, you can use the Container Service console to add nodes automatically or manually. Once added, the nodes will adopt the node pool's configuration by default and will be uniformly managed at the node pool level.
Limits
Limits on instance types
Limit item | Description | Recommended solution |
GPU-accelerated ECS Bare Metal instance | GPU-accelerated ECS Bare Metal instances (ECS instance families ebmgn7 and ebmgn7e) do not support automatic MIG cleanup. Therefore, when ACK adds nodes of this type, the existing MIG settings are reset. The reset time is uncertain, which may cause automatic node addition to fail due to prolonged reset time. | For troubleshooting failure causes, see What to do if adding bare metal instance nodes fails?. |
Quota limits
Limit item | Description | Recommended solution |
Node quota | Ensure that your cluster node quota is sufficient. For more information about the quota limits of ACK clusters, see Quotas and Limits. | To add more nodes, go to Quota Center to adjust. |
Limits on ECS instances
Limit item | Description | Recommended solution |
Account, region, and VPC | Only ECS instances under the same account are supported, and the added ECS instances must be in the same region and VPC as the cluster. | Recreate ECS instances or clusters that meet the requirements. |
Cluster | ECS instances that already belong to other clusters cannot be added. | If ECS instances already belong to other clusters, remove the nodes from the old cluster before adding them to the target cluster. For removal methods, see Remove Nodes. |
Network limits
Limit item |
Description |
Recommended solution |
Terway network plug-in |
The maximum number of pods supported by the ECS instance type to be added has the following limits.
Note that the maximum number of pods supported by different ENI modes depends on the maximum number of ENIs supported by the instance type. For more information on how to calculate the maximum number of pods supported by different ENI modes, see Use the Terway Network Plug-in. For more information on how to view the number of pods supported by instance types, see Use the Terway Network Plug-in. |
|
The following limits apply to ENIs. |
||
When adding nodes that include a new zone, you need to update the vSwitch configuration for Terway. Otherwise, pod IPs will be allocated from the vSwitch corresponding to the primary ENI of the node. |
||
When adding existing nodes to a node pool, the ENI already bound to the node will be retained, and pod IPs will be allocated from the vSwitch associated with the bound ENI. Ensure that the added node contains only one primary ENI. |
If there are pod IPs that do not belong to the vSwitch you configured, remove the node from the cluster, delete the secondary ENIs except for the primary ENI, and then re-add the node to the cluster. |
|
When adding existing nodes to a node pool, ensure that the target ECS instance is bound to the cluster RAM role to avoid incorrect calculation of MaxPod (the maximum number of pods available on the node) due to permission issues, which may lead to incorrect calculation of the number of ENIs. |
||
Flannel network plug-in |
There is a limit on the number of custom route entries in the system route table under the cluster VPC. The number cannot exceed the quota of custom route entries that can be retained in each route table. For more information about custom route entries, see Route Table. |
Adjust in Quota Center. |
IPv4/IPv6 dual-stack |
You must assign an IPv6 address to the primary ENI of the ECS instance. |
For more information on how to assign an IPv6 address to the primary ENI of an ECS instance, see IPv6 Communication. |
Limits on security groups
Limit item | Description | Recommended solution |
Security group type | When adding existing ECS instances to a node pool, the ECS instances will be added to the security group of the node pool. Due to ECS limits, an ECS instance cannot belong to a basic security group and an advanced security group at the same time. Therefore, the type of security group to which an ECS instance belongs must be the same as that of the node pool to which the ECS instance is added. You can click Details next to the target node pool to view the security group information of the node pool on the Basic Information tab. You can log on to the ECS console to view the security group information to which the ECS instance belongs. For more information about security groups, see Security Group Overview. | The security group of a node pool cannot be modified, and the type of security group cannot be changed. If a conflict occurs, resolve it in the following ways.
|
Security group rules | The security group rules of an ECS instance must not conflict with the security group rules of the node pool and the cluster to which the ECS instance is added. | Add existing nodes to the security group of the node pool in advance. Use the security group rule detection feature to check each rule in the security group of the node pool. |
Number of security groups | When adding existing ECS instances to a node pool, the instances will be added to the security group of the node pool. Ensure that the number of security groups to which the ECS instance belongs does not exceed the upper limit (the default upper limit is 5). | For more information about the limit on the number of security groups and how to apply for an increase in the number of security groups that an ECS instance can join, see Security Group. |
Limits on operating systems
Only ECS instances with operating systems of Alibaba Cloud Linux, Red Hat, Ubuntu, and CentOS are supported. We recommend that you choose Alibaba Cloud Linux 3. For more information about the operating system support of ACK clusters, see Operating System.
Precautions
Instance release
ECS instances added to a node pool are not released when the cluster or node pool is deleted. You need to manually remove them. For more information about manual removal, see Remove Nodes.
Related billing
ECS instances added to a cluster are not released when the cluster or node pool is deleted. We recommend that you check the billing of the ECS instances in a timely manner to avoid unnecessary costs. For more information, see Billing Overview.
Instance configuration
When you add existing ECS instances to a node pool, the billing method and instance type remain unchanged.
Operating systems and disks
We strongly recommend that you back up data before performing operations to avoid data loss. For specific operations, see Create Snapshots.
When nodes are automatically added, the original operating system of the node will be reset according to the current operating system of the node pool. When nodes are manually added, the original operating system of the node will be retained. If you want to retain the original operating system of the node, manually add the node.
When nodes are automatically added, the original system disk will be released, but the data disk will not be released. However, the disk ID of the data disk will change, and the stored data will not be affected.
When nodes are automatically added, the user-created snapshots of the original system disk will be retained. Automatic snapshots will be retained or deleted together with the system disk based on the setting of the "Delete Automatic Snapshots While Releasing Disk" attribute of the system disk. You can log on to the ECS console, click Modify Attribute in the disk list to view or modify the attribute value.
To ensure that you have a sufficient snapshot quota to periodically run automaticsnapshot policies, we recommend that you delete user snapshots and automatic snapshots that are no longer needed.
Automatically or manually add nodes
Adding nodes automatically or manually does not affect the existing nodes or applications in your cluster. To avoid compatibility issues, we recommend that you do not initialize ECS instances as worker nodes if services are already deployed on the ECS instances.
Automatically add nodes: The original operating system of the node will be reset according to the current operating system of the node pool. If you do not want to reset the operating system of the node, manually add the node.
Manually add nodes: The original operating system of the node will be retained, and the operating system will not be reset.
Before manually adding nodes, in addition to Limits, pay attention to the following restrictions.
The operating system does not support enabling Swap.
If you want to store container data and images on a data disk, make sure that the data disk uses the ext or XFS file system.
Log on to the ACK console. In the left-side navigation pane, click Clusters.
On the Clusters page, find the cluster that you want to manage and click its name. In the left-side navigation pane, choose .
On the Node Pools page, click More in the Actions column on the right side of the target node pool, and then click Add Existing Nodes.
Manually add nodes
You need to obtain the installation command and log on to the corresponding ECS instance to perform the installation. You can add only one ECS instance at a time.
On the Select Existing ECS Instance configuration item page, select Add Method as Manual Add, select the desired ECS instance from the list of existing ECS instances, and then click Next.
On the Fill In Instance Information page, fill in the relevant information, and then click Next.
Configuration Item
Description
Cluster Id/name
The information about the cluster to which you want to add the ECS instances. This parameter is automatically set.
Data Disk Mounting
Specify whether to store the container and image data on a data disk.
If a data disk is attached to the ECS instance and the file system of the last data disk is not initialized, the system automatically formats the last data disk to ext4 to store content in /var/lib/docker and /var/lib/kubelet (the default data directories for Docker container runtime and kubelet component, respectively).
ImportantData on a disk is lost after you format the disk. Before the system formats the data disk, we recommend that you back up the data on the data disk.
If no data disk is attached to the ECS instance, the system does not automatically attach a new data disk.
Retain Instance Name
When you add nodes, Retain Instance Name is enabled by default. If you do not need to retain the instance name, you can disable Retain Instance Name. In this case, the node will be renamed according to the rule specified by the custom node name.
Instance Information
The IDs and names of the ECS instances that you want to add.
On the Add Complete page, copy the execution command for remote connection to the ECS instance, and then click Complete.
Log on to the ECS console, click Instances & ImagesInstances in the left-side navigation pane, select the region where the cluster resides, and select the ECS instance that you want to add.
Click Remote Connection on the right side of the ECS instance. In the Remote Connection And Command dialog box, select a remote connection method to enter the ECS remote connection interface.
For more information about remote connection methods, see the following table:
Remote Connection Method
Description
VNC Remote Connection
For more information about how to use VNC to remotely connect to ECS instances, see Connect to Instances via VNC.
Send Remote Commands (cloud Assistant)
This is the recommended method. Sending remote commands allows you to quickly execute commands inside the instance without logging on to the instance. You can perform operations such as viewing disk capacity, installing software, and starting or stopping services. This feature is implemented by using the command execution feature of Cloud Assistant. For more information about how to install or activate the Cloud Assistant client, see Install Cloud Assistant Agent.
In the ECS instance remote connection interface, follow the page instructions to enter the command saved in Step 6, and click Execute to start executing the script.
After the script is executed, the ECS instance is successfully added.
After successful addition, you can click Details in the Actions column on the Node Pools page, and then view the added node information on the Node Management tab.
Automatically add nodes
Available ECS instances under the current account will be listed, and installation and deployment will be performed in the console interface, and the instances will be automatically added to the cluster.
On the Select Existing ECS Instance configuration wizard page, select Add Method as Automatic Add, select the desired ECS instance from the list of existing ECS instances, and then click Next.
ImportantIf no ECS instance is displayed in the list, this means that the existing ECS instances do not meet the requirement to be added to the cluster. For more information about the specific reasons for not meeting the requirement, see Limits and Precautions.
You can also select Show Unavailable Instances to view the reasons why instances cannot be added to the cluster. If no corresponding instance is found after selecting, check whether the instance and the cluster are in the same region and VPC.
On the Fill In Instance Information page, fill in the relevant information, and then click Next.
Configuration Item
Description
Cluster Id/name
The information about the cluster to which you want to add the ECS instances. This parameter is automatically set.
Data Disk Mounting
Specify whether to store the container and image data on a data disk.
If a data disk is attached to the ECS instance and the file system of the last data disk is not initialized, the system automatically formats the last data disk to ext4 to store content in /var/lib/docker and /var/lib/kubelet (the default data directories for Docker container runtime and kubelet component, respectively).
ImportantData on a disk is lost after you format the disk. Before the system formats the data disk, we recommend that you back up the data on the data disk.
If no data disk is attached to the ECS instance, the system does not automatically attach a new data disk.
Retain Instance Name
When you add nodes, Retain Instance Name is enabled by default. If you do not need to retain the instance name, you can disable Retain Instance Name. In this case, the node will be renamed according to the rule specified by the custom node name.
Instance Information
The IDs and names of the ECS instances that you want to add.
In the Add Existing Instances To Cluster dialog box, carefully read the precautions, and then click OK.
After successful addition, you can click Details in the Node Pools page's Actions column, and then view the added node information on the Node Management tab.
FAQ
Will subsequent ECS instance upgrades or downgrades affect the cluster business after the ECS instance is added to the cluster?
ECS instance upgrades or downgrades usually involve modifying instance types, modifying public bandwidth billing methods, modifying public bandwidth, modifying data disk billing methods, etc. For more information, see Overview of Upgrade and Downgrade Methods. Different upgrade or downgrade methods have different impacts on ECS instances.
Upgrade or downgrade operations that do not require a restart: Determine the impact on the business based on the actual business situation.
Upgrade or downgrade operations that require restarting ECS instances: Operations such as instance type upgrades or downgrades will cause disruptive changes to the business. Before performing related operations (Upgrade or Downgrade Node Resources), we recommend that you first determine whether you need to scale out redundant nodes to accommodate application pods based on the actual load situation, and drain and remove the nodes that need to be upgraded or downgraded from the scaling group and ACK cluster. For specific operations, see Remove Nodes.
Once the upgrade or downgrade is finished, follow the steps outlined in this topic to reintegrate the node into the cluster.
What should I do if an error occurs after adding existing nodes and a timeout message is displayed?
Verify the network connectivity between the node and the API Server CLB. Initially, ensure that the security group complies with the necessary requirements. For details on security group usage limits when incorporating existing nodes, see Limits on Security Groups. For additional information on network connectivity issues, refer to Network Management FAQ.
Can I add existing nodes of different instance types to an ACK cluster?
Yes, you can specify multiple instance types in a node pool to avoid node scale-out failures due to unavailable instance types or insufficient stock. The procedure is as follows:
Edit or create a new node pool and configure the required instance types. For specific operations, see Create and Manage Node Pools.
For information on removing and draining target nodes, see Remove Nodes (do not release ECS instances at the same time).
Refer to the Limits and Automatically or Manually Add Nodes sections in this topic to add ECS instances of different instance types to the node pool.
How can I move nodes across ACK clusters?
ACK does not support directly moving nodes across clusters, but you can do so by adding existing nodes. The procedure is as follows:
For information on removing and draining target nodes, see Remove Nodes (do not release ECS instances at the same time).
Refer to the Limits and Automatically or Manually Add Nodes sections in this topic to add the target ECS instances to the node pool.
Will the expected number of nodes in the node pool automatically change after adding existing nodes?
Yes. If you enabled Expected Number Of Nodes when you Create and Manage Node Pools, the expected number of nodes will automatically increase by the number of nodes added after adding existing nodes. For example, if the Expected Number Of Nodes configured in your node pool is 5, and you add an ECS instance to the node pool by automatically adding nodes, the Expected Number Of Nodes in the node pool will automatically change to 6.
References
In addition to the console, you can also add ECS instances to an ACK cluster through OpenAPI (Query the Script for Adding Existing Nodes to a Cluster Node Pool) or CLI (Add Existing ECS Instances).
Free nodes may exist in ACK clusters that were created before the node pool feature was released. Free nodes refer to cluster nodes that do not belong to any node pool. You can Migrate Free Nodes to Node Pools to include them in ACK cluster management.
If you want to learn more about the configuration items supported by managed node pools, see Overview of Managed Node Pools and Create and Manage Node Pools.
If your nodes, pods, or other resources encounter errors or run abnormally, you can refer to Node Exception Troubleshooting, Pod Exception Troubleshooting, and Node and Node Pool FAQ for self-troubleshooting.