Elastic Compute Service:Install cloud-init

For more information about cloud-init, see Cloud-init documentation.

Step 1: Check whether cloud-init needs to be installed or upgraded

1. Log on to the source server.

2. Check whether cloud-init is installed on the source server.

   CentOS series

   rpm -qa | grep -i cloud-init 
   pip list | grep -i cloud-init

   Ubuntu series

   dpkg -l | grep -i cloud-init 
   pip list | grep -i cloud-init

   • If no version number is returned or the version of cloud-init is 0.7.9 or earlier, proceed to Step 2: Install cloud-init.

     Important

     cloud-init 0.7.9 is an initial version of the native cloud-init, which is not suitable for ECS instances and must be upgraded.

   • If cloud-init 18 or later is used, cloud-init can automatically configure networks during instance initialization. If the network configurations do not meet your business requirements, you can modify the configurations. For more information, see the Customize network configurations section of this topic.

   • If cloud-init 19.1.21 is used, we recommend that you upgrade to Alibaba Cloud cloud-init 23.2.2. Compared with Alibaba Cloud cloud-init 19.1.21, Alibaba Cloud cloud-init 23.2.2 provides the following improvements and new features:

     • Cloud-init 23.2.2 allows you to access instance metadata in security hardening mode. For information about instance metadata, see Obtain instance metadata.

     • Other feature enhancements, performance improvements, error fixes, and community contributions. For more information, see ChangeLog.

   • If other version numbers of cloud-init are returned, perform the operations that are described in the What to do next section of this topic.

3. To prevent data loss caused by misoperations during cloud-init installation, we recommend that you back up the data of the source server in advance. For example, you can create snapshots to back up the data of the source server.

Step 2: Install cloud-init

• (Recommended) Alibaba Cloud cloud-init 23.2.2: This version requires Python 3.6 or later and uses Aliyun as the data source.

• Alibaba Cloud cloud-init 19.1.21: This version requires Python 3.6 or later and uses Aliyun as the data source.

• Alibaba Cloud cloud-init 0.7.6a17: Install this version of cloud-init on specific earlier operating system versions, such as CentOS 6, Debian 9, and SUSE Linux Enterprise Server (SLES) 12. Alibaba Cloud cloud-init 0.7.6a17 requires Python 2.7 and uses Aliyun as the data source.

  Important

  The Python community no longer provides technical support for Python 2.7. To prevent risks that are associated with dependency libraries, we recommend that you use more recent versions of cloud-init.

• Native cloud-init: The community versions of cloud-init are official versions that are publicly available and maintained by the community of developers who contribute to the project. The most recent version of Alibaba Cloud cloud-init is 23.2.2. If you want to use a more recent version of cloud-init, install a community version of cloud-init. For more information, visit the official cloud-init website.

Alibaba Cloud cloud-init is optimized for and can provide better support for Alibaba Cloud services. To ensure compatibility, we recommend that you use Alibaba Cloud cloud-init. Select a cloud-init version to install based on your business requirements.

# Check whether the python3-pip dependency library is installed. If the python3-pip dependency library is not installed, install the dependency library.
if !  python3 -c 'import setuptools' >& /dev/null; then
  yum -y install python3-pip
fi
# Back up cloud-init of an earlier version.
test -d /etc/cloud && mv /etc/cloud /etc/cloud-old
# Download and decompress the Alibaba Cloud cloud-init installation package.
wget https://ecs-image-tools.oss-cn-hangzhou.aliyuncs.com/cloudinit/cloud-init-19.1.21.tgz
tar -zxvf ./cloud-init-19.1.21.tgz
# Install Alibaba Cloud cloud-init.
issue_major=$( cat /etc/redhat-release | grep -Eo '[0-9]+\.?[0-9]+' | head -1 | awk -F'.' '{printf $1}')
bash ./cloud-init-*/tools/deploy.sh centos "$issue_major"

# Check whether the python3-pip dependency library is installed. If the python3-pip dependency library is not installed, install the dependency library.
if !  python3 -c 'import setuptools' >& /dev/null; then
  yum -y install python3-pip
fi
# Back up cloud-init of an earlier version.
test -d /etc/cloud && mv /etc/cloud /etc/cloud-old
# Download and decompress the Alibaba Cloud cloud-init installation package.
wget https://ecs-image-tools.oss-cn-hangzhou.aliyuncs.com/cloudinit/cloud-init-19.1.21.tgz
tar -zxvf ./cloud-init-19.1.21.tgz
# Install Alibaba Cloud cloud-init.
issue_major=$( cat /etc/os-release | grep VERSION_ID | grep -Eo '[0-9]+\.?[0-9]+' | head -1 | awk -F'.' '{printf $1}')
bash ./cloud-init-*/tools/deploy.sh rhel "$issue_major"

# Check whether the python3-pip dependency library is installed. If the python3-pip dependency library is not installed, install the dependency library.
if !  python3 -c 'import setuptools' >& /dev/null; then
  apt-get install python36 python3-pip -y
fi
# Back up cloud-init of an earlier version.
test -d /etc/cloud && mv /etc/cloud /etc/cloud-old
# Download and decompress the Alibaba Cloud cloud-init installation package.
wget https://ecs-image-tools.oss-cn-hangzhou.aliyuncs.com/cloudinit/cloud-init-19.1.21.tgz
tar -zxvf ./cloud-init-19.1.21.tgz
# Install Alibaba Cloud cloud-init.
issue_major=$( cat /etc/os-release | grep VERSION_ID | grep -Eo '[0-9]+\.?[0-9]+' | head -1 | awk -F'.' '{printf $1}')
bash ./cloud-init-*/tools/deploy.sh ubuntu "$issue_major"

# Check whether the python3-pip dependency library is installed. If the python3-pip dependency library is not installed, install the dependency library.
if !  python3 -c 'import setuptools' >& /dev/null; then
  apt-get -y install python3-pip
fi
# Back up cloud-init of an earlier version.
test -d /etc/cloud && mv /etc/cloud /etc/cloud-old
# Download and decompress the Alibaba Cloud cloud-init installation package.
wget https://ecs-image-tools.oss-cn-hangzhou.aliyuncs.com/cloudinit/cloud-init-19.1.21.tgz
tar -zxvf ./cloud-init-19.1.21.tgz
# Install Alibaba Cloud cloud-init.
issue_major=$( cat /etc/os-release | grep VERSION_ID | grep -Eo '[0-9]+\.?[0-9]+' | head -1 | awk -F'.' '{printf $1}')
bash ./cloud-init-*/tools/deploy.sh debian "$issue_major"

# Check whether the python3-pip dependency library is installed. If the python3-pip dependency library is not installed, install the dependency library.
if !  python3 -c 'import setuptools'>& /dev/null; then
  zypper -n install python3-pip
fi
# Back up cloud-init of an earlier version.
test -d /etc/cloud && mv /etc/cloud/etc/cloud-old
# Download and decompress the Alibaba Cloud cloud-init installation package.
wget https://ecs-image-tools.oss-cn-hangzhou.aliyuncs.com/cloudinit/cloud-init-19.1.21.tgz
tar -zxvf ./cloud-init-19.1.21.tgz
# Install Alibaba Cloud cloud-init.
issue_major=$( cat /etc/os-release | grep VERSION_ID | grep -Eo '[0-9]+\.?[0-9]+' | head -1 | awk -F'.' '{printf $1}')
bash ./cloud-init-*/tools/deploy.sh sles "$issue_major"

# Check whether the python3-pip dependency library is installed. If the python3-pip dependency library is not installed, install the dependency library.
if !  python3 -c 'import setuptools'>& /dev/null; then
  zypper -n install python3-pip
fi
# Back up cloud-init of an earlier version.
test -d /etc/cloud && mv /etc/cloud/etc/cloud-old
# Download and decompress the Alibaba Cloud cloud-init installation package.
wget https://ecs-image-tools.oss-cn-hangzhou.aliyuncs.com/cloudinit/cloud-init-19.1.21.tgz
tar -zxvf ./cloud-init-19.1.21.tgz
# Install Alibaba Cloud cloud-init.
issue_major=$( cat /etc/os-release | grep VERSION_ID | grep -Eo '[0-9]+\.?[0-9]+' | head -1 | awk -F'.' '{printf $1}')
bash ./cloud-init-*/tools/deploy.sh opensuse"$issue_major"

(Optional) Step 3: Configure cloud-init

FAQ

• What do I do if the required dependency libraries are missing after I use Python 3 to install cloud-init?

  The dependency libraries that are missing may vary based on the image. You can use pip to install the missing dependency libraries and then re-install cloud-init.

• What do I do if cloud-init does not run as expected and an error message is returned?

  If different versions of dependency libraries are installed by using the default software package manager (such as YUM) and the pip manager, library version conflicts may occur and cause errors in cloud-init. We recommend that you download dependency libraries based on the error message.

  Error message	Cause	Troubleshooting command
  no setuptools module in python	The python setuptools module is missing.	For example, if you use Python 3.6, run one of the following commands based on the operating system to install the python setuptools module: CentOS and Red Hat: yum -y install python3-pip Ubuntu and Debian: apt-get -y install python3-pip openSUSE and SUSE: zypper -n install python3-pip
  File "/home/testuser/cloud-init/cloudinit/log.py", line 19, in <module> import six ImportError: No module named six )	The six dependency library is missing.	pip3 install six
  File "/home/testuser/cloud-init/cloudinit/url_helper.py", line 20, in <module> import oauthlib.oauth1 as oauth1 ImportError: No module named oauthlib.oauth1 )	The oauthlib dependency library is missing.	pip3 install oauthlib
  No indication of a missing dependency library.	The error message does not indicate a cause.	Run the following command to install all dependency libraries that are displayed in the requirements.txt file of cloud-init: pip3 install -r requirements.txt

• What do I do if cloud-init does not run as expected on an ECS instance after I install a new version of Python 3 on the instance?

  If you install a new version of Python 3 such as Python 3.9 on an ECS instance and create a symbolic link (such as ln -s /usr/bin/python3.9 /usr/bin/python3) for the version to configure the installed version as the default version, cloud-init does not run as expected on the instance. For example, when you run the cloud-init --version command, an error is reported.

  $cloud-init --version
  Traceback (most recent call last):
    File "/usr/local/bin/cloud-init", line 33, in <module>
      sys.exit(load_entry_point('cloud-init==19.1.9', 'console_scripts', 'cloud-init')())
    File "/usr/local/bin/cloud-init", line 22, in importlib_load_entry_point
      for entry_point in distribution(dist_name).entry_points
    File "/usr/lib64/python3.9/importlib/metadata.py", line 524, in distribution
      return Distribution.from_name(distribution_name)
    File "/usr/lib64/python3.9/importlib/metadata.py", line 187, in from_name
      raise PackageNotFoundError(name)
  importlib.metadata.PackageNotFoundError: cloud-init

  You can use one of the following methods to resolve the issue:

  • Method 1: Use the new version of Python 3 to re-install cloud-init.

  • Method 2: Modify the cloud-init executable file to set the Python interpreter path to the path of an earlier Python 3 version. In this example, Python 3.6 is used. Perform the following operations to change the Python interpreter path in the cloud-init executable file:

    1. Run the following command to open the cloud-init executable file:

       vim   /usr/local/bin/cloud-init

    2. Press the I key to enter Insert mode. At the beginning of the executable file, replace the content that follows #! with the path of Python 3.6.

       Example of the new content on the line that starts with #!:

       #!/usr/bin/python3.6

    3. Press the Esc key to exit Insert mode, enter :wq, and then press the Enter key to save and close the file.

• What do I do if the user data that I specified for an instance during instance creation does not or cannot run after I install cloud-init on the instance?

  1. Run the following command on the instance to check whether the user data exists in the instance metadata:

     curl http://100.100.100.200/latest/user-data

     Command output:

     • If the user data exists in the instance metadata, the command output contains the user data. In this case, perform other operations to troubleshoot the issue.

     • If the user data does not exist in the instance metadata, no command output is returned.

  2. Identify the reason why the user data does not or cannot run.

     • Check whether the user data is in a valid format.

       The user data must be in a valid format to allow cloud-init to run the user data. For example, the first line of the user data must start with #!. For more information, see the Formats and running frequencies of user data section of the "Instance user data" topic.

     • Check the execution result of the user data in the /var/log/cloud-init.log file of cloud-init and troubleshoot the issue based on the returned error message.

       Example of the execution result of the user data in the log file:

        util.py[DEBUG]: Running command ['/var/lib/cloud/instance/scripts/part-001'] with allowed return codes [0] (shell=False, capture=False)

     • Check the standard output (stdout) and standard error (stderr) generated when the user data was running.

       In systemd (a system initialization tool of Linux), user data is run by the cloud-final.service system service. You can run the following command to check the stdout and stderr generated when the user data was running. Then, you can troubleshoot the issue based on the command output.

       journalctl -u cloud-final.service

     • If you cannot identify the reason why the user data does not or cannot run after you perform the preceding operations, we recommend that you copy the user data to your on-premises computer and check whether the user data can run on the computer.

• What do I do if the system configurations of an instance cannot be initialized and the user data script cannot run after cloud-init is installed on the instance?

  1. Run the following commands to check whether the four cloud-init services are configured to automatically run on system startup:

     systemctl is-enabled cloud-init-local.service
     systemctl is-enabled cloud-init.service
     systemctl is-enabled cloud-config.service
     systemctl is-enabled cloud-final.service

     If an error message or disabled is displayed for a cloud-init service, the service is not configured to automatically run on system startup.

  2. Run the following commands to configure the four cloud-init services to automatically run on system startup:

     systemctl enable cloud-init-local.service
     systemctl enable cloud-init.service
     systemctl enable cloud-config.service
     systemctl enable cloud-final.service

• What do I do if the root partition of the system disk is not automatically extended after I resize the disk when I create an instance?

  Cloud-init automatically resizes partitions of Linux instances. Make sure that the growpart tool is installed and that the configurations in the cloud.cfg file are correct.

  1. Run the following command to check whether the growpart tool is installed on the instance operating system:

     which growpart

  2. (Conditionally required) If the growpart tool is not installed on the instance operating system, run one of the following commands based on your operating system to install the growpart tool:

     • RHEL series:

       sudo yum -y install cloud-utils-growpart

     • Debian series:

       sudo apt -y install cloud-guest-utils

  3. Run the following command to check whether the /etc/cloud/cloud.cfg file contains the content shown in the following figure:

     cat /etc/cloud/cloud.cfg

     

  4. (Conditionally required) If the /etc/cloud/cloud.cfg file does not contain the content, add the content to the file.

• What do I do if the new hostname of an ECS instance on which cloud-init 0.7.6a16 or earlier is installed does not take effect after I change the instance hostname and restart the instance in the ECS console?

  • Problem description

    After you change the hostname of an ECS instance in the ECS console and restart the instance, the new hostname does not take effect. After you run the cloud-init --version command on the ECS instance, the command output indicates that the version of cloud-init installed on the instance is 0.7.6a16 or earlier.

  • Cause

    When cloud-init uses the update_hostname module to update the hostname, cloud-init runs the hostname command to obtain the current system hostname, reads the hostname in the cloud-init cache from the /var/lib/cloud/data/previous-hostname file, and then checks the obtained hostname against the read hostname. If the hostnames are the same, cloud-init performs the hostname update. If the hostnames are different, cloud-init determines that the hostname was manually changed and does not perform the hostname update.

    In Red Hat operating systems, line feeds are added to the content that is read from the /var/lib/cloud/data/previous-hostname file, but line feeds are not added to the hostname command output. As a result, the read content is different from the command output, and the hostname update fails.

  • Solutions

    You can use one of the following methods to resolve the issue:

    • Upgrade the cloud-init version. For more information, see the Step 2: Install cloud-init section of this topic.

      • For CentOS 6 or earlier, upgrade to cloud-init 0.7.6a17.

      • For CentOS 7, upgrade to cloud-init 19.1.21.

      • For CentOS 8 or later, upgrade to cloud-init 23.2.2 or later.

    • Add line feeds.

      Modify the cloudinit/distros/rhel.py file of cloud-init and add line feeds to the hostname command output. You can run the find / -name rhel.py command to obtain the path of the cloudinit/distros/rhel.py file. For information about how to add line feeds to the hostname command output, see distros/rhel.py: _read_hostname() missing strip on "hostname". Example on how to add line feeds to the hostname command output:

      diff --git a/cloudinit/distros/rhel.py b/cloudinit/distros/rhel.py
      index 0b68414e..65176e99 100644
      --- a/cloudinit/distros/rhel.py
      +++ b/cloudinit/distros/rhel.py
      @@ -143,6 +143,7 @@ class Distro(distros.Distro):
                   return util.load_file(filename).strip()
               elif self.uses_systemd():
                   (out, _err) = util.subp(['hostname'])
      +            out = out.strip()
                   if len(out):
                       return out
                   else: