Community

Blog Events Webinars Tutorials Forum
  1. Blog
  2. Events
  3. Webinars
  4. Tutorials
  5. Forum
Create Account
×
Community Blog Managing ECS Instances with Ansible Dynamic Inventory

Managing ECS Instances with Ansible Dynamic Inventory

In this article, we will learn how to configure and integrate Ansible dynamic inventory with an ECS instance, allowing you to manage your servers through automation.

By Anish Nath, Alibaba Cloud Tech Share Author. Tech Share is Alibaba Cloud's incentive program to encourage the sharing of technical knowledge and best practices within the cloud community.

If your Alibaba Cloud Elastic Compute Service (ECS) inventory fluctuates over time, with hosts spinning up and shutting down in response to business demands, the static inventory solutions will not serve your needs. You may need to track hosts from multiple sources like CMDB, manual query and compilation and all other tedious tasks.

Ansible integrates all of these options via a dynamic external inventory system using Ansible Alicloud Module inventory scripts which can be found here https://raw.githubusercontent.com/alibaba/ansible-provider/master/contrib/inventory/alicloud.py

In this article we will learn how to:

  • Configure ECS Dynamic Scripts
  • Query ECS hosts
  • Integrate ECS with Ansible dynamic inventory
  • Use instance filters
  • Achieve security compliance using ECS dynamic scripts

Prerequisites

There are two ways to install Alicloud provider. However, before installing it, you should ensure Ansible has been installed in your server. If not, please install it.

Log in to your Alibaba Cloud ECS controller node or your managed node and install the following components:

$ ssh root@47.254.17.249 root@47.254.17.249's password:
Welcome to Alibaba Cloud Elastic Compute Service !
$ sudo pip install ansible
$ sudo pip install ansible_alicloud

Note: Make sure your NTP is in sync, otherwise you will be seeing API call failure

In case you are new to Alibaba Cloud, you can get $10 worth in credit through my referral link to get started on an ECS instance.

Authentication

You can specify your Alibaba Cloud authentication credentials (access key and secret key) by passing them as environment variables or by storing them in a vars file.

To pass authentication credentials as environment variables:

export ALICLOUD_ACCESS_KEY='XXXXXXXXXXXXX' 
export ALICLOUD_SECRET_KEY='YYYYYYYYYYYYYZZZZZZZZZZZZ'

The RAM key associated with this account should be having appropriate permission to query Alibaba Cloud resources. The granularity to be determined by the system security professional at minimum it should be having AliyunECSReadOnlyAccess

1

Inventory Script

Download the latest version of the Alibaba cloud dynamic inventory script alicloud.py and make it executable:

$wget https://raw.githubusercontent.com/alibaba/ansible-provider/master/contrib/inventory/alicloud.py 
$chmod +x alicloud.py 
$sudo cp alicloud.py /etc/ansible/hosts

Download the sample Ansible Alibaba Cloud ECS dynamic inventory, modify it to suit your needs and copy it to /etc/ansible/alicloud.ini:

$ wget https://raw.githubusercontent.com/alibaba/ansible-provider/master/contrib/inventory/alicloud.ini  
$vi alicloud.ini  
$sudo cp alicloud.ini /etc/ansible/

You can test the Alibaba Cloud dynamic inventory script manually to confirm it is working as expected:

$ ./alicloud.py --list

After a few moments you should see some JSON output with information about your ECS instances.

[Instance:i-rj9djh1xn7tava07qtam]
{
  "_meta": {
    "hostvars": {
      "i_rj9djh1xn7tava07qtam": {
        "ansible_ssh_host": "47.254.17.249", 
        "auto_release_time": "", 
        "availability_zone": "us-west-1b", 
        "block_device_mapping": [
          {
            "attach_time": "2019-01-04T04:43:26Z", 
            "delete_on_termination": true, 
            "device_name": "/dev/xvda", 
            "status": "in_use", 
            "volume_id": "d-rj9euf5u1ph9l348lre8"
          }
        ], 
        "cpu": 1, 
        "creation_time": "2019-01-04T04:43Z", 
        "credit_specification": "Standard", 
        "deletion_protection": false, 
        "deployment_set_id": "", 
        "description": "demo", 
        "eip": {
          "allocation_id": "", 
          "internet_charge_type": "", 
          "ip_address": ""
        }, 
        "expired_time": "2099-12-31T15:59Z", 
        "gpu": {
          "amount": 0, 
          "spec": "", 
          "specification": ""
        }, 
        "host_name": "demo", 
        "id": "i-rj9djh1xn7tava07qtam", 
        "image_id": "alinux_17_01_64_20G_cloudinit_20171222.vhd", 
        "instance_charge_type": "PostPaid", 
        "instance_id": "i-rj9djh1xn7tava07qtam", 
        "instance_name": "demo", 
        "instance_type": "ecs.t5-lc2m1.nano", 
        "internet_charge_type": "PayByTraffic", 
        "internet_max_bandwidth_in": 100, 
        "internet_max_bandwidth_out": 5, 
        "io_optimized": true, 
        "memory": 512, 
        "network_interfaces": [
          {
            "mac_address": "00:16:3e:00:46:ea", 
            "network_interface_id": "eni-rj99fi72su03d5iqpv26", 
            "primary_ip_address": "172.20.240.111"
          }
        ], 
        "osname": "Aliyun Linux  17.1 64\u4f4d", 
        "ostype": "linux", 
        "private_ip_address": "172.20.240.111", 
        "public_ip_address": "47.254.17.249", 
        "resource_group_id": "", 
        "state": "running", 
        "tags": {
          "tool": "ansible"
        }, 
        "user_data": "", 
        "vpc_id": "vpc-rj9mhs1pv5tbc31khpssv", 
        "vswitch_id": "vsw-rj9bt4ec9zqauhr8vig9q"
      }
    }
  }, 
  "alicloud": {
    "children": [
      "i_rj9djh1xn7tava07qtam"
    ]
  }, 
  "alinux_17_01_64_20G_cloudinit_20171222.vhd": [
    "i_rj9djh1xn7tava07qtam"
  ], 
  "i-rj9djh1xn7tava07qtam": [
    "i_rj9djh1xn7tava07qtam"
  ], 
  "i_rj9djh1xn7tava07qtam": [
    "47.254.17.249"
  ], 
  "security_group_security_group_id": [
    "i_rj9djh1xn7tava07qtam"
  ], 
  "subnet_vsw_rj9bt4ec9zqauhr8vig9q": [
    "i_rj9djh1xn7tava07qtam"
  ], 
  "tag_tool_ansible": [
    "i_rj9djh1xn7tava07qtam"
  ], 
  "type_ecs_t5_lc2m1_nano": [
    "i_rj9djh1xn7tava07qtam"
  ], 
  "us-west-1": [
    "i_rj9djh1xn7tava07qtam"
  ], 
  "us-west-1b": [
    "i_rj9djh1xn7tava07qtam"
  ], 
  "vpc_id_vpc_rj9mhs1pv5tbc31khpssv": [
    "i_rj9djh1xn7tava07qtam"
  ]
}

Refreshing the Cache

Note that the Alibaba cloud dynamic inventory script will cache results to avoid repeated API calls. To explicitly clear the cache, you can run the alicloud.py (or hosts) script with the –refresh parameter: .

$ /alicloud.py --refresh-cache

To see the complete list of variables available for an instance, run the script by itself:

Once you confirm the dynamic inventory script is working as expected, you can tell Ansible to use the alicloud.py script as an inventory file, as illustrated below:

[root@demo ~]# ansible -i alicloud.py all -m ping -k -u root 
SSH password: 
 [WARNING]: Found both group and host with same name: i_rj9djh1xn7tava07qtam
i_rj9djh1xn7tava07qtam | UNREACHABLE! => {
    "changed": false, 
    "msg": "Failed to connect to the host via ssh: ssh: Could not resolve hostname i_rj9djh1xn7tava07qtam: Name or service not known\r\n", 
    "unreachable": true
}
47.254.17.249 | SUCCESS => {
    "changed": false, 
    "ping": "pong"
}

ECS Dynamic Inventory Script Configuration

The above example resulted in one host which is unreachable when generating inventory, Ansible needs to know how to address a ECS instance and these are controlled through ECS dynamic inventory script settings file ansible.ini which provides mappings to instances from several groups

  • Regions: A group of all instances in an ECS region for example cn-beijing,eu-central-1,ap-southeast-1,us-east-1
  • Tags: Each instance can have a variety of key/value pairs associated with it called Tags. Each key/value pair is its own group of instances, again with special characters converted to underscores, in the format tag_TAGNAME
  • HOST VAR: How the VM to be queried

Edit the file alicloud.ini and change the hostname_variable

from
hostname_variable = instance_id
to
hostname_variable = instance_name

Run the script again

[root@demo ~]# ansible -i alicloud.py all -m ping -k -u root 
SSH password: 
 [WARNING]: Found both group and host with same name: demo demo | SUCCESS => {
    "changed": false, 
    "ping": "pong"
}
47.254.17.249 | SUCCESS => {
    "changed": false, 
    "ping": "pong"
}

What about the private VM's which doesn't have the public address, how to manage those VM's. this can be achieved through modifying the alicloud.ini file and changing the destination_variable to use private_ip_address

destination_variable=private_ip_address

The below ansible query will use ECS private address to manage the VM

[root@demo ~]# ansible -i alicloud.py all -m ping -k -u root 
SSH password: 
 [WARNING]: Found both group and host with same name: demo
demo | SUCCESS => {
    "changed": false, 
    "ping": "pong"
}
172.20.240.111 | SUCCESS => {
    "changed": false, 
    "ping": "pong"
}

instance_states

There are five ECS Instances states associated to the ECS VM's ['pending', 'running', 'starting', 'stopping', 'stopped']. By default, only ECS instances in the running status are returned, in order to control these option modify the alicloud.ini file and tune these setting according to your need.

all_instances = True
# instance_states = pending, running, starting, stopping, stopped

Security Compliance

Ansible ECS dynamic inventory script is the right way achieve the security compliance of your dynamic cloud environment, the config can be pushed dynamically without maintaining the static inventory, for example the below cron job configuration will push SSH profile to all the running ECS servers every thirty minutes

*/30 * * * *  ansible-playbook ssh.yml -i alicloud.py

and refresh the cache every four hour through cron job to have an updated List of Inventory, Tune it according to your need.

**0 0 */4 ? * ***   alicloud.py --refresh-cache

Conclusion

  • Use of dynamic Inventory is recommended when ECS CMDB (Configuration Management Database) is dynamic and changes frequently. (VM's spins scale up/down in rapid way or managing huge set of servers )
  • Alibaba Cloud ECS Dynamic Scripts will have always right IP address , if changed manually (EIP assigned to different VM)
  • Never supply Alibaba Cloud credentials in alicloud.ini is not recommended , It is strongly recommended using environment variable
DevOps automation Tech Share Ansible Dynamic Inventory
0 1 1
Share on

Read previous post:

Perseus-BERT: Industry-leading BERT Training Solution Optimized for Performance

Read next post:

Routing Applications in Kubernetes with Nginx-Ingress

Alibaba Clouder

2,599 posts | 764 followers

You may also like

Comments

Alibaba Clouder

2,599 posts | 764 followers

Related Products

More Posts by Alibaba Clouder

See All