Securely Manage Secrets with HashiCorp Vault

By Hitesh Jethva, Alibaba Cloud Community Blog author.

Vault is a free and open-source tool from HashiCorp that can be used for securely storing and accessing secrets. Vault stores and tightly controls access to tokens, passwords, certificates, encryption keys for protecting secrets and other sensitive data using a UI, CLI, or HTTP API. Vault provides an interface to any secret and recording a detailed audit log. You can keep your database credentials, API keys for external services, credentials in the vault. Vault supports multiple storage backends including consul, local disk and cloud storage. Vault enables developers and security professionals to deploy applications in zero-trust environments across public and private data centers.

In this tutorial, we will learn how to manage secretes with HashiCorp Vault on an Alibaba Cloud Elastic Compute Service (ECS) instance with Ubuntu 16.04 installed.

Prerequisites

Before you can begin this tutorial, you need to have the following:

A newly created Alibaba Cloud ECS instance with Ubuntu 16.04 installed.
A root password is set up to your instance.

For reference, check out create a new ECS instance and connect to your instance. Next, once you are logged into your Ubuntu 16.04 instance, you'll need to run the apt-get update -y command to update your base system with the latest available packages.

Install Vault

As the first part of this tutorial, you will need to download the latest version of Vault source from their official website. You can download it along with checksum with the following command:

wget https://releases.hashicorp.com/vault/0.11.4/vault_0.11.4_linux_amd64.zip
wget https://releases.hashicorp.com/vault/0.11.4/vault_0.11.4_SHA256SUMS

Next, check the integrity of the downloaded file with the following command:

grep linux_amd64 vault_*_SHA256SUMS | sha256sum -c -

If everything is fine. You should see the following output:

vault_0.11.4_linux_amd64.zip: OK

Next, extract the downloaded file and copy the extracted binary file to the /usr/local/bin directory so it can accessible from your shell.

unzip vault_0.11.4_linux_amd64.zip
cp vault /usr/local/bin/

Next, you will need to set a Linux capability flag on the binary. You can do this with the following command:

setcap cap_ipc_lock=+ep /usr/local/bin/vault

Configure Vault

First, you will need to create a system user for the Vault daemon to run as. You can create system user with the following command:

useradd -r -d /opt/vault -s /bin/nologin vault

Next, give proper ownership to the /opt/vault directory with the following command:

install -o vault -g vault -m 750 -d /opt/vault

Next, you will need to create a Vault configuration file for storing encrypted secrets in /opt/vault file and listening connections via HTTP.

To do so, create a /etc/vault.hcl file:

nano /etc/vault.hcl

Add the following lines:

backend "file" {
        path = "/opt/vault"
}

listener "tcp" {
        tls_disable = 1

}

Save and close the file. Then, give proper permissions with the following command:

chown vault:vault /etc/vault.hcl 
chmod 640 /etc/vault.hcl

Next, you will need to add your domain name entry in /etc/hosts file to direct requests to Vault to localhost.

You can do this with the nano /etc/hosts command. Then, add the following line:

127.0.0.1 alibabatest.com

Save and close the file, when you are finished.

Create the Vault System Startup File

Next, you will need to create a system service file for Vault. So you can easily manage the Vault service. You can do this by creating the following file:

nano /etc/systemd/system/vault.service

Add the following lines:

[Unit]
Description=Managing Vault Daemon
After=network.target
ConditionFileNotEmpty=/etc/vault.hcl

[Service]
User=vault
Group=vault
ExecStart=/usr/local/bin/vault server -config=/etc/vault.hcl
ExecReload=/usr/local/bin/kill --signal HUP $MAINPID
CapabilityBoundingSet=CAP_SYSLOG CAP_IPC_LOCK
Capabilities=CAP_IPC_LOCK+ep
SecureBits=keep-caps
NoNewPrivileges=yes
KillSignal=SIGINT

[Install]
WantedBy=multi-user.target

Save and close the file. Then, start Vault service and enable it to start on boot time with the following command:

systemctl start vault
systemctl enable vault

You can check the status of Vault service with the systemctl status vault command. Your output will look like:

● vault.service - a tool for managing secrets
   Loaded: loaded (/etc/systemd/system/vault.service; disabled; vendor preset: enabled)
   Active: active (running) since Sat 2018-11-10 11:56:17 IST; 30s ago
     Docs: https://vaultproject.io/docs/
 Main PID: 1964 (vault)
   CGroup: /system.slice/vault.service
           └─1964 /usr/local/bin/vault server -config=/etc/vault.hcl

Nov 10 11:56:21 Node2 vault[1964]: ==> Vault server configuration:
Nov 10 11:56:21 Node2 vault[1964]:                      Cgo: disabled
Nov 10 11:56:21 Node2 vault[1964]:               Listener 1: tcp (addr: "127.0.0.1:8200", cluster address: "127.0.0.1:8201", max_request_durati
Nov 10 11:56:21 Node2 vault[1964]:                Log Level: (not set)
Nov 10 11:56:21 Node2 vault[1964]:                    Mlock: supported: true, enabled: true
Nov 10 11:56:21 Node2 vault[1964]:                  Storage: file
Nov 10 11:56:21 Node2 vault[1964]:                  Version: Vault v0.11.4
Nov 10 11:56:21 Node2 vault[1964]:              Version Sha: 612120e76de651ef669c9af5e77b27a749b0dba3
Nov 10 11:56:21 Node2 vault[1964]: ==> Vault server started! Log data will stream in below:
Nov 10 11:56:21 Node2 vault[1964]: 2018-11-10T11:56:20.401+0530 [WARN]  no `api_addr` value specified in config or in VAULT_API_ADDR; falling b

You can also verify the Vault version with the vault –version command. The output will look like this:

Vault v0.11.4 ('612120e76de651ef669c9af5e77b27a749b0dba3')

Initialize Vault

When you start Vault first time, it will be uninitialized that means it isn't ready to get and store data. As the first step, set an environment variable to tell the vault command how to connect to the Vault server with the following command:

export VAULT_ADDR=http://alibabatest.com:8200

Next, check the vault is in an uninitialized state by running the vault status command. The resulting output will be:

Error checking seal status: Error making API request.

URL: GET https://alibabatest.com:8200/v1/sys/seal-status
Code: 400. Errors:

* server is not yet initialized

Next, initialize the Vault using the vault init -key-shares=3 -key-threshold=2 command. The output will look like this:

WARNING! The "vault init" command is deprecated. Please use "vault operator
init" instead. This command will be removed in Vault 0.12.

Unseal Key 1: GxaoGqVbDRlpDbeZJcf23rUeFzo0XprfhqJ2oGgykcwK
Unseal Key 2: mfhRgTXxTb6jMn3hwEuBEsp7TGbnv0U38qnjqJ4I4Cnc
Unseal Key 3: 0YCai3UWTsHLLhUL8vK9qNVPQ5zNEdCdp+MYi4OvDhtN

Initial Root Token: 5L6ArACs6QTJ4xtz9bVrXsFR

Vault initialized with 3 key shares and a key threshold of 2. Please securely
distribute the key shares printed above. When the Vault is re-sealed,
restarted, or stopped, you must supply at least 2 of these keys to unseal it
before it can start servicing requests.

Vault does not store the generated master key. Without at least 2 key to
reconstruct the master key, Vault will remain permanently sealed!

It is possible to generate new unseal keys, provided you have a quorum of
existing unseal keys shares. See "vault operator rekey" for more information.

It is recommended to save each unseal token and the initial root token in a secure place.

Now, vault is initiated but sealed. So you will need to unseal Vault using the newly created unseal tokens. You will need at least two unseal keys in order to make the service become available and ready to use.

You can unseal it by running the following command:

vault operator unseal

Enter your first unseal token and press Enter:

Unseal Key (will be hidden):

Sealed: true
Key Shares: 3
Key Threshold: 2
Unseal Progress: 1
Unseal Nonce: 6dsa453s-1z34-hj87-1g7j-g89976j69f564

The above output indicates that the unsealing is in progress, but still requires one more unsealing key before Vault is ready for use. Now, run the unseal command again:

vault operator unseal

Enter your second unseal token and press Enter:

Unseal Key (will be hidden): 
Key                Value
---                -----
Seal Type          shamir
Initialized        true
Sealed             true
Total Shares       3
Threshold          2
Unseal Progress    1/2
Unseal Nonce       ad6c44d0-a32e-e1e3-a80f-e4e72f0a0d4e
Version            0.11.4
HA Enabled         false

Vault is now unsealed and ready to use. You can check the status of Vault with the vault status command. Your output will look like this:

Key             Value
---             -----
Seal Type       shamir
Initialized     true
Sealed          false
Total Shares    3
Threshold       2
Version         0.11.4
Cluster Name    vault-cluster-972270a2
Cluster ID      aade8afc-d7a6-70d6-bf53-5809b9bf96f5
HA Enabled      false

Test Vault

Vault is now installed and configured, so now it's time to test how Vault will write, store and read secrets. First, you will need to store the previously generated root token in the environment variable. You can do this with the following command:

root_token=5L6ArACs6QTJ4xtz9bVrXsFR

Next, write a value to a Vault with the following command:

VAULT_TOKEN=$root_token vault write secret/message value=testing

The output will look like this:

Success! Data written to: secret/message

Next, you will need to create a policy file with the nano policy.hcl command. And also add the following lines:

path "secret/message" {
     capabilities = ["read"]
}

Save and close the file. Then, write this policy to Vault with the following command:

VAULT_TOKEN=$root_token vault policy write message-readonly policy.hcl

The output is as follows:

Success! Uploaded policy: message-readonly

Now, create a token with the rights specified in the policy with the following command:

VAULT_TOKEN=$root_token vault token create -policy="message-readonly"

You should see the following output:

Key                  Value
---                  -----
token                e3EsX0o8rHjl1MR0OmJIei01
token_accessor       8po3dqy4V4goDqkcmLhgrfzm
token_duration       768h
token_renewable      true
token_policies       ["default" "message-readonly"]
identity_policies    []
policies             ["default" "message-readonly"]

Next, save the token value from the above output to an environment variable with the following command:

app_token=e3EsX0o8rHjl1MR0OmJIei01

You can now access the data stored in the path secret/message with the following command:

VAULT_TOKEN=$app_token vault read secret/message

The output will look like this:

Key                 Value
---                 -----
refresh_interval    768h
value               testing

Now, try to listing secrets in Vault with the following command:

VAULT_TOKEN=$app_token vault list secret

You can see that unprivileged token cannot perform other operations:

Error listing secret/: Error making API request.

URL: GET http://alibabatest.com:8200/v1/secret?list=true
Code: 403. Errors:

* 1 error occurred:
    * permission denied

Community

Securely Manage Secrets with HashiCorp Vault

Prerequisites

Install Vault

Configure Vault

Create the Vault System Startup File

Initialize Vault

Test Vault

Read previous post:

Read next post:

Alibaba Clouder

You may also like

Comments

Alibaba Clouder

Related Products

ECS(Elastic Compute Service)

OSS(Object Storage Service)