Let's Encrypt ACME with Alibaba Cloud API Gateway and CDN

By John Hanley, Alibaba Cloud Tech Share Author. Tech Share is Alibaba Cloud’s incentive program to encourage the sharing of technical knowledge and best practices within the cloud community.

In this multipart article, we will discuss about SSL certificates in detail to remove any doubts on this topic. We will learn how to use the Let's Encrypt ACME version 2 API using Python to develop software that can create, install, renew and revoke SSL certificates for Alibaba Cloud. Although we have used Alibaba Cloud products for the tutorial, the same principles apply to any computing service that supports X.509 SSL certificates.

In Part 2, we talked about creating your Account Key, Certificate Key and Certificate Signing Request (CSR).

In this section, we will explain about ACME endpoints, getting the ACME directory, creating an ACME account, and retrieving your ACME account information.

Let's Encrypt API Endpoints

Let's Encrypt ACME supports two modes by using different endpoints. A production mode that will issue real certificates and is rate limited, and a staging mode for testing that issues test certificates. The production endpoint limits the number of requests that you can make per day. While developing and software for Let's Encrypt make sure that you are using the staging endpoint otherwise you will reach your API limit and need to wait until the next day to resume testing.

Staging Endpoint: https://acme-staging-v02.api.letsencrypt.org/directory

Production Endpoint: https://acme-v02.api.letsencrypt.org/directory

ACME Directory

The first API call should be to get the ACME directory. The directory is a list of URLs that you call for various commands. The response from get directory is a JSON structure that looks like this:

{
    "LPTIN-Jj4u0": "https://community.letsencrypt.org/t/adding-random-entries-to-the-directory/33417",
    "keyChange": "https://acme-staging-v02.api.letsencrypt.org/acme/key-change",
    "meta": {
        "caaIdentities": [
            "letsencrypt.org"
        ],
        "termsOfService": "https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf",
        "website": "https://letsencrypt.org/docs/staging-environment/"
    },
    "newAccount": "https://acme-staging-v02.api.letsencrypt.org/acme/new-acct",
    "newNonce": "https://acme-staging-v02.api.letsencrypt.org/acme/new-nonce",
    "newOrder": "https://acme-staging-v02.api.letsencrypt.org/acme/new-order",
    "revokeCert": "https://acme-staging-v02.api.letsencrypt.org/acme/revoke-cert"
}

The first line should be ignored. ACME generates a random key and value to dissuade developers from hard coding JSON expectations into their code.

Let's look at each of the returned data parts:

keyChange
This URL is used to change the public key associated with the account. This is used to recover from a key compromise.

meta.caaIdentities
An array of hostnames which the ACME server recognizes as referring to itself for the purposes of CAA record validation. We do not use these records in the example code.

meta.termsOfService
A URL identifying the current terms of service. Take the time to read the referenced document on the Terms of Service at https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf. In the example code, we set the fields accepting the Terms of Service by default.

meta.website
A URL locating a website providing more information about the ACME server. We do not use this record in the example code. You can learn more about it here https://letsencrypt.org/docs/staging-environment/

newAccount
This is an important API and should be the second call that you make to the ACME API. A nonce is a unique random value that protects against replay attacks. Each API call (except for directory) requires a unique nonce value.

newNonce
This endpoint is used to create a new account.

newOrder
This endpoint is used to request this issuance of an SSL certificate.

revokeCert
This endpoint is used to revoke an existing certificate that was issued by the same account.

Example Code to Get the ACME Directory

This is the simplest ACME API example in the examples package. This example just calls the ACME main endpoint. The returned data is a JSON structure that defines the API endpoints as described above. Review the output from this program and become familiar with the various URLs as they will be used in most of the following examples.

Source: get_directory.py

""" Let's Encrypt ACME Version 2 Examples - Get Directory"""

# This example will call the ACME API directory and display the returned data
# Reference: https://ietf-wg-acme.github.io/acme/draft-ietf-acme-acme.html#rfc.section.7.1.1

import    sys
import    requests
import    helper

path = 'https://acme-staging-v02.api.letsencrypt.org/directory'

headers = {
    'User-Agent': 'neoprime.io-acme-client/1.0',
    'Accept-Language': 'en'
}

try:
    print('Calling endpoint:', path)
    directory = requests.get(path, headers=headers)
except requests.exceptions.RequestException as error:
    print(error)
    sys.exit(1)

if directory.status_code < 200 or directory.status_code >= 300:
    print('Error calling ACME endpoint:', directory.reason)
    sys.exit(1)

# The output should be json. If not something is wrong
try:
    acme_config = directory.json()
except Exception as ex:
    print("Error: Cannot load returned data:", ex)
    sys.exit(1)

print('')
print('Returned Data:')
print('****************************************')
print(directory.text)

acme_config = directory.json()

print('')
print('Formatted JSON:')
print('****************************************')
helper.print_dict(acme_config, 0)

Example Code to Create a New Account

The next step is to create a new account on the ACME server. This uses the account.key that we created in Part 2. The ACME server does not track information such as company name in the account database; only a contact email address is needed for notifications.

In this example, change the EmailAddress parameter to be your email address. The example shows how to include more than one email address. The ACME server does not require that you include and email address as this is optional. The ACME server does not verify the email address.

Let's review a couple of key points about the code.

1. Get the ACME directory

    acme_config = get_directory()

2. Get the "newAccount" URL

    url = acme_config["newAccount"]

3. Request a nonce for the first ACME API call
After the first ACME API call, a new nonce in returned in the header "Replay-Nonce" after each ACME API call.

    nonce = requests.head(acme_config['newNonce']).headers['Replay-Nonce']

4. Assemble the HTML headers
The important item is: Content-Type: application/jose+json

    headers = {
        'User-Agent': 'neoprime.io-acme-client/1.0',
        'Accept-Language': 'en',
        'Content-Type': 'application/jose+json'
    }

5. Assemble the HTML body with the ACME API parameters
Creating the HTTP body will be covered in detail in the Part 4.

    payload = {}

    payload["termsOfServiceAgreed"] = True
    payload["contact"] = EmailAddresses

    body_top = {
        "alg": "RS256",
        "jwk": myhelper.get_jwk(AccountKeyFile),
        "url": url,
        "nonce": nonce
    }

6. Assemble the HTML body "jose" data structure
Notice how everything is base64 encoded.

    body_top_b64 = myhelper.b64(json.dumps(body_top).encode("utf8"))
    payload_b64 = myhelper.b64(json.dumps(payload).encode("utf8"))

    jose = {
        "protected": body_top_b64,
        "payload": payload_b64,
        "signature": myhelper.b64(signature)
    }

7. Finally, call the ACME API
This is done with an HTTP POST with a JSON body.

    resp = requests.post(url, json=jose, headers=headers)

8. After the ACME API two items are returned in the HTTP response headers
Location is the URL for your account.

Replay-Nonce is the "nonce" value for the next ACME API call.

    resp.headers['Location']
    resp.headers['Replay-Nonce']

Most ACME API calls require the HTTP headers include:

Content-Type: application/jose+json

Source: new_account.py

""" Let's Encrypt ACME Version 2 Examples - New Account"""

# This example will call the ACME API directory and create a new account
# Reference: https://ietf-wg-acme.github.io/acme/draft-ietf-acme-acme.html#rfc.section.7.3.2

import    os
import    sys
import    json
import    requests
import    myhelper

# Staging URL
path = 'https://acme-staging-v02.api.letsencrypt.org/directory'

# Production URL
# path = 'https://acme-v02.api.letsencrypt.org/directory'

AccountKeyFile = 'account.key'

EmailAddresses = ['mailto:someone@eexample.com', 'mailto:someone2@eexample.com']

def check_account_key_file():
    """ Verify that the Account Key File exists and prompt to create if it does not exist """
    if os.path.exists(AccountKeyFile) is not False:
        return True

    print('Error: File does not exist: {0}'.format(AccountKeyFile))

    if myhelper.Confirm('Create new account private key (y/n): ') is False:
        print('Cancelled')
        return False

    myhelper.create_rsa_private_key(AccountKeyFile)

    if os.path.exists(AccountKeyFile) is False:
        print('Error: File does not exist: {0}'.format(AccountKeyFile))
        return False

    return True

def get_directory():
    """ Get the ACME Directory """
    headers = {
        'User-Agent': 'neoprime.io-acme-client/1.0',
        'Accept-Language': 'en',
    }

    try:
        print('Calling endpoint:', path)
        directory = requests.get(path, headers=headers)
    except requests.exceptions.RequestException as error:
        print(error)
        return False

    if directory.status_code < 200 or directory.status_code >= 300:
        print('Error calling ACME endpoint:', directory.reason)
        print(directory.text)
        return False

    # The following statements are to understand the output
    acme_config = directory.json()

    return acme_config

def main():
    """ Main Program Function """
    headers = {
        'User-Agent': 'neoprime.io-acme-client/1.0',
        'Accept-Language': 'en',
        'Content-Type': 'application/jose+json'
    }

    if check_account_key_file() is False:
        sys.exit(1)

    acme_config = get_directory()

    if acme_config is False:
        sys.exit(1)

    url = acme_config["newAccount"]

    # Get the URL for the terms of service
    terms_service = acme_config.get("meta", {}).get("termsOfService", "")
    print('Terms of Service:', terms_service)

    nonce = requests.head(acme_config['newNonce']).headers['Replay-Nonce']
    print('Nonce:', nonce)
    print("")

    # Create the account request
    payload = {}

    if terms_service != "":
        payload["termsOfServiceAgreed"] = True

    payload["contact"] = EmailAddresses

    payload_b64 = myhelper.b64(json.dumps(payload).encode("utf8"))

    body_top = {
        "alg": "RS256",
        "jwk": myhelper.get_jwk(AccountKeyFile),
        "url": url,
        "nonce": nonce
    }

    body_top_b64 = myhelper.b64(json.dumps(body_top).encode("utf8"))

    data = "{0}.{1}".format(body_top_b64, payload_b64).encode("utf8")

    signature = myhelper.sign(data, AccountKeyFile)

    #
    # Create the HTML request body
    #

    jose = {
        "protected": body_top_b64,
        "payload": payload_b64,
        "signature": myhelper.b64(signature)
    }

    try:
        print('Calling endpoint:', url)
        resp = requests.post(url, json=jose, headers=headers)
    except requests.exceptions.RequestException as error:
        resp = error.response
        print(resp)
    except Exception as ex:
        print(ex)
    except BaseException as ex:
        print(ex)

    if resp.status_code < 200 or resp.status_code >= 300:
        print('Error calling ACME endpoint:', resp.reason)
        print('Status Code:', resp.status_code)
        myhelper.process_error_message(resp.text)
        sys.exit(1)

    print('')
    if 'Location' in resp.headers:
        print('Account URL:', resp.headers['Location'])
    else:
        print('Error: Response headers did not contain the header "Location"')

main()

sys.exit(0)

Example Code to Get Account Information

Now that we have created an account using our account.key, let's communicate with the ACME server to see what information is stored on the server. This example introduces a configuration file "acme.ini" to contain the configuration parameters instead of hard coding them into the source code.

Modify acme.ini to include your specific information such as email address.

Source: acme.ini

 
[acme-neoprime]
UserAgent = neoprime.io-acme-client/1.0

# [Required] ACME account key
AccountKeyFile = account.key

# Certifcate Signing Request (CSR)
CSRFile = example.com.csr

ChainFile = example.com.chain.pem

# ACME URL
# Staging URL
# https://acme-staging-v02.api.letsencrypt.org/directory
# Production URL
# https://acme-v02.api.letsencrypt.org/directory

ACMEDirectory = https://acme-staging-v02.api.letsencrypt.org/directory

# Email Addresses so that LetsEncrypt can notify about SSL renewals
Contacts = mailto:example.com;mailto:someone2@example.com

# Preferred Language
Language = en

Source: get_acount_info.py

""" Let's Encrypt ACME Version 2 Examples - Get Account Information """

############################################################
# This example will call the ACME API directory and get the account information
# Reference: https://ietf-wg-acme.github.io/acme/draft-ietf-acme-acme.html#rfc.section.7.3.3
#
# This program uses the AccountKeyFile set in acme.ini to return information about the ACME account.
############################################################

import    sys
import    json
import    requests
import    helper
import    myhelper

############################################################
# Start - Global Variables

g_debug = 0

acme_path = ''
AccountKeyFile = ''
EmailAddresses = []
headers = {}

# End - Global Variables
############################################################

############################################################
# Load the configuration from acme.ini
############################################################

def load_acme_parameters(debug=0):
    """ Load the configuration from acme.ini """

    global acme_path
    global AccountKeyFile
    global EmailAddresses
    global headers

    config = myhelper.load_acme_config(filename='acme.ini')

    if debug is not 0:
        print(config.get('acme-neoprime', 'accountkeyfile'))
        print(config.get('acme-neoprime', 'csrfile'))
        print(config.get('acme-neoprime', 'chainfile'))
        print(config.get('acme-neoprime', 'acmedirectory'))
        print(config.get('acme-neoprime', 'contacts'))
        print(config.get('acme-neoprime', 'language'))

    acme_path = config.get('acme-neoprime', 'acmedirectory')

    AccountKeyFile = config.get('acme-neoprime', 'accountkeyfile')

    EmailAddresses = config.get('acme-neoprime', 'contacts').split(';')

    headers['User-Agent'] = config.get('acme-neoprime', 'UserAgent')
    headers['Accept-Language'] = config.get('acme-neoprime', 'language')
    headers['Content-Type'] = 'application/jose+json'

    return config

############################################################
#
############################################################

def get_account_url(url, nonce):
    """ Get the Account URL based upon the account key """

    # Create the account request
    payload = {}

    payload["termsOfServiceAgreed"] = True
    payload["contact"] = EmailAddresses
    payload["onlyReturnExisting"] = True

    payload_b64 = myhelper.b64(json.dumps(payload).encode("utf8"))

    body_top = {
        "alg": "RS256",
        "jwk": myhelper.get_jwk(AccountKeyFile),
        "url": url,
        "nonce": nonce
    }

    body_top_b64 = myhelper.b64(json.dumps(body_top).encode("utf8"))

    #
    # Create the message digest
    #

    data = "{0}.{1}".format(body_top_b64, payload_b64).encode("utf8")

    signature = myhelper.sign(data, AccountKeyFile)

    #
    # Create the HTML request body
    #

    jose = {
        "protected": body_top_b64,
        "payload": payload_b64,
        "signature": myhelper.b64(signature)
    }

    #
    # Make the ACME request
    #

    try:
        print('Calling endpoint:', url)
        resp = requests.post(url, json=jose, headers=headers)
    except requests.exceptions.RequestException as error:
        resp = error.response
        print(resp)
    except Exception as error:
        print(error)

    if resp.status_code < 200 or resp.status_code >= 300:
        print('Error calling ACME endpoint:', resp.reason)
        print('Status Code:', resp.status_code)
        myhelper.process_error_message(resp.text)
        sys.exit(1)

    if 'Location' in resp.headers:
        print('Account URL:', resp.headers['Location'])
    else:
        print('Error: Response headers did not contain the header "Location"')

    # Get the nonce for the next command request
    nonce = resp.headers['Replay-Nonce']

    account_url = resp.headers['Location']

    return nonce, account_url

############################################################
#
############################################################

def get_account_info(nonce, url, location):
    """ Get the Account Information """

    # Create the account request
    payload = {}

    payload_b64 = myhelper.b64(json.dumps(payload).encode("utf8"))

    body_top = {
        "alg": "RS256",
        "kid": location,
        "nonce": nonce,
        "url": location
    }

    body_top_b64 = myhelper.b64(json.dumps(body_top).encode("utf8"))

    #
    # Create the message digest
    #

    data = "{0}.{1}".format(body_top_b64, payload_b64).encode("utf8")

    signature = myhelper.sign(data, AccountKeyFile)

    #
    # Create the HTML request body
    #

    jose = {
        "protected": body_top_b64,
        "payload": payload_b64,
        "signature": myhelper.b64(signature)
    }

    #
    # Make the ACME request
    #

    try:
        print('Calling endpoint:', url)
        resp = requests.post(url, json=jose, headers=headers)
    except requests.exceptions.RequestException as error:
        resp = error.response
        print(resp)
    except Exception as error:
        print(error)

    if resp.status_code < 200 or resp.status_code >= 300:
        print('Error calling ACME endpoint:', resp.reason)
        print('Status Code:', resp.status_code)
        myhelper.process_error_message(resp.text)
        sys.exit(1)

    nonce = resp.headers['Replay-Nonce']

    # resp.text is the returned JSON data describing the account

    return nonce, resp.text

############################################################
#
############################################################

def load_acme_urls(path):
    """ Load the ACME Directory of URLS """
    try:
        print('Calling endpoint:', path)
        resp = requests.get(acme_path, headers=headers)
    except requests.exceptions.RequestException as error:
        print(error)
        sys.exit(1)

    if resp.status_code < 200 or resp.status_code >= 300:
        print('Error calling ACME endpoint:', resp.reason)
        print(resp.text)
        sys.exit(1)

    return resp.json()

############################################################
#
############################################################

def acme_get_nonce(urls):
    """ Get the ACME Nonce that is used for the first request """
    global    headers

    path = urls['newNonce']

    try:
        print('Calling endpoint:', path)
        resp = requests.head(path, headers=headers)
    except requests.exceptions.RequestException as error:
        print(error)
        return False

    if resp.status_code < 200 or resp.status_code >= 300:
        print('Error calling ACME endpoint:', resp.reason)
        print(resp.text)
        return False

    return resp.headers['Replay-Nonce']

############################################################
# Main Program Function
############################################################

def main(debug=0):
    """ Main Program Function """
    acme_urls = load_acme_urls(acme_path)

    url = acme_urls["newAccount"]

    nonce = acme_get_nonce(acme_urls)

    if nonce is False:
        sys.exit(1)

    nonce, account_url = get_account_url(url, nonce)

    # resp is the returned JSON data describing the account
    nonce, resp = get_account_info(nonce, account_url, account_url)

    info = json.loads(resp)

    if debug is not 0:
        print('')
        print('Returned Data:')
        print('##################################################')
        #print(info)
        helper.print_dict(info)
        print('##################################################')

    print('')
    print('ID:        ', info['id'])
    print('Contact:   ', info['contact'])
    print('Initial IP:', info['initialIp'])
    print('Created At:', info['createdAt'])
    print('Status:   ', info['status'])

def is_json(data):
    try:
        json.loads(data)
    except ValueError as e:
        return False
    return True

acme_config = load_acme_parameters(g_debug)

main(g_debug)

Summary

In Part 4, we will go deeper into the ACME API and learn how to construct each part of the JSON body, sign the payload and process the results.

Community

Let's Encrypt ACME with Alibaba Cloud API Gateway and CDN – Part 3

Let's Encrypt API Endpoints

ACME Directory

Example Code to Get the ACME Directory

Example Code to Create a New Account

Example Code to Get Account Information

Summary

Read previous post:

Read next post:

Alibaba Clouder

You may also like

Comments

Alibaba Clouder

Related Products

API Gateway

CDN(Alibaba Cloud CDN)

Certificate Management Service