:Introduction

Batch Compute not only allows you to install software on an image directly but also supports application deployment on a Docker image. You can make a Docker image and use the registry tool to upload it to Alibaba Cloud OSS. Then, you can run tasks of your jobs on this image.

1. Support of Docker

The AutoCluster model is used as an example to explain the difference in support for common VMs and Docker containers:

• Using common VMs: Each job you submit can have multiple tasks, each having an image (Linux or Windows) specified.

  When running a task, the system starts the corresponding VM based on the specified image.

  The task then runs on this VM. After the task is finished, the system uploads the task execution result to the specified permanent storage, destroys the VM, and then runs the next task.

• Using Docker: You can specify a Docker image for each task.

  When running a task, the system starts a VM to run the system image (such as Ubuntu) that supports Docker.

  Then, the system downloads the Docker image you specified and starts it in the VM. Your task runs in the Docker container.

  After the task is finished, the system uploads the task execution result to the specified permanent storage, destroys the VM, and then runs the next task.

Currently, one VM can run only one Docker image.

# Use VMs:
---
  |-- job
       |-- task
             |-- VM (specified VM that supports Windows and Linux images)
                  |-- program (your program)

# Use Docker containers:
---
  |-- job
       |-- task
             |-- VM (Ubuntu operating system supporting Docker)
                 |-- Docker-Container (image specified for the Docker container)
                      |-- program (your program)

2. Scenarios with and without Docker

3. Install Docker

3.1. Download from the Docker official

• For a Windows or Mac operating system, download the toolbox at https://www.docker.com/docker-toolbox.

After you install the toolbox, two shortcut icons appear on the desktop:

• For a Linux operating system, download Docker for Ubuntu at https://www.docker.com/docker-ubuntu.

NOTE: Install Docker 1.10 or a later version. Earlier Docker versions may cause compatibility issues.

3.2. Configure booster

The booster can help you obtain an official Docker image more quickly in China:Alibaba Cloud Container Service Development Platform

4. Make a Docker image

This example describes how to make an Ubuntu image containing Python. The image name is myubuntu.

Create a directory dockerUbuntu with the following structure:

dockerUbuntu
  |-- Dockerfile

The content of the Dockerfile is as follows:

FROM ubuntu:14.04

# Replace your_name with your own name and your_email with your email address.
MAINTAINER your_name <your_email>

# Update the source.
RUN apt-get update

# Clear the cache.
RUN apt-get autoclean

# Install Python.
RUN apt-get install -y python

# Run the following command after the system starts:
CMD ["/bin/bash"]

Run the following command to build an image:

cd dockerUbuntu              # Enter the dockerUbuntu directory.
docker build -t myubuntu ./ # Build the image named myubuntu.

• NOTE: In Ubuntu, docker commands can only be executed by a sudo user by default.In a Mac or Windows operating system, start the command line window by using Docker Quickstart Terminal and run docker commands in the window.

After building the image, run the following command to view the image:

docker images

The output information is similar to the following:

5. Upload the Docker image to OSS

To use the Docker service of Batch Compute, upload the Docker image myubuntu you have made to OSS.The system then downloads the image from OSS to run your task programs.

5.4. Install OSS Docker Registry 2

Assume that you want to save Docker in the following directory of OSS:

oss://your-bucket/dockers/

Use the official image of Docker Registry 2 to create a private image repository, and set OSS information including the AccessKey ID, AccessKey Secret, region, and bucket.

The installation procedure is as follows:

Create the config.yml file in the current directory.

version: 0.1
log:
  level: debug
storage:
  oss:
    accesskeyid: your_access_key_id
    accesskeysecret: your_access_key_secret
    region: oss-cn-shenzhen
    bucket: your-bucket
    rootdirectory: dockers
    secure: false
    internal: false
http:
  addr: 0.0.0.0:5000

Replace the following variables:

For more information about OSS configuration, see official Docker documentation.

Run the following command to install OSS Docker Registry 2.

docker pull registry:2
docker run -v `pwd`/config.yml:/etc/docker/registry/config.yml -p 5000:5000 --name registry -d registry:2

• NOTE: In this example, oss-cn-shenzhen represents OSS in the China South 1 (Shenzhen) region. Jobs submitted subsequently can only work normally in this region.

After the installation is finished, run the following command to check the result:

docker ps       # Check the running container.

If the installation is successful, registry:2 is displayed.

5.2. Run the following command to upload the image to OSS:

docker tag myubuntu localhost:5000/myubuntu
docker push localhost:5000/myubuntu

To verify whether the image is uploaded successfully, log on to the OSS console and check for the following directory: oss://your-bucket/dockers/docker/registry/v2/repositories/myubuntu/.

6. Notes

• In a running Docker container, the user name is root, and the default value of the path environment variable is /sbin:/usr/sbin:/bin:/usr/bin, not /usr/local/bin.

• If you do not set the PWD environment variable, its value is ‘/batchcompute/workdir’. All your program packages are decompressed into /batchcompute/workdir.

• The Docker registry never stops after it starts. Therefore, when you submit jobs by specifying cluster IDs, make sure that all jobs submitted to the same cluster have the same BATCH_COMPUTE_DOCKER_REGISTRY_OSS_PATH value.

• InputMapping and OutputMapping cannot be attached to the same directory.

• Batch Compute uses the -privileged=false mode to start Docker containers, and therefore it does not allow a Docker container to start in another Docker container.