This topic describes how to use Docker to build an image that contains an OSS Connector for AI/ML environment.
Prerequisites
Docker is installed.
An access credential configuration file and an OSS connector configuration file are created. For more information, see Configure OSS Connector for AI/ML.
Example
The following example describes how to build a Docker image that contains OSS Connector for AI/ML Python3.12.
Create a text file named Dockerfile in Linux to build a Docker image.
touch Dockerfile
Add the following configurations to the Dockerfile file and save the file.
# Specify the base image. You can replace the base image with a user image. FROM python:3.12.4 # Specify the working directory. WORKDIR /app # Copy the files in the current directory to the /app directory in the container. In most cases, the COPY command is used to copy the project file and the required configuration file. COPY . /app # Install OSS Connector for AI/ML. RUN pip install osstorchconnector
Run the following command to build a Docker image.
your_image_name
in the command specifies the name of the image that you want to build. Specify the parameter based on your business requirements.docker build -t your_image_name .
Run the following command to start the container and mount the access credential configuration file and OSS connector configuration file on the host:
docker run -it --name new-container-name -v /root/.alibabacloud/credentials:/app/credentials -v /etc/oss-connector/config.json:/app/config.json your_image_name bash
After you start the container, you have a containerized environment in which OSS Connector for AI/ML is installed. This environment contains the OSS connector configuration file and the access credential configuration file, as shown in the following figure.
References
When you use a containerized environment in which OSS Connector for AI/ML is installed for training tasks, you can perform the following operations:
Use OssMapDataset to build a map dataset suitable for random read operations. For more information, see Use data in OSS to build a map dataset suitable for random reading.
Use OssIterableDataset to build an iterable dataset for streaming sequential access. For more information, see Use data in OSS to build an iterative dataset that is suitable for sequential streaming reads.
Use OssCheckpoint to implement read and write operations on checkpoints in OSS. For more information, see Store and access checkpoints in OSS.