All Products
Search

This Product

    Batch Compute:Quick start for cli 1

    Document Center

    Batch Compute:Quick start for cli 1

    Last Updated:May 09, 2018

    Note: Make sure that you have signed up Batch Compute service in advance.

    Content:

    1. Install and configure the Batch Compute-cli tool

    Install and configure the Batch Compute-cli tool.

    2. Prepare a job

    This is a summation job. All numbers in input.txt are added and the sum is written to output.txt.

    The calculation process is simple. Therefore, this job requires only one task.

    In this example, the OSS directory is mounted as the VM local directory and files are used for operations.

    2.1. Upload the data file to the OSS

    Create the file input.txt.

    The content of input.txt is as follows and normally one number occupies one line:

    1. 2
    2. 40
    3. 51

    Upload input.txt to:

    1. bcs o up input.txt oss://your-bucket/sum/inputs/
    3. # Check whether the file is successfully uploaded
    4. bcs o ls oss://your-bucket/sum/inputs/

    • your-bucket indicates the bucket created by yourself. In this example, it is assumed that the region is cn-shenzhen.

    • The bcs oss command can complete some typical actions related to your OSS instance. bcs oss -h shows the help information about this command. We recommend that you use this command when only a few data is to be tested. In the case of a large amount of data, the upload or download takes a long time because multithreading is not implemented yet. For more information about how to upload data to OSS instances, see OSS tools.

    2.2. Prepare task programs

    The content of sum.sh is as follows:

    1. #!/bin/bash
    2. t=0
    3. while read LINE
    4. do
    5.  t=$(($t+${LINE}))
    6. done < /home/inputs/input.txt
    7. echo $t
    8. echo $t > /home/outputs/output.txt

    In step 1, input.txt is uploaded to the directory oss://your-bucket/sum/inputs/. In the preceding program, input.txt is read from the VM local directory /home/inputs/, which is implemented by the OSS mounting function of Batch Compute. The function configuration is described in the next step.

    In this sample program, the bash script is used to complete the sum function. You can also run other applications in the script. For more information about how to deploy applications in the Batch Compute environment, see Custom image and Use the Docker.

    3. Submit the job

    Under the directory where sum.sh is located, run the following command to submit the job:

    1. bcs sub "sh sum.sh" -p sum.sh -r oss://your-bucket/sum/inputs/:/home/inputs/ -w oss://your-bucket/sum/outputs/:/home/outputs/

    • The default image and default instance type are used here.

    • -r indicates read-only mounting and the OSS directory oss://your-bucket/sum/inputs/ is mounted to the VM local directory /home/inputs/. In this way, the program can access input.txt under the directory /home/inputs/.

    • -w indicates writable mounting and the OSS directory oss://your-bucket/sum/outputs/ is mounted to the VM local directory /home/outputs/. The file output.txt that is written to the local directory /home/outputs/ is automatically uploaded to the corresponding OSS directory after the program finishes running.

    4. View the job running status and running results

    You can run the following commands to monitor your jobs:

    1. bcs j   # Obtain the job list. The job list obtained each time is cached. Generally, the first job in the cache is the one you only submitted.
    2. bcs j 1  # View the details of the first job in the cache
    3. bcs log 1 # Check the log of the first job in the cache.

    You can run the following command to view the results:

    1. bcs o cat oss://your-bucket/sum/outputs/output.txt