1. Variables for task programs
Batch Compute provides the following variables for task programs:
Variable | Value |
BATCH_COMPUTE_DAG_JOB_ID | Job ID, depending on the actual situation |
BATCH_COMPUTE_DAG_TASK_ID | Task ID, depending on the actual situation |
BATCH_COMPUTE_DAG_INSTANCE_ID | Instance ID, depending on the actual situation |
BATCH_COMPUTE_OSS_HOST | OSS host name, depending on the actual situation |
BATCH_COMPUTE_REGION | Region where the host is located, depending on the actual situation |
BATCH_COMPUTE_CLUSTER_ID | cluster id |
BATCH_COMPUTE_WORKER_ID | worker id |
See the following table for the difference of variables for task programs running in a Docker container:
Variable | Value |
USER | root |
PWD | /batchcompute/workdir |
PATH | /sbin:/usr/sbin:/bin:/usr/bin. |
HOME | /root |
BATCH_COMPUTE_DAG_JOB_ID | Job ID, depending on the actual situation |
BATCH_COMPUTE_DAG_TASK_ID | Task ID, depending on the actual situation |
BATCH_COMPUTE_DAG_INSTANCE_ID | Instance ID, depending on the actual situation |
BATCH_COMPUTE_OSS_HOST | OSS host name, depending on the actual situation |
BATCH_COMPUTE_REGION | Region where the host is located, depending on the actual situation |
2. How to use variables
You can obtain variables from EnvVars in task programs, for example:
2.1. Use variables in a Python program:
task_id = os.environ['BATCH_COMPUTE_DAG_TASK_ID']
instance_id = os.environ['BATCH_COMPUTE_DAG_INSTANCE_ID']2.2. Use variables in a Java program:
String taskId = System.getenv("BATCH_COMPUTE_DAG_TASK_ID");
String instanceId = System.getenv("BATCH_COMPUTE_DAG_INSTANCE_ID");3. Define variables
In addition to variables provided by the system, you can also define new variables when submitting a job.
3.1. Use Python SDK
Code snippet:
env = {
'k1': 'v1',
'k2': 'v2'
}
...
job_desc['DAG']['Tasks']['my-task']['Parameters']['Command']['EnvVars']=env
...3.2. Use Java SDK
Code snippet:
Command cmd= new Command();
cmd.addEnvVars("k1","v1");
cmd.addEnvVars("k2","v2");
...
TaskDescription desc = TaskDescription();
Parameters params = new Parameters();
params.setCommand(cmd);
...
desc.setParameters(params);3.3. Use command line tool
bcs sub "python main.py" -e k1:v1,k2:v2