Parameter description
All classes of parameters are converted to dictionary objects containing the attribute information.
Parameter | Type | Description |
job_desc | JobDescription object, str, dict | Simple description of the job, description of each task of the job object, and DAG dependencies between tasks |
Response description
The create_job method returns a CreateResponse object. The following lists the attributes of the CreateResponse object. You can use response.Id to obtain the ID of the new job.
Attribute | Type | Description |
Id | str | ID of the new job |
For example:
try:
# Init a Client object.
client = ...
# Refer to JobDescription.
job_desc = JobDescription(...)
job = client.create_job(job_desc)
# Print out the job id.
print (job.Id)
except ClientError, e:
print (e)JobDescription class
Parameter description
Parameter | Type | Description |
properties | dict, str, JobDescription object | Object that contains the job description |
Attribute description
Attribute | Type | Description |
Name | str | Job name |
Description | str | Brief description of a job |
Priority | int | The value is an integer ranging from 0 to 1000. Priority increases with the value |
Notification | dict | Notification configuration. You can configure the topic or job related events of the MNS |
JobFailOnInstanceFail | bool | Specifies whether the failure of an instance will cause the failure of the related job |
AutoRelease | boolean | Specifies whether the system automatically releases (deletes) a job after the job runs successfully. Default value: False |
Type | str | Currently, a task can be described only in the form of a directed acyclic graph (DAG) |
DAG | dict, DAG object | DAG description |
2. DAG class
Parameter description
Parameter | Type | Description |
properties | dict, str, DAG object | Descriptions of task mapping and dependencies between all tasks |
Attribute description
Attribute | Type | Description |
Tasks | dict | Mapping between all tasks names and task descriptions |
Dependencies | dict | Dependencies between all tasks |
Method description
Method | Description |
add_task(task_name, task) | Adds a task |
get_task(task_name) | Obtains information about a task, based on the task name |
delete_task(task_name) | Deletes a task |
For example:
# Refer to the TaskDescription class.
find_task = TaskDescription(...)
task_dag = DAG()
task_dag.add_task(task_name='Find', task=find_task)
task_dag.Dependencies = {}3. TaskDescription class
Parameter description
Parameter | Type | Description |
properties | dict, str, TaskDescription object | Description of a single task |
Attribute description
Attribute | Type | Description |
Parameters | dict, Parameters object | Task parameter details |
InputMapping | dict | Mapping from the OSS directory to the local directory |
OutputMapping | dict | Mapping from the local directory to the OSS directory |
LogMapping | dict | Mapping from the local log directory to the OSS directory |
Timeout | int | Task timeout |
InstanceCount | int | Number of instances in a task. It is a positive integer |
MaxRetryCount | int | Maximum number of retries. Default value: 0 |
ClusterId | str | Cluster ID |
Mounts | dict, Mounts object | Network mounting configurations of an instance, which is described by Mounts. Currently, NAS mounting and OSS mounting are supported |
AutoCluster | dict, AutoCluster object | You can select only one of autoCluster and clusterId |
For example:
echo_task = TaskDescription()
# Create map task.
# Refer to Parameters documentation.
echo_task.Parameters.Command.CommandLine = "echo Batchcompute Service"
echo_task.Parameters.Command.PackagePath = ""
echo_task.Parameters.StdoutRedirectPath = "oss://xxx/xxx/"
echo_task.Parameters.StderrRedirectPath = "oss://xxx/xxx/"
echo_task.Timeout = 86400 # 1-day
echo_task.InstanceCount = 3
echo_task.MaxRetryCount = 1
# cluster_id is a cluster created in batchcompute service before.
echo_task.ClusterId = cluster_id
# You can now add this task description to a DAG object through its add_task method.4. Parameters class
Parameter description
Parameter | Type | Description |
properties | dict, str, Parameters object | Description of task parameters |
Attribute description
Attribute | Type | Description |
Command | dict, Command object | Command line parameters related to a user program |
InputMappingConfig | dict, InputMappingConfig object | Configuration of the NFS mounting service |
StdoutRedirectPath | str | OSS directory to which standard output is redirected |
StderrRedirectPath | str | OSS directory to which a standard error is redirected |
5. AutoCluster class
Parameter description
Parameter | Type | Description |
properties | dict, str, AutoCluster object | AutoCluster information |
Attribute description
Attribute | Type | Description |
ECSImageId | str | ECS image ID. Images provided by the system can be used. |
InstanceType | str | Instance type |
ResourceType | str | Resource type. Currently, the OnDemand and Spot types are supported. Default value: OnDemand |
UserData | dict | Key-Value mapping, which is defined by the user and obtained by using metaserver of the ECS instance |
Configs | Configs object | Cluster configurations. For more information, see the introduction to ClusterDescription in Section 4.13. |
SpotStrategy | str | Spot strategy for an instance. It takes effect only when ResourceType is Spot.Values: SpotWithPriceLimit (Set the maximum price for the spot instance) and SpotAsPriceGo (The system provides a price automatically, and the maximum value is the Pay-As-You-Go price). |
SpotPriceLimit | float | Maximum price of an instance per hour (the price of each instance type, not the price per core*hour). The value can contain at most three decimals. It takes effect when SpotStrategy is SpotWithPriceLimit. |
ReserveOnFail | bool | When a task fails, related VMs are not released, and the fees for these resources are still charged until the user deletes the job. The default value is False. It is used only in a survey. |
6. Command class
Parameter description
Parameter | Type | Description |
properties | dict, str, InputMappingConfig object | Configuration of the NFS mounting service |
Attribute description
Attribute | Type | Description |
CommandLine | str | Specifies a command for executing a user program |
PackagePath | str | OSS directory in which a user program resides |
EnvVars | dict | Environment variable for execution of a user program |
7. InputMappingConfig class
Parameter description
Parameter | Type | Description |
properties | dict, str, InputMappingConfig object | Configuration of the NFS mounting service |
Attribute description
Attribute | Type | Description |
Locale | str | Character set used when an OSS object is mounted to the local server. Optional values include GBK, GB2312-80, BIG5, ANSI, EUC-JP, EUC-TW, EUC-KR, SHIFT-JIS, and KSC5601. |
Lock | bool | Specifies whether the NFS mounting service supports the network file lock. |
8. Notification class
Parameter description
Parameter | Type | Description |
properties | dict, str, Command object | Command line parameters related to a user program |
Attribute description
Attribute | Type | Description |
Topic | Topic Object | Message topic |
9. Topic class
Parameter description
Parameter | Type | Description |
properties | dict, str, Command object | Command line parameters related to a user program |
Attribute description
Attribute | Type | Description |
Endpoint | str | Endpoint of the MNS region. The format is http://${your_user_id}.mns.${region}-internal.aliyuncs.com/. Use the intranet endpoint whenever possible. |
Name | str | Topic name |
Events | list | List of events. Make sure that you enter the cluster related events. |
10. Mounts class
Parameter description
Parameter | Type | Description |
properties | dict, str, Command object | Network disk mounting configurations when a cluster is created |
Attribute description
Attribute | Type | Description |
Entries | array | List of network disk mount points, which is described by MountPoint. |
Locale | str | Language used when OSS or NAS storage is mounted |
Lock | bool | Specifies whether the file lock is supported when OSS or NAS storage is mounted |
NAS | dict | NAS configuration |
OSS | dict | OSS configuration |
11. MountPoint class
Parameter description
Parameter | Type | Description |
properties | dict, str, Command object | Network mount point |
Attribute description
Attribute | Type | Description |
Source | str | Source directory for network disk mounting. It can be a string starting with “nas://“ or “oss://“. |
Destination | str | Local mount point directory of the network disk |
WriteSupport | bool | Specifies whether the mount point is writable. |
12. NAS class
Parameter description
Parameter | Type | Description |
properties | dict, str, Command object | NAS configuration |
Attribute description
Attribute | Type | Description |
AccessGroup | list | NAS access group to which the cluster instance must be added |
FileSystem | list | File system to be accessed |
13. OSS class
Parameter description
Parameter | Type | Description |
properties | dict, str, Command object | OSS configuration |
Attribute description
Attribute | Type | Description |
AccessKeyId | str | AccessKey ID used for OSS mounting |
AccessKeySecret | str | AccessKeySecret used for OSS mounting |
SecurityToken | str | Security token used for OSS mounting |
Note: Take the following precautions for Mounts: The Job-level Mounts configuration overwrites the cluster-level Mounts configuration. After a cluster is modified, the RecreateInstance interface must be called to make the new Mounts configuration take effect. During NAS mounting, the source directory must start with “nas:”. Otherwise, an error occurs.