Microservices Engine:Job management

Create a job

1. Log on to Distributed Task scheduling platform.

2. Log on to the EDAS console.

3. In the left-side navigation pane, click Task Scheduling (SchedulerX).

4. In the top navigation bar, select a region.

5. On the Task Overview page, click Jobs in the left-side navigation pane.

6. In the left-side-navigation pane. click Task Management.

7. On the JobsTask Management page, select a namespace and click Create task. In the Basic Configuration step, configure the parameters of the scheduling job and click Next Step.

   

   The following table describes the parameters.

   Parameter	Description
   Task name	The name of the job.
   Description	The description of the job. We recommend that you use simple descriptions to facilitate job management.
   Application ID	The group to which the job belongs. Select an application ID from the drop-down list.
   Task type	The programming language that you want to use to create the job. Valid values: java, xxljob, springschedule, golang, ScriptTask, http, and dataworks. If you select ScriptTask, you must select shell, python, or go for the Script parameter, and enter a script in the editor that appears.
   Class full path	Note This parameter is available only if the Task type parameter is set to java. The full path of the JobProcessor. Example: xxx.xxx.xxx.HelloProcessor.
   Execution mode	The mode in which the job is executed. Valid values: Stand-alone operation: The job is executed on a random worker. Broadcast run: The job is concurrently executed on all workers, and the system waits until all workers complete the job. Visual MapReduce: a Map model. No more than 300 jobs are allowed in the job list. In Professional Edition, a maximum of 1,000 jobs are allowed, and jobs can be queried by keyword. MapReduce: a regular MapReduce model. In this mode, you can process a large number of jobs in parallel. But you can query only the summary information of jobs. We recommend that you select this mode if the number of jobs is less than 1,000,000. Shard run: This mode is similar to the elastic-job model. Shards are evenly distributed to run on multiple agents based on the specified sharding settings. This execution mode supports jobs that use different programming languages. Note The advanced settings of a job vary based on the execution mode of the job.
   Task parameters	An arbitrary string that can be obtained from the context when SchedulerX runs the job.

   The following table describes the parameters for advanced configurations.

   Parameter	Applicable execution mode	Description	Default value
   Instance failed retries	All modes	The number of automatic retries if a job fails. Note If a job is running on a worker and the worker is restarted, the job fails. If you want to immediately rerun the job, you can specify this parameter.	0
   Instance failure retry interval	All modes	The interval between two consecutive retries. Unit: seconds.	30
   Instance concurrency	All modes	The number of instances that run the same job at the same time. A value of 1 indicates that concurrent execution is not allowed.	1
   Number of single-machine concurrent subtasks	Parallel computing Memory grid Grid computing	The number of tasks that concurrently run on a single worker in a distributed model.	5
   Number of failed retries of subtasks	Parallel computing Memory grid Grid computing	The number of automatic retries if a task fails in a distributed model.	0
   Sub-task failure retry interval	Parallel computing Memory grid Grid computing	The interval between two consecutive retries if a task fails in a distributed model. Unit: seconds.	0
   Subtask distribution method	Parallel computing Memory grid Grid computing	Push model: Tasks are evenly distributed to workers. Pull model: Every worker pulls tasks. The Wooden Bucket Theory is not applicable to this model. During the pull process, all tasks are cached on the master node. This puts pressure on the memory. We recommend that you do not distribute more than 10,000 tasks at a time.	Push model
   Number of subtasks pulled per time (applicable only if the Subtask distribution method is set to Pull model)	Parallel computing Memory grid Grid computing	The number of tasks that a slave node pulls from the master node at a time.	5
   Subtask queue capacity (applicable only if the Subtask distribution method parameter is set to Pull model)	Parallel computing Memory grid Grid computing	The size of the queue that caches tasks on a slave node.	10
   Global concurrency of subtasks (applicable only if the Subtask distribution method is set to Pull model)	Parallel computing Memory grid Grid computing	The total number of concurrent tasks on all workers in the pull model. This parameter helps you limit the number of concurrent tasks.	1,000

8. In the Timing configuration step, configure scheduling settings and advanced settings and click Next Step.

   

   The following table describes the parameters for job scheduling.

   Parameter	Description
   Time type	none: The job is triggered by using a workflow. cron: The job is triggered based on a CRON expression. api: The job is triggered by calling an API operation. fixed_rate: The job is triggered at an interval. second_delay: The job is triggered with a delay from 1 to 60 seconds. onetime: The job is triggered only once at the specified point in time. unix cron: The job is triggered based on a CRON expression that is available in the Unix system.
   cron expression	Enter a CRON expression that complies with the cron syntax or use the tool that is provided by SchedulerX to generate an expression. Then, verify the CRON expression. This parameter is available only if the Task type parameter is set to cron.
   Fixed frequency	Specify the interval at which the job is triggered. This parameter is available only if the Task type parameter is set to fixed_rate. The value must be greater than 60. Unit: seconds. For example, a value of 200 indicates that the job is triggered at an interval of 200 seconds.
   Fixed delay	Specify a delay. This parameter is available only if the Task type parameter is set to second_delay. Valid values: 1 to 60. Unit: seconds. For example, a value of 5 indicates that the job is triggered with a delay of five seconds.
   Scheduling time	Specify the date and point in time at which the job is triggered. This parameter is available only if the Time type parameter is set to onetime. Example: 2023-7-13 00:00:00.
   unix cron expression	Enter a CRON expression that is available in the UNIX system. This parameter is available only if the Time type parameter is set to unix cron.

   You can also configure advanced settings. The following table describes the parameters.

   Parameter	Description
   Time offset	The offset between the timestamp of the data to be processed and the time when the job is triggered. You can obtain the offset value from the context when SchedulerX runs a job.
   Time zone	Select the time zone of a country or region. You can also select a UTC time zone based on your business requirements.

9. In the Notification configuration step, configure alert settings and contacts. Then, click Complete.

   

10. Return to the Task Management page and check whether the job exists and uses the specified configurations.

    Note

    By default, a job is enabled after the job is created. You can enable or disable a job based on your business requirements. For more information, see the Enable or disable jobs section of this topic.

Modify a job

1. On the Task Management page, find the job that you want to modify and click Edit in the Operation column of the job.

   

2. In the Edit panel, modify the basic configurations, scheduling settings, and alerting settings.

   Note

   You cannot change the values of the Application ID and Task type parameters of a job. You can configure other parameters in the same way as you configure them when you create a job.

Run a job

On the Task Management page, find the job that you want to run and click Run once in the Operation column to run the job once.

Enable or disable jobs

Enable or disable a single job

1. On the Task Management page, find the job that you want to manage and click the icon in the Operation column. Then, select Disable or Enable.

   

2. In the dialog box that appears, click OK.

Enable or disable multiple jobs at a time

In Professional Edition, you can enable or disable multiple jobs at a time. On the Task Management page, select the jobs that you want to manage and click Batch disable or Batch enable below the job list.

Copy a job

On the Task Management page, find the job that you want to copy and click the icon in the Operation column. Then, select Replication. You can copy the configurations of a job and modify the configurations of the copied job to create a new job.

Update job outputs

If your workloads are updated, you can run the corresponding job again to reprocess the data within a specific time period. For example, if you added a new field to the database or the data that was generated at a point in time of the previous month contains errors, you can update the output of the corresponding job.

If the output of a job contains deviations or some values in the output are missing, you can modify the scheduling parameters of the job and configure a specific job property to update the job output.

1. Log on to Distributed Task scheduling platform.

2. On the Task Management page, find the job whose output you want to update and click the icon in the Operation column. Then, click Rebrush task.

3. In the Rebrush task panel, specify the Start and end date and Data Time parameters and click OK.

   • Start and end date: specifies the time range of the job.

   • Data Time: specifies the point in time of the data that you want to update.

   

   Sample configurations:

   • The current time is 10:00:00 on January 1, 2019.

   • The time range of the job is from October 1, 2018 to October 7, 2018. The default start time is 00:00:00 on October 1, 2018 and the default end time is 23:59:59 on October 7, 2018.

   • The point in time of the data to be reprocessed is 11:11:11.

   The job is rerun seven times and seven job instances are generated.

   Number	Scheduling time	Point in time at which data is reprocessed
   1	2019.1.1 10:00:00	2018.10.1 11:11:11
   2	2019.1.1 10:00:00	2018.10.2 11:11:11
   3	2019.1.1 10:00:00	2018.10.3 11:11:11
   4	2019.1.1 10:00:00	2018.10.4 11:11:11
   5	2019.1.1 10:00:00	2018.10.5 11:11:11
   6	2019.1.1 10:00:00	2018.10.6 11:11:11
   7	2019.1.1 10:00:00	2018.10.7 11:11:11

View the operation records and execution records of a job

• View execution records: On the Task Management page, find the job that you want to view and click the icon in the Operation column. Then, select Historical records.

• View operation records: On the Task Management page, find the job that you want to view and click the icon in the Operation column. Then, select Operation Record.

Delete a job

1. On the Task Management page, find the job that you want to delete and click the icon in the Operation column. Then, select Delete.

   

2. In the Delete task dialog box, click OK.

Import or export a job

1. On the Task Management page, select an application of Professional Edition.

   

2. Click Import Task or Export Tasks.