MaxCompute:Manage jobs

Operations

• Filter jobs

  You can configure filter parameters to query the details of jobs. The following table describes the filter parameters.

  Filter parameters of a job (Expand for details)

  Parameter	Description
  Time Range	Filters jobs by time range. You must specify the start time and end time of a query to filter jobs. This parameter is required. Note This parameter specifies a filter condition to filter all jobs. The chart that displays job statistics and the job list change based on the configuration of this parameter. You can specify the time range to obtain information about the following jobs: Jobs that are complete within this time range. Jobs that are running at the end time of the query or within the last 3 minutes before the end time. You can obtain the job snapshots that are generated within the last 3 minutes. The 3-minute time range indicates the time range for you to observe job snapshots. The default time range is the last 1 hour. The maximum time range is 7 days and the minimum time range is 2 minutes. You can search for only jobs in the last 45 days. You can select a preset time range. You can also click the Time Range field and specify a time range in the panel that appears. The following options are available: 1h: the previous 1 hour 12h: the previous 12 hours 1d: the previous 1 day Select a time range: Select the year, month, and day that you want to query, and click Select Time to select a time range.
  Choose Project	Filters jobs by project. Note This parameter specifies a filter condition to filter all jobs. The chart that displays job statistics and the job list change based on the configuration of this parameter. You can select multiple MaxCompute projects at the same time. This parameter is empty by default.
  Select Quota	Filters jobs by quota group. Note This parameter specifies a filter condition to filter all jobs. The chart that displays job statistics and the job list change based on the configuration of this parameter. You can select only subscription quota groups. This parameter is left empty by default. Note You do not need to specify this parameter when you query pay-as-you-go jobs. For more information about quota groups, see Manage quotas in the new MaxCompute console.
  Job Type	Filters jobs by job type. Note This parameter specifies a filter condition to filter all jobs. The chart that displays job statistics and the job list change based on the configuration of this parameter. Valid values: SQL: SQL jobs SQLRT: MaxCompute Query Acceleration (MCQA) SQL jobs LOT: MapReduce jobs CUPID: Spark or Mars jobs AlgoTask: Platform for AI (PAI) jobs Graph: graph jobs
  Instance ID	Filters jobs by instance ID. You can enter the instance ID of the job that you want to find to perform an exact match. Note You can specify this parameter to filter jobs in the job list. The job list changes based on the configuration of this parameter. This parameter is empty by default. For more information about instance IDs, see View instance information.
  Job Owner	Filters jobs by submitter. A submitter indicates an account that submits MaxCompute jobs. Note You can specify this parameter to filter jobs in the job list. The job list changes based on the configuration of this parameter. This parameter is empty by default. Fuzzy match is not supported. The value of this parameter must be a complete account name, such as ALIYUN$xxx or RAM$xxx.
  ExtNodeId	Filters jobs by job ID of the data source, Note You can specify this parameter to filter jobs in the job list. The job list changes based on the configuration of this parameter. such as a node ID of DataWorks. For more information about node IDs of DataWorks, see Configure basic properties.
  Signature	Filters jobs by the signature of SQL jobs. Note You can specify this parameter to filter jobs in the job list. The job list changes based on the configuration of this parameter. This parameter is available only for SQL jobs. You can use the signature to find the instances on which each time an SQL statement is executed. This parameter is empty by default.
  Latest Status	Filters jobs by job state. Note You can specify this parameter to filter jobs in the job list. The job list changes based on the configuration of this parameter. Valid values: Running: The job is running or is not complete. Success: The job is complete. Failed: The job fails to run. Canceled: The job is canceled. Submitted: The job is submitted and waiting for computing resources. This parameter is empty by default. This indicates that jobs in all states are queried. Note The state in the Latest Status column is the overall state of a job. However, multiple jobs may be run in parallel and each job can be in a specific substate. You can go to the LogView page to view the job states. For more information, see Use LogView V2.0 to view job information.
  Intelligent diagnosis	Filter jobs by the labels from the intelligent diagnosis result. By default, no labels are selected. For detailed meanings of the intelligent diagnosis result labels, see Intelligent diagnostics for jobs.

• Sort jobs

  The job filtering results are sorted by job completion time in descending order, with unfinished jobs appearing at the top. Basic single-column sorting and advanced multi-column sorting are supported.

  • Basic single-column sorting: Sort the column with a sort button in the list in ascending or descending order.

  • Advanced multi-column sorting: Click the Advanced Sorting button in the upper-right corner of the list, add columns by clicking Add Sort, and specify the sort order such as ascending and descending for each column. Click OK to apply the multi-column sorting.

  Note

  When advanced sorting conditions are applied, basic single-column sorting cannot be performed. You need to click the Advanced Sorting button in the upper-right corner of the list, then click Reset and OK before you can perform basic single-column sorting again.

• View job details

  To view the details of a job, perform the following steps: In the job list, find the desired job and click LogView in the Actions column to go to the LogView page. On the page that appears, view the status, details, and results of the job.

• Terminate jobs

  You can terminate one or more jobs that are in the Running state at a time.

• Jobs Insight

  You can perform insight operations on individual jobs to view job summaries, resource consumption, and resource allocation for Quota at a specific point in time, as well as to trigger the Intelligent diagnostics for jobs.

  Note

  • Intelligent diagnostics are available exclusively for SQL jobs.

  • Jobs with a runtime less than 2 minutes or jobs of types other than SQL, MapReduce, Spark, and Mars do not have job-level resource consumption data.

View the chart that displays job statistics

The chart displays the number of jobs on a stacked column chart by time and job state based on the query results. This helps you view the overall status of a job.

Jobs

Query results are obtained based on filter conditions and provide job information for you to manage jobs.

View the details about a job in a specific time range

• Scenario

  You want to view the jobs that are managed on the last day for the Project_1 and Project_2 projects, identify failed jobs, and troubleshoot errors.

• Procedure

  1. Log on to the MaxCompute console. In the left-side navigation pane, click Jobs.

  2. On the Jobs page, set the Time Range parameter to 1d or set the time range from the current time of the last day to the current time of the current day.

  3. Select Project_1 and Project_2 from the Choose Project drop-down list.

  4. In the query results, you can find the desired instance and click LogView in the Actions column to view the details of the job on the LogView page. For more information about LogView, see Use LogView V2.0 to view job information.

View the resources occupied by a job with a subscription quota at a specific point in time

• Scenario

  A large number of resources in the quota group named Subscription Default Quota are occupied. As a result, multiple jobs are waiting for the resources of the quota group. You can use the following method to view the jobs that use the quota.

• Procedure

  1. Log on to the MaxCompute console. In the left-side navigation pane, click Jobs.

  2. Set the Time Range parameter to 1h. Alternatively, specify a custom start time and set the end time to the current time. The end time is the time when you observe the job.

  3. Set the Select Quota parameter to Subscription Default Quota.

  4. Click Search.

     You can view the CPU Utilization Percentage Snapshot and Memory Usage Percentage Snapshot parameters of the jobs whose Snapshot Status is Running in the query results. You can check whether the job that has large values of the CPU Utilization Percentage Snapshot and Memory Usage Percentage Snapshot parameters meets your business requirements. You can determine whether the job runs as expected or whether the job needs to be terminated based on other job information.

     Note

     In the query results, you can find the desired instance and click LogView in the Actions column to view the details of the job on the LogView page. For more information about LogView, see Use LogView V2.0 to view job information.

View details of an MCQA job

• Scenario

  You want to view the status and details of the MCQA job in the last day.

• Procedure

  1. Log on to the MaxCompute console. In the left-side navigation pane, click Jobs.

  2. Set the Time Range parameter to 1d and select SQLRT (Query Acceleration) from the Job Type drop-down list.

  3. Click Search.

     In the query results, you can find the desired instance and click LogView in the Actions column to view the details of the job on the LogView page. For more information about LogView, see Use LogView V2.0 to view job information.

     Note

     For MCQA jobs, multiple SQL statements may be executed in the same session. One session corresponds to one instance ID. You can click an instance ID to view the status of all SQL statements in a session on the LogView page. Take note of the following issues when you query a job of this type on the Operation and Maintenance page:

     • An active session indicates that some SQL statements are still being executed. If a session remains active, the job is in the Running state.

     • If a session expires or is closed, the job is in the Canceled state.

View the resource consumption of a job and the resource allocation of computing quotas at a specific point in time

• Scenario

  If a job is not complete for a long period of time and you cannot locate the cause on the LogView page, you can analyze the job to check whether the issue occurs due to insufficient resources. After the job is complete, if the job runs at a low speed, you can analyze the job to check whether the issue occurs due to insufficient resources.

• Procedure

  1. Log on to the MaxCompute console. In the left-side navigation pane, click Jobs.

  2. On the Jobs page, specify the Time Range and Select Quota parameters and click Search to filter MaxCompute jobs.

  3. In the obtained results, find the desired job and click Insight in the Actions column to go to the Job Insights page.

  4. On the CU Usage tab, you can view the resource consumption in the lifecycle of the job.

     • You can view the trend of the number of used compute units (CUs) and the number of CUs that wait to be used by a job within a specific period of time, and the trend of the CU metrics at the quota group level within a specific period of time based on the resource consumption chart. If the number of CUs used by a job is small, but the number of CUs used by a job in a quota group is large or even continuously reaches the upper limit, the resources in the quota group are insufficient. In this case, other jobs preempt computing resources from the current job.

     • You can click a time point in the horizontal axis of the resource consumption trend chart to view the resource allocation in the quota group at the point in time. You can view the number of jobs that are using CUs and the number of jobs that wait to use CUs and view the statistics on the priorities of existing jobs. You can click the legend that corresponds to the desired priority to go to the job list and view the details of the jobs. This way, you can identify the jobs that preempt computing resources from the current job. You can adjust job priorities or manage computing resources to optimize job execution based on your business requirements. For more information, see Job priority or Manage quotas in the new MaxCompute console.

A job occupies an excessive number of resources and affects the execution of other jobs.

• If the job does not meet your business requirements, you can terminate the job.

• If the job meets your business requirements, invalid settings of the resources in the quota group exist. In this case, you must optimize the resource configuration plan. For more information, see Computing cost optimization.