SchedulerX allows you to perform periodic scheduling for programs, run scripts in different programming languages, and call HTTP operations. SchedulerX also supports scheduling for native Kubernetes jobs and pods. This topic describes how to deploy SchedulerX in a Kubernetes cluster.
Scenarios
When you use SchedulerX to schedule a Kubernetes job, SchedulerX provides the following benefits:
Online script editing in pod mode
In most cases, Kubernetes jobs run in data processing and O&M scenarios based on scripts. If you do not use SchedulerX to schedule a Kubernetes job, you must package the script of the job into an image and configure the script commands in a YAML file. If you want to modify the script, you must build a new image and republish the image. The following sample code provides an example:
If you use SchedulerX to schedule a Kubernetes job, you do not need to build a new image or configure script commands in a YAML file. You need to only edit scripts in the SchedulerX console. Scripts in the following programming languages are supported: Shell, Python, PHP, and Node.js. Then, SchedulerX automatically runs the scripts in pod mode. If you want to modify a script, you need to only re-edit the script in the SchedulerX console. The modifications automatically take effect the next time you schedule the Kubernetes job. This improves the efficiency of developing Kubernetes jobs. In addition, if you schedule a Kubernetes job by using SchedulerX, the operations on containers are imperceptible to you. This allows users who are not familiar with the underlying concepts of containers to schedule Kubernetes jobs with ease.
Visualized job orchestration
Argo Workflows is a mainstream solution for orchestrating the workflows of jobs on Kubernetes. Sample code:
SchedulerX provides a GUI on which you can orchestrate Kubernetes jobs by performing drag-and-drop operations. Compared with Argo Workflows, the mainstream solution for code-based workflow orchestration, SchedulerX provides more convenience. When jobs are running, SchedulerX displays directed acyclic graphs (DAGs) that visualize the workflows of the jobs to help you troubleshoot issues of blocked jobs and improve O&M efficiency.
Monitoring and alerting
If you use SchedulerX to schedule a pod or a job, the monitoring and alerting feature of SchedulerX is available.
Alerting method: SchedulerX sends alert notifications by using text messages, phone calls, emails, and Webhook URLs of DingTalk chatbots, WeCom bots, and Lark bots.
Alerting policy: SchedulerX triggers alerts when a job fails to be executed or the execution of a job times out.
Log collection
If you use SchedulerX to schedule a pod or a job, SchedulerX automatically collects the operational logs of the pod or job. If a pod fails to run, you can troubleshoot the issue based on the logs in the SchedulerX console.
Monitoring dashboard
On the Overview page of the SchedulerX console, you can view your jobs in real time.
Mixed deployment of online and offline jobs
SchedulerX allows you to deploy and schedule online and offline periodic jobs of the Java and Kubernetes types in a mixed manner. A business application may involve multiple periodic jobs. If you frequently schedule periodic jobs, you may deploy them in the same process as your business application. In this case, the CPU resources and memory of the application are consumed. The periodic jobs are not isolated from your online business. If a periodic job consumes a large number of resources and is scheduled at a low frequency, such as once an hour or once a day, you can create a pod to run the job. This allows the job to run in a different process from your online business.
Method 1 (recommended): Deploy SchedulerX by using a Deployment
For non-Java applications, you can deploy schedulerx-agent.yaml by using a Deployment. This way, SchedulerX is deployed as a separate pod. The following figure shows how SchedulerX works in this case.
Prerequisites
A Container Service for Kubernetes (ACK) cluster or a self-managed Kubernetes cluster is created. For more information, see Create an ACK managed cluster.
SchedulerX is activated.
A Kubernetes application is created in the SchedulerX console. For more information, see Create an application.
Step 1: Configure a service account
A service account provides an identity for validation and authorization when a Kubernetes job is scheduled by using SchedulerX. By default, the service account of the namespace to which the Kubernetes job belongs is used.
You need to run schedulerx-serviceaccount.yaml only once in the required namespace of the Kubernetes cluster. If you want to schedule only pods or jobs that belong to the current namespace, run the following YAML code:
If you want to schedule jobs across namespaces, you must call the ClusterRole and ClusterRoleBinding API operations.
Step 2: Install the SchedulerX agent
The following sample code shows the configurations of schedulerx-agent.yaml:
The following table describes the valid values of image.
Chip architecture | Location | Description |
X86_64 | Regions and zones in China | registry.cn-hangzhou.aliyuncs.com/schedulerx/agent:latest-amd64 |
Regions and zones outside China | schedulerx-registry.ap-southeast-1.cr.aliyuncs.com/schedulerx/agent:latest-amd64 | |
ARM64 | Regions and zones in China | registry.cn-hangzhou.aliyuncs.com/schedulerx/agent:latest-arm64 |
Regions and zones outside China | schedulerx-registry.ap-southeast-1.cr.aliyuncs.com/schedulerx/agent:latest-arm64 |
The following table describes the variables contained in env.
Parameter | Description |
${SCHEDULERX_ENDPOINT} | ENDPOINT specifies the endpoint matched with the region where your application is deployed. Example: addr-sh-internal.edas.aliyun.com. For more information, see Endpoints. |
${SCHEDULERX_NAMESPACE} | NAMESPACE specifies the ID of the namespace to which the application belongs. You can view the namespace ID on the Namespace page in the SchedulerX console. |
${SCHEDULERX_GROUPID} | GROUPID specifies the application ID. You can view the application ID on the Application Management page in the SchedulerX console. |
${SCHEDULERX_APPKEY} | APPKEY specifies the application key. You can view the application key on the Application Management page in the SchedulerX console. |
After you deploy the Deployment, you can check whether an instance is available on the Application Management page in the SchedulerX console. If an instance is available, the SchedulerX agent is connected to the Kubernetes cluster.
Method 2: Deploy SchedulerX by using a Helm package
Prerequisites
An ACK cluster or a self-managed Kubernetes cluster is created. For more information, see Create an ACK managed cluster.
SchedulerX is activated.
A Kubernetes application is created in the SchedulerX console. For more information, see Create an application.
The Helm package management tool is installed.
Step 1: Download the SchedulerX Helm package
Run the following command to download the SchedulerX Helm package:
wget https://schedulerx2.oss-cn-hangzhou.aliyuncs.com/helm/schedulerxchart-2.0.0.tgz
Step 2: Install the SchedulerX Helm package
Obtain the access parameters of the application in the SchedulerX console.
Log on to the SchedulerX console.
In the top navigation bar, select a region.
In the left-side navigation pane, click Application Management.
On the Application Management page, click Access configuration in the Operation column. In the upper-left corner of the Access configuration panel, select k8s from the drop-down list.
Run the following installation commands:
NoteReplace the access parameters in the commands with the access parameters of the application that you want to schedule.
By default, the image address in the access configuration is the address of the image of the AMD architecture on the Internet. Specify an image address based on the region where your machine resides and the architecture of your machine.
helm install schedulerxchart schedulerxchart-2.0.0.tgz \ --set SCHEDULERX_ENDPOINT=acm.aliyun.com\ ,SCHEDULERX_NAMESPACE=f856c3f8-a15c-4a7e-9b4e-f812a9f8****\ ,SCHEDULERX_GROUPID=k8s-test3\ ,SCHEDULERX_APPKEY=****\ ,SCHEDULERX_AGENT_IMAGE=registry.cn-hangzhou.aliyuncs.com/schedulerx/agent:latest-amd64
The following figure shows the installation process.
The following table describes the valid values of image.
Chip architecture
Location
Description
x86_64
Regions and zones in China
registry.cn-hangzhou.aliyuncs.com/schedulerx/agent:latest-amd64
Regions and zones outside China
schedulerx-registry.ap-southeast-1.cr.aliyuncs.com/schedulerx/agent:latest-amd64
arm64
Regions and zones in China
registry.cn-hangzhou.aliyuncs.com/schedulerx/agent:latest-arm64
Regions and zones outside China
schedulerx-registry.ap-southeast-1.cr.aliyuncs.com/schedulerx/agent:latest-arm64
The following table describes the variables contained in env.
Parameter
Description
${SCHEDULERX_ENDPOINT}
ENDPOINT specifies the endpoint matched with the region where your application is deployed. Example: addr-sh-internal.edas.aliyun.com. For more information, see Endpoints.
${SCHEDULERX_NAMESPACE}
NAMESPACE specifies the ID of the namespace to which the application belongs. You can view the namespace ID on the Namespace page in the SchedulerX console.
${SCHEDULERX_GROUPID}
GROUPID specifies the application ID. You can view the application ID on the Application Management page in the SchedulerX console.
${SCHEDULERX_APPKEY}
APPKEY specifies the application key. You can view the application key on the Application Management page in the SchedulerX console.
SCHEDULERX_AGENT_IMAGE
The image URL.
After you install the package, you can view the instance in the SchedulerX console.
Method 3: Deploy SchedulerX by using SchedulerX SDK for Java
For Java applications, you may want to schedule both Kubernetes jobs and Java programs. In this case, you can deploy SchedulerX by using SchedulerX SDK for Java. SchedulerX and your online business share the same process. The following figure shows the workflow.
Prerequisites
An ACK cluster or a self-managed Kubernetes cluster is created. For more information, see Create an ACK managed cluster.
A Kubernetes application is created in the SchedulerX console. For more information, see Create an application.
Connect your application to SchedulerX
Use SchedulerX SDK for Java to connect your application to SchedulerX. For more information, see Connect a Spring Boot application to SchedulerX.
Add the schedulerx-plugin-kubernetes
dependency to enable the scheduling of Kubernetes jobs. Sample code:
Replace schedulerx2.version with the latest version of the SchedulerX agent. For more information, see Release notes.
<dependency>
<groupId>com.aliyun.schedulerx</groupId>
<artifactId>schedulerx2-spring-boot-starter</artifactId>
<version>${schedulerx2.version}</version>
</dependency>
<dependency>
<groupId>com.aliyun.schedulerx</groupId>
<artifactId>schedulerx2-plugin-kubernetes</artifactId>
<version>${schedulerx2-plugin-kubernetes.version}</version>
</dependency>
Create a Kubernetes job
Before you can run one of the following scripts, create a Kubernetes job in the Create task panel of the Task Management page. For more information, see Job management.
Run a Shell script
If you want to run a Shell script in pod mode, perform the following operations to create a Kubernetes job: In the Create task panel of the Task Management page, select K8s from the Task type drop-down list, select Shell-Script from the resource drop-down list, and then use the busybox default image or specify a custom image.
After you create the job, click Run once. You can view that the pod named schedulerx-shell-{JobId} is started in the Kubernetes cluster.
On the Task Management page in the SchedulerX console, you can view the historical execution records of the job and the operational logs of the pod.
Run a Python script
If you want to run a Python script in pod mode, perform the following operations to create a Kubernetes job: In the Create task panel of the Task Management page, select K8s from the Task type drop-down list, select Python-Script from the resource drop-down list, and then use the python default image or specify a custom image.
After you create the job, click Run once. You can view that the pod named schedulerx-python-{JobId} is started in the Kubernetes cluster.
On the Task Management page in the SchedulerX console, you can view the historical execution records of the job and the operational logs of the pod.
Run a PHP script
If you want to run a PHP script in pod mode, perform the following operations to create a Kubernetes job: In the Create task panel of the Task Management page, select K8s from the Task type drop-down list, select Php-Script from the resource drop-down list, and then use the php:7.4-cli
default image or specify a custom image.
After you create the job, click Run once. You can view that the pod named schedulerx-php-{JobId} is started in the Kubernetes cluster.
On the Task Management page in the SchedulerX console, you can view the historical execution records of the job and the operational logs of the pod.
Run a Node.js script
If you want to run a Node.js script in pod mode, perform the following operations to create a Kubernetes job: In the Create task panel of the Task Management page, select K8s from the Task type drop-down list, select Nodejs-Script from the resource drop-down list, and then use the node:16
default image or specify a custom image.
After you create the job, click Run once. You can view that the pod named schedulerx-node-{JobId} is started in the Kubernetes cluster.
On the Task Management page in the SchedulerX console, you can view the historical execution records of the job and the operational logs of the pod.
Job-YAML
You can run a native Kubernetes job by using SchedulerX. To create a Kubernetes job, perform the following operations: In the Create task panel of the Task Management page, select K8s from the Task type drop-down list and select Job-YAML from the resource drop-down list.
After you create the job, click Run once. You can view that the job and the pod are started in the Kubernetes cluster.
On the Task Management page in the SchedulerX console, you can view the historical execution records of the job and the operational logs of the pod.
CronJob-YAML
You can run a native Kubernetes cron job by using SchedulerX. To create a Kubernetes job, perform the following operations: In the Create task panel of the Task Management page, select K8s from the Task type drop-down list and select CronJob-YAML from the resource drop-down list.
After you create the job, click Run once. In the Operation column for the job, choose more > Historical records. Then, you can view that the pod is started in the Task instance records panel.
On the Task Management page of the SchedulerX console, you can also query the pod execution history and view the operational logs of the pod.
Pod-YAML
You can run a native Kubernetes pod by using SchedulerX. To create a Kubernetes job, perform the following operations: In the Create task panel of the Task Management page, select K8s from the Task type drop-down list and select Pod-YAML from the resource drop-down list.
After you create the job, click Run once. You can view that the pod is started in the Kubernetes cluster.
On the Task Management page in the SchedulerX console, you can view the historical execution records of the job and the operational logs of the pod.
When you use SchedulerX to run Kubernetes pods, take note of the following items:
We recommend that you do not use a pod that has a long execution cycle, such as the pod of a web application. The execution of the pod never ends after the pod is started.
We recommend that you disable restart for a pod. Otherwise, the pod is repeatedly restarted.