This topic describes how to create a job in Java and submit the job to Lindorm Distributed Processing System (LDPS) for execution.

Prerequisites

  • A Lindorm instance is created and LindormTable is activated for the instance. For more information, see Create an instance.
  • LDPS is activated for the Lindorm instance. For more information, see Activate LDPS and modify the configurations.
  • A Java integrated development environment (IDE) is installed. The version of the Java Development Kit (JDK) is 1.8 or later.

Step 1: Add dependencies

LDPS depends on Spark 3.2.1 to execute Java jobs. You must add the corresponding dependency and set the scope field to provided. The following example shows how to add the dependency on Spark 3.2.1:
<!-- Example -->
<dependency>
  <groupId>org.apache.spark</groupId>
  <artifactId>spark-core_2.12</artifactId>
  <version>3.2.1</version>
  <scope>provided</scope>
</dependency>

Step 2: Configure permissions

To use Spark SQL to access LindormTable, you must configure a username and a password. The following code provides an example on how to configure the username and password. For more information about how to access LindormTable, see Access data in LindormTable.
SparkConf conf = new SparkConf();
conf.set("spark.sql.catalog.lindorm_table.username", "root");
conf.set("spark.sql.catalog.lindorm_table.password", "root");
Parameter Default value Description
spark.sql.catalog.lindorm_table.username root The username that is used to connect to LindormTable.
spark.sql.catalog.lindorm_table.password root The password that is used to connect to LindormTable.

Step 3: Configure parameters

For more information about how to configure parameters for LDPS, see Parameter description.

Step 4: Create a job based on sample code

The development of Java jobs is compatible with Spark 3.2.1. You can click sample Spark job to download the sample code.

Step 5: Submit the Java job

LDPS allows you to submit and manage jobs by using the following two methods: