MaxCompute:Write a Graph job

Prerequisites

• The MaxCompute client is installed, configured, and connected to the MaxCompute instance. For more information, see MaxCompute client (odpscmd).

• The IDEA 2024 development tool is prepared, and MaxCompute Studio is installed and configured (the version update time for MaxCompute Studio must be within 2024). For more information about how to install and configure MaxCompute Studio, see Install MaxCompute Studio and Configure MaxCompute Studio.

• Maven apache-maven-3.5.0 is configured.

• Java Development Kit (JDK) 1.8 or later is installed.

• A data file is prepared. The sssp.txt file is used in this example. The sssp.txt file contains the following data:

  1 2:2,3:1,4:4
  2 1:2,3:2,4:1
  3 1:1,2:2,5:1
  4 1:4,2:1,5:1
  5 3:1,4:1

Procedure

1. Run the MaxCompute client and execute the following statements to create an input table named sssp_in and an output table named sssp_out.

   CREATE TABLE sssp_in (v bigint, es string);
   CREATE TABLE sssp_out (vertex bigint, value bigint);

2. Run the Tunnel command to upload data in the sssp.txt file to the sssp_in table. Separate data in different columns with spaces.

   tunnel u -fd " " sssp.txt sssp_in;

   Note

   In this example, the sssp.txt file is stored in the bin directory of the MaxCompute client. Take note of the actual directory in which the sssp.txt file is stored.

3. Write SSSP code.

   1. Create a MaxCompute Java module named odps-graph-example-sssp in IntelliJ IDEA.

      Note

      Create a MaxCompute Java module. For more information, see Create a MaxCompute Java module.

   2. Create the BaseLoadingVertexResolver class and the SSSP class in odps-graph-example-sssp. For more information about how to create a class, see the sample code for the directed graph that is provided in SSSP. The following code is the configuration code in the pom.xml file.

      <dependency>
          <groupId>com.aliyun.odps</groupId>
          <artifactId>odps-sdk-core</artifactId>
          <version>0.48.0-public</version>
      </dependency>
      <dependency>
          <groupId>com.aliyun.odps</groupId>
          <artifactId>odps-sdk-graph</artifactId>
          <version>0.48.0-public</version>
      </dependency>
      
      <!-- used for local test -->
      <dependency>
          <groupId>com.aliyun.odps</groupId>
          <artifactId>odps-graph-local</artifactId>
          <version>0.48.0-public</version>
      </dependency>

   3. In IntelliJ IDEA, package the Java program into a JAR file by using MaxCompute Studio. For more information about how to package a Java program into a JAR file, see Package a Java program, upload the package, and create a MaxCompute UDF.

      

      Note

      In this example, the JAR package that is deployed in the MaxCompute project is named odps-graph-example-sssp.jar.

4. Run the following command on the MaxCompute client to use the SSSP algorithm:

   jar -libjars odps-graph-example-sssp.jar -classpath <LOCAL_JAR_PATH>/odps-graph-example-sssp.jar SSSP 1 sssp_in sssp_out;

   LOCAL_JAR_PATH: indicates the local path of the odps-graph-example-sssp.jar package.

   After you run the preceding command, run select * from sssp_out; to query the sssp_out table and verify the running result.

   vertex    value
   1        0
   2        2
   3        1
   4        3
   5        2

   • vertex: indicates the current vertex.

   • value: indicates the value of the SSSP from the current vertex to Source Vertex 1.

   Note

   If you want to use the Graph feature, you can directly submit a Graph job.