MaxCompute:Develop a UDF in Java

You can write UDF code in Java. The code must contain the following information:

• Java package: optional.

  You can package Java classes that are defined into a JAR file for future use.

• Base UDF class: required.

  The required UDF class is com.aliyun.odps.udf.UDF. If you want to use other UDF classes or complex data types, follow the instructions provided in Overview to add the required classes. For example, the UDF class that corresponds to the STRUCT data type is com.aliyun.odps.data.Struct.

• @Resolve annotation: optional.

  The annotation is in the @Resolve(<signature>) format. The signature parameter is used to define the data types of the input parameters and return value of a UDF. If you want to use data of the STRUCT data type in a UDF, you cannot use the reflection feature for the com.aliyun.odps.data.Struct class to obtain the names and types of fields. In this case, you must add the @Resolve annotation to the com.aliyun.odps.data.Struct class. This annotation affects only the overloading of the UDF whose input parameters or return value contain the com.aliyun.odps.data.Struct class. Example: @Resolve("struct<a:string>,string->string"). For more information about how to use complex data types in Java UDFs, see Use complex data types in Java UDFs.

• Custom Java class: required.

  A custom Java class is the organizational unit of UDF code. This class defines the variables and methods that are used to meet your business requirements.

• evaluate method: required.

  The evaluate method is a non-static public method and is contained in a custom Java class. The data types of the input parameters and return value of the evaluate method are used as the function signature of a UDF in SQL statements. The function signature defines the data types of the input parameters and return value of the UDF.

  You can implement multiple evaluate methods in a UDF. When you call a UDF, MaxCompute matches an evaluate method based on the data types of the input parameters in the UDF.

  When you write a Java UDF, you can use Java data types or Java writable data types. For more information about the mappings between the data types supported in MaxCompute projects, Java data types, and Java writable data types, see Appendix: Data types.

• UDF initialization or termination code: optional. You can use the void setup(ExecutionContext ctx) method to initialize a UDF and use the void close() method to terminate a UDF. The void setup(ExecutionContext ctx) method is called before the evaluate method. The void setup(ExecutionContext ctx) method is called only once and is used to initialize the resources that are required for data computing or initialize the members of a class. The void close() method is called after the evaluate method. The void close() method is used to clean up data, such as closing files.

Sample code:

• Use Java data types

  // Package the defined Java classes into a file named org.alidata.odps.udf.examples. 
  package org.alidata.odps.udf.examples;  
  // Inherit the UDF class. 
  import com.aliyun.odps.udf.UDF;         
  // The custom Java class. 
  public final class Lower extends UDF { 
  // The evaluate method. String indicates the data types of the input parameters and return indicates the return value. 
      public String evaluate(String s) { 
          if (s == null) { 
          return null; 
      } 
          return s.toLowerCase(); 
    } 
  }

• Use Java writable data types

  // Package the defined Java classes into a file named com.aliyun.odps.udf.example. 
  package com.aliyun.odps.udf.example;
  // Add the class that corresponds to a Java writable data type. 
  import com.aliyun.odps.io.Text;
  // Inherit the UDF class. 
  import com.aliyun.odps.udf.UDF;
  // The custom Java class. 
  public class MyConcat extends UDF {
    private Text ret = new Text();
  // The evaluate method. Text indicates the data types of the input parameters and return indicates the return value. 
    public Text evaluate(Text a, Text b) {
        if (a == null || b == null) {
        return null;
      }
        ret.clear();
        ret.append(a.getBytes(), 0, a.getLength());
        ret.append(b.getBytes(), 0, b.getLength());
        return ret;
    }
  }

Data type mappings

The following table describes the mappings between the data types supported in MaxCompute projects, Java data types, and Java writable data types. You must write Java UDFs based on the mappings to ensure the consistency of the data types. The following table describes the data type mappings.

Hive UDFs

If your MaxCompute project uses the MaxCompute V2.0 data type edition and supports Hive UDFs, you can directly use Hive UDFs whose Hive version is compatible with MaxCompute.

The Hive version that is compatible with MaxCompute is 2.1.0, which corresponds to Hadoop 2.7.2. If a UDF is developed on another Hive or Hadoop version, you must use Hive 2.1.0 or Hadoop 2.7.2 to recompile the JAR file of the UDF.

For more information about how to use Hive UDFs in MaxCompute, see Write a Hive UDF in Java.