MaxCompute:Code-embedded UDFs

Code-embedded UDFs of MaxCompute resolve the following issues in code implementation and maintenance:

• Complicated code implementation: After you create UDFs and develop code, you must compile the code in Java and create resources and functions.

• Inconvenient code maintenance: You cannot directly view the implementation logic of the UDFs that are referenced in SQL scripts or obtain the source code of JAR packages.

• Poor code readability: To implement Java library functions by using user-defined types (UDTs), you need to convert Java code into expressions in long code lines. In addition, you may fail to write some Java code as expressions. Example:

  Foo f = new Foo();
  f.execute();
  f.getResult();

You can use only Janino-compiler to compile embedded Java code. The syntax of the embedded Java code must be a subset of the standard JDK syntax. Embedded Java code has the following limits:

• Lambda expressions are not supported.

• You cannot specify multiple types of exceptions in a single catch block. For example, catch(Exception1 | Exception2 e) is not allowed.

• It cannot automatically infer generic arguments. For example, Map map = new HashMap<>(); is not supported.

• Expressions for type argument inference are ignored. You must use cast expressions to specify the argument type, such as (String) myMap.get(key).

• Assertions are forcefully enabled, even if the -ea option of the Java virtual machine (JVM) is used.

• Code that is programmed in versions later than Java 8 is not supported.

The following code provides an example. You need to submit the SQL statements in script mode. For more information, see SQL in script mode.

SELECT 
  s, 
  com.mypackage.Foo.extractNumber(s) 
FROM VALUES ('abc123def'),('apple') AS t(s);

#CODE ('lang'='JAVA')
package com.mypackage;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
public class Foo {
  final static Pattern compile = Pattern.compile(".*?([0-9]+).*");
  public static String extractNumber(String input) {
    final Matcher m = compile.matcher(input);
    if (m.find()) {
      return m.group(1);
    }
    return null;
  }
}
#END CODE;

The following code provides an example. You need to submit the SQL statements in script mode. For more information, see SQL in script mode.

CREATE TEMPORARY FUNCTION foo AS 'com.mypackage.Reverse' USING
#CODE ('lang'='JAVA')
package com.mypackage;
import com.aliyun.odps.udf.UDF;
public class Reverse extends UDF {
  public String evaluate(String input) {
    if (input == null) return null;
    StringBuilder ret = new StringBuilder();
    for (int i = input.toCharArray().length - 1; i >= 0; i--) {
      ret.append(input.toCharArray()[i]);
    }
    return ret.toString();
  }
}
#END CODE;

SELECT foo('abdc');

The following code provides an example. You need to submit the SQL statements in script mode. For more information, see SQL in script mode.

CREATE TEMPORARY FUNCTION foo AS 'com.mypackage.Reverse' USING 
#CODE ('lang'='JAVA', 'filename'='embedded.jar')
package com.mypackage;

import com.aliyun.odps.udf.UDTF;
import com.aliyun.odps.udf.UDFException;
import com.aliyun.odps.udf.annotation.Resolve;

@Resolve({"string->string,string"})
public class Reverse extends UDTF {
  @Override
  public void process(Object[] objects) throws UDFException {
    String str = (String) objects[0];
    String[] split = str.split(",");
    forward(split[0], split[1]);
  }
}

#END CODE;

SELECT foo('ab,dc') AS (a,b);

The following code provides an example. You need to submit the SQL statements in script mode. For more information, see SQL in script mode.

CREATE TEMPORARY FUNCTION foo AS 'embedded.UDFTest' USING
#CODE ('lang'='PYTHON', 'filename'='embedded')
from odps.udf import annotate
@annotate("bigint->bigint")
class UDFTest(object):
  def evaluate(self, a):
    return a * a
#END CODE;

SELECT foo(4);