AnalyticDB for MySQL lets you use the REMOTE_CALL() function to invoke custom functions that you create in Function Compute. This enables you to use user-defined functions (UDFs) in AnalyticDB for MySQL.
Prerequisites
The kernel version of your cluster is 3.2.1.0 or later.
The cluster and the Function Compute service are in the same region. For more information, see Activate Function Compute.
A user-defined function is created.
NoteFor faster Function Compute startup times when the function is called, select Python, Go, or Node.js for the Runtime Environment when you create the user-defined function.
How it works
The remote UDF feature of AnalyticDB for MySQL uses Function Compute as a remote function server. You can use the REMOTE_CALL() function in AnalyticDB for MySQL to call a UDF that is predefined in Function Compute. The following flowchart illustrates the process.
A client submits an SQL statement to AnalyticDB for MySQL.
After the REMOTE_CALL() function is called, AnalyticDB for MySQL sends data to Function Compute in JSON format.
Function Compute processes the data using the user-defined function.
Function Compute returns the result to AnalyticDB for MySQL in JSON format.
AnalyticDB for MySQL then returns the final result to the client.
Usage notes
The remote UDF feature supports only scalar UDFs.
Syntax
remote_call('returnType', 'func_name', ['{external_config}'|NULL], X1, X2, ..., Xn)Parameters
Parameter | Description |
returnType | The data type of the return value. Only the BOOLEAN, DOUBLE, VARCHAR, INTEGER, TINYINT, BIGINT, TIME, DATE, TIMESTAMP, and DATETIME data types are supported. |
func_name | The name of the user-defined function that you create in Function Compute.
Note Log on to the Function Compute console to view the Function Compute version. |
external_config|null | The extended parameters for the function call. If no extended parameters are used, specify null. The extended parameters must be in JSON format. Separate multiple parameters with commas (,). You can set extended parameters in a query, or configure global parameters using the |
X1.......Xn | The input parameters. Only the BOOLEAN, DOUBLE, VARCHAR, INTEGER, TINYINT, BIGINT, TIME, DATE, TIMESTAMP, and DATETIME data types are supported. |
The following table describes the extended parameters supported by external_config.
Global parameter | Query-level parameter | Required | Description |
XIHE_REMOTE_CALL_SERVER_ENDPOINT | endpoint | Yes | The internal service endpoint of Function Compute. For more information, see Service endpoints. |
XIHE_REMOTE_CALL_SERVER_AK | - | Required under specific conditions | The AccessKey ID of the Alibaba Cloud account or a Resource Access Management (RAM) user that has service-managed permissions on Function Compute.
For more information about how to obtain an AccessKey ID, see Accounts and permissions. Important The XIHE_REMOTE_CALL_SERVER_AK parameter can be configured only as a global parameter. |
XIHE_REMOTE_CALL_SERVER_SK | - | Required under specific conditions | The AccessKey secret of the Alibaba Cloud account or a RAM user that has service-managed permissions on Function Compute.
For more information about how to obtain an AccessKey secret, see Accounts and permissions. Important The XIHE_REMOTE_CALL_SERVER_SK parameter can be configured only as a global parameter. |
XIHE_REMOTE_CALL_COMPRESS_ENABLED | compressed | No | Specifies whether to compress data into GZIP format before AnalyticDB for MySQL sends the data to Function Compute. Valid values:
|
XIHE_REMOTE_CALL_MAX_BATCH_SIZE | max_batch_size | No | The maximum number of data rows that can be sent to Function Compute in each batch after the REMOTE_CALL() function is called. By default, no limit is imposed. A smaller number of data rows reduces the CPU and memory pressure on Function Compute, but increases the query running time. |
Examples
Example of a user-defined function in Function Compute
When the REMOTE_CALL() function is called, AnalyticDB for MySQL sends data to Function Compute in JSON format. After the data is processed, Function Compute returns the result in JSON format. You must write the user-defined function in Function Compute to conform to the specified data sending and returning formats.
In this example, the user-defined function created in Function Compute is named ConcactNumberWithCompress. The following is the sample code:
public class App
implements StreamRequestHandler, FunctionInitializer
{
public static final Logger log = Logger.getLogger(App.class.getName());
public void initialize(Context context)
throws IOException
{
// TODO
}
@Override
public void handleRequest(
InputStream inputStream, OutputStream outputStream, Context context)
throws IOException
{
InputStream notCompressedInputStream1;
notCompressedInputStream1 = tryUnCompress(inputStream);
JSONObject response = new JSONObject();
try {
JSONObject requestJson = JSONObject.parseObject(IOUtils.toString(notCompressedInputStream1));
if (requestJson.containsKey("data")) {
JSONArray result = new JSONArray();
JSONArray data = requestJson.getJSONArray("data");
for (int i = 0; i < data.size(); i++) {
JSONArray row = data.getJSONArray(i);
// Implement the business logic of Function Compute here.
// Assume that the function has two input parameters, a and b. Each row of data sent from AnalyticDB for MySQL contains two numbers. The first number is a, and the second number is b.
if (row.size() == 2) {
result.add(testFunc(row.getInteger(0), row.getInteger(1)));
}
else {
throw new RuntimeException("row size is not 2");
}
// The business logic of Function Compute ends here.
}
response.put("result", result);
response.put("success", true);
response.put("message", "");
}
else {
response.put("success", false);
response.put("message", "no data inside");
}
}
catch (Exception e) {
log.info("error happened" + e.getMessage());
response.put("success", false);
response.put("message", e.getMessage());
}
outputStream.write(tryCompress(response.toJSONString().getBytes()));
}
private String testFunc(int a, int b)
{
// The test function concatenates the input parameters a and b and the number 1 using ampersands (&).
return String.valueOf(a) + '&' + b + '&' + 1;
}
public static byte[] tryCompress(byte[] bytes)
{
ByteArrayOutputStream byteOutputStream = new ByteArrayOutputStream();
try {
GZIPOutputStream gzipOutputStream = new GZIPOutputStream(byteOutputStream);
gzipOutputStream.write(bytes);
gzipOutputStream.close();
}
catch (IOException e) {
throw new RuntimeException(e);
}
return byteOutputStream.toByteArray();
}
public static InputStream tryUnCompress(InputStream inputStream)
throws IOException
{
GZIPInputStream gzipInputStream;
gzipInputStream = new GZIPInputStream(inputStream);
return gzipInputStream;
}
}Examples of calling the REMOTE_CALL function in AnalyticDB for MySQL
If no global parameters are configured, call the ConcactNumberWithCompress user-defined function to concatenate
1and2with an ampersand (&):SELECT remote_call('varchar','ConcactNumberWithCompress','{endpoint:"1234567890000****.cn-zhangjiakou-internal.fc.aliyuncs.com"}',1,2);The following result is returned:
1&2&1If no global parameters are configured, call the ConcactNumberWithCompress user-defined function to concatenate
3and4with an ampersand (&):SELECT remote_call('varchar','ConcactNumberWithCompress','{endpoint:"1234567890000****.cn-zhangjiakou-internal.fc.aliyuncs.com",compressed:false,max_batch_size:5000000}',3,4);The following result is returned:
3&4&1If global parameters are configured, call the ConcactNumberWithCompress user-defined function to concatenate
5and6with an ampersand (&):SELECT remote_call('varchar','ConcactNumberWithCompress',null,5,6);The following result is returned:
5&6&1
FAQ
Why does the java.util.zip.ZipException: Not in GZIP format error occur when I call the REMOTE_CALL function?
AnalyticDB for MySQL compresses data into the GZIP format and sends it to Function Compute. However, the user-defined function created in Function Compute does not contain decompression code. As a result, Function Compute cannot parse the data.
AnalyticDB for MySQL sends uncompressed data to Function Compute. After processing, Function Compute compresses the result using the GZIP format and returns it to AnalyticDB for MySQL. As a result, AnalyticDB for MySQL cannot parse the data.
Why does the parse remote_call config error error occur when I call the REMOTE_CALL function?
The syntax of the external_config parameter in the REMOTE_CALL function is incorrect. Check and correct the SQL statement, and then run it again.