SDKs for data collection
Overview of SDK features
SDKs for data collection provide the class that is used to collect data and push documents.
You can use the SDKs to manage data collection and document pushes for OpenSearch applications. You can push one or more documents at a time.
Release notes of the SDK for Java
Demo of the SDK for Python for pushing behavioral data
Demo of the SDK for Go for pushing behavioral data
Demo of the SDK for C# for pushing behavioral data
Class name: DataCollectionClient
Namespace: OpenSearch\Client
Method description
A constructor.
Method definition
void OpenSearch\Client\DataCollectionClient::__construct(\OpenSearch\Client\OpenSearchClient $openSearchClient)Parameter description
Parameter | Type | Description |
$openSearchClient | OpenSearch\Client\OpenSearchClient | The basic class that is used to calculate signatures, interact with the server, and return the results. |
add
Method description
Adds a document.
After you call this method to add a document, the document is added to the SDK client buffer but is not committed to the server. The document is committed to the server only when you call the commit() method. You can call the add() method multiple times to add multiple documents, and then call the commit() method to commit the documents to the server at a time.
Method definition
\OpenSearch\Generated\Common\OpenSearchResult OpenSearch\Client\DataCollectionClient::add(array $fields)Parameter description
Parameter | Type | Description |
$fields | ARRAY | All fields in a document of behavioral data, user data, or goods data. Example: array("user_id" => "1021468", "bhv_type" => "click"); |
commit
Method description
Commits documents in the SDK client buffer to the server.
Before the documents in the buffer are committed to the server, the documents are cleared from the buffer. Therefore, if the server returns an error and you want to commit the documents again, you must regenerate the documents before you commit them. This prevents the loss of data.
Method definition
\OpenSearch\Generated\Common\OpenSearchResult OpenSearch\Client\DataCollectionClient::commit(string $searchAppName,string $dataCollectionName,string $dataCollectionType)Parameter description
Parameter | Type | Description |
$searchAppName | STRING | The name of the associated OpenSearch application. |
$dataCollectionName | STRING | The name of data collection, which is returned by the OpenSearch console when the feature of collecting behavioral data is enabled. |
$dataCollectionType | STRING | Data collection type: BEHAVIOR |
push
Method description
Pushes multiple documents at a time.
This operation synchronously sends documents to the server.
Method definition
\OpenSearch\Generated\Common\OpenSearchResult OpenSearch\Client\DataCollectionClient::push(string $docJson,string $searchAppName,string $dataCollectionName,string $dataCollectionType)Parameter description
Parameter | Type | Description |
$docJson | STRING | The list of documents in the JSON format. |
$searchAppName | STRING | The name of the associated OpenSearch application. |
$dataCollectionName | STRING | The name of data collection, which is returned by the OpenSearch console when the feature of collecting behavioral data is enabled. |
$dataCollectionType | STRING | Data collection type: BEHAVIOR |
Demo for using the push() method of the SDK for PHP to collect data
<?php
require_once("Config.inc.php");
use OpenSearch\Client\DataCollectionClient;
use OpenSearch\Generated\DataCollection\Command;
$searchAppName = "opensearch_app_name";
$dataCollectionName = "opened_data_collection_name";
$dataCollectionType = "BEHAVIOR";
$docs = json_encode(array(
[
"cmd" => Command::$__names[Command::ADD],
"fields" => [
// The unique user ID.
"user_id" => "1120021255",
// The numeric ID used to distinguish between different search services. A numeric ID corresponds to an OpenSearch application.
"biz_id" => 1365378,
// The value of this field is the value of the request_id parameter that OpenSearch returns in the search results. Pass in the value of the request_id parameter as it is.
"rn" => "156516585419723283227314",
// If the document is searched and collected from OpenSearch, set this field to Alibaba.
"trace_id" => "Alibaba",
// The value of this field is the value of the ops_request_misc parameter that OpenSearch returns in the search results. Pass in the value of the ops_request_misc parameter as it is.
"trace_info" => "%7B%22request%5Fid%22%3A%22156516585419723283227314%22%2C%22scm%22%3A%2220140713.120006678..%22%7D",
// The value of this field is the primary key value of the primary table in the OpenSearch application.
"item_id" => "2223",
// The type of the item is good or commodity.
"item_type" => "goods",
// The behavioral data that is collected on clicks.
"bhv_type" => "click",
// The time at which the behavior occurs. The value is a UNIX timestamp that is accurate to the second.
"bhv_time" => "1566475047"
]
]
));
// Create the DataCollectionClient object and use the OpenSearchClient object as the construction parameter.
$dataCollectionClient = new DataCollectionClient($client);
$ret = $dataCollectionClient->push($docs, $searchAppName, $dataCollectionName, $dataCollectionType);
print_r(json_decode($ret->result, true));Demo for using the commit() method of the SDK for PHP to collect data
<?php
require_once("Config.inc.php");
use OpenSearch\Client\DataCollectionClient;
use OpenSearch\Generated\DataCollection\Command;
$searchAppName = "opensearch_app_name";
$dataCollectionName = "opened_data_collection_name";
$dataCollectionType = "BEHAVIOR";
// Create the DataCollectionClient object and use the OpenSearchClient object as the construction parameter.
$dataCollectionClient = new DataCollectionClient($client);
// Add a document.
// The document is added to the SDK client buffer but is not committed to the server. The document is committed to the server only when you call the commit() method.
// You can call the add() method multiple times to add multiple documents, and then call the commit() method to commit the documents to the server at a time.
$dataCollectionClient->add([
// The unique user ID.
"user_id" => "1120021255",
// The numeric ID used to distinguish between different search services. A numeric ID corresponds to an OpenSearch application.
"biz_id" => 1365378,
// The value of this field is the value of the request_id parameter that OpenSearch returns in the search results. Pass in the value of the request_id parameter as it is.
"rn" => "156516585419723283227314",
// If the document is searched and collected from OpenSearch, set this field to Alibaba.
"trace_id" => "Alibaba",
// The value of this field is the value of the ops_request_misc parameter that OpenSearch returns in the search results. Pass in the value of the ops_request_misc parameter as it is.
"trace_info" => "%7B%22request%5Fid%22%3A%22156516585419723283227314%22%2C%22scm%22%3A%2220140713.120006678..%22%7D",
// The value of this field is the primary key value of the primary table in the OpenSearch application.
"item_id" => "2223",
// The type of the item is good or commodity.
"item_type" => "goods",
// The behavioral data that is collected on clicks.
"bhv_type" => "click",
// The time at which the behavior occurs. The value is a UNIX timestamp that is accurate to the second.
"bhv_time" => "1566475047"
]);
$ret = $dataCollectionClient->commit($searchAppName, $dataCollectionName, $dataCollectionType);
print_r(json_decode($ret->result, true));Demo for using the push() method of the SDK for Java to collect data
package com.aliyun.opensearch.demo;
import com.aliyun.opensearch.DataCollectionClient;
import com.aliyun.opensearch.OpenSearchClient;
import com.aliyun.opensearch.sdk.generated.OpenSearch;
import com.aliyun.opensearch.sdk.generated.commons.OpenSearchResult;
public class PushDataCollectionDoc {
private static String accesskey = "your ak";
private static String secret = "your secret";
private static String host = "your host";
private static String searchAppName = "opensearch_app_name";
private static String dataCollectionName = "opened_data_collection_name";
private static String dataCollectionType = "BEHAVIOR";
public static void main(String[] args) {
// Create an OpenSearch object.
OpenSearch opensearch = new OpenSearch(accesskey, secret, host);
// Use the OpenSearch object as a parameter to create an OpenSearchClient object.
OpenSearchClient client = new OpenSearchClient(opensearch);
// Create the DataCollectionClient object and use the OpenSearchClient object as the construction parameter.
DataCollectionClient dataCollectionClient = new DataCollectionClient(client);
// Push documents.
String docJson = "[{\"cmd\":\"ADD\",\"fields\":{\"user_id\":\"1120021255\","+
"\"biz_id\":1365378,\"rn\":\"156516585419723283227314\","+
"\"trace_id\":\"Alibaba\","+
"\"trace_info\":\"%7B%22request%5Fid%22%3A%22156516585419723283227314%22%2C%22scm%22%3A%2220140713.120006678..%22%7D\","+
"\"item_id\":\"id\",\"item_type\":\"goods\","+
"\"bhv_type\":\"click\",\"bhv_time\":\"1566475047\"}}]";
try {
OpenSearchResult openSearchResult = dataCollectionClient.push(docJson,
searchAppName, dataCollectionName,
dataCollectionType);
System.out.println(openSearchResult);
} catch (Exception e) {
e.printStackTrace();
assertTrue(false);
return;
}
}
}Demo for using the commit() method of the SDK for Java to collect data
package com.aliyun.opensearch.demo;
import com.aliyun.opensearch.DataCollectionClient;
import com.aliyun.opensearch.OpenSearchClient;
import com.aliyun.opensearch.sdk.generated.OpenSearch;
import com.aliyun.opensearch.sdk.generated.commons.OpenSearchResult;
import java.util.HashMap;
import java.util.Map;
public class PushDataCollectionDoc {
private static String accesskey = "your ak";
private static String secret = "your secret";
private static String host = "your host";
private static String searchAppName = "opensearch_app_name";
private static String dataCollectionName = "opened_data_collection_name";
private static String dataCollectionType = "BEHAVIOR";
public static void main(String[] args) {
// Create an OpenSearch object.
OpenSearch opensearch = new OpenSearch(accesskey, secret, host);
// Use the OpenSearch object as a parameter to create an OpenSearchClient object.
OpenSearchClient client = new OpenSearchClient(opensearch);
// Create the DataCollectionClient object and use the OpenSearchClient object as the construction parameter.
DataCollectionClient dataCollectionClient = new DataCollectionClient(client);
Map<String, Object> fields = new HashMap<String, Object>();
// The unique user ID.
fields.put("user_id", "1120021255");
// The numeric ID used to distinguish between different search services. A numeric ID corresponds to an OpenSearch application.
fields.put("biz_id", 1365378);
// The value of this field is the value of the request_id parameter that OpenSearch returns in the search results. Pass in the value of the request_id parameter as it is.
fields.put("rn", "1564455556323223680397827");
// If the document is searched and collected from OpenSearch, set this field to Alibaba.
fields.put("trace_id", "Alibaba");
// The value of this field is the value of the ops_request_misc parameter that OpenSearch returns in the search results. Pass in the value of the ops_request_misc parameter as it is.
fields.put("trace_info", "%7B%22request%5Fid%22%3A%22156516585419723283227314%22%2C%22scm%22%3A%2220140713.120006678..%22%7D");
// The value of this field is the primary key value of the primary table in the OpenSearch application.
fields.put("item_id", "2223");
// The type of the item is good or commodity.
fields.put("item_type", "goods");
// The behavioral data that is collected on clicks.
fields.put("bhv_type", "click");
// The time at which the behavior occurs. The value is a UNIX timestamp that is accurate to the second.
fields.put("bhv_time", "1566475047");
// Add a document.
// The document is added to the SDK client buffer but is not committed to the server. The document is committed to the server only when you call the commit() method.
// You can call the add() method multiple times to add multiple documents, and then call the commit() method to commit the documents to the server at a time.
dataCollectionClient.add(fields);
try {
OpenSearchResult openSearchResult = dataCollectionClient.commit(searchAppName, dataCollectionName, dataCollectionType);
System.out.println(openSearchResult);
} catch (Exception e) {
e.printStackTrace();
return;
}
}
}