Export full data to MaxCompute - Lindorm - Alibaba Cloud Documentation Center

This topic describes how to export full data from HBase to MaxCompute.

Usage notes

This feature is no longer available for LTS instances that are purchased after June 16, 2023. If your LTS instance is purchased before June 16, 2023, you can still use this feature.

Prerequisites

Lindorm Tunnel Service (LTS) is purchased. The username and the password for logging on to the LTS web UI are configured, and you are logged on to the LTS web UI. For more information, see Activate LTS and log on to the LTS web UI.
An HBase data source is added.
A MaxCompute data source is added.

Features

Export full data from HBase to MaxCompute.
Specify the rowkey, the time range, and the columns.

Supported versions

Self-managed HBase V1.x and V2.x
E-MapReduce HBase
ApsaraDB for HBase Standard Edition, ApsaraDB for HBase Performance-enhanced Edition that runs in cluster mode, and ApsaraDB for Lindorm (Lindorm)

Create a task

Log on to the LTS web UI. In the left-side navigation pane, choose Data Export > Archive Full Data to MaxCompute.
On the Lindorm/HBase archive to MaxCompute page, click create new job.
Select the source cluster and the destination cluster, enter the table to be migrated, and then click Create.
Click the task name to view the migration progress.
After the migration is complete, log on to the MaxCompute console to query the data.

Parameters

hbaseTable/odpsTable {"tableMode": "wideTable", "cols":["f1:a", "f1:b", "f1:c"]}
hbaseTable/odpsTable {"tableMode": "wideTable", "cols":["f1:a|string", "f1:b|boolean"]}
hbaseTable/odpsTable {"tableMode": "wideTable", "cols":["f1:a|string", "f1:b|boolean"], "startTs": "20190101000000", "endTs": "20200101000000", "startKey": "a", "endKey": "z"}
hbaseTable/odpsTable {"tableMode": "KVTable"}

You can use the key-value (KV) or wide table mode to export HBase data to MaxCompute.

KV mode

Input parameters: tablexxx {"tableMode": "KVTable"}

Response: MaxCompute

Note

Each row indicates a KV pair in HBase.
rowkey: the rowkey of the KV pair in the HBase table.
version: the version of the KV pair.
op: the type of the KV pair. Valid values: Put, DeleteFamily, and DeleteColumn.
family: the column family.
qualifier: the name of the column.
value: the value of the KV pair. The value is converted to the HexString format.

Wide table mode

Input parameters: hbaseTable/odpsTable {"tableMode": "wideTable", "cols":["cf1:string|string", "cf1:int|int", "cf1:double|double"]}

Note

In wide table mode, you must specify the columns to be exported. For example, you can use "cols":["f1:a", "f1:b", "f1:c"]. By default, the column value is converted to the HexString format. You can specify the data type of the column. For example, you can use "cols":["f1:a|string", "f1:b|boolean", "f1:c|int"]. The columns support the following data types: string, int, long, short, decimal, double, float, and boolean. The default data type is hex. This indicates that byte arrays are converted to the HexString format.

Response:

Specify the range of the data to be exported

Input parameters: table1 {"startKey": "xxx", "endKey": "zzz", "startTs": "20191001153000", "endTs": "20191001233000"}

When you export data from HBase to MaxCompute in KV mode or wide table mode, you can use the startKey and endKey parameters to specify the rowkey range. You can also use the startTs and endTs parameters to specify the time range.

Specify MaxCompute tables for exported data

You can export data to a MaxCompute table whose name is different from the name of the source HBase table. By default, then name of the MaxCompute table is the same as the name of the HBase table. The hyphens (-) and periods (.) in HBase table names are replaced with underscores (_). For example, you can use hbaseTable/odpsTable {"tableMode": "KVTable"} to export data.