Export full data to MaxCompute - ApsaraDB for HBase - Alibaba Cloud Documentation Center

This topic describes how to export full data from HBase to MaxCompute.

Usage notes

This feature is no longer available for Lindorm Tunnel Service (LTS) instances that are purchased after June 16, 2023. If your LTS instance is purchased before June 16, 2023, you can still use this feature.

Prerequisites

LTS is activated. The username and the password for logging on to the LTS web UI are configured. You are logged on to the LTS web UI.
An HBase data source is added.
A MaxCompute data source is added.

Features

Export full data from HBase to MaxCompute.
Specify the rowkey, the time range, and the columns.

Supported versions

Self-managed HBase V1.x and HBase V2.x
Elastic MapReduce (EMR) HBase
ApsaraDB for HBase Standard Edition, ApsaraDB for HBase Performance-enhanced Edition that runs in cluster mode, and Lindorm

Create a task

Log on to the LTS web UI. In the left-side navigation pane, choose Data Export > Archive Full Data to MaxCompute.
On the Lindorm/HBase archive to MaxCompute page, click create new job.
Select the source cluster and the destination cluster, specify the table that you want to migrate, and then click create.
Click the task name to view the migration progress.
After the migration is complete, log on to the MaxCompute console to query the data.

Parameters

hbaseTable/odpsTable {"tableMode": "wideTable", "cols":["f1:a", "f1:b", "f1:c"]}
hbaseTable/odpsTable {"tableMode": "wideTable", "cols":["f1:a|string", "f1:b|boolean"]}
hbaseTable/odpsTable {"tableMode": "wideTable", "cols":["f1:a|string", "f1:b|boolean"], "startTs": "20190101000000", "endTs": "20200101000000", "startKey": "a", "endKey": "z"}
hbaseTable/odpsTable {"tableMode": "KVTable"}

You can use the key-value (KV) or wide table mode to export HBase data to MaxCompute.

KV mode

Input parameters: tablexxx {"tableMode": "KVTable"}

Response MaxCompute

Note

Each row indicates a KV pair in HBase.
rowkey: the rowkey of the KV pair in the HBase table.
version: the version of the KV pair.
op: the type of the KV pair. Valid values: Put, DeleteFamily, and DeleteColumn.
family: the column family.
qualifier: the name of the column.
value: the value of the KV pair. The value is converted into the HexString format.

Wide table mode

Input parameters: hbaseTable/odpsTable {"tableMode": "wideTable", "cols":["cf1:string|string", "cf1:int|int", "cf1:double|double"]}

Note

In wide table mode, you must specify the columns that you want to export. For example, you can use "cols":["f1:a", "f1:b", "f1:c"]. By default, the column values are converted into the HexString format. You can specify the data types of the columns. For example, you can use "cols":["f1:a|string", "f1:b|boolean", "f1:c|int"]. The columns support the following data types: string, int, long, short, decimal, double, float, and boolean. The default data type is hex. This indicates that byte arrays are converted into the HexString format.

Response

Specify the range of the data to be exported

Input parameters: table1 {"startKey": "xxx", "endKey": "zzz", "startTs": "20191001153000", "endTs": "20191001233000"}

When you export data from HBase to MaxCompute in KV mode or wide table mode, you can use the startKey and endKey parameters to specify the rowkey range. You can also use the startTs and endTs parameters to specify the time range.

Specify a MaxCompute table for exported data

You can export data to a MaxCompute table whose name is different from the name of the source HBase table. By default, the name of the MaxCompute table is the same as the name of the HBase table. The hyphens (-) and periods (.) in HBase table names are replaced with underscores (_). For example, you can use hbaseTable/odpsTable {"tableMode": "KVTable"} to export data.