ApsaraDB for HBase:Import incremental data from Log Service

Note

Lindorm Tunnel Service (LTS) no longer supports incremental data import from Log Service from March 10, 2023. If you purchased LTS after March 10, 2023, this feature is no longer available. If you purchased LTS before March 10, 2023, you can still use this feature.

Prerequisites

• You have purchased LTS. You have configured the account name and password for the LTS console and have logged on to the LTS console.

• You have added an ApsaraDB for HBase data source in the LTS console.

• You have added a loghub (SLS) data source in the LTS console.

Supported editions

• Self-managed HBase V1.x and V2.x on which Kerberos is disabled.

• Alibaba Cloud E-MapReduce (EMR) HBase.

• ApsaraDB for HBase Standard Edition and ApsaraDB for HBase Performance-enhanced Edition that run in cluster mode. ApsaraDB for HBase in single-node mode is not supported.

Create a task

1. In the left-side navigation pane of the LTS console, select Data Import > SLS incremental Import.

2. Click create new job. From the Source Cluster drop-down list, select SLS. From the Target Cluster(HBase, Phoenix) drop-down list, select HBase. Configure the table mapping field.

3. Click create. After the job is created, you can view the details about the job on the SLS incremental Import page.

Table mapping parameter description

{
  "reader": {
    "columns": [
      "__client_ip__",
      "C_Source",
      "id",
      "name"
    ],
    "consumerSize": 2, // Specifies the number of consumers that subscribe to the loghub (SLS) data. Default value: 1.
    "logstore": "bds-test"
  },
  "writer": {
    "columns": [
      {
        "name": "f:col1",
        "value": "{{ concat('xx', name) }}" // Specifies the supported expression.
      },
      {
        "name": "f:col2",
        "value": "__client_ip__" // Specifies the mapped column name.
      },
      {
        "name": "f:*" // Specifies that columns that are not matched follow the default matching settings.
      }
    ],
    "rowkey": {
      "value": "{{ concat('idg',id) }}"
    },
    "tableName": "default:sls"
  }
}

• You can use the following simple expression:

{
  "name": "cf1:hhh",
  "value": "{{ concat(title, id) }}"
}