Tunnel Service is a centralized service that uses the Table Store API to allow you to consume full and incremental data. Tunnel Service provides tunnels that are used to export and consume data in the full, incremental, and differential modes. After you create a tunnel, you can use it to consume historical and incremental data that is exported from a specified table.

Create a tunnel

Create a tunnel for a data table.

  • Command syntax
    create_tunnel -n name

    The following table describes the parameters that you can configure to create a tunnel for a data table.

    Parameter Required Example Description
    -t, --table No mytable The name of the data table.
    -n, --name Yes t1 The name of the tunnel.
    -m, --mode No stream_data_only The type of the tunnel. Default value: stream_data_only. Valid values:
    • base_data_only: full tunnel. Only full data is consumed and processed.
    • stream_data_only: incremental tunnel. Only incremental data is consumed and processed.
    • base_and_stream: differential tunnel. After full data is consumed and processed, incremental data is consumed and processed.
  • Examples

    The following sample code shows how to create a tunnel named t1 for a data table: 

    create_tunnel -n t1
    The following result is returned:
    New tunnel created, its id is '9933470d-8a5e-4972-a5b0-b7ae6f836460'.

Query information about a tunnel

Query tunnel information and channel information about a tunnel.

  • Command syntax
    describe_tunnel -n name

    The following table describes the parameters that you can configure to query information about a tunnel.

    Parameter Required Example Description
    -t, --table No mytable The name of the data table.
    -n, --name Yes t1 The name of the tunnel.
    -o, --output No D:\\otstest\\mytunnel.txt The local file in which you want to store the returned result.
  • Examples

    The following sample code shows how to query information about the tunnel named t1: 

    describe_tunnel -n t1
    The following result is returned:
    Tunnel Info:
+--------------------------------------+------------+------------+---------------+---------+
| TunnelId                             | TunnelName | TunnelType | Stage         | Expired |
+--------------------------------------+------------+------------+---------------+---------+
| 9933470d-8a5e-4972-a5b0-b7ae6f836460 | t1         | Stream     | ProcessStream | false   |
+--------------------------------------+------------+------------+---------------+---------+

Channel Info:
+-------------------------------------------------------+-------------+---------------+----------+-------------------------------+
| ChannelId                                             | ChannelType | ChannelStatus | ClientId | ChannelRPO                    |
+-------------------------------------------------------+-------------+---------------+----------+-------------------------------+
| cfd2c05b-54b6-48ec-aa6f-feb427f0ca57_1635771329155688 | Stream      | OPEN          |          | 1970-01-01 08:00:00 +0800 CST |
+-------------------------------------------------------+-------------+---------------+----------+-------------------------------+

Simulate the data consumption of a tunnel

After you create a tunnel, you can simulate the data consumption of the tunnel to preview the data format in the tunnel.

  • Command syntax
    consume_tunnel -n name -m mock_consume

    The following table describes the parameters that you can configure to simulate the data consumption of a tunnel.

    Parameter Required Example Description
    -c, --channel No cfd2c05b-54b6-48ec-aa6f-feb427f0ca57_1635771329155688 The ID of the channel. If you do not configure this parameter, the data in all channels of the tunnel is consumed.
    -t, --table No mytable The name of the data table.
    -n, --name Yes t1 The name of the tunnel.
    -m, --mode Yes mock_consume The consumption mode. Default value: shadow_copy. Valid values:
    • shadow_copy: copies the consumption traffic of an online tunnel.
    • mock_consume: simulates data consumption but does not update checkpoint information.
    • real_consume: consumes data and updates checkpoint information. We recommend that you do not use this mode.
  • Examples

    The following sample code shows how to simulate the data consumption of the tunnel named t1: 

    consume_tunnel -n t1 -m mock_consume
    After you run the command to simulate data consumption, data consumption records are displayed when data is written to the data table. The following sample code shows an example of the returned result:
    Starting consume tunnel 't1' of table 'mytable', it may take a few seconds to start, please wait...
null

Delete a tunnel

You can delete a tunnel that you no longer need.

  • Command syntax
    drop_tunnel -n name -y

    The following table describes the parameters that you can configure to delete a tunnel.

    Parameter Required Example Description
    -t, --table No mytable The name of the data table.
    -n, --name Yes t1 The name of the tunnel.
    -y, --yes Yes N/A Specifies that confirmation information appears. This configuration item must be included in the command.
  • Example

    The following sample code shows how to delete the tunnel named t1: 

    drop_tunnel -n t1 -y