You can use Tunnel Service in the Tablestore console.
Create a tunnel
- Log on to the Tablestore console.
On the Overview page, click the name of the instance that you want to manage or click Manage Instance in the Actions column of the instance.
In the Tables section of the Instance Details tab, click the name of the table that you want to manage. On the page that appears, click the Tunnels tab. You can also click of the table in the Actions column and select Tunnels.
On the Tunnels tab, click Create Tunnel.
In the Create Tunnel dialog box, configure Tunnel Name and Type.
Tunnel Service provides the following types of real-time consumption tunnels for distributed data: Incremental, Full, and Differential. In this example, Differential is selected.
After the tunnel is created, you can click Show Channels in the Actions column to view data in the tunnel, monitor consumption latency, and check the number of consumed rows in each channel.
Preview data types in a tunnel
After you create a tunnel, you can simulate data consumption to preview the data types in the tunnel.
Write data to or delete data from tables in the Tablestore console. For more information, see Read and write data in the console.
Preview the data types in a tunnel.
On the Overview page, click the name of the instance that you want to manage or click Manage Instance in the Actions column.
In the Tables section of the Instance Details tab, click the name of the table that you want to manage. On the page that appears, click the Tunnels tab. You can also click of the table in the Actions column and select Tunnels.
On the Tunnels tab, find the tunnel that you want to manage click Show Channels in the Actions column.
Find the channel that you want to manage and click View Simulated Export Records in the Actions column.
In the View Simulated Export Records dialog box, click Start.
The information about consumed data is displayed.
Enable data consumption for a tunnel
Copy a tunnel ID from the tunnel list.
Use Tunnel Service SDK in any programming language to enable data consumption for the tunnel.
In this example, Tunnel Service SDK for Java is used to enable data consumption for the tunnel.
import com.alicloud.openservices.tablestore.TunnelClient; import com.alicloud.openservices.tablestore.tunnel.worker.IChannelProcessor; import com.alicloud.openservices.tablestore.tunnel.worker.ProcessRecordsInput; import com.alicloud.openservices.tablestore.tunnel.worker.TunnelWorker; import com.alicloud.openservices.tablestore.tunnel.worker.TunnelWorkerConfig; public class TunnelQuickStart { /** * Create a custom data consumption callback to implement the IChannelProcessor operation. Specify the process and shutdown methods. */ private static class SimpleProcessor implements IChannelProcessor { @Override public void process(ProcessRecordsInput input) { System.out.println("Default record processor, would print records count"); System.out.println( String.format("Process %d records, NextToken: %s", input.getRecords().size(), input.getNextToken())); try { // Simulate the processing of data consumption. Thread.sleep(1000); } catch (InterruptedException e) { e.printStackTrace(); } } @Override public void shutdown() { System.out.println("Mock shutdown"); } } public static void main(String[] args) { // 1. Initialize a tunnel client. TunnelClient tunnelClient = new TunnelClient("<ENDPOINT>", "<ACCESS_ID>", "<ACCESS_KEY>", "<INSTANCE_NAME>"); // 2. Create a custom data consumption callback to start automatic data consumption. // We recommend that you share the same TunnelWorkerConfig. TunnelWorkerConfig provides more advanced parameters. TunnelWorkerConfig config = new TunnelWorkerConfig(new SimpleProcessor()); // Configure TunnelWorker and start automatic data processing. // TUNNEL_ID: the ID of the tunnel. TunnelWorker worker = new TunnelWorker("<TUNNEL_ID>", tunnelClient, config); try { worker.connectAndWorking(); } catch (Exception e) { e.printStackTrace(); worker.shutdown(); tunnelClient.shutdown(); } } }
View data consumption logs
After data is consumed, you can view the consumption logs of incremental data, including consumption statistics and the latest synchronization time of incremental data in channels. You can also log on to the Tablestore console or call the DescribeTunnel operation to view the consumption latency and the number of consumed rows in each channel.