All Products
Search
Document Center

Tablestore:Overview

Last Updated:Sep 02, 2024

The TimeSeries model is designed based on the characteristics of time series data. This model is suitable for scenarios such as IoT device monitoring and can be used to store data collected by devices and the monitoring data of machines. The TimeSeries model supports automatic indexing of time series metadata and time series retrieval based on composite conditions. The TimeSeries model uses time series tables to store time series data. This allows applications to write and read petabytes of data concurrently and reduces the storage costs. You can execute SQL statements to query and analyze time series data.

Introduction

The TimeSeries model is designed based on the characteristics of time series data. This model is suitable for scenarios such as IoT device monitoring and can be used to store data collected by devices and the monitoring data of machines.

In the TimeSeries model of Tablestore, a two-dimensional time series table is used to store time series data. Each row represents the data at a point in time in a time series. The time series identifiers and timestamp are the primary key columns of the row, and the data points of the time series under the timestamp are the data columns of the row. A row can contain multiple data columns. You do not need to predefine the schemas of the primary key columns and data columns. You need to only specify the names of the specific data columns when you write data to the time series table.

A time series table can store time series data of different metric types. In the following figure, the time series table stores data of the following two metric types: temperature and humidity.

fig_timestreamsample

In the figure, time series identifiers consist of measurement, data source, and tags. You can call an API operation to update the properties in the metadata of a time series. The properties can be used to retrieve the time series.

After data is written to a time series table, the system automatically extracts the metadata of the time series and automatically creates a metadata index. You can retrieve a time series based on the combination of the metric name, data source, and tags.

The following items describe the benefits of the TimeSeries model:

  • Provides a unified-common modeling method for time series data, which eliminates the need to predefine table schemas.

  • Allows metadata indexes to be automatically created for time series and supports time series retrieval based on composite conditions.

  • Supports queries and aggregation by using SQL.

  • Supports automatic scale-out of service capabilities, high-concurrency writes and queries, and low-cost storage of petabytes of data.

Terms

Term

Description

time series data

Time series data consists of multiple time series. Each time series is a set of data points that are arranged in chronological order. Metadata is required to identify a time series. Therefore, time series data consists of metadata and data.

  • Metadata: records the identifiers and properties of all time series.

  • Data: records the data points of all time series. The data points include the time when the data points are generated and the corresponding data values.

time series metadata

Time series metadata contains the identifiers and properties of a time series. The identifiers are used to uniquely identify a time series. The properties can be modified and can be used to retrieve time series.

time series identifier

Time series identifiers are used to uniquely identify a time series. In the TimeSeries model of Tablestore, time series identifiers consist of the following three parts: metric name, data source, and tags.

metric name

The name of a physical quantity or metric for data in a time series, such as cpu or net, which indicates the CPU usage or network usage.

data source

The identifier of the data source for the time series. This parameter can be left empty.

tag

The tag of the time series. You can specify multiple key-value pairs of the string type.

property

Properties are part of time series metadata and can be used to record modifiable property information of a time series. However, the properties cannot be used as the identifier of a time series and cannot be used to uniquely identify a time series. The properties of a time series are multiple key-value pairs of the string type, which are similar to tags in the format. You can specify or update the properties of a time series to retrieve the time series by using the properties.

data in a time series

A data point in a time series consists of the time when the data is generated and the corresponding data value. If only one value is generated at each moment in a time series, the single-value model is used. If multiple values are generated at each moment in a time series, the multi-value model is used.

The TimeSeries model of Tablestore uses the multi-value model. You can specify multiple data values at one point in time. Each value corresponds to a column in the time series table, including the column name and column value. Column values support the following data types: Boolean, integer, floating-point, string, and binary.

Features

  • Create and manage time series tables

    You can use the Tablestore console, SDKs, or CLI to query all time series tables in an instance, create a time series table, query the configurations of a time series table, update the configurations of a time series table, and delete a time series table.

    When you create a time series table or update the configurations of a time series table, you can specify the time to live (TTL) for the data in the time series table. After the TTL value is specified, the system automatically checks the difference between the current time and the timestamp when the time series data was written. If the difference exceeds the TTL value, the system automatically deletes the expired data.

  • Read and write time series data

    You can use the Tablestore console, SDKs, or CLI to write multiple rows of time series data to a time series table at the same time. After data is written to the time series table, you can specify time series identifiers to query the data in the time series within the specified time range.

  • Retrieve time series

    You can use the Tablestore console, SDKs, or CLI to retrieve time series in a time series table. You can use a composite condition that consists of multiple conditions to retrieve time series. For example, you can retrieve all time series in which the metric name is cpu, the tags contain a tag whose name is region and value is hangzhou, and the properties contain a property whose name is status and value is online. After the time series are retrieved, you can call an API operation to further query the data in the time series.

  • Implement SQL query and analytics

    Time series tables support queries by using SQL. In SQL, you can specify the metadata condition to filter time series and aggregate data based on the aggregation operations in different dimensions. For example, you can query the average value of the sample data that is collected from a batch of devices and aggregate second-level data into minute-level data.

    In addition, you can query only the metadata of time series in SQL. This way, you can manage the metadata of time series by using SQL.

Limits

For more information, see Limits on the TimeSeries model.

Usage notes

The TimeSeries model is supported in the following regions: China (Hangzhou), China (Shanghai), China (Beijing), China (Zhangjiakou), China (Ulanqab), China (Shenzhen), China (Hong Kong), Germany (Frankfurt), US (Virginia), SAU (Riyadh - Partner Region), and Singapore.

API operations

Category

Operation

Description

API operations on time series tables

CreateTimeseriesTable

Creates a time series table.

ListTimeseriesTable

Queries the names of time series tables in an instance.

DescribeTimeseriesTable

Queries information about a time series table.

UpdateTimeseriesTable

Updates the configurations of a time series table.

DeleteTimeseriesTable

Deletes a time series table.

API operations on time series

PutTimeseriesData

Writes time series data to a time series table.

GetTimeseriesData

Queries the data in a time series.

QueryTimeseriesMeta

Retrieves time series metadata.

UpdateTimeseriesMeta

Updates time series metadata.

DeleteTimeseriesMeta

Deletes the metadata of a time series.

Procedure

image

Step

Operation

Description

1

Grant permissions on Tablestore resources to a RAM user

After you create a RAM user, grant minimal permissions to access Tablestore resources to the RAM user. You can use system policies or custom policies to grant the RAM user the permissions to access Tablestore resources.

If you want to use an Alibaba Cloud account or a RAM user that has the required permissions to access Tablestore resources, skip this step.

Important

By default, an Alibaba Cloud account has permissions on all cloud resources. To ensure the security of your resources, we recommend that you create RAM users for your Alibaba Cloud account and authorize the RAM users to access different resources.

2

Activate Tablestore

Before you use the features of Tablestore, you must activate Tablestore.

You need to activate Tablestore only once. You are not charged when you activate Tablestore. If Tablestore is activated, skip this step.

3

Create a Tablestore instance for the TimeSeries model

Important
  • Before you create a Tablestore instance, you must determine the model of the table that you want to create in the instance and the instance type based on the business characteristics and business requirements on read and write performance and costs. For more information, see Billing overview and Instances.

  • If the Create Instance for TimeSeries Model button is not available in a region that supports the TimeSeries model, click Create Instance. For more information, see Create an instance.

Create a Tablestore instance in the selected region based on the model of the table that you want to create in the instance and the instance type.

If an existing Tablestore instance meets your business requirements, skip this step.

4

Create a time series table

Create a time series table to store time series data. When you create a time series table, you can configure the TTL of the time series table based on whether you want to clear historical data.

5

Write time series data

Write time series data to the time series in a batch. Time series data consists of metadata and data. If you do not create metadata before you write the time series data, the system automatically extracts the metadata from the written data.

Note

You can also use the kafka-connect-tablestore package to synchronize data from Apache Kafka to the time series table. For more information, see Data synchronization to time series tables.

6

Query time series data

Important

If you are unsure about information of the time series whose data you want to query, such as the metric name and data source, you can specify multiple conditions to retrieve a time series. For more information, see Manage time series metadata.

After the time series whose data you want to query is retrieved, you can query the time series data that meets the specified conditions such as a time range in the time series.

7

Use SQL to query time series data

You can create three types of mapping tables for a time series table in SQL to query data.

  • After you create a time series table, the system automatically creates a mapping table for time series metadata for the time series table. You can use the mapping table for time series metadata to retrieve time series.

  • After you create a time series table, the system automatically creates a mapping table in the single-value model in SQL for the time series table. You can use the mapping table in the single-value model to query time series data in the time series table.

  • After you create a time series table, you can create a mapping table in the multi-value model in SQL for the time series table. You can use the mapping table in the multi-value model to query time series data in the time series table.

Use the TimeSeries model

Billing rules

The billable items of the TimeSeries model include the read throughput, write throughput, and storage usage of time series data and time series metadata, and the outbound traffic over the Internet. For more information, see Billable items of the TimeSeries model.

FAQ

References

  • To delete historical time series data in a time series table, you can configure the TTL for the time series table. For more information, see TTL of time series data.

  • To back up time series data in Tablestore in a cost-effective manner or export time series data as files to local devices, you can use the Data Integration feature of DataWorks to synchronize time series data from Tablestore to Object Storage Service (OSS) for storage or download. For more information, see Synchronize data from Tablestore to OSS.

  • To visualize time series data, you can connect Tablestore to Grafana. For more information, see Connect Tablestore to Grafana.

  • If you use Realtime Compute for Apache Flink to compute and analyze data, you can use Tablestore time series tables to store the results. For more information, see Tutorial (TimeSeries model).