Tablestore:Data versions and TTL

Max Versions

The Max Versions setting defaults to 1 if not specified during table creation, but a custom value can be provided. After creation, this value can be modified by calling the UpdateTable operation.

Modifying the Max Versions setting has immediate effects on data readability and retention. Once the number of versions for an attribute exceeds the limit, the oldest versions become unreadable, even before they are physically removed by the asynchronous cleanup process. The specific behavior when changing the Max Versions value is as follows:

• Decreasing the value: Triggers an asynchronous cleanup to delete the oldest versions that now exceed the new, lower limit.

• Increasing the value: Can make older, previously unreadable versions accessible again, provided they have not yet been permanently deleted.

Version number

After you configure Max Versions for a data table, each time you update a value in an attribute column, Tablestore generates a new version for the value. The version number is the timestamp in milliseconds.

When writing data to an attribute column, a version number can either be specified or be automatically generated by Tablestore. These version numbers are millisecond-precision timestamps. As TTL and Max Version Offset are measured in seconds, version timestamps must be converted to seconds (by dividing by 1,000) before any comparison.

• By default, the version number that is generated by Tablestore is the current timestamp, which is the number of milliseconds that have elapsed since 00:00:00 UTC on January 1, 1970.

• If you specify a version number for attribute columns, make sure that the version number is a 64-bit timestamp that is accurate to milliseconds and within the valid version range.

Version numbers enable the following features:

• TTL

  The TTL of an attribute column value is determined based on its version number. When data of a specific version in an attribute column is retained for a period of time that exceeds the TTL value, Tablestore automatically deletes the data of the specific version in asynchronous mode.

  For example, the version number of the data in an attribute column is 1468944000000, which is equal to 00:00:00 UTC+8 on July 20, 2016. If you set the value of TTL to 86,400 seconds (one day), the data of this version expires at 00:00:00 UTC+8 on July 21, 2016. Tablestore automatically deletes the data in asynchronous mode after the data expires.

• Maximum number of versions or range of version numbers that you want Tablestore to read from each attribute column

  During a read operation, the maximum number of versions or a specific range of version numbers can be specified for each attribute column. For more information, see Read data.

TTL

TTL defines the validity period of data in a table, specified in seconds. When data of a specific version in an attribute column is retained for a period of time that exceeds the TTL value, Tablestore automatically deletes the data of the specific version in asynchronous mode from the attribute column. If all attribute columns in a row have expired, the entire row is eligible for asynchronous deletion by Tablestore.

For example, you set the TTL of the data in a data table to 86,400 seconds (one day). At 00:00:00 UTC+8 on July 21, 2016, data whose version number is smaller than 1468944000000 expires. The version number 1468944000000 is equal to 00:00:00 UTC+8 on July 20, 2016 after it is divided by 1,000 to convert into seconds. Tablestore automatically deletes the expired data in asynchronous mode.

The TTL setting defaults to -1 if unspecified during table creation, which means the data never expires. A custom TTL value can also be provided. After creation, this value can be modified by calling the UpdateTable operation.

If data is retained for a period of time that exceeds the TTL value, the data becomes invalid and you cannot read the data even if the data has not been deleted by Tablestore.

Modifying the TTL value affects data retention and accessibility. The specific behavior when changing the TTL is as follows:

• Decreasing the value: Triggers an asynchronous cleanup to delete data that has now exceeded the new, shorter retention period.

• Increasing the value: Can make previously expired but not-yet-deleted data accessible again, as it now falls within the new, longer retention period.

Max Version Offset

When writing data with a custom timestamp, the data may expire immediately if its timestamp is too old (current_time - timestamp > TTL). The Max Version Offset feature is designed to prevent this issue.

The Max Version Offset specifies the maximum difference between the current system time and the specified version number in seconds. The value of Max Version Offset is a positive integer that can be greater than the number of seconds that have elapsed since 00:00:00 UTC on January 1, 1970.

When you write data to an attribute column, Tablestore checks the version number of the data. The valid version range of data in an attribute column is calculated by using the following formula: Valid version range = [max{Data written time - Max Version Offset, Data written time - TTL value}, Data written time + Max Version Offset). Data can be written to an attribute column only if its version number (converted into seconds by dividing by 1,000) falls within this valid range.

For example, you set the Max Version Offset of the data in each attribute column of a data table to 86,400 seconds (one day). At 00:00:00 UTC on July 21, 2016, only the data whose version range is between 1468944000000 (00:00:00 UTC+8 on July 20, 2016) and 1469116800000 (00:00:00 UTC+8 on July 22, 2016) can be written. If you attempt to write a row of data in which the version number of data in an attribute column is 1468943999000 (23:59:59 UTC+8 on July 19, 2016), the row fails to be written.

The Max Version Offset setting defaults to 86400 (seconds) if unspecified during table creation, but a custom value can be provided. After creation, this value can be modified by calling the UpdateTable operation.

Configuration methods

Parameters such as Max Versions and TTL can be configured both during table creation and modified afterward. This section describes the methods for modifying these parameters on an existing table.

alter -t mytable --ttl 86400 --version 1

private static void updateTable(SyncClient client) {
    // Specify the TTL. A value of -1 specifies that the data never expires. Unit: seconds. 
    int timeToLive = -1;
    // Update Max Versions to 5. 
    int maxVersions = 5; 
    // Update Max Version Offset to 86400. Unit: seconds. 
    int maxTimeDeviation=86400;
    TableOptions tableOptions = new TableOptions(timeToLive, maxVersions,maxTimeDeviation);
    // Specify the name of the table. 
    UpdateTableRequest request = new UpdateTableRequest("<TABLE_NAME>");
    request.setTableOptionsForUpdate(tableOptions);
    client.updateTable(request);
}

Billing

• When you use the Max Versions feature, you are charged for the storage usage of version numbers and data with various version numbers.

• When you configure TTL to delete data, you are not charged for the delete operations that are performed by Tablestore. When you enable the TTL feature, Tablestore adds timestamps to each attribute column as the version numbers. You are charged for the storage usage of version numbers and data with various version numbers.

For more information, see Data storage.

FAQ

• What do I do if the [table ttl] must be bigger than or equal search index ttl error message is returned when I modify the TTL of a data table?

• Why does the size of a table change after I configure TTL for the table?