Tablestore:Data versions and TTL

Max versions

If you do not specify a value for Max Versions when you create a data table, the default value 1 is used. You can also specify a custom value for Max Versions for data in attribute columns. After you create a data table, you can call the UpdateTable operation to modify the value of Max Versions for the data table.

If the number of versions exceeds the value of max versions that you specify, the data of earlier versions becomes invalid and you cannot read the data even if the data has not been deleted by Tablestore.

• When you decrease the value of max versions, Tablestore asynchronously deletes data of earlier versions if the number of data versions exceeds the new value of max versions.

• When you increase the value of max versions, you can read the data of specific versions if Tablestore has not deleted the data of the versions and the versions are within the new version range.

Version number

After you configure Max Versions for a data table, each time you update a value in an attribute column, Tablestore generates a new version for the value. The version number is the timestamp in milliseconds.

When you write data to an attribute column, you can specify a version number. If you do not specify a version number, Tablestore automatically generates a version number. Version numbers are timestamps in milliseconds. If you want to compare version numbers with TTL values or calculate the max version offset, you must divide the version numbers by 1,000.

• By default, the version number that is generated by Tablestore is the current timestamp, which is the number of milliseconds that have elapsed since 00:00:00 UTC on January 1, 1970.

• If you specify a version number for attribute columns, make sure that the version number is a 64-bit timestamp that is accurate to milliseconds and within the valid version range.

You can use version numbers to implement the following features:

• TTL

  You can specify the TTL of attribute column values in a data table based on version numbers. When data of a specific version in an attribute column is retained for a period of time that exceeds the TTL value, Tablestore automatically deletes the data of the specific version in asynchronous mode.

  For example, the version number of the data in an attribute column is 1468944000000, which is equal to 00:00:00 UTC+8 on July 20, 2016. If you set the value of TTL to 86,400 seconds (one day), the data of this version expires at 00:00:00 UTC+8 on July 21, 2016. Tablestore automatically deletes the data in asynchronous mode after the data expires.

• Maximum number of versions or range of version numbers that you want Tablestore to read from each attribute column

  When Tablestore reads a row of data, you can specify the maximum number of versions or the range of version numbers that you want Tablestore to read from each attribute column. For more information, see Read data.

TTL

TTL specifies the validity period of data in a data table in seconds. When data of a specific version in an attribute column is retained for a period of time that exceeds the TTL value, Tablestore automatically deletes the data of the specific version in asynchronous mode from the attribute column. If data in all attribute columns of a row is retained for a period of time that exceeds the TTL value, Tablestore automatically deletes the row in asynchronous mode.

For example, you set the TTL of the data in a data table to 86,400 seconds (one day). At 00:00:00 UTC+8 on July 21, 2016, data whose version number is smaller than 1468944000000 expires. The version number 1468944000000 is equal to 00:00:00 UTC+8 on July 20, 2016 after it is divided by 1,000 to convert into seconds. Tablestore automatically deletes the expired data in asynchronous mode.

If you do not specify a value for TTL when you create a data table, the default value -1 is used. A value of -1 indicates that data in the data table never expires. You can also specify a custom TTL value. After you create a data table, you can call the UpdateTable operation to modify the TTL value for the data in the data table.

If data is retained for a period of time that exceeds the TTL value, the data becomes invalid and you cannot read the data even if the data has not been deleted by Tablestore.

• When you decrease the TTL value, Tablestore asynchronously deletes the data that is retained for a period of time that exceeds the new TTL value.

• When you increase the TTL value, you can read the data of specific versions if Tablestore has not deleted the data and the data of the versions is retained for a period of time that is within the new TTL value.

Max version offset

If the difference between the current time and the version timestamp that you specify exceeds the TTL value that you specified for the data table, the written data immediately expires. To resolve this issue, Tablestore allows you to configure the max version offset.

The max version offset specifies the maximum difference between the current system time and the specified version number in seconds. The value of max version offset is a positive integer that can be greater than the number of seconds that have elapsed since 00:00:00 UTC on January 1, 1970.

When you write data to an attribute column, Tablestore checks the version number of the data. The valid version range of data in an attribute column is calculated by using the following formula: Valid version range = [max{Data written time - Max version offset, Data written time - TTL value}, Data written time + Max version offset). You can write data to the attribute column only if the version number of the data (converted into seconds by dividing by 1,000) is within the valid version range.

For example, you set the max version offset of the data in each attribute column of a data table to 86,400 seconds (one day). At 00:00:00 UTC on July 21, 2016, only the data whose version range is between 1468944000000 (00:00:00 UTC+8 on July 20, 2016) and 1469116800000 (00:00:00 UTC+8 on July 22, 2016) can be written. If you attempt to write a row of data in which the version number of data in an attribute column is 1468943999000 (23:59:59 UTC+8 on July 19, 2016), the row fails to be written.

If you do not specify a value for max version offset when you create a data table, the default value 86400 is used. You can also specify a custom max version offset for the data in the data table. After you create a data table, you can call the UpdateTable operation to modify the max version offset for the data in the data table.

How to use

When you create a data table, you can configure parameters such as max versions and TTL. You can also modify the parameters such as max versions and TTL after you create a data table. The section describes how to modify the configurations of a data table after the data table is created.

alter -t mytable --ttl 86400 --version 1

private static void updateTable(SyncClient client) {
    // Specify the TTL. A value of -1 specifies that the data never expires. Unit: seconds. 
    int timeToLive = -1;
    // Update Max Versions to 5. 
    int maxVersions = 5; 
    // Update Max Version Offset to 86400. Unit: seconds. 
    int maxTimeDeviation=86400;
    TableOptions tableOptions = new TableOptions(timeToLive, maxVersions,maxTimeDeviation);
    // Specify the name of the table. 
    UpdateTableRequest request = new UpdateTableRequest("<TABLE_NAME>");
    request.setTableOptionsForUpdate(tableOptions);
    client.updateTable(request);
}

Billing

• When you use the max versions feature, you are charged for the storage usage of version numbers and data with various version numbers.

• When you configure TTL to delete data, you are not charged for the delete operations that are performed by Tablestore. When you enable the TTL feature, Tablestore adds timestamps to each attribute column as the version numbers. You are charged for the storage usage of version numbers and data with various version numbers.

For more information, see Storage usage.

FAQ

What do I do if the [table ttl] must be bigger than or equal search index ttl error message is returned when I modify the TTL of a data table?