TairVector is an extended data structure developed by the Tair team to provide high-performance real-time storage and retrieval of vectors.
Overview
TairVector uses a multi-layer hash structure, as shown in the following figure. TairVector provides two indexing algorithms: Hierarchical Navigable Small World (HNSW) and Flat Search.
HNSW: creates graph-based vector indexes and supports asynchronous memory space reclamation to maintain high query accuracy while still being able to handle real-time updates and changes to the index structure.
Flat Search: implements exact match and fast data insertion. This algorithm is suitable for small datasets.
Additionally, TairVector supports multiple distance formulas, such as Euclidean distance, inner product, Cosine distance, and Jaccard distance. Compared with traditional vector retrieval services, TairVector has the following advantages:
Stores all data in memory and supports real-time index updates to reduce read and write latency.
Uses an optimized data structure in memory to make better use of storage capacity.
Functions as an out-of-the-box data structure in a simple and efficient architecture without complex modules or dependencies.
Supports hybrid search that combines vector search and full-text search.
Allows you to create inverted indexes for scalar data such as tag attributes, and provides the k-nearest neighbor (KNN) search feature. During retrieval, TairVector first filters the dataset based on the scalar attributes and then applies the KNN algorithm for vector search on the filtered results.
Release notes
TairVector was released on October 13, 2022 together with Tair DRAM-based instances that are compatible with Redis 6.0.
With the release of Tair 6.2.2.0 on November 22, 2022, two notable additions to TairVector are support for the Jaccard distance formula and enhancement in the TVS.GETINDEX command. This command can provide memory usage statistics for each index by using the
index_data_size
andattribute_data_size
parameters.With the release of Tair 6.2.3.0 on December 26, 2022, notable additions to TairVector include support for cluster instances in proxy mode, introduction of the FLOAT16 data type, and inclusion of the TVS.MINDEXKNNSEARCH and TVS.MINDEXMKNNSEARCH commands.
With the release of Tair 6.2.8.2 on July 4, 2023, two notable additions to TairVector are support for the Cosine distance formula and automatic memory space reclamation of HNSW indexes.
With the release of Tair 23.8.0.0 on August 3, 2023, TairVector introduces commands such as TVS.HEXPIREAT and TVS.HPEXPIREAT that allow you to set a time-to-live (TTL) for keys in indexes, and introduces the TVS.GETDISTANCE command to support vector nearest-neighbor queries on specified keys. TairVector also enhances its capabilities with respect to full-text search by updating commands such as TVS.CREATEINDEX and TVS.KNNSEARCH. These enhancements allow you to combine vector search and full-text search to perform hybrid search.
With the release of Tair 24.5.1.0 on June 6, 2024, the TVS.KNNSEARCHFIELD and TVS.MINDEXKNNSEARCHFIELD commands are added to allow tag attribute information to be returned during nearest-neighbor queries.
Tair 24.7.0.0 was released on July 22, 2024 with added support for HNSW indexes in sparse vectors.
Best practices
Prerequisites
The instance is a Tair DRAM-based instance that is compatible with Redis 6.0 or later. For more information, see DRAM-based instances.
A DRAM-based instance that runs Redis 5.0 cannot be upgraded to one that runs Redis 6.0. To use a DRAM-based instance that runs Redis 6.0, you must create one.
Precautions
The TairVector data that you want to manage is stored on a Tair instance.
Hashtags in Redis cannot be used to control key distribution and slot allocation for TairVector indexes.
TairVector does not support features such as MOVE.
If your business requires high data persistence, we recommend that you enable the semi-synchronous replication feature by setting the #no_loose_tsync-repl-mode parameter to semisync. For more information, see Modify the values of parameters for an instance.
Supported commands
Table 1. TairVector commands
Type | Command | Syntax | Description |
Operations on vector indexes |
| Creates a vector index based on the specified distance formula and algorithm. The algorithm is used to create the index and perform queries. The index can be deleted only by running the | |
| Queries the specified vector index and retrieves the metadata of the index. | ||
| Deletes the specified vector index and all data of the index. | ||
| Scans the vector indexes that meet specific criteria in the Tair instance. | ||
Operations on vectors |
| Inserts a vector into the specified vector index. If the vector already exists, the existing data is overwritten. | |
| Queries all data records of the specified key in the specified vector index. | ||
| Queries the values of attribute keys in the specified key of the specified vector index. | ||
| Deletes the specified vectors from the specified vector index by using the key parameter. | ||
| Deletes the specified attributes of the specified vectors from the specified vector index. | ||
| Scans the keys that meet specific criteria in the specified vector index. | ||
| Increases the value of the specified attribute key for the specified key in the specified vector index by a given integer value (num). | ||
| Increases the value of the specified attribute key for the specified key in the specified vector index by a given floating-point number (num). | ||
| Specifies an absolute expiration time that is accurate to the millisecond for the specified key in the specified vector index. | ||
| Specifies a relative expiration time that is accurate to the millisecond for the specified key in the specified vector index. | ||
| Specifies an absolute expiration time that is accurate to the second for the specified key in the specified vector index. | ||
| Specifies a relative expiration time that is accurate to the second for the specified key in the specified vector index. | ||
| Queries the TTL of the specified key in the specified vector index, accurate to the millisecond. | ||
| Queries the TTL of the specified key in the specified vector index, accurate to the second. | ||
| Queries the absolute expiration time of the specified key in the specified vector index, accurate to the millisecond. | ||
| Queries the absolute expiration time of the specified key in the specified vector index, accurate to the second. | ||
Vector nearest-neighbor queries |
| Queries the nearest-neighbor vectors of the specified vector in the specified vector index. Up to 10,000 vectors can be returned. | |
| Queries the nearest-neighbor vectors of the specified vector in the specified vector index. The search logic of this command is the same as that of TVS.KNNSEARCH, with the additional capability to return tag attributes along with the search results. | ||
| Performs vector nearest-neighbor queries for the specified keys in the specified vector index. | ||
| Queries the nearest-neighbor vectors of the specified vectors in the specified vector index. | ||
| Queries the nearest-neighbor vectors of the specified vector across multiple vector indexes. | ||
| Queries the nearest-neighbor vectors of the specified vector across multiple vector indexes. Tag attributes can be returned along with the search results. | ||
| Queries the nearest-neighbor vectors of the specified vectors across multiple vector indexes. |
The following list describes the conventions for the command syntax used in this topic:
Uppercase keyword
: indicates the command keyword.Italic text
: indicates variables.[options]
: indicates that the enclosed parameters are optional. Parameters that are not enclosed by brackets must be specified.A|B
: indicates that the parameters separated by the vertical bars (|) are mutually exclusive. Only one of the parameters can be specified....
: indicates that the parameter preceding this symbol can be repeatedly specified.
TVS.CREATEINDEX
Item | Description |
Syntax |
|
Time complexity | O(1) |
Command description | Creates a vector index based on the specified distance formula and algorithm. The algorithm is used to create the index and perform queries. The index can be deleted only by running the |
Parameter |
|
Output |
|
Example | Sample commands:
Sample output:
|
TVS.GETINDEX
Item | Description |
Syntax |
|
Time complexity | O(1) |
Command description | Queries the specified vector index and retrieves the metadata of the index. |
Parameter |
|
Output |
|
Example | Run the following commands in advance:
Sample command (with the algorithm parameter set to HNSW):
Sample output:
|
TVS.DELINDEX
Item | Description |
Syntax |
|
Time complexity | O(N), in which N indicates the number of keys in the vector index. |
Command description | Deletes the specified vector index and all data of the index. |
Parameter |
|
Output |
|
Example | Sample command:
Sample output:
|
TVS.SCANINDEX
Item | Description |
Syntax |
|
Time complexity | O(N), in which N indicates the number of vector indexes in the Tair instance. |
Command description | Scans the vector indexes that meet specific criteria in the Tair instance. |
Parameter |
|
Output |
|
Example | Sample command:
Sample output:
Sample output (including the pattern parameter):
Sample output:
|
TVS.HSET
Item | Description |
Syntax |
|
Time complexity | If the operation of inserting or updating data does not involve the creation or update of a vector value, the time complexity is O(1). Otherwise, the time complexity is O(log(N)), where N indicates the number of keys in the vector index. |
Command description | Inserts a vector into the specified vector index. If the vector already exists, the existing data is overwritten. |
Parameter |
|
Output |
|
Example | Sample command:
Sample output:
|
TVS.HGETALL
Item | Description |
Syntax |
|
Time complexity | O(1) |
Command description | Queries all data records of the specified key in the specified vector index. |
Parameter |
|
Output |
|
Example | Sample command:
Sample output:
|
TVS.HMGET
Item | Description |
Syntax |
|
Time complexity | O(1) |
Command description | Queries the values of attribute keys in the specified key of the specified vector index. |
Parameter |
|
Output |
|
Example | Sample command:
Sample output:
|
TVS.DEL
Item | Description |
Syntax |
|
Time complexity | O(1) |
Command description | Deletes the specified vectors from the specified vector index by using the key parameter. |
Parameter |
|
Output |
|
Example | Sample command:
Sample output:
|
TVS.HDEL
Item | Description |
Syntax |
|
Time complexity | O(1) |
Command description | Deletes the specified attributes of the specified vectors from the specified vector index. |
Parameter |
|
Output |
|
Example | Sample command:
Sample output:
|
TVS.SCAN
Item | Description |
Syntax |
|
Time complexity | O(N), in which N indicates the number of keys in the vector index. |
Command description | Scans the keys that meet specific criteria in the specified vector index. |
Parameter |
|
Output |
|
Example | Sample command:
Sample output:
|
TVS.HINCRBY
Item | Description |
Syntax |
|
Time complexity | O(1) |
Command description | Increases the value of the specified attribute key for the specified key in the specified vector index by a given integer value (num). If the specified attribute key does not exist, the attribute key is automatically created and assigned the provided num value. If the attribute key already exists, TairVector updates the value of the attribute key by adding the specified num value to the existing value. |
Parameter |
|
Output |
|
Example | Sample command:
Sample output:
|
TVS.HINCRBYFLOAT
Item | Description |
Syntax |
|
Time complexity | O(1) |
Command description | Increases the value of the specified attribute key for the specified key in the specified vector index by a given floating-point number (num). If the specified attribute key does not exist, the attribute key is automatically created and assigned the provided num value. If the attribute key already exists, TairVector updates the value of the attribute key by adding the specified num value to the existing value. |
Parameter |
|
Output |
|
Example | Sample command:
Sample output:
|
TVS.HPEXPIREAT
Item | Description |
Syntax |
|
Time complexity | O(1) |
Command description | Specifies an absolute expiration time that is accurate to the millisecond for the specified key in the specified vector index. |
Parameter |
|
Output |
|
Example | Sample command:
Sample output:
|
TVS.HPEXPIRE
Item | Description |
Syntax |
|
Time complexity | O(1) |
Command description | Specifies a relative expiration time that is accurate to the millisecond for the specified key in the specified vector index. |
Parameter |
|
Output |
|
Example | Sample command:
Sample output:
|
TVS.HEXPIREAT
Item | Description |
Syntax |
|
Time complexity | O(1) |
Command description | Specifies an absolute expiration time that is accurate to the second for the specified key in the specified vector index. |
Parameter |
|
Output |
|
Example | Sample command:
Sample output:
|
TVS.HEXPIRE
Item | Description |
Syntax |
|
Time complexity | O(1) |
Command description | Specifies a relative expiration time that is accurate to the second for the specified key in the specified vector index. |
Parameter |
|
Output |
|
Example | Sample command:
Sample output:
|
TVS.HPTTL
Item | Description |
Syntax |
|
Time complexity | O(1) |
Command description | Queries the TTL of the specified key in the specified vector index, accurate to the millisecond. |
Parameter |
|
Output |
|
Example | Sample command:
Sample output:
|
TVS.HTTL
Item | Description |
Syntax |
|
Time complexity | O(1) |
Command description | Queries the TTL of the specified key in the specified vector index, accurate to the second. |
Parameter |
|
Output |
|
Example | Sample command:
Sample output:
|
TVS.HPEXPIRETIME
Item | Description |
Syntax |
|
Time complexity | O(1) |
Command description | Queries the absolute expiration time of the specified key in the specified vector index, accurate to the millisecond. |
Parameter |
|
Output |
|
Example | Sample command:
Sample output:
|
TVS.HEXPIRETIME
Item | Description |
Syntax |
|
Time complexity | O(1) |
Command description | Queries the absolute expiration time of the specified key in the specified vector index, accurate to the second. |
Parameter |
|
Output |
|
Example | Sample command:
Sample output:
|
TVS.KNNSEARCH
Item | Description |
Syntax |
|
Time complexity |
N indicates the number of keys in the vector index. |
Command description | Queries the nearest-neighbor vectors of the specified vector in the specified vector index. Up to 10,000 vectors can be returned. |
Parameter |
|
Output |
|
Example | Run the following commands in advance:
Sample command 1:
Sample output 1:
Sample command 2:
Sample output 2:
|
TVS.KNNSEARCHFIELD
Item | Description |
Syntax |
|
Time complexity |
N indicates the number of keys in the vector index. |
Command description | Queries the nearest-neighbor vectors of the specified vector in the specified vector index. The search logic of this command is the same as that of TVS.KNNSEARCH, with the additional capability to return tag attributes along with the search results. |
Parameter |
|
Output |
|
Example | Run the following commands in advance:
Sample command:
Sample output:
|
TVS.GETDISTANCE
Item | Description |
Syntax |
|
Time complexity |
N indicates the number of keys in the vector index. |
Command description | Performs vector nearest-neighbor queries for the specified keys in the specified vector index. |
Parameter |
|
Output |
|
Example | Run the following commands in advance:
Sample command:
Sample output:
|
TVS.MKNNSEARCH
Item | Description |
Syntax |
|
Time complexity |
N indicates the number of keys in the vector index. |
Command description | Queries the nearest-neighbor vectors of the specified vectors in the specified vector index. |
Parameter |
|
Output |
|
Example | Run the following commands in advance:
Sample command:
Sample output:
|
TVS.MINDEXKNNSEARCH
Item | Description |
Syntax |
|
Time complexity |
N indicates the number of keys in the vector index. |
Command description | Queries the nearest-neighbor vectors of the specified vector across multiple vector indexes. |
Parameter |
|
Output |
|
Example | Run the following commands in advance:
Sample command:
Sample output:
|
TVS.MINDEXKNNSEARCHFIELD
Item | Description |
Syntax |
|
Time complexity |
N indicates the number of keys in the vector index. |
Command description | Queries the nearest-neighbor vectors of the specified vector across multiple vector indexes. Tag attributes can be returned along with the search results. |
Parameter |
|
Output |
|
Example | Run the following commands in advance:
Sample command:
Sample output:
|
TVS.MINDEXMKNNSEARCH
Item | Description |
Syntax |
|
Time complexity |
N indicates the number of keys in the vector index. |
Command description | Queries the nearest-neighbor vectors of the specified vectors across multiple vector indexes. |
Parameter |
|
Output |
|
Example | Run the following commands in advance:
Sample command:
Sample output:
|