All Products
Search
Document Center

OpenSearch:Quantized clustering configurations

Last Updated:Feb 27, 2024

QcBuilder

Parameter

Type

Default value

Description

qc.builder.train_sample_count

uint32

0

The volume of training data. If you set this parameter to 0, all data of a document is specified as training data.

qc.builder.thread_count

uint32

0

The number of threads that can be used. If you set this parameter to 0, the number of threads that can be used is equal to the number of CPU cores of an OpenSearch Vector Search Edition instance.

qc.builder.centroid_count

string

Optional

The number of centroids that you want to use for clusters. Hierarchical clusters are supported. Separate levels of hierarchical clusters with asterisks (*).

Sample value for hierarchical clusters that include one level: 1000.

Sample value for hierarchical clusters that include two levels: 100*100.

If you want to specify the number of centroids for hierarchical clusters that include two levels, we recommend that you specify more centroids for the first level than the second level. This ensures a result that is better than the result obtained when you specify less centroids for the first level than the second level. The experience points that can be obtained in the first level are 10 times those in the second level.

If you do not specify the number of centroids, the system automatically infers the appropriate number of centroids. We recommend that you allow the system to automatically infer the number of centroids.

qc.builder.quantizer_class

string

None

The quantizer. By default, the system does not use quantizers. Valid values: Int8QuantizerConverter, HalfFloatConverter, and DoubleBitConverter. In most cases, if you specify a quantizer, performance is improved and the size of an index is reduced. However, retrieval loss may occur in specific scenarios.

qc.builder.quantize_by_centroid

bool

False

Specifies whether to perform quantization based on centroids if you specify a value for the qc.builder.quantizer_class parameter. The qc.builder.quantize_by_centroid parameter takes effect only if you set the qc.builder.quantizer_class parameter to Int8QuantizerConverter.

QcSearcher

Parameter

Type

Default value

Description

qc.searcher.scan_ratio

float

0.01

The maximum ratio of documents that can be scanned during the search to all documents. This parameter is used to calculate the value of the max_scan_num parameter. The following formula is used to calculate this value: Value of the max_scan_num parameter = Total number of documents × Value of the qc.searcher.scan_ratio parameter.

qc.searcher.brute_force_threshold

int

1000

A threshold value. If the total number of documents is less than this threshold value, linear retrieval is performed.