All Products
Search
Document Center

OpenSearch:Quantized clustering configurations

Last Updated:Feb 28, 2024

QcBuilder

Parameter

Type

Default value

Description

qc.builder.train_sample_count

uint32

0

The volume of training data. If you set this parameter to 0, all data of a document is specified as training data.

qc.builder.thread_count

uint32

0

The number of threads that can be used. If you set this parameter to 0, the number of threads that can be used is equal to the number of CPU cores of an OpenSearch Retrieval Engine Edition instance.

qc.builder.centroid_count

string

Optional

The number of centroids that you want to use for clusters. Hierarchical clusters are supported. Separate levels of hierarchical clusters with asterisks (*).

Sample value for hierarchical clusters that include one level: 1000

Sample value for hierarchical clusters that include two levels: 100*100

If you want to specify the number of centroids for hierarchical clusters that include two levels, we recommend that you specify more centroids for the first level than the second level. This ensures a result that is better than the result obtained when you specify less centroids for the first level than the second level. The experience points that can be obtained in the first level are 10 times those in the second level.

If you do not specify the number of centroids, the system automatically infers the appropriate number of centroids. We recommend that you allow the system to automatically infer the number of centroids.

qc.builder.quantizer_class

string

-

The quantizer. By default, the system does not use quantizers. Valid values: Int8QuantizerConverter, HalfFloatConverter, and DoubleBitConverter. In most cases, if you specify a quantizer, performance is improved and the size of an index is reduced. However, retrieval loss may occur in specific scenarios.

qc.builder.quantize_by_centroid

bool

False

Specifies whether to perform quantization based on centroids if you specify a value for the qc.builder.quantizer_class parameter. The qc.builder.quantize_by_centroid parameter takes effect only if you set the qc.builder.quantizer_class parameter to Int8QuantizerConverter.

QcSearcher

Parameter

Type

Default value

Description

qc.searcher.scan_ratio

float

0.01

The maximum ratio of documents that can be scanned during the search to all documents. This parameter is used to calculate the value of the max_scan_num parameter based on the following formula: Value of the max_scan_num parameter = Total number of documents × Value of the qc.searcher.scan_ratio parameter.

qc.searcher.brute_force_threshold

int

1000

A threshold value for linear retrieval. If the total number of documents is less than this threshold value, linear retrieval is performed.