cluster
配置中同時包含線上部分 (ha3) 配置和離線 (build_service) 兩部分配置;離線配置中包含了對builder 和merger 的各種配置。
customized_merge_config
customized_merge_config:使用者自訂配置。使用者可以對不同情境定製不同的merge策略。比如增量階段使用和預設策略不同的merge策略;可以配置多個,多個merge策略會同時生效,相同時間如果觸發了多個merge策略會按照配置中的順序先後觸發merge plan。 使用者除了指定merge_config,還可以配置下面兩個參數:
period: 索引的merge間隔時間,單位為s。或配置在某個固定時刻merge,比如在下午13pm觸發merge,格式是"period": "daytime=13:00"
need_wait_alter_field:如果當前有動態加欄位流程,是否要阻塞merge,預設是true。如果配成false,當前merge 策略將自動換成align_version merge策略生效。
merge_parallel_num: 每個partition參與merge的並發進程數量。
由於新增動態加欄位功能。禁止使用者自訂的merge策略命名為alter_field。
customized_merge_config預設值如下:
{
"full":{
"merge_config":{
"keep_version_count":40,
"merge_strategy":"optimize",
"merge_strategy_param":"after-merge-max-segment-count=20",
"merge_thread_count":4
}
},
"large_segment_reclaim":{
"merge_config":{
"keep_version_count":40,
"merge_strategy":"priority_queue",
"merge_strategy_params":{
"input_limits":"max-segment-size=20480",
"output_limits":"max-merged-segment-size=13312;max-total-merged-size=15360",
"strategy_conditions":"priority-feature=delete-doc-count#desc;conflict-segment-count=2;conflict-delete-percent=8"
},
"merge_thread_count":4
},
"period":"period=1800"
},
"segment_merge":{
"merge_config":{
"keep_version_count":40,
"merge_strategy":"priority_queue",
"merge_strategy_params":{
"input_limits":"max-segment-size=12288",
"output_limits":"max-merged-segment-size=13312;max-total-merged-size=15360",
"strategy_conditions":"priority-feature=valid-doc-count#asc;conflict-segment-count=2;conflict-delete-percent=10"
},
"merge_thread_count":4
},
"period":"period=900"
},
"large_segment_merge":{
"merge_config":{
"keep_version_count":40,
"merge_strategy":"priority_queue",
"merge_strategy_params":{
"input_limits":"max-segment-size=12288",
"output_limits":"max-merged-segment-size=15360;max-total-merged-size=46080",
"strategy_conditions":"priority-feature=valid-doc-count#desc;conflict-segment-count=2;conflict-delete-percent=10"
},
"merge_thread_count":4
},
"period":"period=3600"
},
"small_segment_merge":{
"merge_config":{
"keep_version_count":40,
"merge_strategy":"priority_queue",
"merge_strategy_params":{
"input_limits":"max-segment-size=1536",
"output_limits":"max-merged-segment-size=1536;max-total-merged-size=3072",
"strategy_conditions":"priority-feature=valid-doc-count#asc;conflict-segment-count=2;conflict-delete-percent=20"
},
"merge_thread_count":4
}
}
}
segment_customize_metrics_updater
●segment_customize_metrics_updater:用於產生各個segment的segment metric,可由使用者自訂產生。indexlib內建存在lifecycle updater,配置該updater後,每個segment會組建組態欄位的min,max值,並根據定義的lifecycle_param配置的打標方式,為每個sgement打標(merge階段,build階段按照預設tag打標)。打標可用於load策略和time_series_merge策略。預設值:[]。