E-MapReduce:Parameter configuration

Parameter

Default value

Description

qe_slow_log_ms

5000

The threshold that is used to determine whether a query is a slow query. If the response time of a query exceeds this threshold, the query is recorded as a slow query. Unit: milliseconds.

Parameter

Default value

Description

catalog_try_lock_timeout_ms

5000

The timeout period to obtain the catalog lock. Unit: milliseconds.

edit_log_roll_num

50000

The maximum number of image log entries that can be written before a log file is created for these log entries.

ignore_unknown_log_id

FALSE

Specifies whether to ignore unknown log IDs. Valid values:

• TRUE: ignores unknown log IDs.

• FALSE: The FE exits if unknown log IDs exist.

Some log IDs may be unknown to BEs of earlier versions if an FE is rolled back to an earlier version.

ignore_meta_check

FALSE

Specifies whether to ignore the metadata gap. Valid values:

• FALSE: ignores the metadata gap.

• TRUE: does not ignore the metadata gap.

max_backend_down_time_second

3600

The maximum amount of time for a BE to rejoin after the BE disconnects to the FE. Unit: seconds.

drop_backend_after_decommission

TRUE

Specifies whether to delete a BE after the BE is decommissioned. Valid values:

• TRUE: deletes the BE.

• FALSE: does not delete the BE.

Parameter

Default value

Description

expr_children_limit

10000

The number of IN predicates that can be involved in a query.

expr_depth_limit

3000

The number of layers that can be nested in a query.

max_allowed_in_element_num_of_delete

10000

The maximum number of elements allowed in the IN predicate of a DELETE statement.

max_layout_length_per_row

2147483647

The maximum length of a single row.

disable_cluster_feature

TRUE

Specifies whether to disable the logical cluster feature. Valid values:

• TRUE: disables this feature.

• FALSE: enables this feature.

enable_materialized_view

TRUE

Specifies whether to enable materialized views. Valid values:

• TRUE: enables materialized views.

• FALSE: disables materialized views.

enable_decimal_v3

TRUE

Specifies whether to support the DECIMAL V3 data type. Valid values:

• TRUE: supports this data type.

• FALSE: does not support this data type.

enable_sql_blacklist

FALSE

Specifies whether to enable the blacklist for SQL queries.

• TRUE: enables the blacklist.

• FALSE: disables the blacklist.

dynamic_partition_check_interval_seconds

600

The interval at which new data is checked. If new data is detected, StarRocks automatically creates partitions for the data. Unit: seconds.

dynamic_partition_enable

TRUE

Specifies whether to enable the dynamic partitioning feature. Valid values:

• TRUE: enables this feature.

• FALSE: disables this feature.

max_partitions_in_one_batch

4096

The maximum number of partitions that can be created at a time.

max_query_retry_time

2

The maximum number of query retries on the FE.

max_create_table_timeout_second

60

The maximum timeout period for creating a table. Unit: seconds.

max_running_rollup_job_num_per_table

1

The maximum number of concurrent rollup jobs for each table.

max_planner_scalar_rewrite_num

10_0000

The maximum number of times that the optimizer can rewrite a ScalarOperator.

statistics_manager_sleep_time_sec

60*10

The interval at which statistics tables are created. Unit: seconds.

statistic_collect_interval_sec

120*60

The interval at which statistics are collected. Unit: seconds.

statistic_update_interval_sec

24 *60* 60

The default interval at which statistics are updated. Unit: seconds.

statistic_sample_collect_rows

200000

The number of rows to collect for sampled collection.

enable_statistic_collect

TRUE

Specifies whether to enable the statistics collection feature. Valid values:

• TRUE: enables this feature.

• FALSE: disables this feature.

enable_local_replica_selection

FALSE

max_distribution_pruner_recursion_depth

100

The maximum recursion depth for partition pruning.

Parameter

Default value

Description

load_straggler_wait_second

300

The maximum loading lag that a BE replica can tolerate. Unit: seconds.

If this value is exceeded, data is cloned from other replicas.

desired_max_waiting_jobs

100

The maximum number of pending jobs.

This limit applies to the total number of jobs of all types, such as table creation, loading, and schema change jobs.

max_running_txn_num_per_db

100

The maximum number of concurrent load jobs.

max_load_timeout_second

259200

The maximum timeout period for a load job. This limit applies to all types of load jobs. Unit: seconds.

min_load_timeout_second

1

The minimum timeout period for a load job. This limit applies to all types of load jobs. Unit: seconds.

load_parallel_instance_num

1

The number of concurrent instances on a single BE.

disable_hadoop_load

FALSE

Specifies whether to disable the feature that is used to load data from Hadoop. Valid values:

• TRUE: disables this feature.

• FALSE: enables this feature.

disable_load_job

FALSE

Specifies whether to accept a load job if the cluster encounters an error. Valid values:

• TRUE: accepts the load job.

• FALSE: does not accept the load job.

db_used_data_quota_update_interval_secs

300

The interval at which the quota for a database is updated. Unit: seconds.

history_job_keep_max_second

604800

The maximum retention period for historical jobs. Unit: seconds.

label_keep_max_num

1000

The maximum number of load jobs that can be retained within a period of time.

You can specify the retention period by using the label_keep_max_second parameter.

label_keep_max_second

259200

The retention period of labels. Unit: seconds.

max_routine_load_job_num

100

The maximum number of Routine Load jobs.

max_routine_load_task_concurrent_num

5

The maximum number of concurrent tasks supported in a Routine Load job.

max_routine_load_task_num_per_be

5

The maximum number of concurrent Routine Load tasks on a BE. The value must be less than or equal to the number specified by the routine_load_thread_pool_size parameter.

max_routine_load_batch_size

524288000

The maximum volume of data that can be loaded by a Routine Load task.

routine_load_task_consume_second

3

The maximum amount of time for which a Routine Load task consumes data. Unit: seconds.

routine_load_task_timeout_second

15

The timeout period for a Routine Load task. Unit: seconds.

max_tolerable_backend_down_num

0

The maximum number of faulty BEs allowed. If this value is exceeded, Routine Load jobs cannot be automatically recovered.

period_of_auto_resume_min

5

The interval at which Routine Load jobs are automatically recovered.

spark_load_default_timeout_second

86400

The timeout period for a Spark Load job. Unit: seconds.

spark_home_default_dir

STARROCKS_HOME_DIR/lib/spark2x

The root directory of a Spark client.

stream_load_default_timeout_second

600

The timeout period for a Stream Load job. Unit: seconds.

max_stream_load_timeout_second

259200

The maximum timeout period for a Stream Load job. Unit: seconds.

insert_load_default_timeout_second

3600

The timeout period for an INSERT INTO statement that is used to load data. Unit: seconds.

broker_load_default_timeout_second

14400

The timeout period for a Broker Load job. Unit: seconds.

min_bytes_per_broker_scanner

67108864

The minimum volume of data that can be processed by a Broker Load instance. The default value specifies 64 MB.

max_broker_concurrency

100

The maximum number of concurrent instances for a Broker Load job.

export_max_bytes_per_be_per_task

268435456

The maximum volume of data that can be exported from a single BE by a single data unload job. The default value specifies 256 MB.

export_running_job_num_limit

5

The maximum number of concurrent data unload jobs.

export_task_default_timeout_second

7200

The timeout period for a data unload job. Unit: seconds. The default value specifies 2 hours.

Parameter

Default value

Description

enable_strict_storage_medium_check

FALSE

Specifies whether the FE checks the free storage space of BEs when a table is created. Valid values:

• TRUE: checks the free storage space of BEs.

• FALSE: does not check the free storage space of BEs.

capacity_used_percent_high_water

0.75

The upper limit of disk usage on a BE.

If the disk usage exceeds 75%, table creation or clone jobs will not be sent to this BE until the disk usage decreases to a value lower than the threshold.

storage_high_watermark_usage_percent

85

The maximum space usage in the storage directory of a BE.

storage_min_left_capacity_bytes

2 *1024* 1024*1024

The minimum remaining space in the storage directory of a BE. The default value specifies 2 GB.

storage_flood_stage_left_capacity_bytes

1 *1024* 1024*1024

The remaining space threshold in the storage directory of a BE.

If the remaining space is less than this value, load or restore jobs are rejected. The default value specifies 1 GB.

storage_flood_stage_usage_percent

95

The space usage threshold in the storage directory of a BE.

If the space usage is greater than this value, load or restore jobs are rejected.

catalog_trash_expire_second

86400

The retention period of metadata in the recycle bin after a table or database is deleted. Unit: seconds. The default value specifies one day.

If the retention period is longer than this value, data cannot be restored.

alter_table_timeout_second

86400

The timeout period for a schema change job. Unit: seconds. The default value specifies one day.

balance_load_disk_safe_threshold

0.5

This parameter is valid only when the disk_and_tablet policy is used.

If the disk usage of all BE disks is less than 50%, the disk usage is considered balanced.

balance_load_score_threshold

0.1

• If the be_load_score policy is used, a BE whose load is 10% lower than the average load is in a low load state, and a BE whose load is 10% higher than the average load is in a high load state.

• If the disk_and_tablet policy is used and the difference between the maximum and minimum BE disk usage is greater than 10%, the disk usage is considered unbalanced, and load rebalancing is implemented among tablets.

disable_balance

true

Specifies whether to disable tablet balancing. Valid values:

• TRUE: enable tablet balancing.

• FALSE: disables tablet balancing.

max_scheduling_tablets

2000

The number of tablets being scheduled.

If the number of tablets being scheduled is greater than this value, the load balancing among tablets is skipped.

max_balancing_tablets

100

The number of tablets being balanced.

If the number of tablets being balanced is greater than this value, the load rebalancing among tablets is skipped.

disable_colocate_balance

FALSE

Specifies whether to disable replica balancing for colocate tables. Valid values:

• TRUE: disables replica balancing for colocate tables.

• FALSE: enables replica balancing for colocate tables.

recover_with_empty_tablet

FALSE

Specifies whether to use an empty tablet replica instead if a tablet replica is lost or corrupted. Valid values:

• TRUE: uses an empty tablet replica instead.

• TRUE: does not use an empty tablet replica instead.

You can use an empty tablet replica instead to ensure that a query can be executed even if a tablet replica is lost or corrupted. However, the result may be incorrect because of the missing data.

min_clone_task_timeout_sec

3*60

The minimum timeout period for cloning a tablet. Unit: seconds. The default value specifies 3 minutes.

max_clone_task_timeout_sec

2 *60* 60

The maximum timeout period for cloning a tablet. Unit: seconds. The default value specifies 2 hours.

tablet_create_timeout_second

1

The timeout period for creating a table. Unit: seconds.

tablet_delete_timeout_second

2

The timeout period for deleting a table. Unit: seconds.

tablet_repair_delay_factor_second

60

The interval at which tablet replicas are repaired. Unit: seconds.

consistency_check_start_time

23

The start time when the FE initiates a replica consistency check. The default value specifies 23:00.

consistency_check_end_time

4

The end time when the FE initiates a replica consistency check. The default value specifies 4:00.

check_consistency_default_timeout_second

600

The timeout period for a replica consistency check. Unit: seconds.

Parameter

Default value

Description

plugin_enable

TRUE

Specifies whether to enable the plug-in feature. Valid values:

• TRUE: enables the plug-in feature.

• FALSE: disables the plug-in feature.

max_small_file_number

100

The maximum number of small files that can be stored.

max_small_file_size_bytes

1024*1024

The maximum size of a small file. The default value specifies 1 MB.

backup_job_default_timeout_ms

86400*1000

The timeout period for a backup job. Unit: seconds. The default value specifies one day.

report_queue_size

100

The maximum number of jobs that can wait in a report queue. The report is about disk, task, and tablet information of BEs.

Parameter

Default value

Description

log_roll_size_mb

1024

The maximum size of a log file. If this value is exceeded, a new log file is created. The default value specifies 1 GB.

sys_log_dir

StarRocksFe.STARROCKS_HOME_DIR/log

The directory in which system log files are stored.

sys_log_level

INFO

The severity levels into which system logs are classified.

Valid values: INFO, WARNING, ERROR, and FATAL.

sys_log_verbose_modules

Empty string

The modules for which system logs are generated.

For example, if this parameter is set to org.apache.starrocks.catalog, system logs are generated only for the catalog module.

sys_log_roll_interval

DAY

The interval at which system log files are rotated.

sys_log_delete_age

7d

The interval at which system log files are deleted.

sys_log_roll_num

10

The maximum number of system log files that can be retained within each retention period specified by the sys_log_roll_interval parameter.

audit_log_dir

starrocksFe.STARROCKS_HOME_DIR/log

The directory in which audit log files are stored.

audit_log_roll_num

90

The maximum number of audit log files that can be retained.

audit_log_modules

"slow_query", "query"

The modules for which audit logs are generated. By default, audit logs are generated for the slow_query and query modules.

audit_log_roll_interval

DAY

The interval at which audit log files are rotated. Valid values: DAY and HOUR.

audit_log_delete_age

30d

The interval at which audit log files are deleted.

dump_log_dir

STARROCKS_HOME_DIR/log

The directory in which dump log files are stored.

dump_log_modules

"query"

The modules for which dump logs are generated. By default, dump logs are generated for the query module.

dump_log_roll_interval

DAY

The interval at which dump log files are rotated.

If this parameter is set to DAY, a suffix in the yyyyMMdd format is added to the names of dump log files. If this parameter is set to HOUR, a suffix in the yyyyMMddHH format is added to the names of dump log files.

dump_log_roll_num

90

The maximum number of dump log files that can be retained within each retention period specified by the dump_log_roll_interval parameter.

dump_log_delete_age

30d

The interval at which dump log files are deleted.

Parameter

Default value

Description

frontend_address

0.0.0.0

The IP address of the FE.

priority_networks

Empty string

The CIDR block of the BE, such as 10.10.**.**/24. This parameter can be used for servers that have multiple IP addresses. You can configure this parameter to specify the network that is preferentially used.

http_port

8030

The port of the HTTP server.

http_backlog_num

1024

The length of the backlog queue that is held by the HTTP server.

cluster_name

StarRocks Cluster

The name of the StarRocks cluster, which is displayed for Title on the web page.

rpc_port

9020

The port of the Thrift server on the FE.

thrift_backlog_num

1024

The length of the backlog queue that is held by the Thrift server.

thrift_server_type

THREAD_POOL

The service model that is used by the Thrift server on the FE. Valid values: SIMPLE, THREADED, and THREAD_POOL.

thrift_server_max_worker_threads

4096

The maximum number of worker threads that are supported by the Thrift server.

thrift_client_timeout_ms

0

The timeout period for requests from clients.

The default value 0 specifies that requests from clients never time out.

brpc_idle_wait_max_time

10000

The maximum amount of time for which bRPCs wait in the idle state. Unit: milliseconds. The default value specifies 10 seconds.

query_port

9030

The port of the MySQL server on the FE.

mysql_service_nio_enabled

FALSE

Specifies whether to enable asynchronous I/O for the FE. Valid values:

• TRUE: enables asynchronous I/O for the FE.

• FALSE: disables asynchronous I/O for the FE.

mysql_service_io_threads_num

4

The maximum number of threads that can be run by the MySQL server on the FE to process I/O events.

mysql_nio_backlog_num

1024

The length of the backlog queue that is held by the MySQL server.

max_mysql_service_task_threads_num

4096

The maximum number of threads that can be run by the MySQL server to process tasks.

max_connection_scheduler_threads_num

4096

The maximum number of threads that are supported by the connection scheduler.

qe_max_connection

1024

The maximum number of connections that can be established by all users to the FE.

check_java_version

TRUE

Specifies whether to check version compatibility between the executed and compiled Java programs. Value values:

• TRUE: check version compatibility between the executed and compiled Java programs.

• FALSE: does not check version compatibility between the executed and compiled Java programs.

Parameter

Default value

Description

meta_dir

StarRocksFe.STARROCKS_HOME_DIR/meta

The directory in which metadata is retained.

heartbeat_mgr_threads_num

8

The number of threads that can be run by Heartbeat Manager to run heartbeat tasks.

heartbeat_mgr_blocking_queue_size

1024

The size of the blocking queue that stores heartbeat tasks run by Heartbeat Manager.

metadata_failure_recovery

FALSE

Specifies whether to forcibly reset the metadata of the FE.

edit_log_port

9010

The port that is used for communication among the master, follower, and observer FEs in the StarRocks cluster.

edit_log_type

BDB

The type of edit log that can be generated. Set the value to BDB.

bdbje_heartbeat_timeout_second

30

The timeout period for heartbeats in the BDB JE-based FE. Unit: seconds.

bdbje_lock_timeout_second

1

The timeout period for a lock on the BDB JE-based FE. Unit: seconds.

max_bdbje_clock_delta_ms

5000

The maximum clock offset that is allowed between the master FE and the follower or observer FEs in the StarRocks cluster. Unit: milliseconds.

txn_rollback_limit

100

The maximum number of transactions that can be rolled back.

bdbje_replica_ack_timeout_second

10

The maximum amount of time for which the master FE can wait for ACK messages from a specified number of follower FEs.

master_sync_policy

SYNC

The policy based on which the master FE flushes logs to disk.

replica_sync_policy

SYNC

The policy based on which the follower FE replica flushes logs to disk.

meta_delay_toleration_second

300

The maximum duration by which the metadata on the follower and observer FEs can lag behind that on the master FE. Unit: seconds.

cluster_id

-1

The ID of the cluster to which the FE belongs. FEs or BEs that have the same cluster ID belong to the same cluster.

The default value -1 specifies that a random cluster ID is generated when the master FE of the cluster is started for the first time.

Parameter

Default value

Description

disable_colocate_join

FALSE

Specifies whether to enable colocate joins. Valid values:

• FALSE: disables colocate joins.

• TRUE: enables colocate joins.

enable_udf

FALSE

Specifies whether to enable user defined functions (UDFs). Valid values:

• FALSE: disable UDFs.

• TRUE: enables UDFs.

publish_version_interval_ms

10

The interval at which release validation tasks are issued.

statistic_cache_columns

10_0000

The number of rows that can be cached for the statistics table.

Parameter

Default value

Description

async_load_task_pool_size

10

The thread pool size of load jobs.

load_checker_interval_second

5

The polling interval for load jobs. Unit: seconds.

transaction_clean_interval_second

30

The interval at which finished transactions are cleaned up. Unit: seconds.

label_clean_interval_second

14400

The interval at which labels are cleaned up. Unit: seconds.

spark_dpp_version

1.0.0

The version of Spark Dynamic Partition Pruning (DPP).

spark_resource_path

Empty string

The root directory of the Spark dependency package.

spark_launcher_log_dir

sys_log_dir/spark_launcher_log

The directory in which Spark log files are stored.

yarn_client_path

STARROCKS_HOME_DIR/lib/yarn-client/hadoop/bin/yarn

The root directory of the YARN client package.

yarn_config_dir

STARROCKS_HOME_DIR/lib/yarn-config

The directory in which the YARN configuration files are stored.

export_checker_interval_second

5

The polling interval for unload threads. Unit: seconds.

export_task_pool_size

5

The thread pool size of unload jobs.

export_checker_interval_second

5

The interval at which unload jobs are scheduled. Unit: seconds.

Parameter

Default value

Description

storage_cooldown_second

2592000

The interval for storage media migration. Unit: seconds. The default value specifies 30 days.

default_storage_medium

HDD

The default storage media. Valid values: HDD and SSD.

The default storage media is used if no storage media is specified when you create a table or partition.

schedule_slot_num_per_path

2

The number of tablet-related jobs that can concurrently run in the storage directory of a BE.

tablet_balancer_strategy

disk_and_tablet

The policy based on which load balancing is implemented among tablets. Valid values: disk_and_tablet and be_load_score.

tablet_stat_update_interval_second

300

The interval at which the FE retrieves tablet statistics from each BE. Unit: seconds. The default value specifies 5 minutes.

Parameter

Default value

Description

plugin_dir

STARROCKS_HOME_DIR/plugins

The directory in which the plug-in installation packages are stored.

small_file_dir

STARROCKS_HOME_DIR/small_files

The root directory of small files.

max_agent_task_threads_num

4096

The maximum number of threads in the thread pool of agent jobs.

authentication_ldap_simple_bind_base_dn

""

The base DN that is used to search for the authentication information about users.

authentication_ldap_simple_bind_root_dn

""

The administrator DN that is used to search for the authentication information about users.

authentication_ldap_simple_bind_root_pwd

""

The password of the administrator that is used to search for the authentication information about users.

authentication_ldap_simple_server_host

""

The host on which the LDAP server runs.

authentication_ldap_simple_server_port

389

The port of the LDAP server.

authentication_ldap_simple_user_search_attr

uid

The name of the attribute that identifies users in LDAP objects.

tmp_dir

starrocksFe.STARROCKS_HOME_DIR/temp_ddir

The directory in which temporary files are stored, such as files generated during backup and restore procedures.

locale

zh_CN.UTF-8

The character set.

hive_meta_load_concurrency

4

The maximum number of concurrent threads that are supported for Hive metadata.

hive_meta_cache_refresh_interval_s

4096

The interval at which the cached metadata of Hive external tables is updated. Unit: seconds.

hive_meta_cache_ttl_s

3600 *2

The amount of time after which the cached metadata of Hive external tables expires. Unit: seconds. The default value specifies 2 hours.

hive_meta_store_timeout_s

3600 *24

The timeout period for a connection to a Hive metastore. Unit: seconds. The default value specifies 24 hours.

es_state_sync_interval_second

10

The interval at which the FE obtains Elasticsearch indexes. Unit: seconds.

enable_auth_check

TRUE

Specifies whether to enable the authentication check feature. Valid values:

• TRUE: enables this feature.

• FALSE: disables this feature.

enable_metric_calculator

TRUE

Specifies whether to enable the feature that is used to periodically collect metrics. Valid values:

• TRUE: enables this feature.

• FALSE: disables this feature.

Parameter

Default value

Description

be_port

9060

The port of the Thrift server on the BE. This port is used to receive requests from FEs.

brpc_port

8060

The bRPC port that is used to view the network statistics of bRPCs.

brpc_num_threads

-1

The number of bthreads of a bRPC. The default value -1 specifies the same number as CPU cores.

priority_networks

Empty string

The CIDR block of the BE, such as 10.10.**.**/24. This parameter can be used for servers that have multiple IP addresses. You can configure this parameter to specify the network that is preferentially used.

heartbeat_service_port

9050

The port of the heartbeat service on the BE. This port is used to receive heartbeats from FEs.

heartbeat_service_thread_count

1

The number of threads for the heartbeat service.

create_tablet_worker_count

3

The number of threads that are used to create a tablet.

drop_tablet_worker_count

3

The number of threads that are used to delete a tablet.

push_worker_count_normal_priority

3

The number of threads that are used to process a load job with the NORMAL priority.

push_worker_count_high_priority

3

The number of threads that are used to process a load job with the HIGH priority.

publish_version_worker_count

2

The number of threads that are used to publish a version.

clear_transaction_task_worker_count

1

The number of threads that are used to clear transactions.

alter_tablet_worker_count

3

The number of threads that are used to change a schema.

clone_worker_count

3

The number of threads that are used for cloning.

storage_medium_migrate_count

1

The number of threads that are used for storage media migration. For example, hot data can be migrated from an SSD to an SATA disk by using these threads.

check_consistency_worker_count

1

The number of threads that are used to check the consistency of tablets.

report_task_interval_seconds

10

The interval at which a job is reported. Unit: seconds.

A job can be initiated to create a table, delete a table, load data, or change the schema of a table.

report_disk_state_interval_seconds

60

The interval at which disk status is reported. Unit: seconds.

The disk status includes the status and data volume of each disk.

report_tablet_interval_seconds

60

The interval at which tablet data are reported. Unit: seconds.

The data includes the latest versions of all tablets.

alter_tablet_timeout_seconds

86400

The timeout period for a schema change job. Unit: seconds.

sys_log_dir

${DORIS_HOME}/log

The directory in which system logs are stored.

Levels of systems logs include INFO, WARNING, ERROR, and FATAL.

user_function_dir

${DORIS_HOME}/lib/udf

The directory in which UDFs are stored.

sys_log_level

INFO

The severity levels into which system log entries are classified.

Valid values: INFO, WARNING, ERROR, and FATAL.

sys_log_roll_mode

SIZE-MB-1024

The maximum size of a log file. If this value is exceeded, a new log file is created. The default value specifies 1 GB.

sys_log_roll_num

10

The maximum number of system log files that can be retained.

sys_log_verbose_modules

Empty string

The modules for which system logs are generated. For example, if this parameter is set to OLAP, system logs are generated only for the OLAP module.

sys_log_verbose_level

10

The level of the logs to be displayed. This parameter is used to control the output of logs that start with VLOG in codes.

log_buffer_level

Empty string

The policy based on which logs are flushed. By default, logs are buffered in memory.

num_threads_per_core

3

The number of threads that are started in each CPU core.

compress_rowbatches

TRUE

Specifies whether to compress the row batches in RPCs between BEs. This parameter is used for the data transmission between query layers.

serialize_batch

FALSE

Specifies whether to serialize the row batches in RPCs between BEs. This parameter item is used for the data transmission between query layers.

status_report_interval

5

The interval at which a profile is reported. Unit: seconds. This parameter is used to collect query statistics for FEs.

doris_scanner_thread_pool_thread_num

48

The number of threads in the thread pool that is used by the storage engine for concurrent scan tasks.

doris_scanner_thread_pool_queue_size

102400

The maximum number of tasks that the storage engine supports.

doris_scan_range_row_count

524288

The granularity in which the storage engine splits the range of a query task.

doris_scanner_queue_size

1024

The maximum number of scan tasks that the storage engine supports.

doris_scanner_row_num

16384

The maximum number of data rows that can be returned by each scan thread at a time.

doris_max_scan_key_num

1024

The maximum number of scan keys that can be split.

column_dictionary_key_ratio_threshold

0

The value ratio for the STRING type. If the ratio is less than this value, the dictionary compression algorithm is used.

column_dictionary_key_size_threshold

0

The size of the dictionary compression column. If the size is smaller than this value, the dictionary compression algorithm is used.

memory_limitation_per_thread_for_schema_change

2

The maximum memory for a schema change job.

file_descriptor_cache_clean_interval

3600

The interval at which the cached file descriptors are cleared. Unit: seconds. This parameter is used to clean up file handles that have not been used for a long time.

disk_stat_monitor_interval

5

The interval at which the disk status is checked. Unit: seconds.

unused_rowset_monitor_interval

30

The interval at which expired rowsets are cleared. Unit: seconds.

storage_root_path

Empty string

The directory in which data is stored.

max_tablet_num_per_shard

1024

The maximum number of tablets in each shard. This parameter is used to restrict the number of tablet child directories within each storage directory.

pending_data_expire_time_sec

1800

The maximum retention period of the pending data in the storage engine. Unit: seconds.

inc_rowset_expired_sec

1800

The retention period of loaded data in the storage engine in the incremental cloning scenario. Unit: seconds.

max_garbage_sweep_interval

3600

The maximum interval for garbage collection on disks. Unit: seconds.

min_garbage_sweep_interval

180

The minimum interval for garbage collection on disks. Unit: seconds.

snapshot_expire_time_sec

172800

The interval at which snapshots are deleted. Unit: seconds. The default value specifies 48 hours.

trash_file_expire_time_sec

259200

The interval at which the files in the recycle bin are deleted. Unit: seconds. The default value specifies 72 hours.

file_descriptor_cache_capacity

16384

The number of file descriptors that can be cached.

min_file_descriptor_number

60000

The minimum number of file descriptors in the BE process.

index_stream_cache_capacity

10737418240

The cache capacity for the statistical information of BloomFilter, Min, and Max.

storage_page_cache_limit

0

The capacity of the page cache.

disable_storage_page_cache

TRUE

Specifies whether to disable the page cache.

• TRUE: disables the page cache.

• FALSE: enables the page cache.

base_compaction_check_interval_seconds

60

The interval of thread polling for a base compaction. Unit: seconds.

base_compaction_num_threads_per_disk

1

The number of threads used for base compaction on each disk.

base_cumulative_delta_ratio

0.3

The ratio of cumulative file size to base file size. This is one of the conditions that trigger a base compaction.

base_compaction_interval_seconds_since_last_operation

86400

The interval since the last base compaction. This is one of the conditions that trigger a base compaction.

cumulative_compaction_check_interval_seconds

1

The interval of thread polling for a cumulative compaction. Unit: seconds.

min_cumulative_compaction_num_singleton_deltas

5

The minimum number of singleton files. This is one of the conditions that trigger a cumulative compaction.

max_cumulative_compaction_num_singleton_deltas

1000

The maximum number of singleton files. This is one of the conditions that trigger a cumulative compaction.

cumulative_compaction_num_threads_per_disk

1

The number of threads for cumulative compaction on each disk.

min_compaction_failure_interval_sec

120

The minimum interval that a tablet compaction can be scheduled since the last compaction failure. Unit: seconds.

max_compaction_concurrency

-1

The maximum concurrency of base compactions and cumulative compactions.

The default value -1 specifies that no limit is imposed on the concurrency.

webserver_port

8040

The port of the HTTP server.

webserver_num_workers

48

The number of threads that are used by the HTTP server.

periodic_counter_update_period_ms

500

The interval at which the Counter statistics are collected. Unit: milliseconds.

load_data_reserve_hours

4

The retention period of files generated by small-scale loadings. Unit: hours.

load_error_log_reserve_hours

48

The retention period of data loading logs. Unit: hours.

number_tablet_writer_threads

16

The number of threads used for Stream Load.

streaming_load_max_mb

10240

The maximum size of a file that can be loaded into StarRocks in a stream.

streaming_load_rpc_max_alive_time_sec

1200

The RPC timeout for Stream Load.

fragment_pool_thread_num

64

The number of threads that are used for queries. By default, 64 threads are started. Threads are dynamically created for subsequent queries.

fragment_pool_queue_size

2048

The maximum number of queries that can be processed on a node.

enable_partitioned_aggregation

TRUE

Specifies whether to enable partition aggregation. Valid values:

• TRUE: enables partition aggregation.

• FALSE: disables partition aggregation.

enable_token_check

TRUE

Specifies whether to enable the token check.

• TRUE: enable the token check.

• FALSE: disables the token check.

load_process_max_memory_limit_bytes

107374182400

The maximum size of memory that can be used by all load threads on a node. The default value specifies 100 GB.

load_process_max_memory_limit_percent

30

The maximum percentage of memory that can be used by all load threads on a node.

sync_tablet_meta

FALSE

Specifies whether to enable the synchronization of tablet metadata.

thrift_rpc_timeout_ms

5000

The timeout period for a thrift RPC. Unit: milliseconds.

txn_commit_rpc_timeout_ms

10000

The timeout period for a transaction commit RPC. Unit: milliseconds.

routine_load_thread_pool_size

10

The thread pool size of Routine Load.

tablet_meta_checkpoint_min_new_rowsets_num

10

The minimum number of rowsets to create since the last tablet meta checkpoint.

tablet_meta_checkpoint_min_interval_secs

600

The interval of thread polling for generating a tablet meta checkpoint. Unit: seconds.

brpc_max_body_size

209715200

The maximum body size of a BRPC. The default value specifies 200 MB.

max_runnings_transactions

2000

The maximum number of transactions that the storage engine supports.

tablet_map_shard_size

32

The size of a tablet map shard.

enable_bitmap_union_disk_format_with_set

FALSE

Specifies whether to enable the new storage format of the BITMAP type. This type can improve the performance of bitmap_union.