Updates a Logtail pipeline configuration.

Operation description

The UK (London) region is supported. Supported regions are constantly updated.

Debugging

You can run this interface directly in OpenAPI Explorer, saving you the trouble of calculating signatures. After running successfully, OpenAPI Explorer can automatically generate SDK code samples.

Debug

Authorization information

The following table shows the authorization information corresponding to the API. The authorization information can be used in the Action policy element to grant a RAM user or RAM role the permissions to call this API operation. Description:

Operation: the value that you can use in the Action element to specify the operation on a resource.
Access level: the access level of each operation. The levels are read, write, and list.
Resource type: the type of the resource on which you can authorize the RAM user or the RAM role to perform the operation. Take note of the following items:
- The required resource types are displayed in bold characters.
- If the permissions cannot be granted at the resource level, All Resources is used in the Resource type column of the operation.
Condition Key: the condition key that is defined by the cloud service.
Associated operation: other operations that the RAM user or the RAM role must have permissions to perform to complete the operation. To complete the operation, the RAM user or the RAM role must have the permissions to perform the associated operations.

Operation	Access level	Resource type	Condition key	Associated operation
log:UpdateLogtailPipelineConfig	update	All Resources *	log:TLSVersion	none

Request syntax

PUT /pipelineconfigs/{configName} HTTP/1.1

Request parameters

Parameter	Type	Required	Description	Example
project	string	Yes	The name of the project.	test-project
configName	string	Yes	The name of the Logtail pipeline configuration.	test-config
body	object	No	The content of the Logtail pipeline configuration.
configName	string	Yes	The name of the configuration. Note The value of this parameter must be the same as the value of configName in the outer layer.	test-config
logSample	string	No	The sample log. You can specify multiple sample logs.	2022-06-14 11:13:29.796 \| DEBUG \| __main__:<module>:1 - hello world
global	object	No	The global settings. **
inputs	array<object>	Yes	The input plug-ins. Note You can configure only one input plug-in.
	object	No	The input plug-in. Note For more information about the parameters of the file input plug-in, see the "File input plug-in" section of this topic. For more information about the parameters of other input plug-ins, see the related documentation .	{ "Type": "input_file", "FilePaths": ["/var/log/*.log"] }
processors	array<object>	No	The processing plug-ins. Note Logtail supports native plug-ins and extended plug-ins for data processing. For more information, see Logtail plug-ins overview. Note You can use native plug-ins only to collect text logs. You cannot add native plug-ins and extended plug-ins at the same time. When you add native plug-ins, take note of the following items: You must add one of the following Logtail plug-ins for data processing as the first plug-in: Data Parsing (Regex Mode), Data Parsing (Delimiter Mode), Data Parsing (JSON Mode), Data Parsing (NGINX Mode), Data Parsing (Apache Mode), and Data Parsing (IIS Mode). After you add the first plug-in, you can add one Time Parsing plug-in, one Data Filtering plug-in, and multiple Data Masking plug-ins.
	object	No	The processing plug-in. Note For more information about the parameters of native plug-ins, see the sections of the plug-ins in this topic. For more information about the parameters of extended plug-ins, see the related documentation .	{ "Type": "processor_parse_json_native", "SourceKey": "content" }
aggregators	array<object>	No	The aggregation plug-ins. Note This parameter takes effect only when extended plug-ins are used. You can use only one aggregation plug-in.
	object	No	The aggregation plug-in.
flushers	array<object>	Yes	The output plug-ins. Note You can use only one Simple Log Service output plug-in.
	object	No	The output plug-in. **	{ "Type": "flusher_sls", "Logstore": "test" }

Global settings

Parameter	Type	Required	Default value	Example	Description
TopicType	string	No	Empty	filepath	The type of the topic. Valid values: filepath: The information that is extracted from the log file path is used as the topic of collected logs. This parameter takes effect only when the type of the input plug-in is set to input_file. machine_group_topic: The topic of the machine group to which the configuration is applied is used as the topic of collected logs. custom: You can specify a custom topic for collected logs. For more information, see Log topics.
TopicFormat	string	No (This parameter is required only when TopicType is set to filepath or custom.)	/	/var/log/(.*).log	The format of the topic.
EnableTimestampNanosecond	bool	No	false	false	Specifies whether to extract timestamps that are accurate to the nanosecond.

Input plug-ins

File input plug-in

Parameter	Type	Required	Default value	Example	Description
Type	string	Yes	/	input_file	The type of the plug-in. Set the value to input_file.
FilePaths	[string]	Yes	/	["/var/log/*.log"]	The paths to the files from which you want to collect logs. You can specify only one path. You can use wildcard characters (* and ) to specify a log file path. You can use the wildcard character () only once in a path, and you must place it before the file name.
MaxDirSearchDepth	uint	No	0	0	The number of levels of subdirectories that can be matched for the wildcard character () in the log file path. This parameter takes effect only when the wildcard character () exists in the log file path. Valid values: 0 to 1000.
ExcludeFilePaths	[string]	No	Empty	["/home/admin/*.log"]	The blacklist of file paths. A path must be an absolute path. You can use the wildcard character (*).
ExcludeFiles	[string]	No	Empty	["app*.log", "password"]	The blacklist of file names. You can use the wildcard character (*).
ExcludeDirs	[string]	No	Empty	["/home/admin/dir1", "/home/admin/dir2*"]	The blacklist of directories. A directory must be an absolute path. You can use the wildcard character (*).
FileEncoding	string	No	utf8	utf8	The encoding format of the files. Valid values: utf8 and gbk.
TailSizeKB	uint	No	1024	1024	The size of data that Logtail can collect from a log file the first time Logtail collects logs from the file. If the file size is less than this value, Logtail collects data from the beginning of the file. Valid values: 0 to 10485760. Unit: KB.
Multiline	object	No	Empty	/	The multi-line aggregation options.
Multiline.Mode	string	No	custom	custom	The multi-line aggregation mode. Valid values: custom and JSON.
Multiline.StartPattern	string	This parameter is required when Multiline.Mode is set to custom.	Empty	\d+-\d+-\d+.*	The regular expression that is used to match the beginning of the first line of a log.
EnableContainerDiscovery	bool	No	false	true	Specifies whether to enable the container discovery feature. This parameter takes effect only when Logtail runs in DaemonSet mode and the log file path is an in-container path.
ContainerFilters	object	No	Empty	/	The container filter options. Multiple options are evaluated by using a logical AND. This parameter takes effect only when EnableContainerDiscovery is set to true.
ContainerFilters.K8sNamespaceRegex	string	No	Empty	default	The namespace of the pods to which the required containers belong. The containers refer to Kubernetes containers from which you want to collect data. If you do not specify this parameter, Logtail collects data from all containers. Regular expression matching is supported.
ContainerFilters.K8sPodRegex	string	No	Empty	test-pod	The name of the pods to which the required containers belong. The containers refer to Kubernetes containers from which you want to collect data. If you do not specify this parameter, Logtail collects data from all containers. Regular expression matching is supported.
ContainerFilters.IncludeK8sLabel	map	No	Empty	/	The labels of the pods to which the required containers belong. The containers refer to Kubernetes containers from which you want to collect data. The labels are also referred to as conditions. Multiple conditions are evaluated by using a logical OR. If you do not specify this parameter, Logtail collects data from all containers. Regular expression matching is supported. To create a map, specify the name of a pod label as the key of the map, and specify the value of the pod label as the value of the map. If the value of the map is empty, the pods whose labels contain the key are matched. If the value of the map is not empty, take note of the following information: If the value starts with a caret (`^`) and ends with a dollar sign (`$`), the pods that meet the following conditions are matched: The name of a pod label contains the key of the map, and the value of the pod label matches the value of the map based on a regular expression. In other cases, the pods whose labels exactly match the key and value of the map are matched.
ContainerFilters.ExcludeK8sLabel	map	No	Empty	/	The labels of the pods to which the required containers do not belong. The containers refer to Kubernetes containers from which you want to collect data. Multiple conditions are evaluated by using a logical OR. If you do not specify this parameter, Logtail collects data from all containers. Regular expression matching is supported. To create a map, specify the name of a pod label as the key of the map, and specify the value of the pod label as the value of the map. If the value of the map is empty, the pods whose labels contain the key are matched. If the value of the map is not empty, take note of the following information: If the value starts with a caret (`^`) and ends with a dollar sign (`$`), the pods that meet the following conditions are matched: The name of a pod label contains the key of the map, and the value of the pod label matches the value of the map based on a regular expression. In other cases, the pods whose labels exactly match the key and value of the map are matched.
ContainerFilters.K8sContainerRegex	string	No	Empty	test-container	The name of the required containers. The containers refer to Kubernetes containers from which you want to collect data. If you do not specify this parameter, Logtail collects data from all containers. Regular expression matching is supported.
ContainerFilters.IncludeEnv	map	No	Empty	/	The environment variables of the containers from which you want to collect data. The environment variables are also referred to as conditions. Multiple conditions are evaluated by using a logical OR. If you do not specify this parameter, Logtail collects data from all containers. Regular expression matching is supported. To create a map, specify the name of an environment variable as the key of the map, and specify the value of the environment variable as the value of the map. If the value of the map is empty, the containers whose environment variables contain the key are matched. If the value of the map is not empty, take note of the following information: If the value starts with a caret (`^`) and ends with a dollar sign (`$`), the containers that meet the following conditions are matched: The name of an environment variable is the key of the map, and the value of the environment variable matches the value of the map based on a regular expression. In other cases, the containers whose environment variables exactly match the key and value of the map are matched.
ContainerFilters.ExcludeEnv	map	No	Empty	/	The environment variables of the containers from which you do not want to collect data. Multiple conditions are evaluated by using a logical OR. If you do not specify this parameter, Logtail collects data from all containers. Regular expression matching is supported. To create a map, specify the name of an environment variable as the key of the map, and specify the value of the environment variable as the value of the map. If the value of the map is empty, the containers whose environment variables contain the key are matched. If the value of the map is not empty, take note of the following information: If the value starts with a caret (`^`) and ends with a dollar sign (`$`), the containers that meet the following conditions are matched: The name of an environment variable is the key of the map, and the value of the environment variable matches the value of the map based on a regular expression. In other cases, the containers whose environment variables exactly match the key and value of the map are matched.
ContainerFilters.IncludeContainerLabel	map	No	Empty	/	The labels of the containers from which you want to collect data. Multiple conditions are evaluated by using a logical OR. If you do not specify this parameter, this parameter is left empty by default, which indicates that Logtail collects data from all containers. Regular expression matching is supported. To create a map, specify the name of a container label as the key of the map, and specify the value of the container label as the value of the map. If the value of the map is empty, the containers whose labels contain the key are matched. If the value of the map is not empty, take note of the following information: If the value starts with a caret (`^`) and ends with a dollar sign (`$`), the containers that meet the following conditions are matched: The name of a container label contains the key of the map, and the value of the container label matches the value of the map based on a regular expression. In other cases, the containers whose labels exactly match the key and value of the map are matched.
ContainerFilters.ExcludeContainerLabel	map	No	Empty	/	The labels.of the containers from which you do not want to collect data. Multiple conditions are evaluated by using a logical OR. If you do not specify this parameter, this parameter is left empty by default, which indicates that Logtail collects data from all containers. Regular expression matching is supported. To create a map, specify the name of a container label as the key of the map, and specify the value of the container label as the value of the map. If the value of the map is empty, the containers whose labels contain the key are matched. If the value of the map is not empty, take note of the following information: If the value starts with a caret (`^`) and ends with a dollar sign (`$`), the containers that meet the following conditions are matched: The name of a container label contains the key of the map, and the value of the container label matches the value of the map based on a regular expression. In other cases, the containers whose labels exactly match the key and value of the map are matched.
ExternalK8sLabelTag	map	No	Empty	/	The tags that you want to add to collected logs. The tags are related to the labels of the pods to which the required containers belong. The containers refer to Kubernetes containers from which you want to collect data. To create a map, specify the name of a pod label as the key of the map, and specify the name of the required tag as the value of the map. For example, a map of `app: k8s\_label\_app` is created. If a pod contains the `app=serviceA` label, the system adds the related information to collected logs as a tag. In this example, the __tag__:k8s_label_app: serviceA field is added to collected logs. If a pod does not contain the `app` label, the __tag__:k8s_label_app: field is added.
ExternalEnvTag	map	No	Empty	/	The tags that you want to add to collected logs. The tags are related to the environment variables of the required containers. The containers refer to Kubernetes containers from which you want to collect data. To create a map, specify the name of an environment variable as the key of the map, and specify the name of the required tag as the value of the map. For example, a map of `VERSION: env\_version` is created. If a container contains the `VERSION=v1.0.0` environment variable, the system adds the related information to collected logs as a tag. In this example, the __tag__:env_version: v1.0.0 field is added to collected logs. If a container does not contain the `VERSION` environment variable, the __tag__:env_version: field is added.
CollectingContainersMeta	bool	No	false	true	Specifies whether to enable the container metadata preview feature.
AppendingLogPositionMeta	bool	No	false	false	Specifies whether to add the metadata of a file to a log. The log is included in the file. The metadata includes the __tag__:__inode__ field and the __file_offset__ field.
AllowingIncludedByMultiConfigs	bool	No	false	false	Specifies whether to allow Logtail to collect data from a file that is matched by a different Logtail configuration.

Native plug-ins for data processing

Parsing in regex mode

Parameter	Type	Required	Default value	Example	Description
Type	string	Yes	/	processor_parse_regex_native	The type of the plug-in. Set the value to processor_parse_regex_native.
SourceKey	string	Yes	/	content	The name of the original field.
Regex	string	Yes	/	(\d+-\d+-\d+)\s+(.*)	The regular expression.
Keys	[string]	Yes	/	["time", "msg"]	The fields that you want to extract.
KeepingSourceWhenParseFail	bool	No	false	false	Specifies whether to retain the original field if the parsing fails.
KeepingSourceWhenParseSucceed	bool	No	false	false	Specifies whether to retain the original field if the parsing succeeds.
RenamedSourceKey	string	No	Empty	key	The name of the field that is used to store the original field when the original field is retained. If you do not specify this parameter, the name of the original field is used.

Parsing in JSON mode

Parameter	Type	Required	Default value	Example	Description
Type	string	Yes	/	processor_parse_json_native	The type of the plug-in. Set the value to processor_parse_json_native.
SourceKey	string	Yes	/	content	The name of the original field.
KeepingSourceWhenParseFail	bool	No	false	false	Specifies whether to retain the original field if the parsing fails.
KeepingSourceWhenParseSucceed	bool	No	false	false	Specifies whether to retain the original field if the parsing succeeds.
RenamedSourceKey	string	No	Empty	key	The name of the field that is used to store the original field when the original field is retained. If you do not specify this parameter, the name of the original field is used.

Parsing in delimiter mode

Parameter	Type	Required	Default value	Example	Description
Type	string	Yes	/	processor_parse_delimiter_native	The type of the plug-in. Set the value to processor_parse_delimiter_native.
SourceKey	string	Yes	/	content	The name of the original field.
Separator	string	Yes	/	,	The delimiter.
Quote	string	No	"	"	The quote.
Keys	[string]	Yes	/	["time", "msg"]	The fields that you want to extract.
AllowingShortenedFields	bool	No	true	true	Specifies whether to upload logs whose extracted fields are less than the fields specified in Keys. If you set the parameter to false, the system considers that the parsing fails.
OverflowedFieldsTreatment	string	No	extend	extend	The method that is used to process logs whose extracted fields are greater than the fields specified in Keys. Valid values: extend: retains excess fields and adds each excess field as a separate field to the collected log in the column$i format, where $i represents the sequence number of an excess field. The sequence number starts from 0. keep: retains excess fields and adds the excess fields as a whole field to the collected log in the column0_ format. discard: discards excess fields.
KeepingSourceWhenParseFail	bool	No	false	false	Specifies whether to retain the original field if the parsing fails.
KeepingSourceWhenParseSucceed	bool	No	false	false	Specifies whether to retain the original field if the parsing succeeds.
RenamedSourceKey	string	No	Empty	key	The name of the field that is used to store the original field when the original field is retained. If you do not specify this parameter, the name of the original field is used.

Parsing in Apsara mode

Parameter	Type	Required	Default value	Example	Description
Type	string	Yes	/	processor_parse_apsara_native	The type of the plug-in. Set the value to processor_parse_apsara_native.
SourceKey	string	Yes	/	content	The name of the original field.
Timezone	string	No	Empty	GMT+08:00	The time zone of the log time. Format: GMT+HH:MM or GMT-HH:MM.
KeepingSourceWhenParseFail	bool	No	false	false	Specifies whether to retain the original field if the parsing fails.
KeepingSourceWhenParseSucceed	bool	No	false	false	Specifies whether to retain the original field if the parsing succeeds.
RenamedSourceKey	string	No	Empty	key	The name of the field that is used to store the original field when the original field is retained. If you do not specify this parameter, the name of the original field is used.

Time parsing plug-in

Parameter	Type	Required	Default value	Example	Description
Type	string	Yes	/	processor_parse_timestamp_native	The type of the plug-in. Set the value to processor_parse_timestamp_native.
SourceKey	string	Yes	/	content	The name of the original field.
SourceFormat	string	Yes	/	%Y/%m/%d %H:%M:%S	The format of the log time. For more information, see Time formats.
SourceTimezone	string	No	Empty	GMT+08:00	The time zone of the log time. Format: GMT+HH:MM or GMT-HH:MM.

Data filtering plug-in

Parameter	Type	Required	Example	Default value	Description
Type	string	Yes	processor_filter_regex_native	/	The type of the plug-in. Set the value to processor_filter_regex_native.
Include	map	Yes	/	/	The whitelist of log fields. To create a map, specify the name of a field as the key of the map, and specify a regular expression as the value of the map. The regular expression specifies the conditions that is used to match the content of the field. Multiple conditions are evaluated by using a logical AND. The log is collected only when all conditions are met.

Data masking plug-in

Parameter	Type	Required	Default value	Example	Description
Type	string	Yes	/	processor_desensitize_native	The type of the plug-in. Set the value to processor_desensitize_native.
SourceKey	string	Yes	/	content	The name of the original field.
Method	string	Yes	/	const	The method that is used to mask sensitive content. Valid values: const and md5. If you set the parameter to const, the system replaces the sensitive content with constants. If you set the parameter to md5, the system replaces the sensitive content with the MD5 value of the sensitive content.
ReplacingString	string	No (This parameter is required when Method is set to const.)	/	******	The constant string that is used to replace the sensitive content.
ContentPatternBeforeReplacedString	string	Yes	/	'password:'	The regular expression that is used to match the prefix of the sensitive content.
ReplacedContentPattern	string	Yes	/	[^']*	The regular expression that is used to match the sensitive content.
ReplacingAll	bool	No	true	true	Specifies whether to replace all sensitive content that is matched.

Output plug-ins

Simple Log Service output plug-in

Parameter	Type	Required	Default value	Example	Description
Type	string	Yes	/	flusher_sls	The type of the plug-in. Set the value to flusher_sls.
Logstore	stirng	Yes	/	test-logstore	The name of the Logstore.

Response parameters

Parameter	Type	Description	Example
headers	object
Server	string	The name of the server.	nginx
Content-Type	string	The content type of the response body.	application/json
Content-Length	string	The content length of the response body.	0
Connection	string	Indicates whether the connection is persistent. Valid values: close and keep-alive. close: The connection is non-persistent. A new TCP connection is established for each HTTP request. keep-alive: The connection is persistent. After a TCP connection is established, the connection remains open, and no more time or bandwidth is consumed to establish new connections.	close
Date	string	The time when the response was returned.	Sun, 27 May 2018 08:25:04 GMT
x-log-requestid	string	The request ID.	5B0A6B60BB6EE39764D458B5

Examples

Sample success responses

JSONformat

{}

Error codes

For a list of error codes, visit the Service error codes.