You can use a Logtail plug-in to extract log fields from logs based on a regular expression. The logs are parsed into key-value pairs.
Entry point
If you want to use a Logtail plug-in to process logs, you can add a Logtail plug-in configuration when you create or modify a Logtail configuration. For more information, see Overview.
Configuration description
Parameter | Description |
Original Field | The original field that is used to store the content of a log before the log is parsed. Default value: content. |
Regular Expression | The regular expression that is used to match logs.
|
Extracted Field | The extracted fields. Configure the Key parameter for each Value parameter. The Key parameter specifies a new field name. The Value parameter specifies the content that is extracted from logs. |
Retain Original Field If Parsing Fails | If you select the Retain Original Field If Parsing Fails parameter and parsing fails, the original field is retained. |
Retain Original Field If Parsing Succeeds | If you select the Retain Original Field If Parsing Succeeds parameter and parsing is successful, the original field is retained. |
New Name of Original Field | If you select the Retain Original Field If Parsing Fails or Retain Original Field If Parsing Succeeds parameter, you can rename the original field to store the original log content. |
Configuration example
Raw log
127.0.0.1 - - [16/Oct/2023:12:36:49 +0800] "GET /index.html HTTP/1.1" 200 612 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36"
Logtail plug-in configuration
In this example, the regular expression is set to
(\S+)\s-\s(\S+)\s\[([^]]+)]\s"(\w+)\s(\S+)\s([^"]+)"\s(\d+)(\s\d+)[^-]+([^"]+)"\s"([^"]+).*
.Parsing result