All Products
Search

This Product

    Simple Log Service:Data filtering

    Document Center

    Simple Log Service:Data filtering

    Last Updated:Dec 20, 2023

    You can use a Logtail plug-in to filter logs based on specific field values.

    Entry point

    If you want to use a Logtail plug-in to process logs, you can add a Logtail plug-in configuration when you create or modify a Logtail configuration. For more information, see Overview.

    Configuration description

    Parameter

    Description

    Original Field

    The original field that is used to store the content of a log before the log is parsed.

    Whitelist

    The log whitelist. Only logs that meet the specified conditions in the whitelist can be collected. You must specify new field names and expressions that are used to filter specific logs. The regular expression that you specify supports only full-text matching. Keyword matching is not supported. For more information about how to specify a regular expression, see Getting started with regular expressions.

    The whitelist conditions are in the logical OR relation. Examples:

    • Collect the logs that match the specified filter conditions.

      • If you set Field Name to level and Field Value to WARNING|ERROR, only the logs in which the value of the level field is WARNING or ERROR are collected.

      • If you set Field Name to content and Field Value to .*05\/Jan\/2023.*, only the logs in which the value of the content field contains 05/Jan/2023 are collected.

    • Filter out the logs that do not match the specified filter conditions.

      • If you set Field Name to level and Field Value to ^(?!.*(INFO|DEBUG)).*, logs in which the value of the level field contains INFO or DEBUG are not collected.

      • If you set Field Name to level and Field Value parameter to ^(?!(INFO|DEBUG)$).*, logs in which the value of the level field is INFO or DEBUG are not collected.

      • If you set Field Name to url and Field Value to .*^(?!.*(healthcheck)).*, logs in which the value of the url field contains healthcheck are not collected.