All Products
Search

This Product

    Simple Log Service:Check data by using functions

    Document Center

    Simple Log Service:Check data by using functions

    Last Updated:Aug 09, 2024

    You can check and process data based on specific conditions by using the functions. This topic describes how to use functions to check data in various scenarios.

    Scenario 1: Check whether a field exists

    • Raw log entry

      a: a_value
b:       // Empty string

    • Domain-specific language (DSL) orchestration

      • Solution 1: Use the e_has and e_not_has functions.

        e_if(e_has("a"),e_set("has_a", true))
e_if(e_has("b"),e_set("has_b", true))
e_if(e_has("c"),e_set("has_c", true))
e_if(e_not_has("a"),e_set("not_has_a", true))
e_if(e_not_has("b"),e_set("not_has_b", true))
e_if(e_not_has("c"),e_set("not_has_c", true))

      • Solution 2: Use the e_search function.

        e_if(e_search('a: *'),e_set("has_a", true))
e_if(e_search('b: *'), e_set("has_b", true))
e_if(e_search('c: *'), e_set("has_c", true))
e_if(e_search('not a: *'), e_set("not_has_a", true))
e_if(e_search('not b: *'), e_set("not_has_b", true))
e_if(e_search('not c: *'), e_set("not_has_c", true))
        Note

        In the preceding example, an e_if function is written for each condition to better illustrate the solution. You can simplify the function by including all conditions and the corresponding operations as e_if(condition 1, operation 1, condition 2, operation 2).

    • Result

      a:a_value
b:    // Empty string
has_a: true
has_b: true
not_has_c: true

    Scenario 2: Check whether a field value exists and is not empty

    • Raw log entry

      a: a_value
b:     // Empty string

    • DSL orchestration

      • Solution 1 (recommended): Use the v function that returns a field value.

        e_if(v("a"), e_set("not_empty_a", true))
e_if(v("b"), e_set("not_empty_b", true))
e_if(v("c"), e_set("not_empty_c", true))
        Note

        If the field value extracted by the v function exists and is not empty, the Bool value true is returned. Otherwise, false is returned.

      • Solution 2: Use the e_search function.

        # The field value contains at least one character.
e_if(e_search('a: "?"'), e_set("not_empty_a", true))
e_if(e_search('b: "?"'), e_set("not_empty_b", true))
e_if(e_search('c: "?"'), e_set("not_empty_c", true))

# Regular expression
e_if(e_search('a~=".+"'), e_set("not_empty_a", true))
e_if(e_search('b~=".+"'), e_set("not_empty_b", true))
e_if(e_search('c~=".+"'), e_set("not_empty_c", true))

# The field value exists and is not empty.
e_if(e_search('a: * and not a==""'), e_set("not_empty_a", true))
e_if(e_search('b: * and not b==""'), e_set("not_empty_b", true))
e_if(e_search('c: * and not c==""'), e_set("not_empty_b", true))

    • Result

      a: a_value
b:     // Empty string
not_empty_a: true

    Scenario 3: Check whether a field value exists and is empty

    • Raw log entry

      a: a_value
b:     // Empty string

    • DSL orchestration

      • Solution 1 (recommended): Use the v function that returns a field value.

        e_if(op_and(e_has("a"), op_not(v("a"))), e_set("empty_a", true))
e_if(op_and(e_has("b"), op_not(v("b"))), e_set("empty_b", true))
e_if(op_and(e_has("c"), op_not(v("c"))), e_set("empty_c", true))

# Invalid syntax
e_if(op_not(v("a")), e_set("empty_a", true))
e_if(op_not(v("b")), e_set("empty_b", true))
e_if(op_not(v("c")), e_set("empty_c", true))
        Note

        If the field value extracted by the v function exists and is not empty, the Bool value true is returned. Otherwise, false is returned. The true value is returned if the field value does not exist or if the field value is None.

      • Solution 2: Use the e_search function.

        e_if(e_search('a==""'), e_set("empty_a", true))
e_if(e_search('b==""'), e_set("empty_b", true))
e_if(e_search('c==""'), e_set("empty_c", true))

# Invalid syntax
e_if(e_search('a:""'), e_set("empty_a", true))
e_if(e_search('b:""'), e_set("empty_b", true))
        Note

        In the preceding example of the invalid syntax, the e_search function is used for partial query. In this case, true is returned if the value of the a: "" field exists, regardless of whether the value is empty.

    • Result

      a: a_value
b:       // Empty string
empty_b: true

    Scenario 4: Perform actions based on the logical relationships between field values

    • Raw log entries

      "Log entry 1"
http_host: example.com
status: 200
request_method: GET
scheme: https
header_length: 700
body_length: 1200

"Log entry 2"
http_host: example.org
status: 200
request_method: POST
scheme: https
header_length: 100
body_length: 800

"Log entry 3"
http_host: example.net
status: 200
request_method: GET
scheme:  http
header_length: 700
body_length: 800

"Log entry 4"
http_host: aliyundoc.com
status: 404
request_method: GET
scheme: https
header_length: 100
body_length: 300

    • Requirement 1

      Add the type field to all log entries in which the value of the status field is 200. The value of the type field is normal.

      • DSL orchestration

        e_if(e_match("status", "200"), e_set("type", "normal"))
Or
e_if(e_search('status==200'), e_set("type", "normal"))
        Note

        • You can use one of these solutions in scenarios where the requirements are simple.

        • In this case, status:200 can be used to check whether the value of the status field contains 200. To be more precise, we recommend that you use status==200.

      • Result

        "Log entry 1"
type: normal
http_host: example.com
status: 200
request_method: GET
scheme: https
header_length: 700
body_length: 1200

"Log entry 2"
type: normal
http_host: example.org
status: 200
request_method: POST
scheme: https
header_length: 100
body_length: 800

"Log entry 3"
type: normal
http_host: example.net
status: 200
request_method: GET
scheme: http
header_length: 700
body_length: 800

"Log entry 4"
http_host: aliyundoc.com
status: 404
request_method: GET
scheme: https
header_length: 100
body_length: 300

    • Requirement 2

      Add the type field to all log entries that meet the following conditions: the value of the status field is 200, the value of the request_method field is GET, and the value of the scheme field is https. The value of the type field is normal.

      • DSL orchestration

        e_if(e_search('status==200 and request_method==GET and scheme==https'), e_set("type", "normal"))
Or
e_if(e_match_all("status", "200", "request_method", "GET", "scheme", "https"), e_set("type", "normal"))
        Note

        • You can use the e_search or e_match_all function to match multiple fields. The e_search function is simpler.

        • In this case, status:200 can be used to check whether the value of the status field contains 200. To be more precise, we recommend that you use status==200.

      • Result

        "Log entry 1"
type: normal
http_host: example.com
status: 200
request_method: GET
scheme: https
header_length: 700
body_length: 1200

"Log entry 2"
http_host: example.org
status: 200
request_method: POST
scheme: https
header_length: 100
body_length: 800

"Log entry 3"
http_host: example.net
status: 200
request_method: GET
scheme: http
header_length: 700
body_length: 800

"Log entry 4"
http_host: aliyundoc.com
status: 404
request_method: GET
scheme: https
header_length: 100
body_length: 300

    • Requirement 3

      Add the type field to all log entries that meet one or more of the following conditions: the value of the status field is 200, the value of the request_method field is GET, or the value of the scheme field is https. The value of the type field is normal.

      • DSL orchestration

        e_if(e_search('status==200 or request_method==GET or scheme==https'), e_set("type", "normal"))
Or
e_if(e_match_any("status", "200", "request_method", "GET", "scheme", "https"), e_set("type", "normal"))

      • Result

        "Log entry 1"
type: normal
http_host: example.com
status: 200
request_method: GET
scheme: https
header_length: 700
body_length: 100

"Log entry 2"
type: normal
http_host: example.org
status: 200
request_method: POST
scheme: https
header_length: 100
body_length: 800

"Log entry 3"
type: normal
http_host: example.net
status: 200
request_method: GET
scheme: http
header_length: 700
body_length: 800

"Log entry 4"
type: normal
http_host: aliyundoc.com
status: 404
request_method: GET
scheme: https
header_length: 100
body_length: 1300

    • Requirement 4

      Add the type field to all log entries that meet the following conditions: the value of the status field is 200, the value of the request_method field is GET, and the sum of the values of the header_length and body_length fields is less than or equal to 1000. The value of the type field is normal.

      • DSL orchestration

        e_if(op_and(e_search('status: 200 and request_method: GET'), op_le(op_sum(v("header_length"), v("body_length")), 1000)), e_set("type", "normal"))
        Note

        You can use the e_search function and other expression functions for multiple logical operations.

      • Result

        "Log entry 1"
type: normal
http_host: example.com
status: 200
request_method: GET
scheme: https
header_length: 700
body_length: 100

"Log entry 2"
http_host: example.org
status: 200
request_method: POST
scheme: https
header_length: 100
body_length: 800

"Log entry 3"
http_host: example.net
status: 200
request_method: GET
scheme: http
header_length: 700
body_length: 800

"Log entry 4"
http_host: aliyundoc.com
status: 404
request_method: GET
scheme: https
header_length: 100
body_length: 1300