Simple Log Service:SPL instructions

project

This instruction retains the fields that match the specified pattern and renames the specified fields. During instruction execution, all retain-related expressions are executed before rename-related expressions.

Syntax

| project -wildcard <field-pattern>, <output>=<field>, ...

Parameters

Examples

• Example 1: Retain a field.

  * | project level, err_msg

• Example 2: Rename a field.

  * | project log_level=level, err_msg

• Example 3: Retain fields based on __tag__:* in exact match mode.

  * | project "__tag__:*"

project-away

This instruction removes the fields that match the specified pattern and retains all other fields as they are.

Syntax

| project-away -wildcard <field-pattern>, ...

Parameters

project-rename

This instruction renames the specified fields and retains all other fields as they are.

Syntax

| project-rename <output>=<field>, ...

Parameters

Examples

Rename the specified fields.

* | project-rename log_level=level, log_err_msg=err_msg

extend

This instruction creates fields based on the result of SQL expression-based data calculation. For more information about the SQL functions that are supported by SPL, see SPL-supported SQL functions.

Syntax

| extend <output>=<sql-expr>, ...

Parameters

Examples

• Example 1: Use a computation expression.

  * | extend Duration = EndTime - StartTime

• Example 2: Use a regular expression.

  * | extend server_protocol_version=regexp_extract(server_protocol, '\d+')

• Example 3: Extract JSONPath content and convert the data type of a field.

  • SPL statement

    *
    | extend a=json_extract(content, '$.body.a'), b=json_extract(content, '$.body.b')
    | extend b=cast(b as BIGINT)

  • Input data

    content: '{"body": {"a": 1, "b": 2}}'

  • Result

    content: '{"body": {"a": 1, "b": 2}}'
    a: '1'
    b: 2

where

This instruction filters data based on the result of SQL expression-based data calculation. Data that matches the specified SQL expression is retained. For more information about the SQL functions that are supported by the where instruction, see SPL-supported SQL functions.

Syntax

| where <sql-expr>

Parameters

Examples

• Example 1: Filter data based on field content.

  * | where userId='123'

• Example 2: Filter data by using a regular expression that matches data based on a field name.

  * | where regexp_like(server_protocol, '\d+')

• Example 3: Convert the data type of a field to match all data of server errors.

  * | where cast(status as BIGINT) >= 500

parse-regexp

This instruction extracts the information that matches groups in the specified regular expression from the specified field.

Syntax

| parse-regexp <field>, <pattern> as <output>, ...

Parameters

Examples

• Example 1: Use the exploratory match mode.

  • SPL statement

    *
    | parse-regexp content, '(\S+)' as ip -- Generate the ip: 10.0.0.0 field. 
    | parse-regexp content, '\S+\s+(\w+)' as method -- Generate the method: GET field.

  • Input data

    content: '10.0.0.0 GET /index.html 15824 0.043'

  • Result

    content: '10.0.0.0 GET /index.html 15824 0.043'
    ip: '10.0.0.0'
    method: 'GET'

• Example 2: Use the full pattern match mode and use unnamed capturing groups in a regular expression.

  • SPL statement

    * | parse-regexp content, '(\S+)\s+(\w+)' as ip, method

  • Input data

    content: '10.0.0.0 GET /index.html 15824 0.043'

  • Result

    content: '10.0.0.0 GET /index.html 15824 0.043'
    ip: '10.0.0.0'
    method: 'GET'

parse-csv

This instruction extracts information in the CSV format from the specified field.

Syntax

| parse-csv -delim=<delim> -quote=<quote> -strict <field> as <output>, ...

Parameters

Examples

• Example 1: Match data in simple mode.

  • SPL statement

    * | parse-csv content as x, y, z

  • Input data

    content: 'a,b,c'

  • Result

    content: 'a,b,c'
    x: 'a'
    y: 'b'
    z: 'c'

• Example 2: Use double quotation marks as the quote to match data that contains special characters.

  • SPL statement

    * | parse-csv content as ip, time, host

  • Input data

    content: '192.168.0.100,"10/Jun/2019:11:32:16,127 +0800",example.aliyundoc.com'

  • Result

    content: '192.168.0.100,"10/Jun/2019:11:32:16,127 +0800",example.aliyundoc.com'
    ip: '192.168.0.100'
    time: '10/Jun/2019:11:32:16,127 +0800'
    host: 'example.aliyundoc.com'

• Example 3: Use a combination of multiple characters as the delimiter.

  • SPL statement

    * | parse-csv -delim='||' content as time, ip, req

  • Input data

    content: '05/May/2022:13:30:28||127.0.0.1||POST /put?a=1&b=2'

  • Result

    content: '05/May/2022:13:30:28||127.0.0.1||POST /put?a=1&b=2'
    time: '05/May/2022:13:30:28'
    ip: '127.0.0.1'
    req: 'POST /put?a=1&b=2'

parse-json

This instruction extracts the first-layer JSON information from the specified field.

Syntax

| parse-json -mode=<mode> -path=<path> -prefix=<prefix> <field>

Parameters

Examples

• Example 1: Extract all keys and values from the y field.

  • SPL statement

    * | parse-json y

  • Input data

    x: '0'
    y: '{"a": 1, "b": 2}'

  • Result

    x: '0'
    y: '{"a": 1, "b": 2}'
    a: '1'
    b: '2'

• Example 2: Extract the value of the body key from the content field as different fields.

  • SPL statement

    * | parse-json -path='$.body' content

  • Input data

    content: '{"body": {"a": 1, "b": 2}}'

  • Result

    content: '{"body": {"a": 1, "b": 2}}'
    a: '1'
    b: '2'

• Example 3: Extract information in preserve mode. For an existing field, retain the original value.

  • SPL statement

    * | parse-json -mode='preserve' y

  • Input data

    a: 'xyz'
    x: '0'
    y: '{"a": 1, "b": 2}'

  • Result

    x: '0'
    y: '{"a": 1, "b": 2}'
    a: 'xyz'
    b: '2'