Query string syntax - Simple Log Service - Alibaba Cloud Documentation Center

Query strings are used in the domain-specific language (DSL) for Simple Log Service to filter log data in an efficient manner and simplify condition matching. This topic describes the rules for specifying query strings.

Functions

The following table describes the functions that use query strings.

Category	Function	Scenario
Event check functions	e_search	Query strings are used to check whether the value of a field in an event meets specified conditions.
Resource functions	res_log_logstore_pull	Query strings are used to configure a field blacklist or a field whitelist to filter data from a Logstore and return a table.
Resource functions	res_rds_mysql	Query strings are used to configure a field blacklist or a field whitelist to filter data from a specified table of an ApsaraDB RDS for MySQL database and return a table.

Features

The following table lists the search features that support field search and full-text search.

Feature	Field search	Full-text search
Search for substrings	Supported	Supported
Search for strings by using wildcard characters, which support asterisks (`*`) and question marks (?)	Supported	Supported
Exact match	Supported	Not supported
Search for strings by using regular expressions	Supported	Not supported
Search for strings by comparing numeric ranges	Supported	Not supported
Search for strings by comparing numeric values	Supported	Not supported
Search for strings by using logical operators (AND, OR, and NOT), or a combination of these operators	Supported	Supported

Escape special characters

Special characters, such as asterisks (*) and backslashes (\), must be escaped in query strings.

Escape special characters in a field name
Field names cannot be enclosed in double quotation marks (""). Special characters in a field name must be escaped by using backslashes (\). Examples:
- \*$1+1$\?: abc. Special characters are escaped by using backslashes (\).
- __tag__\:__container_name__: abc. Special characters are escaped by using backslashes (\).
- "content": abc. In this example, the field name is invalid. The field name cannot be enclosed in double quotation marks ("").
Escape special characters in a field value
- To query a field value that contains special characters such as quotation marks (") and backslashes (\), you must escape the special characters by using backslashes (\). Example: content: "abc\"xy\\z".
  Note
  A field value must be enclosed in double quotation marks (""). You can use single quotation marks('') to enclose the string and double quotation marks ("") to enclose the field value. For example, e_search("domain: '/url/test.jsp'") is invalid, and e_search('domain: "/url/test.jsp"') is valid.
- To query a field value that contains special characters such as asterisks (*) and question marks (?), you must escape the special characters by using backslashes (\). If you do not escape the special characters by using backslashes (\), the special characters are used as wildcard characters for matching.
- To query a field value that contains only letters, digits, underscores (_), hyphens (-), asterisks (*), and question marks (?), you do not need to enclose the field value in double quotation marks (""). When other characters are used, you must enclose the field value in double quotation marks (""). Examples:
  - status: "\*\?()[]:=". The field value is enclosed in double quotation marks (""). The asterisk (*) and question mark (?) are escaped by using backslashes (\). Characters other than the asterisk (*) and question mark (?) are not escaped in the field value.
  - content: ()[]:=. The field value is invalid. The field value must be enclosed in double quotation marks ("").
  - status: active\*test and status: active\?test. The field values contain only letters, an asterisk (*) and a question mark (?). The field values do not need to be enclosed in double quotation marks (""). The asterisk (*) and question mark (?) in the field values are escaped by using backslashes (\).

Search for substrings

Full-text search
Search for substrings in all fields.
- Syntax
```
e_search('substring')
```
- Examples
  - e_search('"error"'): searches for a substring.
  - e_search('"active error"'): searches for a substring that contains a space.
  - e_search('active error'): searches for multiple substrings. The logical operator OR is used by default.
Field search
Search for substrings in specific fields.
- Syntax
```
e_search('...')
```
- Examples
  - e_search('status: active'): searches for a substring.
  - e_search('author: "john smith"'): searches for a substring that contains a space.
  Note
  e_search('field: active error'): searches for active in the field field or searches for error in all fields. In this example, the query string is equivalent to field:active OR "error".

Search for strings by using wildcard characters

An asterisk (*) specifies zero or multiple characters. A question mark (?) specifies one character or one wide character.

Full-text search
Search for substrings in all fields.
- Syntax
```
e_search('substring')
```
- Examples
  - e_search('active*test'). The asterisk (*) is used to match zero or multiple characters. The query string does not need to be enclosed in double quotation marks ("") because the query string contains only letters and an asterisk (*).
  - e_search('*error occurs'). The asterisk (*) is used to match zero or multiple characters. For example, the error occurs and critical error occurs strings can be matched.
  - e_search('active?good'). The question mark (?) is used to match one character. The query string does not need to be enclosed in double quotation marks ("") because the query string contains only letters and a question mark (?).
  - e_search('ac*tive?good'). The query string is used to perform an exact match by using an asterisk (*) and a question mark (?).
  - e_search('ac*tive??go*od'). The query string is used to perform an exact match by using multiple asterisks (*) and question marks (?).
Field search
Search for substrings in specific fields.
- Syntax
```
e_search('field name: substring')
```
- Examples
  - e_search('status: active*test'). The asterisk (*) is used to match zero or multiple characters.
  - e_search('status: active?good'): The question mark (?) is used to match one character.

Exact match

In exact match, the entire field value is matched.

Syntax

e_search('field name==string that must be exactly matched')

Examples
- e_search('author== "john smith"'). The value of the author field must be john smith.
- e_search('status== ac*tive?good'). The query string contains wildcard characters and is used for exact match.

Search for strings by using regular expressions

Regular expressions are more efficient than wildcard characters in matching.

Syntax
```
e_search('field name~=regular expression')
```
Note
- Regular expressions may contain backslashes (\). We recommend that you use r to prevent the system from escaping the backslashes (\).
- By default, Simple Log Service performs fuzzy match. To enable exact match, you must specify a regular expression that includes a caret (^) as a prefix and a dollar sign ($) as a suffix.
Examples
- e_search('status~= "\d+"'). The value of the status field contains digits.
- e_search('status~= "^\d+$"'). The value of the status field is a number.

Search for strings by comparing numeric values or numeric ranges

You can search for field values by comparing field values with specified numeric values or numeric ranges.

Numeric value comparison
You can compare field values with specified numeric values by using the following operators. The operators are greater-than (>), greater-than-or-equal-to (>=), equal-to (=), less-than (<), and less-than-or-equal-to (<=).
```
e_search('age >= 18')  #  >=18
e_search('age > 18')   #  > 18
e_search('age = 18')   #  = 18
e_search('age <= 18')  #  <=18
e_search('age < 18')   #  < 18
```
Numeric range comparison
You can search for field values that are within a closed interval. An asterisk (*) can be used to specify an infinite interval.
```
e_search('count: [100, 200]') # >=100 and  <=200
e_search('count: [*, 200]')   # <=200
e_search('count: [200, *]')   # >=200
```

Search for strings by judging logical relationships

Logical operators can be used among multiple search conditions. Parentheses () are used to nest search conditions.

Logical operator	Keyword
AND	`and`, `AND`, and `&&`. The keywords are not case-sensitive.
OR	`or` and `OR`. The keywords are not case-sensitive.
NOT	`not`, `NOT`, and `!`. The keywords are not case-sensitive.

Examples:

e_search('abc OR xyz')    # The logical operator is not case-sensitive.
e_search('abc and (xyz or zzz)')
e_search('abc and not (xyz and not zzz)')
e_search('abc && xyz')    # and
e_search('abc || xyz')    # or
e_search('abc || !xyz')   # or not

Local operators can also be used to match substrings.

e_search('field: (abc OR xyz)')      # The field value contains abc or xyz.
e_search('field: (abc OR not xyz)')  # The field value contains abc or does not contain xyz.
e_search('field: (abc && !xyz)')     # The field value contains abc and does not contain xyz.

Field check

You can use query strings to check fields.

e_search('field: *'): checks whether a field exists.
e_search('not field:*'): checks whether a field does not exist.
e_search('not field:""'): checks whether a field does not exist.
e_search('field: "?"'): checks whether a field exists and whether the field is not empty.
e_search('field==""'): checks whether a field exists and whether the field is empty.
e_search('field~=".+"'): checks whether a field exists and whether the field is not empty.
e_search('not field~=".+"'): checks whether a field does not exist or whether the field is empty.
e_search('not field==""'): checks whether a field does not exist or whether the field is not empty.