Simple Log Service allows you to query and analyze logs in index mode. This feature is integrated with SQL computing. This topic describes the syntax and limits of log query and analysis statements. This topic also describes the SQL functions that you can call when you use the log query and analysis feature.
If you want to query and analyze logs, you must store the logs in a Standard Logstore and turn on Enable Analytics for the required fields when you configure indexes. For more information, see Manage a logstore and Create indexes.
If you want to query tens of billions of log entries, you can repeatedly execute a query statement up to 10 times to obtain all results. For more information, see What can I do if the "The results are inaccurate" error occurs?
By default, Simple Log Service provides reserved fields. For more information about how to analyze reserved fields, see Reserved fields.
Syntax
We recommend that you specify at most 30 query conditions in a query statement.
By default, an analytic statement analyzes data in the current Logstore. You do not need to specify the FROM or WHERE clause.
Analytic statements do not support offsets, are not case-sensitive, and must not end with semicolons (;).
Separate a query statement and an analytic statement with a vertical bar (|). You can use query statements separately. Analytic statements must be used together with query statements. You can use the log analysis feature to analyze all data in a Logstore or only data that meets the specified query conditions in a Logstore.
Query statement|analytic statementType | Description |
Query statement | A query statement specifies one or more query conditions, and then returns the log data that meets the specified conditions. A query condition can be a keyword, a numeric value, a numeric value range, a space character, or an asterisk (*). If you specify a space character or an asterisk (*) as the query condition, all data is returned. For more information, see Search syntax. |
Analytic statement | An analytic statement is used to aggregate or analyze query results or all data in a Logstore. For more information, see Use SPL to query and analyze logs and SQL syntax and functions. |
Examples
* | SELECT status, count(*) AS PV GROUP BY statusThe following figure shows the results of a query statement and an analytic statement.
For more information about how to query and analyze data in Logstores, see Query and analyze logs.
Advanced features
LiveTail: To reduce O&M workloads, Simple Log Service provides the LiveTail feature in the Simple Log Service console. You can use the feature to monitor and analyze logs in real time. For more information, see LiveTail.
LogReduce: When you collect logs, the LogReduce feature can aggregate highly similar logs and extract patterns from the logs. This way, you can have an overview of the logs. For more information, see LogReduce.
Contextual query: You can use the context information to identify and troubleshoot errors in an efficient manner. For more information, see Contextual query.
Field analysis: This feature analyzes the distribution and metrics of each field and provides a time series chart for the top 5 values of each field. You can use the analytic results for data insights and visualization. For more information, see Field analysis.
Event settings: Simple Log Service allows you to configure drilling events for raw logs. This way, you can visualize and manage the raw logs and obtain the detailed information about the raw logs in an efficient manner. For more information, see Event settings.
StoreView: You can use StoreView to query logs across regions and Logstores. For more information, see Use StoreView to query logs across Logstores.
Limits on query statements
Limit | Description |
Number of keywords | The number of keywords that can be used as query conditions is limited. The number of logical operators is not included. You can specify at most 30 keywords in each query statement. |
Field value size | The maximum size of a field value is 512 KB. The excess part is not used in the query statement. If the size of a field value exceeds 512 KB, log data may fail to be returned based on keywords. This does not affect the integrity of the log data stored in the Logstore. |
Query concurrency | Each project supports a maximum of 100 concurrent query statements. For example, up to 100 users can query data in all Logstores of a project at the same time. |
Returned results | The returned log entries are displayed on multiple pages. Each page displays up to 100 log entries. |
Fuzzy query | In a fuzzy query, Simple Log Service matches up to 100 words that meet the specified conditions and returns the log entries that meet the query conditions and contain one or more of these words. For more information, see fuzzy search in Search syntax. |
Query result sorting | By default, query results are sorted in descending order of the query time, which is accurate to seconds. Query results are sorted in descending order of the query time that is accurate to nanoseconds if any result is returned within nanoseconds. |
Limits on analytic statements
Limit | Standard SQL | Dedicated SQL |
Query concurrency | Each project supports a maximum of 15 concurrent analytic statements. For example, up to 15 users can execute analytic statements to analyze data in all Logstores of a project at the same time. | Each project supports a maximum of 100 concurrent analytic statements. For example, up to 100 users can execute analytic statements to analyze data in all Logstores of a project at the same time. |
Data volume | You can analyze up to 1 GB of data in each shard. | An analytic statement can scan a maximum of 200 billion rows of data at the same time. |
Mode | By default, Standard SQL is enabled. | You need to manually enable Dedicated SQL. For more information, see Enable Dedicated SQL. |
Billing | Free of charge. | Billed based on the actual CPU time. |
Applicable scope | You can analyze only the data that is written to Simple Log Service after the log analysis feature is enabled. If you need to analyze historical data, you must re-index the historical data. For more information, see Reindex logs for a Logstore. | You can analyze only the data that is written to Simple Log Service after the log analysis feature is enabled. If you need to analyze historical data, you must re-index the historical data. For more information, see Reindex logs for a Logstore. |
Returned results | By default, an analytic statement returns up to 100 rows of data. If you want to view more data, use a LIMIT clause. For more information, see LIMIT clause. | By default, an analytic statement returns up to 100 rows of data. If you want to view more data, use a LIMIT clause. For more information, see LIMIT clause. |
Field value size | The maximum size of a field value is 16 KB (16,384 bytes). The excess part is not used in the analytic statement. Note The maximum size of a field value is 2 KB (2,048 bytes). If you want to modify the maximum size of a field value, specify the Maximum Statistics Field Length parameter. After you modify the index settings, the new index settings take effect only on incremental data. For more information, see Create indexes. | The maximum size of a field value is 16 KB (16,384 bytes). The excess part is not used in the analytic statement. Note The maximum size of a field value is 2 KB (2,048 bytes). If you want to modify the maximum size of a field value, specify the Maximum Statistics Field Length parameter. After you modify the index settings, the new index settings take effect only on incremental data. For more information, see Create indexes. |
Timeout period | The maximum timeout period for a single analytic statement is 55 seconds. | The maximum timeout period for a single analytic statement is 55 seconds. |
Number of decimal places in the value of a DOUBLE-type field | The value of a DOUBLE-type field can contain a maximum of 52 decimal places. If the number of decimal places is greater than 52, the accuracy of the field value is compromised. | The value of a DOUBLE-type field can contain a maximum of 52 decimal places. If the number of decimal places is greater than 52, the accuracy of the field value is compromised. |
References
For more information about the functions and syntax supported by Simple Log Service for analyzing logs, see the following topics:
For more information about the API operations that you can call to query logs, see the following topics: