All Products
Search

This Product

    Simple Log Service:GetLogs

    Document Center

    Simple Log Service:GetLogs

    Last Updated:Nov 06, 2024

    Queries the logs of a Logstore in a project.

    Operation description

    Usage notes

    Note Simple Log Service allows you to create a Scheduled SQL job. For more information, see Create a scheduled SQL job.

    • Host consists of a project name and a Simple Log Service endpoint. You must specify a project in Host.
    • An AccessKey pair is created and obtained. For more information, see AccessKey pair.

    The AccessKey pair of an Alibaba Cloud account has permissions on all API operations. Using these credentials to perform operations in Simple Log Service is a high-risk operation. We recommend that you use a Resource Access Management (RAM) user to call API operations or perform routine O&M. To create a RAM user, log on to the RAM console. Make sure that the RAM user has the management permissions on Simple Log Service resources. For more information, see Create a RAM user and authorize the RAM user to access Simple Log Service.

    • The information that is required to query logs is obtained. The information includes the name of the project to which the logs belong, the region of the project, and the name of the Logstore to which the logs belong. For more information, see Manage a project and Manage a Logstore.

    • Limits are imposed when you use Simple Log Service to query logs. We recommend that you specify query statements and query time ranges based on the limits. For more information, see Log search overview and Log analysis overview.

    • Indexes are configured before you query logs. For more information, see Create indexes.

    • If the number of logs in a Logstore significantly changes, Simple Log Service cannot predict the number of times that you must call this operation to obtain the complete results. In this case, you must check the value of the x-log-progress parameter in the response of each request and determine whether to call this operation one more time to obtain the complete results. Each time you call this operation, the same number of charge units (CUs) are consumed.

    • After a log is written to a Logstore, you can call the GetHistograms or the GetLogs operation to query the log. The latency of the query varies based on the type of the log. Simple Log Service classifies logs into the following types based on log timestamps:

      • Real-time data: The difference between the time record in a log of this type and the current time on Simple Log Service is within the interval (-180 seconds,900 seconds]. For example, if a log was generated at 12:03:00, September 25, 2014 (UTC) and Simple Log Service received the log at 12:05:00, September 25, 2014 (UTC), Simple Log Service processes the log as real-time data. This type of log is usually generated in common scenarios.
      • Historical data: The difference between the time record in a log of this type and the current time on Simple Log Service is within the interval [-604,800 seconds,-180 seconds). For example, if a log was generated at 12:00:00, September 25, 2014 (UTC) and Simple Log Service received the log at 12:05:00, September 25, 2014 (UTC), Simple Log Service processes the log as historical data. This type of log is usually generated in data backfill scenarios. After real-time data is written to a Logstore, the data can be queried with an approximate latency of 3 seconds.

    Note Simple Log Service calculates the difference between the log time that is specified by the __time__ field and the receiving time that is specified by the __tag__:receive_time field for each log. The receiving time indicates when Simple Log Service receives the log. If the difference is within the interval (-180 seconds,900 seconds], Simple Log Service processes the log as real-time data. If the difference is within the interval [-604,800 seconds,-180 seconds), Simple Log Service processes the log as historical data.

    Authentication resources

    The following table describes the authorization information that is required for this operation. You can add the information to the Action element of a RAM policy statement to grant a RAM user or a RAM role the permissions to call this operation.

    ActionResource
    log:GetLogStoreLogsacs:log:{#regionId}:{#accountId}:project/{#ProjectName}/logstore/{#LogstoreName}

    Debugging

    You can run this interface directly in OpenAPI Explorer, saving you the trouble of calculating signatures. After running successfully, OpenAPI Explorer can automatically generate SDK code samples.

    Authorization information

    There is currently no authorization information disclosed in the API.

    Request syntax

    GET /logstores/{logstore}?type=log HTTP/1.1

    Request parameters

    ParameterTypeRequiredDescriptionExample
    projectstringYes

    The name of the project.

    		ali-test-project
    logstorestringYes

    The Logstore whose logs you want to query.

    		example-logstore
    fromintegerYes

    The beginning of the time range to query. The value is the log time that is specified when log data is written.

    • The time range that is specified in this operation is a left-closed, right-open interval. The interval includes the start time specified by the from parameter, but does not include the end time specified by the to parameter. If you specify the same value for the from and to parameters, the interval is invalid, and an error message is returned.
    • The value is a UNIX timestamp representing the number of seconds that have elapsed since January 1, 1970, 00:00:00 UTC.
    Note To ensure that full data can be queried, specify a query time range that is accurate to the minute. If you also specify a time range in an analytic statement, Simple Log Service uses the time range specified in the analytic statement for query and analysis.

    If you want to specify a time range that is accurate to the second in your analytic statement, you must use the from_unixtime or to_unixtime function to convert the time format. For more information about the functions, see from_unixtime function and to_unixtime function. Examples:

    • * | SELECT * FROM log WHERE from_unixtime(__time__) > from_unixtime(1664186624) AND from_unixtime(__time__) < now()
    • * | SELECT * FROM log WHERE __time__ > to_unixtime(date_parse('2022-10-19 15:46:05', '%Y-%m-%d %H:%i:%s')) AND __time__ < to_unixtime(now())
    		1627268185
    tointegerYes

    The end of the time range to query. The value is the log time that is specified when log data is written.

    • The time range that is specified in this operation is a left-closed, right-open interval. The interval includes the start time specified by the from parameter, but does not include the end time specified by the to parameter. If you specify the same value for the from and to parameters, the interval is invalid, and an error message is returned.
    • The value is a UNIX timestamp representing the number of seconds that have elapsed since January 1, 1970, 00:00:00 UTC.
    Note To ensure that full data can be queried, specify a query time range that is accurate to the minute. If you also specify a time range in an analytic statement, Simple Log Service uses the time range specified in the analytic statement for query and analysis.

    If you want to specify a time range that is accurate to the second in your analytic statement, you must use the from_unixtime or to_unixtime function to convert the time format. For more information about the functions, see from_unixtime function and to_unixtime function. Examples:

    • * | SELECT * FROM log WHERE from_unixtime(__time__) > from_unixtime(1664186624) AND from_unixtime(__time__) < now()
    • * | SELECT * FROM log WHERE __time__ > to_unixtime(date_parse('2022-10-19 15:46:05', '%Y-%m-%d %H:%i:%s')) AND __time__ < to_unixtime(now())
    		1627269085
    querystringNo

    The search statement or the query statement. For more information, see Log search overview and Log analysis overview. If you add set session parallel_sql=true; to the analytic statement in the query parameter, Dedicated SQL is used. For example, you can set the query parameter to * | set session parallel_sql=true; select count(*) as pv. For more information about common errors that may occur during log query and analysis, see How do I resolve common errors that occur when I query and analyze logs?

    Note If you specify an analytic statement in the value of the query parameter, the line and offset parameters do not take effect. In this case, we recommend that you set the line and offset parameters to 0 and use the LIMIT clause to limit the number of logs to return on each page. For more information, see Paged query.
    		status: 401 | SELECT remote_addr,COUNT(*) as pv GROUP by remote_addr ORDER by pv desc limit 5
    topicstringNo

    The topic of the logs. The default value is an empty string. For more information, see Topic .

    		topic
    linelongNo

    The maximum number of logs to return for the request. This parameter takes effect only when the query parameter is set to a search statement. Minimum value: 0. Maximum value: 100. Default value: 100. For more information, see Perform paged queries.

    		100
    offsetlongNo

    The line from which the query starts. This parameter takes effect only when the query parameter is set to a search statement. Default value: 0. For more information, see Perform paged queries.

    		0
    reversebooleanNo

    Specifies whether to return logs in reverse chronological order of log timestamps. The log timestamps are accurate to the minute. Valid values:

    • true: returns logs in reverse chronological order of log timestamps.
    • false (default): returns logs in chronological order of log timestamps.
    Note
    • The reverse parameter takes effect only when the query parameter is set to a search statement. The reverse parameter specifies the method used to sort returned logs.
    • If the query parameter is set to a query statement, the reverse parameter does not take effect. The method used to sort returned logs is specified by the ORDER BY clause in the analytic statement. If you use the keyword asc in the ORDER BY clause, the logs are sorted in chronological order. If you use the keyword desc in the ORDER BY clause, the logs are sorted in reverse chronological order. By default, asc is used in the ORDER BY clause.
    		false
    powerSqlbooleanNo

    Specifies whether to enable the Dedicated SQL feature. For more information, see Enable Dedicated SQL. Valid values:

    • true: enables the Dedicated SQL feature.
    • false (default): enables the Standard SQL feature.

    You can use the powerSql or query parameter to configure Dedicated SQL.

    		false

    Response parameters

    ParameterTypeDescriptionExample
    headersobject
    x-log-progressstring

    The status of the query and analysis results that are returned. Valid values:

    • Complete: The query is successful, and the complete query and analysis results are returned.
    • Incomplete: The query is successful, but the returned query and analysis results are incomplete. To obtain the complete results, you must repeat the request.
    Note You can use the *|select count(*) as count statement to obtain the total number of logs. You can also call the GetHistogram operation to obtain the number of logs that are generated within each subinterval of a time range and add the numbers to obtain the total number of logs. If you do not need to obtain the total number of logs, you can modify the offset parameter in the request and repeat the request multiple times. If the value of the x-log-progress parameter is Complete and the number of returned rows is less than the number of requested rows, all logs are read.
    		Complete
    x-log-countlong

    The number of rows that are returned.

    Note The total number of logs that meet the specified conditions is not returned. To query the total number of logs that meet the specified conditions, you must specify a query statement. For example, the query statement request_method:GET|select count(*) as count can be used to query the total number of logs whose request_method is GET. For more information, see Best practices for query and analysis and Examples for querying and analyzing logs by using SDK for Java.
    		100
    x-log-processed-rowslong

    The number of rows that are processed in the query.

    		10000
    x-log-elapsed-millisecondlong

    The time that is consumed by the query. Unit: milliseconds.

    		5
    Serverstring

    The name of the server.

    		nginx
    Content-Typestring

    The content type of the response body.

    		application/json
    Content-Lengthstring

    The content length of the response body.

    		0
    Connectionstring

    Indicates whether the connection is persistent. Valid values:

    • close: The connection is non-persistent. A new TCP connection is established for each HTTP request.
    • keep-alive: The connection is persistent. After a TCP connection is established, the connection remains open, and no more time or bandwidth is consumed to establish new connections.
    		close
    Datestring

    The time when the response was returned.

    		Sun, 27 May 2018 08:25:04 GMT
    x-log-requestidstring

    The request ID.

    		5B0A6B60BB6EE39764D458B5
    array<object>

    An array of logs. Each element in the array indicates a log.

    object
    		[{'remote_addr': '198.51.XXX.XXX', 'pv': '1', '__source__': '', '__time__': '1649902984'}, {'remote_addr': '198.51.XXX.XXX', 'pv': '1', '__source__': '', '__time__': '1649902984'}, {'remote_addr': '198.51.XXX.XXX', 'pv': '1', '__source__': '', '__time__': '1649902984'}, {'remote_addr': '198.51.XXX.XXX', 'pv': '1', '__source__': '', '__time__': '1649902984'}, {'remote_addr': '198.51.100.XXX', 'pv': '1', '__source__': '', '__time__': '1649902984'}]

    Examples

    Sample success responses

    JSONformat

    [
  {
    "test": "test",
    "test2": 1
  }
]

    Error codes

    For a list of error codes, visit the Service error codes.