通过Scan搜索实现上下文查询

代码示例

1. 在pom.xml文件中添加如下依赖。

   <!-- https://mvnrepository.com/artifact/com.aliyun.openservices/aliyun-log -->
   <dependency>
       <groupId>com.aliyun.openservices</groupId>
       <artifactId>aliyun-log</artifactId>
       <version>0.6.120</version>
   </dependency>

2. 使用如下代码。根据实际情况替换参数project、logStore、endpoint、accessKeyId和accessKeySecret，日志服务的公网域名，获取方式请参见服务接入点，用户身份识别ID，获取方式，请参见创建AccessKey。

   import com.aliyun.openservices.log.Client;
   import com.aliyun.openservices.log.common.QueriedLog;
   import com.aliyun.openservices.log.exception.LogException;
   import com.aliyun.openservices.log.response.GetLogsResponse;
   
   public class DoScanTest {
       // 本示例从环境变量中获取AccessKey ID和AccessKey Secret。
       static String accessKeyId = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID");
       static String accessKeySecret = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET");
   
       //日志服务的服务接入点。此处以杭州为例，其它地域请根据实际情况填写。
       static String host = "cn-hangzhou.log.aliyuncs.com";
       //初始化日志服务client
       static Client client = new Client(host, accessKeyId, accessKeySecret);
   
       public static void main(String[] args) throws LogException {
           String project = "test-project";
           String logStore = "test-logstore";
           int fromTime = 1740985125;      // event-time, [from, to)
           int toTime = 1740985125 + 7200; // event-time, [from, to)
           // String query = "Status:404 | where http_host = 'www.yt.mock.com'";      // 不返回上下文信息
           String query = "Status:404 | where http_host = 'www.yt.mock.com' | with_pack_meta";    // 返回上下文信息
           int totalCount = 0;
           // boolean reverse = false;  // 从前向后查找
           boolean reverse = true; // 从后向前查找
           boolean forward = true;
           int offset = 0;
           while (true) {
               GetLogsResponse resp = client.GetLogs(project, logStore, fromTime, toTime, "", query, 100, offset, reverse, forward, "mode=scan;");
               for (QueriedLog log : resp.getLogs()) {
                   System.out.println(log.GetLogItem().ToJsonString());
               }
               System.out.println("[response of this scan]\tbegin offset: " + resp.GetBeginOffset() + "\tend offset: " + resp.GetEndOffset() + "\tresult logs: " + resp.getLogs().size() + "\tis finished: " + resp.IsScanAll());
               totalCount += resp.getLogs().size();
               if (resp.IsScanAll()) {
                   break;
               }
               offset = forward ? (int) resp.GetEndOffset() : (int) resp.GetBeginOffset();
           }
           System.out.println("totally scanned logs\t: " + totalCount);
       }
   }

   SDK参数说明

   reverse	forward	offset	行为
   false	true	0 或上次请求响应的 endOffset。	从前（时间戳小）往后（时间戳大）翻页。
   true	true	0 或上次请求响应的 endOffset。	从后（时间戳大）往前（时间戳小）翻页。
   false	false	索引命中的日志总数或上次请求响应的 beginOffset。	从后（时间戳大）往前（时间戳小）翻页。
   true	false	索引命中的日志总数或上次请求响应的 beginOffset。	从前（时间戳小）往后（时间戳大）翻页。

3. 运行输出。

   {
       "referer": "www.xxx.xxx.xxx",
       "slbid": "slb-01",
       "scheme": "https",
       "vpc_id": "8c093000-9f68-2c0f-a904-5c612483505a",
       "upstream_addr": "125.36.xx.xx",
       "owner_id": "owner-01",
       "body_bytes_sent": "1733",
       "request_method": "GET",
       "http_host": "www.yt.mock.com",
       "http_user_agent": "Mozilla/4.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/11.0.1245.0 Safari/537.36",
       "remote_user": "0iu9d",
       "upstream_status": "200",
       "vip_addr": "139.207.xxx.xxx",
       "request_time": "24",
       "__pack_meta__": "0|MTczNjkzNzIxNzg2NDA5NT****==|287|277",
       "__tag__:__pack_id__":"5253859C5169****-3",
       "host": "www.xxx.xxx.xxx",
       "client_ip": "58.19.XXX.XX",
       "user_agent": "Mozilla/5.0 (Windows NT 7.1) AppleWebKit/534.30 (KHTML, like Gecko) Chrome/12.0.742.112 Safari/534.30",
       "remote_addr": "183.70.XX.XXX",
       "__topic__": "nginx_access_log",
       "instance_name": "instance-01",
       "time_local": "03/Mar/2025:08:02:34",
       "request_uri": "/request/path-2/file-4",
       "instance_id": "i-01",
       "request_length": "4458",
       "http_x_forwarded_for": "103.96.xxx.xxx",
       "upstream_response_time": "30",
       "network_type": "vlan",
       "region": "cn-shanghai",
       "logtime": 1740988954,
       "server_protocol": "HTTP/2.0",
       "status": "404"
   }
   [response of this scan]	begin offset: 0	end offset: 1	result logs: 1	is finished: true
   totally scanned logs	: 1
   
   Process finished with exit code 0

4. 上下文信息。

   在代码中，当Scan 语法是{Index Search Query} | {Scan Query}时，返回的日志信息中不包含上下文字段。带上下文字段返回的语法是 {Index Search Query} | {Scan Query} | with_pack_meta。

   如上代码示例中，返回的日志上下文字段如下：

   __pack_meta__: 0|MTczNjkzNzIxNzg2NDA5NT****==|287|277
   __tag__:__pack_id__: 5253859C5169****-3

   ​有了这些上下文信息后，您可以参考GetContextLogs获得这条日志在原始文件中的上一条和下一条。