Quick preview - Simple Log Service - Alibaba Cloud Documentation Center

In Quick preview mode, you can check whether the syntax of a data transformation script is valid and whether data is efficiently transformed as expected. You can perform the preceding operations in Quick preview mode free of charge. This topic describes how to configure the Quick preview mode.

Prerequisites

Data is collected. For more information, see Log collection methods.

Procedure

Log on to the Simple Log Service console.
Go to the data transformation page.
1. In the Projects section, click the project that you want to manage.
2. On the Log Storage > Logstores tab, click the Logstore that you want to manage.
3. On the query and analysis page, click Data Transformation.
In the upper-right corner of the page that appears, specify the time range of the data that you want to manage.
After you specify the time range, verify that logs appear on the Raw Logs tab.
In the edit box, enter a data transformation statement.
For more information, see Data processing syntax.
Note
You can add comments to the data transformation statement to debug the statement line by line.
Preview data.
1. In the upper-right corner of the page, click Quick.
2. Click the Data Testing tab.
3. On the Data Testing tab, enter test data.
  Test data includes basic data and dimension table data.
  - On the Data tab, you can enter or specify basic data.
    You can click the Raw Logs tab, find a log entry, and then click Import Test Data. The log entry is added as test data. You can also enter a test log entry.
    Note
    The size of the test data for a single preview cannot exceed 1 MB.
    Each test log entry is separated by a blank line.
    The Markdown syntax is used to indicate cross-line field values. Triple backticks (```) are used to indicate a whole field.
    The test data that you enter or specify on the Data tab must be in the key-value pair or JSON format. Colons (:) are used to connect the field names and field values of data in the key-value pair format.
    - Example 1: The test data includes two log entries. The first log entry is in the key-value pair format and contains a cross-line field named traceback. The second log entry is in the JSON format.
      time_local: 25/May/2020:01:56:22 user agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.18 (KHTML, like Gecko) Chrome/11.0.661.0 Safari/534.18 "request:method": GET ``` traceback: Traceback (most recent call last): File "traceback_print_exc.py", line 20, in <module> produce_exception() File "/home/user/code/test.py", line 16, in produce_exception produce_exception(recursion_level-1) File "/home/user/code/test.py", line 18, in produce_exception raise RuntimeError() RuntimeError ``` { "time_local": "25/May/2020:01:56:22", "user agent": "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.18 (KHTML, like Gecko) Chrome/11.0.661.0 Safari/534.18", "request:method": "GET", "remote user": "john" }
    - Example 2: The test data includes three log entries in the JSON format.
      [ { "time_local": "25/May/2020:01:56:22", "user agent": "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.18 (KHTML, like Gecko) Chrome/11.0.661.0 Safari/534.18", "request:method": "GET", "remote user": "john" }, { "time_local": "25/May/2020:01:56:22", "user agent": "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.18 (KHTML, like Gecko) Chrome/11.0.661.0 Safari/534.18", "request:method": "GET", "remote user": "john" }, { "time_local": "25/May/2020:01:56:22", "user agent": "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.18 (KHTML, like Gecko) Chrome/11.0.661.0 Safari/534.18", "request:method": "GET", "remote user": "john" } ]
  - On the Dimension Table tab, you can enter or specify dimension table data.
    You can use dimension tables to preview the resources that are accessed by resource functions. You can enter sample dimension table data for preview and debugging.
    Note
    If you enter test data on the Dimension Table tab and use the res_rds_mysql or res_log_logstore_pull function to transform the data, the data must be in the CSV format. If you use the res_oss_file or res_local function, the data must be in the CSV or JSON format.
    Example: The test data includes two log entries. The first log entry is in the CSV format, and the second log entry is in the JSON format.
```
ip,country,province
127.0.0.1,China,Shanghai
192.168.0.0,China,Zhejiang

[
  {
    "ip": "127.0.0.1",
    "country": "China",
    "province": "Shanghai"
  },
  {
    "ip": "192.168.0.0",
    "country": "China",
    "province": "Zhejiang"
  }
]
```
4. Click Preview Data.
  Note
  A maximum of 100 log entries can be returned for each data transformation test.
  After you configure the preview settings, you can preview the data transformation results on the Transformation Results tab.
  - If the data fails to be transformed because the syntax of the transformation statement or the permissions are invalid, troubleshoot the failure as prompted.
  - If the data is transformed as expected, you can save the transformation statement as a rule. For more information, see Create a data transformation job.

Quick preview example

Transformation statement

Enter the following transformation statement in the editor:

# e_set("insert_field", "test_value")
e_table_map(
    res_rds_mysql(
        address="rm-uf6wjk5****.mysql.rds.aliyuncs.com",
        username="test_username",
        password="****",
        database="test_db",
        table="test_table",
    ),
    "ip",
    ["country", "province"],
)

Test data
On the Data Testing > Data tab, enter the following content:
```
{
    "id": "1001", 
    "ip": "127.0.0.1"
}
```
Dimension table data
On the Data Testing > Dimension Table tab, enter the following content:
```
ip,country,province
127.0.0.1,China,Shanghai
192.168.0.0,China,Zhejiang
```
Preview transformation results