This topic describes the syntax and parameters of structured data functions. Structured data includes JSON data and XML data. This topic also provides examples on how to use the functions.

Functions

Category Function Description
JSON json_select Extracts or calculates specific values from a JSON expression by using JMESPath.
json_parse Parses a value into a JSON object.
XML xml_to_json Converts XML data into JSON data.

json_select

The json_select function extracts or calculates specific values from a JSON expression by using JMESPath.

  • Syntax

    json_select(value, jmes, default=None, restrict=False)

  • Parameters

    Parameter Type Required Description
    value String and JSON Yes The input JSON expression or field.
    jmes String Yes The JMESPath expression. The expression specifies the field whose value is extracted.
    default String No The value that is returned if the specified field does not exist. Default value: None, which indicates that no fields are returned.
    restrict Bool No Specifies whether to enable the restricted mode if the value of the specified field is in an invalid JSON format. Default value: False. Valid values:
    • False: The invalid format issue is ignored, and the system continues to transform data. The value of the default parameter is returned.
    • True: The invalid format issue is reported, and the system stops transforming data. The log is discarded.

  • Response

    The extracted value is returned.

  • Examples

    • Example 1: Extract the value of the name field from the content field.
      • Raw log 
        content:  {"name": "xiaoming", "age": 10}
      • Transformation rule 
        e_set("json_filter",json_select(v("content"), "name"))
      • Result
        content:  {"name": "xiaoming", "age": 10}
json_filter:  xiaoming
    • Example 2: Extract all values of the name field from the content field.
      • Raw log 
        content:  {"name": ["xiaoming", "xiaowang", "xiaoli"], "age": 10}
      • Transformation rule
        e_set("json_filter", json_select(v("content"), "name[*]"))
      • Result 
        content:  {"name": ["xiaoming", "xiaowang", "xiaoli"], "age": 10}
json_filter:  ["xiaoming", "xiaowang", "xiaoli"]
    • Example 3: Extract the value of the name3 field from the content field. If the name3 field does not exist, the value of the default parameter is returned.
      • Raw log 
        content:  {"name": "xiaoming", "age": 10}
      • Transformation rule 
        e_set("json_filter", json_select(v("content"), "name3", default="None"))
      • Result 
        content:  {"name": "xiaoming", "age": 10}
json_filter: None
    • Example 4: Extract the value of the name-test element from the content field.
      • Raw log 
        content:  {"name": {"name-test":"xiaoming"}, "age": 10}
      • Transformation rule 
        e_set("json_filter", json_select(v("content"), 'name."name-test"', default=None))
      • Result 
        content:  {"name": {"name-test":"xiaoming"}, "age": 10}
json_filter: xiaoming
    • Example 5: Extract the value of the name-test element from the content field. If the name-test element does not exist, no fields are returned.
      • Raw log 
        content:  {"name": {"name.test":"xiaoming"}, "age": 10}
      • Transformation rule 
        e_set("json_filter", json_select(v("content"), 'name."name-test"', default=None))
      • Result 
        content:  {"name": {"name-test":"xiaoming"}, "age": 10}

json_parse

The json_parse function parses a value into a JSON object.

  • Syntax

    json_parse(value, default=None, restrict=False)

  • Parameters

    Parameter Type Required Description
    value String Yes The input field.
    default String No The value that is returned if the specified field does not exist. Default value: None, which indicates that no fields are returned.
    restrict Bool No Specifies whether to enable the restricted mode if the value of the specified field is in an invalid JSON format. Default value: False. Valid values:
    • False: The invalid format issue is ignored, and the system continues to transform data. The value of the default parameter is returned.
    • True: The invalid format issue is reported, and the system stops transforming data. The log is discarded.

  • Response

    A JSON object is returned.

  • Examples

    • Example 1: Extract the JSON value of the content field.
      • Raw log 
        content:  {"abc": 123, "xyz": "test" }
      • Transformation rule 
        e_set("json", json_parse(v("content")))
      • Result
        content:  {"abc": 123, "xyz": "test" }
json:  {"abc": 123, "xyz": "test"}
    • Example 2: Extract the value of the content field. If the value is not in the JSON format, the value of the default parameter is returned.
      • Raw log 
        content: this is not json
      • Transformation rule 
        e_set("json", json_parse(v("content"), default="FFF", restrict=False))
      • Result
        content: this is not json
json: FFF

xml_to_json

The xml_to_json function converts XML data into JSON data.

  • Syntax

    xml_to_json(source)

  • Parameters

    Parameter Type Required Description
    source String Yes The input field.

  • Response

    JSON-formatted data is returned.

  • Examples

    • Raw log 
      str : <data><country name="Liechtenstein"><rank>1</rank><year>2008</year><gdppc>141100</gdppc><neighbor name="Austria" direction="E"/><neighbor name="Switzerland" direction="W"/></country><country name="Singapore"><rank>4</rank><year>2011</year><gdppc>59900</gdppc><neighbor name="Malaysia" direction="N"/></country><country name="Panama"><rank>68</rank><year>2011</year><gdppc>13600</gdppc><neighbor name="Costa Rica" direction="W"/><neighbor name="Colombia" direction="E"/></country></data>
    • Transformation rule 
      e_set("str_json",xml_to_json(v("str")))
    • Result 
      str:<data><country name="Liechtenstein"><rank>1</rank><year>2008</year><gdppc>141100</gdppc><neighbor name="Austria" direction="E"/><neighbor name="Switzerland" direction="W"/></country><country name="Singapore"><rank>4</rank><year>2011</year><gdppc>59900</gdppc><neighbor name="Malaysia" direction="N"/></country><country name="Panama"><rank>68</rank><year>2011</year><gdppc>13600</gdppc><neighbor name="Costa Rica" direction="W"/><neighbor name="Colombia" direction="E"/></country></data>
str_json:{"data": {"country": [{"@name": "Liechtenstein", "rank": "1", "year": "2008", "gdppc": "141100", "neighbor": [{"@name": "Austria", "@direction": "E"}, {"@name": "Switzerland", "@direction": "W"}]}, {"@name": "Singapore", "rank": "4", "year": "2011", "gdppc": "59900", "neighbor": {"@name": "Malaysia", "@direction": "N"}}, {"@name": "Panama", "rank": "68", "year": "2011", "gdppc": "13600", "neighbor": [{"@name": "Costa Rica", "@direction": "W"}, {"@name": "Colombia", "@direction": "E"}]}]}}