Encoding and decoding functions - Simple Log Service - Alibaba Cloud Documentation Center

This topic describes the syntax and parameters of encoding and decoding functions. This topic also provides examples on how to use the functions.

Functions

Category	Subcategory	Function	Description
Encoding and decoding	String	str_encode	Encodes data.
	String	str_decode	Decodes data.
	Base64	base64_encoding	Encodes data by using the Base64 algorithm.
	Base64	base64_decoding	Decodes data by using the Base64 algorithm.
	HTML	html_encoding	Encodes data in the HTML format.
	HTML	html_decoding	Decodes HTML-encoded data.
	URL	url_encoding	Performs URL encoding on data.
	URL	url_decoding	Performs URL decoding on data.
	Protobuf	protobuf_decoding	Parses data into the JSON format by using a specified Protobuf template.
	JSON Web Token (JWT)	jwt_encoding	Encodes JSON data based on the JWT standard.
	JSON Web Token (JWT)	jwt_decoding	Decodes data to raw JSON data based on the JWT standard.
	Hashids	hashids_encoding	Encodes data by using the Hashids library.
	Hashids	hashids_decoding	Decodes data that is encoded by using the Hashids library.
Compression and decompression	Gzip	gzip_compress	Compresses and encodes data.
	Gzip	gzip_decompress	Decompresses compressed data.
	Zlib	zlib_compress	Compresses and encodes data.
	Zlib	zlib_decompress	Decompresses compressed data.
Encryption and decryption	Advanced Encryption Standard (AES)	aes_encrypt	Encrypts data by using the AES algorithm.
Encryption and decryption	Advanced Encryption Standard (AES)	aes_decrypt	Decrypts data by using the AES algorithm.
Hash (digest)	MD5	md5_encoding	Encodes data by using the MD5 algorithm.
	SHA1	sha1_encoding	Encodes data by using the SHA1 algorithm.
	Cyclic redundancy check (CRC)	crc32_encoding	Calculates a CRC code for data.

str_encode

The str_encode function encodes a string by using a specified encoding format.

Syntax

str_encode(value, "utf8", errors="ignore")

Parameters

Parameter	Type	Required	Description
value	Arbitrary (automatically converted to the string type)	Yes	The value that you want to encode.
encoding	String	No	The encoding format. Default value: UTF-8. ASCII is supported.
errors	String	No	The method that is used to process characters if the characters cannot be recognized based on the encoding format. Valid values: ignore (default): The characters are not encoded. strict: An error is reported, and the log is discarded. replace: The unrecognizable characters are replaced with question marks (?). xmlcharrefreplace: The unrecognizable characters are replaced with XML characters.

Response
An encoded string is returned.

Examples

Example 1

Raw log
```
test: asewds
```

Transformation rule

e_set("f1", str_decode(str_encode("Hello", "utf8"), "utf8"))

Result
```
test: asewds
f1: Hello
```

Example 2

Raw log
```
f2: test Test data
```

Transformation rule

e_set("f1", str_encode(v("f2"), "ascii", errors="ignore"))

Result
```
f1:test 
f2:test Test data
```

Example 3
- Raw log
```
f2: test Test data
```
- Transformation rule
```
e_set("f1", str_encode(v("f2"), "ascii", errors="strict"))
```
- Result
  An error is reported during execution.

Example 4

Raw log
```
f2: test Test data
```

Transformation rule

e_set("f1", str_encode(v("f2"), "ascii", errors="replace"))

Result
```
f1:test ????
f2:test Test data
```

Example 5

Raw log
```
f2: test Test data
```

Transformation rule

e_set("f1", str_encode(v("f2"), "ascii", errors="xmlcharrefreplace"))

Result

f1:test &#27979;&#35797;&#25968;&#25454;
f2:test Test data

str_decode

The str_decode function decodes an input value by using a specified encoding format.

Syntax
```
str_decode(value, "utf8", errors="ignore")
```
Note
The str_decode function can process only the data of the byte data type.

Parameters

Parameter	Type	Required	Description
value	Arbitrary (automatically converted to the string type)	Yes	The value that you want to decode.
encoding	Arbitrary (automatically converted to the string type)	No	The encoding format. Default value: UTF-8. ASCII is supported.
errors	Arbitrary (automatically converted to the string type)	No	The method that is used to process characters if the characters cannot be recognized based on the encoding format. Valid values: ignore (default): The characters are not decoded. strict: An error is reported, and the log is discarded. replace: The unrecognizable characters are replaced with question marks (?). xmlcharrefreplace: The unrecognizable characters are replaced with XML characters.

Response
A decoded value is returned.

Examples

Raw log
```
test: asewds
```

Transformation rule

e_set("encoding", str_decode(b'\xe4\xbd\xa0\xe5\xa5\xbd', "utf8", 'strict'))

Result
```
test: asewds
encoding: Hello
```

base64_encoding

The base64_encoding function encodes data by using the Base64 algorithm.

Syntax
```
base64_encoding(value, format=None)
```
Parameters
Parameter Type Required Description
value String Yes The value that you want to encode.
format String No The Base64 encoding scheme. Valid values: RFC3548 and RFC4648. Default value: RFC3548.
Response
An encoded string is returned.

Examples

Raw log
```
str_en : data to be encoded
```

Transformation rule

e_set("str_base64",base64_encoding(v("str_en")))

Result

str_en : data to be encoded
str_base64 : ZGF0YSB0byBiZSBlbmNvZGVk

base64_decoding

The base64_decoding function decodes data by using the Base64 algorithm.

Syntax
```
base64_decoding(value, format=None)
```

Parameters

Parameter	Type	Required	Description
value	String	Yes	The value that you want to decode.
format	String	No	The Base64 decoding scheme. Valid values: `RFC3548` and `RFC4648`. Default value: RFC3548. Note The Base64 decoding scheme in RFC 4648 uses equal signs (=) to pad a decoded value to a multiple of 4 bytes.

Response
A decoded string is returned.

Examples

Raw log
```
str_de: ZGF0YSB0byBiZSBlbmNvZGVk
```

Transformation rule

e_set("str_de_base64",base64_decoding(v("str_de")))

Result

str_de: ZGF0YSB0byBiZSBlbmNvZGVk
str_de_base64: data to be encoded

html_encoding

The html_encoding function encodes data in the HTML format.

Syntax
```
html_encoding(value)
```
Parameters
Parameter Type Required Description
value String Yes The value that you want to encode.
Response
An encoded string is returned.

Examples

Raw log

str : <img class="size-medium wp-image-113" style="margin-left: 15px;" title="su1" src="http://aliyundoc.com/wp-content/uploads/2008/10/su1-300x194.jpg" alt="" width="300" height="194" />

Transformation rule

e_set("str_html_en",html_encoding(v("str")))

Result

str : <img class="size-medium wp-image-113" style="margin-left: 15px;" title="su1" src="http://aliyundoc.com/wp-content/uploads/2008/10/su1-300x194.jpg" alt="" width="300" height="194" />
str_html_en : &lt;img class=&quot;size-medium wp-image-113&quot; style=&quot;margin-left: 15px;&quot; title=&quot;su1&quot; src=&quot;http://aliyundoc.com/wp-content/uploads/2008/10/su1-300x194.jpg&quot; alt=&quot;&quot; width=&quot;300&quot; height=&quot;194&quot; /&gt;

html_decoding

The html_decoding function decodes HTML-encoded data.

Syntax
```
html_decoding(value)
```
Parameters
Parameter Type Required Description
value String Yes The value that you want to decode.
Response
A decoded string is returned.

Examples

Raw log

str : &lt;img class=&quot;size-medium wp-image-113&quot; style=&quot;margin-left: 15px;&quot; title=&quot;su1&quot; src=&quot;http://aliyundoc.com/wp-content/uploads/2008/10/su1-300x194.jpg&quot; alt=&quot;&quot; width=&quot;300&quot; height=&quot;194&quot; /&gt;

Transformation rule

e_set("str_html_de",html_decoding(v("str")))

Result

str : &lt;img class=&quot;size-medium wp-image-113&quot; style=&quot;margin-left: 15px;&quot; title=&quot;su1&quot; src=&quot;http://aliyundoc.com/wp-content/uploads/2008/10/su1-300x194.jpg&quot; alt=&quot;&quot; width=&quot;300&quot; height=&quot;194&quot; /&gt;
str_html_de : <img class="size-medium wp-image-113" style="margin-left: 15px;" title="su1" src="http://aliyundoc.com/wp-content/uploads/2008/10/su1-300x194.jpg" alt="" width="300" height="194" />

url_encoding

The url_encoding function performs URL encoding on data.

Syntax
```
url_encoding(value, plus=False)
```
Parameters
Parameter Type Required Description
value String Yes The value that you want to encode.
plus
Boolean
No
Specifies whether to convert spaces to plus signs (+). Default value: False. Valid values:
True
False
Response
An encoded string is returned.
Examples
- Example 1: Perform URL encoding on data.
  - Raw log
```
content : https://www.example.org/hello/asdah
```
  - Transformation rule
```
e_set("url",url_encoding(v("content")))
```
  - Result
```
content : https://www.example.org/hello/asdah
url: https%3A%2F%www.example.org%2FHello%2Fasdah
```
- Example 2: Perform URL encoding on data. Leave the plus parameter unconfigured to ensure that spaces are not converted to plus signs (+).
  - Raw log
```
content : 1 2+3:4
```
  - Transformation rule
```
e_set("URL",url_encoding(v("content")))
```
  - Result
```
content:1 2+3:4
URL:1%202%2B3%3A4
```
- Example 3: Perform URL encoding on data. Set the plus parameter to True to convert spaces to plus signs (+).
  - Raw log
```
content : 1 2+3:4
```
  - Transformation rule
```
e_set("URL", url_encoding(v("content"), plus=True))
```
  - Result
```
content:1 2+3:4
URL:1+2%2B3%3A4
```

url_decoding

The url_decoding function performs URL decoding on data.

Syntax
```
url_decoding(value, plus=False)
```
Parameters
Parameter Type Required Description
value String Yes The value that you want to decode.
plus
Boolean
No
Specifies whether to convert plus signs (+) to spaces. Default value: False. Valid values:
True
False
Response
A decoded string is returned.

Examples

Example 1: Perform URL decoding on data.

Raw log

content : https%3A%2F%www.example.org%2FHello%2Fasdah

Transformation rule
```
e_set("URL",url_decoding(v("content")))
```

Result

content : https%3A%2F%www.example.org%2FHello%2Fasdah
URL : https://www.example.org/hello/asdah

Example 2: Perform URL decoding on data. Leave the plus parameter unconfigured to ensure that plus signs (+) are not converted to spaces.

Raw log

content : /answer?event_date=2022-06-30+09%3A06%3A53%20123

Transformation rule
```
e_set("URL",url_decoding(v("content")))
```

Result

content:/answer?event_date=2022-06-30+09%3A06%3A53%20123
URL:/answer?event_date=2022-06-30+09:06:53 123

Example 3: Perform URL decoding on data. Set the plus parameter to True to convert plus signs (+) to spaces.

Raw log

content : /answer?event_date=2022-06-30+09%3A06%3A53%20123

Transformation rule

e_set("URL",url_decoding(v("content"),plus=True))

Result

content:/answer?event_date=2022-06-30+09%3A06%3A53%20123
URL:/answer?event_date=2022-06-30 09:06:53 123

protobuf_decoding

The protobuf_decoding function parses data into the JSON format by using a specified Protobuf template.

Syntax

protobuf_decoding(data, protocol, input_format="bytes", message_name=None)

Parameters

Parameter	Type	Required	Description
data	String	Yes	The Protobuf-serialized data.
protocol	String	Yes	The Protobuf template.
input_format	String	No	The data type. Valid values: hex: hexadecimal bytes: bytes raw: bytes base64: Base64-encoded
message_name	String	No	The name of the message that you want to obtain.

Response
If you do not configure message_name, the function returns a JSON list. If you configure message_name, the function returns a JSON object.

Examples

Example 1: Parse the Protobuf data that is encoded in Base64 into the JSON format by using a template.

Raw log

{
  "protocol": 'syntax = "proto3"; message Person {  string name = 1;  int32 id = 2; string email = 3;}  ',
  "data": "Cgl4aWFvIG1pbmcQARoOMTIzMTIzQDEyMy5jb20="
}

Transformation rule

e_set("data", protobuf_decoding(v("data"), v("protocol"), "base64", "Person"))

Result

data:{
  "name": "xiao ming",
  "id": 1,
  "email": "123123@123.com"
}
protocol:syntax = "proto3"; message Person {  string name = 1;  int32 id = 2; string email = 3;}

Example 2: Parse the Protobuf data into the JSON format by obtaining a template from advanced parameters.
- Raw log
```
{
    "data": "Cgl4aWFvIG1pbmcQARoOMTIzMTIzQDEyMy5jb20="
}
```
- Transformation rule
```
e_set("data",
      protobuf_decoding(
          v("data"),
          res_local("protocol"),
          "base64",
          "Person"
      )
)
```
- Advanced Parameter Settings
  In the Create Data Transformation Job panel, configure advanced parameters. Set key to protocol and value to syntax = "proto3"; message Person { string name = 1; int32 id = 2; string email = 3;}. For more information, see Create a data transformation job.
- Result
```
data:{
  "name": "xiao ming",
  "id": 1,
  "email": "123123@123.com"
}
```

Example 3: Parse the Protobuf data into the JSON format by obtaining a template from an Object Storage Service (OSS) file.

Raw log

{
    "data": "Cgl4aWFvIG1pbmcQARoOMTIzMTIzQDEyMy5jb20="
}

OSS file content

syntax = "proto3"; message Person {  string name = 1;  int32 id = 2; string email = 3;}

Transformation rule

e_set(
    "protocol",
    res_oss_file(
        endpoint="http://oss-cn-chengdu.aliyuncs.com",
        ak_id="your_ak_id",
        ak_key="your_ak_key",
        bucket="test-protobuf",
        file="test.json",
    ),
)
e_set("data", protobuf_decoding(v("data"), v("protocol"), "base64", "Person"))

Result

data:{
  "name": "xiao ming",
  "id": 1,
  "email": "123123@123.com"
}
protocol:syntax = "proto3"; message Person {  string name = 1;  int32 id = 2; string email = 3;}

jwt_encoding

The jwt_encoding function encodes JSON data based on the JWT standard.

Note JWT is an open standard that defines a compact and self-contained method to transmit information between parties as JSON objects in a secure manner. For more information, see RFC 7519. The information is digitally signed and can be verified and trusted. JWTs can be signed by using a secret and the HMAC algorithm. JWTs can also be signed by using a public/private key pair and the RSA or ECDSA algorithm. For more information, see Introduction to JSON Web Tokens.

Syntax

jwt_encoding(payload, key, algorithms="HS256", headers=None)

Parameters

Parameter	Type	Required	Description
payload	JSON	Yes	The value that you want to encode. The value must be in the JSON format. The JWT standard defines the following fields. You can also define custom fields. iss: issuer exp: expiration time sub: subject aud: audience nbf: Not Before iat: Issued At jti: JWT ID The following sample code provides an example of raw JSON data: `{ "iss": "localhost", "sub": "name", "aud": "user", "address": { "street": "street number", "city": "hangzhou", "country": "china" } }`
key	String	Yes	The key that you want to use to encrypt a JWT. The key varies based on the encryption algorithm that you select. Asymmetric encryption algorithm: PEM-encoded private key Symmetric encryption algorithm: raw key
algorithms	String	No	The algorithm that you want to use to sign a JWT. Default value: HS256. Valid values: `HS256, HS384, HS512, ES256, ES256K, ES384, ES512, RS256, RS384, RS512, PS256, PS384, PS512, and EdDSA`.
headers	JSON	No	The information about the JWT header. Default value: `{ "typ": "JWT", "alg": "HS256" }`

Response
An encoded string is returned.

Examples

Example 1: Encode the value of the data field.

Raw log
```
data: {"some": "payload"}
```

Transformation rule

e_set("jwt_token", jwt_encoding(v("data"),"secret", algorithms="HS256"))

Result

data:{"some": "payload"}
jwt_token:eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzb21lIjoicGF5bG9hZCJ9.Joh1R2dYzkRvDkqv3sygm5YyK8Gi4ShZqbhK2gxcs2U

Example 2: Use an asymmetric encryption algorithm to encode the value of the data field.

Raw log
```
data: {"some": "payload"}
```

Transformation rule

e_set(
    "jwt_token",
    jwt_encoding(
        v("data"),
        "-----BEGIN RSA PRIVATE KEY-----\nMIICYQIBAAKBgQC1iaUr5cgShCn0127+w14XN297q/IviaewIeIJsKTZF1hBPsLn\nNIPsnqtQ9DFbjIyqyZvdmQFDJCSLpXaVc648yepnFDKbOfs3r+K4Crnpo2SuZmNV\nNDVEi4pECXlBz810zJY1wqVArM7qGAyCcRLBprwXB6wfEhk3CAP3c29+pwIDAQAB\nAoGAARo65I9arbIbxx7fz7BEDAQMK0YaDGvbltg91S07cw4PPSYybNEG1BMKm01A\nV3v9BrR+u9PIDC5WAnsYwiODqEoSyk8OwO1E2kWA6+MNclYYfVjaJeiRJ5PzCud/\niUObonptRzxuTng+u1oGuX7QwUhwGJdXVBUAtJFYwXR2qVECRQC5S+6vdFESRLSX\n7yBZVM6+49lZcdehMv0HwT17UseLvWcjeSbiogvv02HbYilrW9ZydKsixAWP5w/U\nS3L34CS811VcDQI9APrOiL7c1xg5fX8wAWv2d+e+MfZoB3ohb8671W3pmp3JVnjY\ntzhoYNNQmnmRQQWf7n3J63MQz4sYYNn0gwJEE/pl37Dw1MFnn0H/AOKt79LtKkGl\n+BFhSqbBFDzWmvBu4Fo9oQ3Lr63gzSCGSrb6JhkCIptz5hIJmOARozwdeebVozkC\nPQDwOqVmU3c/P8nB6oRiGditw1Jt1yTaSW6jkOyUc73iRngqFkIgqHGd1kWwDX4/\nWfoAyEhalY6Fh5s1COsCRQCy4b3hnQws5zz/gmGNnoyxn9N+A09ySaFtn2WkUlrR\nZ6DwoJz+n6EgjLY8z6ZQyv342iobO5zKkZHFvO9QYGqk7y8lJA==\n-----END RSA PRIVATE KEY-----\n",
        algorithms="RS256",
    ),
)

Result

data:{"some": "payload"}
jwt_token:eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJzb21lIjoicGF5bG9hZCJ9.Ewwls5YPuJCmAR3XR2tcptOLrH83wVCzmUaUpGMzMLcPknRrIvbDmFGlNlQha-PMx0jsxt3t1oxpz7P3z3SR9o4qyWusAb99UG_Jn8oP8W0a5GKSy4UEJB0xgpVvJl5F2JaIPeUSHpV0VeS2WAsGSBBSAaOMkrc-8uie-H4J9M0

Example 3: Set the algorithm parameter to HS256 and configure the headers parameter to encode the value of the data field.

Raw log
```
data: {"some": "payload"}
```

Transformation rule

e_set(
    "jwt_token",
    jwt_encoding(
        v("data"),
        "secret",
        algorithms="HS256",
        headers={"kid": "230498151c214b788dd97f22b85410a5"},
    ),
)

Result

data:{"some": "payload"}
jwt_token:eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiIsImtpZCI6IjIzMDQ5ODE1MWMyMTRiNzg4ZGQ5N2YyMmI4NTQxMGE1In0.eyJzb21lIjoicGF5bG9hZCJ9.gdQ884yjlnLnIrYjfQaClE6rJC2x8v2OP2s_eXOLhZA

Example 4: Specify a custom JWT field for the data field and encode the value of the data field.

Raw log

data: {"some": "payload", "iss": 9, "sub": "name", "nbf": 123, "iat": "22"}

Transformation rule

e_set("jwt_token", jwt_encoding(v("data"),"secret"))

Result

data:{"some": "payload", "iss": 9, "sub": "name", "nbf": 123, "iat": "22"}
jwt_token:eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzb21lIjoicGF5bG9hZCIsImlzcyI6OSwic3ViIjoibmFtZSIsIm5iZiI6MTIzLCJpYXQiOiIyMiJ9.c6MxZHdXsgASchQ1Mdqj208NO_3rNGjYnvo6c7HNAyk

jwt_decoding

The jwt_decoding function decodes data to raw JSON data based on the JWT standard.

Syntax

jwt_decoding(jwt_payload, key="", algorithms=None, options=None, audience=None, issuer=None, leeway=0)

Parameters

Parameter	Type	Required	Description
jwt_payload	String	Yes	The value that you want to decode.
key	String	Yes	The key that you want to use to decrypt data. This key must be the same as the key that is used for encoding.
algorithms	List	No	The algorithm that you want to use to sign a JWT. Example: HS256.
options	JSON	No	The extended options for decoding and verification. Valid values: verify_signature: specifies whether to verify the encrypted JWT signature. Default value: True. require: specifies the fields that must exist in the encoded data. By default, this option is empty. Example: `["exp", "iat", "nbf"]`. verify_aud: specifies whether to check the aud field. Default value: True. verify_iss: specifies whether to check the iss field. Default value: True. verify_exp: specifies whether to check the exp field. Default value: True. verify_iat: specifies whether to check the iat field. Default value: True. verify_nbf: specifies whether to check the nbf field. Default value: True. Example: `{"require": ["aud"], "verify_aud": True}`.
audience	String/List	No	Checks the value of the aud field. You must configure this parameter if the aud field is specified for encoding and `"require": ["aud"]` or `"verify_aud": True` is specified for the options parameter.
issuer	String	No	Checks the value of the iss field. You must configure this parameter if the iss field is specified for encoding and `"require": ["iss"]` or `"verify_iss": True` is specified for the options parameter.
leeway	float	No	The interval at which an expiration check is performed. Unit: seconds.

Response
A decoded string is returned.

Examples

Example 1: Set the signature algorithm for decryption to HS256 to decode the value of the data field.

Raw log

data:eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzb21lIjoicGF5bG9hZCJ9.Joh1R2dYzkRvDkqv3sygm5YyK8Gi4ShZqbhK2gxcs2U

Transformation rule

e_set("data_decoded", jwt_decoding(v("data"), "secret", algorithms="HS256"))

Result
```
data_decoded: {"some": "payload"}
```

Example 2: Use a public key to decode the value of the data field.

Raw log

data: "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJzb21lIjoicGF5bG9hZCJ9.Ewwls5YPuJCmAR3XR2tcptOLrH83wVCzmUaUpGMzMLcPknRrIvbDmFGlNlQha-PMx0jsxt3t1oxpz7P3z3SR9o4qyWusAb99UG_Jn8oP8W0a5GKSy4UEJB0xgpVvJl5F2JaIPeUSHpV0VeS2WAsGSBBSAaOMkrc-8uie-H4J9M0"

Transformation rule

e_set(
    "data_decoded",
    jwt_decoding(
        v("data"),
        "-----BEGIN RSA PUBLIC KEY-----\nMIGJAoGBALWJpSvlyBKEKfTXbv7DXhc3b3ur8i+Jp7Ah4gmwpNkXWEE+wuc0g+ye\nq1D0MVuMjKrJm92ZAUMkJIuldpVzrjzJ6mcUMps5+zev4rgKuemjZK5mY1U0NUSL\nikQJeUHPzXTMljXCpUCszuoYDIJxEsGmvBcHrB8SGTcIA/dzb36nAgMBAAE=\n-----END RSA PUBLIC KEY-----\n",
        algorithms="RS256",
    ),
)

Result

data_decoded: {"some": "payload"}
data:"eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJzb21lIjoicGF5bG9hZCJ9.Ewwls5YPuJCmAR3XR2tcptOLrH83wVCzmUaUpGMzMLcPknRrIvbDmFGlNlQha-PMx0jsxt3t1oxpz7P3z3SR9o4qyWusAb99UG_Jn8oP8W0a5GKSy4UEJB0xgpVvJl5F2JaIPeUSHpV0VeS2WAsGSBBSAaOMkrc-8uie-H4J9M0"

Example 3: Use the default value of the options parameter to decode the value of the data field.

Raw log

data: "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzb21lIjoicGF5bG9hZCIsImlzcyI6OSwic3ViIjoibmFtZSIsIm5iZiI6MTIzLCJpYXQiOiIyMiJ9.DzvqhJd0PrTFk6eeASGZxOoDtrLBt_H3xC7CqOATRRw"

Transformation rule

e_set(
    "data_decoded", jwt_decoding(v("data"), "secret", algorithms="HS256", options=None)
)

Result

data_decoded:{
        "some": "payload",
        "iss": 9,
        "sub": "name",
        "nbf": 123,
        "iat": "22"
    }
data:"eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzb21lIjoicGF5bG9hZCIsImlzcyI6OSwic3ViIjoibmFtZSIsIm5iZiI6MTIzLCJpYXQiOiIyMiJ9.DzvqhJd0PrTFk6eeASGZxOoDtrLBt_H3xC7CqOATRRw"

Example 4: Specify a custom value for the options parameter to decode the value of the data field.

Raw log

data: "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzb21lIjoicGF5bG9hZCIsImlzcyI6Im5hbWUifQ.XwT9jqwofcdSP6olidbiYPC6CnZd36OEqCHZmGmooWM"

Transformation rule

e_set(
    "data_decoded",
    jwt_decoding(
        v("data"),
        "secret",
        algorithms="HS256",
        options={"require": ["iss"], "verify_iss": True},
        issuer="name",
    ),
)

Result

data_decoded:{"some": "payload", "iss": "name"}
data: "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzb21lIjoicGF5bG9hZCIsImlzcyI6Im5hbWUifQ.XwT9jqwofcdSP6olidbiYPC6CnZd36OEqCHZmGmooWM"

hashids_encoding

The hashids_encoding function encodes data by using the Hashids library.

Syntax

hashids_encoding(value, salt="", min_length=0)

Parameters

Parameter	Type	Required	Description
value	Number, list, or tuple	Yes	The value that you want to encode. If you want to specify a number for this parameter, you must specify a number that is no less than 0.
salt	String	No	The salt that you want to add to the value. Default value: an empty string.
min_length	Number	No	The minimum length of the hash value that is generated. Default value: 0.

Response
An encoded string is returned.
Examples
- Example 1: Use the default salt and minimum hash value length to encode a number.
  - Raw log
```
content:test
```
  - Transformation rule
```
e_set("hashid", hashids_encoding(123))
```
  - Result
```
content:test
hashid:Mj3
```
- Example 2: Use a custom salt and minimum hash value length to encode list data.
  - Raw log
```
content:test
```
  - Transformation rule
```
e_set("hashid", hashids_encoding([123, 456], salt="test"))
```
  - Result
```
content:test
hashid:bpBHYO
```
- Example 3: Use a custom salt and minimum hash value length to encode list data.
  - Raw log
```
content:test
```
  - Transformation rule
```
e_set("hashid", hashids_encoding([123, 456], salt="test", min_length=32))
```
  - Result
```
content:test
hashid:O6jN0Z7VARqDzbpBHYOakeowE5Xnr41g
```
- Example 4: Use a custom salt and minimum hash value length to encode tuple data.
  - Raw log
```
content:test
```
  - Transformation rule
```
e_set("hashid", hashids_encoding((123, 456), salt="test", min_length=32))
```
  - Result
```
content:test
hashid:O6jN0Z7VARqDzbpBHYOakeowE5Xnr41g
```

hashids_decoding

The hashids_decoding function decodes data that is encoded by using the Hashids library.

Syntax

hashids_decoding(hashid, salt="", min_length=0)

Parameters

Parameter	Type	Required	Description
hashid	String	Yes	The value that you want to decode.
salt	String	No	The salt that you want to add to the value. Default value: an empty string.
min_length	Number	No	The minimum length of the hash value that is generated. Default value: 0.

Response
A decoded string is returned.
Examples
- Example 1: Use the default salt and minimum hash value length to decode a number that is encoded.
  - Raw log
```
content:Mj3
```
  - Transformation rule
```
e_set("value", hashids_decoding(v("content")))
```
  - Result
```
content:Mj3
value:[123]
```
- Example 2: Use a custom salt and the default minimum hash value length to decode list data that is encoded.
  - Raw log
```
content:bpBHYO
```
  - Transformation rule
```
e_set("value", hashids_decoding(v("content"), salt="test"))
```
  - Result
```
content:bpBHYO
value:[123, 456]
```
- Example 3: Use a custom salt and minimum hash value length to decode list data that is encoded.
  - Raw log
```
content:O6jN0Z7VARqDzbpBHYOakeowE5Xnr41g
```
  - Transformation rule
```
e_set("value", hashids_decoding(v("content"), salt="test", min_length=32))
```
  - Result
```
content:O6jN0Z7VARqDzbpBHYOakeowE5Xnr41g
value:[123, 456]
```
- Example 4: Use a custom salt and minimum hash value length to decode tuple data that is encoded.
  - Raw log
```
content:O6jN0Z7VARqDzbpBHYOakeowE5Xnr41g
```
  - Transformation rule
```
e_set("value", hashids_decoding(v("content"), salt="test", min_length=32))
```
  - Result
```
content:O6jN0Z7VARqDzbpBHYOakeowE5Xnr41g
value:[123, 456]
```

gzip_compress

The gzip_compress function compresses and encodes data.

Syntax

gzip_compress(data, compresslevel=6, to_format="base64", encoding="utf-8")

Parameters

Parameter	Type	Required	Description
data	String	Yes	The data that you want to compress.
compresslevel	Int	No	The compression level. Valid values: 0 to 9. Default value: 6. 1: The compression speed is the highest, and the compression ratio is the lowest. 9: The compression speed is the lowest, and the compression ratio is the highest. 0: Data is not compressed.
to_format	String	No	The encoding format for the compressed data. Valid values: base64 and hex.
encoding	String	No	The encoding format for the raw data. Default value: utf-8. For more information about other encoding formats, see Standard encoding formats.

Response
An encoded string is returned.

Examples

Example 1: Compress a log field and encode the field by using the Base64 algorithm.

Raw log

content: I always look forward to my holidays whether I travel or stay at home.

Transformation rule

e_set("base64_encode_gzip_compress",gzip_compress(v("content"),to_format="base64"))

Result

content: I always look forward to my holidays whether I travel or stay at home.
base64_encode_gzip_compress: H4sIAA8JXl4C/xXK0QmAMAwFwFXeBO7RMQKNREx5kAZDtle/7wbES3rDyRsnoyQmklgNo1/ztzJN08BAhjzqYGCnNCS/tPR4AcgrnWVGAAAA

Example 2: Encode a log field by using the HEX encoding format.

Raw log

content: H4sIAA8JXl4C/xXK0QmAMAwFwFXeBO7RMQKNREx5kAZDtle/7wbES3rDyRsnoyQmklgNo1/ztzJN08BAhjzqYGCnNCS/tPR4AcgrnWVGAAAA

Transformation rule

e_set("hex_encode_gzip_compress", gzip_compress(v("content"), to_format="hex"))

Result

content:H4sIAA8JXl4C/xXK0QmAMAwFwFXeBO7RMQKNREx5kAZDtle/7wbES3rDyRsnoyQmklgNo1/ztzJN08BAhjzqYGCnNCS/tPR4AcgrnWVGAAAA
hex_encode_gzip_compress:1f8b08004a478c6202ff0dc1dd0e43301800d047aa65156e3ff52f4a2ba17649c43255194d4a9f9e73527c64007e2e2426e81485c35628c1c42616535079bc6405e5d1e92ef009b59c906786a879efe1c50fb55d6c5de44cb717b2dae6d4f103f8feecbf4f88a2a441bae618c679575d9bc0e306907876806c000000

gzip_decompress

The gzip_decompress function decompresses compressed data.

Syntax

gzip_decompress(data, from_format="base64", encoding="utf-8")

Parameters

Parameter	Type	Required	Description
data	Arbitrary	Yes	The data that you want to decompress.
from_format	String	No	The encoding format for the compressed data. Valid values: base64 and hex.
encoding	String	No	The encoding format for the raw data. Default value: utf-8. For more information about other encoding formats, see Standard encoding formats.

Response
Decompressed data is returned.

Examples

Example 1: Decode a log field by using the Base64 algorithm.

Raw log

content: H4sIAA8JXl4C/xXK0QmAMAwFwFXeBO7RMQKNREx5kAZDtle/7wbES3rDyRsnoyQmklgNo1/ztzJN08BAhjzqYGCnNCS/tPR4AcgrnWVGAAAA

Transformation rule

e_set("gzip_decompress",gzip_decompress(v("content"),from_format="base64"))

Result

content: H4sIAA8JXl4C/xXK0QmAMAwFwFXeBO7RMQKNREx5kAZDtle/7wbES3rDyRsnoyQmklgNo1/ztzJN08BAhjzqYGCnNCS/tPR4AcgrnWVGAAAA
gzip_decompress: I always look forward to my holidays whether I travel or stay at home.

Example 2: Decode a log field by using the HEX decoding format.

Raw log

content:1f8b0800bff8856202ff0dc1dd0e43301800d047aa65156e3ff52f4a2ba17649c43255194d4a9f9e73527c64007e2e2426e81485c35628c1c42616535079bc6405e5d1e92ef009b59c906786a879efe1c50fb55d6c5de44cb717b2dae6d4f103f8feecbf4f88a2a441bae618c679575d9bc0e306907876806c000000

Transformation rule

e_set("gzip_decompress", gzip_decompress(v("content"), from_format="hex"))

Result

content:1f8b0800bff8856202ff0dc1dd0e43301800d047aa65156e3ff52f4a2ba17649c43255194d4a9f9e73527c64007e2e2426e81485c35628c1c42616535079bc6405e5d1e92ef009b59c906786a879efe1c50fb55d6c5de44cb717b2dae6d4f103f8feecbf4f88a2a441bae618c679575d9bc0e306907876806c000000
gzip_decompress:H4sIAA8JXl4C/xXK0QmAMAwFwFXeBO7RMQKNREx5kAZDtle/7wbES3rDyRsnoyQmklgNo1/ztzJN08BAhjzqYGCnNCS/tPR4AcgrnWVGAAAA

zlib_compress

The zlib_compress function compresses and encodes data.

Syntax

zlib_compress(data, compresslevel=6, to_format="base64", encoding="utf-8")

Parameters

Parameter	Type	Required	Description
data	String	Yes	The data that you want to compress.
compresslevel	Int	No	The compression level. Valid values: 0 to 9. Default value: 6. 1: The compression speed is the highest, and the compression ratio is the lowest. 9: The compression speed is the lowest, and the compression ratio is the highest. 0: Data is not compressed.
to_format	String	No	The encoding format for the compressed data. Set the value to base64.
encoding	String	No	The encoding format for the raw data. Default value: utf-8. For more information about other encoding formats, see Standard encoding formats.

Response
Encoded data is returned.

Examples

Raw log

content: I always look forward to my holidays whether I travel or stay at home.

Transformation rule

e_set("zlib_compress", zlib_compress(v("content"), to_format="base64"))

Result

zlib_compress: "eJwVytEJgDAMBcBV3gTu0TECjURMeZAGQ7ZXv+8GxEt6w8kbJ6MkJpJYDaNf87cyTdPAQIY86mBgpzQkv7T0eAGNshln"
content: "I always look forward to my holidays whether I travel or stay at home."

zlib_decompress

The zlib_decompress function decompresses compressed data.

Syntax

zlib_decompress(data, from_format="base64", encoding="utf-8")

Parameters

Parameter	Type	Required	Description
data	String	Yes	The data that you want to decompress.
from_format	String	No	The encoding format for the compressed data. Set the value to base64.
encoding	String	No	The encoding format. Default value: utf-8. For more information about other encoding formats, see Standard encoding formats.

Response
Decompressed data is returned.

Examples

Raw log

content: "eJwVytEJgDAMBcBV3gTu0TECjURMeZAGQ7ZXv+8GxEt6w8kbJ6MkJpJYDaNf87cyTdPAQIY86mBgpzQkv7T0eAGNshln"

Transformation rule

e_set("zlib_decompress", zlib_decompress(v("content"), from_format="base64"))

Result

content: "eJwVytEJgDAMBcBV3gTu0TECjURMeZAGQ7ZXv+8GxEt6w8kbJ6MkJpJYDaNf87cyTdPAQIY86mBgpzQkv7T0eAGNshln"
zlib_decompress: "I always look forward to my holidays whether I travel or stay at home."

aes_encrypt

The aes_encrypt function encrypts data by using the AES algorithm. The AES algorithm is a common symmetric encryption algorithm that is used. To improve data security, you can use the AES algorithm to encrypt data.

Syntax

aes_encrypt(data, key, mode, pad_style, pad_block, input_format, input_encoding, output_format, iv)

Parameters

Parameter	Type	Required	Description
data	String	Yes	The data that you want to encrypt.
key	String	Yes	The key that you want to use to encrypt data.
mode	String	No	The AES encryption mode. CBC: Cipher Block Chaining (default) ECB: Electronic Code Book CFB: Cipher Feedback OFB: Output Feedback CTR: Counter OPENPGP
pad_style	String	No	The padding mode. Default value: pkcs7. Valid values: iso7816, x923, and pkcs7.
input_format	String	No	The format of input characters. Default value: raw. Valid values: raw: bytes hex: hexadecimal base64: Base64 encoded
input_encoding	String	No	The encoding format for input characters. This parameter is required only if you set the input_format parameter to raw. Default value: utf-8.
output_format	String	No	The format of output characters. Default value: hex. Valid values: raw: bytes hex: hexadecimal base64: Base64 encoded
iv	Bytes	No	The offset that you want to use for encryption.

Response
An encrypted string is returned.

Examples

Example 1

Raw log
```
"test": "aliyuntest"
```

Transformation rule

e_set('result',aes_encrypt(v("test"), "qwertyuiopasdfgd", iv=b"xxywosjdapdiawdk", output_format="base64"))

Result
```
"result": "gXIqu0cBBtZHQxJBK8GLeA=="
```

Example 2

Raw log
```
"test": "aliyuntest"
```

Transformation rule

e_set('result',aes_encrypt(v("test"), "qwertyuiopasdfgh", iv=b"ywisnjaduaqibdqi", mode="OFB"))

Result
```
"result":  "5cac3e9e1c42f713dc6d"
```

aes_decrypt

The aes_decrypt function decrypts data by using the AES algorithm.

Syntax

aes_decrypt(data, key, mode, pad_style, input_format, input_encoding, output_format, iv, output_encoding)

Parameters

Parameter	Type	Required	Description
data	String	Yes	The data that you want to decrypt.
key	String	Yes	The key that you want to use to decrypt data.
mode	String	No	The AES decryption mode. CBC: Cipher Block Chaining (default) ECB: Electronic Code Book CFB: Cipher Feedback OFB: Output Feedback CTR: Counter OPENPGP
pad_style	String	No	The padding mode. Default value: pkcs7. Valid values: iso7816, x923, and pkcs7.
input_format	String	No	The format of input characters. Default value: hex. Valid values: raw: bytes hex: hexadecimal base64: Base64 encoded
input_encoding	String	No	The encoding format for input characters. This parameter is required only if you set the input_format parameter to raw. Default value: utf-8.
output_format	String	No	The format of output characters. Default value: raw. Valid values: raw: bytes hex: hexadecimal base64: Base64 encoded
iv	Bytes	No	The offset that you want to use for decryption.
output_encoding	String	No	The encoding format for output characters. Default value: None.

Response
A decrypted string is returned.

Examples

Example 1

Raw log
```
"test": "gXIqu0cBBtZHQxJBK8GLeA=="
```

Transformation rule

e_set('result', aes_decrypt(v("test"), "qwertyuiopasdfgd", iv=b"xxywosjdapdiawdk", input_format="base64"))

Result
```
"result": "aliyuntest"
```

Example 2

Raw log
```
"test": "5cac3e9e1c42f713dc6d"
```

Transformation rule

   e_set('result', aes_decrypt(v("test"), "qwertyuiopasdfgh", iv=b"ywisnjaduaqibdqi", mode="OFB"))

Result
```
"result": "aliyuntest"
```

md5_encoding

The md5_encoding function encodes data by using the MD5 algorithm.

Syntax
```
md5_encoding(value, format="hex")
```
Parameters
Parameter Type Required Description
value String Yes The value that you want to encode.
format String No Valid values: binary and hex. Default value: hex.
Response
An encoded string is returned.

Examples

Example 1

Raw log
```
str : GeeksforGeeks
```

Transformation rule

e_set("str_md5_en",md5_encoding(v("str")))

Result

str : GeeksforGeeks
str_md5_en : f1e069787ece74531d112559945c6871

Example 2

Raw log
```
str : GeeksforGeeks
```

Transformation rule

e_set("str_md5_en",base64_encoding(md5_encoding(v("str"), format="binary")))

Result

str : GeeksforGeeks
str_md5_en : 8eBpeH7OdFMdESVZlFxocQ==

sha1_encoding

The sha1_encoding function encodes data by using the SHA1 algorithm.

Syntax
```
sha1_encoding(value, format=None)
```
Parameters
Parameter Type Required Description
value String Yes The value that you want to encode.
format String No The encoding type. Valid values: SHA1, SHA224, SHA256, SHA384, and SHA512. Default value: SHA1.
Response
An encoded string is returned.

Examples

Raw log
```
str : GeeksforGeeks
```

Transformation rule

e_set("str_sha1",sha1_encoding(v("str")))
e_set("str_sha512",sha1_encoding(v("str"),format='SHA512'))
e_set("str_sha224",sha1_encoding(v("str"),format='SHA224'))
e_set("str_sha384",sha1_encoding(v("str"),format='SHA384'))
e_set("str_sha256",sha1_encoding(v("str"),format='SHA256'))

Result

str : GeeksforGeeks
str_sha1 : 4175a37afd561152fb60c305d4fa6026b7e79856
str_sha512 : 0d8fb9370a5bf7b892be4865cdf8b658a82209624e33ed71cae353b0df254a75db63d1baa35ad99f26f1b399c31f3c666a7fc67ecef3bdcdb7d60e8ada90b722
str_sha224 : 173994f309f727ca939bb185086cd7b36e66141c9e52ba0bdcfd145d
str_sha384 : d1e67b8819b009ec7929933b6fc1928dd64b5df31bcde6381b9d3f90488d253240490460c0a5a1a873da8236c12ef9b3
str_sha256 : f6071725e7ddeb434fb6b32b8ec4a2b14dd7db0d785347b2fb48f9975126178f

crc32_encoding

The crc32_encoding function calculates a CRC code for data.

Syntax

crc32_encoding(data, input_format="raw", input_encoding="utf-8")

Parameters

Parameter	Type	Required	Description
data	String	Yes	The value for which you want to calculate a CRC code.
input_format	String	No	The format of input characters. Default value: raw. Valid values: raw: bytes hex: hexadecimal base64: Base64 encoded
input_encoding	String	No	The encoding format for input characters. This parameter is required only if you set the input_format parameter to raw. Default value: utf-8.

Response
The CRC code for the input value is returned.

Examples

Example 1: Calculate a CRC code for the test field.

Raw log
```
test: aliyuntest
```

Transformation rule

e_set("str_crc32", crc32_encoding(v("test")))

Result
```
str_crc32:1434103726
test:aliyuntest
```

Example 2: Join the test1 field and test2 field, encode the fields by using the MD5 algorithm, and calculate a CRC code for the fields.

Raw log
```
test1: test1
test2: test2
```

Transformation rule

e_set(
    "str_crc32",
    crc32_encoding(
        md5_encoding(str_join("+", v("test1"), v("test2")), format="binary")
    ),
)

Result

str_crc32:369733261
test1:test1
test2:test2

Example 3: Calculate a CRC code for the test field. The field value is Base64-encoded.

Raw log
```
test: Taloz+e+PzP3NltrEXiCig==
```

Transformation rule

e_set("str_crc32", crc32_encoding(v("test"), input_format="base64"))

Result

str_crc32:1093789404
test:Taloz+e+PzP3NltrEXiCig==

Functions

str_encode

Syntax

Parameters

Response

Examples

str_decode

Syntax

Parameters

Response

Examples

base64_encoding

Syntax

Parameters

Response

Examples

base64_decoding

Syntax

Parameters

Response

Examples

html_encoding

Syntax

Parameters

Response

Examples

html_decoding

Syntax

Parameters

Response

Examples

url_encoding

Syntax

Parameters

Response

Examples

url_decoding

Syntax

Parameters

Response

Examples

protobuf_decoding

Syntax

Parameters

Response

Examples

jwt_encoding

Syntax

Parameters

Response

Examples

jwt_decoding

Syntax

Parameters

Response

Examples

hashids_encoding

Syntax

Parameters

Response

Examples

hashids_decoding

Syntax

Parameters

Response

Examples

gzip_compress

Syntax

Parameters

Response

Examples

gzip_decompress

Syntax

Parameters

Response

Examples

zlib_compress

Syntax

Parameters

Response