Use the Lua plug-in to process and forward network requests - Microservices Engine

Lua is a lightweight and highly efficient scripting language. In gateway development, Lua can be used to write and execute various gateway programs, such as API gateways, message gateways, and reverse proxies. Developers can use Lua to write scripts to implement features such as request routing, filtering, and authentication, and perform customized processing. Lua can be embedded into some proxies, such as NGINX and Envoy for request and response processing, log output, and other customized operations. In this topic, Lua is embedded into the Envoy proxy for request and response processing. This topic describes how to use Lua to print headers and bodies of requests and responses in logs.

Limits

The version of your Microservices Engine (MSE) cloud-native gateway must be 1.2.11 or later.

Usage notes

For security purposes, the following Lua libraries and functions are disabled for MSE cloud-native gateways by default.

debug.debug
debug.getfenv
debug.getregistry
dofile
io
loadfile
os.execute
os.getenv
os.remove
os.rename
os.tmpname

Procedure

Log on to the MSE console, and select a region in the top navigation bar.
In the left-side navigation pane, choose Cloud-native Gateway > Gateways. On the Gateways page, click the name of the gateway.
In the left-side navigation pane of the gateway details page, click Plug-in Marketplace.
On the All Plug-ins tab, click the Custom sub-tab and click the resource card of lua.
On the Plug-in Configuration tab, configure the parameters.
1. In the Application Scope section, select the application scope of the Lua plug-in.
2. In the Plug-in Configuration section, click Edit, enter the Lua code in the code editor, and then click Save.
3. In the Status section, turn on the switch. In the Enable message, click OK.
  The Lua plug-in is enabled.

API references

For more information about the Lua APIs that are provided by cloud-native gateways, see Lua.

If you want to use JSON, you can use the following methods:

Introduce the JSON package: local json = require "json".
Serialize JSON data: local json_str = json.encode(obj).
Deserialize JSON data: local obj = json.decode(json_str).

If an error occurs during the serialization or deserialization, Lua calls the error function to return an error and terminates the current process.

Common use cases

Display complete request and response information in plug-in logs

Log on to the MSE console. In the top navigation bar, select a region.
In the left-side navigation pane, choose Cloud-native Gateway > Gateways. On the Gateways page, click the name of the gateway.
In the left-side navigation pane, click Plug-in Marketplace. On the page that appears, click the desired plug-in.

Click the Plug-in Configuration tab. In the Plug-in Configuration section, enter the following Lua code.

Note

In the sample code, only the following request bodies and response bodies of the content type are displayed. The body size cannot exceed 1,024 bytes (1 KB).

application/x-www-form-urlencoded
application/json
text/plain

local maxBodySize = 1024

function check_content_readable(type)
  if type == nil then
    return false
  end
  if string.find(type, "application/x-www-form-urlencoded",1,true) or string.find(type, "application/json",1,true) or string.find(type, "text/plain",1,true) then
     return true
  end
  return false
end

function envoy_on_request(request_handle)
  local headers = request_handle:headers()
  local headersStr = ""
  local contentType
  for key, value in pairs(headers) do
    if key == "content-type" then
       contentType = value
    end
    headersStr = headersStr  .. key .. "=" .. value .. ", "
  end
  request_handle:streamInfo():dynamicMetadata():set("envoy.lua","request_headers",headersStr)
  local requestBody = ""
  if check_content_readable(contentType) then
    for chunk in request_handle:bodyChunks() do
      if (chunk:length() > 0) then
        requestBody = requestBody .. chunk:getBytes(0, chunk:length())
      end
      if (#requestBody > maxBodySize) then
         requestBody = requestBody .. "<truncated>"
         break
      end
    end
  end
  request_handle:streamInfo():dynamicMetadata():set("envoy.lua","request_body",string.gsub(requestBody,"\n","\\n"))
end

function envoy_on_response(response_handle)
  local headers = response_handle:headers()
  local headersStr = ""
  local contentType
  local contentEncoding = false
  for key, value in pairs(headers) do
    if key == "content-type" then
       contentType = value
    elseif key == "content-encoding" then
       contentEncoding = true
    end
    headersStr = headersStr .. key .. "=" .. value .. ", "
  end
  local responseBody = ""
  if check_content_readable(contentType) and not contentEncoding then
    for chunk in response_handle:bodyChunks() do
      if (chunk:length() > 0) then
        responseBody = responseBody .. chunk:getBytes(0, chunk:length())
      end
      if (#responseBody > maxBodySize) then
         responseBody = responseBody .. "<truncated>"
         break
      end
    end
  end
  local reqHeaders = ""
  local reqBody = ""
  local metadata = response_handle:streamInfo():dynamicMetadata():get("envoy.lua")
  if metadata ~= nil then
    local headers = response_handle:streamInfo():dynamicMetadata():get("envoy.lua")["request_headers"]
    if headers ~= nil then
      reqHeaders = headers
    end
    local body = response_handle:streamInfo():dynamicMetadata():get("envoy.lua")["request_body"]
    if body ~= nil then
      reqBody = body
    end
  end
  response_handle:logInfo("request Headers: [" .. reqHeaders .. "] request Body: [" .. string.gsub(reqBody,"\n","\\n") .. "] response Headers: [" .. headersStr .. "] response Body: [" .. string.gsub(responseBody,"\n","\\n") .. "]")
end

On the Plug-in Logs tab, click Log Shipping Settings. In the dialog box that appears, turn on the switch and click OK.
After you enable Log Shipping Settings, the plug-in logs are shipped to Simple Log Service and can be viewed in the console.
As shown in the following figure, you can use request-id in the access log to find the complete request and response information in the Lua plug-in log.

Add complete request and response information to access logs

Log on to the MSE console. In the top navigation bar, select a region.
In the left-side navigation pane, choose Cloud-native Gateway > Gateways. On the Gateways page, click the name of the gateway.
In the left-side navigation pane, click Plug-in Marketplace. On the page that appears, click the desired plug-in.

Click the Plug-in Configuration tab. In the Plug-in Configuration section, enter the following Lua code.

You can configure custom dynamic metadata for the access log parameters of a gateway. In this case, you must define the metadata in the plug-in. The following sample code defines four types of metadata information, which correspond to the request header, request body, response header, and response body.

envoy.lua:request_headers
envoy.lua:request_body
envoy.lua:response_headers
envoy.lua:response_body

local maxBodySize = 1024

function check_content_readable(type)
  if type == nil then
    return false
  end
  if string.find(type, "application/x-www-form-urlencoded",1,true) or string.find(type, "application/json",1,true) or string.find(type, "text/plain",1,true) then
     return true
  end
  return false
end

function envoy_on_request(request_handle)
  local headers = request_handle:headers()
  local headersStr = ""
  local contentType
  for key, value in pairs(headers) do
    if key == "content-type" then
       contentType = value
    end
    headersStr = headersStr  .. key .. "=" .. value .. ", "
  end
  request_handle:streamInfo():dynamicMetadata():set("envoy.lua","request_headers",headersStr)
  local requestBody = ""
  if check_content_readable(contentType) then
    for chunk in request_handle:bodyChunks() do
      if (chunk:length() > 0) then
        requestBody = requestBody .. chunk:getBytes(0, chunk:length())
      end
      if (#requestBody > maxBodySize) then
         requestBody = requestBody .. "<truncated>"
         break
      end
    end
  end
  request_handle:streamInfo():dynamicMetadata():set("envoy.lua","request_body",string.gsub(requestBody,"\n","\\n"))
end

function envoy_on_response(response_handle)
  local headers = response_handle:headers()
  local headersStr = ""
  local contentType
  local contentEncoding = false
  for key, value in pairs(headers) do
    if key == "content-type" then
       contentType = value
    elseif key == "content-encoding" then
       contentEncoding = true
    end
    headersStr = headersStr .. key .. "=" .. value .. ", "
  end
  response_handle:streamInfo():dynamicMetadata():set("envoy.lua","response_headers",headersStr)
  local responseBody = ""
  if check_content_readable(contentType) and not contentEncoding then
    for chunk in response_handle:bodyChunks() do
      if (chunk:length() > 0) then
        responseBody = responseBody .. chunk:getBytes(0, chunk:length())
      end
      if (#responseBody > maxBodySize) then
         responseBody = responseBody .. "<truncated>"
         break
      end
    end
  end
  response_handle:streamInfo():dynamicMetadata():set("envoy.lua","response_body",string.gsub(responseBody,"\n","\\n"))
end

In the left-side navigation pane, click Parameter Settings. In the Observability Parameters section, modify the log format. Add the metadata to the custom log fields. Then, you can view the required information in access logs.