Function Compute:Use Loggie in custom runtimes

Prerequisites

A project and a Logstore are created in Simple Log Service. For more information, see Create a project and Create a Logstore.

Procedure

Step 1: Create a function

1. Log on to the Function Compute console. In the left-side navigation pane, click Functions.

2. In the top navigation bar, select a region. On the Functions page, click Create Function.

3. On the Create Function page that appear, configure the following parameters, retain the default values for other parameters, and then click Create. For more information, see Create a function.

   • Method to create the function: Web Function

   • Basic Settings: Configure Function Name.

   • Code: Configure the runtime and code-related information of the function.

     Parameter	Example
     Sample Code	Python 3.9
     Code Upload Method	Select Upload Folder. The name of the folder to upload is code. The app.py file is stored in the code directory. The following code snippet provides sample code of app.py: from flask import Flask from flask import request import logging import os REQUEST_ID_HEADER = 'x-fc-request-id' app = Flask(__name__) format_str = '[%(asctime)s] %(levelname)s in %(module)s: %(message)s' logging.basicConfig(filename='/tmp/log/fc-flask.log', filemode='w', format=format_str, encoding='utf-8', level=logging.DEBUG) @app.route("/invoke", methods = ["POST"]) def hello_world(): rid = request.headers.get(REQUEST_ID_HEADER) logger = logging.getLogger() print("FC Invoke Start RequestId: " + rid) logger.info("FC Invoke Start RequestId: " + rid) data = request.stream.read() print(str(data)) logger.info("receive event: {}".format(str(data))) print("FC Invoke End RequestId: " + rid) logger.info("FC Invoke Start RequestId: " + rid) return "Hello, World!" if __name__ == '__main__': app.run(host='0.0.0.0',port=9000) Note You can change filename='/tmp/log/fc-flask.log' to the actual log type and log location in the code. Make sure that the configuration is consistent with the value of sources.paths configured in Step 2.
     Startup Command	/code/bootstrap Note The bootstrap file is created in Step 2.
     Listening Port	9000

Step 2: Create a bootstrap file to serve as the startup command

1. After the function is created, use WebIDE on the Code tab to create a bootstrap file in the code directory.

   The following sample code shows an example bootstrap file.

   #!/bin/bash
   
   #1. Create the pipelines.yml file.
   mkdir -p /tmp/log /code/etc
   cat << EOF > /code/etc/pipelines.yml
   pipelines:
     - name: demo
       sources:
         - type: file
           name: fc-demo
           addonMeta: true
           fields:
             topic: "loggie"
           fieldsUnderRoot: true
           paths:
             - "/tmp/log/*.log"
       sink:
         type: sls
         endpoint: ${LOGGIE_SINK_SLS_ENDPOINT}
         accessKeyId: ${LOGGIE_SINK_SLS_ACCESS_ID}
         accessKeySecret: ${LOGGIE_SINK_SLS_ACCESS_SECRET}
         project: ${LOGGIE_SINK_SLS_PROJECT}
         logstore: ${LOGGIE_SINK_SLS_LOGSTORE}
         topic: ${LOGGIE_SINK_SLS_TOPIC}
   EOF
   
   #2. Create the loggie.yml file.
   cat << EOF > /code/etc/loggie.yml
   EOF
   
   #3. Start Loggie and run it as a background process.
   /opt/bin/loggie -config.system=/code/etc/loggie.yml -config.pipeline=/code/etc/pipelines.yml > /tmp/loggie.log 2>&1 &
   
   #4. Start the application.
   exec python app.py

   The script performs the following operations:

   1. Create the pipelines.yml file, which is the pipeline configuration file.

      • sources

        Specifies the type and path of logs. This example shows how to collect logs from all files that end with .log in the /tmp/log directory.

        addonMeta in sources specifies to add the default state metadata of log collection. For more information about sources, see Overview.

      • sink

        Specifies information about Simple Log Service. The variables in the script are set in Step 4.

   2. Create the loggie.yml file, which is the Loggie configuration file.

      If this file is empty, the default configurations are used. In this example, the default configurations are used. However, the loggie.yml file must exist. If the file is not empty, see Overview for the detailed configurations.

   3. Start Loggie and run it as a background process. Running logs of Loggie are printed to the /tmp/loggie.log file.

   4. Start the application. In this example, a Python runtime is used. The actual runtime that you use shall prevail.

2. Configure the executable permissions on the bootstrap file.

   In WebIDE, choose Terminal > New Terminal and run the chmod 777 bootstrap command to configure the permissions.

3. Click Deploy to deploy the code.

Step 3: Add the Loggie Agent common layer

Step 4: Configure environment variables

Step 4: Verify results

Troubleshooting

Loggie runs independently in a function instance. Function Compute cannot detect whether the Loggie is normal. Execution of functions is not affected even if Loggie is abnormal.

When you query logs that are related to Loggie in Simple Log Service, latency of seceral seconds may occur. If you cannot find logs that are related to Loggie in Simple Log Service, perform the following operations to troubleshoot the issue:

Function runs as expected

In this case, the instance is alive for several minutes after the function is invoked. You can log on to the instance to view the running status and logs of Loggie. For more information about how to log on to an instance, see CLI-based instance management.

• If logs do not exist, you can start Loggie in the command line.

• If logs exist, troubleshoot the issue based on the logs.

  • Check whether the pipelines.yml file is correctly configured.

  • Check whether the Simple Log Service sink is successfully started. Logs are similar to pipeline sink(sink/sls)-0 invoke loop start.

  • Check whether the log file is obtained. Logs are similar to start collect file: /tmp/log/fc-flask.log. If no similar log is available, check whether a log file is generated based on the paths path in the pipelines.yml file configuration.

Function fails to run

If a function fails to run, you can remove the startup logic from Loggie and check whether the function is running as expected. In most cases, Loggie does not affect the running of functions because Loggie is an external extension. If unexpected exits of processes or execution timeouts occur, you can scale up the memory or increase the CPU specifications.

References

• For more information about Loggie, see Loggie.

• In this example, Loggie collects logs and uploads them as is, without any processing. If you want to process logs before you upload them, such as parsing JSON format logs or removing DEBUG logs, you can add Interceptor configurations in the pipelines.yml file. For more information, see Overview.