Function Compute:Use Loggie in custom runtimes

Before you start

• Function Compute: Create a service.

• Simple Log Service: Create a project and Create a Logstore.

  Note

  The log project that you create must be in the same region as the function that you want to create in Step 1.

Procedure

Step 1: Create a function in a custom runtime

1. Log on to the Function Compute console. In the left-side navigation pane, click Services & Functions.

2. In the top navigation bar, select a region. On the Services page, click the desired service.

3. On the Functions page, click Create Function.

4. On the Create Function page, configure the following parameters, use the default values for other parameters, and then click Create. For more information, see Create a function.

   • Creation method: Select Use Custom Runtime.

   • Basic Settings: Enter a function name in the Function Name field and select Event Handler for Handler Type.

   • Code: Configure the runtime environment and code-related information of the function.

     Parameter	Example
     Runtime	Python 3.9
     Code Upload Method	Select Use Folder. The name of the folder to be uploaded is code, and the file in the code directory is app.py. The following sample code shows the content of app.py: from flask import Flask from flask import request import logging import os REQUEST_ID_HEADER = 'x-fc-request-id' app = Flask(__name__) format_str = '[%(asctime)s] %(levelname)s in %(module)s: %(message)s' logging.basicConfig(filename='/tmp/log/fc-flask.log', filemode='w', format=format_str, encoding='utf-8', level=logging.DEBUG) @app.route("/invoke", methods = ["POST"]) def hello_world(): rid = request.headers.get(REQUEST_ID_HEADER) logger = logging.getLogger() print("FC Invoke Start RequestId: " + rid) logger.info("FC Invoke Start RequestId: " + rid) data = request.stream.read() print(str(data)) logger.info("receive event: {}".format(str(data))) print("FC Invoke End RequestId: " + rid) logger.info("FC Invoke Start RequestId: " + rid) return "Hello, World!" if __name__ == '__main__': app.run(host='0.0.0.0',port=9000) Note You can change filename='/tmp/log/fc-flask.log' to the actual log type and log location in the code. Make sure that the configuration is consistent with the sources.paths path configured in Step 2.
     Startup Command	/code/bootstrap Note The bootstrap file is created in Step 2.
     Listening Port	9000

Step 2: Create a bootstrap file as the startup command

1. After the function is created, use WebIDE on the Code tab to create a bootstrap file in the CODE directory.

   The following sample code shows the content of the bootstrap file.

   #!/bin/bash
   
   #1. Create the pipelines.yml file.
   mkdir -p /tmp/log /code/etc
   cat << EOF > /code/etc/pipelines.yml
   pipelines:
     - name: demo
       sources:
         - type: file
           name: fc-demo
           addonMeta: true
           fields:
             topic: "loggie"
           fieldsUnderRoot: true
           paths:
             - "/tmp/log/*.log"
       sink:
         type: sls
         endpoint: ${LOGGIE_SINK_SLS_ENDPOINT}
         accessKeyId: ${LOGGIE_SINK_SLS_ACCESS_ID}
         accessKeySecret: ${LOGGIE_SINK_SLS_ACCESS_SECRET}
         project: ${LOGGIE_SINK_SLS_PROJECT}
         logstore: ${LOGGIE_SINK_SLS_LOGSTORE}
         topic: ${LOGGIE_SINK_SLS_TOPIC}
   EOF
   
   #2. Create the loggie.yml file.
   cat << EOF > /code/etc/loggie.yml
   EOF
   
   #3. Start Loggie and run it as a background process.
   /opt/bin/loggie -config.system=/code/etc/loggie.yml -config.pipeline=/code/etc/pipelines.yml > /tmp/loggie.log 2>&1 &
   
   #4. Start the application.
   exec python app.py

   The script performs the following operations:

   1. Create the pipelines.yml file, which is the pipeline configuration file.

      • sources

        Specifies the type and path of the logs. This example shows how to collect logs from all files that end with .log in the /tmp/log directory.

      • sink

        Specifies information about Simple Log Service. The variables in the script are set in Step 4.

   2. Create the loggie.yml file, which is the Loggie configuration file.

      If this file is empty, the default configurations are used. In this example, the default configurations are used. However, the loggie.yml file must exist. If the file is not empty, see Overview for the detailed configurations.

   3. Start Loggie and run it as a background process. The run logs of Loggie are printed to the /tmp/loggie.log file.

   4. Start the application. In this example, Python is used. In actual scenarios, specify the programming language that you want to use.

2. Obtain the execution permission on the bootstrap file.

   In WebIDE, choose Terminal > New Terminal and run the chmod 777 bootstrap command to set make the file executable.

Step 3: Add the Loggie official common layer

Step 4: Set environment variables

Step 4: Verify results

Troubleshooting

Loggie runs independently in a function instance. Function Compute cannot detect whether Loggie is running as expected. Execution of functions is not affected even if Loggie malfunctions.

When you query logs that are related to Loggie in Simple Log Service, latency of seconds may occur. If you cannot find logs that are related to Loggie in Simple Log Service, perform the following operations to troubleshoot the issue:

Function runs as expected

In this case, the instance is alive for several minutes after the function is invoked. You can log on to the instance to view the running status and logs of Loggie. For more information about how to log on to an instance, see Run commands to manage function instances.

• If logs do not exist, you can start Loggie in the command line.

• If logs exist, troubleshoot the issue based on the logs.

  • Check whether the pipelines.yml file is correctly configured.

  • Check whether the Simple Log Service sink is successfully started. Logs are similar to pipeline sink(sink/sls)-0 invoke loop start.

  • Check whether the log file is obtained. Logs are similar to start collect file: /tmp/log/fc-flask.log. If no similar log is available, check whether a log file is generated based on the paths configuration in the pipelines.yml file.

Function fails to run

If a function fails to run, you can remove the startup logic from Loggie to check whether the function is running as expected. In most cases, Loggie does not affect the running of functions because Loggie is an external extension. If unexpected exits of processes or execution timeouts occur, you can scale up the memory or increase the CPU specifications.

References

• For more information about Loggie, visit the Loggie website.

• In this example, Loggie collects logs and uploads them without any processing. If you want to process logs before you upload them, for example, parse JSON format logs or remove DEBUG logs, you can add Interceptor configurations in the pipelines.yml file. For more information, see Overview.