Use request-level metrics for troubleshooting - Function Compute

Request-level metrics help you track function invocations and executions. You can use the request-level metrics feature to detect and troubleshoot issues at the earliest opportunity and analyze application bottlenecks. After you enable the request-level metrics feature, you can manage and optimize function applications in an efficient manner.

Overview

Request-level metrics refer to metrics that are obtained when a function executes requests. After you enable the request-level metrics feature, the system collects metrics, such as performance metrics, exceptions, and tracing metrics, each time the function is executed. The metrics are delivered to the Logstore that you specify.

Performance metrics include the following information: memory usage, function execution time, initialization time, and cold starts.
Exceptions include execution failures and error details.
Tracing metrics include the following information: time required to execute system key links, sampling of the integrated tracing analysis, path of an HTTP function, status code, and client IP address.

After you enable the request-level metrics feature, Function Compute delivers the execution metrics of each request to Logstores. The following figure shows the log of a request.

Scenarios

View details of function executions, performance metrics, and error messages
Requests of a function are displayed in the Function Compute console. You can go to the console to view information about each request, including the execution status, instance ID that corresponds to a request, time required for execution, memory usage, and the request log. The following figure shows an example of request-level metrics.
Note
You can also view requests even if the request-level metrics feature is disabled. However, exceptions may occur in requests in the following scenarios:
- If an ID of a request appears multiple times, you cannot locate the log of the request. This scenario occurs during retries of asynchronous invocation.
- If FC Invoke Start RequestId: xxx and FC Invoke End RequestId: xxx are printed in a custom runtime or Custom Container runtime, a single invocation may appear twice in the request list.
Search for statistics and create a dashboard
Request-level metrics are delivered to Logstores in the key:val format. You can create a dashboard based on your business requirements. For more information, see Create a dashboard.

Usage notes

Effective scope
When you configure the logs of a function, you can configure the Request-level Metrics parameter in the Function Compute console to configure the feature. If you enable this parameter, a log entry is generated each time the function executes a request.
Billing
You are charged for storing logs in Simple Log Service. For more information about the pricing of Simple Log Service, see Pricing.

Enable the request-level metrics feature

Use the Function Compute console

Before you start

Function Compute
- Create a service
Simple Log Service
- Create a project
- Create a Logstore

Procedure

Log on to the Function Compute console. In the left-side navigation pane, click Services & Functions.
In the top navigation bar, select a region. On the Services page, find the desired service and click Configure in the Actions column.

In the Log Settings section of the Modify Service page, configure the parameters based on your business requirements and click Save.

The following table describes the parameters that are required. Retain the default values for other parameters.

Parameter	Description	Example
Log Settings	Specify whether to enable the logging feature and use Simple Log Service to store function execution logs. For more information, see Configure logging.	Enable
Simple Log Service Project	Select a log project that you configured.	aliyun-fc-hangzhou-2238f0df-a742-524f-9f90-976ba*******
Logstore	Select a Logstore that you configured.	function-log
Request-level Metrics	Specify whether to view request-level metrics. After you enable this feature, invocation metrics for the function are collected and delivered to the specified Logstore.	Enable

Use Serverless Devs

Before you start

Procedure

Create a code directory. The following sample code provides an example of the directory structure:

.
├── code
│   └── index.js
└── s.yaml

Create a code directory. The following sample code provides an example of the directory structure:

In the example, the enableRequestMetrics field is used to configure the request-level metrics feature. Before you enable the request-level metrics feature, you must enable the logging feature and set the logstore and project fields to an existing Logstore and log project within your account.

edition: 1.0.0
name: hello-world-app
access: "default"

vars: 
  region: "cn-hangzhou"
  service:
    name: "hello-world-service"
    description: 'hello world by serverless devs'
    logConfig: # Specify the actually used project and Logstore. Alternatively, set this field to auto to allow the system to create a project and a Logstore for you.
      project: 'your-project'
      logstore: 'your-logstore'
      enableRequestMetrics: true # Enable the request-level metrics feature. After you enable the feature, the system collects the metrics each time the function executes a request.
      enableInstanceMetrics: true
    role: 'acs:ram::<accountID>:role/aliyunfcdefaultrole' # Replace <accountID> with the ID of your Alibaba Cloud account.

services:
  helloworld:
    component: fc 
    props:
      region: ${vars.region} 
      service: ${vars.service}
      function:
        name: "hello-world"
        description: 'hello world by serverless devs'
        runtime: nodejs14
        codeUri: ./code
        handler: index.handler
        memorySize: 128
        timeout: 60

Run the following command to deploy the application:
```
sudo s deploy -y
```
If the execution is successful, you can log on to the Function Compute console to check whether the request-level metrics feature of the deployed function is enabled.

Query logs

Log format

The topics of request-level metric logs follow the following format: FCRequestMetrics:serviceName/functionName. You can use the statement __topic__: "FCRequestMetrics:serviceName/functionName" to filter all request-level metric logs in Simple Log Service. For more information, see Query and analyze logs.

Metrics

The following table describes the metrics on a request.

Metric	Description	Example	Recorded for all invocations
serviceName	The service name.	my-service	Yes
functionName	The function name.	my-function	Yes
versionId	The version name.	12	Yes
qualifier	The service alias. Default value: LATEST.	prod	Yes
requestId	The request ID.	db72ce53-ccbe-4216-af55-642622e01494	Yes
operation	The operation name.	InvokeFunction	Yes
invocationType	The invocation type. Valid values: Sync: synchronous invocation Async: asynchronous invocation	Sync	Yes
memoryMB	The upper limit of memory.	512	Yes
memoryUsageMB	The memory consumed by a function execution.	410	Yes
durationMs	The memory consumed by a function execution.	20.20	Yes
isColdStart	Specifies whether a cold start occurs. Note When a request is sent to Function Compute, the Function Compute system does not have started instances to execute the request. In this case, you must create new instances, download code, and start the execution environment. In this case, a cold start occurs. Function Compute creates instances in advance to mitigate cold starts. A clod start does not occur if an incoming request is sent to a created instance.	false	Yes
instanceEvent	The instance event. Valid value: ColdStart. This metric is recorded only when a cold start request occurs.	ColdStart	No
hasFunctionError	Specifies whether a function error occurs during a function execution.	false	Yes
errorType	The function error type. Valid values: FunctionOOMError: The function runs out of memory. FunctionTimeoutError: The execution times out. FunctionUnhandledError: Other exceptions that are not captured.	FunctionUnhandledError	No. This metric is recorded only when a function execution error occurs (`hasFunctionError:true`).
invokeFunctionLatencyMs	The execution time of the function.	99.00	Yes.
traceContext	The context of tracing analysis.	371d3ff242fcee9:371d3ff242fcee9:0:1	No. This metric is recorded only when Managed Service for OpenTelemetry is configured.
isSampled	Specifies whether the request is sampled by Managed Service for OpenTelemetry.	true	No. This metric is recorded only when Managed Service for OpenTelemetry is configured.
resourceMode	The type of the instance that executes the request. Options: OnDemand: on-demand instance Provision: provisioned instance	OnDemand	Yes
instanceID	The ID of the elastic container instance.	c-65603d8c-37e1bf7123054a77****	Yes
hostname	The instance host.	c-65603d8c-37e1bf7123054a77****	Yes
ipAddress	The IP address of the instance. Note: This IP address is intended for internal use to distinguish between instances.	21.0.XX.XX	Yes
activeInstances	The number of active instances.	1	Yes
activeInstancesPerFunction	The number of active instances of the function.	1	Yes
scheduleLatencyMs	The scheduling latency. When a cold start occurs in the request, the latency in scheduling an instance for execution increases.	10.07	Yes
coldStartStartTimestamp	The timestamp when a cold start begins.	1700806029167	No. This metric is recorded only when cold starts occur.
coldStartLatencyMs	The cold start latency.	487.65	No. This metric is recorded only when cold starts occur.
prepareCodeStartTimestamp	The timestamp when code begins to be downloaded. The value is a 13-digit timestamp in milliseconds.	1700806029167	No. This metric is recorded only when cold starts occur.
prepareCodeLatencyMs	The time consumed to download code.	0.18	No. This metric is recorded only when cold starts occur.
runtimeInitializationStartTimestamp	The timestamp when the runtime begins to be initialized.	1700806029168	No. This metric is recorded only when cold starts occur.
runtimeInitializationMs	The time consumed to initialize the runtime.	487.37	No. This metric is recorded only when cold starts occur.
asyncAttemptStartTimestamp	By default, you can retry up to three times when an asynchronous invocation of a function fails. This metric is the timestamp when you perform the ${retryCount} retry.	1700806028084	No. This metric is recorded only when an asynchronous invocation is performed.
asyncAttemptLatencyMs	By default, you can retry up to three times when an asynchronous invocation of a function fails. This metric is the timestamp when you perform the ${retryCount} retry.	1688.74	No. This metric is recorded only when an asynchronous invocation is performed.
asyncMode	The asynchronous invocation mode. Value description: Stateless Stateful	Stateful	No. This metric is recorded only when an asynchronous invocation is performed.
retryCount	The maximum number of retries.	0	No. This metric is recorded only when an asynchronous invocation is performed.

Query logs

Indexes are required to query logs. Indexes are automatically configured in the Function Compute console. After indexes are configured, you can go to the Simple Log Service console, find the desired Logstore, and then use SQL statements to retrieve and filter requests in logs. You can specify topics to retrieve request-level metric logs in an efficient manner.

For example, you can use the following SQL statement to query the list of requests whose execution duration exceeds 10s.

__topic__: "FCRequestMetrics:s1/f1" AND qualifier: q1 and operation: InvokeFunction and durationMs>1000

For more information about complex query syntax, see Query and analyze logs.

Create a custom dashboard

If you want to create a chart whose X coordinate is t and Y coordinate is a metric value, refer to the following SQL statement:

__topic__: "FCRequestMetrics:s1/f1" AND qualifier: q1 and operation: InvokeFunction | select __time__ - __time__ % 60 as t, count(*) as invocations, avg(durationMs) as avgDurationMs group by t order by t

For more information about how to create a custom dashboard, see Create a dashboard.