How to integrate Serverless Workflow with asynchronous Function Compute invocation - CloudFlow

In Task steps of Serverless Workflow, workflows can asynchronously invoke functions of Function Compute. Asynchronous invocation is ideal for some scenarios such as long-term tasks and manual audits. Asynchronous function invocation can prevent throttling errors and simplify error handling procedures and retry logic. This topic describes the scenarios and integration patterns of asynchronous function invocation. This topic also provides an example on how to integrate Serverless Workflow with asynchronous function invocation.

Prerequisites

The following operations are performed:

Optional: Asynchronous invocation is enabled. For more information, see Overview.
After you enable stateful asynchronous invocation for your functions, you can stop the function instance when the task is being executed. This helps you view the flow from a finer-grained perspective.
Function Compute is granted the permissions to access Serverless Workflow. This ensures that Serverless Workflow can be called back to execute subsequent steps after the asynchronous function execution is complete. The following snippet shows how to configure the permission policy.
```
{
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "fnf:ReportTaskSucceeded",
                "fnf:ReportTaskFailed"
            ],
            "Resource": [
                "*"
            ]
        }
    ],
    "Version": "1"
}
```
For more information about authorization, see Grant Function Compute permissions to access other Alibaba Cloud services.

Background information

By default, Serverless Workflow uses the synchronous pattern when Serverless Workflow orchestrates Function Compute tasks to implement task flows. In the synchronous pattern, Serverless Workflow does not proceed to the next step until the workflow receives the execution results of the previous function. The following snippet shows a flow that uses the synchronous pattern.

version: v1
type: flow
steps:
  - type: task
    name: mytask
    resourceArn: acs:fc:{region}:{account}:services/{serviceName}.{qualifier}/functions/{functionName}

The following issues may occur when you use the synchronous pattern for function invocation:

The resource limits on invocation of functions of Function Compute may cause throttling errors. By default, a maximum of 300 pay-as-you-go instances are allowed in a single region for an Alibaba Cloud account of Function Compute.
When you use Serverless Workflow to invoke a function, the invocation shares the same quota with the function invocations that are initiated by other services. This may cause throttling errors. In this case, you need to define a complex retry policy in your Serverless Workflow flow. However, this does not ensure that the flow is executed successfully.
If the task requires a long time to execute, a persistent connection must be established between Serverless Workflow and Function Compute. Unstable network conditions may cause unexpected errors.

Serverless Workflow allows you to combine the asynchronous function invocation feature with various integration patterns to solve the preceding issues. At the same time, the combination can meet the following requirements:

In some scenarios, the system needs to perform subsequent steps without waiting for the execution completion of the current step.
If unexpected errors occur when the flow is being executed, the system needs to skip the current step and proceed to the next step.

Scenarios

In different scenarios, Serverless Workflow uses the request-response pattern (requestResponse), synchronous pattern(sync), or wait-for-callback pattern (waitForCallback) to asynchronously invoke functions of Function Compute.

Integration pattern	Value of the pattern parameter	Scenario
Request-response pattern (default pattern)	`requestResponse` Sample flow: `- type: task ... pattern: requestResponse serviceParams: InvocationType: Async`	Tasks are long-running and the task execution results are non-consequential.
Synchronization pattern	`sync` Sample flow: `- type: task ... pattern: sync serviceParams: InvocationType: Async`	Tasks are long-running and may be throttled.
Wait-for-callback pattern	`waitForCallback` Sample flow: `- type: task ... pattern: waitForCallback serviceParams: InvocationType: Async`	When a flow arrives at a specific step, the flow needs to proceed to subsequent steps regardless of whether the execution of the current step is complete. A sample scenario is manual audit.

Service parameters

In the following examples, Serverless Workflow uses Function Compute as the task node, uses resourceArn to specify Function Compute as the destination service, and uses serviceParams to specify the parameters that are used to invoke functions of Function Compute. serviceParams supports the following parameters:

InvocationType: the invocation type of functions. The valid values are Sync and Async. Sync indicates synchronous invocation. Async indicates asynchronous invocation.
Optional: StatefulAsyncInvocationID: the ID of a stateful asynchronous invocation. You can use this parameter to search for the name of the destination task in the Function Compute console.
Note
If you set InvocationType to Async and specify a value for StatefulAsyncInvocationID, the function invocation pattern of Serverless Workflow is stateful and asynchronous invocation. The value of the StatefulAsyncInvocationID parameter must be unique for all the functions.

Integration pattern of asynchronous function invocation

Request-response pattern

When a flow arrives at the specified step, Serverless Workflow asynchronously invokes the specified function and executes the function. Then, the flow proceeds to the next step without waiting for a callback or the execution completion of the preceding step.

version: v1
type: flow
steps:
  - type: task
    name: mytask
    resourceArn: acs:fc:{region}:{account}:services/{serviceName}.{qualifier}/functions/{functionName}
    pattern: requestResponse  # Async invocation with sync pattern
    serviceParams:
        InvocationType: Async

The preceding example shows that a function is triggered to execute when the flow arrives at the mytask step. After the function is triggered, the flow proceeds to the next step when the execution of the mytask step may not be complete.

Synchronization pattern

When a flow arrives at the specified step, Serverless Workflow asynchronously invokes the specified function and executes the function. Then, the flow is suspended and does not proceed to the next step until the Serverless Workflow flow is notified that the function execution is complete.

version: v1
type: flow
steps:
  - type: task
    name: mytask
    resourceArn: acs:fc:{region}:{account}:services/{serviceName}.{qualifier}/functions/{functionName}
    pattern: sync  # Async invocation with sync pattern
    serviceParams:
        InvocationType: Async

The preceding example shows that when the flow arrives at the mytask step, a function is triggered to execute. Then the flow is suspended and does not proceed to the next step until the flow is informed that the function execution is complete.

Note

The use of the synchronous integration pattern is the same as the use of the synchronous function invocation feature in terms of operations. Only the function invocation methods are different.

Wait-for-callback pattern

When a flow arrives at the specified step, Serverless Workflow asynchronously invokes a function, executes the function, and passes in a task token. Then, the flow is suspended. Regardless of whether the function execution is complete, the flow does not proceed to the next step until you use the task token to manually notify the flow of the function execution result.

version: v1
type: flow
steps:
  - type: task
    name: mytask
    resourceArn: acs:fc:{region}:{account}:services/{serviceName}.{qualifier}/functions/{functionName}
    pattern: waitForCallback  # Async invocation with sync pattern
    serviceParams:
        InvocationType: Async

The preceding example shows that a function is triggered to execute when the flow arrives at the mytask step. After the function is triggered, the flow is suspended to wait for a callback that is implemented with the invocation of the ReportTaskSucceed or ReportTaskFailed API operation. The flow does not proceed to the next step until the flow receives a callback request and processes the callback request. The callback request is initiated by you. You can initiate the callback request when the function execution is complete or not.

Example: asynchronous function invocation

Serverless Workflow combines the asynchronous function invocation feature and the stateful asynchronous invocation feature to provide supports for job-type scenarios. When you enable the stateful asynchronous invocation feature and use the asynchronous invocation pattern, you can view the function execution status and stop the function execution when and after the function is invoked. This makes the flow observable and easy to operate. The following snippet shows the flow definition language (FDL) of Serverless Workflow.

version: v1
type: flow
steps:
  - type: task
    name: mytask
    resourceArn: acs:fc:::services/{serviceName}.{qualifier}/functions/{functionName}
    pattern: sync  # Async invocation with sync pattern
    inputMappings:
      - target: id
        source: $context.execution.name
    serviceParams:
        InvocationType: Async
        StatefulAsyncInvocationID: $.id