Create a GPU function by using container images to improve development and delivery efficiency - Function Compute

When you need to deploy AI projects that require GPU-accelerated instances for faster computation, we recommend that you set up functions by using container images. This way, images serve as function deliverables, which improves development and delivery efficiency. Examples of the aforementioned AI projects include Stable Diffusion WebUI, ComfyUI, retrieval-augmented generation (RAG), and TensorRT.

Create a function

Log on to the Function Compute console. In the left-side navigation pane, click Functions.
In the top navigation bar, select a region. On the Functions page, click Create Function.

On the Create Function page, select GPU Functions. Configure the following parameters and click Create.

Basic Settings: Configure Function Name.

Code: Configure the runtime and code-related information of the function. The following table describes the parameters.

Parameter	Description	Example
Runtime	Use Sample Images: Use sample images provided by Function Compute for quick deployment. Select the sample images you want from the list of Container Images. Use Container Registry Images: Click Select a Container Registry image below the Container Image field. In the Select Container Image panel, configure Container Image Instance and Container Registry Repository. From the list of images that appears, find the one you want and click Select in its Actions column.	Use Sample Images
Container Images	Select the image you want.	Sample Images of SpringBoot Web Applications
Startup Command	Configure the startup command of the container. This parameter is optional. If you do not configure it, Entrypoint/CMD in the image is used as the default startup command.	None
Listening Port	Specify the port on which the HTTP server in your code listens.	9000

Advanced Settings: Configure instance details and the function execution timeout period.

Parameter	Description	Example
GPU Type	Select a GPU type. For more information, see Instance specifications.	Tesla Series
Specifications	Select values from the GPU Memory Size, vCPU Capacity, and Memory Capacity drop-down lists based on your business requirements. You can manually enter those values. For billing information, see Billing overview. Relationship between GPU Memory Size and region-level instance concurrency A Tesla series GPU has a total memory capacity of 16 GB. If you configure the GPU Memory Size parameter to 1 GB, you can run 16 GPU containers simultaneously on one GPU of this series. By default, the total number of GPUs in a region is limited to 30. Therefore, at any given time, a maximum of 480 Tesla series GPU containers can run within a region. If you set the instance concurrency of your GPU function to 1, a maximum of 480 inference requests can be concurrently processed by your function in a region. If you set the instance concurrency of your GPU function to 5, a maximum of 2,400 inference requests can be concurrently processed by your function in a region. An Ada series GPU has a total memory capacity of 48 GB, and can carry only one GPU container (the GPU Memory Size parameter can only be set to 48 GB). By default, the total number of GPUs in a region is limited to 30. Therefore, at any given time, a maximum of 30 Ada series GPU containers can run within a region. If you set the instance concurrency of your GPU function to 1, a maximum of 30 inference requests can be concurrently processed by your function in a region. If you set the instance concurrency of your GPU function to 5, a maximum of 150 inference requests can be concurrently processed by your function in a region.	GPU Memory Size: 4 GB vCPU Capacity: 2 vCPUs Memory Capacity: 4 GB
Size of Temporary Disk	Specify the size of the disk for temporary file storage based on your business requirements. Function Compute offers a complimentary temporary disk of 512 MB. There will be a charge for using a temporary disk that is larger than this size. Note Data can be written to any directory on the temporary disk, which allocates its space across all directories. The temporary disk has the same lifecycle as the associated function. When a function is reclaimed by the system, all associated data is removed from the temporary disk. To store files persistently, you can mount a File Storage NAS (NAS) system or Object Storage Service (OSS) bucket to your function. For more information, see Configure a NAS file system and Configure an OSS file system.	512 MB
Execution Timeout Period	Specify the execution timeout period of a function. The timeout period is 60 seconds by default, and can be extended up to a maximum of 86,400 seconds.	60
Instance Concurrency	Function Compute provides an instance concurrency feature that allows a function to process multiple requests simultaneously. The Instance Concurrency parameter specifies the maximum number of concurrent requests a function can process. For more information, see Configure instance concurrency.	20
Time Zone	Select the time zone for your function. Once you configure the time zone of your function, an environment variable named TZ is automatically added to the function. The variable value corresponds to the time zone that you configure.	UTC
Function Role	Specify the Resource Access Management (RAM) role of the function. Function Compute uses this role to generate a temporary AccessKey pair to access your Alibaba Cloud resources and passes the AccessKey pair to your code. For more information, see Grant Function Compute permissions to access other Alibaba Cloud services.	mytestrole
Access to VPC	Specify whether to allow your function to access VPC resources. For more information, see Configure network settings.	Yes
VPC	Specify the VPC. This parameter is required if you set Access to VPC to Yes. Create a VPC or select the ID of an existing VPC that you want the function to access from the drop-down list.	fc.auto.create.vpc.1632317****
vSwitch	Specify the vSwitch. This parameter is required if you set Access to VPC to Yes. Create a vSwitch or select the ID of an existing vSwitch from the drop-down list.	fc.auto.create.vswitch.vpc-bp1p8248****
Security Group	Specify the security group. This parameter is required if you set Access to VPC to Yes. Create a security group or select an existing security group from the drop-down list.	fc.auto.create.SecurityGroup.vsw-bp15ftbbbbd****
Allow Default NIC to Access Internet	Specify whether to allow the function to access the Internet through the default network interface controller (NIC) of Function Compute. Important When you use a static public IP address, make sure that Allow Default NIC to Access Internet is set to No. For more information, see Configure static public IP addresses.	Yes
Logging	Specify whether to integrate with Simple Log Service. Valid values: Enable: Function execution logs are persistently stored in Simple Log Service, which facilitates code debugging, fault analysis, and data analysis. Disable: You cannot use Simple Log Service to store or query function execution logs.	Enable

Environment Variables: Configure the environment variables for your function. For more information, see Configure environment variables.

Modify a function

If you want to modify the image after creating a function, go to the Function Details page of your function and click the Configurations tab. Then, in the left-side navigation pane, choose Runtime. Click Modify, and in the Runtime panel that appears, adjust the Container Image parameter as needed. The following figure illustrates this process.

For information about how to modify environment variables and logging, see Function configuration.

Delete a function

Log on to the Function Compute console. In the left-side navigation pane, click Functions. In the top navigation bar, select a region. On the Functions page, find the function that you want to delete and choose More > Delete in the Actions column. In the dialog box that appears, confirm that the function that you want to delete is not bound to any resources such as triggers and reserved instances. Then, click Delete.

References

Function Compute provides four types of functions for different scenarios: event function, web function, task function, and GPU function. For more information about how to select the function type that applies to your specific scenario, see Function type selection.
In addition to the Function Compute console, Function Compute also allows you to use API operations to manage functions. For more information, see Function.
If function execution exceeds the time limit, see What do I do if the function execution times out and the error "Function time out after" is reported?

It may take a long time to invoke a function that is not frequently used. For more information, see Why do functions that are infrequently used take a longer period of time to invoke? You can use provisioned instances to reduce cold start latencies for infrequently used functions. For more information, see How to keep instances alive to eliminate the impact of cold starts?
For more information about how to use an Alibaba Cloud Resource Name (ARN) to locate the corresponding Alibaba Cloud resource in code, see Obtain the ARN of a function.