Object Storage Service:Use Function Compute to download multiple objects as a package

Solution

1. A user invokes a function and specifies the objects to be compressed and the bucket in which the objects are stored.

2. Function Compute downloads the specified objects from OSS and generates a ZIP package that has a random name.

3. Function Compute uploads the ZIP package to OSS.

4. Function Compute returns the URL that is used to download the ZIP package to the user.

5. The user downloads the ZIP package from OSS by using the URL, which remains valid for a long period of time.

Usage notes

• Streaming download and streaming upload are used to transfer the ZIP package due to the limited disk space for running the function. Only a small amount of data is cached in the memory.

• To accelerate the transfer of the ZIP package, Function Compute generates the ZIP package and uploads the package to OSS at the same time.

• Multipart upload is used to upload the ZIP package in parts in parallel to OSS.

• Function Compute takes up to 24 hours to compress objects into a package. Test data shows that Function Compute took approximately 63 seconds to compress 57 objects whose total size is 1.06 GB into a package.

Procedure

In this example, Function Compute is used to download multiple objects in the dir/ directory of the examplebucket bucket in the China (Hangzhou) region to your local computer.

1. Deploy an application and obtain its public domain name.

   Use ROS to deploy an application

   Perform the following steps to use Resource Orchestration Service (ROS) to deploy a Function Compute application and obtain its public domain name:

   1. Use Create Stack wizard in the ROS console.

   2. In the Configure Parameters step, retain the default settings for all parameters and click Next.

   3. In the Check and Confirm step, click Create.

      On the Stack Information tab of the page that appears, the status of the stack is displayed as Creating.

   4. After the stack enters the Created status, click the Outputs tab to view the public domain name.

   Use Serverless Application Center to deploy an application

   1. Go to Serverless Application Center to deploy the start-zip-oss application template.

      Select Directly Deploy for Deployment Type. Select the region of the bucket for the application template. Retain the default settings for other parameters and click Create and Deploy Default Environment. The following figure shows the configuration details.

   2. Obtain the public domain name.

      1. In the Deploy Version panel of the page that appears, click the Result tab.

      2. In the triggers field, obtain the public domain name from the urlInternet parameter.

         

   Use Serverless Devs to deploy an application

   1. Install Serverless Devs.

      curl -o- -L http://cli.so/install.sh | bash

      Check whether Serverless Devs is installed.

      s -v

      The following output is returned if Serverless Devs is installed:

      @serverless-devs/s: 2.1.1, s-home: /root/.s, linux-x64, node-v14.19.3

   2. Configure Serverless Devs.

      s config add --AccessKeyID LTAI******** --AccessKeySecret  KZo1******** --access fc-access

      The following table describes the parameters in the preceding command.

      Parameter	Description
      --AccessKeyID	The AccessKey ID used to access Function Compute.
      --AccessKeySecret	The AccessKey secret used to access Function Compute.
      --Access	The custom key alias.

   3. Deploy the start-zip-oss application template.

      1. Initialize the project.

      s init start-zip-oss -d start-zip-oss

      2. The following table describes sample configurations for the start-zip-oss application template.

      Configuration item	Example
      The ID of the region in which you want to deploy the application	cn-hangzhou
      The service to which the application belongs in Function Compute	start-zip-oss
      The function name of the application	zip-oss-func
      The ARN of the RAM role that is created for the service to which the application belongs in Function Compute	acs:ram::137918634953****:role/test-role

      3. Start the project.

      cd start-zip-oss

      4. Deploy the project.

      s deploy -y

      5. Record the returned value of system_url.

      fc-zip-oss-service:
        region:   cn-hangzhou
        service:
          name: start-zip-oss
        function:
          name:       zip-oss-func
          runtime:    python2.7
          handler:    main.main_handler
          memorySize: 3072
          timeout:    1800
        url:
          system_url: https://zip-oss-func-start-zip-oss-ayouye****.cn-hangzhou.fcapp.run
        triggers:
          -
            type: http
            name: http-test

2. Run the following command to generate the event.json file and specify the bucket name and the directory in which the objects that you want to download are stored:

   cat <<EOF > event.json
   {
     "bucket": "examplebucket",
     "source-dir": "dir/"
   }
   EOF

   Note

   You must set source-dir to a directory other than the root directory of the bucket.

3. Run the following command to trigger the function:

   curl -v -L -o  oss.zip -d @./event.json https://zip-oss-func-zip-oss-ayouye****.cn-hangzhou.fcapp.run

   • /test/oss.zip: the local path used to store objects in the dir/ directory.

   • https://zip-oss-func-zip-oss-ayouye****.cn-hangzhou.fcapp.run: the value of system_url returned when the project is deployed.

   Note

   If you want to change the anonymous non-authenticated HTTP function in the preceding example to the authenticated HTTP function, you can use one of the following OSS SDKs:

   • Java SDK

   • Python SDK

   • Node.js SDK

   • Go SDK

FAQ

References

• For more information about the introduction and billing rules of Function Compute, see Product Introduction and Billing.

• For more information about the template and applications when you use Function Compute together with OSS, see Sample code.