All Products
Search
Document Center

Object Storage Service:Securely and directly upload data from a client to OSS

Last Updated:Oct 12, 2024

A direct upload from a client allows you to generate a signature by using a security token obtained from Security Token Service (STS) on a client and use the signature to directly upload data from the client to Object Storage Service (OSS) without an intermediate data transfer. This way, the workflow is simplified and the upload speed is accelerated. This topic describes how to securely and directly upload data to OSS.

Security risk management

To reduce the workload on the application server of an enterprise and improve efficiency, a web application is designed to directly interact with OSS in the web browser of a user, instead of transferring all requests by using the application server.

Enterprise A plans to build a web application to directly upload data to OSS by using the following solution:

image

Browser environments are on the user side and cannot be controlled by Enterprise A. As a result, Enterprise A faces the following challenges:

  • Credential leaks: Browsers are considered untrusted environments. If you save AccessKey pairs in browsers, high risks of credential leaks may arise. Attackers can use leaked credentials to steal corporate data, such as business secrets, customer data, intellectual property, and key assets. Data leaks may cause significant financial losses and damage to the reputation of a company.

  • Coarse access control: In most cases, RAM users that use the application are granted excessive access permissions to manage and access other cloud services. Leaked AccessKey pairs of RAM users may result in serious consequences. This coarse access control practice can also increase the possibility of corporate resource abuses. Hackers may use leaked AccessKey pairs to deploy malware programs, such as backdoor programs, password stealers, and crypto mining software. These malware programs pose high risks to normal business activities, finance security, and business reputation.

Solution

To handle the risks, Enterprise A can add temporary access authorization to the existing solution. This way, Enterprise A can maintain upload efficiency and gain the following benefits:

  • Enhanced authentication and authorization: STS issues security tokens that are valid within a specific period of time, which helps protect against abuses of long-term access credentials and mitigate the risks of credential leaks.

  • Fine-grained access control: STS allows Enterprise A to configure permissions based on the principle of least privilege. Enterprise A can grant only the required permissions to the web application. Fine-grained access control mitigates the risks of credential leaks.

Eventually, Enterprise A decides to use the following solution to build the web application:

image

Deployment methods

The following deployment is used as an example to describe how to perform a direct client upload from a web application by using OSS and STS.

In this example, the function code is written in Python. Sample project: oss-upload-sts.zip.

Resource Orchestration Service (ROS) template: add-signatures-on-the-client-by-using-JS-and-upload-data.yml.

For more information about the Java code used for obtaining temporary access credentials from STS, see ststoken.zip.

Quick deployment

You can use ROS to configure an Elastic Compute Service (ECS) instance and an OSS bucket, and deploy a browser client on the ECS instance. To use ROS to quickly deploy the web application and perform direct uploads, perform the following steps:

  1. Deploy cloud resources with a few clicks.

    1. Use Create Stack wizard in the ROS console.

    2. In the Configure Parameters step of the Create Stack wizard in the ROS console, enter a stack name and the name of the OSS bucket that you want to create. Specify the zone, instance type, system disk category, and password for the ECS instance that you want to purchase, and then click Next. In the Check and Confirm step, confirm your settings and click Create.

      On the Stack Information tab of the page that appears, the status of the stack is Creating.

    3. After the stack enters the Created state, click the Outputs tab to view the resources, such as the ECS instance and OSS bucket.

  2. Perform a direct upload.

    1. On the Outputs tab, copy the value of UploadPath and open it in your browser.

    2. On the Transfer Data from Web Client to OSS page, click Select File, select a file of a specific type, and then click Upload.

Operation verification and resource cleanup

Operation verification

After the upload operation is complete, check whether the object exists in the bucket.

  1. Log on to the OSS console.

  2. In the left-side navigation pane, click Buckets.

  3. On the Buckets page, find the OSS bucket that you want to manage and click the bucket name.

  4. On the Objects page, check whether the object exists.

Resource cleanup

Release the resources to avoid unnecessary charges.

  1. In the upper-right corner of the stack page, click Delete.

  2. In the Delete Stack dialog box, set Method to Delete the Stack to Release Resources and click OK.

Manual deployment

Preparations

  • Create an OSS bucket to store data that is uploaded by the web application from a browser.

    Parameter

    Example

    Region

    China (Hangzhou)

    Bucket Name

    web-direct-upload

    For more information, see Create a bucket.

  • Create an ECS instance to obtain temporary access credentials from STS.

    Note

    In an actual deployment scenario, you can integrate STS API operations into your application server without the need to create an ECS instance.

    Parameter

    Example

    Billing Method

    Pay-as-you-go

    Region

    China (Hangzhou)

    Public IP Address

    Assign Public IPv4 Address

    Security Group

    HTTP-TCP:80-open

    For more information, see Create and manage an ECS instance in the console (express version).

  • Configure cross-origin resource sharing (CORS) for the OSS bucket.

    Parameter

    Example

    Source

    http://ECS-public-IP-address

    Allowed Methods

    PUT

    Allowed Headers

    *

    For more information, see CORS.

Procedure

Step 1: Create a RAM user in the RAM console

Create a RAM user for which OpenAPI access is enabled and obtain the AccessKey pair of the RAM user. The AccessKey pair is long-term access credentials that are required to access and manage the application server.

  1. Log on to the RAM console by using an Alibaba Cloud account or as an account administrator.

  2. In the left-side navigation pane, choose Identities > Users.

  3. On the Users page, click Create User.

  4. Configure the Logon Name and Display Name parameters.

  5. In the Access Mode section, select OpenAPI Access and click OK.

  6. Click Copy in the Actions section to copy the AccessKey pair and paste the AccessKey pair into a securely stored file.

Step 2: Authorize the RAM user to call the AssumeRole operation

Grant the RAM user the permissions to call the AssumeRole operation. This way, the RAM user can obtain temporary access credentials by assuming a RAM role.

  1. In the left-side navigation pane, choose Identities > Users.

  2. On the Users page, find the RAM user to whom you want to grant permissions and click Add Permissions in the Actions column.

  3. In the Policy section of the Grant Permission panel, select the AliyunSTSAssumeRoleAccess policy.

    Note

    The AliyunSTSAssumeRoleAccess policy allows a RAM user to call the AssumeRole operation. The permissions of the policy are independent of the permissions required for the RAM user to obtain temporary access credentials from STS and initiate requests to OSS.

  4. Click Grant permissions.

Step 3: Create a RAM role in the RAM console

Create a RAM role for the Alibaba Cloud account and obtain the Alibaba Cloud Resource Name (ARN) of the RAM role. The RAM role is assumed by the RAM user later.

  1. In the left-side navigation pane, choose Identities > Roles.

  2. Click Create Role. In the Create Role wizard, set Select Trusted Entity to Alibaba Cloud Account and click Next.

  3. In the Configure Role step, enter a name in the RAM Role Name field and set Select Trusted Alibaba Cloud Account to Current Alibaba Cloud Account.

  4. Click OK. After you create the role, click Close.

  5. On the Roles page, enter the role name in the search box. Example: oss-web-upload. Click the name of the role.

  6. In the Basic Information section, click Copy next to the ARN field to record the ARN of the RAM role.

Step 4: Create a custom policy in the RAM console

Create a custom policy that allows the RAM role to call only the specified API operation on the bucket.

  1. In the left-side navigation pane, choose Permission > Policies.

  2. On the Policies page, click Create Policy.

  3. On the Create Policy page, click JSON. Copy the following sample script and paste it into the code editor. Replace <BucketName> with the name of the bucket that you created.

    Important

    The following sample script is provided only for reference. You must configure fine-grained RAM policies based on your requirements to avoid granting excessive permissions to users. For more information about how to configure fine-grained policies, see Custom policies for OSS.

    {
      "Version": "1",
      "Statement": [
        {
          "Effect": "Allow",
          "Action": "oss:PutObject",
          "Resource": "acs:oss:*:*:<BucketName>/uploads/*"
        }
      ]
    }
  4. Click Next to edit policy information.

  5. In the Basic Information section, specify a policy name and click OK.

Step 5: Attach the custom policy to the RAM role in the RAM console

Attach the custom policy to the RAM role. This way, the RAM role can provide the required permissions when the RAM role is assumed.

  1. In the left-side navigation pane, choose Identities > Roles.

  2. On the Roles page, find the RAM role and click Grant Permission in the Actions column.

  3. In the Policy section of the Grant Permission panel, select Custom Policy from the drop-down list and select the custom policy.

  4. Click Grant permissions.

Step 6: Obtain temporary access credentials from the application server

Integrate STS SDK into the application server to obtain the /get_sts_token operation to obtain temporary access credentials from STS. When the /get_sts_token operation is called by using the HTTP GET method, temporary access credentials are created and returned to the requester.

On the ECS instance, build a web application by using the Flask framework and implement the operation to obtain temporary access credentials from STS:

  1. Connect to the ECS instance.

    For more information, see Create and manage an ECS instance by using the ECS console (express version).

  2. Install Python 3.

  3. Create a project directory and switch to the project directory. Sample commands:

    mkdir my_web_sample
    cd my_web_sample
  4. Run the following command to install dependencies:

    pip3 install Flask
    pip3 install attr
    pip3 install yarl
    pip3 install async_timeout
    pip3 install idna_ssl
    pip3 install attrs
    pip3 install aiosignal
    pip3 install charset_normalizer
    pip3 install alibabacloud_tea_openapi
    pip3 install alibabacloud_sts20150401
    pip3 install alibabacloud_credentials
  5. Write application backend code.

    1. Create a main.py file.

    2. Write the following Python code into the file:

      import json
      from flask import Flask, render_template
      from alibabacloud_tea_openapi.models import Config
      from alibabacloud_sts20150401.client import Client as Sts20150401Client
      from alibabacloud_sts20150401 import models as sts_20150401_models
      from alibabacloud_credentials.client import Client as CredentialClient
      
      app = Flask(__name__)
      
      # Replace <YOUR_ROLE_ARN> with the ARN of the RAM role. 
      role_arn_for_oss_upload = '<YOUR_ROLE_ARN>'
      # Specify the region of STS. Example: cn-hangzhou 
      region_id = 'cn-hangzhou'
      
      @app.route("/")
      def hello_world():
          return render_template('index.html')
      
      @app.route('/get_sts_token', methods=['GET'])
      def get_sts_token():
          # If you do not specify parameters when you initialize CredentialClient, the default credential chain is used. 
          # If you run the program on your local computer, you can obtain the AccessKey pair from the ALIBABA_CLOUD_ACCESS_KEY_ID and ALIBABA_CLOUD_ACCESS_KEY_SECRET environment variables.
          # If you run the program on ECS, Elastic Container Instance, or Container Service for Kubernetes (ACK), you can specify the role of the bound instance by configuring the ALIBABA_CLOUD_ECS_METADATA environment variable. The SDK automatically obtains the temporary access credentials from STS. 
          config = Config(region_id=region_id, credential=CredentialClient())
          sts_client = Sts20150401Client(config=config)
          assume_role_request = sts_20150401_models.AssumeRoleRequest(
              role_arn=role_arn_for_oss_upload,
              # Replace <YOUR_ROLE_SESSION_NAME> with the name of the custom session. 
              role_session_name='<YOUR_ROLE_SESSION_NAME>'
          )
          response = sts_client.assume_role(assume_role_request)
          token = json.dumps(response.body.credentials.to_map())
          return token
      
      app.run(host="0.0.0.0", port=80)
    3. Replace <YOUR_ROLE_ARN> in the sample code with the ARN of the RAM role that you created in Step 3.

    4. Replace <YOUR_ROLE_SESSION_NAME> with the custom name of the session. Example: role_session_test.

  6. Start the application by using the AccessKey pair that you obtained in Step 1.

    ALIBABA_CLOUD_ACCESS_KEY_ID=<YOUR_AK_ID> ALIBABA_CLOUD_ACCESS_KEY_SECRET=<YOUR_AK_SECRET> python3 main.py
  7. In the address bar of your browser, enter http://<ECS-public-IP-address>/get_sts_token.

    Sample success response:

    sts token.png

  8. Press Ctrl+C to stop the application.

Step 7: Use the temporary access credentials in the browser to upload a file to OSS

After you configure the /get_sts_token operation on the application server, import OSS SDK for JavaScript on the frontend page of the web application by using Alibaba Cloud CDN. When you upload data to the bucket, call the /get_sts_token operation to obtain temporary access credentials from STS and use the temporary access credentials to upload the data.

The following steps describe how to integrate OSS SDK for JavaScript into the frontend code of the web application on the ECS instance:

  1. Press Ctrl+C to stop the application.

  2. Create a frontend project file. Sample command:

    mkdir templates
  3. Create an HTML file.

    1. Create an index.html file in the templates directory.

      vim templates/index.html
    2. Write the following HTML code to the HTML file:

      <!DOCTYPE html>
      <html lang="zh-CN">
        <head>
          <meta charset="UTF-8" />
          <meta name="viewport" content="width=device-width, initial-scale=1.0" />
          <title>Upload an object to OSS</title>
          <script src="https://gosspublic.alicdn.com/aliyun-oss-sdk-6.17.0.min.js"></script>
        </head>
        <body>
          <div class="container">
            <form>
              <div class="mb-3">
                <label for="file" class="form-label">Select File</label>
                <input
                  type="file"
                  class="form-control"
                  id="file"
                  name="file"
                  required
                />
              </div>
              <button type="submit" class="btn btn-primary">Upload</button>
            </form>
          </div>
          <script type="text/javascript">
            let credentials = null;
            const form = document.querySelector("form");
            form.addEventListener("submit", async (event) => {
              event.preventDefault();
              // Re-obtain the temporary access credentials when they expire. This reduces the number of STS calls.
              if (isCredentialsExpired(credentials)) {
                const response = await fetch("/get_sts_token", {
                  method: "GET",
                });
                if (!response.ok) {
                  // Handle the errors based on the HTTP status codes.
                  throw new Error(
                    `Failed to obtain STS token: ${response. Status} ${response.statusText}`
                  );
                }
                credentials = await response.json();
              }
              const client = new OSS({
                // Replace <YOUR_BUCKET> with the name of the OSS bucket. 
                bucket: "<YOUR_BUCKET>",
                // Replace <YOUR_REGION> with the ID of the region in which the OSS bucket is located. Example: oss-cn-hangzhou. 
                region: "<YOUR_REGION>",
                // If you are using a locally-obtained ststoken parameter, manually replace credentials.xxx with the specific value of the obtained parameter.
                accessKeyId: credentials.AccessKeyId,
                accessKeySecret: credentials.AccessKeySecret,
                stsToken: credentials.SecurityToken,
              });
      
              const fileInput = document.querySelector("#file");
              const file = fileInput.files[0];
              const result = await client.put('uploads/'+file.name, file);
              console.log(result);
            });
      
            /**
             * Check whether the temporary access credentials expire. 
             **/
            function isCredentialsExpired(credentials) {
              if (!credentials) {
                return true;
              }
              // If you are using a locally-obtained ststoken parameter, manually replace credentials.xxx with the specific value of the obtained parameter.
              const expireDate = new Date(credentials.Expiration);
              const now = new Date();
              // If the remaining validity period of the temporary access credentials is less than 1 minute, the temporary access credentials are considered expired. 
              return expireDate.getTime() - now.getTime() <= 60000;
            }
          </script>
        </body>
      </html>
    3. Replace <YOUR_BUCKET and <YOUR_REGION> in the code with the name of the bucket that you created and the ID of the region in which the bucket is located. For more information about region IDs, see Regions, endpoints and open ports.

  4. Start the application by using the AccessKey pair that you obtained in Step 1.

    ALIBABA_CLOUD_ACCESS_KEY_ID=<YOUR_AK_ID> ALIBABA_CLOUD_ACCESS_KEY_SECRET=<YOUR_AK_SECRET> python3 main.py
  5. Open http://<ECS-public-IP-address> in the address bar of the browser. On the web page, upload a file.

Operation verification and resource cleanup

Operation verification

After the upload operation is complete, check whether the object exists in the bucket.

  1. Log on to the OSS console.

  2. In the left-side navigation pane, click Buckets.

  3. On the Buckets page, find the OSS bucket that you want to manage and click the bucket name.

  4. On the Objects page, check whether the object exists.

Resource cleanup

To build the web application, you created an ECS instance, OSS bucket, RAM user, and RAM role. After you complete verification, you can release these resources to avoid unnecessary charges and eliminate relevant security risks.

What to do next

After you upload objects to the bucket, you can perform the following operations:

  • Allow others to download or preview the objects by using signed URLs. For more information, see Share objects with object URLs.

  • Process image objects. For example, you can add watermarks to images and convert image formats. For more information, see Overview.

FAQ

How do I obtain the URL of an uploaded object?

You can obtain the URL of an uploaded object based on the domain name of the bucket that stores the object and the object path. For more information, see Share objects with signed URLs.