A direct upload from a client allows you to generate a signature by using a security token obtained from Security Token Service (STS) on a client and use the signature to directly upload data from the client to Object Storage Service (OSS) without an intermediate data transfer. This way, the workflow is simplified and the upload speed is accelerated. This topic describes how to securely and directly upload data to OSS.
Security risk management
To reduce the workload on the application server of an enterprise and improve efficiency, a web application is designed to directly interact with OSS in the web browser of a user, instead of transferring all requests by using the application server.
Enterprise A plans to build a web application to directly upload data to OSS by using the following solution:
Browser environments are on the user side and cannot be controlled by Enterprise A. As a result, Enterprise A faces the following challenges:
Credential leaks: Browsers are considered untrusted environments. If you save AccessKey pairs in browsers, high risks of credential leaks may arise. Attackers can use leaked credentials to steal corporate data, such as business secrets, customer data, intellectual property, and key assets. Data leaks may cause significant financial losses and damage to the reputation of a company.
Coarse access control: In most cases, RAM users that use the application are granted excessive access permissions to manage and access other cloud services. Leaked AccessKey pairs of RAM users may result in serious consequences. This coarse access control practice can also increase the possibility of corporate resource abuses. Hackers may use leaked AccessKey pairs to deploy malware programs, such as backdoor programs, password stealers, and crypto mining software. These malware programs pose high risks to normal business activities, finance security, and business reputation.
Solution
To handle the risks, Enterprise A can add temporary access authorization to the existing solution. This way, Enterprise A can maintain upload efficiency and gain the following benefits:
Enhanced authentication and authorization: STS issues security tokens that are valid within a specific period of time, which helps protect against abuses of long-term access credentials and mitigate the risks of credential leaks.
Fine-grained access control: STS allows Enterprise A to configure permissions based on the principle of least privilege. Enterprise A can grant only the required permissions to the web application. Fine-grained access control mitigates the risks of credential leaks.
Eventually, Enterprise A decides to use the following solution to build the web application:
Deployment methods
The following deployment is used as an example to describe how to perform a direct client upload from a web application by using OSS and STS.
In this example, the function code is written in Python. Sample project: oss-upload-sts.zip.
Resource Orchestration Service (ROS) template: add-signatures-on-the-client-by-using-JS-and-upload-data.yml.
For more information about the Java code used for obtaining temporary access credentials from STS, see ststoken.zip.
Quick deployment
You can use ROS to configure an Elastic Compute Service (ECS) instance and an OSS bucket, and deploy a browser client on the ECS instance. To use ROS to quickly deploy the web application and perform direct uploads, perform the following steps:
Deploy cloud resources with a few clicks.
Use Create Stack wizard in the ROS console.
In the Configure Parameters step of the Create Stack wizard in the ROS console, enter a stack name and the name of the OSS bucket that you want to create. Specify the zone, instance type, system disk category, and password for the ECS instance that you want to purchase, and then click Next. In the Check and Confirm step, confirm your settings and click Create.
On the Stack Information tab of the page that appears, the status of the stack is Creating.
After the stack enters the Created state, click the Outputs tab to view the resources, such as the ECS instance and OSS bucket.
Perform a direct upload.
On the Outputs tab, copy the value of UploadPath and open it in your browser.
On the Transfer Data from Web Client to OSS page, click Select File, select a file of a specific type, and then click Upload.
Operation verification and resource cleanup
Operation verification
After the upload operation is complete, check whether the object exists in the bucket.
In the left-side navigation pane, click Buckets.
On the Objects page, check whether the object exists.
Resource cleanup
Release the resources to avoid unnecessary charges.
In the upper-right corner of the stack page, click Delete.
In the Delete Stack dialog box, set Method to Delete the Stack to Release Resources and click OK.
Manual deployment
Preparations
Create an OSS bucket to store data that is uploaded by the web application from a browser.
Parameter
Example
Region
China (Hangzhou)
Bucket Name
web-direct-upload
For more information, see Create a bucket.
Create an ECS instance to obtain temporary access credentials from STS.
NoteIn an actual deployment scenario, you can integrate STS API operations into your application server without the need to create an ECS instance.
Parameter
Example
Billing Method
Pay-as-you-go
Region
China (Hangzhou)
Public IP Address
Assign Public IPv4 Address
Security Group
HTTP-TCP:80-open
For more information, see Create and manage an ECS instance in the console (express version).
Configure cross-origin resource sharing (CORS) for the OSS bucket.
Parameter
Example
Source
http://ECS-public-IP-address
Allowed Methods
PUT
Allowed Headers
*
For more information, see CORS.
Procedure
Step 1: Create a RAM user in the RAM console
Create a RAM user for which OpenAPI access is enabled and obtain the AccessKey pair of the RAM user. The AccessKey pair is long-term access credentials that are required to access and manage the application server.
Log on to the RAM console by using an Alibaba Cloud account or as an account administrator.
In the left-side navigation pane, choose Identities > Users.
On the Users page, click Create User.
Configure the Logon Name and Display Name parameters.
In the Access Mode section, select OpenAPI Access and click OK.
Click Copy in the Actions section to copy the AccessKey pair and paste the AccessKey pair into a securely stored file.
Step 2: Authorize the RAM user to call the AssumeRole operation
Grant the RAM user the permissions to call the AssumeRole operation. This way, the RAM user can obtain temporary access credentials by assuming a RAM role.
In the left-side navigation pane, choose Identities > Users.
On the Users page, find the RAM user to whom you want to grant permissions and click Add Permissions in the Actions column.
In the Policy section of the Grant Permission panel, select the AliyunSTSAssumeRoleAccess policy.
NoteThe AliyunSTSAssumeRoleAccess policy allows a RAM user to call the AssumeRole operation. The permissions of the policy are independent of the permissions required for the RAM user to obtain temporary access credentials from STS and initiate requests to OSS.
Click Grant permissions.
Step 3: Create a RAM role in the RAM console
Create a RAM role for the Alibaba Cloud account and obtain the Alibaba Cloud Resource Name (ARN) of the RAM role. The RAM role is assumed by the RAM user later.
In the left-side navigation pane, choose Identities > Roles.
Click Create Role. In the Create Role wizard, set Select Trusted Entity to Alibaba Cloud Account and click Next.
In the Configure Role step, enter a name in the RAM Role Name field and set Select Trusted Alibaba Cloud Account to Current Alibaba Cloud Account.
Click OK. After you create the role, click Close.
On the Roles page, enter the role name in the search box. Example:
oss-web-upload
. Click the name of the role.In the Basic Information section, click Copy next to the ARN field to record the ARN of the RAM role.
Step 4: Create a custom policy in the RAM console
Create a custom policy that allows the RAM role to call only the specified API operation on the bucket.
In the left-side navigation pane, choose Permission > Policies.
On the Policies page, click Create Policy.
On the Create Policy page, click JSON. Copy the following sample script and paste it into the code editor. Replace
<BucketName>
with the name of the bucket that you created.ImportantThe following sample script is provided only for reference. You must configure fine-grained RAM policies based on your requirements to avoid granting excessive permissions to users. For more information about how to configure fine-grained policies, see Custom policies for OSS.
{ "Version": "1", "Statement": [ { "Effect": "Allow", "Action": "oss:PutObject", "Resource": "acs:oss:*:*:<BucketName>/uploads/*" } ] }
Click Next to edit policy information.
In the Basic Information section, specify a policy name and click OK.
Step 5: Attach the custom policy to the RAM role in the RAM console
Attach the custom policy to the RAM role. This way, the RAM role can provide the required permissions when the RAM role is assumed.
In the left-side navigation pane, choose Identities > Roles.
On the Roles page, find the RAM role and click Grant Permission in the Actions column.
In the Policy section of the Grant Permission panel, select Custom Policy from the drop-down list and select the custom policy.
Click Grant permissions.
Step 6: Obtain temporary access credentials from the application server
Integrate STS SDK into the application server to obtain the /get_sts_token operation to obtain temporary access credentials from STS. When the /get_sts_token
operation is called by using the HTTP GET method, temporary access credentials are created and returned to the requester.
On the ECS instance, build a web application by using the Flask framework and implement the operation to obtain temporary access credentials from STS:
Connect to the ECS instance.
For more information, see Create and manage an ECS instance by using the ECS console (express version).
Install Python 3.
Create a project directory and switch to the project directory. Sample commands:
mkdir my_web_sample cd my_web_sample
Run the following command to install dependencies:
pip3 install Flask pip3 install attr pip3 install yarl pip3 install async_timeout pip3 install idna_ssl pip3 install attrs pip3 install aiosignal pip3 install charset_normalizer pip3 install alibabacloud_tea_openapi pip3 install alibabacloud_sts20150401 pip3 install alibabacloud_credentials
Write application backend code.
Create a
main.py
file.Write the following Python code into the file:
import json from flask import Flask, render_template from alibabacloud_tea_openapi.models import Config from alibabacloud_sts20150401.client import Client as Sts20150401Client from alibabacloud_sts20150401 import models as sts_20150401_models from alibabacloud_credentials.client import Client as CredentialClient app = Flask(__name__) # Replace <YOUR_ROLE_ARN> with the ARN of the RAM role. role_arn_for_oss_upload = '<YOUR_ROLE_ARN>' # Specify the region of STS. Example: cn-hangzhou region_id = 'cn-hangzhou' @app.route("/") def hello_world(): return render_template('index.html') @app.route('/get_sts_token', methods=['GET']) def get_sts_token(): # If you do not specify parameters when you initialize CredentialClient, the default credential chain is used. # If you run the program on your local computer, you can obtain the AccessKey pair from the ALIBABA_CLOUD_ACCESS_KEY_ID and ALIBABA_CLOUD_ACCESS_KEY_SECRET environment variables. # If you run the program on ECS, Elastic Container Instance, or Container Service for Kubernetes (ACK), you can specify the role of the bound instance by configuring the ALIBABA_CLOUD_ECS_METADATA environment variable. The SDK automatically obtains the temporary access credentials from STS. config = Config(region_id=region_id, credential=CredentialClient()) sts_client = Sts20150401Client(config=config) assume_role_request = sts_20150401_models.AssumeRoleRequest( role_arn=role_arn_for_oss_upload, # Replace <YOUR_ROLE_SESSION_NAME> with the name of the custom session. role_session_name='<YOUR_ROLE_SESSION_NAME>' ) response = sts_client.assume_role(assume_role_request) token = json.dumps(response.body.credentials.to_map()) return token app.run(host="0.0.0.0", port=80)
Replace
<YOUR_ROLE_ARN>
in the sample code with the ARN of the RAM role that you created in Step 3.Replace
<YOUR_ROLE_SESSION_NAME>
with the custom name of the session. Example:role_session_test
.
Start the application by using the AccessKey pair that you obtained in Step 1.
ALIBABA_CLOUD_ACCESS_KEY_ID=<YOUR_AK_ID> ALIBABA_CLOUD_ACCESS_KEY_SECRET=<YOUR_AK_SECRET> python3 main.py
In the address bar of your browser, enter
http://<ECS-public-IP-address>/get_sts_token
.Sample success response:
Press
Ctrl+C
to stop the application.
Step 7: Use the temporary access credentials in the browser to upload a file to OSS
After you configure the /get_sts_token operation on the application server, import OSS SDK for JavaScript on the frontend page of the web application by using Alibaba Cloud CDN. When you upload data to the bucket, call the /get_sts_token
operation to obtain temporary access credentials from STS and use the temporary access credentials to upload the data.
The following steps describe how to integrate OSS SDK for JavaScript into the frontend code of the web application on the ECS instance:
Press
Ctrl+C
to stop the application.Create a frontend project file. Sample command:
mkdir templates
Create an HTML file.
Create an
index.html
file in thetemplates
directory.vim templates/index.html
Write the following HTML code to the HTML file:
<!DOCTYPE html> <html lang="zh-CN"> <head> <meta charset="UTF-8" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" /> <title>Upload an object to OSS</title> <script src="https://gosspublic.alicdn.com/aliyun-oss-sdk-6.17.0.min.js"></script> </head> <body> <div class="container"> <form> <div class="mb-3"> <label for="file" class="form-label">Select File</label> <input type="file" class="form-control" id="file" name="file" required /> </div> <button type="submit" class="btn btn-primary">Upload</button> </form> </div> <script type="text/javascript"> let credentials = null; const form = document.querySelector("form"); form.addEventListener("submit", async (event) => { event.preventDefault(); // Re-obtain the temporary access credentials when they expire. This reduces the number of STS calls. if (isCredentialsExpired(credentials)) { const response = await fetch("/get_sts_token", { method: "GET", }); if (!response.ok) { // Handle the errors based on the HTTP status codes. throw new Error( `Failed to obtain STS token: ${response. Status} ${response.statusText}` ); } credentials = await response.json(); } const client = new OSS({ // Replace <YOUR_BUCKET> with the name of the OSS bucket. bucket: "<YOUR_BUCKET>", // Replace <YOUR_REGION> with the ID of the region in which the OSS bucket is located. Example: oss-cn-hangzhou. region: "<YOUR_REGION>", // If you are using a locally-obtained ststoken parameter, manually replace credentials.xxx with the specific value of the obtained parameter. accessKeyId: credentials.AccessKeyId, accessKeySecret: credentials.AccessKeySecret, stsToken: credentials.SecurityToken, }); const fileInput = document.querySelector("#file"); const file = fileInput.files[0]; const result = await client.put('uploads/'+file.name, file); console.log(result); }); /** * Check whether the temporary access credentials expire. **/ function isCredentialsExpired(credentials) { if (!credentials) { return true; } // If you are using a locally-obtained ststoken parameter, manually replace credentials.xxx with the specific value of the obtained parameter. const expireDate = new Date(credentials.Expiration); const now = new Date(); // If the remaining validity period of the temporary access credentials is less than 1 minute, the temporary access credentials are considered expired. return expireDate.getTime() - now.getTime() <= 60000; } </script> </body> </html>
Replace
<YOUR_BUCKET
and<YOUR_REGION>
in the code with the name of the bucket that you created and the ID of the region in which the bucket is located. For more information about region IDs, see Regions, endpoints and open ports.
Start the application by using the AccessKey pair that you obtained in Step 1.
ALIBABA_CLOUD_ACCESS_KEY_ID=<YOUR_AK_ID> ALIBABA_CLOUD_ACCESS_KEY_SECRET=<YOUR_AK_SECRET> python3 main.py
Open
http://<ECS-public-IP-address>
in the address bar of the browser. On the web page, upload a file.
Operation verification and resource cleanup
Operation verification
After the upload operation is complete, check whether the object exists in the bucket.
In the left-side navigation pane, click Buckets.
On the Objects page, check whether the object exists.
Resource cleanup
To build the web application, you created an ECS instance, OSS bucket, RAM user, and RAM role. After you complete verification, you can release these resources to avoid unnecessary charges and eliminate relevant security risks.
Release the ECS instance. For more information, see Release an instance.
Delete the bucket. For more information, see Delete buckets.
Delete the RAM user and RAM role. For more information, see Delete a RAM user and Delete a RAM role.
What to do next
After you upload objects to the bucket, you can perform the following operations:
Allow others to download or preview the objects by using signed URLs. For more information, see Share objects with object URLs.
Process image objects. For example, you can add watermarks to images and convert image formats. For more information, see Overview.