Object Storage Service:Add signatures on the server, configure upload callbacks, and directly transfer data

Background information

When you upload objects to OSS, the information about the upload task and names of the objects must be sent to the application server. When you upload images to OSS, information about the sizes of the images must be sent to the application server. To meet the preceding requirements, OSS provides the upload callback feature.

Process

If a user wants the application server to receive an upload callback request when the user uploads an object to OSS, configure a callback function to ensure that OSS sends the upload callback request to the application server. After the user uploads the object, the application server returns a response for the upload callback request to OSS. Then, OSS forwards the response to the user. This response is the upload result.

Examples

Process analysis

The following process describes the core code that is used and the messages that can be returned.

1. A user sends a request to the application server to obtain the upload policy and callback configurations.

   In the upload.js file of the client source code, the value of the serverUrl variable can be used to configure the URL of the application server. After you configure the URL of the application server, the client sends a GET request to the path specified by serverUrl to obtain the required information.

   // serverUrl specifies the URL of the application server that returns information about the signature and upload policy. Replace the sample IP address and port number with your actual information. 
   serverUrl = 'http:/88.88.XX.XXX:8888';

2. The application server returns the upload policy and the code that is used to configure upload callbacks to the user.

   The application server processes the GET request that is sent from the client for direct data transfer. You can modify the corresponding code to ensure that the application server returns the correct message to the client. The configuration documents for different programming languages provide clear instructions for your reference.

   The following sample code provides an example of the message body that is returned to a client. The message body is important to upload an object from the client.

   {
   "accessid":"LTAI**********",
   "host":"http://post-test.oss-cn-hangzhou.aliyuncs.com",
   "policy":"eyJleHBpcmF0aW9uIjoiMjAxNS0xMS0wNVQyMDoyMzoyM1oiLCJjxb25kaXRpb25zIjpbWyJjcb250ZW50LWxlbmd0aC1yYW5nZSIsMCwxMDQ4NTc2MDAwXSxbInN0YXJ0cy13aXRoIiwiJGtleSIsInVzZXItZGlyXC8i****",
   "signature":"I2u5**********",
   "expire":1446727949,
   "callback":"eyJjYWxsYmFja1VybCI6Imh0dHA6Ly9vc3MtZGVtby5hbGl5dW5jcy5jb206MjM0NTAiLAoiY2FsbGJhY2tCb2R5IjoiZmlsZW5hbWU9JHtvYmplY3R9JnNpemU9JHtzaXplfSZtaW1lVHlwZT0ke21pbWVUeXBlfSZoZWlnaHQ9JHtpbWFnZUluZm8uaGVpZ2h0fSZ3aWR0aD0ke2ltYWdlSW5mby53aWR0aH0iLAoiY2FsbGJhY2tCb2R5VHlwZSI6ImFwcGxpY2F0aW9uL3gtd3d3LWZvcm0tdXJsZW5jb2RlZCJ9",
   "dir":"user-dirs/"
   }

   The content that is returned for a callback in the preceding example is Base64-encoded. The following sample code shows the Base64-decoded content:

   {"callbackUrl":"http://oss-demo.aliyuncs.com:23450",
   "callbackBody":"filename=${object}&size=${size}&mimeType=${mimeType}&height=${imageInfo.height}&width=${imageInfo.width}",
   "callbackBodyType":"application/x-www-form-urlencoded"}

   Note

   The callback method that is used in the preceding example is provided only for reference. You can configure a callback method by modifying the server code.

   Parameter	Description
   callbackUrl	The URL of the application server to which OSS sends the request.
   callbackHost	The Host header that is included in the request sent by OSS to the application server.
   callbackBody	The content that OSS sends to the application server. If you upload a non-image object, the content can be the object name, size, and type. If you upload an image, the content can be the image height and width.
   callbackBodyType	The content type of the request. Valid values: application/x-www-form-urlencoded (default) application/json

3. The user sends an object upload request to OSS.

   In the upload.js file of the client source code, the value of callbackbody is included in the callback body that is returned by the application server to the client in Step 2.

   new_multipart_params = {
        'key' : key + '${filename}',
        'policy': policyBase64,
        'OSSAccessKeyId': accessid,
        // Set the HTTP status code that is returned by the application server to 200. If you do not configure this parameter, the HTTP status code 204 is returned. 
        'success_action_status' : '200', 
        'callback': 　callbackbody,
        'signature': signature,
    };

4. OSS sends a callback request to the application server based on callback configurations.

   After you upload the object to OSS, OSS parses the upload callback configurations of the client and sends a POST callback request to the application server. The following sample code shows the content of the request message:

   Note

   In this example, a V1 signature is used. If you want to use a V4 signature to send the request, see Guidelines for upgrading V1 signatures to V4 signatures.

   Hypertext Transfer Protocol
       POST / HTTP/1.1\r\n
       Host47.97.XXX.XX53\r\n
       Connection: close\r\n
       Content-Length: 76\r\n
       Authorization: fsNx********//a8x6v2lI1********\r\n
       Content-MD5: eiEMyp7lbL8KStPBzMdr9w==\r\n
       Content-Type: application/x-www-form-urlencoded\r\n
       Date: Sat, 15 Sep 2018 10:24:12 GMT\r\n
       User-Agent: aliyun-oss-callback\r\n
       x-oss-additional-headers: \r\n
       x-oss-bucket: signedcallback\r\n
       x-oss-owner: 137918634953****\r\n
       x-oss-pub-key-url: aHR0cDovL2dvc3NwdWJsaWMuYWxpY2RuLmNvbS9jYWxsYmFja19wdWJfa2V5X3YxLnaH****\r\n
       x-oss-request-id: 534B371674E88A4D8906****\r\n
       x-oss-requester: 137918634953****\r\n
       x-oss-signature-version: 1.0\r\n
       x-oss-tag: CALLBACK\r\n
       eagleeye-rpcid: 0.1\r\n
       \r\n
       [Full request URI: http://47.xx.xx.53/]
       [HTTP request 1/1]
       [Response in frame: 39]
       File Data: 76 bytes
   HTML Form URL Encoded: application/x-www-form-urlencoded
       Form item: "filename" = ".snappython.png"
       Form item: "size" = "6014"
       Form item: "mimeType" = "image/png"
       Form item: "height" = "221"

5. The application server returns a response to OSS.

   The application server verifies the signature based on the authorization header included in the message that is sent by OSS. If the verification is successful, the application server returns the following message in the JSON format to OSS:

   {
   "String value": "ok",
   "Key": "Status"
   }

6. OSS forwards the response from the application server to the user.

Parsing of client source code

The following sample code describes common features:

• Specify random names for the objects that you want to upload

  If you want to specify random object names without changing the name extension, you can use the following code to modify the function:

  function check_object_radio() {
      g_object_name_type = 'random_name';
  }

• Retain the original object names

  If you want to retain the original object names, you can use the following code to modify the function:

  function check_object_radio() {
      g_object_name_type = 'local_name';
  }

• Configure the directory to which you want to upload the objects

  The directory to which you want to upload the objects is determined by the server. You can upload objects only to the specified directory. This helps ensure data isolation. The following code shows how to set the directory to abc/ by using OSS SDK for PHP. The directory must end with a forward slash (/).

  $dir ='abc/';

• Configure upload conditions

  You can use Plupload filters to configure upload conditions, such as uploading only images, configuring limits for the size of objects that you can upload, and preventing repeated uploads of an object.

  var uploader = new plupload.Uploader({
      ...
      filters: {
          mime_types : [ 
          // Specify that only images and ZIP objects can be uploaded. 
          { title : "Image files", extensions : "jpg,gif,png,bmp" },
          { title : "Zip files", extensions : "zip" }
          ], 
          // Specify that only objects that are smaller than 400 KB in size can be uploaded. 
          max_file_size : '400kb', 
          // Specify that the repeat upload of an object is denied. 
          prevent_duplicates : true 
      },

  • mime_types: the name extensions of the objects that you want to upload.

  • max_file_size: the maximum size of the objects that you want to upload.

  • prevent_duplicates: specifies whether an object can be repeatedly uploaded.

• Obtain the names of uploaded objects

  You can obtain the names of the objects that are uploaded by using Plupload to call the FileUploaded event.

  FileUploaded: function(up, file, info) {
              if (info.status == 200)
              {
                  document.getElementById(file.id).getElementsByTagName('b')[0].innerHTML = 'upload to oss success, object name:' + get_uploaded_object_name(file.name);
              }
              else
              {
                  document.getElementById(file.id).getElementsByTagName('b')[0].innerHTML = info.response;
              }
      }

  You can use the get_uploaded_object_name(file.name) function to obtain the names of the objects that are uploaded to OSS. The file.name parameter specifies the names of the objects before the objects are uploaded.

• Upload the signature

  You can use the following JavaScript sample code to obtain the policyBase64, accessid, and signature variables from the server.

  function get_signature()
  {
      // Determine whether the value of the expire parameter exceeds the current time. If the value exceeds the current time, you must request a new signature. The signature is returned after three seconds. 
      now = timestamp = Date.parse(new Date()) / 1000; 
      if (expire < now + 3)
      {
          body = send_request()
          var obj = eval ("(" + body + ")");
          host = obj['host']
          policyBase64 = obj['policy']
          accessid = obj['accessid']
          signature = obj['signature']
          expire = parseInt(obj['expire'])
          callbackbody = obj['callback'] 
          key = obj['dir']
          return true;
      }
      return false;
  };

  The following sample code provides an example message that is returned by the server.

  Note

  The message is not in a specific format. However, the accessid, policy, and signature fields must be included in the message.

  {"accessid":"LTAI**********",
  "host":"http://post-test.oss-cn-hangzhou.aliyuncs.com",
  "policy":"eyJleHBpcmF0aW9uIjoiMjAxNS0xMS0wNVQyMDoyMzoyM1oiLCJjxb25kaXRpb25zIjpbWyJjcb250ZW50LWxlbmd0aC1yYW5nZSIsMCwxMDQ4NTc2MDAwXSxbInN0YXJ0cy13aXRoIiwiJGtleSIsInVzZXItZGlyXC8i****",
  "signature":"I2u5**********",
  "expire":1446726203,"dir":"user-dir/"}

  • accessid: the AccessKey ID that is used to send the request.

  • host: the domain name to which you want to send the upload request.

  • policy: the form upload policy that is Base64-encoded as a string. For more information, see POST policy.

  • signature: the signature string that is generated based on the Policy.

  • expire: the point in time when the upload policy expires. The value of this parameter is specified in the server configurations. Before the upload policy expires, you can repeatedly use the policy to upload objects and do not need to obtain signatures from the server for each upload.

  Note

  To reduce the server load, you can obtain the signature each time you initialize an OSSClient instance to upload an object. If you use the OSSClient instance to upload another object, compare the current time with the expiration time of the signature to check whether the signature expires. If the signature is expired, you can send a request to obtain a new signature. If the signature does not expire, you can use the existing signature.

  The following sample code provides an example of a parsed policy:

  {"expiration":"2015-11-05T20:23:23Z",
  "conditions":[["content-length-range",0,1048576000],// By default, the maximum size of the object that you can upload is 5 GB. You can specify a size limit for the object. If the object that you want to upload exceeds the size limit, OSS reports an error. 
  ["starts-with","$key","user-dir/"]]}

  In the preceding example, the starts-with field that is added to the policy specifies that the object name must start with user-dir. You can also specify a prefix based on your business requirements when you upload an object. The starts-with field is added to the policy because each application corresponds to a bucket in most scenarios. To prevent data from being overwritten, you can specify a prefix for the name of each object that you upload to OSS. After a user obtains the policy, the user can upload multiple objects, change the name prefix of an object, and upload the object to an unintended directory when the policy is valid. To resolve this issue, you can specify the starts-with field for the application server. This way, even if a user obtains the policy, the user cannot upload the object to an unintended directory. This helps ensure data security.

• Configure the URL of the application server

  In the upload.js file of the client source code, the value of the serverUrl variable can be used to configure the URL of the application server. After you configure the URL of the application server, the client sends a GET request to serverUrl to obtain the required information.

  // serverUrl specifies the URL of the application server that returns information about the signature and upload policy. Replace the sample IP address and port number with your actual IP address and port number. 
  serverUrl = 'http:/88.88.XX.XX8:8888'

FAQ

How do I upload multiple objects to OSS at a time?

OSS does not provide API operations for you to upload multiple objects at a time. If you want to upload multiple objects to OSS at a time, you can repeatedly perform the steps for uploading a single object.

Can I specify the headers of callback requests when I obtain signature information from the server and configure upload callbacks?

No, you cannot specify the headers of callback requests. You can only specify the custom callback parameters, that is, the body of the callback requests. However, you cannot specify the headers that are carried by OSS callback requests to your callback URLs.