To configure upload callbacks, you need to only add the related callback parameters to the upload request that is sent to OSS. This topic describes how to configure callbacks.
Usage notes
Callbacks are supported in the following regions: China (Hangzhou), China (Shanghai), China (Qingdao), China (Beijing), China (Zhangjiakou), China (Hohhot), China (Ulanqab), China (Shenzhen), China (Heyuan), China (Guangzhou), China (Chengdu), China (Hong Kong), US (Silicon Valley), US (Virginia), Japan (Tokyo), Singapore, Malaysia (Kuala Lumpur), Indonesia (Jakarta), Philippines (Manila), Germany (Frankfurt), UK (London), and UAE (Dubai).
Only the PutObject, PostObject, and CompleteMultipartUpload operations support callbacks.
The default timeout period for a callback request is 5 seconds.
If a callback request fails, the callback request is not retried.
Step 1: Construct a callback parameter
Callback parameter
The callback parameter is a Base64-encoded string that contains multiple fields in the JSON format. To construct a callback parameter, you must specify the URL (callbackUrl) of the server to which OSS sends a callback request and the content (callbackBody) of the callback request.
The following table describes the fields in the JSON format.
Field
Required
Description
callbackUrl
Yes
The URL of the server to which OSS sends a callback request.
After you upload an object, OSS uses the POST method to send a callback request to the URL. The body of the request is the content that is specified by the callbackBody field. In most cases, the server returns the
HTTP/1.1 200 OK
response. The response body must be in the JSON format, and the value of the Content-Length response header must be valid and less than or equal to 3 MB in size.You can specify up to five URLs in a callback request. You must separate URLs with semicolons (;). OSS sends callback requests to each URL until a success response is returned.
HTTPS URLs are supported.
To ensure that Chinese characters can be correctly processed, you must encode the callback URL. For example, you must encode
http://example.com/中文.php?key=value&中文名称=中文值
intohttp://example.com/%E4%B8%AD%E6%96%87.php?key=value&%E4%B8%AD%E6%96%87%E5%90%8D%E7%A7%B0=%E4%B8%AD%E6%96%87%E5%80%BC
.
callbackHost
No
The value of the Host header in the callback request. The value must be a domain name or an IP address.
This field takes effect only if you configure the callbackUrl field.
If you do not configure the callbackHost field, the host values are resolved from the URLs of the callbackUrl field and are specified as the value of the callbackHost field.
callbackBody
Yes
The callback request body. Example:
key=${object}&etag=${etag}&my_var=${x:my_var}
.The callbackBody field supports OSS system parameters, custom parameters, and constants.
In the PutObject and CompleteMultipart operations, custom parameters are passed through the callback-var parameter.
In the PostObject operation, parameters are passed by using form fields.
callbackSNI
No
Specifies whether OSS sends Server Name Indication (SNI) to the origin address specified by callbackUrl when a callback request is initiated from the client. Whether OSS sends SNI depends on the server configurations and region requirements. SNI is recommended for servers that use the same IP address to host multiple TLS/SSL certificates. Valid values:
true
false (default)
NoteWhen a callback request is initiated in the UK (London) region, the SNI is sent regardless of the callbackSNI value. When a callback request is initiated in a region other than the UK (London) region, whether the SNI is sent depends on the callbackSNI field value.
callbackBodyType
No
The content type of the callback request. Valid values:
application/x-www-form-urlencoded (default)
If you set the callbackBodyType field to application/x-www-form-urlencoded, the parameters in the callbackBody field are replaced by URL-encoded values.
application/json
If you set the callbackBodyType field to application/json, the parameters in the callbackBody field are replaced by values in the JSON format.
JSON field examples
Example 1 (includes required and optional fields)
{ "callbackUrl":"172.16.XX.XX/test.php", "callbackHost":"oss-cn-hangzhou.aliyuncs.com", "callbackBody":"{\"mimeType\":${mimeType},\"size\":${size}}", "callbackBodyType":"application/json", "callbackSNI":false }
Example 2 (includes only required fields)
{ "callbackUrl":"172.16.XX.XX:23456/index.html", "callbackBody":"bucket=${bucket}&object=${object}&etag=${etag}&size=${size}&mimeType=${mimeType}&imageInfo.height=${imageInfo.height}&imageInfo.width=${imageInfo.width}&imageInfo.format=${imageInfo.format}&x:my_var=${x:my_var}" }
You can configure system parameters for the callbackBody field. The following table describes the parameters.
System parameter
Description
bucket
The name of the bucket.
object
The full path of the object.
etag
The ETag field of the object. The ETag is returned to the requester.
size
The size of the object. The size is the size of the entire object when you call the CompleteMultipartUpload operation.
mimeType
The resource type. For example, the resource type of JPEG images is image/jpeg.
imageInfo.height
The height of the image. This parameter applies only to image objects. For other objects, this parameter is left empty.
imageInfo.width
The width of the image. This parameter applies only to image objects. For other objects, this parameter is left empty.
imageInfo.format
The format of the image. Examples: JPG and PNG. This parameter applies only to image objects. For other objects, this parameter is left empty.
crc64
The CRC64 value. The value of this parameter is the same as that of the x-oss-hash-crc64ecma header returned after the object is uploaded.
contentMd5
The MD5 value. The value of this parameter is the same as that of the Content-MD5 header returned after the object is uploaded.
ImportantThis parameter is required only if you call the PutObject or PostObject operation to upload an object.
vpcId
The ID of the virtual private cloud (VPC) in which the client that initiates the request resides. If the request is not initiated over a VPC, this parameter is left empty.
clientIp
The IP address of the client that initiates the request.
reqId
The ID of the request that is initiated.
operation
The API operation that is used to initiate the request, such as PutObject and PostObject.
Custom parameters
You can configure custom parameters by using the callback-var parameter. A custom parameter is a key-value pair, such as
my_var=${x:my_var}
. When a POST callback request is initiated, OSS adds the custom parameters and system parameters in the preceding table to the body of the POST request. This way, the requester can obtain these parameters.You can create a custom parameter in the same manner that you create a callback parameter. You must specify custom parameters in the JSON format. The JSON string consists of key-value pairs of all custom parameters.
ImportantThe key of a custom parameter must start with
x:
and must be in lowercase letters. Otherwise, the system cannot assign a valid value to the custom parameter regardless of whether HTTP status code 200 is returned.
Step 2: Configure a callback request
To add the callback and callback-var parameters to a request that you want OSS to send, you must use Base64 to encode the JSON strings that you created in the preceding step, and then add the parameters to the request by using one of the following methods:
Construct the callback (x-oss-callback) parameter
Specify fields that are used to construct the parameter.
{ "callbackUrl": "http://xxx.xxx.22.143/test", "callbackHost": "your.callback.com", "callbackBody": "bucket=${bucket}&object=${object}&my_var=${x:my_var}", "callbackBodyType": "application/x-www-form-urlencoded", "callbackSNI": false }
Encode the preceding fields in Base64.
ewogICAg****bGxiYWNrVXJsIjogImh0dHA6Ly94eHgueHh4LjIyLjE0My90ZXN0IiwKICAgICJjYWxsYmFja0hvc3QiOiAieW91ci5jYWxsYmFjay5jb20iLAogICAgImNhbGxiYWNrQm9keSI6ICJidWNrZXQ9JHtidWNrZXR9Jm9iamVjdD0ke29iamVjdH0mbXlfdmFyPSR7eDpteV92YXJ9IiwKICAgICJjYWxsYmFja0JvZHlUeXBlIjogImFwcGxpY2F0aW9uL3gtd3d3LWZvcm0tdXJsZW5jb2RlZCIsCiAgICAiY2FsbGJhY2tTTkkiOiBmYWxzZQp9
Construct the callback-var (x-oss-callback-var) parameter.
Specify fields that are used to construct the parameter.
{"x:my_var": "var"}
Encode the preceding field in Base64.
eyJ4Om15X3ZhciI6ICJ2YX****==
Add the parameters to the URL
If you add the parameters to the URL, the callback
parameter is required and the callback-var
parameter is optional. If a request contains the callback
or callback-var
parameter, you must use the callback or callback-var parameter as a subresource of CanonicalizedResource when you calculate the signature. For more information, see Creation of CanonicalizedResource.
PUT /your_object?OSSAccessKeyId=LTAI5t7h6SgiLSga****&Signature=vjbyPxybdZaNmGa%2ByT272YEAiv****&Expires=1682484377&callback-var=eyJ4Om15X3ZhciI6ICJ2YX****==&callback=bGxiYWNrVXJsIjogImh0dHA6Ly94eHgueHh4LjIyLjE0My90ZXN0IiwKICAgICJjYWxsYmFja0hvc3QiOiAieW91ci5jYWxsYmFjay5jb20iLAogICAgImNhbGxiYWNrQm9keSI6ICJidWNrZXQ9JHtidWNrZXR9Jm9iamVjdD0ke29iamVjdH0mbXlfdmFyPSR7eDpteV92YXJ9IiwKICAgICJjYWxsYmFja0JvZHlUeXBlIjogImFwcGxpY2F0aW9uL3gtd3d3LWZvcm0tdXJsZW5jb2RlZCIsCiAgICAiY2FsbGJhY2tTTkkiOiBmYWxzZQp9 HTTP/1.1
Host: callback-test.oss-cn-hangzhou.aliyuncs.com
Date: Wed, 26 Apr 2023 03:46:17 GMT
Content-Length: 5
Content-Type: text/plain
Add the parameters to the request as a header
Add the x-oss-callback
or x-oss-callback-var
header to the request. The x-oss-callback-var and x-oss-callback headers are included in CanonicalizedOSSHeaders to calculate the signature.
Example:
PUT /your_object HTTP/1.1
Host: callback-test.oss-test.aliyun-inc.com
Accept-Encoding: identity
Content-Length: 5
x-oss-callback-var: eyJ4Om15X3ZhciI6ICJ2YX****==
User-Agent: aliyun-sdk-python/0.4.0 (Linux/2.6.32-220.23.2.ali1089.el5.x86_64/x86_64;2.5.4)
x-oss-callback: ewogICAg****bGxiYWNrVXJsIjogImh0dHA6Ly94eHgueHh4LjIyLjE0My90ZXN0IiwKICAgICJjYWxsYmFja0hvc3QiOiAieW91ci5jYWxsYmFjay5jb20iLAogICAgImNhbGxiYWNrQm9keSI6ICJidWNrZXQ9JHtidWNrZXR9Jm9iamVjdD0ke29iamVjdH0mbXlfdmFyPSR7eDpteV92YXJ9IiwKICAgICJjYWxsYmFja0JvZHlUeXBlIjogImFwcGxpY2F0aW9uL3gtd3d3LWZvcm0tdXJsZW5jb2RlZCIsCiAgICAiY2FsbGJhY2tTTkkiOiBmYWxzZQp9
Host: callback-test.oss-test.aliyun-inc.com
Expect: 100-Continue
Date: Wed, 26 Apr 2023 03:46:17 GMT
Content-Type: text/plain
Authorization: OSS qn6q**************:77Dv****************
Test
Add the parameters to the form fields in the body of a POST request
You can use only this method to configure callback parameters when you upload objects by calling the PostObject operation.
If you want to add the callback parameter when you use the POST method to upload an object, add the callback parameter by using a separate form field. Example:
--9431149156168 Content-Disposition: form-data; name="callback" eyJjYWxsYmFja1VybCI6IjEwLj****4xN
If you want to add the callback-var parameter to a POST request, you cannot directly add the parameter as a whole. Instead, you must add each custom parameter by using a separate form field. Example:
{ "x:var1":"value1", "x:var2":"value2" }
The following content provides an example on how to add the preceding custom parameters:
--9431149156168 Content-Disposition: form-data; name="callback" eyJjYWxsYmFja1VybCI6IjEwLj****4xN --9431149156168 Content-Disposition: form-data; name="x:var1" value1 --9431149156168 Content-Disposition: form-data; name="x:var2" value2
You can add callback conditions to the policy. If you do not add callback conditions to the policy, the callback parameters are not verified for the upload. Example:
{ "expiration": "2021-12-01T12:00:00.000Z", "conditions": [ {"bucket": "johnsmith" }, {"callback": "eyJjYWxsYmFja1V****4My"}, ["starts-with", "$key", "user/eric/"], ] }
Step 3: Initiate a callback request
If the object upload is successful, OSS sends the content that you specified for the callback and callback-var parameters in the request to the application server by using the POST method. Example:
POST /test HTTP/1.1
Host: your.callback.com
Connection: close
Authorization: GevnM3**********3j7AKluzWnubHSVWI4dY3VsIfUHYWnyw==
Content-MD5: iKU/O/JB***ZMd8Ftg==
Content-Type: application/x-www-form-urlencoded
Date: Tue, 07 May 2024 03:06:13 GMT
User-Agent: aliyun-oss-callback
x-oss-bucket: your_bucket
x-oss-pub-key-url: aHR0cHM6Ly9nb3NzcHVi**********vY2FsbGJeV92MS5wZW0=
x-oss-request-id: 66399AA50*****3334673EC2
x-oss-requester: 23313******948342006
x-oss-signature-version: 1.0
x-oss-tag: CALLBACK
bucket=your_bucket&object=your_object&my_var=var
(Optional) Step 4: Sign the callback request
If you configure the callback parameter in the request, OSS uses the POST method to send a callback request to the application server based on the specified callback URL. To check whether the callback request that is received by the application server is initiated by OSS, you can sign the callback request.
Signature verification is optional. You can specify whether to perform signature verification based on your business requirements.
Step 1: Generate a signature
Signature generation algorithm
OSS uses the RSA encryption algorithm to calculate a signature.
authorization = base64_encode(rsa_sign(private_key, url_decode(path) + query_string + '\n' + body, md5))
NoteIn the preceding code, private_key specifies a private key, path specifies the resource path that is included in the callback request, query_string specifies the query string, and body specifies the message body in the callback request.
Signature generation method
Create the string to sign. The string consists of the resource path that is obtained by decoding the URL, the original query string, a carriage return, and the callback message body.
Sign the created string by using the RSA encryption algorithm and the private key. The hash function that is used to calculate the signature is MD5.
Use Base64 to encode the signed result to obtain the final signature. Then, add the signature to the Authorization header in the callback request.
Example
POST /index.php?id=1&index=2 HTTP/1.0 Host: 172.16.XX.XX Connection: close Content-Length: 18 authorization: kKQeGTRccDKyHB3H9vF+xYMSrmhMZj****/kdD1ktNVgbWEfYTQG0G2SU/RaHBovRCE8OkQDjC3uG33esH2t**** Content-Type: application/x-www-form-urlencoded User-Agent: http-client/0.0.1 x-oss-pub-key-url: aHR0cDovL2dvc3NwdWJsaWMuYWxpY2RuLmNvbS9jYWxsYmFja19wdWJfa2V5X3YxLnsr**** bucket=examplebucket
In the preceding code, the final signature is
kKQeGTRccDKyHB3H9vF+xYMSrmhMZjzzl2/kdD1ktNVgbWEfYTQG0G2SU/RaHBovRCE8OkQDjC3uG33esH2t****
. The signature is generated from the/index.php
path, the?id=1&index=2
query string, andbucket=examplebucket
body.
Step 2: Verify the signature
Signature verification method
The following sample code provides an example on how to verify the signature by using the application server:
Result = rsa_verify(public_key, md5(url_decode(path) + query_string + '\n' + body), base64_decode(authorization))
public_key specifies the public key, and authorization specifies the signature that you want to include in the callback request header.
Signature verification process
The
x-oss-pub-key-url
header in the callback request stores the Base64-encoded URL of the public key. You must decode the Base64-encoded URL to obtain the public key.public_key = urlopen(base64_decode(x-oss-pub-key-url header value))
For example, obtain the URL of the public key by decoding
aHR0cDovL2dvc3NwdWJsaWMuYWxpY2RuLmNvbS9jYWxsYmFja19wdWJfa2V5X3YxLnBlbQ==
in Base64. The decoded URL ishttp://gosspublic.alicdn.com/callback_pub_key_v1.pem
.NoteThe value of the
x-oss-pub-key-url
header in the public_key field issued by OSS must start withhttp://gosspublic.alicdn.com/
orhttps://gosspublic.alicdn.com/
. We recommend that you cache public key content based on the address of the OSS public key to avoid exceptions caused by network failure or server-related errors, for the address of the public key remains unchanged.Obtain the Base64-decoded form of the signature.
signature = base64_decode(authorization header value)
Obtain the string to sign by performing the same procedure that you use to obtain the string after the callback request is signed.
sign_str = url_decode(path) + query_string + '\n' + body
Verify the signature.
result = rsa_verify(public_key, md5(sign_str), signature)
Example
The following Python code provides an example on how an application server verifies a signature. The M2Crypto library is required to run the sample code.
import httplib import base64 import md5 import urllib2 from BaseHTTPServer import BaseHTTPRequestHandler, HTTPServer from M2Crypto import RSA from M2Crypto import BIO def get_local_ip(): try: csock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) csock.connect(('8.8.8.8', 80)) (addr, port) = csock.getsockname() csock.close() return addr except socket.error: return "" class MyHTTPRequestHandler(BaseHTTPRequestHandler): ''' def log_message(self, format, *args): return ''' def do_POST(self): #get public key pub_key_url = '' try: pub_key_url_base64 = self.headers['x-oss-pub-key-url'] pub_key_url = pub_key_url_base64.decode('base64') if not pub_key_url.startswith("http://gosspublic.alicdn.com/") and not pub_key_url.startswith("https://gosspublic.alicdn.com/"): self.send_response(400) self.end_headers() return url_reader = urllib2.urlopen(pub_key_url) #you can cache it,recommend caching public key content based on the public key address pub_key = url_reader.read() except: print 'pub_key_url : ' + pub_key_url print 'Get pub key failed!' self.send_response(400) self.end_headers() return #get authorization authorization_base64 = self.headers['authorization'] authorization = authorization_base64.decode('base64') #get callback body content_length = self.headers['content-length'] callback_body = self.rfile.read(int(content_length)) #compose authorization string auth_str = '' pos = self.path.find('?') if -1 == pos: auth_str = urllib2.unquote(self.path) + '\n' + callback_body else: auth_str = urllib2.unquote(self.path[0:pos]) + self.path[pos:] + '\n' + callback_body print auth_str #verify authorization auth_md5 = md5.new(auth_str).digest() bio = BIO.MemoryBuffer(pub_key) rsa_pub = RSA.load_pub_key_bio(bio) try: result = rsa_pub.verify(auth_md5, authorization, 'md5') except: result = False if not result: print 'Authorization verify failed!' print 'Public key : %s' % (pub_key) print 'Auth string : %s' % (auth_str) self.send_response(400) self.end_headers() return #do something according to callback_body #response to OSS resp_body = '{"Status":"OK"}' self.send_response(200) self.send_header('Content-Type', 'application/json') self.send_header('Content-Length', str(len(resp_body))) self.end_headers() self.wfile.write(resp_body) class MyHTTPServer(HTTPServer): def __init__(self, host, port): HTTPServer.__init__(self, (host, port), MyHTTPRequestHandler) if '__main__' == __name__: server_ip = get_local_ip() server_port = 23451 server = MyHTTPServer(server_ip, server_port) server.serve_forever()
The following table describes the code that you can use to verify a signature on the server in other programming languages.
SDK programming language
Description
Java
Download link: Java code sample
Running method: Decompress the package and run
java -jar oss-callback-server-demo.jar 9000
. You can replace port number 9000 with a different port number.
Python
Download link: Python code sample
Running method: Decompress the package and run
python callback_app_server.py
. The RSA dependencies are required to run the code sample.
Go
Download link: Go sample code sample
Running method: Decompress the package and follow the instructions in README.md.
PHP
Download link: PHP code sample
Running method: Deploy the code to an Apache environment to ensure that specific headers in the code can use the environment as a dependency. You can modify the sample code based on the environment.
.NET
Download link: .NET code sample
Running method: Decompress the package and follow the instructions in
README.md
.
Node.js
Download link: Node.js code sample
Running method: Decompress the package and run
node example.js
.
Ruby
Download link: Ruby code sample
Running method: Run ruby aliyun_oss_callback_server.rb.
Step 5: Return the callback result
The application server returns a response to OSS. The following example shows a response that is returned for a callback request:
HTTP/1.0 200 OK
Server: BaseHTTP/0.3 Python/2.7.6
Date: Mon, 14 Sep 2015 12:37:27 GMT
Content-Type: application/json
Content-Length: 9
{"a":"b"}
The response that is returned by the application server to OSS must contain the Content-Length header. The size of the response body cannot exceed 1 MB.
Step 6: Return the upload result
OSS returns the information that is returned by the application server to the requester.
The following example shows a returned response:
HTTP/1.1 200 OK
Date: Mon, 14 Sep 2015 12:37:27 GMT
Content-Type: application/json
Content-Length: 9
Connection: keep-alive
ETag: "D8E8FCA2DC0F896FD7CB4CB0031BA249"
Server: AliyunOSS
x-oss-bucket-version: 1442231779
x-oss-request-id: 55F6BF87207FB30F2640C548
{"a":"b"}
The response bodies of CompleteMultipartUpload
requests contain content such as data in the JSON format. If you use the upload callback feature, the original body content such as {"a":"b"}
is overwritten.
Error codes
Error code | HTTP status code | Description |
InvalidArgument | 400 | The value of the callback or callback-var parameter is invalid for one of the following reasons:
|
CallbackFailed | 203 | The object is uploaded to OSS, but the callback fails. A callback failure indicates that OSS does not receive the expected callback response, and does not indicate that the application server does not receive a callback request. For example, if the ContentType returned by the application server is application/json but the content is not in the JSON format, a callback failure occurs. |
FAQ
Does OSS send a callback request to the application server when an object fails to be uploaded?
No, OSS does not send a callback request to the application server when an object fails to be uploaded. OSS sends a callback request to the application server only if an object is uploaded. If the object fails to be uploaded, OSS does not send a callback request to the application server but returns an error message.
What do I do if the "Response body is not valid json format" error message is returned?
The error message is returned because the application server throws an exception when the request is being processed. In this case, the response body that is returned to OSS is not in the JSON format. The following figure shows the response that is returned to OSS when this error occurs.
Solution:
Run the following command to confirm the content:
curl -d "<Content>" <CallbackServerURL> -v
Capture packets to confirm the content.
We recommend that you use the Wireshark tool in Windows or run the tcpdump command in Linux to capture packets.
The body of the response returned by the application server to OSS includes a BOM header.
This error is common among application servers that are written by using OSS SDK for PHP. OSS SDK for PHP returns the BOM header. As a result, OSS receives three additional bytes in the response body that is not in the JSON format. The following figure shows that the ef, bb, and bf bytes are the additional bytes in the response body.
Solution: Delete the BOM header from the response body that is returned by the application server to OSS.