:How do I troubleshoot upload callback errors?

About upload callbacks

When a file is uploaded, the OSS can provide a Callback to your callback server. You can carry the relevant callback parameters in the upload request to implement the upload callback. The APIs that support upload callback are PutObject, PostObject, and CompleteMultipartUpload. For more information, see Upload callback and Callback API in the Developer Guide.

Application scenarios

Data stream

The following table describes the data streams.

SDK/PostObject

During the file upload, you can set the callback parameters to specify the URL of the callback server, data to be sent to the callback server, and data format. When the callback server processes a callback, some context information, such as the bucket and object, is specified using system variables. Other context information is specified using custom variables.

Callback := Base64(CallbackJson)
CallbackJson := '{' CallbackUrlItem, CallbackBodyItem [, CallbackHostItem, CallbackBodyTypeItem] '}' 
CallbackUrlItem := '"'callbackUrl'"' ':' '"'CallbackUrlValue'"'
CallbackBodyItem := '"'callbackBody'"' ':' '"'CallbackBodyValue'"'
CallbackHostItem := '"'callbackHost'"' ':' '"'CallbackHostValue'"'
CallbackBodyTypeItem := '"'callbackBodyType'"' : '"'CallbackBodyType'"'
CallbackBodyType := application/x-www-form-urlencoded | application/json

    "callbackUrl" : "http://abc.com/test.php",
    "callbackHost" : "oss-cn-hangzhou.aliyuncs.com",
    "callbackBody" : "{\"bucket\":${bucket}, \"object\":${object},\"size\":${size},\"mimeType\":${mimeType},\"my_var\":${x:my_var}}",
    "callbackBodyType" : "application/json"
                

    "callbackUrl" : "http://abc.com/test.php",
    "callbackBody" : "bucket=${bucket}&object=${object}&etag=${etag}&size=${size}&mimeType=${mimeType}&my_var=${x:my_var}"
                

System variables and custom variables

Variables for CallbackJson, such as ${bucket}, ${object}, and ${size}, in the CallbackJson example are the OSS-defined system variables. During the callback, the OSS replaces the system variables with actual values. The following table lists the OSS-defined system variables.

Variables for CallbackJson, such as ${x:my_var} , in the CallbackJson example are the custom variables. During the callback, the OSS replaces the custom variables with custom values. Custom variable values are defined and carried by the upload request in either of the following two ways:

• The custom variables are carried by x-oss-callback-var in the message header. This is a common and recommended way.

• The custom variables are carried by callback-var in QueryString.

Rules for generating the x-oss-callback-var or callback-var values are as follows:

CallbackVar := Base64(CallbackVarJson)
CallbackVarJson := '{' CallbackVarItem [, CallbackVarItem]* '}'
CallbackVarItem := '"''x:'VarName'"' : '"'VarValue'"'

CallbackVarJson value examples are as follows:

    "x:my_var1" : "value1",
    "x:my_var2" : "value2"
            

SDK usage example

Some SDKs, such as JAVA and JS, encapsulate the preceding steps. Some SDKs, such as Python, PHP, and C, need to use the preceding rules to generate the upload callback parameters and custom variables. The following table lists SDK usage examples.

PostObject usage example

PostObject supports the upload callback, whose callback parameters are carried by the form field callback and custom variables are carried by an independent form field. For more information, see PostObjet.

The following table lists PostObject usage examples.

Callback server

The callback server is an HTTP server that processes callback requests and POST messages sent from the OSS. The callback server URL is the value of the upload callback parameter callbackUrl. You can implement your own processing logic on the callback server for recording, review, processing, and statistics of the uploaded data.

Debugging procedure

The upload callback debugging includes debugging of the client that uploads a file and the callback server that processes the callback. We recommend that you debug the client first and then the callback server. After independently debugging the two parts, perform the complete upload callback.

• Client debugging

  You can use the callback server http://oss-demo.aliyuncs.com:23450 provided by the OSS, that is, the callback parameter callbackUrl to debug the client. The callback server only verifies the callback request signature, and does not process the callback request. For callback requests whose signatures are successfully verified, the callback server returns {"Status":"OK"}. For callback requests whose signatures fail to be verified, the callback server returns 400 Bad Request. For non-POST requests, the callback server returns 501 Unsupported method. For more information about the code of the callback server example, see callback_app_server.py.zip.

• Callback server debugging

  The callback server is an HTTP server that can process the POST request. You can modify the callback server based on the example provided by the OSS or implement it by yourself. The following table describes the examples of the callback server provided by the OSS.

  Language	Example	Running method
  JAVA	AppCallbackServer.zip	Decompress the package and run java -jar oss-callback-server-demo.jar 9000.
  PHP	callback-php-demo.zip	Deploy and run the program to in Apache environment
  Python	callback_app_server.py.zip	Decompress the package and run python callback_app_server.py.
  C#	callback-server-dotnet.zip	Compile the program and run aliyun-oss-net-callback-server.exe 127.0.0.1 80.
  Go	callback-server-go.zip	Compile the program and run aliyun_oss_callback_server.
  Ruby	oss-callback-server	Run ruby aliyun_oss_callback_server.rb.

  The callback server can be debugged by running the cURL command. The following commands may be used:

  # Run the following command to send a `POST` request whose message body is `object=test_obj` to the callback server: 
  curl -d "object=test_obj" http://oss-demo.aliyuncs.com:23450 -v
  # Run the following command to send a `POST` request whose message body is `post.txt` to the callback server: 
  curl -d @post.txt http://oss-demo.aliyuncs.com:23450 -v
  # Run the following command to send a `POST` request whose message body is `post.txt` and which carries the specified message header `Content-Type` to the callback server:
  curl -d @post.txt -H "Content-Type: application/json" http://oss-demo.aliyuncs.com:23450 -v

  Note

  • When debugging the callback server, ignore signature verification because it is difficult for cURL to simulate the signature function.

  • The OSS example already provides the signature verification function. We recommend that you directly use it.

  • We recommend that the callback server provide the logging function to record all messages, facilitating debugging and tracking.

  • After correctly processing a callback request, the callback server must return 200 instead of 20x.

  • The message body sent from the callback server to the OSS must be in JSON format, and Content-Type is set toapplication/json.

Common errors and causes

• InvalidArgument

  <Error>
    <Code>InvalidArgument</Code>
    <Message>The callback configuration is not json format.</Message>
    <RequestId>587C79A3DD373E2676F73ECE</RequestId>
    <HostId>bucket.oss-cn-hangzhou.aliyuncs.com</HostId>
    <ArgumentName>callback</ArgumentName>
    <ArgumentValue>{"callbackUrl":"8.8.8.8:9090","callbackBody":"{"bucket":${bucket},"object":${object}}","callbackBodyType":"application/json"}</ArgumentValue>
  </Error>

  Note

  The callback parameter settings are incorrect, or the parameter format is incorrect. The common error is that the callback parameters in ArgumentValue are not in valid JSON format. In JSON, \ and " are escape characters. For example, "callbackBody":"{"bucket":${bucket},"object":${object}}" must be "callbackBody":"{\"bucket\":${bucket},\"object\":${object}}". For more information about the SDKs, see the upload callback examples in the SDK usage example part.

  Character after escape	Character before escape
  \\	\\\\
  “	\\\”
  \b	\\b
  \f	\\f
  \n	\\n
  \r	\\r
  \t	\\t

• CallbackFailed

  Examples of CallbackFailed error are described as follows:

  • Example 1

    <Error>
      <Code>CallbackFailed</Code>
      <Message>Response body is not valid json format.</Message>
      <RequestId>587C81A125F797621829923D</RequestId>
      <HostId>bucket.oss-cn-hangzhou.aliyuncs.com</HostId>
    </Error>

    Note

    The message body sent from the callback server to the OSS is not in JSON format. You can confirm the content by running curl -d "<Content>" <CallbackServerURL> -v or capture packets. We recommend that you use Wireshark to capture packets in Windows, and use tcpdump to capture packets in Linux. Invalid returned messages include: OK and \357\273\277{"Status":"OK"} (the BOM header containing the ef bb bf bytes).

  • Example 2

    <Error>
      <Code>CallbackFailed</Code>
      <Message>Error status : -1. OSS can not connect to your callbackUrl, please check it.</Message>
      <RequestId>587C8735355BE8694A8E9100</RequestId>
      <HostId>bucket.oss-cn-hangzhou.aliyuncs.com</HostId>
    </Error>

    Note

    The processing time of the callback server exceeds five seconds. Therefore, the OSS determines that a time-out occurs. We recommend that you modify the processing logic of the callback server to asynchronous processing to make sure that it can complete processing within five seconds and returns the result to the OSS.

  • Example 3

    <Error>
      <Code>CallbackFailed</Code>
      <Message> error status:-1 8.8.8.8: 9090 reply timeout, cost: 5000 MS, timeout: 5000 MS (Ernest-4, errno170) </message>
      <RequestId>587C8D382AE0B92FA3EEF62C</RequestId>
      <HostId>bucket.oss-cn-hangzhou.aliyuncs.com</HostId>
    </Error>

    Note

    The processing time of the callback server exceeds five seconds. Therefore, the OSS determines that a time-out occurs.

  • Example 4

    <Error>
      <Code>CallbackFailed</Code>
      <Message>Error status : 400.</Message>
      <RequestId>587C89A02AE0B92FA3C7981D</RequestId>
      <HostId>bucket.oss-cn-hangzhou.aliyuncs.com</HostId>
    </Error>

    Note

    The status code of the message sent from the callback server to the OSS is 400. Check the processing logic of the callback server.

  • Example 5

    <Error>
      <Code>CallbackFailed</Code>
      <Message>Error status : 502.</Message>
      <RequestId>587C8D382AE0B92FA3EEF62C</RequestId>
      <HostId>bucket.oss-cn-hangzhou.aliyuncs.com</HostId>
    </Error>

    Note

    The callback server is not started, CallbackUrl is missing in the callback parameters, or the network between the OSS instance and the callback server is disconnected. We recommend that you deploy the callback server on the ECS, which belongs to the same intranet as the OSS, to save the traffic cost and guarantee the network quality.

• The body of the response is not in JSON format.

  For example:

  

  This error may be caused by the following reasons:

  • The body of the response returned by the application server to OSS is not in JSON format, as shown in the following figure:

    OSS reports the error if resp_body is not in valid JSON format. In addition, this error may be caused by other underlying factors, such as the application server returning a stack trace instead of a normal response to OSS because of exceptions.

  • The body of the response returned by the application server to OSS carries a BOM in the header.

    This problem generally occurs in application servers coded in PHP, which include a BOM header in the response returned to OSS. Therefore, OSS reports the error because three additional bytes (that is, the BOM header) are included in the response, which does not conform to JSON format. The following figure shows the content included in the packet sent by the application server.

    

    In the preceding figure, the ef bb bf bytes are the three additional bytes of the BOM header.

    Note

    To resolve this issue, remove the BOM header in the response returned by the application server to OSS.

• Error status

  Error status codes, such as 502 and 400, are errors that are returned due to incorrect callback functions, as shown in the following figure.

  
  Note

  An error status code, such as 400, 404, or 403, is returned to indicate the HTTP status returned by the application server to OSS. A return of status code 200 indicates the operation is successful.

  Error status code 502 is returned when the Web service is not enabled on the application server, meaning the server cannot receive the callback request sent by OSS.

• Timeout

  The following figure shows a timeout error.

  
  Note

  For security reasons, OSS waits to receive the callback response for a maximum of 5 seconds. If the response is not returned, OSS disconnects from the application server and returns a timeout error to the client. The IP address included in the error message can be ignored.