Object Storage Service:FAQ

The multipart upload task initiated by using OSS SDK for Python failed.

Solutions:

• Determine whether the object is directly uploaded to OSS or uploaded to OSS by using a proxy, such as CDN. If the object is uploaded to OSS by using CDN, expose the ETag of the object and configure cross-origin headers, such as Access-Control-Allow-Origin, Access-Control-Allow-Methods, and Access-Control-Allow-Headers. For more information, see PutBucketcors.

• If the upload task failed due to the network timeout, we recommend that you use resumable upload to upload the object. This way, you can split the object into parts of a custom size and then concurrently upload the parts. If exceptions occur, view and analyze the error message returned by OSS SDK for Python.

• Delete the parts generated by the failed upload task and reupload the object.

The upload and download speeds of ossutil are faster than those of OSS SDK for Python.

• Cause

  ossutil is developed based on OSS SDK for Go and provides better performance in concurrent upload scenarios. If the download speed of OSS SDK for Python is slower than that of ossutil, it may be caused by incorrect installation of crcmod.

• Solution

  Correctly install crcmod. For more information, see Installation.

You can call the multipart upload function of OSS SDK for Python on the CentOS system. If you call the multipart upload function on the Ubuntu system, HTTP status code 403 is returned.

Use tcpdump to capture packets on the client. Analyze the packets to determine whether the calculated signature is different from that on the OSS server due to incorrect header information in the request.

POST /ttsservice%2Fpasswd?uploadId=D468E486D1D94D90A1AB8885A4E32AE0 HTTP/1.1
Host: rokid.oss-cn-hangzhou.aliyuncs.com
Accept-Encoding: identity
Accept: text/html
Content-Length: 137
date: Sat, 29 Dec 2018 07:32:34 GMT
authorization: OSS LTAIknFr:r2KPR0y4E0G5tnU/MYdcvXHP****
Content-Type: application/x-www-form-urlencoded
User-Agent: aliyun-sdk-python/2.6.0(Linux/4.4.0-31-generic/x86_64;3.4.3)

<CompleteMultipartUpload><Part><PartNumber>1</PartNumber><ETag>"3195544E19D99658706D5****"</ETag></Part></CompleteMultipartUpload>HTTP/1.1 403 Forbidden

Server: AliyunOSS
Date: Sat, 29 Dec 2018 07:33:43 GMT
Content-Type: application/xml
Content-Length: 1122
Connection: keep-alive
x-oss-request-id: 5C2723573183****
x-oss-server-time: 0

<?xml version="1.0" encoding="UTF-8"?>
<Error>
 <Code>SignatureDoesNotMatch</Code>
 <Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message>
 <RequestId> 5C2723573183A12D </RequestId>
 <HostId>rokid.oss-cn-hangzhou.aliyuncs.com</HostId>
 <OSSAccessKeyId> LTAXXX </OSSAccessKeyId>
 <SignatureProvided>r2KPR0y4E0G5tnU/MYdc****</SignatureProvided>
 <StringToSign>POST
application/x-www-form-urlencoded
Sat, 29 Dec 2018 07:32:34 GMT
/rokid/ttsservice/passwd?uploadId=D468E486D1D94D90A1AB8885A4E3****</StringToSign>
 <StringToSignBytes>50 4F 53 54 0A 0A 61 70 70 6C 69 63 61 74 69 6F 6E 2F 78 2D 77 77 77 2D 66 6F 72 6D 2D 75 72 6C 65 6E 63 6F 64 65 64 0A 53 61 74 2C 20 32 39 20 44 65 63 20 32 30 31 38 20 30 37 3A 33 32 3A 33 34 20 47 4D 54 0A 2F 72 6F 6B 69 64 2D 6F 70 73 2D 6D 6F 64 65 6C 2F 74 74 73 73 65 72 76 69 63 65 2F 70 61 73 73 77 64 3F 75 70 6C 6F 61 64 49 64 3D 44 34 36 38 45 34 38 36 44 31 44 39 34 44 39 30 41 31 41 42 38 38 38 35 41 34 45 33 32 41 45 30 </StringToSignBytes>
</Error>

If the signature received by the server is different from that calculated on the client, the content of the request is modified. We recommend that you upload the object by using HTTPS.

import base64
import hmac
import sha
mac = hmac.new("<Secretkey>","POST\n\napplication/x-www-form-urlencoded\nSat, 29 Dec 2018 07:32:34 GMT\n/rokid/ttsservice/passwd?uploadId=D468E486D1D94D90A1AB8885A4E3****", sha)
Signature = base64.b64encode(mac.digest())
print(Signature)

If the signatures are consistent, the signature in the request generated by OSS SDK is correct. If the calculated signature is not consistent with the signature in the captured packets, the reason may be that the MD5 hash calculated based on the signature is different on the Ubuntu system due to compatibility issues.

When OSS SDK for Python is used on the macOS system to start multiple threads and perform operations on OSS resources in a subthread, an error occurs if TensorFlow is imported. If TensorFlow is not imported, no errors occur. In addition, TensorFlow can be imported without errors if you do not start multiple threads.

objc[2483]: +[__NSPlaceholderDate initialize] may have been in progress in another thread when fork() was called.
objc[2483]: +[__NSPlaceholderDate initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug.

Solution:

For more information, visit Workarounds for compatibility.

How do I add retry policies?

OSS SDK for Python does not provide retry mechanisms. Requests may fail in poor network conditions. In this case, we recommend that you add retry policies to your code. The following sample code provides an example on how to add a retry policy:

def test_get_object():
    MAX_RETRIES = 3
    retry_count = 0
    while True:
        try:
            retry_count += 1
            # Specify the full path of the object. Do not include the bucket name in the full path. Example: abc/efg/example.jpg. 
            # Specify the full path in which you want to store the downloaded object. Example: /users/local/example.jpg. 
            bucket.get_object_to_file("yourObjectName", "yourFileName")
            break
        except Exception:
            if retry_count >= MAX_RETRIES:
                raise

What do I do if the crcmod extensions in C cannot be installed on some operating systems?

For systems on which the crcmod extensions in C cannot be installed, you can disable CRC-64. For more information about how to disable CRC-64, see Initialization.

How do I ensure that the generated signed URL starts with https?

If you want the generated signed URL to start with https, configure the endpoint to start with https when you initialize OSS SDK for Python, such as https://oss-cn-hangzhou.aliyuncs.com.

How do I use a bucket when multiple processes are running at the same time?

Create a bucket after the processes are initialized.