Object Storage Service (OSS) provides the multipart upload feature. Multipart upload allows you to split a large object into multiple parts to upload. After these parts are uploaded, you can call the CompleteMultipartUpload operation to combine the parts into a complete object.
Usage notes
In this topic, the public endpoint of the China (Hangzhou) region is used. If you want to access OSS from other Alibaba Cloud services in the same region as OSS, use an internal endpoint. For more information about OSS regions and endpoints, see Regions and endpoints.
In this topic, an OSSClient instance is created by using an OSS endpoint. If you want to create an OSSClient by using custom domain names or Security Token Service (STS), see Create an OSSClient instance.
To perform multipart upload, you must have the
oss:PutObjectpermission. For more information, see Attach a custom policy to a RAM user.
Procedure
To upload a local file by using multipart upload, perform the following steps:
Initiate a multipart upload task.
Call the InitiateMultipartUpload operation to obtain a unique upload ID in OSS.
Upload parts.
Call the UploadPart operation to upload the parts.
NoteIn a multipart upload task, part numbers are used to identify the relative positions of the parts in an object. If you upload a part that has the same part number as an existing part, the existing part is overwritten by the uploaded part.
OSS includes the MD5 hash of each uploaded part in the ETag header in the response.
OSS calculates the MD5 hash of uploaded parts and compares the MD5 hash with the MD5 hash that is calculated by OSS SDK for Java. If the two hashes are different, OSS returns the InvalidDigest error code.
Complete the multipart upload task.
After all parts are uploaded, call the CompleteMultipartUpload operation to combine the parts into a complete object.
Sample code of multipart upload
The following sample code provides an example on how to implement a multipart upload task by following the multipart upload process:
#include <alibabacloud/oss/OssClient.h>
#include <fstream>
int64_t getFileSize(const std::string& file)
{
std::fstream f(file, std::ios::in | std::ios::binary);
f.seekg(0, f.end);
int64_t size = f.tellg();
f.close();
return size;
}
using namespace AlibabaCloud::OSS;
int main(void)
{
/* Initialize information about the account that is used to access OSS. */
/* Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. */
std::string Endpoint = "yourEndpoint";
/* Specify the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the region to cn-hangzhou. */
std::string Region = "yourRegion";
/* Specify the name of the bucket. Example: examplebucket. */
std::string BucketName = "examplebucket";
/* Specify the full path of the object. Do not include the bucket name in the full path of the object. Example: exampledir/exampleobject.txt. */
std::string ObjectName = "exampledir/exampleobject.txt";
/* Initialize resources such as network resources. */
InitializeSdk();
ClientConfiguration conf;
conf.signatureVersion = SignatureVersionType::V4;
/* Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. */
auto credentialsProvider = std::make_shared<EnvironmentVariableCredentialsProvider>();
OssClient client(Endpoint, credentialsProvider, conf);
client.SetRegion(Region);
InitiateMultipartUploadRequest initUploadRequest(BucketName, ObjectName);
/* (Optional) Specify the storage class. */
//initUploadRequest.MetaData().addHeader("x-oss-storage-class", "Standard");
/* Initiate the multipart upload task. */
auto uploadIdResult = client.InitiateMultipartUpload(initUploadRequest);
/* Cancel the multipart upload task or list uploaded parts based on the upload ID. */
/* If you want to cancel a multipart upload task based on the upload ID, obtain the upload ID after you call the InitiateMultipartUpload operation to initiate the multipart upload task. */
/* If you want to list the uploaded parts in a multipart upload task based on the upload ID, obtain the upload ID after you call the InitiateMultipartUpload operation to initiate the multipart upload task but before you call the CompleteMultipartUpload operation to complete the multipart upload task. */
auto uploadId = uploadIdResult.result().UploadId();
std::string fileToUpload = "yourLocalFilename";
int64_t partSize = 100 * 1024;
PartList partETagList;
auto fileSize = getFileSize(fileToUpload);
int partCount = static_cast<int>(fileSize / partSize);
/* Calculate the number of parts. */
if (fileSize % partSize != 0) {
partCount++;
}
/* Upload each part. */
for (int i = 1; i <= partCount; i++) {
auto skipBytes = partSize * (i - 1);
auto size = (partSize < fileSize - skipBytes) ? partSize : (fileSize - skipBytes);
std::shared_ptr<std::iostream> content = std::make_shared<std::fstream>(fileToUpload, std::ios::in|std::ios::binary);
content->seekg(skipBytes, std::ios::beg);
UploadPartRequest uploadPartRequest(BucketName, ObjectName, content);
uploadPartRequest.setContentLength(size);
uploadPartRequest.setUploadId(uploadId);
uploadPartRequest.setPartNumber(i);
auto uploadPartOutcome = client.UploadPart(uploadPartRequest);
if (uploadPartOutcome.isSuccess()) {
Part part(i, uploadPartOutcome.result().ETag());
partETagList.push_back(part);
}
else {
std::cout << "uploadPart fail" <<
",code:" << uploadPartOutcome.error().Code() <<
",message:" << uploadPartOutcome.error().Message() <<
",requestId:" << uploadPartOutcome.error().RequestId() << std::endl;
}
}
/* Complete the multipart upload task. */
/* When the multipart upload task is completed, you need to provide all valid PartETags. After OSS receives the PartETags, OSS verifies all parts one by one. After part verification is successful, OSS combines these parts into a complete object. */
CompleteMultipartUploadRequest request(BucketName, ObjectName);
request.setUploadId(uploadId);
request.setPartList(partETagList);
/* (Optional) Specify the ACL of the object. */
//request.setAcl(CannedAccessControlList::Private);
auto outcome = client.CompleteMultipartUpload(request);
if (!outcome.isSuccess()) {
/* Handle exceptions. */
std::cout << "CompleteMultipartUpload fail" <<
",code:" << outcome.error().Code() <<
",message:" << outcome.error().Message() <<
",requestId:" << outcome.error().RequestId() << std::endl;
return -1;
}
/* Release resources such as network resources. */
ShutdownSdk();
return 0;
}To call the CompleteMultipartUpload operation when you obtain uploaded parts, you must provide the ETag value of each part. You can obtain the ETag value of each part by using one of the following methods:
When a part is uploaded, the ETag value of the part is included in the response. You can save and use this ETag value. This method is used in the preceding sample code.
Call the ListParts operation to query the ETag values of the uploaded parts.
List uploaded parts
You can call the ListParts method to list all uploaded parts that use the specified upload ID.
By default, you can list up to 1,000 uploaded parts by using simple list. If the number of uploaded parts exceeds 1,000, list all uploaded parts by page.
List uploaded parts by using simple list
The following sample code provides an example on how to use simple list to list uploaded parts:
#include <alibabacloud/oss/OssClient.h>
using namespace AlibabaCloud::OSS;
int main(void)
{
/* Initialize information about the account that is used to access OSS. */
/* Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. */
std::string Endpoint = "yourEndpoint";
/* Specify the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the region to cn-hangzhou. */
std::string Region = "yourRegion";
/* Specify the name of the bucket. Example: examplebucket. */
std::string BucketName = "examplebucket";
/* Specify the full path of the object. Do not include the bucket name in the full path of the object. Example: exampledir/exampleobject.txt. */
std::string ObjectName = "exampledir/exampleobject.txt";
/* Specify the upload ID. You can obtain the upload ID from the response to the InitiateMultipartUpload operation. You must obtain the upload ID before you call the CompleteMultipartUpload operation to complete the multipart upload task. */
std::string UploadId = "yourUploadId";
/* Initialize resources such as network resources. */
InitializeSdk();
ClientConfiguration conf;
conf.signatureVersion = SignatureVersionType::V4;
/* Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. */
auto credentialsProvider = std::make_shared<EnvironmentVariableCredentialsProvider>();
OssClient client(Endpoint, credentialsProvider, conf);
client.SetRegion(Region);
/* List the uploaded parts. By default, 1,000 parts are listed. */
ListPartsRequest listuploadrequest(BucketName, ObjectName);
listuploadrequest.setUploadId(UploadId);
bool IsTruncated = false;
do {
auto listUploadResult = client.ListParts(listuploadrequest);
if (!listUploadResult.isSuccess()) {
/* Handle exceptions. */
std::cout << "ListParts fail" <<
",code:" << listUploadResult.error().Code() <<
",message:" << listUploadResult.error().Message() <<
",requestId:" << listUploadResult.error().RequestId() << std::endl;
break;
}
else {
for (const auto& part : listUploadResult.result().PartList()) {
std::cout << "part"<<
",name:" << part.PartNumber() <<
",size:" << part.Size() <<
",etag:" << part.ETag() <<
",lastmodify time:" << part.LastModified() << std::endl;
}
}
listuploadrequest.setPartNumberMarker(listUploadResult.result().NextPartNumberMarker());
IsTruncated = listUploadResult.result().IsTruncated();
} while (IsTruncated);
/* Release resources such as network resources. */
ShutdownSdk();
return 0;
}List all uploaded parts by page
The following sample code provides an example on how to specify the maximum number of parts to list on each page and list all parts by page:
#include <alibabacloud/oss/OssClient.h>
using namespace AlibabaCloud::OSS;
int main(void)
{
/* Initialize information about the account that is used to access OSS. */
/* Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. */
std::string Endpoint = "yourEndpoint";
/* Specify the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the region to cn-hangzhou. */
std::string Region = "yourRegion";
/* Specify the name of the bucket. Example: examplebucket. */
std::string BucketName = "examplebucket";
/* Specify the full path of the object. Do not include the bucket name in the full path of the object. Example: exampledir/exampleobject.txt. */
std::string ObjectName = "exampledir/exampleobject.txt";
/* Specify the upload ID. You can obtain the upload ID from the response to the InitiateMultipartUpload operation. You must obtain the upload ID before you call the CompleteMultipartUpload operation to complete the multipart upload task. */
std::string UploadId = "yourUploadId";
/* Initialize resources such as network resources. */
InitializeSdk();
ClientConfiguration conf;
conf.signatureVersion = SignatureVersionType::V4;
/* Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. */
auto credentialsProvider = std::make_shared<EnvironmentVariableCredentialsProvider>();
OssClient client(Endpoint, credentialsProvider, conf);
client.SetRegion(Region);
/* List all uploaded parts by page. */
/* Set the maximum number of parts to list on each page. */
ListPartsRequest listuploadrequest(BucketName, ObjectName);
listuploadrequest.setMaxParts(50);
listuploadrequest.setUploadId(UploadId);
bool IsTruncated = false;
do {
auto listuploadresult = client.ListParts(listuploadrequest);
if (!listuploadresult.isSuccess()) {
/* Handle exceptions. */
std::cout << "ListParts fail" <<
",code:" << listuploadresult.error().Code() <<
",message:" << listuploadresult.error().Message() <<
",requestId:" << listuploadresult.error().RequestId() << std::endl;
break;
}
else {
for (const auto& part : listuploadresult.result().PartList()) {
std::cout << "part"<<
",name:" << part.PartNumber() <<
",size:" << part.Size() <<
",etag:" << part.ETag() <<
",lastmodify time:" << part.LastModified() << std::endl;
}
}
listuploadrequest.setPartNumberMarker(listuploadresult.result().NextPartNumberMarker());
IsTruncated = listuploadresult.result().IsTruncated();
} while (IsTruncated);
/* Release resources such as network resources. */
ShutdownSdk();
return 0;
}List multipart upload tasks
You can call the ListMultipartUploads method to list all ongoing multipart upload tasks. Ongoing multipart upload tasks are tasks that have been initiated but are not completed or canceled.
By default, you can list up to 1,000 multipart upload tasks by using simple list. If the number of multipart upload tasks exceeds 1,000, list all multipart upload tasks by page.
List multipart upload tasks by using simple list
The following sample code provides an example on how to use simple list to list multipart upload tasks:
#include <alibabacloud/oss/OssClient.h>
using namespace AlibabaCloud::OSS;
int main(void)
{
/* Initialize information about the account that is used to access OSS. */
/* Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. */
std::string Endpoint = "yourEndpoint";
/* Specify the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the region to cn-hangzhou. */
std::string Region = "yourRegion";
/* Specify the name of the bucket. Example: examplebucket. */
std::string BucketName = "examplebucket";
/* Initialize resources such as network resources. */
InitializeSdk();
ClientConfiguration conf;
conf.signatureVersion = SignatureVersionType::V4;
/* Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. */
auto credentialsProvider = std::make_shared<EnvironmentVariableCredentialsProvider>();
OssClient client(Endpoint, credentialsProvider, conf);
client.SetRegion(Region);
/* List the completed multipart upload tasks. By default, 1,000 parts are listed. */
ListMultipartUploadsRequest listmultiuploadrequest(BucketName);
bool IsTruncated = false;
do {
auto listresult = client.ListMultipartUploads(listmultiuploadrequest);
if (!listresult.isSuccess()) {
/* Handle exceptions. */
std::cout << "ListMultipartUploads fail" <<
",code:" << listresult.error().Code() <<
",message:" << listresult.error().Message() <<
",requestId:" << listresult.error().RequestId() << std::endl;
break;
}
else {
for (const auto& part : listresult.result().MultipartUploadList()) {
std::cout << "part"<<
",name:" << part.Key <<
",uploadid:" << part.UploadId <<
",initiated time:" << part.Initiated << std::endl;
}
}
listmultiuploadrequest.setKeyMarker(listresult.result().NextKeyMarker());
listmultiuploadrequest.setUploadIdMarker(listresult.result().NextUploadIdMarker());
IsTruncated = listresult.result().IsTruncated();
} while (IsTruncated);
/* Release resources such as network resources. */
ShutdownSdk();
return 0;
}List all multipart upload tasks by page
The following sample code provides an example on how to list all multipart upload tasks by page:
#include <alibabacloud/oss/OssClient.h>
using namespace AlibabaCloud::OSS;
int main(void)
{
/* Initialize information about the account that is used to access OSS. */
/* Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. */
std::string Endpoint = "yourEndpoint";
/* Specify the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the region to cn-hangzhou. */
std::string Region = "yourRegion";
/* Specify the name of the bucket. Example: examplebucket. */
std::string BucketName = "examplebucket";
/* Initialize resources such as network resources. */
InitializeSdk();
ClientConfiguration conf;
conf.signatureVersion = SignatureVersionType::V4;
/* Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. */
auto credentialsProvider = std::make_shared<EnvironmentVariableCredentialsProvider>();
OssClient client(Endpoint, credentialsProvider, conf);
client.SetRegion(Region);
/* List all multipart upload tasks by page. */
/* Set the maximum number of multipart upload tasks to list on each page. */
ListMultipartUploadsRequest listmultiuploadrequest(BucketName);
listmultiuploadrequest.setMaxUploads(50);
bool IsTruncated = false;
do {
auto listresult = client.ListMultipartUploads(listmultiuploadrequest);
if (!listresult.isSuccess()) {
/* Handle exceptions. */
std::cout << "ListMultipartUploads fail" <<
",code:" << listresult.error().Code() <<
",message:" << listresult.error().Message() <<
",requestId:" << listresult.error().RequestId() << std::endl;
break;
}
else {
for (const auto& part : listresult.result().MultipartUploadList()) {
std::cout << "part"<<
",name:" << part.Key <<
",uploadid:" << part.UploadId <<
",initiated time:" << part.Initiated << std::endl;
}
}
listmultiuploadrequest.setKeyMarker(listresult.result().NextKeyMarker());
listmultiuploadrequest.setUploadIdMarker(listresult.result().NextUploadIdMarker());
IsTruncated = listresult.result().IsTruncated();
} while (IsTruncated);
/* Release resources such as network resources. */
ShutdownSdk();
return 0;
}Cancel a multipart upload task
You can call the client.AbortMultipartUpload method to cancel a multipart upload task. If a multipart upload task is canceled, the upload ID cannot be used to upload parts. In addition, the uploaded parts are deleted.
The following sample code provides an example on how to cancel a multipart upload task:
#include <alibabacloud/oss/OssClient.h>
using namespace AlibabaCloud::OSS;
int main(void)
{
/* Initialize information about the account that is used to access OSS. */
/* Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. */
std::string Endpoint = "yourEndpoint";
/* Specify the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the region to cn-hangzhou. */
std::string Region = "yourRegion";
/* Specify the name of the bucket. Example: examplebucket. */
std::string BucketName = "examplebucket";
/* Specify the full path of the object. Do not include the bucket name in the full path of the object. Example: exampledir/exampleobject.txt. */
std::string ObjectName = "exampledir/exampleobject.txt";
/* Specify the upload ID. You can obtain the upload ID from the response to the InitiateMultipartUpload operation. */
std::string UploadId = "yourUploadId";
/* Initialize resources such as network resources. */
InitializeSdk();
ClientConfiguration conf;
conf.signatureVersion = SignatureVersionType::V4;
/* Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. */
auto credentialsProvider = std::make_shared<EnvironmentVariableCredentialsProvider>();
OssClient client(Endpoint, credentialsProvider, conf);
client.SetRegion(Region);
InitiateMultipartUploadRequest initUploadRequest(BucketName, ObjectName);
/* Initiate the multipart upload task. */
auto uploadIdResult = client.InitiateMultipartUpload(initUploadRequest);
auto uploadId = uploadIdResult.result().UploadId();
/* Cancel the multipart upload task. */
AbortMultipartUploadRequest abortUploadRequest(BucketName, ObjectName, uploadId);
auto outcome = client.AbortMultipartUpload(abortUploadRequest);
if (!outcome.isSuccess()) {
/* Handle exceptions. */
std::cout << "AbortMultipartUpload fail" <<
",code:" << outcome.error().Code() <<
",message:" << outcome.error().Message() <<
",requestId:" << outcome.error().RequestId() << std::endl;
return -1;
}
/* Release resources such as network resources. */
ShutdownSdk();
return 0;
}References
For the complete sample code that is used to perform multipart upload, visit GitHub.
A multipart upload involves three API operations. For more information about the operations, see the following topics:
For more information about the API operation that you can call to cancel a multipart upload task, see AbortMultipartUpload.
For more information about the API operation that you can call to list uploaded parts, see ListUploadedParts.
For more information about the API operation that you can call to list all ongoing multipart upload tasks, see ListMultipartUploads. Ongoing multipart upload tasks include tasks that were initiated but are not completed or canceled.