Object Storage Service:マルチパートアップロード

使用上の注意

• このトピックでは、中国 (杭州) リージョンのパブリックエンドポイントを使用します。 OSSと同じリージョンにある他のAlibaba CloudサービスからOSSにアクセスする場合は、内部エンドポイントを使用します。 OSSリージョンとエンドポイントの詳細については、「リージョンとエンドポイント」をご参照ください。

• このトピックでは、OSSエンドポイントを使用してOSSClientインスタンスを作成します。 カスタムドメイン名またはSecurity Token Service (STS) を使用してOSSClientを作成する場合は、「OSSClientインスタンスの作成」をご参照ください。

• マルチパートアップロードを実行するには、oss:PutObject権限が必要です。 詳細については、「RAMユーザーへのカスタムポリシーのアタッチ」をご参照ください。

手順

マルチパートアップロードを使用してローカルファイルをアップロードするには、次の手順を実行します。

1. マルチパートアップロードタスクを開始します。

   InitiateMultipartUpload操作を呼び出して、OSSで一意のアップロードIDを取得します。

2. パーツをアップロード

   UploadPart操作を呼び出して、パーツをアップロードします。

   説明

   • マルチパートアップロードタスクでは、部品番号を使用して、オブジェクト内の部品の相対位置を識別します。 既存の部品と同じ部品番号を持つ部品をアップロードすると、既存の部品はアップロードされた部品によって上書きされます。

   • OSSは、レスポンスのETagヘッダーにアップロードされた各パーツのMD5ハッシュを含めます。

   • OSSはアップロードされたパーツのMD5ハッシュを計算し、そのMD5ハッシュをOSS SDK for Javaによって計算されたMD5ハッシュと比較します。 2つのハッシュが異なる場合、OSSはInvalidDigestエラーコードを返します。

3. マルチパートアップロードタスクを完了します。

   すべてのパーツがアップロードされたら、CompleteMultipartUpload操作を呼び出して、パーツを完全なオブジェクトに結合します。

マルチパートアップロードのサンプルコード

次のサンプルコードは、マルチパートアップロードプロセスに従ってマルチパートアップロードタスクを実装する方法の例を示しています。

#include <alibabacloud/oss/OssClient.h>
#include <fstream>

int64_t getFileSize(const std::string& file)
{
    std::fstream f(file, std::ios::in | std::ios::binary);
    f.seekg(0, f.end);
    int64_t size = f.tellg();
    f.close();
    return size;
}

using namespace AlibabaCloud::OSS;

int main(void)
{
    /* Initialize information about the account that is used to access OSS. */
    
    /* Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. */
    std::string Endpoint = "yourEndpoint";
    /* Specify the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the region to cn-hangzhou. */
    std::string Region = "yourRegion";
    /* Specify the name of the bucket. Example: examplebucket. */
    std::string BucketName = "examplebucket";
    /* Specify the full path of the object. Do not include the bucket name in the full path of the object. Example: exampledir/exampleobject.txt.  */
    std::string ObjectName = "exampledir/exampleobject.txt";

    /* Initialize resources such as network resources. */
    InitializeSdk();

    ClientConfiguration conf;
    conf.signatureVersion = SignatureVersionType::V4;
    /* Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. */
    auto credentialsProvider = std::make_shared<EnvironmentVariableCredentialsProvider>();
    OssClient client(Endpoint, credentialsProvider, conf);
    client.SetRegion(Region);
  
    InitiateMultipartUploadRequest initUploadRequest(BucketName, ObjectName);
    /* (Optional) Specify the storage class. */
    //initUploadRequest.MetaData().addHeader("x-oss-storage-class", "Standard");

    /* Initiate the multipart upload task. */
    auto uploadIdResult = client.InitiateMultipartUpload(initUploadRequest);
    /* Cancel the multipart upload task or list uploaded parts based on the upload ID. */
    /* If you want to cancel a multipart upload task based on the upload ID, obtain the upload ID after you call the InitiateMultipartUpload operation to initiate the multipart upload task. */
    /* If you want to list the uploaded parts in a multipart upload task based on the upload ID, obtain the upload ID after you call the InitiateMultipartUpload operation to initiate the multipart upload task but before you call the CompleteMultipartUpload operation to complete the multipart upload task. */ 
    auto uploadId = uploadIdResult.result().UploadId();
    std::string fileToUpload = "yourLocalFilename";
    int64_t partSize = 100 * 1024;
    PartList partETagList;
    auto fileSize = getFileSize(fileToUpload);
    int partCount = static_cast<int>(fileSize / partSize);
    /* Calculate the number of parts. */
    if (fileSize % partSize != 0) {
        partCount++;
    }

    /* Upload each part. */
    for (int i = 1; i <= partCount; i++) {
        auto skipBytes = partSize * (i - 1);
        auto size = (partSize < fileSize - skipBytes) ? partSize : (fileSize - skipBytes);
        std::shared_ptr<std::iostream> content = std::make_shared<std::fstream>(fileToUpload, std::ios::in|std::ios::binary);
        content->seekg(skipBytes, std::ios::beg);

        UploadPartRequest uploadPartRequest(BucketName, ObjectName, content);
        uploadPartRequest.setContentLength(size);
        uploadPartRequest.setUploadId(uploadId);
        uploadPartRequest.setPartNumber(i);
        auto uploadPartOutcome = client.UploadPart(uploadPartRequest);
        if (uploadPartOutcome.isSuccess()) {
            Part part(i, uploadPartOutcome.result().ETag());
            partETagList.push_back(part);
        }
        else {
            std::cout << "uploadPart fail" <<
            ",code:" << uploadPartOutcome.error().Code() <<
            ",message:" << uploadPartOutcome.error().Message() <<
            ",requestId:" << uploadPartOutcome.error().RequestId() << std::endl;
        }

    }

    /* Complete the multipart upload task. */
    /* When the multipart upload task is completed, you need to provide all valid PartETags. After OSS receives the PartETags, OSS verifies all parts one by one. After part verification is successful, OSS combines these parts into a complete object. */
    CompleteMultipartUploadRequest request(BucketName, ObjectName);
    request.setUploadId(uploadId);
    request.setPartList(partETagList);
    /* (Optional) Specify the ACL of the object. */
    //request.setAcl(CannedAccessControlList::Private);

    auto outcome = client.CompleteMultipartUpload(request);

    if (!outcome.isSuccess()) {
        /* Handle exceptions. */
        std::cout << "CompleteMultipartUpload fail" <<
        ",code:" << outcome.error().Code() <<
        ",message:" << outcome.error().Message() <<
        ",requestId:" << outcome.error().RequestId() << std::endl;
        return -1;
    }

    /* Release resources such as network resources. */
    ShutdownSdk();
    return 0;
}

アップロードされたパーツを取得するときにCompleteMultipartUpload操作を呼び出すには、各パーツのETag値を指定する必要があります。 次のいずれかの方法を使用して、各部品のETag値を取得できます。

• パーツがアップロードされると、そのパーツのETag値がレスポンスに含まれます。 このETag値を保存して使用できます。 このメソッドは、前のサンプルコードで使用されています。

• ListParts操作を呼び出して、アップロードされたパーツのETag値を照会します。

アップロードしたパーツの一覧表示

ListPartsメソッドを呼び出して、指定したアップロードIDを使用するすべてのアップロード済みパーツを一覧表示できます。

#include <alibabacloud/oss/OssClient.h>
using namespace AlibabaCloud::OSS;

int main(void)
{
    /* Initialize information about the account that is used to access OSS. */
    
    /* Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. */
    std::string Endpoint = "yourEndpoint";
    /* Specify the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the region to cn-hangzhou. */
    std::string Region = "yourRegion";
    /* Specify the name of the bucket. Example: examplebucket. */
    std::string BucketName = "examplebucket";
    /* Specify the full path of the object. Do not include the bucket name in the full path of the object. Example: exampledir/exampleobject.txt. */
    std::string ObjectName = "exampledir/exampleobject.txt";
    /* Specify the upload ID. You can obtain the upload ID from the response to the InitiateMultipartUpload operation. You must obtain the upload ID before you call the CompleteMultipartUpload operation to complete the multipart upload task. */
    std::string UploadId = "yourUploadId";

    /* Initialize resources such as network resources. */
    InitializeSdk();

    ClientConfiguration conf;
    conf.signatureVersion = SignatureVersionType::V4;
    /* Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. */
    auto credentialsProvider = std::make_shared<EnvironmentVariableCredentialsProvider>();
    OssClient client(Endpoint, credentialsProvider, conf);
    client.SetRegion(Region);

    /* List the uploaded parts. By default, 1,000 parts are listed. */
    ListPartsRequest listuploadrequest(BucketName, ObjectName);
    listuploadrequest.setUploadId(UploadId);

    bool IsTruncated = false;

    do {
        auto listUploadResult = client.ListParts(listuploadrequest);
        if (!listUploadResult.isSuccess()) {
            /* Handle exceptions. */
            std::cout << "ListParts fail" <<
            ",code:" << listUploadResult.error().Code() <<
            ",message:" << listUploadResult.error().Message() <<
            ",requestId:" << listUploadResult.error().RequestId() << std::endl;
            break;
        }
        else {
            for (const auto& part : listUploadResult.result().PartList()) {
                std::cout << "part"<<
                ",name:" << part.PartNumber() <<
                ",size:" << part.Size() <<
                ",etag:" << part.ETag() <<
                ",lastmodify time:" << part.LastModified() << std::endl;
            }
        }
        listuploadrequest.setPartNumberMarker(listUploadResult.result().NextPartNumberMarker());
        IsTruncated = listUploadResult.result().IsTruncated();
    } while (IsTruncated);

    /* Release resources such as network resources. */
    ShutdownSdk();
    return 0;
}

#include <alibabacloud/oss/OssClient.h>
using namespace AlibabaCloud::OSS;

int main(void)
{
    /* Initialize information about the account that is used to access OSS. */
    
    /* Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. */
    std::string Endpoint = "yourEndpoint";
    /* Specify the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the region to cn-hangzhou. */
    std::string Region = "yourRegion";
    /* Specify the name of the bucket. Example: examplebucket. */
    std::string BucketName = "examplebucket";
    /* Specify the full path of the object. Do not include the bucket name in the full path of the object. Example: exampledir/exampleobject.txt.  */
    std::string ObjectName = "exampledir/exampleobject.txt";
    /* Specify the upload ID. You can obtain the upload ID from the response to the InitiateMultipartUpload operation. You must obtain the upload ID before you call the CompleteMultipartUpload operation to complete the multipart upload task. */
    std::string UploadId = "yourUploadId";

    /* Initialize resources such as network resources. */
    InitializeSdk();

    ClientConfiguration conf;
    conf.signatureVersion = SignatureVersionType::V4;
    /* Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. */
    auto credentialsProvider = std::make_shared<EnvironmentVariableCredentialsProvider>();
    OssClient client(Endpoint, credentialsProvider, conf);
    client.SetRegion(Region);

    /* List all uploaded parts by page. */
    /* Set the maximum number of parts to list on each page. */
    ListPartsRequest listuploadrequest(BucketName, ObjectName);
    listuploadrequest.setMaxParts(50);
    listuploadrequest.setUploadId(UploadId);
    bool IsTruncated = false;
    do {
        auto listuploadresult = client.ListParts(listuploadrequest);
        if (!listuploadresult.isSuccess()) {
            /* Handle exceptions. */
            std::cout << "ListParts fail" <<
            ",code:" << listuploadresult.error().Code() <<
            ",message:" << listuploadresult.error().Message() <<
            ",requestId:" << listuploadresult.error().RequestId() << std::endl;
            break;
        }
        else {
            for (const auto& part : listuploadresult.result().PartList()) {
                std::cout << "part"<<
                ",name:" << part.PartNumber() <<
                ",size:" << part.Size() <<
                ",etag:" << part.ETag() <<
                ",lastmodify time:" << part.LastModified() << std::endl;
            }
        }  
        listuploadrequest.setPartNumberMarker(listuploadresult.result().NextPartNumberMarker());
        IsTruncated = listuploadresult.result().IsTruncated();
    } while (IsTruncated);

    /* Release resources such as network resources. */
    ShutdownSdk();
    return 0;
}

マルチパートアップロードタスクの一覧表示

ListMultipartUploadsメソッドを呼び出して、進行中のすべてのマルチパートアップロードタスクを一覧表示できます。 進行中のマルチパートアップロードタスクは、開始されたが完了またはキャンセルされていないタスクです。

#include <alibabacloud/oss/OssClient.h>
using namespace AlibabaCloud::OSS;

int main(void)
{
    /* Initialize information about the account that is used to access OSS. */
    
    /* Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. */
    std::string Endpoint = "yourEndpoint";
    /* Specify the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the region to cn-hangzhou. */
    std::string Region = "yourRegion";
    /* Specify the name of the bucket. Example: examplebucket. */
    std::string BucketName = "examplebucket";    

    /* Initialize resources such as network resources. */
    InitializeSdk();

    ClientConfiguration conf;
    conf.signatureVersion = SignatureVersionType::V4;
    /* Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. */
    auto credentialsProvider = std::make_shared<EnvironmentVariableCredentialsProvider>();
    OssClient client(Endpoint, credentialsProvider, conf);
    client.SetRegion(Region);

    /* List the completed multipart upload tasks. By default, 1,000 parts are listed. */
    ListMultipartUploadsRequest listmultiuploadrequest(BucketName);
	bool IsTruncated = false;
    do {
        auto listresult = client.ListMultipartUploads(listmultiuploadrequest);
        if (!listresult.isSuccess()) {
            /* Handle exceptions. */
            std::cout << "ListMultipartUploads fail" <<
            ",code:" << listresult.error().Code() <<
            ",message:" << listresult.error().Message() <<
            ",requestId:" << listresult.error().RequestId() << std::endl;
            break;
        }
        else {
            for (const auto& part : listresult.result().MultipartUploadList()) {
                std::cout << "part"<<
                ",name:" << part.Key <<
                ",uploadid:" << part.UploadId <<
                ",initiated time:" << part.Initiated << std::endl;
            }
        }
        listmultiuploadrequest.setKeyMarker(listresult.result().NextKeyMarker()); 
        listmultiuploadrequest.setUploadIdMarker(listresult.result().NextUploadIdMarker());
        IsTruncated = listresult.result().IsTruncated();
    } while (IsTruncated);

    /* Release resources such as network resources. */
    ShutdownSdk();
    return 0;
}

#include <alibabacloud/oss/OssClient.h>
using namespace AlibabaCloud::OSS;

int main(void)
{
    /* Initialize information about the account that is used to access OSS. */
    
    /* Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. */
    std::string Endpoint = "yourEndpoint";
    /* Specify the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the region to cn-hangzhou. */
    std::string Region = "yourRegion";
    /* Specify the name of the bucket. Example: examplebucket. */
    std::string BucketName = "examplebucket"; 

    /* Initialize resources such as network resources. */
    InitializeSdk();

    ClientConfiguration conf;
    conf.signatureVersion = SignatureVersionType::V4;
    /* Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. */
    auto credentialsProvider = std::make_shared<EnvironmentVariableCredentialsProvider>();
    OssClient client(Endpoint, credentialsProvider, conf);
    client.SetRegion(Region);

    /* List all multipart upload tasks by page. */
    /* Set the maximum number of multipart upload tasks to list on each page. */
    ListMultipartUploadsRequest  listmultiuploadrequest(BucketName);
    listmultiuploadrequest.setMaxUploads(50);
    bool IsTruncated = false;
    do {
        auto listresult = client.ListMultipartUploads(listmultiuploadrequest);
        if (!listresult.isSuccess()) {
            /* Handle exceptions. */
            std::cout << "ListMultipartUploads fail" <<
            ",code:" << listresult.error().Code() <<
            ",message:" << listresult.error().Message() <<
            ",requestId:" << listresult.error().RequestId() << std::endl;
            break;
        }
        else {
            for (const auto& part : listresult.result().MultipartUploadList()) {
                std::cout << "part"<<
                ",name:" << part.Key <<
                ",uploadid:" << part.UploadId <<
                ",initiated time:" << part.Initiated << std::endl;
            }
        }  
        listmultiuploadrequest.setKeyMarker(listresult.result().NextKeyMarker()); 
        listmultiuploadrequest.setUploadIdMarker(listresult.result().NextUploadIdMarker());
        IsTruncated = listresult.result().IsTruncated();
    } while (IsTruncated);

    /* Release resources such as network resources. */
    ShutdownSdk();
    return 0;
}

マルチパートアップロードタスクのキャンセル

client.AbortMultipartUploadメソッドを呼び出して、マルチパートアップロードタスクをキャンセルできます。 マルチパートアップロードタスクがキャンセルされた場合、アップロードIDを使用してパーツをアップロードすることはできません。 さらに、アップロードされたパーツは削除されます。

#include <alibabacloud/oss/OssClient.h>
using namespace AlibabaCloud::OSS;

int main(void)
{
    /* Initialize information about the account that is used to access OSS. */
    
    /* Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. */
    std::string Endpoint = "yourEndpoint";
    /* Specify the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the region to cn-hangzhou. */
    std::string Region = "yourRegion";
    /* Specify the name of the bucket. Example: examplebucket. */
    std::string BucketName = "examplebucket";
    /* Specify the full path of the object. Do not include the bucket name in the full path of the object. Example: exampledir/exampleobject.txt.  */
    std::string ObjectName = "exampledir/exampleobject.txt";
    /* Specify the upload ID. You can obtain the upload ID from the response to the InitiateMultipartUpload operation. */
    std::string UploadId = "yourUploadId";

    /* Initialize resources such as network resources. */
    InitializeSdk();

    ClientConfiguration conf;
    conf.signatureVersion = SignatureVersionType::V4;
    /* Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. */
    auto credentialsProvider = std::make_shared<EnvironmentVariableCredentialsProvider>();
    OssClient client(Endpoint, credentialsProvider, conf);
    client.SetRegion(Region);
  
    InitiateMultipartUploadRequest initUploadRequest(BucketName, ObjectName);

    /* Initiate the multipart upload task. */
    auto uploadIdResult = client.InitiateMultipartUpload(initUploadRequest);
    auto uploadId = uploadIdResult.result().UploadId();

    /* Cancel the multipart upload task. */
    AbortMultipartUploadRequest  abortUploadRequest(BucketName, ObjectName, uploadId);
    auto outcome = client.AbortMultipartUpload(abortUploadRequest);

    if (!outcome.isSuccess()) {
        /* Handle exceptions. */
        std::cout << "AbortMultipartUpload fail" <<
        ",code:" << outcome.error().Code() <<
        ",message:" << outcome.error().Message() <<
        ",requestId:" << outcome.error().RequestId() << std::endl;
        return -1;
    }

    /* Release resources such as network resources. */
    ShutdownSdk();
    return 0;
}

関連ドキュメント

• マルチパートアップロードの実行に使用する完全なサンプルコードについては、『GitHub』をご参照ください。

• マルチパートアップロードには3つのAPI操作が含まれます。 操作の詳細については、以下のトピックを参照してください。

  • InitiateMultipartUpload

  • UploadPart

  • CompleteMultipartUpload

• マルチパートアップロードタスクをキャンセルするために呼び出すことができるAPI操作の詳細については、「AbortMultipartUpload」をご参照ください。

• アップロードされたパーツを一覧表示するために呼び出すことができるAPI操作の詳細については、「ListUploadedParts」をご参照ください。

• 実行中のすべてのマルチパートアップロードタスクを一覧表示するために呼び出すことができるAPI操作の詳細については、「ListMultipartUploads」をご参照ください。 進行中のマルチパートアップロードタスクには、開始されたが完了またはキャンセルされていないタスクが含まれます。