All Products
Search

This Product

    Object Storage Service:Copy objects

    Document Center

    Object Storage Service:Copy objects

    Last Updated:Oct 21, 2024

    You can copy objects to another directory in the same Object Storage Service (OSS) bucket or to another bucket that is located in the same region as the source bucket without changing the object content. In ossutil, you can use the cp command to copy objects.

    Usage notes

    • To copy an object, you must have the oss:GetObject, oss:ListObjects, and oss:PutObject permissions. For more information, see Attach a custom policy to a RAM user.

    • For ossutil 1.6.16 and later, you can directly use ossutil as the binary name in the command line. You do not need to update the binary name based on the operating system. For ossutil earlier than 1.6.16, you need to update the binary name based on the operating system. For more information, see ossutil command reference.

    • You cannot run the cp command to copy objects to a bucket that belongs to another Alibaba Cloud account or to a bucket that is located in a different region. If you want to copy objects across Alibaba Cloud accounts or regions, use ossimport or Data Online Migration.

    • You can use the cp command to copy objects, but not parts.

    • Appendable objects cannot be copied.

    • You must have the read permissions on the source object and the read and write permissions on the destination bucket. Otherwise, the copy operation fails. For more information about how to copy an object, see Copy objects.

    Command syntax

    ossutil cp cloud_url cloud_url
[-r, --recursive]
[-f --force]
[-u --update]
[--disable-ignore-error]
[--only-current-dir]
[--bigfile-threshold <value>]
[--part-size <value>]
[--checkpoint-dir <value>]
[--encoding-type <value>]
[--include <value>]
[--exclude <value>]
[--meta <value>]
[--acl <value>]
[--disable-crc64]
[--payer <value>]
[-j, --job <value>]
[--parallel <value>]
[--version-id <value>]
[--start-time <value>]
[--end-time <value>]

    The following table describes the parameters and options in the preceding command.

    Parameter/Option

    Description

    cloud_url

    The paths of the source and destination objects. Format: oss://bucketname/objectname. For example, to copy a source object named srcobject.jpg from the examplebucket bucket to a destination object named destobject.jpg in the same bucket, set the path of the source object to oss://examplebucket/srcobject.jpg and the path of the destination object to oss://examplebucket/destobject.jpg.

    -r, --recursive

    The recursive operations. If you specify this option, ossutil performs operations on all objects in the bucket that meet the specified conditions. If you do not specify this option, ossutil performs operations only on the specified object.

    -f --force

    Specifies that the command is forcibly run without a prompt for confirmation.

    -u, --update

    Specifies that ossutil copies the source object only when the object does not exist in the destination bucket or when the last modified time of the source object is later than that of the destination object.

    --disable-ignore-error

    Specifies that errors are not ignored during batch operations.

    --only-current-dir

    Specifies that only objects in the current directory are copied. Subdirectories and objects in these subdirectories are not copied.

    -bigfile-threshold

    The object size threshold for using resumable upload for the operation. Unit: bytes.

    Default value: 100 MB.

    Valid values: 0 to 9223372036854775807.

    --part-size

    Specifies the part size. Unit: bytes. By default, ossutil determines the part size based on the object size.

    Valid values: 1 to 9223372036854775807.

    --checkpoint-dir

    The directory in which the checkpoint information of the resumable upload copy task is stored. If the task fails, ossutil automatically creates a directory named .ossutil_checkpoint and saves the checkpoint information in the directory. After the task is successful, ossutil deletes the directory. If you specify the --checkpoint-dir option, make sure that the directory can be deleted.

    --encoding-type

    The method that is used to encode the names of objects. Set the value to url. If you do not specify this option, the names of objects are not encoded.

    --include

    Specifies that objects that meet the specified conditions are copied.

    --exclude

    Specifies that all objects that do not meet the specified conditions are copied.

    --meta

    The object metadata. Specify object metadata in the header:value#header:value format. Example: Cache-Control:no-cache#Content-Encoding:gzip. For more information, see set-meta.

    --acl

    The access control list (ACL) of the objects. Valid values:

    • default: The ACL of the object is the same as the ACL of the bucket in which the objects are stored. This is the default ACL setting of the destination object.

    • private: Only the bucket owner can perform read and write operations on the objects in the bucket. Other users cannot access the objects in the bucket.

    • public-read: Only the bucket owner can perform write operations on the objects in the bucket. Other users, including anonymous users, can perform only read operations on the objects in the bucket. This may result in unrestricted access to the data in your bucket and unexpectedly high fees. If a user uploads prohibited data or information, your legitimate interests and rights may be infringed. We recommend that you do not set the object ACL to public-read except in special cases.

    • public-read-write: All users, including anonymous users, can perform read and write operations on the objects in the bucket. This may result in unrestricted access to the data in your bucket and unexpectedly high fees. Exercise caution when you set the object ACL to public-read-write.

    --disable-crc64

    Specifies that CRC-64 is disabled. By default, ossutil enables CRC-64 during data transmission.

    --payer

    The payer of the request. If you want the requester who accesses the resources in the specified path to pay the traffic and request fees, set this parameter to requester.

    -j, --jobs

    The number of concurrent tasks that can be performed on multiple objects. Valid values: 1 to 10000. Default value: 3.

    --parallel

    The number of concurrent tasks that can be performed on a single object. Valid values: 1 to 10000. If you do not configure this option, ossutil specifies a value based on the operation type and the object size.

    --version-id

    The version ID of the object that you want to copy. You can specify this option for objects only in a bucket for which versioning is enabled.

    --start-time

    The beginning of the time range to filter objects based on the last modified time. The value is a timestamp that follows the UNIX time format. Objects whose last modified time is earlier than the UNIX timestamp are ignored.

    Note

    Only ossutil 1.7.18 and later support this parameter. For more information about how to update ossutil, see update.

    --end-time

    The end of the time range to filter objects based on the last modified time. The value is a timestamp that follows the UNIX time format. Objects whose last modified time is later than the UNIX timestamp are ignored.

    Note

    • If both --start-time and --end-time are specified, the command takes effect only on objects whose last modified time is within the range specified by --start-time and --end-time.

    • Only ossutil 1.7.18 and later support this parameter. For more information about how to update ossutil, see update.

    If the default settings of the options do not meet your performance requirements, you can use the -j, --jobs and --parallel options to adjust performance. By default, ossutil calculates the value of --parallel based on the object size. When you copy multiple large objects, the actual number of concurrent tasks is calculated by multiplying the value of -j, --jobs by the value of --parallel.

    • We recommend that you adjust the concurrency value to a value smaller than 100 if the resources such as network bandwidth, memory, and CPUs of your Elastic Compute Service (ECS) instance or server are limited. If resources, such as network bandwidth, memory, and CPUs, are left idle, you can increase the number of concurrent tasks.

    • If the number of concurrent tasks is excessively large, the copy operation performance of ossutil may be degraded because threads switch and compete for resources. In addition, an end-of-file (EOF) error may occur. To prevent these issues, you can adjust the values of -j, --jobs and --parallel based on your available resources. To perform stress testing, specify small values for the two options before you incrementally increase them to the optimal values.

    Sample environment

    The examples in this topic show how to copy objects between different directories in the same bucket and between buckets in Linux. To use the sample commands in your actual business scenario, you need to modify the option and parameter settings based on your operating system and environment. Sample environment:

    • Operating system: Linux

    • Source bucket: examplebucket1

    • Directory 1 in the source bucket: srcfolder1

    • Directory 2 in the source bucket: srcfolder2

    • Destination bucket: examplebucket2

    • Directory in the destination bucket: desfolder

    Specify whether to copy the source directory

    If the source path does not end with a forward slash (/), all objects whose names contain the specified prefix in the source bucket are copied to the destination bucket. If the source path ends with a forward slash (/), only the objects in the specified directory are copied to the destination bucket.

    For example, the following objects are stored in the srcfolder1 directory in the source bucket named examplebucket1:

    srcfolder1/exampleobject1.txt
srcfolder1/exampleobject2.png
srcfolder1/dir1/
srcfolder1/dir1/exampleobject3.jpg
srcfolder1/dir2/
srcfolder1/dir2/exampleobject4.jpg

    • Copy objects from a path that does not end with a forward slash (/)

      ossutil cp oss://examplebucket1/srcfolder1  oss://examplebucket2 -r

      The following objects are copied from the source bucket to the destination bucket named examplebucket2:

      srcfolder1/exampleobject1.txt
srcfolder1/exampleobject2.png
srcfolder1/dir1/
srcfolder1/dir1/exampleobject3.jpg
srcfolder1/dir2/
srcfolder1/dir2/exampleobject4.jpg

    • Copy objects from a path that ends with a forward slash (/)

      ossutil cp oss://examplebucket1/srcfolder1/  oss://examplebucket2 -r

      The following objects are copied from the source bucket to the destination bucket named examplebucket2:

      exampleobject1.txt
exampleobject2.png
dir1/
dir1/exampleobject3.jpg
dir2/
dir2/exampleobject4.jpg

    Perform a simple copy

    • Copy a single object 

      ossutil cp oss://examplebucket1/examplefile.txt oss://examplebucket1/srcfolder2/

    • Copy incremental objects

      If you specify the --update option when you copy multiple objects, ossutil copies the objects only when the destination objects do not exist or when the last modified time of the source objects is later than that of the destination objects. Run the following command to copy incremental objects: 

      ossutil cp oss://examplebucket1/srcfolder1/ oss://examplebucket2/path2/ -r --update

      You can use this option to skip copied objects and copy only objects that fail to be copied.

    • Copy multiple objects at a time

      Run the following command to copy objects in the srcfolder1 directory whose last modified time is within the range of 10:09:18 (UCT+8) October 31, 2023 to 12:55:58 (UTC+8) October 31, 2023: 

      ossutil cp -r oss://examplebucket1/srcfolder1/ oss://examplebucket2/path2/ --start-time 1698718158 --end-time 1698728158

    • Rename an object

      ossutil cp oss://examplebucket1/examplefile.txt oss://examplebucket1/example.txt

      When you run the cp command to rename an object, the original object is retained. You can delete the original object after you rename the object.

    • Copy only objects in the current directory and skip objects in subdirectories 

      ossutil cp oss://examplebucket1/srcfolder1/ oss://examplebucket1/srcfolder2/ --only-current-dir -r

    • Change the metadata of an object

      When you copy an object, you can use the --meta option to change the metadata of the object. The value of this option is in the header:value#header:value... format. 

      ossutil cp oss://examplebucket1/examplefile.txt oss://examplebucket1/ --meta=Cache-Control:no-cache

    • Copy an object from a bucket for which pay-by-requester is enabled to a bucket for which pay-by-requester is disabled 

      ossutil cp oss://examplebucket1/examplefile.txt oss://examplebucket2/desfolder/  --payer=requester

    Copy multiple objects that meet specific conditions

    To copy objects that meet specific conditions, you can specify --include and --exclude.

    • Copy all objects that are not in the JPG format 

      ossutil cp oss://examplebucket1/srcfolder1/ oss://examplebucket2/desfolder/ --exclude "*.jpg" -r

    • Copy all objects whose names contain abc and are not in the JPG or TXT format 

      ossutil cp oss://examplebucket1/srcfolder1/ oss://examplebucket2/desfolder/ --include "*abc*" --exclude "*.jpg" --exclude "*.txt" -r

    Change the storage class of an object

    When you overwrite an object, you can specify the --meta option to change the storage class of the object. Valid values:

    • Standard

    • IA

    • Archive

    • ColdArchive

    • Deep Cold Archive

    For more information about storage classes, see Overview.

    Important

    If you use the -- meta option to change the storage class of an object, the existing user metadata of the object is overwritten by default. If you want to convert the storage class and retain the existing user metadata of the object, you must retain the existing metadata by using the x-oss-metadata-directive:COPY option before the conversion.

    Overwrite the existing user metadata of an object

    • Change the storage class of an object to Archive

      ossutil cp oss://examplebucket1/srcfolder1/examplefile.txt oss://examplebucket1/srcfolder1/examplefile.txt --meta X-oss-Storage-Class:Archive

    • Change the storage class of all objects in a specific directory to Standard

      ossutil cp oss://examplebucket1/srcfolder1/ oss://examplebucket1/srcfolder1/ --meta X-oss-Storage-Class:Standard -r

    Retain the existing user metadata of an object

    • Run the following command to change the storage class of a specific object to IA and retain the existing user metadata of the object: 

      ossutil cp oss://examplebucket1/srcfolder1/examplefile.txt oss://examplebucket1/srcfolder1/examplefile.txt --meta x-oss-metadata-directive:COPY

    • Run the following command to change the storage class of all objects in a specific directory to Standard and retain the existing user metadata of the objects: 

      ossutil cp oss://examplebucket1/srcfolder1/ oss://examplebucket1/srcfolder1/ --meta x-oss-metadata-directive:COPY -r -f
    Important

    • If you change the storage class of an object by running the cp command, you are charged for PUT requests based on the storage class of the source object. The API operation calling fee is included in the bill for the destination bucket.

    • If you change the storage class of an object to IA, Archive, Cold Archive, or Deep Cold Archive, and the object is stored for less than the minimum storage duration, you are charged for the storage usage for the object that is stored for less than the minimum storage duration. For more information, see Storage fees.

    • Before you can change the storage class of Archive, Cold Archive, or Deep Cold Archive objects to Standard or IA by running the cp command, run the restore command to restore the objects. If you enable real-time access of Archive objects, you can change the storage class of Archive objects without the need to restore them. For more information, see Real-time access of Archive objects.

    • When you run the cp command to change the storage class of an object that is larger than 100 MB, ossutil calculates an appropriate part size based on the object size by default. If the part size cannot meet your requirements, you can use --part-size to change the part size. Make sure that the number of parts is smaller than 10,000.

    Modify the tags of an object

    You can add --tagging to the cp command to modify the tags of an object. Separate the tags with ampersands (&). Example:

    ossutil cp oss://examplebucket1/examplefile.txt oss://examplebucket1/ --tagging "abc=1&bcd=2&……"

    For more information about object tagging, see object-tagging.

    Copy and encrypt an object

    When you copy an object, you can specify the server-side encryption method to encrypt the object. For more information, see Server-side encryption.

    • Copy an object and set the server-side encryption method to AES-256 for the destination object 

      ossutil cp oss://examplebucket1/examplefile.txt oss://examplebucket1/srcfolder2/ --meta=x-oss-server-side-encryption:AES256

    • Copy an object and set the server-side encryption method to SSE-KMS for the destination object 

      ossutil cp oss://examplebucket1/examplefile.txt oss://examplebucket2/desfolder/ --meta=x-oss-server-side-encryption:KMS
      Important

      When you use Key Management Service (KMS) to encrypt an object, OSS creates a customer master key (CMK) in KMS for the object. You are charged a small amount of fees when the KMS API operation is called. For more information, see Billing of KMS.

    • Copy an object, set the encryption method of the object to SSE-KMS for the destination object, and then specify a CMK ID

      ossutil cp oss://examplebucket1/examplefile.txt oss://examplebucket2/desfolder/ --meta=x-oss-server-side-encryption:KMS#x-oss-server-side-encryption-key-id:7bd6e2fe-cd0e-483e-acb0-f4b9e1******

    Restore objects in a versioning-enabled bucket

    After you enable versioning for a bucket, objects that are overwritten or deleted in the bucket are saved as previous versions. You can add --version-id to the cp command to overwrite a previous version with the latest version. Example:

    ossutil cp oss://examplebucket1/examplefile.txt oss://examplebucket2/ --version-id  CAEQARiBgID8rumR2hYiIGUyOTAyZGY2MzU5MjQ5ZjlhYzQzZjNlYTAyZDE3MDRk
    Important

    • To use the --version-id option, you need to run the ls --all-versions command to obtain the version IDs of the object.

    • The --version-id option can be used only for objects in versioning-enabled buckets. For more information about how to enable versioning for a bucket, see bucket-versioning.

    Common options

    If you use ossutil to switch to a bucket that is located in another region, add the -e option to specify the endpoint of the region in which the bucket is located. If you use ossutil to switch to a bucket that belongs to another Alibaba Cloud account, add the -i option to specify the AccessKey ID of the specified account, and add the -k option to specify the AccessKey secret of the specified account.

    For example, you can run the following command to copy the srcobject.png object in the root directory of a source bucket named examplebucket to a destination bucket named destbucket. The destination bucket is located in the China (Shanghai) region and is owned by another Alibaba Cloud account.

    ossutil cp oss://examplebucket/srcobject.png  oss://destbucket  -e oss-cn-shanghai.aliyuncs.com -i LTAI4Fw2NbDUCV8zYUzA****  -k 67DLVBkH7EamOjy2W5RVAHUY9H****

    For more information about common options, see Common options.