High-level commands - Object Storage Service - Alibaba Cloud Documentation Center

This topic describes high-level commands in ossutil.

Commands

Command set	Supported option
All high-level commands	--encoding-type string: the encoding method of the object name or file name. Valid value: url. --request-payer string: the payer of the request. If pay-by-requester is enabled, specify this option. Valid value: requester.
High-level commands that support batch processing	--start-after/--end-with: specifies the range of object names that you want to query. The range is a left-open, right-closed interval. filter: specifies filter rules for objects and files by name, path, size, time, and metadata (only for objects). For more information, see Filter options. --limited-num: limits the number of results in the response. --recursive/-r: specifies that the operation is recursively performed on all files or objects, and all subdirectories in the root directory. --dirs/-d: matches files or objects only in the root directory and ignores subdirectories in the root directory. Note If the option is applied to objects, a delimiter is used to simulate a directory, and all objects with the specified name prefix are scanned. The amount of time used to scan objects is directly proportional to the number of the objects with the specified name prefix. --force/-f: forces the operation without requiring confirmation. --list-objects: lists objects by calling the ListObjects operation.
High-level commands that support filtering by destination object	--update: performs the operation only if the last modified time of the source file or object is later than that of the destination object or file. --size-only: performs the operation only if the source file or object has a different size from the destination object or file. --checksum: specifies that the operation is performed only when one of the following conditions is met: a. the size of the source object is different from that of the destination object; b. the CRC-64 of the source object is different from that of the destination object, but the size of the source object is the same as that of the destination object. If the source or destination object does not have a CRC-64 value, the CRC-64 of the source object is different from that of the destination object. This method is only used for copying objects from one bucket to another. --ignore-existing: skips objects that already exist.
High-level commands that apply to individual objects	--version-id string: the version ID of the object.
High-level commands that support the list mode	--list-format: the format of the list file. Valid values: plain and inventory. --list-manifest-from: read the format description of the list file from the file. You are required to specify this parameter if the format is inventory.

Local options

Local option	Type	Description
--encoding-type	string	The encoding method used to encode the object or file name. Valid value: url.
--request-payer	string	The payer of the request. If pay-by-requester is enabled, specify this option. Valid value: requester.
--start-after	string	Matches objects whose names are alphabetically after the value of the --start-after option.
--end-with	string	Matches objects whose names are alphabetically before or the same as the value of the --end-with option.
--page-size	int	The maximum number of objects to return. Default value: 1000. Valid values: 1 to 1000.
--include	stringArray	The inclusion rule for the path or object name.
--include-from	stringArray	Reads inclusion rules from a file.
--exclude	stringArray	The exclusion rule for the path or object name.
--exclude-from	stringArray	Reads exclusion rules from a file.
--filter	stringArray	The filter rule for paths or object names.
--files-from	stringArray	Reads a list of source object names from a file, with empty lines or comment lines ignored.
--files-from-raw	stringArray	Reads a list of source object names from a file, without ignoring empty lines and comment lines.
--max-age	Duration	Excludes objects whose last modified time is earlier than the value of this option. The default unit is s (seconds). You can suffix the numerical value with a unit: ms for milliseconds, s for seconds, m for minutes, h for hours, d for days, w for weeks, M for months, or y for years.
--min-age	Duration	Excludes objects whose last modified time is later than the value of this option. The default unit is s (seconds). You can suffix the numerical value with a unit: ms for milliseconds, s for seconds, m for minutes, h for hours, d for days, w for weeks, M for months, or y for years.
--max-mtime	Time	Excludes objects whose last modified time is later than the value of this option. The time must be in UTC.
--min-mtime	Time	Excludes objects whose last modified time is earlier than the value of this option. The time must be in UTC.
--max-size	SizeSuffix	The maximum size of a file or object that can be transferred. By default, the size is in bytes (B). You can suffix the numerical value with a 1024-based unit: B for byte, K for Kibibyte, M for Mebibyte, G for Gibibyte, T for Tebibyte, or P for Pebibyte.
--min-size	SizeSuffix	The minimum size of a file or object that can be transferred. By default, the size is in bytes (B). You can suffix the numerical value with a 1024-based unit: B for byte, K for Kibibyte, M for Mebibyte, G for Gibibyte, T for Tebibyte, or P for Pebibyte.
--metadata-exclude	stringArray	The exclusion rule for object metadata.
--metadata-include	stringArray	The inclusion rule for object metadata.
--metadata-filter	stringArray	The filter rule for object metadata.
--metadata-filter-from	stringArray	Reads metadata filter rules from a file.
--checksum	/	Specifies that only when one of the following conditions is met, the source object is copied: a. the size of the source object is different from that of the destination object; b. the CRC-64 of the source object is different from that of the destination object, but the size of the source object is the same as that of the destination object.
--size-only	/	Copies a source file or object only if it has a different file size from the destination object or file.
-u, --update	/	Copies a source file or object only if its last modified time is later than that of its destination object or file.
-r, --recursive	/	Specifies that the command performs the operation recursively. If this option is specified, the command performs the operation on all matching objects in the bucket. If this option is not specified, the command performs the operation only on the specified objects in the specified path.
-d, --dirs	/	Matches files or objects and subdirectories in the current directory.
-f, --force	/	Forces the operation without requiring confirmation.
--list-objects	/	Lists objects by calling the ListObjects operation.
--no-progress	/	Hides the progress bar.
-j, --job	int	The number of concurrent jobs for processing multiple files or objects. Default value: 3.
--checkers	int	The number of checkers that can run in parallel. Default value: 16.
--parallel	int	The number of concurrent jobs for processing a single file or object.
--part-size	SizeSuffix	The part size. By default, a part size is automatically calculated based on the object size. The part size ranges from 100 KiB to 5 GiB.
--bigfile-threshold	int	The object size threshold for using multipart upload, download, or copy. Default value: 104857600.
--version-id	string	The version ID of the object.
--storage-class	string	The storage class of the object. Valid values: Standard, IA, Archive, ColdArchive, and DeepColdArchive.
--acl	string	The access control list (ACL) of the object. Valid values: private, public-read, public-read-write, and default.
--cache-control	string	The caching behavior in web browsers when the object is downloaded.
--content-disposition	string	The display mode for the object.
--content-encoding	string	The encoding method used to encode the object.
--content-type	string	The content type of the object.
--expires	string	The absolute expiration time of the cached content.
--copy-props	string	The attributes to copy from the source object. Valid values: none, metadata, and default.
--tagging	string	The object tags in key=value pairs.
--tagging-directive	string	The method used to configure the tags of the destination object. Valid values: COPY and REPLACE.
--metadata	string	The user metadata in key=value pairs.
--metadata-directive	string	The method used to set metadata for the destination object. Valid values: COPY and REPLACE.
--list-format	string	The format of the list file. Valid values: plain and inventory.
--list-manifest-from	string	Read the format description of the list file from the file. You are required to specify this parameter if the format is inventory.

Filter options

Filter options are applicable to commands that support batch processing, such as ls, cp, sync, and rm. Filter options match files or objects by path, file or object name, path list, file list, object list, last modified time, and file or object size. Metadata-based filtering applies only to objects.

Filter options can be used only in command lines. They cannot be specified in the configuration file.

You can use the ls command to test your filters. You can also use the --dry-run/-n option to perform a test run without making any changes.

Filter by path, file name, or object name

You can use the --include and --exclude options to include and exclude files or objects by name or path, as in the --include="*.txt" and --exclude="*" rules.

You can specify inclusion and exclusion rules directly in command lines or by writing them to a rule file. You can combine the --include and --exclude options based on your requirements and specify them multiple times in a single command. All occurrences of the options define a list of inclusion and exclusion rules, which match files or objects based on the following principles:

The order of the options matters. The rules are applied from the left to the right in command lines and top to bottom in rule files.
A detected file, object, or path is checked immediately for inclusion or exclusion.
Rule match checking occurs in order. If a file, object, or path is a match for an inclusion or exclusion rule, it is no longer checked against the subsequent rules.
If a file, object or path does not match any specified rules, the file, object, or path is included.
If a directory in a local file system is excluded, all subdirectories and files in the directory are excluded.
Objects are flatly stored in a bucket and are checked one by one for inclusion or exclusion.

ossutil uses glob matching, in which the following matching patterns are common:

A matching pattern with a leading forward slash (/) searches for matches directly within the specified path. A matching pattern without a leading forward slash (/) searches for matches at any depth within the path.
A matching pattern with a trailing forward slash (/) matches only directories. A matching pattern without a trailing forward slash (/) matches directories and files.
A forward slash (/) in the middle of a matching pattern is a path separator. Example: subdir/test.txt.
The * wildcard character matches any sequence of characters of arbitrary length, excluding /.
The ** element in the end of a matching pattern matches any sequence of characters of arbitrary length, including /. For example, dir/** matches the path dir/subdir/xx/test.txt.
The ? character matches any single character, excluding /.
The square brackets ([]) matches any single character within the defined character class. For example, [a-z] matches a lowercase letter from a to z, and [0-9] matches a digit from 0 to 9.
By default, a pattern must completely align with the end of the path or file name. For example, the foo pattern matches foo and dir/foo, but does not match dir/foo1 or foo/xx.
Matching is based on the integrity of the specified file name, object name, or directory name and does not support name splitting. For example, the foo pattern does not match xxx/afoo, and abc/foo does not match subdir/aabc/foo.

Note

ossutil uses the forward slash (/) instead of the backslash (\) as path separators on Windows.

The following table describes filter rule syntax and options that you can use to filter files or objects by path or name.

Item	Description
Filter rule syntax	rule pattern: glob matching pattern rule filename: the path to the file that contains glob matching patterns rule modifiers pattern: glob matching pattern with a directive. The `+` directive indicates inclusion. The `-` directive indicates exclusion.
Filter options	--include --include-from --exclude --exclude-from --filter --filter-from

The following table describes the filter options.

Option	Description
--include --exclude	Specify a matching pattern after the option name. Each option is followed by only one matching pattern (also called a rule). You can specify multiple matching patterns in a command by repeating the options, as in the --include ".txt" --include ".js" --exclude "*.jpg" settings.
--filter	Specify a matching pattern that contains a directive. Each option is followed by only one matching pattern. You can specify multiple matching patterns in a command by repeating the option, as the in --filter "+ .txt" --filter "+ .js" --filter "- *.txt" settings.
--include-from --exclude-from	Specify the path to a rule file after the option name. Each glob matching pattern is placed in a separate line in the rule file. You can specify multiple rule files in a command by repeating the options. The rule file that follows the --include-from option contains only inclusion rules. The rule file that follows the --exclude-from option contains only exclusion rules. For example, you can specify the following filters in a command: --include-from include-file.txt --exclude-from exclude-file.txt, where the include-file.txt file contains the following patterns: `.txt .js` and the exclude-file.txt file contains the `*.jpg` pattern.
--filter-from	Specify the path to a rule file after the option name. Each directive-based glob matching pattern is placed in a separate line in the rule file. You can specify multiple rule files in a command by repeating the option. For example, you can specify --filter-from filter-file.txt that contains the following patterns. `# a filter rule file + .txt + .js - **`

Notes:

Matching patterns in a rule file are loaded and handled from top to bottom. Empty lines and comment lines that start with the number sign (#) or semicolon (;) are ignored.
If the file path is set to -, data is loaded from the standard input. You can specify only one "-" in a command.
If the --include or --include-from option is specified, ossutil automatically adds the - ** pattern to the bottom of the rule list defined by the option.
If the --filter option is used, ossutil does not automatically add the - ** pattern to the bottom of the rule list defined by the option.

For example, if you want to use the --filter option to include only files whose extension is .jpg, you must specify the --filter "+ *.jpg" --filter "- **" filters.

Note

You cannot use --filter or --filter-from together with --include, --include-from, --exclude, or --exclude-from.

Filter by path list, file list, or object list

You can use the options described in the following table to specify a file that contains a list of paths, object names, or file names to filter.

Option

Description

--files-from

Reads paths, file names, or object names from the specified list file. You can specify multiple list files in a command by repeating the option.

When ossutil reads data from the list file, it ignores leading and trailing spaces in each line, and lines that start with the number sign (#) or semicolon (;).

--files-from-raw

Reads paths, file names, or object names from the specified list file. You can specify multiple list files in a command by repeating the option.

When ossutil reads data from the list file, it does not ignore leading or trailing spaces in each line, or lines that start with the number sign (#) or semicolon (;).

Note

If you filter objects or files by using a list of paths, file names, or object names, other types of filters specified in the command are ignored.

Filter by last modified time

You can filter files or objects by last modified time.

You can specify relative and absolute time filters. The following table describes the options that filter files or objects by last modified time.

Option	Description
--min-age	Excludes objects whose last modified time is later than the value of this option. The default unit is s (seconds). You can suffix the numerical value with a unit of time. For example, 1h indicates one hour.
--max-age	Excludes objects whose last modified time is earlier than the value of this option. The default unit is s (seconds). You can suffix the numerical value with a unit of time. For example, 1h indicates one hour.
--min-mtime	Excludes objects whose last modified time is earlier than the value of this option. The time must be in UTC. Example: 2006-01-02T15:04:05.
--max-mtime	Excludes objects whose last modified time is later than the value of this option. The time must be in UTC. Example: 2006-01-02T15:04:05.

Notes:

The --min-age and --max-age options allow you to suffix the numerical value with a unit of time: ms for milliseconds, s for seconds, m for minutes, h for hours, d for days, w for weeks, M for months, and y for years.
The numerical value before a unit can be a decimal, as in 1.5d (one and a half days).

The following sample command lists files or objects that were last modified more than three days ago.

ossutil ls oss://bucket/ --min-age 3d

Filter by data size

You can filter files or objects by their data size to include files or objects whose sizes are within the specified range.

The following table describes the options that you can use to filter files and objects by data size.

Option	Description
--min-size	The minimum size of a file or object that can be included. By default, the size is in bytes (B). You can specify a 1024-based size unit after the numerical value, as in 1K (1,024 bytes).
--max-size	The maximum size of a file or object that can be included. By default, the size is in bytes (B). You can suffix the numerical value with a 1024-based unit, as in 1K (1,024 bytes).

Notes:

You can specify one of the following short-form size unit: B for byte, K for Kibibyte, M for Mebibyte, G for Gibibyte, T for Tebibyte, or P for Pebibyte.
The numerical value before a unit can be a decimal, as in 0.5K.

The following sample command lists files or objects that do not exceed 1 MiB in size:

ossutil ls oss://bucket/ --max-size 1M

Filter by metadata

Metadata-based filtering applies only to objects in a bucket. You can filter objects by metadata in a similar way you filter files or objects by path, file name, or object name.

Metadata-based filtering uses glob matching and requires key=value representation of metadata filter rules.

You can use the following options to filter objects by metadata:

--metadata-include
--metadata-exclude
--metadata-filter
--metadata-filter-from

You can filter objects by using the following metadata items:

Storage class. To filter objects by storage class, specify x-oss-storage-class=value, where the value can be Standard, IA, Archive, ColdArchive, or DeepColdArchive.
Object type. To filter objects by object type, specify x-oss-object-type=value, where the value can be Normal, Multipart, Appendable, or Symlink.
Restoration status. To filter objects by restoration status, specify x-oss-restore=value, where the value is the actual restoration status that you want to filter by.
Content type. To filter objects by content type, specify content-type=value, where the value is the actual content type that you want to filter by.
User metadata. To filter objects by user metadata, specify x-oss-meta-aaa=value, where aaa is the custom attribute name in lowercase and value is the metadata value.

The following sample command lists objects in the Archive, Cold Archive, and Deep Cold Archive storage classes in the specified bucket:

ossutil ls oss://bucket/ --metadata-include "x-oss-storage-class=*Archive"