All Products
Search
Document Center

Object Storage Service:OSS tools

Last Updated:Sep 23, 2024

Object Storage Service (OSS) provides various management tools that can be used to streamline OSS operations, such as uploads of large objects, signature generation, data migration, and bucket mounting. In addition, third-party tools and plug-ins available in the communities can help provide more enhancements and ease of use.

Official tools

Tool

Description

ossutil 2.0 (in preview)

The version 2.0 of command-line tool ossutil.

  • Supports multi-level commands for object and bucket management, such as high-level commands and API-level commands.

  • Supports external access credentials and OpenID Connect (OIDC) credentials. If you use access credentials for a RAM user, ossutil automatically obtains the role name.

  • Provides the --output-format option, which allows you to format command output in JSON, YAML, or XML.

ossutil 1.0

A command-line tool used to manage objects and buckets.

  • Provides a wide range of easy-to-use and powerful commands for you to manage objects and buckets.

  • Supports concurrent uploads and resumable transmission.

  • Allows you to upload and download directories.

Alibaba Cloud CLI

A command-line tool used to manage Alibaba Cloud resources. You can run commands in the CLI to manage your data in OSS. For more information, see Use Alibaba Cloud CLI to manage data in OSS.

ossbrowser

A graphical management tool.

  • Provides features similar to those of Windows Explorer.

  • Allows you to browse objects.

  • Allows you to upload and download directories.

  • Supports concurrent uploads and resumable transmission.

  • Provides a visualized experience for you to configure authorization policies and grant permissions to RAM users.

  • Supports Windows, Linux, and macOS.

Limits:

  • A graphical management tool whose transmission speeds and performance are not as good as those of ossutil.

  • Only objects smaller than 5 GB in size can be moved or replicated.

  • You can upload a single file up to 48.8 TB in size.

ossimport

A tool used to migrate data to OSS.

  • Allows you to migrate data from a third-party data source to OSS.

  • Supports distributed deployment. You can use multiple servers to migrate data at the same time.

  • Supports migration of terabytes of data.

  • Supports Windows and Linux.

  • Provides compatibility with Java 7.

ossfs

A tool used to mount a bucket to a local file system. After you mount an OSS bucket to the local file system of Linux, you can access and manage objects in the bucket by using the local file system, which facilitates data access and sharing.

  • Supports most features of POSIX-compliant file systems, including file reads and writes, directories, link-related operations, permissions, UIDs, GIDs, and extended attributes.

  • Allows you to use multipart uploads to upload large files.

  • Supports MD5 verification to ensure data integrity.

Limits:

The following limits apply to ossfs when you use ossfs to mount a bucket to a local directory on the Linux operating system:

  • ossfs is not suitable for scenarios that require highly concurrent read and write operations.

    Note
    • Both read and write operations consume the disk capacity. In highly concurrent read/write scenarios, disk performance limits read and write operations.

    • Concurrent read and write requests compete for resources, which affects the bandwidth.

  • ossfs does not support hard links.

  • Archive, Cold Archive, and Deep Cold Archive buckets cannot be mounted to local file systems by using ossfs.

  • If you use ossfs to edit an uploaded object, the object is re-uploaded.

  • The performance of metadata-related operations, such as list directory, is compromised because you must remotely access the OSS server.

  • Errors may occur if you rename an object or a directory. Operation failures may cause data inconsistencies.

  • If an OSS bucket is mounted to multiple clients, you are responsible for maintaining data consistency. To prevent multiple clients from writing the same object at the same time, we recommend that you schedule object access appropriately.

ossftp

An FTP-based tool used to manage objects in OSS.

  • You can use FTP clients, such as FileZilla, WinSCP, and FlashFXP, to manage objects in OSS.

  • ossftp is an FTP server that receives FTP requests and performs operations on objects and directories in OSS.

  • ossftp is based on Python 2.7 and later.

  • ossftp supports Windows, Linux, and macOS.

RAM Policy Editor

A tool used to automatically generate OSS-related authorization policies. We recommend that you use this tool to generate custom authorization policies.

  • Generates an authorization policy based on the specified requirements. You can use this tool to create an authorization policy. You can add the policy to a custom RAM policy.

  • Supports the following browsers: Google Chrome, Firefox, and Safari.

osscmd (retired)

A command-line tool used to manage objects and buckets.

  • Provides a wide range of commands to manage objects and buckets.

  • Supports Windows and Linux.

Limits:

  • osscmd is compatible only with Python 2.5, 2.6, and 2.7. osscmd is incompatible with Python 3.x.

  • osscmd does not support some OSS features, such as the Infrequent Access (IA), Archive, Cold Archive, and Deep Cold Archive storage classes, cross-region replication (CRR), and mirroring-based back-to-origin.

Important

The osscmd operation commands have been integrated into ossutil. osscmd has been unavailable since July 31, 2019. Alibaba Cloud regrets any inconvenience caused, and appreciates your patience and understanding.

OSS Connector for AI/ML

A Python library that is used to efficiently access and store OSS data in PyTorch training jobs.

  • Supports a map dataset that is suitable for random reading and facilitates the quick retrieval of specific data during training.

  • Supports an iterable dataset that is suitable for streaming sequential reading and can process continuous data streams.

  • Creates a checkpoint by using OssCheckpoint to directly load checkpoints to OSS during training.

Limits:

  • Operating system: 64-bit x86 Linux

  • glibc: 2.17 or later

  • Python: 3.8 to 3.12

  • PyTorch: 2.0 or later

  • To use the OSS checkpoint feature, the Linux kernel must support userfaultfd.

Third-party tools and plug-ins

Tool

Description

Big Data Tools

Big Data Tools is a JetBrains IDE plug-in that supports OSS and can be used to optimize big data workflows. You can use the plug-in to:

  • Connect to remote file systems and storage systems such as OSS.

  • Manage files in a way similar to using a file manager, for example, copy, move, rename, delete, and download objects.

  • Preview files and switch between the Text view and Table view of CSV files.

  • Obtain information about files, such as modification time.

  • Preview binary files, such as Parquet.

  • Open storage buckets and directories in separate tabs.

For more information, see Use Big Data Tools to connect JetBrains IDEs to OSS.

Hacklog Remote Attachment

Hacklog Remote Attachment is a WordPress plug-in that allows you to store attachments and media on WordPress websites in OSS instead of local servers for faster website loading and lower storage consumption on hosts. For more information, see How to store remote attachments from WordPress websites to OSS.

FIleZilla

FIleZilla is an easy-to-use FTP client tool that allows you to upload files from local sites to OSS for optimized storage and access and lower storage consumption on local sites. For more information, see Use FileZilla to upload files from local sites to OSS.