Object Storage Service:ossfs 2.0 overview

Performance improvements

ossfs 2.0 delivers significant performance improvements over ossfs 1.0 in sequential reads and writes, and in highly concurrent reads of small files. For more information, see Performance Testing.

• Sequential write performance: In single-threaded, large-file sequential write scenarios, ossfs 2.0 provides nearly 18 times the bandwidth of ossfs 1.0.

• Sequential read performance: In single-threaded or multi-threaded (4 threads) large-file sequential read scenarios, ossfs 2.0 provides more than 3 times the bandwidth of ossfs 1.0.

• Concurrent small-file read performance: In high-concurrency (128 threads) small-file read scenarios, ossfs 2.0 provides more than 20 times the bandwidth of ossfs 1.0.

Scenarios

ossfs 2.0 is suitable for scenarios that require high-performance storage access, such as AI training, inference, big data processing, autonomous driving, and other compute-intensive workloads. These workloads primarily involve sequential and random reads and sequential (append-only) writes, and do not require full POSIX semantics.

Runtime environment

ossfs 2.0 is built on Filesystem in Userspace (FUSE).

Limits

• Permission requirements: The AccessKey must have full permissions on the target bucket or prefix. Otherwise, the mount operation may fail or cause abnormal behavior.

• Storage class limitations: You cannot mount buckets that use the Archive Storage, Cold Archive, or Deep Cold Archive storage classes.

• File name limitations: File names are limited to 255 characters because of a Linux constraint. Files or folders in OSS with names that exceed this limit are not visible in the mount target.

• File read limitations: ossfs uploads new files to OSS only after the corresponding file handles are closed. Reading a file before its handle is closed may cause an error.

• File write limitations: Random writes and concurrent writes to the same file are not supported. The default part size is 8,388,608 bytes (8 MiB). Therefore, the maximum supported file size is 83,886,080,000 bytes (78.125 GiB). You can configure the part size using the upload_buffer_size mount option.

• File rename operations (non-atomic)

  • File: Copies the file and then deletes the original remote file.

  • Folders: This operation first copies all files and then deletes the source files in a bulk operation. By default, a folder rename operation is limited to 2 million descendant files. You can configure this limit using the rename_dir_limit mount option.

• Concurrent write consistency: When multiple clients mount the same bucket and write to the same file concurrently, data consistency cannot be guaranteed.

• POSIX API compatibility: Partially compatible. For more information, see Supported POSIX APIs.

Features

References

• Install ossfs 2.0

• Configure ossfs 2.0

• Mount a bucket to a local file system using ossfs 2.0

• ossfs 2.0 mount options

• ossfs 2.0 Performance Testing

• File and folder recognition mechanism

• Reduce OSS requests and improve mount target performance

• FAQ

• Release notes