To meet the demand for processing increasing volumes of data stored in Object Storage Service (OSS) buckets, Alibaba Cloud provides comprehensive data lake solutions that can handle throughput rates exceeding 10 terabits per second (Tbps). In daily operations, concurrent tasks such as data collection, preprocessing, AI model training, development and debugging on the same data lake platform often lead to uneven resource allocation and bottlenecks. To resolve these issues, OSS has introduced the resource pool Quality of Service (QoS) policy. This policy allows multiple buckets in different business scenarios to share the same resource pool, dynamically adjusting resource allocation based on the characteristics of tasks, which ensures that key services and compute-intensive tasks are prioritized during peak hours and improves the efficiency, resource utilization, and the stability of systems.
Get started with resource pool QoS
The following video provides a quick introduction to resource pool QoS and its features.
Features
Bucket bandwidth throttling
You can incorporate one or more OSS buckets into a resource pool and configure bucket- level threshold settings.
Requester throttling
You can configure bucket-level or resource-pool-level threshold settings for specific RAM users.
Configuration of multiple threshold settings
Total read throughput, total write throughput, internal network read throughput, internal network write throughput, external network read throughput, and external network write throughput.
Quick configuration and immediate effectiveness
You can adjust the threshold setting using ossutil or by calling API operations. Configurations take effect within 2 minutes.
Configuration of throttling events
You can create alert rules for OSS throttling events and specify that notifications are sent to the administrator if throttling is triggered or the threshold values are reached. For more information, see real-time monitoring of OSS throttling information through CloudMonitor.
Monitoring
You can use CloudMonitor to view throttling configurations and monitor performance metrics in real time. For more information, see using monitoring services.
Limitations
Up to one hundred resource pools can be configured for a single Alibaba Cloud account.
You can manage up to 100 buckets within a resource pool.
A single resource pool supports throttling configurations for up to 300 RAM users.
Apply for this feature
Resource pool QoS is in invitational preview. If the throughput of your OSS buckets in a region reaches or exceeds 500 Gbps, you can contact technical support to apply for this feature.