By Zhang Youdong, nicknamed Linqing at Alibaba.
Alibaba Cloud ApsaraDB for Redis hybrid storage instances represents a proprietary database system solution that is fully compatible with the Redis protocol and highly compatible with Redis commands and related features. These hybrid storage instances support storing full data on a disk rather than in the memory and caching hot data in memory, which strikes a balance between access performance and storage costs.
The below diagram provides a general picture of the underlying architecture and features of this service.
Hybrid storage instances are compatible with the majority of Redis commands, however the use of the following Redis commands are either not supported or restricted. If you require use of these commands, submit a service ticket.
When selecting a hybrid storage instance, you must select the appropriate memory and disk configurations. The memory configuration determines the total amount of hot data that can be stored, and the disk configuration determines the total amount of data that can be stored. When an instance is created, the most appropriate CPU resource configuration is selected based on the storage specifications. Currently, the number of CPU cores cannot be customized for this service.
For example, an instance with 64 GB memory and 256 GB disk space can store up to 256 GB of data. Note that these numbers are subject to the total size of physical files stored in the KV storage engine. In addition, 64 GB of data can be cached in the memory.
Recommendations on memory selection: To ensure full compatibility with the Redis native access protocol, ApsaraDB for Redis hybrid storage poses the following requirements:
Recommendations on disk selection: When Redis data is stored in the KV storage engine, each key comes with additional metadata, which increases the storage usage. We recommend that you leave a proper margin when selecting the disk storage size. We estimate the necessary margin to be 20% to 50% of your actual storage demand.
Case 1: User A stores 100 GB of data in an ApsaraDB for Redis cluster. Total queries per second is less than 20,000, and 80% of the data is seldom accessed. User A can use a hybrid storage instance with 32 GB memory and 128 GB disk space to save nearly 70 GB in memory storage and reduce storage costs by more than 50%.
Case 2: To avoid the high storage cost of Redis, User B creates a Pika or SSDB instance in an on-premises data center. The instance stores about 400 GB of data, of which about 10% is actively accessed. However, User B wants to migrate to ApsaraDB to simplify cluster maintenance. In this case, User B can use a hybrid storage instance with 64 GB memory and 512 GB disk space to remove the need for maintenance while keeping the same quality of service (QoS).
The performance of an ApsaraDB for Redis hybrid storage instance is closely related to the memory-disk ratio and service access. The performance of simple Set and Get operations may fluctuate at a level between thousands and tens of thousands, depending on the specifications and service access mode. In the best case scenario, each requested data record can be read from the memory. As such, the performance of an ApsaraDB for Redis hybrid storage instance is basically equivalent to that of an ApsaraDB for Redis in-memory instance when all data is stored in the memory. In the worst case scenario, each requested data record must be read from the disk.
Test scenario: 20 million keys, with a value size of 1 KB. 25% of hot keys can be stored in the memory. The test data for GET requests is as follows.
Live streaming generates a large amount of hot data, most of which is from popular live streaming rooms. You can use ApsaraDB for Redis hybrid storage instances to store data from popular live streaming rooms in the memory and store data from unpopular rooms on disks. As such, you can make full use of the limited memory space.
E-commerce applications have a large amount of commodity data. New products are frequently accessed, whereas older products are accessed less frequently. Using ApsaraDB for Redis hybrid storage instances, you can easily bypass the memory capacity limit by storing a large amount of commodity data on disks. During normal service requests, active commodity data is gradually cached in the memory to meet business needs at the lowest costs.
Online education generates a large amount of data, such as courses, question banks, and data exchanged between teachers and students. Only the latest courses and question banks are frequently accessed. You can use ApsaraDB for Redis hybrid storage instances to store massive course data on disks and transfer frequently accessed courses and question banks to the memory for constant caching. This ensures the access performance of frequently accessed data and strikes a balance between performance and storage costs.
In scenarios where hot and cold data is accessed separately, you can use ApsaraDB for Redis hybrid storage instances to reduce storage costs while maintaining the average performance level needed for systems.
The memory size is too small, resulting in insufficient memory space to store all keys and their metadata. We recommend that you upgrade the instance type in the console to increase the memory size of the instance.
Key values are small in size. ApsaraDB for Redis hybrid storage instances do not trigger storage transfer to disks for small values, such as those values less than 20 bytes. Even after small values are transferred to a disk, their metadata is still stored in the memory, which makes it impossible to reduce memory usage. Fully acknowledging these issues, we are dedicated to further optimizing the hybrid storage kernel to solve this problem and adapt to more scenarios.
ApsaraDB for Redis is a stable and reliable database service with excellent performance and auto scaling capabilities. It is structured on Alibaba Cloud's proprietary Apsara Distributed File System and full SSD high-performance storage, with support for primary-secondary and cluster-based high-availability architectures.
Learn How the Column-Store Meta Scan Accelerates Query Performance in AnalyticDB for PostgreSQL
Alibaba Clouder - March 24, 2021
Alibaba Clouder - January 21, 2021
ApsaraDB - April 28, 2020
Alibaba Clouder - August 7, 2020
ApsaraDB - October 16, 2020
Alibaba Clouder - September 27, 2018
Tair (Redis® OSS-Compatible)
A key value database service that offers in-memory caching and high-speed access to applications hosted on the cloud
Learn More
ApsaraDB for HBase
ApsaraDB for HBase is a NoSQL database engine that is highly optimized and 100% compatible with the community edition of HBase.
Learn More
ApsaraDB for OceanBase
A financial-grade distributed relational database that features high stability, high scalability, and high performance.
Learn More
ApsaraDB for Cassandra
A database engine fully compatible with Apache Cassandra with enterprise-level SLA assurance.
Learn MoreMore Posts by ApsaraDB
