AnalyticDB:Lake cache

Prerequisites

An AnalyticDB for MySQL Data Lakehouse Edition cluster is created.

Overview

How it works

The following section describes how the lake cache feature works:

1. When the lake cache client sends a read request to OSS, the read request is forwarded to a lake cache master node to request for the object metadata.

2. The lake cache master node returns the object metadata to the lake cache client.

3. The lake cache client uses the object metadata to send a request to lake cache worker nodes for OSS objects.

   • If the requested objects are stored on the lake cache worker nodes, the worker nodes return the objects to the lake cache client.

   • If the requested objects are not stored on the lake cache worker nodes, the lake cache feature obtains the objects from OSS. Then, the lake cache feature returns the objects to the lake cache client and stores the objects in the cache space.

Billing rules

After you enable the lake cache feature, you are charged for the cache storage based on the pay-as-you-go billing method. For more information, see Pricing for Data Lakehouse Edition.

Usage notes

• The lake cache feature is supported only in the following regions: China (Hangzhou), China (Shanghai), China (Beijing), China (Zhangjiakou), China (Shenzhen), Singapore, and US (Virginia).

  Important

  If you want to use the lake cache feature in other regions, submit a ticket.

• If a fault occurs on the cache hardware, data queries are not interrupted but slow down. After cached data is pre-prefetched from OSS, the query speed is restored.

• When the cache space used by the lake cache feature reaches the upper limit, the lake cache feature replaces the infrequently accessed objects in the cache space with frequently accessed objects. If you do not want to replace the objects stored in the cache space, you can increase the cache size for the lake cache feature.

Enable the lake cache feature

1. Log on to the AnalyticDB for MySQL console. In the upper-left corner of the console, select a region. In the left-side navigation pane, click Clusters. On the Data Lakehouse Edition tab, find the cluster that you want to manage and click the cluster ID.

2. In the Configuration Information section of the Cluster Information page, click Configure next to Lake Cache.

3. In the Lake Cache dialog box, turn on Lake Cache and specify a cache size.

4. Click OK.

Use the lake cache feature

After you enable the lake cache feature, you can specify the spark.adb.lakecache.enabled parameter for Spark jobs to accelerate OSS data reads. Sample code:

-- Here is just an example of using LakeCache. Modify the content and run your spark program.
SET spark.adb.lakecache.enabled=true;

-- Here are your sql statements
SHOW databases;

{
  "comments": [
    "-- Here is just an example of using LakeCache. Modify the content and run your spark program."
  ],
  "args": ["oss://testBucketName/data/readme.txt"],
  "name": "spark-oss-test",
  "file": "oss://testBucketName/data/example.py",
  "conf": {
    "spark.adb.lakecache.enabled": "true"
  }
}

View the monitoring information about the lake cache feature

After you enable the lake cache feature, you can check whether the submitted Spark applications use the lake cache feature in the CloudMonitor console. You can also view the monitoring information about the lake cache feature, such as the amount of data that is read from the cache space. Perform the following steps:

1. Log on to the CloudMonitor console.

2. In the left-side navigation pane, choose Cloud Service Monitoring > Cloud Service Monitoring.

3. Move the pointer over the AnalyticDB for MySQL card and click AnalyticDB for mysql 3.0 - Data Lakehouse Edition.

4. Find the AnalyticDB for MySQL cluster that you want to monitor and click Monitoring Charts in the Actions column.

5. Click the LakeCache Metrics tab to view the details about the lake cache metrics.

   The following table describes the lake cache metrics.

   Metric	Description
   LakeCache Cache Hit Ratio(%)	The cache hit ratio. Formula: Number of read requests that hit the cache/Total number of read requests.
   LakeCache Cache Usage(B)	The used cache space. Unit: bytes.
   Total Amount of Historical Cumulative Read Data of LakeCache(B)	The total amount of data that is read from the cache space. Unit: bytes.