Data Lake Analytics - Deprecated:Background and preparations

Background

Data Lake Analytics (DLA) serves as a hub for in-cloud data processing. It allows you to query and analyze data in ApsaraDB for RDS, Table Store, and OSS through standard JDBC.

In DLA, you can also create a database (mapping database) connecting to an ApsaraDB for Redis (Redis) database in DLA. Then, you can use a MySQL client or MySQL CLI tool to connect to DLA and execute standard SQL statements to read data from the Redis database.

This topic describes how to read data from the Redis database by using DLA. It uses data in the CSV and JSON formats as an example.

Currently, DLA can only read these two types of data from Redis databases.

Prerequisites

Before using DLA to read data from the Redis database, you must perform the following operations to insert test data into the Redis database.

1. Create a Redis instance

   DLA and the Redis instance must be in the same region. Otherwise, the operations described in this topic are not allowed.

   DLA connects to the Redis database over the VPC of the Redis instance. Therefore, we recommend that you select VPC as the network type when creating the Redis instance. You can click Switch to VPC to switch the network type of the Redis instance from classic network to VPC.

2. Configure the whitelist

3. Connect to the instance

4. Insert test data

In this example, insert the following test data into the Redis database:

# CSV data
set hello_world_1 1,james,10
set hello_world_2 2,bond,20
set hello_world_3 3,lily,30
set hello_world_4 4,lucy,20
# JSON data
set hello_foo_1 '{"id":1,"name":"james","age":110}'
set hello_foo_2 '{"id": 2, "name": "bond", "age": 210}'
set hello_foo_3 '{"id": 3, "name": "lily", "age": 310}'
set hello_foo_4 '{"id": 3, "name": "lucy", "age": 210}'

Notes

Before you create a connection to the Redis database in DLA, you must add the CIDR block 100.104.0.0/16` to the whitelist of the Redis instance.

Your Redis instance is in a VPC, and by default DLA cannot access resources in the VPC. To enable DLA to access your Redis instance, you must use the reverse VPC technology, that is, add the CIDR block 100.104.0.0/16 to the whitelist of your Redis instance.

Permission statement: When you add the CIDR block 100.104.0.0/16 to the Redis instance, you agree that we can use the reverse VPC technology to read data from the Redis database.

Procedure

1. Create a Redis schema

2. Create a Redis table and read data from the table