Community

Blog Events Webinars Tutorials Forum
  1. Blog
  2. Events
  3. Webinars
  4. Tutorials
  5. Forum
Create Account
×
Community Blog Creating a Shop Search System with Geo Indexes

Creating a Shop Search System with Geo Indexes

In this article, we will create a geo management system for 100 million shops based on Table Store.

By Wang Tantan (Si Ming)

Background

The core and bottleneck of a geo management system is the database storage performance and query capability. On one hand, storage services require low-latency data storage and reading that can deal with massive amounts of data; on the other hand, storage services also need to provide efficient geographic and multi-dimensional data retrieval. Table Store is a serverless and distributed NoSQL database that can completely meet the requirements of a geo management system.

In this article, we will create a geo management system for 100 million shops based on Table Store.

Scenario

A certain shop search platform provides information about 100 million shops. Users can search for desired shops based on their own dimensions on the PC application and website of this platform. The platform needs to display the specific locations of shops and shop details on a map, and support redirecting to shop homepages.

Dimension 1: [within 1km] [less than 100 RMB per customer ] [top rating] [milk tea];

Dimension 2: [within Hangzhou city] [top rating] [Shen Jia*] shop;

Implementing fast and multi-dimensional geo queries is the core feature of a geo management solution. The following is a sample:

Note: This sample provides data information about 100 million shops. Sample in the official console: Project sample

1

The preceding screenshot shows an overview of the Shop Search system page based on Table Store. The sample is integrated in the Table Store console. You can log on to the console to experiment with the system. (If you are a new Table Store user, you need to click Activate Now for a trial of this service. The service activation is free. Order data is stored in public instances. A trial doesn't consume storage, network traffic, or CUs.)

Table Store

The SearchIndex solution in Table Store allows you to easily set up a search system targeting 100 million shops. The SearchIndex feature allows creating indexes such as geo indexes and word-breaking string indexes. It provides you with the capabilities such as geo retrieval and multi-dimensional combined retrieval. Indexes can be created at any time. Inventory data and incremental data are automatically synchronized.

Table Store is a fully-hosted and distributed NoSQL data storage service from Alibaba Cloud that provides features such as storage of massive amounts of data, automatic sharding of hot data, and multi-dimensional retrieval of massive amounts of data. Table Store can effectively solve the challenge of large volumes of geo data.

You can create and activate indexes as needed. Table Store ensures the consistency of data synchronization, greatly reducing the work required for your solution design, service maintenance, and code development.

Preparation for Building

If you are satisfied with the shop search system for 100 million shops implemented based on Table Store and want to start to set up your own system, simply follow the following steps:

(1) Activate Table Store

Activate the Table Store service in the console. Table Store is out-of-the-box (post-paid) and billed on a pay-as-you-go basis. Table Store also provides a free quota that is sufficient for functional tests. For more information, visit Table Store Console and Free quota description.

(2) Create an Instance

Create a Table Store instance in the console and select a region that supports SearchIndex. (Currently the SearchIndex feature has not been commercialized and is supported in the following regions:Beijing, Shanghai, Hangzhou, and Shenzhen. This feature will be gradually available in other regions.)

2

After the instance is created, open a ticket to apply for the SearchIndex beta test invitation. (After becoming commercialized, SearchIndex will be enabled by default. No fees will be incurred if the feature is not used.)

  • Beta test invitation request: Open a ticket, select"Table Store" > "Product Features and Characteristics" > "Create a Ticket". The application content is as follows:
  • Question description: Please enter "Apply for SearchIndex beta test invitation"
  • Confidential information: Please enter Region + Instance name, for example, Shanghai + myInstanceName

3

(3) Download SDKs

Use SDKs with SearchIndex (see the official website for more details). Currently, new functions are added for Java, Go, and Node.js SDKs.

Java-SDK

<dependency>
    <groupId>com.aliyun.openservices</groupId>
    <artifactId>tablestore</artifactId>
    <version>4.8.0</version>
</dependency>

Go-SDK

$ go get github.com/aliyun/aliyun-tablestore-go-sdk

Nodejs-SDK

$ npm install tablestore@4.1.0

(4) Design a Table

The shop search system sample only uses one shop table that mainly includes fields such as shop type, shop name, shop location, average shop ratings, and per capita consumption. The table is designed as follows:

Table Name: geo_positon

4

Start Building (Core Code)

(1) Create a Data Table

You only need to create a "shop information table" under the instance for which you have successfully applied for the beta test invitation: Create and manage data tables in the console (or directly by using SDKs):

5

(2) Create a Data Table Index

Table Store automatically synchronizes full and incremental index data: Users can create and manage indexes through the console (or, they can also create indexes using the SDK)

6

7

(3) Data Import

Insert test data (100 million entries are inserted into the sample in the console. You can insert a small amount of test data by yourself).

8

9

(4) Data Reading

Data reading falls into two types:

1.  Primary key reading

The primary key column is obtained based on the native Table Store: getRow, getRange, batchGetRow. Primary key reading is used for index (automatic) reverse lookup. Users can also provide a single query page for the primary key (Order MD5). And the query speed is extremely fast at the scale of 100 million entries of data. Multi-dimensional retrieval is not supported for the single primary key query;

2.  Index reading (shop query)

Query based on the new SearchIndex function: the search interface. Users can freely design multi-dimensional combination queries for index fields. By setting and selecting different query parameters, different query criteria and different sorting methods are built. Currently, exact query, range query, prefix query, match query, wildcard query, phrase match query, and word breaking string query are supported, and they are combined by boolean AND and OR.

The following is the criteria for querying milk tea shops in the vicinity within 1km range at "36.76613, 111.41461":

List<Query> mustQueries = new ArrayList<Query>();

TermQuery termQuery = new TermQuery();
termQuery.setFieldName("type");
termQuery.setTerm(ColumnValue.fromString(milk tea));
mustQueries.add(termQuery);

GeoDistanceQuery geoDistanceQuery = new GeoDistanceQuery();
geoDistanceQuery.setFieldName("pos");
geoDistanceQuery.setCenterPoint("36.76613,111.41461");
geoDistanceQuery.setDistanceInMeter(1000);
mustQueries.add(geoDistanceQuery);

BoolQuery boolQuery = new BoolQuery();
boolQuery.setMustQueries(mustQueries);
NoSQL Database Storage Table Store
0 0 0
Share on

Read previous post:

Alibaba Cloud OSS complies with the compliance requirements of U.S. SEC and FINRA

Read next post:

Implementing Tracking Management and Geo-Fencing on Table Store

Alibaba Cloud Storage

57 posts | 12 followers

You may also like

Comments

Alibaba Cloud Storage

57 posts | 12 followers

Related Products

More Posts by Alibaba Cloud Storage

See All