Alibaba Cloud EMR Serverless StarRocks is a fully managed, storage-compute-separated lakehouse OLAP engine. It natively integrates a three-way fused query capability combining full-text search, scalar filtering, and vector nearest-neighbor search. It supports direct multimodal hybrid retrieval on open lakehouse formats such as Paimon, Iceberg, and Lance, enabling a single SQL statement to complete joint recall for “keyword matching + conditional constraints + semantic similarity.”
Technical implementation: Full-text channel (inverted index + BM25 ranking) and vector channel (HNSW / IVFPQ / DiskANN nearest-neighbor indexes) perform parallel recall. Results are then re-ranked using RRF (Reciprocal Rank Fusion), weighted fusion, or learning-to-rank, while scalar predicates are deeply optimized and executed synchronously via In-Filter. Retrieval results are written back directly to the source table via column-level Partial Update, with data never leaving the lake throughout the process.
Core advantages: The three retrieval channels are fused and scored inside the engine rather than recalled separately and stitched together, improving recall by more than 30%. Self-developed Native Reader/Writer delivers industry-leading lake table read/write performance. One engine plus one storage layer reduces cost by 60% compared with traditional multi-system solutions. Serverless storage-compute separation enables on-demand billing and zero idle cost.
Typical scenarios: autonomous driving corner case mining and training dataset construction, multimodal e-commerce search, RAG knowledge base enhancement, content safety review, and medical image retrieval.
When AI teams filter training datasets from massive unstructured data, a single query often includes three types of requirements:
| Retrieval Type | Problem Solved | Example |
|---|---|---|
| Full-text search | Search descriptions / annotation text by semantic keywords | Images whose description contains “urban road” |
| Scalar filtering | Precisely filter by business tags | Weather = heavy rain AND includes pedestrians AND nighttime |
| Vector similarity | Rank by visual / semantic similarity | Find the Top-N images most similar to the seed image |
Using autonomous driving as an example, a typical requirement from an algorithm engineer is:
“From historical road-capture data, find the Top-100 images that mention ‘construction zone’ in the description, have heavy rain weather, are on urban roads, and are visually most similar to the current misjudgment scenario, then label them and send them directly for training.”
The traditional approach requires three systems to work together: Elasticsearch for full-text search, a vector database for similarity search, and an analytics engine for scalar filtering. Data is moved back and forth among the three systems, the pipeline is long, consistency is hard to guarantee, and the final stitched result may still fail to return enough K results.
Alibaba Cloud EMR Serverless StarRocks solves this with one engine, one SQL statement, and one unified execution path.
Figure 1: Multimodal hybrid retrieval architecture from data source to application
| Capability | Technical Details | Applicable Scenarios |
|---|---|---|
| Full-text search | BM25 ranking + inverted index + intelligent tokenization (Chinese/English) | Log storage, intelligent analytics, text matching, semi-structured data filtering |
| Vector search | ANN approximate search, supporting mainstream algorithms such as HNSW / IVFPQ / DiskANN | RAG retrieval augmentation, semantic similarity search, multimodal image & text retrieval |
| Hybrid search | Joint recall across scalar, vector, and full-text channels, with custom weights and re-ranking | Autonomous driving training data preparation, RAG Q&A, precise e-commerce product search |
Traditional solutions execute the three retrieval modes separately and concatenate results at the application layer. StarRocks, however, performs fusion inside the engine:
Figure 2: Two-way parallel recall → in-engine RRF fusion re-ranking → final Top-K
| Technology | What It Does | Effect |
|---|---|---|
| RRF fusion algorithm | Unifies full-text BM25 scores and vector similarity in the engine for ranking | Improves recall by more than 30%, results are more accurate |
| In-Filter deep optimization | Executes scalar filtering and vector search synchronously within the same operator | Avoids insufficient results caused by “recall first, filter later” |
| Lake Optimizer | Query planning customized for lake table file layout and partition characteristics | Narrows the range with scalar predicates before vector computation |
| Native Reader/Writer | Directly parses and writes lake table files | Industry-leading lake table read/write performance |
| Custom weights + Re-Rank | Adjusts recall weights across full-text, vector, and scalar channels | Flexible tuning of ranking strategy by business scenario |
-- Create a Paimon lake table
CREATE TABLE ai_dataset.scene_data (
id BIGINT,
path STRING, -- OSS image path
description STRING, -- scene description text
weather STRING, -- weather tag
road_type STRING, -- road type
time_of_day STRING, -- time period
has_pedestrian BOOLEAN, -- whether pedestrians are included
speed DOUBLE, -- speed
scene_tag STRING, -- scene tag
embedding ARRAY<FLOAT> -- image vector (1024 dimensions)
) USING paimon
TBLPROPERTIES (
'row-tracking.enabled' = 'true',
'data-evolution.enabled' = 'true'
);
-- Vector index: supports HNSW / IVFPQ / DiskANN
CREATE INDEX idx_vec ON ai_dataset.scene_data (embedding)
USING VECTOR (
"index_type" = "HNSW",
"dim" = "1024",
"metric_type" = "cosine"
);
-- Full-text index: inverted index + BM25 ranking
CREATE INDEX idx_desc ON ai_dataset.scene_data (description)
USING GIN;
-- Scalar indexes: accelerate frequently filtered fields
CREATE INDEX idx_weather ON ai_dataset.scene_data (weather) USING BITMAP;
CREATE INDEX idx_road ON ai_dataset.scene_data (road_type) USING BITMAP;A single table can simultaneously support full-text, vector, and scalar indexes, and the query engine coordinates them automatically.
-- Three-way fusion: full-text keywords + scalar conditions + vector similarity, done in one SQL statement
SELECT
id, path, description, weather, road_type,
approx_cosine_similarity(embedding, @query_vector) AS similarity
FROM ai_dataset.scene_data
WHERE description MATCH 'construction zone' -- full-text search (BM25)
AND weather = 'heavy rain' -- scalar filtering
AND road_type = 'urban road' -- scalar filtering
AND time_of_day = 'night' -- scalar filtering
ORDER BY similarity DESC
LIMIT 100;Execution logic: Lake Optimizer automatically pushes scalar predicates forward. Bitmap indexes first narrow the candidate set from the 100-million scale to the 10-thousand scale; the GIN index performs BM25 full-text matching on the candidate set; the HNSW index performs vector nearest-neighbor search within the filtered range; and the RRF algorithm unifies full-text scores and vector similarity into a single ranking to return the final Top-K.
After retrieval, use Partial Update to tag only the matched records without touching other columns:
-- Column-level write-back: update only the scene_tag column, do not affect original data
UPDATE ai_dataset.scene_data
SET scene_tag = 'corner_case_rain_urban_night'
WHERE id IN (
SELECT id
FROM ai_dataset.scene_data
WHERE description MATCH 'construction zone'
AND weather = 'heavy rain'
AND road_type = 'urban road'
ORDER BY approx_cosine_similarity(embedding, @query_vector) DESC
LIMIT 100
);Write amplification is extremely small, and iterative labeling is supported. Downstream Spark training jobs can read directly from the same Paimon table, and data never leaves the lake.
Perception models often underperform in adverse weather, so large amounts of long-tail scenario data are needed for retraining. But such scenarios are extremely rare in road-capture data, typically less than 0.1%, making manual frame-by-frame screening very inefficient. The following demonstrates how Alibaba Cloud EMR Serverless StarRocks completes the full workflow of “select seed image → hybrid recall → tag write-back → deliver training dataset.”
Upstream Spark processes road-capture images in batch via AI Function, generates labels and vectors, and writes them into the Paimon lake table:
-- Spark batch ingestion into the lake (completed upstream)
INSERT INTO ai_dataset.scene_data
SELECT
monotonically_increasing_id() AS id,
path,
ai_query('Describe the driving scene in the image, including weather, road, vehicles, pedestrians, and other elements',
service_name => 'qwen-vl-plus', data => content) AS description,
ai_query('Determine the weather: sunny/overcast/light rain/heavy rain/fog/snow',
service_name => 'qwen-vl-plus', data => content) AS weather,
ai_query('Determine the road type: highway/urban road/rural road',
service_name => 'qwen-vl-plus', data => content) AS road_type,
ai_query('Determine the time period: daytime/nighttime/dusk',
service_name => 'qwen-vl-plus', data => content) AS time_of_day,
ai_query('Does the image contain pedestrians: true/false',
service_name => 'qwen-vl-plus', data => content) AS has_pedestrian,
NULL AS speed,
NULL AS scene_tag,
ai_embedding_multimodal(content,
service_name => 'tongyi-embedding-vision-plus') AS embedding
FROM read_files('oss://ad-raw/camera_front/2025-*/', suffix => 'jpg,png');After Spark writes the data, StarRocks reads the same Paimon data directly through Native Reader, with no synchronization required.
Choose a typical seed from a known misjudgment case as the query anchor for vector retrieval:
-- Find the embedding of a typical misjudgment frame as the seed
SELECT embedding
FROM ai_dataset.scene_data
WHERE path = 'oss://ad-raw/camera_front/2025-10-15/frame_003812.jpg';Using the seed image as the anchor, recall through full-text keywords, scalar tags, and vector similarity at the same time:
-- Recall the Top-200 most similar samples that mention construction in the description + heavy rain + urban road + pedestrians
SELECT id, path, description, weather, road_type, has_pedestrian,
approx_cosine_similarity(embedding, @seed_embedding) AS similarity
FROM ai_dataset.scene_data
WHERE description MATCH 'construction' -- full-text: description mentions construction
AND weather = 'heavy rain' -- scalar: heavy rain weather
AND road_type = 'urban road' -- scalar: urban road
AND has_pedestrian = true -- scalar: pedestrians present
ORDER BY similarity DESC
LIMIT 200;Why In-Filter is critical here: traditional solutions first take the vector Top-200 and then apply scalar filtering. If heavy rain + urban road + pedestrians account for only 2%, only 4 records may remain after filtering. In-Filter synchronously executes scalar filtering during vector search, ensuring that the result set can still reach 200 records.
One job can batch recall multiple corner cases and write labels directly back to the source table:
-- Scenario 1: heavy rain + construction zone + nighttime
UPDATE ai_dataset.scene_data
SET scene_tag = 'rain_construction_night'
WHERE id IN (
SELECT id FROM ai_dataset.scene_data
WHERE description MATCH 'construction'
AND weather = 'heavy rain' AND time_of_day = 'night'
ORDER BY approx_cosine_similarity(embedding, @seed_1) DESC
LIMIT 200
);
-- Scenario 2: fog + highway + pedestrians
UPDATE ai_dataset.scene_data
SET scene_tag = 'fog_highway_pedestrian'
WHERE id IN (
SELECT id FROM ai_dataset.scene_data
WHERE description MATCH 'highway'
AND weather = 'fog' AND has_pedestrian = true
ORDER BY approx_cosine_similarity(embedding, @seed_2) DESC
LIMIT 200
);
-- Scenario 3: snow + rural road
UPDATE ai_dataset.scene_data
SET scene_tag = 'snow_rural'
WHERE id IN (
SELECT id FROM ai_dataset.scene_data
WHERE weather = 'snow' AND road_type = 'rural road'
ORDER BY approx_cosine_similarity(embedding, @seed_3) DESC
LIMIT 300
);Partial Update writes only the scene_tag column, so write amplification is minimal. Iterative labeling does not affect the original data.
Use StarRocks OLAP analytics to evaluate the quality of the training set on the same table:
-- Number of samples and similarity distribution by scenario
SELECT
scene_tag,
COUNT(*) AS sample_count,
AVG(approx_cosine_similarity(embedding, @seed_1)) AS avg_similarity,
MIN(approx_cosine_similarity(embedding, @seed_1)) AS min_similarity
FROM ai_dataset.scene_data
WHERE scene_tag IS NOT NULL
GROUP BY scene_tag
ORDER BY sample_count DESC;
-- Check for label conflicts (the same image matched by multiple scenarios)
SELECT id, path, COUNT(DISTINCT scene_tag) AS tag_count
FROM ai_dataset.scene_data
WHERE scene_tag IS NOT NULL
GROUP BY id, path
HAVING tag_count > 1;Downstream Spark training jobs directly read from the same Paimon table and filter training data by scene_tag:
-- Spark downstream consumption (no data movement needed)
SELECT * FROM ai_dataset.scene_data
WHERE scene_tag IN ('rain_construction_night', 'fog_highway_pedestrian', 'snow_rural');A closed-loop pipeline: Spark writes → StarRocks retrieves and labels → Spark consumes for training. Data always stays in the same Paimon table, with zero movement and zero redundant copies.
In the era of large models, what enterprises truly lack is no longer compute, but a foundational data platform that can continuously supply high-quality training and retrieval data. Whether it is road-capture images for autonomous driving, product images and text for e-commerce, audio and video for content platforms, or document images for financial risk control, nearly every AI team faces the same set of problems: raw data is scattered across object storage in multimodal forms such as images, text, audio, and video, often at a scale of billions; meanwhile, model training, RAG knowledge bases, and agent applications repeatedly need to filter and refine that data into high-quality subsets. This means constantly performing “semantic retrieval + tag filtering + similarity recall” on the same dataset, and persisting the results for reuse.
In previous architecture practices, to satisfy structured queries, full-text search, and vector similarity matching at the same time, data often had to move and synchronize among multiple systems such as object storage, search engines, dedicated retrieval engines, and data warehouses. This multi-system collaboration model tends to cause data redundancy, rising storage costs, complex pipelines, and inconsistent definitions, forcing AI engineers to spend most of their time on “moving data” and “reconciling data” rather than “using data.” This is exactly the core problem Alibaba Group AI Data aims to solve. It has two goals: prepare data for large-model training, and provide high-quality data for upper-layer AI applications. Based on this, it proposed four hard requirements:
In technology selection, the Alibaba Cloud EMR Serverless StarRocks + DLF Paimon architecture has already been deployed and validated at scale in multiple core Alibaba Group businesses, proving its stability and cost-effectiveness. The AI Data team’s judgment is: rather than introducing a new system specifically for AI multimodal scenarios, it is better to reuse the same mature technology stack and extend it naturally from structured BI analytics to multimodal hybrid retrieval. In this way, existing operational systems, data assets, and engineering experience can be preserved, while BI analytics, AI applications, and Data Agent workloads can all share one architecture and one copy of data. This is also a path that any enterprise aiming to build an AI data foundation at low cost can follow.
Figure 3: AI Data multimodal data lake and hybrid retrieval architecture
| Metric | Result |
|---|---|
| Cost | One engine + one storage format, reducing cost by 60% compared with the original multi-system solution |
| Recall rate | Scalar + vector dual-channel recall, combined with RRF fusion re-ranking, improves overall recall by more than 30% |
| Usability | AI Function is used to vectorize raw files with embeddings, and a unified SQL interface greatly improves usability |
A user uploads a fashion image and simultaneously enters the text description “commute-style dress”:
SELECT product_id, title, price, image_url,
approx_cosine_similarity(embedding, @image_vector) AS visual_sim
FROM mall.products
WHERE description MATCH 'commute dress' -- full-text: match product description keywords
AND category = 'women\'s clothing' -- scalar: category
AND price BETWEEN 200 AND 800 -- scalar: price range
AND stock_qty > 0 -- scalar: in stock
ORDER BY visual_sim DESC
LIMIT 20;The returned products can both “understand the text,” “look like the image,” and “actually be purchasable.”
Retrieve the most relevant document chunks from an enterprise knowledge base under the combined constraints of semantics, keywords, and permissions:
SELECT chunk_id, title, content,
approx_cosine_similarity(embedding, @question_vector) AS relevance
FROM kb.doc_chunks
WHERE content MATCH 'refund process invoice' -- full-text: keyword match
AND dept IN ('finance', 'customer service') -- scalar: knowledge domain restriction
AND access_level <= 2 -- scalar: permission filter
AND publish_date >= '2025-01-01' -- scalar: freshness
ORDER BY relevance DESC
LIMIT 5;One SQL statement returns knowledge chunks that are “most semantically relevant + keyword matched + authorized + within time validity,” ready to be passed directly to the LLM for answer generation.
After discovering a piece of violating content, quickly locate similar items in the historical library:
SELECT content_id, content_type, risk_level,
approx_cosine_similarity(embedding, @violation_vector) AS similarity
FROM safety.content_pool
WHERE description MATCH @violation_keywords -- full-text: violation keywords
AND content_type = 'image' -- scalar: content type
AND status = 'published' -- scalar: still online
ORDER BY similarity DESC
LIMIT 50;Doctors can retrieve similar cases from a historical imaging repository while constraining clinical conditions:
SELECT case_id, patient_age, diagnosis, report_summary,
approx_cosine_similarity(embedding, @current_scan) AS similarity
FROM medical.imaging_cases
WHERE report_summary MATCH 'nodule ground-glass' -- full-text: report keywords
AND body_part = 'lung' -- scalar: exam location
AND diagnosis_category = 'suspected malignant' -- scalar: diagnosis category
ORDER BY similarity DESC
LIMIT 10;Alibaba Cloud EMR Serverless provides hybrid retrieval capabilities with both StarRocks and Spark engines, which complement each other:
| Dimension | StarRocks | Spark |
|---|---|---|
| Retrieval modality | Three-way fusion of full-text + scalar + vector | Dual-channel coordination of scalar + vector |
| Fusion algorithm | RRF / weighted fusion / learning-to-rank re-ranking | Index-coordinated filtering |
| Response latency | Sub-second to seconds | Seconds to minutes |
| Suitable scenarios | Online interactive retrieval, iterative data mining | Large-scale offline batch processing, batch vector generation |
| Data ingestion | Real-time ingestion and immediate retrieval | Batch ingestion, asynchronous index build |
| AI capability | Vectors generated at the application layer and passed in | Built-in AI Function, vectors generated in SQL |
| Post-retrieval processing | Partial Update writes back to source table directly | Batch ETL writes to target table |
| Data format | Direct read/write for Paimon / Iceberg / Lance | Direct read/write for Paimon |
Best combination: a closed-loop pipeline with data never leaving the lake
Figure 4: Spark offline write → StarRocks online retrieval and labeling → Spark downstream consumption, all on the same lake data
Full-text (BM25) + scalar + vector are unified and scored inside the engine through RRF / weighted fusion / learning-to-rank, rather than being recalled separately and stitched together. In-Filter deep optimization ensures scalar filtering and vector search execute synchronously, improving recall by more than 30% while keeping the result size controllable.
Direct read/write support for Paimon / Iceberg / Lance lake tables, with retrieval results written back directly to the source table via Partial Update. Upstream Spark writes, StarRocks retrieval, and downstream training consumption all complete on the same dataset, with zero movement and zero redundant copies. One engine plus one storage layer reduces cost by 60% compared with traditional solutions.
The self-developed Native Reader/Writer directly parses lake table files. Combined with Lake Optimizer’s query planning tailored to file layout, plus multi-level caching and intelligent prefetching, read/write performance on open lake formats remains industry-leading.
Supports mainstream lake formats such as Paimon, Lance, and Iceberg, unifying five data forms in storage: structured data, semi-structured data, vectors, full text, and binary objects (Blob). AI Function can perform parsing, chunking, embedding, and structured extraction directly in the lake, without external pipelines.
Compute resources elastically scale with query load, and there is zero compute cost when idle. Indexes are stored with the data in the lake, so scaling up or down does not require rebuilding.
| Condition | Description |
|---|---|
| StarRocks instance | EMR Serverless StarRocks instance already activated (version 3.5 or later) |
| Lake connection | Paimon Catalog configured to connect to the data lake |
-- Step 1: Create table
CREATE TABLE my_db.docs (
doc_id BIGINT,
title STRING,
content STRING,
category STRING,
embedding ARRAY<FLOAT>
) USING paimon;
-- Step 2: Create indexes
CREATE INDEX idx_vec ON my_db.docs (embedding)
USING VECTOR ("index_type"="HNSW", "dim"="768", "metric_type"="cosine");
CREATE INDEX idx_ft ON my_db.docs (content) USING GIN;
CREATE INDEX idx_cat ON my_db.docs (category) USING BITMAP;
-- Step 3: Three-way fused retrieval
SELECT doc_id, title,
approx_cosine_similarity(embedding, @query_vector) AS score
FROM my_db.docs
WHERE content MATCH 'refund invoice'
AND category = 'finance'
ORDER BY score DESC
LIMIT 10;Q1: How is the performance of three-way fused retrieval?
A: Thanks to In-Filter optimization and Lake Optimizer, scalar conditions are pushed forward to greatly narrow the search range before full-text matching and vector computation. Hybrid retrieval on hundred-million-scale data typically completes in the sub-second to second range.
Q2: Which vector indexing algorithms are supported?
A: HNSW, IVFPQ, DiskANN, and other mainstream algorithms are supported. HNSW is suitable for high-recall scenarios, IVFPQ is suitable for ultra-large vectors with memory constraints, and DiskANN is suitable for large-scale vector retrieval on disk storage.
Q3: Which languages does full-text search support?
A: Chinese and English tokenization are supported, and the tokenizer can be configured through the GIN index. Chinese uses jieba by default, and English uses the standard tokenizer.
Q4: Which lake table formats are supported?
A: Mainstream open lake formats such as Apache Paimon, Apache Iceberg, and Lance are supported, with unified storage for structured data, vectors, full text, semi-structured data, and Blob objects.
Q5: Will Partial Update write-back affect other columns or reads from other engines?
A: No. Partial Update updates only the specified columns and does not touch the original data columns. After write-back, other engines such as Spark can immediately see the updated results, and data consistency is guaranteed by the lake table transaction mechanism.
Want to see how EMR powers elastic, fully managed big data analytics with Spark, StarRocks, and lakehouse-native AI? 👉 Try EMR on Alibaba Cloud or talk to our solution architect to explore how you can build data lakes, real-time warehouses, and Data + AI workflows with seamless scalability.
More Resources：
E-MapReduce Serverless Spark Free Trial:1000 CU*H 3 months !
Alibaba Cloud EMR Serverless Spark AI Function Multimodal & Autonomous Driving Practice
MaxCompute Agentic Ecosystem Debut: Let AI Directly "Read" Your Data Warehouse
25 posts | 0 followers
FollowAlibaba Cloud Big Data and AI - July 6, 2026
Alibaba Cloud Big Data and AI - December 29, 2025
Alibaba Cloud Big Data and AI - July 17, 2026
Alibaba Cloud Big Data and AI - April 15, 2026
Alibaba Cloud Big Data and AI - April 13, 2026
Alibaba Cloud Big Data and AI - July 6, 2026
25 posts | 0 followers
Follow
Big Data Consulting for Data Technology Solution
Alibaba Cloud provides big data consulting services to help enterprises leverage advanced data technology.
Learn More
Big Data Consulting Services for Retail Solution
Alibaba Cloud experts provide retailers with a lightweight and customized big data consulting service to help you assess your big data maturity and plan your big data journey.
Learn More
Realtime Compute for Apache Flink
Realtime Compute for Apache Flink offers a highly integrated platform for real-time data processing, which optimizes the computing of Apache Flink.
Learn More
Message Queue for Apache Kafka
A fully-managed Apache Kafka service to help you quickly build data pipelines for your big data analytics.
Learn MoreMore Posts by Alibaba Cloud Big Data and AI