Community

Blog Events Webinars Tutorials Forum
  1. Blog
  2. Events
  3. Webinars
  4. Tutorials
  5. Forum
Create Account
×
Community Blog Apache Fluss Graduates to a Top Level Project, Making Agentic Lake Fully Real-Time with Lakestream

Apache Fluss Graduates to a Top Level Project, Making Agentic Lake Fully Real-Time with Lakestream

Alibaba Cloud reaffirms its commitment to the open lakehouse ecosystem with fully managed Streaming Lakehouse solutions powered by Apache Fluss

by Jark Wu, Apache Fluss PMC Chair, Staff Engineer at Alibaba Cloud
Alibaba Cloud reaffirms its commitment to the open lakehouse ecosystem with fully managed Streaming Lakehouse solutions powered by Apache Fluss

Apache_Fluss_16_9_

The Apache Software Foundation has officially announced that Apache Fluss has graduated from the incubator to become a Top-Level Project (TLP).The graduation proposal passed with a unanimous vote from the Incubator Project Management Committee (IPMC) and was subsequently approved by the ASF Board of Directors — a testament to the project's technical maturity, community health, and governance excellence.

For Alibaba Cloud, this milestone carries special significance. Apache Fluss originated from our own Apache Flink team's deep experience with real-time data infrastructure at scale. Seeing it grow into a vibrant, independent global community under the Apache Way is a proud moment — and a validation of our belief that the best infrastructure is built in the open.

How We Got Here

Apache Fluss — short for Flink Unified Streaming Storage, and also the German word for "river" — was born out of a practical need. At Alibaba, our Flink team faced persistent challenges in streaming analytics: the lack of unified stream-batch storage, high compute costs, and overly complex data pipelines. Apache Fluss was designed to solve these problems by providing a unified streaming storage layer purpose-built for the lakehouse era.

After more than a year of intensive development and large-scale production validation within Alibaba, we open-sourced Fluss at Flink Forward Asia 2024 in Shanghai. In June 2025, the project entered the Apache Software Foundation incubator. Our decision to donate Fluss to Apache Software Foundation was deliberate: we believed that streaming storage, as foundational infrastructure, should be governed by an open, vendor-neutral community — not owned by any single company.

In just over a year of incubation, Fluss has grown from an Alibaba internal project into a truly global open-source effort — and today, it takes its place alongside the most respected projects in the Apache ecosystem.

Apache Fluss: Achievements and Capabilities

image

A Thriving Community

The numbers tell the story of a healthy, accelerating project:

  • 157 contributors from multiple countries and organizations
  • 2,000+ GitHub Stars
  • 1,700+ merged Pull Requests
  • An active, inclusive community built around the Apache Way

Technical Differentiation

Fluss is lakehouse-native streaming storage designed for real-time analytics and AI workloads. Its core capabilities include:

  • Efficient columnar streaming storage: Built on the Apache Arrow columnar format, with server-side column pruning and pushdown capabilities, Fluss reduces data reads and transfers for efficient streaming analytics.
  • Real-time updates and low-latency queries: Primary-key tables natively support streaming updates, partial updates, changelogs, key-value lookups, and Delta Join, allowing the same real-time data to serve both computation and point queries.
  • Stream and Lakehouse unification: Fluss continuously tiers real-time data into open Lakehouse formats such as Apache Paimon, Apache Iceberg, and Lance, combining data freshness with cost-efficient long-term storage under a unified table abstraction.
  • An open ecosystem with multi-language access: Fluss works with compute engines such as Apache Flink, Apache Spark and StarRocks and provides Java, Rust, Python, and C++ clients, making it easy for real-time analytics and AI applications to connect.

Proven in Production

Apache Fluss is already deployed in production at leading companies including Alibaba, Xiaohongshu, JD.com, Ant Group, Fresha, and iQiyi. Use cases span log analytics, real-time data warehousing, search and recommendation, indexing pipelines, and real-time feature services. During the 618 shopping festival of Alibaba, Apache Fluss successfully handled traffic at the scale of hundreds of billions of events — proving its readiness for the most demanding production environments.

Alibaba Cloud OpenLake: Enterprise Lakestream on the Cloud

For organizations looking to build a real-time Lakehouse in production, Alibaba Cloud offers the OpenLake solution — a comprehensive, fully managed data lakehouse solution where multiple specialized engines operate concurrently on identical datasets through a centralized metadata registry.

At the core of OpenLake, Apache Fluss and Apache Paimon form a unified Lakestream: Paimon preserves long-term historical data, while Fluss serves the latest data with sub-second freshness. Through Lakestream Union Read powered by Serverless Flink, Serverless Spark and EMR StarRocks, users and agents can query historical and real-time data from a single table view, gaining fresh, complete context from the past to the present for production-grade real-time decisions and actions.

This Lakestream capability is a key pillar of Alibaba Cloud's Agentic Lake vision — the evolution toward agent-native data engines with unified semantic layers. OpenLake's differentiation rests on three pillars:

● Open-source leadership: Alibaba Cloud leads contributions to both Apache Paimon and Apache Fluss.

● Multi-engine equality: Flink, Spark, StarRocks, Milvus, ElasticSearch, MaxCompute, and Hologres access the same data through a unified DLF Catalog, engine-neutral, no lock-in.

● Data+AI integration: The lakehouse natively combines with Platform for AI (PAI) for end-to-end capabilities from data preprocessing to model training and inference.

A New Chapter

Graduation is not the finish line — it is the starting line for the next phase. Apache Fluss will continue to evolve as the open streaming storage layer for the lakehouse, powering real-time analytics and AI applications around the world.

Alibaba Cloud remains committed to investing in open-source infrastructure and contributing back to the communities that make it all possible. The river is flowing. Let it run.

Learn more about Apache Fluss

Start building your streaming lakehouse today. Explore the fully managed Apache Fluss service on Alibaba Cloud.

Big Data Apache flink Realtime Compute for Apache Flink Streaming Lakehouse Apache Fluss
0 1 0
Share on

Read previous post:

Apache Flink Agents 0.3 Release Announcement

Apache Flink Community

208 posts | 59 followers

You may also like

Comments

Apache Flink Community

208 posts | 59 followers

Related Products

More Posts by Apache Flink Community

See All