A Review of KubeVela from Launch to Promotion to the CNCF Incubation

This article offers a complete review to sort out the traps in the evolution of the project, aiming to help the development of the entire open-source ecosystem.

By Sun Jianbo and Zeng Qingguo

In February 2023, KubeVela successfully entered CNCF Incubation after all ToC votes. It is the first application-oriented delivery and management platform that is being promoted to incubation in the cloud-native field. The core concept behind KubeVela is the Open Application Model (OAM) jointly released by Alibaba Cloud and Microsoft in 2019. KubeVela has helped hundreds of enterprises (such as DingTalk, China Merchants Bank, Li Auto, Mobile Cloud, and Baidu) build their cloud-native application platforms through its programmable and extensible architecture, good user experience, and a wealth of core ecosystem capabilities , significantly reducing the threshold for using cloud-native technologies.

KubeVela is also different from the conventional open-source model of the Internet giant. It has followed the principle of community initiation, open governance, and international operation since the first day. One of its core concepts is to always take the industry's broadest and most realistic scenarios as the compass for project evolution. Therefore, KubeVela has been listening to the voices of the community in its development path and takes the most common needs as its highest priority. We are honored to experience the whole process of a project, from community initiation to the growth of the user group. We have overcome many difficulties, from technology iteration and function improvement, community operation, and open-source governance to polishing products and establishing ecology. These may be the challenges that all open-source projects will encounter.

Today, we will do a comprehensive review to sort out the traps in the evolution of the project, hoping to help the development of the entire open-source ecosystem.

Project Initiation: Clear Goals and Positioning

When an open-source project is launched, its core is to clarify the project's goals and positioning. In the early stages of OAM/KubeVela's creation in2018-2019, we judged how simplifying and standardizing the operations and functions of application delivery and management will become the next evolution direction and the next focus of the market as cloud-native technologies gradually unify the abstraction at the infrastructure and workload levels. Here, we mainly consider four aspects:

Audience: Most developers (or developers of final business applications) are concerned about application development and deployment rather than computing and storage networks. This means substantial simplification and standardization in the application layer will become a strong requirement.
Positioning and Space: Kubernetes is clear to leave its abstraction at the infrastructure layer, which provides enough space and support for innovation and work at the application layer.
Industry Pattern: As Kubernetes is becoming a de facto standard, most technologies (such as OpenShift) are still packaged in a limited way, but native tools (such as Helm and Kustomize) are too simplistic. This does not meet the fragmented and diversified demands of cloud users, nor does it create a user-friendly experience.
Technical Reserves: The gradual popularization of IaC technologies (such as CRD Operator and Terraform) provides a programmable and modular application management abstraction for rapid delivery. Based on Kubernetes, an independent application management and delivery system can focus on the layer itself without paying attention to the infrastructure layer.

Based on the above trends, Alibaba began to deploy application delivery and management in 2019 and proposed a series of pilot explorations and practices, including Helm/Kustomzie application management and multi-cluster application delivery. We finally set our core goal and vision of making software delivery easier, more efficient, and more reliable in today's popular hybrid and multi-cloud environments. We also set an infrastructure-independent, user-friendly, flexibly extensible application delivery abstraction as our core deliverable. This application delivery abstraction is today's OAM spec. The subsequent KubeVela is a concrete implementation of this layer of abstraction.

Figure 1: What Is KubeVela?

Clear goals and positioning not only support the KubeVela open source team and a large number of community contributors to collaborate in a relatively loose mode for a long time, but also help the team to free themselves from a large number of messy requirements and focus on the most core problem domain. For example, KubeVela does not cover workloads. Users can integrate Kubernetes-native Deployment, scale out CRD Operator, or use a workload management tool (such as OpenKruise). At the same time, the team focuses on the integration capabilities and extensiblity of the project. For example, we have devoted enough energy to the orchestration of Kubernetes APIs, and any Kubernetes resources can be combined, split, checked the status, and passed parameters in the KubeVela system. This feature enables KubeVela to quickly position in the market and gain early users like Li Auto.

Early Evolution: Identifying Core Technical Principles to Be Adhered To

In the early stages of open-source projects, we often supplement core functions based on initially set goals. Prior to this, we may need to answer the question:

What design principles should we follow to differentiate our open-source projects?

The first year of the KubeVela project is the initial formation stage of core technical functions. At the beginning of the design, we set the following key principles:

Kubernetes-Native: OAM is not limited to the Kubernetes ecosystem, and we choose to implement the KubeVela control plane through the Kubernetes addon (CRD) mode. On the one hand, it makes full use of the final-state design concept of Kubernetes declarative APIs to provide users with ease of use and certainty, which is convenient for users to install and use. On the other hand, it helps us use the Kubernetes API ecosystem to quickly implement the core capabilities required by applications (such as Helm delivery, elastic scaling, service gateway, and canary release). The Terraform controller project released by the KubeVela team later proved the feasibility of connecting to the cloud based on the Kubernetes control plane.
Extensible: KubeVela provides user-friendly upper-layer abstractions based on OAM, and the scale-out of these abstractions is supported to meet various requirements of users. This brings huge challenges to the technical implementation, but it is also a very important innovation. It prevents the KubeVela project from becoming a useless project that can only solve the lowest common denominator problem.
Programmable: Among numerous scalability designs, KubeVela finally chose Infra as Code (IaC) programmable scalability approach. Since this approach is easy to use, modularize, and use as an addon. This choice provides the best solution for scaling out KubeVela and lays the foundation for ecological capabilities (such as the addon market).

Based on these principles, the project chooses the CUE configuration language as the cornerstone of dynamic programming capabilities. KubeVela 1.0 and earlier versions provide a flexible OAM implementation. However, the evolution of the project also makes KubeVela and OAM form a clear difference from the earlier version, which poses a big challenge. How can we ensure the compatibility of open-source projects?

Figure 2: KubeVela Programmable and Scalable Modular Design

Continuous Iteration: Maintaining the Compatibility of Open-Source Projects

KubeVela was created under the demands of many users in the OAM community. The engineers who participated in the early contributions were also the platform builders in the company who actively promoted the implementation of OAM. They not only provided a large number of suggestions and code contributions but also helped the community verify through actual scenarios. While promoting new technology, we know it is important to solve the original problems while minimizing the cost of adoption, which means not introducing new problems. Therefore, when KubeVela released its first version, early adopters were more likely to upgrade their OAM practices than to use a brand-new system. Similarly, compatibility considerations have been given priority in subsequent project iterations.

When the first phase of the KubeVela project is implemented, it is being adopted by the open-source community. According to a large number of user feedback and research, we found that in addition to basic workloads and O&M requirements, users also have requirements for application management policies (such as multi-cluster high availability, resource sharing, and resource recycling). However, there is an essential model behind various seemingly fragmented and complex application delivery scenarios, which is workflow. We soon began to simultaneously evolve OAM and implemented a very lightweight workflow engine in KubeVela. The workflow feature is complementary to Kubernetes and GitOps ecology. KubeVela's workflow steps can be scaled out arbitrarily, and the final-state semantics of the declarative API can describe the state machine of the workflow well. This makes KubeVela's workflow meet the needs of almost any delivery scenario. Therefore, this innovation has become the reason for KubeVela's popularity.

Figure 3: KubeVela 1.0 Remains Function and Model-Level Compatibility after Release

This will bring a lot of difficulties for the long-term evolution of the technical solution, so we added the project feature abandonment mechanism. Only a small number of unreasonable features with almost no users were abandoned, and the abandonment was notified in advance of two versions. We observed that user growth during the formation of core capabilities did not meet expectations. Through operational analytics, we found that whether it is GitHub or the official website, the initial access traffic is large, but the actual conversion data is not ideal. At this point, we realized that the project experience may not be satisfactory, and the use and feedback of the community users are important inputs driving the continued growth of our project.

User Conversion: Focus on Ease of Use and First-Time Experience

The philosophy of KubeVela is industry-leading. so we have been doing a lot of evangelism and have attracted many users. However, many users have provided feedback that they did not understand what the project was about after reading the official website of the KubeVela project. Then, we realized that the project's usability was a problem. We took ourselves out of the identity of the project maintainer and began to examine the project as a user. Working as new users exposed to the project for the first time, we asked ourselves three questions:

Can I figure out what the project is solving?

The KubeVela project has a certain understanding threshold. It is based on OAM, and the core idea of OAM separation of concerns divides the platform users into the platform builders and end users, which brings a lot of cognitive costs to users who directly contact KubeVela. During the nearly one-year period from version 1.0 to 1.5, we will continue to reconstruct our official website's documents according to users' feedback, making an analogy with the keyword and scenarios that resonate with the majority of users. At the same time, we have been doing subtraction to hide the advanced functions of the project to a relatively deep position.

On the other hand, we continue to define the target user group in product implementation, positioning the core of the KubeVela controller to platform engineers, and the initial positioning of the VelaUX sub-project is out-of-the-box for business developers. We also provide a reference and basic framework for platform engineers to construct enterprise platforms. Through clearly positioned products to cultivate users' minds, more key information will be disseminated to community users and then through the community to influence the wider ecology.

Can I immediately think of whether my scenario can use it? Can it meet my needs?

The attention of users is limited. Through the analysis of user access data on the official website, we found that the average residence time of new users on the website does not exceed three minutes. How can we quickly capture the interest of target users? The design of the document structure is important. Through a large number of interviews with community users, we realized that most users would look for projects with demand keywords, so we adjusted the sidebar directory to reflect the core functions of the project with familiar keywords to help users match faster.

In addition to project documents, another key point is to refine user cases since user cases of the industry head have a strong driving effect. For example, Li Auto took the lead in adopting KubeVela, and it didn't take long for Xiaopeng to adopt it. Not long ago, Aurora (a giant in the autonomous driving field) even contacted us on Slack concerning the adoption of KubeVela for the whole company. The case of China Merchants Bank also led a large number of users in the financial industry to adopt KubeVela. The cases are accumulating slowly. It takes time and requires the project maintainers to be patient with community users and the companies supporting the development of the project behind the open-source project to be far-sighted. We are grateful to the maintainers of Alibaba Cloud, China Merchants Bank, Napptive, and other companies for their continuous and firm investment.

I want to see the actual running effect of the project. Can I experience the complete Demo in a few minutes?

This problem involves a key feature of open-source projects, which is open-source projects must have the feature of self-help. Users must have the ability to install, use, maintain, and spread in the community. If an open-source project cannot be started by most users themselves, it is difficult to succeed. Therefore, we will check the following two items every time we release a new version.

Can most users install it smoothly? For the international KubeVela project, we need to help users overcome network obstacles, whether accessing documents or downloading and installing domestically or abroad. In order to reduce the installation complexity, we even launched a special project called VelaD to help users one-click pull up a complete experience environment (including Kubernetes from a single server offline), significantly reducing the threshold for users to get started. In addition, there are experience optimizations (such as ARM architecture image support and one-click version upgrade).
Is the first use case simple enough to illustrate the project features? The first use case in the early days of KubeVela was either too simple to see the features or too complex to run if there are no multiple clusters. After continuous polishing, the first use case revolves around core concepts and embodies core capabilities (such as workload abstraction, delivery strategy, multi-cluster, and workflow). It has no additional requirements for the user environment.

Figure 4: VelaD Offline One-Click Installation of Complete Environment, Including the Kubernetes Cluster

Gradual Maturity: Product-Based Operations and Community Users

As the project gradually matures, the community will attract a steady stream of users, and the key to the success of open-source projects is a large number of users' adoption. KubeVela is fortunate to have a large number of scenarios within Alibaba that can help polish and incubate, and we also pay great attention to the needs of community users. In addition to holding regular community meetings at home and abroad every week, we will organize one-on-one exchanges meetings with different enterprises and even help them formulate a complete platform architecture. The accumulation of this process is mutual. The KubeVela maintenance team learns about the current situation and differentiated pain points from the leading enterprises in various industries and can design functions from a broader perspective.

After communicating with community users, we summarized some key points:

How can you give open-source users a sense of security? For infrastructure-based open-source projects, once adopted, they will become part of the core architecture of the enterprise. Compared with self-built projects, adopting open-source makes them feel insecure. It is important to give enterprises a sense of security for the open-source project. The first is the diversification, long-term stability, and high activity of the project maintenance team. KubeVela is an incubation project managed by CNCF and maintained by Alibaba Cloud, China Merchants Bank, Napptive, and other companies. At the same time, KubeVela continuously maintains high requirements on code quality and pays attention to security risk and stability. KubeVela has more than 40 continuous integration test items, with an overall test coverage rate of more than 60% and a coverage rate of more than 90% for core scenarios. Each merged code can guarantee compatibility and has high requirements on the capabilities of code contributors. The team is also highly sensitive to security risks and stability issues and will release a fixed version as soon as it is found. Finally, provide certainty, with clear milestones, release plans, and firm execution. Since KubeVela is released, major versions are released every 2-3 months, and minor versions are released every 1-2 weeks. The community has maintained a high popularity. So far, more than 150 versions have been released, and each version has an accompanying change document.
How can you balance the niche needs of community users? In the landing process, due to the differences in the scenario, community users will produce niche needs. Our strategy for this is to prioritize Blocker Issues, which means if some function settings and implementation directly hinder the scenario landing, such problems should be solved in the first place. For the requirements of adding new functions, the community will evaluate together and wait for more feedback. After confirming it is a common requirement, it will be added to the version plan. For example, in the early stages of KubeVela, some users wanted to connect to the unified authentication platform within their enterprises, which may not be the standard protocol mode. Through the discussion of the community Issue, some contributors pointed out that Dex was adopted as the unified mode of enterprise single sign-on, and different enterprises could connect to Dex by themselves. This allows the project to pool its efforts and focus on long-term development.
How can you obtain overseas users? How to open the international market is an unavoidable topic for open-source projects in the infrastructure field to achieve real success. On the one hand, the project should focus on the forefront of the field, maintain the advanced nature of technology, listen to the usage scenarios and habits of foreign users, and make a keen judgment on the development trend of technology. On the other hand, the core team of KubeVela is all based in China. It has overcome many objective difficulties (including time difference, language, and culture). It has insisted on holding community meetings at relatively appropriate times in North America and continuously advertising at various overseas conferences (such as KubeCon). Documents and articles are published in Chinese and English. At the same time, it actively answers questions in Slack and Issue to meet the needs of foreign users that want peer-to-peer communications. This requires project maintainers to have strong comprehensive strength and passion. The team is actively cultivating overseas contributors. With the continuous prosperity of the KubeVela ecosystem, corporate partners (such as Napptive and Guidewire) are joining the community to assume the responsibility of project maintenance.

Figure 5: OAM/KubeVela Community Meetings Abroad Rain or Shine

Ecological Construction and Community Operation

Finally, let"s talk about community ecosystem. Building a good open-source project is not the same as building cutting-edge technology. It is more like building a good product in the target field. In addition to polishing product features, we need to get more developers' recognition and participation through the continuous operation of the project and community governance. Developers need to form a consensus on KubeVela and actively join and participate in the collaboration to achieve this goal.

KubeVela was created in the OAM community and is the first project to implement the application-centric design concept. Therefore, in the early stage, we must first let everyone understand the idea of OAM before they have the will to understand what KubeVela is. We have done a lot of advertising and jointly released the industry's first cloud-native technology open course with CNCF to introduce how OAM's application-centric concept solves application development problems in the cloud-native era, which has won the resonance of many developers. In May 2021, Alibaba Cloud released the industry's first cloud computing open architecture standard with OAM as the core with China Academy of Information and Communications Technology (CAICT), which turned OAM into a reality and played a key role in the development of KubeVela.

To keep the community alive, KubeVela officially joined CNCF in July 2021, and the project audience was expanded to the world. We have invested a lot of energy in overseas operations, including holding community meetings with ecological partners (such as CNCF TAG App Delivery, Argo, FluxCD, Prometheus, and K3s) and creating team accounts in mainstream overseas developer communities. Gradually, our influence has penetrated North America, Japan, Spain, the United Kingdom, and other countries. In order to make the surrounding ecology flourish so the project can be widely spread in a larger field, we have specially established an addon system so developers in the community can integrate ecological projects and easily make them into an addon package with versions, unified warehouses, discoverability, distribution, and one-click installation.

Figure 6: KubeVela's Addon Ecosystem

KubeVela's technology is advanced, and there are many contributors in the community, but their levels vary. In order to ensure the quality of the project and not discourage the enthusiasm of the contributors, we will respond timely, fully communicate, and give contributors enough respect and patience for each code submission. At the same time, we will actively supplement the developer documentation and systemize the common problems encountered by developers. On the other hand, we have established a perfect promotion mechanism for developers. From the organization's member, reviewer, approver, and finally to the maintainer, there are clear conditions to reach, so developers can have a sense of accomplishment.

Today, KubeVela contributors can be found in more than 70 companies and organizations worldwide. The idea we advocate is that the form of open-source contribution is not only code. One share or one answer is also a contribution to the project. We are also establishing and gradually improving the community promotion model and collaboration mechanism to allow everyone to participate in the community more standardized and efficiently. The KubeVela community is still young, and many things are improving. It is great to have so many friends working together to make the team stronger. We also look forward to more friends joining the KubeVela community to experience the charm of open-source.

Community

A Review of KubeVela from Launch to Promotion to the CNCF Incubation

Project Initiation: Clear Goals and Positioning

Early Evolution: Identifying Core Technical Principles to Be Adhered To

Continuous Iteration: Maintaining the Compatibility of Open-Source Projects

User Conversion: Focus on Ease of Use and First-Time Experience

Gradual Maturity: Product-Based Operations and Community Users

Ecological Construction and Community Operation

Read previous post:

Read next post:

Alibaba Cloud Native Community

You may also like

Comments

Alibaba Cloud Native Community

Related Products

Cloud-Native Applications Management Solution

Managed Service for Prometheus

Container Service for Kubernetes

ACK One