By Li Zhixin and Yu Yu
After becoming open-sourced in 2011, Dubbo has been adopted by a large number of small and medium-sized companies and has also been the most popular RPC framework in China. In 2014, Dubbo was suspended for a period due to the adjustment of Alibaba's internal organizational architecture. After the launch of Spring Cloud, the integration of the two systems promoted the popularity of microservices.
However, this world is changing fast. Since the birth of container technologies represented by Docker and container orchestration technologies represented by Kubernetes, the cloud-native era has arrived. In the cloud-native era, the immutable infrastructure has brought immutable middleware infrastructure to existing middleware. The gRPC unified the underlying communication layer. The protobuf unified the serialization protocol and service mesh represented by Envoy and Istio unified the service control and data layers gradually.
The idea of dubbo-go is "Bridging the gap between Java and Go." While maintaining the interconnection between Go and Java applications, dubbo-go embraces the cloud-native era with the help of the Go language (the de facto, first cloud-native language.) The dubbo-go community strived to create three products in 2020:
In a word, dubbo-go 3.0 has a new communication protocol, serialization protocols, application registration models, and service governance capabilities! This article focuses on the new communication protocol and application-level service registration and discovery model of dubbo-go 3.0.
There is a saying, “Progress is made by knowing yourself and your opponents.” The communication layer of dubbo-go 3.0 mainly uses gRPC for reference.
In short, the gRPC protocol is based on the HTTP2 protocol. It adds a specific protocol header starting with grpc-. It also uses a specific unpacking tool (protobuf) to serialize the data. Therefore, RPC calls are implemented.
As we all know, gRPC has almost no service governance capabilities. The existing Dubbo framework of Alibaba Cloud is equipped with RPC and service governance, which is as powerful as gRPC. However, since gRPC is used by everyone, the new communication protocol of dubbo-go 3.0 must be fully compatible with gRPC and services deployed by developers. It should also retain the existing Dubbo protocol and service governance capabilities and further introduce a series of new policies, such as mesh support and application-level service registration.
Currently, gRPC has already been supported in dubbo 2.7. Developers can use the protoc-gen-dubbo to convert the protobuf IDL protocol into stub supported by the framework. Then, the RPC process of underlying gRPC conn can be used to transfer the existing service governance capabilities to gRPC from top to bottom. By doing so, the support for gRPC services is implemented.
The dubbo-go 1.5.x also supports gRPC stream calling. Similar to unary RPC, by generating the stubs supported by the framework, the streaming RPC capability is incorporated into the framework based on underlying gRPC stream calls. However, since dubbo 2.7.x and dubbo-go 1.5.x do not support stream calling, no upper-layer service governance is supported for gRPC stream calling.
The problem developers face is that they may not feel at ease when using dubbo-go 2.7 for gRPC transmission. The upcoming protocol of dubbo-go 3.0 will solve this problem from the root.
The author believes the support from a service framework provided to the third-party protocol can be divided into three levels: application level, protocol level, and transmission level.
If a framework encapsulates the interface beyond protocol SDK, the support can be considered as application-level. This framework needs to be in line with the interfaces of lower-level SDKs, while the extensibility is poor.
In terms of protocol level, support is provided by the framework from the configuration layer to the service governance layer. The protocol layer and the network transmission layer all use a fixed communication protocol. Such a framework can solve service governance problems, but the framework itself cannot be fully adapted to the third-party protocol. If so, its support for the third-party protocol will be weakened, such as the lack of dubbo-go 1.5 support for RPC stream calling mentioned above.
Efforts need to be made at the transmission layer to further support more third-party protocols. The specific fields of the third-party protocols and the frame models and data streams of the underlying protocols depended (such as HTTP2) must be understood. Then, data exchange modules that are completely consistent with the third-party protocol can be developed as the bottom layer of this framework. By doing so, the extensibility of the protocol is maximized. The required fields can be added optionally while being compatible with existing protocols. Thus, functions that cannot be implemented by existing protocols can be developed, such as the back-pressure policy supported in dubbo-go 3.0.
The transmission process of unary RPC calling in gRPC based on HTTP2 is listed below:
The following figure shows the HTTP2 Header frame that contains the gRPC call information:
When the stream is called in gRPC, Data can be sent multiple times when the server returns the Data. After the call is completed, the Header is sent to terminate the RPC process, and the status information is reported.
The communication layer of dubbo-go 3.0 has the same communication process based on HTTP/2 protocol, ensuring the communication with gRPC underlying layer.
In addition to the communication protocol HTTP2, dubbo-go 3.0 will use the triple protocol (hereinafter referred to as the dubbo3 protocol) based on Google protobuf as the serialization protocol. This will lay the foundation for Dubbo to support more programming languages in the future. The current transmission model of dubbo-go 3.0 is shown in the following figure:
Dubbo-go 3.0 uses the next-generation service registration and discovery system. The original "interface-level registration and discovery" is replaced by "application-level registration and discovery."
Interface-level registration and discovery uses the RPC service as the key and the instance list as the value for data organization in the registry. The new "application-level service registration and discovery" uses an Application as the key and a group of Instances deployed by the application as the value. This brings two differences:
It can be concluded that the amount of data stored and pushed by the application-level model is proportional to the number of applications and instances. The number of applications or instances will only increase after the pressure of address push increases.
For an interface-level model, the data volume is positively correlated with the number of interfaces. Since one application usually publishes multiple interfaces, the data volume is generally dozens of times that of an application-level mode. Another key point is that the interface definition is more about the internal behavior on the business side. The interface-level model will cause opacity of the cluster size assessment. For the application-level model, the instance and application growth is usually planned on the O&M side with good controllability.
Industrial and Commercial Bank of China (ICBC) used to estimate these two models in production. The result showed that the application-level model can reduce the data volume to 1.68% of the original in the registry. The new model also made it easy for ZooKeeper to support 100,000-level services and nodes.
The decrease of data volume in the data center results in the loss of RPC service-related data in the registry. Only the data on application and instance level remains. To ensure that the missing data can still be perceived by the Consumer, a separate communication channel is established between the Consumer and the Provider. Currently, two solutions are available for metadata synchronization:
MetadataService
The entire development process should be more transparent to the users of early-version dubbo-go, while avoiding the impact brought by the specified provider in terms of extensibility. For that goal, a set of mapping relationships between RPC services and application names has been designed. These relationships automatically complete the conversion from RPC services to provider application names on the consumer side.
This design allows dubbo-go 3.0 to maintain the interconnection with dubbo 2.6.x, dubbo 2.7.x, and dubbo 3.0.x.
In terms of routing, a subset of IP addresses is selected based on the specific routing rules from the list of all existing IP addresses. The routing process involves filtering according to the configured routing rules and the result obtaining from the intersection of all the routing rules. Similar to a pipeline, multiple routers form a routing chain, which filters out the final destination address set from all the address tables. Then, the chain selects the address to be accessed based on the load balancing policy.
The logic of the routing chain can be simply understood as target = rn(...r3(r2(r1(src)))). For the internal logic of each router, multiple mutually disassociated address pools, such as addrs-pool-1 ... addrs-pool-n, are split by the addrs-in based on full address addrs-all in the router. The intersection is taken as the output address based on the defined rules. Likewise, the calculation of the entire routing chain is completed.
Failover fields can be configured in the routing rule configuration file. Failover can be triggered when an address cannot be found. Then, other subsets are selected, and the same process is executed. This procedure will be repeated until the address is found. Otherwise, an error indicating that the address cannot be found should be reported.
In the configuration of the routing rules, a match without any conditions can be configured. The final result is that at least one subset is selected for address protection.
Li Zhixin (GitHub ID: LaurenceLiZhixin) is the Development Engineer of Alibaba Cloud's Cloud-Native Middleware Team and a developer in the dubbo-go community. Li is currently a student at Sun Yat-sen University, majoring in software engineering. Li is good at using the Go language, focusing on cloud-native technologies and microservices.
Yu Yu (Github ID: @AlexStocks) is the Director of the dubbo-go project and community and a programmer with more than ten years of experience in infrastructure research and development on the server. Yu Yu has participated in improving Muduo, Pika, Dubbo, Sentinel-go, and other well-known projects. Currently, Yu Yu is engaged in container orchestration and service mesh in the trusted-native department of Ant Financial.
OpenKruise v0.7.0: A Controller for Periodic Task Distribution
506 posts | 48 followers
FollowAlibaba Cloud Native Community - March 6, 2023
Alibaba Cloud Native Community - December 30, 2021
Alibaba Developer - February 4, 2021
Alibaba Cloud Native Community - December 7, 2023
Alibaba Cloud Native - October 9, 2021
Alibaba Cloud Native Community - January 26, 2024
506 posts | 48 followers
Follow
Microservices Engine (MSE)
MSE provides a fully managed registration and configuration center, and gateway and microservices governance capabilities.
Learn More
Function Compute
Alibaba Cloud Function Compute is a fully-managed event-driven compute service. It allows you to focus on writing and uploading code without the need to manage infrastructure such as servers.
Learn More
Elastic High Performance Computing Solution
High Performance Computing (HPC) and AI technology helps scientific research institutions to perform viral gene sequencing, conduct new drug research and development, and shorten the research and development cycle.
Learn More
Quick Starts
Deploy custom Alibaba Cloud solutions for business-critical scenarios with Quick Start templates.
Learn MoreMore Posts by Alibaba Cloud Native Community