By Yuxuan Wang (Apache Dubbo Committer)
With the rapid development of Dubbo3 towards cloud-native microservices, the Go implementation of Dubbo has undergone a comprehensive and significant upgrade, marking the most extensive upgrade since Dubbo3. This upgrade encompasses various aspects, including API, protocol, traffic management, and observability. The latest version of Dubbo-go brings the following upgrades:
• Full upgrade on the Triple protocol: The Triple protocol is now fully upgraded to be compatible with gRPC and standard HTTP clients. It provides a simple and clear API for writing RPC servers and clients, facilitating basic communication between components.
• Comprehensive service governance for microservice scenarios: The upgrade includes enhancements in configuration management, observability, traffic management rules, and ecosystem integration and adaptation.
With the implementation of the Triple protocol in Dubbo-go, you can effortlessly develop browsers and gRPC-compatible RPC services that can run on both HTTP1 and HTTP2 simultaneously.
As shown in the figure above, you can access the backend triple service published by Dubbo-go using the standard "http+json" format. This allows you to access the backend service on the Dubbo client through browser pages and mobile devices using the standard cURL tool. You can also communicate with Dubbo services using applications in the Spring ecosystem.
The Triple protocol is fully compatible with the gRPC protocol, enabling direct communication between Dubbo backend services and standard gRPC services. They can seamlessly communicate with each other, whether in unary or streaming communication modes.
To experience the upgraded Triple protocol, you can create a Dubbo-go server and publish a service based on the Triple protocol.
Create a server and start it. The server will monitor Triple protocol requests at the specified port.
func main() {
srv, err := server.NewServer(
server.WithServerProtocol(
protocol.WithTriple(),
protocol.WithPort(50051),
),
)
if err != nil {
panic(err)
}
if err := greettriple.RegisterGreetServiceHandler(srv, &api.GreetTripleServer{}); err != nil {
panic(err)
}
if err := srv.Serve(); err != nil {
panic(err)
}
}After the Triple service is started, the simplest way is to access the service by using HTTP/1.1 POST, and the parameters are passed as HTTP load in standard JSON format. The following is an example of access using the cURL command:
curl \
--header "Content-Type: application/json" \
--data '{"name": "Dubbo"}' \
http://localhost:50051/greet.GreetService/GreetA major upgrade of the Triple protocol is to support for direct access from standard HTTP tools. By using cURL, the cost of testing, verifying, and accessing Dubbo-go services from the front-end is greatly reduced.
The Dubbo Go SDK supports defining services using specific ways for IDL or programming languages. It provides a lightweight set of APIs to publish or call these services. In the previous section, we have already seen some examples of Dubbo-go API usage. Now, let's delve into a more comprehensive and detailed introduction of the API design in the new version of Dubbo-go.
For RPC communication scenarios, developers can simply use Dubbo-go to write the most basic RPC server or RPC client. In the new version of Dubbo-go, this can be achieved with just a few lines of code.
Usually, Protocol Buffer (IDL) is used to define a Dubbo service.
syntax = "proto3";
package greet;
message GreetRequest {
string name = 1;
}
message GreetResponse {
string greeting = 1;
}
service GreetService {
rpc Greet(GreetRequest) returns (GreetResponse) {}
}Use Protocol Buffers Compiler to generate the stub code from IDL. The code is not displayed here due to the length limit. See Quick Start on the Apache Dubbo official website. Next, implement the greettriple.GreeterClient interface and provide a custom service implementation.
type GreeterServer struct {
}
func (s *GreeterServer) SayHello(ctx context.Context, in *greet.HelloRequest) (*greet.User, error) {
return &greet.User{Name: "Hello " + in.Name, Id: "12345", Age: 21}, nil
}The following is an example of a simple RPC server that executes the protocol information and registers the service with the server:
func main() {
srv, err := server.NewServer(
server.WithServer_Protocol(
protocol.WithTriple(),
protocol.WithPort(50052),
),
)
if err != nil {
panic(err)
}
if err := greettriple.RegisterGreetServiceHandler(srv, &api.GreetTripleServer{}); err != nil {
panic(err)
}
if err := srv.Serve(); err != nil {
panic(err)
}
}As mentioned in the Triple protocol section, you can use cURL to directly test the running of server services. The following is an example of the corresponding RPC client.
func main() {
// for the most brief RPC case
cli, err := client.NewClient(
client.WithURL("tri://127.0.0.1:50052"),
)
if err != nil {
panic(err)
}
svc, err := greettriple.NewGreetService(cli)
if err != nil {
panic(err)
}
common.TestClient(svc)
}If you are developing a microservice application, then in addition to RPC communication, you usually need to configure some service governance for the application, such as the registry, configuration center, observability, etc.
The following shows how to use Dubbo-go to develop a microservice application.
First, create a server that represents a microservice application, register the service with it, and configure service governance such as a registry.
func main() {
// configure global configurations and common modules
ins, err := dubbo.NewInstance(
dubbo.WithName("dubbo_test"),
dubbo.WithRegistry(
registry.WithZookeeper(),
registry.WithAddress("127.0.0.1:2181"),
),
dubbo.WithProtocol(
protocol.WithTriple(),
protocol.WithPort(50052),
),
)
// create a server with registry and protocol set above
srv, err := ins.NewServer()
if err != nil {
panic(err)
}
// register a service to server
if err := greettriple.RegisterGreetServiceHandler(srv, &api.GreetTripleServer{}); err != nil {
panic(err)
}
// start the server
if err := srv.Serve(); err != nil {
panic(err)
}
}The instance here refers to the global configuration introduced in the new version. You can initialize the global configuration of all microservices here. Here, the microservice communication protocol and the registry are configured, as shown in the following code.
ins, err := dubbo.NewInstance(
dubbo.WithName("dubbo_test"),
dubbo.WithRegistry(
registry.WithZookeeper(),
registry.WithAddress("127.0.0.1:2181"),
),
dubbo.WithProtocol(
protocol.WithTriple(),
protocol.WithPort(50052),
),
)From here, the operation becomes more simple. Create a server, register the service with it, and then start the server, as shown in the following code. If you have more services to add, register them with the server one by one, and then start the server.
// create a server with registry and protocol set above
srv, err := ins.NewServer()
// register a service to server
if err := greettriple.RegisterGreetServiceHandler(srv, &api.GreetTripleServer{}); err != nil {
panic(err)
}
// start the server
if err := srv.Serve(); err != nil {
panic(err)
}This is the basic process of developing a microservice application. If your microservice application needs to call some remote Dubbo services, you can use the following method.
As shown in the following code, a client is created and a GreetService remote service proxy is generated. Then, the remote Dubbo service can be called just like a local service. The client automatically discovers server instances based on the registry and applies the load balancing policy to traffic.
func main() {
// configure global configurations and common modules
ins, err := dubbo.NewInstance(
dubbo.WithName("dubbo_test"),
dubbo.WithRegistry(
registry.WithZookeeper(),
registry.WithAddress("127.0.0.1:2181"),
),
)
// configure the params that only client layer cares
cli, err := ins.NewClient()
if err != nil {
panic(err)
}
svc, err := greettriple.NewGreetService(cli)
if err != nil {
panic(err)
}
resp, err := svc.Greet(context.Background(), &greet.GreetRequest{Name: "triple"})
if err != nil {
return err
}
logger.Infof("TRIPLE unary call resp: %s", resp.Greeting)
}In addition to the API mode, Dubbo-go also supports file-driven coding, which is suitable for larger-scale microservice development scenarios. In this mode, we put component configurations such as registry and protocol, and even the service declaration in the dubbogo.yml file. The framework loads the configuration file during the startup process.
The following is an example of developing a microservice application based on dubbogo.yml.
The server.go file is defined as follows.
func main() {
greettriple.SetProviderService(&GreeterServiceImpl{})
if err := dubbo.Load(); err != nil {
panic(err)
}
}The following shows the content of the dubbogo.yml file.
dubbo:
application: # The application information. After the service is started, the information is registered with the registry and can be identified by the client from the URL.
name: myApp
registries:
nacos:
protocol: nacos # Select nacos as the registry.
address: 127.0.0.1:8848 # nacos ip
group: DEFAULT_GROUP # nacos group. The default value is DEFAULT_GROUP.
namespace: 9fb00abb-278d-42fc-96bf-e0151601e4a1 # nacos namespaceID, should be created before. default public
username: abc
password: abc
protocols:
dubbo:
name: tri
port: 20000
provider:
services:
UserProviderWithCustomGroupAndVersion: # The interface trituple: interface name, version number, and group. The client and the server must be consistent.
interface: org.apache.dubbo.UserProvider.Test # The interface name is required.
version: myInterfaceVersion # Empty by default.
group: myInterfaceGroup # Empty by default.Compared to the previous method of API coding, the server.go file here only requires two lines of code. The dubbo.Load() function completes the automatic assembly of all configurations and starts the related components. When starting the application, you only need to specify export DUBBO_GO_CONFIG_PATH=$ABSOLUTE_PATH/conf/dubbogo.yml.
Starting from version 3.2.0, Dubbo-go has upgraded its built-in metrics collection capability. It provides various built-in tracking tags, including RPC calls (such as response time, QPS, call volume, successful requests, failed requests, and concurrent requests), registry, metadata center, and configuration center interaction statistics. It also supports multi-dimensional metric aggregation.
Dubbo-go can export these built-in metrics to the Prometheus and Grafana systems. The figure below shows the monitoring effect of the Dubbo-go 3.2.0 example in Grafana. More specific examples will be published later in the dubbo-go-samples/metrics repository.
Dubbo provides a variety of traffic management policies:
• Service discovery and load balancing. Service discovery enables dynamic publishing and unpublishing of service instances, while load balancing ensures even distribution of traffic across each instance.
• Traffic management based on routing rules. The routing rules match each request with the conditions and route the requests that meet the conditions to the specific service cluster.
Service discovery, relying on registries like Zookeeper, Nacos, and Istio, ensures that callers have access to the latest address of provider instances. Dubbo provides various load balancing policies for consumers, including random load balancing, consistent hashing load, weighted round robin, minimum activity first, and P2C.
Dubbo-go traffic management rules accurately control traffic based on applications, services, methods, and parameters. Traffic is matched against conditions based on the target service, method, and additional parameters in the request. Traffic that meets the matching conditions is then forwarded to a specific address subset based on specific rules. Below are some examples of specific scenarios where Dubbo-go traffic management rules can be applied.
• Weighted traffic distribution
• Canary validation
• Canary release
• Routing by request parameters
• Same zone priority
• Timeout adjustment
• Retry
• Throttling and degradation
The following is an example of Dubbo-go-based end-to-end canary implementation:
The following is an example of Dubbo-go-based weighted traffic forwarding:
As for Dubbo-go traffic management, a complete demo example with a mall system is provided here:
• Details of traffic management rules[1]
• Example interpretation of traffic management in a mall[2]
Dubbo-go generally follows the design principle of framework kernel + plug-in. The framework kernel section on the left defines some core concepts of Dubbo-go as a microservice framework, while the plug-in section on the right provides an extended implementation of the core concepts.
From top to bottom, the framework kernel can be divided into four layers:
• API Layer
Dubbo-go supports the service contracts based on IDL and interface/struct, taking into account cross-language and ease-of-use requirements. It supports the configuration mode of microservices based on YAML files. It provides synchronous, asynchronous, unary, and streaming RPC communication and coding models.
• Service Governance Layer
Dubbo-go has built-in multi-dimensional service governance capability abstraction to meet the core requirements of microservice development and cluster governance, including service discovery, load balancing, metrics, traffic management, and tracing.
• RPC Protocol Layer
The core RPC protocol implemented by Dubbo-go is the Triple protocol, which can run on HTTP1 and HTTP2 at the same time because it supports cURL direct access, and it is compatible with gRPC. In terms of design, Dubbo-go also supports multi-protocol publishing services. You can publish services of different communication protocols such as Triple, Dubbo2, REST, and jsonRPC at the same time in one process.
• Transport Layer
Dubbo-go supports HTTP1/2 and TCP transport layers, taking into account performance and versatility. It also supports multiple serialization methods.
The Dubbo-go plug-in system greatly enriches its features and ecology. The community provides a large number of built-in extension implementations. At the same time, developers can easily add extension implementations as required. Here are some typical plug-in definitions:
• Protocol
Based on the protocol plug-in, Dubbo-go provides built-in support for protocols such as Triple, Dubbo2, and REST. You can also extend more protocols for Dubbo-go.
• Service Discovery
Dubbo-go supports integration with mainstream registries such as Nacos, Zookeeper, and Polaris.
• Traffic Management
Dubbo-go supports the traffic rules defined by the Dubbo system, which can dynamically adjust service behaviors such as timeout period, retry times, and throttling parameters during runtime. By managing traffic distribution, you can implement a variety of services, including A/B testing, canary release, multi-version proportional traffic distribution, condition matching routing, and blacklist and whitelist.
• Metrics
Dubbo-go provides rich built-in tracking tags such as RPC calls (including RT, QPS, number of calls, number of successful requests, number of failed requests, and number of concurrent requests), registry, metadata center, and configuration center interaction statistics. It also supports multi-dimensional metric aggregation.
• Logging
Dubbo-go provides a common log collection interface that supports built-in Zap and Logrus.
• Tracing
Dubbo-go supports distributed tracing. You can use this plug-in to access tracing systems such as Zipkin, Jaeger, and Skywalking.
In the future, we plan to release an official stable version in February. For the detailed Roadmap, please follow the project warehouse: https://github.com/apache/dubbo-go
[1] Details of traffic management rules
https://cn.dubbo.apache.org/en/overview/core-features/traffic/
[2] Example interpretation of traffic management in a mall
https://cn.dubbo.apache.org/en/overview/tasks/traffic-management/
Official Release of IntelliJ IDEA and Apache Dubbo's IDEA Plug-in
508 posts | 49 followers
FollowAlibaba Cloud Native Community - April 23, 2023
Aliware - October 20, 2020
Alibaba Cloud Native Community - September 12, 2023
Alibaba Cloud Native Community - December 7, 2023
Alibaba Cloud Native Community - December 30, 2021
Alibaba Cloud Native Community - May 21, 2021
508 posts | 49 followers
Follow
Microservices Engine (MSE)
MSE provides a fully managed registration and configuration center, and gateway and microservices governance capabilities.
Learn More
Function Compute
Alibaba Cloud Function Compute is a fully-managed event-driven compute service. It allows you to focus on writing and uploading code without the need to manage infrastructure such as servers.
Learn More
Managed Service for Prometheus
Multi-source metrics are aggregated to monitor the status of your business and services in real time.
Learn More
Elastic High Performance Computing Solution
High Performance Computing (HPC) and AI technology helps scientific research institutions to perform viral gene sequencing, conduct new drug research and development, and shorten the research and development cycle.
Learn MoreMore Posts by Alibaba Cloud Native Community