Before answering what Pixiu is, let's briefly explain what Dubbo is. Dubbo is an open-source, high-performance RPC framework with rich service governance capabilities and excellent scalability. Dubbo-go is extended from Dubbo to provide users with the Dubbo solution written in Golang. This bridges the gap between the two languages and makes Dubbo closer to cloud-native.
As a Golang service, Dubbo-go achieves mutual calls with Dubbo services. However, users often have the requirement to expose Dubbo services in a RESTful style while also considering internal Dubbo calls in daily use scenarios. Pixiu (in Chinese: 貔貅, former name: dubbo-go-proxy), the gateway of Dubbo API was created to solve this issue. The name Pixiu was adopted because Java Zuul is a similar product, and its title is based on a fictional monster. As a Chinese product, Pixiu uses the name of a similar mythical animal from China as its project name. Building a whole cloud-native ecosystem also shows the determination of the Dubbo community.
Java represents the best-developed multi-language ecosystem of Dubbo, and Golang is the second. Other languages are poor-developed. The dubbo-go-pixiu project is based on Dubbo-go. Its interface protocol layer supports 7-layer HTTP request calls and plans to support gRPC request calls in version 0.5. Pixiu's other mission is to serve as a new Dubbo multi-language solution.
Pixiu is a cloud-native, high-performance, and scalable microservice API gateway based on Dubbo-go. As a gateway product, Pixiu helps users easily create, release, maintain, monitor, and secure APIs of any size. It also helps users receive and process tens of thousands of concurrent API calls. It provides the services of traffic management, CORS support, authorization and access control, restriction, monitoring, and API version management. In addition, as a derivative of Dubbo, Pixiu can help Dubbo users achieve protocol conversion and cross-system, cross-protocol sharing of service capabilities.
The overall design of Pixiu complies with the following principles:
Non-RESTful APIs and Dubbo services often need to be modified before they can be opened in the RESTful API style. Pixiu provides the protocol conversion function. With Pixiu, developers can configure their HTTP API or Dubbo API and open them in the style of RESTful API. Pixiu v0.2.1 supports the protocol conversion from HTTP to Dubbo based on generic calls. It also supports the forwarding of HTTP protocols. In future versions, the community will add support for the gRPC and HTTP/2 protocols.
The configuration of typical gateways is often difficult and complex. Pixiu is an easy-to-use gateway product. It was designed with three layers of configurations: global configuration at the gateway layer, configuration at the API resource layer, and configuration at the HTTP verbs method layer. Users can achieve deep customization with three different layers of configuration. Pixiu supports unified default configurations, local configuration files, and unified configuration servers. In addition, a console module is also provided to support hot updates of configuration. The console interface of Pixiu is also under development.
General functions, such as retry, circuit breaking, traffic control, and access control no longer need to be achieved repeatedly on each backend service. Pixiu can be used for global control through the configuration of filter or respective configuration based on API. As such, developers can focus on business logic and services instead of spending too much time maintaining infrastructure.
Different scenarios have unique requirements. Pixiu uses the plugin mode to meet the customization requirements of different users. Developers can embed their business logic in the Pixiu gateway in the form of a filter by writing the go plugin to achieve functions, such as enterprise login authentication.
Figure 1: Core Functions of Pixiu
Figure 2: Pixiu Architecture
Since API Gateway derives from Dubbo, Pixiu was built with Golang for the following reasons:
In Pixiu, Listener indicates the way in which an external user can access Pixiu. Users can expose the gateway by configuring properties, such as the specified protocol type, address, and port. Pixiu supports HTTP and will support gRPC in the future.
listeners:
- name: "net/http"
address:
socket_address:
protocol_type: "HTTP"
address: "0.0.0.0"
port: 8888
config:
idle_timeout: 5s
read_timeout: 5s
write_timeout: 5sRouter is the routing module of Pixiu. Pixiu stores exposed URLs in the form of trees in memory based on configuration files. When a request is sent to the router, the corresponding backend service and its API configuration are searched according to the URL and HTTP methods. The information is encapsulated in the request to provide sufficient content for subsequent filters and client calls.
Router provides the following functions:
Filter in Pixiu is a major module used to achieve extra functions and scalability. Filter is implemented in ways similar to the filter in Dubbo-go. The call chain is generated based on the specification of the filter in the configuration. Before the backend service is called, the logic in each filter is run once. Thus, functions are achieved, such as throttling and logging.
Users can write a go-plugin to achieve customized filters. Users can refer to the following configuration, load .so file, and specify the plugin group and plugin name in API config to achieve customized filters.
pluginFilePath: ""
pluginsGroup:
- groupName: "group1"
plugins:
- name: "rate limit"
version: "0.0.1"
priority: 1000
externalLookupName: "ExternalPluginRateLimit"
- name: "access"
version: "0.0.1"
priority: 1000
externalLookupName: "ExternalPluginAccess"
- groupName: "group2"
plugins:
- name: "blacklist"
version: "0.0.1"
priority: 1000
externalLookupName: "ExternalPluginBlackList"Client is responsible for calling specific services. Pixiu supports HTTP and Dubbo backend services. The community will gradually support gRPC and other clients to meet the needs of different protocols.
The implementation of the HTTP client is relatively simple. A request is generated and called through the Golang official package net/http based on the backend service information obtained from the Router.
The Dubbo client is slightly more complicated in implementation than the HTTP client, and it is based on the generic call of Dubbo services. This is a very basic function provided by Dubbo. Dubbo can use the generic call technology to initiate a service call. Users only need the name of a method called, parameter type, and return value type to accomplish this. A generic call of the server from the client can discover services through the registry center or connect with the server directly to dynamically call services.
As shown in the following code, Pixiu dynamically configures referenceConfig and then generates the Generic Client (generic call client) of Dubbo through GetRPCService for calls in the next step.
referenceConfig := dg.NewReferenceConfig(irequest.Interface, context.TODO())
referenceConfig.InterfaceName = irequest.Interface
referenceConfig.Cluster = constant.DEFAULT_CLUSTER
var registers []string
for k := range dgCfg.Registries {
registers = append(registers, k)
}
referenceConfig.Registry = strings.Join(registers, ",")
if len(irequest.DubboBackendConfig.Protocol) == 0 {
referenceConfig.Protocol = dubbo.DUBBO
} else {
referenceConfig.Protocol = irequest.DubboBackendConfig.Protocol
}
referenceConfig.Version = irequest.DubboBackendConfig.Version
referenceConfig.Group = irequest.Group
referenceConfig.Generic = true
if len(irequest.DubboBackendConfig.Retries) == 0 {
referenceConfig.Retries = "3"
} else {
referenceConfig.Retries = irequest.DubboBackendConfig.Retries
}
dc.lock.Lock()
defer dc.lock.Unlock()
referenceConfig.GenericLoad(key)
clientService := referenceConfig.GetRPCService().(*dg.GenericService)In the client of the generic call, the key step in executing the generic call is generic_filter (the following code segment) in Dubbo-go. When you call the Invoke function of generic_filter, the first parameter in the parameter list of invocation is the method name, the second is the list of parameter types, and the third is the list of parameter values. generic_filter converts the requested list of parameter values into a map with unified format (struct2MapAll in the code.) Then, it converts the serialization and deserialization of the class (struct in Golang) into the serialization and deserialization of the map. This makes it possible to inject the hessain library through hard coding without the POJO description to complete the generic call of Dubbo services.
func (ef *GenericFilter) Invoke(ctx context.Context, invoker protocol.Invoker, invocation protocol.Invocation) protocol.Result {
if invocation.MethodName() == constant.GENERIC && len(invocation.Arguments()) == 3 {
oldArguments := invocation.Arguments()
if oldParams, ok := oldArguments[2].([]interface{}); ok {
newParams := make([]hessian.Object, 0, len(oldParams))
for i := range oldParams {
newParams = append(newParams, hessian.Object(struct2MapAll(oldParams[i])))
}
newArguments := []interface{}{
oldArguments[0],
oldArguments[1],
newParams,
}
newInvocation := invocation2.NewRPCInvocation(invocation.MethodName(), newArguments, invocation.Attachments())
newInvocation.SetReply(invocation.Reply())
return invoker.Invoke(ctx, newInvocation)
}
}
return invoker.Invoke(ctx, invocation)
}After the brief description of the four modules and the registry center in the preceding sections, you can see that when a request is received by Pixiu through Listener, the request is transmitted to the router. Based on the interface configuration, the router locates the target backend service from the original request and delivers it with the relevant API configurations to the filter module. The filter module performs operations based on the order of original request, API configuration, and other information. Finally, the request reaches the client, through which the backend service is called.
Figure 3: Pixiu Iteration Stages
As a gateway product, Pixiu's derivative projects will be launched in the future, majorly aiming at making it more usable. For example, Dubbo-go needs to generate service metadata and write it to the registry center through configuration files since the Golang language lacks native annotations. Some employees of Kaikela Education wrote a tool to scan codes. They add corresponding annotations before each RPC service method, generating metadata by scanning annotations before the service starts. Pixiu also plans to provide a package in future versions to allow services to generate API configuration and register it to Pixiu through annotations with golangAnnotations.
Pixiu is a 7-layer protocol gateway, and its original version was defined as a Dubbo service gateway. As a product of the cloud era, Pixiu must be cloud-native. Version 0.2.1 achieved basic Dubbo/HTTP service proxy and some general functions of the gateway. Soon after, Version 0.3.0 added additional features, enhancements, and bug fixes, such as rate limit filer and integrate test. Version 0.4 (under development) and future versions will support gRPC and Spring Cloud service calls. MQ service will be supported in the future. In addition, the community will continue to optimize the configuration method and the official filter to make Pixiu more usable. As such, Pixiu can achieve more general functions of the gateway at the official level.
In the coming year, the community plans to support xDS API and expand Pixiu into a sidecar for Dubbo Mesh. The ultimate goal is to become Proxy Service Mesh from the existing Dubbo Mesh. Based on this form, script languages, such as JavaScript, Python, PHP, Ruby, and Perl, can benefit from the original Dubbo Mesh and improve their performance.
The ultimate goal of Pixiu in Dubbo Mesh is to gradually integrate horizontal traffic and vertical traffic into Pixiu and gradually make itself acquire the capabilities of Application Runtime. This is a key solution for the Dubbo multi-language ecosystem.
Feng Zhenyu, an Apache Dubbo Committer, is currently responsible for managing the entire team of the IT Department of a Hong Kong-based consumer goods company. In the summer of 2020, he accidentally read an article about Dubbo-go. He later joined the Dubbo-go community. Currently, he is leading the development of Pixiu v0.4.0.
Performance Comparison and Stress Testing – Nacos 1.0 vs. Nacos 2.0
506 posts | 48 followers
FollowAlibaba Cloud Native Community - May 31, 2022
Alibaba Cloud Native - October 9, 2021
Alibaba Cloud Native Community - March 6, 2023
Aliware - April 13, 2020
Alibaba Cloud Native Community - January 26, 2024
Aliware - August 18, 2021
506 posts | 48 followers
Follow
Function Compute
Alibaba Cloud Function Compute is a fully-managed event-driven compute service. It allows you to focus on writing and uploading code without the need to manage infrastructure such as servers.
Learn More
Managed Service for Prometheus
Multi-source metrics are aggregated to monitor the status of your business and services in real time.
Learn More
Cloud-Native Applications Management Solution
Accelerate and secure the development, deployment, and management of containerized applications cost-effectively.
Learn More
Lindorm
Lindorm is an elastic cloud-native database service that supports multiple data models. It is capable of processing various types of data and is compatible with multiple database engine, such as Apache HBase®, Apache Cassandra®, and OpenTSDB.
Learn MoreMore Posts by Alibaba Cloud Native Community