A Look Back Into Architecture and Service Mesh in 2020

By Zhang Yuchen (Tongzhao), Alibaba Cloud Delivery Expert

What Is Architecture?

"Architecture" has become a buzzword and has something to do with object-oriented programming, functional programming, modular design, and microservices. The architecture field is full of obscure terms. When these terms are translated from English to Chinese, part of their context is lost, producing different meanings in Chinese, such as "engine", "framework", "architecture", "application", and "system". Of course, everyone understands these terms in basically the same way and knows what others are referring to when we talk about these terms at work. However, when I want to gain deep insights into "architecture" or "software architecture", this question pops into my head: What are we talking about when we use these terms?

In fact, the definition of "architecture" is relatively fixed in a specific context. For example, in the construction sector, "architecture" may refer to a house structure, overall design, and composition. High-level design does not require a complete understanding of underlying details. For example, when we use RestTemplate to call web services, we do not care about Layer 4 socket connection because these details are hidden from us.

The term "architecture" as used in construction is vastly different from the term "software architecture". Software can be viewed as a type of changeable "ware", or product. Its definition is contrary to that of "hardware". We want software that is easy to modify so that we can promptly respond to different needs. Therefore, a software architecture is contrary to a building architecture that is costly to modify once completed.

Alan Kay once said, "Most software today is very much like an Egyptian pyramid with millions of bricks piled on top of each other, with no structural integrity, but just done by brute force and thousands of slaves."

I agree with Alan Kay's opinion but in fact, as the tombs of Egyptian pharaohs, the pyramids have a complete design logic and later evolved into outstanding feats of architecture after some initial attempts and maturation of construction management. For thousands of years, the Egyptian pyramids were the tallest buildings on Earth. There is still controversy over whether the Egyptian pyramids were built by slaves. The preceding figure is sourced from Isabella Jusková from Unsplash.

The capabilities and behaviors of software products are the building blocks of a project and deserve attention from engineers. In addition, engineers also need to focus on the architectural design of software in order to design elastic, high-performance, and high-availability systems that are easy to maintain and able to evolve continuously to adapt to future needs. A well-designed architecture should consider non-functional needs of software, including:

Easy Development: An engineer is often expected to start R&D immediately after joining a team and be able to write readable code and build a simple and unified development environment. Imagine the hard work required to quickly get up to speed on an unfamiliar project. When we started to apply Docker to our development work, the architect at the time required that we write only a one-line command to start the local test environment by using Docker. In addition, this command had to be applicable to all microservices. We worked hard to finally complete the transformation, achieving a significant improvement in efficiency. Three years later, any new member of our team could obtain a local environment with complete dependencies simply by running the ./run-dev.sh command on a terminal.
Easy deployment: A system is not very useful if it is costly to deploy. Even now, many enterprises are still using inextensible systems and expect them to keep running without any major faults. Many other enterprises use advanced orchestration systems like Kubernetes but have to release changes every Thursday when they need to deploy and launch applications. These enterprises do not adopt modern release methods, such as A/B testing, canary release, and phased release, because system transformation is costly or they cannot guarantee the appropriateness of modifications through sufficient automated testing, let alone implement continuous integration and continuous deployment (CI/CD).
Easy O&M: DevOps is designed to shorten the cycle from R&D to O&M so that we can handle problems more easily and look beyond the tasks at hand to focus on products. This does not mean that every O&M engineer is expected to become a Java expert and be able to quickly locate problems through heap analysis. Here, we emphasize closed-loop capabilities during the O&M process. As regards software products, we hope to develop products that are sufficiently independent, autonomous, independently deployable, horizontally scalable, and fully observable. This is based on the fact that hardware costs today are far lower than manpower costs.
Maintenance Costs: It is difficult to add new features to long-existing software. Similarly, it is difficult to determine whether to redevelop or refactor a long-running project. The real risk lies in the increased possibility of feature corruption due to code rewriting and the increase of technical workloads. Even when we redevelop some features and modules, we find it difficult to determine whether the rewritten code covers all the intended features and modules. In short, the principle "don't break anything" is difficult to follow in coding.
Evolution Capability: This is an important point. A well-designed architecture is able to facilitate the evolution of systems. This is like replacing the tire of a car, an easy change where you don't have to worry about the framework, underlying database, and operating system. However, this is difficult in practice. When we select technologies for a project, we may choose some technologies over others to adapt to the features of a database. However, we must separate upper-level design from database features and handle these features as underlying details. We have to work out an approach to organize beans in Spring without dependency injection and decide on a type of system frontend, which can be web, mobile, or both.

According to Robert C. Martin, better known as "Uncle Bob", a software product provides values in two ways: by implementing features and by building architectures. The architecture-related value may be more important because it represents the "soft" aspect of software.

This book provides few examples and lacks suggestions on how to refactor or improve currently popular frameworks. However, it provides enlightening methodologies. Robert C. Martin's opinion on databases, and specifically relational database management systems (RDBMSs), is worth discussing. He views databases as a kind of detail, which should be decoupled from businesses in the architecture. He also emphasizes the independence of business code from databases. When we write code to perform calculations, tables are often not the ideal data structure, so we can use trees and directed acyclic graphs (DAGs) in some scenarios. For example, we often encounter the problem of storing a tree in a database.

Microservices as a Type of Software Architecture, with No Need for Extension

According to our previous discussion, the microservices of a backend system do not affect the system's functional value. There are no functional differences between microservices and monolithic applications. Many of my colleagues prefer to develop monolithic applications to implement the same functions as microservices.

This means microservices are only a type of software architecture with some outstanding benefits, specifically, the ability to meet non-functional needs. However, there are no business needs that can only be met through microservices. Some people describe microservices by using patterns, while others view microservices as a type of granular-level service-oriented architecture (SOA). No matter how microservices are defined, we cannot expect that any single technology will be able to replace Oracle soon.

When we develop enterprise-ready backend applications, we have to deal with many non-business needs. For example, we may be required to develop an application that can run on three types of clients: mobile clients, mobile web clients, and desktop clients, and support different protocols, such as JSON and XML. We may be required to use different middleware to transmit messages. Then, we may have to do refactoring later because a piece of code is poorly written during the R&D process. We want to try the latest technologies but the cost is too high. Systems cannot be scaled out or scaling is costly.

Systems are extremely complicated and cannot run locally, resulting in low efficiency. These daunting issues are the primary reasons for using microservices. From the perspective of software architecture, we want to reduce the costs of inter-component changes by using loosely coupled, independent services. This means users can still view their shopping carts as normal when we update the notification feature. We only need to modify a small amount of code and then launch the new feature within a short time.

However, there is no such thing as a free lunch, so we have to consider the costs of using microservices before we can enjoy the benefits they provide, such as component as a service, loose coupling, independent deployment, service orientation, high maintainability, and high extensibility. For example, assume you have a complete understanding of microservices and are familiar with commercially available frameworks and products, so you are ready to get started with microservices. After you break down several services, you find that there are not enough automated methods, so you have to manually complete testing, coding, deployment, and monitoring, which results in a poor experience. The situation gets even worse if all services have to be launched on the same day due to the lack of an optimized deployment policy.

It is increasingly costly to rewrite the code of monolithic applications as they are scaled out. In most cases, we do not implement the correct practices for microservices models. For example, we used to think that traffic could be directly routed to a service after passing through a gateway, and we also ignored inter-service calls. An ideal microservices model should be a network in which each node is an independent, autonomous service.

We can use monolithic applications to meet the needs of some scenarios, but it is difficult to do so in distributed environments. For example, we can maintain consistency through RDBMS-provided database transactions. If the order placement service is separated from the pricing service, we have to implement distributed transactions to ensure consistency, especially eventual consistency. However, distributed transactions are extremely costly and complicated. In a monolithic environment, we can easily implement permission verification through the facade pattern. However, it is difficult to control inter-service calls in a microservices model.

It is difficult to break down services or define service boundaries. The most effective approach to date may be domain-driven design (DDD). Every natural domain or subdomain seems to map to a service because a sufficient bounded context can maintain service independence and hide service details within boundaries. This makes DDD a good solution. However, many people directly apply DDD to software development without making any adaptations, and successful cases are difficult to replicate.

I often divide services based on business domains, but I do not think this practice has anything to do with DDD. It is not necessary to specify the number of services to be the same as the number of domains or specify whether subdomains are independent of services. I prefer an evolving approach over a top-level design solution that aims to solve problems all at once. To break down a service at low costs, you only need sufficient infrastructure and automatic tools, instead of drawing all the architecture diagrams from the start.

A step-by-step approach always produces better results than trying to do everything at once. Therefore, when you are ready to use microservices, you'd better prepare yourself for all possible failures. Do not attempt to solve all problems through a single solution when you are dealing with technologies.

The Guardian website represents a successful case of microservice transformation. The website still depends on a large monolithic application but provides new features through microservices by calling the APIs of the monolithic application. Microservices can help you develop pages and features specific to frequent campaigns, such as sports events. The pages and features can be deleted or discarded after the campaigns are completed. In one of my earlier projects, I used the strangler pattern to remove a large, outdated JBoss application through equivalent substitution and refactoring.

Mark Cerny, the chief architect of PlayStation, shared his technical experience upon the release of PlayStation 5 this year. He proposed the idea of balancing evolution and revolution in the development of gaming consoles. We want to build on our years of development experience and reuse past successful practices, while hoping to use cutting edge technologies.

The Pros and Cons of Spring Cloud

In the Java world, Spring Cloud seems to be the best approach to implement microservices. I have seen many resumes where the job applicants view Spring Cloud and microservices as the same thing. I also have heard many complaints that it is impossible to implement microservices because we do not use the Java technology stack. Unfortunately, there is no programming language paradigm for software architectures, and many different design mode versions have been developed. Spring Cloud provides an attractive all-in-one solution, allowing you to build a next-generation architecture by adding a few JAR packages.

I used to marvel at the integrity of Spring Cloud, which provides a complete set of solutions for almost all common microservices. A majority of these solutions were developed at the application layer and are highly adaptable. They include Eureka for registration and discovery, Zuul (a gateway service that provides routing), Config Service, and Hystrix circuit breaker. Developed through a unified programming paradigm and based on annotation injection and configuration, these solutions provide abundant function options and effective integration methods. For example, each common function offers two options to choose from. Backed by the popularity of microservices and the large Java community, Spring Cloud became the first-choice of developers after it was successfully applied by Netflix. However, Spring Cloud has its weaknesses, especially in its functional design that deviates from containerization. The integration of Spring Cloud and Kubernetes is a hot topic today. The following sections explain their limitations.

Intrusion and Language Restrictions

Language may be the biggest problem. Currently, Java is still the most frequently used programming language and has caused many arguments in China. Both architects and entry-level programmers must try new technologies or functions to meet future needs. It is difficult to use Ribbon for custom client-side load balancing through a language other than Java. Though Java is a powerful programming language, we should have more languages to choose from in the future. A technical company should not restrict its programmers to a specific language. I believe that Golang, Rust, and Scala will become as popular as Java. Other services such as single sign-on (SSO) and Config Service are too integrated themselves and require many modifications to implement a single adaptation. Fortunately, however, these services are open-source. It is possible that we will have to refactor all frameworks once they have been developed to a certain stage. Ruby on Rails is an example. Intrusion is another problem. When I talked about software architecture, I recommended the separation of top-level design from the underlying framework and details. However, this separation is difficult because we have to reference annotations and JAR packages directly and frequently.

Cloud-native Integration

As regards cloud native, the Cloud Native Computing Foundation (CNCF) and Pivotal (VMWare) emphasize the importance of containerization, microservices, and cloud-oriented environments. Kubernetes is an advanced container orchestration technology that has become increasingly popular and made many contributions to CNCF projects. Kubernetes and Spring Cloud have many features in common, whereas Kubernetes and infrastructure as a service (IaaS) do not have any features in common. Today, few vendors launch purely IaaS products. It is difficult to choose between Kubernetes and Spring Cloud because the latter provides an attractive all-in-one solution.

Centralized Resources

Config Service and Eureka are services of the singleton pattern even though they provide clusterized methods with nearly 100% reliability. From the perspective of logical architecture, all microservices depend on Config Service and Eureka. These centralized resources are strongly coupled and exist until the last system that uses the resources is deprecated. We need to avoid using shared instances and resources in an architecture. Also, we need to ensure that an application keeps running when logging fails and can be started without locally installed Eureka.

The Fear of Platform Binding

Implementing service registration and discovery and load balancing within a process is an effective solution. This approach is platform-independent but cannot use the platform capabilities. Kubernetes binding seems a better approach because Kubernetes is more flexible than Spring Cloud and does not require binding to any basic cloud platform. However, Kubernetes is more difficult to manage and maintain than Spring Cloud. This does not mean Kubernetes is the only tool for microservice implementation. As a container orchestration platform, Kubernetes provides a wide range of functions in addition to running microservices, such as managing databases and middleware.

Methodological Capabilities

Now that the first half of 2020 is over, serverless technology has matured and Kubernetes has become more sophisticated. However, methodologies and architecture designs often lag behind technology development. For example, many of us still maintain the tradition of releasing changes on a certain day every week, still use outdated technologies, still implement Jenkins packaging, and have not yet installed Docker. These problems are not related to frameworks or platforms but due to the fact that we still follow old methodologies.

Constant Evolution

Applications must be developed based on the open-closed principle. This means applications should be open for extension to provide more choices in the future. We can use microservices to evolve our architectures while avoiding excessive costs. A componentized system provides components whose key features can be replaced, upgraded, and refactored independently without affecting other parts of the system. Such independence significantly drives down costs compared with the costly practice of abandoning an old, cumbersome framework and rewriting code to build a new system.

When advancing in the right direction with the right tools, we can control changes more quickly and frequently, choose new technology stacks with more confidence, and integrate two tightly coupled services. With the continuous aggregation and extraction of services, you will have a clearer picture of your system's logical architecture and be more confident in making changes. We can apply different storage technologies to each service. For example, we can store files in Object Storage Service (OSS) instead of storing images and videos only in an Oracle database.

Solutions and Problems of Istio and What Service Mesh Lacks

Though Istio received a great deal of negative feedback in the beginning, the community is still confident about it and finds it easier to use after the latest major modification. Now Istio can be applied to production environments at a moderate cost. However, it is easier to praise a thing than to criticize it.

Service mesh technology was conceptualized in 2017 but now has become the primary choice in the microservice sector. However, only Istio is sophisticated enough to implement the service mesh. I once used sidecars to implement reverse proxy, log collection, performance monitoring, and health check, but the results fell significantly short of the expected performance of service mesh.

I do not want to weigh the pros and cons of service mesh and Istio here. A process-independent solution can ensure system flexibility. We hope that architects can focus on businesses to extract real value and ensure architecture flexibility and then proceed to feature configuration, such as traffic shaping, service governance, end-to-end transmission security, throttling, and service discovery and registration. This is the reason why serverless technology is highly regarded. We still care about containers even when we want to delegate infrastructure control. The solution that integrates Istio and Kubernetes is effective and our team is currently studying it. Before we provide best practices to customers, we need to solve many problems, some of which are explained in the following sections.

Elasticity and Automatic Recovery

Users often need to use system metrics and parameters to trigger auto scaling. We can meet this requirement by using Cloud Monitor or developing a solution on our own. I prefer to decouple data from usage and solve problems in publish/subscribe (pub/sub) mode. For example, alerts and elastic rules are triggered when CPU utilization is too high and related data is displayed on a dashboard. We can add pods to handle part of the service load or start a new pod for automatic recovery when a pod exits abnormally. These operations are done manually.

Monitoring and Observability

Observability deals with logs and alerts, and we have implemented many practices in this area. However, we still have not implemented Application Real-Time Monitoring Service (ARMS), New Relic, and other application performance management (APM) functions on target platforms. We have not considered or designed capabilities to clearly display the real-time performance of each service.

Throttling and Downgrade Practices

Istio provides powerful traffic shaping capabilities. We need to summarize our practical experience in service governance operations, such as service downgrade and throttling. It is necessary to avoid cascade failure and fault escalation when you work with microservices.

Multiple Automatic Deployment Methods

We need to implement a wide range of deployment methods, such as blue-green release, phased release, and canary release, and apply these deployment methods to CI/CD. We may also write minor scripts, such as deployment util. In the future, a deployment tool, a configuration file, and CI/CD will constitute the deployment method for individual applications.

ABAC and OPA

Attribute based access control (ABAC) and Open Policy Agent (OPA) are highly customizable. We can use sidecars to verify permissions independently of processes and study how to customize OPA through Golang.

Saga and Compensation Events

Distributed transactions are a big challenge for microservice implementation. I wrote an article about distributed transactions and, based on my project experience, I prefer to import transactions into a single service. So far, I have not had the opportunity to write code for an online store system. Compensation events are often used to guarantee eventual consistency and unified verification.

High Availability and Chaos Engineering

It is not difficult to guarantee high availability for applications through Kubernetes because containers are easy to start and manage. The ability to kill random pods without affecting system availability is equivalent to the combined capabilities of Server Load Balancer (SLB), Elastic Compute Service (ECS), and auto scaling groups (ASGs). The real challenge comes from non-business pods, such as supervisors in Istio.

Microservice implementation requires architects to consider much more than just meeting business needs. Sufficient preparations in the right areas will make R&D and iteration easier.

Community

A Look Back Into Architecture and Service Mesh in 2020

What Is Architecture?

Microservices as a Type of Software Architecture, with No Need for Extension

The Pros and Cons of Spring Cloud

Intrusion and Language Restrictions

Cloud-native Integration

Centralized Resources

The Fear of Platform Binding

Methodological Capabilities

Constant Evolution

Solutions and Problems of Istio and What Service Mesh Lacks

Elasticity and Automatic Recovery

Monitoring and Observability

Throttling and Downgrade Practices

Multiple Automatic Deployment Methods

ABAC and OPA

Saga and Compensation Events

High Availability and Chaos Engineering

References

Read previous post:

Read next post:

Alibaba Cloud Native Community

You may also like

Comments

Alibaba Cloud Native Community

Related Products

Cloud-Native Applications Management Solution

Container Service for Kubernetes

Architecture and Structure Design

Alibaba Cloud Service Mesh