By Sun Jianbo (Tian Yuan), a technical expert at Alibaba Group.
The idea behind DevOps was proposed as early as 2007. During that time, the idea of cloud and its associated infrastructure had only just started up, but as the Internet and cloud became more popular and widespread, and the need for related applications, especially web applications, exploded. With this, application development needed to transition from a waterfall model to an agile model.
Before DevOps, applications were delivered through a complicated process that involved multiple teams. Minimally, this process needed to start with a developer team and continue on to an IT operations and maintenance team that would take care of deploying and running the application. However, this delivery method was relatively inefficient, and it couldn't keep up with the quick pace needed for the Internet era.
DevOps was a solution to this problem. It connected the development and operations teams, sped up the application delivery cycle, and improved the quality of applications. And by revolutionizing the way in which applications were developed and delivered, DevOps introduced a completely new culture and system of practices that have become widespread in the web application space.
However, now, will the rise of cloud native and application-management with Kubernetes, the question in many people's minds is whether DevOps has any case of staying around. In this blog, Alibaba technical expert Jianbo will discuss his thoughts on DevOps, cloud, and Kubernetes, and will address the question whether Kubernetes will kill off DevOps or not.
In its infancy, DevOps quickly rose to popularity. Its wide adoption, at the dawn and early morning hours of the Internet age, helped to meet the industry's demand for the agile development of web applications with ensured quality. DevOps also ushered in a "common space" where application development and operations teams could collaborate more closely than ever before. At that time, application development still was a relatively new industry.
DevOps was in many ways successful because it learned from the lessons of manufacturing past. Manufacturing, historically speaking, was able to achieve large-scale production thanks to the invention of the assembly line. And, in particular, through the standardation of each step along the assembly line, the manufacturing process came to run more smoothly, with each and every worker knowing exactly what his or her particular job and position was along the assembly line.
The way that DevOps borrowed from the lessons of manufacturing past was through its CI/CD pipeline model. This model helped to simulate the development of a series of automation and semi-automation tools, such as puppet, chef, and ansible. These tools integrated the scalability of compile scripts while also standardizing several of the operations involved in the development and operations management processes.
One point about DevOps to note is that it required several technicians to be involved in a single project, so dedicated "DevOps teams" became a common phenomenon in the IT departments of companies and other organizations alike. These DevOps teams served as a tool and platform where development was able to better approximate the end production environment, allowing for simple, one-click deployment and troubleshooting during the integration and delivery process involved in application development. This process in turn also allowed for issues to be weeded out during the development process so that they wouldn't turn up, or at least would be much less likely to turn up, in the actual production environment.
So, from what we've seen so far, one conclusion that we can make is that DevOps was more or less designed to help simplify the operations and maintenance side of application development. DevOps provided O&M procedures in the production environment through a set of automation tools, and by doing so, got rid of the need of dealing with all the nitty gritty details traditionally associated with the production infrastructure. At the same time, DevOps also helped to expose problems with applications during the initial development stages, again helping to simplify the operations side of things. To sum up things, DevOps was very much built for the benefit of the operations side of the equation.
With all of these advancements and advantages, DevOps at least at first did gain great popularity and become a crucial part of many application development workflows. But, as time went on, the holes in the DevOps world started to become clear. DevOps wasn't built in a totally sustainable way. DevOps doesn't provide any direct profits for an enterprise, nor does it have any products immediately associated with it. In fact, DevOps for many enterprises was associated as a cost, rather than an asset.
DevOps was costly in that it required there to be an often large dedicated team and it didn't necessarily make a company using it more profitable. And so, because of this, many companies weren't willing to invest in DevOps. And, as time went on, DevOps couldn't match the evolving demands of developers, and DevOps's development team also wasn't willing to continue to develop with new industry trends like cloud computing and the rising prominence of the open source community in mind. As such, DevOps slowly became a bottleneck in the application development and delivery process of several enterprises.
How did cloud start anyway? Well, it started with a wise company who made the connection that their needs also represented the needs and demands of the entire industry. That wise company was Amazon, and this was precisely the logic behind how Amazon Web Services, or AWS, was born. Amazon's dip into the world of public cloud services revolutionized the face of IT forever.
As early as 2006, Amazon became the first company in the world to take the basic components of IT infrastructure, like networking, computing and storage hardware, and sell them as an online service to users from across the US and ultimately all over the world. This innovation meant that, as a customer of Amazon, you could have the capacity to host your services online and agilely develop web applications without the need to purchase our own server hardware and related IT infrastructure.
Amazon, and its competitors to emerge to the cloud space, including Azure, and Alibaba Cloud, were able to make cloud a valuable option by making their services cheaper than building and hosting your own IT infrastructure. And, with cloud, also was born the basic services models of cloud, including infrastructure-as-a-service (IaaS), platform-as-a-service (PaaS), and software-as-a-service (SaaS).
In the early days of cloud computing, the main service model was IaaS. With it, customers, mostly enterprise-level customers, could purchase web servers, or virtual machines, as well as the relevant storage they needed from the cloud service provider. And, in the IaaS service structure, customers had full control on how they wanted to operate and maintain the infrastructure provided. The only difference between IaaS with that of traditional IT, of course, was that the object of maintenance was no longer physical servers. So, nothing really changed at the end of the day.
And, as cloud computing quickly advanced to what it is today, the capabilities of the cloud continued to be strengthened and enriched. The products and services offered on the cloud eventually came to cover all the various aspects and capabilities of the operations and maintenance side of IT, including all the services involved in managing the application lifestyle, from coding and hosting, to integration and delivery, all the way to monitoring, alarm systems and auto scaling capabilities.
In this quickly changing world of IT, DevOps was mostly unchanged, but it still had its uses. Connecting to the cloud, especially in the early days, could be difficult. Cloud service providers offer a complex and diverse array of services, and each provider's portfolio of services aren't exactly the same, and so customers need to be aware and understand how cloud services work and make sure that they are not locked into one vendor when migrating to the cloud. These decisions aren't easy.
DevOps, on the other hand, was more or less unchanged during this period, which in turn lead to greater customer confidence and ease of adoption. And, even with the emergence of cloud, DevOps could still help to simplify the complexities of the development environment just as it did in the past. With the only difference being that the basic IT infrastructure to manage was now running on the cloud. So, in other words, given its simplicity, DevOps still had its place in the market.
Now, let's discuss Kubernetes, the next big thing in IT to shake up the DevOps world. From the very get-go, Kubernetes was designed as a modern application infrastructure. Its creators at Google wanted to make something that would integrate applications with the cloud in a more intuitive way, expanding on all of the potential up there in the cloud. And with this, the ultimate goal of Kubernetes was to change the very way in which basic IT works, concentrating on how to make it better suited for applications and be a more efficient system for application delivery.
Realizing the high hopes and lofty dreams of its creators, Kubernetes, along with Docker and Operator came to be crucial open-source projects in the cloud native era of the cloud computing world. They made for a completely different world for application management and delivery. With Kubernetes in particular, developers only needed to use declarative syntax to control the final state of their applications, and Kubernetes would take care of all the rest.
It was this simplicity that made Kubernetes extremely popular. Declarative API actions are its bread and butter because, with them, Kubernetes was powerful enough to disrupt everything about how cloud works. And it was and is a powerful tool. Kubernetes users can declare a large variety of states quickly and easily with declarative API actions. Their capabilities were miles high, yet their responsibilities were greatly simplified.
Nowadays, as application developers, we can not only use the declarative syntax of Kubernetes to control the final running status of our applications, but we can also use it to declare several of the operations-related parts of our applications, too. For example, we can declare that an application be updated using a canary release method, or that, when CPU usage goes higher than 50%, the application must expand to an additional instance.
This all posed a great challenge for DevOps tools and teams. Because in the case that a company can simply use declarative API actions to have complete control over applications as well as several service level features, then wouldn't you just expect everyone to switch right on over to Kubernetes for their application development needs. Well, that was exactly the problem. The big question was then did people still have legitimate reasons to learn DevOps's CI/CD pipeline.
To put things another way, for the longest of times DevOps was the logical "glue" that held together application development to the basic IT infrastructure associated with it. But now, with Kubernetes and its fully equipped declarative API actions having infiltrated all the nooks and crannies of the application development infrastructure, Kubernetes has come to take on the role of the infrastructural "glue" rather well. At the same time, just as DevOps's past status was being challenged by Kubernetes, traditional middleware was also being pounded down by the big Service Mesh tidal wave.
So, now we've got DevOps, cloud, and Kubernetes, so what happens next for DevOps. Well, over the past several years, Kubernetes has been described as the perfect partner to DevOps in the context of cloud.
Similarly, there exists the idea that Kubernetes, like Docker, can solve the problems related to application runtime. On this note, Kubernetes has been likened as a sort of fashionable version of IaaS that has taken the application runtime responsibilities away from the virtual servers and has given them containers. So, in other words, as long as you connect the existing thinking and processes associated with DevOps to Kubernetes, then you'd be able to enjoy the lightweight nature and scalability of container technology. This would, of course, be a good pair for DevOps given DevOps's record for promoting agile development.
However, as far as we can tell now, this isn't the complete story. Kubernetes's roadmap of future development doesn't involve Kubernetes taking on an IaaS-role in the future. Rather, though Kubernetes is focused on changing the capabilities of the underlying technological infrastructure behind application development, Kubernetes is far from being infrastructure in its own right. More to the point, when it comes to application runtime, Kubernetes is more concerned with the statuses of applications and the application lifecycle, rather anything else. And, besides this, Kubernetes provides what could be called the "controller model", which makes the actual status of applications closer to their more ideal status, which goes well beyond typical application runtime management.
In other words, Kubernetes is concerned with the applications themselves, which makes it much different to the basic infrastructure of an IaaS system, and solidifies its status as an infrastructural "glue". Then, of course, if the capabilities of Kubernetes are sufficient, and it can serve as a glue between application development and the underlying infrastructure, then where does DevOps lie in all of this? Does DevOps need to exist? Anf in this so-called cloud-native era, can applications really just be declared, and everything else will fall into place？And will DevOps just vanish off the place of the planet?
Well, this view of things isn't fully in line with reality. Kubernetes still has several shortcomings to overcome before this could happen.
Kubernetes is a classic "Platform for Platform" project, so its API actions are far from being a true development tool. Consider a deployment object for instance, it would include both an image that the development side is concerned about, as well as the resource configurations of the infrastructure side, and even the container security configurations. These aren't things that can be achieved with Kubernetes alone, so the whole declare-and-you're-done idea far from reality. This is also why DevOps still has a use today. For most strings in Kubernetes, they still need to be supplemented by what DevOps does.
Next, the Kubernetes API only includes a little piece of the cloud resources you're need, such as PV/PVC for storage purchases, and Ingress for load balancing. And as such there's a lot of things Kubernetes simply isn't very good at. For example, a developer would struggle to use the declarative syntax of Kubernetes to express something related to a database, VPC, or message queue if he needed to rely on Kubernetes alone. DevOps helps with this. Moreover, existing solutions are dependent on the implementation of specific vendors—something that can lead to problems of vendor lock-in.
Last, Kubernetes's Operator mechanism is the secret to why Kubernetes capabilities could grow as much as they have already. However, one major problem is that operators cannot interact with each other. For example, let's say that you were to take and expand RDS through CRD and Operator to Kubernetes's declarative API system, and then someone else decides he or she wants to write a CRD Operator that regularly backs up RDS persistent files to cooperate with you. Well, it would essentially be impossible to do so without the intervention of DevOps's systems.
From our above discussion, it's pretty clear that existing Kubernetes projects still rely on DevOps to complete high efficiency application iteration and delivery tasks. This is unavoidable. Even though Kubernetes is pushed as an application-centric infrastructure, the reality is that, as a system-level project generated from Google Borg, Kubernetes still revolves around and functions based on several other existing infrastructural elements.
However, one thing that we cannot deny is that Kubernetes is aimed that the pursuit of NoOps. This was clear from the very get-go. Following this, the creation of the CRD and Operator systems brought to life the application-level concerns of Kubernetes. Interestingly, too, we're starting to see many DevOps processes becoming the declarative objects and control loops of Kubernetes. The Tekton CD project is an example of this.
If the future of Kubernetes is all for declarative application management and NoOps, then we also have reason to believe that DevOps will eventually disappear from the technical arena, but it will still remain as a kind of cultural heritage in the IT world. After all, the operations and maintenance engineers at the time that DevOps got big will probably become the compilers or the designers of Kubernetes's Controller and Operator.
A Decade of Container Development: Chronicles of Software Delivery
507 posts | 48 followers
FollowAliware - July 27, 2020
Alibaba Cloud Native Community - July 12, 2022
Alibaba Clouder - September 10, 2020
Alibaba Cloud New Products - November 10, 2020
Alibaba Clouder - September 1, 2020
Alibaba Cloud Native Community - December 29, 2023
507 posts | 48 followers
Follow
ApsaraDB for MyBase
ApsaraDB Dedicated Cluster provided by Alibaba Cloud is a dedicated service for managing databases on the cloud.
Learn More
Alibaba Cloud Flow
An enterprise-level continuous delivery tool.
Learn More
DevOps Solution
Accelerate software development and delivery by integrating DevOps with the cloud
Learn More
Container Service for Kubernetes
Alibaba Cloud Container Service for Kubernetes is a fully managed cloud container management service that supports native Kubernetes and integrates with other Alibaba Cloud products.
Learn MoreMore Posts by Alibaba Cloud Native Community
