Alibaba Cloud has come to implemented cloud native technologies on a large scale. At the KubeCon + CloudNativeCon + Open Source Summit held on June 26, 2019, Xiang Li, a CNCF TOC representative and the senior staff engineer at Alibaba Cloud, delivered a keynote speech. Mr. Li shared Alibaba's experience in scalability, reliability, development efficiency, and migration strategy, and discussed how to implement cloud native technologies and address technical challenges.
This article covers a presentation by senior engineer Xiang Li at KubeCon on how Alibaba Cloud has developed and implemented cloud native technologies on a large scale.
Why go cloud native? What benefits can cloud native technologies bring to us? From finding its way through the dark, to embracing open-source standards and contributing back to the community, what kind of challenges did Alibaba face to implement cloud native technologies? Does Alibaba have technical experiences to share?
Since 2011, Alibaba had begun to put the cloud native technology system into practice by leveraging containers. Alibaba, trailblazing in this industry, over time developed a containerized infrastructure architecture that is now today top-of-the-line among the global leading technology companies. This architecture is now the technological backbone of the entire Alibaba Group. Alibaba believes exploration is intrinsic to developing and discovering new technologies. Through much determination and exploration, Alibaba's technical team has revolutionized many of the ways that technology is used today, becoming a leader in developing cloud native technologies in China.
Alibaba's businesses are large and complex. A suitable starting point must be found to develop cloud native. Motivated by the cost pressure of the 11.11 global shopping festival, Alibaba chooses resource cost and efficiency optimization as the starting point of their journey into cloud native.
Alibaba leverages containers to develop low-cost virtualization and scheduling technologies. It provides flexible and standard deployment units, and changes the resource scheduling mode from static to dynamic and on demand. This improves deployment efficiency, solves the problem of resource fragmentation, and increases the deployment density. Employing technologies such as storage network virtualization and separation of storage and computing, Alibaba not only enhances the portability of tasks and improves resource reliability, but also reduces operating costs.
Motivated to reduce the cost of resources, Alibaba has completed overall containerization and replaced resource allocation with a highly efficient scheduling platform. Alibaba's cloud native exploration is still ongoing, however. Increasing the research and development (R&D) efficiency and accelerating the iteration are key to boosting Alibaba's business. Alibaba hopes to leverage cloud native technologies to improve the efficiency of developers.
To improve automation and simplify application deployment, Alibaba adopted Kubernetes as its container orchestration platform. Since then, Alibaba has dedicated its efforts to improve the performance and scalability of Kubernetes. Kubernetes also enables Alibaba to refine its R&D and deployment processes. To build more cloud-native Continuous Integration and Continuous Delivery (CI/CD) and further implement standardization and automation, Alibaba has introduced standardized application management tools such as Helm to manage the entire process, from R&D all the way to product launch. Alibaba also tries new and innovative deployment patterns like GitOps, and pushes forward the final state-oriented automated construction of the PaaS layer. Additionally, Alibaba is now beginning to explore service mesh, aiming to further improve the universality and standardization of service governance, lower the adoption threshold for developers, and further popularize microservices in multiple languages and environments.
In 2019, Alibaba launched the All-in-Cloud initiative. Through cloud native exploration and reconstruction, Alibaba builds a modern and standard infrastructure system. The container technology decouples applications from hosts when they are running. Kubernetes abstracts resources into Pods and volumes to unify the implementation of various resources. Intelligent scheduling on the PaaS layer makes it possible to automatically migrate applications and fix any instabilities. By using cloud native, Alibaba greatly simplifies the migration to cloud.
In the process of improving resource and personnel efficiency, Alibaba's entire infrastructure has become more open and connected to multiple open-source ecosystems. Alibaba also integrates and shares beneficial concepts, technologies, and ideas with open-source communities. Now, Alibaba Cloud operates China's largest cloud native application – the 11.11 global shopping festival. It also boasts the largest public cloud cluster and image repository in China. As the only vendor in China that was listed in Gartner's Competitive Landscape: Public Cloud Container Services Market, Alibaba Cloud has accumulated the most extensive and valuable customer practices.
At the 2019 DTCC, Dr. Feifei Li, chief database scientist at Alibaba DAMO Academy, gave a presentation on Cloud Native database technology and trends.
As the Cloud Computing tide strikes, the traditional database market is facing a situation of reshuffling. The rise of a group of new forces, including cloud databases, has shaken the monopoly of traditional databases, and the Cloud Native database led by cloud vendors pushes this change to the maximum.
What changes will databases face in the cloud era? What are the unique advantages of Cloud Native databases? At the 2019 DTCC (Database Technology Conference China 2019), Dr. Feifei Li, the vice president of Alibaba, gave a wonderful presentation on the Next Generation of Cloud Native Database Technology and Trend.
The following figure shows a Gartner report on the global database market share. The report shows that the current global database market share is about $40 billion, of which the database market share of China accounts for 3.7%, about $1.4 billion.
In terms of database market distribution, the five traditional database manufacturers, which are Oracle, Microsoft, IBM, SAP, and Teradata, account for 80%, and the cloud database accounts for nearly 10% of the share now and is growing rapidly every year. Therefore, Oracle and MongoDB are also vigorously deploying their competitive positions in the cloud database market.
According to the DB-Engines database market analysis, the database system is developing towards diversification, from the traditional TP relational database to the multi-source heterogeneous database form today. Currently, the database systems we are familiar with, such as commercial databases (like Oracle and SQL Server), and open-source databases (like MySQL and PostgreSQL), are still in the mainstream. However, some newer database systems, such as MongoDB and Redis, have opened up a new track. The traditional sales method for database license is gradually going downhill, while the popularity of open-source and cloud database license is continuously increasing.
As Jeff Bezos, the founder of AWS, said: "The real battle will be in databases". The cloud was first built from IaaS. Therefore, from virtual machines, storage, and networks, to popular intelligent applications nowadays (such as voice recognition, computer vision, and robotics), all are based on IaaS, and the database is the most critical link to connect IaaS with intelligent application SaaS. From data generation, to data storage, and to data consumption, databases are crucial at every step.
This article summarizes the presentation at the Yunqi Community Director's Course and shares some practical tips and development trends of the Cloud Native technology.
The development of container technologies preludes the cloud-native computing:
First, it is the containerization of applications. Many PaaS platforms such as Heroku/CloudFoundry are built based on the container technologies, significantly simplifying the deployment and maintenance of Web applications. Docker puts forward the container packaging specification—Docker Image and creates Docker Hub for global application distribution and collaboration, greatly facilitating the popularization of container technologies and applying container applications in more business scenarios. Communities such as Open Container Initiative and Containerd further facilitate the standardization and normalization of container technologies.
Then it is the orchestration and scheduling of containers. Developers want to optimize/combine and efficiently schedule underlying resources by using container technologies to improve the system usage, application SLAs and the automation level. Kubernetes stands out from among other competing products due to its excellent openness, high scalability, and active community. Google donating Kubernetes to CNCF accelerates the popularization of Kubernetes.
With the establishment of container and container orchestration standards, Kubernetes/Docker hides the differences between underlying infrastructures and provides excellent portability to support multiple clouds/hybrid clouds. The community has begun to build upper-layer business abstraction based on these features. Let's look at the service governance layer. The year 2018 was the beginning of Service Mesh. Istio is a service governance platform launched by Google/IBM/Lyft. Similar to a network protocol stack between microservices in the cloud-native era, Istio can be used to implement a variety of tasks without the need to process them in the application layer, such as dynamic service invocation routing, flow limits, downgrading, end-to-end tracing, and secure transmission. Based on this, cloud-native frameworks targeting specific fields also emerge quickly, such as the cloud-native Kubeflow platform for machine learning and Knative for serverless workloads. These layered architectures allow developers to focus only on their own business logic instead of the complexity of underlying implementations.
Alibaba Cloud demonstrated its self-developed PolarDB database with up to 1,000,000 QPS to European technical experts for the first time at the International Conference on Data Engineering (ICDE) in April 2018. The release of new database falls in line with Alibaba Cloud's efforts towards establishing cloud-native databases as a standard in the industry.
According to Alibaba Cloud's Database Architect Cai Songlu, with the hardware and software of the new 3DXpoint storage media, NVMe SSD, and RDMA, PolarDB can implement the shared storage among nodes and horizontal scalability in just a few seconds. The database can also support seamless scaling up to 100 TB, 2-minute read-only replica scaling, and 1-minute full backup. In standard scenarios, PolarDB has six times higher performance than MySQL, can provide one million QPS, and 100 TB in one single table. Perhaps the best part of all is that all these features come at a cost ten times lower than that of other commercial databases.
In the recent years, exponential data growth and data diversification have led to diversified database needs. In addition to database performance, users now focus more on how to bring offline service data online, how to make services available to customers in real-time, and how to reduce operations cost, and O&M workloads.
Cai Songlu pointed out that "only Cloud-Native Database can meet these diverse user needs." Alibaba Cloud's innovative cloud-native database concept can provide the HTAP, Server-less, and Intelligence features, and help users implement online real-time analytics, automatic SQL statement optimization, automatic parameter adjustment, and fault analysis. Additionally, users can directly utilize the computing power without having to focus on O&M.
At ICDE held this year, Alibaba Cloud innovatively applied the PAXOS algorithm. Therefore, the new-generation POLARDB database can ensure consistency as well as implement error tolerance of nodes. "Instead of the relationship between 0 and 1, the difference between 99% and 100% should be the core of the CAP theorem. Cloud databases will implement perfect CAP balance, just like the Chinese martial art Tai Chi," Cai Songlu said.
Alibaba Cloud Elastic Compute Service (ECS) provides fast memory and the latest Intel CPUs to help you to power your cloud applications and achieve faster results with low latency.
Simple Application Server is a new generation computing service for stand-alone application scenarios. It provides one-click application deployment and supports all-in-one services such as domain name resolution, website publishing, security, O&M, and application management.
This course aims to help IT companies who want to container their business applications, and cloud computing engineers or enthusiasts who want to learn container technology and Kubernetes. By learning this course, you can fully understand what Kubernetes is, why we need Kubernetes, the basic architecture of Kubernetes, some core concepts and terms of Kubernetes, and how to build a Kubernetes cluster on the Alibaba cloud platform, so as to provide reference for the evaluation, design and implementation of application containerization.
This course is designed to help IT companies who want to containerize business applications, as well as cloud computing engineers and operations & maintenance engineers who want to understand and learn about performance testing and optimizing of containerized applications.
ApsaraVideo for VOD supports various video upload methods, including client upload and server upload. However, it may not provide upload SDKs of your required language, such as Go. In this case, you can upload videos through the OSS native SDK.
Before getting started with this topic, make sure that ApsaraVideo for VOD does not provide the upload SDK of your required language. ApsaraVideo for VOD supports the following SDKs:
This topic describes how to achieve seamless migration of your services to the cloud by using Virtual Private Cloud (VPC). In this topic, the source server refers to your on-premises server, virtual machine (VM), or cloud host.
Your source server is able to access the target VPC. If not, you can use VPN Gateway or the physical connection feature of Express Connect to access the VPC. For more information, see Process of creating a dedicated physical connection or Establish a connection between a VPC and an on-premises data center. You will be charged for using Express Connect and VPN Gateway. For more information, see Billing of physical connections and Pay-As-You-Go.
A workload mover software solution, supports Physical-to-Cloud, Virtual-to-Cloud, and Cloud-to-Cloud. The very first on-line migration solution tightly integrated with Alibaba Cloud.
KUSANAGI is a virtual machine that is built to execute WordPress at high speed. It processes commands in 3 milliseconds and can handle 1000 user requests per second without page caching. High performance and advanced security. WAF, IPS/IDS, Selinux, TSL1.3, Vulnerability scanning tool. WordPress, concrete5, Drupal 7, Drupal 8 support to provision via commands.
DevOps Implementation and Testing Strategy: Best Practices and Tools
2,599 posts | 764 followers
FollowAlibaba Cloud Native - October 27, 2021
Alibaba Cloud Native Community - January 5, 2022
Alibaba Clouder - January 5, 2021
Alibaba Clouder - January 4, 2021
Alibaba Clouder - May 18, 2021
Alibaba Tech - July 2, 2019
2,599 posts | 764 followers
Follow
GPU(Elastic GPU Service)
Powerful parallel computing capabilities based on GPU technology.
Learn More
Server Load Balancer
Respond to sudden traffic spikes and minimize response time with Server Load Balancer
Learn MoreMore Posts by Alibaba Clouder
