In the single-tenant big data product architecture, the bottom layer is HDFS2, on which resource control platforms such as Hadoop Yarn and MESOS reside. We can implement specific computing models, such as MR, Hive, HBASE, and Spark over the resource control platforms. In this ecosystem, the IaaS platform is generally available to the same tenant. When new business requirements arise, the tenant can apply for a batch of VM clusters on the IaaS platform, and then deploy open-source products on the clusters. This ecosystem encounters following problems from the perspective of isolation:
To address these problems, MaxCompute offers PaaS multi-tenancy capability using a proprietary system architecture.
In the PaaS multi-tenant architecture, MaxCompute runs over the Apsara operating system. It depends on the Apsara Fuxi module to provide unified resource control, the Apsara Pangu module to provide unified storage, and the Apsara Nvwa module to provide consistency service. MaxCompute uses the same computing engine to offer multiple computing models, including SQL, MR, graph computing, PAI, and near real-time.
MaxCompute uses the following methods to implement multi-tenancy:
For details about these three isolation mechanisms offered by MaxCompute, please go to From Single-tenant IaaS to Multi-tenant PaaS - Multi-tenant Isolation with MaxCompute.
Forrester names Alibaba Cloud MaxCompute as one of the world's leading cloud-based data warehouse in the "Cloud Data Warehouse, Q1 2018" report.
Cloud-based big data services have been in high demand in recent years due to the advantages of security, elastic scalability, rapid deployment, and low costs. Conversely, locally deployed big data analytic solutions are gradually becoming obsolete.
MaxCompute's exabyte-level performance and processing make it the global leader in the field. In October 2017, MaxCompute completed the world's first public cloud-based 100 TB BigBench big data benchmark test, achieving a performance in excess of 7830 QPM.
MaxCompute introduces multi-tenant cloud security isolation technology that upends the security limitations of traditional big data platforms. This technology refines security boundaries to the user, process, and CPU core levels. MaxCompute authorizes and audits millions of tenants and the tens of billions of tasks they perform each day to ensure financial-grade data security.
This article covers a presentation by senior engineer Xiang Li at KubeCon on how Alibaba Cloud has developed and implemented cloud native technologies on a large scale.
Since 2011, Alibaba had begun to put the cloud native technology system into practice by leveraging containers. Alibaba, trailblazing in this industry, over time developed a containerized infrastructure architecture that is now today top-of-the-line among the global leading technology companies. This architecture is now the technological backbone of the entire Alibaba Group. Alibaba believes exploration is intrinsic to developing and discovering new technologies. Through much determination and exploration, Alibaba's technical team has revolutionized many of the ways that technology is used today, becoming a leader in developing cloud native technologies in China.
Multi-tenant management of Kubernetes is another key technical issue for Alibaba Cloud. Considering the limits of Namespaces such as poor scalability and naming conflicts, you can use Kubernetes to set up virtual clusters. In addition to high scalability, Kubernetes can implement strong API-layer isolation. Syncer is used to link virtual clusters and real clusters, and agents are added to nodes to improve multi-tenant management and resource utilization.
MaxCompute is a multi-tenant data processing platform. Distinct tenants have distinct data security requirements. Therefore, MaxCompute provides project-level security configurations to comply with the unique requirements of individual tenants. Project owners can customize their external account support and authentication models.
This topic describes the security model of MaxCompute and that of DataWorks. The security model of MaxCompute can be used by MaxCompute project owners and security administrators for better overall O&M and regular security operations. To ensure better data security, we recommend that you read about the security model before you configure any security functions on Alibaba Cloud.
MaxCompute (previously known as ODPS) is a general purpose, fully managed, multi-tenant data processing platform for large-scale data warehousing. MaxCompute supports various data importing solutions and distributed computing models, enabling users to effectively query massive datasets, reduce production costs, and ensure data security.
Application Configuration Management (ACM) allows you to centralize the management of application configurations. This makes for more convenient management of configurations and enhances service capabilities for such scenarios as microservices, DevOps, and big data.
In addition to isolation policies provided by Spring Cloud, ACM supports multi-tenant, app, data_id, group and other multi-level isolation policies
2,599 posts | 764 followers
FollowAlibaba Clouder - July 26, 2019
Alibaba Clouder - March 30, 2021
Alibaba Cloud MaxCompute - October 31, 2022
Alibaba Clouder - July 11, 2018
Alibaba Cloud MaxCompute - June 23, 2022
Alibaba Cloud MaxCompute - March 7, 2024
2,599 posts | 764 followers
Follow
Container Service for Kubernetes
Alibaba Cloud Container Service for Kubernetes is a fully managed cloud container management service that supports native Kubernetes and integrates with other Alibaba Cloud products.
Learn MoreMore Posts by Alibaba Clouder
