Home > Spotlights

Alibaba Cloud Pangu: a large-scale distributed storage system for infrastructure of digital economy

By Alibaba Cloud Computing Co., Ltd. wuzhenwic.org Updated: 2021-10-20

Computing, storage, and network are the three core components of cloud computing. Pangu, a large-scale distributed storage system, has been one of the most critical kernel components in Alibaba Cloud Apsara Operating System since the very beginning. Over the last decade, Pangu has advanced and evolved in scalability, durability, availability, and versatility: managing exabyte data and trillions of files; achieving high durability and availability; supporting a rich set of workloads of different characteristics. Pangu has been widely deployed as a unified storage platform for Alibaba cloud storage and data analytics services, serving millions of external customers and many internal applications. Continuous innovation in Pangu over the years has helped Alibaba cloud establish a competitive edge over the competition.

阿里云计算公司-主图原始素材_副本.png

Pangu is a large-scale distributed storage platform designed for cloud computing. It features several key technologies, including distributed storage software, high-performance network, hardware architecture, flash storage architecture, accelerator architecture, and intelligent operation and management system. Pangu builds a large-scale distributed and reliable storage system on commodity hardware. To meet the needs of cloud computing, Pangu has redefined the boundary between software and hardware, enhanced data center SSD technical specifications, designed high-speed network protocols, and realized a software/hardware codesign stack. As a result, Pangu has become a unique storage system for cloud computing in the industry, satisfying both high throughput and low latency requirements. The most essential four technical pillars in Pangu are described as follows:

1. Ultra-scale distributed storage software

A distributed algorithm is the key to Pangu. With the help of the distributed computing technology, Pangu orchestrates a large number of commodity computers to offer exabyte-scale, reliable and scalable storage. A single cluster can scale up to hundreds of thousands of machines. Pangu improves data reliability and reduces the cost using erasure coding and many fault tolerance techniques.

2. Extremely low-latency network for storage

A predictable and high-performance network is the cornerstone for distributed storage systems. With the innovation in the fields of network protocol and hardware, which reduces CPU overhead and offers multipath, congestion control, and fast recovery capability, Pangu offers high-performance Cloud storage for the microsecond era.

3. Hardware/software codesign for flash memory storage 

To fully exploit the performance of NAND MaxCompute, an exabyte-scale data storage and processing service based on Pangu won multiple world benchmark records. For years, Alibaba Cloud's market share ranked first in Asia-Pacific and third in the world, serving millions of customers and accelerating their digital transformation. Alibaba Cloud strives to become a digital economy infrastructure, in which Pangu serves as the storage infrastructure. Flash storage media and improve service quality, we jointly proposed the NVMe ZNS technical standard (NVMe 2.0) with industry collaborators. Pangu is currently a leading distributed storage system in the industry featuring deep hardware and software codesign.

4. Intelligent and automated operation and management 

Operating and managing a large-scale cluster is critical to storage durability, availability, and service quality. Pangu has applied AI techniques to improve service quality and reduce operating costs at a large scale.

阿里云计算公司-配图2_副本.png

As a common storage platform, Pangu has been widely used in Alibaba Group, supporting all businesses units, including all core e-commerce transaction systems of Alibaba Group, Ant's key business applications, Alibaba Cloud storage services, as well as Alibaba Cloud MaxCompute big data analytics services. ESSD, the industry-leading performance block storage service, offers extremely low latency and one million IOPS, leading cloud storage to the microsecond era. Object Storage Service (OSS) provides the reliability of 99.9999999999% (12 9's) and massive storage capacity.


ORGANIZED BY
Cyberspace Administration of China
People's Government of Zhejiang Province
CO-ORGANIZED BY
United Nations Department of Economic and Social Affairs
International Telecommunication Union
World Intellectual Property Organization
GSMA
HOSTED BY
Secretariat of World Internet Conference (Preparatory Office)
Cyberspace Administration of Zhejiang Province
Economy and Information Technology Department of Zhejiang Province
Tongxiang Municipal People's Government
National Internet Emergency Center
CONTACT US
Tel: 0086-571-85311391(For Conference) 0086-571-85800770-213(For Exhibition)
Fax: 0086-571-85195207
Email: service@wicwuzhen.cn
QQ: 2092919312

Copyright © World Internet Conference. All rights Reserved
Presented by China Daily. 京ICP备13028878号-23

Copyright © World Internet Conference. All rights Reserved Presented by China Daily. 京ICP备13028878号-23