Storage Class Memory (SCM) has been proposed as a cost-effective enabling technology for large-scale in-memory compute problems. However, to enable such systems at massive scale, we need some mechanism to ensure reliability and consistency of very large pools of distributed memory, even while ensuring low-latency access to remote memory across a network fabric.
I’m proud to announce that today Western Digital Research will help present a breakthrough solution to this problem at the P4 European Workshop (P4EU) in Cambridge, UK. We will demonstrate an in-network high-performance fault tolerant solution for distributed in-memory compute, at latencies that are orders-of-magnitude lower than any currently existing solution.
High Performance Memory Consistency at Scale
SCM is a new class of emerging memory technology that promises performance close to DRAM, but with higher die capacities and lower cost. This new class of memory will enable in-memory compute problems such as in-memory databases or real-time analytics to be solved at a scale which has not previously been economical. A well-known consequence of designing systems at scale is the emergence of new failure mechanisms (e.g. system failure, memory errors) which are less significant at smaller scales. Well established fault-tolerant mechanisms exist to deal with these problems, but traditional implementations impose a severe performance penalty.
This solution implements fault tolerance inside of the network switch, using the emerging P4 network programming language, resulting in many orders of magnitude improvement in performance relative to existing solutions. The rich and open P4 ecosystem empowers network owners, operators and app developers to have greater control of the data plane within the network, enabling a new class of distributed system solutions at performance levels well beyond what is conceivable today.
Presenting a New Approach
The live demonstration at P4EU is a P4 program implementation developed by Western Digital and the Università della Svizzera italiana (USI) using a high-performance Barefoot Tofino processor powering a BF6064X switch from STORDIS. USI and Western Digital use the rich, open and empowering P4 programming language to implement a consensus protocol within the Tofino network switch chip, allowing us to retain multiple copies of remote, non-volatile memory and to manage the consistency protocol within the switch itself. The system treats the SCM-based main memory in each server as a distributed storage system and implements data replication across systems, along with a consensus protocol to keep the replicas consistent.
Early implementations such as the one we are presenting at P4EU already operate at time scales that are faster than traditional replicated storage. We expect an explosion of new system design approaches based on the power of in-network computing, and anticipate that future SCM-based distributed systems will leverage new protocols in the network to enable unprecedented scalability and performance.
The P4EU project team consisted of Dejan Vučinić, Director, R&D Engineering, Non-Volatile Memory Systems Architecture Group at Western Digital, Huynh Tu Dang, PhD Candidate at Università della Svizzera italiana in Lugano, Switzerland, and Professors Fernando Pedone and Robert Soulé, both from the Systems Institute of the Università della Svizzera italiana.
To learn more, please read the Consensus for Non-Volatile Main Memory research paper, co-written by Yang Liu, Marjan Radi and Dejan Vucinic of Western Digital, and Jaco Hoffman of Technischen Universität Darmstadt in Darmstadt, Germany. This work was performed at Western Digital Research in the Next Generation Platform Technologies department led by Zvonimir Bandić.
Stay tuned for further developments and news from Western Digital research @WesternDigiCTO on Twitter. We will have much more to share in the coming months.
Richard New is currently Vice President of Research at Western Digital. Richard grew up in Cambridge, England, and Waterloo, Canada, and received a BS in Electrical Engineering from the University of Waterloo, and a PhD in Electrical Engineering from Stanford University. Richard joined IBM Research in 1995 where he worked as a research staff member and manager in the areas of magnetic recording, signal processing and error correction. In 2007 Richard took on the role of Director of the Hitachi GST San Jose Research Center, working on advanced magnetic recording technologies such as heat-assisted magnetic recording, as well as advanced storage architecture concepts. In 2016 Richard became Vice President of Research at Western Digital, in charge of the Western Digital Research Lab, focusing on exploratory research and advanced technology development in emerging non-volatile memories, microprocessor architecture (RISC-V), storage and memory interface standards, advanced storage architecture concepts and prototyping, and open source software projects relating to storage and memory architecture. Richard currently resides in Palo Alto with his family, and enjoys working on robotics and go-carts with his kids.