By Walt Hinton, Pavilion Data Systems
Supercomputing 2019 will be held in my back yard of Denver, Colorado, from November 17 to 22. Western Digital and Pavilion Data Systems will be working diligently behind the scenes to support the network and storage backbone of leading High Performance Computing organizations across the globe. This network, volunteer-built each year for the conference, is called SCinet’19.
Spearheaded by Caltech Professor Harvey Newman, SCinet’19 showcases how people and organizations gain insights and knowledge in gene sequencing, social and economic analysis, and other big data applications.
Known for continually pushing the envelope of network and storage physics, Mr. Newman’s abstract for SCinet’19 is aptly entitled, “Network Research Exhibition: Global Exascale Workflows for Data-Intensive Science Accelerated by Next Generation Programmable SDN Architectures and Machine Learning Applications.”
It sounds like groundbreaking stuff. And it is.
SCinet’19 features the latest advances in Terabit/sec networks, intelligent global operations and monitoring systems, workflow optimization methodologies with real-time analytics, and state of the art long-distance data transfer methods, with tools and storage designs to meet the challenges faced by leading-edge programs in high energy physics, astrophysics, climate and other fields of data-intensive science.
This network is the backbone for collaboration within global scientific enterprises, each encompassing hundreds to thousands of scientists.
Translation for data storage folks: Western Digital and Pavilion Data Systems are providing the high-performance storage infrastructure with NVMe™ and NVMe-over-Fabrics storage for the network and its participants.
Meeting Extreme Requirements
Pavilion’s Hyper Parallel Flash Array (HFA) was exclusively selected by Professor Newman and team to meet the extreme IOPs and bandwidth requirements of NVMe-oF™ for SCiNet’19. Featuring OpenChoice Storage™, the Pavilion HFA relies on the industry’s proven workhorse, the Western Digital® Ultrastar DC SN200 NVMe SSDs. Seventy-two of them to be precise.
Maybe I’m partial for being part of the Western Digital team that brought the SN200 to market, but when Mr. Newman approached Pavilion about our HFA, I reached out to Western Digital immediately.
The extreme duty cycles of SCinet’19 traffic, combined with very high bandwidth demands and quality of service requirements, made the SN200 an obvious choice.
One of the things to keep a close eye on is the use of SCinet’19 for dynamic regional caches or “data lakes” that unify global data. Building on the successful petabyte cache currently in operation between Caltech and UCSD, this cache concept will extend to more remote sites such as KISTI in Korea, and TIFR (Mumbai) with distributed GPU clusters for machine learning. Blending these data lakes with real-time joint-sky-survey analysis data services will help define the future workflows, software systems, and methods to, well, make the impossible possible.
Partners and contributors to SCinet’19 include physicists, network scientists and engineers from Caltech, Pacific Research Platform, Fermilab, FIU, UNESP, Yale, Tongji, UCSD, UMaryland, LBL/NERSC, Argonne, KISTI, Michigan, USC, Northeastern, Colorado State, UCLA, TIFR (Mumbai), SCinet, ESNet, Internet2, StarLight, ICAIR/ Northwestern, CENIC, Pacific Wave, Pacific Northwest GigaPop, AmLight, ANSP, RNP, REUNA, NetherLight, SURF, and their science and network partner teams, with support from Ciena, Intel, Dell, 2CRSI, Premio/Echostreams, Arista, Mellanox, Color Chip, Pavilion, and Western Digital.
It is an honor to be part of SCinet’19 with Caltech and Western Digital. If you are coming to Supercomputing 2019, be sure to check out Western Digital in booth #1275.
