Last year, SanDisk® pioneered a new industry category termed by IDC as “Big Data Flash” and we’re thrilled to have announced this morning a new collaboration with IBM to combine our InfiniFlash System with the enterprise-ready IBM Spectrum Scale filesystem. This software-defined solution delivers a highly scalable, high performance unified storage system for both file and object storage with integrated data analytics designed to break new ground for:
- Cognitive, Big Data analytics applications using Data Lakes
- Internet of Things and Internet of Everything applications
- Genome sequencing, healthcare and patient data intelligence applications
- High-performance data analytics in the financial services industry
- Content depots, content repositories, media and streaming applications
- Business application backend for Oracle, SAP and SQL Server
- OpenStack private, hybrid, cloud services for data protection, archiving, file sync and share, tiering to Cleversafe and public clouds
Let me share some more about this unique collaboration.
Applications Change, and So Must Storage
Applications are changing dramatically and as they evolve into agile services their underlying infrastructure has to change as well. Be it social, mobile, or Big Data applications, what unites them is their hunger for storage services that must be high performance, scalable, elastic and agile. At the same time, budgets of IT departments are not growing at the pace of data, and breakthrough economics and efficiencies are a must. As workloads become more demanding at scale, systems based on HDDs cannot deliver the performance, TCA and TCO benefits that modern flash solutions can provide.
Yet the biggest challenge with traditional storage solutions is that they usually consist of captive and siloed architectures. These are systems that cannot scale-out and cater only to a single workload, resulting in hardware infrastructure sprawls. Even many of the software-defined solutions in the market are often developed to fit a particular workload only. This model is increasingly failing not just by adding more cost, but also by creating tremendous administrative and management headaches. It’s an IT concept that cannot scale to the challenges of today.
New IT is especially challenging for cloud service providers that want to keep up with the new and growing demands of the market. Solution stacks targeted for Infrastructure-as-a-Service (IaaS), Platform-as-a-Service (PaaS) and Software-as-a-Service (SaaS) mandate new, consolidated architectures that are high performance and scalable to host micro-services yet agile enough for containerized workloads, orchestration and automation.
The Next Generation! Software-Defined All-Flash Unified Storage
If you read this blog, you area likely familiar with SanDisk’s award-winning InfiniFlash system, a next-generation storage platform offering 5x the density, 50x the performance and 4x the reliability, while consuming 80 percent less power than traditional HDD based arrays—starting at less than $1 per gigabyte (GB) before compression or de-duplication technologies.
IBM Spectrum Scale is a software-defined, distributed, parallel filesystem, for high performance, large-scale workloads on-premises or in the cloud with the ability to automatically tier data based on application needs with integrated data analytics.
By combining these solutions, we bring the best of flash, unified storage and software-defined storage together. This solution is a high performance, POSIX-compliant, multi-protocol (NFS, CIFS, HDFS and object) storage system that can cater to a broad range of workload requirements and leverage compute from an ecosystem of industry leading partners. Furthermore, it enables private, hybrid and public cloud customers to deliver Infrastructure as a Service (IAAS), by starting small and easily scaling to multiple PB’s. Right out of the box, you’ll find best in class $/IOPS/TB, with tremendous footprint savings.
Accelerating at Scale
The InfiniFlash for IBM Spectrum Scale Solution is scalable from 512TB in 3U to multiple PBs using 3U appliances. The building block SKU provided 16 GB/s of read throughput and 7 GB/s of write throughput in a replica configuration with 2xNSD Servers and 2x192TB InfiniFlash units.
You can see some recent data points in Christopher Howard’s recent blog about the extreme performance of InfiniFlash for workload testing combining large amounts of sequential streaming data with fast random data access. There are no hot spots, and read and write throughput and IOPS scale linearly with additional clients. InfiniFlash disaggregated deployment allows independent scaling of compute and storage, along with choice of server and networking vendor. Starter (small), performance-optimized and capacity-optimized Rackscale reference architecture consumption models (bundles) meet customers’ varying workload requirements.
By deploying the InfiniFlash for IBM Spectrum Scale Solution customers can take advantage of accelerating software-defined storage for files and objects, using an entirely flash-based architecture. This solution can optimally handle both sequential and random I/O requirements during ingest, store, analyze, query phases, onboard and analyze larger datasets, optimize resource utilization and use vertical scaling to maximize application performance.
Filling a Storage Gap for New Workloads
The SanDisk and IBM collaboration is filling a market gap with a new and revolutionary approach for existing and new workloads, and fits extremely well the IBM market leading FlashSystem portfolio, focusing on Big Data use cases. Some of all-flash unified storage solution unique value proposition includes:
Unified Storage, Data Lakes and In-place Analytics
- Hadoop Transparency Connector supports shared storage by implementing an HDFS RPC protocol. This enables unique “in-place analytics” combining operational and analytical stores, which is not possible with other SDS solutions.
- Unified multi-protocol support enables Data Lake architectures where data can be ingested using File protocols, analyzed using HDFS protocol and presented using objects; ingest object and analyze using HDFS.
IaaS for Private, Hybrid and Public Cloud
- Full OpenStack ecosystem integration (Cinder driver, Heat, Horizon) based on OpenStack Swift and S3 interfaces (Havana, Kilo and Icehouse).
- Transparent Cloud Tiering for Hybrid clouds to Cleversafe or private clouds
Data Management at Scale
- Information Lifecycle Management (ILM) and Hierarchical Storage Management (HSM), Active File Management functionality
- Supports QoS (Quality of Service) to prioritize data path over control path
Unprecedented Performance and Scale
- Line rate performance and throughput per appliance with scale-out to multiple appliances within and across racks
Data Availability, Disaster Recovery and Security
- Flash-based unified storage enables RPOs and RTOs never before possible for data protection use cases, Async DR
- Data availability (snapshots, replication), integrity (end-to-end checksum) and security (NIST/FIPS-compliant data at rest encryption, Secure Erase)
Breakthrough TCA and TCO with Fab Economics
- Policy-based file-level compression, enhances the storage efficiency and provides unprecedented flash based storage at close to 20 cents/GB.
Data Processing: Store, Analyze, Query
- Store – Scale to PBs with Rackscale architecture, very low Annual Failure Rate (AFR) rate, compared to HDD. Store multi-terabyte for long periods.
- Support high throughput batch and low latency real-time queries
- Analyze in-place data analytics in desired file or object format, support for Apache Ambari
- Support real-time queries with msec latencies.
- Support batch/aggregate queries
Go To Market (GTM) Routes Ensuring Ease-of-Doing-Business
SanDisk has a simple GTM approach for this solution. The InfiniFlash for IBM Spectrum Scale Solution is promoted directly to customers and delivered by select channel partners (SIs/VARs), who already deliver IBM Spectrum Family and SanDisk products. Customers have a wide choice of best of breed solution building blocks with server and networking vendors (IP, Infiniband) with worldwide support and services leveraging TSAnet and SanDisk FlashstartTM—for a single SKU.
Make sure to reach out to our team if you are interested in joining the SanDisk Technology Partner Program.
Do More in New Ways
IT teams are continuously challenged to do more, in new ways. Bimodal and data-driven IT is propelling traditional, second platform and cloud-native third platform applications, as IT teams are increasingly challenged to meet these new requirements. There’s a need for delivering storage services in entirely new ways requiring scale and agility that traditional storage infrastructure struggle to deliver.
SanDisk and IBM deliver a unique class of next-generation software-defined all-flash storage solutions for the data center, leveraging SanDisk’s high-capacity and extreme-performance InfiniFlash System and the software-defined Spectrum Scale filesystem from IBM. With innovation and breakthrough technology, we’re addressing the escalating datacenter challenges of scale, performance, agility and economics to enable new services and experiences.
Join our Webinar with IBM
SanDisk & IBM: Breaking New Grounds for Scale-Out Storage Live on March 30th at 1:00 pm PT (or after on demand) Watch it here
Shailesh has 20+ years experience in IoT/IoE, Healthcare and Genome Sequencing, FSI and Enterprise Application verticals.