Organizations are looking for new, cost-effective mass storage solutions to help them contend with the onslaught of data. Innovations such as helium-sealed drives take advantage of lower internal atmospheric density to enable additional, thinner disks for higher capacities, while Shingled Magnetic Recording (SMR) hard drives overlap data tracks to achieve greater storage density.
The first generation SMR drives were intended as a starting point to help develop an ecosystem that can leverage SMR technology. How can organizations successfully adopt these innovations today? We recently worked with Dropbox to implement host-managed SMR technology and today announced that they have qualified and are the first to deploy Western Digital’s Ultrastar® Hs14 host-managed SMR hard disk drive (HDD) at exabyte scale. To better understand why host-managed SMR is so important, let’s start with a basic introduction to the three flavors of deploying SMR.
Three Flavors of SMR
Essentially, SMR comes in three flavors. It is important to understand their differences as the host software requirements and drive performance characteristics differ.
Drive-Managed SMR
Drive-managed SMR, where the drive manages all write commands from the host, allows a plug-and-play implementation, compatible with any hardware and software. However, the background ‘housekeeping’ tasks that the drive must perform result in highly unpredictable performance, unfit for enterprise workloads.
Host-Managed SMR
In contrast to drive-managed SMR, host-managed SMR is an implementation where the host is responsible for everything ranging from managing data streams, to read/write operations and zone management. Host-managed SMR requires host-software modification so that the host system has knowledge of the underlying media and can micro control all elements by employing a new set of commands.
Depending on the system architecture, implementing these modifications may seem like an onerous task, yet once developers gain SMR familiarity and optimize their applications for sequential writing, they can take advantage of unsurpassed levels of reliability and quality. With the ability to deliver predictable, consistent performance comparable to what users expect from traditional PMR drives, host-managed SMR is emerging as the preferred option for implementing shingled magnetic recording.
Host-Aware SMR
Host-aware SMR is like a superset of the aforementioned options. On the surface this may seem like the best of both worlds. However, if predictability and reliability are what you are after, you cannot take any shortcuts in modifying your stack as you would for host-managed SMR. As such, host-managed SMR allows for a smooth, staged transition to Host-aware SMR in a future timeframe.
SMR Capacity isn’t “Free”
While SMR certainly delivers increased storage capacity using essentially the same hardware components as a PMR drive, utilizing this capacity requires a commitment on the part of the customer to invest in software development both in the file system and the underlying applications. However, this investment can pay dividends long term since an SMR drive provides lower cost per TB and better total cost of ownership (TCO) when considering the capital and operating cost of the data center.
Host-Managed SMR Done Right – Dropbox and the Cloud
We’ve seen more and more cloud customers turning to host-managed SMR. Dropbox is a great example of a cloud company that recognized the opportunity of SMR for their file system. They’ve chosen host-managed SMR implementation, since it gives them total control of how the data is placed on the disk, and how and when the necessary “housekeeping” tasks are managed.
To begin with, the Dropbox workload was a good fit for SMR. They write their data sequentially in “chunks” that align well with the SMR zones. User data is split up among many chunks so the sizes of these writes are independent of the individual file sizes. Once written, most of the file access is random reads, which also fits well with SMR technology. Most data is not deleted, but when they are, the deleted data will free up space in the chunks, which will require re-writing new chunks to better utilize the installed capacity, but because they are using host-managed SMR, they have total control of the process and how system resources will be used for these housekeeping tasks without impacting performance.
Nimble Development – Deploying Technology as an Early Adopter
When Dropbox embarked upon this journey, many of the tools required for implementation were in their nascent stages. The ZBC/ZAC libraries were in early release and went through many fundamental stages. While these open-source tools were effectively leveraged by Dropbox in the development of their SMR implementation, a fast moving, flexible development approach allowed them to stay on the leading edge and realize the advantages of SMR ahead of their competitors.
Increasing SMR Capacity by 40%
SMR and helium-sealed drive technology have a long roadmap ahead. Today, we ship the Ultrastar® Hs14 drive – a combination of host-managed SMR and HelioSeal® technology that delivers 14TB of storage at our lowest TCO, a 40% increase in capacity over the prior generation of SMR helium drives. Dropbox’s initial software investment will allow them to take advantage of that increased capacity.
Host-Managed SMR – A Blueprint for Adoption
As enterprises and cloud companies better understand SMR implementation options, they will be able to adopt it more successfully. SMR technology will be pervasive as businesses contend with the data deluge and make more of their archival data active.
Dropbox is a great example of a company that has a deep understanding of their solution stack and could use agile development to quickly adapt and adopt host-managed SMR technology to achieve significant cost savings, allowing them to remain highly competitive in the marketplace.
Our collaboration is a blueprint of how companies can successfully approach SMR technology; a model of great things to come. The host-managed approach may seem like a greater undertaking but today, it’s the best way to reap the greatest results – get it right the first time!
