Device-Managed Shingled Magnetic Recording (DMSMR)

Device-Managed Shingled Magnetic Recording (DMSMR)

Device-Managed Shingled Magnetic Recording (DMSMR) technology is different from Conventional Magnetic Recording (CMR), and can be implemented in various ways. I wanted to share an overview of our DMSMR architecture, and how we apply specific capabilities and configurations.

Logical Block Address Indirection

In CMR, each track is written separately from its adjacent neighboring tracks. In this approach, there is minimum interaction between tracks. Data sectors can be written, and re-written repeatedly. Furthermore, the LBA (Logical Block Address) location is absolute and immovable, after format.

SMR architecture and CMR architecture

Conversely, in SMR drives, data tracks are laid down by overlapping the previous track (thus the name “Shingle”). In this fashion, tracks are packed much closer together, helping to increase areal density. However, using this architecture, data from the “down shingle” track will be erased when writing, thus making laying down new data to a previously written location impossible without destroying old data from the down shingle track. To accomplish the new write, the entire data segment (SMR Zone) needs to be re-written in order to preserve other data.

SMR

This is where things get interesting. In order to re-write the data, all of the data within the same data segment (SMR zone) is relocated elsewhere. In doing so, the absolute address of the data is remapped. This type of movement is called LBA Indirection. It governs how the device works with this underlying recording technology and how it can be tuned to be advantageous for different applications and workloads by orchestrating where, how, and when the data is to be placed.

Dynamic Data and Zones

Our approach to DMSMR is dynamic. We designate various SMR zone sizes for different purposes and applications. Using machine learning and pattern recognition in our system, the drive detects types of data so that we can coalesce similar data types together to maximize performance. For example, logs and metadata for indexing or journaling of small block writes will be placed somewhere more advantageous for garbage collection, while large block transfers will be placed into large zones.

Another example is designating large number of small zones for pure random I/O, as in the case of the NAS applications. Here we can either buffer the data for collation for more efficient flushing into final destination zones, or we can choose to leave the data there for permanent storage, thereby cutting down the need for more background activity.

The configuration of large and/or small shingle zones gives us the ability to keep data movement as a part of background activities, and only when necessary. This is only made possible by the LBA indirection nature of SMR architecture. The number of configurations is vast, the flexibility is greater still.

New Benefits That Emerged

This way of dealing with intelligent data placement has brought about a few interesting benefits:

Data integrity — Because DMSMR drives put data down sequentially, track ECC (accumulation of parity information on a per-track basis) can be placed at the end of each data track. It enables the drive to correct up to 8K bytes of data per track during read, with near on-the-fly performance. This exceptions-handling capability is especially beneficial in event of shock, fan vibration, grown defects or other less-than-ideal operating environments. This development of data protection power during read comes naturally in a SMR architecture.

Furthermore, there are techniques that SMR drives can deploy when running into command faults during write, in the same non-ideal operating environment: It can either place the data with larger track spacing when the ability to track-follow is less than perfect, or by swiftly rectifying the corrupted data due to off-track write and migrate them elsewhere. It can also seamlessly perform write abort recovery by rewriting the same data down on different tracks.

Throughput – The property of SMR architecture also allows us to coalesce the data in cache more freely, mainly due to, again, LBA Indirection. Performance can be good when a burst of random writes is sequentialized into a local zone. Nevertheless, this burst of performance does come at a cost: data needs to be migrated to improve near-sequential read performance. However, with appropriate amount of idle time, such as in a typical NAS environment, this benefit can be easily harvested in the right workload.

DMSMR Considerations

Data movement requires idle time — One of the most discussed about topics in DMSMR drives, is data movement. This is, by LBA Indirection nature, the other side of the same coin. To be able to move the data freely requires the drive to have idle time to perform these tasks in the background. Without the ability to preemptively maintain disk space, the drive, under specific conditions, may take longer to complete a command as its resources dwindle.

A simple analogy: Imagine the drive as a warehouse that organizes all your storage for you. As more cartons of various sizes begin to pile in, we need more time to reorganize them. The more we delay the work, the less open space we have to move it around and tidy it up. If we don’t allow for this time, the disorganization will lead to crammed space, low efficiency, and poor response time to locate the right carton.

Specified workloads — Our drives are built to work in specific environments. The firmware is tailored for a specific application use case and as such is designed with extremely varying zone sizes, buffering and flush policies of how data is committed.

We collect a lot of field data to deeply understand workloads, data usage, idle time, read/write ratio and other characterization so we can design firmware that is highly-optimized for specific segments (e.g. personal computing, NAS, etc.). As our drives are optimized for specific purposes, performance may be impacted if used for a purpose for which it was not designed.

A Paradigm Shift

DMSMR technology is relatively new, and it’s evolving as we continue to advance performance and other drive capabilities. While this type of purpose-built design is exciting in its flexibility, it naturally comes with a lot of questions. We plan to regularly provide technology insight and use case guidance.

14 comments on “Device-Managed Shingled Magnetic Recording (DMSMR)

  1. Hauke Lasinger on

    A NAS means “raid” (most of the time, anyway), and “raid” means “rebuild”, that is, writing the whole disk from start to end in one go.

    I’d be interested in reading what’s the reasoning behind labelling a smr disk “NAS”, given that it, by it’s nature, cannot do this.

  2. Tim Reuscher on

    Right, I think the technology itself sounds like it has promise in certain applications, particularly where performance isn’t critical and cost savings are more appropriate. The issue is putting it in NAS drives, where proven performance and reliability are non-negotiable.

  3. Wong on

    There is so much secondary, PR-related information here, but nothing that addresses the biggest issues the users are talking about now all over, mainly issues on resilvering or compatibility issues with other CMR drives. Face the music and disclose what people really want to know.

  4. Maltz on

    This is getting absurd. The issue isn’t that we don’t understand SMR or DMSMR. It’s that we DO understand it, and that WD has added it to drive models that didn’t have it before, in lines where it is COMPLETELY inappropriate. Reds (and even a Black!) shouldn’t have any form of SMR. Reds are designed for RAID, and rebuilding a RAID array with an SMR drive in it would be madness. And the performance-oriented Black drive being SMR doesn’t make any sense at all.

    The purpose of this is to reduce the number of platters and cut manufacturing costs while passing the savings on to… well, no one, since they didn’t reduce the price of the SMR drives.

  5. Jeff on

    The funny thing is – you would lose a LOT less customers if you simply owned up to your (intentional) mistake, apologized, and tried to make it right.

    What’s so hard about labeling a drive with exactly what’s inside of it? You were selling NAS rated drives with garbage tech inside. Omitting the truth is the same thing as lying. When people pay more money for NAS rated drives, they have a certain expectation that you failed to meet.

    You don’t realize it at the moment but you will be taking a HUGE hit on this from people like myself who, in the past, only considered WD products. Now? I’ll buy anything *but* WD.

    Instead of writing fluff tech pieces like this to try and smooth things over, maybe try owning your mistake?

  6. Johnny Henson on

    I’m seriously inclined NOT to believe the supposed ‘benefits’ of SMR….because Western Digital went to lengths to NOT publicize the practice. No company HIDES a true improvement in their product.

  7. Fexto on

    Can’t believe you’re doubling down on this instead of just apologizing and removing SMR from NAS lines.

  8. Jonasz on

    Are you planning FW update to SMR RED drives for make them back CMR drives? How much space will be lost if SMR drive will be flashed with CMR FW? Or the only option is to return the drive back to you? (Non-compliance – NAS drive is not SMR drive!)

  9. nontasif on

    This is a big blame from WD side! When I ordered my new drive, I ordered a drive named “Red NAS drive”. Under the hood is something different!
    The feeling is awful. I never expected such a “game” from WD. Give us all the info about your products and let us decide what technologies/characteristics we want to pay for! Don’t deceive us! We have already lost so much time because of you. You should announce a replace of all these drives without cost!

  10. cspot78 on

    I was about to update my nas to 4x8tb drives, but with all the talk about SMR and how it was handled….I’m not sure which vendor is safe to go with…clearly not WD.

  11. Paul on

    I was proud to populate my new NAS with (2) WD 6tb Reds

    After a lot of research, their reputation made them sound like the best option. While I didn’t go Pro’s, I also didn’t go cheaper or smaller solutions

    Now, they are kind of embarrassing. It’s embarrassing to have the wrong drives in a NAS, even when you buy NAS drives

    Wish I had gone Iron Wolf now.

  12. Marco Lopes on

    Someone said this is part of a RACE for BIGGER capacities. It can be… BUT, before that happens, WD is probably using the most demanding customers / environments to TEST SMR tech so they can DEPLOY them in the bigger capacity DRIVES: 8, 10, 12, 14TB and beyond (do not currently exist). I say this because, WD has the same “infected SMR drives” using the well known PMR tech! https://documents.westerndigital.com/content/dam/doc-library/en_us/assets/public/western-digital/product/internal-drives/wd-red-hdd/data-sheet-western-digital-wd-red-hdd-2879-800002.pdf

    Why is that? Why keep SMR and PMR drives with the SAME capacity in the same line and HIDING this info from customers? So they can target “specific” markets with the SMR drives? It seems like a marketing TEST!!! How BIG is it?

    Note that currently, the MAX capacity drive using SMR is the 6TB WD60EFAX, with 3 platters / 6 heads… So… is that it?? Is WD USING RAID / more demanding users as “guinea pigs” to test SMR and then move on and use SMR on +14TB drives (that currently use HELIUM inside to bypass the theoretical limitation of 6 platters / 12 heads)??? Is that the next step? And after that, plague all the other lines (like the BLUE one, that already has 2 drives with SMR). I’m thinking YES!! And this is VERY BAD NEWS. I don’t want a mechanical disk that overlaps tracks and has to write adjacent tracks just to write a specific track!!!

    Customers MUST be informed of this new tech, even those using EXTERNAL SINGLE DRIVES ENCLOSURES!!! I have many WD external drives, and i DON’T WANT any drive with SMR!!! Period!

    Gladly, i checked my WD ELEMENTS drives, a NONE of the internal drives is PLAGUED by SMR! (BTW, if you ask WD how to know the DRIVE MODEL inside an external WD enclosure, they will tell you it’s impossible!!! WTF is that??? WD technicians don’t have a way to query the drive and ask for the model number?? Well, i got new for you: crystaldiskinfo CAN!!! How about that? Stupid WD support… )

    So, if anyone needs to know WHAT INTERNAL DRIVE MODEL they have in their WD EXTERNAL ENCLOSURES, install https://crystalmark.info/en/software/crystaldiskinfo and COPY PAST the info to the clipboard! (EDIT -> COPY or CTRL-C). Paste it to a text editor, and voila!!!

    (1) WDC WD20EARX-00PASB0 : 2000,3 GB [1/0/0, sa1] – wd
    (2) WDC WD40EFRX-68N32N0 : 4000,7 GB [2/0/0, sa1] – wd

    Compare this with the “INFECTED” SMR drive list, and you’re good to go!

    P.S. I will NEVER buy another EXTERNAL WD drive again without the warranty to check the internal drive MODEL first!!!! That’s for sure!

  13. Luigi on

    New WD RED EFAX with SMR are orrible (to evitate).
    Hight noise when they aren’t working (I don’t konw what they do in that time) .
    We used many red efrx with no problem in the years.
    Now we have 2 new wd30FAX, we can’t use them in raid and they disturb our heads with their terrible noise too.
    WD have to do something, if they don’t want lose storical and new users…

  14. J. Zhu on

    If you think submarining DMSMR w/o full disclosure is bad, just wait until they ship ePMR.

    We know ePMR is not MAMR or HAMR. But that’s all we know. Which is confusing, since WD said MAMR would be out last year.

    Come on WD, what’s in your new top of the line drives? Where’s the data? Where’s analysis on performance and reliability?

Comments are closed.