For the last couple of years, Western Digital and 451 Research have taken a snapshot of the industry’s data challenges, by collecting the experience and opinions of real IT practitioners. This year we once again reached out to survey IT professionals to learn about current unstructured data trends and challenges, and to gain insight into the modern enterprise.

The results from the 2018 survey of 200 enterprise customers are now available, and there are a few surprises.

Trend 1: Compliance is shaking up the enterprise.

The recent privacy initiatives have forced organizations to change how they think about data. With a strict new set of rules around data sovereignty and private information, the impacts on data management have been far reaching. Compliance has been this year’s biggest unstructured data trend. We can expect to see contextual metadata, as implemented in object storage, serving as a new set of tools for data identification and the management of privacy controls.

Trend 2: Unstructured data is becoming increasingly difficult to manage.

The scope of the problem is increasingly bubbling up as a concern for IT managers.  Looking a little deeper into the survey data it appears that there are multiple aspects for aggravation. Finding the skilled people to manage the growth of data is the biggest challenge across the board; even more pressing than hiring data scientists that can gain useful and actionable insight from it. We can project that this would mean that better tools need to be in place to support this data without runaway headcount growth (and if economics are an issue, the public cloud won’t be the right answer either).

Trend 3: Data lives forever.

Whether for compliance reasons or analytics, people seem to be keeping data longer. The business objective is to get maximum value from your data where you may not initially know all the potential uses of that data. To protect yourself and this perishable resource, the survey suggests many will extend the data shelf life, perhaps as an insurance policy against future useful life of the data.

Trend 4: Traditional file systems are choking with extremely large and/or dense data sets.

They just weren’t designed for petabyte-scale workloads.  This addresses a common problem of the rapid growth of unstructured data where it is being put on an existing NAS system which becomes overwhelmed.  It may also end up on a tier 1 NAS whose performance may be compromised by the sheer volume of unstructured data and a large number of hierarchical structures.

It is interesting to note that the survey report states that the only way to support exabytes of storage in a worldwide model is object storage.  Indeed, the underpinnings of the public cloud are object storage- for several good reasons: economics, scale, management efficiency, metadata flexibility and erasure coding to fix the limitations of RAID. As people keep more data longer, object storage can take a bigger portion of data to address the shortcomings of legacy storage at massive scale and across global locations.

Trend 5: New workloads find a home on object storage.

Obviously, object storage is still very good for archiving. As systems have matured, some object stores have added more performance (check out some interesting testing results in this report). Today you can find some secondary (non-production) workloads that will work just fine on object storage to take advantage of its cost, management and scalability benefits. We’re seeing interesting use cases in media and entertainment, as well as new workloads such as analytics and data lakes finding new homes on object storage.

It will come as no surprise that Western Digital has sponsored the 451 Research survey because we also see the increasing unstructured data trend of object storage adoption.  Our object storage offering, ActiveScale™, has been around several years and has been continually improved to address the changing nature of data in today’s world.  ActiveScale has a number of advanced features that make it a great choice for unstructured data and diverse workloads and applications.

Data is the lifeline of companies in the digital era. The unstructured data trends we’re seeing represent the critical changes organizations are making in the data value chain through visibility, control and automation. Object storage is a key component of the next generation of data infrastructure. A future where unstructured data is contextualized and monetized.

Learn More

Find out more about ActiveScale.

Get a free copy of the 451 Research object storage survey.


Forward-Looking Statements

Certain blog and other posts on this website may contain forward-looking statements, including statements relating to expectations for our product portfolio, the market for our products, product development efforts, and the capacities, capabilities and applications of our products. These forward-looking statements are subject to risks and uncertainties that could cause actual results to differ materially from those expressed in the forward-looking statements, including development challenges or delays, supply chain and logistics issues, changes in markets, demand, global economic conditions and other risks and uncertainties listed in Western Digital Corporation’s most recent quarterly and annual reports filed with the Securities and Exchange Commission, to which your attention is directed. Readers are cautioned not to place undue reliance on these forward-looking statements and we undertake no obligation to update these forward-looking statements to reflect subsequent events or circumstances.

Erik is the Senior Director, Product Marketing of Western Digital's Data Center Systems, with 25+ years of experience in high tech storage.