The Lunar New Year is arriving on February 16th and according to Chinese zodiac it will be the year of the Earth Dog. According to Chinese horoscope, this year will be a year of action; good for change and improvement although it is likely you will encounter some obstacles before your plans can be completed. They say you will need to use your wisdom to remove roadblocks first, then you will see a wide-open road.
What Does All This Have to do with Digital Transformation?
Many enterprises that start down the path to digital transformation find that their paths are blocked. The problem many discover is the inability to access their data. Data is often spread out and locked in isolated storage islands, making it difficult to extract and use. Another issue is that since many of these islands were built for specific purposes (not necessarily for sharing), they contain lots of data that is duplicated, obsolete or no longer needed.
Many data scientists agree that the bulk of the work involved in gaining analytical insight from data is related to acquiring the data and preparing it for analysis (our own chief data scientist, Janet George, shared her insight to this very problem in a previous blog post). It is similar to when you paint something. Most of your effort will be on preparing the surface. The actual painting is the easy part and doesn’t take much time if the preparation is done correctly. The same is true for the concept of a data lake. You can’t just dump a bunch of data into it and expect to get good results out. The data needs to be properly collected, formatted and tagged with metadata before analysis or you will end up with an unusable data swamp instead of a data lake.
Building a Foundation for Your Data with Object Storage
Object storage solutions are a perfect foundation for a data lake as you venture down your digital transformation journey. Object storage systems can store massive amounts of unstructured data (trillions of objects), seamlessly scale over time as the amount of data grows, are able to consolidate content from multiple structured and unstructured data silos and provide the metadata capabilities to allow intelligent analysis through association.
There are several key reasons object storage is being adopted for data lake projects. The first is metadata capabilities. Object storage is the favored storage repository for most of the software applications that are used to automatically extract, classify, enrich and categorize all of an organization’s data. These intelligent applications leverage the metadata capabilities to deliver better results to their clients.
A hidden (but important) benefit of deploying object storage is cost savings. Moving duplicate, obsolete and stale data to low cost object storage frees up space on more expensive primary storage and often delays (or eliminates) the need to purchase additional primary storage systems. Object storage can also replace antiquated magnetic tape systems and their burdensome opex costs associated with handling, transporting and storing hundreds of physical tapes
Make This a Good Year
Although the Chinese horoscope predicts that this year of the Earth Dog will be a good year for change, it could also be quite a challenging year unless you properly prepare before you begin “painting”. If you start with a powerful object storage platform as the backbone of your data lake initiative, you will have a better chance of avoiding the swamps and having a good year.
I wish you all the best of luck with your Digital Transformation efforts in this year of the Earth Dog!
Also known as “The Data Storage Dude”, Victor is a storage industry veteran with expertise in enterprise backup applications, deduplication, traditional and cloud storage infrastructures and hyper-converged solutions.As a former IT professional himself, he understands how organizations approach IT and brings a unique perspective to helping customers design and deploy cost effective solutions.
Victor holds a master’s degree in Computer Engineering from the University of Southern California. When he is not in the office, you can find him surfing, scuba diving and/or coaching youth sports.