brain monitoring - dark data in healthcare
Industries & Applications

What Dark Data Can Teach Us About Healthcare

From the advent of IoT wearables to fitness tracking devices, to the introduction of high-resolution imaging and on-demand digital patient information – healthcare has been undergoing dramatic digital disruption with a staggering growth and diversity of data. With the recent release of the whitepaper “Value of Data: Embracing Dark Data” we interviewed two of our in-house experts on the role of dark data in healthcare and the promise of AI and hyper-personalization.

charlene wan linda zhou
Charlene Wan, Marketing Programs Direction and Linda Zhou, Director of Research and Life Sciences Solutions

What is the opportunity for dark data in healthcare?

LindaWhen you look at research in life sciences, there’s a colossal mountain of dark data across research institutions, service providers and labs. You have massive data results from surveys, papers, research and other projects, but they are archived without contextual information that can help interpret it. What was the methodology used? Why was a particular demographic surveyed? What question was a particular researcher trying to solve?

Without knowing how data was collected and processed, it’s simply useless bits taking up the storage capacity.

This type of data is what we call dark data. It’s valuable data that currently sits dormant, and it’s so massive that we’ll need machine learning to mine and categorize it. Data management like this is one of the key machine learning trends and challenges on its path to realization. Ultimately, the goal is to be able to use dark data in various ways to find patterns, correlations and other ways to use data infrastructure to extract value from it.

Charlene: Dark data is dormant data in archives, and in healthcare there is a lot of it! With HIPPA regulations healthcare data is stored and protected, for years, which can exponentially increase the amount of dark data that will only take up storage space. As sensors become more compact and diverse, more data will be collected from patients, doctors, clinics and hospitals. This includes everyday information as granular and mundane as patients’ heartrate, glucose levels, blood pressure, perspiration, movement, or diet. It can also include clinical research carried out by academic as well as medical foundations on bacteria and particles, other case studies, or high-drama ER treatments, operations, surgeries, medications and unpredicted virus outbreaks.

Imagine if we could pool this vast amount of data and apply artificial intelligence (AI), which can leverage the power of convolutional neural networks to detect, sort, classify and identify logics and patterns. It could quickly and effectively turn dark data into insights for inference and predictions.

What is the role of IoT devices in tapping into that opportunity?

Charlene: IoT devices at the edge are changing patients’ healthcare experience, whether it’s a mobile device collecting patient information at an emergency room (ER) visit or a diabetic’s on-body continuous glucose monitoring system.

We’re starting to see all kinds of IoT devices collecting health data from individuals: from heart and blood pressure monitors to “smart” pills designed with a time-release or electroceuticals that monitor intake[1] to wearables like smart socks with temperature control or smart vests for vital sign monitoring. In addition to monitors, there are also all kinds of devices designed to remind patients to correct their posture, take their medicine, or serve as personal assistants on-the-go.

IoT devices will continue to play a critical role in tapping into the opportunity to turn dark data into something bright and useful, if not life-changing.

Linda: IoT can open a new opportunity for understanding medical care in a more holistic manner. If you think about ER and emergency situations, very often all the data collected about vital signs etc. is not stored or kept. The goal is to ensure the patient is stabilized, and then they are moved to the next station. With the proliferation of wearables, suddenly we have data tracking the patient throughout the medical emergency – from the time leading up to the event and through the recovery. Collecting and mining this data can teach us what are preliminary warning signs for heart attacks, strokes and much more.

What healthcare data should be stored?

Linda: The cost of storing data has dropped immensely particularly with object storage in the cloud or on-premises. If you make data alive and assist doctors to quickly find a cure, the benefits of keeping data is better than saving pennies for a rainy day.  You shouldn’t throw out any data, ever! Moreover, if more data can lead to faster time to find a cure for a disease, you can save or improve the lives of millions of people. How do you put a price on that?

The lifetime of data has changed. Your data infrastructure should facilitate a data-forever architecture.

Can you put a price on saving, or improving, millions of lives?

CharleneOne of the biggest areas where dark data can be turned to light is medical imaging and the bioinformatics workflow. Be it X-rays, mammograms, MRI, or any kind of imaging, this unstructured data needs to be stored so that it can be easily accessed, analyzed and transformed for analysis, diagnosis and appropriate treatments if needed. Being able to compare thousands, if not millions of images, within a short timeframe, to find trends, patterns or anomalies is what will lead to the next big medical breakthrough!

What opportunities do you see for hyper-personalization in healthcare?

CharleneIoT is a key enabler of hyper-personalization. The addition of IoT devices worn by patients not only reduces visits to the doctor’s office since they can be monitored remotely, but also opens the door to new opportunities for hyper-personalization. We see it in future autonomous carswhere embedded sensors can monitor passenger’s conditions such as their alertness, stress level, mood change as well as detect and avoid any distractions for a safe and comfortable ride. In healthcare, a patient can wear smart glasses to potentially reduce eye pressure as a preventive care for their eye health. All of this is made possible by ever-evolving IoT devices and purpose-built edge data architectures.

Linda: When it comes to hyper-personalization, I see more organizations and companies getting involved in healthcare that have traditionally not touched that market. Let’s look at airline industry. When I fly to a conference, the airline knows my itinerary, including any layover. The airline can forewarn me about any flu outbreak, so I can get a vaccination or poor air quality due to forest fires, so I can pack a mask along with my asthma medicine.

How will healthcare embrace dark data? Download our whitepaper for a deeper dive into making hyper-personalization a reality.


[1]https://www.telegraph.co.uk/science/2017/11/14/smart-pill-checks-medication-has-taken-approved-regulators/

Leave a Response