This is part three of a three-part series about data storage in the new space age. Design by Aisha Alkinaey.
“NewSpace” is the moniker of the current space age. An era characterized by the privatization and commercialization of space. Where Silicon Valley-like entrepreneurism stormed a sector once limited to government agencies and billion-dollar aerospace firms.
NewSpace is looking for innovative answers to the cosmic challenges of space (see part I of this series). It’s bringing agile development philosophies and the smaller-faster-cheaper force of Moore’s Law to an industry that was bound to extraordinarily complex, expensive, and long-term projects (see part II of this series).
While billionaire-led space tourism, upcoming moon excavations, and the possibility of multiplanetary human existence are taking up most of NewSpace headlines, it’s the proliferation of satellites and new data from space that is already changing life on Earth.
An Earth selfie
Klaudia Bielińska is an Earth observation engineer with a post-graduate in geographic information systems (GIS). Working in the space sector for the last decade, Bielińska has been at the heart of the rapid growth of data from space.
“In 2009, I was doing a project for the Space Research Center of the Polish Academy of Sciences,” said Bielińska. “We could work using data sets from two satellites that were updated only once every 16 days; the data was very narrow,” she explained.
Fast forward a decade, and more than 40 commercial companies have launched thousands of new satellites. Hundreds of imaging satellites can work in unison to map Earth’s landmass every day. Others can bring detailed images from the same location every few hours.
But it’s not just more of the same. Today’s Earth observation satellites collect richer, diverse, and more detailed data. Startups are regularly launching new satellites with higher resolution capabilities. Radar-based satellites gather more diverse multispectral bands (like infrared, ultraviolet light, or microwaves) capable of detecting surface changes as small as a fraction of an inch.
This wealth of data is helping organizations unearth insights (forgive the pun). As Bielińska explained, “for many climate researchers, the more spectral bands they can use to assess things like moisture levels, photosynthetic pigments, and structure, and the more combination of bands they can apply together, the more insight they can extract from those data sets.”
Managing massive data
Today, Bielińska works for CloudFerro, a cloud service provider focused on the European space sector. CloudFerro was founded to address a burning data infrastructure need. All those satellites traveling across our atmosphere are beaming a massive amount of data to Earth.
The company’s online archive of Earth observation data is already a massive 23 petabytes in size (the equivalent of almost 60 years of binge-watching 4K films) and quickly growing. Managing that much data is no small feat. Marcin Kowalski, the senior manager of products at CloudFerro, would know, as he is responsible for infrastructure products.
For Kowalski, keeping up with the rapid growth of space data comes down to scalable software and the right data storage. As he explained, even when deploying thousands of hard drives, with every hard drive added, system complexity grows, as do the costs of networking and maintenance.
“With our biggest clusters exciting 54 petabytes of raw storage, it’s extremely important to keep the number of hard drives in the system as low as possible,” he said. “Using high-capacity hard drives [from Western Digital] are therefore a necessity in our solutions and they help us by lowering complexity and costs while maintaining highest reliability,” he said.
Letting AI crunch Earth data
CloudFerro creates a storage and computing backbone for Earth observation data. Yet the magic happens when organizations figure out what to do with it.
It’s not surprising that NewSpace became a hotbed for AI applications. The use cases are rich.
Some organizations can use satellite data and combine 100 million data points to create profiles of every vessel at sea. Others crunch space data to predict inventory, retail profits, or agricultural yield.
LiveEO is an AI-powered space monitoring startup from Berlin. The company’s young, space enthusiast founders saw the massive growth in satellite data as an underutilized asset. They looked for ways in which the advancement in machine learning could help organizations take advantage of data from space.
The vulnerability of Earth’s infrastructure
LiveEO saw a vital opportunity in the infrastructure sector. The massive network of electric grids, national pipelines, and railway tracks that are the arteries of modern life.
Each of these networks is huge, essential, and vulnerable. A small seismic change in ground elevation can crack a pipe. A single tree can cause massive electrical outages, or worse, spark deadly wildfires.
“Detecting vegetation from space is not hard or new,” explained Andreas Naujoks a spokesperson at LiveEO. “What is difficult is to identify vegetation on a large scale and create accurate risk prognosis,” he said.
Staying on track
LiveEO’s machine learning algorithms use multispectral and stereoscopic satellite imagery (images taken of the same place from two different locations that together generate a 3D elevation map) to identify the location, species, health condition, and height of plants.
The company’s software can then track vegetation changes, recognize environmental stressors, and even calculate the potential falling curves of individual trees to identify areas that pose a risk to networks. Like Europe’s largest railway operator, Deutsche Bahn (DB),
Around 70 percent of DB’s rail network runs through areas with tree cover, making overgrown vegetations or fallen branches a constant threat. Especially with the growing frequency of extreme weather events.
For infrastructure companies like DB, surveying land for potential risk had been done using helicopters and arduous ground patrol. With a rail network of 20,500 miles, having a remote view of the entire network at once is not just more efficient but it’s also more accurate and safer.
Space for all
With more data available from space, the ability to store massive data sets, and the advancements in cloud computing and AI, Naujoks believes it’s only a matter of time before most companies will leverage satellite data.
Just as no one could foresee how digital payments and data location services would revolutionize transportation through ride-hailing apps, Naujoks anticipates satellite data will disrupt consumer applications in unexpected ways.
In a decade, 50,000 satellites may zoom through our night sky, offering space-based data services, collecting more data than ever about our planet, and driving 52 Exabytes of traffic.
Taking advantage of the data coming in and out of space will require innovative data storage that can survive the challenges of operating in space and meet the meteoric rise of data on Earth.
Header image: Sentinel-1 satellite ©ESA/ATG medialab
