Industries & Applications

MIT and Adobe Bring Down the (Image) Noise

There is no getting away from the fact that smartphones are becoming ever more like pocket computers. They pack substantially more processing power in them than the computers that took man to the moon but there are some areas where all that processing power doesn’t quite match up. One such area is in image processing.

Since smartphones are now the de facto camera of choice for the majority of the population we are seeing the rapid growth in mobile imaging technologies to provide ever better images. And along with that, there is a massive growth in mobile image-processing applications too—a quick look at any of the ‘app’ stores will show you that imaging apps are one of the most popular, and populated, categories.

The challenge, however, is that as image quality improves—and the more data that gets captured in individual photos via RAW mode or simply in higher resolution—image editing becomes even more computationally intensive, which can mean longer processing times and flattening battery times. Waiting for things to happen, and suffering rapidly decreasing battery life are two things smartphone users are not willing to accept, so any technology that can help alleviate these issues is likely to be readily snapped up by end-users.

The combined brains of MIT and Adobe might just have such a solution. At the Siggraph Asia conference, they showed a system that reduced the load on the phone processing by shipping the heavy data crunching off to a cloud server. This, they say reduced power consumption by up to 85% and because of their methodology it reduces bandwidth by as much as 98.5%.

The solution they have is using noise—image noise. The kind photographers normally spend a long time trying to avoid! In essence they create a very low-resolution version of the file to send to the processing server and when it gets there add high frequency noise to re-create the detail.

The image is then broken up into blocks which are each edited using a learning machine algorithm. Currently they use around 25 parameters to adjust each block—things like luminance, saturation, sharpening and so on. Then, instead of sending the resulting image back, they simply send back instructions for how to edit the high-resolution image stored locally on the smartphone, based on these 25 parameters.

While this does still leave some computational requirement to the phone architecture, it is neither as intensive in power needs or in bandwidth as sending a high resolution file would be. It also doesn’t negate the need for high capacity local storage to keep the high resolution files on the device.

At the moment they say it works well with Instagram style filters or global adjustments to the whole image, but is less effective with edits that change the content—by removing an element from the image for example.

Mobile imaging is here to stay and this is just the next step in improving the image quality smartphones can produce and in giving end-users more power in their palm.

Subscribe to this blog

Enter your email address to subscribe to this blog.

Dave Newton
Since taking up photography seriously in 1998, David Newton has resisted the urge to be pigeon-holed into one category of photography. He's a photographer first and foremost and will therefore point his lens at all subjects great and small. A degree in Marine Biology points to a strong love for the natural world and so wildlife, travel and landscape photography form the greatest part of his work, but nothing is off-limits as he regularly turns his hand to everything from sports to portraits and even commercial work. With a strong technical background, David is a well known technical writer and trainer. He is often sought out by major photographic companies to help explain or demonstrate equipment both in online videos and through event seminars and training courses. Since he never knows what his next subject might be, David has to rely on his equipment completely. That means paying attention to the smallest of details so as nothing is left to chance. When it comes to memory cards, David won't trust his images to anything but SanDisk Extreme®.

data infographicToday’s digital economy with mobile, IoT and cloud is based on the value of data. How do you unlock it? Download the infographic to learn more: