The current paradigm of computational imaging is characterized by the co-design of optics and signal processing — eliminating the need for “traditional” human-interpretable optical images. Instead, human-interpretable digital images are computed from a sensed optical image.
Rambus researchers have pioneered a diffractive design strategy, effectively severing a long-standing reliance on traditional optical elements of lenses or curved mirrors. This approach enables the manufacture of small, inexpensive sensors that shift the burden of image formation from the optical to digital domain by capturing data relevant for machine-specific consumption.
“Lensless technology allows sensors to capture information rich images using a low-cost phase grating,” said inventor Dr. Patrick R. Gill. “Although the raw ‘snap’ is indecipherable to the naked human eye, the sensor, which is about the size of pinhead, is designed to capture all of the information in the visual world up to a certain resolution.”
As Gill explains, traditional imaging is typically associated with conventional cameras that capture a simple, straightforward representation of a particular subject or scene. However, lensless technology is (roughly) analogous to the way a human, animal or insect brain perceives the world: the real-time interpretation of a scene or object facilitated by inherent pattern recognition capabilities. More specifically, says Gill, data leaving your retina looks nothing like a bitmap, although it obviously contains all the data needed to interpret an image.
“The realization that machine-viewed images don’t have to be captured and rendered as little pictures has helped inspire the design of new optical systems,” he continued.
“At Rambus, we are developing a new generation of discerning sensors, with individual categories targeting specific tasks, functions and features – just like application specific neurons in the brain. For example, our low power sensors can tell you how many people are in a particular room, recognize a specific gesture performed in close proximity to a smartphone or tablet, gauge light levels and detect when a car is approaching your vehicle’s bumper.”
All of this, says Gill, ties into the Internet of Things (IoT), which is characterized by the rise of smart, connected devices and platforms such as power grids, appliances, wearables, cars and even entire cities.
“We expect inexpensive, low power sensors to become ubiquitous as the IoT rapidly evolves,” he noted. “They are absolutely essential to helping us make sense of the world around us.”
Rambus Fellow Dr. David G. Stork, who leads research in Rambus’ Computational Sensing and Imaging group and has, together with Dr. Gill, developed both the fundamental theory of computational diffractive sensing and imaging, as well as demonstrated a number of novel functions for Rambus lensless sensors, from image estimation to pattern recognition.
The technology behind lensless sensors, says Stork, can also be used to help researchers in other spaces, including physics, medicine, neuroscience, mathematics and even artificial intelligence (AI).
“Ultimately, Rambus algorithms can help make machines more intelligent by allowing them to clearly perceive and understand the world around them,” he explained.
As Stork points out, one of the most important lessons from artificial intelligence research in the past several decades concerns the benefits of using machine learning and pattern recognition methods applied to large amounts of real-world data.
“To the extent that sensors continue to proliferate and extract real-world data, artificial intelligence systems will continue to improve,” he said.
Indeed, we often take our sensory capabilities for granted, not realizing they have incrementally evolved over the millennia. For example, a human can be shown a hand at various angles and still be able to recognize it as a hand.
“Unfortunately, even the best AI is not yet capable of replicating this exercise, which most adult humans can perform quite effortlessly and without much thought,” Stork added.
“Then again, the world is quite complex, so it is unsurprising that various initiatives to create artificial machine intelligence have proven ineffective – especially given the current absence of meaningful sensory data. We’re hopeful that our work at Rambus will help shake up the current equation, both in terms of lensless sensors and their potential applications across multiple spaces.”