42 | New Scientist | 9 November 2019
Seeing around
corners
Hidden scenes are lurking in the shadows.
Jon Cartwright exposes the intriguing
science of seeing the invisible
N
OTHING to see here: just an image
of an empty street. But the investigator
thinks there is more to this than
meets the eye. With a few clicks of his mouse,
he enhances a featureless shadow cast on
the floor, apparently defying the laws of
optics to extract a blurry image of two people
lurking around the corner.
Technical wizardry like this seems far-
fetched. But this isn’t CSI. The investigator is a
computer scientist not a detective, and those
characters are graduate students not suspects.
More importantly, this technology is real, and
it is being developed in labs right now.
The science of seeing around corners is
new, fast-moving and breathtaking. We are
discovering that the shadows are full of visual
information that our eyes can’t see. Now, as
people develop clever ways to make the
invisible visible, they are exposing all manner
of potential applications besides forensics.
Autonomous cars that spot hidden hazards.
Cameras that direct fire crews to people
trapped in burning buildings. Endoscopes that
guide surgery in unreachable parts of the body.
“It could be extremely powerful,” says
Vickie Ye, a computer vision researcher
at the University of California, Berkeley.
“Any information outside the frame could
be interpretable.”
You don’t need novel science to see around
a corner. You could just use a periscope, or any
mirror for that matter. A mirror works because
light rays bounce off the surface in a clean andpredictable way – namely, at the same angle
at which they hit it. As a result, all the visual
information collectively contained within
the light rays is preserved, so that you always
see a clear image of whatever is out of view.
The problem is that most surfaces we
encounter aren’t reflective, at least not in
the sense that a mirror is. When you look at a
painted wall, for example, you are observing
light rays that have bounced, or scattered, from
all sorts of random angles, preventing you
from seeing an image of yourself. In fact, your
image is there, but it is made up of only the tiny
minority of light rays that happen to take the
direct path from your face, into the wall and
back into your eyes. The majority of light rays,
which scatter through alternative paths, wash
these out and thus render any image invisible.
To the human eye, at least. In 2012, computer
scientist Ramesh Raskar at the Massachusetts
Institute of Technology (MIT) and his team hid
an artists’ manikin behind a screen and then
fired laser pulses onto an adjacent wall. They
knew that some of the photons fired by the
laser would scatter off the wall, rebound off the
manikin and then scatter off the wall again,
before finally being picked up by their photon
detector. They also knew that this portion of
photons would be tiny compared with the
zillions taking different routes. The trick was
in the precision of their detection system.
By timing the return of a photon to within a
few trillionths of a second, they could calculate
how far that photon had travelled after it hadFeatures
ELENI DEBO