OPTICAL IMAGING
Time-of-flight 3D imaging through multimode
optical fibers
Daan Stellinga^1 †, David B. Phillips^2 †, Simon Peter Mekhail^1 †, Adam Selyem^3 , Sergey Turtaev^4 ,
TomášCˇižmár4,5, Miles J. Padgett^1 *
Time-of-flight three-dimensional (3D) imaging has applications that range from industrial inspection
to motion tracking. Depth is recovered by measuring the round-trip flight time of laser pulses, typically
using collection optics of several centimeters in diameter. We demonstrate nearÐvideo-rate 3D imaging
through multimode fibers with a total aperture of several hundred micrometers. We implement
aberration correction using wavefront shaping synchronized with a pulsed source and scan the scene
at ~23,000 points per second. We image moving objects several meters beyond the end of an
~40-centimeters-long fiber of 50-micrometer core diameter at frame rates of ~5 hertz. Our work
grants far-field depth-resolving capabilities to ultrathin microendoscopes, which we expect to have
applications to clinical and remote inspection scenarios.
M
ultimode optical fibers (MMFs) repre-
sent an extremely efficient method of
transporting light with a high spatial
information density. They can support
the propagation of thousands of spa-
tial modes—i.e., optical field patterns that
act as independent information channels—
within a cross-sectional area similar to that
of a human hair. These features have led to
much interest in the deployment of MMFs as
microendoscopes, enabling high-resolution im-
aging at the tip of a needle ( 1 – 4 ).
In this work, we extend the imaging cap-
abilities of such microendoscopes to include
depth information. Three-dimensional (3D)
imaging through ultrathin MMFs promises
an array of new applications, including the
3D inspection of the internal chambers of
objects that are difficult to open, such as jet
engines or nuclear reactors, and the 3D vi-
sualization of hollow viscous organs, which
could help surgeons navigate inside the body
during operations.
However, the compact form of MMFs comes
at a cost: Optical signals are subject to modal
dispersion, typically causing input coherent
light patterns to be unrecognizably scrambled
into speckle patterns at the output facet,
formed entirely from nonballistic light that
has scattered multiple times from the core-
cladding interface ( 5 – 7 ). Fortunately, as long
as a MMF remains in a fixed configuration,
the scrambling process is deterministic and
unchanging in nature, so a given input field
will always produce the same output field.
This means that the way a static MMF scram-
bles light can be represented by a linear matrix
operator, known as a transmission matrix
(TM), which maps any possible input field to
the resulting output ( 8 – 13 ).
Measurement of the TM enables the calcula-
tion of how an input field should be preshaped
to generate a desired output—for example a
spot focused to a particular location. This
method is known as wavefront shaping ( 14 , 15 );
thus, by illuminating the proximal facet with
a sequence of carefully prepared input light
fields, a focused spot can be raster scanned
across the distal facet of a MMF. Scanning-
based imaging can then be achieved by re-
cording the total intensity of light returning
through the fiber and correlating this with the
position of the focus ( 16 , 17 ). Recently, wave-
front shaping through MMFs has been em-
ployed for in vivo imaging of neurons deep
inside the brains of mice ( 2 – 4 )—an endeavor
very challenging to achieve in any other such
minimally invasive way. The imaging of ob-
jects some distance from the distal facet of a
MMF is also possible ( 9 , 18 ). However, this is
more demanding because the level of return
signal falls off rapidly in proportion to the
square of the object’s distance.
Here, we augment MMF microendoscopy
with time-of-flight (TOF) light detection and
ranging (LIDAR) techniques to provide depth
information alongside 2D reflectance images
( 19 ). TOF techniques recover depth by mea-
suring the round-trip flight time of a laser
pulse reflecting from the scene. To achieve
this, we implement high-speed wavefront shap-
ing synchronized with a subnanosecond pulsed
laser source. Light fields entering the MMF
are shaped using a digital micromirror device
(DMD) operating at 22.7 KHz. We use knowl-
edge of the TM linking the proximal (near)
end of the MMF to the far field of the distal
facet to raster scan the pulsed source across
the field of view.Extending wavefront shaping through MMFs
from continuous-wave to pulsed illumination
presents an additional complication—the po-
tential for temporal pulse distortion as a result
of chromatic and spatial mode dispersion. In
our system, the latter form of dispersion domi-
nates. Spatial mode dispersion can be under-
stood by considering that an incident pulse will
excite different spatial modes supported by the
fiber, which travel at slightly different velocities
( 20 , 21 ).
We must ensure that the optical path dif-
ference accumulated by these modes as they
propagate through the fiber remains below
the coherence length of the pulse. This places
a constraint on the temporal length of the
pulsetpthat can be used with a given geom-
etry of step-index fiber, leading to (see supple-
mentary materials, section 7 for derivation)
the followingtp≫NA^2 L
cncð 1 ÞwhereLis the fiber length, NA is the numerical
aperture of the fiber,ncis the refractive index of
the core, andcis the speed of light in a vacuum.
Here,weuseastep-indexfiberofL~ 0.4 m,
NA = 0.22,nc= 1.46, and core radiusa= 25mm.
Therefore, Eq. 1 yieldstp≫44ps. To ensure
minimal pulse distortion, we choose a laser
with a pulse duration oftp~ 700 ps, with central
wavelengthl= 532 nm. We note that Eq. 1 pro-
vides a bound on the minimum pulse duration
to achieve arbitrary wavefront shaping at the
output without temporal distortion. However,
each far-field focal spot only requires the exci-
tation of spatial fiber modes with very similar
phase velocities, so for this special case it may
be possible to use a source with substantially
shorter pulse duration than that implied by
Eq. 1. We also note that the use of a graded-
index fiber would place a lower limit ontp.
Our fiber supportsN~(paNA/l)^2 ~ 1000 spa-
tial modes per polarization, setting the lateral
resolution to ~4N= 4000 independently re-
solvable features within each image, according
to the Raleigh criterion (supplementary mate-
rials, section 8) ( 18 , 22 ).
Figure 1 shows a schematic of our prototype
system and some example depth images. Each
frame was recorded by raster scanning 4200
points in 200 ms. A single laser pulse is de-
livered to each spatial position in the field of
view. A second fiber, placed alongside the il-
lumination fiber, is used to collect the back-
scattered light. This collection fiber has a larger
core diameter of 500mmtoincreasethecol-
lection efficiency and thereby the working dis-
tance of the endoscope. The return signal from
each image pixel is coupled directly to an ava-
lanche photodiode (APD) and converted to a
TOF histogram, referenced to the time that the
outgoing pulse entered the illumination fiber.SCIENCEscience.org 10 DECEMBER 2021•VOL 374 ISSUE 6573 1395
1
School of Physics and Astronomy, University of Glasgow,
Glasgow G12 8QQ, UK.
2
School of Physics and Astronomy,
University of Exeter, Exeter EX4 4QL, UK.^3 Fraunhofer Centre
for Applied Photonics, Glasgow G1 1RD, UK.^4 Leibniz Institute
of Photonic Technology, Albert-Einstein-Straße 9, 07745
Jena, Germany.^5 Institute of Scientific Instruments of the
CAS, Královopolská 147, 612 64 Brno, Czech Republic.
*Corresponding author. Email: [email protected]
These authors contributed equally to this work.
RESEARCH | REPORTS