SCIENCE
F
or generations, scientists and
sci-fi fans have inspired each
other to (boldly) go where no
one has gone before to better
understand the world around
them. Now, real-world science has taken
another page right out of the science fiction
handbook, creating an immersive, virtual
reality “holodeck” for animals.
You might remember the holodeck as
one of the most iconic features of Star
Trek: The Next Generation, a high tech VR
arena that could create any imaginable
world in the blink of an eye. Now, scientists
in Vienna and Freiburg have developed a
holodeck-like environment to help us
understand how visual cues influence
behavioral choices in animals.
The platform, called FreemoVR, proj-
ects an environment within the animal
arena (like an ocean floor) and then pre-
cisely follows the animal with multicam-
era tracking software developed by the
Straw Lab. Most importantly, FreemoVR
adapts to the viewpoint of the animal in
real-time according to its behavior. This
means that, from the perspective of the an-
imal, it is moving through the virtual
world and not toward a flat screen.
Using VR to study behavior is nothing
new. Model organisms have been maneu-
vering their way through video game mazes
for years and this work has been incrediblyhelpful for our understanding
of the brain. But until now
these models have required
the animals to be partially or
fully immobilized. Ultimately
this restriction influences the
way the animal would behave
compared to the natural environment. The
advantage of FreemoVR is that there are no
strings attached. Here, animals can explore
the environment without physical restric-
tion or additional visual aids (think: Google
glasses for mice) that might change the way
they interact with their surroundings.
Proof-of-concept tests with FreemoVR
have been published in Nature Methods for
three of the most common model organ-
isms: fish, fruit flies and mice. The team
tracked the animals as they swam, flew or
ran around the virtual objects and situa-
tions and compared this with the animal’s
response to the real thing. Using this tech-
nique, the team could make mice afraid of
virtual heights and make fruit flies and ze-
brafish avoid a virtual pole in a way that mir-
rored their responses to real-world objects.TELEPORTING FISH
Why is this important? Let’s take fish, for
example. Until now, if you wanted to see
how a fish would interact with an object
you would have needed to physically move
the object or the fish.“It is very difficult to
separate the effect of this
movement from the experi-
ment. Now you can project
something, like a plant, and
then take it away instanta-
neously. It gives you a real
feedback loop for your experiment,” says
Prof. Kristin Tessmar-Raible from the Max
F. Perutz Laboratories in Vienna. Her lab is
studying how light and time influence be-
havior and uses FreemoVR for zebrafish
and medaka fish.
FreemoVR can be adapted for each type
of animal, which might mean that a differ-
ent arena style or number of cameras may
be used to capture its unique challenges
(for a home experiment, try tracking a
fruit fly with your eye). In FreemoVR for
fish, the virtual environment is projected
directly on the fishbowl. Four cameras
track the fish, measuring disturbances in
near-infrared light. Using strong contrast-
ing backgrounds like a checkerboard, it is
possible to detect preferences in fish who-
choose to swim close to light or dark
squares, which might give insight into how
they perceive light.
“In general, this system opens up a lot of
new research opportunities to address
questions. related to animal biology. For
example, we have no idea why fish have
photoreceptors in the brain. What are they“This system opens up new research opportunities. For
example, we have no idea why fish have photoreceptors in the brain.”
Prof. Kristin Tessmar-Raible, researcher and group leader at the Max F. Perutz Laboratories in ViennaThe VR arena allows
researchers to study the
behavior of mice (pictured
previous page), fish (left)
and fruit flies (above). It’s
like a “holodeck,” an iconic
feature of Star Trek’s
spaceship Enteprise.PHOTOS: PREVIOUS PAGE: IMP/IMBA GRAPHICS DEPARTMENT; THIS PAGE: IMP/IMBA GRAPHICS DEPARTMENT; NEXT PAGE: LEFT: PARAMOUNT PICTURES AND/OR CBS STUDIOS / WIKIMEDIA COMMONS; RIGHT: STRAWLAB 7 FREEMOVR.