184 HOW THE BRAIN MAKES POTENTIALas in the psy chol ogy of music, is increasingly drawing attention to the
deep correlational structure in it, or “structure in time.” Again, we seem
to appreciate such structure because of its tacit qualities of predictability,
which are also the essence of harmony and the source of the plea sure in
music.
Each of these senses, like vision, plays a crucial part in be hav ior. But, of
course, they can only do so in conjunction with inputs from other senses.
In the way that the world is usually experienced, there will be considerable
correlation between the statistical structures sensed in diff er ent modes:
touch with vision, vision with sound, all three at once, and so on. Th is is
particularly the case as any movement, such as action on objects, will also
change experience in several sense modes si mul ta neously. Consider eating
a meal, or just walking along the road; vision, sound, smell, and even the
stimuli through the soles of the feet will vary together, or correlate.
Th ere are also signals from internal senses. Every time an individual
acts on objects, the brain receives a barrage of signals from receptors in the
body itself. Th ese include stretch receptors in muscles and tendons, vari ous
kinds of touch receptors in skin, pressure receptors in joints and muscles
and from the muscle spindles (providing information about body posture).
Changes in all of these will covary with one another and with those occur-
ring si mul ta neously in visual and auditory experience of the event. As well
as enhancing predictability in experience as a whole, integration of senses
permits expectations from one mode to another. On hearing a sound, for
example, even young babies turn their eyes and heads toward it.^20
Such integration takes place at vari ous levels in the subcortex and cor-
tex. Multisensory cells, sensitive to more than one mode, have long been
identifi ed in the ce re bral cortex. Recordings of receptive fi elds commonly
fi nd neurons in the visual cortex that are also tuned to acoustic frequen-
cies. Cells in auditory cortex in humans are strongly activated by watching
silent videos of speechlike facial movements. Seeing something at the same
time as hearing it enhances reaction times and discriminability in labora-
tory tasks. It has also been noted how the strength of coupling of signals
from diff er ent modalities depends on the statistical co- occurrence be-
tween them.
In the 1980s, Donald MacKay argued that much intentional be hav ior
(e.g., hearing while looking, moving eyes and head to sample a range of
This content downloaded from 139.184.14.159 on Tue, 17 Oct 2017 13:54:00 UTChttp://www.ebook3000.com