presented mismatch. In other words, the visual input actually overrides the acoustics. In more ecologically realistic
situations, thevisuallydetectedmotionofthespeaker's mouth contributes tophoneticperception and actuallyhelps us
hear in noisy environments. Within structure-constrained modularity, the McGurk effect can be attributed to an
additional interface processor that uses visual input to contribute fragments of structure to phonological working
memory. But this interface can't tell phonology about all aspects of phonological structure—only about those
distinctivefeatures that can be detected by visual inspection (lip closure and rounding, and perhaps vowel height) plus
perhaps some weak information about degree of stress. Similarly, its input is not all of visual structure, but only those
aspects that pertain to the external appearance of the vocal tract. So it implements an extremely limited partial
homology between the visual input and phonological structure.
A different partial homology appears in the use of gesture accompanying language. As observed in section 5.3, beat
gestures basically set up a metrical grid; these must be placed in optimal coincidence with the metrical grid in
phonological structure. Every other aspect of phonological structure is irrelevant. A similar though more complex
problem occurs in setting texts to music: the musical and linguistic stresses must coincide (within certain degrees of
latitude). And of course visually detected beat gestures are available to music processing too: that's how the orchestra
follows the conductor. In each of these cases, what is required is a highly bi-domain-specific interface processor that
attempts to correlate metrical grids, ignoring all other aspects of the structures involved.
Reading requires yetanother type of visual-phonologicalinterface.This operationis fast, mandatory (try seeing writing
as mere design!), and subject to domain-specific brain damage—again hallmarks of modular processes. Reading (at
least in alphabetic orthographies) provides phonology with predominantly segmental information; punctuation gives
some information about prosodic bracketing that is also derivable from the auditory signal. Unlike the
auditory–phonological interface, reading does not give information about stress, except through use of underlining
or italics. But on the other hand, reading gives more reliable information about word boundaries than auditory input.
So the output of the reading interface partially overlaps with that of the auditory–phonological mapping, but is far
from complete. The visual information“visible”to the reading interface is also partial. The phonological content of
what you're reading doesn't change if you change the colors on your computer screen or carve the letters in a tree or
put the mup in neon lights. That is, reading does not exploit the full capabilities of the visual syste m, and it overlapsin
complex ways with general-purpose vision. In short, like the