11
THE NEURAL PROCESSING
OF COMPLEX SOUNDS
.
Abstract
This chapter considers the temporal processing of complex sounds relevant to musical analysis.
Functional imaging studies, using positron emission tomography (PET), functional magnetic res-
onance imaging (fMRI), and magnetoencephalography (MEG), and the psychophysical assessment
of patients with lesions allow two different approaches to this. Functional imaging allows the determina-
tion of structures normally involvedin temporal analysis, while patient studies allow inference about the
necessarystructures for temporal analysis. Both approaches suggest a hierarchal organization in
the brain corresponding to the processing of music. The features of individual notes are analyzed in
the pathway up to and including the auditory cortices, while higher-order patterns formed by those
features are analyzed by distributed networks in the temporal lobe and frontal lobes distinct from the
auditory cortices.
Keywords:Neural processing; Complex sounds; Musical analysis; Functional imaging
Introduction
This chapter might have been called ‘The Neural Processing of Complex Sound Features’.
Allsound is processed by mechanisms for the analysis of simple acoustic features (intens-
ity, frequency, onset), complex acoustic features (such as patterns of these simple features
as a function of time), and semantic features (learned association of sound patterns and
meanings). Here, I concentrate on the analysis of complex features likely to be important
in musical analysis, below the level of semantic and affective processing. Such analysis is
represented in the middle box in Figure 11.1.
For the analysis of music, analysis of temporal features is likely to be particularly import-
ant, although I would certainly not dismiss the role of spectral or spatial analysis.
Temporal analysis can be considered at different levels that I will call fine temporal struc-
ture and higher-order temporal structure. Such a categorical distinction works well for
music as a sound with a highly segmented structure. Fine temporal structure corresponds
to temporal structure at the level of milliseconds or tens of milliseconds; this is a ‘window’
for the processing of the temporal regularity within individual notes that is relevant to
complex pitch perception (see Refs 1 and 2 for discussion of mechanisms of pitch percep-
tion). Higher-order temporal structure corresponds to temporal structure at the level of
patterns of the pitch, onset time, or duration of notes. This corresponds to a ‘window’ at