use of typed variables. His agenda is to defuse the extravagant claims that have been made on behalf of a particular
class of spreading activation models, the multi-layer perceptrons of the sort common in connectionist modeling (e.g.
Rumelhart and McClelland 1986a, b; Elman et al. 1996). He shows that for principled reasons these models cannot
encode variables of the sort necessary for two-placerelations such as“X is identical with Y,”“X rhymes with Y,”and
“X is the regular past tense of Y.”Space precludes my repeating his arguments here, not to mention his replies to the
manyreactions hisworkhas produced.Sufficeittosay thatMarcus has testedalltherelevantnetworksintheliterature
on the data sets for which he predicts they will fail, and indeed they fail.^33
This principled failure is fatal to unadorned spreading activationmodels of language, for, as we saw in section 3.2, all
combinatorial rules of language—formation rules, derivational rules, and constraints—require typed variables. Again,
this does not mean that spreading activation plays no role in the brain's storage and processing of linguistic f-
knowledge; in fact it likely does (see Chapter 7). But some further technical innovationis called for in neural net-work
models, which will permit them to encode typed variables and the operation of instantiating them. I think that upon
thedevelopmentof such an innovation,thedialogue between linguistic theory and neural networkmodeling willbegin
to be more productive.
3.5.4 Binding in working memory vs. long-term memory
As alluded to earlier, contemporary neuroscience tends to see transient (short-term) connections among items in
memory as instantiated either by spreading activation through synapses or by the“binding”relation, often thought of
interms offiring synchrony. By contrast, lasting(long-term) connections are usuallythoughtofas encoded interms of
strength of synaptic connections. However, the combinatoriality of language presents the problem that the very same
relation may be encoded either in a transient structure or in one stored in memory.
Consider, for instance, the idiomkick the bucket. This has to be stored in long-ter m me mory, since one cannot predict
its meaning from the meaning of its parts. At the same time, it has the syntactic structure of an ordinary verb phrase
such aslift the shovel, which is built up combinatorially. Hence, whenkick the
COMBINATORIALITY 65
(^33) Marcus's arguments concerning variables do not appear to be adequately appreciated in the connectionist literature; e.g. Ifind no reference to the min a recent issue of
Cognitive Science dedicated to“Connectionist Models of Human Language Processing: Progress and Prospects”(Christiansen etal. 1999).