310 | 29 CONNECTIONISm: COmPUTING wITH NEURONS
Connectionism’s promise
In a famous demonstration of the potential of connectionism in the 1980s, James McClelland
and David Rumelhart trained a network of 920 neurons to form the past tenses of English
verbs.^2 Verbs such as ‘come’, ‘look’, and ‘sleep’ were presented (suitably encoded) to the layer of
input neurons. The automatic training system noted the difference between the actual response
at the output neurons and the desired response (such as ‘came’) and then mechanically adjusted
the connections throughout the network in such a way as to give the network a slight push in
the direction of the correct response.
About 400 different verbs were presented to the network one by one, and after each presenta-
tion the network’s connections were adjusted. By repeating this whole procedure approximately
200 times, the connections were honed to meet the needs of all the verbs in the training set. The
network’s training was now complete, and without further intervention it could form the past
tenses of all the verbs in the training set.
Furthermore, the network had now reached the point of being able to respond correctly
to unfamiliar verbs. For example, when presented for the first time with ‘guard’, it responded
‘guarded’. More impressively still, it replied ‘wept’ to ‘weep’, ‘clung’ to ‘cling’, and ‘dripped’ to
‘drip’ (even down to the double ‘p’). Sometimes, though, the peculiarities of English were just
too much for the network, and it formed ‘squawked’ from ‘squat’, ‘shipped’ from ‘shape’, and
‘membled’ from ‘mail’.
Today, simulated connectionist networks are in widespread use. Some prototype driver-
less cars involve connectionist networks, and they are also used in hearing aids to filter out
figure 29.1 A natural neural network: the method of staining the brain-tissue renders the neurons and their
interconnecting fibres visible.
Posted to Wikimedia Commons by GerryShaw, https://commons.wikimedia.org/wiki/File:Culture_of_rat_brain_cells_stained_with_
antibody_to_MAP2_(green),_Neurofilament_(red)_and_DNA_(blue).jpg. Creative Commons licence.