258 • CHAPTER 9 Knowledge
process. Before learning, the weights are incorrect, as in our example, so activating
“canary” causes an erroneous response in the property units. According to connec-
tionism, these erroneous responses are noted and sent back through the network, by
a process called back propagation. Back propagation is the process by which error
signals are sent back to the hidden and representation units to provide information
about how the connection weights should be changed so that the correct property
units will be activated.
We won’t explain the specifi cs of how these back-propagated error signals pro-
vide information for changing the connection weights, because the answer is complex
and is still a topic of investigation by researchers. Instead we will describe the pro-
cess behaviorally. Consider a young child watching a robin on a branch. Just below
the tree, a cat is approaching. The robin sees the cat and fl ies away. The young child
may have thought the bird would not react to the cat but, in observing this situation,
learned that robins avoid cats. Children also learn about properties of robins from
their parents. If they see a canary and say “robin,” the parent might correct them and
say “That is a canary” and “Robins have red breasts.” Thus, a child’s learning about
concepts begins with little information and some incorrect ideas, which are slowly
modifi ed in response both to observation of the environment and to feedback from
others. Similarly, the connectionist network’s learning about concepts begins with
incorrect connection weights, which are slowly modifi ed in response to error signals.
In this way, the network slowly learns that things that look like birds can fl y, things
that look like fi sh can swim, and things that look like trees stay still while other things
move around them.
The connectionist network’s learning process therefore consists of initially weak
and undifferentiated activation of property units, with many errors (for example, the
input “canary” causing activation of the property unit “tall”). Error signals are then
sent back through the network. These signals result in changes in connection weights,
so the next activation of “canary” results in a new activation pattern. Changes to the
connections are small after each learning experience. The new pattern is closer to the
correct pattern but still isn’t correct, so the process is repeated until the network assigns
the correct properties to “canary.” The pattern of activation distributed across the net-
work that results in activation of the correct property units is the pattern that repre-
sents “canary.”
Although this “educated” network might work well for canaries, what happens
when a robin fl ies by and alights on the branch of a pine tree? To be useful, this net-
work needs to be able to represent not just canaries, but also robins and pine trees.
Thus, to create a network that can represent many different concepts, the network is
not trained just on “canary.” Instead, presentations of “canary” are interleaved with
presentations of “robin,” “pine tree,” and so on, with small connection weights made
after each presentation.
Because the network has to respond correctly to many different concepts, the
network’s learning process has to be designed in such a way that changing the con-
nection weights to obtain a better response to “canary” doesn’t result in a worse
response to “pine tree.” This is achieved by changing the weights very slowly on each
trial, so that changing the weights in response to one concept causes little disrup-
tion of the weights for the other concepts that are being learned at the same time.
Eventually, after thousands of trials, the weights in the network become adjusted so
that the network generates the correct activation of property units for many differ-
ent concepts.
We can appreciate how this learning process occurs over many trials by looking at
the results of a computer simulation. The network in Figure 9.22 was presented with a
number of different concepts and relation statements, one after another, and the activ-
ity of the units and connection weights between units were calculated by the computer.
● Figure 9.23 indicates the activation of the eight representation units in response to
different inputs. At the beginning of the process, the experimenter set the connection
weights so that activity was about the same in each unit (Learning trials = 0). This cor-
responds to the initially weak and undifferentiated activation we discussed earlier.Copyright 2011 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part.