Instrumental Responding 377possible and in retrospect may appear as plausible as the a
priori model that inspired the research.
Nevertheless, models have succeeded in stimulating
experiments that identify new empirical relationships. The
models most successful in this respect are often among the
least successful in actually accounting for behavioral change.
This is because a model stimulates research only to the extent
that it makes unambiguous predictions. Models with many
parameters and variables (e.g., McLaren & Mackintosh,
2000; Wagner, 1981) can be tuned post hoc to account for
almost any observation; hence, few attempts are made to test
such models, however plausible they might appear. In con-
trast, oversimplified models such as Rescorla and Wagner
(1972) make unambiguous predictions that can be tested,
with the result that the model is often refuted. For the fore-
seeable future, a dialectical path towards theory develop-
ment, in which relatively simple models are used to generate
predictions which, when refuted, lead to the development of
relatively complex models that are more difficult to test, is
likely to persist.
INSTRUMENTAL RESPONDING
This chapter has so far focused almost exclusively on
Pavlovian (i.e., stimulus-outcome) conditioning. By defini-
tion, in a Pavlovian situation the contingency between a
subject’s responding and an outcome is zero, but in many
situations outcomes are in fact dependent upon specific
responses. That is, behavior is sensitive to the contingency
between a response and an outcome. It is obvious that such
sensitivity is often adaptive. For example, a rat will quickly
learn to press a lever for food pellets; conversely, a child who
touches a hot stove will rarely do so again. A situation in
which an organism’s behavior changes after exposure to a re-
sponse-outcome contingency is termedinstrumental condi-
tioning. After reviewing Thorndike’s early work on the law
of effect and some basic definitions, this section considers
research on instrumental conditioning from three different
perspectives: associationistic, functional, and ecological-
economic.
Law of Effect: What Is Learned?
Although the idea that rewards and punishments control be-
havior dates back to antiquity, the modern scientific study of
instrumental conditioning was begun by Thorndike (1898).
He placed hungry cats in so-called puzzle boxes in which the
animal had to perform a response (e.g., pulling a loop of cord)
in order to open a door and gain access to food. Over repeated
trials, he found that the time necessary to escape gradually
decreased. To explain this result, Thorndike (1911) proposed
thelaw of effect,which states that stimulus-response (S-R)
connections are strengthened by a “satisfying consequence”
that follows the response. Thus, the pairing of the cats’ escape
response with food increased the likelihood that the cats
would subsequently perform the response. Aversive conse-
quences have symmetric but opposite effects; S-R connec-
tions would be weakened if an “annoying consequence” (e.g.,
shock) followed a response. The law of effect represents the
most importantempiricalgeneralization of instrumental con-
ditioning, but its theoretical significance remains in dispute.
The three perspectives considered in this section (associa-
tionistic, functional, and ecological-economic) provide dif-
ferent interpretations of the law of effect.The Three-Term ContingencyUnlike the contingencies used in Pavlovian conditioning,
which depend on two stimuli (the cue and outcome) scheduled
independently of the subjects’ behavior, the contingencies
considered here depend on the occurrence of a response. Such
contingencies are calledinstrumental(i.e., the subjects’ be-
havior is instrumental in producing the outcome) oroperant
(i.e., the subjects’ behavior operates on the environment).
Because different stimuli can be used to signal particular con-
tingencies (i.e., illumination of a light above a lever signals
that a rat’s pressing the lever will result in the delivery of
food),the three-term contingencyhas been proposed as the
fundamental unit of instrumental behavior: In the presence of
a particular stimulus (discriminative stimulus), a response
produces an outcome (reinforcer;Skinner, 1969).
In an instrumental situation, the environmentally imposed
reinforcement contingency defines a response and, not sur-
prisingly, the frequency of that response ordinarily changes
in a functional manner. Instrumental behavior can sometimes
be dysfunctional (i.e., a different response is observed than
that defined by the functional contingency), but this is the
exception rather than the rule. When dysfunctional acquired
behavior is observed, it usually reflects a prevailing con-
tingency that is unusual to the subject’s ecological niche or
contrary to its prior experience. Two good examples of dys-
functional responding are vicious circle behavior (Gwinn,
1949) and negative automaintenance (D. R. Williams &
Williams, 1969). In the former case, a previously learned re-
sponse obstructs the subject from coming in contact with a
newly introduced contingency, and in the latter case the rein-
forcement contingency (reward omission) imposed by the
experiment is diametrically opposed by a species-specific
predisposition that is highly functional in the natural habitat.