Learning and conditioning

Four names make any case of conditioning easier to read, and Pavlov’s dogs show all four. The food is the unconditioned stimulus: it sets off drooling with no learning at all. That drooling is the unconditioned response. The metronome starts out neutral, setting off nothing in particular. Once pairing has done its work, the metronome has become the conditioned stimulus, and drooling at its sound is the conditioned response.

Unconditioned here means unlearned, and conditioned means learned. So the same drooling can be either, depending on what sets it off: the food, which needed no teaching, or the sound, which did. Pavlov’s own name for the learned reaction was the conditional reflex.

A conditioned response grows over repeated pairings, a stage called acquisition. What happens when the signal keeps coming and the food stops? Pavlov showed it in a lecture, with a dog that had learned to drool at a metronome. He sounded it for 30 seconds every two minutes, with no food after it. The first sounding drew 10 drops of saliva; the seventh drew 3. A conditioned response that weakens once its signal stops being followed by what it predicted is undergoing extinction.

Yet the learning was not gone. Twenty-three minutes later the metronome drew 6 drops again. A response that returns after a rest, with no new training, shows spontaneous recovery. Extinction does not wipe out the old link. It lays a new one over it, that the signal now means nothing, and the old link can still show through.

Is pairing enough? In 1968 the psychologist Robert Rescorla tested this with rats. A rat that fears a sound stops pressing a lever for food while the sound plays, so the pause measures fear. Every rat heard a tone now and then, along with brief electric shocks. For one group, shocks came only while the tone sounded. For another, shocks came just as often at other times as during the tone.

Both groups heard the tone together with a shock about equally often, yet only the first group came to fear it. For the first group the tone predicted a shock. For the second it predicted nothing, since a shock was no more likely with it than without it. Rescorla called this relation contingency: how far one event makes another more likely. Conditioning is learning what predicts what.

Can an emotion be conditioned in a person? In 1920 the psychologist John Watson and his graduate student Rosalie Rayner, at Johns Hopkins University, tried it on a baby they called Albert B. At about nine months he showed no fear of a white rat. At eleven months, as his hand touched the rat, a steel bar was struck with a hammer behind his head. After seven such pairings, the rat alone made him cry and crawl away.

Five days later he cried at a rabbit and shied away from a fur coat and a Santa Claus mask, none of which had come with a noise. A learned response that spreads to things resembling its signal is called generalization. The study frightened a baby on purpose. Knowing he would soon leave the hospital, they used his last month to test whether the fear lasted, and never removed it.

In 1924 Mary Cover Jones, advised by Watson, undid a fear. Peter, aged two years and ten months, was afraid of rabbits. While he ate food he liked, a caged rabbit was brought a little closer, never so close that he stopped eating, until he let it nibble his fingers. Pairing a feared thing with something pleasant until the pleasure takes the fear’s place is now called counterconditioning.

A plainer remedy is to meet the feared thing again and again without harm, so that the fear fades as in extinction. The scholar Abū Zayd al-Balkhī, who died in 934, advised this in Sustenance of the Body and Soul: people who fear something should expose themselves to it repeatedly, though it is unpleasant at first, until the fear is gone. Experienced physicians, he noted, have grown used to wounds and burns that frighten a beginner.

Where does a conditioned fear sit in the brain? In 1995 Antoine Bechara, Antonio Damasio and their colleagues tested patients with damage to particular parts of the brain. A colour or a tone came just before something unpleasant, and fear showed as a slight sweating of the skin that a meter can pick up.

One patient had damage on both sides to the amygdala, a small almond-shaped structure deep in each side of the brain. This patient learned which signal came before the unpleasant event, and could say so, yet the skin never reacted to it. Another, with damage to the hippocampus, which lays down memories of facts and events, showed the opposite: the skin reacted to the signal, but the patient could not say what it had come before. Being conditioned and knowing the facts depend on different parts of the brain.

Most conditioning needs many pairings, with the signal coming just before what it predicts. Taste breaks both rules. From the 1950s the psychologist John Garcia and others found that rats which drank a flavoured water and later fell ill would avoid that flavour, often after one pairing, and even when the sickness came hours after the drink. The result is called a taste aversion.

In 1966 Garcia and Robert Koelling showed that not every pairing is learned so readily. Licking the water spout also set off a click and a flash of light. Rats made ill afterwards came to avoid the taste but not the light and sound; rats given a shock as they drank avoided the light and sound but not the taste. The psychologist Martin Seligman called this readiness to learn some links more easily than others preparedness.

In classical conditioning a signal comes before a reaction the animal cannot help. Much learning runs the other way: an animal acts first, and what follows the act decides whether it is repeated. Edward Thorndike showed this in 1898 with cats that learned to work the latch of a puzzle box. In the 1930s B. F. Skinner built a box in which a rat could press a lever for a pellet of food, each press recorded.

Skinner called this operant conditioning, because the behaviour operates on the world and brings about consequences. A consequence that makes the act it follows more likely is reinforcement; one that makes it less likely is punishment. Both kinds of conditioning build an association, a learned link: classical between two events, operant between an act and its result.

Reinforcement and punishment each come in two forms, and the names trip people up. Positive and negative here say nothing about good or bad. They are arithmetic: positive means something is added after the act, and negative means something is taken away.

So there are four cases. A child praised for tidying up tidies more often: positive reinforcement. A driver who buckles the seatbelt makes the car’s beeping stop, and buckles up sooner each time: negative reinforcement, because taking away something unpleasant strengthens the act. A child who touches a hot pan and feels pain stops touching it: positive punishment. A teenager whose phone is taken away for coming home late starts coming home on time: negative punishment, because taking away something pleasant weakens the act. The commonest mistake is to call negative reinforcement a kind of punishment.

How do you teach an animal something it would never do by chance? In 1943, in a flour mill in Minneapolis, Skinner and two students, Keller Breland and Norman Guttman, tried to teach a pigeon to swipe a wooden ball with its beak. They waited for a swipe they could reward, and none came. So they rewarded anything slightly like a swipe, at first just looking at the ball, and then only movements nearer the real thing. Within a few minutes, Skinner wrote, the ball was bouncing off the walls of the box.

Building a behaviour by rewarding steps that come closer and closer to it is called shaping, and the steps are called successive approximations.

A behaviour need not be rewarded every time. The rule that decides which responses pay is a schedule of reinforcement, and Charles Ferster and Skinner described their effects in a 1957 book. A ratio schedule pays after a number of responses; an interval schedule pays the first response after some time has passed. Either can be fixed or variable. On a variable ratio schedule the reward comes after an unpredictable number of responses, and it draws the fastest, steadiest responding of the four.

Rewarding only some responses also makes a behaviour last longer once rewards stop. This is the partial reinforcement effect. A vending machine that stops working is given up after a try or two, because it always paid before. A slot machine pays out at random, on a variable ratio, so the gambler keeps pulling through a long dry spell: any pull might win.

Must a behaviour be rewarded to be learned? In 1961 Albert Bandura, Dorothea Ross and Sheila Ross showed 72 nursery-school children, aged between three and six, an adult in a room with an inflatable clown, the Bobo doll. For some children the adult spent most of ten minutes attacking it: hitting it with a mallet, sitting on it, punching its nose and shouting “Sock him in the nose.” Others saw a quiet adult, or no adult at all.

Later, playing in another room that also held a Bobo doll, the children who had watched the attack copied much of it, blow for blow and word for word. Those who had not seen it hardly did. Nobody had rewarded them for it. Bandura called this observational learning: learning by watching another person, the model, and doing as they did.

In 1965 Bandura showed children a film of an adult attacking a Bobo doll, with three endings. In one the adult was rewarded, in another punished, and in the third nothing followed. Left with the doll, children who had seen the punishment copied fewer of the attacks.

Then every child was offered juice and stickers for each of the adult’s acts they could show. Now the three groups did about equally well. The children who had seen the punishment had learned the model’s actions as fully as the rest; they had held back from doing them. Bandura later described learning by watching in four steps: attention to the model, retention of what was seen, reproduction of it, and motivation to do it. Reward and punishment work on the last step. Learning is one thing, and doing another.