Reward-Based, Positive Reinforcement, and Clicker-Training

February 8, 2011 § Leave a comment

 

Clicker-trained dogs like Iggy tend to be very attentive.

 

History and Theory: Lures, Rewards, Whistles, Clickers

Contemporary, reward-based animal training has its roots in marine-mammal training. During the mid-twentieth century, people began trying to train captive dolphins and orcas using negative reinforcement, or coercion methods, just as people had been training dogs and horses. The trainers prodded and poked the cetaceans; all of the animals reacted in one of two ways: they would either take a big breath and sink to the bottom of the pool in the center and stay there until they couldn’t hold their breath any longer, or they would swim to the far side of the pool and keep the entire pool between them and the trainer. In no instance did any of the animals actually even attempt to do a behavior in response to being pushed or prodded.

This complete lack of success using coercion methods, combined with the simple practical problem of trying to figure out how to force an orca, or even a dolphin, to jump in the air, or how to punish them for not doing something (imagine hitting a 2,000 pound carnivore with a rolled up newspaper) caused the trainers and biologists to rethink their methods and start from a different place altogether. It took a lot of experimentation, but what they ended up doing worked so well their methods allowed the trainers to teach the animals to do things they hadn’t even considered possible. Most of us have seen, at least on television, coordinated acrobatic displays of dolphins and orcas leaping, flipping, towing humans, even swimming backwards while most of their bodies are out of the water.

 

The trainers taught the animals to do these things by using positive reinforcement and by shaping the performers’ behavior using markers. With marine mammals, the best marker was found to be a whistle, and this is the precursor to the clicker and clicker-training used with dogs, cats, horses and even chickens today. The first step was to cause the cetaceans to identify the whistle as a reward in itself. Like Pavlov with his bells and metronomes, the trainers blew the whistle each time the animals got a fish. Very quickly, the animals associated the whistle with something positive: being fed a fish. This allowed the trainers to use the whistle as an immediate reward (it is difficult or impossible to get a fish into the mouth of a dolphin at exactly the moment it performs a requested action; relatively simple to blow the whistle at just the right moment). The animals quickly learned the sound of the whistle meant fish was coming, just not at that moment, and so the whistle became its own reward, and a marker.

 

To teach a dolphin a simple (for a dolphin) trick like jumping out of the water, the trainers waited until the dolphin did it on its own, then marked that behavior with the whistle and rewarded it with a fish. To teach the dolphin to do a flip, they would lure it by using a fish on a pole. Eventually, the dolphin would do the flip as it followed the fish in the air. When it did, the trainers would mark that with the whistle and reward it with a fish. Very quickly, the tricks got more complicated, and possibly the most amazing part was that other untrained dolphins would mimic the trained ones, hoping for their own whistles and fish.

 

Karen Pryor

 

In 1963, a biologist named Karen Pryor signed on as a dolphin trainer at Sea Life Park in Hawaii. She was an expert dog and horse trainer, who had great success using traditional methods with those animals, but who had to relearn everything when working with cetaceans. When she left Sea Life Park several years later, Pryor tried using the same general principles she had learned in working with sea mammals on her dogs and horses. The effects were immediately obvious, and quite astounding. In one year she trained an adult dog for obedience competition. Conventional wisdom up to that time said that serious obedience competitors needed at least two years, and that starting with an adult dog, instead of an adolescent, was all but a waste of a time. She entered her dog and won, and did it without using any “corrective” techniques or equipment at all: no choke chain, no hitting, not even the use of “No!” To help prove this was not, as many other trainers claimed, a fluke, she taught her methods to another trainer, who used an adult dog with little or no obedience training. This time, it took only six months, and the new trainer and dog won the same competitions Karen Pryor had won the previous year. People began to notice, and asked for more information.

 

Pryor did more experimentation and research, and trained horses, dogs, chickens (turns out chickens are pretty smart, given the chance to learn things) and even people using what she calls “positive reinforcement shaping and training”. She has published many papers and articles, but is best known for her bestselling book, Don’t Shoot the Dog, which lays out the history, principles, and benefits to reward-based, positive-reinforcement training. As with most technological breakthroughs, many people were working on similar theories and ideas at the same time (who really invented the radio, the computer mouse-interface, or the automobile?). It took a while for these general ideas to become mainstream, but now these methods are the norm, while just 20 years ago most dog-training methods involved force and coercion instead of rewards and reinforcement. Don’t Shoot the Dog seems to be the most dramatic and important (but far from the only) catalyst for the movement.

 

Actual Method: Priming

 

To learn clicker training, I recommend working with a professional trainer, or getting a book on the specific topic. The new, revised edition of Don’t Shoot the Dog has a chapter on clicker training, and there are whole books on the subject available at many large booksellers and pet-supply stores. Click for Joy, listed in this site’s blibiography is also a great resource. I will explain how I do it, and I have had some success with it. 

 

The first step is to get the dog (or cat, chicken, what have you) primed for the clicker. This is fun; we get to play Pavlov. It is as simple as putting together a bag of treats and getting a clicker, then feeding the animal the treats, and clicking each time. Timing is important! The click should happen just as the dog puts her mouth on the treat. Wait a few seconds, until the dog has completely finished the treat and put her attention back to you, and then do it again. Ten or so times per session is about right, and do it a few times a day, several days in a row. At the end of a session, pocket the clicker and show your empty hands, saying “That is all!” or “All done!” or something to let your pet know the session is over.

 

It is not essential, but it is helpful to have different kinds of treats for this, and for training in the future as well. I use a combination of small (puppy size) dog biscuits in various flavors, cut up pieces of dog loaf (like Natural Balance Dog Food Roll), and freeze dried liver, heart, and lung. You only need very small pieces for this to be an effective reward. The reason it is better to have different foods in your treat bag is based on a principle which has been shown to apply to most animals and especially people. No matter how much the dog likes their favorite thing, he will get tired of it. And no matter how much they like consistency, randomization is almost always preferred. Apply this randomization to later training, too — it is always best, when convenient. Think about it in terms of a slot machine that always pays something. People blow their life savings on slot machines for a reason — the hoping-for something-great causes us to pull that arm, and deposit our money, again and again. Same with animals! Randomizing their reward causes them to be more interested.

 

Actual Method: Shaping

 

Once the dog recognizes the click as something good, it can be used as a marker for a behavior we like. And the best part is it can be used as a marker for getting close, thus teaching the dog that it is getting the right idea. The first thing I taught my dog, Iggy, to do using a clicker was to catch popcorn in his mouth. First I made a plan. Making a plan is important because it helps avoid confusing the dog. How can we avoid confusing our pets if we are confused ourselves? My goal was for my dog to catch popcorn in his mouth when I threw it to him. My plan was to shape his behavior by rewarding him, at first, for anything that was even vaguely moving in the right direction. I decided this meant at first I would click for him if the popcorn even hit his muzzle (but not any other part of his head). After a few sessions of this, I planned to click only if he attempted to catch it, even if he didn’t succeed. After a few sessions of that, he would only get a click if he actually made the catch. Also important, is that I would not throw the popcorn unless he was in front of me and attentive (looking at me). I decided I would not require him to be sitting, but requiring him to be sitting would have been a reasonable choice.

 

Timing is very important, and I found it challenging to get this right. The click should come at the instant that the behavior happens, so this meant clicking right as the popcorn hit his muzzle, then right as he lunged for it, then right as he caught it. It was harder than it sounds, but I kept trying and Iggy learned this trick in about three days. It is important to limit sessions — how long is appropriate varies with the animal, but 20 minutes would be the maximum for an adult dog who is fairly patient and already somewhat trained. For a cat, I would keep it to 5 at first, and for a puppy I would try 10. Better to end early, than late! It is also always best to end upbeat. I always make the last thing we do in a training session something that Iggy has down cold, like “Sit.” We also begin with things he already knows, to get warmed up. The whole session should be fun; if you get frustrated or pissed, end the session and start again later. All training should be fun!!! 

 

The popcorn trick is a fun one for the dog, because there are a lot of rewards happening, and no downside. If the dog misses the popcorn altogether, he still gets to eat it off the floor. If the dog gets a click because it hit his muzzle, but then eats it off the floor, well great! He is happy — he still gets the popcorn, and he earned a click! With practice, Iggy began to get the secondary part of the reinforcement, though: catching popcorn means the next piece comes faster. He doesn’t have to spend time rooting around on the floor looking for it, or resetting himself for my next throw. As he got better at it, I would only click if he actually caught it — close didn’t count anymore. This is because he now knew the goal, and just needed practice to get good at it. The popcorn is plenty reward!

 

Clicking is Temporary!

 

The clicker is a tool used to teach. It works the same way to simply say “Yes!” or “Good!” at exactly the right moment as it does to click. The reason the clicker is so great is that it is completely consistent, and sounds the same whether or not the trainer is having a bad day, is hoarse from a cold, is depressed, moody, or even if it is a different person training the dog (like a different spouse or sibling). The clicker is also a great tool because the sound is unique, and is not likely to be encountered in the household or out in the world, whereas a dog is bound to hear lots of “Yes!” and “Good!” in its normal course of life. I have taught Iggy lots of things using the clicker, but rarely take one on walks or to the park. He knows how to come to the front, heel, spin etc., and while he learned those things with the clicker, once he knows them, he doesn’t need the clicker anymore! His rewards for these things are either treats (I usually carry a treat bag), affection (he loves to have his chest rubbed), or even verbal praise (Good, good dog! Yes!).  

Sidecar Sidekick

January 21, 2011 § 3 Comments

Strummer in a Mexican pool.

Chris Larson has been my friend for over twenty years, and while I do not see him often, it is always interesting to hear what he is doing. Chris has a special excitement about life. In preparing this article, and reflecting on my friendship with Chris, I realized that I have never known him to be in a bad mood. This is one of the many qualities that make him such a special dog person. Chris grew up on Capitol Hill in Seattle, and still lives there today. When Chris is not hanging out with his dog, Strummer, or girlfriend, Stephanie, he coaches high school soccer, and is the owner of Chris Larson Construction.

Tell us about your dog. Strummer is a 4.75 yr. old female chocolate lab. She weighs 43 lbs. (Can you say runt?) Whatever, we never wanted a show dog, just a friend! She came from a farm in Enumclaw, WA. We specifically looked at litters with small parents, with the idea that we’d get a small dog!

What is the source of her name? She is named after Joe Strummer, from The Clash. Love of The Clash has been a theme in my relationship with Stephanie, my girlfriend of almost 10 yrs.

Out for a ride on a hot day.

Tell us about your ride; it is very unusual. It’s a 1970 Chang Jiang 750. They were produced by the Chinese military until the late ’90’s. Originally designed by BMW in 1938, then the factory tooling was stolen by the Russians during WWII. The Russians gave the design and factory parts to the Chinese. It was used extensively by the People’s Liberation Army.

How did you train Strummer to ride with you? I started training her for the sidecar about a month before I picked it up. I started by putting my ski goggles on her and not allowing her to take them off. When I got the bike home I brought her out to the garage while I worked on it. When she was comfortable with the sound of the motor and vibration I turned the bike off and put her in the sidecar. I continued to work on the bike with the engine off making sure to jostle the rig as much as possible. Once I felt she was ready I fired the motor and she jumped out. I lead her back into the sidecar and continued doing other things and left the motor running. I then revved the motor and she jumped out again. I led her back into the sidecar and held her collar while I made her watch me operate the throttle then she figured it was me causing all the racket and chilled out.

Strummer in full regalia.

Do you use any special equipment to keep her safe? We bought a harness, and I made a 3 point restraint system for the sidecar. She can’t be bounced out of the chair! After all that I had a motorcycle jacket elbow pad sewn into a child size aviator hat, and the rest is history. She never fusses with the safety gear. There are inherent dangers with motorcycling, so I never allowed it to become an excitement or game.

Where all have you travelled with Strummer? Our longest motorcycle trip was an overnight camping trip to Orcas Island, We took her to Vashon Island, but mostly it is just around town or day trip/picnics with Mama Stephanie. We have taken her to Mexico twice and Whbistler, but not on the bike. Commuting to the job sites with her on the bike is a blast!

Atop Mt. Consitution on Orcas Island.

Can you tell if she likes riding in the sidecar? She’s pretty used to it, and sometimes just goes to sleep! She’s had her picture taken so many times she just ignores it now. It doesn’t matter what we do, as long as are all together, Strummer is having a blast! She stays in the sidecar when we go in restaurants and stores and such. I unhook her and take off her gear a she just chills. I’ll also toss my coat down for her outside just about anywhere and she’ll use it as a bed.

The family that plays together...

Does Strummer have other training? As far as training is concerned, she does all the command stuff and is rarely on a leash. I have Steph to thank for that; she’s a natural dog trainer. The only actual trick Strummer does is high five.

Chris, Stephanie, and Strummer in Mexico.

What other activities do you enjoy with Strummer? We run (w/ Mama almost every day, and me less so), hike, fetch (she can’t get enough of the tennis ball. She has them stashed all over Capitol Hill), swim (webbed toes you know), swim and fetch together… and she loves traveling to Whistler and Mexico with us. I can’t imagine my life without my girls: Stephanie, Strummer and Chloe (the cat). I am richer because of them!

A happy trio.

Out for a Walk, or Off to War?

December 29, 2010 § 3 Comments

This video is meant to illustrate one way to learn a behavior called “loose-leash walking”. There are many ways to learn this — if you have a method that is already working well for you, and doesn’t involve hurting your dog in any way, then cheers! I call the game that we play to teach this “leash dancing”. We enjoy giving our dogs some treats, and find it makes teaching easier and faster, but if you are opposed to this, then substitute affection or tug-time or whatever your dog’s assigned currency might be. While playing the leash dance game, it is important not to yank on the lash or get into a pulling contest – leash dancing is a team sport, not a contest of individual will power.

Before leaving the house, before getting fed, before getting a rub-down or anything else really good, the rule with my dogs is that they must be calm. There is no correction, no rebuke, no real negative consequence to exhibiting excitement – they just don’t get the good thing they know could be theirs with a more appropriate demeanor.  A little excitement is fine — what I object to is out of control behavior: jumping up, barking, spinning, or running around the house. If the dog does this kind of thing, just relax and wait by the door, leash in hand. Relax. Break out a book. Take long, exaggerated breaths. It may take awhile at first, but soon the dog will come to you and be relatively calm. Reward them by putting on the leash, and giving them a little treat.  I don’t require the dogs to sit at this stage; I just require them to be more or less stationary. Iggy taught himself within a couple weeks to sit down and look up when he sees the walking collar come out.  Ten months in, and Frankie comes up calmly, but doesn’t sit. That is all right with me – it is the calmness I really want.

A sidewalk will work, but a running track, a parking lot, or a field is best because there is more room and fewer distractions. After working on the leash dance game for a while – maybe 5 minutes (more if it is going well, or less if it is not) — walk the dog a couple hundred feet and let it sniff/pee/snuffle, whatever. Then go back to the leash dance for another session. Alternate 5-minute leash dancing sessions with sniffing opportunities for 30 or so minutes, but only so long as it remains fun for you and the dog.

Wait for Me, or Stay There!

December 22, 2010 § 1 Comment

Separating the two cues wait and stay is not an absolute necessity, of course. Many will just have one standard stay cue to mean “remain where you are and in that position until I release you or tell you to do something else.” This is an adequate way of doing things, but separating the two concepts will usually make each much stronger. 

After a quick display of the dogs playing on cue, this video shows a solid stay, then demonstrates wait as a cue to remain at the current location, in the current position, and with attention on the handler.  If the dog is standing, then it is all right for them to sit or lie down, but if already prone or semi-prone, then they should remain in that position.  Further, wait may be broken or modified by a recall or position- change (such as from sit to lie down) cue from a distance. Wait is different from stay in that stay will only ever be released or transitioned from within touching distance. When on stay, the dog’s attention may wander from the handler, as the dog may learn through repetition that they will be staying put until the handler is within touching distance. When on wait, the dog’s attention should remain on the handler, as a cue or release will be coming soon.

A good way to teach these is to begin with wait. Put the dog in a sit or down position, then turn your hand parallel to the ground with the forefinger and middle finger extended and give the cue wait.  When first teaching this, over-enunciate, and lengthen the vowel sound, “Waaaaaiiiiiiiit.” As in all training, speak clearly but gently and with a positive tone.  The first threshold is just one second and one step backward, then two seconds and two steps, then five, ten, etc. It is helpful to randomly alternate between returning to the dog and clicking and treating or calling the dog to the handler. Once the dog can maintain the wait steadily for a few seconds while the handler is a several feet away, increase the challenge by walking around the dog halfway, then returning to the dog’s front. Once this can be done reliably, walk a full circle around the dog. Gradually extend the distance between the handler and the dog, alternating between calling the dog and returning to the dog.

Learn stay in much the same way, but only use stay for longer periods (like a minute or more, and building up to 30+ minutes), only release or transition the dog from within touching distance. The hand signal is an open hand, palm forward, like a traffic-cop’s “STOP” signal.

Examples: cue the dog to wait when preparing to open a car door, or when feeding, or any time the dog may be called to the handler. Cue the dog stay only when the handler will be returning to the dog, and for longer periods, or when the handler will be out of sight from the dog. 

Preceding the stay and wait demonstrations, this video shows the handler cueing the dogs to play.  We didn’t really teach this in a formal way, it is just a gesture and a word that we said again and again when the dogs were encouraged to run around and be crazy – now they know the gesture and the word as a signal to start frolicking. This is a good way to help them burn off some energy, as many dogs won’t really exercise unless prompted to (which is why leaving a dog in a backyard, even a big yard, is not an adequate way to get the dog exercise).

Lie Down and Be Calm for Meals

December 5, 2010 § 4 Comments

Lie Down and be Calm for Meals

This behavior actually incorporates several tricks, and also requires the dog to exhibit a calm demeanor. This was relatively easy to achieve with Iggy, but a little challenging when working with Frankie. Frankie was slightly possessive of his food, generally more excitable than Iggy, and had a hard time focusing when he knew it was meal time. Both dogs were trained and guided for this behavior in the same way, but Iggy did not need some of the steps, so I will describe how it went with Frankie.                                                                                                                                                                                                                                                                           The first day Frankie came to live with us, following a long, vigorous walk (exercise is critically important for achieving calmness), I fed him by hand. After mixing his dry food with half a teaspoon of  canned pumpkin (helps regularity) and a teaspoon of wet food, I mashed together little bits of it and fed him off the palm of my hand. This prevents accidental nips on the end of one’s fingers, but still associates the person and the person’s hand with food: we become recognized as the provider of something good.

After a couple days of hand-feeding, I divided his meal into four portions and put one in his bowl, then as he finished that, I added more, making sure he was respectful of my hand near his food and mouth. This reinforces that people near his food bowl are the source of good things (more food), not bad things (taking away his food).  In separate sessions, I also started teaching Frankie three specific tricks: sit, wait, and lie down. Training these will be the subject of a different post, but as in everything when teaching a dog, it is helpful to exercise them first, so they are more able to focus and relax. 

As soon as Frankie was comfortable with the general concepts of “sit”, “wait”, and “lie down”, we began working them into meal time. He learned sit before wait, and wait before down, so the progression went like this:  before setting down his bowl, Frankie would sit, and then receive his meal.  After he learned the general concept of “wait”, he was asked to sit, and then asked to wait; his food was put down, but he was required to wait until released, or the process started over again. No correction or punishment was given: the reward was his meal and the consequence for not waiting properly was the delay of his meal.

Frankie was good at lying down and waiting when we practiced as part of our training routine, and he was good at sitting and waiting for his food, but he seemed unable to control himself enough to lie down and wait at mealtime.  The problem turned out to be not the inability to resist the food, but the coldness of our tile floor.  It was October when we were putting all these steps together and the kitchen floor was quite cold. Frankie has very little hair on his underside.  A yoga mat was sacrificed, and Frankie became a very polite diner.

Reward for Elimination

December 4, 2010 § Leave a comment

When my dogs poop I reward them with a single tasty treat. This encourages them to eliminate early in our walk or outing. There is a natural tendency for people to walk a ways and then turn back as soon as our dogs do their business. This is a bad habit to form, because the dogs will pick up on it and postpone going until they have walked a long distance. I make sure that once they go, I reward them with a morsel, and then we keep moving in the direction we were going for at least a block or so (or much longer if they do it early in the outing). It took a few weeks for Iggy to pick up on the connection, but now he goes within a block or two of leaving the house, then sits nicely for his treat and relaxes until I have gotten his mess cleaned up. Frankie still prefers to go in his own backyard, which is also just fine — I reward him there and I expect that over time he will make the connection and then will go on our walks. 

I have also found that a little pumpkin (Tblsp for large dogs, tspn or less for smaller ones) with each meal helps keep them regular and on schedule. They didn’t like it a lot at first, but they got used to it within a couple days and now seem to really enjoy the sweet taste.                                                                                                                                                                                                                                                                                                      

A few weeks ago we had very cold weather and snow here, and neither of my dogs revels in the cold — once Iggy went, he would actually turn in the direction of home and request to go back; often this occured within a block of our house. Frankie would go in the back yard and be ready to come back in within 2-3 minutes.

Target Stick and Platform Practice Video

December 3, 2010 § Leave a comment

Where Am I?

You are currently browsing the Training Specific Behaviors category at Positive Canine Guidance.