Hard to Break: Why Our Brains Make Habits Stick 9780691219837

Citation preview

hard to break

Hard to Break why our brains make habits stick

russell a . poldr ack

p r i n c e to n u n i v e r s i t y p r e s s p r i n c e to n & ox f o r d

c 2021 by Princeton University Press Copyright  Requests for permission to reproduce material from this work should be sent to [email protected] Published by Princeton University Press 41 William Street, Princeton, New Jersey 08540 6 Oxford Street, Woodstock, Oxfordshire ox20 1tr press.princeton.edu All Rights Reserved Library of Congress Cataloging-in-Publication Data Names: Poldrack, Russell A., author. Title: Hard to break : why our brains make habits stick / Russell A. Poldrack. Description: 1st. | Princeton : Princeton University Press, [2021] | Includes bibliographical references and index. Identifiers: LCCN 2020049215 (print) | LCCN 2020049216 (ebook) | ISBN 9780691194325 (hardback) | ISBN 9780691219837 (ebook) Subjects: LCSH: Habit. | Human behavior. | Cognitive psychology. | Neurosciences. Classification: LCC BF335 .P65 2021 (print) | LCC BF335 (ebook) | DDC 152.3/3—dc23 LC record available at https://lccn.loc.gov/2020049215 LC ebook record available at https://lccn.loc.gov/2020049216 British Library Cataloging-in-Publication Data is available Editorial: Hallie Stebbins and Kristen Hop Production Editorial: Natalie Baan Jacket Design: Amanda Weiss Production: Jacquie Poirier and Danielle Amatucci Publicity: Kate Farquhar-Thomson, Sara Henning-Stout, and Katie Lewis Copyeditor: Jennifer McClain This book has been composed in Arno Printed on acid-free paper. ∞ Printed in the United States of America 10 9 8 7 6 5 4 3 2 1

contents

List of Illustrations

ix

Acknowledgments xiii part i. the habit machine : why we get stuck 1

2

1

What Is a Habit?

3

The Poet of Habits

4

The Zoo of Habits

5

Habits and Goals

7

Why Do We Have Habits?

10

Understanding Behavior

11

A Road Map for Understanding Habits and Behavior Change

12

The Brain’s Habit Machinery

16

A System for Conscious Memory

19

Enter the Lizard Brain

23

What Are the Basal Ganglia?

26

Dopamine: It’s Complicated

30

Dopamine and Brain Plasticity

34

What Does Dopamine Mean?

36

What about Pleasure?

40

Selecting Actions in the Striatum

42

v

vi

3

contents

Once a Habit, Always a Habit

45

Old Habits Never Die

46

The Transition to Mindlessness

48

Becoming One: Habits as Chunked Actions

50

Trigger Warning: How Cues Trigger Habits

52

Can’t Look Away: Rewarding Stimuli Capture Attention

55

A Recipe for Stickiness

58

4 The Battle for Me

60

A Competition in the Brain?

61

Memory System Interactions in Humans

64

Formalizing the Goal-Habit Distinction

66

Model-Based versus Model-Free Reinforcement Learning

72

Can Goals become Habitual?

77

Self-Control: The Greatest Human Strength?

81

What’s Up Front?

83

Why Is the Prefrontal Cortex Special?

86

Holding Information in Mind

90

The Waiting Is the Hardest Part

97

Now or Later?

102

Two Minds in One Brain?

106

Controlling Our Impulses

110

Stopping Ourselves

114

The Rise and Fall of Willpower

119

6 Addiction: Habits Gone Bad

123

5

The Intoxicating Allure of Drugs

123

“This Is Your Brain on Drugs. Any Questions?”

126

c o n t e n t s vii

The Transition from Impulse to Habit

130

Stress and Addiction

134

Is Addiction Really about Habits?

137

“My Drug of Choice Is Food”

138

Digital Addiction?

144

Why Do Only Some People Become Addicted?

147

part ii. coming unstuck: the science of behavior change 7

8

151

Toward a New Science of Behavior Change

153

Behavior Change as a Public Health Problem

153

A New Science of Behavior Change

155

A New Approach to Behavior Change

157

Targets for Intervention

159

Planning for Success: Keys to Successful Behavior Change

161

The Architecture of Choice

161

Loss Aversion and Framing

163

Make Rules, Not Decisions

165

Trigger Warning: Intervening on Habits

166

Mindfulness: Hype or Help?

170

Can Self-Control Be Boosted?

171

Training Inhibition

174

Envisioning Change

175

Summing Up

177

9 Hacking Habits: New Tools for Behavior Change

179

Can Bad Habits Be Erased?

179

“I Forgot That I Was a Smoker”

185

viii

contents

Optogenetics in Humans?

187

A Neurochemical “Goldilocks Zone”: Drugs to Improve Executive Function

189

Toward Personalized Behavior Change

190

10 Epilogue

194

Summing Up

195

From Individual to Societal Change

198

Notes 201 Index

213

i l lu st r at i o n s

Figures 1.1 An operant conditioning chamber (or Skinner box)

8

1.2 A schematic for understanding the various factors that go into a choice

12

2.1 MRI scan of the author’s brain

19

2.2 An example of mirror-reversed text

21

2.3 A schematic of the different parts of the basal ganglia, and a graphic that shows the position of the different portions of the striatum

27

2.4 A map of basal ganglia circuitry, showing direct and indirect pathways

28

2.5 Image depicting neurons that produce dopamine within the substantia nigra pars compacta (SNc) and the ventral tegmental area (VTA), with their outputs sent widely across the brain, particularly to the striatum

31

2.6 A schematic of the three-factor plasticity rule

35

2.7 The demonstration of reward prediction error signaling by dopamine neurons

37

4.1 A photo of a plus maze and a schematic of training and testing in Packard’s experiment

62

4.2 An example of a reinforcement learning model using slot machines

70

ix

x

l i s t o f i l l u s t r at i o n s

4.3 Performance of the reinforcement learning model, based on the slot machine example of Figure 4.2

71

4.4 An example of Daw’s two-step learning task

75

4.5 A schematic example of a task used to examine the model-free selection of goals

79

5.1 Photographs of Phineas Gage’s skull and a reconstruction of Gage’s brain injury

84

5.2 The hierarchy of brain systems, with primary systems, unimodal association regions, and heteromodal association regions, and the prefrontal cortex at the top

87

5.3 A schematic of the oculomotor delayed response task used by Goldman-Rakic

91

5.4 A schematic of the “inverted-U” relationship between arousal and performance, first described by Yerkes and Dodson

95

5.5 Examples of discounting functions for two individuals, one who discounts quickly and the other who discounts more slowly

103

5.6 A summary of active brain regions in 99 published studies that mentioned the stop-signal task in the abstract of their publication

117

6.1 A schematic of the brain’s stress systems

135

7.1 Relapse curves showing the percentage of people trying to quit various substances who remained abstinent at various points up to one year

155

9.1 The process of consolidation in solidifying memories, and the blocking of reconsolidation that can result in a loss of memory

183

Boxes 2.1 Excitatory and inhibitory neurons

29

2.2 Controlling the brain with light

33

l i s t o f i l l u s t r at i o n s xi

2.3 Calcium imaging

39

3.1 DREADDs

56

5.1 Imaging white matter using diffusion-weighted imaging

90

5.2 Genome-wide association studies

105

5.3 Brain stimulation

109

5.4 Why small studies can be problematic

112

6.1 Gene regulation and epigenetics

127

6.2 Positron emission tomography

129

8.1 Combining research studies using meta-analysis

172

ack n o w l e d g m e n t s

i’d first like to thank several colleagues who were kind enough to read and provide detailed comments on an early draft of the entire book or large sections of it: Peter Bandettini, Aaron Blaisdell, Julia Haas, David Jentsch, Colin Klein, Trevor Robbins, and Luke Stoeckel. Their honest and detailed feedback was crucial in helping me to reorganize the book in a more effective manner and to get the science right in areas where I am less expert. I would also like to thank a number of colleagues and friends for commenting on early drafts of sections of the book, answering technical questions or providing useful discussion: Amy Arnsten, Joshua Berke, Patrick Bissett, Kyle Burger, Fiery Cushman, Nathaniel Daw, Angela Duckworth, Paul Fletcher, David Glanzman, Kevin Hall, Rob Malenka, Earl Miller, Lisbeth Nielsen, Amy Orben, Paul Phillips, James Proud, Bill Savoie, Tom Schonberg, Katerina Semendeferi, Mac Shine, Eric Stice, Dan Tranel, Uku Vainik, Kate Wassum, and David Zald. My editor at Princeton University Press, Hallie Stebbins, also deserves special thanks. Her input on multiple drafts helped me craft a book that is much better than it would have otherwise been. My greatest thanks go to my wife, Jennifer Ott, who has put up with my workaholism for many more years than either of us would like to admit. Her love and support has given me the gumption to keep going even when the words refused to flow.

xiii

pa rt i

The Habit Machine why we get stuck

1 What Is a Habit?

think for a moment about your morning routine. Mine involves walking downstairs from my bedroom, turning on the espresso machine, putting together my breakfast (plain yogurt, blueberries, and nuts), and firing up my laptop to check email, social media, and news. What is so remarkable is that we can perform these kinds of routines without actually thinking about what we are doing—very rarely do I actually entertain conscious thoughts like “now I need to take out a spoon and scoop the yogurt into the bowl” or “now I need to walk from the refrigerator to the counter.” When people think of habits, they often jump immediately to “bad habits,” like smoking, drinking, or overeating, or “good habits,” like exercise or brushing our teeth. However, these are just the visible tip of a huge iceberg of habits that each of us has. And if you think a bit about what life would be like without them, it’s pretty clear that we would quickly succumb to decision paralysis. In his moving book The Emperor of All Maladies: A Biography of Cancer, Siddhartha Mukherjee describes how we should not think of cancer as something separate from our bodies, because in fact it is a reflection of exactly the biological functions that keep us alive: Cancer, we have discovered, is stitched into our genome. . . . Cancer is a flaw in our growth, but this flaw is deeply entrenched in ourselves. We can rid ourselves of cancer, then, only as much as we can rid ourselves of the processes in our physiology that depend on growth, aging, regeneration, healing, reproduction. (p. 462, Kindle edition) 3

4

chapter 1

We should think of habits in much the same way. We will see how the stickiness of habits can make behavior very hard to change, but it is exactly this stickiness that makes habits essential for navigating our complex world so effectively.

The Poet of Habits William James was the first great American experimental psychologist. Whereas his brother Henry James is renowned as one of the greatest American novelists, William James stands as one of the greatest thinkers ever to have written about the human mind. In his 1890 book Principles of Psychology,1 James wrote what remains one of the most compelling descriptions of habits and their importance, providing a particularly striking picture of just how essential habits are to our everyday lives: The great thing, then, in all education, is to make our nervous system our ally instead of our enemy. . . . For this we must make automatic and habitual, as early as possible, as many useful actions as we can. . . . There is no more miserable human being than one in whom nothing is habitual but indecision, and for whom the lighting of every cigar, the drinking of every cup, the time of rising and going to bed every day, and the beginning of every bit of work, are subjects of express volitional deliberation. Full half the time of such a man goes to the deciding, or regretting, of matters which ought to be so ingrained in him as practically not to exist for his consciousness at all. (p. 122, emphasis in original) For James, the idea of “habit” was defined at its core in terms of automaticity—that is, the degree to which we can perform an action automatically when the appropriate situation arises, without consciously entertaining the intention to do it. Automaticity often only becomes apparent when it makes us do the wrong thing. Nearly all of us have had the experience of intending to make an unusual stop on the way home from work (the dry cleaners is a common example), only to get home and realize that we forgot to make the stop, because our behavior was carried by the automatic habits that we have built up over driving

w h at i s a h a b i t ? 5

the same route many times. Just as cancer is the dark side of our cells’ mechanisms for growth, errors like these are the flip side of our usually safe reliance on habits. James’s notion of making our nervous system “our ally instead of our enemy” becomes particularly clear when we acquire a new skill, by which we mean a highly tuned ability that we can perform without effort—very similar in fact to the concept of a habit. Nearly every aspect of our interactions with the artifacts of our world, from driving a car or riding a bicycle to using a computer keyboard or smartphone touchpad, involves skilled behaviors that develop over a long period of time. Perhaps one of the most unique human skills is reading. Written language has only existed for about 5000 years, a tiny portion of the evolutionary history of humans, and while nearly all humans learn to understand and speak their native language with seemingly no effort, reading is a skill that takes years of education and practice to acquire. However, the skill of reading becomes automatic once it is acquired, in the sense that we can’t help but process the meaning of text that we see. The automatic nature of reading is seen in the well-known Stroop effect, in which a person is shown words written in colored ink and asked to name the ink color as quickly as possible. If we compare how long it takes to name the color of a word when the text matches the color (for example, “red” written in red) versus the same color with a different word (“blue” written in red), we see that people are invariably slower to name the color when it mismatches the word—which means that even when the written language is irrelevant or even harmful to the task at hand, we can’t help but read it. In this way, skills are often very similar to habits in that they are executed automatically without any effort or awareness. As we will see in the next chapter, this relationship between habits and skills has played a central role in our understanding of the brain’s systems that support both habits and skills.

The Zoo of Habits If habits are truly a fundamental aspect of our mind’s functioning, then we should expect to see them everywhere we look, and indeed we do. Each of us has a large number of routines—that is, complex sets of

6

chapter 1

actions that we engage automatically in a particular context, often daily but sometimes more infrequently. We make coffee in the morning, we drive a particular route to work, we set the table before dinner, and we brush our teeth before going to bed. Although each of these actions serves a particular purpose, very rarely do we consciously think about our goal as we do them, or even about the fact that we are doing them at all. The mindless nature of these routines is at odds with a longstanding idea in psychology that our actions are driven primarily by our goals and beliefs.2 However, research by psychologists Judith Ouellette and Wendy Wood has shown that many routine behaviors (especially those we engage in daily) are better explained in terms of how often they have been done in the past (that is, the strength of the habit) rather than in terms of goals or intentions.3 While routines can truly make our brain “our ally instead of our enemy” as James proposed, other habits often seem more like mindless responses to a particular cue or situation. Sometimes these don’t seem to serve any apparent goal at all, as when a person chews their fingernails or twirls their hair. In other cases, as when we devour a bowl of popcorn on the couch while watching a movie, the action seems to be in service of a goal, but again our intentions don’t seem to come into play, and we often realize that we have eaten much more than we would have ever intended. As we will see below, the idea that habits become detached from goals or intentions is one of the central concepts behind our knowledge of how habits work, and we have an increasingly deep understanding of how this comes to be. The habits we have discussed so far all involve physical actions, but it’s important to point out that we can also have habits of mind. My wife and I, having been together almost 30 years, will often find that we end up thinking exactly the same thing in particular situations, or finishing each others’ sentences when telling a story. Our shared experience over decades has led us to develop a set of shared mental responses to common situations. In other cases, habits of mind can become deeply disruptive, as when individuals suffering from obsessive-compulsive disorder become disabled by particular thoughts that they cannot keep out of mind.

w h at i s a h a b i t ? 7

Finally, emotional responses to particular situations can also become habitual. For example, many people develop an intense fear reaction to the prospect of speaking in public, as I did early in graduate school. Just as habitual actions are triggered by particular situations, the psychological and physical responses that occur in a phobia can be thought of as an “emotional habit.”

Habits and Goals While the gamut of habits thus spans from action to thought, most of the research on habits has focused on relatively simple actions. Further, while our interests are ultimately in understanding how habits work in humans, much of the research I discuss has been carried out in species other than humans, particularly in rodents (rats and mice). This is in part because creating new habits in the laboratory in humans is just plain difficult due to the amount of time and experience that is required; because rats live in the laboratory, they can be exposed to training for hours each day. In addition, our scientific interests are often focused on “bad” habits, such as substance use or overeating, but it would not be ethical to give a human a new bad habit for research purposes. Fortunately, the organization of the rodent’s brain is similar enough to the human brain that there is much to be learned from studying them, though we always have to keep in mind that there are differences as well. In addition, rodents are useful species in which to study habits, because they are relatively single-minded, at least when a member of the opposite sex is not present: they just want to eat. More recently, an increasing amount of research has been done using mice instead of rats because of the ability to use powerful genetic tools for dissecting and controlling brain function that are more readily available for mice than rats, which I describe later in the book. One standard way that rodents are studied is to put them in an operant conditioning chamber (Figure 1.1), often called a “Skinner box” after the psychologist B. F. Skinner who popularized them for studying how rats learn. The box has a way for the animal to respond (usually a lever

8 chapter 1

figure 1.1. A rat poking its nose in an operant conditioning chamber (better known as a Skinner box). (Photo courtesy of Aaron Blaisdell)

that they can press or a port they can poke their nose into), along with a food dispenser that can drop pellets of food for the animal to eat. The box is configured so that a certain number of presses of the lever (or alternatively, presses within a certain amount of time) result in food being dropped. Rodents will fairly quickly learn to press the lever in order to obtain food, and this is the basis of many of the studies that have been done to examine how habits are learned. Let’s say that a researcher trains a rat to press a lever to obtain food over many days, so that when they are put into the box they immediately start pressing. How would we know whether this behavior is a “habit”? One influential answer to this question was provided by the

w h at i s a h a b i t ? 9

psychologist Anthony Dickinson of Cambridge University. According to Dickinson, there are two reasons that a rat might continue to press a lever once it has learned to do so. On the one hand, the rat might be pressing the lever because it has in mind the goal of getting some food, and it knows that pressing the lever will obtain the reward; because this behavior is directly in service of a goal, Dickinson called it goal-directed action. On the other hand, the rat may simply press the lever because that’s what it has learned to do when placed in the Skinner box, even if it doesn’t have the goal in mind. This is what Dickinson refers to as stimulus-response, or habitual behavior. Based on this distinction, Dickinson devised a clever way to determine whether a rat had a goal in mind when it was pressing: eliminate the value of the goal and see whether the animal continues performing the behavior. For example, let’s say that the reward is a pellet of rat chow. We can devalue the reward by feeding the rat a bunch of chow just before we put it into the Skinner box, so that it’s sick of that particular food. If the rat no longer presses the lever after having been satiated, then we can be sure that its lever pressing is done with the goal in mind. On the other hand, if the rat continues to press the lever even when it doesn’t want the chow anymore, then we can be sure that the lever pressing is a habit, which for Dickinson means that it is an action that is evoked by a particular stimulus (in this case, the presence of the lever) without any goal in mind. What Dickinson and his colleagues found was that early in the process of learning, the rats behaved as if they were goal directed: when the reward was devalued, the rats stopped pressing the lever. However, with additional training, the rats’ behavior became habitual, such that they continued to press the lever even though they didn’t want the reward. This transition from early reliance on goal-directed control to later reliance on habitual control is a pattern that we will see repeatedly in our examination of habits.4 Thus, habits differ from intentional, goal-directed behaviors in at least two ways: they are automatically engaged whenever the appropriate stimulus appears, and once triggered they are performed without regard to any specific goal. Now let’s ask why evolution would build a brain that is such a habit machine.

10 c h a p t e r 1

Why Do We Have Habits? It’s easy to forget that many aspects of the world that we inhabit are remarkably stable. The laws of physics remain the same from day to day, and the structure of the world also remains largely consistent— your friends don’t start speaking a new language to you out of the blue, and the steering wheel on your car works pretty much the same way every day. On the other hand, there are aspects of the world that change from day to day, such as the particular spot where a person parks their car, or the weather they need to dress for that day. Other aspects of the world are consistent in our local environment but differ in other environments; for example, when I drive a car in the US, I need to drive on the right side of the road, whereas if I were to drive on a trip to the UK, I would need to drive on the left. Our brains are thus stuck on the horns of a tricky dilemma. On the one hand, we would like for our brain to automate all the aspects of the world that are stable so that we don’t have to think about them. I don’t want to spend all of my time thinking “stay in the right lane” when I am driving my car at home in the US, because that’s an aspect of my local world that is very stable. On the other hand, when things change in the world, we want our brain to remember those things; if a particular road is closed for construction, I need to remember that so that I can avoid it on my way to work. An even more challenging wrinkle is that the brain isn’t told which things are stable and which are changing— it has to learn this too, and in particular it needs to make sure that we don’t change too quickly. For example, if I were to drive a car in England on vacation for one day, I wouldn’t want to come home with my brain rewired to drive on the left side of the road. The computational neuroscientist Stephen Grossberg coined the term “stability-plasticity dilemma” to describe this conundrum: How does the brain known how to change at the right time without forgetting everything that it knows? In Chapter 3 I delve much more deeply into how habits are an essential aspect of the brain’s solution to the stability-plasticity dilemma and how this relates directly to the stickiness of habits. The basic strategy that evolution has used to solve the dilemma is to build multiple

w h at i s a h a b i t ? 11

systems into the brain that support different types of learning. The psychologists David Sherry and Daniel Schacter proposed that these separate brain systems evolved because they were needed to solve a set of problems that are “functionally incompatible”—that is, problems that simply cannot be solved by a single system. They argued that the brain’s habit system evolved to learn those things that are stable (or invariant) in the world, whereas another memory system (known as the declarative memory system) evolved to allow us to learn the things that change from moment to moment. The habit system lets us learn how the pedals on the car work (which usually never changes), while the declarative memory system lets us remember where exactly we parked our car today (which changes from day to day). In the next two chapters, I go into much more detail about how these systems work in the brain and how they relate to one another.

Understanding Behavior Any particular choice or action that we make belies a massive amount of computation going on in our brain. Because I spend much of this book discussing the various factors that drive our behavior, it’s useful to have a framework in place for understanding how we behave. Figure 1.2 shows a schematic that guides the organization of this book. Everything we do is influenced by our environment, which allows some kinds of choices and forbids others, and also presents us with stimuli that can trigger our desires and habits. As we will see Chapter 8, many of the most effective ways of changing behavior involve changing the environment. Once we are ready to make a choice, there are several factors that can influence our decision. First, we have our long-term goals—what do we want to do in the future? Second are our immediate desires. These are the things that we want right now, without regard to how they align with our long-term goals. Finally, we have our habits. These are the behaviors that we have learned through experience and that we automatically engage in without thinking. To make this concrete, let’s say that I am attending a party at a colleague’s house, to which I drove my own car, and my colleague offers

12 c h a p t e r 1

Long-term goals

Environment

Habits

Behavior

Immediate desires

figure 1.2. A schematic for understanding the various factors that go into a choice.

me a cocktail. I like cocktails, and my immediate desire is to say, “Thanks, I’d love one.” However, I have the longer-term goal of remaining sober so that I can drive home (which relates to my even longerterm goals of avoiding accidents and staying out of jail), which would lead me to decline the cocktail and drink something more goal relevant instead, such as a glass of water. However, depending on my experience, I might have a habit of drinking cocktails at parties and could find myself with a cocktail in my hand despite my long-term goals. As we will see, all of these different components of a choice are important to understand how we can more effectively change behavior.

A Road Map for Understanding Habits and Behavior Change This book is broken into two parts. The first part, “The Habit Machine,” outlines what exactly scientists mean when they refer to a “habit” and where habits come from in the brain. Different scientists define habits in different ways, but most agree on a few basic characteristics. First, a habit is an action or thought that is triggered automatically by a particular stimulus or situation—it doesn’t require any conscious intention on our part. Second, a habit is not tied to any particular goal; rather, habits

w h at i s a h a b i t ? 13

are engaged simply because of their trigger. This is important, because it means that the habit persists even if the reward that created it is no longer present. Third, habits are sticky: they come back despite our best efforts to suppress them, often when we are at our weakest point. In the next chapter, I turn to describing the brain systems that underlie habits and how they relate to other kinds of learning and memory. Here we will first see that the systems in the brain that underlie the learning of habits are distinct from the systems that help us form conscious memories for the past. We will also have our first encounter with the neurochemical that might be viewed as either the star or the villian of the habit saga, depending on your perspective: dopamine. In particular, we will see how dopamine plays a central role in strengthening actions that lead to reward, ultimately setting the stage for the development of habits. In Chapter 3, I turn to the research on why habits are so sticky. Here we will see that a number of different features of habits conspire to make them particularly persistent. On the one hand, habits become increasingly unitized over time; what was once a set of actions that each required our conscious attention and effort becomes a single unit of action that requires little added thought or intention. On the other hand, the triggers for those habits become increasingly powerful and increasingly draw our attention. Together these mechanisms provide a recipe for behaviors that become very difficult to change. In Chapter 4, I discuss how the different memory systems in the brain work together to let us behave in an intelligent way. Here we will see that our behavior arises from a competition between different learning systems in the brain. I also dive deeply into explaining one of the best-accepted theories that describes the computations the brain performs in order to learn new habits, known as reinforcement learning. We will see how different forms of reinforcement learning can give rise either to habits or to planful (goal-directed) behavior. I also describe how higher-level goals can become habitual, moving beyond simple action habits to more complex kinds of habits. When many people think of habits and why they are so hard to change, their mind often turns immediately to the ideas of self-control

14 c h a p t e r 1

and willpower, which I explore in Chapter 5. This story centers heavily around the brain’s prefrontal cortex, which is the center that helps us resist immediate temptations and instead behave in service of longerterm goals. There are actually several different facets of self-control, which rely upon somewhat different systems in the brain. I also discuss the concept of willpower, which you will see plays a very different role than our intuitions would lead us to expect. The most serious and often tragic impact of habits is often seen in addictions, which I turn to in Chapter 6. It is no accident that all drugs of addiction cause unnaturally strong activation of the dopamine system, given its central role in habit formation. Beyond drugs, I also discuss the idea that one can become addicted to food or digital devices. I also discuss some recent neuroscience research that sheds light on the interesting question of why some drug users become addicted but many others do not—research that suggests that the answer may lie in a biological luck of the draw. The second part of the book, “Coming Unstuck,” focuses on what science tells us about how to change behavior most effectively, realizing that habits will always remain immensely strong. In Chapter 7, I describe how the difficulty of behavior change underlies a number of our most important and difficult public health problems. I outline the shortcomings of previous research on behavior change, and describe a new approach that is attempting to change this by focusing on the basic mechanisms that support behavior change. Many different strategies have been suggested to help change behavior, and in Chapter 8 I discuss research into the effectiveness of many of these approaches. Some of the strategies are supported by science, but for many of them the science is just too weak to support their use. In Chapter 9 I discuss some possible avenues for future interventions based on neuroscience research. None of these have been implemented yet at any sort of scale, but some of them provide promise for the future. I wrap things up in the epilogue, where I offer a synthesis of what the science tells us about the prospects of improving our ability to change our behavior, particularly in the context of major challenges such as the COVID-19 pandemic and the climate crisis.

w h at i s a h a b i t ? 15

Be forewarned: I don’t have any “easy tricks” to offer for breaking bad habits. In fact, many of those magic solutions for habits that you’ve read about in other books tend to vanish when we look at the real science. Instead, you will walk away with a deep understanding of why habits are so sticky and hopefully with a few well-supported ideas about how to improve the chances of making successful changes.

2 The Brain’s Habit Machinery

our thoughts and actions appear to be so seamless that it’s hard to imagine that they arise from a messy cacophony of electrical activity coursing through a few pounds of Jello-like tissue in our head— but that’s exactly what happens. The complexity of the brain is beyond staggering, and what the general public may not know is that many neuroscientists quietly despair as to whether we can ever fully understand how it works. That said, we know a lot of the basics, starting with how cells in the brain process information. The human brain is made up of tens of billions of neurons, the main cells that process information, along with many other supporting cells known as glia. Neurons send electrical signals from one end to the other, and then release chemical signals that affect electrical activity in their neighboring neurons. The signals from the main body of the cell to the other end travel by a wirelike structure called an axon, where they cause the release of chemicals known as neurotransmitters. It is this combination of electrical and chemical signals, along with the architecture of the brain that defines which neurons are connected to which, that results in everything we do. As an example, let’s trace what happens in my cat Coco’s brain when she sees a bird outside the window. The light reflected by the bird hits the retina in Coco’s eye, which contains specialized neurons that sense light; they do this by turning the energy from the light into an electrical signal, through changes in the electrical properties of the cell

16

t h e b r a i n ’s h a b i t m a c h i n e r y 17

that occur when a photon hits a specialized receptor molecule located on the surface of the cell (the cell membrane). This electrical signal is called an action potential, and it travels down the length of the neuron. These light-sensing neurons make contact with other kinds of neurons in the retina, and the action potential causes the release of chemical signals that can either activate or deactivate the next neuron in the chain. These signals are propagated through several layers of neurons in the eye and ultimately travel via a nerve (which is a cable made of many axons) to the brain. The signal first arrives in a structure buried deep in the brain called the thalamus, which can be thought of as the brain’s switchboard—nearly all incoming signals to the brain come in via the thalamus. From the thalamus, the signal travels to the outer surface of the brain known as the cerebral cortex, where much of the advanced information processing in our brain happens. Different parts of the cerebral cortex receive different types of information; in this case, visual information travels to a region in the back of the brain known as the visual cortex and then moves successively forward in the brain. At each stage, the information processing gets a bit more complex. In parts of the visual cortex that receive the inputs from the thalamus, individual neurons are only sensitive to signals coming from small portions of the visual world and are sensitive to relatively simple features, such as edges or lines. Those early regions send their signals to regions toward the front of the brain, which are sensitive to increasingly complex features in the input, such as patterns or whole objects. At some point, these signals lead Coco to identify the pattern of visual stimulation as a bird, which sends signals to other parts of the brain that are involved in emotion, releasing neurochemicals that lead her to become very excited and agitated. Some of these signals also lead to activity in the motor cortex that causes her to run toward the door and make very strange noises. Throughout the book I go into more detail about various aspects of how brain function gives rise to our thoughts and behaviors, because many of those details are essential for understanding how habits are formed and why they persist.

18 c h a p t e r 2

Habits and Conscious Memories A striking feature of habits is how they can be completely divorced from our conscious memory for the past, both in their execution and in our later memory for them. Take the habit of locking the door when we leave the house. Once we learn to lock the door, we would never think back about how to lock the door, or try to remember past times when we locked the door; we just do it without thinking, as it is with all habits. If I ask you which way you turn the key in order to lock the door, you can visualize it and tell me that you turn it to the right, but when you are locking the door you never explicitly think “now I need to turn the key to the right.” And conversely, how many times have you left your house, only later to be unable to remember whether or not you locked the door? We can generally trust our habit system to make sure the door gets locked, but habits often leave us with little explicit trace of the experience. The distinction between habits and conscious experience becomes particularly striking in people who have lost their memory due to brain damage. A famous example comes from the French neurologist Édouard Claparède. Upon meeting one of his patients with a memory disorder, he stuck her hand with a pin that was hidden in his hand. After a few minutes, she no longer remembered having been pricked by the pin, but she nonetheless was reticent to extend her hand when he reached out again. When he asked why she pulled her hand back, she said “Sometimes pins are hidden in people’s hands.” Claparède’s patient clearly retained some record of the experience of being stuck with the pin, even if she didn’t consciously remember the specific episode. Building upon anecdotes like this one, a large body of neuroscience research that began in the 1960s has now established the idea that there are multiple memory systems in the brain. The main distinction lies between the systems that allow us to consciously remember past events (such as remembering where you parked your car this morning) and other types of memory that do not involve conscious recollection of the past (including habits and skills, like how to drive your car).

t h e b r a i n ’s h a b i t m a c h i n e r y 19

Up Left

Right Down

Thalamus

Hippocampus

CA1

Medial temporal cortex figure 2.1. This is an MRI scan of my brain demonstrating the anatomy of the declarative memory system, showing the relative locations of the hippocampus (including area CA1), medial temporal lobe cortex, and thalamus. The top right inset shows the approximate location of the slice through my brain, just in front of my ears. (To learn much more about how MRI works, and how my brain came to be one of the most intensely studied brains to date, see my previous book The New Mind Readers.)

A System for Conscious Memory Conscious recollection of the past relies specifically on the declarative memory system, which involves a set of brain areas in a deep part of the temporal lobe (known as the medial temporal lobe), including the hippocampus and the parts of the cerebral cortex that surround it (see Figure 2.1). Damage to these regions can cause a loss of memory of the past as well as an inability to create new memories. In fact, it doesn’t take much damage to the hippocampus to cause such a memory disorder. The memory researcher Larry Squire and his colleagues demonstrated this by studying the brain of a man known by his initials R. B., who developed severe memory problems after suffering several cardiac events that momentarily starved his brain of oxygen. While R. B. was alive, Squire and his colleagues tested his memory in

20

chapter 2

a number of ways and found that he had problems with many aspects of memory. R. B.’s intelligence remained intact—in fact, his IQ was 111, which is above average. However, his ability to remember new materials was badly impaired. For example, when presented with a prose passage, he was able to repeat the details of the story immediately afterward, but after 20 minutes was unable to remember nearly any details of the story. R. B. was also keenly aware of his memory problems, as described by Squire and colleagues: “He explained that he needed to ask his wife repeatedly to tell him what had gone on and, if he talked to his children on the phone, he did not remember anything about it the following day” (p. 2951).1 R. B. donated his brain to science, so that Squire and his colleagues were able to examine it up close after his death to see exactly how it was damaged. Overall his brain looked healthy, but when they looked at it microscopically they saw that there was damage in a very specific part of the hippocampus known as CA1. This tiny section of the brain contains about 16 million neurons, which may sound like a lot but is a minuscule fraction of the nearly 100 billion neurons in the brain. Nonetheless, damage restricted to these neurons was sufficient to cause R. B. to have an enduring and significant memory problem, while leaving his other cognitive abilities intact. What began to become clear starting in the 1960s was that while hippocampal damage could cause severe deficits in remembering the past, it left some other forms of learning almost completely intact. One of the first such findings came from Brenda Milner and Suzanne Corkin, who studied a man named Henry Molaison, who is much better known by his initials: H. M. H. M. became amnesic following surgery to relieve his severe epilepsy, which would not respond to any of the available medications at the time. The surgeon removed much of H. M.’s medial temporal lobes on both sides of his brain, which greatly reduced his epilepsy but left him with a profound inability to remember his experiences from the many years prior to the surgery, as well as the inability to form new memories in the future. At the same time, the experiences of Corkin and her colleagues interacting with H. M. showed that he retained the ability to learn other things surprisingly

t h e b r a i n ’s h a b i t m a c h i n e r y 21

well. In her book Permanent Present Tense: The Unforgettable Life of the Amnesic Patient H. M., Corkin discussed how H. M. was able to learn new motor skills, such as using a walker after his hip was replaced in 1986. Even though he was unable to remember exactly why he needed the walker—asked why he needed it, he simply responded “so I won’t fall down”—he was able to, with practice, learn to use it properly. In a set of studies conducted in the 1950s and 1960s, Milner and Corkin extensively studied H. M.’s learning abilities. They found that while he remained unable to consciously remember the past, he was able to learn a number of motor skills quite well and retain them over long periods of time. These early studies of H. M. set the stage for a flurry of research in later decades that further delved into the learning abilities that remain intact in people with amnesia. One of the major findings of this work was that people with amnesia could learn not just new motor skills but new perceptual and cognitive skills as well. Neal Cohen, who would later be my PhD mentor, provided a particularly compelling demonstration of this in his early research with Larry Squire at the University of California at San Diego. In their landmark study, they examined the ability of individuals to learn the perceptual skill of reading words that have been mirror reversed (as shown in Figure 2.2).2 Research by the Canadian psychologist Paul Kolers in the 1970s had shown that individuals gradually became faster and more accurate at reading mirrorreversed words with practice, and that once they had learned the skill it persisted for at least one year. In their study, Cohen and Squire presented individuals with triplets of uncommon words in mirror-reversed text and measured how long it took for them to read the words aloud. They examined learning of the skill in three sets of individuals who became amnesic for different reasons, one of which was particularly ffitnialp elggardeb suoicirpac figure 2.2. An example of mirror-reversed text. Try reading the words aloud as quickly as possible, from right to left.

22

chapter 2

macabre, as described by Squire and colleagues in a later paper.3 The patient, known by his initials N. A., had been in the US Air Force when he was accidentally stabbed by a miniature fencing foil that entered his nostril and continued into his brain. MRI imaging by Squire and colleagues showed that the fencing foil damaged the thalamus, which is probably important for memory because of its close connections with the hippocampus. Another group tested by Cohen and Squire were individuals with Korsakoff syndrome, a brain disorder that occurs in some chronic alcoholics due to a deficiency of thiamine (vitamin B1), which also leads to damage to the thalamus. Finally, they studied people undergoing electroconvulsive therapy to treat their chronic depression, which also causes amnesia for a period of time after the treatment. Cohen and Squire first needed to establish that the individuals were indeed amnesic. To do this, they presented each person with 10 pairs of words and then tested them by showing the first word and asking what the other word in the pair had been. After seeing the full set of word pairs three times, healthy individuals were able to recall between 8 and 9 of the words on average. By comparison, the amnesic individuals could only recall about 2 of the 10 words, showing that their ability to consciously remember the past was severely impaired in comparison to the healthy control participants. To test their ability to learn the mirrorreading skill, Cohen and Squire gave the participants practice at reading the mirror-reversed sets of words over three days; they also tested them again about three months later to see how well the skill was retained. Their results showed that amnesic patients had no problem learning how to read the mirror-reversed text, improving their reading times just as fast as the healthy controls. When tested three months later, the amnesic patients also picked up right where they had left off, showing no loss of the skill, and in fact continued to improve on the task. These results provided a striking demonstration of just how much an individual can learn even in the face of amnesia, and they also provided clear evidence that the hippocampus and its related brain system are not necessary in order to learn new skills. But the question remained: If not the hippocampus, then what brain systems are essential for habits and skills?

t h e b r a i n ’s h a b i t m a c h i n e r y 23

Enter the Lizard Brain If you ask the internet, our “lizard brain” seems to be the source of many of our deepest human problems, with search results such as • • • •

How your 200-million-year-old lizard brain is holding you back Don’t listen to your lizard brain How to beat your lizard brain Quieting the lizard brain

The idea that habitual behavior arises from the evolutionary vestiges of the reptilian brain was made popular by the neuroscientist Paul MacLean, who spent several decades (from the 1960s through the 1990s) studying the brains and behavior of lizards in a quasi-natural laboratory environment that he built at the US National Institute for Mental Health. MacLean was interested in how the mammalian brain evolved from the reptile brain more than 200 million years ago. Because there are no surviving species of the lizard-like creatures that mammals ultimately evolved from (known as therapsids), MacLean looked to their closest existing relatives: lizards. His description of the daily life of the blue spiny lizard outlines how its behavior is remarkably habitual yet also oddly familiar to us as humans: In the morning when emerging from its shelter, the blue spiny lizard inches its way slowly and cautiously as though expecting at any moment to be seized by a predator. It proceeds to a preferred basking site where it adopts postures that maximize absorption of heat from the substrate of the rays from the artificial sun. Once its body has warmed to a near-optimum temperature, its next act is to empty its cloaca in an accustomed place near the basking site. In other words, like many kinds of mammals, it has a defecation post. . . . After defecation a blue spiny starts on its way to a favored perching place, pausing perhaps to take a drink of water. . . . Having attained its perch, it performs a brief signature display and then assumes a sit-and-wait posture, scanning the area for any moving prey. Its appearance is not so unlike that of a fisherman waiting to strike. . . .

24

chapter 2

After an anchored sit-and-wait period of feeding, there is a period of afternoon inactivity. . . . As the day wears away, the females begin to return to their favored places in the shelter. Then with eyes closed they gradually settle down for the night with their heads pressed into a crevice. The males eventually do the same, but first there often seems to be a need for further basking, with the dominant lizard soaking up the warmth longer than all the rest. (pp. 106–107)4 Based on his research, MacLean proposed the idea of the triune brain, which breaks the brain into three major sections. The reptilian brain consists of a deep set of brain regions that are present in all vertebrate species, including the brain stem and the basal ganglia, which we discuss in much greater detail later in this chapter. MacLean highlighted the role of the reptilian brain in routine/habitual behavior, as well as in activities such as mating calls and displays of dominance or submission. The limbic system is a set of structures that MacLean thought were novel to mammals, which are involved in the experience of emotion. The neomammalian brain refers to the portion of the cerebral cortex that is most highly developed in mammals and that exploded in size as mammals evolved. In making his case for the role of the “reptilian brain” in habitual or routine behavior, MacLean focused on the basal ganglia in particular, inspired by findings from research on individuals suffering from Huntington’s disease. Huntington’s disease is a genetic disorder with a very simple inheritance pattern: if either of a child’s parents has the disease, then they have a 50/50 chance of inheriting the genetic mutation that causes the disease. And this mutation is very powerful—anyone with the mutation is guaranteed to develop the disease during their lifetime, usually by their 50s. The most evident symptom of Huntington’s disease is the inability to control one’s movements, resulting in jerky movements of the limbs and uncoordinated walking, but the disease is also accompanied by psychiatric symptoms, such as psychosis, irritability, and depression. The brain disorder in Huntington’s disease results from the effects of the genetic mutation on the structure of a protein called huntingtin, so named due to its role in the disease. This

t h e b r a i n ’s h a b i t m a c h i n e r y 25

is a protein that is found in cells throughout the body but is particularly common in certain parts of the brain, most notably in particular neurons within the basal ganglia. The genetic mutation in Huntington’s disease causes the cells to produce a mutant version of the huntingtin protein, which leads to dysfunction and premature death of cells containing the mutant protein. While the disease ultimately attacks much of the brain, its earliest signs appear in the basal ganglia; in fact, in a brain imaging study of young adults who carried the Huntington’s disease mutation but were estimated to be more than 10 years from actually developing any overt symptoms of the disease, signs of changes in the basal ganglia were already evident.5 To establish the relation of Huntington’s disease to the loss of habit or routine, MacLean used evidence from a set of case studies that had been published by Eric Caine and colleagues in 1978.6 For example, one person with the disease complained that she could no longer prepare her usual Thanksgiving dinner, even though she knew how to do all of the individual steps; she got confused about how to perform the individual steps in order. Whereas the authors of the paper had viewed these complaints in terms of “difficulty with organization, planning, and sequencing,” MacLean reframed them in terms of the loss of the ability to engage in routinized behavior. These anecdotes were suggestive but didn’t show directly that people with Huntington’s disease had a problem with learning new habits. Inspired by the research of Cohen and Squire, another group of researchers from the University of California at San Diego was the first to experimentally test whether people suffering from Huntington’s disease were impaired at learning a new skill, which would have been predicted given the close relation between skills and habits. Maryanne Martone and her colleagues used the same mirror-reading task that Cohen and Squire had used, but in addition to testing individuals with Korsakoff syndrome, they also included individuals suffering from Huntington’s disease.7 When Martone and her colleagues tested the Huntington’s and amnesic patients on the mirror-reading task, they saw that the two groups exhibited almost the opposite pattern of deficits. The amnesic Korsakoff patients behaved very similarly to those in

26

chapter 2

the study by Cohen and Squire, showing relatively normal learning of the mirror-reading skill but difficulty remembering the words that had appeared in the mirror-reading task. Conversely, the Huntington’s disease patients showed relatively normal ability to remember the words, and while they did benefit somewhat from practice on the mirrorreading task, their skill learning was substantially poorer than amnesics or controls. This established what we refer to as a double dissociation, in which two different groups show the opposite pattern of normal or impaired performance across two different tasks. This kind of dissociation is generally taken to provide good evidence that the different tasks rely on separate brain systems, and in this case the results provided some of the first evidence that people with basal ganglia disorders have impairments in skill learning. Interestingly, although MacLean’s ideas about the role of the basal ganglia in habitual behavior have stood the test of time, the idea that there is something particularly “reptilian” about these parts of the brain has been largely rejected by neuroscientists. Subsequent studies comparing the anatomy of brains of many different vertebrates (from reptiles to birds to mammals) have shown that the overall plan of brain organization is remarkably similar between these groups;8 even the lamprey, the most ancient living vertebrate, has a similar organization. Thus, the brain of a reptile is not fundamentally different from the brain of a human in its overall organization; the human simply has a lot more tissue, organized in a much more complex manner. As we will see in subsequent chapters, it is this development, particularly in the prefrontal cortex, that allows humans to go beyond the routine and habitual behavior that characterizes many other species such as lizards.

What Are the Basal Ganglia? Deep within the brain sits a collection of brain nuclei (sets of cells bundled together) known as the basal ganglia. The basal ganglia in humans comprise several separate regions, including the caudate nucleus, putamen, and nucleus accumbens (which together are known as the

t h e b r a i n ’s h a b i t m a c h i n e r y 27

Striatum Thalamus Subthalamic nucleus Globus pallidus

C

C

P

P NA

Substantia nigra/ ventral tegmental area

figure 2.3. (Left) A schematic of the different parts of the basal ganglia, depicted in their relative position within an outline of the cerebral cortex. (Right) The position of the different portions of the striatum, including the caudate nucleus (C), the putamen (P), and the nucleus accumbens (NA).

striatum), the globus pallidus (which has two sections, internal and external), and the subthalamic nucleus, shown in Figure 2.3. In addition, the substantia nigra and ventral tegmental area, both of which include neurons that release dopamine, are considered part of the basal ganglia. While they are spread across different parts of the middle of the brain, what holds these areas together is the way in which they are tightly interconnected with each other. A large number of connections come from neurons in the cerebral cortex into the basal ganglia, making contact with the regions of the striatum. Importantly, each part of the striatum receives input from a different part of the cerebral cortex; the putamen receives input from motor and sensory areas, the caudate from the prefrontal cortex and from temporal lobe areas involved in vision, and the nucleus accumbens from regions in the frontal lobe that are involved in the processing of reward and emotion (as well as other subcortical areas, such as the amygdala). It is these different connections that determine the function of each of the regions—for example, the nucleus accumbens plays a central role in addiction, whereas the putamen plays a role in routine actions. When the input from the cortex arrives at the striatum, it generally connects to a specific set of neurons known as medium spiny neurons

28

chapter 2

DIRECT PATHWAY

INDIRECT PATHWAY Excitatory Inhibitory

Striatum

+ _

_

+

Striatum

+

Thalamus

+ _

_ + _

Globus pallidus

Thalamus Subthalamic nucleus

Globus pallidus

figure 2.4. A map of basal ganglia circuitry, showing direct (left) and indirect (right) pathways. Excitatory connections are shown with a pointed arrow and plus sign; inhibitory connections are shown with a rounded end and minus sign. The direct pathway has two inhibitory steps, which results in excitation of the cortex; the indirect pathway includes an additional inhibitory step from the external globus pallidus onto the subthalamic nucleus, resulting in excitation of the internal globus pallidus and inhibition of the cortex.

because of their spiny appearance under a microscope. From here, there are two paths that the signals can take through the basal ganglia, which we refer to as the direct pathway and indirect pathway, both of which are shown in Figure 2.4. The direct pathway goes from the striatum to another region called the globus pallidus, specifically to the internal part of this region, while the indirect pathway takes a more circuitous route through the basal ganglia, as we will see later. From here, the signals are sent to the thalamus and are then sent back to the cerebral cortex, usually to a region that is very close to where the input originally initiated. It is for this reason that we refer to these circuits as corticostriatal loops. Let’s look at what happens as the signal courses through the direct circuit. In order to understand this, it’s important to know that neurons are distinguished by the effect that they have on the neuron that they are connecting to: an excitatory neuron increases the activity in its target neuron, whereas an inhibitory neuron suppresses activity in the target neuron (see Box 2.1 for more details). Later we will also discuss a third class of neurons that have yet a different effect of modulating the responses of these other neurons. When a neuron in the cerebral

t h e b r a i n ’s h a b i t m a c h i n e r y 29 Box 2.1. Excitatory and inhibitory neurons A neuron at rest generally has a negative electrical potential, meaning that the electrical charge on the inside of the cell is less than that outside the cell. This difference is maintained by a set of ion channels, which allow ions like sodium and potassium to travel passively across the cell membrane, and ion pumps, which actively pump ions across the membrane. When one neuron causes another to fire, it does this by releasing molecules (neurotransmitters) into the synapse, which make contact with receptors located on the cell membrane of the target neuron. The excitatory neurotransmitters (such as glutamate) activate ion channels that allow positively charged ions to enter the cell, raising its electrical potential. When the membrane potential reaches a particular level known as a threshold, an action potential occurs, sending an electrical impulse down the axon to the downstream neurons. Inhibitory neurotransmitters (like GABA), on the other hand, allow negative ions (like chloride) to enter the cell, making the electrical potential more negative and preventing the neuron from firing.

cortex sends an input to the striatum, it causes the medium spiny neurons (which receive this input) to become more active, because these particular cortical neurons are excitatory; in fact, nearly all neurons that send long-distance messages from one part of the brain to another are excitatory. The medium spiny neurons in the striatum that receive those inputs are inhibitory, which means that when they fire they cause reduced activity in their target neurons in the globus pallidus. Those neurons in the globus pallidus are also inhibitory, such that when they fire they inhibit activity in their target neurons in the thalamus. And the neurons in the globus pallidus fire a lot— between 60 and 80 times per second when an animal is resting.9 This constant (or “tonic”) inhibition keeps the neurons in the thalamus largely silenced most of the time, preventing them from exciting their targets back in the cortex. Note what happened here: we have two inhibitory neurons in a row, which means that the input to the first one (the medium spiny neuron in the striatum) will reduce the constant inhibition of the second one (in the globus pallidus), leading to excitation in the thalamus and subsequently in the cortex. It’s like multiplying together two negative numbers, which makes a positive number. Thus, we think that the effect of stimulation of the direct pathway is usually to cause the initiation of an action or thought by exciting activity in the cortex at the end of the loop.

30 c h a p t e r 2

The indirect pathway through the basal ganglia has the opposite effect: by inhibiting the neurons in the cerebral cortex that are at the end of the loop, it shuts down action and thought. The pathway starts very similarly to the direct pathway, with a connection from the striatum to the globus pallidus, though in this case to its external section. The external globus pallidus then sends an inhibitory signal to a region that we will encounter on a number of occasions, known as the subthalamic nucleus (STN). The STN sends an excitatory output to the internal globus pallidus—which you will recall is the second inhibitory stage of the direct pathway. By turning on this inhibitory stage, the effect of activity in the STN is to inhibit overall activity in the thalamus and cortex. Thus, by adding an additional step in the circuit, activity in the indirect pathway has the effect of inhibiting action and thought, as we will see in our discussion of response inhibition in Chapter 5. How does the input from the cortex to the striatum know which pathway to take? It turns out that different groups of medium spiny neurons in the striatum send their outputs to either the direct or indirect pathway, and one of the main differences between those two sets of neurons has to do with everyone’s favorite neurochemical: dopamine.

Dopamine: It’s Complicated Dopamine is seemingly everywhere. Science journalist Bethany Brookshire captured this beautifully in a 2013 blog post: In a brain that people love to describe as “awash with chemicals,” one chemical always seems to stand out. Dopamine: the molecule behind all our most sinful behaviors and secret cravings. Dopamine is love. Dopamine is lust. Dopamine is adultery. Dopamine is motivation. Dopamine is attention. Dopamine is feminism. Dopamine is addiction. My, dopamine’s been busy. You have almost certainly read about dopamine in the popular media— in fact, it seems to be their favorite neurotransmitter. It is often portrayed as a “pleasure chemical” in the media, responsible for everything

t h e b r a i n ’s h a b i t m a c h i n e r y 31

Prefrontal Striatum cortex Nucleus accumbens

Meso-cortic pathway

VTA

Meso-limbic pathway SNc Nigrostriatal pathway

figure 2.5. The neurons that produce dopamine reside deep in the middle of the brain, within the substantia nigra pars compacta (SNc) and the ventral tegmental area (VTA). These neurons send their outputs widely across the brain, but a particularly large amount is sent to the striatum. (Image by Arias-Carriòn et al., CC-BY)

from love to addiction, but this is really a misrepresentation of the many complex roles that dopamine plays in the brain. As we will see later, dopamine is fundamental to the development of habits, good and bad alike. First, let’s ask where dopamine comes from and what it does. The great majority of dopamine in the brain is produced in two small nuclei deep in the middle of the brain: the substantia nigra (specifically, a portion of this area called pars compacta) and the ventral tegmental area (see Figure 2.510 ). These neurons send projections to much of the brain, but the projections to the basal ganglia are especially strong. The number of dopamine neurons in the brain is tiny—about 600,000 in humans11 —which belies their outsized effect on nearly every

32

chapter 2

aspect of our thought and behavior. Dopamine is a neuromodulatory neurotransmitter, which means that it doesn’t directly cause excitation or inhibition in the neurons that it affects. Rather, it modulates the effect of other excitatory or inhibitory inputs to those neurons— think of it like a volume knob on a guitar amplifier, which modulates how strongly the input from the guitar affects the loudness of the speaker. As we will see, dopamine also plays a critical role in the changes in the brain that occur due to experience, which neuroscientists call plasticity. One additional complication of dopamine (which also applies to the other neuromodulatory transmitters that we discuss later in the book, such as noradrenaline) is that there are different types of dopamine receptors that are present on neurons. Some of these (known as D1-type receptors) have the effect of increasing the excitability of the neurons where they are present (turning up the volume), while others (D2-type receptors) have the effect of reducing the excitability of those neurons (turning down the volume). Individual neurons tend to express—that is, to create and put those receptors on the surface of the cell—only one of these two types of dopamine receptors. Studies of medium spiny neurons in the direct and indirect pathways have shown that neurons in the direct pathway primarily express D1-type dopamine receptors, whereas neurons in the indirect pathway primarily express D2-type dopamine receptors. For many years this distinction was controversial, but a set of new neuroscience methods known as optogenetics have provided strong evidence for the distinction (see Box 2.2). A landmark study by Alexxai Kravitz and Anatol Kreitzer of the University of California at San Francisco tested the mapping of dopamine receptor types to basal ganglia pathways by using optogenetics to activate striatal neurons that expressed either D1-type or D2-type dopamine receptors in mice, and examined the effects of this activation on the animals’ behavior.12 Remember that striatal neurons with D1-type receptors are thought to fall in the direct pathway, and activating them should increase the animals’ activity. When Kravitz and Kreitzer stimulated neurons expressing D1-type dopamine receptors, they saw that the animals spent more time walking around their cages

t h e b r a i n ’s h a b i t m a c h i n e r y 33 Box 2.2. Controlling the brain with light Optogenetics refers to a set of techniques that allow neuroscientists to control the activity of specific sets of neurons using light. It has long been a holy grail in neuroscience to be able to directly control the activity of individual neurons, because this allows researchers to test many different theories about what those neurons do. In the past, neuroscientists would often inject electrical currents into the brain in order to stimulate neurons, but it’s impossible to make this stimulation very precise, which means that many different types of cells (both excitatory and inhibitory) will be stimulated. In addition, the levels of electrical stimulation are far beyond those that naturally occur in the brain. Starting around 2000 neuroscientists began to experiment with using light to control neurons. Our body already has cells that respond to light, particularly those in our retina that provide us with the ability to see light from the world. These cells express photoreceptors, which are ion channels on the cell surface that change their molecular shape in response to light, such that they allow positive ions to enter the cell and cause an action potential. Other organisms have even more powerful photoreceptors, and it was the discovery of a class of powerful photoreceptors in green algae (known as channelrhodopsins) that was the key to allowing neuroscientists to control brain activity with light. Using the increasingly powerful tools of molecular biology, these channelrhodopsins can be inserted into neurons with a great deal of precision, and the application of light to the brain can then be used to control their activity. Other kinds of photoreceptors can also be inserted that can silence activity rather than excite the neuron. It is no exaggeration to say that optogenetics has revolutionized neuroscience.

and less time sitting still, which is consistent with the idea that these neurons drive increased activity in the cortex. When they stimulated neurons expressing D2-type receptors, they saw the opposite—the mice spent much more time frozen in place and less time walking around the cage. These results cemented the role of these different classes of neurons in either causing or preventing actions. With all of this background, we now have the knowledge to understand why Huntington’s disease causes its particular symptom of uncontrollable movements. For reasons that are not fully understood, Huntington’s disease affects neurons in the indirect pathway well before it damages those in the direct pathway. Thus, while movements are inhibited by the indirect pathway in the healthy brain, in the Huntington’s brain this pathway is damaged, disturbing the balance in favor of the direct pathway and thus leading to uncontrollable movements. We can also understand what happens in Parkinson’s disease, a much more common neurological disorder than Huntington’s disease,

34 c h a p t e r 2

whose cause remains largely unknown. The symptoms of Parkinson’s disease are in some ways the opposite of those of Huntington’s disease: slowed movement, rigid posture, and tremors. Parkinson’s disease involves the degeneration of dopamine neurons in the substantia nigra. The loss of these neurons starves the brain of dopamine, which results in a relative increase in activity in the indirect pathway, since dopamine suppresses activity on those neurons due to their D2-like receptors. Conversely, the lack of dopamine results in a decrease in activity in the direct pathway, since dopamine increases the activity of those neurons due to their D1-like receptors. These two disorders show just how finely activity must be balanced between these two pathways in order to achieve healthy brain function.

Dopamine and Brain Plasticity As I mentioned above, dopamine has many different effects in the brain, and one of these is central to its role in the formation of habits: it modulates the basic mechanism of change in the brain, known as synaptic plasticity. To understand synaptic plasticity, let’s look at what happens when one neuron communicates with another. Say that we have a neuron in the cortex that projects onto a medium spiny neuron in the striatum. When the cortical neuron fires an action potential, this results in the release of an excitatory neurotransmitter (glutamate) from small storage vesicles that are positioned at the end of the axon that forms a synapse with the neuron in the striatum. These molecules are released into the open space between the axon and its target neuron in the striatum; this open space is known as the synapse. After they are released, the neurotransmitter molecules float through the synapse, and some of them make contact with receptors that are present on the surface of the neuron on the other side of the synapse. When that happens, they cause electrical changes in the cell that can ultimately lead it to undergo its own action potential. Importantly, there are many different reasons why one particular neuron might have a stronger effect than others when it comes to causing an action potential: they could release more neurotransmitters, they could have more synapses, or they could

t h e b r a i n ’s h a b i t m a c h i n e r y 35

Both neurons fire but no dopamine present => weakened synapse

Both neurons fire with dopamine present => strengthened synapse

Cortical neuron

Cortical neuron

G

G G G G

Dopamine neuron

G

G G G G

Dopamine neuron

D D D

D

Medium spiny neuron in striatum

Medium spiny neuron in striatum

figure 2.6. A schematic of the three-factor plasticity rule, in which dopamine modulates the plasticity of synapses in the striatum. The cortical neuron releases glutamate (G), which causes the striatal neuron to fire. Changes in the synapse occur, which depend on whether dopamine (D) is present; if dopamine is present (right), then the synapse is strengthened, whereas it is weakened if dopamine is absent (left).

have larger synapses, just to name a few. In addition, the downstream neuron could also have more receptors on its surface. Synaptic plasticity is the process by which experience changes the strength of synapses, so that some neurons become more potent at exciting other neurons and others become less potent. This plasticity is thought to be critical to learning.13 Dopamine does not directly cause synaptic plasticity. Instead, it plays a critical role in modulating plasticity by what is known as the three-factor rule (see Figure 2.6). One of the most common forms of plasticity happens when one cell causes another to fire in quick succession and the strength of their synapse is increased. This kind of plasticity (known as Hebbian plasticity after the neuroscientist Donald Hebb) is often described in the following terms: “Cells that fire together, wire together.” In some regions of the brain, including the striatum, this concept is slightly modified to get the three-factor rule: “Cells that fire together, in the presence of dopamine, wire together; cells

36 c h a p t e r 2

that fire together without dopamine come unwired.” The three factors are the firing of the incoming neuron, the firing of the target neuron shortly thereafter, and the presence of dopamine in the vicinity. In this way, dopamine serves as a gate to the development of new behaviors, including habits.

What Does Dopamine Mean? Wolfram Schultz is a German neuroscientist (now based at Cambridge University) who has spent his career trying to understand dopamine, and whose work has been key to starting to unravel the mystery of what causes dopamine to be released. His research involves recording the activity of the neurons that release dopamine in the brains of monkeys and trying to understand what makes them fire. Dopamine had long been associated with reward, based on earlier studies showing that if an electrode is put in a rat’s brain in a location that stimulates dopamine release, the rat will do almost anything to receive this stimulation. Schultz’s earliest work on monkeys confirmed that rewarding events do indeed cause activity in the dopamine neurons in the monkey’s brain. However, he noticed a phenomenon that would come to revolutionize how we think about the role of dopamine. When the monkey was provided with an unexpected reward, the dopamine neurons would fire. Then Schultz tested a situation where the monkey first received a signal (a flash of light) that occurred prior to the reward (see Figure 2.714 ). In the beginning of the experiment, before the monkey knew that the light was predictive of the reward, the dopamine neurons didn’t fire until the reward appeared. But once the monkey learned that the reward was foreshadowed by the light, the dopamine neurons fired when the light appeared and did not fire when the reward appeared. Further, if the expected reward did not appear after the light, then activity in the dopamine neurons went down below their baseline level of activity. This was the first hint that dopamine neurons are not strictly sensitive to reward, but instead seem to be sensitive to situations where the world is different from our predictions (a concept known as reward prediction error).

t h e b r a i n ’s h a b i t m a c h i n e r y 37

No prediction Reward occurs R Reward prediction Reward occurs CS

R

Reward prediction No reward occurs CS

(no R)

figure 2.7. The demonstration of reward prediction error signaling by dopamine neurons. Each plot shows the activity of dopamine neurons over time within a trial. The top panel shows dopamine responses to an unpredicted reward (R). The middle panel shows the response of these neurons to a “conditioned stimulus” (CS) that predicts the reward, and a lack of response to the predicted reward itself. The bottom panel shows the depression of activity that occurs due to negative prediction error when a predicted reward does not appear. (Adapted from Schultz et al.)

This discovery was critical because it helped link dopamine with a set of ideas from computer science and psychology that ultimately led to what is now the dominant computational framework for understanding the role of dopamine. Within computer science, researchers have long been interested in how to build systems that can learn from experience; this field is now known as machine learning and is the foundation for many of the automated systems we interact with every day. One of the kinds of learning that these researchers have investigated is called reinforcement learning, which basically means learning by trial and error. Imagine that you walk into a casino and have to choose between two slot machines to play. At first, you have no way of knowing which machine might have a better payoff, so you just choose one at random. If you play a few rounds and continue to lose, at some point you will probably move over to the other machine, whereas if you have several initial wins, you will likely stay with that machine. The theory of

38 c h a p t e r 2

reinforcement learning describes how an individual should behave in a situation like this (which we discuss in much more detail in Chapter 4). One of the basic ideas from the theory of reinforcement learning is that learning should proceed on the basis of how well our predictions match the outcomes that we actually experience. After all, if we can perfectly predict the world then there is nothing else to learn! Most theories of reinforcement learning posit that the decision maker chooses an action based on the predicted value of all the possible actions one could take; in the case of our two slot machines, this would mean choosing the machine with the highest predicted value. We then observe the outcome (win or loss), and use this information to update our prediction for the next round. Importantly, it’s not the absolute amount of the win or loss that we use to update our predictions—rather, it’s the difference between the prediction and the observed outcome that we use, which is exactly the prediction error signal that dopamine was shown by Schultz and his colleagues to represent. By showing that dopamine could be understood in terms of the mathematical theory of reinforcement learning, this work provided a powerful framework that continues to be highly influential in the study of decision making in the brain. Einstein is famous for having quipped that scientific theories should be as simple as possible, but no simpler. In this case, it appears that despite its success in explaining many aspects of dopamine function, the reward prediction error theory may be too oversimplified. Ilana Witten is a neuroscientist at Princeton University who has set her sights on understanding the function of dopamine in all its messy detail. She does this by studying mice, because of the powerful techniques available for studying the mouse brain. In order to understand the complexity of dopamine signals, one needs to be able to record from dopamine neurons while the mouse is engaged in complex behaviors. To do this, Witten teamed up with David Tank, another Princeton professor whose group had developed a virtual reality system for mice. In this system, the mouse sits on a small ball (sort of like a ping-pong ball), with its head fixed in place using a small metal helmet, while it watches an immersive video display and runs on the ball. The helmet holds the head in place well enough that a small microscope is able to

t h e b r a i n ’s h a b i t m a c h i n e r y 39 Box 2.3. Calcium imaging: Making neurons “light up” If we want to understand the activity of neurons, the best way to do that is to directly record from individual neurons in the brain. Historically, this was done by placing very small electrodes into the brain and recording the electrical activity of neurons (generally in nonhuman animals, except in rare cases where such electrodes are implanted into human brains to help plan surgery for epilepsy). This approach has led to much of our basic knowledge about how the brain works, but it is also limited because we can only record from a relatively small number of neurons at any one time. However, the optogenetics revolution (discussed in Box 2.2) has also provided neuroscientists with the ability to image the activity of many neurons at once, using a technique called calcium imaging. This technique relies upon the fact that when neurons become active there are changes in the concentration of calcium ions within the cell. Via genetic engineering, researchers can insert a gene into specific types of cells in the brain (such as dopamine neurons) that results in generation of flourescent light whenever a neuron becomes active. Using a microscope that is tuned to this light, researchers can measure the activity of large numbers of neurons at once.

record from neurons in the brain while the animal is behaving, using a technique known as calcium imaging (see Box 2.3). This allows the researchers to determine how the activity of neurons in the dopamine system are related to many different aspects of the animal’s behavior, not just reward prediction errors. In 2019 Witten and her colleagues used the virtual reality technique to show that dopamine neurons behave in ways that are much more complex than we thought before. The mice played a fairly simple game in which they ran down a virtual hallway and then turned to the left or right at the end of the hall; if they turn in the correct direction they get a sip of water (which is quite rewarding for a thirsty mouse), whereas if they turn the wrong direction they get buzzed at and have to sit through a 2-second time-out. As the mouse runs down the hallway, it is presented with virtual “towers” on each side of the hall, which provide the animal with a cue as to which side will be rewarded at the end of the hallway; the more towers on one side, the more likely the reward is to appear on that side. The mice learn to do this, such that after practice their likelihood of turning to a particular side closely matches the proportion of towers on that side. Across 20 mice, Witten and her colleagues were able to record the activity of more than 300 dopamine neurons in the mouse brain while they played the game.

40

chapter 2

If the reward prediction error theory was correct, then the dopamine neurons should fire only in relation to unexpected rewards and to cues that predict rewards. To test this, Witten’s group teamed up with Nathaniel Daw, another Princeton professor, who is one of the world’s experts in the study of how dopamine signals relate to learning (whose work we see again in Chapter 4). The group built a statistical model that would allow them to test how dopamine neurons responded to many different aspects of the mouse’s experience—everything from where the mouse was in the hallway to how fast it was running to whether it had been rewarded on the previous trial. What they found was that there were dopamine neurons that responded to each of these different aspects of the game. There was certainly good evidence that many of the neurons responded as expected by the reward prediction error theory, but this was far from the only factor that caused them to react. Witten’s research has begun to show us just how complicated dopamine really is.

What about Pleasure? When it comes to popular ideas that are scientifically incorrect, the commonly noted link between dopamine and pleasure is almost certainly at the top of the list. The link between dopamine and pleasure certainly makes sense; after all, animals will self-stimulate their dopamine system until they fall over with exhaustion, so they must find it pleasurable, right? Well, sometimes the obvious answer happens to be the wrong one. The brain’s neurochemical systems are remarkably complex and intertwined, and a major discovery over the last two decades is that dopamine is not directly responsible for the pleasurable sensations that occur due to drug use. Instead, the role of dopamine appears to be centered on motivation—or as the neuroscientist Kent Berridge has called it, “wanting,” rather than “liking.” John Salamone is a neuroscientist at the University of Connecticut who has spent his career studying motivation in rats—in essence, trying to turn rats into couch potatoes by manipulating their brain chemistry. To do this, he used an experimental setup where the rat is allowed to make a choice between a small amount of food that they

t h e b r a i n ’s h a b i t m a c h i n e r y 41

can access without any extra effort, or a larger amount of food that they can access only after climbing over a wire barrier. Given the choice, a normal rat will nearly always climb over the barrier to get the extra amount of food. However, in a number of studies Salamone and his colleagues have shown that interfering with dopamine causes the rats to be much more likely to choose the smaller amount of food that doesn’t require them to work for it. This is not because the animals with disrupted dopamine can’t climb the barrier; given the choice between climbing the barrier for food versus no food, they will indeed climb it to get the food. It just seems to reduce their willingness to work for food. Salamone’s work dovetails with a set of ideas from neuroscientists Kent Berridge and Terry Robinson, who have argued persuasively that the role of dopamine is what they call “incentive salience”: rather than determining how much an organism likes a reward, dopamine instead provides signals about how much the organism wants some particular reward in the world and how much they will work to obtain it. As we will see in Chapter 6, this idea is key to understanding some of the changes that occur in addiction. One complexity of dopamine’s role in motivation is that dopamine is released widely across the brain and has different effects depending on which area is receiving the dopamine input. For example, blocking dopamine in the motor portion of the striatum reduces an animal’s overall level of physical activity, just as it leads to a reduction in movement in people suffering from Parkinson’s disease. It is dopamine receptors in the nucleus accumbens (a portion of the striatum that is heavily connected to other parts of the brain involved in emotion) that appear to play a central role in incentive motivation, but its role is complex. Blocking dopamine in the nucleus accumbens doesn’t seem to interfere with the basic appetite for food or with the pleasure that is obtained by eating it. However, it does interfere with the animal’s willingness to engage in behaviors required to obtain food or to work extra for additional food. Whereas dopamine seems to be important for motivation, the pleasurable aspects of reward seem to be signaled by other neurotransmitter systems in the brain, including opioids (targets of opiate drugs

42

chapter 2

such as heroin) and cannabinoids (targets of the active components in cannabis). The best-known evidence for this comes from the work of Kent Berridge, who studied the “hedonic response” of rats (such as licking their lips or paws for sweet foods or shaking their head in response to bitter foods) after blocking either dopamine or opioid neurotransmitters in the brain. While blocking dopamine did not reduce the animals’ expressions of hedonic responses, blocking of opioid neurotransmitters did. These results are consistent with numerous reports on the effects of naltrexone, a drug that blocks opioid transmission and is commonly used for the treatment of alcoholism. Studies have examined the effects of naltrexone on everything from sex to gambling to amphetamine administration, and have generally found that the drug reduces the pleasure that individuals experience from each of these.

Selecting Actions in the Striatum At any particular point in time, there is an almost infinite set of actions that a person could make. Even in the context of a simple action, such as picking up a coffee cup, there are a multitude of ways that one could perform it—quickly or slowly, smoothly or jerkily, in a direct way or a roundabout way, and so on. To help understand why we do what we do, an important question to ask is, What is our goal when we select an action? On the one hand, we want to maximize the reward that we obtain from that action. This might be obtaining a sip of coffee without spilling it or winning the maximum amount of money from a slot machine. On the other hand, we want to minimize the cost of the action, both in terms of physical and mental effort and in terms of time. We could pick up the coffee cup and raise it above our head five times before putting it to our mouth, but no one would ever actually do that, both because of the risk of spilling and the added cost in terms of physical effort and time. The basal ganglia and the dopamine system appear to play a central role in the computations that help us decide what to do at any particular point in time and how to go about doing it.

t h e b r a i n ’s h a b i t m a c h i n e r y 43

In 1999 the neuroscientist Peter Redgrave proposed that the role of the basal ganglia is to act as a sort of “central switchboard” for the selection of actions.15 In this theory, the cortex sends signals to the basal ganglia that represent potential actions. This theory relies upon the strong tonic inhibition within the globus pallidus that we mentioned earlier. Let’s say a person is in a situation where they need to choose between two potential actions (say, reaching for a piece of cake or a carrot). Each of those potential actions will be represented by a signal that is sent from the cerebral cortex to the basal ganglia. Before the signals arrive, the tonic inhibition in the basal ganglia inhibits all actions. When the two signals arrive at the striatum, they compete with one another through a combination of activity within the direct and indirect pathways, with one action ultimately being selected and executed via the direct pathway. For many years this model remained largely speculative, but recent advances in neuroscience methods have started to provide direct evidence for the idea. Rui Costa is a neuroscientist at Columbia University whose work has provided some of the best evidence of how action selection works in the basal ganglia. This work takes advantage of the optogenetics tools that we mentioned earlier, which allow the researchers to identify for any individual neuron in the striatum which pathway it is part of, and then measure the activity in those neurons separately for the two pathways. In their experiment, they taught mice to perform a sequence of lever presses; with practice, the mice became very fast at doing this. When Costa and his colleagues then recorded from neurons in the striatum, they saw that both direct and indirect pathway neurons became active at the beginning of the sequence of movements; this probably reflects the competition between the different possible movements. However, once the sequence of movements had started, it was only the direct pathway neurons that remained active. At the end of the sequence, the indirect pathway neurons once again became active, showing that these neurons are also involved in ending a complex action.16 The research in this chapter has shown us that the basal ganglia are the center for habit learning in the brain, and that dopamine plays a

44

chapter 2

critical role in establishing new habits. From this, we can start to see how these mechanisms can ingrain a habit. Let’s take a simple example: A rat is presented with two possible levers that it could choose to press, only one of which gives a food pellet as a reward. At first, the two actions have roughly equal value, since the rat doesn’t know which one will give a reward (though lab animals have generally spent enough of their life in experiments like this to know that levers are made to be pressed!). The cortex would send commands to the striatum corresponding to each of the two lever presses, and one of them would win the competition in the striatum; for example, the rat may have learned in the past that levers on the right side of the box are more likely to give a reward, so it might go for that one. If it receives food, then that unexpected reward will cause the release of dopamine, which will result in strengthening of the connections between the cortex and the striatum that caused that particular lever press, via the three-factor rule. This increase in the strength of those connections will make it more likely that the cortical neuron will cause the striatum neuron to fire the next time. If the rat doesn’t receive food, then that will cause a decrease in the strength of the connection that caused the response. These changes in connection strength then make it more likely that the rewarded action will win the competition the next time the rat is given the same choice, and over time this becomes cemented into a habit. In the next chapter, we turn to ask one of the central questions of this book: Why are habits so sticky?

3 Once a Habit, Always a Habit

in 2008 my wife and I took a trip to New Zealand, starting in Christchurch on the South Island and making our way north. We rented a car and began the drive north, through beautiful and relatively mountainous territory where one sees many more sheep than people (or other cars). In New Zealand, cars drive on the left side of the road, and this requires a lot of attention for someone from the US, but over a couple of days I started to get used to it. At some point on the trip, we encountered some construction on one of the mountain roads, which had the road narrowed to a single lane (the right lane from our direction). This went on for a good while, and then at some point the construction ended—but I didn’t notice. Instead, I stayed in the right lane. I probably drove a few miles before I was confronted by an oncoming car in my lane. Fortunately, the road was winding enough that we were both going slow and were able to avoid a head-on collision. But the encounter shows just how sticky habits can be and how easy it is for them to return, even when the stakes are very high. Not everyone gets as lucky as we did on that winding road in New Zealand, especially when it comes to habitual use of addictive drugs. Philip Seymour Hoffman was an acclaimed screen actor, winning the Academy Award for Best Actor in 2005 along with many other accolades for his work. After abusing drugs and alcohol in college, Hoffman went through a drug rehab program and remained sober for more than 20 years. However, Hoffman relapsed in 2013 in the wake of 45

46

chapter 3

problems in his personal life, and he could not escape the oncoming collision; despite another attempt at drug rehab, he died less than a year later of an overdose of multiple drugs, including heroin, cocaine, and amphetamine. To understand how someone could go for so long only to relapse into addiction, we need to understand the neuroscience behind why habits are so sticky.

Old Habits Never Die We learned in Chapter 1 that the brain has to determine when to remain stable and when to change, known as the stability-plasticity dilemma. One of the brain’s strategies to solve this dilemma has been uncovered in large part through the work of a scientist named Mark Bouton from the University of Vermont. Bouton has spent the last two decades trying to understand why old habits often return, studying this in rats using approaches very similar to those we discussed in Chapter 1 in the work of Anthony Dickinson. The phenomena that Bouton has studied go by several different names—spontaneous recovery, renewal, reinstatement, resurgence—but all seem to reflect a common effect in which earlier-learned habits return with a vengeance after they appear to have been lost. The phenomenon of resurgence is particularly relevant to many of the bad habits that we are interested in changing, so it’s worth describing it in a bit more detail. Evoking resurgence in rats is fairly simple. The rat is first trained to perform a particular action for food, such as pressing a lever (let’s call it lever A). The rat is then trained to perform a different action (let’s say pressing a different lever, B), while the original behavior is “extinguished,” meaning that the animal is no longer rewarded for pressing lever A. The animal quickly learns to press lever B and stops pressing lever A. What happens if the experimenter then stops rewarding the rat for pressing lever B? If the original habit of pressing lever A had been completely abolished, then one would expect the rats to simply do nothing, since they are not going to receive a reward for pressing either lever. If instead the original habit of pressing lever A is still lurking, then we should expect it to come back as soon as the lever B action

o n c e a h a b i t, a lway s a h a b i t 47

is extinguished. The latter is what happens: once the rat stops being rewarded for pressing lever B, it invariably starts pressing lever A again. This is just one of many examples from the work of Bouton and others showing that the first-learned behavior in any situation remains active in the background, just waiting to pop up again. Bouton’s work on resurgence and related phenomena has helped to cement the idea that when we supplant an old behavior with a new one, we are not actually forgetting the old habit—instead, we are actively inhibiting the original behavior so that the new behavior can emerge. What he has further shown is that this inhibitory learning is much more closely tied to the context in which it is learned than is the original habit, and this idea has important consequences for the treatment of many different disorders that involve habitual thoughts or actions, from phobias to post-traumatic stress disorder to obsessive-compulsive disorder. One of the most common treatments for these disorders is exposure therapy, in which the person is gradually exposed to the thing that they fear most. Michelle Craske is a psychologist at UCLA who has studied the treatment of anxiety disorders, and in 2002 she published a paper in which she and her colleagues treated a group of undergraduates who suffered from spider phobia. They describe the experimental setup in the driest of academic terms: “One nonpoisonous Chilean Rose-Haired tarantula (Phrixotrichus spatulata; legspan approximately 6 inches, or 15.2 cm) served as the phobic stimulus.”1 Over the course of about an hour, the students went from standing near the spider, to touching it with a gloved hand, to ultimately letting it walk on their bare hand. The treatment was very effective, and most of the students were able to let the spider walk on their hand by the end of the session. One important detail: There were two different contexts for the exposure therapy (different locations and other details), and each student was randomly assigned to one of the two contexts. The students returned seven days later to see how well the treatment had worked. Some of the students were tested in the same context where they had initially been exposed, while others were tested in a different context. While both groups showed much less fear of the spider a week after exposure than they had prior to treatment, Craske and

48

chapter 3

her colleagues found that changing the context reduced the effectiveness of the therapy—the students who underwent the follow-up in the same room as the initial treatment showed less fear than those who saw the spider again in a different context. These results, and others like them, provide another clue to the stickiness of habits. The habit system’s solution to the stability-plasticity dilemma is to assume that the world doesn’t change, so that whatever habit happens to get built first (in response to a particular stimulus) is assumed to reflect the enduring structure of the world. Thus, when a habit is developed, it becomes a sort of default behavior, such that it will be expressed across a range of different contexts. Any subsequent learning that aims to supplant the habit will be much more specific to the context in which it is learned, which means that when one ends up in a new context, the original habit is more likely to return. This has some important implications for how we might go about trying to override habits. In particular, it suggests that the effectiveness of exposure therapy can be increased by performing it across multiple contexts— and indeed some work from Craske’s group has shown this to be the case, though with varying levels of effectiveness.

The Transition to Mindlessness One of the most common unwanted habits is biting one’s fingernails— almost half of children and about 20% of young adults engage in the behavior. As habits go, nail-biting is certainly not one of the worst, though it can cause damage to the fingernails as well as dental problems. I was a nail-biter for many years, until my wife pointed out to me just what kind of bacteria live under our fingernails (I’ll spare you the disgusting details). A striking aspect about nail-biting, like many other motor habits, is that one often has no awareness of actually doing it. And an explanation for this is found in the way that habits are established over time in the brain. To understand this, we first need to dive more deeply into the structure of the basal ganglia. Remember from Chapter 2 that the striatum is connected to the cerebral cortex by a circuit known as a corticostriatal

o n c e a h a b i t, a lway s a h a b i t 49

loop. There are two aspects of the anatomy of these loops that are important for understanding how habits develop. First, these loops are fairly specific in their anatomy, such that the outputs of the loop return to roughly the same place in the cerebral cortex from which the inputs to the striatum arose. Second, different parts of the basal ganglia receive inputs from different parts of the frontal lobe, but these inputs are not random: specific regions in the cortex send their connections to specific regions in the basal ganglia. This is important because, as mentioned briefly before, different parts of the prefrontal cortex are involved in different functions. The rearward part of the frontal lobe, known as the motor cortex, is involved in the generation of movements; its projections to the basal ganglia are primarily directed at a part of the striatum known as the putamen. The part of the prefrontal cortex known as the dorsolateral (dorsal means “up,” lateral means “side”) prefrontal cortex is involved in higher cognitive functions, such as planning or holding information in working memory. The dorsolateral prefrontal cortex projects to a more forward part of the basal ganglia called the caudate nucleus. Finally, there are parts of the prefrontal cortex just above the eyes, known as the orbitofrontal cortex, that are primarily involved in emotional and social functions (known generally as “affective” functions), which project to a part of the basal ganglia known as the nucleus accumbens. What we now know is that different parts of the basal ganglia are differently involved in the establishment of habits. Whereas earlier research had treated the different parts of the rat’s basal ganglia interchangeably, a set of studies published in 2004 and 2005 by Henry Yin, Barbara Knowlton, and Bernard Balleine from UCLA demonstrated that these different parts of the basal ganglia actually play different roles in habit creation, at least for motor habits like pressing a button to get food (or, presumably, biting one’s nails). The part of the rat basal ganglia that is analogous to the human putamen is indeed necessary for the development of motor habits, as suggested in earlier work by Mark Packard, which is discussed in more detail in Chapter 4. However, the part analogous to the human caudate nucleus was shown to actually be involved in goal-directed (or what they called “action-outcome”)

50 c h a p t e r 3

learning. Yin and Knowlton subsequently developed a framework for understanding how habit learning proceeds in the brain.2 It starts with goal-directed learning that initially involves the “cognitive“ corticostriatal loop that connects the dorsolateral prefrontal cortex and caudate nucleus. Over time the “motor” circuit involving the motor cortex and putamen starts to learn the habit, and ultimately takes over from the cognitive loop. The transition from cognitive to motor circuits relies upon a particular way in which the striatum is connected to the dopamine system. One of the major inputs to the dopamine cells comes from the striatum, and this projection has a very similar loop-like structure to the corticostriatal connections—there is a topographic organization of these projections such that nearby parts of the striatum project to (and receive projections back from) nearby sets of dopamine neurons. However, there is a bit of a twist, literally—or, in the words of the neuroanatomist Suzanne Haber, a “spiral.” What happens is that portions of the striatum project to the dopamine neurons that connect back to them, but they also send some inputs to cells that project to regions that are closer to the motor system, such that the organization of connections between the striatum and dopamine cells looks like an upward spiral toward the motor system. This means that the striatal neurons that are part of the cognitive loop also send some input to the dopamine cells that ultimately send their outputs to the motor loop. Yin and Knowlton proposed that this feature of the dopamine system allows the cognitive system to slowly ingrain a motor habit by sending dopamine signals and thus modulating plasticity in the motor loop. As the habit becomes ingrained in the motor system, it becomes less amenable to inspection by the cognitive systems, leading to behaviors that we can become completely unaware of.

Becoming One: Habits as Chunked Actions You probably haven’t thought much about tying your shoes recently. When you first learned to tie your shoes as a child, it required attention to each cross of the laces and threading of the loops; now, you

o n c e a h a b i t, a lway s a h a b i t 51

just do it without thinking. This highlights another important aspect of habits, which is that they often consist of sequences of actions rather than a single action. This becomes evident when we look at the kinds of “action slips” that people make when executing habitual behavior. The psychologist James Reason had individuals keep a diary of action slips for two weeks, and reported an average of about 12 slips per person during that period. He identified many different types of errors, but the most interesting for our purposes were those in which habits take over and interfere with another goal. For example, one action slip that nearly all of us have had experience with is when we fail to interrupt a habit midway through: “I intended to drive to Place X, but then I ‘woke up’ to find that I was on the road to Place Y.” Another example is when a habitual action proceeds beyond our goal: “I went up to my bedroom to change into something comfortable for the evening. I stood beside my bed and started to take off my jacket and tie. The next thing I knew I was getting into my pajama trousers.” Anne Graybiel is a neuroscientist at MIT whose work has provided detailed insight into how our brain chunks actions as we acquire a new habit. Using all of the powerful neuroscientific tools we have described so far, her work has shown that as rats develop a new habit, activity in the basal ganglia “bookends” the sequence of actions that comprise the habit, such that once the sequence begins it can run to completion without additional activity. Graybiel’s research poses her rats with a very simple task, using a maze shaped like a T: they must run down a narrow passageway (starting from the bottom of the T) and then turn either left or right. She places food consistently on the same side so that the rats quickly learn to make the correct turn in order to get food as quickly as possible. Graybiel’s early research showed that when rats are first learning this task, there is activity in the striatum throughout their run; but as the turn toward the reward becomes a habit, the activity occurs primarily at the beginning and end of the action. In one particularly clever set of later studies, Graybiel and Kyle Smith examined a phenomenon that they had sometimes observed when their rats performed the task: occasionally, the rat would stop at the choice point (the junction of the T where the starting track meets the crossing track)

52

chapter 3

and look back and forth, which they refer to as “deliberation.”3 In Smith and Graybiel’s study, they measured activity in the striatum as well as a part of the rat’s prefrontal cortex (known as the infralimbic cortex) that is known to be essential for the formation of habits in rats. As the rats developed the habit, the “action-bracketing” pattern (in which activity occurs only at the beginning and end of the run) began to develop in the striatum, and as this happened the rats began to deliberate less. When Smith and Graybiel looked at the relation between the activity patterns in the striatum and the rats’ deliberation, they saw something very interesting: the likelihood of deliberating on any particular trial was related to whether the bracketing pattern had occurred in the beginning of the trial—well before any deliberation had occurred (at the choice point). They also saw that a similar pattern appeared in the infralimbic cortex but not until much later, around the time when the behavior actually became habitual. This result shows that, as habits develop, the striatum and prefrontal cortex work together to transform the action sequence into a single unit of action rather than a set of individual actions, making it much harder to stop in the middle of the sequence once it is triggered.

Trigger Warning: How Cues Trigger Habits How often have you heard someone else’s phone ring or buzz, only to immediately pull your own phone out to check messages? Every owner of a smartphone has almost certainly experienced this phenomenon, which has been given a complicated name by psychologists: Pavlovianinstrumental transfer. It is so named because it combines two different kinds of learning. Pavlovian learning occurs when a stimulus in the world becomes associated with an outcome of value—just as the bell came to signify food for Pavlov’s dogs, causing them to salivate. In the case of the phone, the “value” that we obtain is information (as we discuss further in Chapter 6). Instrumental learning refers to learning to perform particular actions in the context of particular situations or stimuli, just as we discussed in the context of reinforcement learning. Pavlovian-instrumental transfer refers to the fact that a cue associated with an outcome through Pavlovian learning (such as the sound of

o n c e a h a b i t, a lway s a h a b i t 53

someone else’s phone ringing) comes to elicit an action (checking for messages) that was acquired through instrumental learning. Pavlovian-instrumental transfer is thought to be particularly important in triggering bad habits. Simply seeing another person smoking or walking into a smoky bar can cause a smoker to have an immediate urge to light up a cigarette. This has been shown in studies with rats, which have been the most common model for studying the phenomenon. For example, in a study by Peter Holland,4 rats were first exposed to pairings of a sound with food, which led the rat to expect food when the sound was played; this relation between an intrinsically valuable stimulus (food) and an initially neutral stimulus (the sound) reflects Pavlovian learning, just as Pavlov’s dogs associated food with a bell. The rats were then trained to press a lever to receive food pellets (with no sound present); this is known as instrumental learning, reflecting the animal learning to perform a particular action in order to obtain a particular outcome. After this training, Holland first tested for Pavlovianinstrumental transfer by placing the rats in the cage and playing the sound, without any food present. Transfer is present when the rats are more likely to press the lever when the sound is present than when a different sound (not associated with food) is played. What he found was that rats who had experienced many hours of training on the instrumental lever press exhibited strong Pavlovian-instrumental transfer: that is, they were more likely to press the lever if they heard the sound that they had previously associated with food, even though there was no food available during this test. Holland then examined whether Pavlovian-instrumental transfer was affected by devaluation of the reward; remember from Chapter 1 that this is a hallmark of habitual behavior. To do this, he injected some of the rats with lithium chloride, which makes them lose their appetite. Rats who had only received a bit of training on the leverpressing response were less likely to press the lever after being injected with the toxin; this means that their behavior was goal directed, since the feeling of sickness reduced the attractiveness of the value of food. On the other hand, rats who had received many hours of training on the lever press continued to press just as much as the rats who were not

54 c h a p t e r 3

injected with the toxin, which is a hallmark of habitual behavior. When he examined the amount of Pavlovian-instrumental transfer, he saw that it was directly related to whether the response was habitual or not; rats with little training showed little transfer, whereas rats with many hours of training showed a large amount of transfer, and it was unaffected by devaluation of the food. That is, once the behavior becomes a habit, it can be triggered by a related stimulus even though the animal doesn’t actually want the reward anymore! Research like this has established that habits are particularly sensitive to this kind of transfer. Think of a smoker who is trying to quit and walks into a bar. The smell of the smoke, along with many of the other cues in the bar, will evoke the behavior most strongly in the person for whom smoking is a strong habit, even if they don’t actually want a cigarette anymore. One might reasonably ask whether humans exhibit the same kinds of transfer effects as rats, and indeed a number of studies have shown that humans do exhibit Pavlovian-instrumental transfer. In one study by Sanne de Wit and her colleagues,5 human subjects were first trained to press one of two buttons to receive one of two different foods (popcorn or chocolate candies). The subjects were then trained to associate a set of visual cues with the different food items. In order to test the effects of devaluation, the researchers then asked subjects to watch a television show for 10 minutes, during which time they were given a bowl of one of the two foods in order to satiate their desire for that specific food. The subjects then were given a test where they were asked to press buttons in order to receive the food of their choice, which would be given to them at the end of the session. The results demonstrated transfer: when the image associated with popcorn was presented, the subjects were more likely to choose the popcorn, whereas when the image representing the candy was presented, they were more likely to choose the candy. And similar to what Holland found in his rats, this effect was not affected by satiation—the effect of the cues on responding were about the same regardless of which food the person had been allowed to fill up on. Dopamine appears to play a central role in Pavlovian-instrumental transfer. Kate Wassum and her colleagues performed a study very

o n c e a h a b i t, a lway s a h a b i t 55

similar to the one by Holland, but did so while measuring dopamine levels in the rat’s nucleus accumbens using a technique called fast-scan cyclic voltammetry.6 This technique uses a very small carbon fiber electrode inserted in the brain, through which a particular pattern of electrical current is run; changes in dopamine levels result in differences in the electrical response of the electrode, allowing the researchers to quantify how much dopamine is present. Wassum and her colleagues found that the amount of dopamine increased over time after the sound that had been paired with food was presented, compared to presentation of a sound that had not been paired with food. When they examined the timing of the dopamine release, they found that larger dopamine releases often occurred just before the rats pressed the lever, tying dopamine more directly to Pavlovian-instrumental transfer. The results of the study by Wassum and colleagues were suggestive but only correlational; in order to causally link dopamine to Pavlovian-instrumental transfer, it’s necessary to show that interfering with dopamine also interferes with transfer. This was demonstrated in a study by Sean Ostlund, Kate Wassum, and colleagues, who used a new technique known as DREADDs (see Box 3.1) to interfere with dopamine signaling in specific regions of the brain.7 When they specifically interfered with dopamine signaling in the nucleus accumbens, they found that the Pavlovian-instrumental transfer was reduced, whereas interfering with dopamine signaling in a different part of the brain (the middle part of the rat’s prefrontal cortex) did not affect transfer. This shows that dopamine in the nucleus accumbens plays a central role in tying trigger cues to reward-seeking behaviors.

Can’t Look Away: Rewarding Stimuli Capture Attention When we look around the world, it’s only natural that some things are more likely to capture our attention than others. Often this occurs due to the features of the thing itself, such as its size or color, but other times our attention is drawn by more idiosyncratic features. For example, if you are a car aficionado, you are likely to have your eye drawn to

56 c h a p t e r 3 Box 3.1. DREADDs Alongside optogenetics, another set of methods known as chemogenetics have provided neuroscientists with an additional set of tools to manipulate brain function. The best known of these tools is “designer receptors exclusively activated by designer drugs” (DREADDs). The idea behind DREADDs is to insert a specifically designed receptor into the neuron, which can be controlled by a molecule (the “designer drug”) that does not occur naturally in the brain. Thus, the experimenter has complete control over the activity of those receptors and can use them to either excite or inhibit the neuron. A major difference between optogenetics and DREADDs is their timing: whereas optogenetic stimulation has an immediate effect on the neuron, DREADDs can take tens of minutes to kick in and can last for a couple of hours. This might seem like a checkmate for optogenetics, but there are often cases where researchers want to study the effects of activating or deactivating specific neurons over a longer time scale, and this is a place where DREADDs are beneficial. In addition, it can be easier to work with DREADDs, which only require an injection of the drug—optogenetics, on the other hand, requires insertion of an optical fiber into the brain, that must be held in place while the animal behaves, making it much trickier to allow animals to engage in unrestricted behavior.

a 1957 Ford Thunderbird, whereas if you are a bird watcher, you are more likely to be drawn to the green jay sitting on a post next to the car. This is generally known as attentional bias, and it features prominently in discussions of addiction, where individuals often have strong biases to attend to visual cues related to their drug of choice. One way that this has been demonstrated is in an adaptation of the Stroop task that was discussed in Chapter 1. Remember that in this task an individual is presented with words in colored ink and is asked to name the color of the ink. The Stroop effect occurs when it takes longer for someone to name the color of the ink when it is in conflict with the word itself (e.g., “blue” written in red ink). The idea here is that this slowing reflects interference that is automatically evoked due to the salience of the word. A prime example of this is seen in the addiction Stroop task, in which substance abusers are presented with pictures or words related to their addiction and are asked to respond to a simple feature, such as the color of the image. The general finding is that substance abusers are slower to respond to the stimulus compared to stimuli that are not related to their addiction, which is interpreted to show that

o n c e a h a b i t, a lway s a h a b i t 57

their attention is automatically drawn to the substance information, interfering with their ability to perform the more basic task.8 This means that not only do the cues for certain habits gain extra power through Pavlovian-instrumental transfer but they also become much more salient to the individual, making them even more likely to trigger the habit by “jumping out” at us. The mechanism behind this effect has been investigated by the psychologist Brian Anderson from Texas A&M University, who has argued that the attentional biases observed in addiction are not abnormal but are simply a manifestation of a basic psychological mechanism known as value-based attentional capture. In his studies, Anderson shows individuals a set of colored circles on a computer screen, each of which has a line that is either vertical or horizontal. On each trial, the participant is told to search for one of the circles based on its color (for example, “find the red circle”) and then told to report which way the line inside that circle is oriented. After each trial, the participant also receives a small monetary reward, which differs in its size depending on the color of the circle—for example, if the circle is red the subject might get a 5-cent reward 80% of the time and a 1-cent reward the rest of the time, whereas if it is green they only get the large reward 20% of the time. In order to test whether this value has rubbed off, he then gives a test phase in which the subject is asked to search for objects based on their shape rather than their color. In this test phase, the target shape is never a circle, but sometimes circles can appear as distracting alternatives. The question Anderson was interested in was whether having a distracting circle in the same color that was rewarded earlier would be more distracting (causing it to take longer to find the target shape), and whether highly rewarded colors would be more distracting than weakly rewarded colors. He found exactly that: people were slower to find the target shape when there was a distractor in either of the rewarded colors (versus colors they hadn’t experienced before), and the effect was even larger for the highly rewarded colors compared to those related to low reward. The effects are not large—tens of milliseconds—but they are robust, and Anderson has argued that value-based attentional capture

58 c h a p t e r 3

has many parallels with the attentional biases seen in addction: it is long lasting (with a duration of at least 6 months) and difficult to override. One small study has also shown that it appears to be linked to activity in the dopamine system, as people with higher levels of dopamine receptors in their basal ganglia appear to show higher levels of attentional capture. Thus, another reason that habits become difficult to overcome is that it becomes increasingly harder to ignore the cues that trigger the habit.

A Recipe for Stickiness All of these factors come together to make habits especially difficult to change. Let’s take the example of checking your messages on your smartphone. Initially, this begins as a goal-directed behavior, with the goal of gaining new information. Once the habit develops, it will be protected from disruption by the mechanisms uncovered by Mark Bouton’s work, discussed earlier. Over time the behavior moves from initially relying on corticostriatal loops involved in cognitive function to those involved in motor function, essentially removing it from direct oversight by the cognitive system. Say that you then decide to take a “dopamine fast” (the latest trend in Silicon Valley), and avoid checking messages on your device for a week. The habit of checking messages is still there, and you will have to override it in the context of your new goal to avoid the device, which requires effortful executive control; as we will see in Chapter 5, this kind of control is fragile and often fails. Over time this might get easier, but if the context changes (for example, you become stressed about an ongoing public health emergency), then you are likely to fall back into the old habit. The components of the action become cemented into a single unit (or “chunk”) that, once engaged, runs to completion automatically; before you know it, you can find yourself checking the phone for messages without any intention of doing so. Mechanisms including Pavlovian-instrumental transfer and value-based attentional capture also conspire to make it much easier to trigger the habit—your attention will be drawn by the

o n c e a h a b i t, a lway s a h a b i t 59

sound of another person’s phone buzzing with a new message, and you will then find yourself checking your own phone before you even know it. So far we have treated habits and goal-directed behaviors as completely separate things. In the next chapter, we turn to how it is that our brain’s systems for habits and those for goal-directed learning interact with one another to determine our behavior.

4 The Battle for Me

imagine walking into a convenience store with the intention to buy a snack: How do you choose what to buy? You might weigh all of the different options and try to find the one that best matches your desires for healthiness, tastiness, or cost, though that might take a long time. On the other hand, if this is a store you visit regularly, you might just go with the same snack that you usually buy. Or perhaps a new snack grabs your attention and you want some variety. When I say that “I” made a decision, this claim belies the fact that there are a number of systems in my brain that are working together— or sometimes against one another—to determine my actions. Just as there are multiple memory systems, there are also several different pathways to an action, each of which has important implications for our ability to control or change the behavior. First, there are reflexes— behaviors that are built into our nervous system by millions of years of evolution. When you accidentally touch your hand to a hot surface and recoil in pain, that is a reflex in action. Many reflexes rely upon very rudimentary parts of our nervous system; the withdrawal of one’s hand from a painful stimulus relies on the spinal cord rather than the brain, as do many other reflexes. Sometimes reflexes can also become associated with stimuli in the world, such that the stimulus becomes sufficient to elicit the behavior. These “conditioned reflexes” were famously studied by Ivan Pavlov; many readers will be familiar with Pavlov’s dogs, who learned that a bell foretold the arrival of food and came to salivate as 60

t h e b att l e f o r m e 61

soon as the bell was rung. At the other end of the spectrum are actions that are taken with an explicit goal in mind. These actions (which we refer to in Chapter 1 as goal-directed behaviors) include many of our daily activities; we eat a particular food because we think it is healthy or tasty, and we take a particular route to work with the explicit goal of avoiding traffic. As we have seen in the previous chapters, what distinguishes a goal-directed behavior from other kinds of behavior is that once the goal is no longer of interest, the behavior shouldn’t happen; if I eat too much cake with lunch then I won’t want more later in the day, and if I am listening to a radio story and don’t care about getting home quickly then I won’t bother trying to avoid traffic. In between reflexes and goal-directed behaviors lie habits. Habits are actions that at some point may have been goal directed, but with enough repetition become automatic, very much like a reflex—except that while reflexes are basically impossible to stop, habits can often be stopped with sufficient effort and attention.

A Competition in the Brain? The research described in the previous two chapters established that there are distinct brain systems for habits and goal-directed behavior; but our behavior appears to seamlessly combine these different influences, which raises a question: How do the different systems for choice relate to one another? When I began working in this area in the early 1990s, most researchers had thought that they operated completely independently of one another. However, a set of studies by Mark Packard, a neuroscientist now at Texas A&M University, began to suggest that they might actually be in competition with one another. Packard was interested in how different memory systems relate to different kinds of behavior—he didn’t use the terms habit and goal directed because he came from a different research tradition, much more in line with the ideas of multiple memory systems that we discussed in Chapter 2. Because so much of the focus of memory systems research was on the distinction between the basal ganglia and the medial temporal lobe,

62

chapter 4

Test

Training

Food

Response learning

Place learning

figure 4.1. (Left) A photo of a plus maze. (Right) A schematic of training and testing in Packard’s experiment. During training (left diagram), the rat is placed in the southern arm and has to learn to run to the eastern (rightward) arm to obtain food; the northern arm is blocked off. In the test (right diagram), there is no food and the rat is placed in the northern arm, with the southern arm blocked off. If the rat turns left, this means it has learned to go to a place in space, whereas if it turns right, it means the rat has learned to make a particular response. (Photo courtesy of Mark Packard)

Packard targeted those regions to understand their role in different kinds of learning. To test this, Packard set up a very simple task for his rats to perform, called a plus maze (Figure 4.1), which basically looks like it sounds. The maze is situated in a room with various decorations on the wall, so that the rat can tell where it is in the maze, because the arms otherwise look identical. Packard started by putting the rat into one arm of the maze after putting food in one of the other arms, requiring either a right or left turn to get to the food; the arm straight ahead of the rat was closed off, making the maze really more of a T than a plus from the standpoint of the rat, just like the maze used by Graybiel in the work I discussed in the previous chapter. With practice, the rat quickly learned which way to turn to get the food, but Packard reasoned there could be two different ways that this could happen. The rat might learn where in space the food was located, which he referred to as place learning; based on a large body of research tying spatial learning to the hippocampus, he predicted that the hippocampus would be necessary for this kind of learning. On the other hand, the rat might simply learn to turn in a particular direction, which he referred to as response learning. Notice the close parallels with goal-directed versus habitual behavior—in place

t h e b att l e f o r m e 63

learning the animal is moving toward a goal (a place in space), whereas in response learning it is simply repeating the action that produced a reward in the past. To test which kind of learning the rat used, Packard played a simple trick: He started the rat off in the opposite arm with no food present, and looked at where it went (as shown in Figure 4.1). If the rat learned to navigate to the place in space where the food is, then it should go to that location, which now requires turning in the opposite direction from what it had learned in the original arm. Conversely, if it learned to make a particular action, then it should turn in the same direction it had turned before, leading it away from the food. What Packard found was that the rats’ behavior differed depending on how much training they had received. Early in training, the rats exhibited the hallmark of place learning: when put in the opposite arm, they made the correct turn to get to the previous location of the food. However, with more training the behavior changed, with the rats now making the response they had learned before. Thus, habits developed with experience, just as they had in Dickinson’s original experiments. Packard then made an important mental leap. Previous work on memory systems had nearly always assumed that the different systems worked independently of one another, but Packard proposed that both systems were always learning but they then competed to determine how the animal would behave. This would imply that disrupting the brain system involved in one kind of learning should shift the animal to using the other kind of learning. On the other hand, if the other system was not involved, then the rat should simply start exploring the two arms randomly, since it wouldn’t have any way to know which arm had food. The results showed that the systems did indeed seem to be both learning at the same time and competing with each other to control the rat’s behavior: when he deactivated the hippocampus early in learning, he moved the animals toward using a response learning strategy, whereas when he disrupted the basal ganglia later in learning, the rats shifted to a place learning strategy. He also showed that he could shift animals toward using one kind of learning or the other by chemically stimulating the brain area that supports that kind of learning. This work

64

chapter 4

suggested that the brain’s different memory systems were in constant competition to determine how we behave.

Memory System Interactions in Humans Around the same time that Packard published his research, I had just finished my postdoctoral fellowship at Stanford and was starting a new lab at the Massachusetts General Hospital in Boston. My lab was also starting to think that the brain’s different memory systems were interactive rather than independent, as had been previously thought. In particular, it didn’t make sense to me that the habit system and the declarative memory system would operate completely independently from one other; given the highly interconnected and dynamic nature of brain activity, it seemed unlikely that these two systems wouldn’t somehow interact with one another in a healthy brain. I thought that the field’s focus on the study of patients with lesions had led us astray, and that we might be able to use brain imaging to see these interactions in action. Working with Mark Gluck and Daphna Shohamy of Rutgers University, we trained individuals on a task that required them to learn by trial and error to predict an outcome (in this case, whether it was rainy or sunny) based on a set of visual cues (cards with shapes). The cards had a probabilistic relationship to the outcomes—for example, one card might be associated with rain 65% of the time, while another might be associated with sunshine 80% of the time. Healthy subjects were able to learn to accurately perform this “weather prediction” task with practice, whereas earlier work by Barbara Knowlton and Larry Squire had shown that individuals with Parkinson’s disease had trouble learning to perform the task. This suggested to them that the basal ganglia and/or dopamine was necessary for this kind of trial-and-error learning. We intuited that a slight change in the way the task was learned could change the way the brain accomplishes the task, flipping it from using the habit system to using the declarative memory system. In the weather prediction task that Knowlton and Squire had initially developed, the subjects learned through trial and error; on each

t h e b att l e f o r m e 65

trial, they selected one of the two outcomes (rain or sunshine) and were given feedback that they could use to improve their performance over time. Given the role of dopamine in signaling reward prediction errors that we saw in Chapter 2, it was perhaps not surprising that patients with Parkinson’s disease (who have impaired dopamine signaling) were impaired at learning the task. We decided to design a task with one small tweak that was meant to change how the brain would treat the task: we still presented the individuals with the cue cards and with the weather outcome, but instead of having them learn by trial and error, we had them simply try to memorize which outcome went with each set of cues. We referred to this as paired-associate learning, since that term is often used in psychology to refer to situations where a person has to learn pairings between items. We used functional MRI to measure brain activity in the basal ganglia and medial temporal lobe while individuals performed either the trial-and-error or paired-associate version of the weather prediction task. When we compared brain activity across these two different versions of the task, we found results that were consistent with our understanding of the brain’s memory systems: there was greater activity in the basal ganglia for the trial-and-error version of the task, whereas there was greater activity in the medial temporal lobe for the paired-associate version of the task. We also found something that led us to agree with Packard that these two systems are in competition with one another: their activity seemed to move in opposite directions. Across different people, and across time within the same person, we saw that as basal ganglia activity went up, medial temporal lobe activity went down. We published these results in Nature in 2001.1 In later work led by Daphna Shohamy (now a professor at Columbia University), we also tested people with Parkinson’s disease on both versions of the task and found that, while they had problems learning the trial-and-error version (just as Knowlton and Squire had shown), the patients were able to learn the paired-associate version much more easily. Over time we have come to understand the limitations of the weather prediction task that we had used in our early studies—foremost, it didn’t provide a clear way to tell what kind of learning a person was engaged in (unlike Packard’s plus

66

chapter 4

maze). As we will see a bit later, another group of researchers inspired by the goal-habit distinction, along with new developments in computer science, would provide an even clearer way of understanding how these different systems work together in the brain.

Formalizing the Goal-Habit Distinction In San Francisco it’s common to see groups of three or four engineers driving around in small cars adorned with sensors on the roof. What might seem like some kind of nerdy party vehicle is actually a prototype of an autonomous vehicle (better known as a self-driving car), being developed by a number of companies in Silicon Valley. The challenges in building a functioning autonomous vehicle are immense, requiring a fast and effective computer implementation of many aspects of human intelligence. The quest for artificial intelligence began in the 1950s, and for many decades largely focused on developing systems that reasoned like humans on difficult tasks, such as medical diagnosis or chess. These approaches languished, unable to even start to solve human-level problems in a robust and flexible way. But in the twentyfirst century a different approach to artificial intelligence has shown itself to be much more adept at solving the kinds of problems that are necessary to acheive human-level intelligence. These methods, which go by the name machine learning, take advantage of very powerful computers along with large amounts of data in order to learn in a way a bit closer to how humans learn. In particular, an approach known as deep learning has been highly successful at solving a number of problems that vexed computer scientists for many years. When Facebook finds the faces in a uploaded photo and identifies the names of those people, it is using deep learning—not surprisingly, given that one of the godfathers of deep learning, Yann LeCun, now works for Facebook. Machine learning researchers typically distinguish different kinds of problems that a system (be it a human or a computer) has to solve as it learns about the world. On one end are problems known as supervised learning, in which the system is told what the right answer is

t h e b att l e f o r m e 67

and must simply learn to repeat that answer in the appropriate context; think of a child learning the names of different animals from a parent. On the other end are problems known as unsupervised learning, in which the system has no teacher whatsoever and must simply look at the world and try to identify its structure based on those observations. Infants engage in unsupervised learning when they listen to their parent’s speech and identify the speech sounds that are important for the particular language being spoken by the parent. In between these two types of learning sits reinforcement learning, which we have already heard about in the context of dopamine. In reinforcement learning, the system must learn the appropriate actions based on feedback from the world, but it isn’t told explicitly what the right answer is— it simply receives carrots or sticks depending on whether it makes the right choice. Before computer science even existed as a field of study, reinforcement learning was being studied by psychologists interested in how learning works.2 Unlike physics, psychology has very few laws, but one of the best-established laws in psychology is the law of effect, first coined by the American psychologist Edward Thorndike in 1898. This law states that when an action is followed by a pleasant outcome it will occur more often in the future (in the particular context that led to the outcome), whereas if an action is followed by an unpleasant outcome it is less likely to occur in the future. Throughout the twentieth century psychologists (particularly those focused on learning in animals such as rats or pigeons) worked to understand the basis for the law of effect, but one of the foundational ideas came from researchers studying another kind of learning known as classical conditioning—or sometimes as Pavlovian learning since it is the form of learning observed by Pavlov when his dogs began to salivate at the sound of a bell that preceded feeding. In the 1970s the psychologists Robert Rescorla and Allan Wagner were particularly interested in understanding a phenomenon that occurs during learning known as blocking. Previous theories had proposed that animals learn which events go with which others in the world by simply registering their co-occurrence. This would suggest that anytime a reward occurs in association with an action, the animal

68

chapter 4

should learn to perform that action more frequently. However, in 1968 the psychologist Leon Kamin showed that the association between a stimulus and a reward could be blocked if the reward was already associated with another stimulus. For example, in grade school we might come to associate the sound of a particular bell with lunchtime, such that the occurrence of the bell would make us start to salivate. What Kamin showed was that if another stimulus was later added—for example, a flashing light along with the bell—the relationship between that second stimulus and the outcome was blocked, such that if the light occurred later on its own, it would not elicit the same response as the bell. This showed that the brain was not simply recording which stimuli went together in the world. Rescorla and Wagner developed a mathematical theory of learning based on the idea that learning depends on the degree to which their predictions in the world were violated— exactly the same idea of reward prediction error that we encountered in our discussion of dopamine. While this particular theory has largely been superseded by newer approaches, it cemented the concept of error-driven learning in psychology. The idea of a mathematical model of reinforcement learning might sound complicated, but the basic concept is actually quite simple. Let’s say that a person walks into a very small casino with only four slot machines. The person knows that some of the machines in this particular casino are much better than others but doesn’t know which specific machine is good or bad. The reinforcement learning model provides us with the means to describe how a person (or robot) can learn which machine to play in order to maximize their winnings. The most basic model has several major components. The policy describes how actions are chosen in any particular state. In general, this is based on the estimated value of each possible action in that particular state. In our casino example, we would need to estimate the value of the winnings from each of the slot machines. Starting out we don’t really know these values, so we would just assume that they are all the same; the goal of the reinforcement learning model is to learn the values through experience. The simplest policy would be to just choose the machine that has the highest estimated value at any point in time, but as we will

t h e b att l e f o r m e 69

see shortly, this is problematic. Instead, we usually want a policy that allows some degree of exploration, such that we can occasionally pick a machine that we don’t currently think is very good just to make sure that we are right about that. The model also needs a reward signal, which tells it the outcome of its action. In our case, this is simple—we just record whether we won or not for each trial. Let’s look at what happens as the model starts to play the slot machines,3 which is shown in Figure 4.2. If we are the casino owner, then we actually know the true probability of winning for each of the machines (shown at the top of the figure), which in this case varies from 85% for the best machine to 10% for the worst machine, with the others in between. However, in the beginning the model doesn’t know anything about the expected payout from the different machines, so we would set the expected value associated with each of the four machines to zero. Since all of the values are the same, we need some way to break the tie, which we usually do by introducing some degree of randomness into our action selection mechanism. One common way to do this is by using a softmax policy, in which we choose actions with a probability that is proportional to their value relative to all the other actions. In this first trial, since the values are all the same, each of the actions would have a 25% likelihood of being chosen. Suppose that we randomly choose machine 2 on the first trial, and we happen to win $1 (which for that machine happens 40% of the time). The next job of the model is to update its value estimates based on that experience— actually, based on how our experience differs from our expectation. In this case, the value that we expected for machine 2 on the first trial was $0, but the actual reward value was $1; the reward prediction error is thus 1. We update the value estimate for machine 2 by adding the reward prediction error to the existing value estimate, but only after multiplying it by a relatively small number (called a learning rate) to make sure that any particular win doesn’t have a huge impact on our value estimates—this helps prevent our behavior from changing too quickly based on limited evidence, stabilizing our behavior over time. Using a learning rate of 0.1, our updated value estimates would be 0.1 for machine 2 and 0 for the others. We then use this updated value estimate

70

Bandit 1 (85%)

chapter 4

Bandit 2 (40%)

Bandit 3 (35%)

Bandit 4 (10%)

Play Bandit 2, win $1 Starting value

0

0

0

0

Softmax action selection Probability of choosing

0.25

0.25

0.25

0.25

Play Bandit 2, win $1 Prediction error

0

1

0

0

Multiply by learing rate (.1) New value

0

0.1

0

0

Play many rounds Learned value

0.82

0.20

0.08

0.01

Softmax action selection Probability of choosing

0.92

0.04

0.02

0.02

figure 4.2. An example of a reinforcement learning model using slot machines. Each bandit starts with an estimated value of zero, which leads to a 25% chance of choosing any of the four machines. Machine 2 is chosen (based on a random selection), and the player wins a $1 reward. The difference between the actual reward ($1) and the predicted reward ($0) leads to a prediction error value of 1. This value is multiplied by the learning rate (0.1) to give a new value for machine 2, while the values for the other machines remain unchanged. After many rounds, the player comes to choose the highest-paying machine (bandit 1) more often than the others.

on the next trial. Note that the machine with the highest expected value after the first trial (machine 2) is not actually the machine with the highest payoff in the long run (machine 1). However, if we were to simply choose the machine with the highest value estimate (which we refer to as greedy action selection), then we would be stuck choosing machine 2 forever simply due to the fact that we randomly chose it on the first

t h e b att l e f o r m e 71

Probability of choosing each machine

1.0

0.8

0.6

Bandit 1 Bandit 2 Bandit 3 Bandit 4

0.4

0.2

0.0 0

200

400 600 Number of trials

800

figure 4.3. Performance of the reinforcement learning model based on the slot machine example, showing that it increasingly comes to choose the high-value option (solid line) over the other options during the course of many training trials.

trial. Instead, we need to include some degree of exploration, as mentioned before. In this example, the softmax policy that we are using will result in the different actions being chosen with probabilities related to their value estimates, allowing the learner to occasionally explore other options. An example of the behavior of the reinforcement learning model learning to choose slot machines is shown in Figure 4.3. In the beginning, all the different machines are chosen with roughly equal proportions. However, over time the greater proportion of wins for machine 1 leads its value to be increased and thus the probability of choosing it to become much greater than the others. In this way, this very simple reinforcement learning model learns over time to select the action that results in the greatest reward. Why is a mathematical model of learning relevant for our understanding of habits? Remember our discussion in Chapter 2 of the work of Wolfram Schultz, who studied how dopamine cells in the monkey’s

72

chapter 4

brain responded to rewards and the cues that predicted them. His research showed that the firing of dopamine neurons very closely matched the difference between actual outcomes and predicted outcomes—exactly the prediction error that is computed in the reinforcement learning model. Subsequent research by Hannah Bayer and Paul Glimcher made this link even tighter, showing a strong mathematical relationship between the activity of dopamine neurons in the monkey’s brain and the prediction error values obtained from a reinforcement learning model.4 This is an example of what is now a burgeoning research area, known as computational neuroscience, in which models from computer science are used to understand how brains work.

Model-Based versus Model-Free Reinforcement Learning The reinforcement learning model that we described above doesn’t have any knowledge about how the world works—it simply tries all of the possible actions and learns which one leads to the best outcome on average. Researchers refer to this (somewhat confusingly) as model-free reinforcement learning, because the learner does not have a model of how the world works. In a simple situation like our slot machine example, this is fine; but in the real world, it becomes untenable. Let’s say that I want to drive from my house in San Francisco down to the Stanford campus, which is about 40 miles south in Palo Alto. On this route, there are more than 20 street intersections and 20 freeway exits. Without a map, I would have to try each option at each intersection and each exit in my attempt to get to campus, but it’s clear that this would be futile—there are just too many possibilities. Computer scientists refer to this problem as the curse of dimensionality, meaning that as the number of possible choices increases, the number of combinations of those choices increases much faster. With one intersection, there are three possible choices: assuming I can’t make a U-turn, I can either go straight, turn left, or turn right. With two intersections, there

t h e b att l e f o r m e 73

are a total of 9 possible combinations of choices, and the numbers get exponentially larger, such that for 20 intersections there are more than three billion possible combinations of turns (that is, three to the twentieth power) that I would have to try in order to fully map out the route! Because of this, it quickly becomes impossible to try all possible actions in all possible states of the world. The model-free learner also can’t deal well with changes in the world. Let’s say that by some miracle I was able to get to the Stanford campus without a map; I remember that route and it becomes my go-to. However, one day the freeway exit that I had discovered happens to be closed for construction. Without a map, I have no idea how to get to my destination, and I’m stuck trying out all the other options blindly. The model-free learner sounds remarkably silly, but it turns out that it actually seems to provide a good description of how habits work, in the sense that it simply performs the learned response given the situation, without regard to goals or other knowledge. Another kind of reinforcement learning system, known as modelbased reinforcement learning, uses structured knowledge to understand how the world works and make decisions accordingly. When we think of a “model” of the world, we often use the concept of a map. This could be a map of a physical space (like a road map), but it could also be some other kind of “cognitive map” that outlines our knowledge of the world. As an example, let’s say that you need to get from your current location to the airport for a flight. Just as a physical map might show the different roads you could take to get there, a cognitive map also outlines the different ways you could get to the airport (subway, taxi, rideshare) and the different actions needed to acheive each of those. For example, the cognitive map tells you that you will need to pay the taxi driver at the end of the ride, whereas you don’t pay the rideshare driver because the app takes care of it. And it’s also sensitive to the context—for example, the fact that your ridesharing app doesn’t work in some cities and that you are expected to tip the taxi driver in the US but not in Italy. A cognitive map might also model the ways in which your actions change the world. For example, you might be unhappy with how slowly the rideshare driver is driving you to the airport, but

74

chapter 4

you know that complaining to the driver might negatively affect your passenger rating, which could have consequences down the road, so you hold your tongue and be patient. A model-based reinforcement learner uses this kind of cognitive map to determine the best action to achieve the goal, and it’s clear that much of our behavior must utilize model-based reinforcement learning in order to be effective. At the same time, habits seem to be well described by a model-free system; having learned that one particular action is the way to go, we just keep doing it. The Princeton neuroscientist Nathaniel Daw has spent his career trying to understand how these two kinds of learning work together in the human brain. Daw trained as a computer scientist, and then spent several years working with the legendary computational neuroscientist Peter Dayan at University College London. Inspired by ideas from computer science and neuroscience, Daw has used a combination of brain imaging and computational modeling to understand how each system works in the brain and how the systems relate to one another. In order to study the question of how model-based and modelfree reinforcement learning work in the human brain, Daw needed to develop an experimental task in which both of them could be tested. The task he designed has become known as the two-step task and has been used extensively in humans and rodents to study decision making. It is so named because it involves two subsequent decisions; it’s usually done using colored shapes on a computer screen, but I’ll describe it in somewhat more familiar terms (see Figure 4.4 for a depiction of the task). Imagine you are entering a new building on a treasure hunt. You initially have the choice of two doorbells to press—let’s call them the circle and triangle buttons. These buttons cause one of two doors to open, in a probabilistic fashion; for example, the circle button will open the door to the circle room 70% of the time and the triangle room 30% of the time, whereas the triangle button has the converse effect. Once you enter the door that opens, you then have another choice. There are two doors (which we will call door A and door B) in each of the rooms, and behind each of these doors there may be a reward, again with some probability. For example, in the circle room there might be an 80%

t h e b att l e f o r m e 75

A

80%

B

20%

A

20%

B

80%

70%

30%

30%

70%

figure 4.4. An example of Daw’s two-step learning task. The participant first chooses one of the doorbells (left), which leads to one of the doors opening with a particular probability. The participant then chooses either door A or B in the next room (right), and receives a reward with some probability.

chance of reward behind door A and a 20% chance behind door B, with the probabilities reversed in the triangle room. First, let’s think about how a model-based learner would approach this task. In this case, the “model” is a description of the probabilities of transitions between the states (that is, how likely each button press is to open each door) and the probabilities of reward for each action in each final state (that is, which door is best within each room). The learner would first learn how often each of the button presses at the first step leads to each of the two rooms. They would also learn how often each of the doors at the second step leads to a reward in each

76

chapter 4

state. With this knowledge, they would choose the action at the first step that leads to the second step with the most valuable action, and then choose that action at the second step. A model-free learner, on the other hand, doesn’t have a mental model of the task, so it simply learns which sets of actions lead to reward. Daw intuited that one could distinguish between model-based and model-free learning by looking at what happens when an infrequent transition happens at the first step and the chosen action is rewarded—for example, when the triangle button leads to the circle room and door A is rewarded. The modelbased learner would realize that this reward means that the value of the state at the second step (the circle room) is higher (since they received a reward in that state). However, they would also realize that the choice they took at the first step (the triangle button) is relatively unlikely to lead to that state, so they would actually be less likely to choose the triangle button next time—that is, they are learning about how particular states of the world relate to rewards. The model-free learner, on the other hand, would simply record which actions lead to rewards and thus would be more likely to choose the triangle button next time. A number of studies by Daw and his colleagues as well as others have shown that both humans and rats will generally engage in modelbased learning in the two-step task, but that the degree of model-based control differs across individuals. One group of studies has shown that there are individual differences between people in the degree of model-based versus model-free control. However, there are some reasons to think that these differences might reflect situational variables (such as how stressed or tired the person is when they complete the task), rather than necessarily reflecting stable individual differences in model-based control. One study published by my colleagues and me tested 150 people twice, with several months in between, on a version of the two-step task.5 We found that there was a very weak relationship between the degree of model-based control exhibited at the two time points by any particular person, suggesting that there may not be stable differences between people in the degree to which they exhibit model-based control.

t h e b att l e f o r m e 77

There is also evidence that specific situational factors can affect the deployment of model-based versus model-free reinforcement learning. In particular, distraction seems to drive people toward the use of modelfree control. Ross Otto, now a faculty member at McGill University, demonstrated this in a study that was inspired by earlier work from Karin Foerde, Barbara Knowlton, and myself. In our study, we had individuals learn to perform the weather prediction task that I discussed earlier, either while focused on the task or while distracted by a secondary task (keeping a mental count of how many times a particular sound had occurred). What we found was that distraction didn’t reduce the participants’ ability to accurately predict the weather (which we thought relied upon the habit system), but it did substantially reduce their conscious memory for what they had experienced. Otto and his colleagues presented subjects with the two-step decision-making task, which they performed under either focused or dual-task conditions. What he found was that whereas subjects behaved as modelbased learners when they were focused, they were more likely to use model-free learning when they were distracted. There is a large body of research that implicates the prefrontal cortex in multitasking, and Otto’s results are consistent with the idea that the prefrontal cortex is necessary for model-based decision making, such that engaging it in multitasking reduces its effectiveness and allows the model-free system to win the competition.

Can Goals Become Habitual? The habits we have discussed so far are mostly simple and closely tied to motor actions, such as a rat pressing a lever or a person choosing a specific food from a vending machine. However, many of the “habits” that we are concerned about in the real world actually look much more like goal-directed behavior. Perhaps the best example is drugseeking habits (which we explore much more in Chapter 6). The internet is littered with stories by emergency medical professionals of individuals engaging in complex schemes to obtain drugs, such as this one:

78

chapter 4

Recently had a dishevelled female present to a school claiming to have been bitten on the leg by a poisonous snake. A good Samaritan applied a pressure immobilisation bandage to her leg before we arrived. On arrival she had a very vague story of kicking a dead snake in a nearby park which then had bitten her. She had a small single scratch like mark above her ankle. Upon loading she immediately begged for pain relief, citing allergies to everything except opioid. She answered yes to all of my symptom questions and was very vague in her answers. On arrival at hospital, it was revealed she had presented to three other metro hospitals via ambulance that week with the same complaint.6 This individual clearly has a drug habit, but the means by which she attempted to obtain the drug are far from a simple action, suggesting that what is habitual is the overall goal of obtaining and consuming drugs rather than the individual actions (which will necessarily vary depending on the particular circumstances). While most research has focused on habitual actions, there is increasing interest in the idea that goals can become habitual as well. Fiery Cushman is a psychologist at Harvard University whose work has focused on understanding how thoughts and goals can be learned through model-free reinforcement learning, which ties them more directly to habits. In one set of studies, Cushman used an adaptation of Daw’s two-step task to examine whether random large rewards would affect the future choice of a particular goal.7 The experimental setup is rather complex, but the idea can be seen in a real-world example (see Figure 4.5). Let’s say that someone has money to invest and must choose between investing in either tech stocks or financial stocks. The system only allows the user to choose between two stocks on any purchase, one from each category of interest. However, the system also has a bonus feature; on a small proportion of trades chosen completely at random, the user receives a large bonus, which is equally likely to occur regardless of which particular stock the user wished to buy on that round. Let’s say that the buyer is given the choice between two stocks,

t h e b att l e f o r m e 79

Super Bank

MoneyBags

Financials

CompuTech

PearComp

Tech stocks

BONUS!

SETUP TRIAL

TEST TRIAL

SuperBank or CompuTech?

MoneyBags or PearComp?

Choose SuperBank (financial goal)

Does the bonus on the setup trial rub off on the financial goal state?

Recieve BONUS! figure 4.5. A schematic example of the task used by Cushman and his colleague Adam Morris to examine the model-free selection of goals. There are two financial stocks and two tech stocks, and on each trade the trader has the opportunity to purchase one of them. However, on a small number of trades chosen at random, the trader receives a large bonus. The critical test for model-free influence is whether the receipt of a bonus after selection of a stock from one group leads the trader to be more likely to choose the other stock from that group on a subsequent trade; that is, does the bonus increase the value of the goal that was being pursued when it was received?

chooses a financial stock, and then receives a bonus. A model-based system would realize that the bonus is completely random and would not change the value of the financial stock goal. However, a model-free system simply learns to associate choices with outcomes, so the bonus would cause it to increase the value that it places on the goal of buying a financial stock. In a set of studies, Cushman demonstrated exactly this kind of model-free learning of goal states, suggesting that goals may be learned in a way similar to habits. More generally, this study and others like it suggest that we should view reinforcement learning in a more hierarchical way, in which a model-based system can select habits and model-free learning can affect the value we place on goal states. So far we have discussed how habits and goals combine to drive our choices and why habits are so sticky. In particular, habits appear to become unmoored from the goals that initially drive us to engage

80

chapter 4

in them, and become the default behavior that must be overridden in order to change the habit. We have also seen how the distinction between habits and goal-directed behavior appears to map closely onto the concepts of model-free versus model-based reinforcement learning. In each of these examples, we have talked about the need for planning or control over behavior, which involves the prefrontal cortex. In the next chapter, we turn to understanding exactly how the prefrontal cortex plays a role in controlling our behavior, and why it often fails.

5 Self-Control t h e g r e at e st h u m a n st r e n gt h?

the battle between our immediate desires and our long-term goals has been evident ever since humans started thinking about our own minds. Plato, in Phaedrus, likened the human soul to a chariot pulled by two horses—one noble horse who is a “lover of honor with modesty and self-control,” and a beastly horse that fills the owner with “tingles and the goading of desire”—with the charioteer reining in these opposing passions in order to guide our behavior. The idea of a battle between our inner rational thinker and our passionate enjoyer was most famously popularized in the writings of Sigmund Freud, who conceptualized mental life as a battle between the pleasure-seeking Id and the moralistic and logical Superego, with the Ego responsible for mediating the conflict. We now turn to the question of how the brain exerts control in service of our goals. To begin, let’s see what psychologists mean by “self-control.” For each of the following statements, decide whether it accurately describes you or not: • • • • •

I often act without thinking through all the alternatives. I am lazy. I have trouble concentrating. I do certain things that are bad for me, if they are fun. I say inappropriate things. 81

82 • •

• •

chapter 5

I have a hard time breaking bad habits. Sometimes I can’t stop myself from doing something, even if I know it is wrong. I wish I had more self-discipline. Pleasure and fun sometimes keep me from getting work done.

These are all items from a popular survey used to measure self-control in research studies. If you said “no” to all or most of these items, then you would likely fall toward the high end of the self-control spectrum. On the other hand, if you said yes to very many of them, then you would be considered to have relatively low self-control. Part of the reason that psychologists focus so much on self-control is that it seems to have powerful effects on many important life outcomes. Some of the most compelling evidence for this has come from research by Terrie Moffitt and Avshalom Caspi from Duke University, who have spent years following a group of more than 1000 individuals born 1972– 1973 in Dunedin, New Zealand. They first measured self-control when these individuals were children, starting at 3 years old, by simply asking parents, teachers, and the children themselves whether they showed evidence of self-control problems, such as acting before thinking, difficulty waiting or taking turns, tendency to “fly off the handle,” and low tolerance for frustration. In a set of studies, Moffitt and Caspi have examined how these early measures of self-control relate to social, educational, and health outcomes in adulthood. The results are striking, to say the least: Nearly every positive life outcome is better for the children who had higher self-control at a young age. They are more likely to be financially successful, have better physical health, are less likely to have drug and alcohol problems, and are less likely to be convicted of a crime, just to name a few of the outcomes. Perhaps most importantly, higher self-control appeared to help these individuals avoid what Moffitt and Caspi called “snares,” or life choices that end up trapping individuals into undesirable outcomes—such as starting to smoke at an early age or dropping out of school. You might have noticed that the statements in the survey refer to many different aspects of psychological function, including planning,

s e l f - c o n t r o l 83

motivation, concentration, pleasure-seeking, and inhibition, just to name a few. We can think of these as the “psychological ingredients” of self-control, and we will see that all of them involve a part of the brain that is uniquely evolved in humans: the prefrontal cortex.

What’s Up Front? Often the best way to understand the role that a brain region plays in mental function is to observe what happens when it is damaged. Perhaps the best-known case of frontal lobe damage is Phineas Gage, whose story was first told in the Ludlow, VT, Free Soil Union on September 14, 1848: Horrible accident—As Phineas P. Gage, a foreman on the railroad in Cavendish [Vermont] was yesterday engaged in tamkin for a blast, the powder exploded, carrying an iron instrument through his head an inch and a fourth in circumference, and three feet and eight inches in length, which he was using at the time. The iron entered on the side of his face, shattering the upper jaw, and passing back of the left eye, and out at the top of the head. The most singular circumstance connected with this melancholy affair is, that he was alive at two o’clock this afternoon, and in full possession of his reason, and free from pain. The tamping rod did a massive amount of damage to Gage’s face and skull (as shown in Figure 5.11 )—and remember that this was before the days of modern surgery, antibiotics, and painkillers. Yet through the astute care of his physician, John Harlow, he was able to survive the ensuing infections, and within a few months he was able to engage in some daily activities. However, while he could seemingly walk and talk normally, something had changed. Prior to his injury, Gage was described as “One of the most efficient and capable foremen” employed by the contractor, “a shrewd, smart businessman,” and “energetic and persistent in carrying out his plans.” In a later report with the matter-offact title “Recovery from the passage of an iron bar through the head,” Harlow described how Gage’s demeanor changed after the accident:2

84

chapter 5

figure 5.1. (Left) Photographs of Phineas Gage’s skull alongside the tamping rod that caused his brain injury. (Right) A reconstruction of Gage’s brain injury by Van Horn et al., CC-BY

The equilibrium or balance . . . between his intellectual faculties and his animal propensities seems to have been destroyed. He is fitful, irreverent, indulging at times in the grossest profanity (which was not previously his custom), manifesting but little deference for his fellows, impatient of restraint or advice when it conflicts with his desires, at times pertinaciously obstinate, yet capricious and vacillating, devising many plans of future operation, which are no sooner arranged than they are abandoned in turn for others more appealing. . . . In this regard his mind was radically changed, so decidedly that his friends and acquaintances said he was “no longer Gage”. Some authors, particularly Malcolm Macmillan of the University of Melbourne,3 have questioned whether Harlow and others may have

s e l f - c o n t r o l 85

exaggerated Gage’s impairment, noting that Gage did actually hold jobs for most of the period between his accident in 1848 and his death in 1860. Nonetheless, it is clear that his personality changed substantially after the injury, offering a striking example of the psychic changes that can result from damage to the frontal lobe. While some people’s behavior becomes disinhibited and inappropriate after frontal lobe injury, other people show very different kinds of effects. In the 1940s and 1950s a common treatment for major mental illness (such as depression or schizophrenia) was the frontal lobotomy, in which parts of the prefrontal cortex are surgically disconnected from the rest of the brain; this has the same effect as removing that part of the brain altogether. A widely known case of lobotomy was Rosemary Kennedy, sister of US president John F. Kennedy. Rosemary suffered from intellectual disabilities and mental illness, and often exhibited fits of rage as well as what appeared to be epileptic seizures. This was before the advent of psychiatric medications, and without any other treatments available, the Kennedy family decided to have a frontal lobotomy performed on Rosemary at age 23. After the surgery, Rosemary was almost completely debilitated—she lost the ability to walk and talk, and would remain institutionalized for the rest of her life. Some of the most detailed studies of the effects of frontal lobe damage on personality have been performed by Daniel Tranel and his colleagues at the University of Iowa. Tranel is the keeper of the Iowa Neurological Patient Registry, which for almost 40 years has recruited and tracked individuals who have been treated for brain disease or injury at the University of Iowa Hospital. These individuals are contacted regularly to participate in research studies, and the size of the database (currently over 3500 individuals) has made it a one-of-a-kind resource for the study of the effects of brain damage. A study published in 2018 examined 194 individuals from this registry to assess the kinds of personality changes that result from brain damage.4 Looking at individuals with damage across the entire brain, they found that almost half of these individuals showed some kind of change in their personality, which took several different forms. The most common effect was social and emotional dysfunction, similar to what was seen in Phineas

86

chapter 5

Gage. Another common effect was “executive dysregulation,” involving a range of symptoms including lack of judgment, indecisiveness, and social inappropriateness. Another group showed symptoms similar to those observed in Rosemary Kennedy’s case, involving apathy, social withdrawal, and a lack of stamina or energy. Finally, some individuals showed signs of emotional distress and anxiety. Interestingly, sometimes damage to the frontal lobe can actually have a positive effect on one’s personality. In another study made up of 97 patients from the Iowa registry, Marcie King, Tranel, and colleagues asked a relative or friend to rate the individual’s personality characteristics before and after their brain damage.5 Somewhat surprisingly, they found that more than half of the patients had some aspect of their personality that was rated as improved after their brain damage compared to before, and that these individuals were more likely to have damage to the farthest forward (or anterior) parts of the frontal lobe. Some of the examples provided in the paper show just what kind of changes occurred. One patient had been highly irritable and outspoken prior to undergoing surgery for a tumor in her frontal lobe, and was described as “stern” by her husband. After the surgery, she became much happier and outgoing, and her husband noted that she smiled and laughed more. Another patient had been frustrated and angry prior to a brain aneurysm, often complaning about his job and being temperamental with his daughter; he was also described by his wife as being “mopey.” After the aneurysm, which caused damage to part of his prefrontal cortex, he became much more easygoing and content, and both he and his wife described the changes in his personality as being positive. These findings show that some of the deepest aspects of our personalities live in the far reaches of the frontal lobes, even the not-so-good aspects.

Why Is the Prefrontal Cortex Special? If we ask what makes the prefrontal cortex so important for self-control, the key feature is its wiring. To understand this, let’s take a look at the different areas in the cerebral cortex (see Figure 5.2).6 The primary

s e l f - c o n t r o l 87

Heteromodal association (prefrontal)

Unimodal association

Primary

Motor

Visual

figure 5.2. The prefrontal cortex sits at the top of the hierarchy of brain systems, which starts with primary systems (bottom), goes to unimodal association regions (middle), and finally connects to the heteromodal association regions that include the prefrontal cortex. Lines are drawn with arrows in both directions to highlight the fact that connections in the brain are almost always reciprocal.

brain regions are dedicated to processing inputs from a specific sensory modality (such as vision, touch, or hearing) or directly generating output signals to control movement. These are the gateways for information to come into or out of the brain. The association regions integrate information from the primary regions and relate it to existing knowledge. Some association cortices, known as unimodal, primarily process information from a single sensory modality. Others, known as heteromodal, combine information across different sensory modalities. These regions are arranged in a hierarchy, with the prefrontal cortex at the very top, receiving input from each of the lower-level unimodal cortical regions. In addition, even within the prefrontal cortex there

88

chapter 5

is a hierarchy, with regions toward the front processing more complex information. In this way, the regions at the top of the hierarchy (which sit at the very front of the prefrontal cortex) have access to an “executive summary” of all the brain’s available information. It was once thought that the human prefrontal cortex was outsized compared to other primates, but recent evidence using MRI scanning of the brains of many different primate species (monkeys, great apes, and humans) has shown that the size of the human prefrontal cortex relative to our brain size is at least roughly similar to that of other apes. Why, then, do humans seem so different in their ability to plan, wait, and control themselves? For one, it does seem that the sections of the prefrontal cortex that are involved in the most abstract kinds of thinking (at the very front of the brain) may be relatively bigger in humans. Another answer may lie in the wiring of the brain. Underneath the gray matter of the brain, which houses the bodies of all of its neurons, lies the white matter, which serves as the conduit for wiring that connects different parts of the brain to one another. Some studies suggest that parts of the white matter in humans may be larger (relative to our overall brain size) than that of other primates, which could lead to greater connectivity both within the prefrontal cortex and with the rest of the brain. However, if there is a difference, it is nowhere near as striking as the apparent differences in intelligence between humans and other primates. It seems that microscopic differences in how the neurons of our brains are organized, such as how the neurons are spaced out in the tissue and how branched they are in parts of the prefrontal cortex, may hold another key to the differences between species. The ways in which these differences in the architecture of tissue give rise to huge differences in cognitive ability between species are yet to be fully understood.7 The prefrontal cortex is also the last part of the brain to develop (which will be of no surprise at all to anyone who has spent much time around adolescent children). The process of brain development is somewhat counterintuitive, in that an early explosion in the number of both neurons and synapses is followed by a prolonged period of “pruning” in which unnecessary neurons and connections are removed. This growth and subsequent pruning happens earliest in the primary

s e l f - c o n t r o l 89

sensory and motor regions, finishing within a few years after birth, whereas in the prefrontal cortex the pruning doesn’t really kick in until mid-childhood and isn’t complete until early adulthood. One way that we can see this is by looking at the thickness of the gray matter across development, which we can measure using MRI images. The cerebral cortex is generally about 3–4 millimeters thick, though this thickness varies across the brain. One landmark study by Elizabeth Sowell and her colleagues showed that cortical thickness decreases across most of the brain during late childhood (between 5 and 10 years old), but actually increases during the same time in parts of the frontal and temporal lobes that are most associated with language functions. The white matter that connects the prefrontal cortex to the rest of the brain also matures relatively slowly. The brain’s white matter is composed of a set of wirelike appendages (axons) that carry signals from a neuron to its target. Axons are covered with a substance called myelin that insulates the axons, like the plastic coating on an electrical wire. Myelin also helps speed up transmission along the axon, by allowing the signals to jump quickly over sections of the axon. The process of myelination begins in utero and continues throughout childhood, but in the prefrontal cortex it takes much longer, extending into early adulthood. Some of the best evidence for this protracted white matter development comes from studies using a kind of magnetic resonance imaging known as diffusion-weighted imaging (see Box 5.1). In one study led by Peter Kochunov and David Glahn, changes in both the thickness of the cortex and the microstructure of the white matter were measured in more than 1000 people who ranged in age from 11 to 90.8 They found that a measure of the structural integrity of the white matter known as fractional anisotropy, which is thought to relate to myelination, peaked much later than a measure of cortical thickness across the entire brain. The protracted development of white matter in adulthood was especially apparent in white matter regions connecting the prefrontal cortex to the rest of the brain. These results, along with many others, show that white matter continues to develop long after the cortex has reached its adult zenith. Having outlined the reasons that the prefrontal cortex seems to be so well placed to exert control over the rest of the brain, we now turn

90 c h a p t e r 5 Box 5.1. Imaging white matter using diffusion-weighted imaging We often refer to the white matter as “wiring,” but this obscures an important fact about axons: they are both filled with and bathed in fluid that is largely composed of water. In the late 1980s the MRI physicist Mike Moseley realized that it might be possible to image the structure of the white matter by taking advantage of a technique known as diffusion-weighted MRI that allows us to measure the movement of water molecules at the microscopic scale. Individual water molecules move around in a random way as they are bumped into by other molecules, resulting in a phenomenon known as diffusion, in which molecules will travel a particular (very small) distance over time. If a water molecule is in the middle of a bucket of water (far from the sides), it is equally likely to diffuse in any direction. If we look at the average movement of many such molecules, this will look like a round ball, which we refer to as isotropic motion, meaning that it’s the same regardless of which direction we measure it in. However, imagine that you are a water molecule floating around in the microscopic space between several axons that are all aligned in the same direction. The myelin on the axons is a fatty substance that repels water, meaning that the water molecules in between axons are much more likely to diffuse along the direction of the axons and less likely to diffuse in the direction perpendicular to the axons—which we refer to as anisotropic diffusion. If we average the resulting diffusion patterns, they will look more like a cigar than a ball, showing much more diffusion in one direction (the direction of the axons) than the others. Imaging the brains of cats using an MRI technique that measures the diffusion of water along a specific direction, Moseley showed that there was indeed greater diffusion when the direction of the scan was parallel to the known orientation of axons, compared to when it was perpendicular. The discovery that diffusion-weighted imaging could be used to image the structure of white matter led to the development of a number of techniques for measuring the structure of white matter, the most notable of which is called diffusion tensor imaging. This technique involves the collection of diffusion-weighted images along six different directions, which allows us to fit a mathematical model to the data that quantifies the direction and shape of diffusion at each point in the brain. In particular, we can compute a measure known as fractional anisotropy that quantifies the degree to which the diffusion is isotropic or anisotropic. While this is not a pure measure of myelination, it is related, and it has allowed researchers to gain a much better handle on how the structure of white matter relates to many different aspects of brain function and development.

to some of the “basic ingredients” of self-control that are supported by the prefrontal cortex.

Holding Information in Mind One of the essential ingredients for behaving in a planful manner is the ability to hold information about one’s goals in mind over time, and the prefrontal cortex seems uniquely important for doing this. This was

s e l f - c o n t r o l 91

Cue

Delay

Response

figure 5.3. A schematic of the oculomotor delayed response task used by Goldman-Rakic. In the cue phase, the monkey keeps its eyes focused on the fixation point at the center of the screen, but remembers the location of the target (shown in red at the top right). In the delay phase, the monkey remains focused on the fixation point, holding the target location in mind. When the center point disappears, that is the monkey’s cue to move its eyes to the location of the target (which is not shown on the screen).

demonstrated in a long line of research by the late Yale neuroscientist Patricia Goldman-Rakic, who studied how neurons in the prefrontal cortex of the macaque monkey responded while the animal performed a simple task (see Figure 5.3). In this task, the monkey is first trained to fixate its gaze on a point in the middle of a screen, known as a fixation point. While it remains fixated on the point, it is then shown a cue that appears at some location away from the fixation point, in its peripheral

92

chapter 5

vision. The monkey remains fixated until the fixation point disappears, at which point it is supposed to move its eyes to the location where the cue had appeared (but is now gone). If it moves its eyes to the correct location, it receives a squirt of fruit juice in its mouth as a reward. This is called an oculomotor delayed response task because it involves making an eye movement after a delay. When Goldman-Rakic recorded the firing of neurons in a particular part of the prefrontal cortex while the monkeys did this task, she found that some neurons were activated only by the cue and others became active only when the monkey moved its eyes after the fixation point disappeared. However, she also found some neurons whose activity was “delay-selective”—that is, the neurons became active only during the period when the monkey was waiting to move its eyes. The firing of these cells was selective to a particular direction of motion, meaning that any particular cell only fired when the upcoming eye movement was in a specific direction. Goldman-Rakic and her colleagues also established that dopamine was critical for working memory. In one study, they administered a drug directly into the prefrontal cortex that blocked the function of dopamine D1 receptors in monkeys.9 As the drug took effect, the monkeys made many more errors in remembering the intended eye movements, and the errors became greater as the delay between the cue and the response became longer, suggesting that the memory was being degraded over time. In a later study, they applied the dopamineblocking drug while also recording from neurons in the prefrontal cortex.10 While the drug had minimal effects on neurons that were responding to the cue or the movement, it resulted in decreased activity in the cells that responded during the delay. In the decades after Goldman-Rakic and her colleagues demonstrated the presence of sustained activity while animals held information in working memory, it became widely accepted in neuroscience that this persistent activity was responsible for holding information in working memory. However, in recent years the idea that working memory requires sustained activity in the prefrontal cortex has been questioned, and there is increasing reason to think that the story is more

s e l f - c o n t r o l 93

complicated. In particular, research by the MIT neuroscientist Earl Miller and his colleagues has shown that most neurons in the prefrontal cortex do not fire persistently during the delay period in working memory tasks, especially when the tasks become more complicated than those used by Goldman-Rakic. Instead, these neurons seem to exhibit bursts of strongly synchronized activity in between periods of relative inactivity. There are many reasons to think that these bursts of activity are actually much more effective as a means to hold onto information: as Miller and his colleagues said in a 2018 overview of their research, “in the constant chatter of the brain, a brief scream is heard better than a constant whisper.”11 In particular, the patterns identified by Miller and his colleagues allow populations of neurons to hold onto more information, in essence allowing different groups of neurons to “speak” at different points in time. Despite these complications regarding how it is accomplished, it remains clear that neurons in the prefrontal cortex are essential for holding information in mind.

The Biology of Being Frazzled Amy Arnsten was a postdoctoral fellow with Patricia Goldman-Rakic when she started studying a different neuromodulatory system that is in many ways the neglected cousin to dopamine: the noradrenergic system, which releases the neurochemical noradrenaline. Most people are familiar with adrenaline, which is the hormone that kicks in when we get anxious or excited—noradrenaline is just a small chemical change away from that molecule. Noradrenaline is also chemically very similar to dopamine and in fact is created directly from dopamine in the brain; both are members of a class of neurochemicals called catecholamines. This conversion happens in a very small region of the brain known as the locus coeruleus, buried deep in the brain stem; its name is Latin for “blue spot,” due to the fact that the region appears blue when viewed in a dissected brain. Just like the dopamine system, the locus coeruleus sends its projections widely across the brain, especially to the prefrontal cortex. And just like dopamine it also appears to play a central role in working memory.

94 c h a p t e r 5

Arnsten and Goldman-Rakic first identified the role of noradrenaline in working memory by studying what happens as monkeys get older. Older monkeys, just like older humans, have poorer working memory, and they also have decreased numbers of receptors for noradrenaline. Just as dopamine has different classes of receptors, there are also different classes of receptors for noradrenaline as well, known as alpha and beta receptors; and in this case, too, the different classes of receptors have opposite effects. Many people take a type of drug called a “beta-blocker” for high blood pressure, which gets its name from the fact that it blocks a specific version of noradrenergic receptor known as the beta receptor; this also highlights the fact that chemicals like noradrenaline and dopamine play many roles across our entire body, not just in our brain. Beta receptors are responsible for the usual effects of adrenaline that we think of; in fact, many people (including myself at one time) take beta-blockers when they have to speak in public, because they reduce some of the symptoms of anxiety.12 Arnsten and GoldmanRakic focused on the role of the other class of receptors, looking at a specific version called the alpha-2A receptor. This receptor has the opposite effect of the beta receptors, and in fact drugs (such as clonidine) that activate alpha-2A receptors are also used to treat high blood pressure. What Arnsten and Goldman-Rakic found was that activating alpha-2A receptors in the older monkeys improved their ability to remember the location of the intended eye movement over a delay, having its biggest effect at longer delays.13 In the long and illustrious career that has followed, Arnsten (now a professor at Yale University) has worked out in detail the biology of how this works, and in particular has focused on understanding why stress wreaks such havoc on our prefrontal cortex. In particular, Arnsten has argued that there is an “inverted-U” relationship between the level of catecholamines in the prefrontal cortex and the function of the neurons there (see Figure 5.414 ). Just as in the classic story of Goldilocks and the three bears, the level of catecholamines in the prefrontal cortex needs to be “just right” for optimal function; if the level is too low (as it is thought to be when we are sleepy) or too high (as occurs when we are under extreme stress), the prefrontal

Improved task performance Enhanced prefrontal function

s e l f - c o n t r o l 95

Focused

Fatigued

Frazzled

Increasing arousal/stress More dopamine/noradrenaline

figure 5.4. A schematic of the “inverted-U” relationship between arousal and performance, first described by Yerkes and Dodson. As the level of arousal or stress goes up, the amount of catecholamines (dopamine and noradrenaline) in the prefrontal cortex also goes up. This leads to improved functioning of the prefrontal cortex and better task performance, up to a point, where it then starts to break down. (Adapted from Arnsten et al.)

cortex becomes unreliable and our ability to think and plan goes out the window. This relationship is thought to underlie another one of psychology’s basic laws, known as the Yerkes-Dodson law after the two psychologists who described it in 1908. Yerkes and Dodson were interested in learning about how stress, caused by shock to a mouse’s feet, affected its ability to learn how to differentiate the brightness of two stimuli. They found that when the discrimination was easy, increasing shock led to better performance on the task, but when the discrimination was difficult, there was an upside-down U-shaped relationship, such that increasing shock led to better performance until a point when it then started to degrade performance. Although this early study was not done to the standards that we would hold research to today, its results have held up over the last century, and the idea that there is an optimal level of arousal for performance is now ingrained as a basic tenet of psychology.

96 c h a p t e r 5

A particularly compelling demonstration of the effects of stress on cognitive function comes from a set of studies by researchers from the US Army, who studied a group of Navy special operations soldiers (known as SEALs) going through a high-intensity training exercise referred to as “Hell Week.”15 This exercise is so difficult that more than half of the soldiers who start voluntarily withdraw before the end (meaning that they are rejected from the prestigious SEAL program), in part because they are forced to engage in intense physical activity and discomfort with almost no sleep. Examination of cognitive performance after the first three days of Hell Week showed that the soldiers were badly impaired on tests of memory and attention, compared to their performance just before the week started. For example, on a task that required them to learn a sequence of keystrokes on a computer keypad, the solders took more than twice as long to learn the task after three days of intense stress. It is the inverted-U relationship between catecholamines in the prefrontal cortex and neuronal function that explains in part how one’s state of arousal impacts prefrontal function. When we are alert and interested, moderate amounts of noradrenaline are released into the prefrontal cortex, which optimizes the function of neurons by making their patterns of firing more precise. These moderate levels of noradrenaline engage a particular group of noradrenaline receptors (the alpha-2A receptors mentioned above), which strengthens connectivity between neurons in the prefrontal cortex, allowing them to better hold onto information over time. Research by Arnsten and her colleagues showed that applying the drug guanfacine (which activates alpha-2A receptors) directly to the prefrontal cortex in monkeys during a working memory task caused neurons in the area to fire more precisely when a low level of the drug was applied, while high doses of the drug disrupted the ability of neurons to fire during a delay period.16 The high levels of noradrenaline released into the prefrontal cortex with uncontrollable stress also engage a different type of receptor, the alpha1 receptor, which suppresses the firing of neurons in the prefrontal cortex and impairs working memory performance. Interestingly, drugs

s e l f - c o n t r o l 97

that block the alpha-1 receptor are currently being tested to treat posttraumatic stress disorder, though so far the results appear to be mixed. Dopamine is also released into the prefrontal cortex in relation to the level of arousal and appears to have similar effects to noradrenaline. Arnsten and Goldman-Rakic demonstrated this in an experiment with monkeys who were subjected to loud noises while performing the working memory task that I described above. This noise impaired the animals’ ability to perform the working memory task, though they were able to perform other tasks (that did not require working memory) perfectly fine. Drugs that blocked dopamine function in the prefrontal cortex improved performance of the stressed monkeys on the working memory task, even though the same drugs actually reduced performance of nonstressed monkeys, highlighting the degree to which dopamine levels need to be “dialed in” for optimal performance. ( Just when you thought dopamine couldn’t get any more complicated!)

The Waiting Is the Hardest Part If there is one psychological experiment that is familiar to most casual readers, Walter Mischel’s “marshmallow study” might be it. In reality, there was no single study but rather a series of studies, starting in the 1960s when he was on the faculty of Stanford University. In these studies, Mischel tested children who were students at the Bing Nursery School, which sits on the Stanford campus and is heavily populated by the children of its faculty. Children were brought into a “surprise room” where they were told that they would be able to play with some toys after a short time. But before that, the children were shown two different rewards that varied in how desirable they were, such as two pretzels versus two marshmallows. Although it’s widely known as the marshmallow study, there were actually several different rewards used, depending on the desires of the children who had been tested before the study, so that even marshmallow-haters would still have a desired reward. The children were told that the experimenter was going to leave the room, and if they waited for the experimenter to

98

chapter 5

come back they could have the more desired reward (such as the two marshmallows), but if they didn’t want to wait they could ring a bell, which would summon the experimenter. In that case, they would only receive the less desirable reward (the two pretzels). In some cases both foods were present during the waiting period, while in other cases the experimenter took them away during that time. Mischel tested many children on different versions of this problem, from which he learned a number of important things. First, the children actually had remarkably good self-control in one way: very few of them ate the treat without ringing the bell and waiting for the experimenter to return. Second, children’s ability to resist temptation depended on what they had to look at during the delay. If the foods were both there in front of them, then the children were remarkably bad at waiting, persisting for only about a minute on average, whereas if neither of the foods was present, children were much more patient, waiting more than 11 minutes on average. Even if the rewards were hidden, children had trouble waiting if they were cued to think about the rewarding items during the delay, whereas if they were given something distracting to think about, they were able to wait much longer. But the most striking results of Mischel’s studies came when he followed up with his participants more than 10 years later. The teenagers who had been able to wait as children were most likely to be described by parents as being verbally fluent, attentive, competent, and dependable. On the other hand, teenagers who had trouble waiting as children were more often described as immature, stubborn, and tending to “go to pieces” under stress. Mischel was also able to obtain SAT scores for a subset of the children who had participated in his studies, and the results of this analysis have become perhaps the most fabled finding from his research. For the 35 children who had participated in the version of the task that was most taxing on self-control—that is, the version where both rewards were present and the child was not provided with any strategies to help them resist temptation—there was a relatively strong relationship between SAT scores and waiting time. However, this relationship did not exist for the other versions of the test, which for Mischel showed that the

s e l f - c o n t r o l 99

taxing version of the test was most effective at measuring the kind of willpower that was important for later success. Mischel readily admitted that the sample size of 35 children was far too small to make any strong conclusions, but as is often the case, the subsequent discussions of the results dropped this important caveat, and it soon became common parlance that this study had demonstrated that the ability to delay gratification was an essential component of success, right alongside intelligence. Subsequent research has largely confirmed that there is indeed a relationship between the ability to delay gratification and later life outcomes, but it has also shown that the relationship is more complex than often portrayed. In particular, two important studies have taken advantage of a relatively large dataset known as the Study of Early Child Care and Youth Development (SECCYD), which was funded by the US National Institute of Child Health and Development. This study followed more than 1300 children through development and measured a large number of educational and psychological outcomes; it also measured delay of gratification when the children were 4 years old. One major asset of this dataset is that it is much more diverse than the relatively affluent and white population that took part in Mischel’s studies, so it provides a much more generalizable view of the relationship between self-control and life outcomes. On the other hand, a major limitation of the dataset is that the subjects were given only a maximum of 7 minutes to ring the bell, whereas Mischel had provided them with at least 15 minutes. Because of this short waiting time, only about half of the children rang the bell before time was up, which makes analysis of the data difficult compared to Mischel’s design, in which very few of the children waited the entire time when both rewards were present. In a first study, Angela Duckworth and her colleagues examined how the ability to delay gratification in this dataset was related to a number of outcomes when the children were in eighth or ninth grade.17 They found that waiting on the delay of gratification test at 4 years of age was related to higher GPA and standardized test scores in adolescence. Interestingly, they also found that waiting was related to body

100

chapter 5

mass index; children who had trouble waiting were more likely to be overweight. They also tested whether the relationship between selfcontrol and academic outcomes was caused by greater intelligence in children who waited longer, using an additional set of self-control and intelligence measures that had been collected on these children. They found that the relationship between waiting and grade point average was primarily driven by self-control and not by intelligence. However, both self-control and intelligence were important for the relation between the delay task and standardized test scores, probably reflecting the fact that self-control is more important for day-to-day classroom behavior than for a single high-stakes testing session. A second study of the SECCYD dataset by Tyler Watts and his colleagues at New York University subsequently raised bigger questions about just how predictive the ability to delay gratification is for later life outcomes. Watts compared waiting data from age 4 with outcomes measured at age 15 (slightly later than those tested by Duckworth and her colleagues). Importantly, he also split the data according to whether the child’s mother had attained a bachelor’s degree; when he did this, he saw that the children from nondegreed mothers were much more demographically representative of the country compared to those with degreed mothers, who were more likely to be white, affluent, and come from two-parent families. The differences in self-control between the children in these two groups were striking. The children from degreed mothers waited on average about 90 seconds more than the children with nondegreed mothers and were also less than half as likely to ring the bell within the first 20 seconds; they also (unsurprisingly) performed better on standardized academic achievement tests. These differences in waiting were strongly related to socioeconomic variables, but only for children from nondegreed mothers; for the other children, there probably wasn’t enough variability in socioeconomic status between homes to find any relationship with waiting. When Watts compared waiting to academic performance for the children from nondegreed mothers, he found a relationship, but it was about half as strong as the one that had been reported by Mischel. Perhaps

s e l f - c o n t r o l 101

more importantly, he found that the relationship between waiting and academic performance in these children was primarily present for kids who rang the bell in the first 20 seconds; for kids who could wait longer than that, there didn’t seem to be a relation between how much longer they waited and how well they did academically. One complication with these results is that it’s not always clear that waiting is the right thing for a child to do, and in fact sometimes it might be the wrong thing to do. Imagine you are a child living in an impoverished household with food instability in a socially dysfunctional neighborhood, and you are brought into an experimental situation to perform a delay of gratification task. It is likely that children from such an environment have experienced crime and deception by adults, which might lead them to rationally believe that they should take what they can as soon as they can, in order to make sure that they actually receive it. This question was examined in a study by Celeste Kidd and her colleagues.18 Before participating in a version of the marshmallow task, their subjects (children between the ages of 3 and 6) first met with an experimenter as part of an “art project”. The experimenter told the child that they could play with a set of used crayons, or they could wait for the experimenter to return with a set of new art supplies. For one set of children, the experimenter returned as promised with those art supplies, whereas for another set of children, the experimenter returned and apologized that the other art supplies were not available. The children then participated in the delay of gratification task, and as expected, the children who had seen the experimenter to be unreliable were much less willing to wait for the good reward than the children who had seen the experimenter follow through as promised. Other research has also shown that children are much less likely to wait when they don’t trust the experimenter, or when they don’t trust other people in general. These findings show that it can be very difficult to disentangle the different factors that may lead to relationships between delay of gratification and academic performance, but the findings nonetheless provide a good basis for thinking that the ability to delay gratification is a reliable correlate of success later in life.

102 c h a p t e r 5

Now or Later? Much of our understanding of the brain systems underlying patience comes not from studies of children eating marshmallows, but from adults making decisions about monetary rewards available at different points in time, which we refer to as intertemporal choice. In the standard intertemporal choice test, individuals are given a choice between a particular amount of money now (say, $20), and a larger amount of money later (say, $30 in two months). Individuals complete many of these kinds of choices, which vary in the relative goodness of the delayed reward compared to the immediate reward. When people make these kinds of choices, they tend to overweight immediate rewards more strongly than economic theory says they should. One consequence of this is that their choices are “dynamically inconsistent,” meaning that their relative preference for different outcomes changes over time. Let’s say that today I offer you a choice between $20 in two weeks or $30 in four weeks. Nearly everyone will choose the larger/later reward. However, now fast-forward two weeks, such that the same choice becomes $20 today versus $30 in two weeks. In this case, many people will switch their preference and take the immediate reward. This suggests that people will differently value the same outcomes depending on when they are considering them, violating the basic rules of classical economics. It also bears a close resemblance to behavior in other domains relevant to behavior change, such as the decision in the morning to abstain from drinking today that suddenly is reversed once the cocktail hour arrives. We can quantify how patient or impatient a person is by estimating a single number that describes how quickly they discount future rewards, which is generally referred to as k. Figure 5.5 shows examples of discounting functions for two individuals with different levels of k. The patient individual (on the right) discounts future rewards relatively little, whereas the impatient individual (left) discounts them very quickly, leading the two individuals to make different decisions about whether to take $10 now versus $17.50 in two months. With high enough k, discounting becomes so steep that it is as if the impatient person

s e l f - c o n t r o l 103

Future reward amount

High discounting (impatient)

Low discounting (patient)

30.0

30.0

27.5

27.5

25.0

25.0 Wait for later reward

Wait for later reward

22.5

22.5

20.0

20.0

17.5

17.5

15.0

15.0

12.5

12.5

Take $10 now

10.0

Take $10 now

10.0 0

20

40 60 Time (days)

80

100

0

20

40 60 Time (days)

80

figure 5.5. Example discounting curves for two individuals, one (left) who discounts quickly and the other (right) who discounts more slowly. In each plot, the line shows the minimum amount of future reward that the individual would require in order to wait for the reward compared to receiving $10 now. The lightly shaded area highlights combinations of delays and reward amounts where the individual would choose the immediate reward, while the darkly shaded area denotes combinations where the individual would be willing to wait. The star in each plot denotes a specific choice where the two would disagree: $10 now versus $17.50 in two months. The impatient individual on the left would take the immediate reward, whereas the patient individual would wait for the delayed reward.

only cares about immediate rewards; anything in the future is basically worthless to them. When we look across people, we see that k differs widely. For example, one large study measured k in more than 20,000 people and found that the highest k value across these people was more than 1000 times larger than the lowest value! In that study, the most patient person would prefer to wait 30 days for $20.10 over an immediate reward of $20, whereas the most impatient person would require a delayed reward of $167 in 30 days in order to give up an immediate reward of $20. These differences between people in their discounting rates appear to arise from a combination of genetic and environmental influences. We already saw some of the environmental influences above,

100

104 c h a p t e r 5

when we discussed the marshmallow task. If a person doesn’t trust others, then they are more likely to take the immediate reward rather than waiting for a delayed reward that they don’t trust others to actually deliver. Another factor that likely impacts discounting rates is one’s socioeconomic status. The behavioral economists Sendhil Mullainathan and Eldar Shafir have proposed that when someone experiences scarcity (as the poor do on a daily basis), their attention is so focused on solving their immediate problems that thinking about the future just doesn’t make sense. This could be why poor individuals take out high-interest payday loans; as Mullainathan and Shafir showed in a set of studies, scarcity (in this case, in a video game) causes people to focus more on immediate needs and thus be more likely to borrow against their future.19 Consistent with this idea, research has also shown that income has a direct relationship with discounting rates, such that lower-income individuals show faster discounting of delayed monetary rewards compared to higher-income individuals.20 We also know that genetics plays a role in discounting, though the degree of its impact is unclear. Studies of identical and fraternal twins have shown that identical twins are more similar in their discounting rates than are fraternal twins, and from these data those researchers estimated that about half of the variability in discount rates between individuals is due to genetic differences.21 Several previous studies had claimed to find specific genetic differences that were related to discounting, with many of them focused on genes related to dopamine function. However, these studies were relatively small, which (as I discuss in more detail later) can often lead to irreproducible results. Only one large study (with more than 20,000 participants) has analyzed the entire genome in order to find specific genes that are related to discounting (see Box 5.2 for more on genome-wide analysis). This study identified one gene that was related to discounting, though on its own it explained very little of the differences between people in discounting rates—in fact, differences across the entire genome only explained about 12% of the differences in discounting between people. Importantly, this wellpowered study did not find evidence for a relationship between

s e l f - c o n t r o l 105 Box 5.2. Genome-wide association studies The Human Genome Project has provided researchers with new tools to measure how traits of interest (such as personality or behavioral traits) are related to genetic differences across individuals. The human genome is made up of more than three billion individual building blocks known as bases (signified by the letters A, C, G, and T, which refer to the four different nucleic acids that are combined to make up our DNA). About 25% of those bases fall within genes, which are the regions of the genome that contain the instructions for how to create the proteins that make up our cells. One of the important findings from the Human Genome Project was that most of these letters are exactly the same across people, but each individual has differences in specific locations, which are referred to as single nucleotide polymorphisms (SNPs). While everyone has changes to their DNA that are rare, occurring only in themselves or their family, there are only a few million places in the genome where there are different versions of the DNA sequence that are relatively common across people. The technology for determining which version of each common SNP an individual has at each of these points has become relatively cheap, such that anyone can (at time of writing) obtain a “genome-wide” analysis from the company 23andMe for about $200, which examines about 640,000 locations across the genome. When researchers wish to understand the relationship between traits and differences in specific genes, they generally perform what is called a genome-wide association study (or GWAS). In such a study, the researchers obtain information about a large number of SNPs for each individual, as well as measurements of the trait of interest. They then perform statistical tests to determine whether the trait differs, depending on the version of each SNP. This involves a very large number of statistical tests, and in order to avoid false positive results the researchers must use very stringent statistical corrections. However, that stringency means that the researchers can only find results with very large sample sizes, generally at least 10,000 people. In addition, it is common in genetics to require that any finding is replicated in a different sample to ensure that it is robust. This means that a GWAS is very expensive to perform and usually requires the combination of data from many different sites. However, the results are much more reproducible than previous genetics studies that used much less stringent analyses with individual genes rather than the entire genome. In general, GWASs rarely find that differences at any one location in the genome account for more than 1% of the differences across people, particularly for complex psychological traits, but across the entire genome differences can account for much more of the variability across people. This makes sense from an evolutionary standpoint for any behavior that might be related to reproductive fitness: with a few exceptions, any genetic change with a strong positive effect on fitness should quickly sweep the population (as did the genes that allowed humans to start speaking), whereas any change that has a strong negative effect should quickly be weeded out.

discounting rates and any of the genes that had previously been implicated in those small earlier studies of individual candidate genes.

106

chapter 5

Just as behavior on the marshmallow task is related to life outcomes, so is discounting of future monetary rewards. This has been shown most clearly for drug abuse, where a large number of studies have confirmed that individuals with drug addictions show substantially faster discounting than do nonaddicted individuals.22 This doesn’t tell us that discounting causes drug addiction; it may be that addiction causes people to discount more heavily or that a common factor (such as socioeconomic status or childhood trauma) results in both addiction and faster discounting. One bit of evidence pointing toward discounting as a risk factor for addiction, rather than a result of addiction, comes from a study that followed individuals over time and measured the relationship between discounting and smoking.23 The study found that differences in discounting were predictive of later smoking, whereas the advent of smoking did not predict changes in discounting, which was very stable over time. At the same time, the importance of discounting for drug abuse has almost certainly been overstated by researchers, some of whom have gone so far as to claim that fast discounting of monetary rewards is a “behavioral marker of addiction,”24 meaning that measures of discounting could be used to predict who is going to become addicted. In our research, we have found that discounting rates are indeed related to multiple aspects of substance abuse and obesity, but that they account for less than 5% of the differences between people in drug or alcohol use and obesity. Others have also failed to find strong relationships between the patterns of drug abuse and measures of delay discounting. This means that treating discounting rates as a “marker” of addiction would almost certainly label many nonaddicted people as potential addicts, and seems like a potentially dangerous overreach.

Two Minds in One Brain? The sight of children participating in the marshmallow task is so compelling that a video of it went viral on YouTube, reaching more than seven million views.25 When watching these children resist the temptation to eat the marshmallow, it’s hard not to envision their rational brain battling their pleasure-seeking desire centers. Indeed, as I mentioned at

s e l f - c o n t r o l 107

the outset of this chapter, the idea of a battle of reason versus passion appears to be as old as human thought. Within both neuroscience and economics, the idea of a battle between impulsive and rational brain systems has played a major role in explanations of intertemporal choice, in the form of dual systems theories of decision making. These theories propose that there are two brain systems that play a role in decision making. One system, referred to as the “doer” by the economists Thaler and Shefrin,26 the “hot system” by Walter Mischel, and “System 1” by Daniel Kahneman, is an automatic system that drives us toward fast and immediate consumption of rewards without regard to goals. This system is usually associated with brain regions that are engaged by reward, including the nucleus accumbens, the ventromedial prefrontal cortex, and the dopamine system. A second system (referred to variably as the “planner,” “cold system,” or “System 2”) is thought to be a rational, goal-directed, and patient thinker. This system is generally linked to the lateral parts of the prefrontal cortex, which have long been associated with what neuroscientists call cognitive control processes. These processes, which include holding information in working memory, resisting distraction, planning future actions, and inhibiting unwanted actions, are thought to be the basic ingredients of self-control. The initial evidence that intertemporal choice can be understood as a battle between these two sets of brain systems came from a study published in 2004 by Sam McClure and his colleagues. Their study was based on a specific implementation of the dual-system model that had been proposed by the economist David Laibson, known as the “betadelta model.” According to this model, there is one system (“beta”) that only cares about immediate rewards, in effect placing zero value on any future rewards, and another system (“delta”) that discounts rewards over time in a much slower way that is consistent with standard economic theory. They collected fMRI data while subjects made choices between an earlier versus a later monetary reward; sometimes the earlier reward was obtained immediately ($10 today versus $11 tomorrow), while on other trials both rewards were delayed ($10 in 7 days versus $11 in 8 days). To identify areas related to the immediate (“beta”) system,

108 c h a p t e r 5

they compared activity on all trials where an immediate reward was available to all other trials that only included delayed rewards. This analysis identified a set of reward-related regions, including the ventromedial prefrontal cortex and nucleus accumbens, that were more active for choices involving immediate rewards than for choices that only involved delayed rewards. To find the areas involved in the patient (“delta”) system, they simply looked for regions that were active during all decisions, on the assumption that the delta process is engaged for all choices. This analysis identified a number of regions across the brain, including the lateral prefrontal regions that are thought to be involved in executive control. In order to link brain activity directly to choices, they compared brain activity on trials when the subject chose the immediate reward to trials when the subject chose a delayed reward, and found that the “beta” areas were more active when the subject chose the immediate reward, whereas the “delta” areas were more active when the subject chose the delayed reward. Based on these findings, and harking back to Aesop’s fable of the ant and the grasshopper, they concluded that, “Within the domain of intertemporal choice, the idiosyncrasies of human preferences seem to reflect a competition between the impetuous limbic grasshopper and the provident prefrontal ant within each of us.”27 Subsequent research has strongly criticized the findings of this study, leading many in the field to reject the idea that intertemporal choice reflects a competition between impatient and patient brain systems. Leading this charge has been Joe Kable, a neuroscientist at the University of Pennsylvania. Joe’s early work (with the pioneering neuroeconomics researcher Paul Glimcher) showed that, rather than being explained in terms of bias toward immediate outcomes, the socalled impatient regions in McClure’s study were simply responding to the subjective value of the decisions, which differed between the immediate and delayed choices. Other work shows that damage to the ventromedial prefrontal cortex causes individuals to become more impatient, discounting future rewards more heavily; this is exactly the opposite of what one would predict if this region was responsible for

s e l f - c o n t r o l 109 Box 5.3. Brain stimulation In order to know whether a brain region is directly involved in some mental process, it’s necessary to know if disrupting that area leads to a change in behavior. Sometimes we can study this by examining people with brain lesions, as we have already seen. However, it can often be difficult to find enough patients with lesions in any particular location to allow us to study them in a robust way. For this reason, researchers have also developed ways to stimulate the brain noninvasively, in effect creating temporary “virtual lesions.” The most common technique to create virtual lesions is transcranial magnetic stimulation (TMS). This technique takes advantage of a basic law of physics, which states that a changing magnetic field will cause an electrical current in any material that conducts electricity; this is the same law that gives us hydroelectric power generators. TMS is performed by placing a coil against the scalp and then running a brief but strong electrical pulse through the coil, creating a very short-lived magnetic field that causes an electrical current to be induced in the neurons below the coil. The effect of this stimulation on the neurons depends on the timing of the magnetic pulses. A single pulse that is applied during performance of a task will disrupt the activity of neurons at a specific point in time, allowing researchers to determine when that area plays its role in the task. Repeatedly stimulating an area over time can result in longer-lasting changes that either reduce or increase the excitability of that part of cortex, depending on how rapidly it is stimulated. One of the challenges of interpreting results from TMS studies is that the stimulation does not just affect the area that is directly stimulated, but also flows throughout the brain, affecting other areas that are connected to the stimulated region.

impatient decisions, in which case damage to it should cause people to be less impatient. There is, on the other hand, relatively good evidence that the lateral portion of the prefrontal cortex is important for exerting the control that is necessary to make patient decisions. Like the McClure study, a number of other studies have also found that the lateral prefrontal cortex is more active when people make patient decisions compared to impatient decisions. Although there are no published studies of how lesions to the lateral prefrontal cortex affect temporal discounting, a number of studies have used brain stimulation (see Box 5.3) to disrupt the function of this area in healthy individuals. Together these have shown that disruption of the lateral prefrontal cortex generally leads individuals to become more impatient on intertemporal choice tasks, confirming its role in exerting control over our tendency to be impatient.

110 c h a p t e r 5

Controlling Our Impulses Learning to control one’s impulses may be one of the most important aspects of becoming a functioning adult. As young children, our every impulse is indulged, from throwing tantrums to demanding attention from everyone in the room, but as adults we must learn to tamp down these impulses so that we don’t disrupt others. Some people are able to do this better than others, and the lack of impulse control is particularly evident in adults with attention-deficit/hyperactivity disorder, or ADHD. It might be considered cute when an impulsive child butts into a conversation with every idea that comes to mind, but if an adult does this it would be considered just plain annoying. In fact, one of the most persistent problems for individuals with ADHD is social difficulties that arise when they are unable to inhibit or edit themselves in social situations. Impulsivity is a complex phenomenon that has been studied by psychologists for decades, usually through surveys that ask people questions such as • • • •

Do you often act on the spur of the moment? Do you squirm at plays or lectures? Do you change hobbies often? Do you get bored solving problems?

From these questions you can see that the concept of impulsivity covers a broad landscape of mental life, and various researchers have tried to distill it into its separate parts. One well-known and well-supported framework proposes that there are four major aspects to impulsivity:28 •

• •

•

Urgency: The tendency to act without thinking when one is in either a positive or negative mood Lack of perseverance: Failure to follow through on intended plans Lack of premeditation: Failure to think through actions before committing them Sensation seeking: Engaging in activities that are novel and exciting, even if they are dangerous

s e l f - c o n t r o l 111

These different components were inferred using a statistical technique known as factor analysis, which identifies sets of survey questions that are answered similarly across people. Like discounting, impulsivity also has a strong underlying genetic component. And as for discounting, there have been a large number of past studies that tried to link individual genes to discounting, but none of these results have been confirmed in more recent very large studies of the entire genome. One large genome-wide study by Sandra Sanchez-Roige and her colleagues has provided particularly good evidence regarding the genetics of impulsivity.29 This study was conducted in collaboration with the personal genetics company 23andMe, which allowed them to collect data on impulsivity questionnaires along with genetic information from more than 22,000 individuals; if you are a 23andMe user and filled out one of these questionnaires, then you were probably included in this study. They looked across more than 500,000 different locations in the genome to see whether there was a relationship between genetic differences in that location and behavior on the impulsivity questionnaires, as well as whether there was a relationship with drug experimentation. The first thing this study showed was that there is a strong relationship between impulsivity and a number of real-world outcomes. More impulsive individuals had lower household income and education levels, higher body weight, and a greater likelihood of having experimented with drugs. The availability of the genetic data also allowed the researchers to examine what is called the genetic correlation between these different traits—that is, to what degree is similarity in these traits related to similarity in the genomes of different individuals? Intriguingly, this showed that there were strong genetic correlations between impulsivity and a number of negative outcomes, including drug usage and mental health problems like depression and ADHD. This tells us that the genetic risk for all of these negative outcomes is related (at least in part) to impulsivity. Our knowledge of the brain systems underlying impulsivity remains relatively poor at this point. When I reviewed the literature while working on this book, what I found was that there were a large number of studies that had attempted to measure relations between brain function

112 c h a p t e r 5

and impulsivity, but nearly all of these studies were too small to take seriously. Unfortunately, small studies provide results that are highly variable and unreliable (see Box 5.4), especially when they are measuring correlations between behavior and measures of brain function. In fact, one study that attempted to reproduce 17 published correlations between brain structure and various measures of behavior (all of which had relied upon relatively small sample sizes) was not able to replicate any of those previous findings. My current rule of thumb is that a study that aims to look at relations between brain and behavior across people needs to have at least 100 participants in order to even have a chance of being reliable; most of the studies that I found in this area were much smaller. In fact, recent studies suggest that the sample sizes necessary for reliable findings in this kind of study may be in the thousands. The first study to provide a reasonably well-powered analysis of the relationship between brain function and impulsivity was performed by Box 5.4. Why small studies can be problematic It would be wonderful if we could have complete faith in any research study that has passed peer review, but unfortunately this is not the case. In fact, my Stanford colleague John Ioannidis has famously argued that “most published research findings are false”30 —and I believe that his argument is largely correct. One factor that is particularly important in determining whether one can believe a published result is the size of the sample used in the study. When we design a research study, we need to make sure that it has the ability to find the effect we are looking for, if it truly exists. For example, let’s say that we want to do an experiment to test for the difference in height between adult men and women; we don’t really need statistics to tell us that men are taller than women on average, but let’s see how it works if we were to do it. Using a large public dataset, we see that the average height for adult men is about 176 centimeters, while for women it is about 163 centimeters. A difference of 13.5 centimeters sounds big, but we need a way to express its size that doesn’t depend on the particular way that it was measured; after all, we could have measured height in meters instead of centimeters. To do this, we divide the size of the effect by the average amount that individuals differ from the group average, known as the standard deviation. There is a standard scale that we use to roughly determine the importance of an effect, which was developed by the statistician Jacob Cohen. An effect that is less than 0.2 standard deviations is considered negligible, those from 0.2 to 0.5 are considered small, those from 0.5 to 0.8 are considered medium, and those above 0.8 are considered large. The effect size for the difference in height between adult men and women is a whopping 1.8 standard deviations, meaning that the difference is almost twice as big as the variability that we observe across people overall.

s e l f - c o n t r o l 113 Box 5.4. (continued) Once we know the effect size we are looking for, then we can determine the sample size we need in order to find the effect, if it exists; we are usually satisfied if we have sufficient power to find the effect 80% of the time when it truly exists. For an effect as large as our height difference, we only need a sample of 6 men and 6 women in order to detect a difference in height 80% of the time. Now let’s say instead that we are interested in testing for a difference in height between 9-year-old boys and 9-year-old girls. This is a much smaller effect, about 0.25 standard deviations, and in order to reliably find an effect of this size we need a much larger sample, 261 boys and 261 girls. What happens if we run an underpowered study on the 9-year-olds? If we could obtain data from only 20 boys and 20 girls, this study would have about 12% power, meaning that it would likely fail to find the difference that we know to exist between boys and girls. What Ioannidis demonstrated was that underpowered studies are not just unlikely to find an effect if it truly exists; any positive findings that they do report are also likely to be false. You can understand this through a thought experiment. Let’s say we are measuring BMI, but our computer is broken and is simply giving us random numbers. This means that any differences we claim to find between the groups are necessarily false. In statistics, we generally try to limit the likelihood of a false result to 5%, which means that if we were to do 100 studies using the broken scale, we would still expect 5 positive findings—all of which would be false! In essence, a study with very low statistical power is very similar to a study with a broken detector, since it has a very low likelihood of finding the effect even if it exists and most of the positive findings that result from such small studies are likely to be false. Since statistical power is relative to the size of the effect we are searching for, there is no single sample size that we can say counts as “big enough,” and this also differs across types of experiments. Unfortunately, there are still many studies published in peer-reviewed journals with sample sizes that are far too small, which means that you need to read closely in order to determine whether any particular result is believable or not. Ultimately, we also want to see that the result can be replicated by other research groups.

Kent Kiehl and his colleagues, who studied a group of incarcerated juveniles alongside a set of typical young adults.31 The study focused on measurement of brain connectivity using a method called resting state fMRI, in which the individual simply rests in an MRI scanner while their brain activity is measured. From these measurements, the researchers were able to infer how different parts of the brain were connected, focusing in particular on the region called the premotor cortex, which is thought to play an important role in motor control and action planning. What they found was that the premotor cortex was functionally connected to other brain networks involved in executive control in the typical young adults and the less impulsive offenders, whereas

114 c h a p t e r 5

this connectivity was disrupted in the most impulsive offenders. They also observed that a similar pattern was present across development, such that the brains of younger children showed a pattern of lower connectivity that was similar overall to the pattern observed in the more impulsive juvenile offenders. Other studies have, however, found different relationships between brain connectivity and impulsivity. For example, a study published by Daniel Margulies and his colleagues in 2017 collected resting state fMRI from about 200 individuals who also completed a survey that assessed each of the different aspects of impulsivity that I described above. These researchers focused on the region called the anterior cingulate cortex, which is thought to play an important role in cognitive control.32 They found that there were differences in the strength of connectivity between the anterior cingulate and other brain areas depending on a person’s level of impulsivity. In particular, they found that a lack of perseverance was related to connectivity between the anterior cingulate and the lateral portion of the prefrontal cortex, though this relationship went in the opposite direction that one might have predicted, such that people with more problems in perseverance had stronger connectivity between these regions. A full understanding of how brain differences relate to impulsivity must await more and larger studies.

Stopping Ourselves Binge eating is a disorder in which a person’s eating is literally out of control. One binge eater describes her first binge, after also struggling with anorexia: I wanted M&M’s for breakfast. I purchased a big, huge family-size bag of peanut M&M’s from a store at 7 a.m. on the way to school. . . . I cannot remember if I planned to eat the whole bag or if I was fooling myself and thinking I’d only eat a handful, but my hunger was ravenous. It was not just physical hunger. It was emotional hunger. Distressed hunger. I drove to school and started eating the M&M’s on my drive. One by one, at first. I then started eating them quicker. I stopped tasting

s e l f - c o n t r o l 115

them and started practically swallowing them whole. I would shove tens of them into my mouth at once, in a robotic, out-of-control motion. It felt like I was in a trance state. I went into some sort of blind black hole. My body was asking me to stop eating, I was full now. But I ignored the signals. I don’t remember what happened, but pretty soon, the entire bag was gone. A 1 lb. bag of M&M’s was gone.33 The ability to stop ourselves from doing something, either before we do it or after we have started, is an essential aspect of self-control that is referred to as response inhibition. Its failure is thought to underlie many disorders, from binge eating to drug addiction. Response inhibition has been studied for many years using a very simple laboratory task called a stop-signal task. In this task, the participant is shown stimuli and asked to respond to each one as quickly and accurately as possible; for example, they might be shown individual pictures of male or female faces and asked to press one button if the photo is a male and a different button if it’s a female. A task like this will generally take a healthy young adult well under 1 second to perform. However, there is one other critical instruction: there is another signal that could appear (let’s say it’s a loud beeping sound), and if that signal occurs the person is asked not to respond. The stop signal occurs relatively infrequently, and the effectiveness of the signal depends critically on when it occurs. If it occurs late, such that the person has already started moving their finger to make the response, it is very difficult (though not impossible) to stop the action, whereas if it occurs early (say, just a few thousandths of a second after the stimulus appears), then it will be relatively easy to stop. By presenting stop signals at many different delays and examining how successful people are at stopping their behavior, we can use a mathematical model to estimate how much time it takes for a person to interrupt the impending action. And the answer is that it’s generally very quick, around 2 tenths of a second. The person who has done the most to advance our understanding of how people perform the stop-signal task is the Canadian psychologist Gordon Logan, now a professor at Vanderbilt University. In particular,

116 c h a p t e r 5

he developed the mathematical framework that allows us to estimate the amount of time that is required to stop, which is known as the stop-signal reaction time. It’s easy to measure how long it takes for something to happen, but in the stop-signal task we are trying to estimate the amount of time that it takes for something not to happen. Logan developed a framework known as the race model, which proposes that the success or failure of inhibition depends on a race between two different processes. On the one hand, there is the go process, which starts when the stimulus is presented. In the absence of a stop signal, this process will nearly always finish, leading to a motor response. However, when a stop signal occurs, this triggers a separate stop process that races against the go process; if the stop process finishes first then the response will be successfully inhibited, whereas if the go process finishes first then the person will fail to stop. With some additional assumptions, we can use the race model to estimate the amount of time that it takes to do nothing. I have known Gordon Logan since my graduate school days in the 1990s, when he was a professor at the University of Illinois at UrbanaChampaign and my office was just across the hall from his. However, I didn’t start working in earnest on response inhibition until 2003, when a young researcher named Adam Aron joined my lab as a postdoctoral fellow. Aron had grown up in the small African country of Swaziland, and after college in South Africa he went to graduate school at Cambridge University in England, where he worked with the neuroscientist Trevor Robbins to understand how damage to different parts of the brain affected the ability to stop actions on the stopsignal task. Aron’s research had shown that stopping was particularly impaired when people had damage to a part of the right prefrontal cortex known as the right inferior frontal gyrus, or RIFG (see Figure 5.6).34 He came to my lab because he wanted to use brain imaging to better understand how stopping was accomplished in the brains of healthy individuals. Aron developed an initial study that was as simple as possible.35 He imaged the participants’ brains using functional magnetic resonance imaging (fMRI) while they performed a simple version of the

s e l f - c o n t r o l 117

Pre-SMA RIFG

STN

figure 5.6. The regions highlighted on the brain in this image were active across 99 published studies that mentioned the stop-signal task in the abstract of their publication (generated using the Neurosynth.org meta-analysis tool). The regions noted by arrows are those that were first discovered in our 2006 paper, including the right inferior frontal gyrus (RIFG), the pre-supplementary motor area (pre-SMA), and the subthalamic nucleus (STN).

stop-signal task.36 We first analyzed the data to determine what parts of the brain were active when a person made their response on a go trial in the absence of a stop signal; the brain systems involved in simple responses like this are well understood, so this was really just a reality check. This analysis found that all of the expected areas, including the visual and motor cortex and the putamen in the basal ganglia, were activated when a person performed the task. The big question was what areas were more active on the stop trials when a stop signal appeared compared to go trials. The results confirmed Aron’s previous work by showing that the right inferior frontal gyrus was indeed activated by the stop signal, along with another part of the prefrontal cortex called the pre-supplementary motor area. However, we also found activity in another area that should be familiar from Chapter 2: the subthalamic nucleus. You may remember that the subthalamic nucleus is part of the indirect pathway in the basal ganglia and that activating this region results in the inhibition of actions. At the time we were doing this research, there was an emerging idea about an additional route in the basal ganglia, known as the hyperdirect route, by which the prefrontal cortex could directly activate the subthalamic nucleus;37 earlier research in monkeys had shown that electrical stimulation of this route

118

chapter 5

led to the cancellation of ongoing behavior, meaning that it could play exactly the kind of role needed for rapid stopping. However, connections between the prefrontal cortex and the subthalamic nucleus had never been established in humans. In order to determine whether the hyperdirect route might be responsible for stopping, we teamed up with a group of researchers from Oxford University who were experts in the analysis of diffusionweighted imaging, which we encountered earlier in this chapter. Our colleagues from Oxford, Tim Behrens and Steve Smith, had developed software that allowed us to virtually trace the white matter tracts of the brain using a method known as tractography. When we used this tool with the diffusion-weighted imaging data collected from the same individuals who had participated in the fMRI study, we found that both of the frontal lobe areas identified during stopping (the right inferior frontal gyrus and the pre-supplementary motor area) had direct white matter connections to the subthalamic nucleus.38 The results of our neuroimaging study provided some early evidence that the hyperdirect route from the frontal lobe to the subthalamic nucleus was involved in stopping. However, a clearer demonstration of the role of the subthalamic nucleus has since been provided by a set of studies that examined rats and mice as they performed a version of the stop-signal task. In this adaptation, the rodent starts with its nose in a central port, where it waits until it hears a sound. If the sound is high-pitched, the rat must move its nose to a port on one side; if it’s low-pitched, then the rat moves its nose to a port on the other side. If the rodents do this correctly, they are rewarded with a sweet treat. However, just as in the stop-signal task in humans, on about 30% of trials a stop signal is presented (in this case, a brief bit of white noise), telling the rodent not to poke its nose into either of the ports; if it successfully withholds its response, then it receives a sweet reward. Since it’s not possible to tell the rats what the rules of the task are, they must be trained to do the task, which can take several months, but once trained they can do it very well. Josh Berke and his colleagues recorded from neurons in the subthalamic nucleus while rats

s e l f - c o n t r o l 119

performed the stop-signal task, and their recordings confirmed that the subthalamic nucleus was indeed engaged whenever a stop signal occurred.39 However, this activity occurred whether or not the rat was able to successfully stop, whereas another part of the brain (a part of the indirect pathway that receives input from the subthalamic nucleus) was active only when the rat was able to successfully stop. This showed that the role of the subthalamic nucleus was to relay the stop signal to neurons in other parts of the indirect pathway. Their analyses of the timing of responses in these areas also provided a direct validation of Logan’s race model, by showing that the success or failure of stopping was related to the relative timing of activity in the different sets of neurons in the basal ganglia—the stopping-related activity in the subthalamic nucleus had to occur before the movement-related activity in the striatum began in order for stopping to be successful. Subsequent work by Berke and others has shown that the brain mechanisms of stopping are quite a bit more complicated than this story would suggest, but they remain consistent with the idea that the subthalamic nucleus plays a central role in stopping actions.

The Rise and Fall of Willpower When people are asked why they failed to successfully make desired changes in their lifestyle, the top factor that they report is “I don’t have enough willpower.” This belief in willpower even extends to health care providers: One study found that dietitians working with overweight and obese people thought that lack of willpower was one of the most important causes of weight problems, and they gave different advice to individual patients depending on how good they thought the patient’s willpower was. By “willpower,” most people think of a specific aspect of self-control that involves either saying no to something that they want (like an extra serving of dessert) or saying yes to something that they don’t want (like going to the gym). It has long been assumed that people with “good willpower” are those who are good at saying no to their impulses in the heat of the moment—overriding their craving

120 c h a p t e r 5

for a cigarette or actively choosing the carrot rather than the slice of cake. However, there is increasing evidence that this view is just plain wrong. In the previous section, we encountered the stop-signal task, which is meant to measure the ability to exert inhibition over one’s actions. If this kind of inhibition is essential to willpower, then we would expect that a person’s ability to stop in the moment would be related to measures of self-control, which is usually measured using self-report surveys like the ones discussed earlier in the chapter. In one recent study, we measured both response inhibition ability and self-control in more than 500 people.40 When we computed the correlation between the stop-signal reaction time (which quantifies how long it takes a person to stop) and a measure of self-control based on questions like the ones listed at the beginning of the chapter, we found virtually no relationship. In fact, across many different measures we found almost zero relationship between tasks meant to measure executive control and surveys meant to measure self-control, just as a number of studies from other researchers have also found. Given this lack of relationship, it’s hard to see how differences in these basic inhibitory functions could be responsible for the differences that exist between people in their self-control abilities. It appears that, instead of being better at inhibiting their impulses, people who appear to have better self-control are actually better at avoiding the need to exert self-control to begin with. Evidence for this comes from a study by Wilhelm Hofmann and colleagues, which studied how desires, goals, and self-control interact using a method called experience sampling.41 The 208 study participants were given Blackberry devices (the study was performed around 2011), and during the course of the day their device signaled to them every couple of hours to make a report about their experience. They first reported whether they were currently experiencing a desire, or had done so within the last 30 minutes. If they had, they were asked to report what kind of desire it was and to rate how strong the desire was, on a scale from 1 (no desire) to 7 (irresistible). They then answered a number of other questions about whether they had attempted to resist the

s e l f - c o n t r o l 121

desire, how much conflict they felt between the desire and a goal of theirs, and a number of other details. The individuals also filled out survey items about their self-control ability, like the ones we presented earlier. Once all of the data were in, Hofmann looked at how an individual’s reported level of self-control related to each of the different aspects that were recorded by experience sampling. If the role of self-control was to squash desires in service of goals, then the people with better selfcontrol should experience more conflict between their desires and their goals and should resist their urges more often. However, the results showed exactly the opposite: the people with higher self-control exhibited less conflict and reported resisting their desires less often than the people with low self-control. In addition, they found that people with higher self-control actually reported experiencing fewer and weaker desires in general. Another study by Brian Galla and Angela Duckworth from the University of Pennsylvania provides one possible answer to why it is that people who report having higher self-control paradoxically seem to need it less: they are better at establishing good habits. To examine this paradox, they first surveyed a large number of people about their daily habits (including snacking and exercise) and also surveyed their level of self-control. Not surprisingly, they found that people who had better self-control exercised more and ate healthier snacks; but, interestingly, they also reported that their exercise and healthy eating was more habitual, meaning that they just did it automatically without the need to think about it. The researchers also found that the effects of self-control were carried by good habits—better self-control predicted stronger good habits, which in turn predicted less need to exert effortful self-control.42 Similar results were also found in several studies of academic performance and study habits; but perhaps the most interesting finding came from a study they performed that followed 132 individuals going through a five-day meditation retreat. Before the retreat started, Galla and Duckworth measured each individual’s self-control, and then followed up three months later to see how likely the participants were to make a habit of meditation. The people who had higher

122

chapter 5

self-control were more likely to develop a meditation habit after the retreat, and they felt that meditation had become more automatic for them. What this research shows is that willpower is not all that it’s cracked up to be. Next we turn to the question of why some particular habits are so hard to change, where we will also see that willpower doesn’t seem to play the role that many people intuitively believe.

6 Addiction habits gone bad

judging from the titles of popular songs, one can be addicted to just about anything, including love (Robert Palmer), orgasms (the Buzzcocks), danger (Ice-T), and bass (Puretone). However, neuroscientists reserve the term addiction more specifically for the compulsive and uncontrollable engagement in a particular behavior in spite of its harmful consequences to the user.1 This often involves some kind of chemical substance, though as I discuss later in this chapter, the concept of “behavioral addictions” has received increasing interest and support, encompassing such behaviors as problem gambling and “smartphone addiction.” Addiction is obviously an incredibly complex phenomenon that arises for many different reasons across individuals, but addiction researchers have outlined a number of the brain processes that occur during the transition from initial drug use to addiction.

The Intoxicating Allure of Drugs It was instantly fucking amazing. It felt like everything was melting and everything was somehow better. Nothing mattered. It lasted about five or six hours and I felt really floaty and nice. . . . I wanted to do nothing else but feel that way. I decided almost immediately that this was going to be my life.2 123

124

chapter 6

This is how one person described their first experience taking heroin. The process of drug addiction starts with a substance that causes an intoxicating experience, but that experience can vary from the instant peace of heroin to the euphoria of cocaine to the giddiness of alcohol. Despite this variability in the experience, all drugs that are abused by humans appear to cause increases in the level of dopamine, particularly in the nucleus accumbens. Some drugs do so directly by affecting a protein called the dopamine transporter, which is a molecular pump that removes dopamine from the synapse after it has released, sucking it back into the neuron so that it can be recycled. Cocaine blocks the activity of the dopamine transporter, while amphetamines can actually cause it to go in reverse, pumping even more dopamine back into the synapse. Other drugs have their effects by causing dopamine neurons to fire more strongly, by either directly causing them to fire (as nicotine and alcohol do) or indirectly causing them to fire by reducing the activity of other cells that normally inhibit the dopamine neurons (as happens with opioids and cannabis). One of the reasons that these drugs are so powerful is that they cause dopamine neurons to behave in ways that do not occur naturally. It is commonly stated that drugs of abuse cause a much larger dopamine release than natural rewards, but this claim turns out to be remarkably difficult to pin down. As we saw in Chapter 2, dopamine neurons will fire for a short time in response to an unexpected reward or to a cue that predicts the later appearance of a reward. The dopamine that is released is relatively quickly taken up by the dopamine transporter, as well as being broken down by various enzymes present in the synapse, meaning that much of the dopamine will be gone relatively quickly. However, because most drugs of abuse have a longer-lasting effect, they will cause dopamine release that extends well beyond the point at which the drug is consumed. It is less clear whether the amount of dopamine released immediately upon exposure is actually larger for drugs of abuse than for natural rewards. Much of the research examining the size of the dopamine response has used a method called microdialysis, which measures the amount of dopamine from fluids extracted from a region of the brain. Because of the very small amount

a d d i c t i o n 125

of fluid that can be extracted, this method requires several minutes of data collection in order to accurately measure dopamine levels. Studies using microdialysis have generally found that the amount of dopamine release is much greater for drugs than for natural rewards, but this may reflect the fact that they are measuring dopamine over such a long period, and thus cannot distinguish between the amount of dopamine released upon receipt of the reward and the duration of the response. Another method, mentioned in Chapter 3, is fast-scan cyclic voltammetry, which allows much faster measurement of dopamine release, and studies using this technique have generally found that the magnitude of the immediate dopamine release for drugs is not substantially larger than that for natural rewards. Thus, it seems that the ability of drugs to drive habit formation may in part reflect the unnatural duration of the dopamine response rather than the amount of dopamine released when the reward is received. The effects of dopamine release on behavior are very powerful. It has long been known that stimulation of dopamine neurons can drive the development of strongly motivated behavior in animals, and the development of optogenetic tools has allowed researchers to link dopamine to the development of these behaviors even more precisely. In this research, dopamine neurons located in the ventral tegmental area are optogenetically stimulated, which causes release of dopamine to their outputs, including the basal ganglia. Stimulating these neurons directly is similar to injecting the animal with a drug like cocaine, but its effects are much more specific and immediate, so we can be more confident that the results of stimulation reflect the effect of dopamine. Several studies from the laboratory of Karl Deisseroth at Stanford have provided a compelling demonstration of the role of dopamine in driving the behavior of animals. In one study, they allowed mice to simply explore their cage, but stimulated their dopamine neurons when the mouse was in a specific part of the cage. This caused the animals to spend more time in that part of the cage, which is referred to as conditioned place preference. Another study, led by Ilana Witten (whose later work you learned about in Chapter 2), built on decades of research showing that when rats are given the chance to electrically stimulate in

126

chapter 6

their own brains in or near dopamine neurons, they will do so compulsively, in some cases pressing a lever more than 7000 times in an hour.3 Witten provided mice with the opportunity to optogenetically stimulate their own dopamine neurons simply by sticking their nose into one of two ports located in their cage. The mice quickly learned to do this, and within a few days were poking their nose into the stimulating port thousands of times per day! These results show that dopamine stimulation is sufficient to create the kind of compulsive behavior that we often associate with drug abuse.

“This Is Your Brain on Drugs. Any Questions?” Anyone who lived through the 1980s will remember the famous television commercial produced by the Partnership for a Drug-Free America,4 which went as follows: This is your brain (holds up an egg). This is drugs (points to frying pan). (cracks egg into frying pan) This is your brain on drugs. Any questions? The reality of how drugs affect the brain is of course much more complicated, and the drama of this ad fails to match the reality of drug experiences for many people, but it is also clear that drugs of abuse can have both immediate and lasting effects on the brain. Drug use results in changes in the function of many different neurons in the brain, some of which are short term and others of which may have effects far into the future. The immediate aftereffect of drug ingestion is that dopamine neurons become hypersensitive, and these changes appear to occur after ingestion of any drug that has addictive properties.5 These changes, which can last for at least several weeks, appear to be in part the result of the same kinds of synaptic learning mechanisms that we discussed in Chapter 2. In addition, changes occur that can lead to even longer-lived effects. In particular, there are changes in the regulation of gene activity in the brain, including epigenetic changes

a d d i c t i o n 127 Box 6.1. Gene regulation and epigenetics The role of genes is to create the proteins that are the building blocks of the body and brain. The process of creating a protein from a gene, known as gene expression, involves first transcribing DNA into a copy (called RNA) that tells the cell’s machinery what protein to make. This messenger emerges from the nucleus of the cell and is then translated into a protein. While each cell includes the DNA that forms the recipes to make any of the roughly 20,000 proteins in the human body, only a small percentage of those genes is being expressed at any point in time. Which genes are expressed, and how much they are expressed, is controlled in at least two different ways. One is through the activity of a set of proteins known as transcription factors, which can either reduce or increase the transcription of particular genes. One particularly important transcription factor for plasticity in the brain is known as CREB (for “cyclic AMP response element binding protein”). CREB is essential for many different forms of lasting learning and memory across a wide range of species, from sea slugs to mammals, and is known to be involved in the sensitization of dopamine neurons and the nucleus accumbens after exposure to drugs, including stimulants (cocaine and amphetamine) and opiates. Another way gene expression can change in a lasting way is through epigenetic changes. These changes involve the complex packaging in which DNA is enveloped within the cell, which controls access to the DNA by the machinery that does the transcription. By making chemical changes to these proteins or to the DNA itself, lasting changes in the function of the cell can occur by controlling the level at which different genes are expressed. I mentioned in Chapter 1 that bad habits are like cancer, in that they reflect the unwanted expression of a fundamental biological process. Epigenetics provides an even more intriguing link between learning and cancer, in that both seem to rely heavily on epigenetic changes within the cell. Epigenetics also provides a means by which a mother’s experience, such as stress, can have a lasting impact on an offspring’s brain, though this remains an area of significant controversy.

(see Box 6.1), which lead to changes in the expression of genes in regions including the nucleus accumbens. One intriguing idea proposed by Yan Dong and Eric Nestler6 is that exposure to some drugs (such as cocaine) may reawaken some of the molecular mechanisms of brain plasticity that are present during early brain development and that allow the very rapid plasticity that occurs in the first years of life. While they are prevalent in early development, these mechanisms are largely absent in the adult brain. One of these is the “silent synapse,” which is similar to a regular synapse but lacks the particular form of glutamate receptor that is necessary to allow a normal synapse to transmit activity. Although they are rare in the adult brain, silent synapses are highly abundant in developing brains, and

128

chapter 6

their conversion to active synapses is thought to play an important role in the development of early brain connectivity. However, there is evidence that cocaine exposure results in the generation of such silent synapses in the brains of adult rodents. Because these synapses provide a powerful mechanism to quickly establish new connections in the brain, the unsilencing of these synapses may be one of the plasticity mechanisms that leads to the development of lasting drug habits, though this remains under debate. While biological changes leave the brain more sensitive to drug effects after initial or intermittent drug exposure, longer-term exposure leads to another set of changes known as tolerance. These changes primarily reflect the adaptation of the brain in an attempt to maintain its steady state by neutralizing the effects of the drug. Evidence for this in humans comes from an early study by Nora Volkow (currently the head of the US National Institute for Drug Abuse), in which she and her colleagues examined the brain response of cocaine addicts and healthy nonaddicts to an intravenous injection of the drug methylphenidate (know best by its trade name Ritalin), which has similar effects to cocaine in causing the release of dopamine.7 They used positron emission tomography (PET) scanning (see Box 6.2), which allowed them to estimate the amount of dopamine that was released in the striatum. The healthy subjects reported a feeling of being “high” and restless that was greater than that reported by the cocaine addicts, while the cocaine addicts reported feeling strong craving for cocaine during the injection. When Volkow and colleagues measured the function of the dopamine system in each group, they saw that the addicts had a significantly smaller response to the drug compared to the control group, suggesting that their dopamine system had adapted to the presence of cocaine, requiring a greater amount of drug stimulation to obtain the same level of dopamine response in the brain. Other research has shown that there are also structural changes in the brain over time related to drug abuse, with shrinkage of dopamine neurons and extra growth of other neurons in the nucleus accumbens. It is likely these kinds of changes that lead addicts to increase their intake over time, and they may also play a

a d d i c t i o n 129 Box 6.2. Positron emission tomography Positron emission tomography (PET) is an imaging technique that allows us to identify the presence of molecules in the body by tagging them with radioactive material. PET relies upon the fact that when some radioactive particles break down, they emit a positron, which is like an electron but with the opposite charge. When this occurs, the positron travels until it collides with an electron, which results in the annihilation of the particles, resulting in the emission of two gamma rays in opposite directions. The PET scanner is composed of a ring of detectors that measure these gamma rays; based on when and where they are detected, the scanner can reconstruct where the annihilation must have occurred. The images that are produced by the scanner reflect how many of those annihilation events occurred in each location in the brain. In order to use PET imaging to image neurochemicals, such as dopamine, we need to be able to attach a radioactive tag to a molecule. In some cases, we might attach it to a molecule that travels around the body, such as glucose or oxygen, so that we can simply measure the presence of the molecule in different tissues. In other cases, we need to find a molecule that will attach itself to a particular receptor on the cell (which we call a ligand), and then attach the radioactive tag to that molecule using clever chemistry techniques. One ligand that is commonly used to image dopamine is fallypride, which is tagged with a radioactive fluorine atom and injected into the bloodstream. It then makes its way to the brain, where it attaches to dopamine receptors (specifically, the D2-like receptors) and then sticks around. By imaging the resulting radioactive decay, we can estimate how many dopamine receptors are available in each part of the brain.

role in the withdrawal symptoms that are seen when a drug addict tries to quit cold turkey. One of the most striking aspects of drug addiction is the degree to which cues come to elicit powerful cravings. In Chapter 2, I mentioned the idea of “incentive salience” that has been proposed by Terry Robinson and Kent Berridge to describe the degree to which a particular cue results in the motivation to obtain a particular reward—what in popular parlance we would refer to as “wanting.” They have further argued that one of the important changes that occurs in drug abuse is that the brain becomes hypersensitized to the incentive salience of drug-related cues, which leads to compulsive wanting that subsequently leads to compulsive use, even as the drug becomes less pleasant for the user. This “incentive sensitization” process may also lead to the increased attentional salience of drug cues that I described in Chapter 3.

130 c h a p t e r 6

The Transition from Impulse to Habit A popular recent idea in the neuroscience of addiction, which has been proposed by the British neuroscientists Trevor Robbins and Barry Everitt,8 is that the development of an addiction involves a transition from impulsivity to compulsivity. This idea proposes that the early experimentation with drugs is related to sensation-seeking tendencies and impulsivity, while the development of compulsive drug use in addiction is related to a transition from goal-directed to habitual behavior. As we have seen in previous chapters, the concept of “impulsivity” means different things to different researchers, and for Robbins and Everitt the term primarily refers to aspects of cognitive control that we discussed in Chapter 5: response inhibition and waiting. Part of their focus on these concepts, rather than those related to self-reported impulsivity, is that their approach involves the development of animal models alongside their studies of humans. For this reason, they also developed a task specifically meant to measure the ability for rats to “hold their horses,” known as a five-choice serial reaction time task. In this task, the rodents are trained to wait for visual signals in order to obtain food by poking their noses into one of five holes in the box. Impulsivity is measured by testing how often the animals poke their noses before the light appears—for which the animal gets a time-out. Rats vary in their likelihood of doing this, and some impulsive rats will continue to poke early despite the punishment. In a study published in 2007, Robbins and Everitt examined whether this kind of impulsivity was related to dopamine function.9 Using a tiny PET scanner, they scanned the rats to find out how many dopamine receptors each rat had in its nucleus accumbens. The impulsive rats had significantly fewer dopamine receptors available for the PET tracer to attach itself to. They then tested whether these differences were also related to addictive potential. To do this, they gave the rats the opportunity to mainline cocaine by pressing a lever; some of the rats were selected to be especially impulsive (failing to wait more than half the time), while others were selected for being patient. Both groups of rats pressed the lever for cocaine, but the impulsive rats injected themselves

a d d i c t i o n 131

about 50% more often compared to the patient rats, over about 20 days of testing. The findings of this study provided a compelling palette of evidence for the relation between impulsivity, dopamine, and addiction, though it was a relatively small study (with only 6 rats in each group), leaving the results somewhat in question. In addition, other research by this same group has shown that the same relationship does not hold for heroin usage, showing that addictions to different drugs may involve different mechanisms. Shortly after the publication of this paper, I collaborated with Edythe London, an addiction researcher at UCLA, on a study that aimed to test the theory in humans.10 We scanned two groups of people using the same kind of PET scan that Robbins and Everitt had used in their rats, which allowed us to measure the availability of dopamine receptors in their striatum. One group was a set of healthy individuals, and the other was a set of long-term methamphetamine abusers who had been using for more than 10 years on average. The subjects also completed a survey meant to measure their level of impulsivity. Comparing the PET scans, we saw as expected that the drug abusers had lower availability of dopamine receptors across their striatum. And when we looked at the relation between impulsivity and dopamine, we saw that those drug abusers who were most impulsive also had the lowest dopamine availability. In a subsequent study led by Dara Ghahremani, we also found a relationship between dopamine receptors and performance on the stop-signal task—just as in the rat study, the subjects who were worst at stopping their motor responses showed the lowest level of dopamine receptor availability. Together this research provides a confirmation in humans of the results from Robbins and Everitt in rats, establishing a stronger link between response inhibition, dopamine, and drug abuse. The second major claim of the impulsivity-to-compulsivity theory is that the compulsive drug use in addiction arises through a transition from goal-directed to habitual behavior and finally to compulsive drug taking. This is inherently plausible, given the central role of dopamine in addiction as well as in habit learning, and research with rats has allowed neuroscientists to home in on this question. Compulsive drug

132 c h a p t e r 6

taking in rats has been studied using experiments where the rat is first allowed to administer the drug to itself, but then later required to endure a mild foot shock in order to receive continued drug infusion. If the shocks are introduced after a limited amount of experience, then all rats will refuse to endure the punishment in order to selfadminister the drug. However, after a few months of experience with the drug, a subset (about 20%) of rats will continue to take the drug after the punishment is introduced. This model has provided a useful tool to understand the brain systems involved in compulsive drug use; as we will see later, it has also provided interesting insights into why only some drug users become addicts. Research by Barry Everitt and his colleagues found that compulsive cocaine self-administration (that is, continuing to take the drug despite punishment) was abolished when they inactivated the part of the striatum that is involved in habitual behavior.11 The brain’s habit systems, it seems, are critical to the development of compulsive drug use, at least in this particular rodent model. In Chapter 3 I described the idea of the spiral of dopamine signaling in the basal ganglia, in which higher-order regions of the striatum also send dopamine signals to regions that are closer to the motor areas that are essential for habits. It appears that this structure is also important for drug habits. Research by David Belin and Barry Everitt has shown that the development of habitual cocaine seeking in rats requires that the nucleus accumbens be able to trigger dopamine output to the dorsal striatum, following the spiral pattern.12 Other research has confirmed this by recording dopamine release over the course of drug experience, showing that over time there is greater dopamine release in the particular part of the striatum that is essential for habit development. Other research has examined the role of different prefrontal cortex regions in compulsive responding. Previous work has shown that different parts of the rat’s tiny prefrontal cortex are involved in habitual versus goal-directed behaviors, as we noted in Chapter 3: in particular, an area called the prelimbic cortex is required for goal-directed behavior, while an area called the infralimbic cortex is essential for the development and maintenance of habits. Research by Antonello Bonci and colleagues has

a d d i c t i o n 133

shown that the prelimbic cortex plays a central role in the development of compulsive drug use in rats.13 They trained rats to self-administer cocaine and then identified those rats that demonstrated compulsive drug taking. When they examined the activity of neurons in the prelimbic region, they saw that the neurons of the compulsive rats were much less responsive than in the rats who did not use the drug compulsively. When the researchers activated the prelimbic region in the drug-taking rats using optogenetic stimulation, the amount of drug taking was substantially reduced. Conversely, they also found that deactivating the prelimbic region led the noncompulsive rats to behave like the compulsive rats, self-administering even in the face of punishment. This shows how the relative balance of activity in different parts of the prefrontal cortex is responsible for maintaining a balance between habitual and goal-directed behavior. So far we have seen research that implicates both habit systems and systems for goal-directed action in the development of compulsive drug use. But rather than thinking about addiction as either a problem of uncontrolled habit development or a problem of insufficient self-control, there is a growing movement to think about it in terms of a disrupted balance between these two processes.14 Research in this area has been particularly influenced by the ideas of model-based versus model-free reinforcement learning introduced in Chapter 4. In one large study published in 2016, Claire Gillan and Nathaniel Daw asked almost 1500 individuals to complete a survey about various mental health problems, including eating disorders, alcohol addiction, and intrusive thoughts, as well as having them perform the two-step task that I discussed in Chapter 4.15 They found that, across the board, people with higher levels of symptoms on this dimension (which they characterized as “compulsive behavior and intrusive thought”) showed less reliance on model-based learning and more reliance on modelfree learning. Studies of several different groups with addictive disorders, including methamphetamine abusers, alcohol abusers, and people with binge eating disorder, have also consistently shown that individuals with these disorders show lower levels of model-based decision making. Thus, it seems that, rather than reflecting a specific problem

134 c h a p t e r 6

with either habitual or goal-directed behavior, addiction may reflect an imbalance between the two systems.

Stress and Addiction So far we have focused on the reward side of drug use—the intoxication—but it’s well known that drug abuse also has a dark side. Brian Rinker described the “sheer hell” of opioid addiction: For many users, full-blown withdrawal is often foreshadowed by a yawn, or perhaps a runny nose, a sore back, sensitive skin or a restless leg. For me, the telltale sign that the heroin was wearing off was a slight tingling sensation when I urinated. These telltale signals—minor annoyances in and of themselves— set off a desperate panic: I’d better get heroin or some sort of opioid into my body as soon as possible, or else I would experience a sickness so terrible I would do almost anything to prevent it: cold sweats, nausea, diarrhea and body aches, all mixed with depression and anxiety that make it impossible to do anything except dwell on how sick you are.16 Just as our body has finely tuned systems to keep our blood sugar and temperature within safe ranges, the brain has a complex set of processes that aim to keep neurons from being overstimulated, and these systems are central to the dark side of addiction. Some of these changes occur within the reward system itself. While short-term exposure to drugs increases the response of the reward system, over time the brain’s response to dopamine becomes suppressed, in an attempt to normalize activity in the face of high levels of dopamine—essentially trying to prevent neurons from burning themselves out. This is the reason that we observed lower levels of dopamine receptors in the methamphetamine abusers in the PET study I mentioned before. However, there is also another set of changes that occur outside of the reward system that may have even more detrimental impact. These changes occur in several brain systems related to stress. The brain’s

a d d i c t i o n 135

Stress

Hypothalamus CRF

Cortisol

Pituitary gland ACTH Adrenal gland figure 6.1. A schematic of the brain’s stress systems. When it experiences stress, the brain sends signals to the hypothalamus, which result in the release of corticotropin releasing factor (CRF), which then leads the pituitary gland to release adrenocorticotropic hormone (ACTH). This leads the adrenal gland to release cortisol into the bloodstream, which then affects brain function.

primary system for responding to stress is known as the hypothalamicpituitary-adrenal (or HPA) axis (see Figure 6.1). When an individual experiences stress, the hypothalamus releases a hormone called corticotropin releasing factor (CRF), which travels to the pituitary gland and causes it to release another hormone called ACTH (for adrenocorticotropic hormone) into the bloodstream. ACTH travels to the adrenal glands (located on top of the kidneys), which release the stress hormone cortisol into the bloodstream. Cortisol receptors in the brain help control the release of ACTH in a negative feedback loop, to ensure that levels of cortisol don’t get too high. Importantly, there are receptors for CRF in many parts of the brain involved in reward and emotion, and these appear to be sensitized by drug exposure, leading to an enhanced

136 c h a p t e r 6

stress response when the individual is withdrawn from the drug. In particular, research in animal models suggests that CRF activity is particularly involved in the anxiety response that occurs when an individual goes into withdrawal. Experiments that blocked the effects of CRF showed that this reduces the anxiety responses of rodents who are withdrawn from drugs after a long period of exposure. Another prominent aspect of withdrawal is the negative emotional state known as dysphoria, which appears to be related to changes in the brain’s opioid systems. We usually think of opioids as being related to pleasure, but just as with dopamine, there are different versions of the opioid receptor that have different effects. In this case, the kappaopioid receptor is instead related to negative emotional states. Drugs that activate these receptors cause mood disturbances (as well as dreamlike states of consciousness), and blocking these receptors leads to reduced drug withdrawal in rats. These receptors are activated by a hormone called dynorphin, which is present in increased amounts in the brains of drug-addicted animals. The tight link between the brain’s stress systems and reward systems helps explain why stress is such a powerful trigger for relapse in drug users. It seems that this link is a vicious cycle: drug use causes changes in the brain’s stress response, and stress causes increases in the incentive salience of drugs. As we saw in Chapter 5, stress also impairs the ability of the prefrontal cortex to exert control over our behavior; in addition, it appears to modulate the trade-off between habit and goal-directed behavior. Mark Packard showed this in rats, where he found that stressing rats before training them on the plus maze task (for example, by exposing them to the odor of a predator) resulted in greater reliance on the habit system during learning. Finally, stress also appears to enhance the ability of cues to trigger responses through Pavlovian-instrumental transfer. In fact, one study by Kent Berridge and colleagues showed that injections of CRF into the nucleus accumbens in rats enhanced Pavlovian-instrumental transfer similarly to injections of amphetamine, showing the powerful influence of stress on our reward systems.17

a d d i c t i o n 137

Is Addiction Really about Habits? In Chapter 4 I recounted an example of the complex plans that one drug abuser used in her attempt to obtain prescription painkillers. That example suggests that while drug-seeking behavior is definitely sticky, it appears to be intently goal directed and have a degree of flexibility that we don’t expect of habitual routines. The idea that addiction reflects a reduction in the degree of goaldirected control is also evident in the common notion that addiction represents a failure of willpower. However, if you ask addicts about this, as Anke Snoek and her colleagues did, their answers fall less in line with the idea of failed willpower and more in line with the idea of overly powerful goal-directed behavior—where the goal is obtaining and consuming the drug.18 As one of their participants reported: “Very strong-willed (chuckling). That’s my problem, I’m very strong-willed.” The writer Crispin Sartwell described it this way: “Ask yourself what it takes to keep doing [drugs] even while everyone around you is telling you that you need to stop, and so on. It takes a masterful will.” There is, in fact, a growing number of researchers who think that the drug seeking observed in addiction may be goal directed rather than habitual. Animal studies have certainly shown that simple behaviors that deliver a drug or alcohol reward, such as pressing a lever, are initially goal directed but become habitual over time. However, these actions are highly simplified compared to the complex planning and action used by humans to obtain drugs of abuse. Terry Robinson and his colleagues examined what would happen when rats were forced to engage in a more complex behavior—solving a new puzzle every day— in order to receive an opportunity to self-administer cocaine.19 The puzzles were quite complex; as an example, the rats might have to make four presses on one lever, followed by two turns of a wheel—if they do all of these properly, then they get an opportunity to press another lever that gives them cocaine. Rats learn to do this after a lot of training, and once they have learned it, at least some of the rats exhibit the hallmarks of addiction in rodents—they escalated the amount of drug that

138 c h a p t e r 6

they used and how hard they would work to get it, and the drug-seeking response remained strong even after the drug was no longer available— though critics have noted that this is not strong evidence of compulsive drug seeking. Because of the complex nature of the task and the fact that the puzzle changed every day, it is impossible that this behavior is relying on any sort of habitual action, which implies that it must be relying upon goal-directed control. There are several ways in which an over reliance on goal-directed behavior might give rise to addictive behaviors. One is that the goal value of the drug reward may become heightened, such that it overpowers any other possible goals or actions, as proposed by Robinson and Berridge in their idea of incentive sensitization discussed earlier. One suggestion from Lee Hogarth of the University of Exeter is that the goal value of the drug becomes heightened due to its ability to alleviate the negative emotional consequences that occur during drug withdrawal.20 There is also evidence that compulsive drug taking in rodents is related to neural plasticity in connections between the orbitofrontal cortex and the nucleus accumbens, which are thought to underlie learning of the values of various outcomes in the world (such as food or drugs). Another way that this could occur is for the goal of obtaining the drug to become habitual, which would be consistent with the research by Fiery Cushman that I discussed in Chapter 4. The most important point here is that drugs have powerful effects on many systems in the brain, so we shouldn’t expect the many alterations of behavior that occur in addiction to boil down to any single cause. Habits are certainly part of the story, but only part.

“My Drug of Choice Is Food” Unlike illicit drugs, food is something that nearly all humans experience every day; go long enough without it, and we will die. Yet an increasing number of individuals claim to be “addicted” to food. Oprah Winfrey famously said, “My drug of choice is food. I use food for the same reasons an addict uses drugs: to comfort, to soothe, to ease stress.” And her statement resonates with many individuals with weight problems.

a d d i c t i o n 139

It is also clear that there are some individuals whose eating behavior is highly disordered, such as the person with binge eating disorder who we met in Chapter 5. However, the concept of “food addiction” remains controversial from a scientific standpoint, particularly with regard to its relationship to obesity in the broader population. It is worth noting at the outset that obesity is a complex phenomenon, which almost certainly has many different causes across people and places. Psychological factors and eating behavior clearly play a role, but they are far from the only factor in the development of obesity. In fact, studies that have tried to predict who will become obese over time, using many different types of information, have achieved only modest success. For example, one study followed more than 1000 children in Chicago from age 5 to age 35, in order to test how well the researchers could predict obesity at age 35 based on various factors present in childhood, such as the family and neighborhood environment.21 Using a large set of possible factors, the researchers could only explain about 10% of the variability in body weight in adulthood. Similarly, psychological factors appear to play an important but relatively limited role. In a study that we performed on over 500 individuals using a large set of psychological surveys (including specific surveys about eating behavior) as well as self-reporting of body weight, we were able to explain less than 20% of the variability in obesity across people.22 Psychological factors related to eating behavior are thus clearly important, but far from the full story about why individuals are increasingly becoming obese in the developed world. By far the best childhood predictor of obesity is whether the child’s mother is obese;23 a child of an obese mother is more than six times more likely to be obese than the child of a lean mother, which likely reflects a complex intersection of genetic and environmental factors. However, that finding doesn’t give us much insight into the potential causes of obesity, which are our ultimate goal if we want to be able to treat and prevent it. Our food environment certainly plays a major role in the relatively recent surge in obesity in the United States and many other developed countries. It is incontrovertible that the foods we eat today are distinctly different from those present for most of human evolution, in

140

chapter 6

large part because much of what we eat is manufactured rather than grown. Studies of the American diet have found that the majority of calories consumed by Americans are ultra-processed—meaning that they include additives not used in regular cooking, such as artificial flavors, texture enhancers, or other chemicals.24 These ultra-processed foods are particularly high in sugar, for a reason: people like sugar, and if a food is sweet they will generally eat more of it. Glance at the ingredients for nearly any processed food, and you will see some kind of sweetener on the list. Even the unprocessed foods that we eat bear little relation to our ancestral diet. While we can’t be sure about the sugar content of wild fruits thousands of years ago, anyone who has eaten a wild berry knows that they are distinctly less sweet than the berries we buy packaged at the store. In fact, increasing the sugar content of fruit is a major aspect of modern fruit breeding. An article from a horticultural research journal said the following in relation to a recent decrease in the consumption of peaches: “Sugar content is one of the most important quality traits perceived by consumers, and the development of novel peach [breeds] with sugar-enhanced content is a primary objective of breeding programs” to improve sales.25 The term “sugar-enhanced content” could indeed be used to describe the entirety of the standard American diet. The impact of this highly processed food is that it causes us to eat more. Recent evidence for this comes from a study by Kevin Hall and his colleagues at the US National Institutes of Health,26 who recruited a group of participants to live at the NIH Clinical Center for 28 days, during which time their meals were provided and their food intake was closely monitored. Each of the 20 participants in the study was assigned to one of two meal plans for the first 14 days: an “ultra-processed diet” similar to the standard American diet, or an “unprocessed” diet composed of foods cooked from ingredients that were minimally processed. Importantly, the menus were matched so that the total amount of calories and various nutrients (such as carbs or fat) that were offered to the participants was nearly identical between the two diets. The participants were given free access to food, and the researchers measured how much they ate each day, along with their body weight and many

a d d i c t i o n 141

other biological measurements. After the initial 14 days, each participant was then switched to the alternative diet for another two weeks; this is known as a crossover study design, and allows the measurements on the two diets to be compared within each person, which provides greater statistical power. The results showed clearly that individuals on the ultra-processed diet ate more—about 500 calories per day—and gained weight and increased body fat, whereas those on the unprocessed diet lost weight and decreased body fat. Critics have noted correctly that it is a relatively small study, but it nonetheless provides a compelling demonstration of just how effective modern food processing is at causing us to eat more and gain weight. Given that our food environment seems to be purposely designed to cause us to eat more, it’s not a huge leap to think that we might become addicted to these foods. Neuroscience research into the idea of food addiction has relied heavily upon research using rodent models. This research is nascent compared to the very extensive research literature on drug addiction, but nonetheless provides some clues as to the relation between brain responses to highly palatable foods and drugs of abuse. Research by Paul Johnson and Paul Kenny has examined what happens to rats when they are given access to a “cafeteria style” diet, which is much more palatable and dense in energy than their usual rat chow.27 The answer is that they eat a lot of it, and they become obese, consuming almost twice as many calories as rats who only had access to rat chow. In order to test how this affected the rats’ responsiveness to reward, the rats were given the opportunity to stimulate an electrode implanted in their lateral hypothalamus (a brain region involved in feeding), and the researchers measured how much stimulation the rat required in order to keep doing it. This test showed that the obese rats required greater stimulation in order to keep going, showing that their responsiveness to reward was reduced overall. Further, when the researchers looked at the level of dopamine receptors in the striatum of these rats, they saw that the obese rats had much lower levels of dopamine receptors, and the more obese rats had even fewer of these receptors—similar to what is seen in drug abusers. Other research has more specifically examined whether rodents can become

142

chapter 6

addicted to sugar. Rats that are provided with sugar and then deprived for a long period of time will “binge” on sugar when they are given access to it again; they also show evidence of anxiety and depressionlike symptoms after withdrawal from sugar.28 Another interesting fact is that hunger increases the readiness to work for electrical selfstimulation in certain brain regions, and satiety decreases it. This may reflect the fact that both the gut hormone ghrelin, which acts to stimulate appetite, and the adipokine leptin, which acts to decrease food intake, have a direct relationship with dopamine: ghrelin action causes the release of dopamine, while leptin reduces the activity of dopamine neurons. While the animal research on food addiction shows clear overlap between food and drug rewards, there are also clear differences in how the brain responds to drugs versus natural rewards such as sugar.29 In particular, neurons in the nucleus accumbens appear to respond to both food reward and drug rewards, but these responses are distinct from one another30 —that is, while the brain’s reward systems respond to both food rewards and drug rewards, it treats them differently. In addition, it seems that the special circumstances of the laboratory may be important for developing addiction-like behavior in rats, as these behaviors only seem to arise when the animals are given intermittent access to sugar after being deprived for a long period; rats will not binge on sugar when they are given free access to it, which brings into question how generalizable these results are to humans. In contrast to the growing evidence for addiction-like responses to food in animal models, the concept of food addiction in humans remains controversial.31 One set of concerns has been raised by the neuroscientist Paul Fletcher and his colleagues, who have pointed out problems in the way that “food addiction” is defined in research studies, as well as weakness in the evidence that has been used to support the idea of food addiction in humans. Much of this evidence has tried to relate obesity to drug addiction by showing differences in the brain’s reward systems or dopamine system in relation to obesity— for example, by showing differences in dopamine receptors in the brains of obese versus lean individuals.32 However, many of these

a d d i c t i o n 143

findings came from small studies that have failed to replicate when tested in larger samples.33 Research from genome-wide association studies has also generally shown that there is little relationship between the genetic variants that are related to obesity and those that are related to addiction. Better evidence regarding the relation between overeating and drug addiction comes from recent work by Uku Vainik of the University of Tartu in Estonia and Alain Dagher at the Montreal Neurological Institute. They have pointed out that the many different ways in which food addiction and related behaviors have been defined can be understood in terms of a more general concept of “uncontrolled eating,” which they define as a combination of heightened sensitivity to the rewarding aspects of food combined with a poor ability to control one’s eating.34 Vainik and Dagher examined the relationship between obesity, uncontrolled eating, and a number of addictive disorders by measuring their relationship to a range of personality characteristics; in essence, they tested whether the “personality profile” was similar between people in these different groups. This study found that while the personality profiles of those with obesity in general was only weakly related to people with addictive disorders, uncontrolled eating was more strongly related to addictions.35 There also appear to be differences in brain activity related to uncontrolled eating. Combining across a number of brain imaging studies that used a diverse set of measures of uncontrolled eating, Vainik and Dagher observed that activity in the prefrontal cortex (mostly during tasks involving food reward) was related overall to the presence of uncontrolled eating behavior; individuals with greater activity in the prefrontal cortex were less likely to report uncontrolled eating. This result has been interpreted as showing that uncontrolled eating is related to weaker self-control, but this is an example of a problematic form of reasoning that I discussed in detail in The New Mind Readers, known as reverse inference. That is, while self-control certainly has been associated with the prefrontal cortex, so have many other functions, so we can’t infer from the presence of a difference in that area that it necessarily relates to self-control. Overall the research into uncontrolled eating suggests that while some people do exhibit eating

144 c h a p t e r 6

behavior that bears some resemblance to drug abuse, the underlying mechanisms seems to be only partly overlapping. My personal conclusion regarding food addiction is that some individuals certainly do suffer from problems with uncontrolled eating, which I think are largely a consequence of the ultraprocessed food environment in which most people now live. The distress caused by these problems provides a reasonable rationale for referring to this as an addiction. However, it’s also pretty clear that the brain mechanisms underlying drug addiction and uncontrolled eating are far from identical, and that we need much better research to understand the mechanisms of uncontrolled eating in the human brain.

Digital Addiction? If a time traveler from 100 years ago were to visit any country in the developed world today, they would likely be immediately struck by the fact that nearly every human they see walking down the street is hunched over and intensely focused on a small, shiny object. The usage of digital devices by children and adolescents is particularly prevalent; teenagers in the US report more than 7 hours of daily screen use each day, much of it involving social media and watching videos. In fact, use of digital devices is so extreme among teens and young adults that the medical profession has developed a new term—“text neck”—for the spinal problems that arise from unnatural flexion of the neck due to smartphone usage. Concern over the significant usage of smartphones, particularly by the “digital native” generation, has led to increasing discussion of “smartphone addiction.” And whereas the concept of food addiction has been controversial, the question of whether heavy smartphone use is damaging and should count as an addiction has been downright explosive. In fact, some have gone so far as to claim that tech companies have intentionally designed their devices in order to addict users, in what Tristan Harris (a leading critic) has called “a race to the bottom of the brain stem.” There is a case to be made that links smartphone usage to the dopamine system and thus indirectly to drug abuse. In addition to

a d d i c t i o n 145

reward prediction error, the dopamine system seems to be particularly sensitive to novelty in the world. You can think of this as a sort of generalized prediction error—since a novel event is by definition one that we didn’t expect. In one study, Nico Bunzeck and Emrah Düzel examined the response in the brain’s dopaminergic centers to pictures that varied in different ways, including how novel they were.36 They found that these dopamine areas were activated in particular whenever the picture was novel, and that this was related to better memory for those pictures when they were tested later. These novelty signals alert the brain that it should open itself up to change, be it through the creation of new habits or new conscious memories. One way to think of a smartphone is as a continuous generator of novelty; a new text, email, or social media post is always just around the corner, and it is this link that has provided some of the justification for treating excessive smartphone usage as an addiction. While this link is provocative, there is little solid evidence that directly relates smartphone usage to specific changes in brain function. A number of small studies have been published that claim to relate device usage to various aspects of brain structure or function, but none are sufficiently large and well designed to provide a strong basis for this claim. The notion of behavioral addictions first arose around the concept of problem gambling, which seems to clearly fit the mold of an addiction in that it is a compulsive behavior that the individual cannot stop despite severe negative consequences. The question of whether excessive device usage rises to the level of an addiction largely turns on the question of what kind of harm or impairment actually results from excessive smartphone usage. A number of studies have reported anecdotal claims from smartphone users that their lives are negatively impacted by their smartphones, but many researchers in this area are leery that these impacts rise to the level of a true addiction. In fact, there is increasing concern that the definition of smartphone usage as an addiction may reflect what the psychiatrist Allen Frances has referred to as “diagnostic inflation”—in essence, pathologizing behaviors that may be excessive but do not rise to the level of a psychiatric disorder. In particular, it has been proposed that a behavior should not be considered as

146 c h a p t e r 6

an addiction simply because it detracts from other aspects of one’s life; to be considered an addiction, it needs to cause “significant functional impairment or distress” for a person.37 To date there is little evidence showing that such impairments occur due to smartphone usage; further studies may yet find them, but those studies will need to use much more rigorous methods of measuring impairment than have been used in previous studies. It has become fashionable in recent years to decry the effects of device usage and social media on the generation that has grown up around these devices. The psychologist Jean Twenge has labeled this generation “iGen” and has argued that the soaring levels of mental health problems in this generation are directly due to their device usage. It’s always dangerous to make inferences about causality from changes over time, illustrated beautifully by Tyler Vigen in his Spurious Correlations website38 and book, and there are of course many other possible causes for these changes in mental health, such as changes in parenting styles over time. Since we can’t perform randomized controlled experiments to determine whether device usage causes mental health problems, the best that we can do is to look at correlations between these factors, and a set of large-scale studies of digital technology usage and mental health by Amy Orben at Cambridge and Andrew Przybylski at Oxford have provided the best evidence to date on this question. In one of these studies, the researchers examined the relationship between digital technology usage and psychological well-being in several very large samples from the US and UK, totaling over 350,000 kids.39 They found that there was indeed a small negative relationship between digital technology usage and well-being, but their samples also allowed them to put the size of this effect in the context of other effects on well-being. For example, the relationship between being bullied or smoking marijuana and well-being was much stronger than the relation with digital device usage, as was the relationship with wearing glasses to school. In fact, the relationship between digital technology usage and well-being was just barely stronger than the relationship of well-being with eating potatoes! These studies suggest that much of the current concern about the effects of digital device usage on mental health may be overblown.

a d d i c t i o n 147

However, it’s also important to remember that observational studies (that is, studies that measure correlations between different factors in the population) are also limited in what they can tell us about causality, though the results from Orben and Przybylski certainly suggest that if there is an effect, it is not particularly large.

Why Do Only Some People Become Addicted? Of the people who try a drug once, only a small subset will become addicted in the long term. Estimates for the rates of addiction vary across studies and across different drugs, but for all drugs other than tobacco (which addicts about two-thirds of those who try it), the estimated rates appear to be in the range of 10%–20% of individuals who ultimately become addicted. The question of why some people become addicted and others do not is of course a question with many layers, only some of which likely have to do with neuroscience. Poor self-control or “willpower” is commonly invoked as a reason why some individuals become addicted, though as we saw earlier in this chapter, this does not accord with the strong-willed nature of drug seeking. As we saw in Chapter 5, the concept of willpower has not held up well as an explanation for differences in self-control. However, there is evidence for differences in some of the ingredients of self-control, particularly response inhibition, that are related to addiction. Many studies have shown that response inhibition is reduced in people who are addicted to drugs,40 but this doesn’t tell us which way the causal arrow points—that is, do differences in inhibition lead to drug addiction, or does drug abuse lead to decreased inhibition? To answer that question, we need to test whether individuals with reduced inhibition are more likely to develop addictions in the future. One study in rats by David Belin, Trevor Robbins, and Barry Everitt looked at whether rats who are more impulsive (as measured using the “hold your horses” task that I described earlier) are also more likely to become compulsive cocaine users, braving foot shocks in order to obtain the drug.41 They found exactly that: more impulsive rats were much more likely to develop compulsive cocaine-seeking habits. Evidence for this relationship in

148

chapter 6

humans comes from the studies of Moffitt and Caspi that I described in Chapter 5, which have shown that children with poor self-control are more likely to go on to develop alcohol problems as young adults. Thus, response inhibition may play a role in the development of addictions, particularly in the transition from experimentation to compulsive use. Genetics also play a role in the likelihood of becoming addicted, by determining how a particular substance affects each individual differently. One of the strongest genetic predictors of alcohol abuse is whether or not someone carries the genetic variant that results in the alcohol flush reaction. This genetic variant affects the function of the gene that breaks down one of the by-products of alcohol, resulting in an uncomfortable flushing reaction that makes drinking so uncomfortable that they are unlikely to drink heavily. Similarly, some of the strongest genetic predictors of smoking are found in the genes for nicotine receptors, which likely affect the degree to which people find nicotine to be aversive. However, genetics plays only a limited role in determining who becomes addicted in general. Perhaps the best evidence for this is the fact that laboratory mice, who have been bred to be nearly identical in their genetic makeup, still vary across individuals in their likelihood of becoming addicted. Research by Vincent Pascoli and his colleagues published in 2018 gave us new insight into where these differences might come from.42 They first implanted an optogenetic stimulator into the dopamine regions of 109 mice and gave the mice the opportunity to self-stimulate their dopamine neurons, which is somewhat like cocaine on steroids. About 60% of the mice (which the researchers labels “the perseverers”) would self-stimulate even after they had to endure a foot shock, whereas the remainder (“the renouncers”) stopped self-stimulating once the shocks started to occur. Pascoli and his colleagues used a set of state-of-the-art neuroscientific tools to identify which specific set of neurons and connections were responsible for these differences in behavior across animals. In this case, they were able to trace the compulsive self-stimulation to a set of neurons connecting the orbitofrontal cortex to the striatum. They focused in particular on the strength of synapses from the orbitofrontal cortex onto medium spiny neurons in

a d d i c t i o n 149

the striatum, and found that the strength of those synapses was related to the amount of perseverance. Pascoli and his colleagues then examined what would happen if they purposely stimulated plasticity in the specific connections between the orbitofrontal cortex and the striatum, which they accomplished by optogenetically stimulating those neurons in a particular way that induces plasticity. This stimulation made the renouncer mice much more likely to self-stimulate in the face of punishment. Conversely, when they eliminated the plasticity in these same neurons in the persevering mice (using a combination of optogenetic stimulation and a drug that blocked dopamine D1 receptors, taking advantage of the three-factor rule discussed in Chapter 2), they saw that the mice reduced their level of self-stimulation. An important caveat to this study is that the effects of optogenetic stimulation are different from drugs in that they are much more specific and faster acting. Nonetheless, the work provides important insights into why some individuals become addicted while others do not. The neuroscientific tour de force by Pascoli and colleagues provides an answer as to what differs between the persevering and renouncing mice, but it still doesn’t tell us where it comes from; keep in mind that these are mice that are very genetically similar to one another and that have grown up in a very similar lab environment. Pascoli and colleagues propose that it may be a reflection of what they call stochastic individuality—where stochastic basically means “random.” The idea behind this concept is that within a biological system as complex as a brain, there will always be a large amount of difference between individuals that is due simply to random factors that can’t be explained.43 There are many ways in which this variability could arise, from the random happenstance of which neurons connect to which, to random variability in the epigenetic makeup from cell to cell that could result in differences in gene expression. There are also, of course, the effects of experience; even when animals share the same genetic makeup, they will necessarily have different experiences throughout their lives, such as their place in the social hierarchies that develop when rodents are housed together. These experiences leave tracks in the brain that could affect their behavior down the road. In fact, research by Jeff Dalley and

150 c h a p t e r 6

colleagues has shown that socially dominant and subordinate rats differ both in their willingness to self-administer cocaine and in the number of dopamine receptors in their basal ganglia. An additional factor that is known to affect the vulnerability to addiction is early life stress or adversity. One large study by Dean Kilpatrick and his colleagues used data from the National Survey of Adolescents to determine how exposure to violence was related to the development of substance use disorders.44 Teenagers who witnessed violence were more than twice as likely to abuse drugs than those who did not. The effects of sexual assault were even more drastic, with survivors almost six times more likely to develop both depression and drug abuse. These effects even appear to extend to the mother’s stress during pregnancy. Animal models of addiction have shown that the offspring of stressed mothers are more susceptible to addiction, which may be due to the effects of stress hormones from the mother being transmitted to the fetus. In summary, neuroscience tells us several important things about addiction. The development of addictive behaviors relies upon many of the same mechanisms as all habits, but supercharged by the unnaturally extended release of dopamine caused by these drugs. However, addiction also relies upon changes in other brain systems, particularly stress systems, that drag abusers to the dark side of withdrawal. We understand a great deal about the biology that leads to the development of addiction, but it seems that we may never be able to easily predict whether any particular individual is going to become addicted.

Part I of the book has shown you how habits are formed and why they are so hard to break. In the second part of the book, I turn to the question of how we can use this knowledge to help develop a new science of behavior change.

7 Toward a New Science of Behavior Change

in part i of the book, we saw how the neurological deck is stacked against us when it comes to changing our behavior. The brain is a habit machine, intent on automating any routine behavior so that we don’t have to spend time thinking about our every move. These habits are built to be sticky, which usually serves us well, until it doesn’t. In particular, many of the features of the modern world drive levels of dopamine response that are far beyond what we experienced in the course of human evolution, and the central role of dopamine in forming habits means that the resulting behaviors are especially sticky. At the same time, our ability to control behavior in accordance with our long-term goals relies upon a fragile prefrontal cortex that can be easily disrupted by stress or distraction, leading us back to our old habits. In Part II I turn to asking what science has told us about how to make behavior change effective. I start in this chapter by laying out just how important the problem of behavior change is for modern societies.

Behavior Change as a Public Health Problem Before the advent of vaccines and antibiotics, the majority of humans died of infectious diseases. For example, in 1900 a disease called diphtheria was one of the top ten causes of death in the United States, 153

154 c h a p t e r 7

leading to more than 8000 deaths—on par with the rate of deaths from Alzheimer’s disease in 2017. However, it’s likely that you have not even heard of this disease unless you are a trained medical professional, because it has been largely eradicated in the developed world through vaccination. Today most adult deaths in the developed world are due to what some have called “diseases of modernity”—that is, diseases directly related to our modern lifestyles and environments. Take the big killer in 2017: heart disease. The causes of heart disease are manifold, but there are a number of behaviors that we know to directly affect one’s risk of heart attack, with smoking tobacco being at the forefront. Smokers are about twice as likely to die of heart disease as nonsmokers, and this risk is reduced by about half within just a year after quitting. In fact, the substantial reduction in heart disease in the US since the 1960s has been largely attributed to a reduction in the number of smokers in the population. Smoking also plays a role in two of the other major killers: cancer and lung diseases such as chronic obstructive pulmonary disease (COPD). Up to 8 out of every 10 deaths from COPD may be associated with smoking. By simply deciding not to smoke (and following through on that decision), tens of thousands of individuals could reduce their chances of prolonged suffering and premature death. The problem is that quitting is just plain hard, and even if one is able to quit for a short period, it’s very easy to fall back on old habits. Figure 7.1 shows a couple of examples of the pattern of results that is seen in nearly every study of behavioral change: most people cannot maintain lasting change, and the last half century of intense research has done little to change that.1 Studies of smoking and alcohol cessation show consistently that only about one-third of people maintain their abstinence for an entire year. Weight loss is similarly hard—while people nearly always lose weight in the short term (regardless of the particular diet plan), they rarely keep it off for more than a couple of years, and often end up gaining back even more. On the other hand, some people are able to make lasting change, and studies of these individuals have started to provide some insight into the important principles of successful behavior change, as we will see in the next chapter.

a n e w s c i e n c e o f b e h av i o r c h a n g e 155

1971 Percent remaining abstinent

100

2011 100

SUBSTANCE Alcohol Heroin Smoking

80 60

60

40

40

20

20

0

0

50 100 150 200 250 300 350 Time (days)

SUBSTANCE Alcohol Opiates Cocaine Marijuana

80

0

0

50 100 150 200 250 300 350 Time (days)

figure 7.1. Relapse curves showing the percentage of people trying to quit various substances who remained abstinent at various points up to one year. (Left) Data from 1971. (Right) Data from 2011.

A New Science of Behavior Change If so many of our health problems could be alleviated by behavior change, then why is the medical profession so powerless to help us change our behavior? In many areas of medicine, detailed biological knowledge has led to the development of treatments that are often revolutionary in their impact. One striking example is human immunodeficiency virus (HIV) infection, which causes acquired immune deficiency syndrome (AIDS). In the 1980s a diagnosis of AIDS meant that one had a roughly 50% chance of dying within 2 years. However, a deep understanding of how viruses work led to the identification of the virus and ultimate development of a combination therapy that has turned an AIDS diagnosis from a short-term death sentence to a long-term chronic disease, with the majority of treated patients now surviving more than 10 years. Similarly, our understanding of the molecular biology of cancer has led to targeted approaches that are changing the face of treatment for some particular cancers. The success of modern medicine points directly to the utility of understanding basic biological mechanisms in order to develop new treatments.

156 c h a p t e r 7

The fact that our behavior change interventions have remained stubbornly unsuccessful during a time in which other medical treatments have improved outcomes so remarkably might lead one to think that the basic understanding of behavior change that gave rise to those interventions is probably flawed. There is in fact no single theory of behavior change within psychology—one paper counted 117 different theories! Unlike some areas of science in which the field works toward a common theory, psychologists tend to develop their own—leading the late Walter Mischel to quip that “psychologists treat other peoples’ theories like toothbrushes—no self-respecting person wants to use anyone else’s.”2 The most widely accepted theory of behavioral change is known as the transtheoretical model, which outlines a set of six stages of behavioral change: •

•

•

• •

•

Precontemplation: The person is not yet ready to make a change in their behavior Contemplation: The person starts to realize that their behavior is problematic and think about making a change Preparation: The person is ready to make a change and starts taking steps toward implementing it Action: The person implements the intended change Maintenance: The person maintains the change for an extended period (at least six months) Termination: The behavior is completely changed and will not return

These all sound perfectly reasonable and intuitive, but you might notice that there is something very different about this theory compared to theories of most diseases. For example, take our current understanding of cancer, which says that mutations in genes controlling cell growth lead to uncontrolled growth of those cells. This theory describes the biological mechanisms of cancer, and it is our understanding of those underlying mechanisms that has led to the recent development of increasingly successful targeted treatments for certain cancers. The transtheoretical model tells us nothing about the underlying brain or

a n e w s c i e n c e o f b e h av i o r c h a n g e 157

psychological mechanisms that make behavior change more or less effective. In this way, it is akin to having a theory of cancer that doesn’t actually include any knowledge about how and why cancer comes about but simply describes the progress of cancer as it develops— which might be useful for predicting the progression of the disease but is not very useful to understand how to treat the disease. It also seems clear that the transtheoretical model has not actually helped researchers develop effective treatments that increase the success of behavior change. Whenever I want to find the most unbiased results regarding a particular medical treatment, I look to the Cochrane Organization. This British group publishes “systematic reviews,” which analyze research on a particular topic according to a set of rules that are meant to make the review as unbiased as possible. In particular, the reviews focus on results from randomized controlled trials, which provide the best evidence as to whether a treatment is effective. In their most recent review of the effectiveness of treatments for obesity based on the transtheoretical model, they found only three randomized controlled trials that even met the criteria for inclusion in their analysis. The conclusion of this review was that the studies were too poorly performed to make any strong conclusions, and that any evidence provided by these studies was of “very low quality.”3 Thus, nearly 40 years after it was first proposed, there is little evidence that the most widely accepted model in this field has any effectiveness in developing new treatments. Clearly, a new approach is needed, and there is hope that this will begin to change with the advent of a new way of thinking about behavior change that has been championed by a set of researchers within the US National Institutes of Health (NIH).

A New Approach to Behavior Change The NIH is by far the world’s largest funder of biomedical research, putting more than $26 billion into research in 2016; compare this to the next largest funder, the European Union, which spent $3.7 billion. The NIH is largely organized around either diseases or organ systems, with institutes focused on cancer, heart disease, diabetes, drug abuse, and

158 c h a p t e r 7

mental health, among others. This means that the funding priorities for each of those institutes will be focused on addressing the specific diseases that are the primary charge of the institute, prioritizing research that is directly relevant to its disease of interest rather than on issues (like behavior change) that cut across different diseases. To its credit, the NIH leadership recognized this problem and developed a program in 2007 known as the NIH Common Fund, which has a budget of more than $600 million that is explicitly directed at research that spans across institutes. Around 2008 a group of researchers from a number of institutes within the NIH began to discuss the idea of creating a new program that would tackle the general problem of understanding the basic mechanisms of behavior change. In particular, they proposed that research on behavior change move toward an approach much more like the one used in experimental medicine. Rather than simply asking whether a treatment has an effect on a disease, the experimental medicine approach focuses on understanding the mechanisms by which the treatment works. In particular, this approach tries to understand the mechanistic targets of the treatment and assesses the degree to which the treatment has engaged the target, in addition to assessing how well the treatment actually works. Once the engagement of a particular target has been linked to treatment outcomes, then researchers can try to enhance the effectiveness of the treatment by maximizing target engagement. For example, let’s say that we want to develop a treatment that improves the ability to envision future outcomes over immediate rewards, which (as we will see later) is thought to be important for behavior change. Using the experimental medicine approach, we would perform a trial that included a way to directly measure whether the treatment actually improved the ability to wait for future outcomes in the face of immediate rewards, and further whether those improvements were related to improved behavioral change such that people who improve most in their ability to wait also show the greatest improvements in behavior change. This approach has been remarkably successful in other areas of medicine and holds great promise for improving our understanding of behavior change.

a n e w s c i e n c e o f b e h av i o r c h a n g e 159

In 2010 these researchers convinced the NIH Common Fund to initiate a program known as the Science of Behavioral Change (SOBC), which has funded millions of dollars of research into the basic mechanisms of behavioral change (including two projects that my group has been involved in). Under the SOBC program, we have started to develop a new scientific understanding of the basic psychological and brain mechanisms that underlie the ability to change behavior, and we are looking for ways to target those mechanisms to drive behavior change. However, while the SOBC program has been a great start, the amount of the investment is tiny given how central behavior change is to nearly every disease that the NIH is commissioned to treat or prevent.

Targets for Intervention If we adopt the experimental medicine way of thinking regarding behavior change, then we must focus on targets for interventions— that is, what particular social, psychological, or neurobiological mechanisms can we manipulate in order to help improve behavior change? The framework laid out in Chapter 1 provides us with an overall way to divide these possible targets. Environment: The environment drives us toward some behaviors and away from others—it’s much easier to smoke a cigarette in a bar than in a church. By understanding the impact of our environment on behavior, we can optimize our ability to make changes in behavior. Habit: The persistence of habits is a clear impediment to behavior change. As we have seen throughout the book, we now have a deep knowledge of the biology of habits, and in Chapter 9 I discuss some potential ways in which this knowledge could be used in the future to target specific habits. We can also use our knowledge of how habits work to avoid common traps, as we will see in the next chapter. Goal-directed behavior: Action in service of our long-term goals requires attention to those goals as well as self-control to override our immediate impulses or habits. Our detailed knowledge of the neurobiology

160 c h a p t e r 7

of the prefrontal cortex and self-control provides us with actionable tools to potentially improve behavior change. In the ensuing chapters, I outline how strategies addressing these targets can help optimize behavior change, from those available now (Chapter 8) to those that might be enabled in the future by advances in neuroscience (Chapter 9). While we do not touch on many of the potential mechanisms that are relevant to behavior change—such as those related to social support, coping skills, and mind-sets—these chapters should leave you with a road map for how behavior change might be improved.

8 Planning for Success k e y s to s u c ce s s f u l b e h av i o r ch a n g e

make no bones about it: behavior change is, and always will be, hard. However, research across several areas, from neuroscience to psychology to economics, provides some immediately actionable tools that help improve behavioral change. It also provides us with good clues about what kinds of methods don’t work. In this chapter I outline this research, focusing on a set of targets that we have encountered at various points throughout the book: the environment, habits, goal-directed behavior, and self-control. This chapter focuses on ideas from psychological research, while the next chapter looks at ideas from neuroscience.

The Architecture of Choice You might think that our choices are guided by our desires and preferences, but in many cases they are also influenced by the way that options are presented to us. The grocery store is an obvious example: a person is much more likely to buy an apple if it is presented at the checkout stand than if it is hidden away in the back of the store. These effects don’t make sense according to classical economic theories, which suppose that humans make decisions solely on the basis of how much they value various outcomes in the world; that is, we buy an apple solely because we value the apple more than other things that we could buy 161

162

chapter 8

instead. However, since the 1970s a field of research known as behavioral economics has arisen that focuses on how humans actually make decisions, rather than how they theoretically should make decisions. Behavioral economists have coined the term “choice architecture” to refer to the fact that in any decision-making situation there are some choices that are encouraged and others that are discouraged, simply by the design of the environment in which the decision is being made. Choice architecture can have an important impact on the choices that we make, as outlined by Richard Thaler and Cass Sunstein in their outstanding book Nudge.1 The idea of a “nudge” is a change to the choice architecture that encourages a particular behavior but doesn’t limit anyone’s freedom. A particularly impressive example of a successful nudge comes from research into the effects of default options on choices. Eric Johnson and Daniel Goldstein compared organ donation rates in European countries that had an opt-in donation policy (that is, explicit consent is required for donation) versus countries with an opt-out policy (that is, consent for donation is presumed unless explicitly denied).2 The results were striking; across a set of countries that are otherwise very similar in all other respects, countries with an opt-in policy had donation rates of less than 30%, whereas countries with an opt-out policy had donation rates of greater than 85%, with most of them nearing 100%. These differences in donation rates show just how powerful defaults can be when people make choices: particularly when we don’t feel strongly about the choice, we are very likely to just go with the default choice that we are presented. The idea of using nudges to make major improvements in behavior change might sound too good to be true in general—and in fact it probably is. A massive study known as StepUp, led by Angela Duckworth and Katy Milkman at the University of Pennsylvania, tested the effect of 53 interventions on exercise adherence (of which 52 were nudges intended to increase exercise and 1 was a negative control meant to have no effect). To test these, they worked with a national gym chain to recruit 63,000 of their members.3 Over 28 days, the participants received various nudges to help push them to go to the gym (such as reminders), in addition to a small payment for each trip to the gym.

p l a n n i n g f o r s u c c e s s 163

The program worked in the short term, in that people receiving the nudges were more likely to go to the gym than a control group who received no intervention. However, once the program was done, there were no clear, long-lasting effects on gym-going. As Milkman said in an interview in 2019: So after our 28-day program, pretty much we saw nothing in terms of behavior change. All 53 versions of the program, pretty much nothing sticks. And that was the ultimate goal. So that was a major failure.4 There may well be situations where nudges can make a major difference, but the results from this major study show that it’s not a universal solution. However, I would not categorize the study as a “failure”—far from it! Rather, it’s a shining example of how to use solid science to test ideas and see if they work in the wild. It doesn’t tell us what we should do, but it definitely tells us what we should not do, and that’s equally important.

Loss Aversion and Framing Pretend that I walk up to you on the street and offer you the following gamble based on a coin flip: heads I give you $25, tails you give me $20. Economic theory says that a rational human should accept this bet; after all, the expected value of the gamble (that is, how much you would expect to win on average) is $2.50, so in the long run you will come out ahead if you say yes. However, very few humans would actually accept such a gamble; research by the psychologists Amos Tversky and Daniel Kahneman shows that most people require the amount that they could win to be almost twice the amount that they could lose before they will accept such a gamble, a phenomenon that they refer to as loss aversion. Loss aversion is seen not just in the laboratory but also in the real world. It is a well-known phenomenon in the stock market that individual investors are more likely to sell stocks that have gained money relative to their purchase price, compared to those that have lost value. It might seem that this is an example of “selling high,” but in reality the purchase price of a stock should not matter in one’s decision to sell;

164 c h a p t e r 8

if the investor thinks that the discounted future value of the stock is greater than the current price then they should keep it, otherwise they should sell it. Terrance Odean showed in 1998 that this behavior leads to substantial losses for investors in the long term, particularly given the possible tax benefits of selling losing stocks rather than holding onto them.5 Loss aversion also leads us to make different choices depending on whether particular outcomes are described as gains or losses, a phenomenon known as framing. Tversky and Kahneman showed this in a famous experiment known as the “Asian disease study.”6 One group of subjects was presented with the following choice: Imagine that the US is preparing for the outbreak of an unusual Asian disease, which is expected to kill 600 people. Two alternative programs to combat the disease have been proposed. Assume that the exact scientific estimate of the consequences of the programs are as follows: If program A is adopted, 200 people will be saved. If program B is adopted, there is a one-third probability that 600 people will be saved and a two-thirds probability that no people will be saved. In this case, 72% of people choose program A. Another group of subjects was presented with the same options, but framed slightly differently: If program C is adopted, 400 people will die. If program D is adopted, there is a one-third probability that nobody will die and a two-thirds probability that 600 people will die. In this case, the majority (78%) of subjects chose program D. If you look at the two problems closely, you will see that programs A and C are identical, as are programs B and D; the only difference is whether they are described in terms of gains or losses. In this case, the difference in decisions between the two framings seems to be driven by the fact that people are generally more willing to take a chance to avoid a loss, whereas they are less likely to take a chance when there is a sure

p l a n n i n g f o r s u c c e s s 165

gain involved. This finding (which is very robust) demonstrates how the framing of an outcome can radically change the choices that we make. The effects of framing on decision making are also important for choice architecture. In one study, my Stanford colleague Alia Crum and her colleagues performed a large-scale trial in the dining halls of five universities to determine how the labeling of vegetables in terms of either tastiness (“Herb n’ Honey Balsamic Glazed Turnips”) or healthiness (“Healthy Choice Turnips”) would affect students’ choices.7 Across more than 130,000 individual dining decisions, they saw that labeling in terms of taste resulted in a substantial increase in vegetable purchases. There is not yet research on how framing can contribute to individual behavior change, but the robustness of framing effects across many different contexts suggests that we could possibly improve behavior change by framing the options in the most appropriate way.

Make Rules, Not Decisions Another way to design for successful change is to use strict rules to enforce the change. If we allow ourselves to make a decision each time a possible temptation appears, it’s likely that we will fail at least some of the time—it’s just too easy to find an excuse or a justification for any particular lapse. Specifying rules that outlaw particular behaviors in the home or other contexts can help remove the feeling that we actually have a choice about the matter. One large study of smoking cessation found that smokers trying to quit were 10 times more likely to succeed if they lived in a home that was smoke-free, and twice as likely to succeed if their workplace was smoke-free.8 While rules appear to be useful for behavior change, not all rules are created alike. Some diets specify a complex set of rules that the dieter must use in order to adhere to the plan. For example, the popular Weight Watchers diet provides the dieter with a target number of “points” and a guide to how many points each food item is worth. This requires a substantial amount of cognitive work on the part of the dieter, and while it provides a structured set of rules, the dieter is still left with

166 c h a p t e r 8

many individual decisions to make. Compare this to a diet that simply provides recipes for each meal, to be followed exactly. Such a diet would in principle require no decision making on the part of the dieter. A study that compared two such diets found that while there were no major differences in adherence between the diets, a major reason that people reported quitting the more complex diet was its perceived complexity.9 This suggests that simpler rules are likely to be more effective at ensuring behavior change, though the research here is still in its infancy. The idea that people may use simple rules to make choices, rather than weighing all the different features of a choice, has been promoted by the German psychologist Gerd Gigerenzer, who has coined a term to describe this form of decision making: “fast and frugal.” In many different domains, his research has shown that people often use simple rules of thumb (known as heuristics) rather than taking all of the available information into account. There is evidence that this occurs when people make food choices. One study asked participants to rate a number of different dishes on nine different attributes, such as its healthiness, tastiness, convenience, and price, and then had the participants make choices between different pairs of the dishes.10 The researchers then tested how well they could predict the participants’ choices, either using all of the different attributes or using only the most important attribute for each participant (unless there was a tie between the dishes, in which case they used the second most important attribute). They found that they were able to predict the participants’ meal choices just as well using their “fast-and-frugal” model that only uses one or two attributes, compared to the model that used all of the attributes. The fact that people use simplified decision strategies when they make choices provides further evidence that rules should be as simple as possible in order to be effective.

Trigger Warning: Intervening on Habits We saw in Chapter 3 that one of the important mechanisms for the stickiness of habits is the fact that they come to be easily triggered by

p l a n n i n g f o r s u c c e s s 167

cues in the environment. This suggests that one potential strategy for intervening on habits is to prevent the appearance of triggers for the habit, so that it never gets triggered in the first place. A set of studies by Angela Duckworth and her colleagues provides direct evidence for the utility of removing temptations rather than relying on willpower to stop us.11 They examined the ability of high school and college students to achieve their study goals, which required them to avoid distraction. In each study, one group of students was told to implement strategies to remove temptations that might distract them (for example, installing an app to block Facebook usage during their study time), while another group was told to practice exerting willpower to resist the temptation whenever it arose. A week later the students rated how well they had met their goal. The students who had changed their environment consistently reported better achievement of their study goals than those told to use willpower to resist temptations. One particular challenge with avoiding habit triggers is that they powerfully draw our attention, via the value-driven attentional capture mechanism that I discussed in Chapter 3. This bias is particularly strong in the case of addictions, where individuals are strongly biased to pay attention to cues related to their substance of choice. A number of studies have tested whether it is possible to reduce this bias through training, an approach known as attention bias modification. This involves presenting individuals with pairs of stimuli in which one image is related to their addiction (such as images of cocaine for a cocaine addict) and the other is a neutral image, and training them to attend to the neutral image. While this training invariably reduces the amount of attentional bias on the experimental task, the important question is whether it generalizes to the real world, and here the answer seems to be negative. Across several studies that examined attention bias modification training for various addictions, there was no evidence that it was effective in reducing drug use outside of the lab.12 However, the fact that cues are so strong and salient suggests a simple strategy for someone aiming to change behavior: identify the cues that trigger the habit and remove them from one’s environment to the greatest degree possible.

168

chapter 8

A radical but potentially effective solution to avoiding habit triggers is to move to a new location. A qualitative study by Todd Heatherton and Patricia Nichols asked participants to write stories about their successful or failed attempts at making changes in their lives.13 One of the biggest differences between those who had successfully changed and those who had failed to change was moving to a new location; successful changers were almost three times as likely to have moved compared to nonchangers. Wendy Wood and her colleagues examined this more directly by studying how exercise habits changed before and after students transferred to a new university.14 Their results showed that changing one’s location had a particularly large effect on those with initially strong habits, substantially reducing their exercise frequency compared to those with weak exercise habits. On the other hand, for those with initially weak habits, the change actually helped bring their behavior into line with their intentions, so that those with stronger intentions to exercise actually exercised more. Thus, changing one’s environment by moving appears to have potentially powerful effects on changing behavior.

Reversing Habits As many as 20% of children will experience a tic—a specific action or pattern of actions that is made repeatedly. In some cases, this can be as minor as an eye twitch, whereas in the worst cases, it can involve selfinjurious behavior or uncontrollable utterance of curse words. While most cases of tics resolve by adulthood, some individuals are left to struggle throughout their life with this behavioral disorder, which is known as Tourette syndrome. While we don’t understand the causes of this disorder, it is thought that these tics arise through hyperactivity of the same brain mechanisms that normally give rise to motor habits.15 While there is no cure for Tourette syndrome, there is an effective treatment that can substantially reduce the prevalence and severity of tics.16 Known as comprehensive behavioral intervention for tics, or CBIT, this therapy involves multiple components, each of which

p l a n n i n g f o r s u c c e s s 169

provides potential insights into how we might improve behavior change in general. These include •

•

•

•

•

Awareness training: The patient works with the clinician to learn more about their tics and to identify the signals that a tic is about to occur (known as premonitory urges). Competing response training: The patient develops a new alternative behavior that prevents the tic from occurring. For example, if an individual has a tic that involves moving their head to one side, they might tense the muscles on the other side of their neck when they feel the tic coming on, in order to prevent it. Generalization training: The patient practices using the competing response in their daily life, outside the context of the clinic. Self-monitoring: The patient and/or a support person (such as a parent or significant other) monitors and records the occurrence of tics. Relaxation training: The patient learns breathing and muscle relaxation techniques that can reduce tension, which is often a trigger for tics.

Randomized controlled trials have demonstrated that CBIT is more than five times as effective as standard treatment in reducing tics, even though it doesn’t work for everyone and it doesn’t completely eliminate tics. There are a couple of important takeaways from the success of CBIT. First, it confirms what we have already learned in many other contexts: behavior change is hard work! Succeeding at CBIT requires significant effort on the part of both the patient and family members in order to endure the necessary training and to consistently monitor behavior. Second, it highlights the importance of a broad-spectrum approach to behavior change. Treating such a difficult problem requires combining a number of different techniques, none of which would be sufficient on their own. Perhaps most importantly, the success of CBIT provides us with a beacon of hope by demonstrating that it is possible to successfully change even the most problematic behaviors.

170

chapter 8

Mindfulness: Hype or Help? One of the fundamental tenets of Buddhism is that suffering exists because of craving, and that meditation and mindfulness are keys to ending this suffering. This ancient wisdom has now become a billiondollar business, from posh meditation retreats to smartphone apps. In fact, one online magazine announced that “Meditation and Mindfulness Training Is One of the Best Industries for Starting a Business in 2017,” and one of the most lucrative parts of this market is focused on weight loss. Some meditators rave about its ability to reduce their cravings: My practice has helped me with all kinds of cravings and aversions over the years. Meditation helped me quit drinking, quit smoking, and stop eating crappy food. It has also helped me get over my aversion to exercise which led to me loving a good, sweaty workout on a regular basis. . . . All in all meditation has been a tremendously powerful antidote to craving and aversion in my life.17 The internet is also replete with numerous stories touting the power of meditation to improve willpower and self-control, usually with an obligatory mention of how it affects the prefrontal cortex. Most recently, the idea of meditation has been repackaged in the Silicon Valley fad of the “dopamine fast,” in which the individual tries to avoid all stimulation whatsoever. On its face, meditation seems poised to address both of the possible mechanisms of behavior change, by reducing our cravings and improving our executive control. But what does the science tell us about the actual effectiveness of meditation for behavior change? Assessing the science around meditation is challenging, in part because this is an area in which many of the researchers are also advocates for the technique. My skepticism is backed up by a consensus paper published in 2018, titled “Mind the Hype: A Critical Evaluation and Prescriptive Agenda for Research on Mindfulness and Meditation.”18 This paper, whose authors included a number of prominent meditation researchers, outlined many of the problems with meditation research to date, including ambiguity about what “mindfulness” means and how it

p l a n n i n g f o r s u c c e s s 171

is measured, and the very poor quality of most trials that have examined the clinical effectiveness of mindfulness interventions. The authors also highlighted the fact that most studies in this area have not even measured the presence of adverse effects of meditation, but rather have simply assumed that meditation has no harmful effects, even though traditional Buddhist meditation guides discuss common negative outcomes. There is also evidence of biased reporting of outcomes in the field of meditation research. One analysis found that more than half of the clinical trials for meditation interventions that had been registered in a national database remained unpublished 30 months after being completed, suggesting that researchers might be hiding away negative findings in the file drawer.19 This kind of cherry-picking (known as publication bias) can lead to a research literature that appears to provide evidence for a treatment, even when there is in reality no such effect. Many people find meditative practices to be very helpful (I personally do yoga on a regular basis and find it very beneficial both mentally and physically), and the problems with the meditation research don’t undercut that personal utility. However, these problems do deflate some of the overblown claims about the effectiveness of meditation as a treatment for everything from eczema to depression to cancer. More generally, these research findings highlight that we should be very careful about the results of any particular study and look closely to see whether a treatment has been broadly established in a way that is backed up by solid science from multiple research groups without conflicts of interest.

Can Self-Control Be Boosted? Given the importance that many people have placed on the role of willpower in behavioral change, a number of researchers have examined whether self-control can be improved through targeted training. It probably won’t surprise you at this point that a number of studies claim to have found positive effects of various types of training, from physical exertion to controlling one’s speech. However, meta-analyses (see Box 8.1) of these studies have found evidence of publication bias,

Box 8.1. Combining research studies using meta-analysis Meta-analysis is a method that is used widely across science to determine the consensus across a number of published research papers. To perform a meta-analysis, one first needs to find all of the relevant studies that have asked the question of interest. Because it is common for studies that do not show a statistically significant finding to go unpublished (known as the file drawer problem), the researcher will often try to find unpublished studies as well as published studies to include in the meta-analysis. Some studies may also be excluded, for example, due to the poor quality of their methods. Once a set of studies is identified, each study is examined to determine the size of the effect that was reported. When studies are published, they generally include a measure known as an effect size, which describes the size of the effect in relation to the variability in the data. Let’s say that we wanted to determine the relation of regular physical activity to body mass index (BMI), which is a measure of body weight in relation to height and is often used to determine whether a person is overweight or obese. A large dataset known as the National Health and Nutrition Examination Surveys (NHANES) provides these data on about 5000 Americans, which we can use to explore how meta-analysis might work. Let’s suppose that 10 different researchers obtained data from different samples of 200 individuals in the NHANES dataset and compared the BMI of people who reported engaging in regular physical activity versus those who did not. Each group of 200 would have a range of BMI values, and these ranges would differ between the different samples of individuals. In order to quantify the effect size, the researchers first need to determine the difference between the groups by simply subtracting the BMI for the active group from the BMI for the inactive group. Let’s say that we do this and we find that the inactive group has an average BMI of 29.6 and the active group has an average of 27.9, so that the difference between the groups is 1.7. This number is not very useful on its own because we don’t know how to interpret it; if BMI varies widely across individuals, then we might not consider it to be a very large effect, whereas if BMI values for all of the individuals are very close to those mean values, then the difference between the groups would be considered large. We can quantify the variability of the individuals using the standard deviation (discussed in Box 5.4), which is basically the average amount that the individuals differ from the group average. To compute the effect size, we simply divide the average difference by the standard deviation across all of the individuals. If we collect 10 different samples of 200 people and compare BMI for active versus inactive groups, we see that the estimated effect sizes vary from 0.03 to 0.46, and 6 of these 10 studies show a statistically significant difference in BMI between the groups. To perform a simple meta-analysis we can simply take the average effect size across these groups, which is 0.26; this is very close to the value of 0.24 that we get if we take the entire NHANES dataset, showing that meta-analysis has helped us find the right answer. One problem, however, is that researchers often don’t publish results that are not statistically significant, which in this case would leave us with only six results. If we were to perform a meta-analysis on only these studies, we would estimate that the effect size was 0.38, about 50% larger than the true effect size. There are a number of advanced statistical methods that allow researchers to address this problem, which can sometimes change the conclusions of a meta-analysis drastically.

p l a n n i n g f o r s u c c e s s 173

meaning that negative results may have been filed away and thus that the published literature is biased toward positive results. After correcting for that bias, the effects of self-control training appear to be basically zero. There is another more fundamental challenge that any study of cognitive training must grapple with. Let’s say that someone engages in a training program to improve self-control, such as training themselves to resist temptation to eat a sweet by staring at a candy bar in their dining room every morning and resisting the temptation to eat it. The goal of the training is not simply to get better at the particular activity that is being trained, but for the effects of that training to transfer to other contexts—for example, resisting temptation to eat cake at a restaurant. One of the fundamental principles of the science of learning is that this kind of transfer is very difficult to establish. In 1901 the psychologist Edward Thorndike gave a name to this idea: the principle of identical elements, which states that learning in one situation will transfer to another situation only to the degree that those situations share some identical elements. The question this raises is whether something very abstract like “resisting temptation” can count as an identical element, and a great deal of research suggests that it can’t. The challenge of transfer has been front and center in studies of “brain training,” which have examined whether engaging in purposebuilt online cognitive exercises can improve cognitive function more generally. One large study by Adrian Owen and his colleagues followed more than 11,000 people as they participated in a six-week online cognitive training program meant to improve a range of cognitive skills.20 The training was very effective at improving the participants’ performance on the specific tasks that were trained. However, there was no transfer to other tests, even those that were relatively similar to the ones that had been trained. This specific finding has been criticized for not providing a long enough period of training, but the result was backed up by a consensus document published in 2016, which concluded that there was “little evidence that training enhances performance on distantly related tasks or that training improves everyday cognitive performance.”21 This

174

chapter 8

might be the one way in which mental functions really are like muscles: just as training one’s biceps doesn’t generalize to having stronger abs, training on one cognitive task will generally only strengthen the specific skill being trained. The consensus paper also noted that the quality of evidence in most of these studies was low, meaning that one should generally be leery of any particular study claiming to find effects of brain training (a story that by now probably doesn’t surprise you). Another problem with research in this area is that there is money to be made off of an effective training program, meaning that researchers doing this work often have financial conflicts of interest. Another group of studies has focused more directly on training of working memory, which as I discussed in Chapter 5 is the ability to hold information in mind, avoiding distraction and updating that information when the world changes in a relevant way. Here too there has been substantial controversy. For example, one meta-analysis looked at results from 23 studies of working memory training and found that, while working memory training did improve performance on the specific task that was trained, these effects were relatively short-lived and did not transfer to other domains of cognitive function.22 Thus, improving self-control through training remains a promise, not a reality, at this point.

Training Inhibition Another body of research has focused on the role of inhibition and selfcontrol in behavior change, though our discussion in Chapter 5 has already shown us that the relations between basic inhibitory control processes and behavior change may be weaker than many would have expected. Instead of training inhibitory control in general (which we and others have found to be very difficult), most studies have focused on training people to inhibit their responses to specific types of stimuli, such as food or drugs, with the goal of generating a lasting inhibitory response to those items. There have been a number of small studies showing that training to inhibit a response to a particular class of foods

p l a n n i n g f o r s u c c e s s 175

can result in reduced consumption of that food, at least in the laboratory. However, my laboratory and others have failed to replicate these effects, and large-scale trials have also been less positive about the potential for trained inhibition to change behavior outside the lab. For example, one randomized controlled trial compared three different types of inhibitory control training with individuals who wished to reduce their drinking.23 One condition used a “go/no-go” task, in which the subjects were shown pictures (which included alcoholrelated images) with a letter overlaid in the corner of the image; they were then asked to perform a task that required them to respond on all images that contained a particular letter and to withhold their response on images that contained another letter. Unbeknownst to the subjects, the no-go stimulus always appeared for every alcohol-related image. A second condition used a version of the stop-signal task (discussed in Chapter 5), in which stop signals were presented 50% of the time for alcohol images and never for nonalcohol images. The goal of both of these conditions was to try to associate inhibition with the alcohol stimuli and thus reduce later consumption. The participants performed these training sessions online over the course of a month (up to 14 sessions) and also kept a drinking diary. The results showed that while all subjects drank less over the course of the study, the inhibitory training did not result in any more reduction in drinking compared to a control group who simply responded to pictures. There may well be cases in which inhibitory training can change behavior outside the lab, but these remain to be validated in robust clinical trials.

Envisioning Change Your key to planning a successful operation is to anticipate possible future events and to be prepared for contingencies. —us a rmy field ma nua l

A fundamental problem for behavior change is known as the intentionbehavior gap—referring to the fact that many individuals will decide to make a change in their behavior, but then fail to actually take the

176

chapter 8

necessary actions in order to make the change happen. The best intentions to change one’s behavior are useless without planning for how they will be implemented. Let’s say that you wish to stop smoking. What are the situations in which you will be tempted to smoke, and what will you do in order to avoid smoking in those specific circumstances? Within behavior change research, these kinds of detailed if-then plans for how change will be enacted are known as implementation intentions, and there is good evidence that they improve the effectiveness of behavior change. Large meta-analyses of physical activity and healthy eating interventions have shown that implementation intentions have a positive impact on the effectiveness of those interventions. The effects of implementation intentions are relatively small in these studies, but this research also shows that more specific plans are more likely to be effective.24 So instead of “If someone offers me a cigarette, I will say no,” a smoker might think of all the possible temptations and how they would deal with them—for example, “If my friend Tina offers me a cigarette, I will first remind myself of the importance of my goal of not smoking, and then tell her that I appreciate the offer but that I am trying to go one year without smoking.” Even if we have decided to make a change and planned our implementation of the change, we often fail to put the change in practice when the rubber meets the road. In an episode of the popular podcast Radiolab titled “You vs. You,” the hosts interview an eighty-year-old woman named Zelda Gamson. Zelda had been a lifelong activist for racial equality, and in 1984 was visiting her friend and fellow activist Mary Belenky in Vermont. Zelda had also been a smoker for 30 years, having tried and failed many times to quit. When Mary met Zelda at the airport and saw her with a cigarette, she exclaimed “Why Zelda, are you still smoking?,” to which Zelda responded “Yeah, and don’t tell me to stop!” The comment clearly got under Zelda’s skin, and when she was leaving town she said to Mary: “OK Mary, if I ever smoke again, I’m going to give 5000 dollars to the Ku Klux Klan!” Every time she went to smoke a cigarette after that, she was haunted by the idea of the Klan taking her money, and she never smoked again. This type of pledge is known as a commitment device and appears to be an effective means for enhancing behavior change. One study

p l a n n i n g f o r s u c c e s s 177

examined the effectiveness of these devices for weight loss in almost 4000 people, using a web-based platform that allows the participant to make a monetary commitment in service of a weight loss goal.25 If the goal is not met, the pledged amount can be given to a friend, a charity, or an “anti-charity,” like the KKK in Zelda’s example. This study found that individuals who committed money were more successful at reaching their weight loss goals than those who didn’t commit any money, and those who committed their money to an anti-charity were most successful. Some evidence shows that commitment devices to change one’s diet for weight loss are particularly effective if they are made public.26 Another important aspect of implementation is obtaining feedback about whether one’s efforts are actually working—both as a reward when they do work and as a message that it’s time to change things up when they don’t. Evidence shows that monitoring of behavior change appears to be an important aspect of success. Rena Wing of Brown University has studied individuals who are part of the National Weight Control Registry, which follows more than 10,000 individuals who have lost at least 30 pounds and kept it off for at least a year. One of the common features of these “successful losers” is that they monitor their weight closely, with almost half weighing themselves once a day, which is consistent with other research showing that self-monitoring of weight is important for weight loss. Similarly, there is evidence that self-monitoring (involving recording one’s level of consumption) can help with reducing excessive alcohol consumption.

Summing Up We can see a few common takeaways from the various studies described in this chapter. In order to maximize the success of behavior change, individuals should •

•

Look closely at their environment in order to better understand the situations that trigger the unwanted behavior Change their choice architecture to minimize habit triggers and promote wanted behaviors

178 •

•

chapter 8

Prepare a detailed plan for how the change will be implemented, including if-then rules for how specific situations will be handled Closely monitor progress toward the goal, and change the plan when it’s not working

The research outlined in this chapter has evolved over many decades, giving it a head start over neuroscience in providing usable knowledge. However, in the next chapter we look at some new ideas that neuroscience provides about how we might better enable behavior change in the future by more directly targeting particular brain mechanisms.

9 Hacking Habits n e w to o l s f o r b e h av i o r ch a n g e

if we want to move beyond the scattershot approach to behavior change that has characterized most previous approaches, we need to better understand the brain mechanisms underlying behavior change so that we can more directly target them. So far we have seen two possible mechanisms that we might be able to target in order to improve behavior change: the habits that drive the behavior, and the executive function that allows us to engage in goal-directed behavior to avoid the habit altogether or short-circuit it once it is engaged. In this chapter I examine possible ways in which both of these mechanisms might be targeted biologically. There are a number of potential avenues from neuroscience that could one day enable biological targeting, though none is yet supported by strong evidence and some remain in the domain of science fiction.

Can Bad Habits Be Erased? A couple of years ago I was on a trip to Montreal when I noticed a strange bump on my forehead, like a pimple but not painful. It continued to grow over the next few weeks, and when I finally saw my dermatologist about it, she biopsied the bump and gave me the verdict a few days later: I had skin cancer. Not a life-threatening variety, but something that needed to be removed soon. The minor surgery to 179

180

chapter 9

remove the cancer from my forehead took less than an hour, and once it healed the scar is hardly even visible. What if we could do the same thing for bad habits, removing them with surgical precision and with little impact on the rest of our behavior?

Erasing memories When we experience an event in our life, it can seem as if the memory for that event is instantly created, but in reality the creation of lasting, long-term memories requires biological processes that occur over a much longer time scale, known as memory consolidation. The creation of a lasting memory starts with cellular processes involving the activation of molecules known as protein kinases within neurons that have been activated. These protein kinases have several important effects in the neuron, all of which lead to stronger connections between neurons. First, they change the effectiveness of glutamate receptors in the synapse so that the same set of receptors can cause a larger effect when they are activated. They also result in the delivery of new glutamate receptors to the synapse, making it more sensitive to incoming signals. In the longer term, these protein kinases are also involved in changes in the structure of the neuron that can maintain the memory over time. One particular molecule that has gained substantial interest, and controversy, for its role in memory consolidation is known as PKMzeta (referring to the zeta isoform of protein kinase C in mammals), which was discovered by Todd Sacktor from the State University of New York. While most protein kinases are involved in the early phase of memory creation, this protein kinase appears to be specifically involved in the maintenance of memories over time, as Sacktor and his colleagues showed in a remarkable paper published in 2007.1 Most of us have had the experience of getting sick after eating a particular food and then having a lasting aversion to that food. Neuroscientists refer to this as a conditioned taste aversion, and it is a very powerful form of learning; I’m still revolted by the smell of beef barley soup, even though the illness in question happened more than 40 years ago. Sacktor and his colleagues first created an aversion to a

h a c k i n g h a b i t s 181

particular taste by injecting rats with lithium chloride just after experiencing the flavor for the first time (since novel tastes are the most likely to lead to conditioned taste aversion). This kind of learning is known to rely upon a particular part of the brain that is involved in taste, known as the insula. All of the rats exhibited a conditioned taste aversion, reflected in the fact that they avoided the new flavor even after they had recovered from their nausea. Three days after this experience, some of the rats were injected directly to the insula with a drug called ZIP, which inhibits the activity of PKM-zeta, while others were injected with a placebo. Amazingly, the rats who had been injected with ZIP appeared to lose their taste aversion very quickly, and it did not return even after a month. This effect has been replicated a number of times, though it has been somewhat controversial. In 2013 two groups of researchers reported that they had engineered mice who lacked the ability to create PKM-zeta but who still exhibited normal learning and memory, which seemed to suggest that PKM-zeta was not necessary for long-term memory after all. Sacktor and his colleagues showed in 2016 that these genetically engineered mice had actually compensated for the missing PKM-zeta using a different protein kinase (though some researchers still question whether this result fully resolved the controversy). PKM-zeta appears to play its role in learning by helping to stabilize the changes in synapses that occur during neural plasticity. One of the mechanisms that underlies plasticity is the appearance of new glutamate receptors within the synapses that are activated, which help strengthen the synapse by making the postsynaptic neuron more sensitive to input. When PKM-zeta is generated, it gets sent to the dendrites (where the receiving end of the synapse is located); unlike other protein kinases that are turned off soon after they are activated, PKM-zeta remains active for much longer, and it is thought to play a role in preventing those new glutamate receptors from being removed. The manipulation of PKM-zeta to help erase habit memories in humans is still far away, but there is increasing evidence that it may be able to erase the memories that underlie drug addiction. Several studies in rats have found that injecting ZIP into the nucleus accumbens results in a disruption of the place preference that rats usually show for

182

chapter 9

the location where they have received drugs.2 Other research in rats has also shown that ZIP can disrupt habit memories that are stored in the part of the striatum involved in motor habits.3 One can certainly imagine that one day these drugs will be used to try to excise drug memories, but there are a number of concerns that this will raise. In particular, the effects of the drug are likely to be widespread, erasing any memories that rely upon that particular area of the brain in a general way. For this reason, there is greater enthusiasm about another approach that appears to have the potential to much more specifically disrupt particular memories, known as reconsolidation.

Destabilizing memories It was long thought that once memories were consolidated they remained stable, but a phenomenon initially discovered in 19684 and rediscovered by Karim Nader and Joe LeDoux in 20005 flipped this dogma on its head. In their experiment, Nader and LeDoux first trained rats to fear a particular sound by shocking their feet whenever the sound was played. After a bit of experience, rats came to fear the sound, causing them to freeze. It was already known at that time that injecting the rats with a drug that impairs the creation of new proteins would prevent the long-term consolidation of the memory if it was injected just after learning, but not if it was injected hours later. Building off of earlier ideas about the reactivation of memories, Nader and LeDoux had the intuition that the memory trace underlying fear learning might become unstable if the animal was reminded of the earlier experience just before being injected with the drug. To test this, they waited a day after initially training the rats to fear the tone, and then they put them back in the box and played the tone again (without a shock this time), after which they injected them with the drug that blocked creation of new proteins. These rats showed much less freezing the following day, compared to rats who also got the reminder but had been injected with an inert substance, showing that the reminder had made the memory unstable so that its maintenance required further protein synthesis. They labeled this phenomenon reconsolidation, and it has subsequently

h a c k i n g h a b i t s 183

Updated memory Reconsolidation

New memory

Consolidation

Stable memory

Reminder

Unstable memory Blocked protein synthesis extinction

Lost memory

figure 9.1. Memories are initially solidified through the process of consolidation. When they are reactivated, they become unstable and must be reconsolidated in order to become stable again. Blocking this process of reconsolidation, either through drugs or other manipulations, can result in a loss of the memory.

become a topic of great interest in the neuroscience of learning and memory. Figure 9.1 provides a schematic of the reconsolidation idea. Many of the early studies of reconsolidation focused on fear learning, but researchers have also begun to look at whether reconsolidation affects reward-related habits as well. An experiment by Jonathan Lee and Barry Everitt examined this by training rats to press a particular lever to receive cocaine upon seeing a light appear.6 To test for reconsolidation, they reminded the rats with a presentation of the light (without any cocaine) and then injected them with a drug that blocked the synthesis of new proteins. They then looked at whether the rats would learn to press a lever in order to make the light appear (since they should have associated the light with cocaine). Rats that had not received the drug that blocked protein synthesis quickly learned to press the lever to make the cocaine-associated light come on, and so did a group of rats that had not received the reminder before learning. However, the rats that had the reminder and received the protein-blocking drug did not learn to press the lever to make the light appear; it was as if they had forgotten that the light had been earlier associated with cocaine. A study by Lin Yu and colleagues at Peking University took this even further, showing that drug memories could be blocked not just by drugs that interfere with building new proteins but also by experiences.7 Rats

184

chapter 9

first learned to associate a particular part of their cage with injection of either morphine or cocaine. After learning this, the rats were simply reminded of the association by placing them back in the cage for a short period with no drug. Then, after a short delay, the rats were placed back in the cage for a very long period (3 hours) with no drug, which was meant to extinguish the association between the location and the drug. Marie Monfils of the University of Texas had previously shown that this kind of extinction treatment could successfully cause the modification of fear memories but only within a particular time window, from roughly 10 minutes to an hour after the reminder. Yu and colleagues found that this reminder prior to extinction training was able to modify the rats’ memories of the drug and prevent drug-seeking behavior, but only if the reminder occurred 10 minutes before the extinction training; waiting 6 hours to give the extinction training prevented changes in the memory. This shows that reconsolidation relies upon biological processes that take place in a relatively short time window after an experience. The findings of Yu and colleagues in rats led them to ask whether their approach could work in human heroin addicts. In fact, the idea of reconsolidation as a clinical tool had been preceded decades ago by a tantalizing finding that suggested that the method could erase problematic habits of thought. A psychiatrist named Richard Rubin published a short report in 1976 in an obscure psychiatry journal that outlined a study of 28 psychiatric patients, many of whom suffered from obsessions, compulsions, or paranoid delusions.8 At the time, the treatment of choice in psychiatric hospitals for many different mental disorders was electroconvulsive therapy (ECT)—often referred to colloquially as “shock treatment.” Many of these patients had previously undergone ECT with no response, but Rubin hypothesized that one problem was that these treatments had been given while the patients were anesthetized (to prevent injury). He thought that if the patients were instead to act out their obsessions or compulsions during the treatment, this would cause amnesia and thus “cure” them of the harmful thoughts. The results that he reported were impressive: “All patients, following a single ECT . . . improved dramatically for periods of three months to

h a c k i n g h a b i t s 185

ten years to date. One relapsed after nine months but recovered after further treatment.” In the study of heroin addicts by Yu and colleagues, the extinction program involved viewing drug-related videos and pictures and handling drug-related cues, including fake heroin, for an hour; just as in the rat study described earlier, the intuition is that experiencing drug stimuli without actually obtaining any drug is sufficient to interfere with the drug memory. Some subjects received a reminder either 10 minutes or 6 hours before this extinction training, which comprised a short video with drug cues. The researchers measured the subjects’ physiological responses and found that the extinction preceded by a reminder 10 minutes earlier resulted in a lasting reduction in heroin craving up to 6 months later. This was a relatively small study (with only 22 subjects in each group), and to my knowledge it has never been replicated by an independent group, so it remains a tantalizing but preliminary demonstration that needs to be tested in a much larger trial. Reconsolidation has subsequently been tested in a number of clinical disorders (particularly for post-traumatic stress disorder) with somewhat mixed results. A number of questions have been raised about the robustness and generality of these effects, leading one group to conclude in 2017 that “the degree to which disrupting reconsolidation is a viable clinical intervention remains questionable.”9 Thus, reconsolidation remains a potentially promising technique in need of further validation.

“I Forgot That I Was a Smoker” In Chapter 5 I discussed the ways in which brain lesions can sometimes lead to fortuitous changes in psychological function. A study by Antoine Bechara, involving patients from the Iowa Neurological Patient Registry, found another interesting side effect of brain lesions that bears directly on the potential for reducing drug cravings.10 Bechara and his colleagues examined 19 smokers who had sustained damage to the insula, discussed earlier, along with 50 smokers who had damage to other parts of the brain. The insula is particularly

186

chapter 9

involved in integrating different aspects of sensory information, including information from within our bodies (known as interoceptive sensations)—literally, our “gut feelings.” When they examined how smoking behavior changed from before to after the patients’ brain lesions, Bechara found that while very few of those with lesions outside the insula had quit smoking, about two-thirds of those with lesions in the insula had quit smoking. The report from one of the patients (referred to as “N.”) shows how this change appears to relate to the desire to smoke. Before his stroke, he had never tried to stop smoking, and he had had no intention of doing so. N. smoked his last cigarette on the evening before his stroke. When asked about his reason for quitting smoking, he stated simply, “I forgot that I was a smoker.” When asked to elaborate, he said that he did not forget the fact that he was a smoker but rather that “my body forgot the urge to smoke.” He felt no urge to smoke during his hospital stay, even though he had the opportunity to go outside to smoke. His wife was surprised by the fact that he did not want to smoke in the hospital, given the degree of his prior addiction. N. recalled how his roommate in the hospital would frequently go outside to smoke and that he was so disgusted by the smell upon his roommate’s return that he asked to change rooms. He volunteered that smoking in his dreams, which used to be pleasurable before his stroke, was now disgusting. Several other studies have replicated the findings from this initial study, showing that the effects of insula lesions on smoking are highly reliable. If lesions to the insula only caused reductions in smoking, then one might consider psychosurgery to lesions in that area to halt the addiction. However, as with all brain areas, lesions to this area can have many different negative effects, including disruptions of cardiovascular function, taste perception, and pain perception, as well as effects on mood and motivation. Thus, it seems unlikely that psychosurgery to lesions in the insula would be considered ethical given the risks. One might envision the use of a drug like ZIP to disrupt memory consolidation

h a c k i n g h a b i t s 187

for cravings in the insula, just as Sacktor did to erase taste aversion memories. However, this kind of memory disruption has never been demonstrated in humans, and the potential risks are unknown. There is in fact evidence that ZIP is toxic to some cells in the brain, suggesting that there would likely be untoward side effects that might outweigh any benefits in reducing addictive behaviors. Another potential way to manipulate the brain in a safer manner would be to use brain stimulation techniques, such as transcranial magnetic stimulation, or TMS (introduced in Box 5.3). These techniques allow one to either inhibit or enhance the activity of a particular brain region, though only relatively large areas can be targeted. TMS is currently approved for use in the treatment of depression; it provides a much more palatable alternative to electroconvulsive therapy and has been shown to be effective for some people with depression. Its side effects are relatively minimal if performed properly, mostly limited to discomfort due to stimulation of the scalp and scalp muscles. A number of studies have examined treatment with TMS for various addictive disorders, including eating disorders and drug addiction. While the results from some trials are promising, the results are not yet sufficient to support the use of brain stimulation generally for the treatment of these disorders.

Optogenetics in Humans? We have seen at various points just how powerful and precise optogenetic stimulation can be in animal models, such as the example in Chapter 6 where optogenetic stimulation of connections between the orbitofrontal cortex and the nucleus accumbens in mice was able to eliminate compulsive drug taking. The prospect of optogenetic stimulation in humans might seem futuristic, but in fact it is already being tested. As of 2019 two early-phase clinical trials are under way to examine the safety and possible effectiveness of an optogenetic treatment for retinitis pigmentosa, a disease of the retina that causes blindness. To optogenetically stimulate cells, it’s necessary to implant special ion channels that are sensitive to light into those cells. In lab animals this is

188

chapter 9

often done through genetic engineering, but in humans it must be done using a form of gene therapy in which a virus is used to insert a gene for these ion channels into the cell’s DNA. This kind of gene therapy is still in its infancy, and there are always potential unknown risks of genetic modifications. The potential risks of gene therapy have been clear since the widely publicized death of the teenager Jesse Gelsinger in one of the earliest trials of gene therapy in 1999. Gelsinger suffered from a genetic liver disease, and his parents enrolled him in a clinical trial that aimed to replace his dysfunctional gene with a new functional version. However, after injection with the modified cold virus meant to deliver the new gene to his liver, Gelsinger suffered a massive inflammatory response and died several days later due to organ failure. Research on gene therapy recovered after several years, with much greater safeguards, and has since been used successfully to treat diseases including sickle cell disease. If shown to be safe, the ability to optogenetically stimulate specific pathways in the brain could have great potential for changing behavior, though given that this would require brain surgery, the substantial risks would mean that this would only be done in the context of a very severe behavioral disorder. One particular study in rats by Anne Graybiel of MIT and her colleagues provides a glimpse of the potential for using optogenetics to break habits. They trained rats to run a maze like the one used by Mark Packard (which I described in Chapter 4), and confirmed that the rats had developed a habit, as they would continue running to a devalued reward. Using a form of optogenetic stimulation that allowed them to momentarily inhibit neurons in the infralimbic cortex, they saw that they could almost immediately render the behavior sensitive to devaluation by disrupting this area; that is, they turned it from a habit back to a goal-directed behavior. Amazingly, if the rats were allowed to develop a new habit, inactivation of the infralimbic cortex would then return them to their original habit, which shows that the stimulation didn’t erase the original habit but rather modulated the degree to which it could control behavior. One challenge to applying this kind of approach in humans is that we don’t know exactly what the human analog of the rodent infralimbic cortex is (though it probably

h a c k i n g h a b i t s 189

falls within the ventromedial prefrontal cortex), and it would almost certainly function very differently between the species. Nonetheless, this probably provides one of the most promising leads for control of severe addictive behavior in the future, though its implementation in human trials is likely years, if not decades, away.

A Neurochemical “Goldilocks Zone”: Drugs to Improve Executive Function If we want to see just how radically a medication can improve executive function, we can look at what happens when an individual with severe attention-deficit/hyperactivity disorder (ADHD) takes a stimulant medication such as methylphenidate (Ritalin). Mike Berbenes wrote about his experience as an adult with ADHD on and off of the drug:11 The presence or absence of the medication affects every single experience I have. A simple trip to IKEA can either be a productive use of a Saturday or an overwhelming gauntlet of tedium and frustration, depending on whether or not I have prescription amphetamines in my bloodstream. . . . Without my pills I am an amputee without his prosthetic. Tedium becomes torture. IKEA becomes Abu Ghraib. And then he takes his methylphenidate: Twenty minutes later, things are calm. The noise is gone. . . . The energy is still there, but it has purpose. My focus, scattered just an hour ago, has become concentrated. I always tell people that it’s like turning a floodlight into a laser. What is particularly interesting about the effects of methylphenidate is that it is a stimulant drug, closely related to amphetamines (which are also prescribed for ADHD). How can a drug that makes some people jumpy and excited make others focused and calm? Methylphenidate has its effect on executive function by increasing the amount of several neurotransmitters known as catecholamines

190 c h a p t e r 9

(including dopamine and noradrenaline) present in the prefrontal cortex. There is clear evidence that stimulant drugs like methylphenidate can improve executive function, particularly in individuals with ADHD. The evidence for cognitive effects in healthy individuals is less clear-cut, even though many people without ADHD use these drugs for their stimulant properties. A meta-analysis by Martha Farah and her colleagues from the University of Pennsylvania found that the effects of stimulant drugs on cognitive function in healthy individuals are small at best.12 The continued use of these drugs may be due to the drugs’ effects on motivation rather than cognition; for example, one study found that d-amphetamine (another popular drug for ADHD) led healthy subjects without ADHD to work harder on a task requiring rapid button presses for a small reward, compared to when they received a placebo.13 There is no research that I have found that tests whether stimulant drugs can improve behavior change. One major challenge with this idea is that stimulant drugs can themselves be addictive, so there would be a risk of trading one bad habit for another. However, particularly for individuals suffering from ADHD, the benefits of stimulant drugs for increasing executive control could have a potential impact on behavior change.

Toward Personalized Behavior Change There is a great deal of excitement in medicine regarding the potential to personalize treatments for diseases based on more precise diagnostics at the individual level, which has come to be known as precision medicine. This enthusiasm is inspired in part by the realization that for many diseases, there are large differences between how each individual responds to a particular treatment. Nowhere is this clearer than in cancer treatment, where the response of an individual to a particular treatment can vary widely depending upon the particular molecular signature of the genetic mutations in that individual’s cancer cells. Outside of cancer treatment, a growing number of pharmaceuticals are now labeled with information regarding the impact of specific

h a c k i n g h a b i t s 191

genetic variants on the effectiveness of the drug or the likelihood of particular side effects. As of June 2020, the US Food and Drug Administration listed 240 noncancer treatments and 164 cancer treatments whose effects can vary depending on the presence or absence of a particular biomarker 14 —which refers to a marker (such as a genetic variant or other biological difference) that can be easily measured and used to help determine which patients will benefit most from a particular treatment or, conversely, which patients should avoid the treatment due to potential side effects. The challenge for precision medicine in the context of behavioral change is to determine what these biomarkers are and how they relate to possible treatments. There is much enthusiasm about the use of brain imaging to identify biomarkers to help improve outcomes for the treatment of mental health disorders. One prominent example was published by Conor Liston and his colleagues in 2017, which identified potential brain biomarkers for different subtypes of depression using functional MRI.15 The researchers examined patterns of brain connectivity using resting fMRI; by examining correlations in activity between different brain regions, and then clustering patients into groups depending on their patterns of connectivity, the researchers identified four “biotypes” of depression. They further showed that people with these different biotypes differed in their response to treatment with transcranial magnetic stimulation. Some questions have been raised about the reproducibility of the specific biotypes in this study,16 but nonetheless this kind of finding shows the promise of brain imaging for developing biomarkers for various conditions that are related to brain function. Behavioral measurements, such as performance on cognitive tasks, are much less flashy than brain imaging data, but may provide a much lower-cost biomarker. For example, in our research we have found that the combination of different behavioral measures is useful for predicting a number of outcomes relevant to behavior change, such as problem drinking, smoking, and obesity.17 We have not yet determined whether these are useful for predicting the response to various interventions, but this is an obvious next step. The measurement of behavior using mobile devices is of particular interest,

192

chapter 9

as it may allow the noninvasive and low-cost measurement of various “behavioral biomarkers.” One other idea that has gained attention in the context of precision medicine is the N-of-1 clinical trial (where “N” refers to the number of participants in the trial).18 In medicine, the gold standard to determine whether a treatment works has been the randomized controlled trial, in which study participants are randomly assigned to receive either the treatment of interest or a control treatment. This random assignment of participants to treatment groups helps minimize any biases that might occur if patients or their physicians were to choose the treatment. A problem with such a clinical trial is that it assumes that everyone will respond in the same way to the treatment, which we increasingly know to be false for many treatments. An alternative approach is to test a range of treatments for a specific individual, which is the idea behind the N-of-1 clinical trial. As an example of how this might work, take an individual who is trying to reduce their alcohol consumption, whose physician thinks that two medical treatments might be effective: naltrexone (which reduces the rewarding effects of alcohol) or guanfacine (which is thought to improve executive function). To perform an N-of-1 trial, the physician would prescribe one of these drugs for a period of time (say, two months), during which time the patient would record their alcohol use. After that time elapsed, the physician would switch the patient to the other medication; they might also alternate between these drugs several times, perhaps including a period when the patient is given a placebo treatment. Over time it should become clear whether either of these treatments is particularly effective for that patient. Given the many different underlying mechanisms for the difficulty of behavior change, it is likely that such an approach could be effective in the context of behavior change. A small number of such trials have been published, though their quality has been criticized,19 so we must await larger and better-controlled studies before we can determine whether this approach will ultimately be useful. Let’s imagine what the precision behavior change therapy of the future might look like, using the example of our individual who wishes to drink less. Their first visit to their physician would involve a set of

h a c k i n g h a b i t s 193

cognitive tests, a brain scan using functional MRI to measure brain connectivity, and a blood draw to analyze their genome. A genetic analysis would identify the individual’s risk for various problems, including their sensitivity to alcohol reward and their likelihood of executive control difficulties, and the cognitive testing would be tailored to specifically measure those particular aspects of their function. The brain imaging analysis would be combined with those results in order to determine which brain mechanisms are most likely underlying the individual’s drinking problem. Using this information, the physician would select the set of possible treatments that are most likely to benefit that individual and undertake an N-of-1 trial to test those various treatments, either individually or in combination. It is this kind of approach, driven by an understanding of the underlying mechanisms of behavior change problems, that I hope the neuroscience research outlined throughout this book can ultimately provide.

10 Epilogue

we generally assume that our world will be largely the same from day to day, and as I described in Chapter 1, this is a basic fact that is thought to have given rise to the evolution of the habit system. However, sometimes the world can change very quickly, as it did in early 2020 as the SARS-CoV-2 virus quickly spread across the globe, causing the COVID-19 pandemic. In an attempt to prevent an outbreak that could overwhelm medical facilities (as it did in Italy), localities in the United States began to issue orders that resulted in a massive disruption of nearly everyone’s daily routines. When the counties in the San Francisco Bay Area issued a shelter-in-place order on March 16, I went from a daily commute involving cars, trains, and bicycles to spending most of my day in a makeshift home office, and from regular personal interactions with my colleagues and students to days filled with online meetings and lectures. In those early fearful days, as we watched the horror of overflowing emergency rooms in Italy, we also became much more attuned to simple habits like touching one’s face, which became high on the list of public health enemies in the attempt to stop the spread of the disease. We also learned various strategies to help ensure that we washed our hands for the 20 seconds required to eliminate the virus, such as singing “Happy Birthday” twice. In some ways, the COVID-19 pandemic has cemented many of the ideas that I have presented in this book. Behavior change is hard, even when we are highly motivated: despite my strong intention to avoid 194

e p i l o g u e 195

touching my face, I would often find myself doing it anyway, and I have also found over time that my hand-washing habits are creeping back toward the short wash that was standard before the pandemic. It also highlighted just how powerful the environment can be in driving changes in our habits. Back when I commuted to campus, I would often find myself at the campus café in the mid-afternoon, indulging in a chocolate chip cookie and coffee, despite my strongly held goal to avoid eating sugary foods. Once the COVID-19 lockdown started in San Francisco, I could no longer easily walk to a café and grab a treat, and as I write this paragraph roughly three months into the lockdown, I have not eaten any sugary afternoon treats since it started. On the other hand, the pandemic also highlighted how quickly humans can adapt to new situations. Within a few weeks of the lockdown, those of us lucky enough to have jobs that allowed remote work had settled into our new “socially distanced” lifestyle, and it all became shockingly normal. It remains to be seen whether the changes that we made in response to the pandemic will stay with us once we go back to “normal” life, now that several highly effective vaccines have become available. Will we go back to handshakes and hugs and touching our faces? Only time will tell, but I predict that the stickiness of these habits will win out.

Summing Up Evolution provided us with a brain that functioned very well in the Paleolithic world, but the modern world has uncovered a few bugs in its design. Just as cybercriminals can take advantage of computer bugs to hack into computer systems, the drug dealers, food engineers, and technology designers of the modern day have all identified ways to take advantage of the brain’s vulnerabilities. It’s unlikely that these genies can ever be put back into the bottle, which means that behavior change will remain an enduring struggle for humans. However, from my standpoint, the future is bright for the science of behavior change, for a number of reasons.

196

c h a p t e r 10

First, research is increasingly focused on understanding the underlying biological and psychological mechanisms of behavior change. With knowledge of these mechanisms, we can begin to develop studies that can help us understand not just whether a particular treatment works but how it works. We still don’t have the kind of sophisticated understanding of behavior change that we do for many diseases, but neuroscience and psychology are providing increasing clarity on the mechanisms that underlie behavior change. Many of these have been uncovered through the astonishing advances in neuroscientific technology that have occurred in the last two decades, providing us detailed knowledge of the particular biological mechanisms for habits and selfcontrol. But we have also gained insights from purely behavioral studies, such as the work that has provided a new understanding of the role of willpower in habits and behavior change. Second, we are gaining a better handle on how to do reproducible and generalizable research on these questions. At various points in the book, I have outlined ways in which we have discovered that previous scientific practices led to conclusions that were not reliable, such as the use of small sample sizes or the focus on effects of individual genes on behavior. In the last decade, a “reproducibility movement” has burgeoned within science that has developed a number of strategies to address these problems head-on, and these new approaches are increasingly becoming the norm. In this sense, science as a whole has shown its ability to change its behavior relatively quickly (at least compared to other aspects of society), even while individual scientists may struggle with the difficulties of adopting new research practices and ways of thinking. One major improvement is the increasing realization of the need for much larger sample sizes in order to obtain reproducible results. We see this particularly in the growing prevalence of megastudies, such as the StepUp project to understand behavioral nudges and the genomewide association studies that have provided new knowledge of the genetic basis of various aspects of behavior. These large studies can provide much more reliable answers to scientific questions, but this improvement comes at a cost: such studies are so expensive to perform

e p i l o g u e 197

that it simply will not be possible to collect them to address every scientific question of interest. Fortunately, other developments can help ensure the reproducibility of smaller studies as well. One important new practice is known as preregistration, in which researchers outline their research methods and their predictions prior to performing the study and place them in the hands of a third party until the study is completed. Many nonscientists might think that this is naturally how science should work, but in practice researchers often have substantial flexibility in how they analyze their data. In particular, if their planned analyses don’t show the predicted outcome, researchers in the past would often analyze the data in different ways until they found a result that supported their prediction, and the result was the publication of many false positive studies. When the National Heart, Lung, and Blood Institute began requiring clinical trials to register their predicted outcomes in 2000, the effect was striking:1 before this requirement, more than half of all clinical trials reported a positive outcome, while after the requirement less than 10% of all trials did so. It seems that prior to the requirement for researchers to specify exactly how they would define success, they were “moving the goalposts” in order to find some way to show that their treatment was successful. This kind of preregistration is increasingly prevalent within psychology, which has struggled badly in the last decade with a number of prominent failures to replicate previously published findings.2 It has not yet become prevalent in neuroscience research, but the movement is beginning to take hold there as well. I have been particularly engaged in the development of what has come to be called “open science,” which I think has the potential to greatly improve the quality and impact of scientific research. The goal behind the open science movement is to develop a scientific culture focused on transparency and reproducibility, rather than a competitive culture in which researchers battle to be the first to publish a new discovery, often at the cost of research quality. One important aspect of open science is the sharing of research materials, including data and the computer software that is used to analyze them. Our research has shown that, even when given the same data, researchers can come up

198

c h a p t e r 10

with very different answers,3 so it is very important that both data and software be shared so that other researchers can determine how generalizable the findings are. Another important aspect of the open science movement is the push for open access to research publications. Most research is funded by taxpayer funds, yet many research papers are published in journals that charge for access to the publications, limiting the public’s access to the research that they have paid for. The movement toward open access has been particularly successful—all research funded by the US National Institutes of Health must now be made publicly accessible through the Pubmed Central database. In addition, researchers increasingly post their papers on open-access “preprint” websites, which allow anyone to access the research. These preprint sites became highly visible during the COVID-19 pandemic, as thousands of research papers were posted to the sites in the first few months of the outbreak. The challenge with these sites is that the research has not been peer reviewed, meaning that it may suffer from flaws in its methods or interpretation. However, the public availability of this work has allowed research to move very quickly, providing insights into the virus and disease at a much faster rate than in any previous public health emergency.

From Individual to Societal Change I have focused on individual behavior change in this book, but the human species also faces an existential crisis that will likely require major changes in both individual and collective behavior in order to prevent wide-scale disruptions to human lives across the globe. I refer of course to the climate crisis that has resulted from the carbonintensive lifestyles of industrialized societies. The massive wildfires in Australia of early 2020, deadly European heat waves of 2019, and Hurricane Harvey in 2017 are just the latest in a growing number of signals that this crisis will soon have major impacts on the lives of nearly every human. The scale of the behavioral change necessary to solve the climate problem is almost unfathomable, in part because of its worldwide scale.

e p i l o g u e 199

Avoiding the “tipping points” beyond which climate change may be uncontrollable will require major and rapid changes in nearly every aspect of society, far beyond the breadth and speed of the changes that are currently agreed upon by most of the world’s largest nations.4 Many individuals are already making changes in their personal lives in order to address their impact on the planet. Personally, I have nearly eliminated my professional air travel (down from more than 100,000 miles per year), given the major carbon impacts of air travel, and many of my scientific colleagues are doing the same.5 However, it is clear that preventing a climate apocalypse will require societal behavior change in addition to individual change. Although in this book I have focused on individual behavior change, psychological science also provides insights into the factors that underlie support for societal changes that require individual sacrifice.6 I hope that the science of behavior change, applied to these societal problems, can also help us understand how to best move the behavior of human societies toward a more sustainable path.

notes

Chapter 1 1. W James. The Principles of Psychology. Volume 1. New York: Henry Holt and Co., 1890. 2. Icek Ajzen and Arie W Kruglanski. “Reasoned action in the service of goal pursuit.” In: Psychol Rev 126.5 (Oct. 2019), pp. 774–86. doi: 10.1037/rev0000155. 3. Judith A Ouellette and Wendy Wood. “Habit and intention in everyday life: The multiple processes by which past behavior predicts future behavior.” In: Psychological Bulletin (1998), pp. 54–74. 4. A Dickinson. “Actions and habits: The development of behavioural autonomy.” In: Philosophical Transactions of the Royal Society of London, Series B, Biological Sciences 308.1135 (1985), pp. 67–78. http://www.jstor.org/stable/2396284.

Chapter 2 1. S Zola-Morgan, L R Squire, and D G Amaral. “Human amnesia and the medial temporal region: Enduring memory impairment following a bilateral lesion limited to field CA1 of the hippocampus.” In: J Neurosci 6.10 (1986), pp. 2950–67. 2. N J Cohen and L R Squire. “Preserved learning and retention of pattern-analyzing skill in amnesia: Dissociation of knowing how and knowing that.” In: Science 210.4466 (1980), pp. 207–10. 3. L R Squire et al. “Description of brain injury in the amnesic patient N. A. based on magnetic resonance imaging.” In: Exp Neurol 105.1 (1989), pp. 23–35. 4. P D MacLean. The Triune Brain in Evolution: Role in Paleocerebral Functions. New York: Plenum, 1990. 5. Angela Rizk-Jackson et al. “Evaluating imaging biomarkers for neurodegeneration in presymptomatic Huntington’s disease using machine learning techniques.” In: Neuroimage 56.2 (2011), pp. 788–96. doi: 10.1016/j.neuroimage.2010.04.273. 6. E D Caine et al. “Huntington’s dementia. Clinical and neuropsychological features.” In: Arch. Gen. Psychiatry 35.3 (Mar. 1978), pp. 377–84. 7. M Martone et al. “Dissociations between skill learning and verbal recognition in amnesia and dementia.” In: Arch Neurol 41.9 (1984), pp. 965–70. 8. R G Northcutt and J H Kaas. “The emergence and evolution of mammalian neocortex.” In: Trends Neurosci 18.9 (1995), pp. 373–79.

201

202

notes to chapter 3

9. J W Mink. “The basal ganglia: Focused selection and inhibition of competing motor programs.” In: Prog Neurobiol 50.4 (1996), pp. 381–425. 10. Oscar Arias-Carríon et al. “Dopaminergic reward system: A short integrative review.” In: Int Arch Med 3 (2010), p. 24. doi: 10.1186/1755-7682-3-24. 11. Shankar J Chinta and Julie K Andersen. “Dopaminergic neurons.” In: Int J Biochem Cell Biol 37.5 (2005), pp. 942–46. doi: 10.1016/j.biocel.2004.09.009. 12. Alexxai V Kravitz et al. “Regulation of parkinsonian motor behaviours by optogenetic control of basal ganglia circuitry.” In: Nature 466.7306 (2010), pp. 622–26. doi: 10.1038 /nature09159. 13. I should note that despite the fact that it is one of the most widely accepted ideas in neuroscience, the idea that synaptic plasticity is the primary mechanism for learning and memory has come under heavy fire in the last few years. In particular, more recent work has begun to suggest that important aspects of memory may reside in changes within the neuron related to how genes are expressed, which we further discuss in Chapter 6. See Wickliffe C Abraham, Owen D Jones, and David L Glanzman. “Is plasticity of synapses the mechanism of long-term memory storage?” In: NPJ Sci Learn 4 (2019), p. 9. doi: 10.1038/s41539-019-0048-y for an outstanding overview of these recent ideas. 14. W Schultz, P Dayan, and P R Montague. “A neural substrate of prediction and reward.” In: Science 275.5306 (1997), pp. 1593–99. doi: 10.1126/science.275.5306.1593. 15. P Redgrave, T J Prescott, and K Gurney. “The basal ganglia: A vertebrate solution to the selection problem?” In: Neuroscience 89.4 (1999), pp. 1009–23. doi: 10.1016/s03064522(98)00319-4. 16. Xin Jin, Fatuel Tecuapetla, and Rui M Costa. “Basal ganglia subcircuits distinctively encode the parsing and concatenation of action sequences.” In: Nat Neurosci 17.3 (2014), pp. 423–30. doi: 10.1038/nn.3632.

Chapter 3 1. J L Mystkowski, M G Craske, and A M Echiverri. “Treatment context and return of fear in spider phobia.” In: Behavior Therapy 33 (2002), pp. 399–416. 2. Henry H Yin and Barbara J Knowlton. “The role of the basal ganglia in habit formation.” In: Nat. Rev. Neurosci. 7.6 ( June 2006), pp. 464–76. doi: 10.1038/nrn1919. 3. Kyle S Smith and AnnMGraybiel. “A dual operator view of habitual behavior reflecting cortical and striatal dynamics.” In: Neuron 79.2 (2013), pp. 361–74. doi: 10.1016/j.neuron.2013 .05.038. 4. Peter C Holland. “Relations between Pavlovian-instrumental transfer and reinforcer devaluation.” In: J Exp Psychol Anim Behav Process 30.2 (2004), pp. 104–17. doi: 10.1037 /0097-7403.30.2.104. 5. P Watson et al. “Working for food you don’t desire. Cues interfere with goal-directed food-seeking.” In: Appetite 79 (2014), pp. 139–48. doi: 10.1016/j.appet.2014.04.005. 6. Kate M Wassum et al. “Phasic mesolimbic dopamine release tracks reward seeking during expression of Pavlovian-to-instrumental transfer.” In: Biol Psychiatry 73.8 (2013), pp. 747–55. doi: 10.1016/j.biopsych.2012.12.005.

n o t e s t o c h a p t e r 5 203 7. Briac Halbout et al. “Mesolimbic dopamine projections mediate cue-motivated reward seeking but not reward retrieval in rats.” In: Elife 8 (2019). doi: 10.7554/eLife.43551. 8. Matt Field and W Miles Cox. “Attentional bias in addictive behaviors: A review of its development, causes, and consequences.” In: Drug Alcohol Depend 97.1–2 (2008), pp. 1–20. doi: 10.1016/j.drugalcdep.2008.03.030.

Chapter 4 1. R A Poldrack et al. “Interactive memory systems in the human brain.” In: Nature 414.6863 (2001), pp. 546–50. doi: 10.1038/35107080. 2. Richard S Sutton and Andrew G Barto. Reinforcement Learning: An Introduction. Second edition. Adaptive Computation and Machine Learning Series. Cambridge, MA: Bradford Books, 2018. 3. A computational notebook with a working implementation of this model is available at https://github.com/poldrack/reinforcement_learning_example/blob/master/RLexample .ipynb. 4. Hannah M Bayer and Paul W Glimcher. “Midbrain dopamine neurons encode a quantitative reward prediction error signal.” In: Neuron 47.1 (2005), pp. 129–41. doi: 10.1016 /j.neuron.2005.05.020. 5. Ian W Eisenberg et al. “Uncovering the structure of self-regulation through data-driven ontology discovery.” In: Nat Commun 10.1 (2019), p. 2319. doi: 10.1038/s41467-019-10301-1. 6. https://www.reddit.com/r/ems/comments/2auj17/drug_seeker_stories/. 7. Fiery Cushman and Adam Morris. “Habitual control of goal selection in humans.” In: Proc Natl Acad Sci USA 112.45 (2015), pp. 13817–22. doi: 10.1073/pnas.1506367112.

Chapter 5 1. John Darrell Van Horn et al. “Mapping connectivity damage in the case of Phineas Gage.” In: PLoS One 7.5 (2012), e37454. doi: 10.1371/journal.pone.0037454. 2. John M Harlow. “Recovery from the passage of an iron bar through the head.” In: Publications of the Massachusetts Medical Society 2.3 (1868). 3. Malcolm Macmillan. An Odd Kind of Fame: Stories of Phineas Gage. Cambridge, MA: MIT Press, 2000. 4. Joseph Barrash et al. “‘Frontal lobe syndrome’? Subtypes of acquired personality disturbances in patients with focal brain damage.” In: Cortex 106 (2018), pp. 65–80. doi: 10.1016/j.cortex.2018.05.007. 5. Marcie L King et al. “Neural correlates of improvements in personality and behavior following a neurological event.” In: Neuropsychologia (2017). doi: 10.1016/j.neuropsychologia. 2017.11.023. 6. M M Mesulam. “From sensation to cognition.” In: Brain 121 ( Pt 6) (1998), pp. 1013–52. 7. Kate Teffer and Katerina Semendeferi. “Human prefrontal cortex: Evolution, development, and pathology.” In: Prog Brain Res 195 (2012), pp. 191–218. doi: 10.1016/B978-0-444 53860 -4.00009-X.

204

notes to chapter 5

8. P Kochunov et al. “Fractional anisotropy of cerebral white matter and thickness of cortical gray matter across the lifespan.” In: Neuroimage 58.1 (2011), pp. 41–49. doi: 10.1016 /j.neuroimage.2011.05.050. 9. T Sawaguchi and P S Goldman-Rakic. “D1 dopamine receptors in prefrontal cortex: Involvement in working memory.” In: Science 251.4996 (1991), pp. 947–50. 10. G V Williams and P S Goldman-Rakic. “Modulation of memory fields by dopamine D1 receptors in prefrontal cortex.” In: Nature 376.6541 (1995), pp. 572–75. doi: 10.1038/376572a0. 11. Earl K Miller, Mikael Lundqvist, and André M Bastos. “Working Memory 2.0.” In: Neuron 100.2 (Oct. 2018), pp. 463–75. doi: 10.1016/j.neuron.2018.09.023. 12. I should note that there is evidence that doing this can be counterproductive, and it was only when I started giving talks without the drug that I was able to finally conquer my anxiety about public speaking. 13. A F Arnsten and P S Goldman-Rakic. “Alpha 2-adrenergic mechanisms in prefrontal cortex associated with cognitive decline in aged nonhuman primates.” In: Science 230.4731 (1985), pp. 1273–76. 14. Amy F T Arnsten, Min J Wang, and Constantinos D Paspalas. “Neuromodulation of thought: Flexibilities and vulnerabilities in prefrontal cortical network synapses.” In: Neuron 76.1 (2012), pp. 223–39. doi: 10.1016/j.neuron.2012.08.038. 15. Harris R Lieberman et al. “The fog of war: Decrements in cognitive performance and mood associated with combat-like stress.” In: Aviat Space Environ Med 76.7 Suppl (2005), pp. C7–C14. 16. Min Wang et al. “Alpha2A-adrenoceptors strengthen working memory networks by inhibiting cAMP-HCN channel signaling in prefrontal cortex.” In: Cell 129.2 (2007), pp. 397– 410. doi: 10.1016/j.cell.2007.03.015. 17. Angela L Duckworth, Eli Tsukayama, and Teri A Kirby. “Is it really self-control? Examining the predictive power of the delay of gratification task.” In: Pers Soc Psychol Bull 39.7 (2013), pp. 843–55. doi: 10.1177/0146167213482589. 18. Celeste Kidd, Holly Palmeri, and Richard N Aslin. “Rational snacking: Young children’s decision-making on the marshmallow task is moderated by beliefs about environmental reliability.” In: Cognition 126.1 (2013), pp. 109–14. doi: 10.1016/j.cognition.2012.08.004. 19. Anuj K Shah, Sendhil Mullainathan, and Eldar Shafir. “Some consequences of having too little.” In: Science 338.6107 (2012), pp. 682–85. doi: 10.1126/science.1222426. 20. L Green et al. “Temporal discounting in choice between delayed rewards: The role of age and income.” In: Psychol Aging 11.1 (1996), pp. 79–84. 21. Andrey P Anokhin et al. “The genetics of impulsivity: Evidence for the heritability of delay discounting.” In: Biol Psychiatry 77.10 (2015), pp. 887–94. doi: 10.1016/j.biopsych.2014 .10.022. 22. James MacKillop et al. “Delayed reward discounting and addictive behavior: A metaanalysis.” In: Psychopharmacology 216.3 (2011), pp. 305–21. doi: 10.1007/s00213-0112229-0. 23. Janet Audrain-McGovern et al. “Does delay discounting play an etiological role in smoking or is it a consequence of smoking?” In: Drug Alcohol Depend 103.3 (2009), pp. 99–106. doi: 10.1016/j.drugalcdep.2008.12.019.

n o t e s t o c h a p t e r 5 205 24. Samuel M McClure and Warren K Bickel. “A dual-systems perspective on addiction: Contributions from neuroimaging and cognitive training.” In: Ann NY Acad Sci 1327 (2014), pp. 62–78. doi: 10.1111/nyas.12561. 25. https://www.youtube.com/watch?v=QX_oy9614HQ . 26. Richard H Thaler and H M Shefrin. “An economic theory of self-control.” In: Journal of Political Economy 89.2 (1981), pp. 392–406. http://www.jstor.org/stable/1833317. 27. Samuel M McClure et al. “Separate neural systems value immediate and delayed monetary rewards.” In: Science 306.5695 (2004), pp. 503–7. doi: 10.1126/science.1100907. 28. S Whiteside and D Lynam. “The five factor model and impulsivity: Using a structural model of personality to understand impulsivity.” In: Personality and Individual Differences 30.4 (2001), pp. 669–89. 29. Sandra Sanchez-Roige et al. “Genome-wide association studies of impulsive personality traits (BIS-11 and UPPS-P) and drug experimentation in up to 22,861 adult research participants identify loci in the CACNA1I and CADM2 genes.” In: J Neurosci 39.13 (2019), pp. 2562–72. doi: 10.1523/JNEUROSCI.2662-18.2019. 30. John P A Ioannidis. “Why most published research findings are false.” In: PLoS Med 2.8 (2005), e124. doi: 10.1371/journal.pmed.0020124. 31. Benjamin J Shannon et al. “Premotor functional connectivity predicts impulsivity in juvenile offenders.” In: Proc Natl Acad Sci USA 108.27 (2011), pp. 11241–45. doi: 10.1073/pnas .1108241108. 32. Johannes Golchert et al. “In need of constraint: Understanding the role of the cingulate cortex in the impulsive mind.” In: Neuroimage 146 (Feb. 2017), pp. 804–13. doi: 10.1016/j .neuroimage.2016.10.041. 33. https://www.leefromamerica.com/blog/bingehistory. 34. Adam R Aron et al. “Stop-signal inhibition disrupted by damage to right inferior frontal gyrus in humans.” In: Nat Neurosci 6.2 (2003), pp. 115–16. doi: 10.1038/nn1003. 35. Adam R Aron and Russell A Poldrack. “Cortical and subcortical contributions to stop signal response inhibition: Role of the subthalamic nucleus.” In: J Neurosci 26.9 (2006), pp. 2424–33. doi: 10.1523/JNEUROSCI.4682-05.2006. 36. For a deeper introduction to fMRI and neuroimaging in general, see my previous book The New Mind Readers. 37. Atsushi Nambu, Hironobu Tokuno, and Masahiko Takada. “Functional significance of the cortico-subthalamo-pallidal ‘hyperdirect’ pathway.” In: Neurosci Res 43.2 (2002), pp. 111–17. doi: 10.1016/s0168-0102(02)00027-5. 38. Adam R Aron et al. “Triangulating a cognitive control network using diffusion-weighted magnetic resonance imaging (MRI) and functional MRI.” In: J Neurosci 27.14 (2007), pp. 3743– 52. doi: 10.1523/JNEUROSCI.0519-07.2007. 39. Robert Schmidt et al. “Canceling actions involves a race between basal ganglia pathways.” In: Nat Neurosci 16.8 (2013), pp. 1118–24. doi: 10.1038/nn.3456. 40. Ian W Eisenberg et al. “Uncovering the structure of self-regulation through datadriven ontology discovery.” In: Nat Commun 10.1 (2019), p. 2319. doi: 10.1038/s41467-019 -10301-1.

206

notes to chapter 6

41. Wilhelm Hofmann et al. “Everyday temptations: An experience sampling study of desire, conflict, and self-control.” In: J Pers Soc Psychol 102.6 (2012), pp. 1318–35. doi: 10.1037 /a0026545. 42. Brian M Galla and Angela L Duckworth. “More than resisting temptation: Beneficial habits mediate the relationship between self-control and positive life outcomes.” In: J Pers Soc Psychol 109.3 (2015), pp. 508–25. doi: 10.1037/pspp0000026.

Chapter 6 1. The term addiction has been superseded in clinical psychiatry by the term substance use disorder, but I will use addiction here since it is more commonly understood. 2. https://www.vice.com/en_us/article/kwxkbv/ex-users-describe-the-first-time-they-tried -heroin. 3. J Olds. “Self-stimulation of the brain; its use to study local effects of hunger, sex, and drugs.” In: Science 127.3294 (1958), pp. 315–24. doi: 10.1126/science.127.3294.315. 4. https://www.youtube.com/watch?v=GOnENVylxPI. 5. Christian Lüscher. “The emergence of a circuit model for addiction.” In: Annu Rev Neurosci 39 ( July 2016), pp. 257–76. doi: 10.1146/annurev-neuro-070815-013920. 6. Yan Dong and Eric J Nestler. “The neural rejuvenation hypothesis of cocaine addiction.” In: Trends Pharmacol Sci 35.8 (2014), pp. 374–83. doi: 10.1016/j.tips.2014.05.005. 7. N D Volkow et al. “Decreased striatal dopaminergic responsiveness in detoxified cocainedependent subjects.” In: Nature 386.6627 (1997), pp. 830–33. doi: 10.1038/386830a0. 8. B J Everitt, A Dickinson, and T W Robbins. “The neuropsychological basis of addictive behaviour.” In: Brain Res Rev 36.2–3 (2001), pp. 129–38. doi: 10.1016/s0165-0173(01)00088-1. 9. Jeffrey W Dalley et al. “Nucleus accumbens D2/3 receptors predict trait impulsivity and cocaine reinforcement.” In: Science 315.5816 (2007), pp. 1267–70. doi: 10.1126/science.113 7073. 10. Buyean Lee et al. “Striatal dopamine d2/d3 receptor availability is reduced in methamphetamine dependence and is linked to impulsivity.” In: J Neurosci 29.47 (2009), pp. 14734–40. doi: 10.1523/JNEUROSCI.3765-09.2009. 11. Sietse Jonkman, Yann Pelloux, and Barry J Everitt. “Differential roles of the dorsolateral and midlateral striatum in punished cocaine seeking.” In: J Neurosci 32.13 (2012), pp. 4645–50. doi: 10.1523/JNEUROSCI.0348-12.2012. 12. David Belin and Barry J Everitt. “Cocaine seeking habits depend upon dopaminedependent serial connectivity linking the ventral with the dorsal striatum.” In: Neuron 57.3 (2008), pp. 432–41. doi: 10.1016/j.neuron.2007.12.019. 13. Billy T Chen et al. “Rescuing cocaine-induced prefrontal cortex hypoactivity prevents compulsive cocaine seeking.” In: Nature 496.7445 (2013), pp. 359–62. doi: 10.1038/nature12024. 14. Youna Vandaele and Patricia H Janak. “Defining the place of habit in substance use disorders.” In: Prog Neuropsychopharmacol Biol Psychiatry 87.Pt A (Dec. 2018), pp. 22–32. doi: 10.1016 /j.pnpbp.2017.06.029. 15. Claire M Gillan et al. “Characterizing a psychiatric symptom dimension related to deficits in goal-directed control.” In: Elife 5 (2016). doi: 10.7554/eLife.11305.

n o t e s t o c h a p t e r 6 207 16. https://khn.org/news/what-dope-sick-really-feels-like/. 17. Susana Peciña, Jay Schulkin, and Kent C Berridge. “Nucleus accumbens corticotropin-

releasing factor increases cue-triggered motivation for sucrose reward: Paradoxical positive incentive effects in stress?” In: BMC Biol 4 (2006), p. 8. doi: 10.1186/1741-7007-4-8. 18. Anke Snoek, Neil Levy, and Jeanette Kennett. “Strong-willed but not successful: The importance of strategies in recovery from addiction.” In: Addict Behav Rep 4 (2016), pp. 102–7. doi: 10.1016/j.abrep.2016.09.002. 19. Bryan F Singer et al. “Are cocaine-seeking ‘habits’ necessary for the development of addiction-like behavior in rats?” In: J Neurosci 38.1 ( Jan. 2018), pp. 60–73. doi: 10.1523 /JNEUROSCI.2458-17.2017. 20. Lee Hogarth. “Addiction is driven by excessive goal-directed drug choice under negative affect: Translational critique of habit and compulsion theory.” In: Neuropsychopharmacology 45.5 (2020), pp. 720–35. doi: 10.1038/s41386-020-0600-8. 21. Lauren Eales, Arthur J Reynolds, and Suh-Ruu Ou. “Childhood predictors of adult obesity in the Chicago Longitudinal Study.” In: Prev Med 132 (Mar. 2020), p. 105993. doi: 10.1016 /j.ypmed.2020.105993. 22. Ian W Eisenberg et al. “Uncovering the structure of self-regulation through datadriven ontology discovery.” In: Nat Commun 10.1 (2019), p. 2319. doi: 10.1038/s41467-019 -10301-1. 23. Brenda L Rooney, Michelle A Mathiason, and Charles W Schauberger. “Predictors of obesity in childhood, adolescence, and adulthood in a birth cohort.” In: Matern Child Health J 15.8 (2011), pp. 1166–75. doi: 10.1007/s10995-010-0689-1. 24. Eurídice Martínez Steele et al. “Ultra-processed foods and added sugars in the US diet: Evidence from a nationally representative cross-sectional study.” In: BMJ Open 6.3 (2016), e009892. doi: 10.1136/bmjopen-2015-009892. 25. Marco Cirilli, Daniele Bassi, and Angelo Ciacciulli. “Sugars in peach fruit: A breeding perspective.” In: Hortic Res 3 (2016), p. 15067. doi: 10.1038/hortres.2015.67. 26. Kevin D Hall et al. “Ultra-processed diets cause excess calorie intake and weight gain: An inpatient randomized controlled trial of ad libitum food intake.” In: Cell Metab 30.1 (2019), p. 226. doi: 10.1016/j.cmet.2019.05.020. 27. Paul M Johnson and Paul J Kenny. “Dopamine D2 receptors in addiction-like reward dysfunction and compulsive eating in obese rats.” In: Nat Neurosci 13.5 (2010), pp. 635–41. doi: 10.1038/nn.2519. 28. Nicole M Avena, Pedro Rada, and Bartley G Hoebel. “Evidence for sugar addiction: Behavioral and neurochemical effects of intermittent, excessive sugar intake.” In: Neurosci Biobehav Rev 32.1 (2008), pp. 20–39. doi: 10.1016/j.neubiorev.2007.04.019. 29. Margaret L Westwater, Paul C Fletcher, and Hisham Ziauddeen. “Sugar addiction: The state of the science.” In: Eur J Nutr 55 Suppl 2 (2016), pp. 55–69. doi: 10.1007/s00394-016 -1229-6. 30. E M Bowman, T G Aigner, and B J Richmond. “Neural signals in the monkey ventral striatum related to motivation for juice and cocaine rewards.” In: J Neurophysiol 75.3 (1996), pp. 1061–73. doi: 10.1152/jn.1996.75.3.1061. 31. Paul C Fletcher and Paul J Kenny. “Food addiction: A valid concept?” In: Neuropsychopharmacology 43.13 (Dec. 2018), pp. 2506–13. doi: 10.1038/s41386-018-0203-9.

208

notes to chapter 7

32. G J Wang et al. “Brain dopamine and obesity.” In: Lancet 357.9253 (2001), pp. 354–57. doi: 10.1016/s0140-6736(00)03643-6. 33. Linh C Dang et al. “Associations between dopamine D2 receptor availability and BMI depend on age.” In: Neuroimage 138 (2016), pp. 176–83. doi: 10.1016/j.neuroimage.2016 .05.044. 34. Uku Vainik, Isabel García-García, and Alain Dagher. “Uncontrolled eating: A unifying heritable trait linked with obesity, overeating, personality and the brain.” In: Eur J Neurosci 50.3 (Aug. 2019), pp. 2430–45. doi: 10.1111/ejn.14352. 35. Uku Vainik et al. “Obesity has limited behavioural overlap with addiction and psychiatric phenotypes.” In: Nat Hum Behav 4.1 ( Jan. 2020), pp. 27–35. doi: 10.1038/s41562 -019-0752-x. 36. Nico Bunzeck and Emrah Düzel. “Absolute coding of stimulus novelty in the human substantia nigra/VTA.” In: Neuron 51.3 (2006), pp. 369–79. doi: 10.1016/j.neuron.2006 .06.021. 37. Daniel Kardefelt-Winther et al. “How can we conceptualize behavioural addiction without pathologizing common behaviours?” In: Addiction 112.10 (2017), pp. 1709–15. doi: 10.1111/add.13763. 38. https://www.tylervigen.com/spurious-correlations. 39. Amy Orben and Andrew K Przybylski. “The association between adolescent well-being and digital technology use.” In: Nat Hum Behav 3.2 (2019), pp. 173–82. doi: 10.1038/s41562-018 -0506-1. 40. James David Jentsch and Zachary T Pennington. “Reward, interrupted: Inhibitory control and its relevance to addictions.” In: Neuropharmacology 76 Pt B (2014), pp. 479–86. doi: 10.1016/j.neuropharm.2013.05.022. 41. David Belin et al. “High impulsivity predicts the switch to compulsive cocaine-taking.” In: Science 320.5881 (2008), pp. 1352–55. doi: 10.1126/science.1158136. 42. Vincent Pascoli et al. “Stochastic synaptic plasticity underlying compulsion in a model of addiction.” In: Nature 564.7736 (Dec. 2018), pp. 366–71. doi: 10.1038/s41586-018-0789-4. 43. Kyle Honegger and Benjamin de Bivort. “Stochasticity, individuality and behavior.” In: Curr Biol 28.1 ( Jan. 2018), R8–R12. doi: 10.1016/j.cub.2017.11.058. 44. Dean G Kilpatrick et al. “Violence and risk of PTSD, major depression, substance abuse/dependence, and comorbidity: Results from the National Survey of Adolescents.” In: J Consult Clin Psychol 71.4 (2003), pp. 692–700. doi: 10.1037/0022-006x.71.4.692.

Chapter 7 1. Data for 1971 (left panel of Figure 7.1) comes from the following source: W A Hunt, L W Barnett, and L G Branch. “Relapse rates in addiction programs.” In: J Clin Psychol 27.4 (1971), pp. 455–56. doi: 10.1002/1097-4679(197110)27:43.0.co;2-r. Data for 2011 (right panel of Figure 7.1) comes from the following source: Rajita Sinha. “New findings on biological factors predicting addiction relapse vulnerability.” In: Curr Psychiatry Rep 13.5 (2011), pp. 398–405. doi: 10.1007/s11920-011-0224-0. 2. W Mischel. “The toothbrush problem.” In: APS Observer 21 (2008), p. 11.

n o t e s t o c h a p t e r 8 209 3. Nikolaos Mastellos et al. “Transtheoretical model stages of change for dietary and physical exercise modification in weight loss management for overweight and obese adults.” In: Cochrane Database Syst Rev 2 (2014). doi: 10.1002/14651858.CD008066.pub3.

Chapter 8 1. Richard H Thaler and Cass R Sunstein. Nudge: Improving Decisions about Health, Wealth, and Happiness. Revised and expanded edition. New York: Penguin Books, 2009. 2. Eric J Johnson and Daniel Goldstein. “Medicine. Do defaults save lives?” In: Science 302.5649 (2003), pp. 1338–39. doi: 10.1126/science.1091721. 3. https://freakonomics.com/podcast/live-philadelphia/. 4. https://freakonomics.com/podcast/live-philadelphia/. 5. Terrance Odean. “Are investors reluctant to realize their losses?” In: Journal of Finance 53.5 (1998), pp. 1775–98. doi: 10.1111/0022-1082.00072. e-print: https://onlinelibrary.wiley.com /doi/pdf/10.1111/0022-1082.00072. url: https://onlinelibrary.wiley.com/doi/abs/10.1111/0022 -1082.00072. 6. A Tversky and D Kahneman. “The framing of decisions and the psychology of choice.” In: Science 211.4481 (1981), pp. 453–58. doi: 10.1126/science.7455683. 7. Bradley P Turnwald et al. “Increasing vegetable intake by emphasizing tasty and enjoyable attributes: A randomized controlled multisite intervention for taste-focused labeling.” In: Psychol Sci (2019), pp. 1603–15. doi: 10.1177/0956797619872191. 8. Chung-won Lee and Jennifer Kahende. “Factors associated with successful smoking cessation in the United States, 2000.” In: Am J Public Health 97.8 (2007), pp. 1503–9. doi: 10.2105 /AJPH.2005.083527. 9. Jutta Mata, Peter M Todd, and Sonia Lippke. “When weight management lasts. Lower perceived rule complexity increases adherence.” In: Appetite 54.1 (2010), pp. 37–43. doi: 10.1016 /j.appet.2009.09.004. 10. Benjamin Scheibehenne, Linda Miesler, and Peter M Todd. “Fast and frugal food choices: Uncovering individual decision heuristics.” In: Appetite 49.3 (2007), pp. 578–89. doi: 10.1016/j.appet.2007.03.224. 11. Angela L Duckworth et al. “A stitch in time: Strategic self-control in high school and college students.” In: J Educ Psychol 108.3 (2016), pp. 329–41. doi: 10.1037/edu0000062. 12. Melvyn Zhang et al. “A systematic review of attention biases in opioid, cannabis, stimulant use disorders.” In: Int J Environ Res Public Health 15.6 ( June 2018). doi: 10.3390/ijerph 15061138. 13. Todd F Heatherton and Patricia A Nichols. “Personal accounts of successful versus failed attempts at life change.” In: Personality and Social Psychology Bulletin 20.6 (1994), pp. 664– 75. doi: 10.1177/0146167294206005. e-print: https://doi.org/10.1177/0146167294206005. url: https://doi.org/10.1177/0146167294206005. 14. Wendy Wood, Leona Tam, and Melissa Guerrero Witt. “Changing circumstances, disrupting habits.” In: J Pers Soc Psychol 88.6 (2005), pp. 918–33. doi: 10.1037/0022-3514.88.6.918. 15. A M Graybiel and S L Rauch. “Toward a neurobiology of obsessive-compulsive disorder.” In: Neuron 28.2 (2000), pp. 343–47. doi: 10.1016/s0896-6273(00)00113-6.

210

notes to chapter 9

16. Joseph F McGuire et al. “Behavior therapy for tic disorders: An evidenced-based review and new directions for treatment research.”’ In: Curr Dev Disord Rep 2.4 (2015), pp. 309–17. doi: 10.1007/s40474-015-0063-5. 17. https://deconstructingyourself.com/overcoming-craving.html. 18. Nicholas T Van Dam et al. “Mind the hype: A critical evaluation and prescriptive agenda for research on mindfulness and meditation.” In: Perspect Psychol Sci 13.1 ( Jan. 2018), pp. 36–61. doi: 10.1177/1745691617709589. 19. Stephanie Coronado-Montoya et al. “Reporting of positive results in randomized controlled trials of mindfulness-based mental health interventions.” In: PLoS One 11.4 (2016), e0153220. doi: 10.1371/journal.pone.0153220. 20. Adrian M Owen et al. “Putting brain training to the test.” In: Nature 465.7299 (2010), pp. 775–78. doi: 10.1038/nature09042. 21. Daniel J Simons et al. “Do ‘Brain-Training’ Programs Work?” In: Psychol Sci Public Interest 17.3 (2016), pp. 103–86. doi: 10.1177/1529100616661983. 22. Monica Melby-Lervåg and Charles Hulme. “Is working memory training effective? A metaanalytic review.” In: Dev Psychol 49.2 (2013), pp. 270–91. doi: 10.1037/a0028228. 23. Andrew Jones et al. “A randomized controlled trial of inhibitory control training for the reduction of alcohol consumption in problem drinkers.” In: J Consult Clin Psychol 86.12 (2018), pp. 991–1004. doi: 10.1037/ccp0000312. 24. Isabel Carrero, Irene Vilà, and Raquel Redondo. “What makes implementation intention interventions effective for promoting healthy eating behaviours? A meta-regression.” In: Appetite 140 (2019), pp. 239–47. doi: 10.1016/j.appet.2019.05.024. 25. Lenard I Lesser, Caroline A Thompson, and Harold S Luft. “Association between monetary deposits and weight loss in online commitment contracts.” In: Am J Health Promot 32.1 ( Jan. 2018), pp. 198–204. doi: 10.1177/0890117116661157. 26. Nia Coupe et al. “The effect of commitment-making on weight loss and behaviour change in adults with obesity/overweight; a systematic review.” In: BMC Public Health 19.1 (2019), p. 816. doi: 10.1186/s12889-019-7185-3.

Chapter 9 1. Reut Shema, Todd Charlton Sacktor, and Yadin Dudai. “Rapid erasure of long-term memory associations in the cortex by an inhibitor of PKM zeta.” In: Science 317.5840 (2007), pp. 951–53. doi: 10.1126/science.1144334. 2. Jose A Crespo et al. “Activation of PKCzeta and PKMzeta in the nucleus accumbens core is necessary for the retrieval, consolidation and reconsolidation of drug memory.” In: PLoS One 7.2 (2012), e30502. doi: 10.1371/journal.pone.0030502; Yan-qin Li et al. “Inhibition of PKMzeta in nucleus accumbens core abolishes long-term drug reward memory.” In: J Neurosci 31.14 (2011), pp. 5436–46. doi: 10.1523/JNEUROSCI.5884-10.2011; D Shabashov, E Shohami, and R Yaka. “Inactivation of PKM in the NAc shell abolished cocaine-conditioned reward.” In: J Mol Neurosci 47.3 (2012), pp. 546–53. doi: 10.1007/s12031-011 -9671-7.

n o t e s t o c h a p t e r 9 211 3. Wolfgang M Pauli et al. “Inhibiting PKM reveals dorsal lateral and dorsal medial striatum store the different memories needed to support adaptive behavior.” In: Learn Mem 19.7 (2012), pp. 307–14. doi: 10.1101/lm.025148.111. 4. J R Misanin, R R Miller, and D J Lewis. “Retrograde amnesia produced by electroconvulsive shock after reactivation of a consolidated memory trace.” In: Science 160.3827 (1968), pp. 554–55. doi: 10.1126/science.160.3827.554. 5. K Nader, G E Schafe, and J E LeDoux. “Fear memories require protein synthesis in the amygdala for reconsolidation after retrieval.” In: Nature 406.6797 (2000), pp. 722–26. doi: 10.1038/35021052. 6. Jonathan L C Lee et al. “Disrupting reconsolidation of drug memories reduces cocaineseeking behavior.” In: Neuron 47.6 (2005), pp. 795–801. doi: 10.1016/j.neuron.2005.08.007. 7. Yan-Xue Xue et al. “A memory retrieval-extinction procedure to prevent drug craving and relapse.” In: Science 336.6078 (2012), pp. 241–45. doi: 10.1126/science.1215070. 8. R D Rubin. “Clinical use of retrograde amnesia produced by electroconvulsive shock. A conditioning hypothesis.” In: Can Psychiatr Assoc J 21.2 (1976), pp. 87–90. doi: 10.1177/ 070674377602100205. 9. Michael Treanor et al. “Can memories of traumatic experiences or addiction be erased or modified? A critical review of research on the disruption of memory reconsolidation and its applications.” In: Perspect Psychol Sci 12.2 (Mar. 2017), pp. 290–305. doi: 10.1177 /1745691616664725. 10. Nasir H Naqvi et al. “Damage to the insula disrupts addiction to cigarette smoking.” In: Science 315.5811 (2007), pp. 531–34. doi: 10.1126/science.1135926. 11. https://www.vice.com/en_us/article/gqwnex/with-and-without-my-ritalin. 12. Irena P Ilieva, Cayce J Hook, and Martha J Farah. “Prescription stimulants’ effects on healthy inhibitory control, working memory, and episodic memory: A meta-analysis.” In: J Cogn Neurosci 27.6 (2015), pp. 1069–89. doi: 10.1162/jocn_a_00776. 13. Margaret C Wardle et al. “Amping up effort: Effects of d-amphetamine on human effort-based decision-making.” In: J Neurosci 31.46 (2011), pp. 16597–602. doi: 10.1523 /JNEUROSCI.4387-11.2011. 14. https://www.fda.gov/drugs/science-and-research-drugs/table-pharmacogenomic-bio markers-drug-labeling. 15. Andrew T Drysdale et al. “Resting-state connectivity biomarkers define neurophysiological subtypes of depression.” In: Nat Med 23.1 ( Jan. 2017), pp. 28–38. doi: 10.1038/nm.4246. 16. Richard Dinga et al. “Evaluating the evidence for biotypes of depression: Methodological replication and extension of Drysdale et al. (2017).” In: Neuroimage Clin 22 (2019), p. 101796. doi: 10.1016/j.nicl.2019.101796. 17. Ian W Eisenberg et al. “Uncovering the structure of self-regulation through data-driven ontology discovery.” In: Nat Commun 10.1 (2019), p. 2319. doi: 10.1038/s41467-019-10301-1. 18. Elizabeth O Lillie et al. “The n-of-1 clinical trial: The ultimate strategy for individualizing medicine?” In: Per Med 8.2 (2011), pp. 161–173. doi: 10.2217/pme.11.7. 19. Jonathan A Shaffer et al. “N-of-1 randomized intervention trials in health psychology: A systematic review and methodology critique.” In: Ann Behav Med 52.9 (Aug. 2018), pp. 731–42. doi: 10.1093/abm/kax026.

212

notes to chapter 10

Chapter 10 1. Robert M Kaplan and Veronica L Irvin. “Likelihood of null effects of large NHLBI clinical trials has increased over time.” In: PLoS One 10.8 (2015), e0132382. doi: 10.1371 /journal.pone.0132382. 2. Open Science Collaboration. “Psychology. Estimating the reproducibility of psychological science.” In: Science 349.6251 (2015). doi: 10.1126/science.aac4716. 3. Rotem Botvinik-Nezer et al. “Variability in the analysis of a single neuroimaging dataset by many teams.” In: Nature 582.7810 (2020) pp. 84–88. doi: 10.1038/s41586-020-2314-9. 4. Timothy M Lenton et al. “Climate tipping points—too risky to bet against.” In: Nature 575.7784 (Nov. 2019), pp. 592–95. doi: 10.1038/d41586-019-03595-0. 5. Adam R Aron et al. “How can neuroscientists respond to the climate emergency?” In: Neuron 106.1 (2020), pp. 17–20. doi: 10.1016/j.neuron.2020.02.019. 6. Adam R Aron. “The climate crisis needs attention from cognitive scientists.” In: Trends Cogn Sci 23.11 (Nov. 2019), pp. 903–6. doi: 10.1016/j.tics.2019.08.001.

index

amnesia, 21, 22, 184 anterior cingulate, 114 Arnsten, Amy, 93–97 Aron, Adam, 116

Graybiel, Anne, 51, 52, 62, 188 H. M., 20, 21 hippocampus, 19, 20, 22, 62, 63 Huntington’s disease, 24–26, 33, 34 hyperdirect route, 117, 118

basal ganglia, 24–28, 30–32, 42, 43, 48, 49, 51, 58, 61, 63–65, 117, 119, 125, 132, 150 Behrens, Tim, 118 Bouton, Mark, 46, 47, 58

implementation intentions, 176 impulsivity, 110–112, 114, 130, 131 intertemporal choice, 102, 107–109

caudate nucleus, 26, 27, 49, 50 Cohen, Neal, 21, 22, 25, 26 Corkin, Suzanne, 20, 21

James, William, 4–6 Kable, Joe, 108 Knowlton, Barbara, 49, 50, 64, 65, 77

Daw, Nathaniel, 40, 74–76, 78, 133 Dayan, Peter, 74 Dickinson, Anthony, 9, 46, 63 diffusion-weighted imaging, 89, 90, 118 dopamine, 13, 14, 27, 30–32, 34–44, 50, 55, 58, 64, 65, 67, 68, 71, 72, 92–95, 97, 104, 107, 124–132, 134, 136, 141, 142, 144, 145, 148–150, 153, 190 dopamine fast, 58, 170 Duckworth, Angela, 99, 100, 121, 162, 167

law of effect, 67 limbic system, 24 Logan, Gordon, 115, 116, 119 machine learning, 37, 66 MacLean, Paul, 23–26 Martone, Maryanne, 25 meditation, 121, 122, 170, 171 Miller, Earl, 93 Milner, Brenda, 20, 21 mindfulness, 170, 171 Mischel, Walter, 97–100, 107, 156

Everitt, Barry, 130–132, 147, 183 executive function, 179, 189, 190, 192 exposure therapy, 47, 48 functional MRI, 65, 191, 193

noradrenaline, 32, 93–96, 190 nucleus accumbens, 26, 27, 41, 49, 55, 107, 108, 124, 127, 128, 130, 132, 136, 138, 142, 181, 187

globus pallidus, 27–30, 43 Goldman-Rakic, Patricia, 91–94, 97

213

214

index

optogenetics, 32, 33, 39, 43, 56, 188

Robbins, Trevor, 116, 130, 131, 147

Packard, Mark, 49, 61–65, 136, 188 Parkinson’s disease, 33, 34, 41, 64, 65 Pavlovian-instrumental transfer, 52–55, 57, 58, 136 PKM-zeta, 180, 181 positron emission tomography, 128–131, 134 prefrontal cortex, 14, 26, 27, 49, 50, 52, 55, 77, 80, 83, 85–97, 107, 109, 114, 116–118, 132, 136, 143, 153, 160, 170, 189, 190 putamen, 26, 27, 49, 50, 117

Schultz, Wolfram, 36–38, 71 Shohamy, Daphna, 64, 65 Smith, Steve, 118 Squire, Larry, 19–22, 25, 26, 64, 65 stop-signal task, 115–119, 131, 175 striatum, 27–31, 34, 35, 41–44, 48–52, 119, 128, 131, 132, 141, 148, 149, 182 Stroop effect, 5, 56 substantia nigra, 27, 31, 34 subthalamic nucleus, 27, 28, 30, 117–119 synaptic plasticity, 34, 35

reconsolidation, 182–185 reinforcement learning, 13, 37, 38, 52, 67, 68, 70–74, 77–80, 133 reptilian brain, 23, 24 response inhibition, 30, 115, 116, 130, 131, 147, 148 resting fMRI, 191 resurgence, 46, 47 reward prediction error, 36–40, 65, 68, 69, 145

Tourette syndrome, 168 transcranial magnetic stimulation, 109, 187, 191 triune brain, 24 ventral tegmental area, 27, 31, 125 ventromedial prefrontal cortex, 108 willpower, 14, 99, 119, 120, 122, 137, 147, 167, 170, 171, 196

a note on the type This book has been composed in Arno, an Old-style serif typeface in the classic Venetian tradition, designed by Robert Slimbach at Adobe.