Why Brains Don't Compute 3030710637, 9783030710637

Citation preview

Dale Purves

Why Brains Don’t Compute

Why Brains Don’t Compute

Dale Purves

Why Brains Don’t Compute

Dale Purves Duke Institute for Brain Science Duke University Durham, NC, USA

ISBN 978-3-030-71063-7    ISBN 978-3-030-71064-4 (eBook) https://doi.org/10.1007/978-3-030-71064-4 © The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Cover image: courtesy of R. Beau Lotto This Springer imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

Preface

This book examines what seems to me the central challenge in neuroscience today: understanding how subjective experience generated by the human brain and the rest of the nervous system is related to the physical world in which we live. The 25 short chapters that follow summarize the argument and evidence that brains address this problem using a wholly empirical strategy. The goal is to encourage neuroscientists, computer scientists, philosophers, and other interested parties to think about this concept of neural function and its implications, not least the conclusion that brains don’t compute. Indeed, if brains operate empirically there is nothing to compute. Durham, NC, USA

Dale Purves

v

Acknowledgement

I am grateful to Jan Troutt for her work on the art.

vii

Contents

Part I Solving Problems in Different Realities 1

Solving Problems��������������������������������������������������������������������������������������    3 1.1 Introduction ��������������������������������������������������������������������������������������    3 1.2 Two Perspectives ������������������������������������������������������������������������������    3 1.3 Consequences������������������������������������������������������������������������������������    4 1.4 Intelligence����������������������������������������������������������������������������������������    4 1.5 Artificial Intelligence������������������������������������������������������������������������    5 1.6 Common Ground������������������������������������������������������������������������������    6 1.7 Conclusion����������������������������������������������������������������������������������������    6 Further Readings����������������������������������������������������������������������������������������    7

2

 Objective and Subjective Reality������������������������������������������������������������    9 2.1 Introduction ��������������������������������������������������������������������������������������    9 2.2 Newtonian Reality����������������������������������������������������������������������������    9 2.3 Biological Measurements������������������������������������������������������������������   10 2.4 What, Then, Do We Perceive?����������������������������������������������������������   11 2.5 Discrepancies Between Objective and Subjective����������������������������   11 2.6 What About Machines����������������������������������������������������������������������   12 2.7 Conclusion����������������������������������������������������������������������������������������   12 Further Readings����������������������������������������������������������������������������������������   13

Part II Algorithmic Computation 3

Algorithms������������������������������������������������������������������������������������������������   17 3.1 Introduction ��������������������������������������������������������������������������������������   17 3.2 Instantiation of Algorithms in Machines������������������������������������������   17 3.3 Turing Machines ������������������������������������������������������������������������������   18 3.4 Boolean Algebra��������������������������������������������������������������������������������   19 3.5 Electronic Computers������������������������������������������������������������������������   20

ix

x

Contents

3.6 The Emergence of Modern Computation�����������������������������������������   20 3.7 Conclusion����������������������������������������������������������������������������������������   21 Further Readings����������������������������������������������������������������������������������������   21 4

Coding ������������������������������������������������������������������������������������������������������   23 4.1 Introduction ��������������������������������������������������������������������������������������   23 4.2 Computer Codes��������������������������������������������������������������������������������   23 4.3 Programming Languages������������������������������������������������������������������   24 4.4 Personal Computers��������������������������������������������������������������������������   24 4.5 Neural Coding ����������������������������������������������������������������������������������   25 4.6 Action Potentials ������������������������������������������������������������������������������   25 4.7 Conclusion����������������������������������������������������������������������������������������   26 Further Readings����������������������������������������������������������������������������������������   27

Part III Neural Networks 5

Neural Networks��������������������������������������������������������������������������������������   31 5.1 Introduction ��������������������������������������������������������������������������������������   31 5.2 History����������������������������������������������������������������������������������������������   31 5.3 A Problem for Artificial Networks����������������������������������������������������   32 5.4 Failure to Take Off����������������������������������������������������������������������������   32 5.5 Perceptrons����������������������������������������������������������������������������������������   33 5.6 Conclusion����������������������������������������������������������������������������������������   34 Further Readings����������������������������������������������������������������������������������������   34

6

 Resurrection of Neural Networks ����������������������������������������������������������   35 6.1 Introduction ��������������������������������������������������������������������������������������   35 6.2 Error Correction by Back Propagation ��������������������������������������������   35 6.3 Unsupervised Credit Assignment������������������������������������������������������   37 6.4 Search Spaces������������������������������������������������������������������������������������   37 6.5 Conclusion����������������������������������������������������������������������������������������   38 Further Readings����������������������������������������������������������������������������������������   38

7

Learning Empirically������������������������������������������������������������������������������   39 7.1 Introduction ��������������������������������������������������������������������������������������   39 7.2 An Algorithmic Beginning����������������������������������������������������������������   39 7.3 Games that Cannot Be Won Algorithmically������������������������������������   40 7.4 Conclusion����������������������������������������������������������������������������������������   42 Further Readings����������������������������������������������������������������������������������������   42

Part IV Perception 8

What We Perceive������������������������������������������������������������������������������������   47 8.1 Introduction ��������������������������������������������������������������������������������������   47 8.2 Traditional Assumptions ������������������������������������������������������������������   47 8.3 The Dilemma������������������������������������������������������������������������������������   48 8.4 A Solution ����������������������������������������������������������������������������������������   48

Contents

xi

8.5 Caveats����������������������������������������������������������������������������������������������   49 8.6 Conclusion����������������������������������������������������������������������������������������   49 Further Readings����������������������������������������������������������������������������������������   50 9

Spatial Intervals ��������������������������������������������������������������������������������������   51 9.1 Introduction ��������������������������������������������������������������������������������������   51 9.2 Discrepancies������������������������������������������������������������������������������������   51 9.3 Determining Frequencies of Occurrence������������������������������������������   53 9.4 An Example��������������������������������������������������������������������������������������   54 9.5 Explanation ��������������������������������������������������������������������������������������   54 9.6 Conclusion����������������������������������������������������������������������������������������   56 Further Readings����������������������������������������������������������������������������������������   56

10 Angles��������������������������������������������������������������������������������������������������������   57 10.1 Perceived Angles ����������������������������������������������������������������������������   57 10.2 Determining the Frequency of Angle Sources��������������������������������   57 10.3 Reason for Different Frequencies of Angle Projections ����������������   59 10.4 The Systematic Misestimation of Angles ��������������������������������������   60 10.5 Alternative Explanations ����������������������������������������������������������������   60 10.6 Conclusion��������������������������������������������������������������������������������������   62 Further Readings����������������������������������������������������������������������������������������   62 11 Lightness and Darkness��������������������������������������������������������������������������   63 11.1 Introduction ������������������������������������������������������������������������������������   63 11.2 Seeing in Black and White��������������������������������������������������������������   63 11.3 A Possible Physiological Explanation��������������������������������������������   64 11.4 A Different Approach����������������������������������������������������������������������   65 11.5 Conclusion��������������������������������������������������������������������������������������   67 Further Readings����������������������������������������������������������������������������������������   67 12 Empirical Ranking ����������������������������������������������������������������������������������   69 12.1 Introduction ������������������������������������������������������������������������������������   69 12.2 Stimuli as Recurrent Patterns����������������������������������������������������������   69 12.3 Luminance and Lightness as Examples������������������������������������������   70 12.4 Explaining More Complex Patterns������������������������������������������������   70 12.5 Conclusion��������������������������������������������������������������������������������������   72 Further Readings����������������������������������������������������������������������������������������   73 13 Color����������������������������������������������������������������������������������������������������������   75 13.1 Introduction ������������������������������������������������������������������������������������   75 13.2 Seeing Color������������������������������������������������������������������������������������   75 13.3 Color Explained Empirically����������������������������������������������������������   79 13.4 Conclusion��������������������������������������������������������������������������������������   80 Further Readings����������������������������������������������������������������������������������������   80 14 Color Psychophysics��������������������������������������������������������������������������������   81 14.1 Introduction ������������������������������������������������������������������������������������   81 14.2 Psychophysics ��������������������������������������������������������������������������������   81

xii

Contents

14.3 An Example������������������������������������������������������������������������������������   81 14.4 The Bezold-Brucke Effect��������������������������������������������������������������   83 14.5 Conclusion��������������������������������������������������������������������������������������   85 Further Readings����������������������������������������������������������������������������������������   85 15 Motion Speed��������������������������������������������������������������������������������������������   87 15.1 Introduction ������������������������������������������������������������������������������������   87 15.2 Physical Motion������������������������������������������������������������������������������   87 15.3 The Problem������������������������������������������������������������������������������������   88 15.4 An Example������������������������������������������������������������������������������������   90 15.5 The Explanation������������������������������������������������������������������������������   90 15.6 Conclusion��������������������������������������������������������������������������������������   92 Further Readings����������������������������������������������������������������������������������������   93 16 Motion Direction��������������������������������������������������������������������������������������   95 16.1 Introduction ������������������������������������������������������������������������������������   95 16.2 Apertures ����������������������������������������������������������������������������������������   95 16.3 Effect of a Circular Aperture����������������������������������������������������������   96 16.4 Effect of a Vertical Aperture�����������������������������������������������������������   98 16.5 Explanation ������������������������������������������������������������������������������������   99 16.6 Conclusion��������������������������������������������������������������������������������������  100 Further Readings����������������������������������������������������������������������������������������  100 17 Object Size������������������������������������������������������������������������������������������������  101 17.1 Introduction ������������������������������������������������������������������������������������  101 17.2 Classical Size “Illusions”����������������������������������������������������������������  101 17.3 Object Sizes in Scenes with 3-D Cues��������������������������������������������  102 17.4 Relevance to a Long-Standing Puzzle��������������������������������������������  104 17.5 Conclusion��������������������������������������������������������������������������������������  105 Further Readings����������������������������������������������������������������������������������������  105 18 Stereopsis��������������������������������������������������������������������������������������������������  107 18.1 Introduction ������������������������������������������������������������������������������������  107 18.2 Image Versus Anatomical Correspondence������������������������������������  107 18.3 Binocular Circuitry ������������������������������������������������������������������������  108 18.4 Ocular Dominance��������������������������������������������������������������������������  109 18.5 Relevance to Perception������������������������������������������������������������������  110 18.6 Conclusion��������������������������������������������������������������������������������������  111 Further Readings����������������������������������������������������������������������������������������  111 Part V Linking Objective and Subjective Domains 19 Stimuli and Behavior ������������������������������������������������������������������������������  115 19.1 Introduction������������������������������������������������������������������������������������  115 19.2 What, Then, Are Stimuli? ��������������������������������������������������������������  115 19.3 Behaviors as Understood by Physiologists ������������������������������������  116 19.4 Behaviors as Understood by Psychologists������������������������������������  116

Contents

xiii

19.5 The Common Strategy��������������������������������������������������������������������  117 19.6 Conclusion��������������������������������������������������������������������������������������  118 Further Readings����������������������������������������������������������������������������������������  118 20 Associations����������������������������������������������������������������������������������������������  119 20.1 Introduction������������������������������������������������������������������������������������  119 20.2 Associations Wrought by Evolution ����������������������������������������������  119 20.3 Associations Wrought by Lifetime Learning����������������������������������  120 20.4 Associations Wrought by Culture ��������������������������������������������������  121 20.5 Conclusion��������������������������������������������������������������������������������������  122 Further Readings����������������������������������������������������������������������������������������  122 21 Mechanisms����������������������������������������������������������������������������������������������  123 21.1 Introduction������������������������������������������������������������������������������������  123 21.2 Neural Plasticity������������������������������������������������������������������������������  123 21.3 Short-Term Changes ����������������������������������������������������������������������  124 21.4 More Enduring Changes ����������������������������������������������������������������  124 21.5 Reward��������������������������������������������������������������������������������������������  125 21.6 Conclusion��������������������������������������������������������������������������������������  126 Further Readings����������������������������������������������������������������������������������������  127 22 Reflexes ����������������������������������������������������������������������������������������������������  129 22.1 Introduction������������������������������������������������������������������������������������  129 22.2 Behavioral Responses as Reflexes��������������������������������������������������  129 22.3 Is All Behavior Reflexive?��������������������������������������������������������������  131 22.4 Counterarguments ��������������������������������������������������������������������������  132 22.5 Conclusion��������������������������������������������������������������������������������������  133 Further Readings����������������������������������������������������������������������������������������  133 Part VI Other Theories 23 Feature Detection ������������������������������������������������������������������������������������  137 23.1 Introduction������������������������������������������������������������������������������������  137 23.2 Feature Detection����������������������������������������������������������������������������  137 23.3 Neurons that Respond to more Specific Stimuli ����������������������������  139 23.4 Perception in Monkeys�������������������������������������������������������������������  140 23.5 Back to Sherrington������������������������������������������������������������������������  140 23.6 Conclusion��������������������������������������������������������������������������������������  140 Further Readings����������������������������������������������������������������������������������������  141 24 Statistical Inference���������������������������������������������������������������������������������  143 24.1 Introduction������������������������������������������������������������������������������������  143 24.2 Statistical Inference������������������������������������������������������������������������  143 24.3 Bayes’ Theorem������������������������������������������������������������������������������  144 24.4 The Problem with a Bayesian Approach����������������������������������������  146 24.5 Conclusion��������������������������������������������������������������������������������������  146 Further Readings����������������������������������������������������������������������������������������  146

xiv

Contents

Summing Up ��������������������������������������������������������������������������������������������  149 25 25.1 In Brief��������������������������������������������������������������������������������������������  149 Figure Sources��������������������������������������������������������������������������������������������������  151 Glossary: Definitions of Some Relevant Terms ��������������������������������������������  155 Index������������������������������������������������������������������������������������������������������������������  163

About the Author

Dale Purves  is the George B. Geller Professor of Neurobiology Emeritus in the Duke Institute for Brain Sciences, where he remains Research Professor with additional appointments in the department of Psychology and Brain Sciences and the department of Philosophy at Duke University. After earning a B.A. from Yale, an M.D. from Harvard and additional postdoctoral training at Harvard and University College London, he joined the faculty at Washington University School of Medicine in 1973. In 1990, he became the founding chair of the Department of Neurobiology at Duke Medical Center and was subsequently Director of Duke’s Center for Cognitive Neuroscience. He also served as the Director of the Neuroscience and Behavioral Disorders Program at the  Duke-NUS Graduate Medical School  in Singapore. Best known for his work on neural development and synaptic plasticity, Purves’ research during the last 20 years has sought to explain visual perception and auditory perception in the context of music. He is a member of the National Academy of Sciences, the National Academy of Medicine, a fellow of the American Academy of Arts and Sciences and is the author, and co-author or editor of 11 previous books on neuroscience.

xv

Part I

Solving Problems in Different Realities

Basic to the argument that follows is that the major challenge in contemporary neuroscience is understanding how solving objective problems differs from solving problems in the subjective world that we and other animals perceive. Part I introduces these issues.

Chapter 1

Solving Problems

1.1 Introduction Broadly speaking, there are two ways biological or artificial agents can solve problems: (1) by following a set of logical rules; or (2) by trial-and-error learning—i.e., empirically. Do biological brains operate like electronic computers governed by logical rules, and, if they do, what exactly are they computing? Or do they operate simply by instantiating neural connections and functions that have proven successful in the past?

1.2 Two Perspectives These questions have no agreed upon answers and, for the moment at least, science seems to have dug itself a rather deep hole when it comes to dealing with them. Neuroscientists and psychologists do not really know how brains work; computer scientists do not know how empirically based computers function; and mind-brain philosophers have yet to come up with an idea that could help resolve these issues. Until relatively recently computers have solved problems by following logical rules (algorithms) encoded in programs running on mainframes, laptops, smartphones, and other devices. This approach has become less compelling, however, as computer scientists have increasingly attacked problems that entail large data sets empirically, using artificial neural networks that learn from experience. This latter approach does not depend on rules other than preliminary guidance about how trial and error responses are executed and how success is “rewarded.”

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. Purves, Why Brains Don’t Compute, https://doi.org/10.1007/978-3-030-71064-4_1

3

4

1  Solving Problems

1.3 Consequences Because algorithmic and empirical strategies are fundamentally different, these two ways of problem solving have tended to segregate their practitioners. Neuroscientists and psychologists generally do not know much about computer science or the skills needed in that field; and computer scientists, mathematicians and theoreticians generally do not know much about biology, let alone the extraordinary anatomical, physiological and biochemical complexity of brains. And philosophers without much expertise in any of these domains are caught in the middle. One camp—computer scientists and neuroscientists attracted to mathematical and logical theories and methodologies—like to explain the function of the brain and the rest of the nervous system as a computational machine that employs sophisticated algorithms. Conversely, biologically inclined neuroscientists—probably the majority—regard this conception as unrealistic. The counter argument is that the brain is an organ like any other and must operate according to strategies evolved by natural selection over the last 3 or 4 billion years. Given the nature of the problems organisms face, computation based on logical rules may not be up to the challenges that biological actors routinely encounter in the real world.

1.4 Intelligence At the level of primates and many other animals the challenges facing organisms (or artificial entities) requires something loosely referred to as “intelligence.” But what that is in either biological or computational terms is difficult to say. Many people think of intelligence as the ability to deal effectively with logical problems and imagine that is what brains, in the main, must be doing. Psychologists showed long ago, however, that logical thinking is not a human forte. When high school students were asked to estimate the products of the sequence 1 × 2 × 3 × 4 × 5 × 6 × 7 × 8 versus the sequence 8 × 7 × 6 × 5 × 4 × 3 × 2 × 1, the median answers given for the first series was 512 and for the second 2250. The students failed to recognize that the sequences are multiplications the same numbers and must therefore generate the same product (which is actually 40,320). Another infamous example is the “Monty Hall Problem” named after the TV game show host. In this scenario a contestant is presented with three doors behind one of which is a prize (Fig. 1.1). The contestant chooses one of the doors, say door number 1, but it remains closed. Monty Hall then opens door number 3 showing the player that the prize is not behind that door. The problem is whether the contestant is better off sticking with the initial choice of door 1 or switching to door number 2. The answer is that the probability of winning increases if the player’s choice is switched to door 2, but few of us have the wherewithal to explain why. When it comes to logical problems we animals are poor performers. And, from a biological perspective, it is not hard to see why. Since rationality and logic play little part in survival and reproduction, a generally feeble ability to solve logical problems might be expected. Reasoning is a ponderous strategy for answering most

5

1.5  Artificial Intelligence

1/3

1

2/3

2

2/3

0

Fig. 1.1  The Monty Hall Problem. In this scenario the goal of the contestant is to win a prize that is behind one of the three doors. The contestant decides to open Door 1 but has not yet done so. Door 3 is then opened by the host showing that the prize is not behind that door. The question is if the contestant switches his or her choice to Door 2 will that increase the odds of winning? The answer is that it does. Although the original odds of winning do not change for Door 1 (they remain 1 in 3), the odds for choosing Door 2 are now 2 in 3 instead of 1 in 3

problems encountered during life, and problems whose solution demands reasoning and logic are relatively rare outside the classroom or Martin Gardner’s puzzle page in an old Scientific American magazine. The concept of “intelligence” typically refers to talents that most of us lack, such as the ability to do higher mathematics or to solve difficult challenges such as the Monty Hall problem. But this idea is misleading: is the accomplished athlete solving complex problems on the playing field any less “intelligent” than the math prodigy? This definitional deficiency has led to endless arguments, not least the value of IQ tests and other dubious measures of a vague concept whose determination is subject to social history, economic status and many other biases. Getting along in world is not generally based on rules, even for simple organisms that have been succeeding in the game of life for much longer than we have.

1.5 Artificial Intelligence Artificial intelligence refers to non-biological neural networks instantiated in computers that can address a variety of problems. This approach has been hugely successful in tasks that range from identifying faces, understanding how proteins fold up to predicting winning poker hands. This impressive success does not mean, however, that brains operate in this way, that neurons are computing elements, or that

6

1  Solving Problems

the problems we typically confront are being solved in this way. The goal of any science is explaining phenomena that are otherwise mysterious. Missing, however, is a clear statement of what phenomenology AI seeks to explain. There are of course plenty of observations that need to be rationalized, the most obvious being numerous well-documented psychophysical/perceptual phenomena that human brains generate as outputs that mediate successful behavior. But that goal does not seem to be on the AI agenda. Given the complex situations we face on a daily basis, the alternative is that nervous systems solve the vast majority of problems encountered by continually updating brain structure and function on basis trial and error experience learned by us as a species and as individuals.

1.6 Common Ground Of course there is some common ground among the various fields trying to sort out these concerns. The word “problem” defined as a practical or conceptual challenge that has (or might have) a solution seems clear enough. Likewise defining “machine” as any material entity that performs work obeying the laws of physics will probably not raise anyone’s hackles. While in centuries past the definition of machines was often limited to human artifacts that used levers, cogs, pulleys, and so on, it seems safe to assume that scientists today accept that nervous systems operate mechanically. Nevertheless, seeking answers by plumbing the detailed organization of brains with the remarkable technology now available may put the cart before the horse. Without some general idea of what brains seek to accomplish and how they are doing it, knowing the genetic, molecular, anatomical and electrophysiological details of nervous systems may be relatively unrewarding. The same goes for understanding what computers that use artificial neural networks and “deep learning” strategies are actually doing to solve the problems they are presented with. Finally, given our current state of ignorance, constructing ad hoc mathematical and theoretical “models” of brain function may be barking up the wrong tree altogether.

1.7 Conclusion There are two different ways that human brains and those of other animals might operate: (1) using logical rules to solve biological problems; or (2) using trial and error responses that have led to successful outcomes over the course evolutionary and individual time. In recent decades these two ways of solving problems and the entities that carry out the relevant strategies have become ever more entangled. Understanding why depends on recognizing that two different realities are pertinent to human behavior.

Further Readings

7

Further Readings Bowler PJ (2003) Evolution: the history of an idea. University of California Press, Berkeley, CA Dyson F (1999) The origins of life. Cambridge University Press, New York, NY Feynman R (2001) The character of physical law (Messenger Lectures, 1964). MIT Press, Cambridge, MA Gleick J (2011) The information. Pantheon Books, New York, NY Gould SJ (1977) Ever since Darwin. W. W. Norton, New York, NY Harari YN (2014) Sapiens: a brief history of humankind. Vintage, New York, NY Krimsky S, Gruber J (eds) (2013) Genetic explanations: sense and nonsense. Harvard University Press, Cambridge, MA Tattersall I (2012) Masters of the planet: the search for human origins. St. Martin’s-Griffin, New York, NY

Chapter 2

Objective and Subjective Reality

2.1 Introduction Unless one goes looking for trouble, it is reasonable to assume that objective reality exists and that physical instruments measure it accurately, or at least as accurately as currently possible. This reality is defined by laws of physics and, with respect to the understanding the behavior of biological agents on Earth, we need not worry much about deeper issues that physicists are still grappling with (e.g., how quantum events are related to macroscopic events; the role of observation in reification; whether information subsumes matter and energy). For present purposes objective reality can be restricted to classical physics. In contrast, perceptions—whether conscious or as unconscious equivalents—are subjective.

2.2 Newtonian Reality The physical world animals deal with routinely can be measured with relatively simple instruments: rulers, protractors, photometers, spectrophotometers, laser range finders and the like. But even ignoring relativity and quantum physics, objective reality is not straightforward. Take the infamous case of the butterfly flapping its wings in Brazil in relation to a tornado in Texas, a scenario first considered by the mathematician and meteorologist Edward Lorenz in the 1970s. Once a tornado has struck in Texas one can, in principle, trace a chain of determined causal events back from the storm to show how the tornado is linked to the butterfly. But the fact that the butterfly flapped its wings does not predict the tornado. Only after an event has happened does a causal chain exist. At this point in the trajectory of neuroscience, however, we first need to sort out some more common sense issues.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. Purves, Why Brains Don’t Compute, https://doi.org/10.1007/978-3-030-71064-4_2

9

10

2  Objective and Subjective Reality

2.3 Biological Measurements A good start is taking to heart the fact that we and other animals cannot measure the environment. Even if Newton’s concept of time and space were all we had to be concerned with, biological agents cannot measure physical reality because they do not have the instruments needed to do so, i.e., rulers, protractors, photometers, and so on. Such devices did not evolve in the sensory systems we animals possess. To complicate further understanding how we behave effectively in the physical world, the information supplied by our sensory systems conflates the properties of stimulus sources (Fig.  2.1). For example, the light reaching our eyes—the luminance of visual stimuli—is determined by illumination, object reflectance, and atmospheric transmission. The combined influence of these physical properties in stimuli cannot be disentangled. Thus any biological stimulus could have been generated by many possible physical sources. This quandary—called the “inverse problem”—arises whenever the information relevant to the solution of a problem could have arisen in multiple ways.

Fig. 2.1  The entanglement of physical factors that give rise to stimuli (e.g., illumination, surface reflectance and atmospheric transmittance in the case of vision). The physical sources of stimuli cannot be known by logical operations carried out by nervous systems

2.5  Discrepancies Between Objective and Subjective

11

The fact that biological sensors cannot measure physical parameters and that whatever information biological agents do get conflates the possible sources of stimuli means that animals are prevented from apprehending the objective properties of the world.

2.4 What, Then, That What We Perceive? When biological agents respond to a stimulus using their nervous systems, on what basis then does behavior succeed? When pondering such questions Theodosius Dobzhansky’s admonition that “nothing in biology makes sense except in the light of evolution” should never be far from mind. Our senses, the perceptions they produce and ability to use subjective information in order to act and think evolved, as has the wherewithal to learn from the behaviors executed over individual lifetimes. For animals the physical world is simply an arena in which the utility of the neural connections between inputs and outputs at any given moment are tested empirically. Although it may seem obvious that we perceive objective reality, philosophers and others have long realized that this impression is false. George Berkeley got it right in the eighteenth century when he argued that the perception of objects and their properties exists in our heads rather than in the physical world. If we touch red hot ember neither the pain we feel nor the color we see are properties of the object. The pain is a sensation generated by the nervous system that animals have evolved to protect us from dangerous circumstances. The color is likewise a subjective construct that allows us to perceive the environment in more informative ways than if we only saw it in black and white. The implication is that neural responses to stimuli are based on the trial-and-error consequences of past experience just like the evolution of any other phenotypic feature. A further point is that the neural connections and functions needed to respond to stimuli are already in place, having been determined by the success or failure of past experience. As implied in Chap. 1, the strategy animals use is apparently to make trial-and-error responses that continually refine neural circuitry and function in the quest to survive and reproduce. The beauty of this strategy is that it works without having to recover the physical properties of the world.

2.5 Discrepancies Between Objective and Subjective Responding to physical reality in this way means that subjective experience will always be at odds with physical measurements. All perceptions are therefore illusory in the sense that they never correspond to measurements of the physical world, a fact makes it hard to understand how actions and other behaviors can routinely succeed.

12

2  Objective and Subjective Reality

The phenomena that must be accounted for are subjective effects that have been systematically studied over the last century or two by testing the responses of human and non-human animals to stimuli in different sensory modalities. To take an example, when projected onto a sensor array like the retina, a ruler in 3D space is transformed into a 2D length that is upside down and right-left reversed. Moreover, if the ruler in space is angled towards or away from the observer, the projection onto the retina no longer conveys the same length it did when the ruler was presented in the frontal plane: the projection of the ruler is foreshortened, in the limit to a point. Although a handful of psychologists and philosophers may still think we perceive the world as it “really is” we clearly do not apprehend what instruments measure. Therefore what we perceive cannot be representations or “models” of the physical world. These psychophysical data provide a rich body of facts that need explaining.

2.6 What About Machines What then about non-biological machine—computers—used in studies of “artificial intelligence”? Whether artifacts such as computers could ever have the subjective experiences that we routinely enjoy is naturally debated. But if brains could be mimicked with sufficient accuracy in electronic or other hardware there would be little ground for concluding that the resulting “machines” could not have subjective experiences like ours. Computation continues to advance rapidly; and hominid brains have been evolving for several million years and will go doing so driven by natural selection and the more rapid changes driven by social learning. Barring the extinction of life on Earth these forces will drive the sophistication of agents far beyond what we know today. Indeed, given the Universe as presently understood, these advances may well have happened already elsewhere in our galaxy. Such speculations aside, a practical question in the current context is whether non-biological machines that evolved by trial and error training would end up perceiving the world the strange way that we do.

2.7 Conclusion Biological agents do not have the wherewithal to make the physical measurements carried out with instruments. The result is difficulty understanding how brains generate useful behaviors. A corollary is that developing “intelligent” machines that solve problems “the way we do” will be stymied until we understand a lot better how brains work. Since biological agents lack even the simplest tools used in physics, animals must be representing and responding appropriately to physical reality empirically. But how is this challenge met by biological or non-biological machines? To answer this question it is important to understand how computers that depend on

Further Readings

13

logic (PartI II) differ from an alternative method that, while in some sense computational, is fundamentally empirical (Part III).

Further Readings Dawkins R (1986) The blind watchmaker. W.W. Norton, New York, NY de Waal F (2016) Are we smart enough to know how smart animals are? Norton, New York, NY Dobzhansky T (1973) Nothing in biology makes sense except in the light of evolution. Am Biol Teach 35:125–129 Heyer D, Mausfeld R (2002) Perception and the physical world: psychological and philosophical issues in perception. Wiley, New York, NY Purves D (2010) Brains: how they seem to work. Pearson/Financial Times Press, New York Tasker E (2017) The planet factory: exoplanets and the search for a second earth. Bloomsbury Sigma, New York, NY

Part II

Algorithmic Computation

Reviewing mechanical and electronic computation carried out over the centuries by following sets of rules called algorithms seems the place to start before delving into what computers and nervous systems can and cannot do.

Chapter 3

Algorithms

3.1 Introduction Following a set of logical rules—an algorithm—remains the basis for most (but certainly not all) computation today. The history of algorithms as a basis for computing goes back a long way. The word itself derives from the Latinization of the ninth century Persian mathematician Al-Khwarizmi to “Algoritmi” when his book on numbers was translated in the twelfth century. In the seventeenth century the term became “algorithm” in English although its meaning as following a set of rules emerged only in the nineteenth century.

3.2 Instantiation of Algorithms in Machines The abacus, a counting frame with beads strung on wires that can perform arithmetical operations, can be traced back millennia and remains in use today in less developed regions of the world. The abacus was only superseded by mechanical calculators in the seventeenth century based on a device constructed by French philosopher and mathematician Blaise Pascal to help in his father’s tax collecting business. The transition in the nineteenth century to something approaching a modern computer is usually attributed to Charles Babbage and Ada Lovelace. Babbage was born into a wealthy London family in 1791, and, after graduating from Cambridge in 1814, pursued a variety of passions that included mathematics, engineering and politics during a very public life. Babbage was appointed Lucasian Professor of Mathematics at Cambridge in 1828 and interacted with nearly every notable scientific and intellectual figure of his day. Exasperated with inaccuracies in the logarithmic tables then in use, Babbage began working on his “Difference Engine” in 1822. The machine was intended to compute polynomial functions (i.e., functions based © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. Purves, Why Brains Don’t Compute, https://doi.org/10.1007/978-3-030-71064-4_3

17

18

3 Algorithms

on addition, multiplication and integer exponents) to solve numerical problems. Comprising an estimated 25,000 parts the machine was not completed in Babbage’s lifetime despite substantial government funding and the help of first-rate engineers and machinists (the “Engine” was finally built in in the early 2000s and resides in the British Museum of Science). Subsequently Babbage designed his “Analytical Engine” taken to be the first attempt to realize a programmable general-purpose computer. Instrumental in this project was Ada Lovelace who is credited with getting Babbage to extend his thinking from un-programmed machinery to a device that could be instructed by rules using punch cards suggested by the operation of Jacquard looms. Lovelace was the only legitimate daughter of the poet Lord Byron who abandoned the family when she was an infant. Her tutor recognized Lovelace’s mathematical talent when she was a teenager and introduced her to Babbage. They began a long correspondence in 1842 when Lovelace undertook the translation of a paper in Italian about Babbage’s work. Her extensive notes in the translated manuscript and their voluminous correspondence subsequently led to a much broader perspective than the eclectic Babbage had initially. Lovelace—who died of cancer at age 36  in 1852—is regarded by many as a computer pioneer and the first “computer programmer.”

3.3 Turing Machines In 1900 the German mathematician David Hilbert made a list of 23 challenges that initiated a period of mathematical formalism (axioms) often credited with establishing the logical framework in which computation grew from its nineteenth beginnings to relative maturity. Hilbert’s “10th Problem” asked whether an algorithm could determine in a finite number of steps whether a certain class of polynomial equations had an integer solution. In 1931 Hilbert’s student Kurt Goedel showed that no fully mathematical system for solving Hilbert’s 10th Problem was possible. His “incompleteness theorem” proved that no set of mathematical axioms can demonstrate its own adequacy. One of the fruits of Goedel’s theorem was the work of Alonzo Church at Princeton and his student, Alan Turing. Turing’s paper in 1936 titled “On computable numbers, with an application to the Entscheidung’s problem” (the German word means “decidability”) is generally recognized as a turning point in the concept of computation. The paper laid out how computation using a binary code could work in principle, leading to the central idea that drove computer science over the following decades. Turing envisioned an endless tape with units called “squares” that were fed into a reader one at a time (Fig. 3.1). Each square contained an instruction in binary code that the machine had to carry out before going forward (or backward) to the next (or previous) square on the tape, showing that any numerical operation can be implemented in this way. And since any logical operation can be represented numerically and any number of Turing machines can be concatenated, in principle almost any logical problem can be solved by a computer operating this way (thus the moniker “Universal Turing Machine”).

3.4  Boolean Algebra

19

Fig. 3.1  A “Turing machine.” An infinitely long tape with a single instruction on each square is fed through a “reading head.” The head has to interpret the instruction and execute it before the next square can be read and, according the instruction, the tape moves backward or forward. The device is not an operational computer (no actual machine operates on an endless tape); rather it is the physical demonstration of a logical concept

3.4 Boolean Algebra The construction of working computers, however, depended on much more than Turing’s seminal conception of how such machines could operate in principle. Constructing computers required logical switches, at first mechanical and eventually electronic. The principles underlying this complementary work are attributed to George Boole. Boole was born in Lincoln in 1815 to a cobbler and his wife. Unlike his contemporary Babbage, Boole was self-taught and never attended a university; until he was about 30 he made a living teaching and running secondary schools. Despite this inauspicious beginning, Boole produced a series of papers when he was in his 20s that gained recognition by several important mathematicians, including Augustus de Morgan at Cambridge with whom he carried on a correspondence. The excellence of his papers and the support of de Morgan and others allowed him to be appointed Professor of Mathematics at Queens College Cork in Ireland in 1849. The gist of Boole’s work was to show that logical expressions could be formalized algebraically, enabling mechanical switches that could carry out logical functions, leading to the electronic switches that modern computers depend on. The optimization of logical signaling was taken up in the 1940s by Claude Shannon who used Boolean algebra as a basis for the information theory he developed and applied to both telecommunications and computation.

20

3 Algorithms

3.5 Electronic Computers Work on the first general purpose electronic computer began in 1943 based on the conceptual ideas of Babbage, Boole, Shannon, and, most importantly, Turing. The computer was called ENIAC standing for “Electronic Numerical Integrator and Computer” and was intended to calculate artillery firing tables that used quadratic equations rather than polynomials as in Babbage‘s Analytical Engine. As it turned out, however, first significant application was determining the parameters of thermonuclear explosions for the Manhattan Project, then going on secretly in Los Alamos. ENIAC was constructed at the Moore School of Engineering at the University of Pennsylvania based on a design by engineers John Mauchly and Presper Eckert implemented by a team recruited mainly from the School. The machine used vacuum tubes, diodes, and relays and, when complete, weighed 30 tons and occupied 1800 square ft. of floor space. The input was punch cards and the operating programs entailed setting electrical switches. ENIAC could perform about 5000 operations per second, a great increase over what expert humans at the Moore School were then doing for the military. The human programmers—called the “computers”—were women also recruited from the School who had been doing ballistic calculations by hand. These expert calculators would be recognized today as computer programmers. Ad hoc improvements to ENIAC were made a more or less daily, many of them suggested by John von Neumann who understood the importance of ENIAC in advancing the work at Los Alamos. Von Neumann, remarkable by any standard, made fundamental contributions not only to computer science but to quantum mechanics and several branches of mathematics, all while being critically involved in advising the government on defense policy. ENIAC began formal operation in 1946 when it was moved to the Aberdeen Proving Grounds in Maryland and officially taken over by the Army. In 1944, not long after the construction of ENIAC had begun, Mauchly and Eckert proposed a new design called EDVAC (“Electronic Discrete Variable Automatic Computer”). Construction started in 1946, again at the Moore School of Engineering. EDVAC started operating at Aberdeen in 1951 and continued until 1962. Similar efforts at the same time were the “Colossus” computers being developed in England for decryption. This work was spearheaded by research telephone engineer Tommy Flowers with the help of Turing (their more famous computer that cracked the German “Enigma code” was called the “Electromechanical Bombe”).

3.6 The Emergence of Modern Computation These devices laid the groundwork for the rapid advances in computation that followed. Unlike the beginnings at Penn, however, the enormous progress from the 1950s to the present had more to do with advances in electronics and programming than conceptual ideas about computation per se. This progress was initially

Further Readings

21

motivated by the “space race” and military concerns during the Cold War, with financial support coming from the newly created Defense Advanced Research Agency (today’s DARPA). It did not very long, however, for consumer economics to begin driving computational progress based on programming ever more complex algorithms for dedicated tasks. The mainframe computers for non-military endeavors were pioneered by International Business Machines (IBM) which long dominated the field. At first the market for these machines outside the military was limited to institutions and corporations that wanted to make their operations more efficient and could afford them (in 1964 dollars the IBM 360 cost about $250,000). The next step, during which IBM gradually lost its lead, was the production of smaller less costly computers by the Digital Equipment Corporation (DEC). Its PDP-8 introduced in the early 1960s was the first computer that could be owned by individuals who were rich enough—and computer savvy enough—to use it (the cost in 1964 was still high at about $20,000). The market changed again in the late 1960s and early 1970s with the invention of integrated circuits, computer chips, and random access memory, again driven primarily by the military, this time for the “Minutemen” system of ICBMs. This progress spawned a roster of new companies dedicated to much smaller computers that were, among other uses, essential to the Moon landing in 1969. The final steps to commercialization were the advent of minicomputers, user-friendly programming, and eventually personal computers.

3.7 Conclusion An “algorithm” is often defined as a set of instructions that can solve problems when the rules are implemented in a Universal Turing machine. However, the question of how this enormously successful approach is related to the nervous systems remains unanswered. Brains are often considered to be computers because we sometimes solve problems by following rules. In biology, however, this idea is at best debatable.

Further Readings Berlinski D (2001) The advent of the algorithm: the 300-year journey from an idea to the computer. Harvest Books, New York, NY Chabert J-L (1999) A history of algorithms: from the pebble to the microchip. Springer, New York, NY Cormen TH, Leiserson CE, Rivest RL, Stein C (2009) Introduction to algorithms, 3rd edn. MIT Press, Cambridge, MA Dayan P, Abbott LF (2001) Theoretical neuroscience: computational and mathematical modeling of neural systems. MIT Press, Cambridge, MA

Chapter 4

Coding

4.1 Introduction Scientists who think about the brain in terms of algorithmic computation—or anyone else for that matter—must consider coding and its practical complement, programming. In addition to the logical, mathematical, and electronic advances that led to today’s computational devices, the ways of getting information into, through, and out of computers are fundamental. For present purposes, the key question is whether nervous systems also need a code and if so, what it is.

4.2 Computer Codes Coding refers to instructions in languages that electronic computers can deal with, usually digital instructions made understandable to human users in English. Although most computer codes today use binary digits—the 0 s and 1 s that in electronic form specify program instructions—it was not always so. For nineteenth century weavers the code was patterns of holes in cards; for Babbage and Lovelace, it was punch cards that instructed concatenations of wheels and gears in the Analytical Engine; for Tommy Flowers it was electrical mechanisms driven by telegraph keys; for ENIAC and Turing’s Bombe it was setting electronic sensors with lights that indicated whether a particular switch was on or off. It was evident from the outset that these cumbersome processes could be automated and that this would require “languages” adapted to electronics by Boolean algebra.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. Purves, Why Brains Don’t Compute, https://doi.org/10.1007/978-3-030-71064-4_4

23

24

4 Coding

4.3 Programming Languages The first programming language is generally considered IBM’s “short code” created in the early 1950s. The language specified operations, mathematical expressions, and variables in a form that could be put together in different ways as programs by “compilers.” The compilers were initially human workers, but these tasks were soon automated as well. The pioneers in the emergence of automated compilers were Grace Hopper in the USA and Alec Glennie in the UK. Hopper had been trained in mathematics at Yale and eventually became a Rear Admiral in the Navy. Glennie worked with Turing at Bletchley Park on decryption and developed a compiler called Autocode. The first widely used computer language, however, was FORTRAN (standing for “formula translation”) created for IBM mainframes during the 1950s by a team led by John Backus. Backus got off to an odd start as a radio engineer, but eventually earned a PhD in math at Columbia and started working for IBM. At the outset (circa 1953), FORTRAN comprised 32 statements such as GO TO, ASSIGN, PAUSE, STOP, BACKSPACE on punch cards that were arranged in blocks with a single entry and exit point. This seminal code has undergone numerous modifications since with the latest iteration used in high-performance “supercomputers.” Other coding languages soon followed, the most important being BASIC (“Beginner’s All-purpose Symbolic Instruction Code”) created in 1964 by John Kemeny and Thomas Kurtz at Dartmouth. The code was quickly taken up by users of the increasingly popular (and affordable) PDP-11 computers being produced by DEC (see Chap. 3). This is the language that would soon be used for programming minicomputers and was a driving force in the “personal computer” revolution that took off in the 1970s.

4.4 Personal Computers The revolution was initiated by the s home computer. In 1969 Ed Roberts and Forest Mims founded a company called Micro Instrumentation and Telemetry Systems (MITS) in Roberts’ garage in Albuquerque, New Mexico. The company sold Altairs as mail-order do-it-yourself kits. The kit quickly became popular among hobbyists and by 1974 could be purchased unassembled for about $500. The popularity of the Altair led Bill Gates and Paul Allen to write Roberts asking if he might be interested in a version of the BASIC computer language they were then working on at a start­up they called Microsoft. Roberts agreed and since they did not have an Altair, Gates simulated the operating system. The BASIC language they created for the Altair was a hit and was soon adopted by another personal computer start-up called Apple. The first Apple computer using BASIC was demonstrated at the Homebrew Computer Club by Steve Jobs and Steve Wozniak in 1976.

4.6  Action Potentials

25

4.5 Neural Coding When it comes to coding and programing in nervous systems, the question is how coding practices in algorithmic computation apply in biology, if they apply at all. Here again some definitions are essential. First, it is important not to confuse two meanings of the “information” being encoded. In Shannon’s usage, the word refers to the carrying capacity of a communication system, whether telephonic or neural. A different usage refers to the information in the message being conveyed. Making transmission in a telecommunication system or a neural system efficient bears little or no relation to the meaning of the messages sent and received. While there is certainly no reason to dispute the importance of efficiency as a means of minimizing metabolic and other biological costs—much as in reducing the costs of telecommunication and computation—the concept of efficient coding does not pertain to understanding how nervous systems generate what we experience subjectively, or how animals behave in response. The subjective experiences elicited by stimuli have to do with the meanings of the messages conveyed, not their efficiency. Nonetheless, as in telecommunications, reducing signal redundancy is clearly advantageous. The person who first pursued this line of thinking in neuroscience was the British vision scientist Horace Barlow working at Cambridge. In the 1960s Barlow began thinking about how information theory might be used to reduce redundancy in the conveying visual information. Barlow, and subsequently others, contributed far-ranging ideas about how this might happen, leading over time to a theory of neural compression that today is called “sparse coding.” Sparsity refers to the density of neural responses elicited by stimuli—i.e., number of neurons activated or deactivated in local circuits. The neural response to a stimulus might spread throughout much the brain (called “dense coding”) or, in the limit, be constrained to the activity of one or a few cells (the former case being Barlow’s infamous “grandmother cell”). Neither of these extremes would presumably be optimal. It thus followed the goal of sparse coding in nervous systems would be finding an optimally efficient neural “representation.” The measure of sparseness is generally described as the “peakedness” of neural activity in response to a stimulus. Activity that is sharply peaked (“tuned”) when recorded from neuronal populations in some region of the brain defines to sparse coding, whereas broadly tuned activity signifies widely distributed effects (i.e., dense coding). In any event, relative sparsity is commonly used as a measure of neural efficiency.

4.6 Action Potentials It is widely assumed by neuroscientists that carrier of any neural code is action potentials, the “all or none” electrical signals used by neurons to convey information from one place to another in nervous systems. Many neurons, but certainly not all, generate these brief self-propagating signals that travel to targets where they

26

4 Coding

depolarize other neurons, muscle, or gland cells to excite them or to inhibit them by holding the membrane potential of a target cell below threshold. Despite the importance of action potentials and the ease with which they can be monitored, their role in neural coding is uncertain. Action potentials are needed only when the distances between the relevant cells exceed a few millimeters; over smaller distances information can be conveyed passive electrical spread. Thus the neural processing carried out in tiny animals such as the roundworm C. elegans does not depend on action potentials. Nor are action potentials used in the human retina, a well-studied region of the brain where the distances between the neurons doing the processing are also small. Some interneurons in the human cerebral cortex also function without action potentials. Thus action potentials are primarily an elegant solution to the problem of rapidly linking cause and effect over distances where passive electrical conduction is inadequate. Neuroscientists interested in coding by action potentials have tended to divide into two camps. The “rate coding” camp favors the idea that the frequency of action potentials per unit time (the “firing rate”) conveys information. Conversely, in “temporal coding” theory the ordering of action potentials over time is thought to carry information. More plausible—and probably the majority view today—is that the significance of the messages being shuttled around the nervous system are determined by the relative activity of populations of neurons. When considering human brains and most others, putting all the money on one or a few neurons to carry out function would be ill advised. Redundancy is a boon and there is lots of evidence for it. For example, surprisingly large regions of the cortex of a rat can be destroyed without much influence on complex behaviors such as running a maze. It makes sense that information be distributed for resilience in the face of neural damage, as well as for generating optimal responses by using many different brain regions to optimize behavior. Rarely if ever would biological success be well served by responding to only a single stimulus parameter or a single modality of input information, let alone to the activity of one or a few nerve cells.

4.7 Conclusion Neural coding exists in the sense that the greater the rate of action potential generation in response to a stimulus, the more strongly a muscle, gland, or target neuron reacts. But how that fact relates to coding in electronic computers is a tougher question. If the operation of nervous is empirical, the concept of a code may be superfluous, an issue taken up in Parts V and VI.

Further Readings

27

Further Readings Greis D (1981) The science of programming. Springer, New York, NY Hunt A, Thomas D, Cunningham W (1999) The pragmatic programmer. From journeyman to master. Addison-Wesley Longman, Amsterdam Kernighan BW (1999) The practice of programming. Pearson, New York, NY Spratling MW (2017) A review of predictive coding algorithms. Brain Cogn 112:92–979

Part III

Neural Networks

Machines that solve problems empirically—whether biological or artefactual—are called “neural networks.” This alternative to following a series of rules can be slow and the strategies impenetrable. As a result, deriving answers in this way is unappealing to many computer scientists, leading to the up and down history of this approach. Recently, however, the trend has been decidedly upward.

Chapter 5

Neural Networks

5.1 Introduction A different strategy for solving problems uses artificial neural networks that operate by altering the connectivity of their elements based on failing or succeeding at some task. Despite some downsides—the need to train, slowness, inscrutable operation— this approach has proven to be remarkably powerful.

5.2 History The history of “computation” carried out by artificial neural networks is far more recent than algorithmic computation, dating back only some 80  years (the scare quotes emphasize that whether this approach qualifies as computation in the usual sense depends on how the term is defined). The generally recognized starting point is a paper published in 1943 by MIT psychologist/psychiatrist Warren McCulloch (his training was in medicine) and logician Walter Pitts. It may be hard to see how the straightforward idea that problems can be solved empirically by altering the strength of network connections emerged from the content: the article is a wild ride aimed largely at psychiatric issues, rife with equations and speculations of uncertain merit. McCulloch—described as a “rebel genius” in his biography—was also an alcoholic, leading his MIT colleague Norbert Weiner (who invented the field of feedback control called “cybernetics”) to express the hope that McCulloch would “drink himself to death.” Pitts evidently was a genius, but mentally unstable and did die of alcoholism at age 40. These failings notwithstanding, the paper pointed out that instead of depending on a series of logical steps implemented in a Turing machine, problems can also be solved using modifiable networks of connected units (“neurons” in their © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. Purves, Why Brains Don’t Compute, https://doi.org/10.1007/978-3-030-71064-4_5

31

32

5  Neural Networks

biologically inspired terminology). In this way the strength of the connections between the network elements (called “nodes”) changes progressively according to feedback from success or failure dealing with the problem at hand. The key attribute of such systems—which quickly came to be called “artificial neural networks” (ANNs or just “neural nets”)—is the ability to solve a problem without knowledge of the answer, the steps needed to reach it, or even a conception of how in principle the problem might be addressed. In effect, neural nets reach solutions by trial-and-error learning rewarded by success. The outcome is ever better responses generated by gradually building on the connectivity that mediated the improved “behavior.” Over a large number of trials the network output comes ever closer to producing the answer being sought. The gradually changing connectivity of the network—analogous to gradually modifying biological neural circuitry in evolutionary or lifetime learning—is entirely a consequence of the network’s experience.

5.3 A Problem for Artificial Networks In biological brains, anatomical, physiological, and biochemical evidence all indicate that feedback from behavioral outcomes—a “reward” of some kind—is needed for learning to be instantiated in neural processing machinery. Between the “lower order” regions that process sensory inputs and the regions ultimately responsible for the ensuing behaviors is an enormous amount of interposed circuitry whose function is not easily categorized. This “association cortex” in the brains of humans and other animals is presumably where most of the progressive changes in connectivity occur. In ways that are still not fully understood experience modifies the strength of the brain’s connectivity so that behavior is a little better the next time around (think of learning a language, how to play music, a sport, etc.). These mechanisms and the biological importance of input-output repetition (practice) are taken up in Part V.

5.4 Failure to Take Off Despite what should have been its strong appeal in principle, the concept of neural networks languished in the two or three decades that followed publication of the McCulloch and Pitts paper. One reason for this interval was presumably the rapidly increasing enthusiasm for algorithmic computing using electronic computers. The extraordinary progress described in Chaps. 3 and 4 in creating these machines and the coding languages that drove their operation understandably took center stage. It was not lost on many scientists, however, that while algorithmically grounded machines were referred to as “electronic brains,” they were a poor representation of biological brains and had little to say about how brains operate or the feats they accomplish. Neural networks are slow and how they were “computing” answers, if

5.5 Perceptrons

33

indeed they were computing at all in the usual sense, put off engineers, mathematicians, and logicians whose talents and interests lay elsewhere. Nevertheless, neural networks were clearly more “brain-like” than electronic computers, and people in several fields began to think about machine-biological parallels and how neural networks might be made to function better.

5.5 Perceptrons But reaching this goal was not so easy. For starters, the designers or programmers would have to provide feedback about whether the output the network was heading in the right direction, and it was not immediately clear how this could be done. An intersection between neural networks and electronic computers that had some appeal to both camps is that nodes—or neurons—might act as elements in a logic circuit giving rise to renewed interest in McCulloch and Pitt’s original idea. By 1950 it was apparent from electrophysiological studies that biological neurons algebraically summed excitatory and inhibitory inputs in generating outputs to other neurons and non-neural targets (e.g., muscles and glands). Moreover, action potentials with their all or none character traveling over axons by electrochemical means implied the use of a “code” not very different from the electronic 0 s and 1 s used by computers (see Chap. 4). In any event, these ideas laid the groundwork for creating simple neural networks that executed logical functions. The first of these networks were called “perceptrons.” Perceptons are neural networks that typically comprise elements in a two-layer array (Fig.  5.1). Although there is some controversy about who actually came up with this idea, a consensus is that psychologist Frank Rosenblatt devised the basic concept in 1958 while working at the Cornell Aeronautical Laboratory under the auspices of the Office of Naval research. The two-layer version represents a computational algorithm for the binary classification of sets according to “supervised learning” (i.e., the answer is known by the programmer and used to correct mistakes in the output). An example is differentiating dogs from cats. The supervisor knows the difference between these species and trains the perceptron with examples in which the network is told whether its output was right or wrong. Despite its limitations, Rosenblatt was quick to ballyhoo the perceptron in the media as a key step toward the creation of an “artificial brain,” leading to equally rapid blowback from other scientists. Nonetheless, in 1960 an electronic version was built by IBM to carry out image recognition. In addition to its limitation as a linear classifier, the perceptron’s abilities were restricted in other ways. A book by artificial intelligence enthusiasts Marvin Minsky and Seymour Papert in 1969 pointed out that a perceptron could not learn a key logical function (the Boolean “exclusive-or” function), hastening the already diminishing enthusiasm for this approach. Rosenblatt, who was undeniably creative, went on to study “memory transfer” by injecting rats with brain extracts (perhaps not a particularly good choice in retrospect).

34

5  Neural Networks Input

Associative units

Summation unit

A

a

A

a

A

a

A

a

A

a

Response unit

Σ

Fig. 5.1  The “Rosenblatt perceptron.” A randomly selected set of input units (“retinal cells” in Rosenblatt’s model) is connected to the layer marked “A” that creates associations. Each “A” unit is a building block whose connection “weights” could be 0 or 1 represented in (“a”). The output of the “A” units multiplied by their weight in “a” was fed to a “summation unit” which in turn drove the “response unit.” The task of the perceptron was to distinguish two sets of input patterns by associations that exceeded a threshold. Notice the absence of any feedback from the output

5.6 Conclusion Using artificial neural networks to solve problems began in the early 1940s and progressed more or less in parallel with the more influential advances in algorithmic computation, although at a much slower pace. The discrepancy between hyperbolic claims and the reality of early neural networks—together with the ongoing success of algorithmic computation—led to a decline in interest during the 1960s and 1970s. In the last 20 years, however, solving problems using artificial neural networks has revived and grown steadily.

Further Readings Anderson JA, Rosenfeld E (2000) Talking nets: an oral history of neural networks. MIT Press, Cambridge, MA Crevier D (1993) AI: the tumultuous search for artificial intelligence. Basic Books, New York, NY McCorduck P (2004) Machines who think, 2nd edn. A. K. Peters, Ltd., Natick, MA McCulloch WS, Pitts W (1943) A logical calculus of the ideas immanent in nervous activity. Bull Math Biophys 5:115–133 Minsky M, Papert S (1972) Perceptrons: an introduction to computational geometry, 2nd edn. MIT Press, Cambridge, MA Russel S, Norvig SJ (2003) Artificial intelligence: a modern approach, 2nd edn. Prentice Hall, Upper Saddle River, NJ

Chapter 6

Resurrection of Neural Networks

6.1 Introduction Renewed enthusiasm for neural networks in the 1970s was largely based on a computational means of altering the weights of network connections by a mathematical feedback technique called “back propagation.” This advance introduced a way of continuously modifying the weights of network connections based on feedback from the network outputs. Enthusiasm for this approach continues unabated today because, at least in part, this is what brains seem to be doing.

6.2 Error Correction by Back Propagation An obstacle holding back progress using artificial neural networks to solve problems was how to credit to successful outputs by changing the weights of network connections, the challenge back propagation addressed. Several people had suggested back propagation in the early 1960s as a useful strategy in the context of control theory (cybernetics), but Paul Werbos applied the concept directly to neural networks in his PhD thesis on economics in 1974. Back propagation, or “back prop” as it soon came to be called, was promoted more widely in the 1980s in an influential book on “connectionist” theory by David Rummelhart and James McClelland in 1986. Before long this method became routine in neural network theory and practice. Back propagation uses a “chain rule” that iteratively computes the “gradients” at each of the nodes in the hidden layers in a fully connected network like the one shown in Fig. 6.1. The strategy of ongoing error correction creates internal weights that match inputs with progressively more successful outputs. Based on the difference between the desired outcome and the actual outcome, back propagation assigns new weights throughout the network. To accomplish this a mathematical technique © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. Purves, Why Brains Don’t Compute, https://doi.org/10.1007/978-3-030-71064-4_6

35

36 Input layer

6  Resurrection of Neural Networks Hidden layer 1

Hidden layer 2

Hidden layer 3

Output layer

Fig. 6.1  A typical neural network comprising an input layer, an output layer, and several fully connected “hidden” layers. The common denominator of this and more complex artificial neural networks is a richly interconnected system of nodes or “neurons.” During training the strengths of the initially random connectivity of the nodes changes progressively according to the relative success of the trial-and-error responses indicated by the output, which is fed back to change the weights of the network’s connectivity. As a result, the system deals ever more effectively with the problem it has been given

called “gradient descent” identifies the direction that must be taken to reach an energy minimum, i.e., moving in the direction indicated by the slope and its derivative. The values obtained in this way specify the weights of network connections, continually improving the overall function of the network in its progress toward a “solution.” Back prop is supervised in that the programmer knows the error that specifies how to instantiate the weights which then allow the network to progress toward the solution of a complex problem. Back propagation, however, is not doing what evolution or lifetime learning accomplishes in biological neural networks. Although it works as a mathematical technique, back propagation is biologically implausible: brains would have to generate specific weights for each of an enormous numbers of neural connections by recurrent neuronal feedback, which is difficult to imagine even in the simplest animals. A further obstacle is that the answer to the problem being posed must be known to the experimenter in order to specify the error. These concerns about the plausibility of back propagation led to second decline in the enthusiasm for neural networks that lasted from the 1990s until the last decade or so.

6.4  Search Spaces

37

6.3 Unsupervised Credit Assignment There is, however, an alternative strategy that neural networks could use. Credit assignment in biology is based on success or failure established by evolution according to survival and reproduction. The equivalent in individual lifetimes is learning by trial and error rewarded by empirical success ultimately measured in the same terms. As animal phenotypes improve over the eons, there is no supervisor; the “supervisor,” so to speak, is simply behavioral success and ultimately the reproduction rate over geological and/or individual time. Since evolution includes the eventual appearance of nervous systems, it makes sense to regard unsupervised reward learning as the guiding principle underlying the operation of brains and the rest of the nervous system. Evolution can be simulated using genetic algorithms with reward (improved fitness) being established in much the same way it is in biology. Accordingly nothing stands in the way of exploring how brains might operate in this way (see Parts V and VI).

6.4 Search Spaces Another reason brains would be well served by trial-and-error learning rather than algorithmic computation is the size of the relevant “search spaces”—the number of possible pathways to the solution of the ecological problems that routinely confront us and other animals. For problems where the number of possible pathways to success is relatively small (e.g., 1010) computation following a set of rules can be an efficient way to proceed. But for problems where the “search space” of possible answers is very large (e.g., 10100 or more) proceeding by trial-and-error learning— the key feature of solutions achieved by artificial neural networks—seems the only viable option. Searching for answers algorithmically would simply take too much time and energy even with supercomputers or quantum computing as currently conceived. Since the number of routes to useful answers in real life is often large beyond reckoning, solving these challenges empirically seems the most plausible strategy for biological nervous systems to use. Moreover, proceeding by trial-and-error learning provides a degree of incremental success from the outset: the path to a solution begins to reveal itself from the very first trial even though an empirical alternative may never get to an “ultimate” answer. The more “tries” an agent makes, the closer it advances along a pathway to greater success even in situations like evolution where there is presumably no ultimate “right” answer. Whether artificial or biological a network that begins to accumulate useful information from the beginning has an obvious advantage. Moreover, experience in particular domains which takes biological organisms eons to accumulate in solving the problems presented in real life can be acquired by computers in days or even hours (see Part V).

38

6  Resurrection of Neural Networks

6.5 Conclusion Because the operations of artificial neural networks are carried out using computers, empirical solutions by reinforcement learning may seem a form of algorithmic computation. But to reiterate a point made in Chap. 1, algorithmic and empirical approaches are fundamentally different ways of solving problems. There are at least two a priori reasons why biology would evolve an empirical rather than an algorithmic strategy: (1) animals cannot measure the parameters that define physical reality; and (2) the problems encountered in life entail exceedingly large search spaces.

Further Readings Anderson JA, Rosenfeld E (2000) Talking nets: an oral history of neural networks. MIT Press, Cambridge, MA Nielsen MA (2015) How the backpropagation algorithm works. In: Neural networks and deep learning. Determination Press, San Francisco, CA. (free download) Rojas R (1996) The backpropagation algorithm. In: Neural networks: a systematic introduction. Springer, Berlin Rumelhart DE, Mcclelland JL (1986) Parallel distributed processing (2 vols). MIT Press, Cambridge, MA Werbos P (1974) The roots of backpropagation: from ordered derivatives to neural networks and political forecasting. PhD thesis, Harvard

Chapter 7

Learning Empirically

7.1 Introduction Vindication for the renewed enthusiasm in neural networks as a superior way to deal with complex problems has come largely from playing games. What began as a competition between human and machine “intelligence” has turned into a major effort to understand the power of trial-and-error learning and its relevance to animal brains. Although work exploring how algorithmic computation can win playing relatively simple games began decades ago, using neural nets in game play supports the conclusion that brains operate empirically.

7.2 An Algorithmic Beginning Artificial intelligence in the guise of mimicking computations that brains might be making dates from a conference of mathematicians, computer scientists, information theorists held at Dartmouth College in 1956. The consensus then was that the creating machine “intelligence” would be relatively easy. Sixty plus years on, however, this goal has not been met. The central problem now, as then, is that no one knows what intelligence or the operating principle of brain is. Nevertheless, success in game play has long pointed a way to explore the possibilities. One place to begin is with Rubik’s cube, the familiar game created by architect Erno Rubik in the early 1970s (Fig. 7.1). Since the elements that make up the cube can be manipulated in ~1019 possible ways, getting tiles into the desired configuration by fiddling is highly unlikely. However, by using reinforcement learning and self-play without specific instructions, computer scientists Alexander Shmakoff and Pierre Baldi showed that a simple algorithm can lead to a solution. Making all the faces the same color requires only 20 specific steps starting from any arrangement © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. Purves, Why Brains Don’t Compute, https://doi.org/10.1007/978-3-030-71064-4_7

39

40

7  Learning Empirically

Fig. 7.1  Rubik’s cube. Starting from a random mix of colored tiles on the eight faces of a cube— each of which can move independently—the goal is to make every face a single color

of the face colors. Kids today learn this sequence to impress their peers, and compete to see how fast these moves can be made (the current record is slightly less than 5 s).

7.3 Games that Cannot Be Won Algorithmically Far more interesting examples of the challenges facing biological organisms, however, are board games. The “game tree complexity” of tic-tac-toe—i.e., an estimate of the number of possible positions that would need to be evaluated to determine the worth of any initial move—is about 2 × 104. This number of possibilities is easily searched by an algorithm, as demonstrated by the chicken pecking one of the tic-­ tac-­toe squares in Fig. 7.2, “playing” in what amounts to a cleverly designed Skinner box. Following a set of rules in relatively simple games like tic-tac-toe, however, quickly runs out of steam as board games become more complex. To understand the advantages of an empirical approach playing board games, imagine the search spaces involved in more complex tests. There are several different ways to measure the complexity of more difficult games, leading to different estimates. But no matter the method used, as the complexity of board games grows, the relevant search spaces become unimaginably large. The game tree of checkers is estimated to be about 1020, of chess about 10120 and of Go an astonishing 10320. For comparison, the estimated number of atoms in the universe is on the order of 1087. Unlike tic-tac-toe, fully searching spaces of this magnitude is not feasible. There is, however, another way to meet such challenges. Spearheaded by a team at the artificial intelligence company Google DeepMind, a series of papers published over the last few years reported ongoing improvement using a strategy that now beats players at complex board games. Beginning in 2015 with a program that beat top human opponents at all 49 games in the Atari “2600” suite (challenges that include Pong, Space Invaders, and Pac-Man), the group progressed to a program

7.3  Games that Cannot Be Won Algorithmically

41

called AlphaGo that beat the European and World Champions at Go, a territorial game that is far more complex than chess (Fig. 7.3). The latest versions in this ongoing effort, called AlphaZero and MuZero easily beat the best players—human or machine—in chess and shogi (Japanese chess) as well as Go. That machines can beat expert human players in board games is not news. The achievement of this long-standing goal in computer science first attracted

Fig. 7.2  The search space of tic-tac-toe is small enough to allow an algorithm to always beat (or draw) a human opponent. The chicken has been trained to peck the square lights up on the floor of the box guided by an algorithm that successfully counters any move made by the player Fig. 7.3  The game of Go. This territorial board game renders algorithms useless in dealing with the search space of this game. If an algorithm evaluated a move once every nanosecond over the age of the Universe, it would still not have completed the task

42

7  Learning Empirically

widespread attention in 1997 when IBM’s program “Deep Blue” beat Garry Kasparov, then the world chess champion. Human players have since been shown to be weak opponents in such games compared to a variety of machine programs. What is different about the DeepMind’s current programs is that the neural networks used learn by trial and error. Previous game-playing machines, including “Deep Blue” and earlier versions in the AlphaGo series, used a combination of brute force logic, tree search, the history of successful moves made by experts in response to specific board positions, and other “hand-crafted” expedients. In contrast, AlphaZero uses only the empirical results of self-play. The starting point is simply a randomly initialized “deep” neural network (an artificial neural network with multiple hidden layers; see Fig. 6.1) that gradually improves using only the rules of the game and reinforcement learning. The connectivity underlying winning moves is instantiated in the network as it plays a competing machine setup in the same way. The machines playing millions of games over a relatively short time become unbeatable. The weights in the network, however, are still changed using back propagation (see Chap. 6). The relevance of all this to neuroscience is the size of the search spaces and the implications that follow. Organisms—and ultimately animals with brains—must contend with even larger search spaces as they play “the game of life” making “moves” that are rewarded by survival and reproductive success. Although the complexity of this biological challenge for humans is hard to imagine, it clearly dwarfs the complexity of board games like Go. Note further that understanding how the connectivity of a trained network achieves its goals may be impossible.

7.4 Conclusion The implication of this evidence is that animals contend with the challenge of surviving in the real world by generating input-output associations empirically rather than logically. If this mode of operation is what brains like ours are doing, then this strategy will have to be taken into account, the difficulty of exploring nervous system functions in this way notwithstanding. Solving problems by unsupervised reinforcement learning using neural networks has until recently been overshadowed by the immense success of rule-based computation which has proven its worth in numerous applications. The path to success in complex games, however, suggests that behavioral responses made empirically is the way animals optimize winning in life.

Further Readings Allis LV (1994) Searching for solutions in games and artificial intelligence. PhD thesis. Univ. Limburg, Maastricht Churchland PM, Churchland Churchland PS (1990) Could a machine think? Sci Am 262:32–37

Further Readings

43

Hsu F-H (2002) Behind deep blue: building the computer that defeated the world chess champion. Princeton University Press, Princeton, NJ Purves D (2019) What does AI’s success playing complex board games tell brain scientists? PNAS 116:14785–14787 Saxe A et al (2021) If deep learning is the answer, what is the question? Nat Rev Neurosci 22:55–67 Silver D, and 16 others (2017) Mastering the game of go without human knowledge. Nature 550:354–359

Part IV

Perception

The properties of local environments cannot be apprehended by biological sensing systems. Thus reactions to stimuli cannot be determined by rules that pertain to physical reality. How, then, can nervous systems generate appropriate behavioral responses in a world whose physical parameters are hidden? Explaining perceptual phenomena—the most thoroughly studied human responses to stimuli—is a critical test for any theory of neural operation.

Chapter 8

What We Perceive

8.1 Introduction An obvious way nervous systems differ from electronic computers is that brains give rise to perceptions, defined as what we subjectively experience, while computers do not. This fact, however, does not mean that neuroscientists know how perceptions arise, what they are or whether non-biological machines could ever give rise to them. For many investigators the nervous system is taken to construct representations of the physical world. Based on what has already been said, however, this option is implausible. Moreover, there are profound discrepancies between the physical sources of stimuli and what we perceive.

8.2 Traditional Assumptions The “information” processed by the brain is traditionally thought of as the output of “lower order” neurons that detect and encode the elemental properties of stimuli. This information is then decoded by circuits in “higher-order” brain areas, constructing neural representations that serve as a basis for motor, cognitive, or other behavioral responses. Since the mid-twentieth century, a major focus in neuroscience has been understanding the characteristics of neurons and neural circuits of experimental animals and humans that carry out this processing. An assumption underlying this project is that perception arises from neural mechanisms that encode the physical sources of sensory input, at the same time filtering out redundant or otherwise less important information. What remains is a “sparse” model or representation of external reality mediated by the population activity of neurons in the relevant brain regions (see Chap. 4). A further assumption

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. Purves, Why Brains Don’t Compute, https://doi.org/10.1007/978-3-030-71064-4_8

47

48

8  What We Perceive

in most accounts is that representing the world in neural terms is carried out by algorithmic strategies analogous to those used by computers. In late 1970s theoretical neuroscientist David Marr outlined this approach for visual system in a comprehensive computational theory that is still widely cited. Although other theories followed, Marr’s effort continues to stand as a singular attempt to explain the computational processes underlying perception rather than simply describing the pertinent anatomy and physiology of the systems and hoping that a rational understanding would emerge. Whether due to Marr’s influence or simply the general drift of thinking in neuroscience, determining how sensory inputs are detected, processed, and represented based on computational algorithms remains a central theme in modern neuroscience. What, then, is wrong with the idea that sensory systems and their higher order effects generate perceptions of the world as it is, and that this seemingly useful goal is achieved by neuronal hardware that detects the elemental features of stimuli in order to construct a representation of the external world based on a set of logical rules?

8.3 The Dilemma As already pointed out, the problem with these assumptions is that neither the properties of the world nor their significance for behavior is specified by the patterns of energy that impinge on biological sensors. Stimuli arising from electromagnetic radiation, atmospheric pressure changes, odorant molecules, or any other form of biologically useful energy can always have arisen from many combinations of physical conditions in the environment. This fact precludes mapping stimulus features back onto physical reality, even if that were desirable. The information needed to do this is simply not available to biological sensing systems, even though “knowledge” about physical environment seems what animals need to succeed in a complex and dangerous world. The primary concern with this or any rule-based scheme is that the sources of stimuli (and thus their significance for subsequent behavior) are unknowable. Any element of a stimulus could have arisen from many—indeed infinitely many—different objects and conditions in different circumstances (see, for example, Fig. 2.1). It is therefore difficult to understand how information provided to the central stations of peripheral sensors could generate percepts and other behavioral responses that succeed in the real-world.

8.4 A Solution As implied in Chap. 7, an alternative would be brains that operate empirically by accumulating and updating neural function according to the relative success or failure of responses made by trial and error. By continually refining the link between

8.6 Conclusion

49

percepts and responses to stimuli on the basis of the success or failure in an evergrowing body of experience, successful responses in any situation could be made without information about physical reality or how to react to a given situation in logical terms. The brain—or any other machine—would not need rules that might be followed to produce a useful reaction. Rather, the behavioral responses would be determined by the relative success or failure of all previous reactions to the same or similar inputs. Given some way of profiting from experience—i.e., learning by retaining neural connections and functions that worked—performance would gradually improve as feedback from particular responses progressively altered the organization of the nervous system.

8.5 Caveats At the outset this empirical strategy would generate chance-level success since little or no experience about useful responses would have been incorporated as a guide to behavior. Over time, however, performance would gradually improve, and, given enough experience, would ultimately become an effective way of operating in the world. Each response would represent the weighted influence of experience of the species and the individual up to that moment, generating useful responses even in exceptional or even unique circumstances. Even in this more sophisticated state, however, an “analysis” of the altered structure of the brain or non-biological system, depending on the complexity of the challenge, might give little or no insight into how the problem had been mastered. Despite the widespread opinion that a full understanding perception will only emerge from a deeper and more thorough analysis of the cellular and molecular mechanisms underlying neural processing, deciphering the significance and purpose of brain circuitry may advance more efficiently once this overarching strategy of perception is accepted. If experience is indeed the Rosetta stone for understanding how neural responses are generated, then what observers perceive should be explainable in these terms.

8.6 Conclusion If the idea that percepts and other behavioral responses are the result of accumulated experience with the inherently unknowable sources of stimuli is correct, then perceptions should be predictable on this basis. Rather than ignoring the puzzling discrepancies between stimuli and percepts described in the following chapters, explaining them in empirical terms would validate that this is the strategy brains use.

50

8  What We Perceive

Further Readings Berkeley G (1709/1976) A new theory of vision. Everyman’s Library, Dutton, New York, NY Heyer D, Mausfeld R (2002) Perception and the physical world: psychological and philosophical issues in perception. John Wiley and Sons, New York, NY Purves D, Lotto RB (2011) Why we see what we do redux. Sinauer, Sunderland, MA Purves D, Monson BB, Sundararajan J, Wojtach WT (2014) How biological vision succeeds in the physical world. PNAS 111:4750–4755 Robinson JO (1998) The psychology of visual illusions. Dover, New York, NY Rock I (1995) Perception. Scientific American Library, New York, NY

Chapter 9

Spatial Intervals

9.1 Introduction If nervous systems operate empirically, then accumulated data that reflect what we and our predecessors have experienced should explain the full-range perceptual phenomena that psychophysicists have described over the last century or more, ideally in quantitative terms. One such category is the perception spatial intervals. Although perceiving this aspect of the physical world in a way that enables successful behavior is critical, there striking differences between what we see and physical reality.

9.2 Discrepancies Discrepancies between measurements of intervals made with rulers and the corresponding perceptions provide plenty of puzzles in rationalizing perceived geometry. Figure  9.1 shows that lines generated by objects of different lengths in different orientations and at different distances can project the same interval in retinal images. How, then, can observers respond usefully in the face of such ambiguity? If neural circuitry is determined empirically, then what we perceive should be accounted for by accumulated experience. By sampling data sets pertinent to the relevant category of experience, it is possible to ask whether experience does indeed determine the intervals we see. To understand this approach, take the perceived length of a line compared to its length in retinal images. Because the possible sources of the intervals are conflated, it would be of little use to perceive the length of the line on retina as such. It would make sense, however, to perceive lengths determined by the frequency of occurrence of a particular length in the retinal image

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. Purves, Why Brains Don’t Compute, https://doi.org/10.1007/978-3-030-71064-4_9

51

52

9  Spatial Intervals

Fig. 9.1  The uncertain sources of object geometry projected onto the retina. The same projection can be generated by objects of different sizes and orientations at different distances from an observer

Fig. 9.2  An empirical scale based on the frequency of occurrence of line lengths in retinal images (red) compared to a scale of lengths measured with a ruler (black). Because any given line length (e.g., 25 units) has a different rank on these two scales (dotted lines), intervals perceived empirically should always differ from the length of a line measured physically

relative to all the projected lengths in the same orientation experienced by human observers. If we and other animals use this strategy, the lengths seen should differ predictably from measured lengths (Fig. 9.2). For instance, if in past human experience 25% of the linear distances in retinal projections generated by objects in the world were shorter than or equal to the length of the stimulus line in question, the rank of that projected length on an empirical scale would be the 25th percentile. A consequence of this way of perceiving spatial intervals would be universal discrepancies between measurements of intervals made with rulers and their perception. Perceiving

9.3  Determining Frequencies of Occurrence

53

qualities according to their empirical rank would, however, allow successful behavior despite the different geometry of the world. The discrepancies would be signatures of the way nervous systems contend with objects they cannot measure.

9.3 Determining Frequencies of Occurrence This explanation can be assessed by tallying the frequency of occurrence of linear projections generated by physical objects measured in laser-scanned scenes (Fig. 9.3). The frequency of projections of particular lengths and orientations on the retina depends on the lengths, orientations, and distances of lines in the physical world. In an empirical framework, this accumulated experience will determine what observers see. If this idea is right, then the relevant probability distributions should predict the psychophysical results.

Fig. 9.3  The frequency of occurrence of lines projected on the retina arising from linear sources in the physical world. (a) The pixels in an image from the database of laser-scanned scenes are represented diagrammatically by the grid squares; the connected black dots indicate a series of templates used to determine the frequency of occurrence of linear projections at different orientations in images generated by lines in the world. (b) Examples of templates overlain on a typical image in which the corresponding distance and direction of each pixel are known. White lines indicate sets of points that corresponded to straight lines in the world, and red lines indicate sets that did not. By repeating such sampling millions of times in natural images, the frequency of occurrence of straight-line projections of different lengths in different orientations projected onto the retina can be determined

54

9  Spatial Intervals

9.4 An Example It has long been known that the perceived length of the same projected line varies as a function of its orientation (Fig. 9.4): the same line looks longer when presented vertically than horizontally. Oddly, however, the maximum length is seen when the stimulus line is oriented about 30° from vertical. How, in empirical terms, can this peculiar psychophysical result be explained? As shown in Fig.  9.4, the apparent length of a line projected on the retina is indeed predicted by the rank of projected lines on an empirical scale determined by the frequency of occurrence experienced by humans over evolutionary and individual time.

9.5 Explanation Why, then, are there more sources in the world—and thus a greater frequency of stimuli—of relatively short vertical lines compared to horizontal lines? And why are there even more sources that project as relatively shorter lines 20°–30° away from vertical? The answer is that straight lines in the physical world are typically components of planes, a statement that may seem odd since we are very much aware of lines that form contrast boundaries (e.g., the edges of things). But whereas lines generated by contrast obviously provide useful information about edges, surfaces are by far the more frequent sources of geometrical straight lines that we experience. Thus when considering the probability distribution of the physical sources of straight lines that project onto the retina in different orientations, the most important factor is the extension of surfaces in the world. Horizontal line projections in retinal images typically arise from the extension of planar surfaces in the horizontal axis, whereas vertical lines are typically generated by the extension of surfaces in either the vertical or the depth axes. Vertical line projections arising from the extension of surfaces in depth, however, are inherently limited because the depth axis is perpendicular to the image plane: such lines on planes are thus foreshortened by perspective (i.e., they produce shorter lines on the retina). An inspection of the world also shows that the extension of surfaces in the vertical axis is limited by gravity, adding to the prevalence of shorter vertical projected lines. Since neither of these restrictions affects the generation of horizontal line projections, humans experience more short vertical line projections than short horizontal ones. As a result, a vertical line on the retina will always have a higher empirical rank than a horizontal line of the same length. Lines shorter than the vertical line in Fig. 9.4a will have been projected on the retina more often, giving the vertical line a relatively higher rank in the “perceptual space” of apparent lengths than a horizontal line of the same length. By the same token, objects extending above the Earth’s surface tend to be taller than wider, explaining the maximum line length at 20° or 30° off vertical.

9.5 Explanation

55

Fig. 9.4  Differences in the apparent length of the same line as a function of its orientation. (a) A horizontal line looks somewhat shorter than vertical or oblique lines, despite the fact that all the lines are physically identical. (b) Graph of the perceived lengths of the lines in (a). The maximum length seen occurs when the line is oriented approximately 30° from vertical, where it appears 10–15% longer than the minimum length seen when the orientation of the line is horizontal. (c) Prediction of perceived lengths based on the frequency of occurrence of oriented lines projected onto the retina from natural scenes (see Fig. 9.3)

56

9  Spatial Intervals

9.6 Conclusion Although a ruler accurately measures physical length, the visual system cannot do this. If, however, the visual brain orders the perceptions of projected intervals according to accumulated experience, this deficiency would be resolved.

Further Readings Howe CQ, Purves D (2005) Perceiving geometry: geometrical illusions explained by natural scene statistics. Springer, New York, NY. Chapter 3 Howe CQ, Purves D (2005) The Müller-Lyer illusion explained by the statistics of image-source relationships. PNAS 102:234–1239 Purves D, Lotto RB (2011) Why we see what we do redux: a wholly empirical theory of vision. Sinauer Associates, Sunderland, MA. Chapter 4

Chapter 10

Angles

Another challenge in rationalizing geometrical percepts empirically is the way we perceive angles made by two straight lines that meet at a point. Like the apparent length of lines measured with a ruler, intuition suggests that angles should scale directly with measurements made with a protractor. This is not, however, what we see.

10.1 Perceived Angles Observers overestimate the magnitude of acute angles and underestimate obtuse ones by a few degrees (Fig. 10.1). Just as the physical source of a projected linear interval is ambiguous, an angle projected on the retina could arise from many real-­ world angles. Like lengths, however, perceiving angles based on accumulated experience allows observers to contend with a physical world that cannot be measured by biological sensors.

10.2 Determining the Frequency of Angle Sources Understanding angle perception in empirical terms depends on much the same ideas as understanding the perception of line lengths. The frequency of occurrence of projected angle generated by the physical world can also be determined by laser range scanning. The first step is to identify regions of the scanned scenes that contain a valid physical source of the two straight lines that form an angle. The frequency of occurrence of a valid angle can then be determined by overlaying templates in different orientations on natural images (Fig. 10.2a). By systematically repeating this procedure, the relative frequency of occurrence of different angles projected onto the retina can be tallied (Fig. 10.2b). © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. Purves, Why Brains Don’t Compute, https://doi.org/10.1007/978-3-030-71064-4_10

57

58

10 Angles

Fig. 10.1  Discrepancies between measured angles and their perception. These psychophysical results show that acute angles are systematically overestimated and obtuse ones are underestimated

Fig. 10.2  Measuring the frequency of occurrence of angles generated by the physical world. (a) The pixels in an image are represented by grid squares; the black dots indicate a reference line in the template and the red dots sampling for a second line that makes an angle with the reference line. (b) The white line indicates the occurrence of valid reference lines the scene. Blowups of the boxes show examples of the second template (red) overlain on the same area of the image to test for the presence of a second straight line making an angle with the reference line

10.3  Reason for Different Frequencies of Angle Projections

59

Fig. 10.3  Frequency of occurrence of angles projected onto the retina. The graphs show the frequency of occurrence of projected angles with different orientations of the sampling template (indicated by the icons below the graphs). The upper row shows the results obtained from natural scenes and the lower row the results from environments that contained some (or mostly) human artifacts

The frequency of occurrence of different angles projected on the retina is shown in Fig.  10.3. Regardless of the orientation of the reference line (indicated by the black lines in the icons under the graphs) or the type of real-world scene, the likelihood of angle projections is always least for 90° and greatest for angles that approach 0° or 180°. Thus the occurrence of real-world sources of an angle decreases as the two lines become increasingly perpendicular.

10.3 Reason for Different Frequencies of Angle Projections The bias evident in these statistical observations can, like the biased projection of line lengths in different orientations, be understood by considering the provenance of straight lines in the physical world. Intersecting straight lines, like individual lines in Chap. 9, are typically components of surfaces. Accordingly, the region of a surface that contains two physical lines whose projections intersect at 90° will, on average, be larger than a surface that includes the source of two lines of the same length intersecting at any other angle (Fig.  10.4). Since larger surfaces include smaller ones, smaller planar surfaces in the world are inevitably more frequent than

60

10 Angles

Fig. 10.4  The physical source of two lines of the same length intersecting at or near 90° must be a larger planar surface (dashed line) than the source of the same two lines making larger or smaller angles. This geometrical fact explains the lower probability of 90° projections in Fig. 10.3 compared to other angles

the larger ones. In consequence sources capable of projecting angles at or near 90° are less prevalent in human experience than the sources of any other angles.

10.4 The Systematic Misestimation of Angles If anomalous perceptions of angle magnitude in Fig.  10.1 are generated by past experience with the frequency of occurrence of projected angles, then the perception of any angle should always accord with its empirical rank in human experience. The rank of any projected angle between 0° and 90° is shifted slightly in the direction of 180° compared to measurements of its physical sources, whereas the opposite is true for any angle between 90° and 180° (Fig.  10.5a). The upshot is that accumulated experience with the relative frequency of angle projections on the retina generated by the geometry of the world predicts the psychophysical results (Fig. 10.5b, c).

10.5 Alternative Explanations As might be expected, many theories have been proposed over the last century or more to explain the anomalous perception of intervals and angles. In the nineteenth century attempts to rationalize these peculiarities included asymmetries of eye anatomy, the ergonomics of eye movements, and cognitive compensation for foreshortening in perspective. More recent investigators have suggested that geometrical perceptions of geometry arise from statistical inferences. For example, the anomalous perception of line length in the Mueller-Lyer illusion has been taken to arise from the different distances implied by angles at real-world corners, the assumption being that convex corners implied by the arrow heads would have been nearer to observers than concave corners implied by arrow tails. Although such explanations are empirical, analysis of real-world geometry contradicts intuitions that seem

10.5  Alternative Explanations Fig. 10.5 Predicting perceived angles based on the frequency of occurrence of retinal images generated by real-world sources. (a) The red curve shows the frequency of occurrence (expressed as cumulative probability) of projected angles derived from the data in Fig. 10.3. For comparison, the black line shows the frequency that would be generated if the probability of any projected angle was the same (inset). (b) The red curve shows the perception of angle magnitude predicted from the information in (a) (the dashed diagonal indicates the angle projected on the retina). (c) The predictions in (b) compared to psychophysical measurements of angle perception in Fig. 10.1

61

62

10 Angles

obvious. Laser scanning of buildings and rooms shows that there is no significant difference in the distance from observers of convex and concave corners. Neurobiological explanations of these effects have considered the receptive field properties of visual neurons, suggesting, for example, that lateral inhibitory effects among orientation-selective cells in the visual cortex could underlie the misperceptions in Fig. 10.1. In this interpretation the perception of an angle might differ from its actual geometry because orientation-selective neurons co-activated by the two arms of the angle inhibit each other. This effect would shift the distribution of cortical activity toward neurons whose respective selectivity would be “farther apart,” thus accounting for the perceptual overestimation of acute angles. But no evidence for any of these ad hoc explanations has been forthcoming.

10.6 Conclusion An empirical explanation of perceived lines and angles includes a range of geometrical phenomena that need to be accounted for (only a few examples have been described here). An analysis of the accumulated experience with the frequency of projected angles can, like the frequency of projected lines, produce successful behavior without physical measurements.

Further Readings Howe CQ, Purves D (2005) Perceiving geometry: geometrical illusions explained by natural scene statistics. Springer, New York, NY. Chapter 4 Howe CQ, Purves D (2005) Natural scene geometry predicts the perception of angles and line orientation. PNAS 102:1228–1233 Purves D, Lotto RB (2011) Why we see what we do redux: a wholly empirical theory of vision. Sinauer Associates, Sunderland, MA. Chapter 5

Chapter 11

Lightness and Darkness

11.1 Introduction Of the basic qualities that define perception the most important are darkness and lightness. An animal may see forms poorly, have little or no color vision, minimal depth perception, or even be effectively blind (e.g., some species of bats, moles, and mole rats). But most animals, many plants and even bacteria discriminate light and dark.

11.2 Seeing in Black and White “Lightness” and “darkness” are the terms applied to the appearance of surfaces that reflect light; the terms “brightness” and “dimness” refer to the appearance of sources of light such as a light bulb. The physical measure of lightness-darkness and brightness-dimness is luminance, which refers to the amount of light falling on a surface determined by a photometer. Lightness, darkness, brightness and dimness, however, are perceived qualities. These responses to stimuli are subjective and can only be evaluated by asking observers to report the appearance of one surface or light sources relative to another (or on an imagined scale over the range of the quality in question). A logical expectation is that lightness or brightness should closely follow the luminance of a stimulus; that is, the more light falling on some region of the retina, the lighter or brighter the target in the scene should appear and vice versa. But that expectation is not met. Indeed, we never see luminance as a photometer measures it. A simple example of the disconnect between lightness-darkness and luminance is shown in Fig. 11.1. The two central circles have the same measured luminance, and thus return the same amount of light to the eye. They are nevertheless perceived © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. Purves, Why Brains Don’t Compute, https://doi.org/10.1007/978-3-030-71064-4_11

63

64

11  Lightness and Darkness

Fig. 11.1  Example of the discrepancies between the measured luminance and the perception of lightness-darkness. When measured with a photometer, the light coming to the eye from the circles in the dark and light surrounds give identical readings. Nonetheless the target circle on the dark background appears lighter than the target on the light background. The key shows that when the targets are presented on the same background they look the same

differently, the patch on the darker background appearing lighter or brighter than the same patch on the lighter background (in the jargon of the field, this phenomenon is called “simultaneous lightness/brightness contrast”).

11.3 A Possible Physiological Explanation The subjective values we see are generated by the neurons in the visual system, and by the 1950s physiologists were hard at work studying these cells in experimental animals. Based on the evidence that emerged, one way of rationalizing the effects elicited by the stimulus in Fig. 11.1 is to suppose them an incidental consequence of neuronal processing. The central region of the receptive fields of retinal output cells were already known have a surround with an opposite functional polarity, an organization that presumably enhances the detection of the edges (Fig. 11.2). As a result, retinal neurons whose receptive fields lie over a light-dark boundary but with their central regions either within or without the dark region of an image would be expected to generate action potentials at a somewhat different rate. This observation made it attractive to suppose that the patch on the dark background in Fig.  11.1 looks lighter/brighter than the patch on the light background because of this difference in the retinal output to the rest of the visual brain.

11.4  A Different Approach

65

Fig. 11.2  The receptive field properties of retinal neurons and their output to processing centers in the rest of the visual system. The upper panel shows the receptive fields of several retinal output neurons (called “ganglion cells”) overlying a light–dark boundary. The lower panel shows the different firing rates of ganglion cells as a function of their position with respect to the boundary. In principle, this difference might cause the lightness/brightness contrast effects in Fig. 11 as an incidental consequence of neural processing at the input stages of vision

11.4 A Different Approach But that explanation is incorrect. In 1929, the German psychologist Adhemar Gelb described a room in which he hung a piece of paper that was illuminated by a hidden lamp. Depending on the level of the illumination of the objects in the rest of the room, the piece of paper could be made to look nearly white or nearly black, despite the same amount of light reaching the observer’s eye from the paper. Figure 11.3 shows similar effect generated by presenting patches with identical luminance in a scene that implies different illumination, as in Gelb’s room. The implication is that the different lightness or brightness values perceived in response to the same luminance of a patch in a light versus a dark surround—or in any stimulus pattern—are determined by accumulated experience with natural surfaces in different contexts, and not by the luminance of surfaces as such.

66

11  Lightness and Darkness

Fig. 11.3  Using scenes to show that lightness and brightness are determined by accumulated experience. (a) Patches of the same luminance (the diamonds) are located in surrounds similar to those in Fig. 11.1. In this case, however, they are presented with contextual information implying that the left diamond lies in shadow and the right one in light. (b) The same test targets and surrounds presented as if they were painted on surfaces that are identically illuminated. (c) Difference in the lightness of the identical test targets and surrounds reported by observers. The upper bar shows the average adjustments made to equate the lightness of the diamond on the dark surround with that of the diamond on the light surround in response to scene (a). The lower bar shows the adjustment made to equalize the appearance of the two diamonds in response to scene (b)

Further Readings

67

11.5 Conclusion If the lightness-brightness values we see were simply proportional to luminance values, the result would be a poor guide to successful behavior. However, if perceptions of lightness-darkness and brightness-dimness are generated empirically—i.e., by trial-and-error—the problem could be circumvented. Cumulative experience would determine the anatomical and physiological links between sensory inputs and behavior, as explained in the next chapter.

Further Readings Morgenstern Y, Rukmini DV, Purves D (2014) Properties of artificial neurons that report lightness based on accumulated experience with luminance. Front Comput Neurosci 8:134 Ng C, Sundararajan J, Hogan M, Purves D (2013) Network connections that evolve to circumvent the inverse optics problem. PLoS One 8(3):e60490 Purves D, Lotto B (2011) Why we see what we do redux: a wholly empirical theory of vision. Sinauer Associates, Sunderland, MA. Chapter 2

Chapter 12

Empirical Ranking

12.1 Introduction Although stimuli are typically thought to convey a representation of the local ­venvironment, that idea is not supported by the evidence described in the preceding chapters. Nonetheless, because humans and other animals must succeed in the physical world, stimuli provide something that promotes apposite perceptions and behaviors. But if not the physical properties of the environment, what information do perceptions and other responses to stimuli rely on?

12.2 Stimuli as Recurrent Patterns Taking vision as example, because images cannot specify the the world—and because natural images activate the full array of retinal receptors in exactly the same way only rarely—biological vision presumably evolved to depend on simple recurring patterns in images to accumulate empirical information. As implied in the last few chapters, since the frequency of occurrence of stimulus patterns is systematically related to the world sensory systems can take advantage of this relationship to shape the connections underlying perceptions that work. The utility of responses in this scenario are not evaluated by whether they convey the measured properties of reality but rather by the degree to which they ultimately promote survival and reproduction. Although this independence from the measured properties of the environment relying on feedback from reproductive success may seem odd, it is commonplace in objective–subjective relationships in neurobiology. Consider, for example, stimuli that elicit autonomic responses and an emotion. A loud, unexpected noise elicits vasoconstriction, piloerection, pupillary dilation, and a sense of fear. These © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. Purves, Why Brains Don’t Compute, https://doi.org/10.1007/978-3-030-71064-4_12

69

70

12  Empirical Ranking

autonomic behaviors and the related emotion evolved because they promote biological success. Even the most subtle neural responses to complex stimuli can be interpreted in this way.

12.3 Luminance and Lightness as Examples As indicated earlier, the significance of a stimulus for behavioral success cannot be conveyed by luminance as such: the light falling on any part of the retina is the product of illumination, surface reflectance, atmospheric transmittance, and these can combine in innumerable ways in any particular image. By relying on experience, however, the lightness values elicited by the luminance of any element in a stimulus pattern can be ranked according to biological success in the past. Whereas perceiving lightness values in proportion to measured luminance values on the retina would not be much help, perceiving lightness values ranked according to their impact on biological success would. Take, for example, the two patterns in Fig. 11.1 in which the central patches look differently light even though their luminance is the same. Figure 12.1 explains why this is so. In natural scenes central luminance values that are the same as the surround occur more often than those that are not. Although a multitude of factors are at play, several aspects of the natural world underlying this difference in the frequency of occurrence of such patterns are plain: (1) nearby surfaces in the world tend to have similar reflectance properties because they are often made of the same stuff; (2) nearby surfaces tend to be similarly illuminated. As indicated in the figure, perceptions (lightness in this case) based on ranks of the occurrence stimulus patterns on an empirical scale lead to biological success. If the percentile rank of a stimulus on such scales is high, more stimuli will have occurred below the rank of that stimulus than above it, the target patch should appear lighter and vice versa. Ranking the luminance values on this basis leads to behavioral success despite the inability of the visual system to measure or otherwise recover the physical properties of objects and conditions in the environment. Once evolution and lifetime learning have linked stimulus patterns with successful behaviors by reinforcement of trial-and-error responses, the frequencies of occurrence of those patterns determine what we perceive.

12.4 Explaining More Complex Patterns The empirical rank of the frequencies of co-occurring luminance relationships in natural scenes can also explain perceptions of complex stimuli that stymie other attempts at rationalization.

71

12.4  Explaining More Complex Patterns

2

3

Frequency of Occurrence of Central Luminance Values (%) 20 Stimulus pattern

T 10

5 3

0

Enhanced Survival

T

Darker

Luminance

2

2

Feedback

Lighter

100

40 20

?

Central Lightness Value Elicited (Percentile Rank)

Percentile rank of target

1

Lighter

100

5

?

T 20

40 20 10

Percentile rank of target

3

0

Enhanced Survival Darker T

Luminance

Fig. 12.1  Diagram illustrating why the lightness values perceived at the center of the two stimulus patterns at the left of the diagram differ. The fact that sources in human visual environment are typically made of the same material in similar illumination means that the luminance value of the central square in the dark surround will be lower, on average, than that luminance value in center of the lighter surround. For biological success the perception of lightness is assigned according to the relative rank of the frequency of occurrence of the luminance values experienced at this locus rather than by the luminance of the central patches

Consider, for instance, the perception of the stimulus in Fig. 12.2. By sampling the images arising from natural environments using the relevant template configurations, the percepts elicited can be explained on the same empirical basis. Fig. 12.3 shows that when targets have an intermediate range of luminance values (i.e., between the luminance values at the two crossover points), the percentile rank of the target luminance that abuts the dark rectangles laterally (red line) is higher than the percentile rank when the targets abut the two light rectangles laterally (blue line). If the percentile in the frequency of occurrence of target luminance values within any specific pattern determines the lightness values perceived, the target with an intermediate luminance on the left in Fig. 12.2 should appear lighter than the equiluminant target on the right, as it does. The upshot is that the empirical rank of luminance values in any stimulus pattern—and thus the lightness values perceived—bear no direct relationship to the properties of sources in the physical world. Rather it is the frequency of occurrence of the luminance values in stimulus patterns that determines perceptions of lightness and darkness.

12  Empirical Ranking

72

Fig. 12.2  The perception of complex luminance patterns rationalized by empirical ranking. The surround of the four gray targets on the left is more luminant overall than the that of the three identical patches on the right. The targets on the left, however, appear lighter than the identical targets on the right. This perception is opposite of the effect elicited by the stimulus in Fig. 11.1 100

Percentile rank

80 60 40 20 0

0

50

100 150 Luminance

200

250

Fig. 12.3  Empirical explanation of the perceptions elicited by the stimulus in Fig. 12.2. See text for explanation. Note that these empirical ranks reverse when the luminance of the stimuli are higher or lower, causing to perceptions to reverse as well

12.5 Conclusion This understanding of perception abandons the assumption that what we see is based on the properties of the physical world. In empirical ranking theory, the perceptual quality elicited by any particular stimulus is determined by the relative

Further Readings

73

frequency of occurrence of that parameter in relation to all the other instances of the same pattern witnessed over evolutionary and individual time. Empirical ranking thus differs from the idea that percepts correspond to physical source of a stimulus, the most likely source of the stimulus, or the receptive field properties of neurons that process sensory input.

Further Readings Purves D, Lotto B (2011) Why we see what we do redux: a wholly empirical theory of vision. Sinauer Associates, Sunderland, MA. Chapter 7 Purves D, Monson BB, Sundararajan J, Wojtach WT (2014) How biological vision succeeds in the physical world. PNAS 111:4750–4755

Chapter 13

Color

13.1 Introduction Color vison confers a modest advantage to animals that have this ability in that only species with color vision can distinguish image regions that have the same luminance values. Our sense of color provides further evidence that brains work empirically.

13.2 Seeing Color The perceptions of lightness and brightness concern what we see in response to light that stimulates the three human cone types more or less equally. Seeing color concerns the perceptual qualities generated when the energy in a stimulus pattern is unevenly distributed. Colors—qualitatively described in terms of hue, saturation, and lightness—are created by brains from whole cloth for reasons of biological advantage, just as nervous systems create all other perceptual qualities (Fig. 13.1). Much of the fascination with color over the centuries stems from the puzzles it presents. For instance, when going from red to green the perception of color must pass through the primary category of either blue or yellow. Moreover, as indicated by the black dots in Fig. 13.1, observers perceive a particular red, green, blue, or yellow as unique; that is, a value in each of the four color domains perceived as having no admixture of any other color category (in contrast to the way observers see orange as a mixture of red and yellow, or purple as a mixture of blue and red). The different activation of the three retinal cone types by light offers no explanation of these perceptual phenomena; nor is it apparent why we should see these four “primary” colors.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. Purves, Why Brains Don’t Compute, https://doi.org/10.1007/978-3-030-71064-4_13

75

76

13 Color

Fig. 13.1  Diagram of human perceptual “color space.” The vertical axis in the upper panel indicates that any particular light intensity evokes a corresponding sense of color lightness/brightness; movement around the perimeter of the relevant plane corresponds to changes in hue, and movement along the radial axis corresponds to changes in saturation (i.e., the progressive approximation to gray). The lower panel is one of the planes face on. Each of the four primary color categories—red, green, blue, and yellow—elicit the sense of a unique hue (black dots), i.e., a color percept that has no apparent admixture of the other three color categories

Another challenge is explaining color constancy and contrast. Under the same illumination, say white light, a patch in a collage of colored surfaces—for instance one that looks reddish—provides a substantially different spectral return to the eye than a surface that looks yellowish, as would be expected from the different physical properties of the two surfaces (Fig. 13.2a). But if the illumination of the surfaces is adjusted so that the yellowish surface now provides exactly the same light coming to the eyes as the spectrum that originally came from the reddish surface, the surfaces continue to look reddish and yellowish, respectively (Fig. 13.2b). Common sense implies that the yellowish surface in the collage should now look the reddish and that the color of the reddish should also have changed. But that is not how they appear.

13.2  Seeing Color

77

Fig. 13.2  Color constancy. (a) The appearance of a collage when illuminated by white light. Because patches 1 and 2 in the key below have different physical properties, they return different spectral stimuli to the eye; and, as expected, they look differently colored. (b) The same collage as it would appear after readjustment of the illumination such that the spectrum returned from patch 2 is now the same spectrum initially returned from patch 1. Despite the change in the spectral information coming from the patches, patch 2 continues to look reddish and patch 1 continues to look yellowish

Some investigators have supposed that the basis of these effects is “adaptation” to the predominant spectral contrasts in a scene. But as in the perception of lightness and darkness, other stimuli elicit perceptions that are inconsistent with this explanation. In Fig.  13.3, for example, the spectra of the circles in (a) and the crosses in (b) are identical in all four panels; and the spectra of the orange and yellowish backgrounds in the upper and lower panels are also the same. Nevertheless the targets on the left in both the upper and lower panels appear to be about the same

78 Fig. 13.3  Color contrast and constancy are not due to adaptation. (a) A color contrast stimulus in which two identical central targets embedded in different spectral surrounds appear differently colored. (b) In this configuration, the central targets are physically the same as those in the upper panels (see the key below), but the backgrounds have been switched. Despite the switch, the perceived colors of the left and right targets are about the same in the upper and lower panels

13 Color

13.3  Color Explained Empirically

79

color, as do the targets on the right. These similar color perceptions of the targets despite the fact that the surrounds are opposite in the upper and lower panels rules out adaptation.

13.3 Color Explained Empirically If nervous systems operate empirically, these puzzling effects should be explained by accumulated experience with spectral patterns. The idea is that the way we see color is also the result of trial-and-error experience with spectral relationships over evolutionary and individual time, thus circumventing our inability to apprehend the physical world as such. As in explaining black and white phenomena in Chaps. 11 and 12, perception is generated by associating the frequency of occurrence spectral stimuli with perceptual responses that worked in the past. If this idea is correct, then the nature of accumulated empirical information should explain contrast and constancy. This explanation is summed up in Fig. 13.4. Although the spectra coming from the central squares on the two faces of the cube in Fig. 13.4a are identical, their

Fig. 13.4  Color anomalies explained empirically. The two panels demonstrate the effects on color perception when two physically similar target surfaces (as in a) or two physically different target surfaces (as in b) are presented in the same context. Because the spectral information in the scene is consistent with different intensities of light falling on the upper and lateral surfaces of the cube, a color difference (“contrast”) effect is seen in (a) and color similarity (“constancy”) in (b). The appearances of the relevant target surfaces in the same “neutral” context are shown in the keys below

80

13 Color

colors look different because of what humans have found successful when interacting with differently reflective surfaces in spectrally different illuminants. Conversely, the same two targets on the faces of the cube in Fig.  13.4b appear similar even though the spectra from the targets are in fact different: the information in the scene is what humans and other animals with color vision like ours would have experienced when interacting with physically similar surfaces under different illumination. In both instances this empirical strategy would have promoted useful responses.

13.4 Conclusion This sort of evidence indicates that the colors we see are not the result of the spectra in retinal images per se, but of associating spectra with perceptions discovered empirically that led to successful behavior. To regard these color effects as “illusions” (a common interpretation) is not only wrong but misses the biological significance of these phenomena and the explanatory advantage of understanding color in empirical terms. Another implication is that color percepts cannot be computed.

Further Readings Brainard DH, Freeman WT (1997) Bayesian color constancy. J Opt Soc Am A Opt Image Sci Vis 14:1393–1411 Long F, Yang Z, Purves D (2006) Spectral statistics in natural scenes predict hue, saturation and brightness. PNAS 103:6013–6018 Purves D, Lotto RB (2011) Why we see what we do redux: a wholly empirical theory of vision. Sinauer Associates, Sunderland, MA. Chapter 3

Chapter 14

Color Psychophysics

14.1 Introduction Perhaps the ultimate challenge for any theory of color vision is rationalizing the complex relationships of hue, saturation, and lightness seen by observers as these parameters are varied. This chapter provides some examples of how empirical ranking can explain perceptual functions that algorithmic computation cannot.

14.2 Psychophysics The aim of color psychophysics is to assess how perceptual responses to elemental color stimuli vary as the parameters relevant to hue, saturation, and lightness-­ brightness are systematically changed. An empirical approach uses the spectral relationships humans have always witnessed to predict the observed psychophysical functions. Analyzing color databases entails extracting the frequency of occurrence of the relationships among hue, saturation, and color lightness in millions of image patterns (Fig. 14.1). If colorimetry functions are determined empirically, the information derived in this way should predict the relationships that have been described.

14.3 An Example An example is explaining in empirical terms the results of classical color discrimination tests such as the just noticeable differences (JNDs) humans can discriminate as the wavelength of a test stimulus is systematically changed. The function in © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. Purves, Why Brains Don’t Compute, https://doi.org/10.1007/978-3-030-71064-4_14

81

82

14  Color Psychophysics

Fig. 14.1  The frequency of occurrence of spectral relationships in natural scenes. By analyzing millions of patches from hundreds of natural scenes like these, the accumulated experience of humans with the spectral relationships can be approximated

Fig. 14.2a was generated in the 1950s and indicates the apparent change in color perception as the wavelength of a monochromatic stimulus is varied. The result is anything but simple, rising and falling over the range of visible wavelengths. Where the slope of the function is not changing much people make finer color discriminations; relatively little change in the wavelength of test stimuli is needed before the subject reports an apparent difference in color. Where the slope is steep, however, a relatively large change in wavelength is needed before observers see a color

83

14.4  The Bezold-Brucke Effect

Wavelength change (nm)

10 8 6 4 2 0

Wavelength change (nm)

16

12

8

4 0 400

500 600 Wavelength (nm)

700

Fig. 14.2  Predicting a colorimetric function based on empirical data. The top panel shows the change in the wavelength of a monochromatic stimulus needed to elicit a just noticeable difference in color perception when tested across the full range of light wavelengths. The bottom panel shows the funtion predicted by an empirical analysis of the spectral characteristics of millions of points (pixels) in a database of natural scenes (see Fig. 14.1); the black dots are the predictions from the empirical data and the red line a best fit

difference. The reason for these complex variations in sensitivity to wavelength change has been anybody’s guess.

14.4 The Bezold-Brucke Effect Another peculiar way color perception varies was described in the 1870s by Wilhelm Bezold and Ernst Brucke—the so called the “Bezold-Brucke shift.” In low intensity illumination the color percepts elicited by wavelengths below ~500 nm are shifted toward blue in perception, whereas wavelengths above ~500 nm are shifted toward yellow. Conversely in high-intensity light, wavelengths below ~500 nm are shifted toward green, while wavelengths above ~500 nm elicit percepts shifted toward red.

84

14  Color Psychophysics

Four wavelengths are resistant to these shifts, and these roughly correspond to the wavelengths of four unique hues that define four categories of human perception (see Fig. 13.1). These effects are obvious when the same surfaces are partly in light and partly in shadow. The phenomena are also well-known to artists who have long been aware that to represent dim illumination in paintings reds and yellows must be darkened. Conversely, when representing strong light in a painting blues and greens must be lightened. As shown in Fig. 14.3(top panel), some monochromatic wavelengths—the locations where the curve intersects the 0 line—are relatively resistant to changes in hue at different light intensities, creating a complex psychophysical function that divides the spectrum into regions characterized by oppositely directed hue shifts. Figure 14.3(bottom panel) shows the function predicted from the co-occurrence of the physical correlates of hue and of brightness corresponding conditional

Wavelength change (nm)

20 10 0 –10 –20 –30 –40

Wavelength change (nm)

40

20

0

–20 –40 400

500

600

700

Wavelength (nm)

Fig. 14.3  Empirical basis for the Bezold-Brücke effect. (Top panel) Psychophysical data showing the changes in wavelength needed to match the hue of a monochromatic light presented at a higher intensity. (Bottom panel) the predicted hue shifts for the indicated points from the joint probability distribution of the physical correlates of hue and brightness obtained from the database derived from scenes such as the one in Fig. 14.1 (the solid line is a polynomial fit)

Further Readings

85

probability distributions at a higher and a lower value of luminance. Comparison of Fig. 14.3 shows reasonable agreement with the psychophysical data. Predicting these and other colorimetric functions on empirical grounds depends on the fact that the light experienced at each point in spectral stimuli will have varied in particular ways determined by the qualities of object surfaces, illumination, and other characteristics of the physical world. For instance, middle wavelength light is more likely than short or long wavelength light at any point because of the spectrum of sunlight and the typical properties of object surfaces. The wavelengths arising from a scene will also have varied as the illumination changes because of the time of day, the weather, and shadowing from other objects. These influences cause the spectral qualities of each point in visual stimuli to have a typical frequency of occurrence. Humans will always have experienced these relationships and, based on this empirical information, have evolved neural circuitry that promoted successful behavior in response to the frequency of occurrence of natural spectral patterns.

14.5 Conclusion If the organization of human perceptual color space has been determined empirically, then the effects of routinely experienced spectral information should predict the relevant psychophysical functions. Although certainly not perfect, there is general agreement between the human responses and the functions predicted from spectral data. In contrast, intuitions and physiological or computational explanations of psychophysical color functions are stymied.

Further Readings Long F, Yang Z, Purves D (2006) Spectral statistics in natural scene predict hue, saturation, and brightness. Proc Natl Acad Sci 103(15):6013–6018 Purves D, Lotto B (2003) Why we see what we do: an empirical theory of vision. Sinauer, Sunderland, MA. Chapters 5 and 6

Chapter 15

Motion Speed

15.1 Introduction Responding to object motion—or motion arising from self-movement—is typically explained in computational and/or mathematical terms. However, if accumulated experience is to replace algorithmic processes as the basis of neural function, then motion is another perceptual quality that should be rationalized empirically.

15.2 Physical Motion In physical terms, motion refers to the speed and direction of objects in a three-­ dimensional frame of reference. In psychophysical terms, however, motion is defined subjectively. We do not see the motion of a clock’s hour hand or a bullet in flight, even though both objects move at physical rates that are easily measured. The range of object speeds projected onto the retina that humans have evolved to appreciate is from roughly 0.1°–175° of visual angle/s. Below this range objects appear to be standing still, like the hour hand; and as speeds approach the upper end of the range they begin to blur, and then, like the bullet, become invisible. A deeper puzzle is the way perceived motion is affected by context: depending on the circumstances, the same speed and direction projected on the retina can elicit very different perceptions. Although such phenomena have often been treated as “illusions” or simply ignored, the issue here is whether the perception of motion has the same empirical basis as the perceptions of geometry, lightness, and color.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. Purves, Why Brains Don’t Compute, https://doi.org/10.1007/978-3-030-71064-4_15

87

88

15  Motion Speed

15.3 The Problem As in these other domains, the speeds and directions we perceive can be understood as a means of getting around our biological inability to measure motion in physical terms. For behavior to succeed, observers must respond correctly to the real-world speeds and directions of objects on the retina. But as illustrated in Fig. 15.1, when objects in three-dimensional space project onto a two-dimensional surface, speed and direction are conflated. As a result, the sequence of the actual positions 3-D space that define motion in physical terms are always ambiguous in retinal images. If contending with this problem depends on the empirical framework used to rationalize other perceived qualities, then the sense of motion elicited by image sequences should be predicted by the frequency of occurrence of experience with the retinal image sequences humans have always witnessed. An obstacle in exploring this idea, however, is acquiring a database that relates 2-D projections to the speeds and directions of objects moving in 3-D space. Although data relating retinal

Fig. 15.1  The problem perceiving the speed and direction of moving objects. Objects (black dots) at different distances moving in different directions (arrows) at different speeds can all generate the same image sequence on the retina (the diagram depicts a horizontal section through the right eye). Thus projected speeds and directions cannot specify the speeds and directions of the real-world sources that observers must deal with

Physical speeds and directions of 3-D objects

Medial

Head To optic chiasm

Image speed

15.3  The Problem

89

Fig. 15.2  The frequency of occurrence of image sequences generated by moving objects in a virtual environment. Diagram of a simulated visual space (red outline) embedded in a larger environment; objects moving randomly in different directions over a range of speeds enter visual space and project onto an image plane (blue outline). The frequency of occurrence of the speeds and directions in image sequences generated by objects whose physical motion is known can then be approximated

projections to real-world geometry is relatively easy to obtain for static scenes (see Chaps. 9 and 10), there is no simple way to collect information about the speed and direction of moving objects. Human experience with motion can, however, be approximated by simulation in a virtual world in which moving objects are projected onto a plane (Fig. 15.2). By sampling projections on the image plane, the frequency of occurrence of the projected speeds and directions arising from 3-D sources can be approximated. These data can then be used to predict the perceived speeds and directions that should be seen in response to motion stimuli. As in other perceptual domains, explaining anomalies of motion perception whose causes have been debated for decades tests the merits of an empirical explanation.

90

15  Motion Speed

15.4 An Example An example of the challenge in explaining perceived speed is the “flash-lag effect.” When a continuously moving stimulus is presented in physical alignment with a flash that marks a point in time and space, the flash is seen lagging behind the moving stimulus (Fig. 15.3a). And the faster the speed of the stimulus, the greater the lag (Fig. 15.3b, c). The effect is one of several related phenomena is apparent when people observe stimuli in which a moving object is presented together with an instantaneous marker. For instance, if the flash occurs at the start of the trajectory of an object, the flash is seen displaced in the direction of motion (the so-called Fröhlich effect). And if observers are asked to specify the position of a flash in the presence of nearby motion, the flash is displaced in that direction (the “flash-drag” effect). These discrepancies all reveal a systematic difference between the speed projected on the retina and the speed seen.

15.5 The Explanation In empirical terms, the flash-lag and related effects can be explained by the frequency of occurrence of image sequences arising from 3-D object motion transformed by projection onto the retina as shown in Fig. 15.2. A first step is to determine the relevant psychophysical function by having observers align a flash with the moving bar, varying the speed of the bar over the full range that elicits the effect. To test whether the function accords with an empirical explanation of perceived speed, the image sequences generated by objects moving in the simulated environment can be sampled to measure the frequency of occurrence of the different projected speeds generated by millions of possible sources moving in the simulated 3-D world. If the flash-lag effect is a signature of visual motion determined empirically, then the lag reported by observers for different stimulus speeds should be predicted by the relative ranks of different image speeds arising from the influence of experience. As shown in Fig. 15.4, the psychometric function in Fig. 15.3c accords with the frequency of occurrence of speeds on the retina generated by moving objects. Since increases in image speed correspond to higher ranks in experience with speed (i.e., more and more sources will have generated image sequences that traverse the retina more slowly than the retinal speed of the stimulus in question), the magnitude of the flash-lag effect should increase as a function of image speed. And, since the flash is stationary, it will always have a lower empirical rank than a moving stimulus. Accordingly, a flash coincident with a moving object should appear to lag behind the position of the object projected onto the retina, as it does.

15.5  The Explanation

91

2.5

Perceived lag (deg)

2

1.5

1

0.5

0

0

5

10

15

20 25 30 35 Image speed (deg/s)

40

45

50

Fig. 15.3  The flash-lag effect. (a) When a flash of light (indicated by the asterisk) is presented in physical alignment with moving stimulus (the red bar), the flash is perceived as lagging behind the moving bar. (b) The apparent lag increases as the speed of the moving object increases. (c) The lag as a function of object speed determined by asking subjects to adjust the position of the flash until it appeared to be in alignment with the moving stimulus

92

15  Motion Speed

Fig. 15.4  Empirical prediction of the flash-lag effect. The graph shows the perceived lag reported by observers (see Fig.  15.3c) plotted against the empirical rank (i.e., the relative frequency of occurrence) of the projected image speeds generated by moving objects. The black diamonds are the empirically predicted lags; the dashed line indicates perfect correlation between the predicted lags and the amount of lag reported by observers

15.6 Conclusion This empirical explanation of perceived speed is much the same as that given for geometry, lightness, and color. Like the perception of these other qualities, the biological inability to measure physical speeds requires that the speeds we see be organized according to the frequency of occurrence of projected speeds on the retina, which then enables successful behavior.

Further Readings Groh JM (2014) Making space: how the brain knows where things are. Harvard University Press, Cambridge, MA Hildreth EC (1983) The measurement of visual motion. MIT Press, Cambridge, MA

Further Readings

93

Sung K, Wojtach WT, Purves D (2009) An empirical explanation of aperture effects. PNAS 106:298–303 Purves D, Lotto RB (2011) Why we see what we do redux: a wholly empirical theory of vision. Sinauer Associates, Sunderland, MA. Chapter 6 Ramachandran VS, Anstis SM (1986) The perception of apparent motion. Sci Am 254:102–109 Weiss Y et al (2002) Motion illusions as optimal percepts. Nat Neurosci 5:598–604 Wojtach WT, Sung K, Truong S, Purves D (2008) An empirical explanation of the flash-lag effect. PNAS 105:16338–16343 Wojtach WT, Sung K, Purves D (2009) An empirical explanation of the speed-distance effect. PLoS One 4:e6771. https://doi.org/10.1371/journal.pone.0006771

Chapter 16

Motion Direction

16.1 Introduction The other basic puzzle in understanding perceived motion is direction. The phenomenon that needs to be understood in this case is the effect of contexts on the apparent direction of motion. This complementary characteristic of moving objects can also be explained empirically and would be difficult to account for algorithmically.

16.2 Apertures The perceived direction of motion varies when a moving object like a rod seen in different frames (called “apertures”). For example, when an obliquely oriented rod moving from left is viewed through a circular opening that hides its ends, its perceived direction of motion changes from horizontal to about 45° downward from the horizontal axis (Fig. 16.1). The change in direction occurs instantly when the frame is applied, depends on the shape of the aperture. If the same oriented rod moving from left to right is seen through a vertical slit, the perceived direction of motion is nearly straight down (called the “barber pole effect”). One way of thinking about the effects of apertures in empirical terms is to consider the frequency of occurrence of the projected directions of un-occluded lines generated by objects moving in 3-D space (Fig. 16.2). The movement of a projected line with a given length and orientation on the retina is about equal in all directions. This uniform distribution of directions in the absence of occlusion describes, to a first approximation, how humans have always experienced the projections of linear objects in full view.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. Purves, Why Brains Don’t Compute, https://doi.org/10.1007/978-3-030-71064-4_16

95

96

16  Motion Direction

Fig. 16.1  The effects of a frame on the perceived direction of motion. The black line in the aperture moves horizontally from left to right in the physical world, as indicated by the yellow arrow. When viewed through the aperture, however, the rod is seen moving downward to the right (red arrow)

16.3 Effect of a Circular Aperture When a line moves behind an aperture, however, this uniform distribution of projected directions is changed. The different frequencies of projected directions that humans will always have seen in an occluding frame thus offers a way of asking whether the perceived directions elicited by different apertures can also be accounted for in empirical terms. If the perceived direction of motion is determined by past experience, then effects of any particular aperture should be predicted by the frequency of occurrence of the various directions generated by object motion projected through that frame. The simplest effect to explore in this way is the altered direction of motion induced by a circular aperture (Fig.  16.3a). The frequency of occurrence of projected directions that observers have always experienced in this circumstance can be determined by systematically applying a circular template to the image plane of the virtual environment and measuring the frequency of occurrence of the 2-D directions of lines that generate image sequences in the aperture whern the ends of the lines remain hidden. The mode of this distribution (red arrows in Fig. 16.3b) is the direction humans will have experienced most often whenever moving lines are seen through a circular aperture. The green arrows are the directions that subjects report

16.3 Effect of a Circular Aperture

97

Fig. 16.2  The frequency of occurrence of projected lines moving in different directions. (a) Diagram of a simulated environment populated by linear objects in different orientations moving in different directions. (b) The projected lines move in directions that are different from the actual movements of the objects in (a) as indicated by the dashed arrows. (c) The frequency of occurrence determined by sampling all the sequences generated by objects in the simulated environment projecting at an angle of 45° as images. The directions of movement are indicated around the perimeter of the graph; the distance from the center to any point on the jagged line indicates the frequency of image sequences moving in that direction

98

16  Motion Direction

Fig. 16.3  Psychophysical results compared to empirical predictions of how moving lines in different orientations would be seen through a circular aperture. (a) As in Fig. 16.2, the orientation (θ) of the line the aperture is measured from the horizontal axis; the direction of movement is shown as positive or negative deviations from the direction perpendicular to the moving line (0°). (b) The ovals indicated by the jagged black lines show the frequency of occurrence of the projected directions constrained by circular aperture for the different orientations. The green arrows show the directions reported in psychophysical testing; the red arrows are the directions predicted empirically

in psychophysical testing. The predicted directions (red arrows) closely match the directions seen.

16.4 Effect of a Vertical Aperture This kind of analysis also explains the effects elicited by additional frames that are otherwise difficult to account for. A case in point is the perceived direction of the horizontally moving lines is seen through a vertical slit (Fig. 16.4a). As mentioned, the stimulus generates the perception of vertical movement. Moreover, as the orientation of the line becomes steeper, the perceived downward direction deviates increasingly from straight down (Fig. 16.4b). To test an empirical explanation, the frequency of occurrence of projected directions as a function of orientation can be determined by applying a vertical slit template millions of times to the images produced in the virtual environment in Fig. 16.2. The analysis again predicts the effects apparent in psychophysical testing (compare the red and green arrows).

16.5 Explanation

99

Fig. 16.4  Comparison of the psychophysical results and empirical predictions for lines moving in different orientations projected through a vertical slit. (a) The aperture. (b) Distributions of the frequency of occurrence of the projected directions of moving lines in different orientations (jagged ovals) when constrained by a vertical aperture. As in Fig. 16.3, the green arrows are the results of psychophysical testing, and the red arrows are the empirical predictions (the black arrows below indicate vertical)

16.5 Explanation To understand these effects on the apparent direction of motion considers the biased directions of image sequences projected through a circular aperture. As shown in Fig. 16.5, for a line in any particular orientation on an image plane there is a direction of motion (black arrow) that entails the minimum projected line length (the red line) that can fully occupy the aperture. A projected line traveling in any other direction (e.g., the blue line) must be longer to fill the aperture; one end or the other of any shorter line moving in that direction would fall inside the aperture boundary at some point, producing a different stimulus and a different perceived direction of motion. Because a line of any length includes all shorter lines, far more moving lines that satisfy the aperture are projected in the direction orthogonal to their orientation than in other directions. As a result, the distribution of directions that satisfies a circular aperture is strongly biased in the direction orthogonal to the orientation of any line (see Fig. 16.3). These facts explain the direction of motion seen through a circular, vertical, or any other aperture.

100

16  Motion Direction

Fig. 16.5  Explanation of the biased directions generated by projection of lines onto an image plane. Because a line of any given length includes all shorter lines, the occurrence of projections that fill a circular aperture generated by the red line moving in the direction indicated by the black arrow in the left panel will always be more frequent than the occurrence of projections generated by the blue line moving in any other direction, indicated by the black arrow in the right panel. The most frequent projected direction when linear objects are seen through a circular aperture will always be the direction orthogonal to the orientation of the projected line (gray arrows)

16.6 Conclusion The empirical accounts of perceived motion here and in Chap. 15 argue that human experience with the projections of moving objects determines the speeds and directions seen. When other empirical influences on perception are brought to bear, the behavioral responses elicited have a high chance of success.

Further Readings Purves D, Lotto RB (2011) Why we see what we do redux: a wholly empirical theory of vision. Sinauer Associates, Sunderland, MA. Chapter 6 Sung K, Wojtach WT, Purves D (2009) An empirical explanation of aperture effects. Proc Natl Acad Sci 106:298–303 Yuille AL, Grzywacz NM (1998) A theoretical framework for visual motion. In: Watanabe T (ed) High-level motion processing: computational, neurobiological, and psychophysical perspectives. MIT Press, Cambridge, MA

Chapter 17

Object Size

17.1 Introduction Adding to this list of perceptual anomalies, the perceived size of objects in different contexts is also at odds with physical reality. Examples are classic 2-D size “illusions” described in the early twentieth century, the anomalous size of objects when distance is represented, and the still unexplained “Moon illusion.” It is difficult to imagine how any of these responses to stimuli could arise from computation, whereas they can all be explained empirically.

17.2 Classical Size “Illusions” The stimuli first studied more than a century ago were simple patterns such as the example in Fig. 17.1 in which identical central circles appear somewhat different in size as a consequence of surrounding circles that are larger or smaller. The common denominator of these “size-contrast” effects is the different apparent sizes of identical objects presented in the context of surrounding objects with different dimensions. As in Chaps. 9 and 10 the frequency of occurrence of the relevant retinal projections can be determined by the application of appropriate templates (i.e., stimuli like those in Fig. 17.1) to laser-scanned images of natural scenes where the geometry of the real is made explicit. The result is that the same central circle occurs more often in images with smaller circles as the context than with larger circles (or small circles farther from the central circle). This analysis shows that the larger the context the less frequently it will be seen relative to experience with smaller contexts. The empirical reason is that the larger surrounding circles—or smaller circles farther from the central target—necessarily arise from larger planes in the physical world that project onto the retina less often than smaller ones because larger planes include © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. Purves, Why Brains Don’t Compute, https://doi.org/10.1007/978-3-030-71064-4_17

101

102

17  Object Size

Fig. 17.1  Size-contrast effects. In the classic “Ebbinghaus effect” (named after the late nineteenth century German psychologist Hermann Ebbinghaus) observers perceive the central circle surrounded by smaller circles on the left as appreciably larger than the identical circle surrounded by bigger circles on the right

smaller ones (see Fig. 10.4). These different frequencies of occurrence generate different empirical ranks in accumulated experience, and thus different perceptions of the identical central objects.

17.3 Object Sizes in Scenes with 3-D Cues Unlike the two-dimensional presentations in Fig. 17.1, many geometrical projections generate a sense of distance arising from cues in more realistic scenes. The anomalous size effects in two-dimensional stimuli are augmented when the information in images implies distance. An example is the gold figures in the depicted foreground and background in Fig. 17.2. The figures are the same size although they appear very different in this scene. This phenomenon is also referred as the “size-distance paradox” because, due to perspective, objects farther away generate smaller projections on the retina than when the same objects are nearer. Figure 17.3 illustrates the variety of cues that give rise to a sense of distance. The most familiar of these is perspective arising from projective geometry: the closer an object is to the observer, the larger its projection on the retina. A second empirical cue is occlusion: when one object, such as the sphere in the foreground, occludes another object, the un-occluded object is always closer than the occluded one. A more subtle cue is aerial perspective: distant objects appear fuzzy and bluish because of the intervening atmosphere. Another cue is motion parallax: when the head and body move, the objects in the foreground are displaced more on the retina than objects in the background. These and other “monocular cues” to distance are presumably learned as in the perception of lines, angles, lightness, color, and motion. In Fig. 17.2, for instance, the rank of the size the far objects will, as a result of perspective and other factors, be lower on the empirical scale of size than near objects. Thus when objects that have the same physical size are compared in the foreground and background, the percentile rank of the background object is higher than that of the same object in the foreground, and thus perceived as larger.

Fig. 17.2  The anomalous perception of object size in a context that includes distance cues. As can be verified with a ruler, the dimensions of the two gold figures are identical

Fig. 17.3  Monocular information pertinent to apparent distance. (a) Illustration of object occlusion, effects of the atmosphere (“aerial perspective”), and the diminution of projected size with distance (perspective). (b) Motion parallax (differences in the apparent motion of near and far objects)

104

17  Object Size

17.4 Relevance to a Long-Standing Puzzle A special challenge in the perception object size is the “Moon illusion.” Observers have always noted that the Moon appears larger by about half when seen on the horizon than when high in the sky. However, simply holding out a finger as a measuring tool shows that there is no difference in the projected size of the Moon on the retina in these two circumstances. Explanations of this phenomenon have been offered by Aristotle, Ptolemy, and numerous others since and continue to be debated in books, articles, and blogs. Despite much thought and experimental effort over the centuries, there is no consensus about why the horizon Moon looks larger than the Moon at its zenith. The Moon illusion can also be explained by the frequency of occurrence of the stimulus patterns humans have always experienced. In terrestrial views, projected sizes near the visible horizon are strongly biased toward smaller surface areas as a result of perspective (the projection of objects diminishes inversely with the square of distance) (Fig. 17.4a). In consequence, an apparently distant object with an ~0.5°

Projected Surface Area (m2)

a

0.03

0.02

0.01

0 1

2

3

4

5

7

6

8

9

10

Distance (m)

b

c 1 Visible Horizon Overhead

0.8 0.6 0.4 0.2 0

0

5

10

15

20

25

30

Projected Surface Area (visual angle degree)

Cumulative Probability

Frequency of Ocurrence

1

0.8 0.6

Visible Horizon Overhead

0.4 0.2

Empirical Rank 1

0 0

Empirical Rank 2 15 5 10

20

25

30

Projected Surface Area (visual angle degree)

Fig. 17.4  The projected surface areas of terrestrial objects as a function of distance from the observer. (a) Since projected surface areas decrease inversely as the square of their distance, the projected sizes of distant objects are progressively diminished. (b) Frequency distributions of projected surface sizes near the horizon (black) compared with those in the sky (blue). (c) Human experience with retinal projections of the horizon and zenith Moon. The projected size of the Moon (~0.5°) will always rank higher among surfaces viewed near the horizon (Empirical Rank 1) than in views of the Moon in the sky (Empirical Rank 2). (Courtesy of C.Q. Howe)

Further Readings

105

diameter such as the Moon on the horizon necessarily ranks relatively high on an empirical scale generated by terrestrial objects whose projected sizes diminish progressively. In the illusion, the rank of the projected Moon in a terrestrial context is compared with the apparent size of the zenith Moon. In the latter case the context is the sky, which fills most of the visual field. Accordingly, the projected size of the zenith Moon—the same projected 0.5° as the Moon on the horizon—now ranks relatively low on the relevant empirical scale (Fig. 17.4b, c). Because the rank of the zenith Moon in accumulated experience is lower than the rank of the horizon Moon in the context of terrestrial projections, the zenith Moon should look smaller when compared to the Moon near the horizon. Although the Moon illusion is only a provocative puzzle, its rationalization in empirical terms adds to the evidence that perceptions of size are molded by experience.

17.5 Conclusion The way we see geometry is rife with examples of objective measurements of the world that differ from the subjective perceptions. As with other qualities, the empirical ranks that determine perceptions of size are based on the success of behavior rather than neural computations.

Further Readings Howe CQ, Purves D (2004) Size contrast and assimilation explained by the statistics of natural scene geometry. J Cogn Neurosci 16:90–102 Purves D, Lotto RB (2011) Why we see what we do redux: a wholly empirical theory of vision. Sinauer Associates, Sunderland, MA. Chapter 4 Purves D, Monson BB, Sundararajan J, Wojtach WT (2014) How biological vision succeeds in the physical world. PNAS 111:4750–4755

Chapter 18

Stereopsis

18.1 Introduction In contrast to cues about geometry, the perception of stereoscopic depth is a topic unto itself. Sorting out this phenomenon entails understanding how information from the right and left eyes is fused, how fusion generates stereopsis, and why “ocular dominance” is characteristic of animals with stereo vision. Although the approach to these issues has been decidedly computational, depth perception can also be explained empirically.

18.2 Image Versus Anatomical Correspondence Theories that address stereopsis are based on the assumption that image points on one retina are matched with corresponding points on the other retina algorithmically (Fig.  18.1). A difficulty in this formulation, however, is explaining how left and right eye neurons responding to the same physical point in space could be linked, a challenge referred to as the “correspondence problem.” To deal with this concern, a “spatio-temporal energy model” was proposed about 30 years ago, adapted from a theory that addressed the motion correspondence problem (i.e., the problem that arises in a computational explanation of motion discussed in Chaps. 15 and 16). The model describes how monocular contrast is computed by cortical neurons that are related to one eye or the other, and suggests how matching monocular information in binocular neurons in the primary visual cortex could be achieved by cross-­ correlating the two retinal images. The alternative empirical strategy suggested by the way other perceived qualities are generated is to have stereo vision depend on differences in neuronal activity at anatomically corresponding retinal points on the two retinas (i.e., points a and a’, b © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. Purves, Why Brains Don’t Compute, https://doi.org/10.1007/978-3-030-71064-4_18

107

108

18 Stereopsis

Fig. 18.1  Diagram illustrating the difference between corresponding anatomical points on the retinas and corresponding image points. (a) The horopter (black semi-circle) is defined by all the points in visual space that fall on corresponding retinal points (a and a’ and b and b’). Points f and f’ are the point of fixation (black dot) that falls on the center of each fovea. (b) A light ray arising from a point nearer than the horopter (blue dot) projects onto point (p) in the left eye but onto point (q) in the right eye whose position differs from the location of the anatomically corresponding point (p’). (c) Similarly, a light ray arising from a point more distant than the point of fixation falls on point (m) in the left eye but on point (n) in the right eye that also differs from the anatomically corresponding point (m’). Image correspondence and anatomical correspondence are thus different ways of conveying information about the depth of points in visual space relative to the horopter.

and b’, p and p’, and m and m’ in Fig. 18.1). In this scenario the routinely different activity at anatomically corresponding retinal loci conveys empirical information about depth that could be used to generate successful behavior.

18.3 Binocular Circuitry One way to examine the merits of this alternative understanding of stereo vision is to ask whether the circuitry required in an empirical scenario matches the circuitry found in experimental animals. To this end, populations of artificial neural networks can be evolved in a simulated environment on the basis of information at corresponding anatomical points on the two retinas. The simulated environment in this approach includes two contrast boundaries (Fig. 18.2). One boundary includes the point of fixation, while the other varies in depth along the line of sight of the fused stimuli. The stimuli are projected onto sensor arrays generating activity at corresponding “anatomical” locations on the right and left arrays (stand-ins for the retinas of the right and left eyes). Thus one pair of corresponding neurons receives information from the point of fixation, while the other pair receives information from the contrast boundary of the surface whose distance is variable. The compiled sum of these transformations is the output of the

18.4  Ocular Dominance

109

Fig. 18.2  The strength of connections after artificial neural networks learned to report differences in stimulus strength at anatomically corresponding positions on the two “retinas.” (a) Top: Evolution of networks trained to respond to points farther than the horopter. Black line is the average error across all the populations tested; the gray overlay shows the absolute deviation from the mean. Bottom: The strength of the connections is indicated by the thickness of the lines; solid lines with arrowheads represent excitatory connections and dotted lines connections with little or no effect. Inhibitory connections are denoted by the end-stopped lines. (b) Evolution and average evolved connectivity of networks trained to respond to points nearer than the horopter

network’s monocular neurons, which would in turn provide inputs to second-order binocular neurons. After a few thousand generations, the output of the second-order neurons had learned to specify whether a point in visual space is nearer or farther than the horopter and by how much. Networks that identified contrast boundaries farther than the horopter evolved a strong excitatory connection from the contralateral monocular neuron, but not from the ipsilateral neuron (Fig. 18.2a). Conversely, networks that discriminated stimuli arising from boundaries nearer than the horopter evolved a strong excitatory connection from the ipsilateral neuron, but not from the contralateral neuron (Fig. 18.2b). In both instances the other monocular input to the second-­ order neuron is inhibitory. This arrangement is similar to the visual circuitry observed in animals that see stereoscopically.

18.4 Ocular Dominance A related phenomenon in stereo vision that needs to be explained is “ocular dominance.” Ocular dominance refers to the fact that most binocular neurons in the primary visual cortex of animals with well-developed depth perception are more strongly driven by one eye or the other. Moreover, right- or left-eye-dominated neurons segregate into columns or stripes that are readily seen in the primary visual

110

18 Stereopsis

Fig. 18.3  Ocular dominance stripes looking down on the primary visual cortex of a monkey after injecting a marker into one eye

cortex of humans and other animals with stereo vision (Fig. 18.3). Although ocular dominance has been widely used since its discovery more than 60 years ago as an index of cortical connectivity during development and to understand and treat strabismus in clinical ophthalmology, its purpose, if any, has remained unclear. In the present argument, ocular dominance arises from the different activity at anatomical corresponding retinal loci (see Fig. 18.1). Because they are excited by one retinal input and inhibited by the other, the evolved units in Fig. 18.2 resemble the right versus left eye dominance of cortical neurons found in experimental animals. Moreover, these second-order units would organize into groups if tiled across a representation of the whole visual field in the primary visual cortex. Stereopsis on the basis of anatomical correspondence also accords with observation in experimental animals that neurons responding to differences to depth tend to be dominated by one eye or the other, whereas neurons that do not tend to be equally driven by each eye.

18.5 Relevance to Perception Stereoscopic depth is a perceptual quality. An important question, therefore, is how a theory depth perception based on anatomical correspondence accords with the empirical understanding of other perceptual qualities such as lightness, color, object geometry, motion, and size. In the case of these other qualities, perceptions arise from the frequency of occurrence of stimuli accumulated over species and individual experience. The result is empirical “scales” that determine what is actually seen. The explanation of stereo vision may be that the frequency of occurrence of differences in activity at anatomically corresponding retinal points and their higher-order consequences are empirically linked to differences in perceived depth based on successful behavior.

Further Readings

111

18.6 Conclusion The frontal eyes and overlapping visual fields of carnivores and primates require that points in visual space project to anatomically corresponding points on the right and left retinas. As a result, unequal activity is generated in the neurons at retinotopically corresponding loci whenever the generative points in visual space are nearer or farther than the horopter. When conveyed to binocular neurons in the visual cortex, this differential activation can specify the magnitude and direction of both absolute and relative depth. This interpretation of stereopsis is consistent with observations in experimental animals, circumvents the image correspondence problem and explains why ocular dominance is only found in animals with stereo vision. Finally, it is consistent with the strategy of empirical ranking evident in other perceptual domains.

Further Readings Adelson EH, Bergen JR (1985) Spatiotemporal energy models for the perception of motion. J Opt Soc Am A 2:284–299 Horton JC, Adams DL (2005) The cortical column: a structure without a function. Philos Trans R Soc Lond B 360:837–862 Marr D, Poggio T (1979) A computational theory of human stereo vision. Proc R Soc Lond B 204:301–328 Ng CJ, Purves D (2019) An alternative theory of binocularity. Front Comput Neurosci 13:71. https://doi.org/10.3389/fncom.2019.00071 Purves D, Lotto RB (2011) Why we see what we do redux: a wholly empirical theory of vision. Sinauer Associates, Sunderland, MA. Chapter 5

Part V

Linking Objective and Subjective Domains

Understanding the relationship between the objective domain of physical reality and the subjective domain of perceptual and other behavioral responses to stimuli is arguably the major challenge in contemporary neuroscience. This section examines the mechanisms that nervous systems use to make this link, and what they imply about how brains operate.

Chapter 19

Stimuli and Behavior

19.1 Introduction The concept of stimuli as energy from the external or internal environment impinging on receptors is not as simple as it may seem. Sensory systems include pre-neural filters and amplifiers that alter the energy before it reaches receptor neurons. Once the receptors have transduced the result, peripheral neural signals interact with other neurons in an increasingly complex hierarchy. What is passed on to these poorly defined higher order systems—the welter of neural connections that lie between inputs and outputs—are “stimuli” in their own right. To complicate matters even further, these higher order effects elicit feedback to the lower order stations in a never-ending loop.

19.2 What, Then, Are Stimuli? Environments present animals with ever-changing energy states, a small minority of which the species in question has evolved to respond to. Right off the bat, then, that the meaning of the term “stimulus” is more complicated than usually thought. As already emphasized, nervous systems allow animals to get along in the world without apprehending reality. These facts indicate that stimuli are determined by the “laws” of biology rather than the laws of physics and chemistry (Fig. 19.1). Once “within” the nervous system, neural signals are further changed by the processing stations that modify signals on the way to the cortex and back down again. This widespread “processing” is mediated by neural connections whose “strength” and ultimate effects depend on inheritance and lifetime learning. The world simply presents energy that a species may evolve to respond to or not, perception depending on how often particular stimulus patterns have contributed to © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. Purves, Why Brains Don’t Compute, https://doi.org/10.1007/978-3-030-71064-4_19

115

116

19  Stimuli and Behavior Natural Selection Pre-neural and Neural Apparatus

Energy

Useful Stimuli

Impact on Survival

Biological Agent

Frequency of Occurrence of Stimulus Patterns

Time

Empirical rank

Perceptions

Fig. 19.1  The frequency of occurrence of empirically useful stimulus patterns. Over evolution and individual lifetimes, animals discover that a subset of neural inputs lead to useful behaviors. Although states in the physical world are determined by physics and chemistry, the frequency of occurrence of biologically determined stimulus patterns gives rise to perceptions and other neural responses that promote survival and reproduction

the well-being of the animal in question. This is the gist of the evidence reviewed in Part IV.

19.3 Behaviors as Understood by Physiologists Given this framework, behavior is the response animals ultimately make to biologically determined stimuli. Physiologists and neurobiologists tend to think of behavior as motor actions. However, behaviors are any observable reaction to the complex chains of stimuli initiated by peripheral sensors. In addition to motor acts, behaviors include perception, attention, emotion, memory, autonomic responses, thought processes and more. Each of these categories presents a research field in itself, and each remains incompletely understood.

19.4 Behaviors as Understood by Psychologists In contrast, psychologists tend to consider behavior from the perspective of classical and operant conditioning. Classical conditioning occurs when a more or less fixed behavior in response to a stimulus—often called a reflex—is modified by associating its natural trigger with an unrelated stimulus. By virtue of repeated experience, the unrelated stimulus eventually triggers the innate response using the various mechanisms of synaptic plasticity and association described in the following chapters. This type of conditioning was famously studied by the Russian physiologist Ivan Pavlov in experiments with dogs and other animals carried out early in the

19.5  The Common Strategy

117

twentieth century. In the best known of many studies, the innate reflex was salivation (the “unconditioned response”) in reaction to the sight or smell of food (the natural or “unconditioned stimulus”). A novel association is created by repeatedly pairing the presentation of food with the sound of a bell (the “conditioned stimulus”). A “conditioned reflex” is considered established when the conditioned stimulus (the bell) elicits salivation (the conditioned response) in the absence of food. Operant conditioning, on the other hand, refers to the increased probability of a more or less random behavioral response being made by rewarding it. In Edward Thorndike’s original experiments carried out as part of his thesis work at Columbia University in the 1890s, cats learned to escape from a box by pressing the lever that opened a door, allowing the cat to reach a food reward. Although the animals initially pressed the lever only occasionally and more or less by chance, the probability of their doing so increased as the animals learned to associate this action with escape and reward. In Frederick Skinner’s more complete and better-known experiments at Harvard a few decades later, pigeons or rats learned to associate a stimulus with receiving a reward by pressing a bar next to a food dispenser (an apparatus known as the “Skinner box”).

19.5 The Common Strategy An important question is whether these seemingly distinct categories of lifetime learning differ in any fundamental way from each other, or from the way inherited associations are made over evolutionary time. In both classical and operant conditioning, a neural association is forged between the repeated occurrence of a stimulus and a behavior that is in some way rewarding. In classical conditioning, the association is gradually made between sound of a bell and salivation, the bell having assumed the place of food that triggers the innate association. In operant conditioning, response and reward are gradually linked by the animal’s own trial and error actions, the experimenter having stepped aside after specifying the response to be learned. The underlying causes are the same: a set of neural connections is formed and/or strengthened between stimulus and response pathways according to the frequency of occurrence of the stimulus–reward conjunction. In either case the frequency occurrence of stimuli to which animals have responded to in the past endorses the enhancement of neural connections that lead to success, whether in nature or in the laboratory. The same biological strategy operates in evolution. Imagine a population of primitive organisms whose behavior is dictated by rudimentary collections of sensors and primitive neural connections. As stipulated in Darwinian theory, the organization of both sensors and their connections in the population is subject to small, random variations in structure and function. By the same token, recurring energy patterns impacting sensory receptors trigger responses to patterns that also vary. Insofar as perceptions and behaviors promote reproduction in the population, natural selection instantiates the underlying variations of pre-neural and neural

118

19  Stimuli and Behavior

configurations in the nervous systems of progeny. As this process is repeated down the generations, increasingly sophisticated behavioral responses gradually emerge. The point in this routine account is that natural selection—driven by trial-and-­ error responses to repeated energy patterns that are ecologically useful—eventually aligns perceived qualities with biological success rather than physical reality. Behaviors are the end product of stimuli that begin to be shaped before energy impinges on receptors and proceeds with higher order neural processing that has led to what we do, perceive, think, and feel.

19.6 Conclusion In biology, stimuli and the responses to them—motor actions, perceptions, emotions, etc.—are determined by the frequency of occurrence stimulus–response associations which in turn reflects the degree to which the linkage has promoted success ultimately measured by survival and reproduction. But how, more specifically, is this strategy implemented, and how does it differ from computation?

Further Readings Crick F (1995) The astonishing hypothesis: the scientific search for the Soul. Scribner, New York, NY Pavlov IP (1927/1960) Conditioned reflexes: an investigation of the physiological activity of the cerebral cortex. (Anrep GV, translator and editor). Dover, New York, NY Searle JR (2010) A theory of perception: seeing things as they are. Oxford University Press, New York, NY Sherrington SC (1947) The integrative action of the nervous system. Second edition. Yale University Press, New Haven, CT

Chapter 20

Associations

20.1 Introduction Electronic computers generally operate algorithmically by following a set of logical rules. In contrast, biological brains and artificial neural networks are guided by operational utility as described in the preceding chapter. In the case of nervous systems, the primary function is making empirical associations between neurons whose success is ultimately measured by survival and reproduction. There are three general ways of establishing these associations: evolutionary “learning,” lifetime learning from routine experience, and social learning from cultural instruction.

20.2 Associations Wrought by Evolution The advent of nervous systems in evolution on Earth seems to have been more or less contemporaneous with the appearance of animals. In fact, other than nervous systems, there seems nothing much that distinguishes “animal” from “non-animal” organisms. If that makes sense, then how should biologists define nervous systems? A first take might be that nervous systems link information from sensory inputs to motor outputs. Indeed, the early nervous systems presumably comprised networks that were not neurons but excitable cells derived from muscles. This sensory to motor association, however, leads to a decidedly limited concept of neural function. These muscle-like cells would eventually have evolved into neurons to distribute information over longer distances, including information pertinent to non-motor as well as motor behaviors. The impetus for ever more complex nervous systems over evolutionary time is the more sophisticated circuitry required to succeed in additional niches, eventually giving rise to nervous systems like ours. The most obvious consequence of this progress is an ever-growing brain size to © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. Purves, Why Brains Don’t Compute, https://doi.org/10.1007/978-3-030-71064-4_20

119

120

20 Associations

body size ratio. This ratio in modern humans has increased by a factor of about three compared with ancient hominins, and even more when compared with great apes who are our closest primate ancestors. How associations are instantiated over the course of evolution is, in principle, clear enough as indicated in Chap. 19. Inherited changes in the organization and function of ancestral neural systems persisted—or not—according to how well novel connections served the survival and reproductive success of progeny harboring a particular variant. Any configuration of receptors and neural circuitry that associate sensory inputs with more useful behavioral responses would tend to increase in the population, whereas less useful neural associations would not. The associations generated by evolution are not, however, limited to general tendencies, with details to be filled in only by individual experiences in the world. Observations made by ethologists and developmental biologists have shown that surprisingly detailed behaviors can be inherited. Preeminent among these pioneers in the mid-twentieth century were Konrad Lorenz, Nikolas Tinbergen, Karl von Frisch, and Viktor Hamburger. In addition to many other contributions, Lorenz and Tinbergen reported that after emerging from their shells, newly hatched herring gulls crouched down in the nest in response to shadows made by hand-held silhouettes of birds of prey but ignored the shadows of innocuous birds. Von Frisch showed in his work on the “waggle dance” of honeybees, even in insects inherited behaviors can be remarkably sophisticated. Hamburger, the foremost embryologist of the mid-­ twentieth century, demonstrated that nervous system connectivity in chick embryos can be established in the absence of input from the external world, or even sensory input arising from the embryo itself. Even more remarkable are structures built by animals working together without any “master plan.” Far from being a hovel of mud, the nests of some termite species are carefully engineered constructions several meters in height, with specialized compartments dedicated to different functions as well as an airflow system that maintains an appropriate temperature in each unit despite large variations in local weather. Such edifices are the outcome of coordinated actions among millions of individual termites. These and many other inherited behaviors must have been “learned” by the species over the course of evolution.

20.3 Associations Wrought by Lifetime Learning The genesis of neural associations, however, obviously continues after hatching or birth by means of associations made by the mechanisms underlying lifetime learning. Obvious examples in humans are learning how to crawl and eventually walk, how to grasp and manipulate objects, how to attract attention and innumerable others. The mechanistic details are taken up in the next chapter.

20.4  Associations Wrought by Culture

121

20.4 Associations Wrought by Culture In contrast, culture is defined as social learning passed on to conspecifics by specific instruction evident in many animals in addition to humans. Some monkey species use a stick as simple tool to extract termites (a key part of their diet) from dead wood, a skill they pass on by teaching their progeny. Other monkeys use specific calls to warn tribe members about the presence of a particular predator, a skill they also learn from their contemporaries. Pigs show their young how to use pieces of bark as shovels to dig for food, and even crows and some seabirds show an impressive degree of social learning. Social learning—culture—greatly increases the pace at which biological agents acquire new information. Like the termite nest, a human enterprise such as building skyscraper depends on the collective interaction of large numbers of individuals with expertise in different areas. The difference is that the expertise of the termite subgroups in nest construction is inherited, whereas the expertise of human workers making skyscrapers (or anything else) is learned over individual lifetimes. The information can be passed on to others and the corpus of knowledge generally grows from one generation to the next. In humans learning to communicate by language is the preeminent example underlying this process. Infants begin learning almost immediately from their social environment and gain fluency over a few years. Although human culture is certainly an extreme case, the difference with respect to other animals is a matter of degree not kind. Human culture and our status in the world thus boil down to the ability to an extraordinary degree of social learning—in large part due to our ability to use language—and to pass on its fruits without the rudimentary communicative bottleneck that limits social learning in other species. Because we pass on information so much more effectively, it is easy to imagine that we differ in kind rather than degree from the rest of the animal kingdom. From a biological perspective, however, this conclusion is more self-aggrandizing than real. Due to the large number of neural associations and their complexity in our relatively larger brains, human culture is far advanced compared to other species. But there is no compelling evidence that our brains and the rest of our nervous systems are qualitatively different from those of other animals. The distinction is thus between neural associations that change over evolutionary and individual time according to reproductive success, and associations that are put in place and carried forward by “social learning” including ever better ways of storing information and transmitting it to the next generation by oral, written, and now digital means. Nonetheless the driving force underlying social learning is also survival and reproductive success, remote though this relationship may seem.

122

20 Associations

20.5 Conclusion Understanding neuronal associations gleaned over the last century by neuroanatomical, neurophysiological, and behavioral studies point in the same general direction: neural connectivity is continually being updated by the empirical value of behavioral outcomes. This framework means that behavioral responses—including actions, perceptions, emotions, thoughts, or any other consequence of neural activity triggered by exogenous or endogenous stimuli—arise from synaptic connectivity already established, even if only moments before the response occurs. Whether created by natural selection over the course of evolutionary time, by neural plasticity during life or by cultural lessons, the overarching strategy is the same: instantiate neural associations that promote biological success on a trial-and-error basis by ranking the frequency of occurrence of useful behavior to circumvent the inability of biological systems to measure the real world.

Further Readings Bonner JT (1980) The evolution of culture in animals. Princeton University Press, Princeton, NJ Galef BG, Laland KN (eds) (2009) The question of animal culture. Harvard University Press, Cambridge, MA Gould SJ (1989) Wonderful life: the burgess shale and the nature of history. Norton, New York, NY Hunt GR, Gray RD (2003) Diversification and cumulative evolution in New Caledonian crow tool manufacture. Proc R Soc Lond B 270:867–874 Lorenz K (1937) On the formation of the concept of instinct. Nat Sci 25:289–300 Seyfarth RM, Cheney DL (2018) The social origins of language. Princeton University Press, Princeton, NJ Tinbergen N (1953) The herring Gull’s world. Harper and Row, New York, NY

Chapter 21

Mechanisms

21.1 Introduction In thinking about brains as computers a recurring point as been that biological sensors cannot measure or otherwise apprehend physical parameters in the environment. The alternative is that nervous systems make empirical associations based on reward (ultimately survival and reproduction). What, then, are the neurobiological mechanisms used to do so?

21.2 Neural Plasticity The mechanisms fall under the rubric of “neural plasticity.” In addition to the myriad mechanisms that evolution and development use to create organisms in the first place, neural plasticity refers to strengthening or weakening synaptic connections over animal lifetimes by virtue of neural activity. The mechanisms of neural plasticity that have been identified over the past 80 years or so include synaptic facilitation, depression, habituation, sensitization, desensitization, post-tetanic potentiation, long-term potentiation, changes in the size and shape of dendritic spines, the production and elimination of synapses in both developing and adult animals, changes in glial cell functions, and changes in gene expression that may be short or long lasting. And in all likelihood others that remain to be discovered. This plethora indicates the extraordinary importance of modifying neuronal connectivity during life.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. Purves, Why Brains Don’t Compute, https://doi.org/10.1007/978-3-030-71064-4_21

123

124

21 Mechanisms

21.3 Short-Term Changes Short-term plasticity refers to changes in synaptic transmission that last up to a few minutes or in some cases hours. By and large, these changes in synaptic “strength” are due to increased neurotransmitter release when a synapse is activated in rapid succession over a few tens of milliseconds (called synaptic facilitation). Decreased transmitter release occurs when a synapse goes on being activated over several ­hundred milliseconds (called synaptic depression). Marked changes in transmitter release also occur when the synapse is tested after a prolonged period of synaptic activity (post-tetanic potentiation or depression). These effects are typically initiated by altered levels of calcium ions in presynaptic terminals, Ca++ being the mediator of neurotransmitter release in animal nervous systems.

21.4 More Enduring Changes Changes in the strength of synaptic associations that last hours, days, or longer depend on additional mechanisms. In invertebrates, these phenomena are evident in reflex changes called habituation in the case of long-lasting depression, and sensitization in the case of long-lasting potentiation. In mammals, the synaptic change of greatest interest has been long-term potentiation (LTP). In addition to its long duration, the focus on LTP is its occurrence in the hippocampus, a region of the mammalian brain that figures in memory and spatial navigation. In humans, if the hippocampus on both sides of the brain is destroyed—as it was in the famous case of Henry Molaison (referred to for decades in the neurological literature as “patient H.M.”)—the ability to learn and store memories of events that can be expressed verbally is severely compromised. The implication of such cases is that LTP underlies, at least in part, the ability to recall and report events in recent experience (called “declarative memory”). LTP in the hippocampus requires synchronous activity of the pre- and postsynaptic neurons and is mediated by increased postsynaptic calcium entry, which in turn stimulates the incorporation of receptor proteins into the postsynaptic cell membrane. LTP can not only strengthen the synaptic association between a presynaptic and a postsynaptic cell but can generate association with a second presynaptic cell that affects the postsynaptic cell too weakly to produce LTP in its own right. In vivo microscopic imaging shows that neural plasticity can also entail the creation of new synapses (and the disappearance of existing ones) over minutes to months. Longer lasting changes (months or years) are presumably due to altered gene expression mediated by transcription factors and maintained by one or more trophic factors operating at synapses.

21.5 Reward

125

21.5 Reward A general perspective on the mechanisms that foster neural associations is the concept of reward. Understanding reward in neurobiological terms harks back to the work of James Olds and Peter Milner in the 1950s at McGill University. Olds and Milner showed that if a stimulating electrode is placed in a specific region of the brainstem of a rat behaving in a Skinner box, the animal continues to self-stimulate to the detriment of its well-being. This observation implied that the stimulated area was a “reward center” mediating what the animal experienced as wanting and/or liking (in effect, generating motivation and a positive emotional reaction). Olds and Milner—and later many others—eventually showed that several brain areas can give rise to pathological self-stimulation including the lateral hypothalamus, the medial forebrain bundle, the nucleus accumbens in the basal forebrain. It is generally accepted today that these loci are part of a widely connected subcortical system (Fig. 21.1). The shared property of the system is innervation from neurons that use dopamine as a neurotransmitter and arise from a region of the brainstem called the “ventral tegmental area” (VTA). Further studies showed that the injection of dopamine into the areas innervated by the VTA have the same effect as electrical stimulation, and that addictive drugs like cocaine and heroin activate this dopaminergic network. Moreover, many of the drugs used in treating psychoses are antagonists of dopamine receptors, or at least a particular class of them. The receptors for these drugs are found in the regions to which the midbrain dopamine neurons project, the idea being that their activation enhances the effects of dopamine at these

Fig. 21.1  The ventral tegmental area and its projections. See text for explanation. The abbreviations indicate transmission by glutamine, gamma-amino butyric acid, or dopamine

126

21 Mechanisms

sites. The upshot is that this dopaminergic complex provides a physiological and/or emotional “reward” for at least some behaviors. Although this dopaminergic complex remains somewhat murky, any answer to the question of physiological reward today would include the role of dopamine. Most neuroscientists today—and the science media—accept that the release of dopamine is the “reward” provided when a “desire” is satisfied. Dopamine, however, is just one of hundreds of neurotransmitter chemicals in the brain, and dopamine receptors are doing a job that is no different from other receptor molecules that respond to transmitters in complex ways. Of the estimated 86 billion neurons in the brain, less than 0.5 million secrete dopamine, a miniscule 0.00005%. It seems more likely that dopamine rewards only a particular category of behavioral responses. A theoretical concept of dopaminergic reward that has arisen over the last 10 or 15 years is based on the work of neurophysiologist Wolfram Schultz and his collaborators at Cambridge University. Schultz has promoted what he calls the “reward prediction error hypothesis.” The idea is that the release of dopamine conveys the difference—the “error”—between an expected reward and what actually happens. The occurrence of a predicted reward elicits no response from the dopaminergic system. On the other hand, if an unexpected reward occurs, the difference error is “positive” and the behavior is reinforced by dopamine release. If, however, an expected reward fails to occur, then the behavior is suppressed. The “reward value” of a behavioral pathway is thus strengthened by the simultaneous release of dopamine locally. This series of events would presumably lead to the repetition of successful behaviors thus strengthening biologically useful connectivity and further promoting the relevant input–output association. An alternative perspective is that the only reward involved is the strengthening and weakening of synaptic connections mediated by synaptic activity, as described earlier. In this case dopamine would simply be producing the same effects initiated by neurotransmitters in any neural pathway.

21.6 Conclusion Nervous systems change over time by the enhancement or weakening of neural connections during life by one or more of the many mechanisms of neural plasticity. The connectivity underlying behaviors that work is gradually strengthened and the behaviors that fail are weakened or lost. From this perspective the nervous system is not providing “rewards” in the way this term implies. Any rewards, whether short or long term, must ultimately come from biological success in the world, and the mechanisms described here are simply means to that end. In any event, neural plasticity sidesteps the concept of neural function based on computation, coding, and representation. These mechanisms are evidently the biological realities underlying conceptual stand-ins such as backpropagation or other computational theories.

Further Readings

127

Further Readings Dittrich L (2016) Patient H.M.: a story of memory, madness, and family secrets. Random House, New York Olds J, Milner P (1954) Positive reinforcement produced by electrical stimulation of septal area and other regions of rat brain. J Comp Physiol Psychol 47:419–427 Schultz W (2016) Dopamine reward prediction error signaling: a two-component response. Nat Rev. Neurosci 17:183–195 Simon HA (1962) The architecture of complexity. Proc Am Phil Soc 106:467–482

Chapter 22

Reflexes

22.1 Introduction An important question in sorting out whether brains operate algorithmically or empirically is whether any response to a stimulus differs from a “reflex.” Many neuroscientists take a reflex to be an automatic link between stimulus and response that is outdated and/or inadequate, given the sophistication of cognitive and much other behavior. The implication of the argument here, however, is that all behaviors are reflexive.

22.2 Behavioral Responses as Reflexes The classical work on reflexes was carried out early in the twentieth century by Charles Sherrington and his collaborators using sensory-motor associations such as the knee-jerk (myotatic) reflex and other responses largely determined by neurons in the mammalian spinal cord (Fig. 22.1). For Sherrington, a reflex had three parts which he called the “reflex arc”: a sensory receptor, an effector of some sort and neural connections between the two. This definition has often been interpreted to mean that reflexes are limited to links between stimuli and behavioral responses that are, by some criterion, “simple.” Sherrington, however, warned that everything in the brain and the rest of the nervous system is directly or indirectly connected to everything else. Moreover, “simple” reflexes can be concatenated to generate arbitrarily complex reactions to sensory or other neural inputs. The advantages of reflex responses for an animal are clear enough: the underlying circuitry links sensory inputs to outputs as directly as possible, allowing the nervous system to respond with speed and accuracy to significant stimuli that occur often. In the case of the knee-jerk reflex in Fig. 22.1, the biological advantage is © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. Purves, Why Brains Don’t Compute, https://doi.org/10.1007/978-3-030-71064-4_22

129

130

22 Reflexes

Extensor muscle

Muscle sensory receptor

(afferent) axon

3A

Cross section of spinal cord 2B 2A

1 Flexor muscle

Motor (efferent) axons

4

1 Hammer tap stretches tendon, which, in turn, stretches sensory receptors in leg extensor muscle

3B

2

(A) Sensory neuron synapses with and excites motor neuron in the spinal cord (B) Sensory neuron also excites spinal interneuron

2C

Interneuron

3 (A) Motor neuron conduts action potential to synapses on extensor muscle fibers, causing contraction

4

Leg extends

(C) Interneuron synapse inhibits motor neuron to flexor muscles

Fig. 22.1  The neural circuitry underlying a spinal reflex. The boxes describe the sequence of events

rapid extension of the leg that prevents a fall after having tripped. It does not follow, however, that reflex responses are simple that they are limited to motor acts or that they entail only “lower order” neural circuitry such as that in the spinal cord. Sherrington was well aware that the concept of a “simple” reflex is, in his words, a “convenient…fiction.” His point was that since every part of the nervous system is in some measure connected to every other part, there is little sense in trying to distinguish “simple” reflexes from any other responses to stimuli. An example of a concatenated reflex is shown in Fig. 22.2. Having stepped on a tack and withdrawn the injured foot, the muscles of the opposite leg must extend to keep the victim from falling when standing on one leg. Is there, then, a level of the nervous system where, in principle, reflexes stop being “simple” and more complex concatenation begins? That question seems nonsensical. Stepping on the tack sends information centrally to various cortical regions that elicit further responses whose complexity is ultimately unlimited including motor reactions to extract the tack, thoughts about who left the tack on the floor, what a suitable punishment might be in order for that bit of negligence, and so on. In a “simple” animal like the fruit fly where the complete wiring of the brain is known, the complexity of its reflex responses is mind boggling.

22.3  Is All Behavior Reflexive?

131

Fig. 22.2  A concatenation of simple reflexes that generates a more complex behavioral response. See text for explanation

22.3 Is All Behavior Reflexive? Although it is difficult to imagine how this degree of anatomical complexity could generate responses to sensory inputs in a fraction of second by a series of algorithmic computations, if neural responses arise on the basis neural associations already in place by virtue of experience, the responses just need to be triggered. Moreover, there is no evidence that any response to sensory input and its sequelae differ from a spinal reflex other than the number and extent of the underlying synaptic connections involved. What, then, is a better way of describing what is actually being associated and why? Since sensory inputs cannot convey the physical parameters of the local environment, it cannot be the characteristics of the “real world” that are linked to responses. Whether one likes it or not, the most plausible answer is the response to any stimulus is activation of the continually updated connectivity based on the success or failure of accumulated experience. It does not follow, however, that the responses will always be identical or that making reflexive responses would forestall novelty and/or creativity. Given size of the search spaces that biological problems entailed, even the knee-jerk response would never involve exactly the same neurons and synaptic connections.

132

22 Reflexes

22.4 Counterarguments The idea that the brain and the rest of the nervous system are fundamentally reactive is unappealing to most neuroscientists and probably everyone else. We humans imagine that we add self-generated thoughts, “creativity” and other egocentric inputs to whatever goes on reflexively. Thus one counter argument to the assertion that responses to stimuli arise automatically from previously established associations is that cognitive behavior is “flexible,” whereas reflexes are not. We can, for instance, alter or even reverse a decision already embarked upon, much to our advantage. Indeed, behavioral flexibility is sometimes taken to be the hallmark of “cognition.” But there is nothing special in changing an earlier course of action on the basis of subsequent sensory input. When a bacterium “decides” to swim up a nutrient gradient, it stops when the nutrient in that direction peters out. It is not the concept of a reflex that is suspect, but the idea that we or other animals make “decisions” that transcend reflex behavior. A related argument is that associations can be affected by “cognitive strategies” that arise at higher levels neural processing. Thus the commonly drawn dichotomy in psychology between “bottom-up” and “top-down” operations that underlie behavioral responses. This interpretation of what the human nervous system is doing again rests on a false distinction: there is no evidence that belies Sherrington’s conclusion that the distinction between lower-order reflexes and higher-order “cognitive” processing is fictitious. The responses to stimuli we think of as perception, attention, emotion, memory, trains of thought, and so on are no different in principle than any other responses to stimuli. The idea that cognition must have a different basis pertaining as it does to animals with larger and more complex nervous systems is not credible. There is no evidence that reflexive behaviors and “cognitive” responses differ in kind. The attraction of this historical distinction has more to do with justifying human exceptionalism than biological fact. Finally, since different sensory modalities must interact to produce optimal behavior, and since the outputs of sensory systems must in turn interact with other neural functions to insure the most useful behavioral outcomes, an empirical strategy is likely to be a general principle of neural operation. Understanding sensory stimuli and their consequences in empirical terms implies that neural circuitry—whether at the level of the peripheral receptors, subcortical nuclei, or the cerebral and cerebellar cortices—operates on the same basis and serves the same purpose: to promote biological success in a world whose physical properties cannot be conveyed by the transduction of energy, much less extracted by operations at any later stage of neural processing.

Further Readings

133

22.5 Conclusion The implication that we are automata, that behavior is robotic, and that we lack “free-will” is of course anathema to most of us. But if this is what science points to, then sooner or later we will have to accept it. Nervous systems are machines— remarkable though they may be—that make modifiable associations based on evolution and lifetime reinforcement learning. In both evolution and life, the frequency occurrence of stimuli endorses neural connections that have led to success, whether in nature, the school room, or the laboratory.

Further Readings Skinner BF (1953) Science and human behavior. Macmillan, New York, NY Sherrington SC (1947) The integrative action of the nervous system, 2nd edn. Yale University Press, New Haven, CT Purves D (2019) Brains as engines of association: an operating principle for nervous systems. Oxford University Press, New York, NY

Part VI

Other Theories

There are of course other ideas about how brains work that differ from the empirical argument here. This section considers the most prominent of these alternatives, which are explicitly or implicitly underpinned by algorithmic computation.

Chapter 23

Feature Detection

23.1 Introduction Of other ideas about how subjective experience is elicited by sensory inputs, the most prevalent is detecting image features that represent stimulus sources in the environment. This interpretation is based on the properties of neurons determined by electrophysiology.

23.2 Feature Detection If one were to ask neuroscientists today how the objective world is related to the subjective world of perceptions and other behavioral responses, the answer would probably be by detecting, representing, and responding to features of the environment elaborated as models by a variety of sensory modalities, receptors, and neuronal receptive field properties that we and other animals possess. In addition to common sense, enthusiasm for this interpretation stems largely from electrophysiological studies of the visual systems of experimental animals that got going about 70 years ago. In a paper published in 1953, neuroscientist Steven Kuffler showed how the output cells in the retina (retinal ganglion cells in cats) organize the information from photoreceptors that is sent on to the rest of the visual brain. Earlier work by Ragnar Granit working on horseshoe crabs in the 1930s and by Horace Barlow working on frogs at about the same time as Kuffler made the same general point about the organization of retinal receptive fields in invertebrates and amphibians, respectively. When Kuffler moved to Harvard from Johns Hopkins in 1959, he brought with him two postdoctoral fellows—David Hubel and Torsten Wiesel—who were intent on pursuing how the higher stations of the primary visual pathway that processed © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. Purves, Why Brains Don’t Compute, https://doi.org/10.1007/978-3-030-71064-4_23

137

138

23  Feature Detection

the output of the retina. Over the next 25 years, they did just that, contributing a body of work that remains highly influential. Like Kuffler, Hubel and Wiesel initially used anesthetized cats placed in front of a screen on which patterns of light were presented (e.g., spots, bars, and circular patches). Although under anesthesia, neurons in the visual system still respond to stimulus properties by increasing (or decreasing) their baseline firing rates. Figure 23.1, for example, shows the responses of a neuron to a series of differently oriented bars. A typical neuron in the primary visual cortex fires vigorously in response to a bar in a particular place and orientation but not otherwise. The evidence from this and many similar experiments that examined responses of cortical and subcortical neurons in cats and later monkeys led to the idea that the activity of a neuron represents features of stimulus sources at a given locus in visual space, orientation in this example. Acting in concert, neuronal populations responding to different stimulus features were imagined generate a “model” of the world that we ended up seeing. But as detailed in Part IV, what we actually perceive—and how neural responses are related to sources in the world—raises issues that Hubel and Wiesel largely ignored. Although a book summarizing this work published in 2005 is titled Brain and Visual Perception, it includes very little about perception. Hubel and Wiesel presumably surmised that understanding perception and other behaviors would emerge once the physiology and anatomy of the visual brain were sufficiently understood. Furthermore, since perceptual responses are difficult to quantify let alone relate to neurophysiology, many neuroscientists no doubt felt then (and perhaps now) that perception should be left for psychologists and philosophers to struggle with. Stimulus Stimulus orientation presented (B)

(C)

Spike rate

(A) Experimental setup Light bar stimulus projected on screen

Record Recording from visual cortex

Stimulus orientation

0

1 2 3 Time (s)

Fig. 23.1  Determining the receptive field characteristics of a neuron in the primary visual cortex of an anesthetized cat. (a) The recording setup. (b) Panels showing the relative activity of the neuron (action potentials elicited per unit time) in response to an oriented bar of light moving across a screen within the region of visual space to which the neuron is sensitive. The tick marks are the action potentials generated by different orientations of the bar. (c) The neuron’s responses to different stimulus orientations shown as a “tuning curve”

23.3  Neurons that Respond to more Specific Stimuli

139

To survive, biological agents like us and other animals must behave usefully with respect to the objective reality of stimulus sources. As already described, however, the lack of measuring tools in biological sensory systems and the inevitable conflation of input information preclude directly implementing this logical strategy. But if the image on the retina cannot reveal the orientation or other physical characteristics of objects in the world, how can the visual system generate perceptions and other behaviors that work?

23.3 Neurons that Respond to more Specific Stimuli More recent electrophysiological studies have added to the idea that brains operate by detecting features, often quite specific ones. An example is neurons in brain regions that respond to especially important objects such as faces. This ability was first reported in studies of neurons in the inferior temporal cortex of monkeys in the 1970s. These findings were confirmed and extended in studies of patients who showed specific deficits in feature recognition following damage to the same general regions of the brain, and by increasingly detailed electrophysiological and functional imaging studies of non-human primates and eventually normal human subjects over the following decades. In theory, such percepts might be generated by a single cortical neuron at the apex of a processing hierarchy—a so-called grandmother cell. This idea was first suggested by Horace Barlow in the 1970s, although William James had raised the same possibility at the turn of the nineteenth century. Barlow suggested grandmother cells more as a debating point rather than a plausible idea. Nonetheless the debate had merit. Could a single neuron encode all the aspects that define your grandmother in a compilation of her various attributes? Although that scenario seems improbable, single cortical neurons can indeed respond to remarkably specific stimuli. These studies were carried out by neurosurgeon Itzak Fried, neuroscientist Rodrigo Quiroga and their colleagues at the University California at Los Angeles. During the course of surgery performed for the excision of epileptic foci, patients were exposed to stimuli of “famous” people while the team recorded the responses of individual neurons. The example often cited concerns on neuronal recordings in a patient that showed activity in response to a picture of TV and movie star Jennifer Aniston from any angle, her voice, or her name in auditory or written form (not surprisingly the patient in this case was a big fan). Although this may seem unequivocal evidence for a grandmother type of cell, the interpretation is more complicated. The fact that “Jennifer Aniston neurons” were found so readily implies that there are many such cells in a given individual. Moreover, it seems unlikely that the particular cells studied functioned only in response to a single well-known star. As the authors point out, there are likely to be populations of neurons in all of us that become active not only to particular stimuli while at the same time contributing to responses elicited by other stimuli. Although there are billions of neurons in the human brain, even that number would be insufficient if every object, condition, or other response was predicated on uniquely dedicated neurons.

140

23  Feature Detection

23.4 Perception in Monkeys More recently, work on face perception in monkeys has extended this sort of evidence. The idea suggested by vision scientists Le Chang and Doris Chao at UC Berkeley is that neurons that respond to faces are actually encoding by facial features as locations in spatial matrix. In this conception, specific faces are not identified as such; rather a coordinate system is used to measure and ultimately compile individual features. Other investigators are taking advantage of specific neurons identified by light stimulation following transfection with genes that produce proteins that can make active neurons visible. The result of this work in mice is that perceptions can be elicited by the stimulation of just a handful of neurons, perhaps as few as one or two. At this point what all this means is hard to know. Although the results are consistent with the general idea of “grandmother” cells, a single spark can start a fire; similarly, activation of one or a few neurons can affect a large number of connected cells. Another approach adding to this evidence was provided by Bill Newsome and his colleagues at Stanford. In the so-called MT (medio-temporal) area of the rhesus monkey cortex, it is possible to both record and stimulate groups of neurons while the monkey indicates what it is perceived. After sufficient training, the animal can use eye movements to report when a stimulus comprising a collection of dots moving randomly gradually made to move coherently in one direction or another. In this way the monkey can indicate its threshold for perceiving motion direction. The result is that stimulating neurons in area MT interfered with the monkeys’ perceptual ability, implying that the neurons are indeed responsible for perceiving this stimulus feature.

23.5 Back to Sherrington These observations accord with Sherrington’s dictum that every region in brains is directly or indirectly connected to most if not all other regions, the idea being that optimal behavior at any moment must be based on the full range of sensory input, emotions, memories, etc. The concept that small numbers of nerve cells are responsible for detecting features of the world is probably not viable, some evidence to the contrary notwithstanding. Brains would not work to the advantage of animals if they did not take into account all the information available at any given moment.

23.6 Conclusion The pros and cons of these electrophysiological studies aside, neural reporting of stimulus features as a means of generating successful behaviors in the world will always be stymied by the inability of nervous systems to measure real-world

Further Readings

141

properties. The most plausible picture of how brains generate behavioral responses that succeed is by using the conjoint activity neurons in many functionally different regions of the nervous system to link accumulated experience with that responses that have been successful.

Further Readings Barlow HB (1995) The neuron doctrine in perception. In: Gazzaniga MS (ed) Cognitive neurosciences. MIT Press, Cambridge, MA, pp 415–436 Hubel DH (1988) Eye, brain and vision. WH Freeman, Baltimore, MD Hubel DH, Wiesel TN (1962) Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. J Physiol 160:106–154 Hubel DH, Wiesel TN (2005) Visual perception and the brain. Oxford University Press, Oxford, UK Quiroga RQ, Reddy L, Kreima G, Koch C, Fried I (2005) Invariant visual representation by single neurons in the human brain. Nature 435:1102–1107 Sugrue LP, Corrado GS, Newsome WT (2005) Choosing the greater of two goods: neural currencies for valuation and decision making. Nat Rev Neurosci 6:363–375

Chapter 24

Statistical Inference

24.1 Introduction The complex relationship of stimuli and their real-world provenance presents a puzzle for any theory of neural operation. This challenge has led some investigators to ask whether brains might operate according a statistical appreciation of the world, offering an alternative to detecting features that cannot be measured by biological sensors.

24.2 Statistical Inference The first and most influential advocate of statistical inferences as a way of contending with the uncertain sources of stimuli was the German physicist and physiologist Hermann von Helmholtz. Helmholtz summarized his conception by proposing that “raw sensations” arising from peripheral sense organs could be modified by information derived from individual experience. He described this process as making “unconscious inferences” about reality, thus generating perceptions that more nearly represented stimulus sources than ambiguous input-level “sensations.” Despite Helmholtz’s sensible arguments in the second half of the nineteenth century, sensory science during most of the twentieth century was understandably dominated by the enormous success of modern neuroanatomy and neurophysiology. A plausible assumption in much of this research was that understanding perception would be achieved through increasingly sophisticated studies of the properties of neurons and their connections. As a result, the possible role of the statistics in determining perceptions received relatively little attention. The last two or three decades, however, have witnessed a resurgence of interest in statistical approaches driven © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. Purves, Why Brains Don’t Compute, https://doi.org/10.1007/978-3-030-71064-4_24

143

144

24  Statistical Inference

primarily by neuroscientists with backgrounds in computer science and mathematics. The most popular methodology underlying this way of thinking is Bayesian decision theory.

24.3 Bayes’ Theorem Thomas Bayes was an eighteenth century minister and amateur mathematician whose short paper in 1763 proved a theorem showing how conditional probabilities can be used to make inferences. Although Bayes’ purpose in elaborating this theorem is unclear, it has provided a framework for addressing statistical problems whose solution depends on an assessment of hypotheses that are only more or less likely to be true. An example is making a clinical decision based on a medical test that produces false positives. Suppose a test for diagnosing a disease is indicative of the disorder in most cases but gives a false positive result in a significant number of patients who have no problem. Bayes theorem provides the quantitative rationale for dealing this sort of situation. The theorem is usually written in the form P(H|E) = P(H) . P(E|H)/P(E) where H is a hypothesis, E the evidence pertinent to its validity and P probability. The first term on the right side of Bayes’ equation, P(H), is referred to as the prior probability distribution or simply the prior, and it is a statistical measure of confidence in the hypothesis, absent any present evidence pertinent to its truth or falsity. Taking visual perception as an example, the prior describes the relative probabilities of different physical states of the world pertinent to retinal images, i.e., the relative probability of various illuminants, surface reflectance values, object sizes, and so on having given rise to the retinal image. The second term, P(E|H), is called the likelihood function. If hypothesis H were true, this term indicates the probability that the evidence E would have been available to support it. In vision, given a particular state of the physical world (i.e., a particular combination of illumination, reflectance properties, and object sizes), the likelihood function describes the probability that the state generated the retinal image in question. The product of the prior and the likelihood function, divided by a normalization constant, P(E), gives the posterior probability distribution, P(H|E). The posterior probability distribution defines the probability of hypothesis H being true, given the evidence E. Thus the posterior distribution indicates the relative probability of a given stimulus having been generated by one or another of the different physical realities that could have been the source of the stimulus in question. To illustrate how Bayes’ theorem can be used to rationalize the percept elicited by a stimulus, take the apparent lightness of a patch having a particular luminance value in the retinal image. Under natural conditions, the luminance in some region of the stimulus is, to a first approximation, a product of the amount of light falling on object surfaces in the world (the illumination) and the reflectance efficiency functions of those surfaces (Fig. 24.1). To further simplify the situation, consider illumination (Ω) and reflectance (R) the only parameters needed to specify the

145

24.3  Bayes’ Theorem A

B

40 60 Reflect ance (%

20

)

80

100

0

100 80 40 20

40 Reflecta

60 nce (% )

20 80

100

ti o n

60 0 0

mi na

40 20

0.01

Illu

0 0

na ti

60

on

80

Probability

100

I ll u mi

Probability

Ω xR=L 0.01

0

C

Fig. 24.1  Characterizing the relationship between the luminance of a visual target and its possible physical sources in Bayesian terms. (a) A prior distribution of illuminant (Ω) and reflectance values (R) in the physical world. The distributions in this example show illumination on an arbitrary scale from 0 to 100 and reflectance on a scale from 0% to 100%. (b) The dashed red line on the prior distribution indicates the set of points at which the product of illumination and reflectance equals the luminance of the target (L). The posterior distribution P(Ω, R|L) obtained by multiplying the prior by the likelihood function identifies a section of the prior indicated by dashed line. (c) The posterior is a section of the prior whose colors indicate the relative probabilities

physical reality underlying the image (in fact, the amount of light reaching the retina is affected by the transmittance of the intervening atmosphere, the characteristics of the ocular media, and a host of other physical factors). Applying Bayes’ theorem, the lightness perceived by an observer in this example is predicted by the posterior probability distribution P(Ω, R|L), where L stands for the luminance in the image. In making this prediction, the first step is to determine the prior distribution P(Ω, R), i.e., the probability distribution of different conditions of illumination occurring together with different surface reflectance values in the physical world. The next step is to derive the likelihood function P(L|Ω, R) which describes for each possible combination of illumination and reflectance the probability that the combination would generate the luminance value of the image patch in question. The posterior distribution—which describes the relative probabilities of the physical sources that could have generated the image under consideration—is obtained by multiplying the prior distribution by the likelihood function and dividing by the normalization factor in Bayes’ theorem.

146

24  Statistical Inference

24.4 The Problem with a Bayesian Approach Using Bayesian decision theory is an important advance that formalizes Helmholtz’s qualitative proposal about “visual inferences” as a means of contending with the inevitably uncertain sources of stimuli. Its implementation, however, presents difficulties. Because the posterior distribution indicates the relative probabilities of a set of possible sources in the world, a particular source (i.e., a particular combination of illumination and reflectance in the example above) must be selected from this set if the aim is to predict what lightness or brightness an observer will actually see. The usual way of addressing this further requirement is to assume that the visual system makes this “choice” according to the behavioral consequences associated with each perceptual “decision.” The influence of behavioral consequences is typically expressed in terms of the discrepancy between the decision made and the actual state of the world, which over the full range of the possible choices defines a gain-loss function. Since there is no a priori way to model this function—indeed, given the enormous number of variables involved, a realistic gain-loss function for some aspect of vision would be extraordinarily difficult to determine—the relative cost of different behavioral responses is also assumed. A common assumption is that observers will “choose” the percept that corresponds to the maximum value in posterior probability distribution, since this choice would generally minimize the discrepancy between the percept and the actual state of the world. Bayesian theory, however, cannot convey the actual states of the world because animal sensory systems have no way of making such measurements. This biological deficiency means there is no information on which to base valid inferences.

24.5 Conclusion Although statistics is an extraordinarily powerful way of addressing otherwise intractable problems, Bayesian theory fails because the physical data that need to be analyzed statistically are not available to biological agents. In contrast, empirical ranking theory supposes that visual percepts represent an ordering of stimuli (brighter vs. dimmer, larger vs. smaller, louder vs. softer, and so on) according to accumulated experience—i.e., the frequency of occurrence of all the relevant stimulus patterns experienced in the past.

Further Readings Bayes T (1763) An essay toward solving a problem in the doctrine of chances. Philos Trans R Soc 53:370–418 Clark A (2013) Whatever next? Predictive brains, situated agents, and the future of cognitive science. Behav Brain Sci 36:181–204

Further Readings

147

Geisler WS, Kersten D (2002) Illusions, perception and Bayes. Nat Neurosci 5:508–510 Turner RS (1994) In the eye’s mind: vision and the Helmholtz-Hering controversy. Princeton University Press, Princeton, NJ Kersten D, Yuille A (2003) Bayesian models of object perception. Curr Opin Neurobiol 13:150–158 Lotter W, Kreiman G and Cox D (2018) A neural network trained to predict future video frames mimics critical properties of biological neuronal responses and perception. https://arxiv.org/ abs/1805.10734v2

Chapter 25

Summing Up

25.1  In Brief Despite the enormous number of neurons in human brains and the complexity of their connectivity and functions, understanding nervous systems may be simpler than imagined. The key points are as follows: 1. There are two ways to solve problems: algorithmically versus empirically. 2. Biological sensing systems cannot measure the objective world. 3. Thus the nervous system cannot link physical reality and subjective experience directly. 4. The implication is that nervous systems must operate empirically. 5. This conclusion is borne out by the ability to predict perceptions on the basis of accumulated experience. 6. A signature of empirical operation is that perceptions always differ from physical measurements. 7. Over evolutionary time empiricism is mediated by natural selection. 8. Over animal lifetimes empiricism is mediated by neural plasticity. 9. Since accumulated experience continuously updates neural connectivity and functions, responses are already determined when stimuli occur. 10. A major reason for the evolution of an empirical strategy of neural operation is the enormous complexity of the problems that animals routinely face. 11. Given this argument, artificial neural networks trained in a sufficient variety of environments should eventually equal and ultimately outdo the abilities of biological brains.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. Purves, Why Brains Don’t Compute, https://doi.org/10.1007/978-3-030-71064-4_25

149

Figure Sources

Figure 2.1. After Figure 3.3 in Purves D, Lotto RB (2003) Why We See What We Do. An Empirical Theory of Vison. Sunderland, MA: Sinauer. Figure 9.1. After Figure 7.1 in , Purves D, Lotto RB (2003) Why We See What We Do. An Empirical Theory of Vison. Sunderland, MA: Sinauer. Figure 9.2. After Figure 2.4 in Howe CQ Purves D (2005) Perceiving Geometry: Geometrical Illusions Explained by Natural Scene Statistics. New  York, NY: Springer. Figure 9.3. After Figure 3.2 in Howe CQ Purves D (2005) Perceiving Geometry: Geometrical Illusions Explained by Natural Scene Statistics. New  York, NY: Springer. Figure 9.4. After Figure 4.8 in Purves D, Lotto RB (2011) Why We See What We Do Redux. A Wholly Empirical Theory of Vison. Sunderland, MA: Sinauer. Figure 10.1. After Figure 4.2 in Howe CQ Purves D (2005) Perceiving Geometry: Geometrical Illusions Explained by Natural Scene Statistics. New  York, NY: Springer. Figure 10.2. After Figure 4.3 in Howe CQ Purves D (2005) Perceiving Geometry: Geometrical Illusions Explained by Natural Scene Statistics. New  York, NY: Springer. Figure 10.3. After Figure 4.4 in Howe CQ Purves D (2005) Perceiving Geometry: Geometrical Illusions Explained by Natural Scene Statistics. New  York, NY: Springer. Figure 10.4. After Figure 4.5 in Howe CQ Purves D (2005) Perceiving Geometry: Geometrical Illusions Explained by Natural Scene Statistics. New  York, NY: Springer.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. Purves, Why Brains Don’t Compute, https://doi.org/10.1007/978-3-030-71064-4

151

152

Figure Sources

Figure 10.5. After Figure 4.6 in Howe CQ Purves D (2005) Perceiving Geometry: Geometrical Illusions Explained by Natural Scene Statistics. New  York, NY: Springer. Figure 11.1. After Figure 3.1A in Purves D, Lotto RB (2003) Why We See What We Do. An Empirical Theory of Vison. Sunderland, MA: Sinauer. Figure 11.2. After Figure 3.1B in Purves D, Lotto RB (2003) Why We See What We Do. An Empirical Theory of Vison. Sunderland, MA: Sinauer. Figure 11.3. After Figure 3.4 in Purves D, Lotto RB (2003) Why We See What We Do. An Empirical Theory of Vison. Sunderland, MA: Sinauer. Figure 12.1. After Figure 3.1 in Purves D, Lotto RB (2003) Why We See What We Do. An Empirical Theory of Vison. Sunderland, MA: Sinauer. Figure 12.2. After Figure 3.2 in Purves D, Lotto RB (2003) Why We See What We Do. An Empirical Theory of Vison. Sunderland, MA: Sinauer. Howe CQ and Purves D (2005) Perceiving Geometry: Geometrical Illusions Explained by Natural Scene Statistics. New York, NY: Springer. Figure 12.3. After Figure 3 in Yang Z, Purves D (2004) The statistical structure of natural light patterns determines perceived light intensity. Proc Natl Acad Sci 101(23): 8745–8750. Figure 13.1. After Figure 3.5 in Purves D, Lotto RB (2011) Why We See What We Do Redux. A Wholly Empirical Theory of Vison. Sunderland, MA: Sinauer. Figure 13.2. After Figure 5.7 in Purves D, Lotto RB (2003) Why We See What We Do. An Empirical Theory of Vison. Sunderland, MA: Sinauer. Figure 13.3 After Figure 3.11 in Purves D, Lotto RB (2011) Why We See What We Do Redux. A Wholly Empirical Theory of Vison. Sunderland, MA: Sinauer. Figure 13.4. After Figure in Lotto, RB, Purves D (1999) The effects of color on brightness. Nature Neurosci 2: 1010–1014. Figure 14.1. After Figure 3.22 in Purves D, Lotto RB (2011) Why We See What We Do Redux. A Wholly Empirical Theory of Vison. Sunderland, MA: Sinauer. Figure 14.2. After Figure 3.23 in Purves D, Lotto RB (2011) Why We See What We Do Redux. A Wholly Empirical Theory of Vison. Sunderland, MA: Sinauer. The data in [A] are redrawn from Wyszecki and Stiles, 1958; [B] is from Long et al., 2006. Figure 14.3. After Figure 5 in Long F, Yang Z, Purves D (2006) Spectral statistics in natural scene predict hue, saturation, and brightness. Proc Natl Acad Sci 103(15): 6013–6018. Original data are from Purdy, D. M. (1931) Am. J. Psych. 43, 541–559. Figure15.1. After Figure 1 in Wojtach WT, Sung K, Truong S, Purves D (2008) An empirical explanation of the flash-lag effect. Proc Natl Acad Sci, 105(42): 16338–16343.

Figure Sources

153

Figure15.2. After Figures 8 in Wojtach WT, Sung K, Truong S, Purves D (2008) An empirical explanation of the flash-lag effect. Proc Natl Acad Sci, 105(42): 16338–16343. Figure15.3. After Figure 2 in Wojtach WT, Sung K, Truong S, Purves D (2008) An empirical explanation of the flash-lag effect. Proc Natl Acad Sci, 105(42): 16338–16343. Figure15.4. After Figure 6 in Wojtach WT, Sung K, Truong S, Purves D (2008) An empirical explanation of the flash-lag effect. Proc Natl Acad Sci, 105(42): 16338–16343. Figure 16.1. After Figure 6.12 in Purves D, Lotto RB (2011) Why We See What We Do Redux. A Wholly Empirical Theory of Vison. Sunderland, MA: Sinauer. Figure 16.2. After Figure 2 in Sung K, Wojtach WT, Purves D (2009) An empirical explanation of aperture effects Proc Natl Acad Sci, 106 (1): 298–303. Figure 16.3. After Figure 3 in Sung K, Wojtach WT, Purves D (2009) An empirical explanation of aperture effects Proc Natl Acad Sci, 106 (1): 298–303. Figure 16.4. After Figure 4 in Sung K, Wojtach WT, Purves D (2009) An empirical explanation of aperture effects Proc Natl Acad Sci, 106 (1): 298–303. Figure 16.5. After Figure 6 in Sung K, Wojtach WT, Purves D (2009) An empirical explanation of aperture effects Proc Natl Acad Sci, 106 (1): 298–303. Figure 17.1. After Figure in After Figure 4.6 in Purves D, Lotto RB (2011) Why We See What We Do Redux. A Wholly Empirical Theory of Vison. Sunderland, MA: Sinauer. Figure 17.2. Courtesy of Mark Williams. Figure 17.3. After Figure 5.1 in Purves D, Lotto RB (2011) Why We See What We Do Redux. A Wholly Empirical Theory of Vison. Sunderland, MA: Sinauer. Figure 17.4. Courtesy of Catherine Q. Howe. Figure 18.1 After Figure 1 in Ng CJ and Purves D (2019) An Alternative Theory of Binocularity. Front Comput Neurosci 09 https://doi.org/10.3389/fncom.2019.00071 Figure 18.2. After Figure 4 in Ng CJ and Purves D (2019) An Alternative Theory of Binocularity. Front Comput Neurosci 09 https://doi.org/10.3389/fncom.2019.00071 Figure 18.3 After Figure  14  in Purves D and Lichtman JW (1985) Principles of Neural Development. Sunderland, MA: Sinauer. Courtesy of D.H.  Hubel and T. Wiesel. Figure 19.1. After Figure 3 in Purves D, Morgenstern Y and Wojtach WT (2015) Perception and reality: Why a wholly empirical paradigm is needed to understand Vision. Front Syst Neurosci 9: 156. https://doi.org/10.3389/fnsys.2015.00156

154

Figure Sources

Figure 22.1. After Figure  1.7  in Purves et  al. (2012). Neuroscience (5th ed). Sunderland, MA: Sinauer. Figure 22.2. After Figure  16.14  in Purves et  al. (2012). Neuroscience (5th ed). Sunderland, MA: Sinauer. Figure 23.1. After Figure  12.8  in Purves et  al. (2012). Neuroscience (5th ed). Sunderland, MA: Sinauer. Figure 24.1. After Figure 2 in Howe CQ, Lotto RB, Purves D (2006) Comparison of Bayesian and empirical ranking approaches to visual perception. J Theor Biol 241: 866–875.

Glossary: Definitions of Some Relevant Terms

Action potential  The electrical signal conducted along neuronal axons by which information is conveyed from one place to another in the nervous system. Algorithm  A set of rules or procedures specified in logical notation, typically (but not necessarily) carried out by a computer. Animal  Organisms with nervous systems. Artificial intelligence An ill-defined phrase used to describe mimicking brain functions in non-biological machines. Artificial neural network  A computer architecture for solving problems by changing the connectivity of nodes according to feedback from the output. Association  In the context of nervous systems, the synaptic connections that link neurons or groups of neurons together. Association cortex  Regions of cerebral neocortex defined by their lack of involvement in primary sensory or motor processing. Autonomic nervous system  The neural apparatus that controls visceral behavior. Includes the sympathetic, parasympathetic, and enteric systems Bayes’ theorem  A theorem that formally describes how valid inferences can be drawn from conditional probabilities. Binocular  Pertaining to both eyes. Bottom-up  A term that loosely refers to the flow of information from sensory receptors to cerebral cortex. Brain  The cerebral hemispheres, brainstem, and cerebellum. Brainstem  The portion of the brain that lies between the diencephalon and the spinal cord; comprises the midbrain, pons, and medulla. Brightness  Technically, the apparent intensity of a source of light; more generally, a sense of the effective overall intensity of a light stimulus (see lightness). Cell  The basic biological unit in plants and animals, defined by a cell membrane that encloses cytoplasm and the cell nucleus. Cell body  The portion of a neuron that houses the nucleus.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. Purves, Why Brains Don’t Compute, https://doi.org/10.1007/978-3-030-71064-4

155

156

Glossary: Definitions of Some Relevant Terms

Central nervous system  The brain and spinal cord of vertebrates (by analogy, the central nerve cord and ganglia of invertebrates). Cerebellum  Prominent hindbrain structure concerned with motor coordination, posture, balance, and some aspects of cognition. Cerebral cortex  The superficial gray matter of the cerebral hemispheres. Cerebrum  The largest and most rostral part of the brain in humans and other mammals, consisting of the two cerebral hemispheres. Circuitry  In neurobiology refers to the connections between neurons; usually pertinent to some particular function (as in “visual circuitry”). Cognition  A poorly defined term that refers to “higher-order” neural functions such as perception, attention, and memory. Color  The subjective sensations elicited in humans by different distributions of energy in the spectrum of light stimuli. Cones  Photoreceptors specialized for high visual acuity and the perception of color. Consciousness  A contentious concept that includes the ideas of wakefulness, awareness of the world, and awareness of the self as an actor in the world. Context  The information provided by the surroundings of a “target.” The division of a scene into target and surround is useful, but arbitrary, since any part of a scene provides contextual information for any other part. Contrast  The difference, usually expressed as a percentage, between the luminance (or spectral distribution in the case of color) of two surfaces. Corpus callosum The midline fiber bundle that connects the two cerebral hemispheres. Cortex  The gray matter of the cerebral hemispheres and cerebellum, where most of the neurons in the brain are located. Critical period A restricted developmental period during which the nervous systems of humans or other animals are particularly sensitive to the effects of experience. Culture  Information passed on by social learning rather than heredity. Dendrite  A neuronal process extending from the cell body that receives synaptic input from other neurons. Depth perception  General term used to indicate the perception of distance from the observer (can be either monocular or stereoscopic). Disparity  The geometrical difference between the view of the left and right eye in animals with frontal eyes and stereoscopic depth perception. Electrophysiology  Study of the nervous system by means of electrical recording. Empirical  Derived from past experience, effectively by trial and error (the opposite of analytical). Feature  Physical characteristic of an object or a stimulus. Feature detection  The idea that sensory systems detect and represent the characteristics of stimuli and/or the objects and conditions that give rise to them. Fixation  Looking steadily a particular point in visual space; the fixation point is where the lines of sight from the left and right eyes intersect. Fovea  Area of the human retina specialized for high acuity; contains a high density of cones and few rods.

Glossary: Definitions of Some Relevant Terms

157

Frequency  How often something occurs over a unit of time or space. Frequency distribution  Histogram or other graphical representation showing the relative frequency of occurrence of some event. Frontal lobe  One of the four lobes of the brain; includes all the cortex that lies anterior to the central sulcus and superior to the lateral fissure. Functional magnetic resonance imaging (fMRI)  A functional imaging technique that reveals relative brain activity based on paramagnetic differences between saturated and unsaturated blood oxygen levels. Gene  A hereditary unit located on a chromosome that encodes the information needed to construct a particular protein. Genetic algorithm  A computer-based scheme for simulating evolution of artificial neural networks. Genome  The complete set of an animal’s genes. Genotype  The genetic makeup of an individual. Geometrical illusions  Discrepancies between the measured geometry of a visual stimulus (i.e., measurements of length and angle) and the resulting perception. Gestalt (psychology)  A school of psychology founded by Max Wertheimer in the early twentieth century in which the overall qualities of a scene are taken to be determinants of its perception; a “gestalt” in German means “an integrated perceptual whole.” Gray matter  Term used to describe regions of the central nervous system rich in neuronal cell bodies; includes the cerebral and cerebellar cortices, the nuclei of the brain, and the central portion of the spinal cord. Heuristic  A rule of thumb that can be used to help solve a problem. Hierarchy  A system of higher and lower ranks; in sensory systems, the idea that neurons in the input stages of the system determine the properties of higher-order neurons. Higher-order  Neural processes and/or brain areas taken to be further removed from the input stages of a system; sometimes used as a synonym for cognitive processes. Higher-order neurons  Neurons that are relatively remote from peripheral sensory receptors or motor effectors. Hippocampus  A specialized cortical structure located in the medial portion of the temporal lobe; in humans concerned with short-term declarative memory, among other functions. Homeostasis  Maintaining organismal function in a stable state. Illumination  The light that falls on a scene or surface. Illusions  A confusing term that refers to discrepancies between the physically measured properties of a visual stimulus and what is actually seen. In fact, all percepts are in this sense illusory. Image  The representation in art, on the retina or in perception of an external form and its characteristics. Information  A variable presented in a way that allows an observer (or some other receiver) to extract a signal from background noise.

158

Glossary: Definitions of Some Relevant Terms

Information theory  Theory of communication channel efficiency elaborated by Claude Shannon in the late 1940s. Innervate  Establish synaptic contact with another neuron or target cell. Innervation  Referring to the synaptic contacts made on a target cell or larger entity such as a muscle. Input  The information supplied to a neural or artificial processing system by some form of energy. Intelligence  A vague descriptor referring to problem solving ability; the harder the problems solved, the more intelligent an organism is taken to be. Inverse problem The impossibility of knowing the world directly through the senses because of the conflation of information at the level of sensory receptors. Invertebrate  An animal without a backbone (technically, a member of the subphylum Invertebrata) Learning  The acquisition of novel behavior through repeated experience. Light  The range of wavelengths in the electromagnetic spectrum that elicits visual sensations in humans (photons having a wavelengths of about 400–700 nm). Lightness  The subjective sense of an object surface elicited by luminance, experienced as achromatic values ranging from white through grays to black (see brightness). Line of sight  An imaginary line from the center of the fovea through the point of fixation. Lobes  The four major divisions of the cerebral cortex (frontal, parietal, occipital, and temporal). Long-term potentiation (LTP)  A particular type of long-lasting enhancement of synaptic strength as a result of prior activity. Luminance  The physical (photometric) intensity of light returned to the eye or some other detector, adjusted for the sensitivity of the average human observer. Machine  Any man-made device or more broadly any apparatus that accomplishes a purpose through by the operation of a series of causally connected parts. Mammal  An animal the embryos of which develop in a uterus, and whose young suckle at birth (technically, a member of the class Mammalia). Map  A systematic arrangement of information in space. In neurobiology, the ordered projection of axons from one region of the nervous system to another, by which the organization of a relatively peripheral part of the body (e.g., the retina) is reflected in the organization of the nervous system (e.g., the primary visual cortex). Microelectrode  A recording device (typically made of wire or a glass tube pulled to a point and filled with an electrolyte) used to monitor electrical potentials from individual or small groups of nerve cells. Mind  The content of consciousness at any point in time. Although used in everyday speech (e.g., “This is what I have in mind” or “My mind is a blank”), it has no scientific meaning. Modality  A category of function; for example, vision, hearing, and touch are different sensory modalities. Monocular  Pertaining to one eye.

Glossary: Definitions of Some Relevant Terms

159

Monocular cues  Term used to describe information about distance arising from the view of a single eye. Motion  The changing position of an object defined by speed and direction in a frame of reference. Motor  Pertaining to biological movement. Motor cortex  The region of the cerebral cortex in humans and other mammals lying anterior to the central sulcus concerned with motor behavior. Motor neuron  A nerve cell that directly innervates skeletal or smooth muscle. Motor system  Term used to describe the central and peripheral structures that support motor behavior. Muscle fibers Cells specialized to contract when their membrane potential is depolarized. Nerve  A collection of peripheral axons that are bundled together and travel a common route in the body. Nerve cell  Synonym for neuron. Neural circuit  A collection of interconnected neurons dedicated to some neural processing goal. Neural network  Typically refers to an artificial network of nodes whose connections change in strength as a means of solving problems by learning. Neural plasticity  The ability of the nervous system changes as a function of experience; typically applied to the changes in the efficacy or prevalence of synaptic connections. Neural processing  A general term used to describe operations carried out by neural circuitry. Neural system  A collection of peripheral and central neural circuits dedicated to a particular function (e.g., the visual system and the auditory system). Neuroglial cell  (glial cell) Includes several types of non-neural cells in the peripheral and central nervous system that carry out a variety of functions that do not directly entail signaling. Neuron  Cell specialized for the conduction and transmission of electrical signals in nervous systems. Neuronal receptive field  The properties of stimuli that elicit a change in the activity of a sensory neuron. Neuroscience  Study of the structure and function of nervous systems. Neurotransmitter  A chemical agent released at synapses that affects the signaling activity of the postsynaptic target cells. Neurotransmitter receptor  A molecule embedded in the membrane of a postsynaptic target cells that bindes a neurotransmitter. Ontogeny  The developmental history of an individual animal; sometimes used as a synonym for development. Orientation  The arrangement of an object in the three dimensions of Euclidean space. Parallel processing  Processing information simultaneously by different components or pathways in a sensory (or other) system.

160

Glossary: Definitions of Some Relevant Terms

Perception  Typically refers to the subjective awareness of the external or internal environment. Can be conscious or unconscious. Peripheral nervous system  The nerves and neurons that lie outside the brain and spinal cord (i.e., outside the central nervous system). Phenomenology  Word used to describe the observed behavior of something. Phylogeny  The evolutionary history of a species or other taxonomic category. Primary motor cortex  A major source of descending projections to motor neurons in the spinal cord and cranial nerve nuclei, located in the precentral gyrus and essential for the voluntary control of movement. Primary sensory cortex  Any one of several cortical areas in direct receipt of the thalamic or other input for a particular sensory modality. Primary visual cortex  See striate cortex. Primate  Order of mammals that includes lemurs, tarsiers, marmosets, monkeys, apes, and humans (technically, a member of this order). Probability  The likelihood of an event, usually expressed as a value from 0 (will never occur) to 1 (will always occur). Probability distribution  Probability of a variable having a particular value, typically shown graphically. Processing  A general term that refers to the neural activity underlying some function. Psychology  The study of higher-order brain processes in humans and other animals. Psychophysics  The study of higher-order processes by quantitative methods that typically involve reports by human subjects of subjective responses to stimuli. Rank  Location on a scale, often expressed as a percentile. Real-world  Phrase used to convey the idea that there is a physical world even though it is directly unknowable through the senses. Receptive field  The area of a receptor surface (e.g., retina or skin), the stimulation of which cause a sensory nerve cell to respond by increasing or decreasing its baseline activity. See also neuronal receptive field. Receptor cells  The cells in a sensory system that transduce energy from the environment into neural signals (e.g., photoreceptors in the retina and hair cells in the inner ear). Reflectance  The percentage of incident light reflected from a surface. Reflex  A stereotyped response elicited by a defined stimulus. Usually taken to be restricted to “involuntary” actions. Retina  Neural component of the eye that contains the photoreceptors (rods and cones) and the initial processing circuitry for vision. Retinal ganglion cells  The output neurons of the retina whose axons form the optic nerve. Retinal image  Pattern focused on the retina by the optical properties of the eye. Since an image is seen only after processing by the visual system, the term can be misleading when this caveat is not understood. Rods  System of photoreceptors specialized to operate at low light levels.

Glossary: Definitions of Some Relevant Terms

161

Saturation  The aspect of color sensation pertaining to the perceptual distance of a color from neutrality (thus an unsaturated color is one that approaches a neutral gray). Scale  An ordering of quantities according to their magnitudes. Sensation  The subjective experience of energy impinging on an organism’s sensory receptors (a word not clearly differentiated from perception). Sensitivity  The relative ability to respond to the energy in a sensory stimulus. Sensory  Pertaining to sensing the environment. Sensory system  All the components of the central and peripheral nervous system concerned with sensation in a modality such as vision or audition. Sensory transduction  Process by which energy in the environment is converted to neural signals. Signal  A function that conveys information about some phenomenon. Somatic sensory cortex  That region of the mammalian neocortex concerned with processing sensory information from the body surface, subcutaneous tissues, muscles, and joints; in humans located primarily in the posterior bank of the central sulcus and on the post central gyrus. Somatic sensory system  Components of the nervous system that process sensory information about the mechanical forces that act on both the body surface and on deeper structures such as muscles and joints. Sparse coding  The idea that the detailed information from sensory receptor organs is abstracted and represented by the specific activity of relatively few neurons in the cerebral cortices. Species  A taxonomic category subordinate to genus; members of a species are defined by extensive similarities and the ability to interbreed. Spectrophotometer  A device for measuring the distribution of power across the spectrum of light. Spectrum  A plot of the amplitude of a stimulus such as light or sound as a function of frequency over some period of sampling time. Spinal cord  The portion of the central nervous system that extends from the lower end of the brainstem (the medulla) to the cauda equina in the lower back. Stereopsis  The special sensation of depth that results from fusion of the two views of the eyes when they regard relatively nearby objects. Striate cortex  The primary visual cortex in the occipital lobe in humans and other primates (also called Brodmann’s area 17 or V1). So named because the prominence of layer IV in myelin-stained sections gives this region a striped appearance. See primary visual cortex. Stimulus  Generic term for a pattern of energy that activates sensory receptor cells. Sulcus  (pl. sulci) Valleys between gyral ridges that arise from folding of the cerebral hemispheres. Synapse  Specialized apposition between a neuron and a target cell; transmits information by release and reception of a chemical transmitter agent. Temporal lobe  The hemispheric lobe that lies inferior to the lateral fissure.

162

Glossary: Definitions of Some Relevant Terms

Thalamus  A collection of nuclei that forms the major component of the diencephalon. Although its functions are many, a primary role of the thalamus is to relay sensory information from the periphery to the cerebral cortex. Transduction  The cellular and molecular process by which energy is converted into neural signals. Universal turing machine  A computer that using a series of steps can solve any problem than can be formulated in numerical (logical) terms. Velocity  Vector defined by the speed and direction of a moving object. Vertebrate  An animal with a backbone (technically, a member of the subphylum Vertebrata). Visual field  The area of visual space normally seen by one or both eyes (referred to, respectively, as the monocular and binocular fields). White matter  A general term that refers to large axon tracts in the brain and spinal cord; the phrase derives from the fact that axonal tracts have a whitish cast when viewed in the freshly cut material.

Index

A Accumulated empirical information, 79 Action potentials, 25, 26 Acute angles, 57, 62 Adaptation, 77–79 Algorithm definition, 21 Algorithmic computation, 32, 34, 37 Algorithmic processes, 87 Algorithms, 3, 4 in machines, 17, 18 Al-Khwarizmi, 17 AlphaGo, 41, 42 AlphaZero, 41, 42 Altair home computer, 24 Altair kit, 24 Alternative perspective, 126 Ambiguity, 51 Amplifiers, 115 Analytical engine, 18, 23 Anatomical correspondence, 108 Angle magnitude, 61 Angle misperception, 62 Angle perception, 57 Angle projections, 60 Animal nervous systems, 124 Anomalous perception, 60 Arithmetical operations, 17 Artificial brain, 33 Artificial intelligence, 5, 6, 12, 39, 40 Artificial neural networks (ANN), 3 attribute, 32 back progress, 35 computation, 31

nodes/neurons, 36 operations, 38 problem solving, 31, 34 Association cortex, 32 Associations animals, 120 behavioral responses, 120 biological perspective, 121 culture, 121 evolution, 119, 120 human culture, 121 motor association, 119 nervous system connectivity, 119, 120 neural associations, 120, 121 organization and function, 120 social learning, 121 survival and reproduction, 119 Atari “2600” suite, 40 Autocode, 24 Autonomic behaviors, 70 B Babbage, 17–20, 23 Back propagation biological neural networks, 36 chain rule, 35 definition, 35 gradient descent, 36 network connections, 35 neural network theory and practice, 35 plausibility, 36 BASIC computer language, 24

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. Purves, Why Brains Don’t Compute, https://doi.org/10.1007/978-3-030-71064-4

163

164 Bayes theorem, 144, 145 behavioral consequences, 146 distributions, 145 illumination, 144 posterior distribution, 145 probability, 144, 145 quantitative rationale, 144 retinal images, 144 visual perception, 144 Bayesian decision theory, 146 Bayesian theory, 146 Beginner's All-purpose Symbolic Instruction Code (BASIC), 24 Behavioral responses, 49 Behavioral success, 70 Behaviors, 70 Bezold-Brücke effect, 83, 84 Binary digits, 23 Binocular circuitry artificial neural networks, 108 horopter, 109 inhibitory connections, 109 networks, 109 simulated environment, 108 Biological agents, 11 Biological measurements, 10–11 Biological neural circuitry, 32 Biological sensing systems, 149 Biological sensors, 48, 57 Biological success, 70, 132 Black diamonds, 92 Board games, 40, 41 Boole, 19, 20 Boolean “exclusive-or” function, 33 Boolean algebra Shannon, 19 Brain/non-biological system, 49 C Cellular and molecular mechanisms, 49 Chess, 40, 41 Classical color discrimination tests, 81 Classical conditioning, 116, 117 Coding definition, 23 electronic computers, 26 languages, 24 neural, 25 practical complement, 23 rate coding, 26 Cognition, 132 Color constancy, 76–78 Color contrast, 78 Color databases, 81

Index Color discriminations, 82 Color psychophysics, 81 Color space, 76 Color vision theory, 81 Color vison, 75 Colorimetric functions, 83, 85 Common sense, 76 Communication system, 25 Compilers, 24 Comprehensive computational theory, 48 Computation, 4, 31 Computational machine, 4 Computers, 20 Concatenated reflex, 130 Conditional probability distributions, 84–85 Contrast, 65, 76 Co-occurring luminance relationships, 70 Cornell Aeronautical Laboratory, 33 Credit assignment, 37 Cumulative experience, 67 Cybernetics, 31 D Darwinian theory, 117 Decidability, 18 Deep neural network, 42 Defense Advanced Research Agency (DARPA), 21 Dense coding, 25 Depression, 124 Desensitization, 123 Difference Engine, 17 Digital Equipment Corporation (DEC), 21 Dim illumination, 84 Discrepancies, 51, 53, 58 Distance cues, 103 Dopaminergic reward, 126 E Ebbinghaus effect, 102 Electromagnetic radiation, 48 Electromechanical Bombe, 20 Electronic brains, 32 Electronic computers, 20, 33, 47 Electronic Discrete Variable Automatic Computer (EDVAC), 20 Electronic Numerical Integrator and Computer (ENIAC), 20, 23 Electrophysiological studies, 33, 140 Empirical explanation, 62, 92 Empirical framework, 53, 88 Empirical information, 69

Index Empirical ranking, 72, 81 Empirical scale, 52, 54 Empirical strategy, 49 Enthusiasm, 35 Evolution, 37 Evolutionary learning, 119 Experiences, 48, 49 F Facilitation, 123 Fascination, 75 Feature detection anesthesia, 138 animals, 140 animals possess, 137 biological agents, 139 cortical and subcortical neurons, 138 cortical neurons, 139 electrophysiological studies, 139 invertebrates and amphibians, 137 Jennifer Aniston neurons, 139 medio-temporal, 140 nervous systems, 140 neuronal populations, 138 neuroscientists, 137 perception in monkeys, 140 perceptual responses, 138 photoreceptors, 137 sensory inputs, 137 Sherrington’s dictum, 140 stimuli, 139 visual pathway, 137 Feedback, 49, 69 Flash-lag effect, 90–92 Formula translation (FORTRAN), 24 Frequency of occurrence, 51, 58, 59, 71, 85 Fröhlich effect, 90 Full-range perceptual phenomena, 51 G Game and reinforcement learning, 42 Game tree complexity, 40 Ganglion cells, 65 Gelb, A., 65 Geometrical perceptions, 57, 60 Geometrical straight lines, 54 Go game, 40–42 Goedel’s theorem, 18 Google DeepMind, 40 Gradient descent, 36 Gradients, 35 Grandmother cell, 25, 139

165 H Habituation, 124 Hidden layers, 35 Higher-order brain areas, 47 Homebrew Computer Club, 24 Horizontal line projections, 54 Horizontal lines, 54 Horopter, 111 Hubel and Wiesel, 138 Hue, 75, 76, 81 Human experience, 60 Human perception, 84, 85 Human visual environment, 71 Human vs. machine intelligence, 39 I IBM’s “short code”, 24 IBM’s program “Deep Blue”, 42 Identical luminance, 65 Identical patches, 72 Illumination, 65, 76, 77 Illusions, 87 Image sequence, 88–90 Images, 53 Incompleteness theorem, 18 Information, 47 Input–output associations, 42 Input–output repetition, 32 Intelligence, 4, 5 International Business Machines (IBM), 21 Intersecting straight lines, 59 Inverse problem, 10 J Jennifer Aniston neurons, 139 Just noticeable differences (JNDs), 81 K Knee-jerk reflex, 129 L Languages, 23 Laser range finders, 9 Laser range scanning, 57 Laser scanning, 53, 62 Lateral inhibitory effects, 62 Lifetime learning, 70, 120 Lighter/brighter, 63, 64 Light-dark boundary, 64, 65 Lightness, 75

166 Lightness-brightness sense of color, 76 Lightness-darkness accumulated experience, 66 incidental consequence, 65 luminance, 63 perception, 63, 64 physical measure, 63 responses, 63 values, 67 visual system, 64 Linear classifier, 33 Linear projections, 53 Linear sources, 53 Logic, 4 Logical rules, 3, 4, 6 Long-term potentiation (LTP), 124 in hippocampus, 124 postsynaptic cell membrane, 124 Lovelace, 17, 18, 23 Luminance, 65, 67, 70 Luminance patterns, 72 Luminance values, 70, 71 M Machine intelligence, 39 Machines, 12 definition, 6 Mainframe computers, 21 Marr’s effort, 48 Marr’s influence, 48 Mathematical and logical theories, 4 McCulloch and Pitts, 31–33 Micro Instrumentation and Telemetry Systems (MITS), 24 Microsoft, 24 Mind-brain philosophers, 3 Minutemen system, 21 Modern computation, 20–21 Modern neuroscience, 48 Monochromatic stimulus, 82, 83 Monochromatic wavelengths, 84 Monty Hall Problem, 4, 5 Moon illusion, 104, 105 Motion definition, 87 direction, 90 flash-lag effect, 90 human experience, 89 measuring, 88 perceived, 87 perception, 87 sense of motion, 88

Index stimuli response, 89 3-D object, 90 Motion direction aperture, 95, 98 characteristic, 95 circular aperture, 96, 99, 100 image sequences, 96, 99 movement, 97 projections, 100 psychophysical results, 99 uniform distribution, 95 vertical aperture, 98 Motion parallax, 102 MuZero, 41 N Natural images, 69 Natural spectral patterns, 85 Nervous systems, 126 Neural associations, 125 Neural circuitry, 51 Neural coding, 25, 26 Neural mechanisms, 47 Neural networks back propagation, 35 computing, 32 concept, 32 electronic computers, 33 enthusiasm, 36 feedback, 35 perceptrons, 33 Neural plasticity, 123 Neural processing, 26, 32 Neural representation, 25 Neurobiological explanations, 62 Neuronal associations, 122 Neurons, 31, 36 Neuroscience, 3, 4, 42, 48 Newtonian reality, 9 Nodes, 32 Non-biological machine, 12, 47 Non-biological neural networks, 5 O Object geometry, 52 Object motion, 87 Object size empirical scale, 105 factors, 102 geometry, 102 illusion, 101 images, 101

Index monocular cues, 102 object occlusion, 103 provocative puzzle, 105 retina, 101 size-contrast, 101 stimuli, 101 two-dimensional presentations, 102 2-D size, 101 visible horizon, 104 zenith Moon, 105 Object speeds, 87 Objective–subjective relationships, 69 Obtuse, 57, 58 Ocular dominance, 109, 110 stereo vision, 110 Operant conditioning, 117 Orientations, 59 Orientation-selective neurons, 62 P PDP-8, 21 Peakedness, 25 Perceived angles, 61 Perceived geometry, 51 Perceived lengths, 55 Perceptions, 9, 11, 33, 47–49 Perceptual qualities, 75 Perceptual space, 54 Peripheral sensors, 48 Personal computers, 24 Photometers, 9, 10 Physical length, 56 Physical motion, 89 Physical source, 57, 60 Physics, 9–12 Physiological/computational explanations, 85 Planar surfaces, 59, 60 Post-tetanic potentiation, 124 Practical/conceptual challenge, 6 Precludes mapping stimulus features, 48 Primary color categories, 75, 76 Primary color perception, 79 Prior probability distribution, 144 Programming language, 24 Projected angles, 61 Projected line, 95 Projected speeds, 88–90, 92 Projected surface areas, 104 Protractors, 9, 10 Psychologists, 4 Psychophysical relationships, 81 Psychophysics, 60, 61 Puzzling effects, 79

167 R Rate coding, 26 Rationalization, 70 Reality definition, 9 Newtonian, 9 objective, 9, 11, 12 physical, 10–12 subjective, 11, 12 Real-world sources, 59 Rebel genius, 31 Receptive fields, 64, 65 Recurring patterns, 69 Redundancy, 26 Reference lines, 58, 59 Reflexes, 129 advantages, 129 classical work, 129 cognitive, 132 counter argument, 132 local environment, 131 nervous system, 130 neural circuitry, 130 neurons and synaptic connections, 131 responses, 130, 131 simple, 129 spinal reflex, 131 Reproductive success, 69 Retinal images, 51, 52, 54 Retinal projections, 52 Reward, 32 Rosenblatt, 33, 34 Rubik’s cube, 39 Rule-based computation, 42 Rulers, 9, 10 Rules, 5 S Sampling projections, 89 Saturation, 75, 76, 81 Scientific American magazine, 5 Search spaces, 37 Seeing color, 75 Self-movement, 87 Sense of color, 75, 76 Sensitization, 123 Sensory modalities, 12 Sensory stimuli, 132 Sensory systems, 10, 48 Sherrington, 129 Short-term plasticity, 124 Simultaneous lightness/brightness contrast, 64 Size-distance paradox, 102

168 Skinner box, 40 Social learning, 12, 121 Solution, 36 Space race, 21 Sparse coding, 25 Sparse model, 47 Sparsity, 25 Spatial intervals, 51, 52 Spatio-temporal energy model, 107 Spectra, 77, 79, 80 Spectral contrasts, 77 Spectral information, 79, 85 Spectral patterns, 79 Spectral qualities, 85 Spectral relationships, 81, 82 Spectral return, 76 Spectral stimuli, 77, 79, 85 Spectral surrounds, 78 Spectrophotometers, 9 Squares, 18 Statistical inferences assumption, 143 biological sensors, 143 Helmholtz’s sensible arguments, 143 methodology, 144 role, 143 sensations, 143 Stereopsis, 110 correspondence problem, 107 horopter, 108 image points, 107 perception, 107 stereo vision, 108 Stereoscopic depth, 110 Stimuli, 10–12, 69, 70 concept, 115 conditioned reflex, 117 conditioning, 116 energy patterns, 117 environments, 115 natural selection, 118 nervous system, 115 operant conditioning, 117 patterns, 116 peripheral sensors, 116 physiologists and neurobiologists, 116 plasticity, 116 responses, 118 survival and reproduction, 118 Stimulus line, 54

Index Stimulus patterns, 70 Straight-line projections, 53 Subjective values, 64 Sunlight, 85 Supercomputers, 24 Supervised learning, 33 Supervisor, 37 Systematic misestimation, 60 T Telecommunication system, 25 Template configurations, 71 Temporal coding, 26 Three-dimensional space project, 88 Tic-tac-toe squares, 40, 41 Trial and error, 11, 12 Trial-and-error experience, 79 Trial-and-error learning, 3, 6, 32, 37, 39 Trial-and-error responses, 36, 70 Turing, 18, 20 Turing machine, 18, 19, 21, 31 Turing’s Bombe, 23 U Unconscious equivalents, 9 Unconscious inference, 143 Unexpected noise, 69 Universal discrepancies, 52 Unsupervised credit assignment, 37 Unsupervised reinforcement learning, 42 Unsupervised reward learning, 37 V Valid inferences, 146 Ventral tegmental area (VTA), 125 Vertical line projections, 54 Vertical movement, 98 Vertical projected lines, 54 Virtual environments, 89 Visual brain, 56, 64 W Wavelengths, 83–85 Werbos, 35 World sensory systems, 69