The Oxford Handbook of Philosophy of Perception 9780199600472, 0199600473

Citation preview

The Oxford Handbook of 

PHILOSOPHY OF PERCEPTION

The Oxford Handbook of 

PHILOSOPHY OF PERCEPTION Edited by

MOH A N M AT T H E N

1

3 Great Clarendon Street, Oxford, ox2  6dp, United Kingdom Oxford University Press is a department of the University of Oxford. It furthers the University’s objective of excellence in research, scholarship, and education by publishing worldwide. Oxford is a registered trade mark  of Oxford University Press in the UK and in certain other countries © The several contributors 2015 The moral rights of the authors have been asserted First Edition published  2015 Impression:  1 All rights reserved. No part of this publication may be reproduced, stored  in a retrieval system, or transmitted, in any form or by any means, without  the prior permission in writing of Oxford University Press, or as expressly permitted by law, by licence or under terms agreed with the appropriate reprographics rights organization. Enquiries concerning reproduction outside the scope  of  the above should be sent to the Rights Department, Oxford University Press,  at  the address  above You must not circulate this work in any other  form and you must impose this same condition on any acquirer Published in the United States of America by Oxford University  Press 198 Madison Avenue, New  York, NY 10016, United States of America British Library Cataloguing in Publication  Data Data available Library of Congress Control Number: 2015931387 ISBN 978–0–19–960047–2 Printed and bound  by CPI Group (UK) Ltd, Croydon, cr0  4yy Links to third party websites are provided by Oxford in good faith  and for information only. Oxford disclaims any responsibility for the materials contained in any third party website referenced in this  work.

Contents

List of Contributors 

Introduction  Mohan Matthen

ix 1

PA RT I   H I S TOR IC A L BAC KGROU N D 1. Perception in Ancient Greek Philosophy  Victor Caston

29

2. Perception in Medieval Philosophy  Dominik Perler

51

3. Skepticism and Perception  Baron Reed

66

4. Perception in Early Modern Philosophy  Alison Simmons

81

5. Perception in Philosophy and Psychology in the 19th and Early 20th Centuries  100 Gary Hatfield 6. Sense-Data  Paul Snowdon

118

7. Phenomenological Approaches  Charles Siewert

136

PA RT I I   C ON T E M P OR A RY PH I L O S OPH IC A L  A PPROAC H E S 8. Perceptual Representation/Perceptual Content  Bence Nanay

153

9. Perception and the First Person  Christopher Peacocke

168

vi   Contents 10. Nonconceptual Content  Wayne Wright

181

11. Disjunctivism  Heather Logue

198

12. Action-Based Accounts of Perception  Pierre Jacob

217

13. Perceptual Reports  Berit Brogaard

237

PA RT I I I   T H E  SE N SE S 14. Vision  David R. Hilbert

257

15. Audition  Matthew Nudds

274

16. Touch  Frédérique de Vignemont and Olivier Massin

294

17. The Chemical Senses  Barry C. Smith

314

18. The Bodily Senses  J. Brendan Ritchie and Peter Carruthers

353

19. Unconscious Perception  Jesse J. Prinz

371

PA RT I V   W H AT W E PE RC E I V E 20. Object Perception  Roberto Casati

393

21. Primary and Secondary Qualities  Peter Ross

405

22. Colour Perception  Kathleen Akins and Martin Hahn

422

Contents   vii 23. Perception and Space  Jérôme Dokic

441

24. Perception and Time  Robin Le Poidevin

459

25. Speech Perception  Casey O’Callaghan

475

26. Musical Perception  Charles Nussbaum

495

27. Own-Body Perception  Alisa Mandrigin and Evan Thompson

515

28. Perception of Pain  Valerie Gray Hardcastle

530

29. Perceiving Nothings  Roy Sorensen

542

PA RT V   I N T E GR AT I NG SE N S ORY I N F OR M AT ION 30. The Individuation of the Senses  Mohan Matthen

567

31. Perceptual Attention  John Campbell

587

32. Multisensory Perception  Tim Bayne and Charles Spence

603

33. Perceptual Constancy  Jonathan Cohen

621

34. How Do Synaesthetes Experience the World?  Malika Auvray and Ophelia Deroy

640

35. Substituting the Senses  Julian Kiverstein, Mirko Farina, and Andy Clark

659

viii   Contents

PA RT V I   F R A M E WOR K S F OR PE RC E P T ION 36. Similarity Spaces  Diana Raffman

679

37. Bayesian Perceptual Psychology  Michael Rescorla

694

38. Signal Detection Theory  E. Samuel Winer and Michael Snodgrass

717

39. Information Theory  John Kulvicki

734

40. Modularity of Perception  Ophelia Deroy

755

PA RT V I I   BROA DE R PH I L O S OPH IC A L  I S SU E S 41. The Epistemology of Perception  Susanna Siegel and Nicholas Silins

781

42. Perceptual Learning  Robert L. Goldstone and Lisa A. Byrge

812

43. Perception and Demonstratives  Imogen Dickie

833

44. Nonhuman Animal Senses  Brian L. Keeley

853

45. Perception and Art  Dominic  McIver Lopes

871

Author Index  Subject Index

885 907

List of Contributors

Kathleen Akins  Simon Fraser University Malika Auvray CNRS-CR1 Tim Bayne  University of Manchester Berit Brogaard  University of Miami Lisa A. Byrge  Indiana University John Campbell  University of California, Berkeley Peter Carruthers  University of Maryland Roberto Casati  École Normale Supérieure, Paris Victor Caston  University of Michigan Andy Clark  The University of Edinburgh Jonathan Cohen  University of California, San Diego Ophelia Deroy  University of London Imogen Dickie  University of Toronto Jérôme Dokic  École des Hautes Études en Sciences Sociales and Institut Jean-Nicod Mirko Farina  Macquarie University Robert L. Goldstone  Indiana University Martin Hahn  Simon Fraser University Valerie Gray Hardcastle  University of Cincinnati Gary Hatfield  University of Pennsylvania David R. Hilbert  University of Illinois at Chicago Pierre Jacob  École Normale Supérieure, Paris Brian L. Keeley  Pitzer College Julian Kiverstein  University of Amsterdam John Kulvicki  Dartmouth College Robin Le Poidevin  University of Leeds

x   List of Contributors Heather Logue  University of Leeds Dominic McIver Lopes  University of British Columbia Alisa Mandrigin  University of Warwick Olivier Massin  Université de Genève Mohan Matthen  University of Toronto Bence Nanay  University of Antwerp and University of Cambridge Matthew Nudds  University of Warwick Charles Nussbaum  University of Texas at Arlington Casey O’Callaghan  Washington University in St Louis Christopher Peacocke  Columbia University Dominik Perler  Humboldt-Universität zu Berlin Jesse J. Prinz  The Graduate Center, CUNY Diana Raffman  University of Toronto Baron Reed  Northwestern University Michael Rescorla  University of California, Santa Barbara J. Brendan Ritchie  University of Maryland Peter Ross  California State University, Pomona Susanna Siegel  Harvard University Charles Siewert  Rice University Nicholas Silins  Cornell University; Yale-NUS College, Singapore Alison Simmons  Harvard University Barry C. Smith  University of London Michael Snodgrass  University of Michigan Paul Snowdon  University College London Roy Sorensen  Washington University in St Louis Charles Spence  University of Oxford Evan Thompson  University of British Columbia Frédérique de Vignemont CNRS E. Samuel Winer  Mississippi State University Wayne Wright  California State University, Long Beach

I n troduction Mohan Matthen

Perception is the ultimate source of knowledge about contingent facts.1 We know about our surroundings because we are able to experience them through perception; we know about scientific phenomena because they are observed. Epistemology is therefore very much concerned with the evidential value of perception. Analytic epistemology is concerned with the rational grounding that perception lends belief; empiricist philosophy of science erects the entire edifice of scientific knowledge on the back of perceptual observation. The rationality of perceptual grounding is contested, of course. On one side, it is contested by those, like Hume, who think we never have reason to believe in contingencies. According to him, we arrive at contingent beliefs about matters beyond mere sense-impressions by the association of ideas, and not by reason. On the other side, it is impugned by rationalists like Plato and Descartes, who think that perception is incoherent—both complain, for example, that it fails to intimate shape in a way that suffices to ground geometry, the authoritative science of shape—and far too evanescent to offer genuine and secure knowledge. Still even the opposition focuses on a critique or reinterpretation of observation and its epistemic value. Given the centrality of perception to epistemology, one would expect that the philosophical study of perception would be a focal philosophical topic. It has not been: neither traditional epistemology nor traditional philosophy of science has been particularly concerned to engage in a determined and scientifically informed investigation of the nature of perception itself. Both articulate puzzles and theories that come out of deep and original thinking about the problem of knowledge yoked to relatively superficial and dogmatic thinking about perception. In the last part of the twentieth century, this situation began to change. Philosophers began to use sensory psychology as a source of new insights about the nature of perception. Thanks to growing interest in perception—how it operates, what it reveals—and the development of new analytic tools, the philosophy of perception is, once again, a vital area of philosophical inquiry. Taken together, the chapters of this Handbook are an introduction to new philosophical thinking about perception. This Introduction presents an overview of some global 1 I  am grateful to Ophelia Deroy and Barry Smith for critically reading through the whole Introduction. Lynne Godfroy was (as always) very helpful in matters of exposition. Other more specific debts are recorded in other notes below.

2   Mohan Matthen issues, with the aim of contextualizing perception within broader philosophical concerns. It does not attempt to summarize or discuss individual chapters. Without exception, these are intellectually sophisticated introductions to sub-areas and as such they stand alone, requiring no additional exposition here. Accordingly, individual chapters are discussed only when they are directly relevant to the more synoptic topics taken up in the Introduction, though each is at least mentioned to show their place in the whole. Thus, the Introduction does not attempt to touch, even very lightly, on all of the many original and often surprising insights that readers will find in each and every individual chapter. It also suppresses bibliographical references; these are found in the relevant chapters.

I Until very recently, and to an extent even now, the epistemologist’s paradigm of perception remains much unchanged since the seventeenth century. According to this view, what we directly perceive—the message given to us by perception unsupplemented by inference from other sources—is a conscious presentation that closely parallels the excitation of sensory receptors. Call this the Receptoral Image Model (RIM). RIM takes slightly different forms in psychology and philosophy, as I shall now explain. RIM traces back at least as far as Johannes Kepler’s theory of the eye. As David Hilbert (Chapter 14) explains, the great astronomer came to realize that the lens of the eye refractively focuses all the light rays arriving from any given external location onto a single retinal place; thus, it creates an image on the retina. According to Kepler, this image is what we directly see. This discovery of the retinal image was greatly impressive to those who followed, though it took more than 200 years for the realization to dawn that optics is not enough. It was not until the early nineteenth century that the eminent German physiologist, Johannes Müller, came to realize that the optical image is not directly the starting point of vision. (It is, rather, the last item in the external causal chain that links object to perception.) For the optical image that is focused on the retina to affect conscious sensation, it must first be converted into a pattern of nerve energy. The retina is packed with neurons that are activated proportionately to the amount of light that falls on each; the activity of these neurons determines visual consciousness, Müller proposed. This is an important advance on Kepler, though it made little, if any, impact on philosophical theory. Philosophers still show little awareness that the conversion into nerve energy destroys much of the wavelength-specific information that is available in the optical image—information that could be extracted by a spectrometer. (See section VI of this Introduction.) Müller’s realization is the basis for generalizing Kepler’s theory beyond vision. Corresponding patterns of receptor activation could be assumed, and do indeed exist, for the other senses—the senses all have specialized receptors that are differentially activated by environmental energy incident on them. RIM assumes that receptor activations correspond closely to what we perceive in each modality—the activation of auditory neurons corresponds to what we hear; the activation of haptic neurons corresponds to what we feel, and so on. The receptoral images in these other senses do not have exactly the same

Introduction   3 properties as the retinal image—the auditory image in particular is poor in spatial information. Nonetheless, the sensory neurons provide us with what we now call sensory information. RIM identifies this information with what philosophers call the perceptual given, in other words, with what we perceive directly and non-inferentially. Psychology and philosophy worked with somewhat different versions of RIM. • Psychologists assumed that conscious awareness in each sense modality corresponds to the receptoral image, and they tended to assume, though less explicitly, that perception beyond the receptoral image is indirect, or inferred by learned association. For example, since the visual receptoral image is two-dimensional, they assumed that direct visual awareness must also be of a two-dimensional array. Perceptual awareness of depth and of three-dimensional objects is inferred. Nineteenth-century psychology realized that we possess two such retinal images, and they tried to work out how the discrepancies between these images could provide a basis for the inference of depth. • Philosophers made a convergent assumption. In early modern philosophy, and for a long time after, it was commonly assumed that the perceptual given—what we directly see, or hear, etc.—is that which is certain or indubitable given our state of sensory awareness. In the case of vision, two coloured regions might be seen side by side, but it is uncertain, given visual evidence, how far away each is. Consequently, philosophers too assumed that depth was not directly given in vision. Berkeley made this inference explicitly, but it is implicit in Descartes and others. The important difference between philosophers and psychologists is that the former are concerned with rational justification and the latter with physiology and conscious awareness. However, they arrive at comparable conclusions. Both conclude that visual awareness of three-dimensionality is indirect, the result of learned association or (according to some philosophers) rational inference. Psychologists and philosophers use Kepler’s theory differently, because philosophy is supposed to be a priori, which means that it cannot use a scientific theory as a foundation for knowledge. For this reason, philosophers cannot explicitly appeal to Kepler’s optics. Nevertheless, philosophers as diverse as Descartes and Berkeley commonly used the retinal image as a heuristic: it serves for them (though not explicitly) as a model of what we see and as the basis for generalizing to the other senses. RIM encourages many misconceptions regarding perception; collectively these misconceptions constitute a traditional view of perception that is slowly eroding away. RIM implies, for example, that: • Perceptual experience is necessarily unimodal (because the receptors and nerve energies are). (See Tim Bayne and Charles Spence, Chapter 32, for a re-evaluation.) • Objects can be made to look a different colour simply by shining coloured light on them (because this changes the colour of light that reaches the visual receptors). (Jonathan Cohen discusses the limitations of this notion in Chapter 33, as well as distortions of shape and size in the optical image; Kathleen Akins and Martin Hahn discuss the case of colour in Chapter 22.) Similar assumptions can be made in other modalities, though it was unusual for them to be explicitly worked out. For example, it could be assumed that since the activation of auditory receptors is changed by new sounds, direct auditory

4   Mohan Matthen awareness of continuing sounds would be modified by new sounds. Again, it could be assumed (with somewhat greater empirical justification) that gustatory awareness of the taste of honey would be modified by taking lemon into one’s mouth. • Like the retinal array, the visual image is a two-dimensional ‘colour mosaic’—that is, it consists of a two-dimensional matrix of minimally sized coloured dots—that does not contain depth information. (How do ‘coloured dots’ match up with the representation of colour in the system? See Akins and Hahn, Chapter 22, for discussion.) Analogously, audition offers us a confused melange of sound that does not directly inform us of spatially separated sources of sound. (Roberto Casati, Chapter 20, Jérôme Dokic, Chapter 23, and Matthew Nudds, Chapter 15, have relevant discussions.) • Flavour is sensed through the tongue alone, for the only receptors that are specialized for flavour perception reside in the tongue. (Barry Smith, Chapter 17, reconceptualizes flavour as a multisensory quality, and recounts, in particular, how wrong it is to think that it is restricted to taste receptors in the tongue—olfactory receptors are involved too, but in an unusual way.) • Because sensory receptors can in principle be excited without any external stimulus, perception must be subjectively indistinguishable from hallucination. Since hallucination is not about external events, perception cannot be either. Thus, RIM encourages the idea that ordinary perceptual experience is of sensory events internal to consciousness, not of the world beyond. (See Baron Reed, Chapter 3, and Paul Snowdon, Chapter 6, for critical discussion.) Additionally, philosophers have often assumed that since the receptoral image is constantly in flux, perception is momentary; temporally extended experience is a fusion of successive receptoral images, and involves memory, which is epistemically on a different footing. Consequently, they assume that: • Experience of change and movement are due to fused temporal sequences of momentary experiences of positions or qualities. (See Robin Le Poidevin, Chapter 24, about temporally extended perception.) • By the same token, insofar as speech and music are perceived, it is by fusing experience of a stream of momentary tones. (See Casey O’Callaghan, Chapter 25, on speech perception, and Charles Nussbaum, Chapter 26, on music perception.) These assumptions are, for the most part, gross oversimplifications; some (such as those concerning the colour mosaic, flavour, and speech) are completely false.

II What explains the dominance and long persistence of momentary RIM in philosophical theorizing? In large part, the answer is historical. RIM, in particular the claims (a) that certain aspects of perception, such as depth, are fallibly inferred, and (b) that hallucination is subjectively indistinguishable from ordinary perceptual experience (Snowdon, Chapter 6), leads quite naturally to the problem of scepticism (Reed, Chapter 3). Scepticism is one of the

Introduction   5 central problems of epistemology, with proponents vying with opponents who quest for theories of justification and of knowledge that can withstand the sceptical threat. Philosophers concentrated over the years on the ins and outs of the sceptical threat to knowledge, leaving unexamined the route by which they arrived at this dangerous place. Scepticism is a real philosophical problem, but it does not necessarily rest on RIM. In fact, as explained earlier, the explicit basis for this and other epistemological theories is a consideration of the role of inference in situations of uncertainty; preconceptions regarding sensory receptors play a merely heuristic role. Nevertheless, theoretical progress in epistemological conceptions of perception was retarded by RIM, because this framework provided a familiar context for motivating scepticism in relation to perception. The philosophy of perception has long been dominated by the so-called ‘problem of perception,’ the problem of how perception, which is often misleading about the external world, can nonetheless be a foundation for knowledge about the external world. Much less effort has gone into figuring out the nature of perception. To wit: is it really true that direct awareness is as RIM would have it? Traditionally, epistemologists took their main problem with regard to perception to be the uncertainty of beliefs that are based on perception. One might think, however, that epistemologists should be at least as vitally concerned to understand how we arrive at ordinary perceptual beliefs—how we get to a belief is, after all, at least partially independent of why it might be mistaken. Take this very simple question. Do we recognize a musical beat by internally timing successive pulses, or do we feel the beat more holistically? This is a positive question about how we arrive at a belief; it is relevant to whether the perception of musical rhythm depends entirely on a sense of temporal duration, and whether, if it does, this would show that it rests on memory. This question is independent of the sceptical question of whether what we hear is real or merely an illusion, and of the question whether we can ever be absolutely certain that a piece of music has a particular time signature. It is a question about the perceptual basis for the belief that a piece of music is a waltz. Is this belief directly delivered to us by perception, or does it rely on a more complex calculation? Considered in this context, the problem regarding the traditional RIM paradigm of perception is not that it encourages scepticism—there is nothing wrong with this—but rather that it offers an incomplete and often misleading account of quotidian perceptual belief and knowledge. It is true that colour mosaics sometimes simulate ordinary visual perception. This is precisely how object perception and motion perception is simulated on TV. Nevertheless, RIM is uninformative about the normal process of forming rational beliefs about objects and their movements in three-dimensional space. And here it is relevant that the psychological heuristic of momentary receptoral activation is based on false assumptions. For instance, it turns out to be false that the visual experience of movement is created by a post-perceptual summation of momentary experiences of objects in successive positions. The fact is rather that a specialized part of the visual brain detects distal motion (differentiating it from shifts of the retinal displays that are due to the subject’s own motion) without the intervention of the subject’s rational acuity in inference. It is also not true that we perceive objects by summing up retinal colour pixels; the brain has specialized pattern-detecting mechanisms for this (Roberto Casati, Chapter 20). As it turns out, our perception of movement and of objects does not depend on perceiving all of the temporal or spatial parts of these entities.

6   Mohan Matthen The philosophical theory about the uncertainty of inference from perception to belief could have been, should have been, and was maintained even after the psychological theory of sensory receptoral images had been long abandoned. This divergence, however, makes it all the more urgent to give a theory of the formation of ordinary perceptual beliefs. The best psychological theories of sensory awareness urge that consciousness presents us with something more substantial than receptoral arrays. At the same time, it is acknowledged that this sensory given is uncertain. (In fact, one important way of probing the perceptual given is to study the illusions that occur in normal perceptual situations.) This gives epistemologists strong motivation to offer better theories of how we ordinarily justify perceptual beliefs. (Susanna Siegel and Nicholas Silins, Chapter 41, discuss reason giving for fallible perceptual belief; Michael Rescorla, Chapter 37, E. Samuel Winer and Michael Snodgrass, Chapter 38, and John Kulvicki, Chapter 39, explain frameworks relevant for posing the problem of the perceptual given.) In a similar vein, the model of speech perception implicit in the music-analogy mentioned earlier gives us a false idea of how we come to know what people around us are saying (O’Callaghan, Chapter 25). Phoneme perception is not a summation of temporally punctate auditory experience; phonemes are temporally extended (though very brief) sound patterns—phonemes are minimal meaning-bearing units of spoken language; no part of a phoneme is heard as a speech sound, yet they are so heard as a whole. By itself, this is proof that perceptual experience is not merely a summation of temporally punctate sensations. As far as music is concerned, rhythm and phrasing is temporally extended (and of relatively long duration, compared to phonemes) but these too are perceived as wholes, not as units that the perceiver has to put together by her own agency (Nussbaum, Chapter 26). As well, vision is involved in speech perception; perception of speakers’ facial gestures modifies what we seem to hear. Speech and musical perception thus contradict the notion that intermodal and cross-temporal integration are always extra-perceptual mental operations. Auditory perception, in general, is experience of temporally extended objects—sounds—that often appear to change (Matthew Nudds, Chapter 15), or at least to have a non-uniform temporal profile. For instance, we might experience a single voiced melody as ululating; or a siren as rising in pitch. (These questions about speech and music perception are illuminated by the considerations about time discussed in Le Poidevin, Chapter 24.) Something like this is true of flavour too: when we savour food or wine, there is an early attack followed by extended finish. Think of spearmint gum: it might start sweet, then become cool, and finish with a slightly bitter ‘aftertaste’. This has to do in part with the dissipation of sugar, and it could be argued that spearmint offers us a progression of flavour experiences, rather than perception of one temporally extended flavour. However that might be, such progressions are predictable; they are, moreover, repeatable, and therefore important in the identification and evaluation of food and wine. In short, they are ecologically salient. (Barry Smith, Chapter 17, discusses.) Expert tasters become able to use such temporal profiles to refine expert discriminations by perceptual learning. (Robert Goldstone and Lisa Byrge, Chapter 42, provide a general introduction to perceptual learning.) The scientific study of perception was, until early in the twentieth century, rooted in many of the same philosophical assumptions that led to the wrong assumptions recounted in section I (and many others). For example, visual perception was thought, in the nineteenth century, to consist first in the transference of retinal images to the primary

Introduction   7 visual cortex, and then the extraction of information from this so-called ‘cortical retina’ by exploiting associations between retinal stimulations and distal features. This is a model that applies to the brain in a not very much modified version of Berkeley’s theory of vision. This model has two fatal weaknesses. First, it misconstrues the nature of the retinal image: the relevant entity is not the image thrown by the lens, but the firing of sensory neurons caused by this pattern of light. Secondly, the model does not transfer smoothly to other modalities. For example, the auditory cortex is a frequency-intensity mapping, not a spatial image. That is, the activation-level of different regions of this cortex corresponds to energy levels associated with a particular frequency, not with energy levels coming from a particular spatial direction.

III The laboratory study of perception began in the middle of the nineteenth century. (See Gary Hatfield, Chapter 5.) At first, it was dominated by the RIM paradigm: Müller held that the retinal image was transmitted to consciousness by the physical action of nerves. One of the great controversies of the late nineteenth century was the battle between Ewald Hering and Hermann von Helmholtz about the extraction of depth information from the disparity of the two retinal images. Helmholtz thought that the perception of three-dimensional space was learned by the association of ideas; Hering was more of a nativist. Science does not stand still. Gradually new discoveries began to widen the research focus. • In the Russo-Japanese War and the First World War, surgery had progressed to the point where soldiers who had been shot in the head could survive; often they survived with severe but localized brain lesions along the path of the bullet that penetrated their skulls. Starting in the early part of the twentieth century, many specialized perceptual deficits were discovered in patients with such lesions, some produced by injury, others by other adverse events, such as strokes or prolonged hypoxia. It was found, for example, that some people with normal visual acuity with respect to colours and lines were nonetheless unable to recognize forms that are composed of the lines they could see normally: familiar objects, such as shapes, faces, places, motion, speech, and alphanumeric symbols. These deficits are specialized—for example, the inability to recognize faces does not predict the inability to recognize shapes, and vice versa. • The perceptual deficits just mentioned are known as ‘agnosias’—they are recognitional failures with regard to a restricted domain of ‘higher-level’ (or composed) objects and features sitting on top of normal acuity with regard to their ‘lower-level’ components. The agnosias directly contradict the Receptoral Image Theory. They indicate perceptual deficits in the absence of receptoral defects. The existence of agnosia shows that awareness of objects is not entirely dependent on awareness of the parts out of which these objects are constructed. Agnosia cannot be a failure of inference as such, at least not if inference is construed as a general purpose capacity, since each agnosia is domain specific—a patient who fails to recognize faces may have no difficulty discerning motion and vice versa. Each agnosia is associated with a brain lesion

8   Mohan Matthen











in a specific area; each highlights a part of the brain that is specialized to extract from receptoral images content about a particular kind of higher-level perceptual object. • The agnosias indicate what is known as modular function in perception—separated processes for the extraction of distinct perceptual features (Ophelia Deroy, Chapter  40). As time passed, it became clear that there are separate processes for the extraction of even the lowest-level features, for example colour and form in vision. • Psychological evidence for awareness of higher-level perceptual objects was accumulated by the Gestalt psychologists, who articulated principles of object awareness (Hatfield, Chapter 5 and Casati, Chapter 20). They demonstrated that certain types of displays result in seeming awareness of objects, while others, though very similar, either do not or result in awareness of very differently shaped or configured objects. It can be inferred that object awareness is not learned by relations of association among receptoral arrays. • Along similar lines, Albert Michotte demonstrated that certain very simple types of temporal sequence have the look of causal connectedness—for example, one in which a simple shape (such as a square) approaches another simple shape, stops when the two touch, at which point the second shape starts moving. Perceivers typically find it hard to resist the appearance of causation (and even of animacy) in such displays. This shows that Hume was wrong to say we have no impression of ‘power’, and that the appearance of causality is nothing but a projection onto displays of the mind’s propensity to infer one event from another. There is a primitive impression of causal connection. • Single neuronal-cell electrical recordings and fMRI imaging demonstrated that certain brain areas are active when certain types of higher-level object are presented to perceivers: for example, colour (the fusiform gyrus), motion (visual area 5), faces (the fusiform face area), places (the hippocampal formation), speech (Wernicke’s area). This bolsters the conclusions drawn from agnosias above, inasmuch as it shows that specialized neuroanatomical structures are dedicated to extracting content about higher-level objects of specific kinds. (Deroy, Chapter 40 discusses the significance of the anatomical localization of this kind of function; see also Hilbert, Chapter 14). • Starting in the last two decades of the twentieth century, greater attention has been paid to multisensory integration (Bayne and Spence, Chapter 32; Smith, Chapter 17, O’Callaghan, Chapter 25). For example, when two simple shapes move along straight intersecting lines, they are seen as passing each other, describing an X. However, when a sound such as a pop or beep is played at the moment of intersection, the two shapes are seen as bouncing off one another (rather than as continuing along their own prior trajectories undisturbed). (This is as a multisensory version of Michotte’s experiments on the perception of causation.) Again, when a subject’s hand is hidden from view and stroked with a brush, while a clearly visible rubber toy hand is synchronously stroked with a brush, the feeling of stroking is spatially shifted to the visible rubber hand. This is, again, an integration of sensory stimulations in two modalities resulting in a single unified percept. Additionally, subjects report feeling that the rubber hand is their own, so there is also an involvement of interoception. (See Frédérique de Vignemont and Olivier Massin, Chapter 16; Alisa Mandrigin and Evan Thompson, Chapter 27; and Brendan Ritchie and Peter Carruthers, Chapter 18 for discussion of the rubber hand illusion.)

Introduction   9 These findings indicate that perception is not simply a matter of receiving external influences, but is rather a process which filters and analyses incoming data in a search for indications of significant external occurrences and states of affairs. Moreover, they indicate that consciousness can be perceptual; it is awareness of external objects and processes, not merely a reproduction of the state of our receptors within our bodies. Thus, RIM is increasingly giving way to the idea that perception presents us with a rich scene: objects of many sorts with properties that do not directly affect the sensory receptors. We literally and directly see objects, faces, places, and shapes; we hear melodies, voices, and phonemes, once again directly and not by painstakingly piecing them together by the use of learned experiential associations. We sense the location of events by both touch and sight working together; the modalities are not correlated by learned associations. Neuroscience and psychology are not the only sources of models for perception. Functional models treat perceptual systems as performing a ‘task’ without being specific about the physical means by which the task is performed. One particularly fruitful idea in this arena has been to treat of sensory receptor response as data, and the task as data processing with the aim of providing the organism with the means to respond productively to a changing environment. Note that data here are abstractly conceived. New analytic tools have also come to the forefront in the last few years to substantiate this conception. The models that are employed to understand perception have a significant constraint. Sensory data processing is ‘analogue’, in the sense that the system (a) has states that causally respond to sensory inputs, and (b) vary in a roughly continuous way with firing rates of neurons etc. Consequently, digital models that are widely used to model thought processes are of limited utility in the perceptual domain. The following have become entrenched in philosophical thinking: • Bayesian reasoning from probabilities, which neural sensory processing is found widely to implement. (Michael Rescorla writes about this in Chapter 37.) • Signal detection theory, which provides a framework for understanding how conscious perception is influenced by the context of inquiry, and perhaps by voluntary attention (E. Samuel Winer and Michael Snodgrass, Chapter 38). • Information theory, which gives an account of what kind of use can be made of environmental signals to adjust to the events from which they emanate (John Kulvicki, Chapter 39). Other analytic tools either are emerging or have receded from prominence—predictive coding is a candidate for future prominence, while connectionism seems to have faded; the current status of dynamic systems theory is clouded. The philosophy of perception tends to be conservatively selective in its attention to such structures, being more in tune with cognitive neuroscience, broadly speaking, than with the abstract mathematics of data processing. No doubt, this has a lot to do with the history of the subject; there are established problem areas in the field that arise from thinking about psychological function and neurological implementation. Abstract modelling has been slower to yield fruitful approaches to established philosophical problem areas—this could be the fault of philosophers, of course—the above-mentioned tools being exceptions. This could well change in the next few decades.

10   Mohan Matthen

IV In Europe, the very idea of perception emerged relatively late. Victor Caston contends in his chapter on Ancient Theories (Chapter 1) that the early Greek thinkers did not clearly distinguish perception from cognition, possessing only the verb ‘perceive’ to carve out the category—in Greek as in English, this verb does not necessarily connote sense perception. It was perhaps Plato who first attempted an analysis, saying that perception is passive, related specifically to certain bodily organs, and outside of rationality and thus shared with animals. This initiates a very long tradition of distinguishing between sense perception and discursive rationality: even today, it is not entirely clear what part of our awareness should be attributed to the senses, and what to learning and rational inference. It is this distinction, precisely, that is at issue when we speak of literally and directly seeing faces, hearing melodies, and (following Michotte) of apprehending causality perceptually. The developments related above tend to shift such entities from the realm of rational inference or learned association to that of perception. How does perception inform us of our surroundings? Two issues dominated the ancient debate and continue to have considerable importance today.2 The first concerns the causal influence of the object. For though it has always been agreed that objects make themselves known by causally influencing the sense organs and (further downstream) the mind, the exact nature of this influence has been hotly debated. A bright light makes me blink; a sudden loud sound startles me. This is perception. On the other hand, the Sun makes my skin get darker; it also makes heliotropic flowers change their orientation. It seems that these organic responses are not perception. Why not? Like the bright light, the Sun causally evokes an organic response. Aristotle charged his predecessors with being insufficiently mindful of the distinction between perception and changes of the latter sort. Aristotle himself considered perception to be the transference of form to the sense organ without matter: for example, when we see a coloured object, the colour is transferred to the sense organ, but without the physical substrate in which it resides. The sense organ has, according to this view, a neutral state that is disturbed by the reception of sensory form; when the organ is no longer in contact with the object, it returns to its neutral state. It ceases to reflect the form of the object with which it is causally connected, and returns to a state of receptivity to a fresh object of perception. Perception is a state of a specific kind, which must be maintained by the ongoing causal influence of the object. Aristotle’s form-without-matter theory appears to be modelled on the so-called telic senses, vision and audition, which record the presence and characteristics of distant objects and events. When we see or hear them, Aristotle wants to say something within us takes on the colour and sound of distant objects. It is unclear how this is meant to apply to the contact senses of touch, smell, and taste. Aristotle posits a medium for these, and presumably he thought that form is transmitted through the medium; in effect, this seems

2 

Brad Inwood contributed a great deal, including stretches of text, to my discussion of ancient theories. His help was indispensable because there is no separate chapter on the Epicureans and Stoics in this Handbook, and the Introduction serves partially to fill the gap.

Introduction   11 to subsume the contact senses to a telic model—just as colour is transmitted through ‘the transparent’ to the eye, so the fuzziness of a peach is transferred to the tactile sense organ by flesh. What could it mean to say that we receive the form of fuzziness when we feel a peach? Is there supposed to be something in us that becomes fuzzy, though without taking in the matter of the peach? This is implausible—even more so than to hold that the eye takes on the colour of the peach without its matter. (As noted earlier, the doctrine about colour is mistaken because, while it is true that the peach throws an orange image on the retina, the only thing relevant to perception is the neural activation caused by the image of the peach. Neural activation has no colour.) However this may be, why is the Sun’s influence on my skin not perception in Aristotle’s system? Perhaps, because my skin does not take on the Sun’s form in this causal transaction, but takes on a different form, a dark colour. This in turn means that the skin, by contrast with the sense organs, lacks the right sort of responsiveness. Perhaps more importantly, skin does not have a neutral state or ‘perceptual mean’, to which it immediately returns when it is not under the influence of the Sun. We perceive colour when the visual organ is pulled off its naturally colourless state by receiving an object’s form of colour. When the coloured object goes out of sight, its effect disappears; the visual organ immediately returns to the mean, and is then ready to take on the colour of whatever else comes into view. This is not true of my skin’s darkness when the Sun sets. Plausibly, though, my skin might perceive the Sun’s warmth. When it is warmed by the Sun, it does take on the Sun’s form, warmth; at night when the Sun has gone down, my skin ceases to register its warmth. Epicureans and Stoics, causal theorists along broadly similar lines, also built their theories of perception on the basis of their distinctive physical theories—atomic films shed by objects entering the sense organs for the Epicureans, qualitative changes in the organs of perception transmitted through a continuous physical medium for the Stoics. These philosophers do not use the distinctively Aristotelian apparatus of form and matter, but they echo Aristotle, nevertheless, by adopting a causal model based on the telic senses. A second issue in the ancient debate concerns the mental significance of perception from the point of view of the perceiver. According to Plato, perception competes with reason; according to Aristotle, it complements it. The senses ‘tell’ us that certain states of affairs obtain; reason and memory can equally well ‘tell’ us that perception is mistaken, or that it is correct. In short, perception delivers a message that can be evaluated as true or false. Both Plato and Aristotle are thereby committed to the view that just as reason delivers propositions for our consideration, so also does perception—what they disagree about is the reliability and coherence of the propositions delivered. Supposing that perception carries propositional content, what is the nature of the content? Both Plato and Aristotle restricted the features of which we are perceptually aware to those that the senses are especially attuned to. Colour, shape, and pitch are properties to which the senses are peculiarly attuned; being and unity are not. Colour, then, is a (or rather the) ‘proper sensible’ for vision. Both Plato and Aristotle insist, however, that the sensible qualities must come together in a central cognitive faculty. I apprehend the pale woman singing, her hand on my shoulder: this is a synthesis of what vision, audition, and touch tell me, a synthesis that cannot be performed by the individual senses since, for instance, colour is not special to touch and pressure not special to vision. It must,

12   Mohan Matthen therefore, be performed by some facility for sensory confluence. I recognize, moreover, that the woman’s pallor is different from the pitch of the notes she emits. This is an act of rational differentiation. Epicurus disagreed with Plato and Aristotle about the first point; he did not think that perception could be false. Provided that we properly focus our thoughts, he held, perception always leads us to the truth; ‘falsehood and error are always located in the opinion we add.’ It is likely that he blamed the faculty of judgement for false propositional content: if we form judgements in an appropriately receptive way, we will not err, but if we add something to ‘impression', then we risk error. (It would have been odd for Epicurus to think that falsity, but not truth, was ‘located in the opinion we add’. Most likely, this is what he thought—error is due to wrongful insistence. It was also open to him to take the position that perception is neither true nor false. Propositional content comes with judgement, or opinion.) Like Plato and Aristotle, the Stoics held that, in humans, perceptual states (which they called phantasiai, or ‘impressions’) convey articulable propositional content that leads to perceptual belief when accepted or rejected. They held, in addition, that these impressions ‘reveal themselves and their cause’, but here their position is nuanced. They hold that perception is generally reliable, but acknowledge that some impressions bring misinformation about the world. Sceptics conclude that this impugns the entire class of impressions. The Stoics disagree. For them, an important subset of perceptual impressions is marked by a self-validating clarity and reliability. When I look at something attentively in good light, for example, I can be certain that I see it as it actually is. Such ‘apprehensive impressions’ (katalēptikai phantasiai) are the foundation and touchstone by reference to which they believe that we can achieve certainty about the physical world. On the other hand, a square tower viewed from the distance looks round—this impression carries on its face its failure to be apprehensive, because the tower looks far away and shimmery. By taking this position, it should be noted, the Stoics align themselves with the idea that rational belief requires certainty. They both underestimate how illusions can take place even in the best conditions, and are also myopic with regard to the efficacy of probable reasoning, which was recognized by the Academic sceptic, Arcesilaus. On the other hand, they were prescient, as it turns out, in suggesting that the perceptual given includes self-regarding reliability estimates—vision doesn’t just tell you that a shape is concave; in many instances, it also includes an estimate of the reliability of this attribution, depending on the goodness of the illumination, the sharpness of resolution, and so on. RIM might suggest that estimates of reliability are necessarily post-perceptual, but there is reason to doubt this. (Rescorla, Chapter 37, discusses Bayesian models of perceptual processing, in which such estimates play a role.) The thinkers we have considered so far evaluate perception as true or as false. (I suggested that Epicurus had a bit of wiggle room here.) One could deny this; one could hold that perception, or rather sensation, is simply an effect created in consciousness by the outside world. We draw inferences about the external world from these effects, it could be held, but the accuracy or error of these inferences is not to be attributed to perception itself. This attitude becomes more prominent in the early modern period, when (as Alison Simmons notes in Chapter 4) Descartes distinguished between physical motions in the sense organs, the sensations occasioned by these, and judgements that we make on the back of these sensations. Only the last of these has propositional content. Descartes’s

Introduction   13 position on the issue of propositional content is thus importantly different from both Epicureanism and Stoicism, though his focus on clarity and distinctness is something he shares with both Hellenistic schools. These questions continue into medieval philosophy as Dominik Perler recounts in Chapter 2. He focuses on three key problems, each of which continues and draws on ancient philosophical discussion: What is the object of perception? What is the nature of the cognitive faculties that we need in order to apprehend these objects? And how trustworthy are the perceptual faculties with respect to what they reveal?

V Aristotle and Epicurus are in the same camp about the reliability of the senses. As we noted in section IV, Epicurus is the more optimistic—he does not think that there is false perception—but Aristotle too thinks that perception leads to rational knowledge (see especially Posterior Analytics II, 19). The Stoics too held that perception could be the foundation for knowledge, since for them, the ‘apprehensive impression’ is, once successfully identified, a reliable foundation for all further cognition. Plato was much more pessimistic on this matter. According to him, the senses deliver constantly shifting and contradictory information. They deliver a vague and confused message, which cannot be a foundation for knowledge even where they serve as a rough and ready guide to ordinary talk and action. The ancient sceptics (Reed, Chapter 3) took Plato’s pessimism to an extreme. Reacting against Stoics, Epicureans, and Aristotle alike, they sought to show that the senses are not to be trusted. Up until now, they say—that is, up until the moment of speaking—they have never been convinced of anything, neither by the senses nor by rational argument. Perhaps some day they will encounter an apprehensive impression that is so clear and distinct that it is self-validating, as the Stoics claim—but so far they haven’t experienced anything like this. Perhaps they will, sometime in the future, encounter a convincing argument that knowledge can be founded on the senses, but so far all that they have experienced is doubt. The sceptical ideology prevents them from making positive pronouncements about the reliability of the senses—they know nothing, not even that knowledge is impossible. They parade a comprehensive armoury of arguments against all possible claims to knowledge, but they do not affirm the completeness of their armoury against all possible challenge. There is, of course, a certain irony in this show of modesty. Though scepticism never completely slipped out of view as a philosophical tradition, it grew less important in the medieval period, with its emphasis on religious faith. Even then, the influential Persian Muslim philosopher, al Ghazali, had considerable affinity with scepticism, and sceptical aperçus are occasionally to be found in such Christian thinkers as Augustine. But outright scepticism was not an option for these thinkers in their culture. The ancient sceptics held that it was fine to play along with religious practice as a matter of ‘custom’, but not as a matter of belief. ‘Bend your head to worship: to do otherwise, would be to defy society, thereby showing epistemic arrogance. But do not believe in God.’ Such prevarication would have gone down as smoothly in medieval centres of Islam and Christianity as it does in contemporary South Carolina or Qom. Much later, scepticism re-emerged forcefully, first in fifteenth-century Italy, then when Henri Estienne translated

14   Mohan Matthen Sextus into Latin in 1562, and shortly thereafter in France with the essays of Montaigne. Descartes’s methodological scepticism, which owes much to Sextus, had a profoundly disruptive and revolutionary influence in the early modern period. Alison Simmons says in Chapter 4 that it led to a re-examination of ‘almost all aspects of perception’. Simmons points out, interestingly enough, that received views of perception came under attack, at this juncture, from science. The Scientific Revolution of the sixteenth and seventeenth centuries posited a world that was in its essence very different from the world that perception reveals. Atoms do not, for example, have colour or smell, not even unperceived colour and smell. If, as was increasingly popular to suppose, atoms are the ultimate stuff of physical reality, how can the larger objects composed of atoms have colour or smell? And what does this say about the veracity of our senses? This disjuncture between science and perception was addressed by a distinction between primary and secondary qualities discussed by Simmons and by Peter Ross (in Chapter 21). The idea traces back to Democritus: ‘by convention sweet and by convention bitter, by convention hot, by convention cold, by convention color; but in reality atoms and void’. The term ‘by convention’ is meant here to contrast with ‘by nature’. The idea is that things are not sweet, hot, or coloured in themselves, or by nature. Rather, they are so relative to the perceptual response of the observer. As Ross shows, Democritus’ idea and its successor, Locke’s distinction between primary and secondary qualities, has many different incarnations. Ontologically, some such distinction is needed to bridge the divide between the physical world as posited by science, and the ‘manifest image’ by which we initially know it. Perhaps the most enduring legacy of scepticism is the notion of a ‘veil of ideas’. Why should we refrain from accepting the evidence of our senses? One major reason is that this evidence is held to be equivocal, much more so than it appears to the naive observer. Objects of different sizes or colours can create the same sense perception if they are at different distances or in different conditions of illumination; the same thing can sound loud close up and faint further away; touch is modulated by pressure exerted; and so on (but see Cohen, Chapter 33, on Perceptual Constancy). According to the sceptic, this implies that any given perceptual state betokens many different real world situations, and thus fails to validate inference to any one of these. What is the similarity that defines sameness of perceptual state? The standard view in both medieval and early modern philosophy was that sameness was determined by the ideas entertained by the perceiver. These ideas constitute an intermediary or ‘veil’ through which we observe the world; perception offers us certainty with regard to the ideas, but not with regard to the world that lies beyond. The power of this doctrine is attested to by discussions in Perler (Chapter 2), Simmons (Chapter 4), Paul Snowdon (Chapter 6), Bence Nanay (Chapter 8), and Heather Logue (Chapter 11). Susanna Siegel and Nicholas Silins (Chapter 41) discuss how perception is reason-giving with respect to belief, and the implications for scepticism.

VI It has now become almost commonplace to note that the philosophy of perception suffered, until recently, from an excessive concentration on vision, which was taken as the proxy for all of the other senses. The result is, as David Hilbert puts it, that ‘vision itself,

Introduction   15 with its own peculiarities and distinctive features has a tendency to fade from view and what we are left with is a generic sense’ (Chapter 14). As Aristotle urged, it is necessary to bring the senses under a unified rubric. Otherwise, we will be unable to differentiate them from other information-gathering facilities such as the immune system. (Matthen, Chapter 30, and Ritchie and Carruthers, Chapter 18, take different approaches to this problem.) However, this unified rubric is insufficiently informative about the ‘peculiarities and distinctive features’ of individual sense modalities. Aristotle was aware of this. He gave a comprehensive characterization of sense perception (form without matter, neutral state of sense organs), but he also discusses the medium, special objects, and limitations of the individual senses separately. In actual fact, the senses are very diverse in character.3 It helps to remember that they are biological systems that evolved to give organisms an advantage by providing them with the means by which to respond effectively to the challenges of living and reproducing in surroundings that are constantly in flux. Thought of in this way, the senses are not simply information sinks—organs that happen to receive ambient information at their sensory receptors, leaving their possessors to determine how to use this information. Nor are they engineered to seek information optimal for the organism’s pre-existing needs. Rather, they are evolved systems, with all of the random fit to the environment that such systems display. The evolution of sensory systems usually begins modestly with an ecologically sensitive receptor that allows an organism to modify its behaviour to suit circumstances. The evolution of vision, for example, begins with molecules known as opsins, possibly derived from molecules involved in photosynthesis. These molecules afforded primitive organisms access to information carried by light; at first, the information available from this source is minimal—perhaps just enough to regulate circadian rhythms. In the case of audition, evolution starts from a fluid filled chamber that picks up vibrations from bones and other rigid structures; again, an organism would benefit from this, miniscule though the quantity of available information would have been. In both cases, evolution needs to add nerves that can communicate the state of these receptors to behaviour controllers. In time, it adds facilities to refine information collection. Given that each such step is a small advantage that a particular population of organisms manages to gain in its local environment, it is path dependent and unpredictable in advance how these systems evolve. The complex utilization of information carried by light and sound that we find in more recently evolved animals, such as mammals, is the outcome of a historically contingent evolutionary pathway from a starting point that could have presaged different outcomes if chance had played differently. Vision and audition illustrate the path dependency and contingency of these developments. Vision receives information from light, audition from sound. The wavelength of light closely matches the size of the molecules that make up the everyday objects light bounces off. Consequently, light interacts with the molecules of things that it encounters and is modified when it is reflected; reflected light is informative about enduring characteristics of the objects. Moreover, because of its short wavelength, light can be, and is,

3  I am grateful to Jonathan Cohen, Yasmina Jraissati, and Diana Raffman for critical discussion of this section.

16   Mohan Matthen focused by a lens; the resulting image recapitulates the spatial distribution of environmental sources of light (including self-luminous objects and reflecting objects). Combining these, vision extracts from light a ‘map’ or image of luminous and reflecting objects together with informational content concerning certain characteristics of these objects. Sound is very different. First, auditory information does not depend on ambient energy. Visual images come (in the main) from light reflected off objects. The source of this light, for example daylight, is constant and enveloping; it is a background condition of the information that arrives at the eye. Sound, on the other hand, has too long a wavelength to be focussed by a lens. Thus, the energy that carries auditory information comes from myriad local events, and is highly variable. (Right now, I am looking at a calm blue sky that illuminates everything outside my window and, more diffusely, everything inside. Sonically, however, I  hear only the banging of a garbage truck, which will shortly be replaced by silence.) Secondly, reflection scarcely (if at all) modifies sound, so reflected sound carries information about its ultimate source, not about the objects off which it is reflected. Finally, biology (and also, for that matter, human engineering) has not succeeded in constructing a lens that would produce a spatial image of the objects from which sound is reflected. (Ultrasound machines produce such images, but they have to produce sound, or ultrasound, and capture the returning echo. Ambient sound, which is unpredictable in direction and amplitude, will not do. Bats, of course, do the same, as do some blind people who produce a stream of sonic clicks, which affords them very rough echo-location.) Put all of this together, and the result is that sound carries information about events that produce it, but hardly any about objects off which it is reflected. Turning now to the receptors, the frequency composition of light cannot be exactly analysed by any biological system, while frequency composition of sound is much more easily analysed, using mechanical resonators. The basilar membrane in the inner ear consists of fibres, each of which has a different resonance frequency. To exaggerate just a little, it incorporates a dedicated receptor for each and every acoustic frequency. The analogue for light—specialized receptors for every frequency in the visible spectrum—is not biologically feasible. The visual system computes colour from the responses of just a few types of cells (three in the case of most humans), each of which responds to a broad visiblefrequency range. Colour vision does little more than register total energy in these broad ranges. (More precisely, it registers normalized differences between energy levels in these ranges. For more about colour vision, particularly about the extraction of information from cone cells, and the derivative character of colour perception in many situations, see Kathleen Akins and Martin Hahn, Chapter 22.) Audition extracts information about material objects by analysing the frequency composition of the sounds they produce. It identifies voices by timbre in speech; it identifies musical instruments and materials by the sounds they emit when struck, plucked, or bowed. This is event based: we recognize humans by the timbre of their voices, but only when they speak. The quality of this act of speech—somebody reciting ‘Mary had a little lamb’—reveals something about the voice—that it is a rich baritone from Lancashire—and enables us to recognize the voice when it sings something entirely different. Vision is different; the shape and colour of the face is directly revealed when light falls on it—these characteristics are not computed via the character of events. Light yields up only crude information about frequency composition—colour is subjectively one of the more salient characteristics of vision, but frequency (colour) information

Introduction   17 is actually surprisingly sparse by comparison to sound, and surprisingly little used by the visual system by comparison with brightness contrasts. Primates and birds—animals that have ‘good’ colour vision—share many perceptual discriminatory capacities with animals that do not possess equally good colour vision, and typically do not use colour information in exercising these capacities. In particular, visual perception of depth, three-dimensional shape, movement, and spatial layout are all available in black and white, as one can tell by looking at a black-and-white movie. To summarize: audition is concerned primarily with sound-producing events (Nudds, Chapter 15), the temporal order of these events, and properties of material objects that can be computed from the acoustic frequency composition of the sounds they produce. Light, on the other hand, yields to the visual system information primarily about the spatial configuration and distribution of objects, and their brightness relative to other things seen at the same time. (See Jérôme Dokic, Chapter 23, for a rich account of the structure of the visual representation of space.) Because vision is dependent on constant ambient illumination, and not so much on events involving objects, it engages more directly with the place and character of objects. The spatial character of both vision and touch give these senses dominant roles in our identification of particular objects. They are associated with demonstratives and pointing—‘that object,’ we say, pointing with our fingers or our eyes, and this attracts the gaze of our auditors. (How do vision and touch enable us to think about individual objects? Imogen Dickie poses the question in Chapter 43.) Vision and audition go in different directions in their engagement with the environment because they have different information-gathering resources available to them. Within their respective parameters, they target specialized objects, depending on the interests of species. Both birds and humans are specialized for identifying others of their species both visually and auditorily—humans are adept at recognizing human faces and voices, and at understanding human speech; birds tend to be specialized for producing and recognizing identifying song. These are ‘higher-level’ capacities. Let’s suppose, for the sake of argument, that humans and birds are equally good at discriminating lines and colours. It would not follow that they are equally good at recognizing birds and humans. Birds recognize bird song, not human speech; humans have it the other way around. This kind of specialization holds true for other sense modalities. Consider de Vignemont and Massin, Chapter 16, who argue that the proper object of touch is pressure. (They exclude temperature perception as a separate modality, as do Ritchie and Carruthers, Chapter 18.) Superficially, this conforms to the Aristotelian framework of proper sensibles. Their point is that the tactile perceptions of ‘texture, vibration, weight, contact, hardness, solidity’, etc. ‘depend on the perception of pressure and tension.’ ‘There is no sui generis sense of texture distinct from the sense of pressure,’ they write, ‘for we feel the texture of a surface by feeling a spatio-temporal pattern of pressure when stroking it.’ However, de Vignemont and Massin do not think that we go from experience of pressure to awareness of texture by inference. They are fully aware that the sense of touch delivers such haptic properties as texture and weight to sensory experience. This marks a departure from the traditional articulation of the special object view as proposed by philosophers as diverse as Aristotle and Hume—these traditional philosophers imply that we have sensory consciousness only of the proper sensibles such as pressure, and inferred knowledge of other properties such as texture. What scientists began to understand with the discovery of the agnosias (section II above) was that the ability to

18   Mohan Matthen recognize certain complex objects of the senses and awareness of these objects is simply provided to us by the perceptual brain. It does not require learning or inference in the classic sense of those terms. The specialized objects of perception are explored in Part IV of this Handbook. Flavour perception is quite a different entity from these senses. (See Smith, Chapter  17, for full discussion.) Whereas vision and audition are shaped by receptors and the kind of energy they receive, flavour is unified by its object: food and drink. The tongue has receptors for basic tastes—sweet, sour, bitter, salty, etc. These basic tastes differentiate many foods; the bitter warns of unhealthy toxins, and sweetness attracts us toward energy-rich foods. Flavour comprises a great deal more than these basic tastes. Take two sherbets, one of cherry, one of raspberry. The basic tastes differentiate them, but not by much. And why should they? Both are healthy, at least as far as our savannah-dwelling ancestors go. Both are sweet, and that is pretty much all the information the tongue gives us. As far as flavour goes, however, the two confections are very different. They are very easy to tell apart. The puzzle is this: if the tongue and its basic tastes do not differentiate, what does? The additional qualities come mainly from olfactory receptors in the nasal cavity. These receptors provide smell when we breathe and sniff. When we take substances into the mouth, their vapours rise to the nasal cavity and flow over the olfactory receptors. The flow is in the direction opposite to that when we breathe through the nose. Olfaction from the mouth is known as ‘retronasal’ olfaction. ‘Smells’ detected retronasally are experienced not as smells; rather, they contribute to the flavour properties of food. They are responsible for the experience of food over and above the basic tastes—but note that they meld in with taste and provide a unitary experience. The ‘taste’ of a cherry . . . no, that is just the sweetness and bit of sourness. The flavour of cherry, that’s the whole thing. Though olfaction contributes to the flavour of cherry, we cannot phenomenologically separate out the contribution of the olfactory receptors. We don’t experience a smell plus a taste; we experience a unified percept. Flavour perception is different from vision and audition in that it does not have information-humble beginnings. It has recruited an evolutionarily well-developed nasal system very late in evolutionary history—humans are almost unique in their use of retronasal olfaction—to work conjointly with another well-developed glottal system. Olfaction is well developed in animals; so is the taste system. Flavour perception in humans is the result of a coalition between the two. Flavour recruits olfactory receptors to enhance a preexisting system that already has a sophisticated nutrition-regulation function. The result is a system that discriminates food and drink far more finely than we need to discern what is healthy to eat and what is not. It seems as if all possible senses get involved when we put food in our mouths—taste, olfaction, touch (which plays a role in binding taste and olfactory qualities), the trigeminal pain system (Smith, Chapter 17)—and arrive together at a much finer discriminatory ability than even the higher primates can deploy. What is the biological point? Why do we not just make do with the system that the higher primates use? It may well be that, like language, flavour plays a role in the highly creative sociocultural system in which only humans participate. Perhaps, humans are more adaptable because of their flavour sense, and this could have contributed to their spread across the planet. Or perhaps flavour does not simply have utility for nutrition, but has as well a social and communicative role.

Introduction   19 The bodily senses (Ritchie and Carruthers, Chapter 18) provide another example of heterogeneous sensory processes that are nonetheless highly integrated. Models based on the external senses do not work smoothly with these. For, first, some internal sense events seem to lack a characteristic phenomenology. Consider the vestibular sense, which determines how we are oriented relative to gravitational forces. Ritchie and Carruthers (Chapter 18) suggest (though they do not come to a firm conclusion about this) that this sense may not present us with a direct message (e.g. that we are upright). It may, instead, feed this information to vision, in the form of visual field orientation, and proprioception, in the form of information about the positioning of our bodily parts. In other words, we may sense gravitational orientation only in terms of the orientation of objects, external and bodily. This puts the status of the vestibular sense into question. Is it a sense modality if it does not have a separate phenomenology? A second source of conceptual cloudiness is that it is difficult to decide whether the internal senses represent objective states of affairs, that is, events or states of the body, as opposed to presenting us merely with sensations. (Bence Nanay, Chapter 8, offers an overview of questions about perceptual representation.) Is an itch, for example, a perception (or misperception) of a certain type of objective event in the body, or is it merely a feeling? Recall the view (Simmons, Chapter 4, and section IV above) that perception is neither true nor false, but only a conscious event, or sensation, that we use to make judgements about the world outside us. This view, applied to bodily feelings, is thought by many to be unavoidable given the facts about bodily awareness. The question is closely related to another: if pains and other bodily feelings are perceptions of occurrences in the body, then it should be possible for this kind of occurrence to go unperceived, just as it is possible for a sound to occur unheard. Is it possible for me to have a pain in my finger of which I am completely unconscious? Many think that this makes no sense—pain is essentially conscious. What does this do to the idea that pain is perception? Valerie Hardcastle discusses these questions in Chapter 28. Jesse Prinz (Chapter 19) has a wide-ranging discussion of further questions about unconscious perception. For example, it is well demonstrated that subjects fail outwardly to respond to or inwardly to notice certain large events in their visual fields. Should we say that they nonetheless see these events? The answer is far from obvious. One important feature of internal perception is that it carries a certain feeling of ownership—the internal states of our bodies are felt as our own, and moreover our bodies are felt as the subjects of internal states. This intimate connection between ourselves and the things that happen in our bodies can be disrupted, but it has been argued, notably by Wittgenstein, that it is ultimately immune to error. Mandrigin and Thompson review the issue in Chapter 27, and Christopher Peacocke, Chapter 9, takes us through philosophically vital related issues of the first person perspective on objects of perception.

VII Despite the differences among individual senses discussed in section VI, commonalities of function and functioning should also be noted. It is characteristic of all of the senses that they present at least some of their content as a continuously varying quantity. As Diana Raffman writes, Chapter 36: ‘one object will

20   Mohan Matthen look bluer or larger, or darker or brighter, than another, and one tone will be louder than another, or sound more stable or more tonally centred in a given musical key.’ Plato was the first explicitly to notice this, in the Philebus. He argued that it was an indication of the incoherence of the senses (and also of pleasure) that their content is presented comparatively, in terms of ‘more and less’. It is a virtue of reason, he argues, that it imposes absolute limits on this indefinite substrate of the more and the less. Syllables and numbers mark well-defined absolute measures on pitch and tone and on indefinite quantity, he claims; they are contributed by the rational mind. Putting aside the normatively loaded claim about reason, Plato was prescient. In speech, pitch, tone, and timbre are comparative, while phonemes are not. In the realm of phonemes, there is ‘categorical perception’, by which a phoneme like /ba/ can be recognized as the same across different speakers, despite differences among them as to how they voice these phonemes—how loud, how high-pitched, whether in a bass or baritone, and so on. There is a somewhat analogous situation in colour perception too, where the named ‘basic’ colours (blue, red, green, etc.) look sharply different from one another, despite a continuously graded difference underlying these boundaries. It is because of this perceptual jump across colour boundaries that the rainbow appears banded, though it is actually a continuous wavelength gradient—there is a phenomenological jump between wavelengthadjacent shades of blue and green, but no such jump within blue or within green. Harmony might also be an instance of this: a gradient of gradually less discordant chords abruptly jumps to harmony. Diana Raffman (Chapter 36) deals with these issues; she also introduces us to the representation of similarity relations as abstract ‘spaces’ in which closeness represents similarity. Another characteristic of all of the senses is that they display ‘constancies’ (Cohen, Chapter 33). Constancy is most apparent and has been most studied in vision. The retinal image is a product not only of characteristics of external objects, but also of the circumstances of viewing. A white cloth will, for example, throw a red image onto the retina when it is bathed in red light. Yet, visual system function is to extract from this image a message about the characteristics of the colour of the object independently of the illumination in which it stands; the white object should look white, and within limits, it usually does. (Akins and Hahn, Chapter 22, have a nuanced discussion, relevant here, of how we come to see things as being of a colour.) The same sort of thing is true of the other senses. In audition, a voice can be heard as possessing constant qualities despite interfering noise from other sources; the sound of the car is kept separate from that of your passenger’s voice. In touch, a granite countertop is felt to be hard even when it is stroked with a soft polishing cloth, which presents its yielding structure to the hand. Constancy is generally thought of as revealing the orientation of the senses toward detecting properties of a stable external environment. As Cohen (Chapter 33) writes: ‘it seems clear that constancy is an absolutely fundamental aspect of perception, [which] will figure centrally in our ultimate understanding of mind–world interaction.’ Attention is a feature of perceptual systems, the facility by means of which we are able consciously to extract information from a perceptual state. There are many forms of attention. In the 1980s, Anne Treisman and co-workers showed that one form enabled ‘binding’. In the traditional RIM paradigm, form is extracted from colour and brightness, and this view is intuitively plausible because it seems, phenomenologically, that the boundaries of form and shape are in fact colour and brightness boundaries. However, it was becoming

Introduction   21 increasingly clear by the 1980s that form and colour were separately detected by separate parts of the brain. Treisman’s discovery was that, having been separately detected, colour and form are ‘bound’ together when the subject attends to them. This shows that the fundamental phenomenology of vision—that of the coincidence of colour and form boundaries—is in fact a product of attention, not just of simple perception. Attention and perception work together to produce characteristic visual appearances. John Campbell, Chapter 31, is on the trail of analogous synergies between perception (mainly vision) and attention. He explores how it makes knowledge possible and how it modifies perceptual experience. Though, as we saw in section VI, the senses are very different from one another in how they process information and how they present the world around us, there are clear communicative channels between them. Earlier we noticed that the perception of speech and of flavour are multi-modal; some say that touch is multi-modal as well. In Chapter 32, Bayne and Spence discuss forms of multi-modal perception. (Their view is nuanced, as they argue that we might never be conscious of more than one modality at any given time. Barry Smith also discusses multi-modal perception in Chapter 17.) One consequence of multi-modal interactions is that the modalities can sometimes get mixed up. One such confusion is synaesthesia. In a significant number of people, perception in one modality gets expressed in another. For example, some experience a particular colour whenever a particular musical note is played. There is also within-modality synaesthesia: some associate individual alphanumeric characters with individual colour, always experiencing, for instance, red when they read ‘6’. Malika Auvray and Ophelia Deroy discuss the varieties of the phenomenon and its proper philosophical description in Chapter 34. Another cross-modal ‘confusion’, often constructively employed, is the ability to use one modality in place of another—for instance, to retrieve visual information from specially arranged tactile stimulation. The phenomenon was discovered by Paul Bach-y-Rita in the 1970s. He converted the brightness levels in a scene into a vibratory field projected onto subjects’ backs. (Small vibrators were set to respond proportionally to the brightness of their field positions.) The result was astonishing. The subjects began to discern characteristically visual phenomena such as perspective and occlusion in the scenes before them, and were able to do so using just the tactile image projected onto their backs. This is the phenomenon called ‘sensory substitution’. Can we say that Bach-y-Rita’s patients saw the scenes in front of them? Julian Kiverstein, Mirko Farina, and Andy Clark explore the ramifications of this and other questions in Chapter 35. One particularly interesting feature about perception is that we are not all equally good at it. Some wine consumers cannot tell the difference between white and red wine; expert tasters can make fine distinctions regarding the origin and the age of wine. (There has been a good deal of scepticism evinced on this topic lately; some of it should be quashed by watching the documentary film Somm, which follows the trials of four candidates vying for the designation of master sommelier. However that might be, my point is quite simple—there are some who cannot make even the simplest distinction in this domain; there are others who can make quite a few more.) There are those who can recognize in a glance the provenance of an old painting; there are others who can barely tell whether it is Indonesian or French. Robert Goldstone and Lisa Byrge, Chapter 42, argue that at least some of these differences in discernment arise from ‘perceptual learning’. Perceptual discrimination is sharpened by repeated practice and exposure. Or, as Goldstone and Byrge write, ‘Perception can be learned. Experience shapes the way people see and hear.’

22   Mohan Matthen To end this section, a word about pictures. What is it about visual perception that allows us to see three-dimensional scenes in two-dimensional arrays of pigment? What is it about auditory perception that makes a musical sequence of tones bear emotional content? Why do we find value in these seemingly ephemeral exercises of our perceptual capacity? Dominic Lopes explores these questions in Chapter 45.

VIII The two preceding sections took up questions about sensory processes. For instance, we asked about visual processing and whether it is property based or event based; we asked about the vestibular sense and whether it has a distinctive phenomenology. Philosophers of perception also ask, more broadly, about the nature of perception as a general faculty. We have touched on some aspects of this question earlier, especially in connection with the history of the subject in sections IV and V above. Further questions remain. What is the nature of the connection between perceiver and the world they perceive? How does perception relate to belief and the rational justification of belief? Let us put idealism to one side—the position that there is nothing outside minds—and also scepticism—the position that we have no reason to believe anything about the external world. On neither of these views does the question arise of how perception rationally grounds belief about the external world: for the idealist, we do not perceive an external world; for the sceptic, beliefs about the external world are not justified by perception. The question then is this. Suppose we take it for granted that we have, or can have, rational beliefs about the external world. How would perception justify such a belief? The standard view in the seventeenth century up until the beginning of the twentieth, and among empiricists, for much longer, was that perception was directly of a realm of ideas, or sense data, that come between us and the external world, and only indirectly of the latter. As we saw in our earlier discussion of scepticism (section V), indirect realism is motivated by the argument from illusion—the idea that perception can fail. If two perceptual states are the same—say a mirage and a veridical perception of a distant body of water—then they must have the same object. Since the mirage fails, then, and by definition lacks an external object, the veridical perception must lack an external object too. Both lack an external object. They must both be directed to their common apparent object, which is internal. They are both directed toward a sense-datum; the difference is that the veridical perception happens to be validated by the actual presence of water on the horizon. As Paul Snowdon relates in his chapter, Chapter 6, this seems to posit an unanalysed psychological act-object relation and a realm of immaterial objects, sense-data. Sense-data are decidedly queer: when I see something blue, sense-datum theorists say, my visual state is directed at something in the mind. Things in the mind don’t have extension or colour. They are not literally blue, or literally any other colour. This rather mystical approach to the objects of perception did not sit well with the increasingly dominant mid-century ethos of naturalism and materialism. A. J. Ayer tried to wriggle out of the quandary by saying that the sense-datum theory was just a way of talking, an ‘alternative language’, as he put it, not a substantive ontological proposal. Coming from a strong partisan of the theory, this seemed like a desperate stratagem, for it is unclear how exactly the supposed

Introduction   23 sameness of two perceptual states is accommodated by a simple terminological shift away from ordinary object talk. Sense-datum theory passed out of fashion after Ayer; by 1980, despite some revisionary attempts to revive it by two Australians, Frank Jackson and Brian O’Shaughnessy, it was essentially gone from the scene. Some credit J. L. Austin and Ludwig Wittgenstein for the defeat of sense-datum theory. It is equally plausible to lay the blame at the feet of its last great proponent. Another view of the perceptual relation originates from the works of Franz Brentano and Edmund Husserl, treated together with the philosophy of Maurice Merleau-Ponty by Charles Siewert in his chapter on Phenomenological Approaches, Chapter 7. At some level, Brentano’s approach is similar to that of sense-datum theory. He proposes that, in a way, the ‘referent’ of perceptual states exists in the mind, and this is just the sense-datum view. However, Brentano introduced an important analogy between perception and linguistic affirmation. When I judge that there is a body of water on the horizon, I affirm something. There is something that I affirm, regardless of whether it is true or false. Something of the sort holds also for perception. The mirage of water on the horizon and the veridical perception of an actual wadi both ‘affirm’ the presence of water—that is what they have in common. The mirage is false; that is how it differs from the veridical perception. Husserl brought a sophisticated theory of meaning to Brentano’s basic insight and was thus able to evade the idea that the sameness of perceptual states must consist in the sameness of their ‘referent’. According to him, perceptual states have a noêma as sentences have a meaning, or Fregean sense. (Nanay, Chapter 8, shows how this approach is elaborated in contemporary analytic philosophy of perception, which deals with perceptual representation in terms taken from analytic philosophy of language.) This enabled him to show how two different perceptual states could have the same noêma, but, when external circumstances change, have different external referents. According to this way of thinking, the situation is analogous to the following: ‘Versailles is where the King of France lives,’ had the same meaning in the eighteenth century as it does now, but it was true then (the circumstances made it so) but false now. This provided Husserl with a new approach to defusing failures of perception. Perceptual states can also have different meanings but the same external referent. This is something that both Brentano and sense-datum theorists are unable to accommodate, but crucial to showing how perceptual states can have external referents. Husserl rids himself of intermediate entities in between the mind and the external world. He is a direct realist who has no need for an intermediate realm of immaterial entities. As Siewert tells us, there is an anti-scientistic strain in phenomenology, though this is later much tempered by Merleau-Ponty. Brentano and Husserl both insist that the meaning of perceptual states is directly available to perceivers. This attitude is the underpinning of Husserl’s later notion of a Lebenswelt or ‘life-world’: a socially constructed but deeply entrenched way of understanding the world given in perception. Scientific concepts, such as those that were employed by the psychologists of the time, were out of place in the psychological description of perceptual states. We do not, for example, perceive ordinary things as possessing reflectance or as emitting compression waves; rather, we perceive them as coloured and as noisy. This aspect of phenomenology has received less attention in analytic circles than it deserves. Two new approaches to the relationship between perception and world have entered the field in the last few decades:  ‘naive realism’, or disjunctivism, which Heather Logue writes about in Chapter 11, and ‘enactive’ accounts, which are dealt with in Pierre Jacob’s

24   Mohan Matthen chapter, Chapter 12. Disjunctivism is a return to a reference-only account of the sort that Husserl tried to escape with his notion of noêma. A perceptual state is, on this view, partially constituted by its object (or referent). As a consequence, a perceptual relation that I bear to one thing is as such different from the same perceptual relation borne to another, or to nothing. It follows that a hallucination, which has no object, is as such different from a veridical perception, which has an object. Perceivers may not be able to discern this difference, but it exists nonetheless. (Hallucinations and veridical perceptions may be indiscernible, but still they share no specific commonality; thus, they can be united only by a disjunction—hence the name, ‘disjunctivism’.) Logue details four different contexts in which this position has been advanced. John Campbell, in his chapter on attention, Chapter 31, argues that this phenomenon can be properly understood only on a disjunctive approach. Pierre Jacob introduces ‘action based accounts’ in Chapter 12—accounts which have also been called ‘sensorimotor’ and ‘enactivist’. Unlike disjunctivism, the primary motivation for these approaches is empirical; the idea is that enactivism best makes sense of certain experimentally determined facts about perception. This line of thought goes back to the work of J. J. Gibson, who claimed that we perceive ‘affordances’, or the possibilities for action that objects ‘afford’ us. For example, birds perceive branches of trees as possible places to perch; we perceive chairs and stools in an analogous way. An even earlier antecedent is Jakob von Uexküll’s influential notion that animals perceive the world in terms only of how it affects their actions, which is faintly echoed in Husserl’s notion of the life-world. Enactivism is sometimes applied to specific sorts of perception, rather than across the board. Jacob illustrates this with the perception of the actions of others. According to one important paradigm, we observe action by covertly simulating it. For instance, Giacomo Rizzolati and co-workers at the University of Parma have developed a paradigm in which perceiving somebody else reaching for an object is inwardly re-enacting oneself reaching for an object. They found, for example, that when a monkey looked at somebody reaching for an object or grasping it, the corresponding motor neuron, that is, the one by which the monkey would control the same action, is activated. By extension, we might perceive speech by inwardly mimicking it, attribute thoughts to others by rehearsing how we would ourselves think in their situation, and so on. Perceiving action is intimately tied up with performing it.

IX How we talk about perception—how we use verbs like ‘see’, for example—offers us some clues about how we use it. Berit Brogaard writes about perceptual reports in Chapter 13. She observes that the verb ‘seem’, which we often use non-perceptually, is an etymological cousin of ‘see’, and that the other perceptual verbs similarly have non-perceptual uses. These perceptual verbs are often used ‘epistemically’, that is, they are used to report beliefs, usually (but not always) beliefs that are perceptually supported—for example, ‘It looks as if the lecturer is late.’ (This, interestingly enough, reports an absence—Roy Sorensen, Chapter 29, discusses ways of approaching ‘Perceiving Nothings’.) As we noted earlier, perception was probably not recognized as a distinct faculty in early Greek philosophy; it was not clearly marked off in linguistic terms then, and it still is not

Introduction   25 in modern languages. As well, we use learned associations to describe beliefs, as Roderick Chisholm’s ‘comparative use’ shows: ‘The cliff looked like a dried-out body,’ to repeat a slightly macabre example of Brogaard’s. It seems plausible, then, to say that perception and belief are not clearly distinguished in ordinary talk. The philosophical question of how perception relates to and rationally supports belief is of relatively recent origin, and should be regarded not as intuitively founded, but the product of science and philosophical analysis. Ordinary speech does not make a clear distinction. If there is a rational connection between perception and belief, what does this tell us about perception? Some say that since belief is conceptually articulated, perception must be so as well. If my perceptual state is to rationalize the belief that this ball is white and smooth, and if white and smooth are concepts applied to the ball, then the perceptual state must contain these concepts too. How else would it rationalize the belief? John McDowell affirms this connection in a particularly strong form, holding (after Kant) that perception itself would not be possible if concepts were not drawn upon in the ‘receptivity’ that leads to perceptual experience. There are doubts. Some point out that animals would not be capable of perception on this account since they do not possess concepts. It could be argued in response that the explicit possession of concepts may not be needed for the reception of conceptually articulated content—there may be ways of registering the application of perceptual concepts to the ball that do not demand this. It is well to note that McDowell’s version of conceptual content would not be mitigated by this stipulation, since he requires that the concepts be drawn on in receptivity itself. The nest of issues surrounding perception and concepts are fully discussed by Wayne Wright, Chapter 10; animal perception is treated of by Brian Keeley in Chapter 44, who discusses the comparability of animal perception to that in humans. Let this suffice as a review of some broad issues about perception and to demonstrate the richness of the study and its utility for philosophical inquiry. The authors of this Handbook have produced original and searching, but at the same time introductory, surveys of issues at the forefront. I hope that you, the reader, benefit from their efforts.

Acknowledgements I am very grateful to the Canada Research Chairs programme, the Social Sciences and Humanities Research Council of Canada, and the University of Toronto for sustained support of my research. My post-graduate fellows, Dustin Stokes and Stephen Biggs, and my graduate students, Matthew Fulkerson and Kevin Connolly, have been an inspiration and a source of instruction. (Kevin served as post-doctoral fellow as well.) Peter Momtchiloff of Oxford University Press suggested this project and offered encouragement at times when things seemed especially difficult; Barry Smith, Ophelia Deroy, and the Centre for the Study of the Senses in London drew me in, pushed the project forward, and introduced me to the world beyond. Lana Kühle brought the project to completion with her invaluable editorial assistance and taught me a lot about phenomenological approaches while she was at it. Steven Coyne took on the preparation of the index with calm competence, and I am hugely appreciative.

Pa rt  I

H ISTOR IC A L BAC KGROU N D

Chapter 1

Perception i n A ncien t Gr eek Phil osoph y Victor Caston

Many of the central questions in the Western philosophical tradition about perception— regarding the metaphysics of perception, the nature of perceptual content, and the role of perception as a basis for empirical knowledge—were first raised in ancient Greece and Rome, where they were the subject of detailed discussion and debate. This chapter will concentrate on the first two concerns, the metaphysics of perception and its content, from the beginnings of Greek philosophy through Plato and Aristotle, when the main lines of inquiry are initially formed. The subsequent development of these issues in later antiquity—most notably, the treatment of propositional content in the Stoics; the Epicureans’ distinction between perceptual belief and what is given in perception; the sceptics’ worries about illusions and phenomenal indiscernibility; and the role of concepts in perception in later Platonism—is too large a subject to cover here.1 For each set of philosophers, I will touch briefly on four issues: how perception is related to the body and soul; the nature of perception itself, including accounts of individual senses; what can be perceived; and perceptual awareness.2

1  The emergence of perception as a philosophical topic There can be no doubt that the Greeks, from Homer on, frequently speak of seeing, hearing, smelling, tasting, and touching. But did early Greek philosophers regard these as

1  For a fuller treatment of these issues from the beginnings of Greek philosophy to later Neoplatonism, with translations and more extensive references to primary sources and scholarship, see my Perception in Ancient Philosophy (in progress). 2  In what follows, all translations are my own. I use standard abbreviations for the titles of ancient works, along with citations for Presocratic fragments and testimonia from H. Diels and W. Kranz, Die Fragmente der Vorsokratiker, 6th edn (= DK).

30   Victor Caston activities that naturally belong together and are distinct in kind from the other activities of living things? It might seem odd to question this, given the obviousness of organs like the eyes, ears, nose, tongue, and hands, not to mention the familiar manipulation of our bodies to get a better look at something or to come within earshot, as well as the inevitable impairments and injuries that frustrate such access. In light of these facts, it is easy to take talk about the senses for granted, as necessarily presupposing the concept of perception. But this presumption has been challenged in two opposing ways. Some have argued that perception was not originally viewed as a distinctive type of cognition. The early Greeks understood all cognitive functions on the model of the senses, taking thought and emotion to be closely identified with specific bodily organs similar to the eyes and ears, though internal to the body. Higher cognitive functions were only distinguished from perception gradually over time.3 At the other extreme, it has been argued that it is perception that is the latecomer here. Remarkably late, in fact. On this view, the original notion of perception is due to Plato (427–347 bce), developed for philosophical purposes in a specific passage of the Theaetetus (184–186). It is something confined to the organs, passive in nature, and non-cognitive, in contrast with the cognitive, rational activity of the soul by itself.4 So there is a question as to when a concept of perception first emerges, whether it is present from the beginning of reflection on the topic, or only a later development. Both views agree, however, that perception is not distinguished from other forms of cognition by earlier thinkers. This observation is not new. Aristotle (384–322 bce) and Theophrastus (371–287 bce), his colleague and former student, claim that their predecessors believed that perception and thought were ‘the same’ and a kind of bodily alteration.5 This is an overstatement in several regards, as Theophrastus seems to have recognized himself.6 But there is an element of truth as well in it that needs to be acknowledged. The verb aisthanesthai, which is standardly used by later philosophers for perceiving as a specific type of cognitive activity, occurs early on predominantly in a broad epistemic sense for noticing, realizing, or grasping some fact, much like broader uses of the English ‘perceive’. In such uses, what is noticed or recognized need not be an object of direct observation, but may be something arrived at by testimony or inference.7 The noun aisthêsis, in contrast, is used more narrowly for sense perception. But it only begins to appear somewhat later in the fifth century bce, much as the two views we considered above would predict. Still, it would be a mistake, methodologically speaking, to rely so heavily on a single term or family of cognate terms. There is no good reason to think that the use of a concept is ever tied so closely to a single word: terminology often develops later, well after conceptual distinctions have emerged and begun to firm up. Restricting our scope to specific terms would unnecessarily blinker our investigation. To appreciate this, we need only think of one of the more central themes in early Greek philosophy, the opposition between experience and reason, where the former is usually expressed simply by reference to the eyes and the ears. This opposition is unintelligible 3 

See esp. Snell, 1960: ch. 1 (‘Homer’s View of Man’). For a healthy corrective, see the excellent articles of Lesher (1994) and Hussey (1990). 4  Frede (1987). 5 Arist. DA 3.3, 427a21–7; Metaph. 4.5, 1009b12–15; Theophr. De sens. 4, 23. 6  For an excellent critical examination of these claims, see Laks, 1999: 255–62 and Lesher, 1994: 11–12; see also Caston, 1996: 25–7, 33–8. 7  For wide-ranging and detailed examination of the use of this family of terms before Plato, in both Ionic and Attic, philosophical and non-philosophical authors, see Schirren (1998).

Perception in Ancient Greek Philosophy    31 unless there is some significant contrast between two broad types of cognition, where the eyes, ears, and other sense organs represent a more or less unified group, whether or not they are denominated by a single noun like aisthêsis. On their own, we are repeatedly warned, our eyes and ears lead us into confusion and error; if they are to be of any use, we must bring our powers of understanding to bear. Heraclitus (6th–5th century bce) complains that the eyes and ears are ‘bad witnesses’ for those who do not understand the language of the senses, literally, those who have ‘barbarian’ souls (DK 22 B107). Epicharmus (early 5th century bce) goes further. He claims that it is the understanding (nous) which sees and hears, in contrast with ‘the others’—the ears and eyes themselves—which are paradoxically said to be ‘deaf and blind’ (DK 23 B12). Parmenides (early 5th century bce) suggests that we may have to disregard our sensory experience in an even more radical way. The goddess in his poem admonishes us not to follow our ‘aimless eye and echoing ear’, but to judge her argument solely by reason (logos),8 a trope echoed later by Empedocles (c.495–435 bce).9 Later in the fifth century, the Hippocratic treatise On Art claims that the causes of disease elude the ‘sight of the eyes’ and can only be grasped by the ‘sight of the mind’ (têi tês gnômês opsei, 11.1–2). The point of the contrast is not simply rhetorical or protreptic. The same opposition can be seen in Melissus’ argument (mid-5th century bce) that our experience of sensory qualities conflicts with the principles of logic, which can only be resolved by accepting monism (DK 30 B8). The contrast is likewise central in the epistemology of Democritus (mid-late 5th century bce). In his Canons, he distinguishes two forms of cognition (gnômê), one ‘legitimate,’ the other ‘illegitimate’ (skotiê, literally, ‘born in the shadows’), to which ‘all of these belong: sight, hearing, smell, taste, contact’ (DK 68 B11).10 The list of all five canonical senses, including touch, is striking and he even uses the verb aisthanesthai later in the fragment, making it clear that is ‘perceiving by contact’, rather than just coming into contact. Unlike Melissus, though, Democritus does not see the relationship between these two types of knowledge as a simple either/or choice. In another fragment he imagines the senses in a court of law, accusing reason of taking its evidence from them and using it to refute them; in so doing, the senses warn, reason will only undermine itself (DK 68 B125). The sophist Critias (460–403 bce) also uses the verb in a restricted sense, when he contrasts what is known by the mind (gignôskei) with ‘what is perceived by the rest of the body’ (aisthanetai, DK 88 B39). The noun aisthêsis likewise occurs before Plato in the relevant sense. It may be used as early as Alcmaeon of Croton (early 5th century) in a fragment that contrasts understanding with sense perception: he claims that while only humans possess understanding, all animals have perception (DK 24 B1a). Even if one questioned whether this is a verbatim quotation though, the general distinction is not in doubt. And the noun certainly occurs in the Pythagorean Philoloaus of Croton (c.470–380s bce), to mark a similar distinction between animals, humans, and plants, where the heart is the organ that governs animals and perception (DK 44 B13). A broad distinction between perception and reason, then, which had 8 

DK 28 B7.4–5. Something similar is reported for Xenophanes (DK 21 A32). In DK 31 B17.21, he urges us to look on the cosmological role of Love, not with ‘confused eyes’, but with our understanding instead. 10  For discussion, see Kahn, 1985:  19–21, although I  see Democritus as more continuous with the tradition than Kahn does. 9 

32   Victor Caston begun to emerge early in the 5th century bce, is firmly in place by the end of that century, well before Plato wrote. This is not to say that there are not competing conceptions of perception or of its range. When Plato offers a theory of perception in support of Protagorean relativism—which he hyperbolically claims is shared by nearly all of his predecessors (see section 3)—he not only characterizes the activities of canonical sense modalities like seeing, hearing, smelling, and feeling heat or cold as ‘perceptions’ (aisthêseis), but also pleasures and pains, desires, fears, and ‘countless others without a name’ (Tht. 156b2–7), a list that suggests a broader category of feeling or experience.11 But the existence of competing views over the extension of the concept, or even over its exact nature, is compatible with there already being to hand some broad notion of perception, as distinct from thought or reasoning.

2  The presocratics As far as we can tell from our sources, the Presocratics’ discussions of the senses were devoted largely to the physics and physiology of perception.12 Much of our evidence concerns how information about distant objects is transmitted by physical means across the intervening medium and into the orifices of various organs,13 along with a certain amount of detail on the structure and material constitution of these organs. One of the more elaborate and interesting accounts occurs in Empedocles, who describes humans as having narrow ‘receptors’—literally, ‘palms’ or devices for grasping (palamai)—spread throughout the body, though he complains that they are limited in what they are exposed to and wear down over time (DK 31 B2.1–3). He urges us to make use of every type of receptor and not to favour sight over hearing or hearing over taste or any others through which there is a ‘conduit for understanding’ (poros noêsai), so that we may understand ‘how each thing is manifest’ (DK 31 B3.9–13). The idea that perception involves conduits or channels (poroi) in the body is common among the Presocratics and goes back at least to Alcmaeon, who claimed that the peripheral sense organs were connected by conduits to a central organ, which he located in the brain (DK 24 A5; cf. A11). In Empedocles, the channels lie instead at the interface between subject and object. Their openings take in the ‘emanations’ (aporrhoiai) or streams of matter that flow from external objects through the intervening medium and enter not only obvious orifices such as ears or nostrils, but also tiny, imperceptible passageways that make up the crystalline lens of the eye and the porous membrane of the tongue.14

11 

Solmsen (1968) generalizes this point, in fact, arguing that the Greek aisthêsis and Latin sensus are ambiguous between perception and feeling throughout ancient philosophy. 12  Beare (1906), though questionable and outdated on many points, is still useful as a compendium for a preliminary survey of the evidence. Much of it derives from critical summaries in Aristotle and above all from Theophrastus’ De sensibus, an invaluable resource for early theories of perception. There is an English translation and commentary of the latter by the psychologist G. M. Stratton (1917), though it too is very dated and needs to be redone in light of recent advances. For an examination of the methodology of Theophrastus’ treatise, see Baltussen (2000). 13  On the ‘topology of sensation’, see Laks, 1999: 263–7. 14  The longest (and most famous) description of the poroi is not attributed to Empedocles directly, but to Gorgias as the student of Empedocles (DK 31 A92). But it is confirmed for Empedocles as the mechanism

Perception in Ancient Greek Philosophy    33 The conduits for each sense differ in gauge, so that only matter of a ‘commensurate’ size (summetros) can ‘fit’ into them snugly (enharmottein). This, Empedocles claims, explains why each sense has its own proper objects and cannot perceive those of another. A sense will not perceive qualities whose material is either too large to enter or so small that it passes through without making contact (DK 31 A86 §7; A90; cf. A87). Fitting into a conduit is merely a necessary condition, however. For perception to take place, the emanation must further encounter material of a similar kind within the subject, on the principle that ‘Like is known by Like’ (DK 31 A86 §15).15 This principle has both a causal dimension and an intentional one, concerning the content of perception: what we perceive on a given occasion must be like material inside our organs because (1) a perceptible object can only affect something like itself and thereby (2) bring about perception of itself. Both Aristotle and Theophrastus contrast this principle with Anaxagoras’ view that only unlike or contrary materials can affect each other and so stimulate perception (which they also connect to his view that all perception involves a kind of irritation or pain).16 In Empedocles, the likeness principle has a further significance, though. It is not possible without the analysis or separation of compounds into their constituent elements that he associates with the cosmological force of Strife. At the same time, perception also brings disparate things together, by fitting of matter into orifices and joining subject to object, and to this extent performs the work of Love (not unlike ‘knowing’ in the biblical sense). In his poem, Empedocles offers detailed and colourful descriptions of the mechanisms involved for the individual senses, most famously comparing the eye to a lantern, whose light passes through screens made of horn into the surrounding darkness. It has often been thought, starting with Aristotle, that this was part of extromission theory of vision, prefiguring the theory in Plato’s Timaeus (see section 3). But it is more likely a part of Empedocles’ account of night vision, which he appears to have discussed in some detail. Nocturnal animals must compensate for the surrounding darkness with larger amounts of fire inside the eye, which we can observe when we see their eyes flash in the night; diurnal animals, in contrast, require more water in order to compensate for the increased brightness during the day.17 He compared the ear to a bell, in which the sounds from our environment echo (De sens. 9 = DK 31 A86), though it is unclear exactly how an internal sound is supposed to help. As Theophrastus rightly objects, how would we in turn hear it? The same thing needs to be explained all over again, he complains, and a homuncular regress looms.18 for perception in general at DK 31 A86 (§§7, 9) and A87. On emanations (aporrhoiai) specifically, see DK 31 B89; also A86, §7. 15 

For the principle that ‘like is known by like’: DK 31 B109 (also Arist. DA 3.3, 427a27–9); DK 31 A86 §§1, 2, 10, 15, 17. Cf. DK 31 B90. 16  On contraries: Theophr. De sens. 27 (cf. 31–3); cf. Arist. DA 2.5, 416b35–417a2. On pain: Theophr. De sens. 17, 29. For a thorough examination of Anaxagoras’ general views on perception, especially his views that all perception involves pain or irritation and that there are least perceptible differences beyond which we cannot discriminate (DK 59 B21), see Warren, 2007: 19–36. 17  Lantern fragment: DK 31 B84; cf. A86 §7. Day and night vision: DK 31 A86 §§8, 18; cf. A91. For a defence of the interpretation here, see Caston (1986) and, along somewhat different lines, Sedley (1992). Katerina Ierodiakonou (2005) has demonstrated convincingly that Empedocles’ account of colour vision acknowledges only two primary colours, white and black (or more exactly, light and dark), the rest being the result of their mixture in various proportions (DK 31 A86 §59; cf. also DK 68 A135 §79). 18  De sens. 21 (= DK 31 A86 §21), reading to gar auto with the mss. Empedocles has little to say about the remaining senses. According to Theophrastus, he only links smelling to breathing and says nothing about

34   Victor Caston In virtually every ancient theory of perception, despite differences in physical theory and many details, one can find the basic framework of a causal theory of perception, the idea that the sensible characteristics of an object are in some way transmitted to the animal and affect its sense organs so as to produce a perception of those very characteristics and the object to which it belongs. Some of the variations, however, are significant and influential in their own right. The sophist Gorgias (early 5th century to early 4th bce), for example, is represented in Plato’s Meno as having been a student of Empedocles’ and accepted his theory of vision (76c4–e1 = DK 31 A92). Nevertheless, in his own work, the Encomium of Helen, Gorgias speaks of the soul being ‘impressed’ or ‘moulded’ (tupoutai) through sight with profound emotional and motivational effects (§15 = DK 82 B11), because of the way sight ‘inscribes on the mind likenesses of the things seen’ (eikonas tôn horômenôn pragmatôn hê opsis enegrapsen en tôi phronêmati, §17 = DK 82 B11), analogies which recur in Plato, Aristotle, and the Stoics. Democritus similarly speaks of an ‘impression’ (entupô­ sis) in explaining how sight comes about, comparing it to the moulds or imprints made in wax (hôsper kai autos legei paraballôn . . . hoion ei ekmaxeias eis kêron, DK 68 A135 §51, cf. 52).19 The image (emphasis) is not produced directly in the eye, however, but is impressed (tupousthai) into the air in front of it, which gets compressed between the eye and the emanation from the object (§50; cf. 74, 80). At first glance, this seems like a quite different theory than the one commonly ascribed to Democritus, which appeals to so-called ‘simulacra’ (eidôla) or ‘replicas’ (deikela), as he also called them (DK 68 B123): thin surface layers which are continually shed by objects and more or less preserve their shape as they move through space. But the two mechanisms are plainly compatible, and it is arguable that both are necessary. Distant objects cannot directly make an impression on the air in front of the eye, given their location. But their surface layers can, once they have become detached, so long as the arrangement of atoms retains a similar contour to the original object. On the other hand, it is plausible to think that only something fine and light like air could enter the eye, rather than the vast assortment of atoms emitted by objects. Democritus appears to have discussed the structure of the eye in extraordinary detail and paid special attention to the material conditions that would allow it to receive the impression. Among other things, the surface of the eye must be moist and have a thin exterior coating, free of thick grease or flesh, with fine, straight, empty passages to adapt to the impression’s shape (DK 68 A135 §50, cf. 52). Through these mechanisms, Democritus hoped to explain how the contours of an object could be transferred to the eye’s interior, along with information about perspective and possibly distance as well.20 Democritus devotes a great deal of attention to perceptible qualities such as colours and flavours, as well as certain tactile qualities like weight and hardness.21 He consistently appeals to the microstructure of objects to explain these qualities and geometric properties in particular. Most often these are characteristics of molecular structures: their overall size and shape, as well as the position and arrangement of specific types of atoms and void touch and taste beyond the general claim that all perception is due to emanations fitting into conduits (DK 31 A86 §§9, 20). 19 

Accepting Burchard’s correction of sklêron; cf. apomattetai kathaper kêros in §52. On the role of perspective, see Rudolph (2011); on the perception of distance, see Avotins (1980); on the structure of the eye, see Rudolph (2012). 21  DK 68 A135 §§61–82. The list of Democritus’ works includes titles for separate essays on colours and flavours (DK 68 B5g & h; A33). 20 

Perception in Ancient Greek Philosophy    35 within them (DK 68 A135 §§60, 67). In fact, it is unclear whether his explanations ever turn on the geometric features of individual atoms. Molecular structures are indisputably at issue in his explanations of the weight and hardness, which are functions not merely of the size and density of macroscopic objects, and so the amount of void included, but also whether the atoms are arranged in regular patterns. Iron is lighter than lead because it contains more void, but harder because of the irregular distribution of atoms, with certain areas more densely packed than others (DK 68 A135 §§61–2). But the ‘shapes’ (skhêmata) of molecular structures are also plainly at issue in his explanation of colours, which appeals to conduits and passages, or to lattices involving alternating pairings of atoms, or to large or small conglomerations of atoms (§§73–6, cf. 82).22 Democritus distinguished four primary colours:  white, black, red, and yellow-green (§§73–5). He seems to have allowed for something like metamerism, at least in the case of white: not only are the inner surfaces of shells white, because of their hard, smooth surfaces and straight conduits, but also substances that are soft and crumble easily, which are composed of lattices of spherical atoms in alternating offset pairs (§73, cf. 79). Black, in contrast, is due to rough, uneven surfaces, with crooked and tangled conduits (§§74, 80); red to the fine-textured atoms that cause heat, though only in larger agglomerations; and yellow-green to various specific arrangements of atoms and void (§75). The remaining colours result from mixtures of other colours, each mixed colour corresponding to a distinct proportion (§78). Democritus thinks there are an infinite number of them, but gives specific combinations for at least eight types of greens, blues, and browns (§§76–8). Flavours are also explained by reference to microscopic shapes (§68), though Theophrastus’ report never makes clear whether these are shapes of molecules or individual atoms. A spicy or piquant flavour, for example, is due to tiny, jagged, angular shapes, while sweet flavours are due to somewhat larger, round ones: the former tear at our organs and create spaces, thereby heating them, while the latter permeate our body slowly and gently, moistening it and causing other atoms to flow. His explanations for sour, sharp, salty, and acrid flavours all turn on the extent to which the microstructures in food heat or cool the organs, dry or moisten them, solidify or loosen them, pass through them, or plug them up.23 It is because of such explanations, no doubt, that Aristotle claimed that Democritus effectively reduced perceptible qualities to geometric properties. According to Aristotle, he refers all perceptible qualities back to what Aristotle calls ‘common perceptibles’ such as shape and size (eis tauta anagousin, eis ta skhêmata anagei, Sens. 4, 442b4–12) and so ‘makes them all tangible’ (hapta poiousin, 442a29–b1). But Theophrastus complains that these explanations contradict other things Democritus says, which effectively make perceptible qualities into ‘modifications of the senses’ (pathê poiôn tês aisthêseôs); he even goes so far as to say that on Democritus’ account there is no nature to any of the perceptible qualities (oudenos phusin), despite the detailed accounts of the microstructures involved that Theophrastus reports (DK 68 A135 §§60–1, 63, 71; cf. also A49). For Democritus also seems to have maintained that perceptible objects do not appear the same to every perceiver, and that would have to be explained, Theophrastus argues, by appealing to the 22 

See Fritz, 1953: 95–9, who was the first to emphasize this. For the explanation of different flavours, see DK 68 A135 §§65–7; A129. For a general statement of the underlying explanatory strategy in terms of physical effects, see DK 68 A130. 23 

36   Victor Caston different constitutions perceivers have or the different conditions they are presently in, rather than the nature of the objects themselves—as the Epicurean Colotes would later charge, they will ‘not be qualified one way rather than another’ (ou mallon toion ê toion).24 This might even be taken to suggest a subjectivist reading of Democritus’ notorious saying ‘by convention bitter, by convention sweet; but in reality, nothing but atoms and the void’ (DK 68 B9). On this reading, perceptible qualities just belong to our experience of objects and not to the objects themselves; if we believe they do, we will be in error quite globally.25 It is clear from Theophrastus’ reasoning, however, that Democritus is anything but a subjectivist about perceptible qualities, much less an eliminativist. His explanation of conflicting perceptual appearances is objective and causal: how things appear perceptually is a result of the way objects affect different perceivers, in their current condition (DK 68 B9 §136). This is also evident from the detailed explanations Theophrastus cites, which take the geometrical properties of objects to be the central explanatory factor of the qualitative character of our experiences, along with Democritus’ general view that not only perceptions, but the content of beliefs and other mental states are a function of our bodily condition, to be understood ultimately in terms of its microstructure.26 Finally, he also seems to have claimed that whatever appears to us perceptually is true.27 So he cannot have thought that we were generally in error about perceptible qualities. How can these different reports be reconciled? Here is one suggestion. First, distinguish perceptible qualities and perceptible objects, both of which can equally be expressed by the Greek aisthêta, and accept the detailed explanations we find in Theophrastus at face value. Then on Democritus’ view, whenever we have an experience of a certain quality, we have been affected by a specific type of microstructure that impinges on our organs.28 If he further identifies the perceptible quality with this microstructure, then every perception will be true of something in our environment, namely, the object which possesses the microstructure affecting us. On the other hand, even if it does belong to the object, it may not be the only microstructure the object possesses. Wine contains microstructures that taste sweet as well as those that taste sour; a pigeon’s neck possesses structures that look green as well as those that look purple. Which ones happen to affect us on a particular occasion will be a function of the conditions in our environment, how we are situated in it, and the nature and state of our organs. The jagged character of spicy particles of food cannot do anything other than tear animals’ tongues. But it may be prevented from achieving this effect, or at least lessened, because of the yogurt that presently coats my tongue. Similarly, if I eat a dessert between glasses of wine, I will no longer be affected predominantly by the sweet structures in the wine that moisten my tongue, but only the tannic ones. But it will not be any more (ou mallon) true to say that 24 

DK 68 A135 §§63–4, 69 and the context of DK 68 B156 (= Plut. Adv. Colot. 4, 1108f–1109a). our ancient sources, a subjectivist reading is most strongly suggested by Galen’s gloss on the quotation at Elem. Hipp. 1.2 (= DK 68 A49). 26  Perception: DK 68 A135 §64. Belief and other mental states: DK 68 B7; A135 §58; cf. A101. 27  DK 68 A112; A101; A135 §69. Some reports suggest that Democritus held the contrary view that no perception was true and so was caught in contradiction; but the view they cite as evidence for this only claims that no one class of perceptions has any greater claim to truth than any other, which is plainly compatible with them all being equally true. 28  In speaking of a ‘microstructure’ in a Democritean context, I mean a molecular compound, which is a part of something larger. What matters is structure and parthood, not absolute size. 25  Of

Perception in Ancient Greek Philosophy    37 the wine is sweet rather than sour (DK 68 A134; A112). It has both kinds of structure and so my perceptions are both equally true of it, since they accurately correspond to the flavour structures in it that are then affecting me. Whether it is true to say that the wine is sweet (or sour), therefore, is precisely a matter of ‘convention’ (nomos): it is a matter of which characterizations of wine are accepted predominantly by perceivers in the relevant community. But the truth of the matter, as Democritus says, is ‘in the depths’ (DK 68 B117). It depends on which microstructures an object possesses and which ones are actually affecting a perceiver on a given occasion, detailed facts from which we are ‘removed’ (DK 68 B6). Hence, we quite frequently do not grasp how each object truly is or is not (DK 68 B10). The perceptual variability Democritus invokes thus concerns how objects can appear differently to different perceivers or on different occasions. The specific microstructures, in contrast, will always ‘appear the same to everyone and so be a test of their truth’, a remark Theophrastus is otherwise at a complete loss to explain (DK 68 A135 §69). Where all causal conditions are the same, including the situation and state of the observer, how things qualitatively seem to a perceiver will be the same, ‘as he himself seems to attest’ (hoper kai autos an dox­ eien epimarturein), because the underlying nature of these qualities is the same (§70). The appearance of qualities is thus invariant. The only variation is in how objects appear, about the qualities they seem to have, which is a function of just which microstructures in an object are actually affecting a subject in a given set of circumstances.

3 Plato Perception looms large in the writings of Plato, not least because of the contrast he draws time and again between the ever-changing realm of perceptible objects and the stable realm of intelligible Forms.29 But his remarks about perception in its own right are also of interest independently of this and cast a long shadow over the subsequent tradition. Because most of his works are in dialogue form, it is difficult to determine whether the views expressed in them can be attributed to Plato himself, or even whether the views expressed in one work can be conjoined with those in another. But thankfully these questions do not need to be answered here. It is sufficient that Plato found such views worth exploring, critically and at length, whether or not he subscribed to them himself or even found them tempting. A feature that recurs in many of Plato’s works is a very strong form of dualism of soul and body, for which he offers some of the most striking statements that can be found in the Western tradition. Not only does he regularly distinguish between the two, but he also stresses the independence and autonomy of the soul in relation to the body. On several occasions he argues not only for the soul’s continued existence after death, but its also having existed prior to this life, separate from a body and knowing things by itself on its own. But if we ask where perception is to be located within this scheme, interestingly Plato’s works differ widely as to whether it is primarily the work of the body, the soul, or both body and soul; and, if it belongs to both, as to how their respective roles are to be distinguished.

29 E.g., Phd. 78b4–79e7; Rep. V, 475e6–480a13; VI, 509d1–511e3; Tim. 27d5–31b3, 51b6–52d1.

38   Victor Caston In the Phaedo, for example, the senses are treated as powers of the body. When Socrates urges us to leave sense experience behind and to use reason purely by itself, he characterizes this as a separation of the soul from the body. When, moreover, he disparages the senses unremittingly as not offering any clear or accurate information and as leading the soul into error and confusion, he assumes that perceptions have content.30 Thus, perceptual activity both belongs to the body and has content on this view. Several later dialogues, in contrast, regard the soul as the locus of perceptual awareness. Even if the body and the soul are both affected in perception (Phlb. 33d4–6, 34a3–5), we are not conscious of this stimulation unless this affection is transmitted through the body to the soul (epi tên psukhên, mekhri tês psukhês) or mind (epi ton phronimon).31 In a pivotal passage of the Theaetetus, Socrates argues that we do not perceive with (hois) our senses, but through (di’ hôn) them (184c1–e7). Instead, we perceive with our souls (184d4–5, 185d3–4). The senses on this view are merely ‘instruments’ or ‘tools’ (organa) through which we access the world.32 He then goes on to argue, even more strongly, that perceptions cannot themselves constitute knowledge because they cannot ‘attain truth’ (186d2–e10). The interpretation of this last claim is controversial. But some take it to mean that perceptions do not have content (for discussion, see below in this section). Republic VII falls somewhere between these extremes. Like the Phaedo, it contrasts the senses with the soul (523a10–525a2). The senses also plainly transmit information: they are said to ‘report’ to the soul (parangellei têi psukhêi, 524a3; têi psukhêi hai hermêneiai, b1). But sometimes they do this in seemingly contradictory ways. When touch, for example, ‘says’ (legei) that my finger is both hard and soft, or sight says it is both large and small, the soul puzzles over what the relevant sense ‘means’ (sêmainei) by ‘hard’ (524a6–10) and calls upon thought and reasoning to sort it out (524b3–d5; cf. X, 602c4–603a9). But other features can be fully discerned by the senses (ta hupo tês aisthêseôs krinomena), which ‘reveal’ (dêloi) their nature, as for example when I see that my finger is a finger (523a10–b4, cf. c3, d5). In both cases, the activity of the senses has content and in the second they can even attain truth (cf. Phdr. 249b6–c1). In various dialogues, there is also discussion of the nature of perception as such and how it comes about. In the Theaetetus, Plato examines at length a causal theory of perception, which holds that our perceptions are about the very objects that bring them about.33 Discussion of it is complicated by the highly dialectical context in which it occurs. Socrates offers it in support of Protagorean relativism—the view that things actually are just as they appear for each person, at least as far as perception is concerned (152a1–c6)— to which the first half of the dialogue is devoted. But he also makes clear that this causal theory is not something the historical Protagoras was ever known to advocate. Socrates mischievously presents it as an esoteric ‘secret doctrine’ that Protagoras revealed only to his inner circle (152c8–10; 155d9–e1) and a ‘mystery’ which only the initiated know about (156a2–4, cf. 155e3), and in developing it draws heavily on Heraclitean notions of flux. 30 

65a9–b7; 65e6–66a8; 66d7–67b2; 79a1–c8; 83a1–b4; cf. 83d4–e3. Tim. 43c4–7, 45d1–3, 64b3–c7; Phlb. 33d2–34a5; cf. Tht. 186b11–c2. In these dialogues Plato seems to be saying that the perceiver is aware not merely of external objects, but of how they affect our body as well. On the bifurcated character of perception that results, see Ganson, 2005: esp. 1–2, 3–4. 32  Tht.184d4, 185c7–8; cf. 184d7–e7, 185d1–2, e7. 33  There are interesting parallels with Aristotle’s own theory, although they differ on several points. See section 4. 31 

Perception in Ancient Greek Philosophy    39 There is no decisive evidence that Plato accepted this causal theory himself either, though this is often assumed.34 It is explicitly offered as an amalgam of views shared by virtually all of his predecessors, from Homer on down, with Parmenides being the sole exception (152e1–153a4; cf. 155e–156a). The core idea is that perception arises when a perceptible object and a perceiver’s sense organ encounter each other and causally interact (153d8–154a3, 160c4–5). The object has the power to act, the organ the power to be acted upon (156a6–7, 182a6–9), and when they have ‘intercourse’, they produce a pair of ‘twins’ as offspring, an act of perceiving (aisthêsis) and a perceptible quality (aisthêton, 156a7–b1)—in fact, Plato coins the word ‘quality’ (poiotês) to characterize the latter more generally when he later summarizes this view (182a9–b1). The twins in turn qualify the ‘parents’: when the eye, for example, encounters a stone or a piece of wood and they together generate the twins whiteness and sight, the eye ‘becomes, not sight, but a seeing eye’ and the stone or wood ‘becomes not whiteness, but white’ (156d3–e9, 182a7–9). This sort of view, Plato argues, entails that objects only have perceptible qualities while a perceiver perceives them on a given occasion (156d5, 160b1–8). They are ‘private’ (idion) in the sense that they exist only in these individual encounters, in relation to the perceiver to whom it appears (154a2–3; cf. 166c4–7). No one else, he claims, can perceive the very same quality, just as that particular act of perceiving can only be directed at it (159e7–160a3, 160c4–5). But if perception is always directed at the object it is presently interacting with, as it is qualified through the interaction, then perception necessarily corresponds to its object and is unerringly true (160c7–9). In Plato’s view, the causal theory of perception underlying his predecessors’ views thus leads to and supports Protagoras’ relativism (160d1–3), as least with regard to perceptible qualities.35 Plato also explores a causal theory of perception in the Timaeus, where one likewise perceives objects as having certain qualities as a result of those very qualities acting on the relevant sense, but adds extensive details about the physical mechanisms of perception and the nature of the perceptible qualities themselves. He distinguishes between those affections which are common to the body as a whole—essentially those relating to touch, along with pleasure and pain—and those which are exclusive to particular organs, such as the tongue, the nose, the ears, and the eyes (65b4–c1; cf. 64a2–5). Like Democritus, all of his accounts are based on the magnitudes and geometric structure of physical objects, together with the causal powers they have as a consequence, except that Plato appeals to the geometrical properties of elemental particles rather than molecular structures. He seems to take perceptible qualities, moreover, to be the effect (pathêma) of these geometric properties on our body, rather than the properties themselves. Among tactile qualities, for example, the searing quality of heat is due to the sharpness of the fire atoms responsible for it and the way they cut and penetrate our body (61d5–62a5); things feel soft when their particles yield and give way, because they have a smaller width, and hard when they are more 34  Day (1997) offers an extended and detailed critique of this assumption, though the position is anticipated in basic outline by Fine (1988). The following, though, can perhaps be admitted. Because the ‘secret doctrine’ of the Theaetetus is embedded in a discussion of Heracliteanism, the exact nature of the theory is something of a moving target (appropriately enough). The extreme form it ultimately takes in the Theaetetus (181d9–182a1), therefore, and which makes it vulnerable to refutation, may not imperil the more moderate version that is first introduced — a view, in fact, that Plato never expressly rejects. 35 Socrates twice concedes that the theory might be more defensible if restricted to perceptible properties, especially those with which one is presently interacting (171d9–e3, 179c2–7).

40   Victor Caston rectangular and resistant (62b7–8); other tactile qualities are handled in a similar way.36 In bringing about perception, most qualities either consolidate or disperse the elements composing bodies (65a3–5); and because the same mechanism is involved, the quality spaces of the different sense modalities share certain correspondences, despite producing different phenomenal experiences (67d6–e4). In the case of flavours, these changes are due mostly to roughness and smoothness, and whether these characteristics melt or cut the tongue, produce tiny blisters, or moisten and assuage it (65c1–66c7).37 The sense of smell constitutes an interesting exception. Since none of the elemental shapes is inherently suited to the nostrils, on his view odour does not belong to any of the elements on their own, but only compound bodies undergoing a change of state, to liquid or gas (66d1–67a6).38 Sound is air striking the brain and blood, via the ears, which is transmitted to the soul, and the change caused by this is hearing. It is high-pitched when the blow occurs quickly, and low when it occurs slowly (67a7–b6).39 Plato devotes the most space to vision. There is a special kind of fire inside the eye that flows out through the pupil in a ‘visual stream’ (to tês opseôs rheuma) to meet an emanation from the object, which together form a single continuous, material link to the object and transmit changes originating from the object back to the subject’s soul, where seeing occurs (45b–d; cf. 58c5–7). Depending on the size of the particles emanating from an object, different colours are seen. Those which are bigger than the ones in the visual stream consolidate the stream and are dark; those which are smaller fragment or disperse it and are light; those which are equal in size cannot be seen (67d2–e6). When fire particles penetrate into the eyes’ channels and meet the fire inside, a dazzling effect occurs (67e6–68b1). Various hues are explained as mixtures of the brilliance that produces this together with white and then subsequently with other colours, although Plato insists that it is impossible for mortals to give a precise account of these matters (68d2–7; cf. b6–8).40 Plato’s views on perceptual content are harder to elicit. But many recent commentators have taken one of the central arguments in the Theaetetus to have profound consequences for his conception of it, at least at one stage. As has been mentioned, Socrates argues that we do not perceive qualities like white and black or high and low with our sense organs (184b7–c8). Instead, we perceive through the senses: the effects perceptible objects have on them are transmitted through the body to a single ‘form, soul, or whatever it ought to be called’.41 If they did not converge on some single thing, the senses would be like the different soldiers inside the Trojan Horse (184d1–2), rather than parts of a unified subject of experience. It would be impossible to differentiate or compare the different things we do perceive, since none of the senses is able to perceive the objects of another (184e8–185c8). Any features these things exhibit in common, moreover—such as being, number, sameness or difference, 36  See the analyses for cold (62a5–b6), hard (62b6–c3), rough (63e10), and smooth (63e10–64a1). Plato also discusses heavy and light, understood in terms of the heft of an object, at length: 62c3–e4. 37  Plato offers detailed explanations for why foods taste acrid (65d3), sour (65d4), bitter (65d4–e1), salty (65e1–4), spicy (65e4–66a2), acidic (66a2–b7), and sweet (66b7–c7). 38  There are only two types of odour, according to Plato, neither of which has a name, but one is pleasant, the other painful (67a1–6). 39  Plato also distinguishes between smooth and rough sounds (67b7) and loud and soft ones (67c1). 40 Plato offers explanations for red (68b3–5), golden (68b5–6), purple (68b8–c1), indigo (68c1–2), amber (68c3), gray (68c3–4), yellow (68c4), blue (68c5–6), azure (68c6–7), and green (68c7). For a general discussion of the theory of perception in the Timaeus, see Brisson (1997), including the appendix (167–76), which provides a running commentary on the passage on colours. 41  Tht. 184d3–5, 185a5, c7–8; cf. 185d8.

Perception in Ancient Greek Philosophy    41 similarity or dissimilarity, and being admirable or shameful, good and bad—are things which the soul is aware of on its own, not through any bodily organ (185c4–d3, 186a2–b1).42 One of these common features in particular is pivotal to Socrates’ argument. Being, he reiterates, is not something grasped through perception, but only by the soul on its own (186a2–c5; cf. 185d8–e2). But since one cannot attain truth without ‘attaining being’ (186c7–d1), it follows that perception cannot attain truth (d4–5, e4–5); and given that one cannot attain knowledge without truth, Socrates argues, knowledge cannot be found in perception itself, but only in our reasoning about perception (186d2–3, e4–7). What is it to ‘attain’ or ‘grasp’ being, though? If we follow the interpretations most prevalent today,43 what is at issue is the predicative or copulative use of the verb ‘to be’, that is, whether we can perceive that a given object is a certain sort of thing or perceive it as being some sort of thing— in short, whether perception has propositional content or indeed any content at all. If things do not seem to be a certain way in perception, then perceptions cannot even be true and so a fortiori cannot constitute knowledge. Only the soul’s judgements about perception will have content. The resulting view radically degrades perception. It is hard to see how it can involve awareness or even ‘registering’, as some of these interpretations claim, much less the transmission of information or ‘reports’, such as we find in the Republic. If touch does not enable us to feel something as hard or soft (cf. 186b2–4), for example, and so does not have the power to discriminate between different tactile qualities, it is not clear what the activity of the senses involves beyond a mere bodily affection. This might be a reason why Socrates goes on to speak about the soul becoming aware of the affections in the body which are transmitted to it (186c1–2). Another possibility, though, is that Socrates is not denying that perception in general has content, but only that it has content involving the ‘common’ features he focuses on, of which being is just one. His concern with being, moreover, is not a concern with predication and truth generally, but rather with what things really are—that is, with what their essence or nature is—along with other ‘common’ concerns, such as whether they are the same or different as other things, similar or dissimilar, good or bad, and so forth. In short, he is concerned with knowledge of deeper, underlying truths about things that go beyond mere appearance.44 This would not only fit dialectically with the rest of the dialogue, as part of a general stance against Protagoras and his sympathizers, but would connect this discussion with one of the main themes that runs throughout earlier Greek philosophy, namely, the search for the real 42  Several

of these topics—namely, the unity of consciousness, the ability to discriminate qualities from different modalities, and the ability to perceive features common to the different modalities—are developed in somewhat different ways by Aristotle: see section 4; see also Turnbull (1978). 43  Most current readings are influenced in one way or another by a trio of papers, originally written in the early 1970s: Burnyeat (1976), Frede (1987), and Cooper (1970). Burnyeat and Frede both take the hard line traced out here, while Cooper explores more moderate, hybrid positions. 44  This reading is especially natural if we read the gloss on ‘being’ (ousia) at 186b6, kai hoti eston, as ‘that is, what they are’ rather than ‘that is, the fact that they are’, as it is standardly rendered. (McDowell (1973) is an exception: for his defence of the former reading, see 111, cf. 191.) One need not think that some ‘rational intuition’ of Platonic Forms is intended here, as is sometimes claimed. All that is required is that the being (ousia) of things should be understood, as so often, in terms of what something really is, its nature or essence. Nothing further need be intimated here about the metaphysics involved, such as that they are Platonic Forms with certain distinctive characteristics. As for how we know them, Plato is quite explicit in this passage that it is not through some kind of rational ‘vision’, but rather by reasoning: making comparisons, drawing inferences, and testing (see also Rep. X, 602c4–603a9).

42   Victor Caston nature of things beneath the play of conflicting appearances. On this alternative reading, the senses do transmit information about the world and do have content. But they cannot grasp the deeper truth about things on their own, because they cannot compare and sift through our experiences, to reach an underlying explanation. To accomplish this, we must work over the information that comes through the senses with the reasoning powers of the soul. If something like this is right, then this passage is not so far off views we have already come across in the Republic or even the Phaedo. There is reason, moreover, to think that in the later dialogues perception does have content of some kind. In the Philebus, Plato discusses how we can be puzzled over what appears to us, for example, about whether something near a rock under a tree is a human or not (38c12–d10). This sort of reflective questioning and weighing up of alternatives is how Plato elsewhere characterizes thinking more generally, as a ‘silent dialogue’ the soul has with itself, uttering its thoughts in a kind of silent speech—a language of thought, if you will—which finally issues in a belief, if the soul succeeds in arriving at a conclusion.45 In the Philebus, he playfully compares the soul to a book and suggests that there is a scribe inside of us writing down what appears to us to be the case. But he elucidates this metaphor by saying that it is our perceptions, memories, and experiences which ‘inscribe’ these words in our souls (hoion graphein hêmôn en tais psukhais logous, 38e12–39a7). Perception is not a passive, inarticulate affection, about which we form interpretations separately in thought. On the contrary, perception articulates a view about how things are, which we may or may not accept, based on how it agrees or disagrees with our other perceptions, memories, and experiences. The resulting discourse in the soul is not pure and abstract either. He suggests that there is also a painter in our soul who illustrates it, providing images from sight or other senses which can in some sense be seen within ourselves (39b3–c1). In the Sophist, belief similarly does not arise just from the soul’s silent thought on its own, but from perception as well, in which case it is called ‘representation’ (phantasia, 264a4–6). But he emphasizes that all of these states are ‘akin to language’ (tôi logôi sungenôn) and so capable of falsehood (264b1–4). There is no strict division of labour here with regard to content, where perception is viewed as entirely non-cognitive and interpreted separately by thought—all of these states have articulable content. The contrast that interests Plato is rather between how things immediately appear to us and how they appear on reflection, through the cooperation of all of our faculties, especially rational ones. Plato has a few, mostly critical things to say about perceiving that we perceive, made in passing, since it serves mainly as a foil to self-knowledge, which is his primary concern. In Alcibiades I, Socrates suggests that our eyes could follow the Delphic inscription to ‘Know Thyself’ by looking in a mirror at ourselves, in particular our pupils, where seeing is said to occur (132d5–133b5). But we cannot achieve self-knowledge in this way, since the self, considered on its own, cannot be identified with the body or with the composite of body and soul, but only with the soul alone (129b1–131a3). In the Charmides, Plato considers reflexive mental attitudes more generally to see whether it is even coherent to speak about selfknowledge. Using sight and hearing as his prime examples, he says that we could not see that we see or hear that we hear, unless seeing or hearing were themselves visible and audible and so possessed colour or sound respectively (168d3–169a7; cf. 167c8–d10). None of this, however, precludes the possibility that we might perceive that we see or hear, without 45 

Tht. 189e4–190a6; Soph. 263e12–264b1.

Perception in Ancient Greek Philosophy    43 literally seeing or hearing these activities, a suggestion Aristotle would make in due course when taking up very similar objections.46

4 Aristotle In the opening chapter of his treatise On the Soul, Aristotle considers whether any psychological state—in which he explicitly includes ‘perceiving generally’ (holôs aisthanesthai)— occurs exclusively (idion) in the soul. He argues that none does. Rather, they are all ‘shared’ (koinon) with the body, which in each case does or undergoes something at the same time (1.1, 403a3–7, a16–19).47 An adequate definition of perception must therefore refer not only to the kinds of formal characteristics the analytic philosopher (ho dialektikos) focuses on, but also the underlying material structures and changes the natural philosopher (ho phusikos) studies (403a29–b9). Thus, while Aristotle distinguishes between what it is to be a sense—that is, the power to perceive—and what it is to be a sense organ, he nonetheless regards a sense and the corresponding sense organ as ‘one and the same’ (2.12, 424a24– 8). Like the soul and the body, they are one in the way that matter and form in general are one (2.1, 412b6–9; cf. 412b17–413a3). As with other psychological states, Aristotle distinguishes between the power (dunamis) to perceive and its exercise or activity (energeia). Having this power, whether it is being exercised or not, is what distinguishes animals from plants and is the basis for other related powers that they share, such as the capacity for feelings of pleasure and pain, desire, representation, and dreams.48 Perceiving is the exercise of this power: it is not a modification or alteration away from this power, in the way that acquiring or losing the ability would be, but rather the realization of its own nature (2.5, 417b2–16).49 Perceiving is a change brought about by the object of perception itself: the object has the active power to affect the sense organ and by acting on it brings it into activity. As with other agent–patient interactions, the activity of the perceptible object and the activity of the sense are not two parallel events, but a single event, taking place in the organ acted upon. They are ‘one and 46  DA 3.2, 425b17–25; Somn. 2, 455a15–22; see section 4. For discussion of the Charmides passage and Aristotle’s response to it, see Caston, 2002: 768–73, esp. 772–3 and 79. 47  The only possible exception Aristotle considers is understanding (noein), and even here he insists that if understanding requires representation (phantasia)—something he later acknowledges, at least for humans—it will be inseparable from the body and its activities (1.1, 403a7–16). For understanding’s dependence on representation, see DA 3.7, 431a16–17; 3.8, 432a8–10; Mem. 1, 449b31. This will not be true of God, who is nothing but understanding (Metaph. 12.7 & 9), or true of the second understanding mentioned in DA 3.5. But it is arguable that the latter just is God: for a defence of this reading, see Caston (1999). 48  What distinguishes animals from plants: e.g., DA 2.2, 413b1–4; Sens. 1, 436b10–12; GA 1.23, 731a30– b4, though see Lloyd, 1996: ch. 3 on possible exceptions to this rule, like the jellyfish, which Aristotle regards as an animal but without perception (PA 4.5, 681a19–20). On the relation to pleasure, desire, and representation to perception: DA 2.2, 413b22–3; 2.3, 414b4–5; 3.11, 434a2–3. 49 Some have thought that this distinction precludes any other modification or alteration being involved in perception, in particular underlying material changes (e.g., Burnyeat, 1995: 19, 21–2). But Aristotle believes that perceptible objects as such produce a material change in the organ when they produce perception, as bright colours do on the moisture in the eye (GA 5.1, 779b26–780a7; cf. DA 1.1, 403a16–19), even though perceiving is not to be identified with the material changes underlying it: each is a distinct type of change. For more detailed discussion, see Caston, 2005: esp. 265–9 and 288–90.

44   Victor Caston the same’, even though their ‘being’ is different (to einai heteron), since what it is to perceive differs from what it is to be perceived (3.2, 425b26–426a19; cf. Phys. 3.3). What can be perceived—colours, sounds, flavours, odours, heat, and moisture, for example—continue to exist apart from perception, because they exist even when they are not exercising their power to affect perceivers. They are objective features of the world, independent of being perceived. Aristotle criticizes his predecessors for failing to realize this, when they assumed that the objects of perception cannot exist unperceived (426a20–6).50 Because perceiving depends on the objects of perception to bring it about and the latter are external to the perceiver, it follows that perception is not something that is ‘up to us’, but depends upon what is furnished by our environment.51 What can perceive and what can be perceived are thus relative (pros ti) to one another (Categ. 7, 6b3, b35–6), where the latter causally acts on the former and is therefore prior in nature.52 Each sense is a power to perceive a certain type of object and its essence is defined by reference to it. The object is thus the sort of thing that, as such, is intrinsically (kath’ hauto) capable of causing that sense exclusively (idion) to perceive it (DA 2.6, 418a24–5). The power of sight, for example, is defined in terms of what can be seen. This turns out to be colour—not by definition, Aristotle is quick to add, but because colour possesses within itself what is responsible for being seen (2.7, 418a29–b2). To be visible and to be a colour are thus not the same (Phys. 3.1, 201b2–4; Metaph. 11.9, 1065b32–3), even if colours are in fact the sorts of things that can be seen.53 Aristotle offers extensive accounts of the different senses,54 the perceptible qualities they are directed at,55 and the various organs that subserve them.56 As with any other agent–patient interaction for Aristotle, the object of perception causes the sense to become like itself. The active quality or form of the agent—in this case, the 50  One naturally thinks of Protagoras here, although there is no explicit mention of him in the text. But Aristotle’s criticism is certainly relevant to the theory of perception Plato offers in support of Protagoras in the Theaetetus, as part of his ‘secret doctrine’ (see section 3): an object is only white while it is being perceived as white by someone. Also note that on this theory, the causal interaction of subject and object always generates ‘twins’—whiteness and seeing, for example—in contrast with Aristotle’s view, where the resulting activity of the power to perceive and the activity of the perceptible object are ‘one and the same’ (though differing ‘in being’). For close comparison of these two texts, see Turnbull (1978). 51  DA 2.5, 417b19–25; cf. Sens. 2, 438b22–3; 6, 445b7–8; Insomn. 2, 459a24–5. 52  Metaph. 4.5, 1010b30–1011a2; cf. Categ. 7, 7b35–8a12; 8, 9a28–b9, esp. b5–7. On relatives which are causally prior (or prior ‘in nature’), see Categ. 12, 14b11–13. 53  Colour is not the only thing that is visible either. Phosphorescent objects—for which, Aristotle notes, there is no Greek word—are visible in the dark, although this is not their proper colour. The proper colour of objects is only manifest in the light (418a27–8, 419a1–7). 54  The senses: sight (DA 2.7, Sens. 2; cf. GA 5.1); hearing (DA 2.8; cf. GA 5.2); smell (DA 2.9); taste (DA 2.10); and touch (DA 2.11). There are interesting discussions in the last chapter about whether touch is a single sense, given the different range of qualities it is sensitive to (422b17–33); and whether flesh is the organ of touch or rather a medium, as the other senses have, the organ being deeper within (422b34–423b26). 55  Perceptible qualities: colour (Sens. 3); flavours (Sens. 4); odours (Sens. 5); sounds (DA 2.8; cf. Sens. 4). For discussion of tactile qualities, see DA 2.11, 423b27–424a15; see also GC 2.2 and PA 2.2, 648b11–649b8, esp. 648b14–17 (cf. b30–3). On light, darkness, and phosphorescence, see DA 2.7. For analogies between the quality spaces of odours and flavours, see also Sens. 5, 443b3–16 and DA 2.9, 421a26–b3. 56  For sense organs in general, see PA 2.1, 647a3–33; see also assorted comments on individual sense organs in HA 4.8. For discussions of specific sense organs: eyes (PA 2.13; GA 5.1; cf. Sens. 2); ears (PA 2.11–12; GA 5.2); nostrils (PA 2.16); tongue (PA 2.17); and flesh (PA 2.8, 653b19–33; cf. DA 2.11, 422b34–423b26). For a comprehensive examination of Aristotle’s discussions of the sense organs and possible implications for his theory of perception, see Johansen (1998).

Perception in Ancient Greek Philosophy    45 perceptible quality itself—is transmitted to the patient, which ‘receives’ it or takes it on. Thus, having been unlike the object initially, the sense organ becomes similar to the object (DA 2.5, 417a17–20, 418a3–6; 2.11, 424a1–2). But not every patient perceives the agent acting on it. Plants, for example, are made warm by the sun, but they do not feel warmth, even though according to Aristotle they have a soul and warmth is a perceptible quality (2.12, 424a32–b1). Plants are unable to perceive because they lack a ‘mean’ or balance (mestotês) along a range of sensible qualities that would make them sensitive to the various qualities along that range. Consequently, they are modified by a sensible quality like warmth ‘along with the matter’ and transformed as a result (424b1–3). Each sense, in contrast, is capable of receiving the perceptible form of the object ‘without the matter’, something Aristotle compares to the way sealing wax receives the insignia from a signet ring, without the ring’s iron or gold (424a17–24). Just how to understand this contrast is quite controversial. Some, like Aquinas, take it to mark off a special kind of reception, where the form is received ‘immaterially’—or, as he also says, spiritually or intentionally—which consists in nothing more than perception’s coming to be directed at the object. Against this, others think that the perceptible quality is literally instantiated in the sense organ (cf. 3.2, 425b22–4), but without the matter of the object in which the form was originally instantiated. It seems unlikely that literal instantiation is at issue, though, since that would no longer offer a contrast with plants, as Aristotle intends, given that plants also become warm without receiving matter from the source of the heat. On two occasions, moreover, Aristotle actually ridicules the view that cognition requires a literal replica of the object within ourselves (1.5, 410a7–11; 3.8, 431b28–9). On the other hand, it seems unlikely that receiving form ‘without the matter’ could merely consist in its being directed at an object, without any further underlying material change, given the comparison to a signet ring’s producing a seal in wax. A third alternative lies between these extremes, however. Perception might be directed at an object in virtue of material changes in the sense organ without the object’s perceptible form being literally embodied. To receive the form ‘without the matter’ would instead be to embody only certain essential features of the form in a different type of material, without producing a replica. For example, according to Aristotle each colour is defined by a ratio of black to white (Sens. 3, 440b14–26). On the present suggestion, the eye would receive a colour ‘without the matter’ by embodying the same numerical ratio in a different pair of opposites—hot and cold, say, or runny and viscous—in the vitreous jelly, which Aristotle believes is the sensitive part of the eye, without the jelly needing to change colour at all. In effect, the senses would act as transducers: they would preserve certain essential features of the perceptible form in a new medium and in so doing transmit information about the character of the objects in the world acting on our senses.57 What is perceptible according to Aristotle, at least in the basic or fundamental case, are (1) the qualities that are intrinsically (kath’ hauto) perceptible to a single sense exclusively (idia), or ‘exclusive’ perceptibles, for short: colours, for example, are intrinsically perceptible exclusively to sight, tones to hearing, odours to smell, flavours to taste, and temperature and moisture to touch (DA 2.6, 418a11–15, a24–5). Since on Aristotle’s theory a perception is about what brings it about, the resulting perceptions of these qualities cannot be mistaken, 57  For a full defence of this position, see my ‘Receiving Form without the Matter:  Aristotle on the Transmission of Information’ (unpublished).

46   Victor Caston although he adds we can be mistaken about the coloured object, with regard to what it is, for example, or where.58 As these remarks suggest, Aristotle does not think that exclusive perceptibles are the only thing we can perceive, even if they are what is fundamentally perceptible (kuriôs aisthêta, 418a24). There are also features of objects that are (2) intrinsically perceptible to more than one sense, the so-called shared or ‘common’ (koina) perceptibles, such as change and rest, number and unity, shape, extension, and duration.59 Finally, there are (3) ‘extrinsic’ (kata sumbebêkos) perceptibles, features that are extrinsic to features that are intrinsically perceptible to a given sense.60 This last category includes features that are intrinsically perceptible to a different sense, but it extends much more widely, to include perceiving the son of Diares and even a universal like colour.61 It seems that many of the features relevant for human action and animal behaviour would fall in this last category, such as when a dog and a lion smell or hear their prey (EN 3.10, 1118a16–23). With the last two kinds of perceptible, error is not only possible, but frequent.62 Aristotle makes clear, moreover, that in such cases the perceptions themselves are false, as distinct from representations generated from them (DA 3.3, 428b25–30), not to mention the judgements subsequently formed on their basis. Given Aristotle’s physics and the range of things he thinks can be perceived, the aetiology of perception in these three cases must be very different. This is especially clear with the last kind, where Aristotle explicitly says that our senses are not affected by the extrinsic characteristics in question as such (DA 2.6, 418a23–4). But Aristotle still regards them all as genuine forms of perception, even if he regards one, the perception of exclusive qualities, as the fundamental case (2.6, 418a24–5).63 Whenever we mis-see or mis-hear, Aristotle says, we nonetheless see or hear something real; it is just not what we take it to be (Insomn. 1, 458b31–3).64 This suggests that in all perception, what we perceive are the individuals acting on us, but we always perceive them as being certain sorts of things, a point on which we can be mistaken. Thus, we not only perceive an individual, like Callias; our perception is of a certain sort of thing, ‘of a human, Callias’ (An. Post. 2.19, 100a16–b1; cf. 1.31, 87b28–30). That is why perception is a discriminative power (kritikê, 99b35): it allows us to distinguish between different types of things in our environment and thereby contributes 58 

DA 2.6, 418a12, a15–16; 3.3, 427b12, 428b21–2; 3.6, 430b29–30; Sens. 4, 442b8–10; Metaph. 4.5, 1010b2–3. But compare DA 3.3 428b18–19, where Aristotle seems to qualify this claim:  the perception of such perceptibles is ‘true, or possesses the least possible amount of falsehood’. For a possible explanation of this apparent exception, see Caston, 1998: 272 n. 56. 59  DA 2.6, 418a17–20 (cf. a10–11); 3.1, 425a14–20; Sens. 1, 437a8–9; 4, 442b4–10; Mem. 450a9–12. Apart from number, this list of common objects is quite different, we should note, from the ones Plato speaks of at Tht. 185c–186b; see section 3. 60  DA 2.6, 418a20–4; 3.1, 425a21–27, a30–b3; cf. An. Pr. 1.27, 43a33–5. 61  For objects intrinsically perceptible to another sense: DA 3.1, 425a30–1; cf. a21–4. Other examples: DA 2.6, 418a21 (Diares’ son); 3.1, 425a24–7 (Cleon’s son); Metaph. 13.10, 1087a19–20 (the universal colour). 62  DA 3.1, 425b3; 3.3, 428b19–25. Aristotle notes in the second passage that error is most frequent where common perceptibles are involved, although without further explanation. 63 Sometimes this sentence is taken to imply that only the perception of exclusive perceptibles is perception ‘strictly speaking’ (a mistaken construal of the Greek kuriôs), while the other two kinds are called ‘perceptions’ by an extended use of the term. But in general Aristotle does not tend to legislate that one use of a term is correct and the others incorrect or metaphorical; rather, he standardly acknowledges different uses of terms and notes where one is basic or fundamental or primary (kuriôs). Exclusive perceptibles on this reading, then, are not the only genuine perceptibles, but rather the basic or fundamental ones. 64  In contrast with dreams and hallucinations, where we do not perceive anything at all (458b33–459a1).

Perception in Ancient Greek Philosophy    47 to our survival and well-being (Sens. 1, 436b19–437a15).65 But then perceptual content must somehow involve universals as well, even in cases where these are not grasped conceptually. In fact, on Aristotle’s view it must be possible to grasp these contents non-conceptually, since our most basic concepts first arise from earlier perceptual experiences, which are said to ‘implant’ the universal in us (An. Post. 2.19, 100b5), while other animals, though completely without concepts, are still capable of perceptual discrimination.66 Although Aristotle does occasionally speak of the after-effects of sensory stimulation, like after-images (Insomn. 2, 459b7–13), as something perceptible (460b2–3), in general he speaks about phenomenal qualities as belonging to the objects themselves. He regards colours, tones, flavours, odours, warmth, and moisture as public, objective features of the world, external to us, that cause us to perceive them, rather than as properties of our own experiences (Sens. 6, 446b17–26).67 Yet he also seems to think that whenever we perceive, we perceive that we perceive (EN 9.9, 1170a29–b1; Sens. 7, 448a26–30). In one passage, he argues that we accomplish this by means of a ‘common power’ of perception, shared by all the senses, by means of which we are also aware and discriminate the perceptibles of different senses (Somn. 2, 455a15–22).68 But at the beginning of On the Soul 3.2, he offers a more extended discussion and arguments (425b12–25). The interpretation of this passage is controversial, but on the most common reading he argues that this higher-order perception cannot be the function of a distinct power of perception, on pain of infinite regress. On an alternative reading (favoured by Brentano), the argument concerns the activity of perception rather than the power (the Greek aisthêsis being ambiguous between the two). The argument would then be that perceiving that we see or hear cannot be a distinct activity from the original act of seeing or hearing, on pain of an infinite regress. Every act of perception, in addition to being directed at an external object, must also be reflexively directed at itself ‘on the side’ (en parergôi, Metaph. 12.9, 1074b25–26).69

5 Conclusion From even this preliminary survey, it should be clear that an interest in perception specifically, as distinct from other forms of cognition, develops relatively early in Greek philosophy. The Presocratics are preoccupied with getting past the manifold appearances the world takes on in experience to achieve a deeper understanding of the nature of things. Their self-conscious appeal to reason, as a way of sorting out the puzzling and conflicting 65  On Aristotle’s use of krinein for perceptual discrimination, rather than judgement (as it is sometimes translated), see the landmark article by Ebert (1983). 66  Aristotle would thus subscribe to a ‘state’ view of non-conceptual content, rather than a ‘content’ view. For a full exploration and defence of these issues, see my ‘Aristotle on Perceptual Content’ (unpublished). 67 It is worth noting in this context that Aristotle emphatically rejects Democritus’ reduction of perceptible qualities to quantitative features like shape and extension (Sens. 4, 442a29–b26), a criticism that would apply equally to Plato’s Timaeus. 68  For the latter function, see DA 3.2, 426b8–427a16; cf. Somn. 2, 455a12–22; Sens. 6, 448b17–449a20, which constitutes a striking departure from Plato, who argued this function could not be performed by any of the senses, but only by something distinct from all of them (see section 3). For Aristotle’s conception of the ‘common sense’, see Gregoric (2007). 69  For close discussion of these passages and a defence of Brentano’s activity reading, see Caston 2002.

48   Victor Caston elements in our experience, leads naturally to a broad contrast between the deliverances of the senses and what we make of them through reflection and argument. It is not surprising that by the mid-fifth century bce there is increasing attention to the causal mechanisms underlying the different sense modalities and to the features of objects responsible for stimulating our sense organs: for example, the confluence of similar material elements inside and outside the subject in Empedocles; the production of impressions by the object of vision in both Gorgias and Democritus; and the latter’s appeal to geometric and structural properties of an object quite generally to explain the qualitative character of the resulting perceptual experience. Identifying these causes is a natural first step towards determining what precisely the senses can tell us about the world—or cannot, as the case may be. Both the epistemological pessimism we find in Democritus and Protagoras’ optimistic turning of the tables in favour of relativism (if Plato is right) are equally due to reflections on perception as a form of causal interaction. Epistemological concerns are still evident in Plato. But the nature of perception and of perceptible qualities are already coming to be of interest in their own right: what exactly perception’s relation is to the body or to causation more generally; how we should understand the awareness we have of objects and indeed the awareness of perceiving itself; and finally the ontological status of perceptible qualities and their role in explaining the character of perceptual experience. The specific answers that Plato and Aristotle devised had much influence in the subsequent tradition. But it is the framing of the questions that would have enduring value.

References Avotins, Ivars (1980). ‘Alexander of Aphrodisias on vision in the Atomists’. The Classical Quarterly, 30, 429–454. Baltussen, Han (2000). Theophrastus against the Presocratics and Plato: Peripatetic Dialectic in the De sensibus. (= Philosophia Antiqua, vol. 86.) Leiden: Brill. Beare, J. I. (1906). Greek Theories of Elementary Cognition from Alcmaeon to Aristotle. Oxford: Clarendon Press. Brisson, Luc (1997). ‘Plato’s theory of sense perception in the Timaeus: How it works and what it means’. With commentary by Dana Miller. Proceedings of the Boston Area Colloquium in Ancient Philosophy, 13, 147–185. Burnyeat, M. F. (1976). ‘Plato on the grammar of perceiving’. The Classical Quarterly n.s., 26, 29–51. Burnyeat, M. F. (1995). ‘Is an Aristotelian philosophy of mind still credible?’ (A draft). In Martha C. Nussbaum and Amélie Oksenberg Rorty (eds), Essays on Aristotle’s De anima (pp. 15–26). Oxford: Oxford University Press. First paperback edition, including an additional essay by M. F. Burnyeat. Oxford: Clarendon Press, 1992. Caston, Victor (1986). ‘Empedocles’ Theory of Vision’. MA thesis, The University of Texas at Austin. Caston, Victor (1996). ‘Why Aristotle needs imagination’. Phronesis, 41, 20–55. Caston, Victor (1998). ‘Aristotle and the problem of intentionality’. Philosophy and Phenom­ enological Research, 58, 249–298. Caston, Victor (1999). ‘Aristotle’s two intellects: A modest proposal’. Phronesis, 44, 199–227.

Perception in Ancient Greek Philosophy    49 Caston, Victor (2002). ‘Aristotle on consciousness’. Mind, 111, 751–815. Caston, Victor (2005). ‘The spirit and the letter: Aristotle on perception’. In Ricardo Salles (ed.), Metaphysics, Soul, and Ethics: Themes from the Work of Richard Sorabji (pp. 245–320). Oxford: Oxford University Press. Caston, Victor (Unpublished). ‘Aristotle on perceptual content’. Caston, Victor (Unpublished). ‘Receiving form without the matter: Aristotle on the transmission of information’. Cooper, John M (1970). ‘Plato on sense-perception and knowledge (Theaetetus 184–186)’. Phronesis, 15, 123–146. Day, Jane M. (1997). ‘The theory of perception in Plato’s Theaetetus 152–183’. Oxford Studies in Ancient Philosophy, 15, 51–80. Ebert, Theodor (1983). ‘Aristotle on what is done in perceiving’. Zeitschrift für philosophische Forschung, 37, 181–198. Fine, Gail (1988). ‘Plato on perception: A reply to Professor Turnbull, Becoming and Intelligibility’. Oxford Studies in Ancient Philosophy, suppl. vol., 15–28. Frede, Michael (1987). ‘Observations on perception in Plato’s later Dialogues’. In Frede, Essays in Ancient Philosophy (pp. 3–8). Minneapolis: University of Minnesota Press. Fritz, Kurt von (1953). ‘Democritus’ theory of vision’. In E. Ashworth Underwood (ed.), Science, Medicine, and History: Essays on the Evolution of Scientific Thought and Medical Practice, written in honour of Charles Singer, vol. 1 (pp. 83–99). London: Oxford University Press. Ganson, Todd Stuart (2005). ‘The Platonic approach to sense-perception’. History of Philosophy Quarterly, 22, 1–15. Gregoric, Pavel (2007). Aristotle on the Common Sense. Oxford: Oxford University Press. Hussey, Edward (1990). ‘The beginnings of epistemology: From Homer to Philolaus’. In Stephen Everson (ed.), Epistemology (pp. 11–38). (= Companions to Ancient Thought, vol. 1.) Cambridge: Cambridge University Press. Ierodiakonou, Katerina (2005). ‘Empedocles on colour and colour vision’. Oxford Studies in Ancient Philosophy, 29, 1–37. Johansen, T. K. (1998). Aristotle on the Sense-Organs. Cambridge: Cambridge University Press. Kahn, Charles H. (1985). ‘Democritus and the origins of moral psychology’. American Journal of Philology, 106, 1–31. Laks, André (1999). ‘Soul, sensation, and thought’. In A. A. Long (ed.), The Cambridge Com­ panion to Early Greek Philosophy (pp. 250–270). Cambridge: Cambridge University Press. Lesher, J. H. (1994). ‘The emergence of philosophical interest in cognition’. Oxford Studies in Ancient Philosophy, 12, 1–34. Lloyd, G. E. R. (1996). Aristotelian Explorations. Cambridge: Cambridge University Press. McDowell, John (1973). Plato, Theaetetus. Translated with Notes. Oxford: Clarendon Press. Rudolph, Kelli (2011). ‘Democritus’ perspectival theory of vision’. The Journal of Hellenic Studies, 131, 67–83. Rudolph, Kelli (2012). ‘Democritus’ opthamology’. The Classical Quarterly, 62, 496–501. Schirren, Thomas (1998). ‘Aisthesis vor Platon: Eine semantisch-systematische Untersuchung zum Problem der Wahrnehmung’. (= Beiträge zur Altertumskunde, vol. 117.) Stuttgart: B. G. Teubner. Sedley, D. N. (1992). ‘Empedocles’ theory of vision and Theophrastus’ De sensibus’. In William W. Fortenbaugh and Dimitri Gutas (eds), Theophrastus: His Psychological, Doxographical,

50   Victor Caston and Scientific Writings (pp. 20–31). (= Rutgers University Studies in Classical Humanities, vol. 5.) New Brunswick, NJ: Transaction Publishers. Snell, Bruno (1960). The Discovery of the Mind: The Greek Origins of European Thought. Translated by T. G. Rosenmeyer. New York: Harper. (Translated from Die Entdeckung des Geistes, 2nd edn, Hamburg, 1948.) Solmsen, Friedrich (1968). ‘αἴσθησις in Aristotelian and Epicurean thought’. Mededelingen der Koninklijke Nederlandse Akademie van Wetenschappen, AFD Letterkunde, Nieuwe Reeks, 24 (1961), 241–262. Reprinted in his Kleine Schriften, 3 vols, vol. 1 (pp. 612–633). (= Collectanea, vol. 4.) Hildesheim: Georg Olms Verlagsbuchhandlung. Stratton, George Malcolm (1917). Theophrastus and the Greek Physiological Psychology before Aristotle. London: George Allen & Unwin Ltd. Turnbull, Robert G. (1978). ‘The role of the special sensibles in the perception theories of Plato and Aristotle’. In Peter K. Machamer and Robert G. Turnbull (eds), Studies in Perception: Interrelations in the History of Philosophy and Science (pp. 3–26). Columbus, OH: Ohio State University Press. Warren, James (2007). ‘Anaxagoras on perception, pleasure, and pain’. Oxford Studies in Ancient Philosophy, 33, 19–54.

Chapter 2

Perception i n M edieva l Phil osoph y Dominik Perler

1 Introduction: Three problems of perception All medieval philosophers in the Aristotelian tradition agreed that perception provides the foundation for our knowledge of material things: had we no sensory access to them, we would know neither that they exist nor how they exist. Moreover, many medieval authors claimed that perception is also indispensable for our knowledge of non-material things. We would never acquire knowledge of mathematical objects if we did not see instances of, say, triangles and if we were not able to use this perception as our starting point for a process of abstraction. Even our knowledge of God would be impossible if we were utterly unable to perceive the world of creatures—an activity that motivates our reasoning about the first cause of this world.1 Given this tendency to find a starting point for all kinds of knowledge, it is hardly surprising that perception played a crucial role in scholastic debates, especially in the period after 1255 when Aristotle’s works in natural philosophy (among them De anima and De sensu et sensato, which explicitly deal with perception) became part of the university curriculum.2 These debates closely examined the origin of perception, its structure, and its function in epistemic processes. They took place in the context of a faculty psychology, i.e. of a theory that appealed both to sensory and intellectual faculties in order to explain the origin and the content of perception. When dealing with single cases of perception, philosophers in the Latin West focused on three key problems. The first problem concerned the object of perception. What exactly is this object: a material thing, a property of a thing (e.g. colour or shape), the mere appearance of a thing, or some other entity? This question immediately gave rise to a second problem: how are we able to perceive these objects, whatever they are? What cognitive faculties 1 

Thomas Aquinas went so far as to provide an a posteriori proof of God’s existence; see Summa theologiae (= STh) I, q. 2. art. 3, corp. To be sure, not all medieval authors chose this approach. Some, most famously Anselm of Canterbury, attempted to give an a priori proof that does not appeal to perceptual evidence. 2  In 1255, Aristotle’s works in natural philosophy were officially accepted at the University of Paris. On the reception of Aristotle’s writings, see Pasnau 2010, 793–797; on the presence of De anima, see Perler 2008.

52   Dominik Perler and devices are required? And how do all the faculties work together to bring about a perceptual state? These questions inevitably led to a third problem that concerned the reliability of perceptual processes:  why can we be certain that our faculties and cognitive mechanisms enable us to perceive an object as it really is? What entitles us to claim that sense perception provides a robust and trustworthy foundation for our knowledge of the material world? Since this type of knowledge was considered the starting point for all kinds of knowledge, even for our knowledge of mathematical objects and God, it is not surprising that problems of perception opened the door to far-reaching debates in epistemology. At stake in these debates was not only what we know on the basis of perception, but also whether we know anything at all on this basis. This chapter examines the way scholastic authors dealt with the three problems of perception. It goes without saying that it can shed light only on some aspects of a complex debate. It is limited to a discussion of Latin philosophers in the late thirteenth and early fourteenth centuries.3 This is a rather short but highly productive period in medieval philosophy, because it was at this time that various traditional theories became fully available: Aristotelian accounts of perception, Neo-Platonic theories of light, Arabic theories of vision, and ancient sceptical debates that questioned the reliability of sense perception. It should also be noted that this chapter only discusses theories of external perception, leaving aside those dealing with internal perception (sensory self-awareness and inner self-perception).4 Finally, this chapter is confined to philosophical approaches to perception and does not enter into an analysis of scientific debates, for instance in optics and physiology; nor does it examine theological controversies.5

2 The object of perception Inspired by Aristotle, most late medieval philosophers claimed that our acts of perception have two immediate objects: ‘proper sensibles’ and ‘common sensibles’.6 Proper sensibles are properties of material things, perceived by one and only one of the five external senses. Thus, colour is perceived by sight, sound by hearing, and odour by smell. By contrast, common sensibles are properties perceived by two or more senses. For example, the shape or the size of a material thing can be seen as well as touched. Medieval Aristotelians had no doubt that both proper and common sensibles are real properties, present in material things, and not just properties that somehow arise in the perceiving person. They even tried to give an ontological classification of these properties by assigning them to the category of qualities and by emphasizing that they exist in (technically speaking, inhere in) material substances.7 3 For a broader overview that covers both Arabic and Latin discussions, see Knuuttila and Kärkkäinen 2008. 4  On internal perception, see Putallaz 1991, Heller-Roazen 2007. 5  On scientific aspects, see Lindberg 1996; on theological debates, see Denery 2005. 6 See De anima II.6 (418a 11–19). 7 See, for instance, William Ockham, Exp. in libr. Praedicamentorum, cap.  14 (OPh II, 276–285); John Buridan, Quaestiones in Praedicamenta, q. 16 (ed. Schneider 1983, 121). To be sure, not all medieval authors agreed that these qualities necessarily exist in a substance. Inspired by theological debates about

Perception in Medieval Philosophy    53 This account of the immediate objects of perception poses a problem. How can it be that we perceive things and not just loose bundles of properties if all that we immediately grasp are proper and common sensibles? For instance, why do I see a red, round thing and not just the properties redness and roundness when I am looking at an apple? To answer this question, Thomas Aquinas (and after him many other medieval authors) appealed to the inner senses that work on the material provided by the external senses. As soon as sight and touch apprehend redness and roundness, they transmit this information to the common sense, which is located in the brain. Aquinas held that this sense is ‘a certain faculty at which the affections of all the senses terminate’.8 That is, all the properties apprehended by the five external senses come together in this internal sense. Their connection and unification enables the faculty of imagination (phantasia), another internal sense, to come up with a ‘phantasm’, i.e. a sensory image that presents a unified whole. So, I am able to see a red, round thing and not just a loose series of properties because I have a sensory image of something unified. No predication is necessary for this kind of unification. Various properties are merely seen as perfectly cohering, without being conceived as qualities that, technically speaking, inhere in a thing. And the image of all the unified properties is not something I deliberately or even consciously produce. It arises spontaneously as soon as an input stemming from the external senses arrives in the internal senses, and it can be stored and reactivated at a later moment when the apple is no longer present. But why do I see the red, round thing as an apple? Or more generally, why am I able to see something as a thing of a certain type? To answer this question, Aquinas invoked the distinction between ‘sensibles per se’ and ‘sensibles per accidens’.9 Properties like colour and shape are ‘sensibles per se’ that are directly grasped by the external senses and transmitted to the internal ones, without there being any need for an additional faculty that interprets or conceptualizes the sensory input. By contrast, being an apple is a property that is grasped along with the directly apprehended properties as soon as they are interpreted in a certain way. What makes this interpretation possible? The cogitative power, Aquinas affirmed: a special faculty that mediates between senses and intellect.10 It uses the concepts formed by the intellect and applies them to the information provided by the senses. Aquinas called it ‘particular reason’ because it is not the faculty of reason in general, which deals exclusively with concepts and their inferential relations, but a faculty that is concerned with concepts insofar as they are applied to perceived properties. Aquinas’ appeal to this special faculty is significant because it shows that he did not conceive of perception as a purely sensory process. Reason is always involved when we perceive something as such and such. That is why there is a clear difference between brute animals and human beings. Animals also have a common sense and the faculty of imagination. Therefore, they are also able to produce sensory images that present unified things. But they lack the faculty of reason and hence also the use of particular reason that would enable them to apply concepts to what is present in a perceptual situation. Consequently, transubstantiation, some assumed that they can be separated, and others even argued that they can exist on their own; see Pasnau 2011, 179–220. 8 

10 

Sentencia libri De Anima (= SDA) II.13, 119. SDA II.13, 121–122, and STh I, q. 78, art. 4, corp.

9 

SDA II.13, 120.

54   Dominik Perler animals are unable to interpret what they see.11 Thus, a cat sitting in front of an apple is able to produce an image of a coloured thing. But it is utterly unable to see it as an apple because it lacks the concept that would enable it to categorize what it sees. Only human beings, endowed with reason, can go beyond the stage of merely having sensory images: we see unified things and we see them as things of a certain type. Aquinas and many other thirteenth-century authors took it to be unproblematic that the appropriate use of senses and reason enables us to perceive things in the material world. Since proper and common sensibles are properties of real substances, we are always in touch with apples, trees, and many other material things, and we more or less adequately perceive how they are. But can we be certain that these things are in fact our objects of perception? In the early fourteenth century, Peter Aureol raised this question and argued that all we immediately perceive are things with ‘apparent’ or ‘intentional being’ (esse appar­ ens or esse intentionale), which do not exist in nature but are produced by our intellect.12 Aureol adduced cases of sensory illusion to illustrate this fact. When we are travelling on a boat and see moving trees on the shore, we perceive, strictly speaking, only apparent trees and not material ones. Why? The apparent trees clearly have a property, namely being in movement, which the material trees on the shore lack. Similarly, when we see a stick partly submerged in water, we immediately perceive an apparent stick that is distinct from the real one. To be sure, Aureol did not claim that we are, as it were, imprisoned in a world of apparent things and that we never have access to material things. In a normal situation, the apparent and the real thing coincide.13 So, if I see a stick taken out of the water, present in the best possible light, I see it the way it is, namely as an unbent piece of wood. But even then it is, strictly speaking, only the apparent stick I immediately perceive. Whether or not the apparent and the material thing coincide is not something I can tell at first sight. I need to evaluate the perceptual conditions and will be certain about a coincidence or convergence of the two types of things only after having excluded the possibility of a deception. It is clear that this account amounts to a radical redefinition of the objects of perception: nothing but apparent things are our immediate objects, and we can only make reasonable guesses about the relation between these entities and material things. However, Aureol’s position was not widely accepted. His immediate successors, among them Walter Chatton and William Ockham, decidedly rejected it, emphasizing the traditional thesis that material things and their properties are our immediate objects.14 According to Chatton, there is a devastating fallacy in Aureol’s reasoning. Aureol had started by describing an object as a relational thing, i.e. as a thing that is related to a perceiving person, but then turned it into an absolute thing, i.e. a thing that has its own existence. In the logical literature, this was called a ‘relational fallacy’.15 Chatton illustrated it with a 11  Aquinas conceded that animals have an ‘estimative faculty’ that enables them to evaluate things. His example is the sheep that takes the wolf to be dangerous (STh I, q. 78, art. 4). But this is simply a natural reaction and not a conceptual form of evaluation. There is, as it were, a mechanism built into the sheep that inevitably triggers a negative reaction when a certain sensory image is present. 12  Scriptum I.3, sect. 14 (ed. Buytaert 1956, 696–697). 13  Scriptum I.3, sect. 14 (ed. Buytaert 1956, 698). 14 Ockham, Ordinatio I.27, q.  3 (OTh IV, 238–258); Chatton, Reportatio et Lectura, prol., q.  2, art. 2 (ed. Wey 1989, 87–91). On further authors, see Tachau 1988. 15  It was already mentioned by Aristotle in Soph. El. 5 (166b 36–167a 14) and often discussed in medieval logic handbooks, for instance in Peter of Spain’s Tractatus VII, n. 120 (ed. de Rijk 1972, 157–158).

Perception in Medieval Philosophy    55 classical example. If someone were to say ‘Homer exists in an opinion; therefore this person, characterized with a number of specific features, exists as such’, he would first talk about Homer insofar as he is thought about and described by readers of Greek literature, and then falsely conclude that Homer-in-the-mind is a distinct entity. But there is no such entity. All that exists is the real Homer, a person of flesh and blood, and all characterizations apply to this entity. Sensory illusions ought to be explained on the same line. To say that one sees moving trees or bent sticks does not mean that a number of special entities exist. This only amounts to saying that some material things are present under a certain aspect. Consequently, all judgements and descriptions apply to material things only. Thus, ‘There are moving trees’ simply means ‘There are material trees that are present as being in movement’. This special presence is due to the circumstances under which the person is looking at the trees. Should she leave the boat and look at them on the shore, the very same trees would be present as being motionless. This explanatory strategy shows that Chatton argued for the rejection of apparent things by pointing out that one and the same object can be present in different ways and hence can also be described in different ways. Or, technically speaking, there can be different ‘extrinsic denominations’ of the same thing. But types of denominations should not be conflated with types of things. Accepting apparent things in addition to material ones would have devastating consequences. For if we assumed that acts of seeing are only directed at apparent things, we could no longer affirm that we are perceptually related to things in the world. Consequently, we could no longer claim that we acquire knowledge of these things. To avoid this consequence, Ockham and Chatton endorsed the thesis that we directly perceive material things in the world, not a number of inner ‘doppelganger’.16 That is why we do not need to make dubious inferences from internal to external objects. Nor do we need to explain how internal, merely apparent objects can signify or represent external ones. Finally, we do not need to provide special evidence for our certainty about the existence of external things. The mere fact that we directly see them, no matter how many false or misleading descriptions we give, shows that we are in immediate touch with them.

3 The perceptual process The Aristotelian thesis that properties such as colour, size, and shape are the first and most basic objects of perception gives rise to the question of how they can become objects of our perception. What physical, physiological, or psychological processes are required? Since philosophers in the Latin West were strongly influenced by Arabic optics, they mostly discussed this question with respect to vision.17 Their crucial problem was: how do colours

16 Ockham, Ordinatio I.27, q. 3 (OTh IV, 241); Chatton, Reportatio et Lectura, prol., q. 2, art. 2 (ed. Wey 1989, 88). 17  Clear examples are Roger Bacon, De multiplicatione specierum (in Lindberg 1983), and John Pecham, Perspectiva communis (in Lindberg 1970). On the Arabic influence, see Tachau 1988, 3–26, and Denery 2005, 82–100. Although priority was given to visual perception, other sense modalities were not completely neglected; on hearing, see Pasnau 2000.

56   Dominik Perler become objects of visual perception? In their answer to this question they referred to a complex causal process that involves at least three steps. In a first step, colours existing in material things are transmitted through a medium, namely through light or air, to the sense organ. Following al-Kindi, many scholastic philosophers claimed that special entities, so-called ‘species in medio’, are required for this transmission. These are physical entities, propagated through the air, that somehow transport the forms of colours to the eyes and affect them. In a second step, the eyes undergo a change and thereby receive the forms. As a consequence, in a third step additional entities, so-called ‘sensible species’, are produced in the sense organ. As soon as they are combined with species stemming from other sense organs, the perceiver will be able to come up with a complex sensory image of the external thing and see that thing. The analysis of all three steps provoked controversies. In philosophical contexts, steps two and three were closely examined.18 First of all, what kind of change does the sense organ undergo? Thomas Aquinas, following his teacher Albert the Great, gave a clear answer: it is a ‘spiritual change’ that should not be conflated with a natural one.19 In a spiritual change it is just the form and not the matter of the colour that is received in the sense organ. That is why the eyes simply assimilate the form but do not literally become coloured. Thus, a person who is affected by a red apple simply receives the form of redness, but her eyes do not turn red. By contrast, in a natural change the form is received together with matter. Should one paint someone’s eyes with red colour, the eyes would receive the entire form-matter compound and literally become red. For Aquinas, it is of crucial importance that the eyes undergo a mere spiritual change, which makes vision the highest among the five senses. Other senses partly undergo a spiritual change, partly also a natural one. For instance, someone tasting or touching water receives the form of wetness but also literally becomes wet. Aquinas’ opposition of spiritual and natural change may give rise to the impression that he referred to an entirely immaterial process that makes the act of seeing possible. But this impression is misleading, because even the spiritual change takes place in the eyes, which clearly are a material sense organ. A purely immaterial process can only take place in a purely immaterial being, for instance in an angel. Whatever takes place in a material being is bound to material conditions. So, a change that occurs in human eyes is bound to the conditions of these special sense organs, which differ from the conditions to be found in the eyes of a dog or a cat. Aquinas’ point is not that the spiritual change somehow transcends all material conditions and that one could ignore the organ in which it takes place. He rather stressed that there can be two kinds of change inside a material organ: a process that consists in the mere reception of information, and a process that consists in a transformation of the organ itself. A modern comparison may help to clarify this point. A radio playing music receives information from the radio station. The radio clearly is a material thing, and it receives the information through a material medium, namely acoustic waves. It is materially affected when it receives these waves, but it is not materially transformed by them; it does not change its size or shape. It only ‘takes in’ music or speech programmes, and given an inner mechanism, it will broadcast these programmes. Similarly, the eyes are materially affected by light rays that transport the form of a colour, but they are not 18 

Step one gave rise to debates in optics. For an overview, see Smith 1981.

19 See STh I, q. 78, art. 3, SDA I.10, 50, and II.24, 169.

Perception in Medieval Philosophy    57 materially transformed by them; they do not change their own colour. They simply ‘take in’ information about a certain colour, and given their inner constitution, they will make this information present so that a person will be able to see that colour. It would be different if the eyes were painted with the colour. In that case they would be materially transformed because they would receive the entire form-matter compound and thereby be affected in their basic structure—like a radio that would not just receive the speech programmes but also incorporate the air emitted by the speakers.20 But in which way does the information become present once it has been received? An answer to this question amounts to an explanation of step three in the causal process, namely the production of ‘sensible species’. These are special devices that transport forms through the external senses and make them accessible to the inner senses. Were there no species, forms present in external things would never become available and never be apprehended. But what exactly is apprehended: the forms transmitted by the species or the species themselves? At first sight, the answer seems clear. Since species are mere devices in the perceptual process, they are only that by which something is apprehended (the medium quo), not that which is apprehended (the terminus ad quem). And in fact, in an influential passage Aquinas unequivocally held that a species is something ‘in virtue of which sight sees’, not that which is seen.21 It therefore seems clear that species are not apprehended, at least not in a normal situation. Only when we reflect upon the way we make external things perceptually available, we focus on the species and make them our cognitive objects. But even then, they will not be seen or otherwise perceived, but intellectually cognized. Yet, Aquinas’ position is not as clear as it might seem. In a number of passages he remarked that the species are in fact immediately apprehended in a basic act of perception, without there being any reflection. Thus, he spoke about ‘the first thing seen, which is the species of the visible thing existing in the pupil’, and even claimed that there is ‘an apprehensive power that apprehends the sensible species when the sensible thing is present’.22 These passages seem to speak in favour of the thesis that species are the terminus ad quem: one first and foremost apprehends inner species and only secondarily forms that are somehow conveyed by the species. It even seems as if Aquinas committed himself to a ‘veil of species’ theory, similar to the ‘veil of ideas’ theory that was to become prominent in early modern debates. For he did not only claim that the species are the first things we see but also affirmed that they are ‘likenesses’ (similitudines) of external properties.23 This seems to imply that a person who is affected by a red apple apprehends a mere likeness of redness, some kind of inner copy or simulacrum, and that she can only make inferences about the external colour. The real colour seems to be hidden behind the ‘veil of species’. Should Aquinas be committed to this inner-veil theory, he would not only openly contradict his own official position about the first and immediate objects of perception but also pave the way for sceptical attacks. Why are we entitled to claim that we perceive colours and many other properties of external things if all we have immediate access to are our inner ‘sensible species’? Why can we be so sure that there are external properties 20 

For a detailed analysis, see Perler 2002, 42–59, and for a slightly different account Burnyeat 2001. STh I, q. 85, art. 2; similarly, Summa contra Gentiles (=ScG) II.75, n. 1550, 218. 22  In I Sent., dist. 35, q. 1, art. 2, and Quaestiones disputate De Veritate (= QDV) q. 1, art. 11, corp., 35. These passages are thoroughly discussed by Pasnau 1997, 200–208. 23 See STh I, q. 17, art. 2, and q. 85, art. 2. 21 

58   Dominik Perler at all? All we can be certain about are our inner likenesses that are supposed to correspond to or signify external properties. But perhaps there are no external properties. Perhaps we only assume that our inner likenesses relate us to something external. Since we have no immediate access to the properties themselves, we can never verify their existence. Aquinas never worried about these problems. Why not? He did not think that internal likenesses are set apart from external properties. Consequently, he did not assume that these likenesses constitute some kind of veil that prevents us from having access to the external properties. According to his technical terminology, every item that shares the form with something else can be its likeness.24 Thus, a red apple can be the likeness of another red apple because they share the accidental form of redness, or a human being can be the likeness of another human being because they share the essential form of humanity. Likewise, a sensible species can be a likeness of an external colour because they share the sensible form.25 And when one apprehends the species, one grasps the form that is also present in the material thing—no inference is required. ‘Even when something is seen through the likeness of another thing’, Aquinas remarked, ‘it can still happen that someone seeing the thing through the medium considers the thing immediately, without its cognition being turned toward anything else’.26 Why is the cognition not turned to the mere likeness, i.e. to the sensible species? Because even though one apprehends the species, one sees in it and through it the form that also exists in the external thing. The species is a mere medium that makes the form available. (Compare: if you listen to the radio and hear a voice, you can very well say that you immediately hear it. You do not go through inferential steps. The voice is immediately accessible through the medium.) Given the crucial claim that sensible forms can be present both in material things and in perceivers who produce sensible species, Aquinas did not doubt that we have direct, noninferential access to colours, sounds, etc. However, not all medieval authors were so optimistic. Peter John Olivi, who studied in Paris shortly after Aquinas’ death, and William Crathorn, who taught in Oxford around 1330, clearly saw that the species theory can easily be turned into some kind of inner-veil theory.27 One simply needs to give up the thesis about form sharing and assume that the species inside and the sensible property outside the perceiver are two distinct items that have nothing in common. How then can we be certain that we really see the colour and not the species? Crathorn phrased his critique as follows: ‘Someone seeing something white sees at once and without any difference whiteness itself and the species of whiteness. On the basis of the mere fact of seeing, one cannot distinguish between whiteness and the species of whiteness.’28 Imagine a person walking 24 

STh I, q. 4, art. 3. Note that the relation between two red apples or two human beings is symmetrical, whereas that between species and external property is asymmetrical: the species makes the form which it shares with the external colour present, not the other way around. That is why Aquinas adds that we are dealing with a special form of similitudo: the species is ‘the principle leading to a cognition’ of the form it shares with the external property; see QDV, q. 8, art. 11, ad. 3. 26  In IV Sent., dist. 49, q. 2, art. 7, ad. 8 (quoted by Pasnau 1997, 206). 27 Olivi, Quaestiones, q. 74 (ed. Jansen 1926, vol. 3, 122–123); Crathorn, Quästionen, q. 1 (ed. Hoffmann 1988, 102–126). Both are discussed by Pasnau 1997, 236–247, and Perler 2002, 109–127. 28  Quästionen, q. 1 (ed. Hoffmann 1988, 123). 25 

Perception in Medieval Philosophy    59 down a street covered with fresh snow. Why would she be entitled to claim that she sees the colour of the snow? Given that she has a sensible species of whiteness and that this species is also apprehended, even primarily, she could as well say that she merely sees her own species. Of course, she may come up with some causal reasoning, arguing that the species could not have come into existence if it had not been caused by the external colour. But then she would only have inferential knowledge of the colour. And the causal reasoning could always be challenged, because one could always invoke a deviant cause. Could it not be that the omnipotent God caused the species?29 And could it therefore not be that someone had a species of whiteness without there being any external colour? Since this possibility can never be ruled out, a person having a species can never be sure that she is really in touch with an external property. To avoid this absurdity, Crathorn’s teacher Ockham had already called for an ontologically parsimonious explanation. In his view, all one needs to admit is an external thing endowed with a set of properties that affects the sense organs. Given certain natural laws, this change in the external senses immediately triggers the inner senses, thereby producing an act of perception that is immediately directed at the external thing.30 This explanation does not only dispense with sensible species, but with the entire idea of transmitting and receiving forms. To put it in a nutshell: an explanation invoking forms and formal causation was replaced with an explanation that only appealed to efficient causation. Ockham thus paved the way for a break with the traditional explanatory framework—a break that was to become prominent in the early modern period when the Aristotelian idea of assimilating forms was finally rejected. All debates about the perceptual process focused on the cause and the object of an act of perception. But what about the phenomenal experience? Doesn’t seeing something red or tasting something sweet stimulate feeling in a certain way? And does this feeling not call for an explanation? Given the framework of hylomorphism, no medieval author saw a need to introduce special entities (e.g. phenomenal qualities).31 Phenomenal experience is constituted entirely by the formal and material change which a perceiver undergoes. Thus, when I am tasting a piece of sugar, my tongue receives the form of sweetness and is also materially affected by a mixture of moist and dry. This triggers an act of experiencing something sweet—nothing more happens. Of course, this parsimonious explanation presupposes that a hylomorphic change is a very special type of change: we do not only get information about an object when we are affected by it, but we also have some kind of awareness of this information and therefore an immediate experience. While insisting on the immediacy and the non-intellectual character of this experience (brute animals have it as well as human beings), some medieval authors emphasized that special sensible properties are responsible for it. Peter Aureol is a telling example. He pointed out that there is a clear difference between seeing and thinking, even if one deals with one and the same object. In an act of seeing, one cognizes an object ‘under conditions

29 

Crathorn explicitly referred to this possibility in Quästionen, q. 1 (ed. Hoffmann 1988, 124). Reportatio III, q. 2–3 (OTh VI, 64–65 and 114–129). Ockham conceded that an object can even act at a distance, without being in immediate touch with a sense organ, and trigger the sensory faculties; see Reportatio II, q. 12–13 (OTh V, 309). 31  It would therefore be misleading to look for a solution to the famous ‘qualia’ problem in medieval texts. This simply was not a medieval problem; see King 2007. 30 

60   Dominik Perler of quantity’ and therefore has an experience that is lacking in an act of thinking.32 Suppose that you are seeing a red apple in front of you. Why does this seeing feel differently from merely thinking about it? Because you see the apple at a certain distance, at a certain angle, in a certain light, etc. That is, you do not simply grasp redness, but a red thing in a specific situation. Redness is, as it were, enriched by many other properties. Given this richness, seeing the red apple right here in this light feels a certain way. This will be lost in an act of thinking that focuses just on redness and abstracts from all other properties. Aureol’s attempt to explain this crucial difference shows that he was well aware of the phenomenal aspect of perceiving. But he did not invoke special inner qualities to account for it. He rather referred to something external, namely to the complex bundle of sensible properties that are present in a given situation. Of course, these properties are only apparent ones. For Aureol, the immediate objects of perception can never be material things or properties, as has been explained in section two. Therefore, he cautiously remarked that ‘things appear under conditions of quantity’ and that it is this appearance that marks the difference between perceiving and thinking.33 But in a normal situation, apparent things correspond to material ones and even ‘coincide’ with them.34 For instance, I would not perceive something red, present at a certain distance and in a certain light, if I had not been affected by a material thing that is in fact red and in fact present at a certain distance and in a certain light. That is why it is the presence of a rich bundle of external properties that gives rise to the presence of apparent properties and hence also to a perceptual experience.

4 The reliability of perception Every account of the perceptual process, no matter how many physiological or psychological sub-explanations it includes, must address a fundamental problem. Can we ever be sure that it is a reliable process that yields veridical perception? Given their empiricist tendencies, medieval philosophers took this problem seriously. For clearly, only a perceptual process that is in principle reliable provides the foundation for a solid body of knowledge. Should perception constantly mislead us and provide distorted or impoverished information about the material world, our knowledge would stand on shaky grounds. It is therefore hardly surprising that scholastic authors did not only spell out all the steps involved in a perceptual process, but also attempted to explain why we can and even should trust this process. First of all, why should we be confident that we perceive colours, sounds, and other ‘proper sensibles’ the way they really are? Aquinas unmistakably answered: ‘It is proper to sight that it cognizes colour, proper to hearing that it cognizes sound, and proper to taste that it cognizes flavor’.35 Simple and harmless as this statement may seem, it should not be dismissed as a mere reference to the scope of each sense. It is part of an all-embracing 32 

Scriptum I.35, part 1, art. 1 (ed. Friedman 2009, 8). also marks the difference between imagining and thinking; see Scriptum I.35, part 1, art. 1 (ed. Friedman 2009, 8). 34  Scriptum I.3, sect. 14 (ed. Buytaert 1956, 698). 35  SDA II.13, 118. 33  It

Perception in Medieval Philosophy    61 teleological theory.36 For Aquinas, each sense is a capacity that has its natural goal, and if it is appropriately actualized, it unfailingly reaches this goal. Thus, sight is designed to be actualized by the presence of a colour and to bring about an act of seeing that colour. No agent can arbitrarily alter or delete this natural goal. Even God, who could do everything by means of his omnipotence, does not intervene and alter the goal at this or that moment, because doing so would amount to changing the natural order. But it is precisely this order that God chose when creating the world. In fixing a new goal he would thwart his own creation. Given this teleological assumption, Aquinas did not only claim that each sense has its own object, but also that it apprehends this object correctly, at least under normal circumstances. ‘With regards to the proper sensibles’, he affirmed, ‘sense does not have a false cognition, unless this happens by accident as it is seldom the case’.37 The qualifying clause is significant. A sense is not some kind of cognitive automaton that brings about a correct perception in each and every situation. Exceptions are possible. That is why a number of conditions need to be taken into account, among them internal ones (the sensory apparatus needs to be in a healthy state) and external ones (the sensible properties need to be directly present).38 But if the conditions are met, each sense yields the perception it is designed to yield—its veridical character is not to be questioned. Since it only rarely happens that the relevant conditions are not fulfilled, one should not constantly worry whether or not one’s perceptions are correct. Given this teleological picture, it is not surprising that Aquinas and many other medieval authors did not enter into discussions about the famous criterion problem that had been prominent among ancient sceptics.39 How can we ever be certain that our perceptions are correct, the sceptics had asked, if we never possess a criterion that would enable us to check their correctness? All we can do is compile and compare a number of perceptions, but we can never test them from a neutral point of view. That is why it could very well be that some or even most of them are incorrect. For Aquinas, this problem did not arise because he did not see the necessity to test each and every perception. Since our senses are designed to work correctly, the perceptions they bring about can in principle be trusted. Using modern terminology, one could say that someone seeing colours or hearing sounds is in a default position. She does not need to justify the correctness of her perceptions unless there are special circumstances that might impair the functioning of her senses. But what about common sensibles and sensibles per accidens, which are not objects designed to be apprehended by distinctive senses? Aquinas conceded that errors can occur.40 This is most evident in the case of sensibles per accidens. As has been explained in section two, the cogitative power is involved in the grasping of these objects as things of a certain type. Since it is always possible that this power does not apply the appropriate concept (e.g. it may use the concept of pear when being provided with the sensory 36 

On the metaphysical framework of this theory, see Schmid 2011, 35–105. STh I, q. 17, art. 2. 38 In QDV, q. 1, art. 11, 3, and STh I, q. 17, art. 2, Aquinas explicitly mentioned possible failings. A person may be sick so that her tongue interprets sweet things as bitter (a violation of the internal condition), or an object may be present behind a green glass so that its colour is taken to be greenish (a violation of the external condition). 39 They were familiar with this problem thanks to Cicero’s Academica and Augustine’s Contra Academicos. On the reception of ancient debates, see Perler 2006, 15–27, and Lagerlund 2010. 40 See STh I, q. 17, art. 2. 37 

62   Dominik Perler image of an apple), mistakes are possible. But these mistakes are not more serious than other cases of misinterpretation. Like all other natural faculties, the cogitative power is a well-functioning faculty that yields correct states when used under normal conditions. That is why we should not worry about occasional false interpretations that occur under abnormal conditions. We simply need to assess the special conditions under which we mistakenly take a thing, say an apple, to be a pear, and we then need to correct our false interpretation. And here, again, not each and every case is to be tested. In principle, there is a reliability built into the capacity that enables us to form concepts and to apply them to what is present to our senses.41 If one assesses Aquinas’ statements about the reliability of perception against the background of his teleological account of cognitive capacities, it is quite understandable why he did not see a threat in the sceptical challenges. He neutralized them by invoking a general framework that guarantees the appropriate functioning of senses and reason. Consequently, the scenario of global error was no option. However, this optimistic account gave rise to a number of objections. Let me mention two of them that were heatedly debated in the fourteenth century. The first problem concerns the claim that we do not only have reliable perception of single properties, but also of things having these properties. What makes Aquinas so confident that we are correctly able to see a red thing and not just redness? Lurking in the background is the ontological assumption, mentioned in section two, that properties are dependent entities that always exist in substances. So, when perceiving a particular instance of redness, we are also related to the substance in which this colour is present. But is it appropriate to say that we perceive this substance? This is precisely the question raised by Nicholas of Autrecourt, a philosopher who studied and taught in Paris between 1327 and 1340. Launching an attack on all Aristotelians, he claimed that ‘Aristotle never possessed evident knowledge about any substance other than his own soul—taking “substance” as a thing other than the objects of the five senses, and other than our formal experiences.’42 To be sure, Nicholas did not question the assumption that we reliably perceive sensible properties; he was no sceptic about the perceptual foundation of knowledge.43 His point was that we do not perceive substances. We rather infer their existence by arguing that each sensible property must inhere in a substance, a hidden but nevertheless really existing entity. But can we be certain about that? If sensible properties are considered to be real qualities that have their own existence, and if one assumes that they can be taken away from the substances in which they happen to be present (when explaining the sacrament of the altar, many scholastic authors made this assumption, arguing that colour and shape can be separated from the substance of bread), then it is not self-evident that perceiving a sensible property amounts to perceiving a thing having that very property. The alleged thing is merely postulated but not seen.44 41  Concepts concerning the essence of particular things are even unfailingly correct, as Aquinas claimed in STh I, q.  85, art. 6.  On this astonishing thesis, see Kretzmann 1991; on the metaphysical foundation of the reliability thesis, see Perler 2012. 42  Nicholas of Autrecourt, Second Letter to Bernard (ed. de Rijk 1994, 73). 43  He was even a foundationalist, as Grellard 2005 convincingly argued: knowledge always needs to have a firm foundation that consists in ‘formal experience’ (i.e. in the apprehension of logical principles such as the principle of non-contradiction) and in perceptual experience. 44  On fourteenth-century debates on this problem, see Robert 2006 and Pasnau 2011, 124–129.

Perception in Medieval Philosophy    63 There is still another problem that concerns Aquinas’ claim that the natural order is not changed, not even by God. Why should we assume that God never changes the given order? After all, our world is only one of many possible worlds God could create at any moment. And why should the possibility be excluded that he intervenes in this world? After all, miracles show that he does so from time to time. He could do it in such a way that we cannot tell any difference from a natural event. Thus, God could destroy a star in the heavens while maintaining in us the act of seeing that star. William Ockham explicitly mentioned this example, thereby making clear that deviation from the natural order can never be ruled out.45 Yet Ockham did not draw a sceptical conclusion. In his view, we are not deceived. Should the star be destroyed, we would correctly judge that it does not exist. But why should we come up with a correct judgement? Given that God is omnipotent, he could bring it about that there is not the slightest difference between the act of seeing we have when the star is present and the act we have after it was destroyed. Why should the first act give rise to a positive judgement of existence and the second to a negative one? A number of fourteenth-century authors were puzzled by this problem, one of them being Peter of Ailly. He acknowledged that there may be no phenomenal difference between perceptual acts that are caused by immediately present material things and those that are caused by God. Furthermore, he conceded that there is no special cognitive mechanism that makes us automatically produce correct judgements. This led him to the conclusion that we can never be entirely certain about our judgements based on perception. All we can have is ‘conditional evidence’ that should be carefully distinguished from ‘absolute certainty’.46 That is, all we can say is: if everything happens according to the natural order, without any divine intervention, then we have no reason to doubt that our perceptual judgements are correct. By contrast, when we make judgements about logical principles or about our own mental acts, we can be absolutely certain about their correctness. For instance, when we are in a state of fear we can be absolutely certain that the judgement ‘I am in fear’ is correct. No matter what caused this state, a natural event or a divine act, it is a real fact that I am afraid—no conditional clause needs to be added. Peter of Ailly’s opposition of absolute and conditional evidence shows that the distinction between internal and external perception became more and more important in late medieval debates. Only internal perception was considered to be infallible and entirely trustworthy, while external perception was exposed to doubt, despite the general reliability of the cognitive capacities. Why did this doubt arise? Most obviously because of the use of the theory of divine omnipotence as a sceptical scenario: if it is conceivable that the natural order can be suspended at every moment, there can be no absolute trust in our perceptions of material things. It is always possible that they have no natural cause. But there is also a deeper reason for a growing doubt about external perception. Aquinas had still assumed that the objects of perception are, strictly speaking, not material things but the sensible forms of these things, and he had taken for granted that these forms can be transmitted to and received in the perceiver. Peter of Ailly, inspired by William Ockham, no longer spoke about a transfer of forms, but only about a relation of efficient causation: material things affect the senses, thereby triggering cognitive capacities and bringing

45 

46 

Ordinatio, prol., q. 1 (OTh I, 39). Quaestiones I, q. 1, fol. dv. For a discussion, see Perler 2006, 188–191.

64   Dominik Perler about perceptual acts. But if this triggering is all that is required, and if there is no sensible form that needs to be both in the material thing and in the perceiver, then it is only a small step to the hypothesis that there could be all kinds of triggering causes—God or malicious demons as well as material things. To be sure, late medieval authors did not spell out this hypothesis. But in gradually giving up the traditional model of form sharing they paved the way for theories of perception motivated by scepticism.47

References Aquinas, Thomas (1856). In quatuor libros Sententiarum. Parma: Fiaccadori. Aquinas, Thomas (1952). Summa theologiae, (ed.) Petrus Caramello. Rome and Turin: Marietti. Aquinas, Thomas (1961). Summa contra Gentiles, (ed.) Ceslaus Pera. Turin and Rome: Marietti. Aquinas, Thomas (1970). Quaestiones disputatae De veritate, (ed.) Leonina XXII. Rome: S. Sabina. Aquinas, Thomas (1984). Sentencia libri De anima, (ed.) Leonina XLV/1. Rome and Paris: Commissio Leonina and Vrin. Aristotle (1958). Topica et Sophistici elenchi, (ed.) W. D. Ross. Oxford: Clarendon Press. Aristotle (1961). De anima, (ed.) W. D. Ross. Oxford: Clarendon Press. Buridan, John (1983). Quaestiones in Praedicamenta, (ed.) Johannes Schneider. Munich: Verlag der Bayerischen Akademie der Wissenschaften. Burnyeat, Myles F. (2001). ’Aquinas on “Spiritual Change” in Perception’. In Dominik Perler (ed.), Ancient and Medieval Theories of Intentionality. Leiden: Brill, 129–153. Chatton, Walter (1989). Reportatio et Lectura super Sententias, (ed.) Joseph C. Wey. Toronto: Pontifical Institute of Mediaeval Studies. Crathorn, William (?) (1988), Quästionen zum ersten Sentenzenbuch, (ed.) Fritz Hoffmann. Münster: Aschendorff. Denery, Dallas G. (2005). Seeing and Being Seen in the Later Medieval World: Optics, Theology and Religious Life. Cambridge: Cambridge University Press. Grellard, Christophe (2005). Croire et savoir. Les principes de la connaissance selon Nicolas d’Autrécourt. Paris: Vrin. Heller-Roazen, Daniel (2007). The Inner Touch:  Archaeology of Sensation. New  York: Zone Books. King, Peter (2007). ’Why Isn’t the Mind-Body Problem Medieval?’ In Henrik Lagerlund (ed.), Forming the Mind: Essays on the Internal Senses and the Mind/Body Problem from Avicenna to the Medical Enlightenment. Dordrecht: Springer, 187–205. Knuuttila, Simo and Kärkkäinen, Pekka (eds) (2008). Theories of Perception in Medieval and Early Modern Philosophy. Dordrecht: Springer. Kretzmann, Norman (1991). ‘Infallibility, Error, and Ignorance’. In Richard Bosley and Martin Tweedale (eds), Aristotle and His Medieval Intepreters. Canadian Journal of Philosophy, suppl. vol. 17,159–194. Lagerlund, Henrik, (ed.). (2010), Rethinking the History of Skepticism: The Missing Medieval Background. Leiden: Brill.

47  I am grateful to Martin Lenz, Mohan Matthen, Martin Pickavé, and Stephan Schmid for valuable comments on earlier versions of this chapter.

Perception in Medieval Philosophy    65 Lindberg, David (1970). John Pecham and the Science of Optics. Madison, WI: University of Wisconsin Press. Lindberg, David (1983). Roger Bacon’s Philosophy of Nature. Oxford: Clarendon Press. Lindberg, David (1996). Roger Bacon and the Origins of Perspectiva in the Middle Ages. Oxford: Clarendon Press. Nicholas of Autrecourt (1994). His Correspondence with Master Giles and Bernard of Arezzo, (ed.) Lambert M. de Rijk. Leiden: Brill. Ockham, William (1967–1986). Opera Theologica (= OTh), (ed.) Gedeon Gál et  al. St Bonaventure: Franciscan Institute. Ockham, William (1974–1988). Opera Philosophica (= OPh), (ed.) Gedeon Gál et  al. St Bonaventure: Franciscan Institute. Pasnau, Robert (1997). Theories of Cognition in the Later Middle Ages. Cambridge: Cambridge University Press. Pasnau, Robert (2000). ‘Sensible Qualities: The Case of Sound’. Journal of the History of Philosophy, 38, 27–40. Pasnau, Robert (ed.). (2010). The Cambridge History of Medieval Philosophy. Cambridge: Cambridge University Press. Pasnau, Robert (2011). Metaphysical Themes 1274–1671. Oxford: Clarendon Press. Perler, Dominik (2002). Theorien der Intentionalität im Mittelalter. Frankfurt am Main: Klostermann. Perler, Dominik (2006). Zweifel und Gewissheit. Skeptische Debatten im Mittelalter. Frankfurt am Main: Klostermann. Perler, Dominik (ed.) (2008). Transformations of the Soul: Aristotelian Psychology 1250–1650. Leiden: Brill (issue 46.3 of Vivarium). Perler, Dominik (2012). ‘Scepticism and Metaphysics’. In John Marenbon (ed.), The Oxford Handbook of Medieval Philosophy. Oxford: Oxford University Press, 547-565. Peter of Ailly (1490, reprint 1968). Quaestiones super libros Sententiarum. Frankfurt am Main: Minerva. Peter Aureol (1956). Scriptum super Primum Sententiarum, (ed.) M. Buytaert. St Bonaventure: Franciscan Institute. Peter Aureol (2009). Scriptum super primum Sententiarum, (ed.) R. L. Friedman et al. The Peter Auriol Homepage, . Peter John Olivi (1922–1926). Quaestiones in secundum librum Sententiarum, 3 vols., (ed.) Bernardus Jansen. Quaracchi: Collegium S. Bonaventura. Peter of Spain (1972). Tractatus called afterwards Summule Logicales, (ed.) Lambert M. de Rijk. Assen: Van Gorcum. Putallaz, François-Xavier (1991). La connaissance de soi au XIIIe siècle. Paris: Vrin. Robert, Aurélien (2006). ‘Jamais Aristote n’a eu de connaissance d’une substance: Nicolas d’Autrécourt en contexte’. In Stefano Caroti and Christoph Grellard (eds), Nicolas d’Autrécourt et la Faculté des Arts de Paris. Cesena: Stilgraf, 153–174. Schmid, Stephan (2011). Finalursachen in der frühen Neuzeit. Eine Untersuchung der Transformation teleologischer Erklärungen. Berlin and New York: W. de Gruyter. Smith, A. Mark (1981). ‘Getting the Big Picture in Perspectivist Optics’. Isis, 72, 568–589. Tachau, Katherine H. (1988). Vision and Certitude in the Age of Ockham: Optics, Epistemology and the Foundations of Semantics 1250–1345. Leiden: Brill.

Chapter 3

Sk epticism a n d Perception Baron Reed

The history of epistemology has been dominated by engagement with skepticism, particularly through developments in the two main flowerings of that tradition—first in the Hellenistic era and then in the Early Modern period. It is somewhat surprising, then, that skepticism often remains poorly understood. There are several reasons for this. First, much recent discussion of skepticism has been almost entirely anti-skeptical in nature. Second, anti-skeptical epistemologists often argue against the generic ‘skeptic’ without considering the views or arguments advanced by any particular skeptic. Third, anti-skeptical epistemologists often choose as their target a single skeptical argument as if an answer to it were all that needs to be given in responding to skepticism in general. Finally, many of the historical figures who were concerned with skepticism were systematic thinkers. Skepticism often played a role in their broader systems—one that is often missed by epistemologists, who usually have narrower, specialized interests. To proceed in these ways, however, is to miss much of what is most interesting in the skeptical tradition. In what follows, then, I shall try to highlight some of the differences among the major skeptical challenges to perceptual knowledge, while keeping in view their place within the broader philosophical contexts in which they occur. As we shall see, these arguments work by pointing to one or more potential sources of error: that experience can be misleading—either in any particular instance or in general—or that any particular belief can be mistaken, or that the concepts we use to make sense of experience may be misleading. Given the variety of ways in which these arguments work, it will become clear that no single anti-skeptical response can hope to be adequate.

1  Academic criticism of stoic epistemology Although our understanding of the history of skepticism is complicated by the lack of surviving material from early figures like Pyrrho, we do have a clear grasp of the earliest known skeptical arguments.1 Some of these can be found in Cicero’s Academica, which 1 See

Bett (2010) for state-of-the-art scholarship on ancient skepticism, including a thorough bibliography of recent work. See also Thorsrud (2009).

Skepticism and Perception    67 we have in two partial versions.2 In this dialogue, Cicero is recounting recent developments in the Academy in light of the two-centuries-long dispute between the Stoics and the Academics. Although the Academic skeptics claimed Socrates as the original skeptic, it was Arcesilaus (head of the Academy in the middle of the third century bce) who moved the school clearly in a skeptical direction. His arguments targeted the epistemology of the Stoics, whose school had recently been founded by Zeno of Citium. The fundamental element of Stoic epistemology is the cognitive or cataleptic impression: they took it to be the criterion of truth, and they said the sage—who assents only to cognitive impressions—can never go wrong. Zeno initially held that the cognitive impression is one that comes from what is, stamped and impressed exactly in accordance with what is. Arcesilaus then asked Zeno whether a true impression could be just like a false one. Zeno replied that no impression could be cognitive if there could be a false one just like one that comes from what is. Arcesilaus agreed that this would need to be added to the account of the cognitive impression and then set about arguing that no impressions could be cognitive because this last condition could not be satisfied.3 The Academics generally used two different arguments to motivate the line of attack Arcesilaus initiated. The first relies on the existence of pairs of indistinguishable objects, like twins or eggs.4 A rather dramatic example of this sort of cognitive failure occurs when a brood parasite, like a European cuckoo, lays its eggs in the nests of other birds, such as reed warblers.5 To the warbler, the cuckoo egg is indistinguishable from its own—even though, in both cases, the warbler is getting impressions of the eggs exactly in accordance with their size, shape, mottling, etc. Given that the cuckoo chick turns out to be a rather impolite guest, pushing the other eggs and chicks out of the nest, the warbler has every incentive (evolutionary and otherwise) to detect the intruding egg. Yet the cuckoo’s strategy is often quite successful, even though the warbler has ample opportunity to inspect the eggs in its nest. The cuckoo’s ability to deceive the reed warbler indicates that the warbler is unable to know its own egg, even when it is sitting on it. More generally, the fact that a false impression can be just like a true one shows that it isn’t enough for impressions to come from what is, exactly in accordance with what is.6 So long as it is possible for there to be false impressions that are indistinguishable from our best true impressions, none of our impressions can be cognitive. The Stoics replied to this line of argument by saying that nature doesn’t contain pairs of objects that are entirely indistinguishable. If no two objects are completely identical, 2  Cicero (2006/46–4 bce). See the introduction to this translation, by Charles Brittain, for a very helpful overview of Academic skepticism. 3  For this account of the dispute, see Cicero, 2006/46–4 bce: 45 (ii. 77). It is controversial whether Zeno meant to add an additional condition to his account of the cognitive impression or merely intended to clarify what he meant by his initial statement. For more on these interpretive options, see Reed (2002). 4  Cicero, 2006/46–4 bce: 32–35 (ii. 54–58) and 49–50 (ii. 84–86). 5  For more on brood parasitism, see Olivia Judson’s excellent article, ‘Cuckoo! Cuckoo!’, in the New York Times, 1 June 2010. 6  There is an interesting question here regarding the sense in which the warbler’s impression of the cuckoo egg is false. After all, in many respects, she sees it as it is; she is not deluded as to its size or shape or mottled colouring. Nevertheless, the bird apparently sees the egg as being a warbler egg (or as its own egg) and in that sense is mistaken. Some philosophers (e.g. Descartes) would be inclined to treat this as a mistake of judgement rather than as a flaw in the impression itself. Even so, Arcesilaus seems to be right in saying that the impression is inadequate as a basis for knowledge, given that it does not allow the warbler to distinguish between the cuckoo’s egg and its own.

68   Baron Reed then an impression that is ‘craftsmanlike’—i.e. one that reflects all of the object’s characteristics—will indeed be cognitive. But, even if one accepts the Stoics’ version of the Principle of the Identity of Indiscernibles, it is clear that their view places unreasonable demands on the capacity for cognition.7 Ptolemy Philopator, the ruler of Egypt, fooled Sphaerus (one of Zeno’s pupils) with a wax pomegranate; Sphaerus’s reply was that his impression of the pomegranate had not been cognitive but merely reasonable.8 That sort of retreat will be universally required, however, because it will always be possible for there to be two objects that differ only in ways that are imperceptible to humans. The second argument used by the Academics makes use of altered psychological states, like dreams and insanity.9 False impressions, after all, do not need to come from genuinely existing objects. Even if it is granted that no two objects will yield the same impressions, it can still be true that a false but indistinguishable impression can be found for every true impression. The Stoics insisted that the impressions we have in dreams, in drunkenness, and in madness are intrinsically different from those we have while in our right minds. But the Academics replied that all of these impressions are equally assented to, when we have them.10 What is noteworthy about this debate between the Stoics and the Academics is that it is the first time a systematic epistemology—one in which perception plays a central role—meets sustained skeptical criticism. In the end, the Stoics may have made it true by definition that some true impressions are such that there couldn’t be indistinguishable false ones. Chrysippus, the leader of the Stoics in the generation after Arcesilaus, seems to have moved in the direction of saying that the awareness made possible by a cognitive impression includes within itself the object that caused the impression.11 On this way of understanding perception, then, cognition is grounded in a direct awareness of the perceived object. The Stoics have avoided the letter of the Academics’ criticism: the awareness enabled by cognitive impressions is metaphysically different from the awareness given by false impressions because only the former includes an extra-mental object. But it is far from clear that the spirit of the revised view is satisfactory—to the subject, this difference is itself unavailable to consciousness. As we shall see, however, a variant of this sort of answer is one that will be given again by other anti-skeptics in the history of epistemology. 7  What is worse, this principle would be insufficient to solve the problem, even if it were true. Suppose the true and the false impression are discernible in some way; the subject might be able to tell them apart, but that does not mean she could identify which is true and which is false. (I am grateful to Mohan Matthen for this point.) 8  Reported by Diogenes Laertius; see Long and Sedley, 1987: 40F. 9  Cicero, 2006/46–4 bce: 29–32 (ii. 47–53) and 50–52 (ii. 88–90). 10  One might also point out, as the eleventh-century Muslim philosopher Al-Ghazali did, that, even if we reject our earlier dream experiences upon waking, ‘it is possible that a state will come upon you whose relation to your waking consciousness is analogous to the relation of the latter to dreaming’; quoted in Lagerlund, 2010: 11–12. In that case, we will reject our present waking experience, just as we did our dreams. 11  See Reed (2002). Part of the evidence for this claim is a passage from Aetius in which Chrysippus is reported to have provided different accounts for veridical and nonveridical perception, in which they make one aware of ontologically different sorts of objects; see Long and Sedley, 1987: 39B. This is the same sort of account offered by so-called disjunctivists like McDowell (1988).

Skepticism and Perception    69

2  The ten modes of pyrrhonism The Ten Modes were argument forms, perhaps developed by Aenesidemus in the first century bce. They survive in the work of Sextus Empiricus and of Philo of Alexandria.12 The Ten Modes point out ways in which the perceptual beliefs a perceiver has are relative—to species, to individuals, to circumstances, etc. In each of these cases, there will be a disagreement between perceivers who vary in the relevant way. For example, honey will taste sweet to me but bitter to people who have jaundice. In order to know how things really are, the disagreement must be adjudicated. But that judgement cannot come from any of the perceivers who are currently parties to the dispute; that would be question-begging. What is needed is an impartial judge. But any perceiver not currently involved in the dispute will add only another perceptual belief to the disagreement. Her judgement will be no better, and no more impartial, than those of the other perceivers. Hence, these kinds of disagreements cannot be resolved in a way that is not question-begging. There is a wealth of anecdotal information about perceptual relativity gathered in Sextus’s version of the Ten Modes. We hear, for example, about Demophon, Alexander the Great’s waiter, ‘who used to shiver when he was in the sun or the baths, and felt warm in the shade,’ and we learn that ‘the same wine appears sour to people who have just eaten dates or figs, but it seems to be sweet to people who have consumed nuts or chickpeas’.13 This material turned out to be the strongest grounds for the distinction made in the Early Modern period, by philosophers like Galileo, Descartes, and Locke, between (to use Locke’s terminology) primary and secondary qualities.14 The primary qualities—extension (or having spatial dimensions) and motion—are mathematically describable quantities, and our representations of them accurately reflect the qualities as they exist in objects. By contrast, the secondary qualities—such as colours, smells, and tastes—do not seem to admit of mathematical description, and there is nothing in the objects that matches our representations of these qualities. The wine that tastes sour to one person and sweet to another cannot be both. Hence, those divergent perceptual representations of it must depend in large part on the differing circumstances of those perceivers. The philosophers who borrowed from the Pyrrhonists the material on perceptual relativity to motivate the distinction between primary and secondary qualities did not intend to be make a skeptical argument. And yet there clearly are skeptical implications that follow from that distinction. If we grant to Galileo, Descartes, and Locke that material objects are characterized only by the primary qualities, then much of our perceptual experience is misleading. As Pierre Bayle put it, if these philosophers are right, then ‘I know that bodies are not at all as they appear to me’.15 He also recognized that the point can be pushed further: ‘if the objects of our senses appear colored, hot, cold, odoriferous, and yet they are 12  See the translation of Sextus Empiricus’s Outlines of Pyrrhonism in Mates, 1996:  94–110 (book I, section 14); Annas and Barnes (1985) provides both the relevant material from Sextus and from Philo, arranged by mode. 13  Annas and Barnes, 1985: 54 and 80. 14  Locke, 1979/1690: book II, chapter viii. 15  Bayle, 1965/1697: 197 (art. ‘Pyrrho’, rem. B).

70   Baron Reed not so, why can they not appear extended and shaped, in rest and in motion, though they are not so?’ 16 As Berkeley would later argue, we find the same sort of perceptual relativity in our perception of primary qualities as we do in our perception of the secondary qualities.17 So, they are equally doubtful. If the science of the Early Modern period seems to show that perception is unable to reveal the true nature of things, science of our own times has only made the problem starker. Most of the space around us is empty, contrary to how it appears. The world as described by physicists is extraordinarily different from the world as we perceive it.

3  The modes of agrippa Immediately after presenting the Ten Modes, Sextus mentions a set of five modes handed down by ‘more recent Skeptics’ and now usually attributed to Agrippa, about whom nothing else is known.18 Where the Ten Modes are based in an assortment of particular facts about different perceptions, Agrippa’s modes are general argument strategies meant to force beliefs of any sort—perceptual and otherwise—into a problematic structure of justification. Two of the modes, deriving from dispute and from relativity, allow the skeptic to raise a problem with the belief in question. Once the belief has been called into question, there are only three possible ways the believer can defend it; the final three modes come into play depending on which way the believer chooses. Call the target belief B. Suppose she offers another belief, C, in support of B. The skeptic can use one of the first two modes to call C into question. The believer might then defend C by mentioning another belief, D, which can itself be called into question. Either the believer will continue using new beliefs to defend those she has already mentioned, or she will eventually use one of the previously mentioned beliefs again. If the former, the skeptic will use the mode of infinite regress; if the latter, the skeptic will use the mode of circularity. In either case, the skeptic is essentially pointing out that the believer never succeeds in justifying the target belief B because justification never enters the chain of beliefs—it is endlessly deferred, either to an unending series of new beliefs, or to the same beliefs over and over. The final way the believer might defend B is to put forward a belief (either B itself or some other belief offered in support of B) as not needing any further support. In that case, the skeptic will rely on the mode of hypothesis: if the believer can hypothesize, say, C, there is nothing to prevent the skeptic from hypothesizing the denial of C. There is no reason to prefer one to the other. Given the abstract and perfectly general nature of Agrippa’s modes, it is natural to think that they have no special relevance to understanding perception. However, many twentieth-century epistemologists used Agrippa’s modes to argue that the structure of justification has to be foundationalist in nature, and they took perceptual beliefs to be a central component of the foundational level.19 The logic of foundationalism dictates that we must 16 

Bayle, 1965/1697: 197. See also p. 365 (art. ‘Zeno of Elea’, rem. G). Berkeley, 1982/1710: pt. I, sections 14–15. 18  Mates, 1996: 110 (book I, section 15), and Annas and Barnes, 1985: appendix C. 19  This is often called the regress argument for foundationalism. See Price (1932), Lewis (1946), Russell (1948), Chisholm (1977), and Fumerton (1985 and 1995). For a very early version of this sort of view, see Aristotle’s Posterior Analytics, ii. 19 (1993/4th century bce). 17 

Skepticism and Perception    71 identify perceptual beliefs that do not depend on support from other beliefs and yet are not mere hypotheses. Perceptual beliefs that draw on a variety of different experiences— e.g. my belief that most lions are tawny—are not plausible candidates to play this role. But those beliefs that are somehow directly grounded in one’s current perceptual experience can, perhaps, serve as the bottom layer of beliefs in a foundationalist structure: they do not receive justification from other beliefs but do provide it to others. There are many subtle issues that arise in trying to understand what C. I. Lewis called the ‘given element in experience’, which is supposed to ground basic beliefs. Some are metaphysical in nature—e.g., is our experience best understood as consisting of discrete sense-data whose reality is exhausted by their appearance, or should we take experience to be simply properties of the perceiving subject? The more serious problems, though, are epistemological. First, some aspects of our experience are too complex to justify our beliefs. So, how can we make a principled distinction between those perceptual beliefs that are well supported by the relevant experiences and those that are not?20 Second, even if we do draw a principled distinction that allows us to see which perceptual beliefs can be grounded in experience, it seems very unlikely that we will be able to move much beyond those basic beliefs. The examples of foundational beliefs usually given have extremely simple contents—this is blue, or I am being appeared to redly. It is very hard to see how we do or even could use beliefs of that kind to justify beliefs about more interesting objects like tables and chairs or other people. If traditional foundationalism were the only game in town, a skeptical conclusion might seem to be inevitable.21 But many epistemologists have abandoned this sort of foundationalism in favour of one that takes the bottom-level beliefs to be justified in virtue of being reliable, in the sense that they are likely to be true. Reliability, however one understands it, is an externalist property, meaning that the subject who has the belief need not be aware that it is, in fact, reliable. Given this sort of view, there is no reason to limit beliefs at the foundational level to those that conform closely to one’s simple experiences. Beliefs about tables and chairs may turn out to be reliable enough to count as knowledge. Once again, it looks as though the response to skepticism is grounded in something that is unavailable to the subject herself.

4  Descartes’s three arguments In the First Meditation, Descartes offers three arguments for skepticism.22 The first of these—that the senses have deceived him and he should never trust anything that has ever misled him—is not one that he takes very seriously. If he did, he surely would have tried to offer a more substantial reply to it later in the Meditations. The purpose of the first 20 

This is the ‘problem of the speckled hen’, which Gilbert Ryle raised for A. J. Ayer; see Chisholm (1942). See Fumerton (1985 and 1995) for an example of someone who follows the route from foundationalism to skepticism. 22  Descartes, 1984/1641: 12–15 (AT vii 17–23). All three arguments are couched in the first person, and I will follow Descartes in presenting them in that way. Nevertheless, it is helpful to keep in mind that, when Descartes says ‘I’, he is not straightforwardly referring to himself. Rather, he is leaving open space for the reader to follow the meditator’s thoughts. 21 

72   Baron Reed argument is, I think, rhetorical. By first presenting and then immediately discarding the argument, Descartes’s meditator is able to pass himself off as a reasonable person—he’s not someone who will allow himself to be bullied by skeptical arguments. Seeing that the meditator will push back when necessary makes it easier for the reader to follow the path the meditator takes through the Meditations. The same pattern holds when Descartes raises the second of his three skeptical arguments—viz., that he can have the same experiences while dreaming that he has when he is awake. The meditator first considers the possibility that waking experience is intrinsically different from dreams and then rejects it: ‘there are never any sure signs by means of which being awake can be distinguished from being asleep’.23 So far, it looks as though Descartes is largely reprising the argument the Academics used against the Stoics, which worked by calling into question the veridicality of any particular experience a person might have.24 But a careful reading of the entire Meditations shows that Descartes has something else in mind. As this point cannot be fully appreciated without understanding what Descartes is doing in his third skeptical argument, however, let us turn our attention to it. Philosophers often take Descartes’s third argument to rest on the possibility that an evil demon is deceiving him, but this is not entirely correct. It’s true that Descartes does try to encapsulate the argument in the hypothesis that an evil demon is deceiving him, and he does make use of that hypothesis when he brings forth the certainty that he exists in the Second Meditation. But when Descartes introduces the skeptical problem in the First Meditation, he considers first the possibility that he has been created by an all-powerful God and then the possibility that he has come into existence as a matter of random chance. What is really driving his skeptical doubt is the fact that he is fundamentally ignorant of his origin—and of what that origin means for his cognitive abilities. Thinking of this argument as grounded in the evil demon hypothesis also tends to cause confusion in that many philosophers have remarked that it could be updated by changing the possibility of being the victim of an evil demon to the possibility that one is a brain in a vat. But the latter scenario seems to work by calling into question one’s experiences. As illustrated so vividly in The Matrix, the world could be utterly different from the run of experiences one has. Understood in that way, Descartes’s third argument is not very different from the second; in effect, it merely adds the possibility that one might be dreaming all the time. To be sure, Descartes’s third argument does make that addition, but it also goes much further. Notice that, when Descartes begins the argument, he says that ‘since I sometimes believe that others go astray in cases where they think they have the most perfect knowledge, may I not go similarly wrong every time I add two and three or count the sides of a square, or in some even simpler matter, if that is imaginable?’ 25 What Descartes calls into question here is not merely our perceptual experience but our capacity for clear and distinct perception.26 That is, the ideas or concepts we use to make sense of our perceptual

23 

Descartes, 1984/1641: 13 (AT vii 19). is probably why Hobbes accused Descartes of rehashing ‘ancient material’ in the First Meditation; see Descartes, 1984/1641: 121 (AT vii 171). 25  Descartes, 1984/1641: 14 (AT vii 21). 26  For an account of how Descartes can call into question clear and distinct perception without falling into the Cartesian Circle, see Reed (2012). 24 This

Skepticism and Perception    73 experience might turn out to be entirely wrong. It’s not just that I might somehow make a calculation error when adding two and three—the real worry is that the world might not be mathematically describable at all. My fundamental ideas—extension, substance, causation, etc.—might all fail to correspond to anything in reality. The same sort of skeptical worry was developed in a variety of ways by some of the Early Modern skeptics who came after Descartes. For example, Bayle says that, ‘if it were true that the real existence of extension contained contradictions and impossibilities,’ as he has argued in his article on Zeno of Elea, ‘it would be absolutely necessary to have recourse to faith to be convinced that there are bodies’.27 Faith, rather than knowledge, would be the most we could hope for, given the way our ideas impeach themselves. Descartes, of course, thinks that this skeptical problem can be resolved. The path he follows—from the knowledge that he exists to the knowledge that God exists and is not a deceiver—is well known and largely unconvincing to philosophers now. But let us suppose for a moment that it succeeds. God’s guarantee turns out to work differently for clear and distinct perception on the one hand and sense perception on the other. God’s guarantee of clear and distinct perception is perfect because, if we did fall into error through using it, we would have no way of correcting it and God couldn’t fail to be a deceiver. The same cannot be said for most of our perceptual beliefs: they can and should be corrected through the use of clear and distinct perception.28 We are at last in a position to return to the difference between the Academics’ challenge to perception and Descartes’s. Where the former seem to call into question the veridicality of perceptual experiences on an individual basis, Descartes wants to reduce the general epistemic status of perception and remove it from its central position in Aristotelian and Scholastic epistemology. The guarantee he thinks God provides for perception is conditional on our correcting it through the use of clear and distinct perception. Sense perception works well enough for practical purposes, but it is not an adequate way of coming to know the true nature of things—that is the correction that needs to be made. Descartes can manage this sort of limited vindication of perception because he, unlike the Academics, is not a skeptic. It is the combination of his skeptical and anti-skeptical arguments that allows him to manoeuvre perception into the reduced role he thinks it is suited for. Descartes returns to the dream argument at the very end of the Sixth Meditation. His response is rather unsatisfactory—it’s abrupt, inaccurate (in that he now identifies this as the principal reason for doubt, despite the fact that he has devoted far more space to replying to the third argument), and weak. He says that he can ‘now notice that there is a vast difference between’ being asleep and being awake; all he needs to do is to use all of his senses, together with his memory and his intellect, to check the veracity of his perceptions.29 Granted, the difficulties of ordinary life often make this practically impossible, so 27 

Bayle 1965/1697: 377 (art. ‘Zeno of Elea’, rem. H). Bayle sets out the ‘contradictions and impossibilities’ in remark G of this article (pp. 359–372). 28  One exception seems to be the most general perceptual beliefs we have—viz., that there are corporeal things. Descartes says that these objects ‘may not all exist in a way that exactly corresponds with my sensory grasp of them, for in many cases the grasp of the senses is very obscure and confused. But at least they possess all the properties which I clearly and distinctly understand’ (1984/1641: 55 (AT vii 80)). Descartes allows this exception because God has given us a strong natural tendency to believe in mind-independent corporeal objects and no faculty that allows us to correct that tendency (1984/1641: 55 (AT vii 80)). 29  Descartes, 1984/1641: 61–2 (AT vii 89–90).

74   Baron Reed Descartes closes by saying that ‘it must be admitted that in this human life we are often liable to make mistakes about particular things, and we must acknowledge the weakness of our nature’.30 When this ‘discovery’ comes without the support of any argument or investigation, one is tempted to ask why he didn’t just come out with it in the First Meditation. In reality, though, Descartes is simply far less interested in what we may know about particular things—and how well we may know it—than he is in putting science on a more secure footing.31 This is well worth keeping in mind, for two reasons. First, Descartes’s limited goal—and his confidence in his own anti-skeptical arguments—allows him to make the confrontation with skepticism an all-or-nothing affair. He pushes his readers into the depths of despair in the First Meditation. This is not the aim of those who identify themselves as skeptics (e.g., the Pyrrhonists, the Academics, and Montaigne). Second, later epistemologists have tended to follow Descartes in seeing the skepticism/anti-skepticism dispute as one that must end in either of two extreme outcomes—total doubt or the complete vanquishing of skepticism. But they have also turned their focus primarily to ordinary sorts of knowledge, not to the foundations of science. That has both magnified the conflict and made it difficult for many to take the skeptical side of the debate seriously, resting, as it appears to, on the outlandish possibility that an evil demon is deceiving me right now into thinking that I have a human body and am sitting in a chair, etc. To be fair, this is one way of coming to terms with the legacy of skepticism. But it is far from the only or most interesting way to understand it.

5  Moorean common sense If skepticism is the result of carrying the critical reflection characteristic of philosophy to its logical extremes, common sense—as defended by Thomas Reid, G. E. Moore, and others—is the diametrically opposed tendency. Common sense seeks to limit the reach of philosophy, to keep it safely within narrow bounds so that dangerous conclusions can be forestalled or ignored. It is, in that sense, a far more pessimistic approach to philosophy than skepticism is. Moore’s quintessential commonsense reply to skepticism can be seen in his treatment of an argument advanced by Bertrand Russell: Russell’s view that I do not know for certain that this is a pencil or that you are conscious rests, if I am right, on no less than four distinct assumptions . . . And what I can’t help asking myself is this: Is it, in fact, as certain that all these four assumptions are true, as that I do know that this is a pencil and that you are conscious? I cannot help answering: It seems to me more certain that I do know that this is a pencil and that you are conscious, than that any single one of these four assumptions is true, let alone all four.32

30 

Descartes, 1984/1641: 62 (AT vii 90). He is clear at the outset that his goal is to establish something ‘in the sciences that was stable and likely to last’; Descartes, 1984/1641: 12 (AT vii 17). 32  Moore, 1959: 226, emphasis in the original. Moore is responding to Russell (1927). 31 

Skepticism and Perception    75 I’ve left out Russell’s assumptions because it doesn’t really matter to Moore’s argument what they are. As William Lycan would say, the assumptions ‘are only philosophy stuff’.33 They can never be so secure as to overturn the convictions of common sense, to which we are so strongly committed. This response to skepticism, which begins and ends with a simple comparison of some philosophical claims to a commonsense belief, betrays a startling lack of curiosity. As Bryan Frances has pointed out, it allows us to defeat the skeptic ‘without showing just what it is that makes it true that Joe knows he has hands or even showing where the sceptic’s argument has gone wrong. We get the anti-sceptical result without doing any of the work!’ 34 When Moore says that he is more certain that he knows he has a pencil in front of him than he is certain of the truth of any of Russell’s assumptions (or, for that matter, the assumptions used in any skeptical argument), it sounds like he is making a plausible claim. If it is to be an answer to skepticism, though, it is itself more ‘philosophy stuff’, and we need to assess it on those terms. What does it mean to say that he is more certain that he has a pencil? If he could give an epistemological account that shows precisely how the commonsense belief is more certain than the skeptical assumptions, that would be wonderful—but it would be a proper philosophical refutation of skepticism, not one that relies on the simple comparative strategy. If, on the other hand, he cannot explain—philosophically—how he is more certain that he has a pencil, then it is hard to see why the skeptic, or anyone else, should take his claim seriously. The same dilemma can be posed to Moore’s other famous response to skepticism. He holds up a hand and says, ‘Here’s one hand,’ and then he holds up his other hand and says, ‘Here’s another.’ Because hands are things that exist outside the mind, Moore takes himself to have proven that an external world exists.35 If this is a proof, it has to be more than a valid argument, for of course people who remain locked in a disagreement might nevertheless agree that a particular argument is valid. What we need to know is whether the premisses, and therefore the conclusion as well, are true. Can Moore say anything on behalf of the premisses? If he could, he would be offering a proper philosophical response to skepticism. But, if he can’t do this (and, in fact, he doesn’t), it is hard to see how he can legitimately claim to have proven anything. Common sense, in its attempt to bring a premature close to philosophical investigation, is not so much a philosophical outlook as it is an evasion of philosophy.

6  Dogmatism, externalism, and circularity If we are to move beyond Moorean handwaving, we will need to have some way of explaining how the premisses used in responding to skepticism are justified. One possibility is to claim that perceptual experience simply provides justification for one’s perceptual beliefs; 33 

34  Frances, 2005: 167. Lycan, 2001: 41. is the argument Moore gives in ‘Proof of an External World’, also in Moore (1959). Some philosophers have argued that Moore’s target in that paper is idealism rather than skepticism; see Sosa, 2009: ch. 1. Most, however, have regarded it as of a piece with his commonsense reply to skepticism. If necessary, the argument could be strengthened so that it is more obviously epistemological rather than metaphysical. On the strengthened version, his premisses would be that he knows he has hands, and the conclusion would be that he knows there is an external world. 35  This

76   Baron Reed an experience represents the world as being a certain way, and it also justifies one’s belief that the world is that way.36 There are at least two reasons why this view can be called dog­ matism. First, the subject’s justification rests solely on the experience; the subject does not need to be able to cite the experience or anything else as justification for the belief. And, second, the philosophical claim that experience provides this sort of immediate justification is posited but not argued for. Dogmatism about perceptual experience thus threatens to take us into a bad epistemological neighbourhood, where the residents each claim about their own favoured ways of forming beliefs—divine revelation, augury, use of a crystal ball, etc.—that they provide immediate justification. This is a problem to which we shall return. For now, though, let us grant the dogmatist about perceptual experience that it does confer justification on the relevant beliefs. Dogmatism about perceptual experience is an internalist theory, in the sense that it takes justification to stem from the mental states of the subject. One of the major trends in epistemology of the last several decades—and a second way of moving beyond bare Mooreanism—is a move toward epistemological externalism. Views of this sort take justification and knowledge to be grounded in properties that are not internal to the subject—the relevant properties are neither mental states nor easily accessible to the subject’s consciousness. There have been a variety of externalist properties suggested as the ground of justification, including at least the following:  a causal relation obtaining between the belief and its object, the reliability of the process that forms the belief, and the belief’s counterfactual tracking of the truth.37 To take a simple example, suppose that a subject looks out of her window, sees a pine tree, and thereby comes to believe and to know that there is a pine tree nearby. The causal theorist would account for her knowledge by pointing to the complex causal relation that includes the light reflected from the tree to her retinas and the neurological activity that eventuates in the relevant belief. The reliabilist would also require that this process be one that leads the subject to a true belief, not only now, but as a usual matter of course. The tracking theorist, finally, would take the subject’s belief to be knowledge if it is one that would be held in the right circumstances; when there is a pine tree there, she believes it, and when there isn’t she doesn’t.38 It is consistent with any of these forms of externalism that a subject could have a perceptual belief that is justified and counts as knowledge, even though she may not be able to offer any reasons or evidence on its behalf. In that regard, externalism and dogmatism agree. If either dogmatism or externalism were correct, it might seem that the debate with skepticism could be easily resolved. Moore’s belief that he has a hand could be justified in virtue of his perceptual experience of it or in virtue of the reliability of his perceptual belief-forming processes. He may not be able to offer any further defence of it, but at least there will be an account of how he knows the premisses in his argument to be true. This is not, however, the end of the story. The most plausible form of dogmatism holds that perceptual experience provides prima facie justification, where this can be defeated

36 

See Pryor, 2000: 519. See Armstrong (1973), Goldman (1979), and Nozick (1981), respectively. 38  I am ignoring some subtleties here. For example, the causal theorist has to require that the causal relation is not deviant, the tracking theorist pays attention to the method used to form the belief when he evaluates the counterfactual matching of belief with truth, etc. These are important details, but they are largely irrelevant to my purposes here. 37 

Skepticism and Perception    77 by counterevidence.39 The same is true of externalism; the external properties that feature in those views can be defeated when the subject has counterevidence for her belief.40 (Defeaters can themselves be defeated by further evidence. When this happens, the subject’s original justification is restored or replaced with something stronger.) There are several different ways in which counterevidence can provide a defeater for a particular belief.41 Sometimes, the counterevidence indicates that the belief is false. For example, a friend might tell you that it is raining outside, but when you look out of the window you see that it isn’t. Alternatively, the counterevidence might not say anything directly about the truth or falsity of the belief, but it might indicate that the belief is not justified. For instance, a neighbour may tell you that he saw a coyote at the end of the block, but his son informs you that his father can’t see that far without his glasses, which he is not wearing. It remains possible that a coyote really is at the end of the block; even so, you are not justified in thinking this. Finally, counterevidence might indicate that the belief in question may not be justified. It’s not that the subject ought to regard her belief as definitely unjustified—say, because the source is unreliable. Rather, the subject ought to regard the source as too risky to rely upon. For example, you may have believed the results of a recent study that said that eating egg yolks does not significantly raise the risk of a heart attack, but then you learn that the study was conducted by the Poultry Farmers Association. Knowing this last fact does not warrant rejecting the study as unreliable or false, but it would be foolish to rely on the study without seeing independent confirmation of its results.42 The skeptic is now in a position to introduce a defeater for Moore’s belief that he has a hand (or, of course, for any perceptual belief).43 We have seen arguments that show that perceptual experience can be misleading in any particular instance (e.g. the dream argument) or in general (e.g. Decartes’s third argument, grounded in ignorance of one’s own origin and nature); we have also seen arguments that conclude perceptual experience distorts or hides the true nature of things (e.g. Bayle’s use of perceptual relativity and Descartes’s correction of sense perception). These arguments may not positively show that perception is incapable of providing us with justified beliefs, but they do seem to be enough for us to doubt whether our perceptual beliefs are justified. What we need, then, is something that will defeat these defeaters. Only then will our perceptual beliefs count as ultima facie justified and as knowledge. In response to this skeptical challenge, the dogmatist or the externalist may offer a track record argument to establish the reliability of perception. The material for it will be provided by further uses of the subject’s perceptual capacities. Both the dogmatist and the externalist will hold that each resulting belief is justified—in virtue of being grounded in experience or in virtue of having been reliably produced. Having amassed enough 39

  Pryor 2000: 534.   See, e.g., Goldman (1986), Nozick (1981), and Plantinga (1993). The need to add a no-defeater condition to externalist theories was made clear through a series of counterexamples in BonJour (1985: ch. 3), in which a subject has a belief with the requisite externalist property (e.g. reliability) but also has evidence that the belief is false or unjustified. 41  See Lackey, 2008: 44–46, for more on defeaters. 42  For this sort of case, see Reed (2006). 43  See Reed (2006) for this sort of argument. Pryor (2000: 534), says that ‘a priori skeptical arguments do not standardly introduce defeating evidence’ of the sort that he would like to countenance, but he doesn’t justify this stipulation. This is a surprising omission, given how central it is to his view. In any case, typical skeptical arguments are not entirely a priori. Part of what makes the dream argument, for example, compelling is our experience of having vivid, realistic dreams. 40

78   Baron Reed instances of perceptual knowledge, the subject can then inductively infer that perception is a reliable means of acquiring true beliefs. There are two problems with this sort of response. First, the dogmatist and the externalist are warranted in claiming only that these perceptual beliefs have prima facie justification. The skeptic might still argue that the defeater in question applies to all of them. In that case, the track record argument cannot get off the ground because none of the premisses is actually available for use. Second, the response given by the dogmatist and the externalist involves using a belief-forming method to justify itself. But this sort of epistemic circularity undermines the conclusion that the dogmatist and the externalist are trying to reach.44 It would be like asking a used-car salesman if he is honest—an affirmative answer does nothing to settle the question. As I mentioned earlier, there are many different ways of forming beliefs about which one might be dogmatic. The proponents of divine revelation, crystal-ball reading, and so forth might now argue that they, too, can prove that their favoured methods are reliable. After all, when they ask God if he is a reliable guide to the truth, he always says yes! Is the dogmatist about perception, or the externalist who takes perception to be reliable, really no better off than the revelationists and the occultists? At this point, argument seems to give out. There is nothing further that could be said that wouldn’t, at some point, involve us in this sort of epistemic circularity. Still, some epistemologists will say that there is a crucial difference, which we should not overlook. Even though the argument vindicating perception is epistemically circular, it has the virtue of having true premisses and a true conclusion.45 The other dogmatists can’t say the same.46 In the end, how far have we advanced beyond Moore’s commonsense response to skepticism? A more elaborate argument has been provided, but it comes to rest, ultimately, on the same sort of bare assertion that seemed so unsatisfactory in the context of Moore’s shorter proof. In assessing the dispute between skepticism and anti-skepticism, it may be helpful to keep in mind that there is no single conclusion that all skeptics have been driving towards. In the most extreme, and pessimistic, case, the outcome of skepticism is the conclusion that no one knows—or even has reason to believe—anything. But it is also possible to be a more moderate sort of skeptic—one who accepts perception as among the best ways of forming beliefs available to us, even if it cannot be put on as secure a footing as we would like.47 Skepticism understood in this way is not a pessimistic conclusion to philosophical reasoning, but rather a persistent goad to try to carry it further.48

References Annas, J. and Barnes, J. (1985). The Modes of Scepticism. Cambridge: Cambridge University Press. Aristotle (1993/4th century bce). Posterior Analytics (2nd edn), trans. J. Barnes. Oxford: Oxford University Press. 44 

See Fumerton 1995: 173–180, and Vogel (2000) for this complaint. 46  Well, they will say the same. But they’ll be wrong. See Sosa, 2009: 206–208. 47  See Reed (2012). 48  For helpful comments on this chapter, I am grateful to Jennifer Lackey and Mohan Matthen. 45 

Skepticism and Perception    79 Armstrong, D. (1973). Belief, Truth and Knowledge. Cambridge: Cambridge University Press. Bayle, P. (1965/1697). Historical and Critical Dictionary, trans. R. Popkin. Indianapolis: Bobbs-Merrill. Berkeley, G. (1982/1710). A Treatise Concerning the Principles of Human Knowledge. Indianapolis: Hackett. Bett, R. (ed.) (2010). The Cambridge Companion to Ancient Scepticism. Cambridge: Cambridge University Press. BonJour, L. (1985). The Structure of Empirical Knowledge. Cambridge, MA:  Harvard University Press. Chisholm, R. (1942). ‘The problem of the speckled hen’. Mind, 51, 368–373. Chisholm, R. (1977). Theory of Knowledge (2nd edn). Englewood Cliffs, NJ: Prentice-Hall. Cicero (2006/46–4 bce). On Academic Scepticism [Academica], trans. C. Brittain. Indianapolis: Hackett. Descartes, R. (1984/1641). ‘Meditations on first philosophy’. In The Philosophical Writings of Descartes, vol. ii, trans. J. Cottingham, R. Stoothoff, and D. Murdoch. Cambridge: Cambridge University Press. Frances, B. (2005). Scepticism Comes Alive. Oxford: Oxford University Press. Fumerton, R. (1985). Metaphysical and Epistemological Problems of Perception. Lincoln, NE: University of Nebraska Press. Fumerton, R. (1995). Metaepistemology and Skepticism. Lanham, MD: Rowman & Littlefield. Goldman, A. (1979). ‘What is justified belief?’ In G. Pappas (ed.), Justification and Knowledge (pp. 1–23). Dordrecht: Reidel. Goldman, A. (1986). Epistemology and Cognition. Cambridge, MA: Harvard University Press. Lackey, J. (2008). Learning from Words: Testimony as a source of knowledge. Oxford: Oxford University Press. Lagerlund, H. (2010). ‘A history of skepticism in the Middle Ages’. In H. Lagerlund (ed.), Rethinking the History of Skepticism: The missing medieval background (pp. 1–28). Leiden: Brill. Lewis, C. I. (1946). An Analysis of Knowledge and Valuation. LaSalle, IL: Open Court. Locke, J. (1979/1690). An Essay Concerning Human Understanding, ed. P. H. Nidditch. Oxford: Oxford University Press. Long, A. A. and Sedley, D. N. (eds) (1987). The Hellenistic Philosophers, vol. i. Cambridge: Cambridge University Press. Lycan, W. (2001). ‘Moore against the new skeptics’. Philosophical Studies, 103, 35–53. McDowell, J. (1988/1982). ‘Criteria, defeasibility, and knowledge’ (rev. edn). In J. Dancy (ed.), Perceptual Knowledge (pp. 209–219). Oxford: Oxford University Press. Mates, B. (1996). The Skeptic Way. Oxford: Oxford University Press. Moore, G. E. (1959). ‘Four forms of scepticism’. In his Philosophical Papers. London: George Allen & Unwin. Nozick, R. (1981). Philosophical Explanations. Cambridge, MA: Harvard University Press. Plantinga, A. (1993). Warrant and Proper Function. Oxford: Oxford University Press. Price, H. H. (1932). Perception. London: Methuen & Co. Pryor, J. (2000). ‘The skeptic and the dogmatist’. Noûs, 34, 517–549. Reed, B. (2002). ‘The stoics’ account of the cognitive impression’. Oxford Studies in Ancient Philosophy, 23, 147–180. Reed, B. (2006). ‘Epistemic circularity squared? Skepticism about common sense’. Philosophy and Phenomenological Research, 73, 186–197.

80   Baron Reed Reed, B. (2012). ‘Knowledge, doubt, and circularity’. Synthese, 188, 273–287. Russell, B. (1927). An Outline of Philosophy. London: George Allen & Unwin. Russell, B. (1948). Human Knowledge: Its scope and limits. New York: Simon and Schuster. Sosa, E. (2009). Reflective Knowledge: Apt Belief and Reflective Knowledge, vol. ii. Oxford: Clarendon Press. Thorsrud, H. (2009). Ancient Scepticism. Berkeley and Los Angeles: University of California Press. Vogel, J. (2000). ‘Reliabilism leveled’. Journal of Philosophy, 97, 602–623.

Chapter 4

Perception i n E a r ly Moder n Philosoph y Alison Simmons

The senses were subject to considerable scrutiny during the seventeenth and eighteenth centuries (traditionally called the ‘early modern’ period). No early modern philosopher would have denied that the senses are an important source of knowledge about the world, but many worried that they are a systematically misleading source. Consider Malebranche’s ominous warning: I shall teach you that the world you live in is not at all as you believe it to be, because actually it is not as you see it or sense it. You judge on the basis of the relation of your senses to all the objects surrounding you, and your senses beguile you infinitely more than you can imagine . . . there is no precision, no truth in their testimony.1

Descartes before him was less melodramatic but similarly critical:  the senses ‘do not, except occasionally and accidentally, show us what external bodies are like in themselves’.2 It’s not all bad news for the senses, however. Hand in hand with this worry about their ability to show us what the world is really like came an extensive re-examination of almost all aspects of perception. Along the way, the early moderns made important advances in our understanding of the perceptual process, established some of the classic questions with which philosophers and perceptual psychologists wrestled for centuries, and even offered a new vision of the proper function of the senses. Why did the senses come under a cloud of suspicion in the first place? The senses had been eyed cautiously since ancient times: ancient atomists cast colours and flavours into the mind of the perceiver, Plato charged the senses with acquainting us with only shifting appearances, and ancient sceptics challenged the ability of sensory appearances to show us with any certainty what the physical world is really like, or indeed whether there really is one. But something quite dramatic and lasting happened in the early modern period. The rediscovery of ancient writings, and especially ancient sceptical writings, certainly provided one source of renewed concern about the senses.3 An even more pervasive 1 

3 

2  Descartes (1644: II.3). Malebranche (1688: Dialogue I). See Popkin (1979) for an overview of the reception of sceptical writings in the early modern period.

82   Alison Simmons challenge, however, came from developments in natural philosophy or physics, which advanced what I will call the ‘mechanical hypothesis’. Details varied from thinker to thinker, but the basic shared idea was that the physical world is made up of a single kind of stuff, matter, that is (a) fully characterized by a handful of privileged properties including size, shape, position, and local motion, and (b) divisible into insensibly small parts that are characterized by those same properties. Any change in the physical world was to be explained by the motion and impact of variously sized and shaped bodies.4 This conception of the physical world forced a rethinking of almost every aspect of sense perception: the objects of perception; the causal process that gives rise to perceptual experience; the structure of perceptual experience itself; the reliability of the senses for showing us what the world is like; and even the function of the senses. This chapter considers each topic in turn. Although a great many thinkers contributed to the discussion about sense perception in this period, the chapter is limited to philosopher–scientists in Europe who had an especially lasting influence on the topic. It restricts its scope to philosophical questions and debates, leaving aside more technical discussions in optics, anatomy, and physiology, though developments in these areas certainly informed the philosophical discussion. Finally, it focuses on the ‘external’ senses (vision, audition, olfaction, gustation, and touch) to the neglect of the ‘internal’ senses (or bodily senses, like proprioception, kinesthesis, hunger, thirst, etc.), and among the external senses, vision takes pride of place as it did in the period.

1  The objects of perception Early modern philosophers continued to explore the ancient and medieval question how a substance, like a kumquat, can be the object of perception when all we immediately perceive are its sensible properties (its colour, odour, flavour, shape, size, etc.), but two other issues arose that dominated discussions about the object of perception: (a) the distinction between primary and secondary qualities and (b)  the question whether the immediate objects of perception are things or ideas of things. This section examines the primary–secondary quality distinction. Section 4 takes up the issue whether ideas are objects of perception. Boyle and Locke introduced the terms ‘primary quality’ and ‘secondary quality’ into the philosophical discussion in the latter part of the seventeenth century, but Galileo and Descartes made the distinction in the early part of the century.5 The distinction is roughly coextensive with the Aristotelian distinction between common and proper sensibles so far as the lists are concerned: primary qualities include many of the common sensibles (size, shape, position, and local motion and rest); secondary qualities include most of the proper sensibles (colour, sound, odour, flavour, hot and cold). The nature of the distinction, however, is quite different. While both of the Aristotelian common and proper sensibles were 4 

For some classic statements of the mechanical hypothesis, see Boyle (1666 and 1674); Hobbes (1655) and Descartes (1644 and 1664a). 5  See Peter Ross, Chapter 21, this volume.

Perception in Early Modern Philosophy    83 thought to be real properties present in bodies in just the way they appear to be,6 the early modern primary and secondary qualities differ from one another ontologically. Primary qualities were thought to be the fundamental intrinsic properties of body, and so they were present in bodies in just that way they appear to be. Secondary qualities were not. (What they were supposed to be we’ll consider in a moment.) The proposal that there is an ontological difference between a kumquat’s shape and its colour does not suggest itself to perceptual experience: both look to be out there in the kumquat. Arguments for distinguishing their ontologies therefore did not typically rely on introspecting perceptual experience. They piggybacked on arguments for the mechanical hypothesis itself, with its proposal that size, shape, position, and motion are ontologically privileged properties of body that are solely responsible for the movements and interactions of bodies. Some of these arguments for the mechanical hypothesis took the form of a priori conceptual analysis which alleged that size, shape, position, and motion are somehow constitutive of the very concept of body in a way that colour, odour, and the like are not.7 Others insisted on a posteriori grounds that the mechanical hypothesis made for an especially intelligible, simple, or fruitful physical theory.8 Thomas Reid was an interesting exception. He accepted the primary–secondary quality distinction, but thought one could argue for it simply on the basis of introspection. Reflection on our sense-based conception of size and shape, he insisted, shows it to be perspicuous and to inform us what these qualities are ‘in themselves’ while reflection on our sense-based conception of like colour, odour, and the like, he further insisted, shows it to be ‘relative and obscure’, informing us only how these qualities affect us.9 Once size, shape, position, and local motion were given pride of place in the ontology of the physical world, a problem was born about the ontology of the remaining sensible qualities. Just how positively to characterize the ontology of colour, sound, odour, and the like differed from philosopher to philosopher (and sometimes from sentence to sentence within a philosopher). The options included: (a) they are reducible to the (microscopic arrangement of) primary qualities in bodies; (b) they are to be identified with the powers of those (microscopic arrangement of) primary qualities in bodies to produce sensations in the mind of the perceiver; and (c) they are merely sensations in the mind of the perceiver that are produced by the (microscopic arrangements of) primary qualities in bodies. Few thinkers firmly opted for (a), though Descartes and Locke occasionally say things that suggest this reductive view.10 More common were versions of (b) and (c). Locke famously cast secondary qualities as ‘nothing in the objects themselves, but powers to produce various sensations in us by their primary qualities, i.e., by the bulk, figure, texture, and motion of their insensible parts’.11 Boyle and Reid followed Locke in portraying colours, odours, and the like as powers or dispositions of bodies to produce distinctive sensations in human perceivers (and Reid, as I suggested, seemed to think this is what the senses themselves suggest to us).12 Galileo and Hobbes, by contrast, opted for the view that colours, sounds, and the like are simply sensations in the mind of the perceiver. Here is Galileo: ‘[colour, odour, 6  They were distinguished by whether more than one sense modality had access to them. See Dominik Perler, Chapter 2, this volume. 7  See Galileo (1623); Descartes (1644: II.4); Locke (1690: II.viii.9); and Malebranche (1712: I.10). 8  See Descartes (1644: I.69–70 and IV.200–3) and Boyle (1674). 9  Reid (1785: II.17). 10  See Descartes (1637: Discourse 1, 1644: IV.199) and Locke (1690: II.viii.15 and II.viii.17). 12  See Boyle (1666) and Reid (1764: VI.4, 1785: II.17). 11  Locke (1690: II.viii.10).

84   Alison Simmons flavours, and sound] which are supposed to be qualities residing in external objects have no real existence save in us, and outside ourselves are mere names’.13 However unsettled the ontology of secondary qualities was in the period, two commitments were clear and common: (a) secondary qualities like colour and sound are not in physical objects in the way they sensorily appear to be, and (b) according to most parties, they depend as much on the perceiver as on the body to which they are attributed.14 To put the point in somewhat different language:  primary qualities are objective features of the physical world, while secondary qualities are in some sense subjective. Berkeley was a rare opponent to the primary–secondary quality distinction, which he considered both philosophically indefensible and an affront to common sense. His most memorable (if unconvincing) argument against the distinction came in the form of a simple challenge: try to conceive a body that has size and shape without conceiving it to have any colour or tactile qualities. He trusted we will find that we cannot, that the distinction is therefore inconceivable, and that it rests on a false abstraction of the mind.15 Both colours and shapes, he insisted, are ontologically on a par. Curiously, and much to the consternation of later philosophers, he defended this commonsense view not by returning colours and the like to a mind-independent world of bodies, but rather by drawing shapes and sizes into the mind alongside colours: all the sensible qualities (and bodies themselves, for that matter), Berkeley argued, exist as mind-dependent ideas (or, in the case of bodies, collections of ideas).16 Few followed him in this thesis.

2  The origins of perception: Sensory processing Just as the early moderns replaced the Aristotelian conception of the physical world and so objects of perception, so they replaced their account of how it is that we come to perceive it. The task for both was to explain how sensible properties out in the world are causally responsible for sense perceptual experiences in perceivers, a process that often occurs over a considerable distance. On the Aristotelian view, sensible species that in some way resemble the sensible properties of bodies are propagated through the medium and received by the sense organs of a sentient perceiver thereby ‘informing’ her experience of the world.17 Descartes explicitly rejected the species theory: the process, he wrote, cannot involve ‘images transmitted by objects to the brain’.18 Hobbes is downright hostile: ‘the 13  Galileo (1623). See also Descartes (1641: 6th Replies, 1644: I.71); Hobbes (1655: XXV.3 and 10, 1658: II.9); and Locke (1690: II.viii.19). 14  If there are genuine adherents to the reductionist position (a), they may say that the orange colour of the kumquat does not depend on the perceiver (the orange colour is nothing but, say, the having of a certain surface texture that can be cashed out in terms of size, shape, position, and motion), but even the reductionist will say that the appearance of the colour that we associate with the word ‘orange’ is dependent on the distinctive sensations that the kumquat produces in the perceiver. 15  Berkeley (1710, §10). 16  See Wilson (1982) for a discussion of the mind-dependence of sensible qualities in Berkeley. 17  See Dominik Perler, Chapter 2, this volume. For a more detailed discussion of this process, see Simmons (1994). 18  Descartes (1637: Discourse 4 and 1; 1641: 6th Replies; 1644: IV.198).

Perception in Early Modern Philosophy    85 introduction of species . . . passing to and fro from the object, is worse than any paradox, as being a plain impossibility’.19 The problem stemmed from the mechanical hypothesis: if bodies operate only by local motion and impact, then the production of, say, an orange species in the medium or eye is unintelligible; no amount of pushing or turning or jiggling of variously sized and shaped particles of matter is going to result in the production of something orange. Species had to go. The early moderns replaced the species theory with a thoroughly mechanical theory. Instead of species of colour and sound propagated through the medium, they proposed the impact of (insensibly small) bodies in motion. Hobbes offered a nice schematic account of the auditory processing of a ringing bell: . . . the clapper hath no sound in [the bell], but motion, and maketh motion in the internal parts of the bell; so the bell hath motion, and not sound, that imparteth motion to the air; and the air hath motion, but not sound; the air imparteth motion by the ear and nerve unto the brain; and the brain hath motion but not sound . . .20

The explanatory rules here are clear: from object to brain there is nothing but the local motions of bodies.21 A critical moment in early modern theorizing about the sensory process was Kepler’s discovery in 1604 that the function of the lens (or ‘crystalline humor’ as it was called) is not to sense, but rather to refract the light coming into the eye in such a way that all the rays of light coming from one point on a focal object are reassembled at a single point on the retina, resulting in a two-dimensional (upside-down and backward) image of the object on the retina.22 The role of the retinal image, and the puzzle about how the proper orientation and the three-dimensional properties of the object are recovered in perceptual experience, became central questions in the study of vision.23 Descartes accepted Kepler’s account of the lens and retinal image.24 Indeed he enthusiastically gives instructions on how to demonstrate it to ourselves with ‘the eye of a freshly dead human being—or failing that, the eye of an ox or some other large animal’.25 Descartes put the retinal image to work in his mechanical account of sensory physiology, speculating that the retinal image is reproduced in the brain by the motion of filaments in the optic nerve that connect the retina to the brain: when filaments jiggle on the retina they simultaneously jiggle at the other end on the ‘inner surface’ of the brain; in so doing they open pores in a pattern matching the retinal image. The open pores that constitute that brain image in turn draw animal spirits—tiny fast-moving bits of matter that perpetually flow from the pineal gland at the centre of the brain—so that the spirits trace yet another replica of the retinal image on the 19 

Hobbes (1658: II.4). See also Malebranche (1712: III.ii.2) and Reid (1784: II.8). Hobbes (1658: II.9). 21  See Descartes (1644:  IV.198); Locke (1690:  II.viii.11–13); Malebranche (1712:  I.11); and Reid (1785: II.2–3). 22  Kepler (1604: 5.3). 23  Berkeley, for instance, noted: ‘the solution of this knot about inverted images seems the principal point in the whole optic theory, the most difficult perhaps to comprehend, but the most deserving of our attention, and, when rightly understood the surest way to lead the mind into a thorough knowledge of the true nature of vision’ (1733: §52). 24  He further noted that the lens ‘accommodates’ to distance by changes in its shape as one focuses on nearer or further objects, thereby serving as an important cue to the object’s distance. 25  Descartes (1637: Discourse 5). Malebranche repeats the instructions at (1712: I.12). 20 

86   Alison Simmons surface of the pineal gland.26 An image of the object is thus transmitted mechanically from retina to brain to pineal gland.27 Although Descartes occasionally spoke as if the mind sees objects by inspecting these retinal and pineal images,28 he quite emphatically insisted in his Dioptrics that the mind in no way sees objects by inspecting images on the retina or in the brain ‘as if there were yet other eyes within our brain with which we could perceive’ them.29 It is rather that the motions of the filaments in the brain, and the flow of animal spirits from the pineal gland, cause our perceptual experience of the object.30 Writing more than a century later, Reid rejected Descartes’ idea that the retinal image is reproduced further back in the brain (he was dubious that anyone had any idea what was going on in the brain), but he continued to insist, with Descartes, that the mind in no way sees objects by inspecting images of them on the retina: ‘No man ever saw the pictures in his own eye, nor indeed the pictures in the eye of another, until it was taken out of the head and duly prepared.’ 31 The retinal image, and any images further back in the brain, were thought to be links in the mechanical causal chain. The million-dollar question, of course, is: what happens next? How do we get from local motions in the brain to a conscious perceptual experience of a bright orange, tangy, ovoid kumquat on the table three feet away? The question is really threefold: (a) how do we get from motions to conscious experiences in general (from something physical to something mental), (b) how do we get from local motions to colours and sounds in the case of secondary quality perception in particular, and (c) how do we get from local motions that constitute a two-dimensional image of the object in the head to a perception of three-dimensional objects outside the head and at a distance from us? The first two questions land us squarely in the mind–body problem. The third is a topic for perceptual psychology, and so will be considered in the next section. The problem how motions in the brain give rise to conscious perceptual experiences in the mind is clearest for Descartes. According to his dualism, mind and body are distinct and utterly heterogeneous substances:  the mind is an immaterial, consciously thinking and perceiving thing while the body is a material and unconscious thing devoid of thought and perception. The mystery is how two things that are so different could have any causal commerce with each other. Materialists didn’t have it any easier: it’s no more obvious how motions in the brain constitute conscious perceptual experiences than it is how they cause them in an immaterial mind.32 Unfortunately, the early moderns did not do much to illuminate this crucial step. Descartes’ treatment was typical: ‘we know that the nature of our soul is such that different local motions are quite sufficient to procure 26 

Descartes (1664b: §67). For an excellent introduction to Descartes’ sensory physiology, see Hatfield (1992). 28  Descartes (1664b: §70, 1641: Meditation 6). 29  Descartes (1637: Discourse 6). 30  See Wilson (1991) for a discussion of these two ways of reading the relationship between pineal image and mind. 31  Reid (1764: VI.12); see also Reid (1785: II.4). 32 Hobbes suggested that conscious perceptual experiences are constituted by a kind of outward directed resistance to motions coming into the brain from the external world: ‘from the brain, [the motion] reboundeth back into the nerves outward, and thence it becometh an apparition without, which we call sound’ (Hobbes, 1658: II.9; see also Hobbes, 1655: XXV.2). It’s not at all clear that this is an improvement on the dualist’s response. 27 

Perception in Early Modern Philosophy    87 all the sensations in the soul’.33 He added that God established a psycho-physiological law (he called it an ‘institution of nature’) joining types of brain motions with types of sensations in the mind,34 but that still doesn’t tell us how the causal exchange occurs.35 The early modern period was not without creative metaphysical ways around the problem of mind–body interaction:  Malebranche’s occasionalism (which was prevalent among the Cartesians) limited all causal efficacy to God, so that the causal exchange between mind and body, like all causality in the created world, was understood to be powered by God;36 Spinoza’s parallelism asserted that mind and body were not separate substances after all, but two expressions a single substance (God or Nature) and that their relation is not causal but representational;37 and according to Leibniz’s pre-established harmony, God set things up so that the internal (and internally caused) events in the mind would be cosmically coordinated with the internal (and internally caused) events in the body.38 Berkeley took Malebranche a step further and held that God directly causes ideas of bodies in us without the need for any extra-mental bodies existing at all.39 Most philosophers, in other words, backed away from saying there is a straightforward causal interaction between mind and body. In response to the second question how motions in the brain result in sensations of colour, flavour, and the like that bear no resemblance to anything in objects or the brain, the early moderns tended to deny that this creates any special problem by pointing out that physical causes produce mental effects that bear no resemblance to them all the time: the written word ‘aardvark’ results in our imagining a critter with a long nose; a blow to the eye results in seeing stars; and a knife slicing flesh results in pain. These things don’t give us pause, so why should we be puzzled at local motions giving rise to bright orange visual sensations and tangy flavour sensations?40 We are, once again, offered no real insight into how this transformation from motions to colours and flavours is supposed to work, and we are left with an explanatory gap. A handful of philosophers, like Locke and Reid, who were especially modest in their estimation of the range of human knowledge, conceded the explanatory gap between mind and body. Reid, for example, wrote: how are the sensations of the mind produced by impressions upon the body? Of this we are absolutely ignorant . . . there is a deep and a dark gulf between [mind and body] which our understanding cannot pass; and the manner of their correspondence and intercourse is absolutely unknown.41

Since intelligibility and explanatory success were meant to be precisely the advantages that the mechanical philosophy had over Aristotelian physics, this concession made at the 33 

34  Descartes (1641: Meditation 6, 1637: Discourse 6). Descartes (1644: IV.198). had a similar view but acknowledged the explanatory gap: ‘we must often be satisfied with knowing that certain things are connected, and invariably follow one another, without being able to discover the chain that goes between them. It is to such connections that we give the name of laws of nature’ (1764: VI.12). 36  See Malebranche (1688: Dialogue IV.10). 37  See Spinoza (1677). 38  See Leibniz (1695). 39  See Berkeley (1710: §26). 40  See, e.g. Descartes (1644: IV.197) and Locke (1690: Essay II.viii.13). Of course, the natural response is that these phenomena should give us pause! 41  Reid (1764: VI.21). See also Reid (1785: II.4) and Locke (1690: Essay IV.iii.13). 35  Reid

88   Alison Simmons body–mind junction was significant. Berkeley did not miss the opportunity to point out exactly that.42

3  The structure of perceptual experience Once in the mind, there is still work to be done to get us to perceptual experience. Descartes famously distinguished three ‘grades of sensory response’: (a) motions in the sense organs and brain, (b) sensations, which are the immediate result in the mind of those motions, and (c) a host of judgements that embellish those sensations in a variety of ways.43 Why suppose that judgements are involved in perception? After all, it doesn’t feel like I am making any judgements when I look at the trees out my window. I just open my eyes, turn my head, and I see them. The trouble stems from the fact that everyone in the period granted that the sensations we receive from the world are impoverished by comparison with the perceptual experience we actually have of it. Visual sensations, for example, were thought to amount to a series of kaleidoscopic images of two-dimensional colour patches. What we see, however, are three-dimensional objects of constant size, shape, distance, and colour. Visual experience must therefore have a more complex structure than meets the eye: it is, the early moderns proposed, determined by a combination of sensations received from without the mind and judgements produced from within it. Descartes’s account of the judgemental component of perceptual experience was a bit of a hodge-podge: it included judgements that attribute colours and flavours and the like to bodies, judgements that flesh out the three-dimensionality of visual objects, judgements that provide size and shape constancy, judgements by which objects appear to be ‘outside’ us, and also the epistemologically precarious judgements that things in the world are as they sensorily appear. He also rather implausibly suggested that while in adulthood we make all these judgements so habitually that we don’t notice them, we originally made them deliberately.44 Malebranche cleaned up the Cartesian picture by separating what he called ‘natural’ judgements that affect the way the world perceptually appears to us from ‘free’ judgements that result in beliefs about how things are in the world, for example, that they are as they appear.45 Only the latter free judgements, he thought, involve an act of will and result in beliefs for which we are epistemologically responsible, since they are the only ones we have any control over. Natural judgements that result in perceptual experience itself, he proposed, are hard-wired into our nature: that colours appear to be on the surfaces of objects and that bodies appear to have three dimensions not two are the result of judgements that occur in us ‘independently of us and even in spite of us.’46 That judgements contribute to perceptual experience was generally accepted. The nature of those judgements, however, was hotly debated. The debate centred largely on spatial perception in vision, for the problem to solve in this case was both clear and pressing: how is it that from two-dimensional retinal and sensory images we come to see three-dimensional 42 

For discussion, see Wilson (1982). Descartes (1641: 6th Replies). For a similar breakdown of the process, see Malebranche (1712: I.9). 44  See Descartes (1641: 6th Replies, 1644: I.71). 45  See Malebranche (1712: I.10 and I.14). 46  See Malebranche (1712: I.7, I.9, I.11, and I.14). 43 

Perception in Early Modern Philosophy    89 objects? Berkeley opened his hugely influential work, An Essay Toward a New Theory of Vision (1709), with a puzzle about distance perception in particular: 2. It is, I think, agreed by all that Distance, of itself and immediately, cannot be seen. For, distance being a line directed endwise to the eye, it projects only one point in the fund of the eye [the retina], which point remains invariably the same, whether the distance be longer or shorter. 3. I find it also acknowledged that the estimate we make of the distance of objects considerably remote is rather an act of judgement . . . than of sense.47

Everyone agreed that if the retinal image did not contain information about distance then any sensation produced in the mind as a result of it could not either.48 Judgements must fill in the distance. The Cartesians are routinely thought to cast these judgements as innate to the mind (witness Malebranche’s description of them as ‘natural’) and as intellectual in nature: we calculate the distance of an object ‘as if by a natural geometry’, Descartes famously wrote; thus, for example, from the knowledge we have of the distance between our eyes and their angular convergence on an object, we calculate by a process of reasoning the distance of the object by angle-side-angle.49 Berkeley, by contrast, argued that the judgements in question are learned not innate, and associative not intellectual: we immediately perceive distance only through touch (by reaching for or walking toward the object), and thus we must learn to ‘see’ distance by associating our visual sensations of an object with tactile perceptions of its distance; distance perception is an intermodal phenomenon and geometry has nothing to do with it.50 This classic portrayal of the debate does an injustice to the Cartesians, who in fact proposed many means by which we perceive distance, some purely psycho-physiological (and so involving no judgements at all), some involving learned associative judgements, and only occasionally ones that smack of innate geometrical reasoning.51 There is nevertheless a real disagreement between them over whether vision needs to be supplemented by touch in order for visual objects to be represented at a distance, and this debate had repercussions into the twentieth century.52 Reid was another important figure in the debate about the structure of perceptual experience. He rejected an important assumption shared by the Cartesians and Berkeley alike: that judgements effectively finish the representational job begun by sensations to provide us with a full-blooded perceptual experience of the world. First, Reid drew a sharp distinction between sensation and perception. Sensation, he proposed, is a wholly 47 

Berkeley (1709: §§2–3). But see below, fn. 51, and Hatfield (1992) for a possible exception in Descartes. 49  See Descartes (1637: Discourse 6; 1641: 6th Replies; and 1666: §48). See also Malebranche (1712: I.9). 50  Berkeley (1709: §45). 51  Even the ‘natural geometry’ in Descartes was sometimes simply a metaphor to describe a purely psycho-physiological process that operates ‘as if’ our brains were calculating by a kind of natural geometry. For his part, Malebranche was crystal clear that the ‘natural geometry’ is part of the system of ‘natural judgments’ hard-wired into the mind so that it involves no actual reasoning (Malebranche, 1712: I.9). For a helpful discussion of the many Cartesian cues to distance, including the natural geometry, see Hall’s commentary in the translation of Descartes (1664b), Hatfield (1992), Simmons (2003b), and Wolf-Devine (1993). 52  For a helpful review of the early modern debate, and a detailed discussion of Berkeley’s side of it, see Atherton (1990). For a discussion of the debate over the course of 300 years, see Degenaar (1996). 48 

90   Alison Simmons non-representational mental state that has no object but only distinctive qualitative character.53 Perception, by contrast, is a mental state that always has an object and that involves both a conception of the object and an irresistible belief in its present existence.54 The two mental states are not two stages in the building of a sensory representation of the world, but two fundamentally different kinds of mental state that are confused together in our ordinary perceptual experience of the world. Second, Reid thought that the transition from having a sensation to perceiving an object occured not by any kind of innate reasoning (contra the Cartesian account of distance perception) and not by learned associations (contra Berkeley’s account of the same), but simply, if mysteriously, ‘by the constitution of our nature’.55 Sensations, he proposed, are nothing but ‘natural signs’ of the properties of bodies that never bear any resemblance to their objects, but that trigger perceptions of those properties ‘by a kind of natural magic’.56 The resulting perceptions are ‘original perceptions’, which Reid distinguished from ‘acquired perceptions’ which depend on learned associations between one original perception and another, as when I ‘see’ that the bag of groceries is heavy.57 On the topic of spatial perception in particular, he granted Berkeley that it is an acquired perception: we must learn to see spatial properties through associations between original perceptions of visual figure (which are always only two-dimensional) and original perceptions of tangible figure (which is three-dimensional).58 The lively debate about spatial perception touched on another famous debate that surfaced in this period: the ‘Molyneux problem’. William Molyneux, an Irish philosopher, posed the following problem to Locke: suppose a man born blind is acquainted by touch with a globe and a cube of similar size and made of the same metal; take the globe and cube away; restore the man’s sight; can he on the basis of sight alone determine which object is the globe and which the cube? Although there are differences in the details, Locke and Berkeley both answered no: it takes experience to associate the look of the objects with the feel of their shapes.59 Leibniz, in a ‘Cartesian’ sounding moment, answered yes: both vision and touch give us access to an innate intellectual geometry, and once that is in place the patient can (at least in principle) visually discern the sphere and cube ‘by applying rational principles’, viz., geometry, to his new visual sensations; there is no need to associate the visual and the tactile here.60 Reid took on the problem too, but gave different answers in different places depending on slightly different construals of the Molyneux question: for a newly sighted blind boy not steeped in the principles of mathematics, he agreed with Berkeley 53 

See Reid (1785: I.1 and II.16, 1764: VI.20–21). See Reid (1785: I.1, II.5, and I.16, 1764: VI.20). Malebranche is often thought to have anticipated Reid’s distinction between sensation and perception (see Nadler, 1994; Jolley, 1995; and even Reid, 1785 II.7; but cf. Simmons). 55  Reid (1764: VI.12). 56  Reid (1764: 5.3, VI.21 and 6.24, 1785: II.16). 57  Reid (1764: VI.20). 58  Reid (1764: VI.3 and VI.21–22; 1785: II.19 and 21). His account is complicated by the fact that he thought our original perceptions of visible figure themselves skipped the sensational stage, proceeding directly from impressions in the eye to original perceptions of (two-dimensional) shapes. Moreover, contrary to Berkeley, who took visual and tactile figure to be heterogeneous properties, Reid insists that our original visual perceptions of figure are homogeneous but incomplete or ‘partial’ perceptions of the same properties perceived completely by touch (see Reid 1785, II.19). 59  Locke (1690:, II.ix); Berkeley (1709: §§41 and 110); and Berkeley (1710: §43). 60  Leibniz (1704, II.ix). 54 

Perception in Early Modern Philosophy    91 that the boy would not be able to distinguish the sphere from the cube and that discerning three-dimensional shapes through vision typically depends on cross-modal associational learning;61 for a sophisticated mathematician, however, he sided with Leibniz, arguing that she could figure out the shapes by applying the principles of mathematics to what is presented in visual sensation.62

4  A veil of ideas? So far we have focused on the question of how perceptual experience comes about. Let’s turn to perceptual epistemology: what is perception like as a source of knowledge about the world? There are two main questions to tackle here: (a) did the early moderns think that things or ideas of things are the proper objects of perception and (b) do the senses yield any useful knowledge about the world? This section treats the first question; the next and final section treats the second. It used to be taken for granted that the early moderns were committed to a ‘veil of ideas’ theory of perception according to which all we immediately see, hear, and smell are ideas that represent objects and their properties, and from which we can only infer the existence of objects and properties that lie on the other side of the ideational veil.63 Ideas, on this view, are both ontological and epistemological intermediaries standing between perceiver and world. The trouble with such a theory of perception is that it invites external world scepticism: if all I immediately perceive are ideas, then how can I be sure there is anything on the other side? It is also an affront to common sense: when I say that I see a vase of lilacs on the table, what I take myself to be in perceptual contact with is a physical vase of lilacs and not a mental representation of it. The evidence that the early moderns committed themselves to a veil of ideas theory is considerable: Descartes opened his Meditations on First Philosophy (1641) with a battery of sceptical arguments involving illusions, dreams, and a malicious demon that question our sensory access to a mind-independent world; a systematic disparity between sensory appearance and physical reality is built into the mechanical hypothesis suggesting a distinction between what we see and what there is; talk of ‘ideas’ was everywhere in the texts; talk of what is immediately perceived and what is only mediately perceived (or judged) was common; Berkeley went so far as to suggest that kumquats and their kin are themselves nothing but collections of ideas; and Reid charged almost every one of his predecessors with holding the veil of ideas theory that culminated in Berkeley’s idealism.64 There is, nevertheless, good reason to resist this reading of the early moderns. Ideas entered the theory of perception on behalf of explaining how it is that we perceive things 61 

Reid (1764, VI.3). Reid (1764: VI.7 and VI.11). Reid thought that visual and tangible figure were inter-derivable, since they are not, contra Berkeley, heterogeneous properties. For a helpful discussion of Reid’s complex answers to the problem, see Nichols (2007: ch. 9). 63  This reading of the early moderns has been around since Reid (1764 and 1785), but it was reintroduced with vigour in the twentieth century by Bennett (1971) and Rorty (1979). 64  The only philosopher (from Plato to Hartley) that Reid hesitates to attribute the veil theory to is the Cartesian philosopher Antoine Arnauld. See Reid (1785: II.13). 62 

92   Alison Simmons like lilacs. Any theory of perception has to work within certain constraints. For most of the early moderns those constraints included (a) no action at a distance, so information about the lilacs has to get to the perceiver somehow; (b) immaterial minds do the perceiving, so information about the lilacs ultimately has to get into the mind in an immaterial form;65 and (c) the world consists of matter in motion, so something has to account for the disparity between the way the lilacs physically are in the world and the purple, fragrant way they appear.66 Given these constraints, ideas might be thought of as immaterial carriers of information that enable the human mind to perceive things like lilacs in the colourful and fragrant way that it does. If that’s right, then early modern ideas were doing much the same work that Aristotelian species did, serving not as the objects of perception but rather as the means of perceiving objects in the way that we do.67 One way of arriving at this alternative, and increasingly popular, reading of the texts is to contextualize the term ‘idea’.68 It is clear that ideas are supposed to be mental, and so, for most early moderns, immaterial. It is also clear that we cannot think or perceive without them and that they are in some sense representations of objects. But are they mental objects? In his mature work, Descartes distinguished two ways to understand the term ‘idea’: (a) as an act of thinking or perceiving and (b) as a representation of an object.69 What is important is that these are two aspects of one and the same mental state and not two distinct mental items that stand to each other as act to object. The Cartesian Antoine Arnauld made this point emphatically: ‘I . . . take the idea of an object and the perception of an object to be the same thing.’ 70 If that’s right, then perceptions are themselves representations of objects; they are not directed to representations of objects. Properly speaking we do not perceive ideas; we have ideas; and in having them we perceive objects. Malebranche, by contrast, explicitly took ideas to serve as objects of perception: we have to perceive ideas if we want to perceive lilacs, for there is nothing representational about our own mind to make lilacs present to it. Malebranchean ideas, however, were purely intellectual representations residing in the mind of God, and nothing like the epistemologically private sensory images at play in the ‘veil of ideas’ theory. When he said that sense perception involves the perception of ideas, he had in mind that there is something intellectual at the core of it, and that there is something outside our minds to which all our perception is directed when we perceive an object.71 Interestingly, in the very public debate between Arnauld and Malebranche on the nature of ideas, each accused the other’s conception of ideas as falling prey to a scepticism-inducing veil between perceiver and world, and each prided himself on avoiding any such thing.72 65 

66  Berkeley was an exception to (c). Hobbes, a materialist, was an exception to (b). challenges to the ‘veil of ideas’ reading of the early moderns, see O’Neil (1974), Arbini (1983), Yolton (1984), Nadler (1994), Yolton (1996), Lennon (2001), and Bolton (2004). For some interpretive push back, see Chappell (1994) and Wilson (1994). 68  For a helpful overview of the many meanings and uses of the term in the period, see Yolton (1996, ch. 2). 69  He gives different labels to them in different places: ideas considered ‘materially’ and ‘objectively’ (Descartes, 1641:  Preface to the Reader); ideas considered ‘formally’ and ‘objectively’ (Descartes, 1641: Meditation 3); ideas considered ‘materially’ and ‘formally’ (Descartes, 1641: 4th Replies); and without any explicit labels (Descartes, 1644: I.17). 70  Arnauld (1683: ch. 5). 71  For discussion, see Jolley (1990: ch. 4); Nadler (1994); and Wahl (2004). 72  For discussion of this aspect of the debate, see Simmons (2009). 67  For

Perception in Early Modern Philosophy    93 Locke initially looks committed to the veil: the mind, he wrote, ‘hath no other immmediate object but its own ideas’73 and ‘’Tis evident, the Mind knows not Things immediately but only by the intervention of the Ideas of thing.’ 74 But then again he also wrote, like Arnauld, that having ideas and perceiving things amount to the same thing 75 and he vehemently rejected Malebranche’s reification of ideas as objects distinct from the perceiving mind.76 It is possible that in saying that ideas are the immediate objects of the mind, Locke was simply pointing out that our access to objects comes not by the objects hurtling themselves into the mind, but rather by way of ideas, that is, by our perceiving them.77 As for Berkeley, he insisted that he was ‘not changing things into ideas, but rather ideas into things’.78 In saying this he was trying to distance himself precisely from the veil theory: we immediately perceive ideas, but since those ideas constitute things like lilacs there is no veil between mind and world; ideas only introduce a veil when there is something else lurking behind them (like collections of insensibly small bits of [mind-independent] matter).79 Clearly the notion of an idea was not fixed in the period. But far from conceiving ideas as epistemological barriers to the world, the early moderns seem to have taken them to be an essential means of access to it (or, in Berkeley’s case, as parts of it) and seem to have been staunchly committed to avoiding the implications of any veil theory.

5  Sensory epistemology How, then, does perception fare as a source of knowledge in the early modern period? In the Aristotelian tradition, the senses were the starting point for our knowledge of both the existence and the nature of the physical world.80 The early moderns largely agreed that the senses are the source of our knowledge that a physical world exists. They took external world scepticism seriously, but responded to it with a variety of arguments for the existence of a physical world based on one or another feature of perceptual experience. Only Reid refused to give an argument, saying instead that our sense-based belief in the physical world is at once ‘unaccountable’ and unassailable.81 Most conceded to the sceptic, however, that the conclusion is not known with absolute certainty.82 Berkeley, 73 

Locke (1690: IV.i.1). 75  Locke (1690: II.i.9). 76  Locke (1706). Locke (1690: IV.iv.2). 77  For a representation of the interpretive dispute, see Chappell (1994); Lennon (2001); and Bolton (2004). 78  Berkeley (1713: Dialogue 3). 79  Berkeley took collections of insensibly small bits of matter to be just mere collections of ideas that are reliably correlated with collections of, in this case, lilac ideas, and not as something lurking behind the sensory appearances. 80  They were also the starting point for our knowledge of immaterial things like the soul and God (see Dominik Perler, Chapter 2, this volume). The early moderns had mixed views concerning our knowledge of immaterial things: some (like Descartes) allowed for purely intellectual knowledge of them that is independent of the senses; some (like Locke) adopted the posture of modesty about our ability to have genuine knowledge of immaterial things, though nonetheless offered arguments for the existence of God; some (like Berkeley) introduced a new category of cognition (cognition by ‘notion’) to talk about the knowledge we have of immaterial things; and some (like Hume) purported to have no idea (literally) what is meant by such terms. 81  Reid (1785: II.20). 82  For some of these discussions, see Descartes (1641: Synopsis and Meditation 6); Arnauld (1683: ch. 28); Locke (1690: IV.ii.14); Malebranche (1712: I.10 and Elucidation 6); and Reid (1785: II.20). 74 

94   Alison Simmons as usual, was an exception in thinking that the senses give us conclusive evidence for the existence of a physical world, but no evidence whatsoever for the existence of a mind-independent physical world. He was as concerned as the others with scepticism, but found a decisive means for refuting (or undermining) it in idealism. (Of course, many thought that Berkeleyean idealism amounted to a simple acceptance of the sceptic’s conclusion.) Where the early moderns departed sharply from their predecessors was in denying that the senses are reliable guides to the nature of the physical world. It is not that they thought the senses show us nothing about the physical world, but what they show us is limited and qualified. The early moderns generally agreed that our perception of primary qualities like size and shape shows us something about what the world is like, but there are qualifications: it is subject to occasional errors about particulars (square towers look round from a distance and straight sticks in water look bent); it is markedly perspectival (coins look perfectly round when viewed from above, but in some sense look elliptical when viewed obliquely); it is relational (what looks to be to your left looks to be to my right); and it is limited (we can’t see anything smaller than a mite or very far away). These caveats worried Malebranche a great deal: ‘the judgments we form on the testimony of our eyes concerning extension, figure, and motion are never exactly true’.83 Even Malebranche, however, allowed that our perception of primary qualities ‘include some measure of truth’.84 It represents to us the sorts of properties that really are out there in the world, even if any particular representation inevitably contains something false or misleading about it. Secondary quality perception is another matter. Locke famously argued that while our sensory ideas of primary qualities ‘resemble’ the qualities in the world that cause them, our sensory ideas of secondary qualities do not.85 Berkeley retorted that nothing mental can resemble anything physical (what could they possibly have in common?), but it seems clear that what Locke had in mind was that our perceptual experience of size, shape, position, and motion shows us (at least in general) what these properties are like in bodies, whereas our perceptual experience of colours, sounds, smells, flavours, and hot and cold does not show us what these qualities are like in bodies. Not even in general. Colour vision may show us that the surface of a lemon is different from the surface of a lime, but it does not reveal the true nature of that difference; that is, it does not show us what that difference consists in physically on the surface of the fruits. Descartes put it this way: ‘If someone says he sees color in a body . . . this amounts to saying that he sees . . . something there of which he is wholly ignorant, or in other words, that he does not know what he is seeing.’ 86 His persistent characterization of secondary quality perception as ‘obscure and confused’ underscored this point. Reid agreed with his predecessors on this point: our senses give us a distinct conception of primary qualities and ‘inform us what they are in themselves’ while they give us only ‘a relative and obscure notion’ of secondary qualities and ‘as to what they are in themselves, our senses leave us in 83 

84  Malebranche (1712: I.10). Malebranche (1712: I.10). Locke (1690: II.viii.15). Descartes and Malebranche similarly spoke of a lack of resemblance between mind and world in the case of colour, odour, sound, etc. (see Descartes, 1641: Meditation 6, 1664a: ch. 1, and Malebranche, 1712: I.12). 86  Descartes (1644: I.68). 85 

Perception in Early Modern Philosophy    95 the dark’.87 Only Berkeley thought our perception of secondary qualities quite true to the nature of reality. It should come as no surprise, then, that so many of the early moderns treated the senses as epistemic troublemakers. Because we typically believe what we see, hear, smell, taste, and feel, we wind up with a lot of false beliefs. On instinct, we believe that pineapples are yellow in just the way they appear to be; that where we see no bodies there are no bodies; that the sun rises and sets while the earth remains still; and that the stars are small and nearby. These are all false beliefs, and it takes something other than unaided sense perception to get us to appreciate the truth of the matter: pineapples may be yellow but not independently of their effect on colour vision; there are insensibly small bodies all over the place; the sun is stationary while the earth rotates; and the stars are very large and very far away. Even once we learn the truth, it is difficult to resist our habitual sense-based beliefs: Descartes notes that even once we study astronomy, it is hard to maintain that belief that stars are large and distant when we actually look at them.88 It is the prevalence and persistence of these false sense-based beliefs that prompted Descartes’s full-scale sceptical attack on the senses at the opening of the Meditations and Malebranche’s exhaustive and lengthy inquisition of the senses at the start of the Search After Truth. Their advice is to withdraw from the senses and rely instead on the deliverances of the intellect in our search for truth about the nature of the physical world. As an empiricist, Locke was friendlier to the senses, taking them as an essential starting point for knowledge, but even he thought the knowledge we can reap from them about the nature of the physical world is extremely limited by the ‘dull and narrow information’ we receive from our ‘not very acute ways of perception’.89 His advice was not to turn to the intellect (since he didn’t think it could do anything other than manipulate sensory ideas), but rather to ‘look a little into the dark side, and take a view of our ignorance’ 90 and to recognize with modesty that ‘we are not capable of a philosophical knowledge of the bodies that are about us’.91 Even Reid, a self-appointed apologist for the senses, concedes that the senses get us into epistemic troubles when we make hasty judgements on their basis.92 All of this raises a question: if the senses lead us into false beliefs about the world, then why on earth do we have them? It can’t be an unfortunate accident. In the theological context of the period, God created us and gave us our senses. What was he thinking? What kind of benevolent creator would equip his creatures with such distorting lenses on the world? The early moderns had an interesting answer: God gave us senses not to show us the true nature of the physical world, but rather to help us to get around safely in the world. As Malebranche put it: ‘the senses were not given to us to know the truth about things in themselves, but only for the preservation of our body’.93 The early moderns thus reconceived the very function of the senses. According to Aristotelian epistemology, the senses are something like under-labourers to the intellect, providing it with the raw materials for knowledge about the nature of the physical world. The early moderns reconceived them as guides to survival. Whatever disagreements the early moderns had on the details 87 

Reid (1785: II.17). See Descartes (1644: I.72 and 1641: 6th Replies). See also Malebranche (1712: I.14) and Reid (1785: II.22). 89  Locke (1690: IV.iii.6). 90  Locke (1690: IV.iii.22). 91  Locke (1690: IV.iii.29). 92  Reid (1785: II.22). 93  Malebranche (1712: I.5). 88 

96   Alison Simmons of sensory processing and the structure of sense perception, they all agreed on this. Even Berkeley, who disagrees with so much of what his predecessors say about perception, agreed that the function of the senses is to facilitate bodily self-preservation: through them ‘we are instructed how to regulate our actions in order to attain those things that are necessary to the preservation and well-being of our bodies, as also to avoid whatever may be hurtful and destructive of them.’94 With this in mind, let’s think again about the senses as sources of knowledge about the physical world. They may do a messy job of showing us the fundamental nature of physical reality, but if their job is to guide us safely through the world, what they need to show us is how it pertains to our bodily well-being. When I’m thirsty, I need to know not that the glass of milk is situated at a certain latitude and longitude but whether it is within reach of my hand. I need to know not that it has a certain physical constitution but whether it is good to drink. The senses reveal precisely these sorts of things to me: the glass of milk looks to be within arm’s reach to my right and smells fresh not sour, so I can drink it. The senses, we might say, provide a self-interested or even ‘narcissistic’95 view of the world: if I’m sick, that same glass of milk may well taste terrible, as well it should since milk is not what I need at the moment. The way the Cartesians put this point was by saying that the senses represent the world not ‘as it is itself’ but ‘as it is related to and can benefit or harm my body’.96 Note that this holds of both primary and secondary quality perception: the spatial properties of the glass of milk are represented egocentrically and perspectivally, and its secondary qualities (like its taste) as ‘agreeable’ or ‘disagreeable’.97 Both are helpful in guiding action. Locke went so far as to suppose that our senses are perfectly fitted ‘to the conveniences of life and the business we have to do here’, so that if we were given more acute sense organs, like ‘microscopical eyes’, that show us more of the detail of the physical world, it would only be to our detriment.98 Reid concurred, insisting that ‘by diminishing or increasing [the acuity of our senses] we should not mend but mar the work of Nature’.99 (Reid thought that our acquired sensory appreciation of fine wine and gourmet food, for instance, constituted such an increase that inevitably leads to a more miserable life.) When viewed as guardians of the body, the epistemological criticism of the senses is replaced with accolades. Descartes wrote that in matters concerning our self-preservation the senses ‘report the truth much more frequently than not’.100 Even Malebranche wrote that they are ‘faithful witnesses’ concerning what’s good for the body; that they are ‘accurate and precise’ in informing us about the relations between our own body and other 94  Berkeley (1709: §147). In other figures, see Descartes (1641: Meditation 6); Locke (1690: II.xxiii.12); and Reid (1785: II.16, 20, and 22). 95  See Akins (1996). 96  Descartes (1644: II.3) and Malebranche (1712: I.6, I.10, I.18 and Conclusion First Three Books, 1688: Dialogues IV.13–14 and XII.2). 97  See Descartes (1641: Meditation 6); Malebranche (1712: I.5); and Reid (1785: II.16 and II.21). Descartes went so far as to say that even colour perception represents things as agreeable or disagreeable (green, he suggested is the most agreeable colour). Malebranche and Reid would have none of that, and instead added a third category of ‘indifferent’ sensations that serve to help us readily distinguish objects. For a more detailed discussion of the self-interested nature of sensory representation in Descartes and Malebranche, see Simmons (2003a and 2008). 98  Locke (1690: II.xxiii.12). 99  Reid (1785: II.21). 100  Descartes (1641:, Meditation 6), emphasis mine; see also 6th Replies.

Perception in Early Modern Philosophy    97 bodies; and that they lead to ‘quite correct’ judgements concerning the preservation of the body.101 Reid insisted that they ‘neither require nor admit of improvement’ in this area.102 If the senses get us into epistemological trouble (i.e. false beliefs), it is because we use them hastily and improperly to construct theories about the nature of the physical world when they are designed to be guides to action.103 The problem is not so much with the senses as with our misuse of them. The scrutiny to which the senses are subject in this period, then, should be read not as an indictment against their epistemological credibility, but rather as part of an effort to recast their role in our cognitive economy.

References Akins, Kathleen (1996). ‘Of Sensory Systems and the "Aboutness" of Mental States’. Journal of Philosophy, 93, 337–372. Arbini, Ronald (1983). ‘Did Descartes have a Philosophical Theory of Sense Perception?’ Journal of the History of Philosophy, 22, 317–337. Arnauld, Antoine (1683). Des vrayes et dees fausses idées. Cologne. Translation: On True and False Ideas. Translated by E. J. Kremer. Lewiston: The Edwin Mellen Press, 1990. Atherton, Margaret (1990). Berkeley’s Revolution in Vision. Ithaca, NY: Cornell University Press. Bennett, Jonathan (1971). Locke, Berkeley, Hume: Central Themes. New York, Oxford: Clarendon Press. Berkeley, George (1709). An Essay Towards a New Theory of Vision. Dublin. Berkeley, George (1710). Treatise concerning the Principles of Human Knowledge. Dublin. Berkeley, George (1713). Three Dialogues between Hylas and Philonous. London. Berkeley, George (1733). New Theory of Vision Vindicated and Explained. London. Bolton, Martha (2004). ‘Locke on Sensory Representation’. In R. Schumacher (ed.), Perception and Reality: From Descartes to the Present (pp. 146–167). Paderborn: Mentis. Boyle, Robert (1666). Origin of Forms and Qualities. Oxford. Boyle, Robert. (1674) About the Excellency and Grounds of the Mechanical Hypothesis. London. Chappell, Vere (1994). ‘Locke’s Theory of Ideas’. In V. Chappell (ed.), The Cambridge Companion to Locke (pp. 26–55). New York: Cambridge University Press, 1994. Degenaar, Marjolein (1996). Molyneux’s Problem: Three Centuries of Discussion of the Perception of Forms. Translated from the Dutch by M. Collins. Boston: Kluwer. Descartes, René (1637). La Dioptrique, published with his Discours de la method. Leiden. Descartes, René (1641). Meditationes de prima philosophia, published along with six sets of objections and the author’s replies. Paris. Descartes, René (1644). Principia Philosophiae. Amsterdam. References to book and section as I.1. 101 

Malebranche (1688: Dialogue 1 and Dialogue 4); see also Malebranche (1712: I.12, I.20, and Conclusion of the First Three Books). 102  Reid (1785: II.21). 103  That is not to say that the senses are useless in constructing a theory of the physical world. Even for a rationalist like Descartes, they have an important role to play in scientific observation and experiment: the intellect determines the nature of the physical world in general, and offers up possibilities for the way the world might be, but the senses are needed to determine which among the possibilities are actual. See Hatfield (1986).

98   Alison Simmons Descartes, René (1664a). Le Monde de M. Descartes ou le traité de la Lumière. Paris. Originally written before 1637, but only published posthumously. Descartes, René (1664b). L’Homme. Paris. Originally written before 1637, but only published posthumously. Translation: Treatise on Man. T.S. Hall. Cambridge, MA: Harvard University Press, 1972. References are to section numbers in the French edition, which is reproduced in Hall. Galilei, Galileo (1623). Il Saggiatore. Rome. Translation: The Assayer in The Controversy on the Comets of 1618. Translated by S. Drake and C. D. O’Malley. Philadelphia: University of Pennsylvania Press, 1960. References are to pages in Drake and O’Malley. Hatfield, Gary (1986). ‘The Senses and the Fleshless Eye: The Meditations as Cognitive Exercises’. In A. Rorty (ed.), Essays on Descartes’ Meditations. Berkeley: University of California Press: 45–79. Hatfield, Gary (1992). ‘Descartes’s physiology and its relations to his psychology’. In J.  Cottingham (ed.), The Cambridge Companion to Descartes (pp. 335–370). New York: Cambridge University Press. Hobbes, Thomas (1655). Elementorum philosophiae section prima De Corpore. London. References are to chapter and section as I.1. Hobbes, Thomas (1658). Elementorum philosophiae section secunda De Homine. London. References are to chapter and section as I.1. Jolley, Nicholas (1990). The Light of the Soul: Theories of Ideas in Leibniz, Malebranche, and Descartes. Oxford: Clarendon. Jolley, Nicholas (1995). ‘Sensation, Intentionality, and Animal Consciousness: Malebranche’s Theory of the Mind’. Ratio (New Series), 7, 128–142. Kepler, Johannes (1604). Ad vitellionem paralipomena. Frankfurt. Translation: Optics: Paralipomena to Witelo & Optical Part of Astronomy. Edited and translated by W. Donahue. Santa Fe, NM: Green Lion Press, 2000. References are to chapter and section as 1.1. Leibniz, Gottfried (1695). ‘System mouveau de la nature et de la communication des substances, aussi bien que de l’union qu’il y a entre l’ame et le corps’. Journal des savants. Published anonymously. Leibniz, Gottfried ([1704]/1765). Nouveaux essays sur l’entendement humain. Amsterdam and Leipzig. Leibniz wrote the text in 1704, but suppressed publishing it when he heard of Locke’s death that year; it was published posthumously in 1765. References are to book, chapter, and section as I.i.1. Lennon, Thomas (2001). ‘Locke and the Logic of Ideas’. History of Philosophy Quarterly, 18(2), 155–177. Locke, John (1690). An Essay Concerning Human Understanding. London. References are to book, chapter, and section as I.i.1. Locke, John (1706). ‘An examination of P. Malebranche’s opinion of seeing all things in God’. In P. King (ed.), Posthumous Works of Mr. John Locke. London. Malebranche, Nicolas (1688). Entretiens sur la métaphysique et la religion. Paris. Translation: Dialogues and Metaphysics and Religion. Translated and edited by N. Jolley and D. Scott. Cambridge: Cambridge University Press, 1997. Malebranche, Nicolas (1712.) De la recherche de la vérité. Où l’on traitte de la nature de l’esprit de l’homme, et de l’usage qu’il en doit faire pour eviter l’erreur dans les sciences. 6th and final edition. Paris. First published in 1674–1675. Translation: The Search After Truth. Translated and edited by T. Lennon. Columbus, OH: Ohio State University Press, 1980. References to book, (part where applicable), and chapter as I.i.1.

Perception in Early Modern Philosophy    99 Nadler, Steven (1994). ‘Malebranche’s Theory of Perception’. In E. Kremer (ed.), The Great Arnauld (pp. 108–128). Toronto: University of Toronto Press. Nichols, Ryan (2007). Thomas Reid’s Theory of Perception. New York: Oxford University Press. O’Neil, Brian (1974). Epistemological Direct Realism in Descartes’ Philosophy. Albuquerque: University of New Mexico Press. Popkin, Richard (1979). History of Skepticism from Erasmus to Spinoza. Berkeley, CA: University of California Press. Reid, Thomas (1764). An Inquiry into the Human Mind on the Principles of Common Sense. Edited by D. R. Brookes. University Park: Pennsylvania State University Press, 1997. References are to chapter and section as I.1. Reid, Thomas (1785). Essays on the Intellectual Powers of Man. Edited by D. R. Brookes. University Park, PA: Pennsylvania University Press, 2002. References are to essay and chapter as I.1. Rorty, Richard (1979). Philosophy and the Mirror of Nature. Princeton: Princeton University Press. Simmons, Alison (1994). ‘Explaining Sense Perception: A Scholastic Challenge’. Philosophical Studies, 73, 257–275. Simmons, Alison (2003a). ‘Spatial Perception from a Cartesian Point of View’. Philosophical Topics, 31(1&2), 395–423. Simmons, Alison (2003b). ‘Descartes on the Cognitive Structure of Sensory Experience’. Philosophy and Phenomenological Research, 67(3), 549–579. Simmons, Alison (2008). ‘Guarding the Body: A Cartesian Phenomenology of Perception’. In P. Hoffman, D. Owen, and G. Yaffe (eds), Contemporary Perspectives on Early Modern Philosophy: Essays in Honor of Vere Chappell (pp. 81–113). Buffalo: Broadview Press. Simmons, Alison (2009). ‘Sensation in a Malebranchean Mind’. In J. Miller (ed.), Topics in Early Modern Philosophy of Mind (pp. 105–129). Dordrecht: Springer. Spinoza, Baruch (1677). Ethica: Ordine Geometrico demonstrata, et in quinque partes distincta. Amsterdam. Wahl, Russell (2004). ‘Malebranche—The Senses, Representation, and the Material World’. In R. Schumacher (ed.), Perception and Reality: From Descartes to the Present (pp. 108–121). Paderborn: Mentis. Wilson, Margaret (1982). ‘Did Berkeley Completely Misunderstand the Basis of the Primary– Secondary Quality Distinction in Locke?’ In C. M. Turbayne (ed.), Berkeley: Critical and Interpretive Essays (pp. 198–223). Minneapolis: University of Minnesota Press. Wilson, Margaret (1987). ‘Berkeley on the Mind-Dependence of Colors’. Pacific Philosophical Quarterly, 68(3–4), 249–264. Wilson, Margaret (1991). ‘Descartes on the Origin of Sensation’. Philosophical Topics, 19(1), 293–323. Wilson, Margaret (1994). ‘Descartes on Sense and "Resemblance"’. In J. Cottingham, Reason, Will, and Sensation: Studies in Cartesian Metaphysics (pp. 209–228). Oxford: Clarendon. Wolf-Devine, Celia (1993). Descartes on Seeing: Epistemology and Visual Perception. Carbondale, IL: Southern Illinois University Press. Yolton, John (1984). Perceptual Acquaintance from Descartes to Reid. Minneapolis: University of Minnesota Press; Oxford: Blackwell. Yolton, John (1996). Perception and Reality: A History from Descartes to Kant. Ithaca, NY: Cornell University Press.

Chapter 5

Perception i n Phil osoph y a n d Psychol ogy i n the 19th a n d E a r ly 20th Cen tu r ies Gary Hatfield

The latter nineteenth century and the first decades of the twentieth century was a period of intense investigation of sensory perception in Germany, Great Britain, and then the United States. New experimental techniques were applied to the study of perception, and there was lively debate within philosophy about the objects of perception, the character of perceptual knowledge, and the structure of the perceptual act. In sensory physiology and sensory psychology, psychophysical methods seemed to allow states of the soul itself to be measured. More generally, laboratory control was established over aspects of sensory perception, especially colour vision, spatial perception, and auditory perception. Investigators such as Wilhelm Wundt (1862, 1874), Hermann Helmholtz (1867), Ewald Hering (1868, 1874), and William James (1890) offered theoretical syntheses or overviews. Philosophically inclined scientists, including Helmholtz (1878) and Ernst Mach (1886), explored the implications of sensory physiology and psychology for the theory of knowledge, and they were answered by NeoKantians seeking to preserve some questions for philosophy. In Britain, scientifically informed philosophers, such as William Hamilton (1861) and John Stuart Mill (1865), debated questions about our knowledge of an external world. These debates sparked an explosion of philosophical argumentation about perception in Britain from 1890 on, which criss-crossed the Atlantic in the first decades of the new century. These discussions concerned not only whether we have knowledge of a mind-independent reality, but especially how such knowledge is constituted, what its objects might be, and whether such objects are directly as opposed to indirectly perceived. These empirical advances and philosophical discussions did not arise spontaneously or without precedent. The topic of perception had enjoyed attention throughout the seventeenth and eighteenth centuries and into the nineteenth, from both philosophers and natural philosophers, many trained as physicians. From this earlier work, the new experimentalists and the theorists of the latter nineteenth century inherited a complex

Perception in Philosophy and Psychology    101 of empirical questions, theoretical ideas, and philosophical problems. These concerned whether spatial perception is innate or learned, whether the fundamental processes in the psychology of vision involve calculative judgement or bare association, epistemological problems concerning perception of an external world, the distinction between primary and secondary qualities, and the interaction of such issues with theories of the relation between mind and body. This chapter begins with a sketch of the empirical, theoretical, and philosophical background, focusing on visual perception. It then considers in depth German sensory physiology and psychology in the nineteenth century and its reception. It offers a brief look at the interaction between perceptual theory and philosophical issues in epistemology and the metaphysics of mind in Britain and America, followed by an overview of psychological theories of perception in the early twentieth century and the Gestalt reaction, culminating with J. J. Gibson.

1 Philosophy and psychology of perception before 1850 Seventeenth-century discoveries set a new framework for theories of visual perception. Kepler’s discovery of the retinal image demanded a new physiology of visual stimulation and Newton’s discovery of the spectral composition of light radically altered the physics of colour vision. Kepler showed how a point-for-point, two-dimensional perspective image is cast upon an opaque surface in the eye. His theory replaced a ‘quasi-optical’ conception of processes in the optic nerve, in which Aristotelian ‘forms’ of light and colour in a two-dimensional arrangement are transmitted along a hollow optic nerve filled with translucent fluid. Kepler retained an ontology of forms and qualities but was undecided whether the image is sensed in the eye or conveyed into the brain and sensed there (Kepler 1604/2000, ch. 5.2).1 As the nerves came to be viewed mechanistically (and, subsequently, electrically or chemically), theorists had to describe how light (or other forms of sensory stimulation) could affect the nerves and how nerve activity that enters the brain results in a sensation or other sensory experience. Newton’s discovery in the 1660s of spectral composition—that white light is composed of component rays having differing refrangibility, which yield the various colours of the rainbow—gave a new physical basis for a theoretical outlook that was already gaining ground: that physical light is to be conceived as a mechanistic process that produces colour sensations as a subjective response in perceivers (see Newton 1704). A related view had been expressed as the distinction between primary and secondary qualities by Boyle and Locke (and, without using those terms, by Galileo and Descartes). Newton’s work added new knowledge of the physical characteristics of light and of their relation to the perceptual experience of colour.2 1 

2 

On Kepler’s optical theory and previous theories, see Lindberg (1976) and Hatfield and Epstein (1979). On early modern distinctions between primary and secondary qualities, see Nolan (2011).

102   Gary Hatfield These findings were accommodated within a larger framework of visual theory. Since antiquity, the science of optics had been a complete theory of vision, applying geometrical analysis not only to optical stimulation but also in accounts of size and distance perception. During the seventeenth century, debates arose concerning whether distance perception is acquired or innate, positions later dubbed ‘empirism’ and ‘nativism’. During the eighteenth and early nineteenth centuries, this discussion was incorporated into the medical and psychological literatures (Hatfield 1990, chs. 2–4). Philosophers, especially Descartes, Berkeley, and Reid, produced natural philosophical works on vision that engaged this issue and others, such as whether the processes underlying vision are fundamentally associational or include an unnoticed act of judgement or calculation. Generally, participants in these debates drew a distinction between an immediate object of vision—characterized as ‘sensation’ by some and consisting of a two-dimensional array of colours—and a percep­ tion that arises through associational or judgemental processes and includes an experience of distance.3 Some theorists, such as Descartes (1637, disc. 6), posited innate physiological mechanisms that directly yield an experience of distance (the third dimension). But the majority view in the eighteenth century was that the original and immediate product of visual stimulation is a two-dimensional array of colours reproducing the spatial structure of the retinal image.4 Philosophers raised questions about the ontology of sensory experience and about the relation between such experience and an external, material world. The proper interpretation of the resulting philosophical theories is a matter of ongoing debate. In my estimation (Hatfield 2009a), Descartes viewed all sensory ideas, including, in vision, the experience of both spatial properties and colour, as intrinsically representational. As had been usual since Aristotle, he considered sensory perception as functioning to preserve the health of the animal by allowing for bodily goods and evils to be detected and approached or avoided. Notoriously, he did not hold sensory experience to be the paradigm of knowledge (which was intellectual knowledge, including the purely intellectual contemplation of essences), but he did regard it as good enough for bodily preservation and as a source of observational knowledge in natural science. It is controversial whether Descartes was a critical direct realist, who held that we directly perceive external objects through sensory ideas, or a representative realist, who held that we perceive our ideas and infer external objects from them. This question is also disputed for Locke, but the grounds for regarding him as a representative realist may be stronger.5 Other positions in the metaphysics and epistemology of sensory perception were legion. Leibniz regarded body (matter) as a well-founded phenomenon, with well-foundedness consisting in the agreement of perceptual representations among monads, the individual substances 3 

On the notion of immediate or direct objects of vision, see Perler, Chapter 2, and Simmons, Chapter 4, in this volume. 4 On Descartes’ various accounts of distance perception, including a direct psychophysiological account tied to accommodation and convergence, see Hatfield (1992) and Wolf-Devine (1993). On theories of distance perception more generally, see Boring (1942), chs. 7–8; Pastore (1971), chs. 2–7; and Hatfield (1990), ch. 2. 5  On epistemological direct realism in Descartes, see O’Neil (1974) and Hoffman (2002). Whether Locke counts as a representative realist depends on the criterion. If it suffices to hold that a three-dimensional visual world is inferred from a two-dimensional sensation or sensory idea, Locke (1690), II.10, meets the criterion; if representative realism requires positing ideas as third things, distinct from minds, then the case is by no means clear (Rogers 2004).

Perception in Philosophy and Psychology    103 with which he populated his universe. Notoriously, monads do not have causal or other relations to one another and are not in space; rather, space is in them as a phenomenal structure. Berkeley supported a clearer and purer idealism, in which only minds, their ideas, and God exist; spatial structures exist only as objects of perceptual experience. Hume adopted an attitude of sceptical naturalism, in which our only sure knowledge is of sensory and emotive impressions and ideas, although he countenanced a natural belief in persisting bodies beyond the impressions. Kant famously limited knowledge to a phenomenal realm, caused by unknowable ‘things in themselves’ that appear to us according to our modes of perception but cannot be known as they are in themselves.6 These various positions (excluding Leibniz and Kant) were examined by Thomas Reid (1785), who contended that Berkeley’s idealism and Hume’s scepticism rested on a shared mistake. Reid read the major philosophers from Descartes to Hume as erroneously committed to a representative theory of perception. In his view, Berkeley and Hume simply revealed that any attempt to infer an external world from behind a veil of perception must fail. Reid offered instead a common-sense realism, in which perceivers naturally believe in and perceive a world of objects in three-dimensional space (natural belief supplants inference). Reid’s diagnosis, its interpretation, and its assessment were closely discussed in the nineteenth century, by the Scottish philosopher-psychologists Thomas Brown and William Hamilton, whose disagreements were examined by John Stuart Mill (1865). Brown rejected Reid’s diagnosis because it foisted an implausible form of representative realism onto figures such as Locke, who, in Brown’s view, spoke only of perceptions and their objects and did not interpose ideas as third things; for Brown’s Locke, ideas are perceptions (1820, 2: 8), a position he assimilates to Reid’s. In contrast, Hamilton interpreted Reid as saying that the mind is in direct and intuitive relation to the external world, unmediated by any mental state; consciousness takes an external entity directly as its object. For Hamilton (1861, 2: 130), this object was, in vision, the light on the retina. Finally, Mill (1865) argued instead that sensations are the immediate object of perception and that beliefs in external objects are in fact psychologically derived beliefs in ‘permanent possibilities of sensation’. Kant’s (1781, 1783) distinction between appearances and things in themselves was closely scrutinized in Germany. Even during Kant’s lifetime, critics complained about the role of the thing in itself in his philosophy, some finding it an unnecessary and unwelcome posit (and so, endorsing idealism proper, as with Fichte), others claiming that more could be known about the thing in itself. In the early decades of the nineteenth century, the philosophers Johann Friedrich Herbart and Jakob Friedrich Fries contended that the thing in itself, or a reality beyond perception, could be known (Hatfield 1990, ch. 4). During the nineteenth century, sensory physiology and sensory psychology in Germany was heavily influenced by Kant. His claim (1781, 1783) that space and time are a priori forms of sensibility, which condition all possible human experience, was given a psychological and naturalistic interpretation. For Kant, an a priori form of intuition need not be regarded as innate because it could unfold with experience.7 He invoked such forms in philosophical

6  On epistemological aspects of theories of perception from Descartes to Kant, see Yolton (1984, 1996); on Descartes to Reid, see also Simmons, Chapter 4, in this volume. 7  On Kant’s notion of a priori forms of intuition and their relation to notions of innateness and to spatial knowledge, see Falkenstein (1990); Hatfield (1990), ch. 3; 2006; and Kitcher (1995).

104   Gary Hatfield arguments about the grounds for knowledge, framed within his ‘transcendental philosophy’. Sensory physiologists and psychologists rendered questions about the status of spatial perception into empirical questions about the innateness of spatial representation. Johann Georg Steinbuch and Caspar Theobald Tourtual offered physiological and psychological theories to explain how spatial representation arises. Steinbuch (1811) espoused a radical empirism, according to which perceivers first acquire the ability to represent spatial structures and then to see things as arrayed in space through processes developed in early experience. Tourtual (1827) offered a nativist account of the origin of spatial representation and of three-dimensional visual perception. Some authors also naturalized the question of whether things in themselves can be known. Steinbuch in particular held that his account, in which spatial perception is acquired through causal interaction with external spatial objects so as to establish a correspondence between spatial perception and real spatial objects, could support the claim that we know things in themselves as they are. Among philosophically inclined scientists, the question of whether things in themselves are knowable continued through the nineteenth century, some finding them unknowable (Helmholtz 1878) and others holding that, because they would be unknowable, they are not needed (Mach 1886). The desire to avoid positing unknowable things in themselves was echoed in British thought.

2 German sensory physiology and psychology The nineteenth century was a golden age for sensory physiology and psychology, especially regarding vision, but also hearing and touch. Research blossomed after 1850, when Helmholtz, Wundt, Hering, G.  E. Müller, and others worked.8 German philosophical writers were aware of this work, whether they considered it to have strong implications for philosophy (as did Mach) or not (the feeling of NeoKantians such as Cohen and Natorp; see Edgar 2008). Scientific investigation of the senses explicitly considered theoretical questions about the relation between nerve activity and sensation, and the processes by which sensations (or elemental sensory representations) are combined to yield perceptions. Empirical techniques gained sophistication, as physicists and physiologists adapted or created instrumentation to allow precise measurement of subjects’ responses to stimuli. The best known are the psychophysical methods of E. H. Weber and Gustav Fechner, which include several experimental paradigms (reviewed below). Theoretically, a basic question concerned the relations among the physical properties of the stimulus, the inception and transmission of nerve activity, and the production of elementary sensations or representations. Although knowledge of the eye’s optical properties continued to be refined, the stimulus to vision was taken to be the retinal image. Earlier theorists had known that the images differ between the two eyes and that these differences are relevant for depth perception, but in the 1830s Wheatstone and Brewster

8 

On these figures, see Hatfield (1990), ch. 5; Turner (1994); and Heidelberger (2004).

Perception in Philosophy and Psychology    105 demonstrated that binocular disparity yields a powerful depth effect. Stereoscopic vision quickly became a major research topic in German sensory physiology.9 Another major topic concerned eye movements, with some theorists holding that the visual system uses eye motion to ‘measure’ visual space. As regards light, the physical agent of retinal stimulation, its wave or particle status was debated (waves were winning), but the basics of Newtonian spectral analysis were widely accepted, Goethe’s (1810) efforts to the contrary notwithstanding.

Nerve–sensation relations and spatial perception In psychological theories of vision, a crucial theoretical choice concerned the conception of the nerve–sensation juncture. Nineteenth-century theorists inherited the standard eighteenth-century conception that the primitive sensation of vision is a two-dimensional correlate of the retinal image. This was variously conceived. The most commonly accepted eighteenth-century position supposed that the two-dimensional sensation arises because the separate nerve fibres running from the retina into the brain retain their order. They project into the brain retinotopically, with each nerve producing a punctiform elemental sensation; these sensations are collectively experienced to have the same spatial order as do the nerve fibres (e.g. Gehler 1787–1796, 4: 11–12). Hence, they constitute a two-dimensional phenomenal image structure, which is what infants experience. Subsequently, as a result of learning to ‘interpret’ the various sources of information about depth and distance—including accommodation of the lens and convergence of the eyes (muscle cues), known size in conjunction with image size, awareness of continuous sequences of ground intervals, and atmospheric perspective and the clarity of details—this two-dimensional image is transformed (via unconscious or unnoticed psychological operations, whether associative or judgemental) into an experience of objects at a distance with a shape and size in three-dimensional space (Hatfield 1990, 35–44). Although this conception of the psychophysiological relations between nerves and sensations was standard fare in textbooks, another theory was proposed by the prominent physiologist Johannes Müller. To this standard theory he added the possibility that the retinal image may be ‘felt’ in its two-dimensional form at the retina and that this feeling may be transmitted via the nerves into the brain (Müller 1833–1840, 2: 350). Other theorists denied that the spatial order of the nerves, either in the retina or in the brain, could in itself account for the spatial order of vision. According to them, the mere spatial order of the nerves cannot determine the spatial order of sensations. Such theorists accepted that each nerve fibre produces a single sensation of colour, but they held that such sensations are intrinsically without spatial order, in just the way (they sometimes asserted) that tones can be heard with different pitches but without spatial location. The radical empirist Steinbuch (1811) held that a two-dimensional visual representation is first achieved through learning. The nativist Tourtual (1827) held that unordered sensations are, by means of ‘specific signs’ that mark the sensations of each nerve fibre, ordered into a two-dimensional representation according to an innately given law, and that depth 9  On early work on binocular vision, see Wade (1983, 2003); on early work in Germany, see Turner (1994), ch. 2.

106   Gary Hatfield and distance arise through a mixture of innate and learned factors. The physiologist and philosopher Hermann Lotze (1852) initially adopted a position similar to Tourtual’s (except that he called the nerve-specific markers ‘local signs’ and had the third-dimension arise entirely through learning), but he later endorsed an empiristic theory according to which perceivers must learn to localize sensations by ‘measuring’ positions through eye-movements and bodily motions, first for two dimensions and then for three (1886, 58–59).10 Despite their differences, these theorists agreed that a two-dimensional image constitutes a stage in the psychological process of visual perception (or does at some point during development). By contrast, the two major German visual theorists of the final third of the century, Helmholtz and Hering, saw no need for such a representation. Helmholtz, like Steinbuch, was a radical empirist, and he shared the assumption that individual visual nerves yield individual sensations that are not originally ordered in space. Unlike Steinbuch, he found no reason to posit a two-dimensional planar image as a representational stage: although ‘most physiologists have regarded it [a planar image] as the kind of vision which results most directly from sensation’ and have ‘looked upon ordinary solid vision as a secondary way of seeing things’, Helmholtz held that ‘solid vision’—that is, the three-dimensional perception of things in a direction at a distance—is the primary mode of representation (1868/1995, 185). For purposes of exposition, he allowed that one might speak of perceiving a ‘surface’ in the field of view. However, although perception of direction may be indefinite about distance, that fact does not call for a planar representation (whether the plane be flat or spherical). In actual perception, direction and distance are joined (1867/1924–25, 3: 235). Helmholtz’s nativist counterpart, Hering, also held that three-dimensional perception, or perception of things as in relief, is basic; but he attributed it to innate factors that are only fine-tuned through learning (Turner 1994, 65–66). The debate between Helmholtz and Hering and their followers over whether spatial perception is innate or learned raged on into the twentieth century.

Psychophysics and size constancy The development of psychophysics by Weber and Fechner is among the best-known achievements of German sensory psychology. The most widely discussed aspect of this work, which has become emblematic of it, is the empirical finding that just discriminable changes of intensity arise with a constant ratio of difference between the intensities of stimuli, a relation that holds across a wide variety of stimulus types. Weber obtained such ratios when he asked subjects to compare different weights and to judge the smallest perceptible differences between the pitches of tones (among other tasks). Reviewing this work, Fechner (1860) formulated ‘Weber’s law’, according to which the proportional relations between stimuli needed to produce a ‘just noticeable difference’ (or jnd) are constant (where R is the Reiz, or physical stimulus, ΔR/R = constant, for the jnd). In other words, the stimulus differences between just discriminable pitches are not

10 

On these theorists, see Hatfield (1990), ch. 4.

Perception in Philosophy and Psychology    107 absolute intervals but vary proportionately. Similarly, with weights there is no set amount (for example, 1 gram) that produces the least discriminable difference. We might barely discriminate a 4.0-gram from a 3.9-gram weight (absolute difference, 0.1 gram) but be unable to discriminate a 39.9-gram weight from a 40.0-gram weight (absolute difference, 0.1 gram). We should be able just to discriminate a 39-gram weight from a 40-gram weight, because the proportion 39/40 is preserved. Fechner transformed Weber’s law into Fechner’s law by assuming that each jnd differs in felt quality or intensity by the same amount as any other jnd and by assigning a value of 1 unit to any physical stimulus at the threshold of detection. Fechner’s law states that the magnitude of sensation (S) is equal to the logarithm of the stimulus times a constant (S = k log R).11 Fechner’s efforts spurred other quantitative work on sensation and perception, facilitating the declaration of a ‘new psychology’ founded on quantitative experimental methods. The best known of these is the ‘method of limits’, which seeks to establish the discriminable intervals (jnds) by regularly varying the intensity (or other stimulus dimension) upwards or downwards until the threshold of sensitivity is crossed. But Fechner offered other methods that also were influential. He described a ‘method of right and wrong cases’, which determines the threshold of stimulus sensitivity and the value of just-discriminable intervals by presenting stimuli randomly rather than in an orderly progression, as with the first method. Fechner’s third method, the ‘method of average error’, became widely adapted (with some modifications) in work on perception. In this method, a standard stimulus is presented and the subject’s task is to match a comparison stimulus to it so that they look the same (are phenomenally indistinguishable). In Fechner’s original conception, this would be done by continuously varying a comparison stimulus until a subjective (apparent) match is obtained. In studying spatial perception, the apparent spatial attributes of the standard are matched to a comparison stimulus. Thus, a subject might be presented with a horizontal white line of a certain length (say, 50 mm) and asked to adjust the length of a second white line off to the side so as to match the first one (Titchener 1905, 1: 70). After 100 trials, the subject’s deviations from the physical value are averaged, yielding the ‘constant error’ (which could be zero or another value, depending on how accurate the subject is). The experimenter also calculates a measure of the average variable error. The first value concerns how close the subject comes to making a physical match between the two stimuli, the second measures the variability of the subject’s responses, that is, the consistency among the settings. If a subject always achieved a perfect match, both error values would be zero. If the subject’s matches varied greatly, sometimes underestimating and sometimes overestimating, the constant error might still be small but the variable error would be large. The study of size constancy shows a trend away from the method of limits toward the method of average error and its descendants. Fechner had noted that perceived size does not vary directly with visual angle but has a tendency to remain constant despite moderate increases in the distance to the object, a phenomenon known as ‘size

11  Accounts of Fechner’s work, its relation to Weber, and its reception, may be found in Titchener (1905); Woodworth (1938), chs. 17–18; Boring (1942), 34–45; and Heidelberger (2004), ch. 6.

108   Gary Hatfield constancy’. Indeed, this observation is found frequently in the optical literature (e.g. in Ptolemy, Ibn al-Haytham, and Descartes) and was widely repeated in the nineteenth century (Ross and Plug 1998). In Wundt’s lab, Götz Martius (1889) studied the problem using psychophysical methods. Subjects matched standard rods of 20, 50, and 100 cm, viewed at 50 cm, to comparison stimuli placed at 300 and 575 cm. Martius varied the comparison stimuli by small increments (as in the method of limits) until a match was obtained. At 300 cm, subjects chose a size slightly larger than the standard as a match, and at the still farther distance they selected a somewhat larger comparison stimulus. From this it can be inferred that distant comparison objects that were equal in physical size to the nearby standard would look slightly smaller than that standard (hence a larger comparison object was needed to match phenomenally the nearby standard). The subjects came nowhere near to matching the retinal sizes or visual angles of the standard and comparison stimuli, which would require greatly enlarged comparison settings, as the visual angles subtended by comparison stimuli of equal physical size to the standard would be, respectively, 1/6 and less than 1/11 of that of the standard (so that, in order to match the standard in visual angle, the nearer of the comparison stimuli would need to be six times as large as the standard). Instead, their responses were closer to ‘size constancy’, that is, to matching a distant comparison to the objective size of the nearby standard. Subsequent authors adopted methods related to the method of average error, in which subjects adjust the comparison to match a standard. Franz Hillebrand (1902) investigated the orientation required of cords lying on a table to make them appear parallel to one another for a subject viewing them from one end. Here, the phenomenal parallel is an assumed standard supplied by the subject. In order to obtain a structure that appeared phenomenally parallel, subjects made the cords diverge. Hillebrand repeated the experiment with many threads hung vertically to yield phenomenal walls and had subjects adjust the walls so that they appeared parallel. Again, they were set to diverge with distance from the subject’s vantage point. The British psychologist Robert Thouless (1931, 1932) examined shape perception by having subjects make drawings of a standard circle or diamond that was rotated in depth, or match the standard to one of a series of nearby comparison shapes. Subsequent experiments on size and shape constancy in Britain and the United States provided variations on these themes, with subjects adjusting the comparison to match a standard or matching the standard to one of a series of comparison sizes or shapes (Epstein and Park 1963; Wagner 2006, 103). The utilization of such methods in studying size perception and other domains of sensory experience amounts to a form of controlled introspection that remains in use today.12

Colour matching and colour theories When Newton established that white light is composed of various spectral colours (separable by a prism), he opened the door to questions about how colours are formed by mixing spectral lights (or by mixtures of mixtures). Various eighteenth-century authors, predating

12 

On various types of introspection, see Titchener (1912) and Hatfield (2005).

Perception in Philosophy and Psychology    109 Thomas Young, speculated that all colours can be formed by mixing three primaries. Just after 1800, Young proposed red, yellow, and blue as primaries, but subsequently altered this list to red, green, and violet (Kaiser and Boynton 1996, 18–22).13 In the study of light, psychophysical methods can be applied to some stimulus relations, including changes of intensity. It had long been conjectured (Bouguer 1760/1961, 51) that discriminable changes in light intensity follow a proportional structure like that encoded in Weber’s law. For light intensity, the concept of altering the physical intensity and noting how the perceived intensity responds is analogous to other domains of psychophysics, in which one charts the relation between physically defined dimensions of variation and perceived values. The method of limits can also be applied to changes along the spectrum, by asking how far two spectral lights must be separated (in ‘refrangibility’ or, in modern terms, wavelength) to be perceived as different. With colour mixture, the logic of measuring the relation between a physically specified dimension of variation and a perceived value breaks down. In the context of nineteenth-century investigations (prior to any physical isolation and direct measurement of cone-pigment sensitivities), there was no prior basis for conjecturing what should happen when lights are mixed. The relevant dimensions of variation are not given by physics (as with intensity), but must themselves be discovered psychophysically. Young’s conjecture that there are three types of sensitive element in the retina that respond best to his three primaries (receptor trichromacy) was based on the empirical result that the primaries can be used to mix (most) other colours. Near the time of the advent of psychophysics, Helmholtz, Hermann Grassmann, and James Clerk Maxwell established laws of colour mixture (Boring 1942, ch. 4; Kaiser and Boynton 1996, chs. 1, 5). They showed, with some known exceptions, that any spectral light or mixture of lights can be matched by a mixture of three suitably chosen primaries. Which primaries are suitable, and how they have to be mixed in order to match a given light, is entirely an empirical matter. On the basis of subjects’ responses, these investigators formulated tables revealing laws of colour mixture. Accordingly, the concept of constant error has no meaning in relation to the physical properties of light that enter colour mixtures. Physical facts about light do not predict that mixtures of red and green light yield the perception of yellow. Thus, there is no meaning (prior to the establishment of an empirical law) to saying that the perceiver is in error, or not, in responding to the physical mixture as yellow; yellow is the ‘correct’ value only relative to a certain type of perceiver. Indeed, for much of evolution, mammals were dichromats and responded in a different manner to mixtures of spectral lights. Consequently, experiments in colour mixture are episodes of discovering the perceptual response characteristics of particular types of subjects. In this sense, colour is a subjective (subject-based) response—which is not incompatible with the objectivity of colour as a property of lights and objects (Hatfield 2003). For normal human trichromats, the average responses to various colour mixtures were worked out with great precision. As usual, the variable error (the distributed differences among one subject’s response to repeated stimulation, or across different subjects) is simply a measure of consistency of response. For colour-matching experiments, the variable error is remarkably small.

13  On these topics, see also Akins and Hahn, Chapter 22, this volume, on colour, and Ross, Chapter 21, this volume, on primary and secondary qualities.

110   Gary Hatfield In the latter half of the nineteenth century and well into the twentieth century, two different accounts of the physiology and psychology of colour vision vied for acceptance. The first, a psychological trichromacy theory, was made prominent by Helmholtz. According to this theory, experienced colours are the product of three types of sensations that are combined unconsciously to yield a perceived colour. One type of sensation corresponds to each of the three types of colour-sensitive elements in the human eye. Helmholtz postulated sensations of red, green, and violet (1868/1995, 158–63). Experienced colours arise from a mixture of the phenomenal characters of the three primary sensations; ‘sensation’ must here, as usually in Helmholtz, be regarded as a phenomenally characterized psychological state, not as mere neural activity. Phenomenal blue is a mixture of the phenomenally characterized sensations of green and violet, and yellow is a mixture of the sensations of red and green. Hering raised phenomenological objections against this theory, leading to his opponent-process theory. He noted that, phenomenally and psychologically, yellow seems to be a primary colour, on a par with red, blue, and green (Hering 1905–11/1964, 41–50). Yellow does not seem to be a mixture of green and red; indeed, it is difficult to see how it could be regarded as such, phenomenally. Hering therefore posited that there are four types of receptors involved in colour vision (‘chromatic’ colour vision, leaving aside black and white) that feed into two physiological channels, a red–green channel and a blue–yellow channel. Each channel responds in an ‘opponent’ manner: the red–green channel may produce either red or green, but not both, which means that it can be physiologically driven toward red or green. When balanced, it produces no effect, and the chroma is determined by the other channel (blue or yellow). These channels can interact to yield the physiological basis for various experienced colours: if a red-process in the one channel interacts with a yellow in the other, phenomenal orange results. This does not come about through a mixture of phenomenally characterized sensations, but through a physiological combination that then produces an experience of orange (Hering 1905–11/1964, ch. 10; Turner 1994, ch. 7). The Hering and Helmholtz schools confronted one another well into the twentieth century (Turner 1994, chs. 10–11, 14; Hatfield 2009b). Today’s handbooks portray a resolution to this controversy through ‘two stage’ or ‘zone’ theories (Kaiser and Boynton 1996, 24–25). In such a theory, Helmholtz gets credit for being right about the three cone-types (retinal receptor trichromacy), while Hering’s opponent theory describes subsequent processes that underlie visual experience. There is something correct about this picture, and even Hering (1905–11/1964, 48 n. 2) seems to imply that such a division might allow some parts of the two theories to coexist. Historically speaking, however, this cannot count as ‘both theories being right’. For the theories to coexist, the Helmholtz theory must concede that the trichromatic processes are merely physiological, feeding into subsequent opponent processes; hence, the individual receptor types do not produce three fundamental phenomenally characterized sensations that are then mentally combined to yield a perceived colour. For the Hering school, this means that a fundamental tenet of Helmholtz’s theory, his generation of all hues by combining three basic sensations, must be abandoned in favour of Hering’s physiological theory (Hurvich and Jameson 1951; Turner 1994, 229). There are other dimensions of thought in German sensory psychology and related philosophical works, including the rise of the phenomenological approach of Franz Brentano (1874) and Mach’s (1886) analysis of sensations. These approaches will be regarded through their British and American receptions.

Perception in Philosophy and Psychology    111

3 Perception in British and American philosophy and psychology Nineteenth-century British writings on perception continued the controversy between Hume and Reid over the belief in and perception of an external world. Most authors accepted Berkeley’s theory that the immediate object of vision is a two-dimensional correlate of the retinal image, and that the perception of depth and distance by sight is acquired through interaction with touch (but Abbott 1864 rejected both tenets). The theory that touch educates vision was further refined by British thinkers through the development of the concept of the muscle sense as a sixth sense beyond the traditional five. Thomas Brown provided an early statement of this theory (Scheerer 1987). He ascribed the origin of the idea of extension not to touch or vision, but to the infant’s experience of moving its arm in three dimensions and feeling, through an awareness of muscle-induced motions, the three-dimensionality of the limb’s path. Brown thereby proposed an empirism as radical as that of Steinbuch or Helmholtz. He pre-dated Helmholtz in foregoing construction of a two-dimensional image in sensation. While acknowledging that, since Berkeley, theorists had ‘universally supposed’ that superficial extension, or visible figure, is immediately available to sight and subsequently serves as the basis for learning to see distance, Brown contended that we learn to see in three dimensions without passing through a stage with a two-dimensional image (1820, 2: 66–97). Brown also speculated that, through the surprise engendered when the usually free motion of an infant’s limb is resisted by an external object, the infant acquires the conception of and belief in an external world (1820, 1: 511–517). Alexander Bain (1855), in one of the first works of the ‘new psychology’ in Britain, offered an account of spatial vision and the belief in an external world bearing similarity to Brown’s theory. The problem of the external world was extensively discussed in Britain throughout the second half of the nineteenth century. Hamilton (1861), intending to support Reid (though perhaps misinterpreting him), held that in perceptual consciousness we are directly aware of both the act of perception and a non-mental object of that act. In the case of vision, we have seen that he regarded the retinal image itself, as a physical image of light, to be the immediately discerned external object. He dubbed his position an ‘intuitive’ theory of perception (1861, 2: 66–71), by contrast with the ‘representative’ theory he (with Reid) ascribed to Locke (ibid., 53–59). He also called it a ‘natural realism’ (ibid., 65). Although challenged by Mill (1865), Hamilton’s position was repeated with some variation by the Oxford philosophers Thomas Case (1888) and L. T. Hobhouse (1896). The notion that perception (as cognitive) is structured so as to include an act directed upon a distinct object became widely accepted, even if divorced from Hamilton’s ‘intuitive’ natural realism. The act–object structure was taken to embrace all mental phenomena by Brentano (1874/1995, 89–91), who was a close student of Hamilton’s writings, and whose writings in turn attracted the attention of G. F. Stout (1896). The early sense-data theories of Moore and Russell accepted an act–object structure, while rejecting Hamilton’s ‘natural realism’ and adopting, at certain times at least, a representative theory of perception.14 14  On the problem of the external world in British philosophy from Brown to Russell, see Hatfield (2013b).

112   Gary Hatfield The question of whether perception is direct or mediated was widely discussed by British and American philosophers in the early decades of the twentieth century. The positions they held included: (1) naïve direct realism, which held that the immediate objects of perception are external material things (Alexander 1909–10, 2; Cook Wilson 1926, 780); (2) representative realism, which in a standard sense-data version rendered an interposed third thing as the object (Russell 1912); (3) neutral monism, which reduced both mind and body to constructions out of neutral particulars (James 1904; Russell 1921); and (4) critical direct realism, which regarded perceptions as mental states that are not themselves the objects of perception but are the means by which we become directly aware of external objects (Sellars 1920, 1961).15

4 Psychology of perception after 1900 In the nineteenth century, the psychology of visual perception witnessed a great atomizing tendency, toward positing non-spatial colour sensations as the primitive sensations in vision.16 From Steinbuch to Helmholtz, this outlook reigned. Hering (1868), Mach (1886), and James (1890) started a counter-reaction, arguing that spatiality is originally given in vision. The atomizing account was paired with another theoretical tendency, which was to see visual spatial perception as the constructed product of unconscious or unnoticed processes of association. Early in the twentieth century, the Gestalt psychologists mounted an ever more effective attack on this unholy pair. Köhler (1913) and subsequently Koffka (1935, 84–87) insisted that the associational processes were only needed because of the hypothesis of atomic sensations, which was itself allied to a particular conception of the relation between nerves and phenomenal sensations: a one-fibre, one-sensation view that they labelled the ‘constancy hypothesis’.17 In place of this web of ideas, they constructed a theory in which three-dimensionality is the primary mode of visual experience (there is no two-dimensional-image stage); visual experience tends toward organization (as in figure– ground structures, or grouping of proximate or similar items); and these aspects of visual experience are produced by (or are identical with) brain processes that are ‘isomorphic’ in structure with the experiences (Koffka 1935; Köhler 1947). The Gestaltists also emphasized that perception tends to track objective properties of things, as in size constancy. They refused to treat the stimulus to vision as a coherent (unified) two-dimensional image, 15  On sense-data, perception, and various ‘realisms’ from 1860 to 1950, see Hatfield (2013a). Price (1932, ch. 3) called naïve realists such as Alexander and Cook Wilson ‘modified naïve realists’ because they took into account perceptual relativity while affirming that the perceiver immediately apprehends objects at a distance. Mach (1886) is not included among the neutral monists, as his restriction to phenomenal elements was epistemic, whereas James (1904) and Russell (1921) proposed a fundamental metaphysics of ‘neutral stuff’ or ‘momentary particulars’. 16  This psychological atomism can be found earlier in Hume (1739–40), bk. 1, pt. 2, sec. 3. 17  Here, the term ‘constancy’ describes the relation between an isolated neural response to a stimulus and the resultant sensation; its connotation is opposite to that of ‘size constancy’, in which the visual system’s global response tracks the properties of external objects rather than those of local proximal stimulation. For this reason, Gilchrist (2012, 107)  suggests renaming the constancy hypothesis as ‘the doctrine of local determination’.

Perception in Philosophy and Psychology    113 but rather regarded it as a mosaic of independent light intensities that individually interact with neural processes but are not themselves perceived. Stimulation from the eyes conditions central neural processes that produce an ordered perceptual world in three dimensions. Order arises only in perception; it is not given in the mosaic. The final assault on the notion of a static retinal image as the stimulus for vision, and of a processing stage that consists of a two-dimensional representation of the visual field, came through the work of J. J. Gibson, himself a fan of the Gestaltists. Gibson (1950) contended that perception of a visual world in three dimensions is the primary and normal mode of perception. There is no two-dimensional stage of processing; the two-dimensional visual field is an atyptical experience that arises from adopting a particular viewing attitude, sometimes called the ‘painter’s attitude’. Considered as a perceptual system, the visual system is built to track the three-dimensional visual world. It does so by responding to dynamic patterns of change within the optic array (Gibson 1950, 1966). Gibson argued that associative or cognitive processes are not needed to respond to the higher-order, complex features of the optic array, as these features are rich enough to specify directly the three-dimensional visual world. Gibson’s outlook is controversial, especially his denial of the need for psychologically interesting mechanisms to respond to the optic array (he granted the need for physiological mechanisms of reception). But however one reacts to that aspect of his theory, the notion that the stimulus to vision is a complex and dynamic pattern of energy, as received by a visual system that includes mobile eyes and a mobile body, has entered the mainstream of visual theory. The notions of a static two-dimensional image and of spatially atomized sensations have been surpassed. The closest counterparts to the latter are the many specialized detectors with small ‘receptive fields’, such as disparity detectors, edge detectors, or motion detectors, that encode retinal stimulation. But even here, the problem of perception lies in understanding how the pattern of activation in locally responsive neurons can be synthesized to yield a global representation of surfaces at a distance, as in Marr’s (1982) 2½ D sketch. The problem of how to theorize the processes that respond to these mechanisms as they encode dynamic patterns of optical stimulation is ongoing.18 Still, the old idea that perception begins from a static 2D image is a thing of the past. From dynamic patterns of information, we see a world of three dimensions, organized into discrete regions according to Gestalt figure–ground relations and through processes of object-recognition. The landscape of visual theory is transformed, which should mean that the philosophical landscape includes new territories and possibilities.

References Abbott, Thomas K. (1864). Sight and Touch: An Attempt to Disprove the Received (or Berkeleian) Theory of Vision. London: Longman, Green, Longman, Roberts, and Green. Alexander, Samuel (1909–10). ‘On Sensations and Images’, Proceedings of the Aristotelian Society, 10, 1–35. 18 

For a survey of recent visual theories, including the legacy of Gestalt, ‘constructivist’ (judgmental or associationist), and Gibsonian approaches, see Palmer (1999), chs. 1–2; for an appreciation of major theoretical trends in relation to Koffka and the Gestaltists, see Epstein (1994).

114   Gary Hatfield Bain, Alexander (1855). The Senses and the Intellect. London: Parker. Boring, Edwin G. (1942). Sensation and Perception in the History of Experimental Psychology. New York: Appleton-Century-Crofts. Bouguer, Pierre (1760). Traité d’optique sur la gradation de la lumière. Paris:  Guerin & Delatour. Translation: Pierre Bouguer’s Optical Treatise on the Gradation of Light, (trans.) W. E. Knowles Middleton. Toronto: University of Toronto Press, 1961. Brentano, Franz Clemens (1874). Psychologie vom empirischen Standpunkt. Leipzig: Duncker and Humblot. Translation:  Psychology from an Empirical Standpoint, (trans.) Antos C. Rancurello, D. B. Terrell, and Linda L. McAlister. London: Routledge, 1995. Brown, Thomas (1820). Lectures on the Philosophy of the Human Mind, 4 vols. Edinburgh: Tait. Case, Thomas (1888). Physical Realism, Being an Analytical Philosophy from the Physical Objects of Science to the Physical Data of Sense. London: Longmans, Green. Cook Wilson, John (1926). ‘Letter in criticism of a paper on primary and secondary qualities’. In Statement and Inference, with Other Philosophical Papers, 2 vols. Oxford: Clarendon Press, 764–800. Descartes, René (1637). La Dioptrique, published with his Discours de la méthode. Leiden: Maire. Edgar, Scott (2008). ‘Paul Natorp and the Emergence of Anti-Psychologism in the Nineteenth Century’. Studies in History and Philosophy of Science, 39, 54–65. Epstein, William (1994). ‘ “Why do things look as they do?”: What Koffka Might have Said to Gibson, Marr, and Rock’. In Stefano Poggi (ed.), Gestalt Psychology: Its Origins, Foundations and Influence. Florence: Olschki, 175–189. Epstein, William, and Park, John N. (1963). ‘Shape Constancy: Functional Relationships and Theoretical Formulations’. Psychological Bulletin, 60, 265–288. Falkenstein, Lorne (1990). ‘Was Kant a Nativist?’. Journal of the History of Ideas, 51, 573–597. Fechner, Gustav Theodor (1860). Elemente der Psychophysik. Leipzig: Breitkopf und Härtel. Partial translation: Elements of Psychophysics: Volume 1, (trans.) H. E. Adler. New York: Holt, 1966. Gehler, Johann Samuel Traugott (1787–96). Physikalisches Wörterbuch, oder Versuch einer Erklärung der vornehmsten Begriffe und Kunstwörter der Naturlehre, 6 vols. Leipzig: Schwickert. Gibson, James J. (1950). The Perception of the Visual World. Boston, MA: Houghton Mifflin. Gibson, James J. (1966). The Senses Considered as Perceptual Systems. Boston, MA: Houghton Mifflin. Gilchrist, Alan (2012). ‘Objective and Subjective Sides of Perception’. In Gary Hatfield and Sarah Allred (eds), Visual Experience: Sensation, Cognition, and Constancy. Oxford: Oxford University Press, 105–121. Goethe, Johann Wolfgang von (1810). Zur Farbenlehre. Tübingen: Cotta. Translation: Goethe’s Theory of Colours, (trans.) Charles Lock Eastlake. London: Murray, 1840. Hamilton, William (1861). Lectures on Metaphysics, (ed.) Henry L. Mansel and John Veitch, 2nd edn, 2 vols. Edinburgh: Blackwood. Hatfield, Gary (1990). The Natural and the Normative: Theories of Spatial Perception from Kant to Helmholtz. Cambridge, MA: MIT Press. Hatfield, Gary (1992). ‘Descartes’s Physiology and its Relation to his Psychology’. In John Cottingham (ed.), Cambridge Companion to Descartes. Cambridge: Cambridge University Press, 335–370. Hatfield, Gary (2003). ‘Objectivity and Subjectivity Revisited: Colour as a Psychobiological Property’. In Rainer Mausfeld and Dieter Heyer (eds), Colour Perception: Mind and the Physical World. Oxford: Oxford University Press, 187–202.

Perception in Philosophy and Psychology    115 Hatfield, Gary (2005). ‘Introspective Evidence in Psychology’. In Peter Achinstein (ed.), Scientific Evidence: Philosophical Theories and Applications. Baltimore: Johns Hopkins University Press, 259–286. Hatfield, Gary (2006). ‘Kant on the Perception of Space (and Time)’. In Paul Guyer (ed.), Cambridge Companion to Kant and Modern Philosophy. Cambridge: Cambridge University Press, 61–93. Hatfield, Gary (2009a). ‘The Sixth Meditation: Mind-Body Relation, External Objects, and Sense Perception’. In Andreas Kemmerling (ed.), Meditationen über die Erste Philosophie. Berlin: Akademie, 123–146. Hatfield, Gary (2009b). ‘What Can the Mind Tell Us about the Brain? Psychology, Neurophysiology, and Constraint’. In Perception and Cognition: Essays in the Philosophy of Psychology. Oxford: Clarendon Press, 434–455. Hatfield, Gary (2013a). ‘Perception and Sense Data’. In Michael Beaney (ed.), The Oxford Handbook of the History of Analytic Philosophy. Oxford: Oxford University Press. Hatfield, Gary (2013b). ‘Psychology, Epistemology, and the Problem of the External World: Russell and Before’. In Erich Reck (ed.), The Historic Turn in Analytic Philosophy. London: Palgrave Macmillan, 171–200. Hatfield, Gary and Epstein, William (1979). ‘The Sensory Core and the Medieval Foundations of Early Modern Perceptual Theory’. Isis, 70, 363–84. Heidelberger, Michael (2004). Nature from Within:  Gustav Theodor Fechner and His Psychophysical Worldview, (trans.) Cynthia Klohr. Pittsburgh, NJ:  University of Pittsburgh Press. Helmholtz, Hermann von (1867). Handbuch der physiologischen Optik. Leipzig: Voss. As republished in the 3rd German edn, 3 vols. Leipzig: Voss, 1910. Translation: Helmholtz’s Treatise on Physiological Optics, (ed. and trans.) James P. C. Southall, 3 vols. Rochester, NY: Optical Society of America, 1924–25. Page citations are to the 3rd German edn; these numbers are shown in the Southall translation. Helmholtz, Hermann von (1868). ‘Die neueren Fortschritte in der Theorie des Sehens’. Preussische Jahrbücher, 21, 149–170. Translation: ‘Recent Progress in the Theory of Vision’, in Helmholtz (1995), 127–203. Citations are to the translation. Helmholtz, Hermann von (1878). Die Thatsachen in der Wahrnehmung. Berlin: Hirschwald. Translation: The Facts in Perception, in Helmholtz (1995), 342–380. Helmholtz, Hermann von (1995). Science and Culture: Popular and Philosophical Essays by Hermann von Helmholtz, (ed.) David Cahan. Chicago, IL: University of Chicago Press. Hering, Ewald (1868). Die Lehre vom binocularen Sehen. Leipzig: Engelmann. Translation: The Theory of Binocular Vision, (trans.) Bruce Bridgeman. New York: Plenum Press, 1977. Hering, Ewald (1905–11). Grundzüge der Lehre vom Lichtsinn. Leipzig:  Engelmann. Translation: Outlines of a Theory of the Light Sense, (trans.) Leo M. Hurvich and Dorothea Jameson. Cambridge: Harvard University Press, 1964. Hering, Ewald (1874). Zur Lehre vom Lichtsinne. Vienna: Gerold’s Sohn. Hillebrand, Franz (1902). ‘Theorie der scheinbaren Grösse beim binokularen Sehen’. Denkschrift der Kaiserlichen Akademie der Wissenschaften Wien, mathematischnaturwissenschaftliche Classe, 72, 255–307. Hobhouse, L. T. (1896). Theory of Knowledge. London: Methuen. Hoffman, Paul (2002). ‘Direct Realism, Intentionality, and the Objective Being of Ideas’. Pacific Philosophical Quarterly, 83, 163–179. Hume, David (1739–40). Treatise of Human Nature. London: John Noon.

116   Gary Hatfield Hurvich, Leo M. and Jameson, Dorothea (1951). ‘The Binocular Fusion of Yellow in Relation to Color Theories’. Science, 114, 199–202. James, William (1890). Principles of Psychology, 2 vols. New York: Holt. James, William (1904). ‘Does “Consciousness” Exist?’, in Journal of Philosophy, Psychology and Scientific Methods, 1, 477–491. Kaiser, Peter K. and Boynton, Robert M. (1996). Human Color Vision, 2nd edn. Washington DC: Optical Society of America. Kant, Immanuel (1781). Kritik der reinen Vernunft. Riga: Hartnoch. Kant, Immanuel (1783). Prolegomena zu einer jeden künftigen Metaphysik. Riga: Hartnoch. Kepler, Johannes (1604). Ad vitellionem paralipomena. Frankfurt:  Marnium & Aubrii. Translation: Optics: Paralipomena to Witelo and Optical Part of Astronomy, (ed. and trans.) William H. Donahue. Santa Fe, NM: Green Lion Press, 2000. Kitcher, Patricia (1995). ‘Revisiting Kant’s Epistemology: Skepticism, Apriority, and Psychologism’. Noûs, 29, 285–315. Koffka, Kurt (1935). Principles of Gestalt Psychology. New York: Harcourt, Brace. Köhler, W. (1913). ‘Über unbemerkte Empfindungen und Urteilstauschungen’. Zeitschrift für Psychologie, 66, 51–81. Translation: ‘On Unnoticed Sensations and Errors of Judgment’, (trans.) H. E. Adler, in Selected Papers of Wolfgang Köhler, (ed.) Mary Henle, 13–39. New York: Liveright, 1971. Köhler, W. (1947). Gestalt Psychology: An Introduction to New Concepts in Modern Psychology. New York: Liveright. Lindberg, David C. (1976). Theories of Vision from al-Kindi to Kepler. Chicago, IL: University of Chicago Press. Locke, John (1690). An Essay Concerning Human Understanding. London: Bassett. Lotze, Rudolph Hermann (1852). Medicinische Psychologie, oder, Physiologie der Seele. Leipzig: Weidmann. Lotze, Rudolph Hermann (1886). Outlines of Psychology, (trans.) George T. Ladd. Boston: Ginn. Mach, Ernst (1886). Beiträge zur Analyse der Empfindungen. Jena:  Fischer. Translation: Contributions to the Analysis of Sensations, (trans.) C. M. Williams. La Salle, IL: Open Court, 1897. Marr, David (1982). Vision: Computational Investigation into the Human Representation and Processing of Visual Information. San Francisco, CA: Freeman. Martius, Götz (1889). ‘Ueber die scheinbare Grösse der Gegenstände und ihre Beziehung zur Grösse der Netzhautbilder’. Philosophische Studien, 5, 601–617. Mill, John Stuart (1865). An Examination of Sir William Hamilton’s Philosophy. London: Longman, Green, Longman, Roberts & Green. Müller, Johannes (1833–40). Handbuch der Physiologie des Menschen. Coblenz: Hölscher. Newton, Isaac (1704). Opticks, or a Treatise of the Reflexions, Refractions, Inflexions and Colours of Light. London: Smith and Walford. Nolan, Lawrence (ed.) (2011). Primary and Secondary Qualities: The Historical and Ongoing Debate. Oxford: Oxford University Press. O’Neil, Brian (1974). Epistemological Direct Realism in Descartes’ Philosophy. Albuquerque, NM: University of New Mexico Press. Palmer, Stephen E. (1999). Vision Science:  Photons to Phenomenology. Cambridge, MA: MIT Press. Pastore, Nicholas (1971). Selective History of Theories of Visual Perception:  1650–1950. New York: Oxford University Press.

Perception in Philosophy and Psychology    117 Price, H. H. (1932). Perception. London: Methuen. Reid, Thomas (1785). An Inquiry into the Human Mind, 4th edn. Edinburgh: Bell and Creech. Rogers, G. A. J. (2004). ‘Locke and the Objects of Perception’. Pacific Philosophical Quarterly, 85, 245–254. Ross, Helen E. and Plug, Cornelis (1998). ‘The History of Size Constancy and Size Illusions’. In Vincent Walsh and Janusz Kulikowski (eds), Perceptual Constancy: Why Things Look As They Do. Cambridge: Cambridge University Press, 499–528. Russell, Bertrand (1921). Analysis of Mind. London: Allen and Unwin. Russell, Bertrand (1912). Problems of Philosophy. London: Williams and Norgate. Scheerer, Eckart (1987). ‘Muscle Sense and Innervation Feelings: A Chapter in the History of Perception and Action’. In Herbert Heuer and Andries F. Sanders (eds), Perspectives on Perception and Action (pp. 171–194). Hillsdale, NJ: Erlbaum. Sellars, Roy Wood (1920). ‘Knowledge and its Categories’. In Durant Drake (ed.), Essays in Critical Realism (pp. 187–219). London: Macmillan. Sellars, Roy Wood (1961). ‘Referential Transcendence’. Philosophy and Phenomenological Research, 22, 1–15. Steinbuch, Johann Georg (1811). Beytrag zur Physiologie der Sinne. Nurnberg: Schragg. Stout, G. F. (1896). Analytic Psychology, 2 vols. London: Swan Sonnenschein. Thouless, Robert H. (1931). ‘Phenomenal Regression to the Real Object, I’. British Journal of Psychology, 21, 339–359. Thouless, Robert H. (1932). ‘Phenomenal Regression to the “Real” Object, II’. British Journal of Psychology, 22, 1–30. Titchener, Edward Bradford (1905). Experimental Psychology:  A  Manual of Laboratory Practice, Vol. 2: Quantitative Experiments. London: Macmillan. Published in two parts. Titchener, Edward Bradford (1912). ‘The Schema of Introspection’. American Journal of Psychology, 23, 485–508. Tourtual, Caspar T. (1827). Die Sinne des Menschen in den wechselseitigen Beziehungen ihres psychischen und organischen Lebens:  Ein Beitrag zur physiologischen Aesthetick. Münster: Coppenrath. Turner, R. Steven (1994). In the Eye’s Mind: Vision and the Helmholtz–Hering Controversy. Princeton, NJ: Princeton University Press. Wade, Nicholas J., (ed.) (1983). Brewster and Wheatstone on Vision. London: Academic Press. Wade, Nicholas J. (2003). Destined for Distinguished Oblivion: The Scientific Vision of William Charles Wells (1757–1817). New York: Kluwer. Wagner, Mark (2006). Geometries of Visual Space. Mahwah, NJ: Erlbaum. Wolf-Devine, Celia (1993). Descartes on Seeing:  Epistemology and Visual Perception. Carbondale, IL: Southern Illinois University Press. Woodworth, Robert (1938). Experimental Psychology. New York: Henry Holt. Wundt, Wilhelm (1862). Beiträge zur Theorie der Sinneswahrnehmung. Leipzig: Winter. Wundt, Wilhelm (1874). Grundzüge der physiologischen Psychologie. Leipzig: Engelmann. Yolton, John W. (1984). Perceptual Acquaintance from Descartes to Reid. Minneapolis, MN: University of Minnesota Press. Yolton, John W. (1996). Perception and Reality:  A  History from Descartes to Kant. Ithaca, NY: Cornell University Press.

Chapter 6

Sense-Data Paul Snowdon

The term ‘sense-datum’ entered the language of philosophy towards the end of the nineteenth century, and employments of it by Royce and William James (in 1882 and 1890 respectively) are cited, in dictionaries, as early examples of its use. It was then taken up by Russell and Moore and became a part of standard philosophical terminology, retaining a central place in the vocabulary employed by philosophers of perception ever since. Despite holding this central place the term ‘sense-datum’ has had a somewhat chequered career. It has had, or perhaps it is useful to think of it as having had, three main interpretations. According to the first two, we are to understand ‘sense-datum’ as standing for an object, of some kind or other. These two attitudes might be divided into a dominant and a less dominant interpretation. On the dominant interpretation the words were linked to a particular conception of the objects of perceptual experience (or the objects involved in perceptual experience), a conception which had existed amongst philosophers (and scientists) for a long time before this particular terminology emerged. In earlier times this conception had been expressed using different terms, notably in the use of the word ‘idea’ to stand for what we (directly) perceive. On what I shall call the less dominant object interpretation, the term ‘sense-datum’ was used to express a supposed conception of an object that it is undeniably present in perception, the only question being what kind of object it is. People using the term in this second way quite often ended up ultimately agreeing or being close to agreeing with those who used it in the first way, but at least they reserved the option of using the term ‘sense-datum’ in a positive way without ending up affirming the traditional view. Moore, who championed this second use, spent his entire philosophical life wondering whether to take this option (or something like it) or not.1 Towards the middle of the century the third usage emerged, associated with A. J. Ayer. According to this usage it is somehow beneficial to employ the terminology of ‘sense-datum’ as part of a stipulated re-expression of appearance claims, where its employment in such a re-expression does not signal the postulation of entities of any kind whatsoever. The principal question that this last development raised was what advantage for philosophy was generated by such a linguistic stipulation or recommendation. I shall describe these different employments in a little more detail below. These second two moves away from 1 

Moore’s version of what I am calling the less dominant interpretation is described in section 3.

Sense-Data   119 the dominant interpretation are both attempts to reduce the theoretical commitments of ‘sense-data’ talk. Very roughly, in the first half of the twentieth century the dominant research strategy in the philosophy of perception was the development of approaches to perception centred on what we might call the postulation of sense-data, by philosophers such as Russell, Moore, Broad, Price, and Ayer. In the second half of the century the tide turned and the dominant research goal was to think about perception in a way that avoided their postulation—an approach endorsed and developed by Wittgenstein, Austin, Ryle, Armstrong, and Pitcher, amongst many others.2 As with all tides, there have been those swimming against this later one, such as Frank Jackson, Brian O’Shaughnessy, and Howard Robinson, just as there in fact had been those who swam against the earlier tide as well. The debates about this type of theory are very familiar to anyone who has worked on the philosophy of perception, and it is unlikely that anything significantly new can be said about them. My hope here is to present and structure this rather familiar material in a way that illuminates and clarifies these debates. Let us begin, then, by outlining the all too familiar and very long-standing model, or theory, to which the sense-datum terminology has been dominantly assigned.

1  The dominant sense-datum model The types of occurrence to start with are those regular events which we naively think of as events of perceiving the world, say, seeing the book you currently think you are seeing. We can here, in our choice of example, but also explicitly, register the fact that philosophers have tended to think primarily about vision when they consider perceptual experience, a practice I shall follow initially. Now, we, normal people, think of that event as one which enables us to pick out, or discern, an object—the book—which is a regular, three-dimensional external object. That is the object which the occurrence enables us to scrutinize and focus on. It is also thought of as a public object, in the sense that it is there for others also to perceive and see. It is part of our shared and communally perceivable environment. But according to the model of this event that philosophers accepted it actually, and in truth, involves the subject standing in a mental or psychological relation which was called

2  As a rough piece of history we can say that once the opposition to sense-data became a common shared assumption, the first response was to be fairly untheoretical about perception and simply oppose the sense-datum account, a description that applies to Austin, and perhaps Wittgenstein. Once the theorizing ambitions returned an early approach was that of the belief theorists. It is obvious that the dominant constraint on that sort of theory was the avoidance of sense-data; these were avoided by taking the psychological primitive in their analysis to be belief, a state the presence of which does not seem to require sense-data. The second advantage of belief is that it is a state with content, and hence a state which promises the possibility of grounding the sort of content that appearances involve. A crucial drawback to basing an analysis on belief is that belief seems to be a psychological state which can be present without involving experiences, whereas perception seems to be an experience. It is therefore not surprising that the next idea to emerge was an analysis of perception based around a distinctive experience-involving contentful state.

120   Paul Snowdon apprehending, or perceiving, or directly perceiving, or being acquainted with (a relation clearly patterned on that of seeing itself, as we understand it) to a quite different sort of object, to which the name ‘sense-datum’, once it emerged, was assigned.3 What is important is the theoretical conception of this postulated type of object or item. Now, by a theoretical conception of an object is meant the general properties it is supposed to possess and the fundamental conditions or modes of existence it has according to the theory. In the case of sense-data, the item was thought of as possessing certain sensible qualities—in the case of visual experience, it possessed colour, and given the overall structure of colours it also exhibits features such as lines and shapes. In fact, the best way to think of them in this sort of case is as picture-like objects—objects that possess the basic properties that pictures have. (I shall argue later that the existence of actual pictures and our familiarity with them is an important psychological source of the long-standing attraction of such ways of thinking.) However, their mode of existence is not like that of real pictures in the world. They are thought of as not being in public space, but as having a subjective or mental existence. Classically, this was captured in Berkeley’s slogan that their ‘esse’ (being) was ‘percipi’ (to be perceived). Along with this conception of their mode of existence went the idea that sense-data are private to a subject—each subject when having such experiences has his or her own sense-data. It should be clear that a theory with this structure can be found in much earlier philosophical theorizing about perception, in periods when the sense-datum terminology had not yet emerged. Thus, it seems reasonable to interpret the following passage in Locke as basically expounding the model just sketched. In his famous discussion of primary and secondary qualities Locke says: Whatsoever the Mind perceives in it self, or is the immediate object of Perception, Thought or Understanding, that I call Idea . . . Let us suppose at present, that the different Motions and Figures, Bulk and Number of such Particles, affecting the several organs of our Senses, produce in us those different sensations, which we have from the Colours and Smells of Bodies, v.g. that a Violet . . . causes the Ideas of the blue Colour. . .  Flame is denominated Hot and Light; Snow White and Cold; and Mana white and Sweet from the Ideas they produce in us. Which qualities are commonly thought to be the same in those Bodies, that those Ideas are in us, the one the perfect resemblance of the other, as they are in a Mirror. . .4

It is, of course, a matter of dispute how best to interpret this famous passage, but it seems Locke’s view is that we directly perceive ideas, which are inner mental items, the type of 3 Russell

introduced the terminology of acquaintance, but that was an odd appropriation of an expression in ordinary language. Russell means it to stand for a relation to an item in virtue of which the subject can, when it holds, scrutinize and attend there and then, to the item. But in normal speech I can be acquainted with an object at a time without at that time being able to attend to it. I am currently acquainted with many people I am not able to concentrate on now. 4  Locke (1975), Bk 2, ch 8 extracted from ss 13 and 15. A complication in the terminology here and one that comes out in the passage quoted is that Locke uses the term ‘idea’ to stand for a type of item present to the mind in both perception and thinking. This reflects his assumption that we should model both occurrences along what I call ‘act/object’ lines. By using the same term, ‘idea’, to cover both cases Locke invited the question, later discussed by Hume, as to what the difference is between experiences which are perceptual and those which are thoughts. Locke failed to face up to this issue, whereas it stared Hume in the face. Despite that insight, Hume’s answer, encoded in the dual terminology of ‘impressions’ and ‘ideas’, is hardly satisfactory.

Sense-Data   121 item present when a sensation is felt, and that this item presents to the percipient such qualities as whiteness and blueness. Locke is concerned to emphasize that ordinary people think that the sensible quality presented by and so belonging to the idea mirrors (or resembles) the same quality in the causing object, but they are, according to him, wrong to think this.5 It is surely clear that Berkeley would have said the same. Hume is slightly more complicated because of his scepticism about subjects, but his characterization of impressions would have attributed to them the same sort of features. Impressions are for him, as it were, independent picture-like images somehow involving consciousness. This conception persisted well into the last century and here at its beginning is Russell speaking along very similar lines. ‘Let us give the name of “sense-data” to the things that are immediately known in sensations: such things as colours, sounds, smells, hardnesses, roughnesses, and so on. We shall give the name “sensation” to the experience of being immediately aware of these things. Thus, whenever we see a colour, we have a sensation of the colour, but the colour itself is a sense-datum, not a sensation.’6 Russell’s way of speaking is somewhat different but the model being proposed is surely the same. It is obvious that many other examples of the expression of this model could be cited.

2  Some aspects of the dominant model Four aspects (out of a number of important ones) of this general approach merit noting immediately. The first is that proponents of the theory or model tended to regard it as deeply revisionary of ordinary opinion. The idea is that ordinary people think of the events they conceptualize as perception of the public world as having a certain structure—the public object, as it were, enters or gains access to the consciousness of the percipient. This conception is labelled naive realism, reflecting the conviction that it is the theory of the naive. The espousal of the new picture is then often regarded as an instance of commonsense thinking being superseded by what is, in a sense, a scientifically and properly worked-out account. In the marvellous terminology of Sellars the new theory represents the triumph of a scientific image over the manifest image. It needs adding that not all proponents of sense-data viewed matter that way, for example Moore did not. The second aspect is that in its general structure it is what philosophers have called an ‘act-object’ theory. It analyses the experiential occurrence as consisting of an object, the sense-datum, with its features, and a mental relation towards that object, which relation can be called an ‘act’. This helpful terminology should not, of course, be read as committing the model to treating the so-called act as being an active doing by the subject, an exercise of the subject’s (voluntary) agency. Once this structure has been noted it becomes a major question whether it is obligatory to adopt an account of the nature of such experiences with this structure. But this is a more general issue than simply whether the experiences we tend to think of as perceptual should be so analysed. Should we think of experiences in general this way—including experiences of pain, after-images, and itches? Are we constrained to 5 

It is highly questionable whether Locke characterizes the naive view accurately here. The naive view is not so much a resemblance view, as the view that what is presented as coloured is the external object. 6  Russell (1912: 12).

122   Paul Snowdon apply it completely generally? Must we think, for example, of a pain as an object that we apprehend when we feel a pain? This is an aspect of the sense-datum theory that obsessed the later Wittgenstein. The third thing to note is that there is a sense in which this model is not, strictly speaking, a theory of perception at all. The reason for saying this is that built into the theory itself is a postulated relation, expressed in such terms as ‘apprehending’ or ‘directly perceiving’, which seems to be more or less equivalent to a relation of perceiving. No analysis or decomposition is offered of this relation. It is, within the model at least, a primitive basic component which the analysis relies on. It can be said that the theory is a theory about the objects of perception. It certainly says what we (directly) perceive. But it clarifies things, I want to suggest, to regard the model as more correctly described as a theory of appearance. The idea is that when subjects have such experiences (which we think of as perceptual) certain appearance facts obtain, such as we would report by saying: ‘It looks to the subject as if there is a red round tomato.’ The fundamental explanatory role of the model is to decompose such facts into an act-object state of affairs, in which the nature, or as one might say, the content, of the appearance is explained by assigning as qualities to the object, the sense-datum, ones which ground or yield that appearance. In the case in question it would postulate that the sense-datum is actually red and that the red patch exhibits a round shape. It is obvious that such a model cannot in any straightforward way account for the truth of the appearance as being of a tomato, since the sense-datum cannot itself be a tomato. To explain that aspect of appearance facts needs more to be built into the account, or perhaps the development of a more sceptical attitude to the idea that appearances can really involve a tomato aspect. The fourth aspect of the theory to mention here is that in order for this model to work as a theory of appearance the central relation between the subject and the postulated item has to be thought of as one which is, in a sense, perfectly transparent. That is, in the obtaining of the relevant relation, the postulated ‘act’, the properties or qualities that the sense-datum has must be revealed without any distortion or subtraction or addition, since if what we might call the generation of illusions about the nature of the sense-datum were possible the basic model would not in any determinate way generate appearances at all. Thus given that possibility of an illusion to be told that the experience consisted of an apprehending of a red patch would merely generate the further question: yes, but how did the red patch appear? Red, or perhaps another colour?

3  Moore’s interpretation of sense-data I have tried to present what I am calling the dominant employment of the sense-datum terminology, as a term standing for an element in a theory of (perceptual) experience, and to draw attention to some aspects of that theory. I shall now briefly sketch the first of the two less dominant uses, the one particularly associated with G. E. Moore. In Moore’s use it is taken to be simply totally obvious that there are sense-data. In the dominant use, in contrast, it is taken that the introduction of sense-data into our theory requires arguments and a defence. Moore’s attitude is that there is a way to introduce the terminology of sense-data which means that no one would deny there are sense-data—that is as obvious

Sense-Data   123 as the fact that there are experiences at all. How can this be? Moore’s assumption is that to have an experience is to be in a position, there and then, to pick out some presented item—a sense-datum. The only real question is: what sort of item is the undeniably or obviously present (or presented) object? That is the question that needs arguments to settle it. There is not the space in this chapter to investigate Moore’s use of the term ‘sense-datum’ at any length, but I  wish to make three observations about it. First, in one way the difference between Moore’s type of use and the dominant one can be regarded as simply a difference in terminological packaging. Moore assumes there is obviously an object presented to the subject, or apprehended by the subject, and labels it a sense-datum, but then he argues for a certain characterization of it, which could be, even if it actually is not, precisely the same as that built into the employment of the term ‘sense-datum’ by the dominant use. And the person arguing for the existence of sense-data, understood in the dominant way, will in effect assume there is an object present within the experience, a fact that Moore employs the term ‘sense-datum’ to record. Moore, for reasons of his own, does not accept that the object depends for its existence on being apprehended or sensed, and so does not actually believe that sense-data understood in Moorean usage are sense-data understood in the dominant use.7 This is, however, unconnected with his use of ‘sense-data’. Second, Moore evidently thought that all experience involves the presentation of an object to a subject. This assumption sustained his conviction that one can discern an object, the sense-datum, within the experience. This assumption is, however, not evidently true at all. It is a significant question whether experience is like that.8 This means that Moore is not entitled to his supposedly ‘non-theoretical’ sense-data. Third, Moore’s idea is that it is obvious that there is an object but its nature remains to be determined. Evidently if he thinks this he must be very cautious about characterizing the object. Since he does not know what kind of object it is, how does he know what the experience entitles him to say about it? Moore is not in fact cautious enough, and when he argues that, say, the sense-datum is not the surface of a physical object he relies on assumptions about its character which he is not entitled to make, unless he already knew that it was not a surface. For example, in thinking about double vision Moore simply assumes there are two objects, two sense-data, when maybe there is only one object seen twice, and hence only one sense-datum. Moore falls into the two traps that his approach sets him. The first is that it rests on a general theory and is not truly a-theoretical, and the second is that he does not preserve his neutrality when he characterizes sense-data.

4 Ayer’s use In 1940 A. J. Ayer published his book The Foundations of Empirical Knowledge. Perhaps the chief importance of this book now is that it became the target of Austin’s famous criticisms of sense-data. However, Ayer’s own aim was to give an account of our perception-based 7 

This is the main point of Moore’s highly influential and famous, but also rather obscurely argued, Refutation of Idealism. 8  In section 7 I shall argue that the assumption is actually false.

124   Paul Snowdon knowledge of our environment (so-called ‘empirical knowledge’). In the early stages of his argument Ayer endorses an employment of the terminology of ‘sense-data’, according to which it is not a terminology the use of which involves novel (or indeed, any) ontological commitments. It simply represents a ‘new verbal usage’.9 This is how Ayer puts it: In order to avoid these ambiguities, what the advocates of the sense-datum theory have done is to decide both to apply the word ‘see’ or any other words that designate modes of perception to delusive as well as to veridical experiences, and at the same time to use these words in such a way that what is seen or otherwise sensibly experienced must really exist and must really have the properties that it appears to have. . .  This procedure is in itself legitimate; and for certain purposes it is useful. I shall adopt it myself. But one must not suppose that it embodies any factual discovery. . .  What he is doing is simply to recommend a new verbal usage. He is proposing to us that instead of speaking, for example, of seeing a straight stick which looks crooked . . . we should speak of seeing a sense-datum which really has the quality of being crooked . . .10

Ayer himself credits G. A. Paul with bringing out that we should understand the introduction of ‘sense-data’ talk as merely a new ‘verbal usage’. How should we view this development? Ayer’s attitude represents, I think we can first say, a complete misunderstanding of Paul’s point in his famous and influential paper.11 Paul’s point was that in the philosophical tradition in which sense-data were talked about—a tradition of which he had had first-hand knowledge during his period in Cambridge—the proponents of such talk, as he puts it, ‘have not spoken as if what they were doing was introducing merely an alternative way of saying this same thing over again, but as if the new sentence which they substitute were in some way nearer the facts’.12 Despite them not thinking this, it is Paul’s view that a scrutiny of their new terminology and how they use it reveals it cannot be ‘nearer to reality’. Paul’s attitude is, then, at bottom a criticism of the proponents of sense-data. Now, my interest here is not in assessing the force of Paul’s critical argument, but in noting that what was in Paul’s hands meant to be a criticism is taken by Ayer to be a characterization of a favourable way of understanding the endorsement of ‘sense-data’ talk as a positive but purely linguistic proposal.13 Further, it is quite clear that Ayer’s approach is inviting trouble. It seems obvious that if there is a tradition of theorizing in which a technical term, ‘sense-datum’, is taken to stand for an object of some kind, it will be hard to maintain the attitude that a new usage is being proposed according to which it has no such semantic role. There is surely a danger that the old understanding will enter how the new talk, in its argumentative context, is taken. It seems a rather imprudent proposal. And, sure enough, we find Ayer almost immediately saying things that betray such a misunderstanding. Thus Ayer himself says that the new terminology commits itself to the idea that ‘what is seen or otherwise sensibly experienced must really exist and must really have the properties that it appears to 9 

Ayer (1940: 25). Ayer (1940: 24–25). I have left out parts of the two paragraphs in the quotation. 11  See Paul (1936)—page references to the reprint in Flew (1960). 12  Paul (1936: 107). By this ‘same thing’ Paul means something along the lines of: that stick looks bent. 13  Paul’s general approach is clearly inspired by Wittgenstein’s attitude to sense-datum talk. It is a mark of the significance of Paul’s article on sense-data that Flew chose to reprint it in a collection published in 1960, twenty-four years after it came out. 10 

Sense-Data   125 have’.14 But this seems to be a total misunderstanding of his own proposal. When the user of the new language re-expresses what he or she would have said in the words ‘that looks bent’ by using the words ‘I see a sense-datum that is bent’ they are not committing themselves to something which ‘really has’ the property of being actually bent. The word ‘bent’ in the new usage can occur in true sentences without there being anything that is actually really bent, at least, insofar as it can so occur in the former locution. The coupling of the term ‘bent’ with ‘sense-datum’ shows that it can have a role in a true sentence without anything being really bent, just as it can in such a sentence as ‘S thinks that is bent’. So Ayer immediately falls into the very trap his proposal is setting. We can also ask what the point is supposed to be. Ayer explains how he sees it in these words: ‘since in philosophizing about perception our main object is to analyse the relationship of our senseexperiences to the propositions we put forward concerning material things, it is useful for us to have a terminology that enables us to refer to the contents of our experiences independently of the material things that they are taken to present. And this the sense-datum terminology provides.’ 15 It is clear, though, that this is a mistake. The new language can only be understood on the basis of knowing what the translation rules are from the new sentences to old sentences. This means that the new terminology cannot provide ways of talking about experience which are more independent of material object notions than the original sentences were. There cannot then be any advantage in employing it for philosophical purposes. Rather, the new terminology will disguise or hide what those relations are. Ayer’s third way with the ‘sense-datum’ terminology, then, seems like other recent ‘third ways’ of speaking, to represent a bad way.

5  Epistemology and ontology Before investigating the arguments in favour and those against the dominant model, something needs to be said about the main focus of the model. The need arises because the chief function (for us) of perception is to gain information about the world (and ourselves). Perception centrally has an epistemological role. This means that philosophical investigations of perception can be motivated by a desire to understand how it yields the information that it does for us, or a desire to understand what the structure of the perceptual relation itself is. In current jargon the latter interest is sometimes called ‘metaphysical’. Now, it seems true to me to say that these two roles have often been confused in the discussion of philosophers. This confusion, if it has occurred, is perennial, but it has been fostered by the use of the term ‘sense-datum’ which is normally taken to stand for a postulated object, but by talking of a ‘datum’ people are encouraged to think it stands for something fact-like that can be learnt or discovered. Once this distinction is blurred confusion breaks out. We can, I think, illustrate this by looking at selections from a passage of Ayer’s. In this passage Ayer is not presenting an argument but is, rather, introducing what he calls the ‘problem of perception’ to the reader. Here is what Ayer says about the problem.

14 

Ayer (1940: 24).

15 

Ayer (1940: 26).

126   Paul Snowdon The problem of perception, as the sceptic poses it, is that of justifying our belief in the existence of the physical objects which it is commonly taken for granted that we perceive. In this, as in many other cases, it is maintained that there is a gap, of a logically perplexing kind, between the evidence with which we start and the conclusions that we reach. . .  The starting point of the argument is, as we have seen, that our access to the objects whose existence is in question must be indirect. In the case of perception, however, it may be well be doubted whether this premise is acceptable. . .  It is certainly not obvious that there is any question here of a passage from one type of object to another. Nevertheless, a great many philosophers have held that this was so. . .  Taking the hard data to be securely known, they have regarded the existence of physical objects as being relatively problematic. . . . What, according to them, is immediately given in perception is an evanescent object called an idea, or an impression, or a presentation, or a sense-datum, which is not only private to a single observer but private to a single sense.16

There are two sorts of oddities, it seems to me, in this passage. One sort are the remarks which, considered on their own, are problematic. For example, Ayer seems to think that the problem of perception is the question as to how we can justify our belief in external objects. Now, that question, or problem, is not posed by the sceptic. It can be asked by anyone. Ayer also talks about ‘the argument’, but it remains unclear as to what, at this stage, the role of any argument is. Does Ayer mean an argument designed to show that there is no justification, or to show that there is something we can think of as our evidence which itself is not a claim about external objects? However, the second, more general oddity, in this passage as a whole, is that no clear conception of the problem of perception emerges. Thus, it is clear that Ayer thinks the problem of perception is an epistemological problem, a problem about justifying a certain class of beliefs. Now, as has been pointed out, Ayer regards the background here as an argument, but it has to be wondered why, if the argument is thought of as promoting the view that there is no justification, any such argument needs to rest on the claim that ‘access to [external] objects’ must be indirect? Indeed, what does that mean? If ‘access’ means way of knowing then presumably the notion of indirectness simply means that knowledge of the relevant facts is based on evidence of a different kind. In that sense the idea that the sceptical argument requires indirectness seems correct but trivial. But Ayer seems to assume that it must involve the idea of a passage, of some sort, from one kind of object to other. What does that mean—and why is it required? And that idea of indirectness then gets aligned to the postulation of a new kind of object—a sense-datum. It seems, on the contrary, obvious that epistemological indirectness does not require this other sort of indirectness. It remains problematic too whether the epistemological indirectness itself follows from the other sort of indirectness. Standing back from Ayer’s introduction, it seems clear that he writes as if there is some sort of argument about perception the interest of which is that it creates an epistemological problem. But no clear idea is presented as to what the epistemological problem really rests on—does it rest on the soundness of the object introducing argument or not?—nor is there any clear recognition that there are two very different notions of ‘indirectness’ involved, nor does Ayer indicate that the introduction of ‘objects’ might be interesting for other reasons or be motivated by 16 

Ayer (1956: 84–85). The quotation in the text draws on three paragraphs from Ayer’s text.

Sense-Data   127 quite different considerations. I think myself that no clear impression of a problem and its interlocking parts is created in this passage. This passage, I am suggesting, is confused. But this allegation has another significance for the sense-datum debate. One important question about the contribution of Austin to the sense-datum debate is to what extent he properly uprooted the approach. Now, one aspect of that issue is the extent to which the people he identified as his opponents and whom he intellectually stalked are the best supporters of the model. It weakens Austin’s contribution if the people he destructively criticizes are not exemplary defenders of the theory. It is, of course, primarily Ayer that Austin criticizes. The worry is that Ayer is himself too mired in confusion to be the best exponent to criticize. There are two important respects in which Ayer is not a good target. The first is that his discussion exhibits the lack of clarity between epistemology and ontology as, I hope, brought out above. The second is that he operated with the notion of sense-datum talk as useful without regarding sense-data as entities—what I called the third way with sense-data. Now this is, as I have argued, a bad approach. That Ayer exhibits both confusions considerably weakens his role as the principle target in Austin’s attempted destruction of the problem of perception.

6  Arguments for the dominant model The standard arguments in favour of the dominant model can be thought of as having a two-part structure. The first part takes a limited range of cases and argues or claims that the model applies to them. The second stage generalizes the conclusion to all cases. These arguments are usually named on the basis of the limited range of cases in the first part. One famous argument—the argument from illusion—takes them as the basic case, and another, the argument from hallucination, generalizes from hallucinations. Arguments with this structure are pervasive in the philosophy of mind.17 Now, these arguments in the philosophy of perception face basically two questions. The first question is whether the claim that the model applied to the basic cases is well supported. The second question is whether the generalizing step is legitimate. With these questions in mind, consider first the argument from illusion. The notion of an illusion as it is used here is that of a case where an object is, as we say, perceived but looks in some way how it is not. The case that was often chosen is that of a straight stick partly seen in water; the stick looks bent but is straight. Why was this case (or type of case) considered interesting? The answer is that it is being used to show that there is a sense-datum, with certain qualities and its ontological status, of which the subject is aware. It can do this if the case enables us to recognize the presence of such an object. Basically, it works by eliciting the acknowledgement that there is something that is bent, which cannot be the external object, which is, of course, agreed not to be bent, and which is within the subject’s range of awareness. The acknowledged presence of the thing, exhibiting bentness, requires the introduction into the model of what is happening of an object distinct from the physical object. Now, it would in fact require extra argument to

17  One interesting example is the pattern of argument aimed at showing that all bodily action involves trying. This first argues that there is trying in the case of failure and then generalizes that claim to all cases.

128   Paul Snowdon show that the postulated object has to have the nature of a sense-datum, but an equally serious question is why a case like this requires us to postulate a bent entity at all. There are two ways of thinking about this. One is that it is simply obvious to the subject (and hence to the theorist) that there is something bent. This claim, however, invites the response of asking why this fact is obvious. The counter-suggestion is that it is obvious that it looks as if there is something bent, but it is not obvious that anything is actually bent. That dialogue, it seems, to me, stops that argument in its tracks. We should also invite anyone who is tempted to claim this is obvious to weigh up the consequences of accepting that conclusion. If something is, say, bent in that occurrence then we have to locate somewhere a bent thing, and that commitment is serious and difficult to fulfil. These considerations need weighing up by anyone who finds the ontological claim obviously true. The second way to look at the conviction that something is bent is that it is the result of an application of a plausible general principle that might be expressed thus; if something looks F then there is something that is F. This general principle is called, by Howard Robinson, the Phenomenal Principle.18 However, invoking a principle merely invites the response of asking why that should be accepted. Further, the current principle is nowhere near the truth. No one would agree that if someone looks old then there must be something old. Considered either way, as evident truth or application of a principle, there is no warrant to introduce the entity in a correct account of such cases.19 This means, I  suggest, that the legitimacy of the spreading (or generalizing) step in the case of the argument from illusion can be left uninvestigated. But the same problem faces the argument from hallucination as it is employed to support the general truth of the sense-datum theory. The issue is why the treatment of the base case of hallucination as involving sense-data is legitimate. If one hallucinates, say, a bent stick, we cannot, of course, treat that as was proposed for the illusion of a bent stick case, as, namely, a straight stick looking bent. However, that does not mean that we need to regard it as involving a bentness-presenting sense-datum, unless we already subscribe to something like the Phenomenal Principle, which we have been given no reason to do, or because it is simply thought to be obvious that there is a bent entity present to the subject’s consciousness, which idea is no more obviously correct in this case than the illusion case. In some other cases that have been appealed to as the ground for introducing sense-data into the account of the base case they have not been based on the straightforward idea of trying to locate an entity bearing a property which is in some sense in the appearance, but rather because it is felt there are two entities within the subject’s consciousness, but only one physical object, for example in the case of seeing double. However, such cases can be treated without invoking sense-data. One way is to regard double vision as simply a double sighting of a single entity, hence no new entity needs postulating. Alternatively, if it is thought that that account is deficient, then one can treat what one might think of as one of the sightings, or maybe, both, as experiences having a structure akin to that involved in hallucination, whatever that is. I am proposing that there is no cogent way to introduce sense-data into any of the base cases, and so there is no interim conclusion about sense-data that can be generalized across 18 

See Robinson (1994: ch. 2). A fuller analysis of the argument from illusion with a different emphasis from the present analysis can be found in Snowdon (1992). 19 

Sense-Data   129 all cases. Two things need adding. First, the generalizing case itself is plainly dubious in the case of the argument from hallucination. Austin’s lovely example of seeing a lemon and seeing a lemon-like bar of soap bring out that the fact that subjects cannot tell (if they in fact cannot) that a hallucination is not a perception simply on the basis of the experience itself does not imply that the two occurrences are not of quite a different nature.20 Second, even if these standard cases—such as illusions and hallucinations—do not yield any support for the sense-datum theory, it does not follow that reflection on them does not yield significant conclusions about perception.

7  Problems for the dominant model The tide swung against the dominant model in the middle of the twentieth century. One important development in philosophy around the time of the Second World War was the emergence of what might be called theories of ‘anti-theory’. By that I mean the conviction shared by both the approaches of Wittgenstein and Austin that philosophy should not engage in strong positive theorizing. It is not entirely easy to say why these influential philosophers accepted this idea, but both of them took what I have been calling the dominant model as an example of bad positive philosophizing. Austin’s Sense and Sensibilia is a more or less totally negative engagement with the dominant model in which he displays his massive talents as a critic. Wittgenstein hammered away at the model repeatedly in his discussion of philosophy of mind.21 Although the philosophy of anti-philosophy did not last it did leave as its legacy the abiding conviction that the sense-datum theory was not a good theory. Moreover, one very fundamental conviction of the philosophy that succeeded it was a commitment to materialism, and the sense-datum model did not fit that approach. Sense-data as characterized in the theory did not seem to be entities that a materialist could countenance. This shift cemented the fate of the model. An important point to note at this stage is that although there is a branch of philosophy called ‘the philosophy of perception’ its own proper home is as a branch within the philosophy of mind. Perception is simply a very basic and central psychological phenomenon. In many ways the fundamental constraints that shape our thinking here have to be those which are central to our understanding of the mind generally. And it is quite clear that no proper psychological theory can postulate an unanalysed psychological relation with mysterious properties at the heart of its analysis of any psychological phenomenon. These are remarks about intellectual history and basic assumption, which are, I hope, illuminating. But I want to develop what I think of as another real problem for the theory.

20  Austin’s

example blocks basing the generalizing step (in the argument from hallucination) on anything like subjective similarity or indistinguishability. More recently, the generalizing step has been grounded in causal considerations (see Robinson, 1994: ch. 6). The attitude to that development implied by the present argument is that his strengthening, should it be a strengthening, cannot aid the case for sense-data, since they do not get into a correct account of the base cases. It might, however, aid the derivation of other conclusions based on hallucinations. 21  I have tried to analyse Wittgenstein’s views in Snowdon (2011).

130   Paul Snowdon I have suggested that the sense-datum theory is best thought of as a theory of appearances. Now, when we focus on appearance reports it seems quite appropriate to talk of the content of an appearance. Thus, if I say that it looks to S as if there is a red patch I can gloss that report as saying that the content of the appearance is that there is a red patch. This is as natural as saying, as practically all philosophers do, that the content of the belief that P is (that) P. This allows us to say that what the sense-datum model offers is a theory of the content of appearances. But the structure of the theory can be conveyed using a terminology, that of content and vehicle, that has recently emerged and which is, I suggest, very helpful.22 We can talk of the sense-datum as the vehicle, the item, to which properties are being ascribed to generate the content of appearance. The idea is that the properties of the vehicle match and explain the content. Now given the commitments of the model under investigation to the idea of the transparency of the psychological relation, the act bearing on the object, it follows that there is a tight relation between the content and the postulated vehicle. Roughly, any element in the content of the appearance must correspond to the presence of a corresponding quality or feature belonging to the vehicle, otherwise that aspect of content has not been explained, and any quality of the vehicle must have a corresponding role in the content, otherwise the ‘act’ has lost its transparency. This leads to two enormous difficulties for the model. The first is that there are properties ascribed to the object that are not mirrored in appearance. Thus the vehicle is supposed to be private, inner, and dependent for its existence on awareness, but no one would suggest that in normal appearances anything is present corresponding to those aspects. Now, the obvious reply to this is to suggest that some aspects or properties of the sense-datum are ‘conveyed’ or ‘captured’ by the transparent act, whereas other, for example the ontological status of the entity, are not. There is no simple way to block this proposal. But, first, not all ‘sense-datum’ theorists do draw this distinction.23 Second, such a distinction cannot just be asserted but needs properly explaining, and it remains unclear what that explanation would be. What is going on here is that the theorist simply assumes they can draw a distinction which does apply to ordinary perception, that is the distinction between the features or aspects of things that perception acquaints us with and those that it does not. But the explanation of that distinction rests on assumptions about the nature and processes involved in ordinary perception which cannot be drawn on when thinking about the pure apprehending relation. More important, though, is that there are, or seem to be, properties of appearance which are not mirrored by matching qualities in the object. One very interesting case is that of what might be called the indeterminacy of appearance. Consider the following case. I can see, as we would say, two lines, A and B, on a surface in front of me. Imagine 22  This terminology is now popular and its employment widespread but a very helpful exposition of it can be found in Millikan (1991). 23  See for example Hume (1964), Bk 1, Pt 4, s 2, p. 185. As Hume puts it: ‘Add to this, that every impression, external or internal, passions, affections, sensations, pains, and pleasures, are originally on the same footing; and that whatever other differences we may observe among them, they appear, all of them, in their true colours, as impressions or perceptions.’ The last part of this sentence is, in effect, endorsing the implication of the model that since they are impressions they exhibit their status as impressions.

Sense-Data   131 that A and B are quite far apart. I look at the lines carefully, but there and then I could not say whether the lines are equal in length or different. If they are different I cannot say which is longer. How should we describe the appearances in such a case? Since, however carefully I attend, I cannot tell what the relation is between the lengths of the lines, it seems to me that the correct appearance or looks judgement is that A and B do not look to be the same length, but they also do not look to be different lengths. Put simply, in appearance A and B are neither the same nor different. We can put this by saying that there is an indeterminacy in the perceptual appearance; appearances leave their relative lengths indeterminate. Now, since it is important in developing this criticism that the indeterminacy of appearance claim is granted, it is helpful to compare appearances with the case of belief and with the case of the objects themselves. It is clear, surely, that the beliefs of a single subject might be described as being indeterminate about the relative lengths of A and B. This would be the case if the believer believed that A and B both have lengths but he or she had no further beliefs about their lengths. It can thus be indeterminate according to a belief system what their relative lengths are. If we think, though, about the lines themselves we would be inclined to say that it cannot be indeterminate in fact how their lengths are related. Either A is longer, or B is longer, or they are the same length. We have, therefore, no indeterminacies in the world (at this level) but we can have indeterminacies of content within belief systems (or theories). The question is into which category do episodes of perceptual appearing belong. Now it clearly does not settle this question to point out that perceptual episodes are natural physical episodes and hence are determinate occurrences. The same is true, we can say, of belief and of speech, cases where content can be indeterminate. Basically, the rule, I  suggest, is that if an appearance (in a perceptual episode) does not enable the subject to come to an opinion as to whether a claim P is true or not, given the ability to concentrate and take their time (and told to take nothing else into account other than how things appear), then it cannot be correct to characterize the appearance as being one as of P’s being true. Thus, if in an appearance to a subject the subject simply cannot estimate which of two objects is further away, we conclude that neither of the objects looks to the subject further away. I am suggesting that that is a basic principle (or close to a basic principle) of appearance ascription. If that is accepted the sense-datum model itself faces a deep problem. Given that the act or psychological relation is itself purely transparent, and given that the actual lines on the internal vehicle have to be determinately related, it cannot be that there is an indeterminacy in appearance. Although the language that has been employed to express this criticism is located in current theories of mind the point being made is one that proponents of act/object analyses of mental phenomena have been grappling with for centuries. It is precisely this type of problem that proponents of imagistic or idea-based theories of thought and language understanding, such as Locke, Berkeley, and Hume, faced in the seventeenth and eighteenth centuries. Berkeley’s key insight in his dispute with Locke about abstract ideas is that single determinate images cannot be the vehicles for general thought. His question is: how can the presence before the mind of a determinately coloured red image ground that relatively indeterminate thought that something is, simply,

132   Paul Snowdon red?24 This problem struck these thinkers in relation to the content of thought (and language) because the generality of thought is hard to miss, whereas, what we might call the generality of perception (and experience) is not quite so obvious. What is the correct response to this problem? The correct response, I want to suggest, is to abandon the act/object structure as the form of the analysis of experience. In fact we can then employ what is called an adverbial analysis. In effect that name stands for nothing more definite than approaches which do not treat it as involving an object and an act. It should not be thought, though, that this means no experiences have an act/object structure, but when they do the act component ought not to be thought of as transparent in the sense that it reveals in a perfect way the character of the object.25 It is essential that we think of such cases, if we wish to acknowledge them, involving an ‘act’ that permits illusions and distortions. There are two other, and related, problems with the model that I wish to highlight. I have been concentrating in the discussion and exposition of the sense-datum theory on the visual case, as is standard with that model, and I want to continue to do so in order to make the next point. When sense-datum theorists think about visual experience they deny that in it things look to be located in three-dimensional space.26 No object can strictly look to be behind or further away than another. Here for example is Locke struggling with the problem: So that from that, which truly is variety of shadow or colour, collecting the figure, it makes it pass for a mark of Figure, and frames to it self the perception of a convex Figure, and an uniform colour; when the Idea we receive from thence, is only a Plain variously colour’d, as is evident in Painting.27

In fact, in this passage Locke is trying to be true to his sense of the manifest character of visual experience, which is that it is three dimensional, but also to be true to what his model fundamentally commits him to, which is that it is not. The commitment of the model is clear towards the end, where Locke says that the idea is what we would describe as a ‘plane’, a two-dimensional structure, which can be compared to a painting, and what it presents is ‘truly’ a variety of colours. But he tries to escape this problem, a task that is a measure of Locke’s own commitment to respecting the truth, by suggesting just before this quotation, that ‘the Ideas we receive by sensation, are often in grown People alter’d by the Judgement, without our taking notice of it’. In effect, in making this suggestion Locke is grafting onto his basic model a modification (or addition) that it cannot incorporate. For, how can an idea be modified by judgement? Indeed, what modification along spatial lines is possible for the idea? Locke’s real commitment comes out when he tells us what ‘truly’ the idea we receive is like, namely a ‘variety of colour’. For Locke the real significance of Molyneux’s 24  I do not mean to suggest that Berkeley himself had a solution to this problem which was better than Locke’s. The general conclusion should in fact be that the basic model in terms of which they were thinking of the problem is fatally flawed and should be abandoned. 25  Talk of transparency in the theory of experience is tricky. Some mean by it that the experience is itself, as one might say, invisible. Whereas others mean the experience simply opens up the complete reality of the object. It is the second sort of transparency that needs eliminating. 26  Mohan Matthen has correctly reminded me that Frank Jackson is an exception to this description. His theory of sense-data located them in public space. I  can here only voice the suspicion that this characterization does not cohere with the other supposed properties of sense-data, nor with our understanding of public space.    27  Locke (1975), Bk 2, ch 9, s 8.

Sense-Data   133 famous question, to which he immediately turns, is that there is no depth in basic visual experience, and it is only after experience and learning, that someone can ‘distinguish’ on the basis of sight between a sphere and a cube. The upshot of E. J. Gibson’s famous Visual Cliff experiment is that it suggests that visual perception of depth does not depend on learning, contrary to one commitment of Locke’s theory.28 The moral of this discussion, illustrated by the very interesting case of Locke, is that the standard sense-datum model, in which the visually apprehended entity is a two-dimensional picture-like plane, cannot accommodate the rich and obvious three-dimensional character of visual experience (and not simply of adult visual experience). This is serious, because ultimately any model of perception has to be judged by its ability to explain the evident role that perception has for us, which is to inform us of our environment. It is a serious failing that the sense-datum model cannot satisfactorily do this for a central aspect of vision. We can, I  think, by reflecting on the consequences of this problem reveal two other serious drawbacks to the sense-datum model. The sense-datum theorists face the further question; if visual experience is two dimensional, is any human mode of experience three dimensional? If it is said that none is the one problem that arises is how we are led to think of our environment as spatially three dimensional. If no experiences indicate that why should we think that way? The second problem is that it is very hard to deny that tactile experience and our awareness of our own bodies is three dimensional. Indeed, those who viewed vision as, strictly, two dimensional usually regarded tactile experience as the source for us of awareness of three-dimensional space. But that element in their view brings with it a deep puzzle. If the quality of experience within an act/object model is explained by the real character of the ‘object’ how can we conceive of the object in such three-dimensional experience? It is not genuinely in space or spatial, being a sense-datum, but it cannot be conceived of as plane-like, since then no three-dimensional layout can be presented. It is a manifestly mysterious thing—a thing so mysterious that we can only say that there is no coherent model for this sort of case.29 With that problem in mind—the problem of no coherent model for the tactile/bodily case—we can consider another case, say that of olfaction. In this case the sense-datum theorist has to postulate an inner mental entity that ‘presents’ olfactory features to the subject. That specification fixes a role for the postulated object, but it does not enable us to envisage its nature. How does it do this? What kind of thing is it? The truth seems to be that we have no way of understanding an inner mental item which presents to the subject the qualities within olfactory experience other than by simply thinking of it as an ‘inner smell’. But it is clear that there is something deeply unsatisfactory about such a conception. The problem I want to highlight is not to do with the idea of an ‘inner smell’ which depends for its existence on the awareness of it, something quite unlike normal smells and, arguably, a genuine difficulty, but rather with making sense of the possibility of an item which is outside real

28  For

a description and very interesting discussion of the visual cliff experiment, see Gibson (1986: 157–159), and also Gibson and Walk (1960: 67–71). 29  The difficulty in this case for the sense-datum model perhaps lies behind a persistent tendency, when thinking about the ‘location’ of sensations, such as pain, for philosophers to treat their ‘location’ as something not felt or experienced, unlike the ‘painfulness’, but rather, as it were, added solely by the spatial reaction of the subject, in, for example, rubbing a certain part of the body in response to the pain. In truth, the pain is experienced as located.

134   Paul Snowdon space, outside all normal causal relations, and without the real nature, whatever it is, of real smells, but which has the property of being, as it were, ‘smelly’. I suggest that we should wonder whether this is a model that makes sense. It involves ascribing a property when it is completely detached from the circumstances and groundings and dispositions which sustain our understanding of its presence normally. It has to be conceded that since there are no established principles about what really makes sense, it cannot be demonstrated that this hypothesis lacks sense. The issue that remains is what sort of object is it that presents to the subject the qualities of smell (but which is not a real smell)? It seems to me that no answer to this question is forthcoming. Why did a similar problem not occur to the sense-datum theorists when they were thinking about the visual case? It is as hard, really, to understand the presence of colour on an inner item quite detached from those conditions in the world which ground or are associated with colour, as it is to understand the presence of smells. What disguised this, I want to suggest, is that they possessed a real model for their theoretical model, which is the existence of pictures, which are two-dimensional surfaces (or planes) which give us visual experiences that can resemble those that vision normally gives us. The familiarity of this phenomenon generates the impression that we understand the way an internal picture, a sense-datum, can generate visual experience. In reality this overlooks three important things. The first thing overlooked is that if the existence of real pictures enables some sense to be made of an ‘inner picture’ model of visual cases there simply is nothing analogous to pictures in non-visual experience which might confer sense on the role of an ‘inner’ representation in those cases. There is, for example, nothing that stands to a real sound as a picture stands to a real scene.30 Second, as suggested above, there is no obvious way to credit ‘inner’ sense-data with the properties of colour which are what actual pictures have. And, third, as Wittgenstein pointed out, there is much in our experience of pictures that is not fixed by the picture itself. Thus, I can see it as a rabbit or as a duck. It is simply an illusion to think that visual experience can come down to a transparent apprehension of a coloured plane. What I have been suggesting is that as we think our way deeper into the model it stands revealed as unable to cope with the difficulties that emerge.

8 Conclusion We have known at least since Austin how to respond to the arguments in favour of sense-data, and we have gradually seen the difficulties in the conception, something that Austin explored less. We have, perhaps, also developed some understanding of the appeal

30 

Mohan Matthen has pointed out to me that this comparison between vision and hearing is much more complicated than the argument in the text acknowledges. The existence of pictures in our lives depends on the fact that the three-dimensional world around us actually contains two-dimensional surfaces, that these surfaces have causal properties that enable us to mark them in certain ways, and that our perceptual system responds to such marked objects in certain ways. In the world of sound there is nothing like the first contrast, nor do sounds have similar causal properties. But it is true that we can create sounds which seem to be of things they are not of. What needs more thought is whether this fact weakens the contrast.

Sense-Data   135 of the view. However, the main conclusion, which is not news, is that the sense-datum conception is unacceptable, and should be rejected. This is, though, in many ways a rather limited conclusion. Roughly, we can say that the sense-datum tradition in philosophy arose from combining two fundamental ideas. The first idea is that the experiential core of perception is the occurrence in subjects of states which can be described as sensations; the second idea is that any experience has to include an object of which the subject is aware and which presents the qualities inherent in the experience to the subject. The drift of the present discussion is that this second, highly appealing but disastrous model is wrong, but in itself that claim leaves the first idea untouched. How we should think of the nature of perceptual experience other than as not consisting in the transparent apprehension of a sense-datum remains open.31

References Austin, J. L. (1962). Sense and Sensibilia. Oxford: Oxford University Press. Ayer, A. J. (1940). Foundations of Empirical Knowledge. London: Macmillan. Ayer, A. J. (1956). Problems of Knowledge. London: Penguin. Baldwin, T. (1990). G. E. Moore. London: RKP. Berkeley, G. (1734). The Principles of Human Knowledge. In M. R. Ayers (ed.), George Berkeley Philosophical Works. London: Dent. Flew, A. (ed.) (1960). Logic and Language (First Series). Oxford: Blackwell. Gibson, E. J. and Walk, R. D. (1960). ‘The visual cliff’. Scientific American, 202. Gibson, J. J. (1986). The Ecological Approach to Visual Perception. Hillsdale, NJ: Lawrence Erlbaum Associates. Hume, D. (1964). A Treatise of Human Nature. London: Dent. Locke, J. (1975). Essay Concerning Human Understanding. Oxford: Clarendon Press. Millikan, R. (1991). ‘Perceptual content and Fregean myth’. Mind, 100(4), 439–459. Moore, G. E. (1903). ‘The refutation of idealism’. In G. E. Moore (ed.), Philosophical Studies (pp. 1–30). London: RKP. Paul, G. A. (1936). ‘Is there a problem about sense-data?’ Aristotelian Society Supplementary Proceedings. Pitcher, G. (1971). A Theory of Perception. Princeton: Princeton University Press. Robinson, H. (1994). Perception. London: RKP. Russell, B. (1912). Problems of Philosophy. London: Oxford University Press. Snowdon, P. F. (1992). ‘How to interpret "direct perception"’. In T. Crane (ed.), The Contents of Experience. Cambridge: Cambridge University Press. Snowdon, P. F. (2011). ‘Private experience and sense data’. In O. Kuusela and M. McGinn (eds), The Oxford Handbook of Wittgenstein. Oxford: Oxford University Press.

31  I wish to thank Mohan Matthen for the invitation to contribute to this volume and also for his encouragement and help with the chapter itself. I wish to thank Craig French, Mike Martin, Mark Kalderon, and Howard Robinson for discussions over the past few years which have influenced my thinking about sense-data.

Chapter 7

Phenom enol ogica l A pproaches Charles Siewert

1 Introduction Phenomenology (as a philosophical movement or tradition) originated in the late nineteenth century, partly in an effort to find philosophy’s place in a culture increasingly dominated by experimental science and technology—and in dialogue (sometimes in tension) with an emerging academic psychology. Beginning with Franz Brentano, phenomenology seeks an elucidation of just what the phenomena are that psychology purports to explain, via inquiry anchored in an understanding of mind available from the first person point of view. From this perspective experience or consciousness is seen as fundamentally ‘intentional’: it refers to or is directed at objects. Just how to describe this ‘intentionality’ and its forms becomes a basic theme. Beginning with Edmund Husserl, the intentionality of perception is investigated by asking: how can experience, itself in near constant flux, nonetheless be of stable objects, so that meaning and knowledge might be possible for us? The key to answering this question, he proposes, is to see perceptual consciousness as dynamic and prospective—a process wherein the needed constancies are achieved via the successful anticipation of further experience through movement and direction of attention. This conception of Husserl’s—with its emphasis on the experience of one’s own body—helped inspire Maurice Merleau-Ponty’s important mid-twentieth-century contribution to phenomenology. The following is a summary of approaches to perception (in Brentano, Husserl, and Merleau-Ponty) that are central to the phenomenological tradition, closing with a brief reference to recent work that interprets, stems from, or neighbours it.

2 Brentano Brentano’s philosophy has such continuity with what his student Husserl called ‘phenomenology’ that the term is fitting for both, though Brentano himself rarely used it, and

Phenomenological Approaches   137 generally preferred to call his approach ‘descriptive psychology’. But this label could be misleading. Psychological description in his sense does not (as one might think) consist in saying what happens at a particular time in an individual’s mind. Rather Brentano purports to describe mental phenomena by identifying their fundamental kinds—to tell us what distinguishes them, and how they are necessarily related to one another. And he conducts ‘psychology’ not by performing experiments, but largely through philosophical dialectic reliant on first-person reflection—necessary, he thought, for clarity about the domain of experimental research. In Brentano’s view descriptive psychology or phenomenology is in this way foundational for explanatory or ‘genetic’ psychology, and crucial to the development of logic, ethics, and aesthetics. It must, he thought, draw on a first person understanding of the phenomena to be taxonomized, because this gives us our basic grasp of what we are talking about when we talk about, for example, consciousness, perception, or judgement (Brentano 1972: 29, 128). He attempts to justify his account of mental kinds by illustrating them in ways we are invited to confirm with reference to our own first person experience, and by continually seeking out puzzles and objections, responding to these with detailed argument. But what is the general notion of perception that Brentano proposes? He was convinced by modern philosophy and science that the actual denizens of space and time do not, in fact, bear the qualities (such as colour or smell) that sensibly appear to us. But his irrealism was also shaped by Aristotle’s doctrine that in perception the sense organ receives the ‘form’ of a sensible thing (e.g. the form of a colour) without its ‘matter’. In adapting ancient and scholastic conceptions, Brentano was led to the single most important aspect of his approach to perception—a revival of the notion of intentionality or (as he would put it) ‘intentional inexistence’ (1972: 88). For Brentano, red (for instance) exists ‘in mind’, as an accusative of ‘mental reference’, even though no individual, mental or otherwise, is in fact red. Thus he thinks, more generally, what is perceived in the case of ‘outer’ (e.g. visual) perception is not an individual private to someone’s mind that unfailingly is just the way it appears, like the ‘sense-data’ of the empiricist tradition. On Brentano’s view, the colours that are presented to us in vision exist ‘in our minds’, but this does not mean that there are coloured things to be found there: when red appears to you, the way it exists in your mind is nothing like the way it is supposed to exist in a tomato. (It should be noted, however, that the interpretation of Brentano’s view on this key point is problematic, and some accounts (see Brandl 2005) would place him closer to sense-data theories than is done here.) The notion of existence ‘in the mind’ or ‘intentional in-existence’ gives rise to a number of logical and ontological quandaries variously addressed by philosophers Brentano directly influenced. Brentano himself eventually abandoned the idea that mental reference should be understood in terms of a distinctive ‘intentional’ mode of existence. But his neo-Scholastic turn had this lasting modern legacy: he implanted in phenomenology the notion that perceiving can be understood as a ‘directedness’ or ‘reference’ to what putatively exists beyond the mind, rather than as an immediate awareness of what actually exists inside it. To understand Brentano’s ‘intentionalist’ view of perception, one needs a grip on his distinction between judgement and ‘presentation’. Not only perception, but all mental phenomena, he held, are intentional in his sense, because they either consist in or include a presentation that refers to an object—an ‘appearance’ of it in the broadest sense (1972:  81,  198). This encompasses the appearing of something one sees, or visualizes in

138   Charles Siewert imagination, or even only entertains a thought about. Merely to have a presentation of an object involves no commitment to its existence. To make this commitment, to judge there is a blue sphere, say, one must, in addition, affirm or accept the object so apparent—a blue sphere (and this is at least part of what distinguishes perceiving from imagining). For Brentano all perception is a form of judgement, in which something presented is affirmed. When a blue sphere is visually apparent to you, and you accept what is apparent, you perceive a blue sphere. And for Brentano, this very perceiving also presents itself to you, and you accept (and thus perceive) that as well. Thus he maintains that in addition to ‘outer’ perception of physical phenomena (like colour and shape) there is ‘inner’ perception of mental phenomena (such as the appearance of colour and shape). Brentano holds that, as a matter of fact (though not of necessity) all of our outer perceptions implicitly contain inner perceptions of themselves. And on his account, for a mental act to be internally perceived is just what it is for it to be conscious (1972: 100–29, 275–277). In response to the worry that he over-intellectualizes perception by making it a form of judgement, Brentano argues that the mere acceptance of an object presented is an effortless, cognitively primitive feat; it does not require an act of ‘synthesis’ whereby one subsumes an object, together with others, under a concept, in virtue of their similarity (1972: 141–142). In this sense judgement (hence perception) can be ‘non-conceptual’. This point holds for both inner and outer perception. The two, however, do differ in (methodologically) significant respects, according to Brentano. Whereas we can, through outer perception, make observations, we cannot, in inner perception, strictly speaking, concurrently ‘observe’ our own minds at all. This is because observing something involves increasing attention to it so as to discover how it already is, and any attempt to thus attend to one’s own current mental phenomena will (Brentano claims) alter what one wants to reveal (1972:  29–30). (This does not mean, for Brentano, that one cannot ‘notice’ one’s own perceiving and thinking as it occurs.) But even though inner perception cannot, like outer perception, constitute observation, it has the following advantage over it. In outer perception there generally is no real object that is just as it is presented, while in inner perception appearance and reality inevitably coincide (1972: 3, 10, 19–20). So, our understanding of the terms by which we describe our minds (unlike our understanding of those by which we describe external things) is grounded in a kind of constant implicit self-perception wherein things invariably appear (and are accepted) exactly as they are in reality. However, this infallible inner perception does not guarantee the correctness of whatever sincere first-person judgements we happen to express about what kinds of minds we have. While inner perception guarantees descriptive psychology a real subject matter, it cannot ensure this will be correctly categorized.

3 Husserl Brentano was certainly not the only major influence on Husserl’s first mature philosophical work, the Logical Investigations. But he was a singularly important one, insofar as the phenomenological approach heralded there, its aims, ‘intentionality’ as a philosophical theme, and its application to perception, all carry over ideas from Brentano—albeit substantially transformed through criticism. Although Husserl’s views on perception

Phenomenological Approaches   139 developed importantly after his Investigations, some grasp of this work is necessary to understand the basics of his approach. For Husserl phenomenology remains ‘descriptive’ in something like Brentano’s sense—it relies on critical first person reflection for understanding the categories needed for studying the mind. But Husserl thought Brentano had not fully recognized that we must sharply distinguish this enterprise from empirical psychology by explicitly rejecting all ‘psychologism’. As Husserl understood it, psychologism in its most basic form holds that principles of logical inference ultimately concern (or are justified by reference to) inductively discoverable processes in actual psychological subjects. Such an attempt to subordinate logic to psychology he saw as an intellectual disaster, for it leads to a relativism about truth that would undermine the rational presuppositions of all theoretical inquiry, and circularly assumes, in its search for the laws of mental processes, the very norms of reason it purports to legitimize. The defining task of logic, as Husserl saw it, is to investigate the presuppositions of theoretical knowledge—including those relating to meaning and perception (2001a: Prolegomena, §§1–16). Thus ‘logic’ in his sense extends well beyond the formal study of valid inference and includes (phenomenological) philosophy. To completely avoid psychologistic error, it must, Husserl thought, in all its subdivisions remain as steadfastly a priori as geometry, and hold its justification aloof from empirical claims (2001a: Prolegomena, 75–76, 149–150, 153–154, 165–166, 168–170). Husserl thus approaches perception via an a priori investigation of the possibility of knowledge. Like Brentano, he makes clarifying the ways in which perception and other mental phenomena ‘refer to objects’ central. Unlike his teacher, he bases his conception of the intentionality of perception on the idea that the objects of perceptual appearance are sensory constants presented through changes in experience. For Husserl perceptual experience is an extended temporal process with a special sort of unity constitutive of its intentionality—the understanding of which is essential to making sense of the possibility of knowledge. Husserl early on illustrates his conception of sensory constancy with reference to colour, shape, and sound (2001a: V, §§2, 14). The surface appears to you uniform, unchanging in colour, even as you experience that appearance differently depending on changes in viewing conditions. The box appears to remain the same cubical shape as you experience this appearance differently with changes in perspective. Your experience as you hear the violin’s adagio or the twittering of birds can also undergo change with attention, though the sounds heard do not thereby appear to change. This is the kind of constancy amid experiential flux that makes sensory appearance refer to an object—an object that goes beyond (‘transcends’) what is strictly speaking a part of (something ‘immanent to’) the experience. It must be emphasized that, in making perceptual constancy crucial to sensory intentionality, Husserl remains within the sphere accessible to first person reflection. He does not conceive of visual constancy as a uniformity of what is seen through change in correlative retinal stimulation. He is interested in object constancy amid experi­ ential flux, our awareness of which does not depend on our knowing anything about (e.g.) our retinas and their condition. Before seeing how Husserl developed this account further, we must consider how to align it with his general view of intentionality, the goal of which is to elucidate the intentionality of experience—where an ‘experience’ is for him a component of someone’s ‘stream of consciousness’, which he takes to include episodes of conceptual thought and volition as well as sense experience and imagery (2001a: V, §1). He starts from paradigms

140   Charles Siewert of ‘intentional mental acts’ Brentano would recognize, including perception, but also judgement, doubt, love, imagination, etc., and the Brentanian observation that in each case we may look to the accusative of the corresponding verb to identify what the act is ‘directed toward’—its object of reference: in judgement something is judged (about), in loved something loved, in imagination something imagined, etc. (2001a: V, §10). The fundamental points for Husserl then are these. First, we must distinguish between the object to which the act refers, and the object as it is referred to—that is, the manner in which the act is directed towards it (2001a: V, §17). Husserl illustrates this with an example in which the object to which two thoughts refer—the German Kaiser—is the same, though he is differently thought of: alternately as the son of Emperor Frederick III, and as the grandson of Queen Victoria. He then goes on to distinguish two important general ways in which the manner of reference may be the same or differ—one (just illustrated) involves a difference in the ‘matter’ or ‘interpretive sense’ of the act, a difference in ‘as what’ the object is referred to. The other involves a sameness or difference in ‘act quality’. (2001a: V, §§20– 21). For example, one may judge that the Kaiser is the grandson of Queen Victoria—and one may also doubt or wonder whether he is. Here we see the matter/sense of the mental acts remains the same, though they differ in quality. Likewise, quality may remain the same while matter varies. We see here too that the matter of an act can correspond to a whole proposition. (Husserl would say that the object to which the act refers is typically not a proposition, but the state of affairs that makes it true.) Finally, every intentional experience must have both quality and matter, and in virtue of this has intentionality or mental reference. However, it is not necessary that there exist an object to which such an act refers. It may be true that you are thinking of the god Jupiter, and what your experience refers to is Jupiter, but this does not entail that there is someone—the god Jupiter—of whom you are thinking (2001a: V, §11). This basic picture is elaborated and revised in elusive ways post-Investigations, when (in Ideas I (1982)) Husserl replaces his ‘matter/quality’ terminology with talk of an act’s ‘noema’ and its components. But leaving these complications aside, we can see how Husserl’s quality/matter schema raises questions about the relationship between thought and judgement on the one hand, and sense experience on the other. On his view, much as what is thought about is thought of as something, what is perceptually apparent also appears to one as something—the appearance involves an ‘interpretation’; it has an ‘interpretive sense’. But are we in either case to regard this as a ‘conceptualization’ of the object? Is the content of perception also ‘conceptual’? And just as we say someone judges that the paper is white, and distinguish the matter/sense/content of this judgement from the state of affairs judged, may we also say someone sees that the paper is white, attributing to one’s visual experience the very same content and object as that of the judgement? Husserl’s Sixth Logical Investigation bears on such questions, but the discussion is tentative and its interpretation uncertain. For present purposes we may limit ourselves to the following. Husserl discusses perception’s role in understanding the reference of demonstrative expressions such as ‘this’ and ‘that’ (and so its role in forming the thoughts they express)— and he holds that the way that we experience the object picked out by ‘this’ so as to understand the reference cannot be expressed in terms of general concepts applied to the object; intentional experience here involves what he calls a ‘non-attributive’ sense (2001a:  I, §§24–27, VI, §5) Further, there is indeed, for Husserl, an experience of seeing that the paper is white, as distinct both from one of judging that the paper is white, and of merely seeing

Phenomenological Approaches   141 the white paper. He uses this ‘seeing that’ locution not to invoke a ‘factive’ sense of ‘see’, but to indicate that the experience is ‘categorially’ structured, and ‘predicative’—which is not guaranteed, for Husserl, by an object’s merely appearing ‘as’ something in a ‘straightforward’ perception. Seeing that—a ‘categorial’ perception—essentially involves (is ‘founded on’) such ‘straightforward’ perception of an object—for example seeing the white paper. This latter, basic object identification is made possible, again, through the sort of constancy phenomenon discussed earlier, distinctive of sensory appearance. This sensory appearance constitutes one form of ‘intuition’ of an object, where ‘intuitive’ intentional experiences have the distinctive role of ‘fulfilling’ other intentional experience, in the sense in which the visual appearance of white paper ‘fulfils’ the judgement that there is some white paper by (to some extent) confirming or warranting it. Husserl recognizes that to get from merely seeing the white paper to seeing that the paper is white (so as to confirm the corresponding judgement)—that is, to move from straightforward to ‘categorial’ perception—one cannot simply pile up object perceptions—as if (absurdly) seeing that the paper is white consisted in seeing three items corresponding to: ‘the paper’, ‘is’, and ‘white.’ Husserl proposes to understand the epistemically crucial link between mere object perception and categorial perception via a kind of sensory attention in which an individual aspect of the object is ‘cast into relief’—you may, he says, be struck by the ‘peculiar colouring’ or ‘noble form’ of an object seen, in a way that does not necessarily involve applying a ‘general presentation’ of the specific type of colour or form involved. Now, however, not just seeing the white paper, but in this special way attending to the paper—seeing the white in the paper—makes it possible for us to acquire concepts from experience and—once those concepts are possessed—it enables one also to see that something is the case, that is, to have a form of categorial perception (2001a: II, §21, VI, §§40–43, 45–48). Subsequent to the Investigations, Husserl enriches his conception of the type of unity in experience over time that makes object perception possible, and allows us to confirm or disconfirm (‘fulfil’ or ‘frustrate’) prior experience through later experience. His idea that all perception involves an ‘indeterminate’ ‘horizon’ whereby future experience is ‘anticipated’ is crucial here (1982: §§24, 35, 41, 113; 2001b: §§1–3). Husserl’s notion of horizon includes a recognition that there is a ‘field’ of visual experience with a ‘fringe’, ‘margin’, or ‘periphery’ where what appears, appears less determinately in its features—for example, shape, size, location—than does what one is focusing on, what one is looking at. But the area of what is less determinately apparent is not confined to some region at the limits of what is experienced—a ‘fringe’. For example, as one looks at a series of words in a text as one is reading, the horizon of one’s experience would include the area immediately surrounding whatever bit of the text one is looking at, as well as the area within it—for the individual letters and their parts are not as determinately apparent as they would be if one focused on each letter individually, as typically one does not. Further, the visual horizon ordinarily includes not only the indeterminately but still visually apparent surfaces and areas before one, but also non-apparent sides and parts of what appears. For on Husserl’s view, part of what makes your perspectivally varying, temporally extended visual experience refer to an object at least partially constant in its spatial features is your successful ongoing ‘anticipation’ of the appearance of its as-yet-hidden, unapparent aspects. Specifically what shape, texture, etc. you visually anticipate on the far side of the object you are looking at is left fairly open—quite indeterminate relative to the appearance of the sides so far facing you. On the other hand, you do not merely anticipate ‘some surface contour or

142   Charles Siewert other’. Some future appearances would reveal the earlier experience to be illusory—by running contrary to what was anticipated—one would, we might say, ‘experience (visual) disillusionment’. This links to the idea that perceptual experience involves an ‘interpretive sense’ (a part of what Husserl came to call the ‘noema’). The sense of your visual experience—what you experience the thing seen as—will ‘predelineate’ the range of how it can appear from additional perspectives while these appearances still remain in ‘harmony’ with those that have gone before. In texts such as Thing and Space (1997) and Ideas II (1989) Husserl emphasizes the uniqueness of the experience of one’s own body and connects this to his notions of perceptual horizons and anticipation. He draws a distinction between the Körper—your body as an object among others, site of various physico-chemical processes—and the Leib—your body as you yourself experience it in normal active life. The latter sort of bodily self-experience is involved in the sort of the anticipation essential to sensory intentionality. For how one anticipates the hidden or less determinately apparent aspects of an object will appear is somehow contingent on one’s movement with respect to it—and one experiences the fulfilment (or frustration) of such anticipations through the experience of one’s own body and its movement. For Husserl then, what is experienced as in space is experienced as having hidden or relatively more determinately experiencable aspects; spatial experience is thus, in this sense, essentially partial, perspectival. He then takes this, in turn, to justify his conception of proper phenomenological method, since it shows that no fulfilment of past spatial experience through satisfied anticipations of future experience is ever complete—ever, in his terms, ‘adequate’. Future experience can always offer past experience something more in the way of confirmation, and the prospect of disconfirmation of past appearances is never entirely ruled out. Husserl seems to infer from this that it is possible to philosophize about experience while suspending commitment to the existence of particular objects in the natural world that it reveals—and by means of this ‘phenomenological reduction’ achieve the sort of epistemic independence from assertions about them he thought his a priori investigation needed (1982: §§42–50). This understanding of the essence of spatial experience and its methodological significance joins with a view of self-awareness that reworks themes from Brentano’s account of inner perception. For multiple reasons Husserl rejects Brentano’s presentation/judgement analysis of perception generally, as well as the idea that our perceptual appearings themselves appear to us in a univocal sense of ‘appear’ and are continually objects of reflexive judgement (2001a: V, ch. 3). Thus he rejects Brentano’s notions of inner perception and consciousness. However, Husserl does maintain analogues of Brentano’s contrast between inner and outer perception, and his notion that all consciousness involves inner perception. For Husserl holds that while it is conceivable that there are no spatial objects one has actually experienced though one has an intact stream of consciousness—even then, one retains a primitive kind of consciousness of one’s own experience that is present in all experience. And Husserl says that when one is, in this way, conscious of experience, one cannot conceivably lack actual experience that is just as one is conscious of it being. A key difference between Brentano and Husserl here lies in the fact that for Husserl this primitive, ubiquitous consciousness of one’s own experience does not render it a continual object of appearance or of judgement—this is a ‘non-objectivating’ consciousness of one’s own experience. Nonetheless, this basic ‘self-givenness’ of experience, in which it—unlike

Phenomenological Approaches   143 its spatial objects—are ‘adequately’ (i.e. incorrigibly) evident, plays a role in grounding the conception of consciousness that his phenomenology aims to articulate, similar to that played by Brentanian self-presentation (1982: §§33, 34, 42, 46, 49, 138). The views just sketched might seem to oppose what in contemporary terms would be styled an ‘externalist’ doctrine about perceptual experience and its content. That is, it might seem that, for Husserl, no finite spatial experience, understood phenomenologically, ever guarantees that any actual spatial object has been perceived. Thus no ‘transcendent’ concrete particular to which a given perceptual experience may refer and by which it is fulfilled could ever be an essential constituent of such experience itself, or of its sense or noema. However, this interpretation is contested. Some (e.g. A.D. Smith 2002 and Zahavi 2003) see Husserl as committed to a ‘relationalist’ conception of perceptual experience, on which objects in space are indeed part of the content of perceptual experience, and are not left behind by methodological reduction, but retained to be considered merely under a certain ‘reduced’ aspect—namely, ‘as perceived’.

4 Merleau-Ponty Merleau-Ponty saw his main work, Phenomenology of Perception, as developing the insights of Husserl, and many of his central concepts are clearly Husserl-inspired. Merleau-Ponty’s approach is avowedly ‘descriptive’ in the sense that descends through Husserl from Brentano: phenomenology proposes to describe perception from ‘the perspective of consciousness’, rather than try to explain it from ‘the perspective of science’ (Merleau-Ponty 2012: xxi–xxii). And for Merleau-Ponty, as for Husserl, this is an effort to characterize the nature of perception as it is experienced by a perceiver—partly as a corrective to traditional theories that allegedly neglect or distort this perspective. As with Husserl, this is motivated partly by a desire to make intelligible the constancy-amid-flux sensory consciousness ordinarily exhibits—for again, perceptual object constancy is viewed as a basic form of intentionality, underlying an objective conception of the world. How is it possible for a perspectivally limited, ever-fluctuating experience to make stable objects apparent to us, constant in, for example colour, size, and shape? How can vision, for example, ‘be brought into being from somewhere, without being locked within its perspective’—so that objective thought, which common sense and science both take for granted, becomes possible? (2012: 69–74). Husserl-derived notions of ‘horizon’, of the ‘indeterminacy’ of experience, and of ‘anticipation’ all contribute to Merleau-Ponty’s answer. However, in certain respects Merleau-Ponty seems to depart significantly from Husserl’s approach. Merleau-Ponty’s text (unlike Husserl’s) is replete with references to experimental and clinical studies—particularly ones having to do with psychological deficits and pathologies—and these feature prominently in his argument. He does not see himself as bound by a strict a priorist methodology, and regards the distinction between the a priori and a posteriori as in some sense a relative matter. He holds that Husserl’s methodological reduction ‘cannot be completed’, and purports to motivate a phenomenological attitude towards perception, in which we attend to perception as we experience it, not by means of some global suspension of commitment to the reality of perceived things, but by recognition of the divergence between the character of objective sensory stimuli, proximal

144   Charles Siewert and distal, and the world as it perceptually appears to us—a divergence he thinks modern psychology helps make evident (2012: xxiv–xxviii, 7–10, 47, 51–54, 60–62). This perspective informs Merleau-Ponty’s Phenomenology from its opening pages, in which he attacks the idea of the ‘sensation as a unit of experience’—that is, of primitive non-intentional elements in experience, subject to processes of association or interpretation through judgement. Here he draws on the early Gestalt psychologists (who themselves had been influenced by Brentano’s school and by Husserl), and on the phenomenologist Aron Gurwitsch, whose lectures introduced Merleau-Ponty to Gestalt Psychology. Merleau-Ponty maintains that even simple perceptual experiences involve a distinction between ‘figure’ and ‘ground’ that goes beyond putative ‘meaningless’ sensory elements— ‘sensations’—and he argues that there is no compelling reason to posit such components of experience as subjective correlates of the perceived objects’ qualities (e.g. colours), or of proximal stimuli (e.g. retinal ‘images’)—as did those psychologists who held the ‘constancy hypothesis’ criticized by the Gestaltists. In objecting to sensations in this sense, Merleau-Ponty also in effect rejects Husserl’s idea of a primitive uninterpreted, non-intentional sensory ‘stuff’ of experience (‘sensory’ or ‘hyletic’ data). But he motivates this opposition in part by appeal to the Husserlian idea that experience is pervaded by a kind of indeterminacy incompatible with supposing its character is constituted out of a set of definite sensory qualities (2012: 6–7, 10–11). For instance, two lines, both apparent, may appear to you neither equal nor unequal in length; a person’s eyes may commonly look somehow coloured to you, even when (without a closer look) there is no specific colour they then appear to you; a many-sided crystal may well appear to you regular in its shape—although there is no specific number of sides it appears to you to have. Merleau-Ponty regards this indeterminacy of appearance as a ‘positive phenomenon’. By this he seems to mean that appearances that are similarly indeterminate with respect to specifically what is apparent (with respect, e.g., to apparent shape, size, colour) may nevertheless really differ in ways accessible to reflection. This idea figures importantly in his view of attention. He objects to the assumption that the direction of attention in perception involves merely bringing to consciousness what was already determinately in the mind—as one might shine a light on what was already there in the attic. What this misses, he thinks, is the aspect of attention exhibited constantly by the movement of our gaze, through which we learn about what is before us by enriching our experience of it: reflection reveals that what is at first only dimly prefigured in experience moves from indeterminacy to determinacy, so as to emerge more clearly for us, while what was more determinately apparent dissolves into the background (2012: 31–34). The ‘positive indeterminacy’ of experience, though ubiquitously evident in shifts of sensory attention, is nonetheless prone to neglect, since perception promotes its own oblivion by plunging our thought into the things we perceive, which makes us liable to read the determinacy of these things back into their manner of appearance. All this prepares us to understand better the notion of a visual (or more broadly a phenomenal) ‘field’. For we now see that there can be for us an indefinitely bounded region of space variously apparent in ways that cannot be cashed out in terms of a definite set of objects identified and attributed a set of properties. This connects with Merleau-Ponty’s appropriation of the Husserlian notion of ‘horizon’, and shapes his interpretation of the guiding ‘transcendental’ question he gets from Husserl, heir to Kant’s legacy, of how consciousness of objects is possible for us. While Merleau-Ponty, steeped in the Kantianism

Phenomenological Approaches   145 of his French teachers, certainly recognizes this heritage, he is determined to avoid what he sees as the ‘intellectualist’ pitfalls of their approach. This requires getting a proper phenomenological understanding of our actual experience, such as emerges from the critique of ‘traditional prejudices’, and using this to frame the challenge. When we do so, and take proper account of the positive indeterminacy of experience, we see that perspective should not be construed—in, for example, the visual case—merely in terms of what parts of the surfaces of objects are exposed to view, given one’s location. For it is far from true that even everything to be found in the facing surfaces is apparent to one. Much that is right before one’s eyes is, in a sense, ‘hidden’ by being not yet as determinately apparent as it can be. And this kind of ‘hidden-ness’ (not just the way things are hidden by being occluded) also contributes to the perspectival limitation in which perception escapes being ‘locked up’ (2012: 69–74). Just when Merleau-Ponty has posed the problem of how this escape is possible—to put it somewhat paradoxically, how there can be more to what one experiences than one experiences—his thought takes a curious turn. He does not address this central question directly. Rather he proposes we first concentrate at length on what is special about the perception of one’s own body—conjecturing that this will furnish the key to understanding our experience of everything else (2012: 74). What is special about the way I experience my body? Partly it is this: I experience a body as my own insofar as it is the body I experience whose movements seem to determine my indefinitely varying perspective on all other things, while affording me a uniquely limited capacity to vary my perspective on it (2012: 92–99). But this still leaves open questions about just how I experience this body’s movement. We might suppose that, even allowing for the special character just mentioned, my mind otherwise represents my own movements much as it represents the position and movements of other objects. The main difference is just that, in the case of my body, the position and movements represented figure in the execution of my intentions uniquely and fundamentally:  my body’s movements are those my mind chooses as basic means to the ends it sets. This is precisely the picture Merleau-Ponty defines his view against. On his account, my ordinary experienced bodily movement in the service of everyday tasks is not in this way merely the effect of some such separate internal planning, nor is it just this plus a cause of inputs to mental processes. Rather, my experienced movement is itself non-derivatively a way of understanding things; it is a way of being conscious of them, a form of intentionality, no less than the thoughts in which I may engage when reasoning—though this ‘motor intentionality’, as he calls it, is distinct in kind from the operations of the intellect. So, it’s not that I am only ‘directed at’ things through representations in my mind, which on their basis formulate commands to a certain body to attain goals it also represents. Rather, I am directed at things through the experienced (‘lived’) movements themselves—looking at things, and touching them; it is literally true then that my body thereby understands things perceived. In this sense I am one with my body: in experiencing my own bodily movement as I am engaged in looking at, touching, reaching, grasping, etc., I am conscious of myself ‘qua subject’—that is, as one who actively understands (2012: 248). His case for this position is elliptical, indirect, and still far from adequately analysed. But it seems to turn on the following considerations. First, there is an appeal to our ordinary experience of movements in executing tasks. From a phenomenological point of view, when I reach to touch my knee, I don’t need to think of how to move, and I am aware, in

146   Charles Siewert the way I am feeling myself reach, of my success (or failure) in executing my intention. When the phone rings and I answer, I ordinarily experience my effortless adjustment of posture, and reach for the phone in a way appropriate to my situation with no awareness of a selection of these movements from others possible as means to a given end. Nevertheless, I am able to smoothly adapt my movements to varying circumstances to secure the same end—all such movements equally experienced as (e.g.) ‘reaching for the phone’—as varying manifestations of a unified skill or bodily ‘habit’. Generally, I may say I experience an indefinite range of my own movements as in this and similar ways functionally equivalent. Thus I have a ‘body schema’: a systematic but open-ended capacity for engaging in patterns of movement, experienced as functionally equivalent, such as my task and situation require (2012: 100–105, 142, 149–151). But one might still think such movements are generally to be regarded as responses to commands issuing from the choices of some inner planning faculty—even if we do not experience them as such. Merleau-Ponty argues this would be a mistake. For there are several phenomena he thinks this would leave us no satisfactory way of understanding. For example, he asks, how can creatures as primitive as insects adapt their behaviour to serve their ends without deliberation (as when a beetle substitutes the action of another limb for that of one that has been severed)? Why do our habits sometimes persist when our avowed beliefs should make the futility of our alleged ‘choice’ of movement obvious (as when an amputee tries to stand on his phantom leg)? Why do severe deficits in subjects’ ability to perform bodily ‘movements to order’ nevertheless leave largely intact their capacity to exhibit similar motor activity as their everyday tasks demand it? (2012: 80–85, 105–113). Suppose we agree with Merleau-Ponty that to respond properly to such questions we must give up thinking of skilled movement as always the product of planning and commands. How should we think of it instead? His alternative is rooted in the phenomenological claim that the practical demands of your tasks are part of what you ordinarily perceive in your situation, as what you are trying to do makes certain things perceptually salient for you. For instance, as you are about to use the scissors, you see where they are to be grasped, and then you see where the material is to be cut. Generally, much as you are conscious of what is required for the completion of a pattern—such as a melody you begin to hear—you see what is to be done with what lies before you, given your goals, and you are (without deliberation, conscious or unconscious) motivated by such experience to do what is called for (to complete this ‘melody’). Moreover, given your projects, you see opportunities for action—for example, something appears to you as reachable, a space appears as a way through. (Here Merleau-Ponty’s account joins with later Gibsonian talk of perceived ‘affordances’ 2012: 108–109, 113–115). And, as you acquire more skills, your capacity to spontaneously recognize and respond to the potential offered in your situation becomes less constrained by circumstances. You acquire a liberty lacking altogether in non-human animals locked into more stereotyped responses to their environment, a sort of spontaneity degraded in human subjects whose brain damage sharply diminishes what opportunities they can perceive. Such pathological cases cast into relief our normal, culturally shaped and loosely constrained perception of the practical significance of what we encounter, which partly constitutes what it is for us not just (like other animals) to have an environment, but—as Merleau-Ponty puts it (alluding to Heidegger)—to be in the world (2012: 89–91, 131–141). On this basis Merleau-Ponty argues that it makes sense to regard movement itself as a form of understanding. For we see that movement is highly flexible or adaptive in pursuit of

Phenomenological Approaches   147 one’s goals, and experienced in a way sensitive to norms (of success or failure). And though it reflects and is guided by one’s plans and intentions, it does not simply derive its teleologically adaptive, norm-sensitive status from a causal relation to a separate goal-oriented mental activity, such as might be found in some planning subsystem. This suffices to make movement itself an exercise of ‘understanding’ in a non-trivial sense (2012: 143–148). We will resist this, if we cling to a theoretical tradition for which what is understood always includes some general idea or concept that one can understand and employ in thought even when not applying it to perceived instances. But we should recognize that what is ‘understood’ through sensorimotor activity is not a concept or rule whose formulation we might apprehend independently of its concrete application, but belongs rather to the category of ‘style’. In this sense ‘style’ is involved in understanding a work of art. We recognize that what is expressed in an artwork strongly resists paraphrase or translation into other media or languages, since it is so closely bound up with a specific sensible manner of expression. Accordingly, one cannot grasp the specific style belonging to the work without perceiving this manner of expression in a concrete instance. Similarly, we can understand the ‘style’ of our movements through which things are perceptually accessible to us, and the ‘style’ of appearance they present to us, only in performing such movements ourselves and encountering what appears; no formulation comprehensible in abstraction from such sensorimotor engagement will provide the same understanding. In this sense, the experienced unity of one’s own body in action is like the unity perceived in a work of art (2012: 149–155). In Part Two of Phenomenology of Perception, Merleau-Ponty proposes to put this conception to work illuminating a range of basic perceptual phenomena:  sensory integration; perceived orientation, depth and motion; and perceptual constancies. We can see how sensory integration is achieved within a modality—as when we resolve our double vision to see a single thing—if we think of this as the exercise of a motor understanding of how to focus and coordinate our eyes (2012: 239–242). And synaesthesia and what are now called ‘cross modal’ phenomena are most intelligible if we understand the boundaries of the sensory modalities in terms of the motor skills they involve, for this can account for how the senses interpenetrate and mutually condition one another in experience as they do (2012: 237–239, 242–244). Perception of orientated space—specifically, of ‘up and down’—also becomes, Merleau-Ponty maintains, more intelligible from the perspective of sensorimotor understanding. Working from George Stratton’s late nineteenth-century experiments, Merleau-Ponty observes that a subject wearing lenses that invert the usual pattern of retinal stimulation says (after some initial confusion), that things ‘appear upside down’. But gradually, after bodily interaction with their surroundings, he maintains that things begin to appear to him oriented more or less as before (2012: 254–265). It is legitimate to accept (as we should) the accuracy of these reports, Merleau-Ponty argues, provided that the perception of up/down orientation is constitutively dependent on the perceiver’s motor skills for dealing with what is thus oriented. Finally, consider the phenomenology of perceptual constancy. The perspectival experience of a size or shape, for instance, is not an experience of a determinate measureable size or shape, and the object does not appear to shrink as I move away, or to morph as I tour it. Thus, while I am given perspectival variation in the experience, I am not given data from which the true objective constant size or shape of a thing could be inferred—as per ‘intellectualist’ accounts of perception. Nonetheless, my perspectivally variant experience of size and shape somehow bears

148   Charles Siewert on my experience of an invariant size and shape. We can understand this, according to Merleau-Ponty, if we see it as essential to experiencing a given constant size or shape that we exercise and are set to exercise a capacity for movements (in looking at, in touching) that appropriately vary systematically with these perspectival changes in experience, so as to generate appearances of constancy. Size and shape constancy is not an inference from data, but an achievement of motor understanding (2012: 312–331). We can now see how all this yields Merleau-Ponty’s answer to the question of how experience can escape being ‘locked within its perspective’, and his take on Husserl’s notion of the sort of ‘anticipation’ of experience whose fulfilment makes sensory intentionality (and thus an objective conception and knowledge of the world) possible. One experiences a stable thing by anticipating the style of its future appearances in the style of one’s movements. This movement (of, e.g., looking and touching) is not merely derivatively prospective—it is not guided by a prediction, as by a separable representation of ‘what will happen if . . .’. Rather the movement itself constitutes the anticipation of what will appear, as when one shapes one’s grip in reaching for something ‘in anticipation’. And by successfully anticipating experience so as to sustain the appearance of a complex style of constancy (in shape, colour, position, etc.), experience is confirmed as the perception of things in the world, rather than illusion or hallucination (2012: 310–311, 349–360). Merleau-Ponty agrees with Husserl that no such fulfilment is ever complete; no experience affords us absolute certainty there is an object of which it is a correct experience, and precludes all possibility of any future reason to doubt this. However (and here perhaps he parts with Husserl) this does not mean that we can remain generally certain of the character of our own experience, while judgements about ‘the external world’ are either doubted or suspended en masse. For in recognizing the possibility of visual error in a particular case, phenomenologically I am left with a merely disjunctive characterization of my experience: either this is a case of genuine (‘factive’) seeing (there is actually (e.g.) an ashtray that I see)—and consequently my experience is essentially relational, not ontologically independent of the thing seen—or else this is a case where my experience is merely as if I am seeing a thing (I ‘seem to see it’). But I wouldn’t understand the second disjunct at all, if I did not think such experience similar to some case in which the first sort of disjunct obtains, where I took myself to genuinely see something, and the experience to be object-involving. (Merleau-Ponty’s views here might be compared to more recent ones in the neighbourhood—see Heather Logue’s entry on disjunctivism.) This, Merleau-Ponty thinks, shows that I cannot rationally, even in philosophical reflection, globally withdraw commitment to the reality of things perceived, and retreat to some certainty regarding my experience: to question whether there really is something I see is equally to put into question what sort of experience I am having, and the reality of a world transcending my experience of it is not something I can ultimately intelligibly doubt (2012: 308–311, 359–360, 393–396).

5  Related contemporary work It remains to indicate briefly some writings where one may find the phenomenological authors discussed above situated with respect to more recent philosophy, and see how the

Phenomenological Approaches   149 tradition to which they belong inspires new work. General treatments of topics in phenomenology, many relevant to philosophy of perception, and incorporating reference to recent debates, can be seen in Luft and Overgaard (2011). A diverse selection of essays representing contemporary phenomenology (several bearing on the themes discussed here) are collected in Zahavi (2012). Helpful detailed overviews of Brentano’s and Husserl’s views of perception sensitive to contemporary concerns are found in Mulligan (1995, 2004). Hopp (2011) interprets and develops Husserl’s Investigations-era views on perception to engage with current debates—particularly those stemming from the ‘conceptualist’ view of John McDowell and opposed accounts of ‘non-conceptual content’. Yoshimi (2011) develops a framework for showing how ‘the dynamics of neural activity, as described using a connectionist formalism, relate to the dynamics of consciousness, as described by Husserl’. A. D. Smith (2002, 2008) defends a theory of perception influenced by Husserl as well as an externalist reading of his position. Opposing treatments of the question regarding Husserl’s internalism or externalism are found in the recent general accounts of Husserl offered in D. W. Smith (2009) and Zahavi (2003). Controversies over the alleged ‘intellectualism’ of McDowell’s understanding of perception and action, raised by Hubert Dreyfus from his Husserl-averse, and Heideggerand Merleau-Ponty-inspired viewpoint, are variously discussed in a collection of papers (Schear 2013) in which interpretation and extension of the phenomenological tradition often figures importantly. Dreyfus (2004) explains how he takes his view of Merleau-Ponty to bear on contemporary cognitive science. Kelly (2004), Wrathall (2004), and Carman (2008) propose and defend interpretations of Merleau-Ponty influenced by Dreyfus’ perspective. Romdehn-Romluc (2007) and Siewert (2005, 2013), partly responding to Dreyfus, offer their takes on crucial aspects of Merleau-Ponty’s account. Gallagher (2005) works out his own conception of ‘body schema’, against the backdrop of his reading of phenomenology. Husserl’s and Merleau-Ponty’s writings generally provide an important background to recent work in ‘embodied cognition’. Thompson (2007) proposes one such ambitious, explicitly phenomenological ‘embodied’ view of perception, building on Husserl and Merleau-Ponty and developed in the context of the philosophy of biology and dynamic systems approaches in neuroscience. Noë’s (2004, 2012) sensorimotor view of perception and of the ‘problem of presence’, though less directly engaged with Merleau-Ponty’s account, significantly resembles it in certain respects, and raises similar issues in its challenge to orthodoxy.

References Brandl, J. (2005). ‘The Immanence Theory of Intentionality.’ In D. W. Smith and A. Thomasson (eds), Phenomenology and the Philosophy of Mind (pp. 167–182). Oxford: Oxford University Press. Brentano, F. (1972). Psychology from an Empirical Standpoint, trans. T. Rancurello, D. Terrell, and L. McAlister. London: Routledge. Carman, T. (2008). Merleau-Ponty. London: Routledge. Dreyfus, H. (2004). ‘Merleau-Ponty and Recent Cognitive Science’. In T. Carman and M. Hansen (eds), The Cambridge Companion to Merleau-Ponty (pp. 129–150). Cambridge: Cambridge University Press.

150   Charles Siewert Gallagher, S. (2005). How the Body Shapes the Mind. Oxford: Oxford University Press. Hopp, W. (2011). Perception and Knowledge:  A  Phenomenological Account. Cambridge: Cambridge University Press. Husserl, E. (1982). Ideas Pertaining to a Pure Phenomenology and to a Phenomenological Philosophy: First Book, trans F. Kersten. Dordrecht and Boston MA: Kluwer. Husserl, E. (1989). Ideas Pertaining to a Pure Phenomenology and to a Phenomenological Philosophy: Second Book, trans. R. Rojcewicz and André Schwer. Dordrecht and Boston MA: Kluwer. Husserl, E. (1997). Thing and Space: Lectures from 1907, trans. and ed. Richard Rojcewicz. Dordrecht and Boston, MA: Kluwer Academic Publishers. Husserl, E. (2001a). Logical Investigations, trans. J. Findlay. London: Routledge. Husserl, E. (2001b). Analyses Concerning Active and Passive Synthesis, trans. A. J. Steinbock. Dordrecht and Boston MA: Kluwer. Kelly, S. (2004). ‘Seeing Things in Merleau-Ponty’. In T. Carman and M. Hansen (eds), The Cambridge Companion to Merleau-Ponty (pp. 74–110). Cambridge:  Cambridge University Press. Luft, S. and Overgaard, S. (eds) (2011). The Routledge Companion to Phenomenology. London: Routledge. Merleau-Ponty, M. (2012). Phenomenology of Perception, trans. Donald A. Landes. London: Routledge. Mulligan, K. (1995). ‘Perception’. In B. Smith and D. W. Smith (eds), The Cambridge Companion to Husserl (pp. 168–238). Cambridge: Cambridge University Press. Mulligan, K. (2004). ‘Brentano on the Mind’. In D. Jacquette (ed.), The Cambridge Companion to Brentano (pp. 66–97). Cambridge: Cambridge University Press. Noë, A. (2004). Action in Perception. Cambridge, MA: MIT Press. Noë, A. (2012). Varieties of Presence. Cambridge, MA: Harvard University Press. Romdehn-Romluc, K. (2007). ‘Merleau-Ponty and the Power to Reckon with the Possible’. In T. Baldwin (ed.), Reading Merleau-Ponty (pp. 44–58). London: Routledge. Schear, J. (ed.) (2013). Mind, Reason, and Being-in-the-World: the McDowell/Dreyfus Debate. Abingdon: Routledge Press. Siewert, C. (2005). ‘Attention and Sensorimotor Intentionality’. In D. W. Smith and A.  Thomasson (eds), Phenomenology and the Philosophy of Mind (pp. 270–294). Oxford: Oxford University Press. Siewert, C. (2013). ‘Intellectualism, Experience and Motor Understanding’. In J. Schear (ed.), Mind, Reason, and Being-in-the-World: the McDowell/Dreyfu Debate (pp. 194–226). Abingdon: Routledge Press. Smith, A. D. (2002). The Problem of Perception. Cambridge MA: Harvard University Press. Smith, A. D. (2008). ‘Husserl and Externalism’. Synthese, 160, 313–333. Smith, D. W. (2009). Husserl. London: Routledge. Thompson, E. (2007). Mind in Life. Cambridge, MA: Harvard University Press. Wrathall, M. (2004). ‘Motives, Reasons and Causes’. In T. Carman and M. Hansen (eds), The Cambridge Companion to Merleau-Ponty (pp. 111–128). Cambridge: Cambridge University Press. Yoshimi. J. (2011). ‘Phenomenology and Connectionism’. Frontiers in Psychology, 2, 1–12.. Zahavi, D. (2003). Husserl’s Phenomenology. Stanford, CA: Stanford University Press. Zahavi, D. (ed.) (2012). The Oxford Handbook of Contemporary Phenomenology. Oxford: Oxford University Press.

Pa rt  I I

C ON T E M P OR A RY PH I L O S OPH IC A L A PPROAC H E S

Chapter 8

Perceptua l R epr esen tation/ Perceptua l Con ten t Bence Nanay

There are two major questions about perceptual representations: (1) Do they exist? (2) What are they?

Are there perceptual representations? Do perceptual representations exist? Twenty years ago, the vast majority of philosophers of perception would have agreed that they do, but this is no longer so. In fact, this issue seems to be one of the most widely discussed questions about perception these days (see Brogaard 2014 for the current state of the debate). I will give (a) some reasons to think perceptual representations do exist, (b) some reasons to think that they do not, and (c) some potential ways of resolving this debate. An important clarification and disclaimer before we begin: (1) and (2) are about perception in general: not just about conscious perceptual experiences but all perception, conscious and unconscious. Both debates have sometimes been hijacked by those primarily concerned about consciousness, but this is a mistake unless one is happy to deny that there are unconscious perceptual processes (see also Prinz, Chapter 19 of this volume). Those who are interested in the peculiarities of conscious perception can read Siewert, Chapter 7 and Peacocke, Chapter 9 of this volume. This chapter is about perception in general.

Representationalism Many of our mental states are representations: my belief that it is raining outside represents a putative state of affairs: that it is raining outside. If I am afraid of a tiger, this fear is also directed at, or is about, something: a tiger. In other words, many mental states refer to something, they are about something: they have content. But then it is tempting to assume that perceptual states are also representations: they also have content: if I see a cat, it would

154   Bence Nanay be natural to say that my perceptual state is about this cat. The cat (or some of the cat’s properties) is part of the content of my perceptual state.1 A further reason for being representationalist (see also Pautz 2010): it allows us to give simple explanations for illusions and hallucinations, as well as for perceptual justification. If we think of perceptual states as representations, then veridical perception amounts to having a correct perceptual representation, whereas illusion and hallucination amount to perceptual misrepresentations (not everyone thinks this is an advantage; see Brewer 2006 and see Logue, Chapter 11 of this volume for analysis). And as perceptual states are at least sometimes capable of justifying beliefs, thinking of them as representations allows us to explain perceptual justification as a relation between two different representations: a perceptual and a non-perceptual one (see also Siegel and Silins, Chapter 41 of this volume). Thinking of perceptual states as representations is also a default assumption of (mainstream) perceptual psychology and vision science (see Burge 2005 and Nanay 2014 for summaries). But some philosophers (and some psychologists) are not convinced and they conceive of perception in a non-representational manner.

Anti-representationalism Anti-representationalism is the view that there are no perceptual representations. As it is a merely negative view, the question arises: what happens when we perceive, according to the anti-representationalist? There are a number of different suggestions, which fall into two broad categories: enactivism and relationalism. According to relationalism, perceptual states are, in part, constituted by the actual perceived objects. Perception is a genuine relation between the perceiver and the perceived object—and not between the agent and some abstract entity called ‘perceptual content’ (Travis 2004, Brewer 2006, Campbell 2002, Martin 2004, 2006, but see also criticism by Byrne and Logue 2008, and Burge 2005, as well as Crane 2006 and Logue, Chapter 11 of this volume). One reason why relationalism may seem appealing is that it captures the particularity of perception, the intuitively plausible assumption that the object of perception is always a particular token object, better than representationalism (see Soteriou 2000 for a summary). Suppose that I am looking at a pillow. What happens if someone, unbeknownst to me, replaces this pillow with another, indistinguishable, pillow? Most representationalists will say that my perceptual state is still the same, as this replacement does not make a difference to the content of my perceptual state. (Note that not all versions of representationalism are committed to this claim: those, for example, that conceive of content as Russellian, gappy, or ‘singular when filled’ [see, e.g., Tye 2007, Schellenberg 2010] are not.) But I am looking at two entirely different token objects before and after the swap. The relationalist thus insists that I have two completely different perceptual states. Like relationalists, enactivists also deny that there are perceptual representations, but they give a different (but not incompatible, see Noë 2004) positive account of

1  Most of my examples are about the visual-sense modality, but all the discussed views apply to all the sense modalities.

Perceptual Representation/Perceptual Content    155 perception (Brooks 1991, Ramsey 2007). According to one version of enactivism, perception is an active and dynamic process between the agent and the environment and this dynamic interaction doesn’t have to be (or maybe even couldn’t be) mediated by static entities like representations (Chemero 2009, Port and Van Gelden 1995). Another version of enactivism emphasizes that when we see a scene, the whole scene in all its details is not coded in our perceptual system. Only small portions of it are: the ones we are attending to. The details of the rest of the scene are not coded at all, but they are available to us all along—we have immediate perceptual access to them without representing them (O’Regan 1992, Noë 2004, esp. pp. 22–24). One may wonder whether these enactivist arguments give us reason to abandon the idea of perceptual representations per se or maybe only to conclude that they are not static or not detailed. In short, some of the enactivist arguments may give us good reason to prefer certain kinds of perceptual representations over others within the representationalist framework. But that is clearly not the aim of most enactivists, who want to reject the whole idea of perceptual representations.2

Ways of resolving this debate The debate about perceptual representation is a subtle one and both sides should be taken seriously. I offer four possible ways of resolving this debate: by (i) capturing some anti-representationalist intuitions within the representationalist framework; by (ii) disputing anti-representationalism on empirical grounds; by (iii) exploring the possibility that the two camps are talking about different phenomena; and by (iv) finding a framework where the two views can co-exist as different explanations in different explanatory projects.

Capturing anti-representationalist intuitions within the representationalist framework One way of resolving the representationalism versus anti-representationalism debate would be to account for the most important anti-representationalist considerations within a representationalist theory. The enactivist considerations against static and detailed, snapshot-like representations may be easier to accommodate within the representationalist framework (by conceiving of perceptual representations as dynamic and dependent on attention, see Clark 1997, Nanay 2010a) than the relationalist ones. As we have seen, a crucial relationalist argument comes from the particularity of perception: we always perceive token objects and the perception of different token objects constitute very different perceptual states. Representationalism, the argument goes, cannot account for this.

2  A

consequence of both versions of anti-representationalism is that perceptual illusion cannot be accounted for as misrepresentation. If I see an object as P but it is in fact Q, this cannot be explained in terms of perceptually misrepresenting the properties of this object. Different anti-representationalists give different alternative explanations for perceptual illusions. I will not say much about this—the question is discussed at length in Logue, Chapter 11 of this volume (see also Travis 2004, Brewer 2006).

156   Bence Nanay But is this argument conclusive? It has been argued that if we interpret perceptual content as ‘Russellian’, ‘gappy’, ‘Russellian gappy’, ‘Fregean gappy’, ‘singular’, ‘object-involving’, or ‘singular-when-filled’ (see, e.g., Soteriou 2000, Martin 2002, Loar 2003, Tye 2007, Schellenberg 2010, and see Chalmers 2004, 2006, Siegel 2006b, Bach 2007 for discussion), then we can account for the particularity of perception within the representationalist framework. The general idea is that perceptual content is different from the content of beliefs in that it depends constitutively on the perceived token object. This dependence can take various forms (see below), but the general idea is that this allows for a difference in our perceptual content when we are looking at the two pillows in the example I used above.3 According to a somewhat different suggestion, we could use the old idea that perception attributes tropes (property-instances that are logically incapable of being instantiated by two different entities at the same time) and not universals (Mulligan 1999, Mulligan et al. 1984, Campbell 1990) to argue for the particularity of perception within a representationalist framework (Nanay 2012b).

Disputing anti-representationalism on empirical grounds We can also bring in empirical considerations to decide this debate. There seem to be at least two well-documented empirical phenomena that are difficult to account for in an anti-representationalist manner: dorsal vision and the multimodal character of perception (see Nanay 2014). The first one is dorsal vision. Humans (and other mammals) have two visual subsystems that use different regions of our central nervous system, the ventral and dorsal streams. To put it very simply, the ventral stream is responsible for identification and recognition, whereas the function of the dorsal stream is the visual control of our motor actions. In normal circumstances, these two systems co-function, but if one of them is removed or malfunctioning, the other can still function relatively well (see Milner and Goodale 1995, Goodale and Milner 2004, Jacob and Jeannerod 2003 for overviews, see also Brogaard forthcoming, as well as Jacob, Chapter 12 of this volume). In healthy humans the way the dorsal and the ventral stream works can come apart in some circumstances, as in the case of the three-dimensional Ebbinghaus illusion. The twodimensional Ebbinghaus illusion is a simple optical illusion. A circle that is surrounded by smaller circles looks larger than a circle of the same size that is surrounded by larger circles. The three-dimensional Ebbinghaus illusion reproduces this illusion in space: a poker chip surrounded by smaller poker chips appears to be larger than a poker chip of the same diameter surrounded by larger ones. The surprising finding is that although our judgement and experience of the comparative size of these two chips is incorrect as we judge the first chip to be larger than the second one, if we are asked to pick up one of the chips, our grip-size is barely influenced by the illusion (Aglioti et al. 1995, cf. Gillam 1998, and Franz et al. 2003, see also Daprati and Gentilucci 1997 and Bruno 2001). The usual way of explaining this 3 Another way of accounting for the particularity of perception within the representationalist framework is to make a distinction between the demonstrative meaning (a la Kaplan 1989) and the content of a perceptual state, where the latter is different, but the former is the same when we are looking at the two pillows.

Perceptual Representation/Perceptual Content    157 finding is that our dorsal stream represents more or less correctly, but the ventral stream misrepresents. This is the representationalist way of describing the 3D Ebbinghaus case: we have two perceptual representations, a dorsal and a ventral one, and they represent the poker chip as having different size properties. But what can the anti-representationalist say? If perception is a relation between the perceiver and the properties of the perceived token object, then we have one perceptual relation here: the one between the perceiver and the perceived token poker chip. But then which property of the perceived object constitutes the other one of the two relata of this relation? The property we experience the chip as having or the one that our grip-size seems to be tracking? These two perceptual episodes are both relations to the very same token object: the same poker chip, and the properties of this same poker chip. And two different perceptual episodes cannot be constituted by the very same perceptual relation. If, on the other hand, as the enactivist says, ‘the world is our external memory’, then what serves as our external memory here, the property we experience the chip as having or the one that our grip-size seems to be tracking? It is difficult to see what would even be meant by having two different ‘worlds as our external memory’. The second empirical phenomenon that may cast doubt on the anti-representationalist framework is multimodal perception—the fact that our sense modalities interact in a variety of ways (see Spence and Driver 2004 for a summary as well as O’Callaghan 2008b, 2012 and Matthen, Chapter 30 of this volume, for philosophical overviews). Information in one sense modality can influence the information processing in another sense modality at a very early stage of perceptual processing (often in the primary visual cortex in the case of vision, for example, see Watkins et al. 2006). A simple example of this is ventriloquism, where vision influences our audition: we experience the voices as coming from the dummy and not from the ventriloquist (see Bertelson 1999). But there are more surprising examples: if there is a flash in your visual scene and you hear two beeps while the flash lasts, you experience it as two flashes (Shams et al. 2000). What is the most important for us from this literature is that the multimodality of perception presupposes that information from two different sense modalities is unified in a shared framework (see, e.g., Vroomen et al. 2001, Bertelson and de Gelder 2004). Noise coming from above and from the left and visual information from the upper-left corner of my visual field are interpreted by the perceptual system as belonging to (or bound to) the same sensory individual (whatever that may be—see below). This is easy for the representationalist to analyze: vision attributes a property to a part of the perceived scene and audition attributes a different property to the same perceived scene. The two different sense modalities represent the same scene as having different properties. To put it very simply, multimodal perception seems to require matching two representations, a visual with the auditory one. If we cannot talk about perceptual representation, how can we talk about what is being matched? The auditory-sense modality gives us a soundscape and vision gives us a visual scene, and our perceptual system puts the two together. It is difficult to explain this without any appeal to representations. The enactivist arsenal seems insufficient:  enactivists can appeal to the active exploration of the multimodal environment, but this is unlikely to help here: we are actively exploring the world that is given to us in both sense modalities—but this in itself requires multimodal integration. In short, the active exploration of the environment presupposes multimodal

158   Bence Nanay integration, which, in turn, seems to presuppose representations. They can also insist that the active exploration of the environment happens separately in each sense modality—but this is in conflict with the findings about multimodal integration very early in perceptual processing (as early as the primary visual cortex, see Watkins et al. 2006). The relationalist version of anti-representationalism also seems powerless, as the relation between the perceiver and the token perceived object that constitutes perception seems to be the outcome of this process of unifying multimodal information: our experience of the perceived token object (thus, presumably, the perceptual relation) is brought about by this unification process. The argument from multimodality seems to show that the phenomena anti-representationalists emphasize, be they the active and dynamic exploration of the environment or the relation to a token object, presuppose the coordination of information in the different sense modalities, but this can only be accounted for in representational terms.

Different explanada? The most promising strategy for the anti-representationalist to counter these empirical considerations is probably to insist that the claim that there are no perceptual representations is to be understood as a claim about perceptual experiences: it is perceptual experiences that are not representations; unconscious perceptual states may well be. In fact, at least some of the relationalist accounts are explicitly about perceptual experiences and not about perceptual states. Enactivists would be less happy with this proposal as they are often explicit about not limiting their attention to conscious or even personal-level phenomena (see esp. Ballard 1996, Noë 2004, 28–32). The suggestion then would be:  representationalism for unconscious (or maybe subpersonal-level) perception and anti-representationalism for conscious (or maybe personal- level) perception (I leave aside the differences between these two distinctions as well as the general worries about the personal/subpersonal divide (see esp. Bermúdez 2000). In fact, John McDowell could be interpreted as endorsing a version of this proposal: he argued that while a representationalist picture is the correct one for the sub-personal level, we should accept J. J. Gibson’s claims with regards to the personal level, which would make his view (at least in this respect) a version of enactivism (McDowell 1994). One important problem with this view is that the differential treatment of conscious and unconscious perception is difficult to square with the general aim of both the representationalist and the relationalist camp to give a general account of perception—that is, not just conscious perception, but perception per se.

Different explanatory projects? Finally, one could argue that we need both representationalism and antirepresentationalism, as they will be able to help us in different explanatory projects about perception (Nanay forthcoming). We can think of the representationalism versus antirepresentationalism debate as a debate about how to individuate perceptual states. As we have seen in the pillow example above, representationalism (or at least some versions thereof) lumps together the two perceptual episodes, whereas relationalism thinks of them as two very different perceptual states. Hence, the real question is whether these

Perceptual Representation/Perceptual Content    159 two perceptual states belong not just to the same type but whether they belong to ‘the same fundamental kind’ (Martin 2004, 39 and 43). The representational view says they do; the relational view says they don’t. Belonging to a ‘fundamental kind’ is supposed to ‘tell what essentially the event or episode is’ (Martin 2006, 361). It is easy to spot the essentialist assumptions in this way of characterizing the representationalism versus anti-representationalism debate. And the hope is that if we discard this essentialist assumption, we may be able to reconcile the two views. Why should we always individuate perceptual states in the same way? It seems that the individuation of biological traits in general is dependent on the explanatory project at hand; why would perceptual states constitute an exception? There are at least three ways of individuating biological traits: the functional (in terms of what they are for), the morphological (in terms of their structural properties), and the homological (in terms of their history). But depending on the explanatory project, biologists use different ways of individuating trait-types. Paleontologists do not consider the forelegs of an ancient amphibian to be wings. But embryologists do consider the morphologically very similar trait of the embryos of birds to be wings. Biologists and philosophers of biology then gave up on trying to find one unified theory of trait-type individuation: in different explanatory contexts, we should use different criteria for individuating biological traits (Nanay 2010b). The suggestion is that philosophers of perception would be well advised to make the same move. Our perceptual system is an evolved mechanism, just like birds’ wings. Thus, if we have good reasons to doubt that there is one and only one way of individuating wings, we also have a prima facie reason to doubt that there is one and only one way of individuating perceptual states. If the individuation of other biological traits depends on the explanatory project, we should expect that so does the individuation of perceptual states (see also Matthen 1998). In the case of some explanatory projects, we should individuate perceptual states according to representationalist criteria. If, for example, a vision scientist is doing research on the shape-recognition mechanisms of the human perceptual system, this may be the natural way to proceed. But in the case of some other explanatory projects, the relationalist way of individuating perceptual states is more appropriate: if a psychologist or philosopher is, for example, enquiring into the differences and similarities between vision and visual imagery, then thinking of perceptual states in a relationalist manner may be more helpful.

1 What are perceptual representations? Let us suppose that there are perceptual representations. The question now is: what are they? There are two very different approaches to characterizing perceptual representations (these are not two kinds of theories, but rather two kinds of general approaches). The first one is to start out with non-perceptual representations, typically beliefs or other propositional attitudes, and see how what we know about representations of this kind can be modified in order to apply to the perceptual case. Some think that there is no need for any modification: perceptual content is exactly the same as belief content. But most philosophers who think of perceptual content in this way allow for some differences—while

160   Bence Nanay nonetheless maintaining that we should use propositional content as a model for understanding perceptual content. Many of these proposed modifications aim to address the problem of the particularity of perception that I mentioned above. The general idea is that unlike the content of beliefs, perceptual content somehow depends constitutively on the token perceived object. These ‘Russellian’, ‘gappy’, ‘singular’, ‘object-involving’, or ‘singular-when-filled’ conceptions of perceptual content, however, are nonetheless conceptions of propositional content—as David Chalmers says, these accounts are thinking about perceptual content as a ‘structured complex’ (Chalmers 2006, 54—Thompson 2009 describes them even more aptly as ‘structured propositions’). The second approach to characterizing perceptual representations is to resist the temptation to start out with belief content and instead use a more basic way of thinking about content in general that can subsume both belief content and perceptual content. We have no reason to believe that all mental representations are linguistically or propositionally structured (see Crane 2009, but see also Siegel 2010a and 2010b). Some (but not all) mental states have content. Some of these (but not all of them) have conceptual content (see also Wright, Chapter 10 of this volume). And some of these (but not all of them) have propositional content. But perceptual states don’t. What would then be a general enough way of thinking about mental representations in a manner that is not necessarily propositional? A reasonable suggestion is to think of them as attributing properties to entities. And if we think of mental representations in general as attributing properties to entities, then we should think of perceptual representations as perceptually attributing properties to the perceived scene. Clarifications: (a) What are these properties? (b) What is the ‘perceived scene’? (c) What makes the attribution perceptual?

Perceptually attributed properties What are these properties that are perceptually attributed when we perceive? We have seen one way of understanding this question: are these properties tropes or universals? I  address two more ways in which the nature of the perceptually attributed properties needs to be specified: (i) What is the range of properties that are perceptually attributed? (ii) Are they determinates or determinables (or maybe super-determinates)?

Which properties are perceptually attributed? Beliefs can represent their objects as having any property. Perceptual states, in contrast, represent their objects as having a limited set of properties. Some plausible candidates include having a certain shape, size, colour, and spatial location. The list may be extended but it will not encompass all properties. The property of having been made in 2008 in Malaysia is unlikely to be represented perceptually—it is a property that is likely to be attributed by a non-perceptual state. The question is then about which properties are represented in perception and which ones are not. A couple of quick examples: it has been argued that we perceive objects as

Perceptual Representation/Perceptual Content    161 trees and tables (Siegel 2006a), as being causally efficacious (Siegel 2005, 2009), as edible, climbable, or Q-able in general (Nanay 2011a, 2012a), as having action-properties (Nanay 2013), as agents (Scholl and Tremoullet 2000), as having some kind of normative character or value (Kelly 2010, Matthen 2010), as having dispositional properties (Nanay 2011b), as having moral value (Kriegel 2007), and as affording certain actions (for very different versions of this claim, see Gibson 1966, 1979, Bach 1978, esp. 368, Jeannerod 1994, esp. section 5, Jeannerod 1997, Jacob and Jeannerod 2003, esp. 202–204, Humphreys and Riddoch 2001, Riddoch et al. 1998, esp. 678). Depending on our view on what range of properties we attribute perceptually, we end up with a very different view of perceptual content and, as a result, of perception in general.

The determinacy of perceptually attributed properties Another important question about perceptually attributed properties concerns their degree of determinacy (Johnston 1921, Funkhouser 2006). Being red is determinate of being coloured, but determinable of being scarlet. The determinable-determinate relation is a relative one: the same property, for example, of being red, can be the determinate of the determinable being coloured, but the determinable of the determinate being scarlet. Properties with no further determinates, if there are any, are known as super-determinates. Are the perceptually attributed properties super-determinates? It has been argued that quite often they are not (Dennett 1996). Some of the properties we perceptually attribute to the perceived scene are determinates or even super-determinates. But some others are determinable properties. Our peripheral vision is only capable of attributing extremely determinable properties. But even some of the properties we perceptually attribute to the objects that are in our fovea can be determinable. The perceptually attributed properties differ in their determinacy and, as we shall see below, part of what this difference in determinacy depends on is a difference in our perceptual attention.

Sensory individuals If we have clarified what properties are attributed in perception, we need to ask what our perceptual system attributes these properties to. In other words, what are the individuals that we perceptually represent as having these properties? Following Cohen 2004, I call these individuals ‘sensory individuals’. I’ll address two questions about sensory individuals: what they are and how they show up in perceptual content. One widespread view about sensory individuals is that they are ordinary objects like apples and chairs (Matthen 2005, Pylyshyn 2007, Cohen 2004, Matthen 2004, 2010). Another, much less widespread, one is that they are regions in space-time (Clark 2000, 2004). The ordinary-object view is seen as the more promising one, both on conceptual (Cohen 2004, Matthen 2012, Siegel 2002) and on empirical (Blaser et al. 2000) grounds (see also Casati, Chapter 20 of this volume). But, to make things even more complicated, it is not clear that sensory individuals of different sense modalities are the same. It has been argued that while the sensory individuals of vision are ordinary objects, in the auditory-sense modality they are sounds

162   Bence Nanay (O’Callaghan 2008a, Nudds 2001, 2010). This suggestion, in turn, raises various questions about what sounds are (Kulvicki 2008, O’Callaghan 2007, Pasnau 1999, Nudds and O’Callaghan 2009, Casati and Dokic 1994). Similar suggestions have been made about olfaction, where the sensory individuals are supposed to be odours (Lycan 2000, Batty 2010, 2011). See Nudds, Chapter 15 and Smith, Chapter 17 of this volume on these sense modalities. Another important question about sensory individuals is about how they show up in perceptual content. The classic representationalist view is that they are also represented by our perceptual states: both the attributed properties and the sensory individuals that these properties are attributed to are part of our perceptual content. But, partly under pressure to account for the particularity of perception, it has been suggested that the sensory individual does not need to be represented: only the properties are and there is a ‘gap’ in the perceptual content where actual objects stand in for sensory individuals (Tye 2007, Schellenberg 2010). Although, as we have seen above, most of these proposals consider perceptual content to be propositional, this is not a necessary feature of this general strategy. If we, as I suggest, think of perceptual content as the perceptual attribution of properties, this leaves open the possibility that the entity these properties are attributed to is not part of the perceptual content, but ‘fills the gap’ that is in the perceptual content. This would be a move equivalent to the one made by the advocates of the ‘Russellian’, ‘gappy’, ‘singular’, or ‘singular if filled’ accounts of perceptual content and it would address the problem of the particularity of perception in a similar manner.

Perceptual content What makes this attribution of properties perceptual? The sentence ‘The cat I am looking at is wet’ also attributes properties to the perceived object, but it nonetheless does not do so perceptually. Without intending to come up with a necessary and sufficient condition for perception, it needs to be pointed out how perceptual content differs from the content of this sentence. There is no easy way to draw this distinction. One important potential difference is that while the entity that the properties are attributed to is propositionally identified in the sentence, it is identified spatially in the perceptual case (see Peacocke 1989, 1992). Perceptual content, in short, is not propositionally but spatially organized (on this classic difference see, e.g. Kosslyn et al. 2006). And this leads to another difference between the content of this sentence and perceptual content: the different role that attention plays. It has been argued that perceptual attention is a necessary feature of perceptual content (Nanay 2010a). More precisely, attention makes the attended property more determinate (see also Yeshurun and Carasco 1998 for empirical evidence and Findlay and Gilchrist 2003 for a summary). If I am attending to the colour of my office telephone, I attribute very determinate (arguably super-determinate) properties to it. If, as it is more often the case, I am not attending to the colour of my office telephone, I attribute only determinable properties to it (of, say, being light-coloured or maybe just being coloured). In short, attention makes the perceived property more determinate. If this is indeed so, that would constitute a genuine and unique feature of perceptual content. The concept of attention plays a more and more important role in philosophy of perception (see Prinz 2010, and

Perceptual Representation/Perceptual Content    163 Chapter 19 of this volume). One important question is whether and how it characterizes perceptual content.

References Aglioti, S., DeSouza, J. F. X., and Goodale, M. A. (1995). ‘Size-Contrast Illusions Deceive the Eye but not the Hand’, Current Biology, 5, 679–685. Bach, Kent (1978). ‘A Representational Theory of Action', Philosophical Studies, 34: 361–379. Bach, Kent (2007). ‘Searle against the World: How Can Experiences Find their Objects?'. In S. L. Tsohatzidis (ed.), John Searle’s Philosophy of Language: Force, Meaning, and Thought (pp. 64–78). Cambridge: Cambridge University Press. Ballard, D. H. (1996). ‘On the Function of Visual Representation'. In Kathleen Akins (ed.), Perception (pp. 111–131), ed. Kathleen Akins. New York: Oxford University Press. Batty, C. (2010). ‘Scents and Sensibilia’. American Philosophical Quarterly, 47, 103–118. Batty, C. (2011). ‘Smelling Lessons’. Philosophical Studies, 153, 161–174. Bermúdez, José Luis (2000). ‘Personal and Subpersonal: A Difference Without a Distinction’. Philosophical Explorations, 3(1), 63–82. Bertelson, P. (1999). ‘Ventriloquism: A Case of Cross-Modal Perceptual Grouping'. In G. Aschersleben, T. Bachmann, and J. Müsseler (eds), Cognitive Contributions to the Perception of Spatial and Temporal Events (pp. 347–362). Amsterdam: Elsevier. Bertelson, P. and de Gelder, B. (2004). ‘The Psychology of Multimodal Perception’. In C. Spence and J. Driver (eds), Crossmodal Space and Crossmodal Attention (pp. 141–177). Oxford: Oxford University Press. Blaser, E., Pylyshyn, Z. W., and Holcombe, A. O. (2000). ‘Tracking an Object through Feature Space, Nature'. 408, 196–199. Brewer, Bill (2006). ‘Perception and Content’. European Journal of Philosophy, 14, 165–181. Brogaard, B. (2011). ‘Are there Unconscious Perceptual Processes?’. Consciousness and Cognition, 20, 449–463. Brogaard, B. (forthcoming). ‘Conscious Vision for Action versus Unconscious Vision for Action?’. Journal of Philosophy. Brogaard, B. (2014). Does Perception Have Content? New York: Oxford University Press. Brooks, R. A. (1991). ‘Intelligence without Representation’. Artificial Intelligence, 47, 139–159. Bruno, Nicola (2001). ‘When Does Action Resist Visual Illusions?’. Trends in Cognitive Sciences, 5, 385–388. Burge, Tyler (2005). ‘Disjunctivism and Perceptual Psychology’. Philosophical Topics, 33, 1–78. Byrne, Alex and Logue, Heather (2008). ‘Either/Or’. In Adrian Haddock and Fiona MacPherson (eds), Disjunctivism: Perception, Action, Knowledge (pp. 57–94). Oxford: Oxford University Press. Campbell, John (2002). Reference and Consciousness. Oxford: Oxford University Press. Campbell, Keith (1990). Abstract Particulars. Oxford: Blackwell. Casati, R. and Dokic, J. (1994). La Philosopie du Son. Paris: Chambon. Chalmers, D. J. (2004). ‘The Representational Character of Experience’. In B. Leiter (ed.), The Future for Philosophy (pp. 153–181). Oxford: Oxford University Press. Chalmers, D. J. (2006). ‘Perception and the Fall from Eden’. In T. Gendler and J. Hawthorne (eds), Perceptual Experience (pp. 49–125). New York: Oxford University Press.

164   Bence Nanay Chemero A. (2009). Radical Embodied Cognitive Science. Cambridge, MA: MIT Press. Clark, Andy (1997). Being There. Cambridge, MA: MIT Press. Clark, Austen (2000). A Theory of Sentience. Oxford: Clarendon Press. Clark, Austen (2004). ‘Sensing, Objects, and Awareness: Reply to Commentators’. Philosophical Psychology, 17, 563–589. Cohen, Jonathan (2004). ‘Objects, Places, and Perception’. Philosophical Psychology, 17, 471–495. Crane, Tim (2006). ‘Is there a Perceptual Relation?’. In T. Gendler and J. Hawthorne (eds), Perceptual Experience (pp. 126–146). Oxford: Oxford University Press. Crane, Tim (2009). ‘Is Perception a Propositional Attitude?’. Philosophical Quarterly, 59, 452–469. Daprati, E. and Gentilucci, M. (1997). ‘Grasping an Illusion’. Neuropsychologia, 35, 1,577–1,582. Dennett, D. C. (1996). ‘Seeing is Believing: Or is it?’. In K. Akins (ed.), Perception (pp. 158–172). Oxford: Oxford University Press. Findlay, John M. and Gilchrist, Iain D. (2003). Active Vision: The Psychology of Looking and Seeing. Oxford: Oxford University Press. Franz, V. H., Bülthoff, H. H., and Fahle, M. (2003). ‘Grasp Effects of the Ebbinghaus Illusion: Obstacle Avoidance is Not the Explanation’. Experiental Brain Research, 149, 470–477. Funkhouser, E. (2006). ‘The Determinable-Determinate Relation’. Nous, 40, 548–569. Gibson, James J. (1979). An Ecological Approach to Visual Perception. Boston, MA: Houghton Mifflin. Gibson, James J. (1966). The Senses Considered as Perceptual Systems. Boston, MA: Houghton Mifflin. Gillam, Barbara (1998). ‘Illusions at Century’s End'. In Julian Hochberg (ed.), Perception and Cognition at Century’s End (pp. 95–136). San Diego, CA: Academic Press. Goodale, M. A. and Milner, A. D. (2004). Sights Unseen. Oxford: Oxford University Press. Humphreys, Glyn W. and Riddoch, M. Jane (2001). ‘Detection by Action: Neuropsychological Evidence for Action-Defined Templates in Search'. Nature Neuroscience, 4, 84–88. Jacob, Pierre and Jeannerod, Marc (2003). Ways of Seeing: The Scope and Limits of Visual Cognition. Oxford: Oxford University Press. Jeannerod, M. (1997). The Cognitive Neuroscience of Action. Oxford: Blackwell. Jeannerod, M., Decety, J., and Michel, F. (1994). ‘Impairment of Grasping Movements Following a Bilateral Posterior Parietal Lesion’. Neuropsychologia, 32, 369–380. Johnston, W. E. (1921). Logic: Part I. Cambridge: Cambridge University Press. Kaplan, David (1977/1989). ‘Demonstratives'. In Almog, Perry, and Wettstein (eds), Themes from Kaplan (pp. 481–504). Oxford: Oxford University Press. Kelly, Sean (2010). ‘The Normative Nature of Perceptual Experience'. In Bence Nanay (ed.), Perceiving the World (pp. 146–159). New York: Oxford University Press. Kosslyn, Stephen M., Thompson, William L., and Ganis, Giorgio (2006). The Case for Mental Imagery. Oxford: Oxford University Press. Kriegel, Uriah (2007). ‘The phenomenologically manifest'. Phenomenology and the Cognitive Sciences, 6, 115–136. Kulvicki, J. (2008). ‘The Nature of Noise’. Philosophers’ Imprint, 8, 1–16. Loar, B. (2003). ‘Transparent Experience and the Availability of Qualia'. In Q. Smith and A. Jokic (eds), New Philosophical Perspectives (pp. 77–96). New York: Oxford University Press.

Perceptual Representation/Perceptual Content    165 Lycan, W. (2000). ‘The Slighting of Smell'. In N. Bhushan and S. Rosenfeld (eds), Of Minds and Molecules: New Philosophical Perspectives on Chemistry (pp. 273–289). Oxford: Oxford University Press. Martin, M. G. F. (2002). ‘Particular Thoughts and Singular Thought’. Royal Institute of Philosophy Supplement, 51, 173–214. Martin, M. G. F. (2004). ‘The Limits of Self-Awareness’. Philosophical Studies, 120, 37–89. Martin, M. G. F. (2006). 'On Being Alienated'. In T. Gendler and J. Hawthorne (eds), Perceptual Experience (pp. 354–410). Oxford: Oxford University Press. Martin, M. G. F. (forthcoming). Uncovering Appearances. Oxford: Oxford University Press. Matthen, Mohan (1998). ‘Biological Universals and the Nature of Fear’. Journal of Philosophy, 95, 105–132. Matthen, Mohan (2004). ‘Features, Places and Things’. Philosophical Psychology, 17, 497–519. Matthen, Mohan (2005). Seeing, Doing, and Knowing:  A  Philosophical Theory of SensePerception. Oxford: Clarendon Press. Matthen, Mohan (2010). ‘On the Diversity of Auditory Objects’. Review of Philosophy and Psychology, 1, 63–89. Matthen, Mohan (2012). ‘Visual Demonstratives'. In Athanasios Raftopoulos and Peter Machamer (eds), Perception, Realism, and the Problem of Reference (pp. 43–67). Athanasios Raftopoulos and Peter Machamer. Cambridge: Cambridge University Press. McDowell, J. (1994). ‘The Content of Perceptual Experience’. Philosophical Quarterly, 44, 190–205. Milner, A. D. and Goodale, M. A. (1995). The Visual Brain in Action. Oxford: Oxford University Press. Mulligan, Kevin (1999). 'Perception, Particulars and Predicates'. In D. Fisette (ed.), Con­ sciousness and Intentionality: Models and Modalities of Attribution (pp. 163–194). Dordrecht: Kluwer. Mulligan, Kevin, Smith, Barry, and Simons, Peter (1984). ‘Truth Makers’. Philosophy and Phenomenological Research, 44, 278–321. Nanay, Bence (2010a). ‘Attention and Perceptual Content’. Analysis, 70, 263–270. Nanay, Bence (2010b). ‘A Modal Theory of Function’. Journal of Philosophy, 107, 412–431. Nanay, Bence (2011a). ‘Do we See Apples as Edible?’ Pacific Philosophical Quarterly, 92, 305–322. Nanay, Bence (2011b). ‘Do we Sense Modalities with our Sense Modalities?’ Ratio, 24, 299–310. Nanay, Bence (2012a). ‘Action-Oriented Perception’. European Journal of Philosophy, 20, 430–446. Nanay, Bence (2012b). ‘Perceiving Tropes’. Erkenntnis, 77, 1–14. Nanay, Bence (2013). Between Perception and Action. Oxford: Oxford University Press. Nanay, Bence (forthcoming). ‘The Representationalism versus Relationalism Debate: Explanatory Contextualism about Perception’. European Journal of Philosophy. Nanay, Bence (2014). ‘Empirical Problems with Anti-Representationalism’. In B. Brogaard (ed.), Does Perception Have Content? New York: Oxford University Press. Noë, Alva (2004). Action in Perception. Cambridge, MA: MIT Press. Nudds, M. (2001). ‘Experiencing the Production of Sounds’. European Journal of Philosophy, 9, 210–229. Nudds, M. (2010). ‘What Are Auditory Objects?’ Review of Philosophy and Psychology, 1, 105–122. O’Callaghan, C. (2007). Sounds. Oxford: Oxford University Press.

166   Bence Nanay O’Callaghan, C. (2008a). ‘ObjectPerception: Vision and Audition’. Philosophy Compass, 3(4): 803–829. O’Callaghan, C. (2008b). ‘Seeing What you Hear: Crossmodal Illusions and Perception’. Philosophical Issues, 18, 316–338. O’Callaghan, C. (2012). ‘Perception and Multimodality'. In E. Margolis, R. Samuels, and S. Stich (eds), The Oxford Handbook of Philosophy and Cognitive Science (pp. 92–117). Oxford: Oxford University Press. O’Regan, K. (1992). ‘Solving the “Real” Mysteries of Visual Perception: The World as an Outside Memory’. Canadian Journal of Psychology, 46, 461–488. Pasnau, R. (1999). ‘What is sound?’. Philosophical Quarterly, 49, 309–324. Pautz, Adam (2010). ‘An Argument for the Intentional View of Visual Experience'. In Bence Nanay (ed.), Perceiving the World (pp. 254–309). New York: Oxford University Press. Peacocke, C. (1989). ‘Perceptual Content'. In J. Almong, J. Perry, and H. Wettstein (eds), Themes from Kaplan (pp. 297–329). Oxford: Oxford University Press. Peacocke, C. (1992). A Study of Concepts. Cambridge, MA: MIT Press. Port, R. and van Gelder, T. J. (eds) (1995). Mind as Motion: Explorations in the Dynamics of Cognition. Cambridge, MA: MIT Press. Prinz, Jesse (2010). ‘How Do Perceptual States Become Conscious?’ In Bence Nanay (ed.), Perceiving the World. New Essays on Perception (pp. 310–332). New York: Oxford University Press. Pylyshyn, Zenon W. (2007). Things and Places:  How the Mind Connects with the World. Cambridge, MA: MIT Press. Ramsey, W. M. (2007). Representation Reconsidered. Cambridge: Cambridge University Press. Riddoch, M. Jane, Edwards, Martin G., and Humphreys, Glyn W. (1998). ‘Visual Affordances Direct Action: Neuropsychological Evidence from Manual Interference'. Cognitive Neuropsychology, 15, 645–693. Schellenberg, Susanna (2010). ‘Particularity and Phenomenology of Perceptual Experience’. Philosophical Studies, 149, 19–48. Scholl, B. J. and Tremoulet, P. D. (2000). ‘Perceptual Causality and Animacy’, Trends in Cognitive Sciences, 4: 299–309. Shams, L., Kamitani, Y., and Shimojo, S. (2000). ‘What You See Is What You Hear’. Nature, 408, 788. Siegel, Susanna (2002). ‘Review of Austen Clark'. A Theory of Sentience. Philosophical Review, 111, 135–138. Siegel, Susanna (2005). ‘The Phenomenology of Efficacy’. Philosophical Topics, 33, 65–84. Siegel, Susanna (2006a). ‘Which Properties are Represented in Perception?’ in T. Gendler and J. Hawthorne (eds), Perceptual Experience (pp. 481–503). Oxford: Oxford University Press. Siegel, Susanna (2006b). ‘Subject and Object in the Contents of Visual Experience’, Philosophical Review, 115: 355–388. Siegel, Susanna (2009). ‘The Visual Experience of Causation’. Philosophical Quarterly, 59, 519–540. Siegel, Susanna (2010a). ‘Do Visual Experiences Have Contents?. In Bence Nanay (ed.), Perceiving the World (pp. 333–368). Oxford: Oxford University Press. Siegel, Susanna (2010b). The Contents of Visual Experience. New York: Oxford University Press. Soteriou, Matthew (2000). ‘The Particularlity of Visual Perception’. European Journal of Philosophy, 8, 173–189.

Perceptual Representation/Perceptual Content    167 Spence, C. and Driver, J. (eds) (2004). Crossmodal Space and Crossmodal Attention. Oxford: Oxford University Press. Thompson, Brad (2009). ‘Senses for Senses’. Australasian Journal of Philosophy, 87, 99–117. Travis, Charles (2004). ‘The Silence of the Senses’. Mind, 113, 57–94. Tye, Michael (2007). ‘Intentionalism and the Problem of No Common Content'. Philosophical Perspectives, 21, 495–541. Vroomen, J., Bertelson, P., and de Gelder, B. (2001). ‘Auditory-Visual Spatial Interactions: Automatic versus Intentional Components'. In B. de Gelder, E. de Haan, and C. Heywood (eds), Out of Mind (pp. 140–150). Oxford: Oxford University Press. Watkins, S., Shams, L., Tanaka, S., Haynes, J. D., and Rees, G. (2006). ‘Sound Alters Activity in Human V1 in Association with Illusory Visual Perception’. NeuroImage, 31, 1,247–1,256. Yeshurun, Y. and Carrasco, M. (1998). ‘Attention Improves or Impairs Visual Performance by Enhancing Spatial Resolutio’. Nature, 396, 72–75.

Chapter 9

Perception a n d the First  Person Christopher Peacocke

Perception and the first person stand in a relation that is both close and puzzling. Here are six questions on the subject: (1) You characteristically perceive objects and events as standing in various relations to you. Your perceptions have a first person content; their correctness conditions concern you. What does this involve, given that this feature can be present even when you perceive no part of yourself? We can call this ‘The puzzle of perceiving in relation to yourself’. (2) You need to keep track of objects in your environment if you are to learn in part from perception that an object that is now thus-and-so is the same as the one that was such-and-such in the past. You do not, it seems, need in any way to keep track of yourself to have such knowledge. How is this possible? (‘The no-tracking puzzle’.) (3) You do perceive your own body from the inside, as when you experience pressure on the palm of your hand, experience your left arm as extended, or feel the inside of your mouth as hot. But you do not merely experience a body from the inside as having these properties: you also experience these body parts as your own. What is it to perceive something as yours? (‘The puzzle of perceived ownership’.) (4) There seems to be something in Hume’s point that he can never catch himself in introspection. Even when Hume is perceiving his body, either from the inside, or as he might perceive another’s body in vision, the thought ‘this body is mine’ is potentially informative. It is informative in the case in which he needs to consider the possibility that his experiences are caused by the states of the perceptual organs and proprioceptive apparatus of someone else’s body. So experiences as of one’s own body being a certain way may mislead. Moreoever, there does not seem to be such a thing as simply and merely perceiving oneself, as opposed to perceiving some body or bodily part that is experienced as one’s own. How can all this be so? If it is so, why is it so, and how do we reconcile the fact with our answers to the earlier questions? (‘The puzzle of perceiving oneself’.)

Perception and the First Person    169

(5) If our perceptual states are veridical, we both have a spatial location and are embodied. But it seems that we can conceive of first person sequences of conscious states that do not require or involve embodiment at all. Is there a tension here, and if so, how is it to be resolved? Is embodiment as given in perceptual experience, both proprioceptive and by other senses, fundamental to being a subject, or not? (‘The puzzle of embodiment’.) (6) Ordinary experiences with a first person content entitle us to form first person beliefs, containing the first person concept. In a wide range of cases, such ordinary experiences lead to knowledge of the form I am F, that is, knowledge of a first person conceptualized content. Yet it is not at all clear that the entitling experience has a conceptualized content at all. What is the right conception of the nexus of relations between perception, the first person concept, its reference, and first person knowledge? (‘The puzzle of first person content, concepts, and entitlement’). I take these questions, which vary in character from the merely puzzling to the near paradoxical, in turn. Adequate answers to them must, as elsewhere in philosophy, integrate metaphysics with the theory of intentional content. In the present case, the metaphysics in question is the metaphysics of the subject of experience, the self. The intentional contents in question are the first personal, so-called de se contents. What is distinctive of de se content in a mental state or event is that the correctness condition for the state or event concerns the very subject who enjoys that state or event, and it concerns that very subject in a non-derivative way. There are complex contents such as the subject of this experience that necessarily and a priori concern the subject of the experience, but they do so derivatively, by way of the subject satisfying a complex descriptive and demonstrative property, that of being the subject of the experience in question. By contrast, when you have a perception with the content That window is straight ahead of me, the perception does not have a content to the effect that the subject of such-and-such experience is thus-and-so. Its content is just that you are thus-and-so. It is a task of the philosophy of mind to integrate the metaphysics of the self with de se intentional content so understood.

1  Perceiving in relation to yourself It has long been recognized that spatial content and de se content are, in a wide range of experiences, intimately related. Many would agree that what J.  J. Gibson said about information applies to spatial perceptual content: ‘Information about the self accompanies information about the environment, and the two are inseparable . . . One perceives the environment and coperceives oneself’ (1986: 126). This applies both to experience of a stationary array, and to motion. One may experience oneself as moving in relation to the environment, or experience things as moving around one. Experiences with these de se contents may involve perception of one’s own body parts, either in proprioception or through seeing one’s limbs, torso, and nose. But they can also occur without any such perception of one’s own body at all. Spatial experiences represent the world around oneself as being a certain way. It is natural to capture that way by specifying it as a spatial type: the way the world around the

170   Christopher Peacocke perceiver, as given in relation to the perceiver’s body, must be filled in if the experience is to be veridical. This is sometimes called scenario content (Peacocke, 1992), and it can be elaborated along several different lines. Common to such elaborations will be the notion of an origin (the point of view from which the world is perceived, in purely visual perception), axes (derived from body type), and some account of what the ‘filling in’ is. It may be specified what properties and relations hold between the regions in the spatial type. Those who think that perceptual content is dependent upon the identity of particular objects that the perception is about can insert objects into a notion of scenario content should they so wish. The apparatus can serve many comers. Consider the spatial perceptual content (1) That object is in that direction from this place

where this place picks out the origin of the scenario content in the perception, a way of thinking of the place made available by the perception itself (unlike the way of thinking whatever place it is at which I’m located, which is available in thought even if one is not perceiving at all); and consider by contrast the de se perceptual content (2) That object is in that direction from me.

What is the relation between these two? They cannot be identical, for they have distinct correctness conditions. As a result of the insertion of some radio device into your optic nerves, in such a way that the nerves receive input from some other person’s retinas, you may temporarily be perceiving the scene around someone else, in a wholly distinct environment from your own (Dennett, 1978). In such a case, it can be that (1) is true but (2) is false. ‘I’m not really in this place’, you may truly think if you find yourself in such circumstances. Purely spatial correctness conditions are one thing; de se correctness conditions concerning the subject enjoying the experience are a different thing. Different—but it seems that the two are related. It is plausible that for a spatial or material perceptual content F(me) to be true is for F to hold of my body. It is also plausible that my body is the one from which I  perceive in normal circumstances, circumstances in which the scenario-origin this place refers to the place which is in fact the location of my body. Call the conjunction of these two plausible claims ‘the normality thesis’. The normality thesis is neutral as between two radically different competing explanations of why it is true. One proposal is that the normality thesis is explained by the fact that having a point of view on a spatial world from a body located in that world is constitutive of being a conscious subject (Strawson, 1966; Evans, 1982; Campbell, 1994). A  rival proposal does not accept that view of subjecthood, but does hold that the normality thesis is explained by what it is for a conscious subject to have a body it represents as its own (Peacocke, 2008). Some examples suggest that it is too strong to hold that any perceptual states with spatial content must also have a de se content. There could be an organism that perceptually represents events and objects as having properties and standing in spatial relations to the place that is the origin in its scenario contents, an organism that does not have any states with de se contents at all (neither conceptual nor nonconceptual). The organism could have a cognitive map of its environment, keep track of a ‘here’ on that map, and engage in such actions as changing its colour or the electric charge it generates, in response to threats or

Perception and the First Person    171 opportunities. The creature would not be capable of bodily movements; it may live in a fluid. This creature has a perceptual apparatus and control centre that is in fact in the spatial world; but because it does not represent itself at all, a fortiori it does not represent itself as having a spatial location. Shoemaker (1984) has argued that for any conscious subject at all, there must be an account of what it is for it to be embodied. If that is true, we have to add that for an organism of the sort just considered, any shape, size, and material kind of embodiment is consistent with the contents of its representational states, provided the embodiment allows for its perceptions and its limited range of action-types. If the example is a genuine possibility, then not all perceiving, embodied subjects have to self-represent (Peacocke, 2014).

2  Absence of self-tracking Suppose that yesterday you perceived yourself to be in a sunny street. If you remember being there, you can today know that you were in a sunny street. You can know this without reliance on any identity of yourself with something given in a non-first-personal way. Contrast your knowledge of ‘This pen [as given demonstratively in perception now] is the one I bought in the sunny street’. To know this, you need either to have kept track of the pen (it has been in your pocket since the purchase); or to know some identifying marks of that pen, which you know both the purchased pen, and the one you now perceive, to possess. Nothing analogous holds in the first person case. You can know you were in a sunny street without keeping track of yourself, and without relying on any identifying marks of yourself. What explains the difference? First person content in a mental state or event refers, de jure, to the subject who enjoys that state or event. Necessarily and a priori, if one and the same subject both held I am F in the past and now accepts I was F, then what he thereby accepts now will be true if what he held earlier was true. There is no need to rely on a risky identity: no risky identity is involved. By contrast, any transition by a subject to a content of the form ‘This [perceptually given] object is F’ from an earlier acceptance of ‘That [perceptually given] object is F’ needs further information if it is to be a rational transition. The further information required is that it is the same object that is given in the two perceptual presentations. The point of difference in this respect between perceptual demonstratives and the first person shows up in what would be adequate subpersonal mechanisms for registering, and as bases for knowledge, in the two kinds of case. In perceptual tracking of an object, we need to acknowledge not only perceptual object files, labelled by the egocentric locations of objects. We also need something that functions to keep track of when successive files are of the same object. This is the function for which Pylyshyn proposes his ‘FINSTS’, subpersonal pointers which underlie the perception of identity of an object over time (Pylyshyn, 2007). By contrast, we do not need a subpersonal analogue of FINSTS for de se contents. A subject can have a subpersonal self-file, which contains representations of properties he perceives himself to possess. As time passes, what was a present tense predicate in this file merely needs to be temporally updated in the appropriate way to a past tense predication of the same notion. If the earlier representation ‘is F’ in the subject’s self-file is correct, then so will the later representation ‘was F’ in the same file also be correct.

172   Christopher Peacocke These points are completely independent of the metaphysical nature of subjects of consciousness. They apply whether subjects are essentially embodied and perceptible both from the inside and from the outside, or are not essentially embodied, or are essentially such that embodiment makes sense for them. The point turns only on the nature of the reference rule for the de se, whatever the metaphysics of its referent may be. A concern sometimes expressed about this treatment is that in a world in which there is frequent fission and fusion of conscious subjects, then one really would need to keep track of oneself in some additional way if the transition from {I was F, I was G} to I was both F and G is to be soundly made. Now, if in a world of fission and fusion I was F is allowed to be true if one of the subject’s ancestors was F, and I was both F and G requires some ancestor to have been both F and G in the past, then indeed that transition fails. But on that understanding, the transition is not an application of the valid rule of conjunction-introduction within complex predicates. It is rather an instance of the clearly fallacious form {Something standing in R to me was F, Something standing in R to me was G} / Something standing in R to me was F and G. In a world of repeated fission and fusion, memories of experiences may mislead a subject on whether she really has sufficiently strong premisses needed for the application of the conjunction-introduction rule within complex predicates. Given that this explanation of the difference in respect of the need for a certain kind of tracking between the first person and the perceptual cases holds entirely independently of the nature of subjects, the explanation is consistent with the thesis that subjects are things that can be perceived by themselves and others. A mechanism that gives genuine awareness and knowledge of identity over time, of an entity that can in fact be perceived, does not itself have to be a perceptual mechanism. Temporal updating of a subject’s file on itself is precisely such a non-perceptual mechanism. On this view, then, Kant seems to make a mistake when, in his treatment of the Third Paralogism in the First Critique, he moves from the true point that certain kinds of apparent awareness of oneself do not involve an intuition of oneself (through which one is given ‘as object’) to the conclusion that such non-perceptual awareness cannot ‘signify the identity of the person’ (point (3) in B408, Kant (1998)). It does signify the identity of the person, but not by way of any kind of identification or tracking that is involved in purely perceptual-demonstrative cases.

3  Experienced ownership For a body to be yours is for it to be the one from which you perceive, the one some of whose movements are your actions, and the one in which you experience bodily sensations as located. For a body part to be yours is for it to be part of the body so identified. Given these conditions on ownership, the nature of ownership involves a subject who perceives, acts, and senses. These conditions on ownership specify what is required for the correctness of the content of any experience in which a body or a body part is experienced as your own. Most of us experience a particular body and certain body parts as our own, and normally these experiences are veridical. But in less common cases, there are also striking and theoretically revealing illusions of ownership. The phantom limb phenomenon experienced by some of the injured, by amputees, and by some of those born without a particular limb, are well known. Almost equally well known today is the rubber hand illusion, in

Perception and the First Person    173 which tactile stimulation of a subject’s unseen hand, when matched by seen touching of a rubber hand, can produce the illusion that the rubber hand is the subject’s own (Botvinick and Cohen, 1998). A more radical illusion can be produced to the effect that the rear of a head, seen by a subject, is that subject’s own head (Lenggenhager et al., 2007). In the condition of apotemnophilia, a person may experience a healthy, undamaged, functioning limb, such as his arm or lower leg, as not belonging to him, as intrusive, and insist that it be amputated (Ramachandran, 2011). The subject suffering from apotemnophilia nonetheless experiences sensations in the limb in question, and may employ it in such actions as walking. It is widely agreed that the explanation of these illusions must involve a mental representation of the body, called the body schema. The body schema can represent there being a body part when there is not (phantom limb). It may fail to represent as part of one’s body something that really is so (apotemnophilia). The body schema is involved in a persisting representational state that concerns what kind of body the subject possesses. It is to be distinguished from what is sometimes called the body image, which represents the particular spatial properties and internal relations of the subject’s body parts at any one time (Gallagher, 2005). The body schema influences the conditions and character of these illusions (Costantini and Haggard, 2007). These illusions should constrain our constitutive account of the phenomenology of ownership. In a paper on the sense of ownership, Martin makes an important and convincing case that bodily sensations have a representational content concerning the subject’s body (1995). But he also further argues that ‘we should think of apparent ownership not as being a quality additional to the other qualities of experience but as somehow already inherent within them’ (1995: 278). The proposal is that ‘when one feels a sensation, one thereby feels as if something is occurring within one’s body’ (1995: 267). The subject suffering from apotemnophilia is a counterexample to this: that subject feels a sensation in what is in fact his lower arm (say), but experiences that arm as not his own. It follows that, no doubt surprisingly, that we must distinguish sharply between experiences that represent a body part indexically, from the inside, and experiences that represent that body part as one’s own. It also follows from the nature of these illusions that we cannot characterize a sense of ownership reductively in terms of the subject’s body schema, if that schema is characterized only in indexical terms—not even if the schema is required functionally to have a connection with action (see de Vignemont, 2007). The subject with apotemnophilia with respect to his lower leg may still use that leg in the action of walking. (Using something in action, even a basic action, is not the same as experiencing it as being one’s own.) If, by contrast, it is supposed to be part of the representational content of the body schema that it also labels various body parts as one’s own, such a body schema could then contribute to the explanation of these various pathologies. But it would do so then by taking for granted the notion of ownership by a subject, rather than by offering some kind of reductive explanation of the notion. None of these phenomena, nor their explanations, establish that you do not perceive your own body. The contrary is argued by Metzinger: ‘At this point into our investigations into consciousness, it seems obvious that we are never in direct contact with our physical bodies but rather with a particular kind of representational content. . . What you experience is not reality but virtual reality, a possibility. Strictly speaking, and on the level of conscious experience alone, you live your life in a virtual body and not in a real one’ (2009: 113–14). Metzinger’s argument for this conclusion is that ordinary

174   Christopher Peacocke perception uses the same kind of representational systems as virtual reality systems that can be attached to a perceiver’s head. This position is open to two objections. First, to say that you are in ‘direct contact’ only with a certain kind of representational content is to confuse the level of intentional content with the level of reference. A perceptual state can be of an object, including one’s own body, consistently with the perceptual state being a result of computations involving representational contents. (Arguably, it must so result if the state is to be genuinely perceptual rather than merely sensational: see Burge 2010: ch. 9.) If Metzinger’s arguments on this issue were sound, they would equally establish that one does not perceive any physical objects at all. Second, and more generally, the states of consciousness involved in enjoying a virtual reality system have the representational content they do because they stand in certain similarity relations to ordinary perceptual states that do for the most part correctly represent objects, events, and the subject’s own real body in ordinary cases. Genuine perception cannot be elucidated without circularity in terms of its relations to something whose nature is parasitic on genuine perception itself.

4  Perceiving yourself In one of philosophy’s famous passages, Hume wrote, ‘when I enter most intimately into what I call myself, I always stumble on some particular perception or other, of heat or cold, light or shade, love or hatred, pain or pleasure. I never can catch myself at any time without a perception, and never can observe anything but the perception’ (Hume, 2000: Book 1, Part 4, Section 6, p. 165). Hume was surely well aware that he could perceive his own body and his body parts. Evidently he did not take this obvious fact as a counterexample to his intended thesis. Many have been similarly tempted by Hume’s claim even if they have found it hard to say exactly what is right in it. The claim has an affinity to a thought of the young Wittgenstein, when in the Notebooks he wrote ‘I objectively confront every object. But not the I.’ (1979, entry for 11 August 1916, p. 80e). We can learn about perception and its relation to the first person by identifying what is right in Hume’s intended claim. What Hume rightly found is that there is a sense in which he could not attend to himself. Whenever a subject attends to something, the subject attends to it as given in a certain way. When hiking, you may attend to a mountain as given in a certain distance and direction from you. That is to be distinguished from attending to the same mountain, which happens also, unobviously, to be the same one as the one presented in a different direction, some of its parts being obscured by an intervening hill. We can then formulate Hume’s claim thus: there is no such thing as attending to something as given in a purely first personal way. Attending to something given as one’s body, or as one’s hand, is no counterexample to this formulation. In such cases, one’s body and one’s hand are given in perception in perceptual-demonstrative ways that are not purely first personal. There is, for example, a perceptual component in your seeing what is in fact your hand which is common both to your experience of your hand, and the experience of the hand by someone suffering from apotemnophilia with respect to that hand. The latter sees and feels it, but not as his own. Attending to oneself seems to be possible only when there is an experience of something not given purely as oneself.

Perception and the First Person    175 If Hume’s claim, so understood, is true, what is the explanation of its truth? The explanation lies jointly in the nature of attention and the nature of de se content. Attention must always be to something apparently given in perception or sensation (or, perhaps, in memory or imagination). But de se content is arguably individuated, like any other element of intentional content, by its fundamental reference rule, which in the case of the first person is this: as a de jure matter, in any mental state or event in whose content the first person occurs, it refers to the subject who enjoys that state. This rule does not require that the subject of the state with first person content also be presented in any other way, in perception or by any other conscious state or event. There are mental states and events with first person content in which the subject is not presented in any other way. As we noted, you can see something as coming towards you, or as at a certain distance from you, even if you are not perceiving or sensing your body at the time at all. You experience your hand as your own, you experience a sensation as your own, but neither of these requires that you be given in some further way a non-first-personal way in these conscious events. Since anything to which you can attend must be given in a perceptual or sensational way (or in imagination or memory), it follows that there is no such thing as attending to something given purely in a first personal way, and not in any other. That was precisely Hume’s claim. Since the claim seems to follow from the nature of attention and the nature of de se content, the claim is not a merely contingent matter. I do not say that these points were clearly in Hume’s own mind. My position is rather that even those who think that Hume’s conclusion about the nonexistence of a subject distinct from its experiences was a nonsequitur nonetheless often feel that there is something right about his starting point. I offer this account in terms of attention and its relation to de se content as a way of stating what is right in his starting point. So construed, Hume’s insight concerns de se intentional content. No conclusions can be drawn from what is right in Hume’s claim about the reference of the de se component, the subject or owner of conscious experiences. The argument just given for Hume’s claim is consistent with the thesis that persons, Hume included, are essentially embodied. The argument is equally consistent with Shoemaker’s thesis that for any given subject, there must be an account of what it is for the subject to be embodied. The argument does not involve any commitment to Hume’s further view that each of his ‘perceptions’ is an independent entity which requires nothing else for its existence. The argument outlined here for Hume’s claim is equally consistent with the view that all mental events must have subjects, subjects whose existence cannot be reduced to subject-free entities. Nor is there any support in the argument for Hume’s view for the younger Wittgenstein’s conclusion that subjects of consciousness are not in the world. From the fact that veridical conscious states do not and could not present their subject as an entity in the world, it does not follow that their subject is not in the world.

5  Objective perception as constitutive of the subject? Is perceiving an objective world from a spatial standpoint a necessary part of either what it is to be a subject, or (what is not quite the same) what it is to be capable of mental states with

176   Christopher Peacocke first person contents? An affirmative answer to either question results in a Kantian thesis. Both theses are endorsed by Strawson (1966). More than one kind of argument has been offered for such Kantian theses. The argument in Strawson (1966) is built from the following elements. A subject enjoys unity of consciousness; this requires the possibility of self-ascription by the subject of the experiences enjoyed by the subject; experiences involve the recognition of entities as falling under kinds; this recognitional component requires a seems/is distinction, whose basis is present in the experiences themselves; and this in turn requires ‘the distinction between the subjective order and arrangement of a series of such experiences on the one hand and the objective order and arrangement of the items of which they are experiences on the other’ (Strawson 1966: 101). The argument is meant to show that there could not be a subject with a series of conscious experiences consisting only of states with no content about the objective world—with merely such objects as ‘red, round patches, brown oblongs, flashes, whistles’ (Strawson 1966: 99), all of which Strawson takes to be subjective. The links and underlying principles of Strawson’s argument are not entirely clear, and this ambitious argument remains a target of identification and discussion. It is clear that the argument would need to meet the following objections. Prima facie, ownership of an experience is more fundamental than self-ascription of that experience. Ownership explains proper self-ascriptions. That would mean that the early stages of the argument would need revision. Identity of subject over time may also be founded in identity of the subpersonal apparatus that integrates information and yields multiple and complex states of consciousness at a given time in a given subject. Such an identity of integrating apparatus can underlie a sequence of experiences that lack objective representational content. Even when experience is as of objects and events on a path through a spatio-temporal world, the unity of such a consciousness is underlain by identity of integrating apparatus. This suggests that we can make sense of a notion of a subject of consciousness under which a subject may or may not have perceptions of, or even as of, an objective world. Ordinary subjects need not have any conception of an integrating apparatus. That identity of subject consists in identity of integrating apparatus is a constitutive, metaphysical thesis. The ordinary subject simply enjoys or suffers states with de se content, such as the content that he recently experienced an acrid smell. The thesis about integrating apparatus concerns the nature of the entity that is the reference of the de se component in the content of a given mental state. The thesis is not meant to be something immediately obvious from the nature of de se intentional content. Under the metaphysical thesis, a subject can be under an illusion that he was recently thus-and-so. Even if someone else was recently thus-and-so, an apparent memory of that event may be transmitted to a subject whose identity involves a different integrating apparatus. So seeming identity is not sufficient for genuine identity (a requirement on which Kant would also insist). But there is also no obstacle to genuine awareness of identity over time, in good cases. To say that identity of subject is not to be explained in terms of perception of an objective world is not to imply that there are not major insights in the work of those who have endorsed these Kantian theses. The theses are also implied by the treatment in Evans (1982). Evans’s book contains accounts of why various bodily and spatial self-ascriptions, made on the basis of proprioception and outer senses respectively, can be knowledge, knowledge

Perception and the First Person    177 that does not rely on any substantive belief I am identical with m, for some singular concept m other than the first person. The accounts in Evans of self-ascriptive knowledge can be retained, consistently with rejection of the Kantian theses. The accounts can be seen as elaborations of the epistemology and metaphysics of bodily and spatial predications in the first person, when the subject does in fact enjoy perception of an objective world and enjoy embodiment within it. That would be consistent with admitting the possibility (denied by Evans 1982: 250–1) that there can be a subject whose conscious states are realized in the states of a brain permanently in a vat, a subject who cannot locate himself in the objective world, but who still succeeds in referring to a subject—himself—when he thinks ‘I wonder if this apparent world is real’ (or even ‘Cogito ergo sum’). This subject in fact has an integrating apparatus whose operations explain his complex experiences at a time and over time. The condition for an entity to be the reference of the de se component of the intentional content of his experiences is met, viz. the condition that it be the subject of those experiences. It is met by the subject itself, a subject realized in a real integrating apparatus. We can distinguish between this subject having correct and having incorrect apparent memories of his earlier experiences. All this is consistent with a philosophical account of what it is for his experiences to have the representational content needing to be cast in terms of their environmental causes and effects when the subject’s integrating apparatus is properly connected to the world, rather than being sustained in the vat.

6  De se content, first person concept, entitlement, and knowledge An animal without language can see something as moving away from it, can hear something as coming towards it. These perceptions have a de se content, and it is very implausible that their occurrence in any way presupposes the possession of concepts in any strict use of that term. Conceptual content is content of the sort that a thinker can accept or reject for reasons. The exact nature of the connection between concepts and reasons is a matter of some disagreement. Some thinkers would write the connection immediately into the individuation of each particular concept. Under that approach, a concept is individuated by what gives reasons for making certain judgements whose content contains the concept, or is individuated by what certain judgements involving the concept in turn give reasons for judging (Brandom, 1994). Concepts with the same reason-involving powers are identical, as an immediate consequence of this approach. Other approaches to conceptual content would regard the connection between concepts and reasons for making certain judgements as derivative. Many truth-conditional approaches to the individuation of content and concepts would regard the connection between a concept and reasons for judging contents containing the concept as explicable from the concept’s contribution to the truth-conditions of contents in which it occurs (Peacocke, 2008). Whatever the source of the connection between reasons and concepts, it is highly plausible that it exists. But the de se perceptual states, and other de se states of some animals, such as intention, memory, and action awareness, seem to lie below the level of anything essentially involved with, or individuated by, relations to reasons. Yet we nevertheless use the first person concept, as

178   Christopher Peacocke expressed by ‘I’ and ‘me’ in English, both in describing and in expressing these perceptual states with nonconceptual de se content. ‘It looks as if the cyclist is moving away from me’ is the entirely natural expression of the de se content of a visual experience we can enjoy. ‘It looks as if the ball is being thrown far away from him’ is an entirely natural description of the dog’s visual experience. That in itself is a puzzle, but it also leads to at least three other questions. What more generally is the relation between the first person concept and nonconceptual de se content? Can experiences with nonconceptual de se content entitle a thinker to make judgements with conceptual content? Can those experiences in appropriate circumstances lead not just to rational judgement, but to knowledge? I will outline a position on these issues, but readers should be alerted that I am not a neutral party in current discussions on these issues. (A very different approach, one which treats at least all human perceptual content as conceptual, is given in McDowell (1994).) Suppose a thinker makes a first person judgement I am F, and suppose too he also enjoys various perceptual experiences with nonconceptual de se contents. A  creature without concepts could also have experiences with such nonconceptual de se contents. Then the object our thinker is judging to be F is identical with the object his de se perceptual states represent to be thus-and-so when they have the nonconceptual content that he is thus-and-so. This identity is not an a posteriori truth. Rather, the identity follows from the very nature of the first person concept and the nature of de se nonconceptual content. The first person concept is individuated by the condition that in any judgement involving it, it refers to the subject who is doing the judging. The reference of the de se component of the intentional content of his experience is the subject enjoying the experience. That is the rule of reference that individuates the nonconceptual de se notion. But we have already said the case is one in which the judger is enjoying the experience. It follows that in such a case, the reference of the thinker’s use of the first person concept is identical with the object represented as thus-and-so in his de se perceptual experiences of being thus-and-so. So the reference of the first person concept, as used by the thinker, is identical with the reference of the nonconceptual de se in such a case. This identity of reference is a necessary condition for a stronger claim, the claim that nonconceptual de se contents of perceptual experience defeasibly entitle the thinker, in the absence of good reasons for doubt, to judge corresponding first person conceptual contents. A perceptual experience with the nonconceptual content that a tree is in a certain direction (picked out in the scenario content) from oneself can entitle a thinker to judge a corresponding conceptual content that tree is in that direction from me. This is structurally parallel to the way in which a perceptual experience in which something perceived nonconceptually as a four-sided equilateral figure, and as symmetrical about the bisectors of its opposite angles, can entitle a thinker to judge the conceptual content that’s regular diamond-shaped (Peacocke, 1992). In both this shape case, and in the de se examples, if the nonconceptual content is correct, then so too is the content of the conceptualized content to whose judgement it provides an entitlement. This conditional holds in virtue of the nature of the contents in question. It is because the conditional holds in virtue of the nature of the contents involved that the entitlement exists. The identity of the reference of the thinker’s first person concept, as employed by him, with the reference of the de se component of his perceptual states, is also a necessary condition for the conditional to hold. This treatment of the relations between the nonconceptual de se contents of perceptual experience and conceptualized first person judgements is available both to those who hold

Perception and the First Person    179 that embodiment is fundamental to subjecthood, and to those who hold that it is not so fundamental. The latter can hold that a conscious subject may not need a bodily realization, but when the subject does, then these entitlement relations just identified are derivative from what it is for the subject to have a particular embodiment. The treatment above also contributes to an explanation of why we so naturally employ the first person concept in describing the de se perceptual states of creatures that do not possess concepts at all. Since our own de se nonconceptual perceptual states entitle us to make first person judgements about ourselves and our relations to our environment, it is natural to identify the perceptual representational states of a nonconceptual creature by the contents those states would entitle us to judge. We engage in such identification not only in the case of the first person concept, but also with many spatial, temporal, and other concepts that have constitutive entitlement relations with the nonconceptual content of perception. The point applies as much to the linguistic expression of our own perceptual states as it does to the description of those of others. Perceptual experiences with de se contents can lead not merely to correspondingly entitled judgements, but also to knowledge. A necessary condition for knowledge is one of the several requirements that have been called ‘safety’, viz., that the method by which the belief is attained leads to truth in other circumstances that could easily have obtained (Peacocke, 1986; Sosa, 1999). The de se contents of perceptual experience exhibit the phenomenon of constancy. You perceive the door as being in the same direction from you, even if your eyes move in their sockets, or your head moves, and the result is a different pattern of local (in this case retinal) stimulation. Constancy implies that in a range of circumstances that can easily obtain, you will have an experience with the same representational content if the relevant objective features of the world remain the same. This in turn means that if you make a judgement which a corresponding perceptual experience with a nonconceptual content entitles you to make, the resulting belief will be safe with respect at least to the range of circumstances under which constancy holds. First person beliefs appropriately based on de se nonconceptual contents of perception can thereby meet this necessary condition for knowledge. Though the self and the de se have many distinctive features, some so briefly outlined in this chapter, these most recent considerations show that they do share with the other contents of perception the general nature of many of their entitlement relations.

References Botvinick, M. and Cohen, J. (1998). 'Rubber hands "feel" touch that eyes see'. Nature, 391, 756. Brandom, R. (1994). Making It Explicit: Reasoning, Representing, and Discursive Commitment. Cambridge MA: Harvard University Press. Burge, T. (2010). Origins of Objectivity. Oxford: Oxford University Press. Campbell, J. (1994). Past, Space, and Self. Cambridge MA: MIT Press. Costantini, M. and Haggard, P. (2007). 'The Rubber Hand Illusion: sensitivity and reference frame for body ownership'. Consciousness and Cognition 16(2), 229–240. Dennett, D. (1978). 'Where am I?' In Brainstorms. Montgomery VT: Bradford Books. Evans, G. (1982). The Varieties of Reference. Oxford: Oxford University Press. Gallagher, S. (2005). How the Body Shapes the Mind. Oxford: Oxford University Press. Gibson, J. J. (1986). The Ecological Approach to Visual Perception. Hillsdale NJ: Erlbaum.

180   Christopher Peacocke Hume, D. (2000). A Treatise of Human Nature, ed. D. Norton and M. Norton. Oxford: Oxford University Press. Kant, I. (1998). Critique of Pure Reason, trans. A. Wood and P. Guyer. Cambridge: Cambridge University Press. Lenggenhager, B., Tadi, T., Metzinger, T., and Blanke, O. (2007). 'Video Ergo Sum: Manipulating bodily self-consciousness'. Science, 317, 1096–1099. McDowell, J. (1994). Mind and World. Cambridge MA: Harvard University Press. Martin, M. (1995). 'Bodily awareness: A sense of ownership'. In J. Bermúdez, A. Marcel, and N. Eilan (eds), The Body and the Self (pp. 267–289). Cambridge MA: MIT Press. Metzinger, T. (2009). The Ego Tunnel:  The Science of the Mind and the Myth of the Self. New York: Basic Books. Peacocke, C. (1986). Thoughts: An Essay on Content. Oxford: Blackwell. Peacocke, C. (1992). A Study of Concepts. Cambridge MA: MIT Press. Peacocke, C. (2008). Truly Understood. Oxford: Oxford University Press. Peacocke, C. (2014). The Mirror of the World: Subjects, Consciousness, and Self-Consciousness. Oxford: Oxford University Press. Pylyshyn, Z. (2007). Things and Places: How the Mind Connects with the World. Cambridge MA: MIT Press. Ramachandran, V. (2011). The Tell-Tale Brain: A Neuroscientist’s Quest for What Makes Us Human. New York: Norton. Shoemaker, S. (1984). 'Embodiment and behavior'. In Identity, Cause, and Mind. Cambridge: Cambridge University Press. Sosa, E. (1999). 'How to defeat opposition to Moore'. Philosophical Perspectives, 13, 141–153. Strawson, P. (1966). The Bounds of Sense. London: Methuen. de Vignemont, F. (2007). 'Habeus Corpus: The sense of ownership of one’s own body'. Mind & Language, 22(4), 427–449. Wittgenstein, L. (1979). Notebooks 1914–1916, ed. G.H. von Wright and G. E. M. Anscombe. Oxford: Blackwell.

Chapter 10

Nonconceptua l Con ten t

1

Wayne Wright

1 Introduction The main debate about whether perceptual experience has nonconceptual content begins with agreement that experience represents the world and that an experience’s content is the way it represents things as being. Assuming this much, the debate centres on the question:  Is the range of content-bearing perceptual experiences a creature can enjoy independent of its conceptual resources? Conceptualists answer negatively, nonconceptualists affirmatively. Bill Brewer (1999, 203) provides an example of conceptualism, claiming that ‘[perceptual] experiences . . . have representational contents which are characterizable only in terms of concepts which the subject himself must possess.’ John McDowell (1994, 10; original emphases) proposes that ‘when we enjoy experience conceptual capacities are drawn on in receptivity, not exercised on some supposedly prior deliverances of receptivity’; receptivity is our capacity to be affected by and form representations of objects (Kant A19/ B33). McDowell’s remark indicates that conceptualism does not take concepts merely to impose a top-down constraint on the range of permissible experiential contents. They are intimately involved in the production of that content. In contrast, nonconceptualists hold that creatures can have experiences with contents that are not limited by their arsenal of concepts. Michael Tye (2006, 507) characterizes nonconceptualism in terms of the specification of correctness conditions. Tye’s idea is that while concepts would have to be exercised in providing a (theory-relative) canonical statement of a given content’s correctness conditions, having an experience with that content does not require possessing or deploying any of those concepts. Consider the fine-grained nature of the properties encountered in experience, which many nonconceptualists emphasize. Gareth Evans (1982, 229) asked, ‘Do we really understand the proposal that we have as many colour concepts as there are shades of colour that we can sensibly discriminate’? Let us assume momentarily that re-identification by means of stored representations 1  I thank Mohan Matthen for much useful feedback on a draft of this chapter. I am also grateful for discussions of these issues with Michael Tye, Gerald Vision, and the late David Welker.

182   Wayne Wright is a criterion of concept possession. Subjects simultaneously viewing two uniformly coloured surfaces of similar shades of red (say, Munsell chips 2.5R 6/12 and 5R 5/14) can discriminate them from one another in terms of colour despite not possessing a stored icon for the specific shade of either sample, being ignorant of the Munsell conceptual system or any other colour order system, and so forth. If such performance depends on differences in how the surfaces’ fine-grained colours are represented, experience has nonconceptual representational content. Endorsing nonconceptualism does not automatically commit one to claiming that a creature could have perceptual experiences with nonconceptual contents without possessing any concepts whatsoever or that perceptual experience cannot also have conceptual content. Although the dispute between conceptualists and nonconceptualists looks easy to get a handle on, there are several complications. The debate turns on notions about which there is ample controversy: the nature of concepts and their possession conditions, the grounds for attributing states with representational content, and what counts as a perceptual experience. Another problem is that conceptualists and nonconceptualists often are motivated by different concerns. Adina Roskies (2008, 634) notes that conceptualists such as Brewer and McDowell are chiefly wrangling with epistemological considerations. They charge that perceptual experiences could not justify beliefs about the world if their contents were not fully conceptual. Nonconceptualists tend to stress either features of the relationship between concepts and experience besides justification (see Peacocke 1992, 2001; Roskies 2008, 2010) or a range of empirical findings and introspective observations that clash with the conceptualist’s thesis (see Evans 1982; Kelly 2001a; Peacocke 1992 and 1998; Raftopoulos 2009; Tye 2006). The varied nature of the goals and motives in play raises the possibility that researchers are sometimes talking past one another or misjudging the resources available to their opponents. There is also the appearance that central terms of the debate are unclear in a way that obscures the positions available. Richard Heck (2000) points out that there are two different ways of being nonconceptual (each with a corresponding way of being conceptual) and it is not always clear which is on the table (see also Crowther 2006; Speaks 2005). Content nonconceptualism is a matter of a state’s representational content not being constituted by concepts. State nonconceptualism has it that being in a given representational state is independent of whether one possesses concepts that reflect the state’s content. Perhaps there is one kind of content, but two kinds of content-bearing state. One kind of state (exemplified by belief) has a content-relative requirement of concept possession while the other (exemplified by experience) does not. Although reasons to pare down the slate of choices might subsequently emerge, we initially have to recognize all four possible pairings of state and content views as options, rather than just simple conceptualism and nonconceptualism. This chapter examines some of the arguments made on both sides of the nonconceptual content debate and the complications just introduced. The primary goal is to shed light on the motivations for and challenges facing both conceptualism and nonconceptualism. I also aim to show that each side has to grant significant concessions to the other. I suspect that this result diminishes the significance of the debate, but the more ecumenically minded might interpret this as a situation in which the truth falls somewhere between opposing views.

Nonconceptual Content   183

2 State and content views Brewer’s and McDowell’s aforementioned remarks reveal that conceptualism as typically understood encompasses both state and content conceptualism. According to this ‘pure’ conceptualism, experiential content is constituted not just by concepts, but by concepts the experiencing creature possesses and which are deployed. This fits with a common approach to attributions of representational content to a range of states like belief and desire that we can group together as ‘thoughts’: thought contents are constituted by concepts and content attributions must capture how things are from the point of view of the subject. The ‘point of view’ requirement would be violated if a thinker were attributed a thought with a content featuring a concept she either did not possess or possessed but did not employ. For example, it would be wrong to use concepts from modern genetics to characterize an eighteenth-century farmer’s beliefs about annual crop variation, and MOTHER 2 ought not figure in the specification of Oedipus’s thoughts about Iocasta prior to his encounter with the herdsman. If perceptual experience has the same sort of conceptual content as our thoughts about the world, we can literally believe what we see, see what we want, and so forth. In such a circumstance, it seems unproblematic to hold that perceptual experiences are capable of both justifying beliefs and feeding their contents into our decision-making and action-planning processes. Matters with nonconceptualism are more complicated. Some nonconceptualist statements suggest a ‘pure’ (i.e. state and content) nonconceptualism. Gareth Evans (1982, 227) claimed that ‘[the] process of conceptualization or judgement takes the subject from his being in one kind of informational state (with a content of a certain kind, namely, nonconceptual content) to his being in another kind of cognitive state (with a content of a different kind, namely, conceptual content)’. Different accounts of a nonconceptual form of content have been offered; viz. Russellian propositions (Tye 2006), sets of possible worlds (Stalnaker 1998), and specifications of how the space around the perceiver is filled out in terms of surfaces and their properties (this is the scenario content of Peacocke 1992). However, arguments for nonconceptualism often neglect what constitutes the content of perceptual experience. Instead, they target what is required to have a content-bearing experience and are relevant only to state nonconceptualism. Recall the argument from fine-grained detail. Alex Byrne (2005, 235–236) notes that a conclusion about what constitutes experiential content does not directly follow from the claim that having an X-representing experience carries no demand that the perceiver is able to conceive of instances of X qua instances of X. This applies mutatis mutandis to other prominent arguments for nonconceptualism (Speaks 2005, 366). These points might be taken as exposing confusion or ineffectiveness in nonconceptualists’ opposition to conceptualism. However, Josefa Toribio (2008) argues that the distinction between state and content nonconceptualism is untenable for those who hold that content attributions must reflect the way the world seems to the subject. Toribio (2008, 360) contends that content-talk earns its keep through its explanatory usefulness, such explana­ tory usefulness requires that content attributions capture how things seem to be from the 2 

I adopt the practice of entirely capitalizing words used to refer to concepts.

184   Wayne Wright perspective of the creature in question, and the range of ways things can seem to a creature is constrained by that creature’s cognitive abilities. This package of ideas appears essential to the relevance of how the world is represented in experience to explanations of intentional behaviour, the formation of perceptual beliefs, and discriminatory abilities. Toribio concludes that this perspective on content attribution makes the pairing of state nonconceptualism and content conceptualism incoherent, as it has the consequence that a creature could be attributed conceptual capacities (in order to fill out the content specification) that it does not possess. Thus should there be a convincing case for state nonconceptualism, there is a path to content (and ‘pure’) nonconceptualism. While the thesis that content attributions must reflect the subject’s point of view has dissenters, conceptualists such as Brewer and McDowell embrace it. Hence the most interesting challenge to conceptualism would begin with agreement on that thesis and move to developing a case for there being a way of representing the world in experience that is not determined by the subject’s conceptual capacities but that can interact with such capacities in the requisite ways. Many nonconceptualists’ efforts fit this mould. This is certainly true of nonconceptualists who build their case on appeals to the empirical literature that are supposed to establish either that there is a species of perceptual output that is genuinely representational and encapsulated with respect to information from epistemic/semantic centres (Bermúdez 1995; Raftopoulos 2009) or that there is not the right sort of match between the conceptual capacities humans possess and the concepts necessary to characterize experiential content in a way that captures how things seem to the subject (Dokic and Pacherie 2001; Tye 2006).

3 Why be a conceptualist? Since nonconceptualism is often positioned as reacting against conceptualism, it is routinely introduced in terms of what it is not (Vision 1997, 244; Bermúdez and Cahen 2011, section 2). So, it will be helpful to get a sense of conceptualism’s appeal. McDowell (1994) serves as a useful point of focus. His formulation of conceptualism turns on the axiom that perceptual belief is justified by perceptual experience. Without an ‘external rational constraint’ (p. 25) on the beliefs we form about our surroundings, we are left with an unsatisfying view of our epistemic circumstances. What is sought is an account of our perceptual contact with the world that makes it a source of justification while standing outside the realm of thought. Unfortunately, the two positions commonly taken up regarding the epistemic relationship between experience and belief put us in bad straits. McDowell locates the failures of these different views in the same source: in the course of making experience distinct from thought, both place experience outside the conceptual realm. On the one hand, we could acknowledge only causal relations between extra-conceptual experiences and beliefs while taking justification to come from some concept-involving source other than experience. Donald Davidson’s coherentism is representative of this sort of thinking. This abandonment of ‘external friction’ (McDowell 1994, 11) faces worries about how thought can get any bearing on empirical reality, as on this approach we ‘cannot get outside our beliefs’ (p. 16). Thus the world’s impacts on our senses place no rational constraints on our thoughts about the state of the world. This view also offers an implausible

Nonconceptual Content   185 picture of belief formation. Intuitively, beliefs are formed for reasons. However, on this account it is unavoidable that many of a subject’s beliefs are causally foisted on her, as though (for example) she had taken a pill that produces in her the belief that her spouse’s shirt is blue (Heck 2000, 51). These problems are significant enough to make the alternative tempting. This other approach has it that experience—understood as unstructured or unprocessed ‘bare presences’ (McDowell 1994, 19)—can justify belief despite being extra-conceptual. This extension of the ‘space of reasons’ beyond the ‘space of concepts’ is the Myth of the Given that Wilfrid Sellars (1956) is supposed to have demolished. The Myth is alluring because it promises the external grounding of belief that is lacking in Davidson’s coherentism, but it cannot deliver on that promise. We can make sense of an experience being a subject’s reason for believing something about the state of the world, conceptualists say, only if there are conceptual relations between the experience and the belief (Brewer 1999, 149–152; McDowell 1994, 7–9). McDowell regards Evans’ (1982) nonconceptualism as a post-1956 lapse back into the Myth. McDowell (1994, 46) presents his conceptualism as a way out of the problems facing the Myth of the Given and Davidsonian coherentism. The core of his proposal is that perceptual experience is able to justify belief because the same conceptual capacities are operative in both, despite experience and belief being distinct. For McDowell, experience draws on our conceptual capacities to impose the sort of order on the deliverances from our senses required for rational relations between experience and thought. McDowell’s rejection of the two traditional options and the motivation for his own view largely turn on his handling of a famous quote from Kant: ‘Thoughts without content are empty, intuitions without concepts are blind’ (A51/B75). For Kant, intuitions are the outputs of the sensibility, the mind’s faculty for being affected by the world, while thoughts are products of our faculty of understanding. According to McDowell’s Kant, representational content requires the joint contribution of concepts and intuitions. Thoughts unconnected to intuitions are a mere shuffling around of concepts, making them incapable of representing anything (McDowell 1994, 3–4). As for intuitions not brought under concepts, their blindness is a failure to present to the subject something which is intelligible to her as a feature of an objective world (McDowell 1994, 54). They lack representational significance and cannot justify beliefs about the world. McDowell claims that while concepts are actively deployed in reflective thought, they operate passively at the level at which external objects impinge on us. We can choose what we think, but not how things are conceptually represented in our experiences (McDowell 1994, 11). Kant’s account of experience is supposed to offer ‘precisely the picture’ McDowell recommends (1994, 41), but McDowell breaks with Kant on matters of critical importance. McDowell’s interpretation of Kant’s dictum about concepts and intuitions leads him to state that ‘[we] should understand what Kant calls “intuition”—experiential intake—not as a bare getting of an extra-conceptual Given, but as a kind of occurrence that already has conceptual content’ (1994, 9). Kant takes intuitions and concepts as cooperative with respect to achieving empirical knowledge, but he also portrays them as fundamentally distinct, as they have different relations to the objects that affect our senses. Intuitions have ‘immediate relations’ to objects (A19/B33, A320/B377) whereas concepts are ‘never immediately related to an object’ (A68/B92–93). This difference is irreconcilable with the idea that intuitions include a conceptual element.

186   Wayne Wright The idea here is not to advance a Kantian refutation of conceptualism, but to point out that central features of Kant’s actual views do not square with what is supposed to be the most persuasive case for conceptualism. Thus it is unclear what Kantian basis there is for conceptualism. According to McDowell, his conceptualism alone offers a means of grounding justification for belief about the world in perceptual experience. However, Robert Hanna (2005, 262–265) argues that Kant held that extra-conceptual intuition has a structure that endows it with non-inferential ‘evidential force’ and that intuition’s immediate relation to objects enables it to serve as the basis for fixing demonstrative reference. Consequently, the list of options McDowell considers—Davidsonian coherentism and the Myth of the Given—looks profoundly incomplete from the Kantian perspective. Gerald Vision (1998) argues that empirical research also reveals McDowell has underestimated the range of options. The evidence at hand demonstrates that our perceptual faculties do not hand over to our cognitive centres an unstructured Given that is disconnected from the world. It is also compelling to think that at least some of the structured outputs of perceptual processes do not result from contributions of concepts that the perceiver is able to deploy in thought; i.e. at least some experiential content is cognitively encapsulated and not subject to top-down influences from stored semantic/epistemic resources. It will emerge later that this does not decisively refute conceptualism. For now, it suffices to note that the central defect of the Myth of the Given—its commitment to a bizarre picture on which a slurry of experiential intake could somehow provide reasons for beliefs about distinct objects, properties, and events in the world—does not obviously attach to an account of nonconceptual content developed around structured outputs of cognitively encapsulated perceptual processes. A story is required about how these perceptual contents might be connected with beliefs in the required ways, but there are no prima facie grounds for thinking such a story is impossible (Heck 2000, 511–520; Peacocke 1992, 74–80). Additionally, perhaps McDowell’s conceptualism does little more than push a bump under a rug from one spot to another. McDowell’s solution to the epistemic challenge at hand is to put the same conceptual capacities to work in both thought and experience. However, more needs to be said about how conceptual capacities are recruited to do the work in experience that McDowell assigns them. This concern is not about the psychological or neural plausibility of McDowell’s proposal. Rather it targets how we can get a grip on the application conditions of concepts at the level of experience. While talk of ‘reasons for’ might be reserved only for states like belief and judgement, something similar should apply when it comes to the deployment of concepts in experience. Otherwise, it seems as though conceptual experiential content is produced in willy-nilly fashion and McDowell’s account looks no better off than those he rejects. Certainly, if conceptual experiential contents are supposed to justify empirical beliefs, providing reasons for a perceiver to believe one thing or another, the relevant concepts must have some reason-conducing basis for figuring in those experiential contents. Let us call this basis the ‘grounds for’ the application of a concept in the content of perceptual experience. Consider a visual illusion like that induced by viewing the Müller-Lyer figure. Whatever concepts constitute the content of such an experience, conceptualism requires that the two lines are represented as having different lengths by means of those concepts. On what grounds are those concepts, rather than others, passively drawn into the content of that visual experience? Since the actual state of the world is that the lines are of the same length, it looks as though the answer cannot involve an appeal to how the world is (cf. McDowell

Nonconceptual Content   187 1994, 39 and 143). In fact, reflection on cases of veridical experience should suffice to reveal that an appeal to the state of the world will not immediately quell the current concern. Despite misgivings, I will grant McDowell that the state of the world can always be conceptualized. Even if the world does not stand outside the boundaries of what can be conceived, we would still require an account of how the world impresses on our senses in a way that allows it to serve as the grounds for the application of the concepts that figure in the content of experience. Relevant here are McDowell’s (1982/1988) epistemological disjunctivism and themes from the work of J. J. Gibson (1979) that find a place in McDowell’s thinking. McDowell, like Gibson, takes our perception of the world to be direct and emphasizes that perceiving creatures (and not brains) inhabit an environment. Drawing on Daniel Dennett’s (1969) distinction between sub-personal and personal levels, McDowell holds that the computational operations performed in the visual system are no more than syntactic transformations carried out by physical processes. This sub-personal activity begins with the retinal stimulus and never makes contact with the environment itself. Consequently, the outputs of those processes cannot tell us anything that would bear on the justification of empirical beliefs. The world itself is what informs us of how things are (McDowell 1994/2002, 450). McDowell states that an information-processing framework such as David Marr’s (1982) is appropriate for studying a syntactic mechanism like the visual system. McDowell has no objection to talking of nonconceptual content for that purpose, but he regards such content attributions as merely ‘as if’ (1994, 55, and 1994/2002, 452; see Bermúdez 1995 on nonconceptual content and sub-personal computational states). The processes and contents that figure in theories pitched at this level are claimed to be only causally relevant to the ‘real’ (i.e. genuinely semantic or intentional) content that figures in perception and thought (McDowell 1994/2002, 452). They play no role in constituting the contents of experiences of and thoughts about the world, as by their very nature they are blind to the world. Gibson’s ecological approach, with its focus on the perceiving creature coming into unmediated contact with its environment, is apt for the study of vision, understood as a personal-level achievement of finding out how things are in the world. For McDowell, since vision puts us in direct contact with our environment, there is no problem with the world itself serving as the grounds for the concepts that figure in the content of experience. Of course, the way the world appears to be in experience can be mistaken and we might be unable to distinguish deceptive and accurate experience on the basis of how things appear. Undetectable misperception has fuelled many of the theories of perception McDowell sets himself against. McDowell (1994/2002, 450–451) explains misperception causally, in terms of things ‘going well’ with the sub-personal processes that begin at the retina. If those processes do not unfold appropriately, we have mere appearance rather than direct perception of the state of the environment. Since the epistemic differences between perception and misperception get a purely causal explanation, there is no room for a reason-conducing intermediate link between perception and the world (see also McDowell 1982/1988, 218 n.15). Thus the above worry dissolves. Empirical thought is rationally constrained by facts about the world due to experience being conceptual and putting us directly in touch with our surroundings. Appeals to epistemological disjunctivism and direct perception will not defuse the problem at hand. Heck (2000, 497–498) argues that the central thesis of epistemological disjunctivism—that from the standpoint of justification there are two kinds of experiential

188   Wayne Wright state, one veridical and the other nonveridical—cannot be extended to the representational content of experience. The content of an experiential state cannot depend on whether or not it is veridical, as the veridicality or nonveridicality of the experience depends on whether the world is as it is represented in the experience. The same reasoning also blocks disjunctivism about what fixes or grounds experiential content. Even if McDowell is correct that veridical and nonveridical experience are distinct states, there has to be a common content-fixing factor between them. That factor cannot be the state of the world itself, as the world is other than how it is represented when we misperceive it (see also Tye 2006, 523). In that case, it is still unclear why an experience would have one conceptual content rather than another. This is a serious problem for McDowell, as the advertised chief advantage of his conceptualism is that it secures reason-based relations between experience and belief.

4 Motivations for nonconceptualism Jeff Speaks (2005, 362–363) identifies seven kinds of arguments that have been offered for nonconceptualism and places them into two categories: ‘arguments from features of perception’ and ‘arguments from general theses about the conceptual’ (see also Bermúdez and Cahen 2011, section 4.1). In this section I focus on one argument from each of the categories Speaks identifies: the arguments from fine-grained detail and concept acquisition. The argument from fine-grained detail has already been introduced. Perceivers are able to perform discriminations, similarity judgements, and categorizations involving determinate features of their experiences that go well beyond their conceptual capacities. Granting that subject performance in these tasks depends on the representation of the determinate qualities in question, the representational content of experience is not constrained by the subject’s conceptual capacities. A related line of reasoning concerns the perceptual faculties of human infants and (some) nonhuman animals. It is compelling to think that these creatures have experiences much like our own in some respects, which should be reflected in continuities between (some of) the contents attributed to their experiences and to adult human experiences. However, infants and nonhuman animals, if they possess concepts at all (see Bermúdez 2003), do not have conceptual resources nearly as rich as those possessed by adult humans. If one is at all tempted by the fineness of grain argument to think that perceptual contents can outrun the conceptual resources of adult humans, the cases of infants and nonhuman animals offer further reason to suppose that a notion of nonconceptual content is needed. Leading conceptualists deny that animals and infants possess concepts and consequently hold that such creatures do not have experiences with genuine representational content (Brewer 1999, 177–179; McDowell 1994, 114– 123, 182–183). Conceptualists have countered the argument from fine-grain by appealing to demonstrative concepts (Brewer 1999, 170–174; McDowell 1994, 56–60). Our capacity for demonstrative thought and general concepts that we antecedently possess are supposed to allow us to pick out fine-grained features for which we lack concepts as such. McDowell (1994, 56–57) offers the following regarding colour experience:

Nonconceptual Content   189 In the throes of an experience of the kind that putatively transcends one’s conceptual powers—an experience that ex hypothesi affords a suitable sample—one can give linguistic expression to a concept that is exactly as fine-grained as the experience, by uttering a phrase like ‘that shade’, in which the demonstrative exploits the presence of the sample.

McDowell contends that these resources are always available and that they are applicable to all features of experience. Thus constructions such as ‘that figure’, ‘that scent’, and ‘that texture’ should suffice for a conceptual representation of the detail present in experience. Nonconceptualists have responded to the appeal to demonstrative concepts in a variety of ways (see Dokic and Pacherie 2001; Heck 2000; Kelly 2001b; Peacocke 2001; Roskies 2010; Tye 2006). One worry is that McDowell’s way of putting things clashes with the claim that conceptual capacities are passively deployed in experience. What McDowell describes suggests the spontaneous engagement of conceptual resources in reflective judgement about what one is currently experiencing (cf. also Brewer 1999, 172 and 227 n.7). The invocation of linguistic expression and utterance, as well as the suggestion that the relevant conceptual resources are directed at an experience, is hard to reconcile with the idea that conceptual capacities are passively at work in experience. This leads into a common reaction to the appeal to demonstrative concepts. It is natural to think that exposure to certain kinds of experiences is central to a causal explanation of the possession of demonstrative capacities. Heck (2000, 493) stresses that one reason Evans was interested in nonconceptual content is that he saw it as essential to an account of demonstrative reference. Conceptualists look to be assuming that demonstratives can successfully point to their targets, without much consideration of what makes that possible. The idea shared by nonconceptualists who pursue this line is that a perceptual demonstrative concept gets a fix on its referent by means of a subject isolating that item in experience and displaying an appropriate sensitivity to information about that item gathered through experience (Heck 2000, 493; Levine 2010, 191; Roskies 2010, 119–122). The challenge to the appeal to demonstrative concepts is that on pain of circularity one cannot account for the fine-grained details represented in experience by appealing to demonstrative concepts for those details while also claiming that possession of demonstrative concepts for fine-grained details depends on those details being present in experiential content. If experiential content is nonconceptual, the circularity can be avoided. Brewer (2005) replies to the charge of circularity by rejecting the understandings of demonstrative concepts and experience it turns on. He takes the relation between experience and demonstrative concepts to be constitutive, not causal. Brewer claims that ‘experience of a colour sample, R, just is a matter of entertaining a content in which the demonstrative “thatR shade” figures as a constituent’ (Brewer 2005, 222). He goes on to offer an account of content fixation for demonstrative concepts that draws on ideas similar to those from McDowell discussed before. Instead of relying on the relevant fine-grained feature itself being isolated in experience by attention, Brewer contends that a demonstrative expression has the content it does because the subject is appropriately sensitive to the feature itself—out in the world—in a way that ‘in large part depends upon [the perceiver’s] normal neurophysiological perceptual processing’ (ibid). This obviates the need for nonconceptual content. Brewer recognizes that his account has to deal with misperception. Heck (2000, 495–499) and Tye (2006, 523)  develop their misperception-based arguments with the

190   Wayne Wright demonstrative concepts gambit in mind, but the preceding section shows that their points apply equally well to general concerns about what fixes the conceptual content of experience. In responding to Heck’s objection from misperception, Brewer (2005, 223) fills in the details of how demonstrative content is fixed by appealing to Evans’ views. He proposes that demonstrative reference requires the subject’s ability to keep track of an item and suitably modify her attitudes toward it through changes over time due to its own movement, the perceiver’s movement, alterations in ambient conditions, and so forth. There is veridical experience when this tracking succeeds. When it fails, we have misperception, which will be causally explained in terms of limitations or malfunction of the perceptual system. This reply misses the most important point of the misperception objection. McDowell and Brewer can, in some sense, give an account of what is supposed to make perceptual error possible. For both, it’s attributable to purely causal processes. However, that does not address, at least consistently with the conceptualist’s other commitments, why an instance of misperception has the particular content it does. As before, direct perception may help in dealing with certain issues that suggest a need for a notion of nonconceptual content, but it brings its own challenges for the conceptualist. Nonconceptualists’ arguments based on concept acquisition also bear on the conceptualists’ appeal to demonstrative concepts. Christopher Peacocke (2001) argues that conceptualists are unable to account for the learning of new observational concepts, such as the shape concept PYRAMID. Peacocke reasons that in the course of acquiring this concept, the subject must have experiences with contents that, for someone who already has PYRAMID, provide an appropriate basis for applying that concept. Suppose the representational contents of those experiences include PYRAMID. That would imply, in keeping with the discussion of content attribution in section 2, that the subject in question already had PYRAMID. In that case, learning does not occur. Unless one wants to abandon the idea that observational concepts are learned, it looks like the way to go is to ‘acknowledge that there is such a thing as having an experience of something as being pyramid shaped that does not involve already having the concept of being pyramid shaped’ (Peacocke 2001, 252). Hence, nonconceptualism should be endorsed. To deal with concept acquisition, Brewer (2005) again invokes demonstrative concepts. Brewer grants Peacocke the claim just quoted, but offers a different conclusion: ‘What such an experience will have is a conceptual content involving the demonstrative concept, “that (shape)”, referring to the pyramid shape of the object in question’ (p. 224). The demonstrative expression’s content is fixed by means of the tracking relations introduced before. Over a sufficient run of learning opportunities, the demonstrative contents of those experiences become linked with a new observational concept, PYRAMID. Obviously, this response depends on the appeal to demonstrative concepts tackled above. Roskies (2010) has further argued that the demonstrative concepts implicated in this response require that experience has a nonconceptual content. As the earlier talk of ‘keeping track’ indicates, attention is crucial to the intentional selection of the target of demonstration. At the core of Roskies’ argument is that such demonstration involves ‘delimiting . . . the referent of the demonstrative by focusing attention’ (2010, 127). This delimiting can be plausibly explained only by taking experience to already be structured in terms of properties, boundaries, locations, and so forth. Otherwise, we are left with a ‘magic coloring book’ account of demonstrative concept formation: that someone could successfully pick out the intended object of demonstration without access to such properties is on a par with the

Nonconceptual Content   191 suggestion that a child managed to ‘stay within the lines’ by colouring as she wished on a blank page and, once she was done, lines magically appeared on the page in a manner that fit what the child produced (ibid). The structuring of experiential content that Roskies and other nonconceptualists have in mind is further examined next.

5 Perceptual structure and concepts In this concluding section, I introduce additional considerations and offer an appraisal of where the conceptualist/nonconceptualist debate stands. Instead of taking a side in the debate, I aim to show that neither view is fully satisfactory. Nonconceptualists might be heartened by empirical evidence (introduced presently) that supports the claim that there is a way of experiencing the world that is both genuinely representational and not determined by the conceptual capacities we bring to bear in thought. On the other hand, the same empirical evidence also strongly suggests that our most familiar way of experiencing the world is dependent on concept-involving cognitive resources. Moreover, there is room to argue that our perceptual systems, which generate the pre-cognitive experiential states mentioned above, have their own conceptual vocabulary and that the contents of those pre-cognitive experiential states are also conceptual. Although this point seems to favour conceptualism, it is unlikely to be welcomed by conceptualists such as Brewer and McDowell. Recall that they insist that the conceptual capacities which shape conceptual content are the same ones we spontaneously engage in thought. In short, the situation is much more complicated, and the significance of the debate is much more uncertain, than is usually appreciated. The ensuing discussion of empirical findings focuses on vision and follows much of Athanassios Raftopoulos’s (2009) synthesis of a diverse body of research, as he assembles the extant empirical evidence in a credible way. Also, the view he develops serves as a useful stepping-off point for the points I wish to advance. While an oversimplification in several respects, it suffices for now to say that the picture which emerges from this research is that as one moves through the stages of processing that begin with light striking the retina and culminate with full-blown visual experience, information about the retinal stimulus is progressively discarded in the service of generating structures that are ultimately interpretable in terms of three-dimensional objects and their properties. This is achieved by means of specialized neurons that take ‘measurements’ of certain parameters (e.g. edges, direction of motion, faces) based on the input they receive and output only a verdict about their parameter of interest (Cavanagh 2011, 1539; see also Matthen 2005, 44–54). Since these measurements provide only ‘hints at what might be out there’ (Cavanagh ibid), inference (or interpretation) is required to form from them the visual percept of a distal world. The relevant operations begin with pre-attentive activity confined to the canonical visual system (see Lamme 2003). Initially, there is bottom-up processing, known as the feedforward sweep (FFS). The FFS is quick, parallel, and independent of feedback from higher visual areas or sources beyond the visual system. It extracts features using series of filters (cells with specialized receptive fields) into which is fed information about patterns of light intensity at the retina. Nothing at this point has cognitive significance.

192   Wayne Wright After the FFS, top-down and horizontal interactions arise through recurrent processing (RP). RP integrates disparate pockets of information and enables more complex, highly structured information to be extracted. The first stage of RP takes place within the visual system (hence its designation as local; LRP) and yields representations of shapes and spatial relations, individuation of objects or proto-objects (e.g. figure-ground segmentation), and the binding of some different features to the same object (‘the binding problem’). These object segmentation processes are dominated by spatiotemporal information and produce object files for discrete objects, to which information about the object’s properties can subsequently be added. (See Rensink 2000 on proto-objects and Kahneman et al. 1992 on object files.) Object files allow an object to be treated as the same across changes in its perceived qualities (e.g. colour, shape, size), location, and relative position. The representations at the LRP stage are highly volatile, as they are not encoded ‘in any kind of memory other than the visual sensory memory’ (Raftopoulos 2009, 114). They are subject to overwriting as the changing pattern of retinal stimulation suggests new parsings into distinct objects. Beyond LRP, the ‘attentional bottleneck’ becomes relevant. Object-based attention selects some of the objects delivered by LRP for further, cognitively informed processing. Focusing on tasks in which the subject voluntarily searches for a target, this is driven by top-down effects from working memory that allow pre-attentive objects to be registered as relevant or irrelevant. For relevant objects, the associated structures from the FFS and LRP receive enhanced activity and are subjected to further processing involving global RP (GRP). Irrelevant objects are suppressed and overwritten. GRP allows the object to be entered into working memory and connections to be established with epistemic/semantic centres. This both (a) makes the object available for report and use in further cognitive processes, and (b) gives it a coherence and stability that was lacking when only LRP was present. GRP thus facilitates object recognition/classification and representation of 3D object shape, as objects can be compared to representations included in the perceiver’s stored knowledge. Crucially, instead of including the fine-grained detail of pre-attentive content, GRP-involving states have abstract contents; e.g. encoding object colour in terms of categories rather than determinate shades. These states also represent properties beyond those extracted at prior stages; e.g. function. The transition from operations involving only LRP to those in which GRP figures can be thought of along the lines of moving from Marr’s (1982) 2-½D sketch to 3D model or from Zenon Pylyshyn’s (2003, 2007) ‘early vision’ to ‘late vision’ (see Raftopoulos 2009, 51). Following Victor Lamme (2003) and Ned Block (2007), Raftopoulos takes RP to be the neural marker of awareness and uses the two different kinds of RP to distinguish between two different kinds of awareness. Employing Block’s phenomenal/access distinction, Raftopoulos aligns LRP with the former and GRP with the latter. The objects of phenomenal awareness are fleeting and unavailable for report. These representations also are nonconceptual, as they arise independently of resources employed in higher cognitive functions. Raftopoulos (2009, 134) contends, understandably, that this phenomenal awareness is not our familiar mode of experience. Phenomenal awareness is the sort of awareness that subjects in partial-report tasks (Sperling 1960) are supposed to be talking about when they claim to see all the stimuli in very brief displays. Our ordinary experience is due to late vision’s stable representations. Because late vision requires contributions from concept-involving higher cognitive resources, its content is conceptual (Raftopoulos

Nonconceptual Content   193 2009, 148). The content of late vision is conceptual not because it is merely the nonconceptual content of early vision now brought under concepts; i.e. the conceptual content of late vision is not just a conceptualized reiteration of what is nonconceptually represented in early vision. Rather, the conceptual contents of late vision ‘reflect but do not record’ the nonconceptual contents of early vision (ibid, 164); Heck (2000, 514–515) offers similar remarks. Thus not only are early and late vision different kinds of states, but their contents are fundamentally different. The interpretation of the empirical literature just sketched and the view Raftopoulos develops based on it appear to bolster nonconceptualism. However, setting aside a number of other issues worth exploring,3 the friendliness of this story to nonconceptualism can surely be questioned. Two points merit attention at this juncture. First, it is a major concession to the conceptualist to admit that our everyday experience is conceptual. The contents of early vision are inaccessible for report or any other voluntary cognitive task. We cannot form beliefs about the objects of early vision, because any attempt to isolate them brings in attention and concepts, resulting in late vision. As Raftopoulos points out, GRP radically transforms pre-attentive content along a number of dimensions. Beyond the differences between early and late vision when it comes to the stability of their representations, early vision deals with distinct objects qua distinct objects (i.e. in terms of individuated objects having generic surface shapes and without concern for their 3D shape, token identity, or kind membership), while our beliefs based on experience tend to be about tokens of things like mountains, chickens, and hammers considered as such. Whatever evidential relation there is between phenomenal experience and empirical belief has to be indirect and not dependent on how perceivers cognitively grasp the former’s content, as perceivers can do no such thing (see also Lyons 2010). Late vision does seem apt to serve as the immediate evidential basis for empirical beliefs. Since late vision is conceptual, we have arrived at a position much like Brewer’s and McDowell’s conceptualism. The other point is that one might object that demonstrating that (some) experiential content is independent of the conceptual resources deployed in thought is not sufficient for establishing that no conceptual resources whatsoever determine that content (see Laurence and Margolis 2012; Matthen 2005). As a start, consider what Raftopoulos says in fleshing out the evidential relationship between early and late vision, which is relevant to avoiding the charge that early vision content is just another form of the Given. Raftopoulos (2009, 195) claims that ‘the structure of nonconceptual content renders possible the evidential relation between perceptual states with nonconceptual content and perceptual judgments made on the basis of those states’. Our visual system by its very nature parses input from the world into objects and (certain of) their properties, and such structuring (presumably non-coincidentally) turns out to be well suited for exploitation by our higher cognitive faculties. This point is made in varying ways by Heck (2000, 522), Rainer 3  One issue is the debate about whether LRP alone forms the neural basis of phenomenal consciousness. This is brought out in several commentaries on the ‘mesh’ argument of Block (2007) and by Phillips (2011). For now, I’ll note that whether (a) LRP is insufficient for awareness but provides the phenomenal aspect of awareness once GRP is established, or (b) GRP alone is the neural basis of phenomenal consciousness, the contents of early vision are likely still to be relevant to a nonconceptualist thesis. (a) is basically a variation on the ‘LRP alone’ account. In the case of (b), early vision states could be construed as subpersonal, pre-cognitive visual states with contents upon which the contents of experience are systematically dependent.

194   Wayne Wright Mausfeld (2003, 387–388), and Vision (1998, 413). Suppose object recognition is a matter of template matching or matching to an internal description formed by decomposing objects into simpler generic components. In either case, the structured early vision content that feeds into recognition is highly relevant to the application conditions for the concepts that figure in late vision. Since (a) the stored representations of objects consist (at least in part) of the templates or descriptions to be matched to early vision content, and (b) early vision content is structured in terms of objects (or object parts) and their features, this relevance seems semantic or epistemic, not merely causal. How does experiential content gets its structure? A persuasive answer is that the visual system imposes that structure through its own categories. The above discussion of visual processing naturally lends itself to being understood in terms of classification, from the talk of neurons with receptive fields selective for particular features (the classificatory nature of which is emphasized by Matthen 2005), to the segmentation of figure from ground, to the creation of object files to which information about object properties can be added. This is not to say that the categories of the visual system are all hard-wired or permanently fixed, that they can be recruited for use in thought, or that they are arbitrary with respect to the nature of the mind-independent world. The idea only is that the visual system employs a set of proprietary classifications in building representations that are suitably structured to form the basis of useful inferences about the distal scene. Should these classifications be counted as conceptual? Evans (1982, 100–105) argued that what distinguishes thought as conceptual and sensation as nonconceptual is that thought has compositional structure while sensation does not. Matthen (2005, 77–85), however, contends that perceivers can conceptually ‘grasp’ features present in sensation (basically, Pylyshyn’s early vision), where such grasp is understood in terms of stimulus generalization and stimulus discrimination. Matthen’s point is that if we are to make sense of (for example) the disposition of a bird that has been trained to peck at blue squares to also peck at a blue disc, sensational content must be understood as consisting of separable and recombinable elements. In that case, an appeal to compositional structure will not support a conceptual/nonconceptual distinction between the contents of thought and sensation. Matthen’s response to arguments based on compositional structure does not address other nonconceptualist reasons for rejecting that the visual system’s classifications are conceptual. For example, Tye (2006) contends that nonconceptual content is structured. However, he does not count the classificatory activity of the visual system as genuinely conceptual on the grounds that perceivers might be unable either to recognize a previously encountered fine-grained perceptual quality when they experience it again or employ a representation of that fine-grained quality (as such) in thought (Tye 2000, 75; 2006, 506–507). Pylyshyn’s work on object tracking informs the account of the empirical literature presented herein and he is adamant that early vision is nonconceptual because it does not rely on top-down contributions from cognitive centres (Pylyshyn 2003, 214–215). Heck (2000, 487–489) points out that much of what Evans had to say in connection with the argument from compositional structure deals with the perceiver’s cognitive resources. Note also that McDowell (1994, 11) states that the capacities ‘in play in experience . . . would not be recognized as conceptual capacities at all unless they could also be exercised in active thinking’. One reply to Matthen, therefore, is that if concepts are restricted to the cognitive realm and the preceding account of early vision is on the mark, sensation is

Nonconceptual Content   195 nonconceptual despite having compositional structure. Matthen, however, is unlikely to simply grant that sort of narrow understanding of concepts. At this point, it is tempting to cast about for an account of concepts that would decide matters. Such an endeavour could prove unfruitful for any number of reasons (Laurence and Margolis 2012). Picking up on Vision’s (1998, 424) remark that ‘[it] is . . . difficult to motivate [the conceptual/nonconceptual] distinction if . . . we think of what we experience as already featured’, I suggest that any effort along these lines will add little to the discussion. The empirical research discussed in this section shows that the concerns which stimulate so much of the interest in the conceptual/nonconceptual distinction can be addressed (and perhaps are approaching resolution) independently of sorting out where, if at all, to draw the line between them. Where one places the conceptual/ nonconceptual border is likely to affect the terms in which answers are stated to questions about, inter alia, the epistemic and semantic relationships between perception and belief, continuities between human and animal perception, and the theory-ladenness of perception. Nothing about the substance of those answers, though, would change. One researcher might have a perfectly legitimate reason to use a notion of concepts that prioritizes classification and shows little or no concern for a cognitive/noncognitive divide. Another researcher, driven to understand the nature and extent of influence of our cognitive faculties, may very well limit talk of concepts to the cognitive domain. It is hopeless to try to argue for one of these (or any number of others) as the single correct understanding of concepts. It is also unmotivated. Whichever understanding one opts for, the classifications of early vision remain importantly distinct from the concepts employed in thought, as early vision content is not determined by top-down contributions from cognitive centres, perceivers are not able directly to access cognitively early vision contents, and GRP’s involvement in late vision radically transforms early vision content. It also continues to be true that early vision content is structured in a way that underwrites semantic/epistemic connections with full-blown experience and thought. It is likely that many (admittedly, not all) parties to the nonconceptual content debate are much more interested in these results—fleshing them out, piecing them together, contesting them, and so forth—than they are in the question of how to mark a principled distinction between the conceptual and the nonconceptual. Thus the downbeat assessment offered here applies only to the significance of the nonconceptual/conceptual distinction, not to the significance of the issues that lead many researchers to engage in the nonconceptual content debate.

References Bermúdez, J. L. (1995). ‘Nonconceptual Content: From Perceptual Experience to Subpersonal Computational States’. Mind and Language, 10, 333–369. Bermúdez, J. L. (2003). Thinking Without Words. Oxford: Oxford University Press. Bermúdez, J. L. and Cahen, A. (2011). ‘Nonconceptual Mental Content’. Stanford Encyclopedia of Philosophy. Block, N. (2007). 'Consciousness, Accessibility, and the Mesh between Psychology and Neuroscience'. Behavioral and Brain Sciences, 30, 481–548. Brewer, B. (1999). Perception and Reason. Oxford: Oxford University Press.

196   Wayne Wright Brewer, B. (2005). 'Perceptual Experience Has Conceptual Content'. In E. Sosa and M. Steup (eds), Contemporary Debates in Epistemology (pp. 217–230). Oxford: Blackwell. Byrne, A. (2005). 'Perception and Conceptual Content'. In E. Sosa and M. Steup (eds), Contemporary Debates in Epistemology (pp. 230–250). Oxford: Blackwell. Cavanagh, P. (2011). 'Visual Cognition'. Vision Research, 51, 1538–1551. Crowther, T. (2006). 'Two Conceptions of Conceptualism and Nonconceptualism'. Erkenntnis, 65, 245–276. Dennett, D. (1969). Content and Consciousness. London: Routledge. Dokic, J. and Pacherie, E. (2001). 'Shades and Concepts'. Analysis, 61, 193–202. Evans, G. (1982). The Varieties of Reference. Oxford: Oxford University Press. Gibson, J. J. (1979). The Ecological Approach to Visual Perception. New York: Psychology Press. Hanna, R. (2005). 'Kant and Nonconceptual Content'. European Journal of Philosophy, 13, 247–290. Heck, R. (2000). 'Nonconceptual Content and the Space of Reasons'. Philosophical Review, 109, 483–523. Kahneman, D., Treisman, A., and Gibbs, B. (1992). 'The Reviewing of Object Files: ObjectSpecific Integration of Information'. Cognitive Psychology, 24, 175–219. Kant, I. (1998). Critique of Pure Reason, eds. P. Guyer and A. Wood. Cambridge: Cambridge University Press. Kelly, S. (2001a). 'The Non-Conceptual Content of Perceptual Experience: Situation Dependence and Fineness of Grain'. Philosophy and Phenomenological Research, 62, 601–608. Kelly, S. (2001b). 'Demonstrative Concepts and experience'. Philosophical Review, 110, 397–420. Lamme, V. (2003). 'Why Visual Attention and Awareness are Different'. Trends in Cognitive Sciences, 7, 12–18. Laurence, S. and Margolis, E. (2012). 'The Scope of the Conceptual'. In E. Margolis, R. Samuel, and S. Stich (eds), The Oxford Handbook of Philosophy of Cognitive Science (pp. 291–317). Oxford: Oxford University Press. Levine, J. (2010). 'Demonstrative Thought'. Mind and Language, 25, 169–195. Lyons, J. (2010). 'Cognition and Perception: How Do Psychology and Neural Science Inform Philosophy?' Notre Dame Philosophical Reviews. Downloaded 21 February 2011. . Marr, D. (1982). Vision. New York: W. H. Freeman and Company. Matthen, M. (2005). Seeing, Doing, and Knowing. Oxford: Oxford University Press. Mausfeld, R. (2003). ' “Colour” As Part of the Format of Different Perceptual Primitives: The Dual Coding of Colour'. In R. Mausfeld and D. Heyer (eds), Colour Perception: Mind and the Physical World (pp. 381–430). Oxford: Oxford University Press. McDowell, J. (1982/1988). ‘Criteria, Defeasibility, and Knowledge'. In J. Dancy (ed.), Perceptual Knowledge (pp. 209–219). Oxford: Oxford University Press. McDowell, J. (1994). Mind and World. Cambridge, MA: Harvard University Press. McDowell, J. (1994/2002). 'The Content of Perceptual Experience'. In A. Noë and E. Thompson (eds), Vision and Mind: Selected Readings in the Philosophy of Perception (pp. 443–458). Cambridge, MA: MIT Press. Peacocke, C. (1992). A Study of Concepts. Cambridge, MA: MIT Press. Peacocke, C. (1998). 'Nonconceptual Content defended'. Philosophy and Phenomenological Research, 58, 381–388.

Nonconceptual Content   197 Peacocke, C. (2001). 'Does Perception Have a Nonconceptual Content?' Journal of Philosophy, 98, 239–264. Pylyshyn, Z. (2003). Seeing and Visualizing:  It’s Not What You Think. Cambridge, MA: MIT Press. Pylyshyn, Z. (2007). Things and Places: How the Mind Connects with the World. Cambridge, MA: MIT Press. Raftopoulos, A. (2009). Cognition and Perception. Cambridge, MA: MIT Press. Rensink, R. (2000). 'The Dynamic Representation of Scenes'. Visual Cognition, 7, 17–42. Roskies, A. (2008). 'A New Argument for Nonconceptual Content'. Philosophy and Phenomenological Research, 76, 633–659. Roskies, A. (2010). ' “That” Response Doesn’t Work: Against a Demonstrative Defense of Nonconceptualism'. Noûs, 44, 112–134. Sellars, W. (1956). 'Empiricism and the Philosophy of Mind'. In H. Feigl and M. Scriven (eds), Minnesota Studies in the Philosophy of Science, vol. 1. Minneapolis, MN: University of Minnesota Press. Speaks, J. (2005). 'Is There a Problem about Nonconceptual Content?' Philosophical Review, 114, 359–398. Sperling, G. (1960). 'The Information Available in Brief Visual Presentations'. Psychological Monographs: General and Applied, 74,1–29. Stalnaker, R. (1998). 'What might Nonconceptual Content be?' In E. Villanueva (ed.), Philosophical Issues, 9 (pp. 339–352). Atascadero, CA: Ridgeview Press. Toribio, J. (2008). 'State versus Content: The Unfair Trial of Perceptual Nonconceptualism'. Erkenntnis, 69, 351–361. Tye, M. (2000). Consciousness, Color, and Content. Cambridge, MA: MIT Press. Tye, M. (2006). 'Nonconceptual Content: Richness and Fineness of Grain'. In T. Gendler and J. Hawthorne (eds), Perceptual Experience (pp. 504–530). Oxford: Oxford University Press. Vision, G. (1997). Problems of Vision. Oxford: Oxford University Press. Vision, G. (1998). 'Perceptual Content'. Philosophy, 73, 395–427.

Chapter 11

Disj u ncti v ism Heather Logue

1

Take an ordinary perceptual experience in which I see a yellow, crescent-shaped banana, and it is as it looks to me to be—what philosophers call a veridical experience. Plausibly, it is possible (at least in principle) for a scientist to stimulate my brain so as to produce a subjectively indistinguishable experience, i.e. one that I couldn’t tell apart from the veridical experience by introspection alone. (I could tell such an experience apart from the veridical one by testimony if the scientist told me what she was up to. But if I restrict myself to introspection—the mode of access one has to one’s own mental states—I couldn’t notice any difference.) The scientist could generate such an experience in the total absence of yellow, crescent-shaped bananas, and it would nevertheless seem to me that there’s a yellow, crescent-shaped thing before me. Philosophers call this sort of experience a hallucination, which is one kind of non-veridical experience. Although these experiences differ in their causal antecedents, it’s natural to suppose that they are fundamentally the same kind of experience—that they are at bottom the same type of mental state. (One might suggest: after all, the subject can’t tell them apart, and what more could there be to a mental state than what the subject can discern through introspection?) Indeed, for much of the history of philosophy of perception, no alternative view was on anyone’s radar. Up until fairly recently, philosophers assumed that a veridical experience and a subjectively indistinguishable hallucination are fundamentally the same. However, in recent years some have felt compelled to reject this tempting claim. Although the veridical experience and the hallucination described above are both experiences as of a yellow, crescent-shaped thing, they think we have good reason to say that they are radically unalike. A common way of expressing this idea is to say that perceptual experience is supposed to be in some sense disjunctive: either an experience is of one kind (including veridical experiences), or of another radically different kind (including at least some non-veridical experiences). Hence, this broad type of view has come to be known as ‘disjunctivism’. The claim that veridical experiences are radically different from at least some non-veridical experiences can be fleshed out in a variety of ways, and the project of doing so raises two questions: 1 

Thanks to Mohan Matthen and Alex Byrne for helpful comments.

Disjunctivism   199 A. In what way are the two classes of experiences unalike? (By ‘radically unalike’, presumably the disjunctivist doesn’t mean totally unalike—e.g. the veridical experience and the hallucination are alike in that they are both perceptual experiences as of a yellow, crescent-shaped thing. So she must mean that they are dissimilar in certain crucial respects; but which ones?) B. Why on earth would someone endorse such a claim? (Given that non-veridical experiences can seem exactly like veridical ones, it might seem pretty implausible that they are radically unalike.)

As it turns out, the answer to (A) depends upon the answer to (B). In this chapter, I will discuss four answers to (B), and discuss how they affect the answer to (A). I will discuss a version of disjunctivism motivated by a desire to defend Direct Realism (section 2): one motivated by a desire to defend the claim that the content of some perceptual experiences is object-dependent (section 3); one motivated by a desire to respond to a certain kind of argument for scepticism about the external world (section 4); and one motivated by a desire to defend Naïve Realism (section 5). First, though, we must unpack the terminology employed in stating the core disjunctivist claim (section 1).

1 Veridical and non-veridical experiences Before evaluating the claim that veridical experiences and non-veridical experiences are radically unalike, an elaboration of the distinction between veridical and non-veridical experiences is in order.2 Veridical experiences are those in which (i) the subject perceives things in her environment; (ii) her environment is as it appears to her to be; and (iii) for all properties F, if something the subject perceives appears to be F, this is because the subject perceives its being F. An example of a veridical experience is one in which I see a yellow banana, it looks yellow to me, and it looks yellow to me because I perceive its yellowness. (As we’ll see shortly, there are other reasons why a yellow thing might look yellow.) By contrast, non-veridical experiences are experiences in which at least one of these three conditions fails to obtain. Illusions are one of two broad categories of non-veridical experiences. In all cases of illusion, condition (i) is met—the subject perceives things in her environment. But (iii) isn’t—either something she perceives appears to be F but isn’t, or it appears to be F even though she doesn’t perceive its being F. An example of an illusion of the first sort is an experience in which I see a green banana that looks yellow to me as a result of unusual lighting conditions. An example of the second sort is an experience in which I see a yellow banana that looks yellow to me, but the lighting in the room is such that the banana would look green if it weren’t for the precise wavelength of light that happens to be coming 2 I

should note that some would object to the distinction between veridical and non-veridical experiences—namely, those who think that the distinction implies that perceptual experience has content, and that the claim implied is false (see, e.g., Travis 2004 and Brewer 2008). This debate is beyond the scope of this chapter; suffice it to say that I see no obstacle to translating the distinction I’m after into terms that don’t imply that experience has content.

200   Heather Logue through the window. Arguably, since there is a very nearby possible world in which the banana doesn’t look yellow to me, the connection between the yellowness of the banana and the banana’s looking yellow to me isn’t robust enough for me to count as perceiving the yellowness of the banana.3 Hallucinations are the other main category of non-veridical experiences. In all cases of hallucination, condition (iii) fails—there is some property F such that it appears to the subject that there is something F in her environment, but not as a result of the subject perceiving a thing’s being F. This is because what makes an experience hallucinatory is that some aspect of how things appear is generated without the appropriate stimulation of the relevant sensory organs (which is plausibly a necessary condition on perceiving a thing’s being F). For example, a visual hallucination has at least some aspects that aren’t the normal causal upshot of light hitting the subject’s retina. One could (at least in principle) tinker with my brain in a way that results in my hallucinating a pink patch on a wall. I would seem to see pink, but not because light of the relevant wavelength was hitting my retina—and hence not because I perceive the pinkness of something. In a case of total hallucination, condition (i) fails as well. A stock example of such an experience is one had by a ‘brain in a vat’, stimulated so that it has experiences that purport to be of the world around it, when in fact it doesn’t perceive anything in its environment at all. In the usual philosophical examples of hallucination condition (ii) fails as well, but this isn’t definitional of hallucination—it could be that the brain-in-a-vat’s environment happens to be exactly as it appears. In a case of partial hallucination, (i) is satisfied, but (iii) isn’t—e.g. an experience in which I perceive the room around me and the things in it, but hallucinate a pink patch on the wall.4 All forms of disjunctivism hold that hallucinations are radically different from veridical experiences. Some forms hold that illusions are too, while others deny this. Now that we’ve clarified the terminology that features in the basic disjunctivist claim, let us move on to the task of elaborating it.

2 The Direct Realist’s disjunctivism Let’s begin our path to the first version of disjunctivism by considering the following question: what does the subject of a total hallucination perceive? By definition, she doesn’t perceive anything in her environment. A  historically popular answer to this question was that she perceives mind-dependent, immaterial things (given that if she doesn’t perceive anything in her mind-independent, material environment, whatever she perceives must be mind-dependent and immaterial). Such things have been traditionally called 3  Another example of an illusion of this sort is the version of the ‘Ames Room’ illusion outlined in Johnston (2006: 272–273), in which the typical effects of the Ames Room are cancelled out by another illusion. In Susanna Siegel’s terminology (2010: 339–340), such illusions are weakly and strongly veridical but not superstrongly veridical, as the latter implies perception of the relevant objects and properties. By contrast, weak veridicality requires only ‘matching’ how things are in the environment, and strong veridicality requires matching and perception of the relevant objects (but not their properties). 4  The difference between an illusion in which only (iii) fails and a hallucination in which only (iii) fails has to do with the reason why (iii) fails. Very roughly, in the latter case the failure is due to goings-on in the subject’s brain, rather than in the subject’s environment.

Disjunctivism   201 sense-data.5 Moreover, since a total hallucination and a veridical experience can seem exactly alike, and can involve the same kinds of neural states, many have been led to the conclusion that the subject of a veridical experience must be aware of sense-data too. But then a question arises: how do the mind-independent, material things the subject of a veridical experience perceives fit into the picture?6 One answer is to chuck the mind-independent, material things out of the ontological picture altogether. That way lies Idealism—the claim that all that exists are our own ‘ideas’ (including our sense-data).7 A less radical answer is Indirect Realism—the view that when a subject perceives mind-independent, material things, she does so indirectly, in virtue of perceiving sense-data that they cause in her. For example, when I perceive a banana, I do so indirectly, by way of direct perception of something else (a sense-datum caused by the banana). Direct Realism, as the name suggests, rejects the claim that perception of mindindependent, material objects is indirect in this way. The Direct Realist holds that when I perceive a banana, there’s nothing I perceive more directly than the banana—I don’t have to perceive something other than the banana (such as a sense-datum) in order to perceive the banana itself. So there’s no room for sense-data in the Direct Realist’s account of veridical experience. But if we don’t perceive sense-data in having veridical experiences, what should we say about what we perceive when having total hallucinations? This is where our first form of disjunctivism comes in—disjunctivism about the objects of experience (see Thau 2004: 195). The basic idea is that the objects of experience (i.e. the things one perceives) in the course of veridical experience and an indistinguishable total hallucination are different: in the case of veridical experience, the subject perceives mind-independent, material things in her environment, but in the case of total hallucination, the subject doesn’t perceive such things. So veridical experiences and at least total hallucinations are radically unalike in that they have different objects. Note that this is compatible with the experiences having other features in common: e.g. their phenomenal character, or ‘what it is like’ for the subject to have the experience. Plausibly, what it is like for someone to have a veridical experience of a yellow, crescent-shaped banana could be the same as what it’s like for someone to have a total hallucination as of a yellow, crescent-shaped banana. Of course, the Direct Realist owes us an account of what one does perceive in the course of a total hallucination. One option is to claim that while the subject of a veridical experience perceives mind-independent, material entities, the subject of a total hallucination 5  When the term ‘sense datum’ was coined, it was intended to refer to whatever one is directly aware of in having an experience—be it a mind-dependent, immaterial entity or a mind-independent, material one. For example, G. E. Moore entertained the possibility that sense-data in this sense are the facing surfaces of mindindependent, material objects (Moore 1953). However, since most of the parties to the debate at the time the phrase was introduced came to the conclusion that sense-data are mind-dependent, immaterial objects (see, e.g., Russell 1912, Broad 1925, Price 1950, and Ayer 1956), the label eventually came to refer to such entities (see Huemer 2011 for this point, and Snowdon chapter 6, in this volume for a general discussion of sense-data). 6  One could run a similar line of thought concerning partial hallucinations or illusions in which the subject perceives things in her environment, but some of them appear to be a way they’re not—for example, an experience in which a green banana looks yellow to her. The subject doesn’t perceive a yellow banana, so we may ask: what does the subject perceive that accounts for the fact that her environment looks to contain a yellow thing? One historically popular answer: a yellow sense-datum. 7 Note that this also involves chucking out the distinction between veridical and non-veridical experiences—without mind-independent, material things out there for us to perceive, misperceive, or fail to perceive, we can’t make sense of the categories of illusion and hallucination.

202   Heather Logue perceives sense-data.8 However, this option is self-defeating. Plausibly, the subject of a veridical experience and the subject of a total hallucination could be in the same type of brain state. To see this, just hold fixed the neural goings-on in a case of veridical experience, and change the story about how those neural goings-on came to pass (instead of being generated by light reflected off a banana hitting the retina and so forth, let’s say that they’re generated by the tinkerings of a mad neuroscientist). Now, if such a brain state gives rise to perception of sense-data in the case of total hallucination, there’s no good reason to claim that it wouldn’t do so in the case of veridical experience. Hence, it appears that if we account for total hallucination in terms of perception of sense-data, we can’t achieve the Direct Realist aim of keeping sense-data out of our account of veridical experience.9 Fortunately, there are other options available. One is to say that the subject of a total hallucination perceives what Mark Johnston calls ‘sensible profiles’, i.e. complexes of properties and relations that are instantiated by things in the subject’s environment if her experience is veridical (2004: 134–135).10 On Johnston’s view, the subject of a veridical experience perceives a sensible profile and the (mind-independent, material) things that instantiate it. The subject of an illusion or a partial hallucination perceives a sensible profile that’s only partially instantiated by the things one perceives. Finally, the subject of a total hallucination perceives a sensible profile that’s not instantiated by anything one perceives.11 On Johnston’s view, veridical experiences, illusions, and partial hallucinations have objects that total hallucinations don’t (i.e. mind-independent, material particulars). But since all of these experiences have sensible profiles as objects, we don’t have complete dissimilarity across the disjuncts with respect to the objects of experience. Does this disqualify Johnston’s view from being a version of disjunctivism about the objects of experience? Arguably, no. If disjunctivism about x requires complete dissimilarity across the disjuncts with respect to x, then Johnston’s view isn’t a version of disjunctivism about the objects of experience. However, I see no reason to accept the antecedent. We should define disjunctivism about x as comprised of only the bare minimum set of claims required to preserve the view that motivates it. Direct Realism is the view that motivates disjunctivism about the objects of experience, and Johnston’s view illustrates a way in which a Direct Realist could accept that the experiences across the disjuncts have the same kinds of objects (just not all and only the same kinds of objects).12 8  This is the view labelled ‘Austinian disjunctivism’ in Byrne and Logue (2008), inspired by Austin (1962: 32). 9  This is in the vicinity of a line of thought endorsed by, e.g., Robinson (1994), Smith (2002), and Martin (2004). Robinson uses it to argue for the sense-datum theory, while Smith and Martin take the modus tollens route and conclude that one shouldn’t account for hallucination in terms of sense-data. 10  Although Johnston never uses the word ‘perceive’ to refer to a subject’s relation to sensible profiles, he does use the phrase ‘visually aware of’—which is presumably a determinate of the determinable perceiving relation. 11  On the face of it, Johnston’s account doesn’t have the resources to distinguish veridical experiences from illusions or partial hallucinations that happen to ‘match’ the subject’s environment. Moreover, it appears that Johnston holds that we can perceive abstracta: since the properties and relations that make up the perceived sensible profile may be uninstantiated in a hallucinating subject’s environment, sensible profiles cannot be complexes of property instances or tropes. I suspect this counterintuitive consequence of Johnston’s account of hallucination is what has kept most Direct Realists from endorsing it. 12  Although Johnston accepts disjunctivism about the objects of experience, he rejects a different view that goes by the name ‘disjunctivism’—namely, negative disjunctivism about the metaphysical structure of experience (which will be discussed in detail in section 5). His unqualified rejection of what he calls

Disjunctivism   203 Another option for the Direct Realist is to insist that the subject of a total hallucination perceives absolutely nothing. This might seem like an implausible suggestion on the face of it—after all, there’s something it’s like to have a total hallucination as of a yellow, crescent-shaped banana. How could this be if the subject of the experience doesn’t perceive anything at all? The reply is to deny that the only way of accounting for what it’s like to have a perceptual experience is in terms of what the subject perceives in the course of having it. One popular alternative is to account for experience in terms of the subject representing her environment as being a certain way (e.g. as containing a yellow, crescent-shaped thing). This is the core claim of Intentionalist theories of perceptual experience.13 The basic idea is that perceptual experience is structurally similar to belief, in that it is a certain kind of attitude to a proposition. Just as I can doxastically represent that there is a yellow, crescent-shaped thing before me, I can perceptually represent that there is a yellow, crescent-shaped thing before me. This allows the Intentionalist to account for total hallucination while denying that the subject perceives anything. Roughly speaking, according to Intentionalism, a total hallucination as of a yellow, crescent-shaped banana is a matter of the subject perceptually representing the proposition that there is a yellow, crescent-shaped thing before her—something she can do even if she doesn’t perceive such a thing. This view yields another version of disjunctivism about the objects of experience, as veridical experiences, illusions, and partial hallucinations have mindindependent, material things as objects, whereas total hallucinations have no objects whatsoever.14 Disjunctivism about the objects of experience is relatively uncontroversial—at least, it’s only as controversial as denials of Indirect Realism and Idealism. So these days, this form of disjunctivism has the status of orthodoxy.15 Let us now turn our attention to versions of disjunctivism that are currently more contentious.

3 An Intentionalist’s disjunctivism Another form of disjunctivism emerges from a division among Intentionalists. As I said in the last section, all Intentionalists hold that having a perceptual experience consists in the subject representing her environment as being a certain way (in a manner

‘disjunctivism’ (Johnston 2004: 122–127) reflects the fact that the label is more often used to refer to the latter view than the former. 13  There are other versions of this alternative besides the Intentionalist one—e.g. the accounts of total hallucination given by negative disjunctivists (see section 5). I’ve just picked one for the purposes of illustration. Intentionalist theories are defended by Peacocke (1983), Harman (1990), Dretske (1995), Block (1996), Tye (2000), and Byrne (2001), among others. The various versions of Intentionalism defended by these authors diverge from each other in many significant ways; but they all subscribe to the core claim identified in the main text. 14  I should note that there is another Direct Realist theory of perceptual experience—Naïve Realism, to be discussed in section 5. This view can be elaborated in terms of Johnston’s account, or in terms of the denial that a subject of a total hallucination perceives anything. For the sake of space, I won’t go into the details of a Naïve Realist disjunctivism about the objects of experience. 15  Recent expressions of dissent can be found in Jackson (1977), Robinson (1994), and Foster (2000).

204   Heather Logue distinctive of experiencing, as opposed to, say, believing).16 Another way of putting this core Intentionalist idea is to say that perceptual experience involves representing a proposition to the effect that one’s environment is thus-and-so. The proposition represented is called the content of the experience.17 Some Intentionalists hold that the content of a veridical experience is different from the contents at least some subjectively indistinguishable non-veridical experiences. Such a view is disjunctivism about the content of experience. Why might an Intentionalist endorse disjunctivism about the content of experience? One potential reason concerns the connection between perception and demonstrative thought.18 Perceiving the banana on my desk puts me in a position to have demonstrative thoughts about it, e.g. to think that this banana is ripe enough to eat. But since a total hallucination as of a yellow banana wouldn’t involve my perceiving any particular thing in my environment, it couldn’t put me in a position to have demonstrative thoughts about any such thing. Some Intentionalists think that in order to account for this difference between experiences which involve perceiving things in one’s environment and those that don’t, we should say that the contents of the former are object-dependent, i.e. that they are propositions whose truth depends on how things are with a particular object (e.g. the proposition that that thing is yellow). By contrast, object-independent propositions can be made true by different particular objects. The proposition that there is a yellow banana before me is true as long as there is some yellow banana or other before me—it doesn’t matter which one. These propositions are suited to be the contents of total hallucinations. A total hallucination as of a yellow banana involves the subject representing that there is a yellow banana before her, but not any particular one. The result is disjunctivism about the content of experience: veridical experiences, illusions, and partial hallucinations (all of which involve perceiving something in one’s environment) have object-dependent contents, whereas total hallucinations have object-independent contents.19 Note that this version of disjunctivism, like disjunctivism about the objects of experience, is compatible with there being plenty of similarities across the disjuncts. Indistinguishable experiences from different disjuncts are radically different in that they have different contents, but are supposed to be similar in respect of, for example, phenomenal character and broad metaphysical structure (they’re all propositional attitudes). Indeed, on some versions of disjunctivism about content, experiences involving perception have object-independent content in addition to object-dependent content, which means that all experiences have object-independent content.20 Of

16  For brevity’s sake, I’ll often leave out this qualification in what follows, and just talk of perceptual experience involving the subject representing her environment as being a certain way. 17  One might hold that experiences have content in a weaker sense; roughly, that there are propositions associated with them, but they aren’t perceptually represented by the subject (see Siegel’s Content View [2010], Schellenberg’s association thesis [2011], and my Mild Content View [Logue, 2014]). One who holds such a view could be a disjunctivist about the content of experience. However, since few people occupy this region of logical space, I’ll set this possibility aside. 18  See Brewer (1999). 19 For the view that perceptual experience involves representation of only object-independent propositions, see, e.g., McGinn (1982) and Davies (1992). For the view that perceptual experience involves representation of object-dependent propositions, see, e.g., Chalmers (2006) and Tye (2007). 20  See Byrne and Logue (2008) for such a view.

Disjunctivism   205 course, this wouldn’t count as a version of disjunctivism about content if that meant there must be no common content across the disjuncts whatsoever. But as I  said in the last section, disjunctivism about x should be only as strong a claim as is needed to preserve the motivation for it. And what’s important to the motivation for disjunctivism about content I  outlined is that only experiences involving perception have object-dependent content, not that there’s no other kind of content all experiences have in common. Is disjunctivism about content true? Obviously, it’s true only if experiences have content to begin with, and this debate is beyond the scope of this chapter.21 But suppose for the moment that it does. In that case, I see no good reason to reject disjunctivism about content, because I see no good reason to reject the claim that the contents of experiences involving perception are object-dependent. One might resist this claim on the grounds that it is possible for a total hallucination to be subjectively indistinguishable from a veridical experience—i.e. to be such that the subject cannot tell it apart from a veridical experience of a certain kind by introspection alone. For example, a brain-in-a-vat’s hallucination as of a yellow banana may be subjectively indistinguishable from a veridical experience of one. Doesn’t this mean that the experiences have the same content? Well, only if we expect that introspection of our experiences to reveal the entirety of their metaphysical structures to us (e.g. whether or not the content of an experience is objectdependent). And I see no reason to think that introspection is this powerful. Moreover, there are a number of arguments against the claim that the content of experience is exclusively object-independent, which I don’t have the space to summarize here.22 But I think it’s pretty safe to say that if perceptual experience has content, then disjunctivism about content is true.

4 An anti-sceptic’s disjunctivism Let us work our way into the third form of disjunctivism by reflecting on a well-known argument for scepticism about the external world. The argument begins with the premise that I don’t know that I’m not a brain-in-a-vat (BIV). Why don’t I know this? Well, if I were a BIV, things could perceptually appear to be exactly as they do now (i.e. it could perceptually appear to me that there’s a yellow thing before me). So it seems that the perceptual evidence I have underdetermines whether I’m a normal embodied subject having a veridical experience of a yellow banana, or I’m a BIV being stimulated to have a subjectively indistinguishable hallucination—the evidence afforded by my experience provides just as much support for the first hypothesis as it does for the second. The second premise is that if I don’t know that I’m not a BIV, then I don’t know anything about the external world. Plausibly, in order to know things about the external world, I’ve got to know that I’m not ‘perceptually disconnected’ from it; that is, I’ve got to know that the experiences generating my beliefs about the world involve perceiving things in it, rather than resulting from someone (or something) tinkering with my brain to generate experiences that 21 

22 

But see, e.g., Travis (2004), Crane (2009), Siegel (2010), Schellenberg (2011), and Logue (2014 But see Tye (2007).

206   Heather Logue for all I know could be completely unreliable guides to what’s going on in the world.23 So from this premise, and the claim that I don’t know that I’m not a BIV (the first premise), it follows that I don’t know anything about the external world. One form of disjunctivism is supposed to undermine the first premise of this argument. Recall that the first premise was supported by appeal to the idea that I’d have the same perceptual evidence regardless of whether I’m having a veridical experience of a yellow banana or a total hallucination as of one. Disjunctivism about perceptual evidence (also known as epistemological disjunctivism) rejects this claim.24 The first key component of the view is that a subject of a veridical experience has more perceptual evidence concerning the goings-on in her environment than the subject of an indistinguishable illusion or hallucination. The latter kinds of experiences are epistemically defective—even if one hallucinates a yellow banana when there is a yellow banana before one, or one is the subject of an illusion where two illusion-generating features of a situation cancel each other out so that a yellow banana looks yellow, intuitively having these experiences wouldn’t put one in a position to know that there’s a yellow banana before one. Disjunctivism about perceptual evidence accounts for this defectiveness in terms of non-veridical experiences providing less evidence than veridical experiences do for claims about one’s environment. The second key component of this version of disjunctivism is that this special kind of perceptual evidence one gets when having a veridical experience is good enough to put one in a position to know things about one’s environment—which requires that it favours the proposition that there’s a yellow banana before me over the proposition that I’m a BIV being stimulated so as to have a hallucination of one in the total absence of bananas. Roughly, the idea is that a veridical experience provides me with evidence that enables me to know things about my environment (e.g. that there’s a yellow banana before me). This claim, in conjunction with the sceptic’s claim that if I don’t know that I’m not a BIV, then I don’t know anything about the external world, entails that I do know that I’m not a BIV after all.25 23  One could also support this premise by appeal to the principle that knowledge is closed under known entailment (i.e. if S knows that p, and S knows that p entails q, then S knows that q). If I know something about the external world, e.g. that I have hands, then I know that I’m not a BIV—since I know that my having hands entails that I’m not a (handless) BIV. Contraposing the conditional gets us the second premise. However, this closure principle won’t support the second premise of all analogous sceptical arguments. For example, if I don’t know that I’m not dreaming, then I don’t know anything about the external world either. But since my having hands doesn’t entail that I’m not dreaming, I don’t know that my having hands entails that I’m not dreaming. Hence, we can’t apply the closure principle to get us to this claim. The case for the second premise in the main text has wider applicability (cf. Stroud 1984: 29). 24 

The label ‘epistemological disjunctivism’ was coined in Snowdon (2005) and propagated in Byrne and Logue (2008). The view labelled was originated by John McDowell (see, e.g., his 1982 and 2008), and defended in Pritchard (2008) and Neta (2008). See Williamson (2000: ch. 8) for a broadly similar view. 25  Note the similarity of this line of thought to G. E. Moore’s response to the sceptic (Moore 1962)—in both cases, I can deny the sceptic’s conclusion (that I don’t know anything about the external world), and use the sceptic’s second premise in order to conclude that I know that I’m not a BIV. (This is why Pritchard 2008 calls epistemological disjunctivism ‘McDowellian Neo-Mooreanism’.) The difference between this view and Moore’s reply to the sceptic consists in the grounds given for rejecting the sceptic’s conclusion. Moore relies on his infamous proof, whereas the epistemological disjunctivist relies on the claim that the subject of a veridical experience has more and better perceptual evidence than a hallucinating or illuded counterpart. One might view epistemological disjunctivism as a way of defending the premises of Moore’s

Disjunctivism   207 The preceding is just a broad outline of epistemological disjunctivism. Two crucial issues have yet to be discussed: in particular, what exactly is the perceptual evidence the subject of a veridical experience has that the subject of an indistinguishable non-veridical experience lacks, and does it put the subject in a position to know things about her environment? For simplicity’s sake, let’s just stick to the answer given by Pritchard in his elaboration of McDowell’s view (analogous issues arise for other answers one could give). According to Pritchard, the relevant evidence is of the form: that one sees that such-and-such is the case (2008: 291).26 For example, when I have a veridical experience of a yellow banana on my desk, one item of perceptual evidence I have is that I see that there is a yellow banana before me. At this point, there are two broad routes an epistemological disjunctivist can take. The first is to provide an account of how one can know that one sees that p despite the sceptic’s argument to the contrary.27 The other is to claim that one can have the proposition that one sees that p as evidence even if one doesn’t know it, or isn’t even in a position to know it. However, this strategy raises the question of what makes this proposition part of one’s evidence, if not one’s knowing it. Surely one has to bear some cognitive relation to it for it to count as evidence one has, and merely believing it isn’t sufficient (believing that the earth is flat doesn’t make that proposition part of one’s evidence). Presumably, one has to at least be justified in believing it. In that case, the epistemological disjunctivist owes an account of how one can be justified in believing that one sees that p. So, is epistemological disjunctivism true? It is arguably a less revisionary anti-sceptical strategy than others on offer (e.g. reliabilism about knowledge, denying that knowledge is closed under known entailment, contextualism about knowledge attributions, and so forth). However, for the reasons just given, it’s true only if we can provide an acceptable account of either how one can know that one sees that p, or how one can be justified in believing that one sees that p—so further investigation is required.

5 The Naïve Realist’s disjunctivism Let us now turn to our fourth version of disjunctivism concerning perceptual experience. This version of disjunctivism aims to preserve a theory known as Naïve Realism. Naïve Realism holds that veridical experience fundamentally consists in the subject bearing the perceptual relation to things in her environment (and some of their properties). For example, according to the Naïve Realist, the veridical experience I’m currently having of the yellow, crescent-shaped banana on my desk fundamentally consists in my bearing the perceptual relation to the banana, as well as its yellowness and its crescent shape.28 (As for what the Naïve Realist says about non-veridical experience, we’ll turn to that question shortly.) proof (i.e. ‘here is a hand; here is another’). In any case, epistemological disjunctivism is intended to avoid the faults commonly attributed to Moore’s anti-sceptical strategy (e.g. the charge of question-begging). 26 

For other potential answers, see Logue (2011). proposals, see Johnston (2006:  287–288), McDowell (2008:  387), Millar (2008:  342), Kennedy (2011: 78–80), and Byrne (2012). 28  It may not be immediately obvious to readers familiar with Naïve Realism that this characterization captures the various formulations offered by its proponents. For discussion that aims to allay this suspicion, see Logue (2013). 27  For

208   Heather Logue The ‘fundamentally’ in the statement of Naïve Realism is important, as without it the claim on the table would simply be that veridical experience involves perceiving things in my (mind-independent, material) environment—which everyone but the idealist accepts. Naïve Realism is a much stronger claim. It is a claim about the metaphysical structure of veridical experience, i.e. about the ultimate psychological facts in virtue of which facts about experiences obtain. To say that an experience fundamentally consists in x is to say that x is the most basic psychological characterization of the experience, in the sense that all of its other psychological features (e.g. its phenomenal character, its epistemological role) are grounded in x. So according to the Naïve Realist, the ultimate psychological fact in virtue of which an experience has the phenomenal character it does, and gives rise to the beliefs and the behaviours it does, is that it consists in the subject bearing the perceptual relation to things in her environment.29 For example, my bearing the perceptual relation to the banana on my desk, and its yellowness and its crescent shape, is the most basic (personal-level) psychological explanation of why I believe that there’s a yellow, crescent-shaped banana on the table. (Of course, there are further neurological and information-processing facts by virtue of which I bear the perceptual relation to the banana and certain of its properties. But these are not the explanantia we’re after in a theory of the metaphysical structure of experience.) By contrast, strong versions of Intentionalism hold that all perceptual experiences, and hence veridical experiences, fundamentally consist in the subject representing her environment as being a certain way (e.g. as containing a yellow, crescent-shaped banana).30 Notice that both Intentionalism and Naïve Realism are compatible with Direct Realism as I described it earlier. For the idea that the subject of a veridical experience directly perceives things in her environment can be elaborated in two different ways: in terms of the experience fundamentally consisting in the subject (directly) perceiving such things, or in terms of the experience fundamentally consisting in the subject representing such things.31 Given that one wants to defend Direct Realism, why would one opt for the Naïve Realist version? This question has been given a variety of answers by proponents of Naïve Realism, and I don’t have space to do any of them justice here. One answer, from which the view presumably derives its name, is that it is the ‘naïve’ or ‘common’ conception of veridical 29  Note that this claim isn’t incompatible with a veridical experience and a hallucination having the same phenomenal character. For even if a veridical experience has its phenomenal character ultimately in virtue of the subject perceiving things in her environment, it could still be the case that a hallucination has the same phenomenal character ultimately by virtue of a different psychological fact. (More on this idea below.) 30  Some Intentionalists balk at explaining the phenomenal character of experience in terms of how the subject perceptually represents her environment as being, usually on the grounds that the representational content and phenomenal character can vary independently of each other. These Intentionalists typically explain the epistemological role of experience in terms of its representational content, while giving a separate account of what grounds its phenomenal character in terms of intrinsically non-representational ‘qualia’ (see, e.g., Block 1990 and 1996). 31 It’s important to keep in mind that the Intentionalist doesn’t hold that a subject perceives the representation in which her experience fundamentally consists—that claim wouldn’t be compatible with Direct Realism. The Intentionalist’s representation (unlike the Sense-Datum Theorist’s) is a state of the subject that enables perception of things in one’s environment without being perceived itself, much as the eye sees without being seen by its subject.

Disjunctivism   209 experience.32 Naïve Realists typically put little weight on this consideration, and they’re right not to do so—even if it is the common-sense view, it’s far from obvious why the deliverances of common sense should dictate our views about the metaphysics of veridical experience. And as Hume famously noted, the ‘slightest philosophy’ (1777/1993: 104) shakes our commitment to this particular naïve belief.33 The bit of philosophy Hume alludes to is, of course, reflection on non-veridical experiences, especially total hallucinations. For a moment’s thought reveals that Naïve Realism cannot be true of total hallucinations: these experiences by definition don’t involve perception of things in one’s environment, so they can’t fundamentally consist in perceiving things in one’s environment. Hence, Naïve Realism is committed to disjunctivism about the metaphysical structure of perceptual experience (metaphysical disjunctivism for short): veridical experience and at least total hallucination have radically different metaphysical structures. Veridical experience fundamentally consists in the subject perceiving things in her environment and certain of their properties, whereas total hallucination fundamentally consists in something else entirely. What about partial hallucinations and illusions? These involve the subject perceiving things in her environment, but misperceiving some of their properties.34 Even if they fundamentally consist in perceiving things in one’s environment, it’s not obvious that this fact could afford an exhaustive account of such experiences. This sort of account seems particularly inappropriate for partial hallucinations. For example, take a partial hallucination in which the subject perceives a green banana, which nevertheless looks yellow because a neuroscientist is fiddling with her brain. The facts in virtue of which the banana looks yellow have nothing to do with the way the banana is, so any attempt to account for the subject’s experience solely in terms of perception of the banana seems completely off-base. However, this isn’t obviously the case when it comes to illusions, which are generated by complicated interactions between properties of the things perceived, the circumstances in which they’re perceived, and the way the subject’s perceptual system normally functions in those circumstances (very roughly speaking). For example, take a garden-variety illusion in which a green banana looks yellow due to strange lighting conditions. We need to account for the fact that the banana looks yellow to the subject. Of course, we can’t account for this fact in terms of the subject perceiving the banana’s colour—it’s green, not yellow. But perhaps there’s another property the banana has (perhaps one closely related to colour properties) such that perceiving it would explain why it looks yellow to the subject. I don’t have the space to 32 

For discussion, see Fish (2009: ch. 1, section 3). For more compelling motivations for Naïve Realism, see Martin (2002), Campbell (2002: ch. 6), and (especially) Fish (2009: 75–79). It’s easy to get the false impression that Martin endorses the motivation described in the main text, as he claims that Naïve Realism is ‘ . . . the best articulation of how our experiences strike us as being to introspective reflection on them’ (2004: 42). If the introspective reflection is that of a philosophically naïve subject, then we have the motivation just outlined. In the footnote immediately following this claim, he refers the reader to his paper ‘The Transparency of Experience’ as the place where he develops it in detail. However, in this paper, he argues that introspection of perceptual experience alone doesn’t support Naïve Realism over Intentionalism (2002: 402). Rather, the motivation offered is a sophisticated philosophical argument utilizing premises about introspection of sensory imagination (not perceptual experience). So if we follow Martin in regarding his 2002 paper as the definitive statement of his motivation for Naïve Realism, he doesn’t endorse the one described in the main text after all. 34  Or, in the case of ‘veridical’ illusions and partial hallucinations, failing to perceive some property one of the things appears to have (see section 1). 33 

210   Heather Logue explore this possibility here; but suffice it to say that some fancy footwork is required in order make Naïve Realism work for illusions.35 In any case, the point to note for our purposes is that metaphysical disjunctivists divide into those who put illusions in the same disjunct as veridical experience (VI v H disjunctivists), and those who put them in the same disjunct as hallucinatory experiences (V v IH disjunctivists).36 (In what follows, we’ll set illusions and partial hallucinations aside for ease of exposition.) Of course, the Naïve Realist’s work is still far from over. It’s not enough to say that the metaphysical structure of hallucination is radically different from that of veridical experience. The Naïve Realist owes us an account of what exactly the metaphysical structure of hallucination is. Before turning to that issue, though, let us pause to reflect on what metaphysical disjunctivism is committed to. For arguably much of the resistance to this form of disjunctivism is the result of taking it to be a much stronger claim than it has to be in order to defend Naïve Realism. I suspect that one source of misunderstanding are characterizations of metaphysical disjunctivism as claiming that veridical experience and at least hallucination have no mental commonalities (e.g. Hinton 1973: 62; Martin 2004: 37; and Byrne and Logue 2009: ix).37 However, such characterizations are misleading. One thing a veridical experience of a yellow banana and a total hallucination as of one have in common is that they are perceptual experiences as of a yellow banana. Another thing they have in common (at least on the face of it) is their phenomenal character—what it’s like to have them is the same.38 Arguably, these are mental commonalities if anything is. So a more charitable formulation of metaphysical disjunctivism is the following: the ultimate psychological facts that ground a veridical experience are radically different from those that ground a subjectively indistinguishable hallucination. This claim allows for a wide range of mental commonalities across experiences that fall in different disjuncts—commonalities which can be employed in responding to critics of the view.39 Some Naïve Realists make a further claim that has raised eyebrows—namely, that their view requires not just disjunctivism about the metaphysical structure of experience, but also what we might call disjunctivism about phenomenology. The latter is the claim that what it is like to have a veridical experience is different from what it is like to have a subjectively indistinguishable hallucination. That is, even though the subject of a hallucination as of a yellow banana couldn’t tell her experience apart from a veridical one of a yellow banana upon the 35 

For fancy footwork along these lines, see Brewer (2008), Antony (2011), and Kalderon (2011). The ‘VI v H’ and ‘V v IH’ terminology was introduced in Byrne and Logue (2008: 69). Metaphysical disjunctivists tend to go for a V v IH version with little in the way of argument for it (see Martin 2006: 360 and Campbell 2002: 117), but for VI v H versions, see Snowdon (1979/80: 185) and Brewer (2008). 37  More precisely, the claim is usually that a veridical experience of (say) a yellow banana and a subjectively indistinguishable hallucination as of one have no reasonably specific commonalities, e.g. commonalities specific enough to not characterize an experience as of (say) a red tomato (see Byrne and Logue 2009: ix). The point I am about to make shows that even this qualified claim is too strong. 38  Some think that Naïve Realism entails that hallucinations are phenomenally different from veridical perceptions—I’ll discuss this claim shortly. 39  One quick example:  Tyler Burge objects to metaphysical disjunctivism on the grounds that it is incompatible with a principle presupposed by empirical study of perceptual experience—roughly, that there are commonalities across veridical experiences and illusions that have the same proximal cause which are relevant to explaining what the subject believes and does (Burge 2005). First, note that this is no objection at all to VI v H metaphysical disjunctivism. But more importantly, no V v IH disjunctivist in her right mind would deny there are mental commonalities between a veridical experience and an illusion with the same proximal cause—e.g. the property of being an experience as of a yellow banana, which is the sort of thing we appeal to all the time in explaining people’s beliefs and behaviours. 36 

Disjunctivism   211 most careful introspection, what it is like for her to have this hallucination is nevertheless different from what it would be like for her to have a veridical experience of the relevant kind. Indeed, some have gone so far as to deny that hallucinations have phenomenal character at all. According to William Fish (2009: 98), there’s nothing it’s like to have a total hallucination; rather, it’s just that hallucinators are prone to form the mistaken belief that they’re in states that have phenomenal character because these states tend to have the same behavioural and cognitive effects as states that do have phenomenal character (i.e. veridical experiences). Why would a Naïve Realist make such an eyebrow-raising claim? Disjunctivism about phenomenology follows if we identify phenomenal character with relational properties—e.g. if we identify what it’s like to experience yellowness with the property of seeing it (a property one couldn’t instantiate in the absence of a yellow thing).40 Given that the subject of a hallucination doesn’t see anything yellow, it follows that there’s nothing it’s like to be in that state. But the Naïve Realist need not account for phenomenal character in this way. Even though the Naïve Realist holds that a veridical experience has the psychological features it does in virtue of the subject bearing the perceptual relation to things in her environment, this doesn’t mean that all (or even any) of its psychological features are identical to the obtaining of this relation. In particular, when it comes to phenomenal character, the Naïve Realist could say that it is multiply realizable. Just as a functionalist about pain says that it can be realized by C-fibres firing or, say, a radically different state of an alien, the Naïve Realist could say that the phenomenal character associated with experience of yellowness can be realized by seeing an instance of it, or something else entirely (i.e. whatever’s going on in the case of total hallucination). Of course, this alternative isn’t entirely unproblematic; but it’s considerably more palatable than the claim that there’s nothing it’s like to have a total hallucination.41 Let us now turn to what the Naïve Realist really ought to say about hallucination. The options divide into two broad categories. First, we have positive (metaphysical) disjunctivism, which characterizes hallucination in terms that are independent of veridical experience. For example, a positive disjunctivist might say that hallucination fundamentally consists in perception of sense-data, perception of sensible profiles, or representing one’s environment as being a certain way—what these candidate accounts of hallucination have in common is that none of them makes reference to veridical experience. By contrast, a Naïve Realist might opt for negative (metaphysical) disjunctivism, which characterizes hallucination in terms of some relation to veridical experience. For example, one such account holds that hallucination fundamentally consists in the property of being subjectively indiscriminable from a veridical experience of a certain kind (e.g. a veridical experience of a yellow banana).42 One might find negative disjunctivism difficult to accept—the idea that there’s nothing to a total hallucination beyond being subjectively indiscriminable from a veridical experience 40 

Pace Johnston (2004). For further elaboration and defence of this alternative, see Logue (2013); for a (later but more quickly published) repudiation, see Logue (2012). 42  Negative accounts of hallucination along these lines are defended by Martin (2004, 2006) and Fish (2008, 2009). Their accounts differ with respect to what the relevant sort of subjective indiscriminability is. Positive disjunctivism is less popular, thanks to an argument of Martin’s that will be discussed below. But a version of it is endorsed in Johnston (2004) (although he wouldn’t call it that, since he takes ‘disjunctivism’ to pick out what I’m calling ‘negative disjunctivism’). The ‘positive’ and ‘negative’ terminology comes from Byrne and Logue (2008). 41 

212   Heather Logue of a certain kind strikes some as a rather unsatisfying account. To outline just one potential source of dissatisfaction: one might have hoped that whatever account of hallucination we give will specify the facts in virtue of which a hallucination is subjectively indistinguishable from a veridical experience of a yellow banana. A positive account would yield such an explanation: for example, if hallucination fundamentally consisted in representing one’s environment as being a certain way, we could then say that a hallucination as of a yellow banana is subjectively indistinguishable from a veridical experience of one in virtue of the fact that both kinds of experience fundamentally consist in representing that there’s a yellow thing before one. But if we say, as the negative disjunctivist does, that all there is to a total hallucination as of a yellow banana is being subjectively indiscriminable from a veridical experience of one, we’ve denied ourselves the resources to account for this fact. In spite of this worry, most metaphysical disjunctivists are negative disjunctivists—they have been persuaded by a powerful argument against positive disjunctivism given by M. G. F. Martin.43 The argument has two basic parts, the first of which we’ve already seen in section 2. Recall that the subject of a veridical experience and the subject of a total hallucination could be in the same type of brain state; and if such a brain state gives rise to a certain kind of psychological state in the case of total hallucination, there’s no good reason to claim that it wouldn’t do so in the case of veridical experience. Hence, whatever is going on in hallucination is going on in veridical experience too—if hallucination involves perception of sense-data, sensible profiles, or representing one’s environment as being a certain way, so does veridical experience.44 This fact in itself might not seem like a problem for the Naïve Realist—why couldn’t he say that veridical experience consists in the subject bearing the perceptual relation to things in her environment and something else? This is where the second stage of the argument comes in. It begins with the fact that a veridical experience of a yellow banana and a subjectively indiscriminable hallucinatory counterpart have doxastic, behavioural, and phenomenal commonalities. For example, they both tend to give rise to the belief that there’s a yellow banana before one, they both cause one to reach in a certain direction if one fancies a snack, and (pace Fish and Logue 2012) what it is like to have them is the same. Now, suppose for the sake of a reductio that some form of positive disjunctivism is true—say, that hallucinations fundamentally consist in the subject representing her environment as being a certain way. By the argument summarized in the last paragraph, veridical experience involves the subject representing her environment as being a certain way too. Given that the veridical experience and the hallucination have this representational commonality, it seems that it’s best suited to explain the doxastic, behavioural, and phenomenal features of the experiences. For example, if the hallucination gives rise to the belief that there’s a yellow banana before one in virtue of the fact that it consists in representing one’s environment as containing a yellow banana, presumably this is the case for veridical experience as well. The perceptual relation between the subject and things in her environment is effectively ‘screened off’ from explaining the veridical experience’s doxastic effects. So too with its behavioural effects and its phenomenal character. Naïve Realism cannot allow that the perceptual relation between subject and object is 43 See

Martin (2004) for the argument, and Brewer (2008:  173)  and Fish (2009:  84–85) for endorsements of it. 44  This stage of the argument is laid out in detail in Martin (2004: 52–58).

Disjunctivism   213 screened off in this manner, as it holds that a veridical experience has all of its psychological features ultimately in virtue of this perceptual relation.45 Hence, given that we’re interested in metaphysical disjunctivism for the purpose of preserving Naïve Realism, it appears that positive disjunctivism won’t do the trick—negative disjunctivism is our only option.46 Or is it? Suppose a positive disjunctivist says something like the following:  yes, a veridical experience of a yellow banana consists in (say) representing one’s environment as containing a yellow crescent-shaped banana. And yes, this representational fact explains the experience’s doxastic, behavioural, and phenomenal features. But this doesn’t screen off the obtaining of the perceptual relation from explaining these things too, because the subject is in the representational state in virtue of bearing the perceptual relation to the banana and certain of its properties. That is, while veridical experience consists in the subject representing her environment as being a certain way, it doesn’t fundamentally consist in this. The doxastic, behavioural, and phenomenal features of the experience are ultimately explained by the subject bearing the perceptual relation to the banana and certain of its properties. By contrast, the hallucination fundamentally consists in the subject representing her environment as being a certain way; there’s no further psychological fact in virtue of which she is in this representational state. Of course, this proposal raises a number of worries I don’t have the space to address here, for example: can a hallucination fundamentally consist in something while a veridical experience consists in the same sort of thing, but not fundamentally so? And what are the grounds for saying that the common property is instantiated in virtue of the perceptual relation obtaining in veridical experience?47 Nevertheless, since negative disjunctivism strikes many as an unsatisfactory account of hallucination, this positive disjunctivist proposal is worth pursuing.48 So is metaphysical disjunctivism true? If Naïve Realism is true, then yes. But Naïve Realism is true only if some form of metaphysical disjunctivism can provide an adequate account of hallucination. These issues are subjects of intense debate in contemporary philosophy of perception. In my view, the jury’s still out.

References Antony, L. (2011). ‘The Openness of Illusions’. Philosophical Issues, 21, 25–44. Austin, J. (1962). Sense and Sensibilia. Oxford: Oxford University Press.

45 

This stage of the argument is laid out in detail in Martin (2004: 58–63). course, on negative disjunctivism, both a veridical experience of a yellow banana and a hallucination as of one are subjectively indiscriminable from a veridical experience of a yellow banana. Martin argues that the indiscriminability property doesn’t screen off the obtaining of the perceptual relation because the explanatory power of the former depends on that of the latter (Martin 2004: 69–70). 47  For a fuller discussion of this positive disjunctivist proposal and objections to it, see Logue (2013). Another more radical positive disjunctivist proposal (one which involves denying that total hallucinations have phenomenal character) is elaborated and defended in Logue (2012). 48  For more nuanced objections to negative disjunctivism than the one presented earlier in this section, see Siegel (2004 and 2008), Johnston (2004: 124–127), and Sturgeon (2006: 208–210). 46  Of

214   Heather Logue Ayer, A. J. (1956). The Problem of Knowledge. Harmondsworth: Penguin. Block, N. (1990). ‘Inverted Earth’. Philosophical Perspectives, 4, 53–79. Block, N. (1996). ‘Mental Paint and Mental Latex’. Philosophical Issues, 7, 19–49. Brewer, B. (1999). Perception and Reason. Oxford: Oxford University Press. Brewer, B. (2008). 'How to Account for Illusion'. In A. Haddock and F. Macpherson (eds), Disjunctivism: Perception, Action, Knowledge. Oxford: Oxford University Press. Broad, C. D. (1925). The Mind and its Place in Nature. London: Routledge and Kegan Paul. Burge, T. (2005). ‘Disjunctivism and Perceptual Psychology’. Philosophical Topics, 33, 1–78. Byrne, A. (2001). ‘Intentionalism Defended’. Philosophical Review, 110, 119–240. Byrne, A. (2012). 'Knowing what I See'. In D. Smithies and D. Stoljar (eds), Introspection and Consciousness. Oxford: Oxford University Press. Byrne, A. and Logue, H. (2008). 'Either/Or'. In A. Haddock and F. Macpherson (eds), Disjunctivism: Perception, Action, Knowledge. Oxford: Oxford University Press. Byrne, A. and Logue, H. (2009). 'Introduction'. In A. Byrne and H. Logue (eds), Disjunctivism: Contemporary Readings. Cambridge, MA: MIT Press. Campbell, J. (2002). Reference and Consciousness. Oxford: Oxford University Press. Chalmers, D. (2006). 'Perception and the Fall from Eden'. In T. Gendler and J. Hawthorne (eds), Perceptual Experience . Oxford: Oxford University Press. Crane, T. (2009). 'Is Perception a Propositional Attitude?' Philosophical Quarterly, 59, 452–469. Davies, M. (1992). ‘Perceptual Content and Local Supervenience’. Proceedings of the Aristotelian Society, 92, 21–45. Dretske, F. (1995). Naturalizing the Mind. Cambridge, MA: MIT Press. Fish, W. (2008). 'Disjunctivism, Indistinguishability, and the Nature of Hallucination'. In A. Haddock and F. Macpherson (eds), Disjunctivism: Perception, Action, Knowledge . Oxford: Oxford University Press. Fish, W. (2009). Perception, Hallucination, and Illusion. Oxford: Oxford University Press. Foster, J. (2000). The Nature of Perception. Oxford: Oxford University Press. Harman, G. (1990). ‘The Intrinsic Quality of Experience’. Philosophical Perspectives, 4, 31–52. Hinton, J. M. (1973). Experiences:  An Inquiry into Some Ambiguities. Oxford:  Oxford University Press. Huemer, M. (2011). ’Sense-Data'. In E. Zalta (ed.), Stanford Encyclopedia of Philosophy. (accessed July 2011). Hume, D. (1777/1993). An Enquiry Concerning Human Understanding, (ed.) E. Steinberg, 2nd edn. Indianapolis, IN: Hackett Publishing Company. Jackson, F. (1977). Perception:  A  Representative Theory. Cambridge:  Cambridge University Press. Johnston, M. (2004). ‘The Obscure Object of Hallucination’. Philosophical Studies, 120, 113–183. Johnston, M. (2006). 'Better than Mere Knowledge? The Function of Sensory Awareness'. In T. Gendler and J. Hawthorne (eds), Perceptual Experience. Oxford: Oxford University Press. Kalderon, M. E. (2011). ‘Color Illusion’. Nous, 45, 751–775. Kennedy, M. (2011). ‘Naive Realism, Privileged Access, and Epistemic Safety’. Nous, 45, 77–102. Logue, H. (2011). ‘The Skeptic and the Naive Realist’. Philosophical Issues, 21, 268–288. Logue, H. (2012). 'What Should the Naïve Realist Say about Total Hallucinations?' Philosophical Perspectives, 26, 173–199. Logue, H. (2013). ‘Good News for the Disjunctivist about (one of) the Bad Cases’. Philosophy and Phenomenological Research, 86, 105–133.

Disjunctivism   215 Logue, H. (2014). 'Experiential content and Naive Realism: A Reconciliation'. In B. Brogaard (ed.), Does Perception Have Content? Oxford: Oxford University Press. Martin, M. G. F. (2002). ‘The Transparency of Experience'. Mind and Language, 17, 376–425. Martin, M. G. F. (2004). ‘The Limits of Self-Awareness’, Philosophical Studies, 120: 37–89. Martin, M. G. F. (2006). 'On Being Alienated'. In T. Gendler and J. Hawthorne, Perceptual Experience. Oxford: Oxford University Press. McDowell, J. (1982). ‘Criteria, Defeasibility, and Knowledge’. Proceedings of the British Academy, 68, 455–479. McDowell, J. (2008). 'The Disjunctive Conception of Experience as Material for a Transcendental Argument'. In A. Haddock and F. Macpherson (eds), Disjunctivism: Perception, Action, Knowledge. Oxford: Oxford University Press. McGinn, C. (1982). The Character of Mind. Oxford: Oxford University Press. Millar, A. (2008). 'Perceptual-Recognitional Abilities and Perceptual Knowledge'. In A. Haddock and F. Macpherson (eds), Disjunctivism: Perception, Action, Knowledge. Oxford: Oxford University Press. Moore, G. E. (1953). Some Main Problems of Philosophy. London:  George, Allen, and Unwin. Moore, G. E. (1962). 'Proof of an External World'. Philosophical Papers. New York: Collier Books. Neta, R. (2008). 'In Defense of Disjunctivism'. In A. Haddock and F. Macpherson (eds), Disjunctivism: Perception, Action, Knowledge. Oxford: Oxford University Press. Peacocke, C. (1983). Sense and Content. Oxford: Oxford University Press. Price, H. (1950). Perception, 2nd ed. London: Methuen. Pritchard, D. (2008). 'McDowellian Neo-Mooreanism'. In A. Haddock and F. Macpherson (eds), Disjunctivism: Perception, Action, Knowledge. Oxford: Oxford University Press. Robinson, H. (1994). Perception. London: Routledge. Russell, B. (1912). Problems of Philosophy. Oxford: Oxford University Press. Schellenberg, S. (2011). ‘Perceptual Content Defended’. Nous. 45, 714–750. Siegel, S. (2004). ‘Indiscriminability and the Phenomenal’. Philosophical Studies, 120, 91–112. Siegel, S. (2008). 'The Epistemic Conception of Hallucination'. In A. Haddock and F. Macpherson (eds), Disjunctivism: Perception, Action, Knowledge. Oxford: Oxford University Press. Siegel, S. (2010). ‘Do Visual Experiences Have Contents?’ In B. Nanay (ed.), Perceiving the World. Oxford: Oxford University Press. Smith, A. D. (2002). The Problem of Perception. Cambridge, MA: Harvard University Press. Snowdon, P. (this volume). 'Sense-Data'. In M. Matthen (ed.), Oxford Handbook of the Philosophy of Perception. Oxford: Oxford University Press. Snowdon, P. F. (1979/80). ‘Perception, Vision, and Causation’. Proceedings of the Aristotelian Society, 81, 175–192. Snowdon, P. F. (2005). 'The Formulation of Disjunctivism: A Response to Fish'. Proceedings of the Aristotelian Society, 105, 129–141. Stroud, B. (1984). The Significance of Philosophical Skepticism. Oxford: Clarendon Press. Sturgeon, S. (2006). ‘Reflective Disjunctivism’. Proceedings of the Aristotelian Society, Supplement, 80, 185–216. Thau, M. (2004). ‘What is Disjunctivism?’. Philosophical Studies, 120, 193–253.

216   Heather Logue Travis, C. (2004). ‘The Silence of the Senses’. Mind, 113, 57–94. Tye, M. (2000). Consciousness, Color, and Content. Cambridge, MA: MIT Press. Tye, M. (2007). ‘Intentionalism and the Argument from no Common Content’. Philosophical Perspectives, 21, 589–613. Williamson, T. (2000). Knowledge and Its Limits. Oxford: Oxford University Press.

Chapter 12

Action-based Accou n ts of Perception Pierre Jacob

According to a prevalent view in twentieth-century analytic philosophy of action and perception, an agent’s ability to act on the world and her ability to perceive the world are two distinct abilities (cf. e.g. Davidson, 1980; Searle, 1983). For an agent to act is to change the world by executing bodily movements as a means to fulfil some intention or other. An agent’s intention is a psychological cause of her movements. It has, in Searle’s (1983) words, a world-to-mind direction of fit: to form an intention is to represent a possible (non-actual) state of affairs that the agent’s intended action would turn into a fact. To perceive the world is to register or record an actual state of affairs or a fact as it is, through one of the five fundamental sensory modalities: sight, hearing, touch, smell, or taste. There are two sides to perception, both of which reflect the mind-to-world direction of fit of perceptual experiences: an epistemic objective side and a phenomenological subjective side. On the objective side, perceptual experiences enable humans to form reliable beliefs about the way the world is. On the subjective side, perception in each sensory modality offers a distinctive phenomenological awareness of the world: there is something it is like to seeing a white leaf of paper that is different from what it is like to smelling it, tasting it, touching it, or hearing it being ripped. To a large extent, the dualism between action and perception seems to fit the rejection of the behaviourist approach to perception characteristic of the cognitive revolution of the 1950s and the subsequent rise of the computational paradigm in the scientific investigation of perception, advocated by David Marr (1982) and others. The computational paradigm in turn was in broad agreement with one of the dominant trends in twentieth-century analytic philosophy of perception, that is representationalism (also known as intentionalism), which stands in sharp contrast with disjunctivism. While advocates of disjunctivism endorse a version of direct realism and stress that in perception, an agent is directly related to mind-independent objects in the world, advocates of representationalism argue that perceptual experiences have content or conditions of satisfaction and depict the perceiver’s surroundings as being thus and so by representing some of the properties of perceivable objects. For example, visual experiences represent the sizes, shapes, spatial positions, colours, and textures of perceivable objects.1 1 For

a defence of disjunctivism, see Hinton (1973) and McDowell (1982). For a defence of representationalism, see Dretske (1995) and Tye (1995).

218   Pierre Jacob In recent years, the reciprocal interactions between action and perception have been the topic of intense investigations. As a result, many prominent cognitive scientists and philosophers of perception have engaged in the exploration of action-based approaches to perception. Not only does the success of an agent’s goal-directed action seem to depend on her ability to select and locate her target, but also an agent’s executed action has direct consequences on her own sensory experience. Action-based accounts of perception, however, do not content themselves with these causal interdependencies. They argue instead for stronger constitutive links between action and perception. There are two main motivations to action-based approaches to perception: one is their parsimonious assumption that, far from being two autonomous systems, each of which contributes to the function of the other, action and perception belong to a single overlapping functional system. The other one is the tendency to minimize the load of internal processing and the role of mental representations in perception. The goal of the present chapter is to assess the scope and limits of action-based approaches to perception, mostly in the visual modality. Many advocates of action-based approaches to perception link their rejection of the computational/representational approach to one or another version of the embodied cognition approach to perception, to which the first section of this chapter is devoted. The second section will examine the support that action-based approaches to perception can expect to derive from the so-called two-systems model of visual processing. The third section will be devoted to the problems faced by one of the recent most prominent action-based accounts of object-perception named enactivism. Finally, the last section will examine action-based accounts of social perception.

1  Embodied perception One of the earliest defences of the embodied perception research programme was Gibson’s (1979) ecological approach to visual perception, which contributed two related fundamental insights. One is that much object-perception consists in detecting possibilities for action, which Gibson called affordances. Different things afford different possibilities for action to different organisms: for example, the surface of a lake affords different possibilities of spatial navigation to insects, birds, and mammals. The second one is that much visual perception consists in an agent’s movements that enable her to extract and manipulate information lying in the optic array. It is also distinctive of embodied perception both to embrace an empiricist account of concepts and to reject the computational approach to perception.

Concept-empiricism Concept-empiricism, advocated by, for example, Barsalou (1999), Barsalou et al. (2003), and Prinz (2002, 2005), involves three basic ingredients. First, it is based on the rejection of the linguistic model of concepts as amodal symbols in some language of thought, whose meanings (or contents) stand to their non-phonetic forms (or vehicles) in the same arbitrary relation as the meanings of words of natural languages stand to their phonetic properties (cf. Barsalou et al., 2003). Secondly, on the empiricist view, concepts are (visual,

Action-Based Accounts of Perception    219 auditory, olfactory, tactile, or motor) images encoded in the various perceptual and motor systems. Finally and correlatively, conceptual processing is taken to be a re-enactment or simulation of basic perceptual and motor processing. For example, retrieving the concept of a dog would consist in re-enacting or simulating some perceptual representation or mental image of a dog (for further discussion, see Machery, 2006, 2007). While some philosophers (e.g. McDowell, 1994) have denied the distinction between representations with conceptual and nonconceptual content on the grounds that the content of perceptual experiences is fully conceptual, advocates of concept-empiricism also deny the same distinction, but for the opposite reason: they first assume that the representations of all perceivable objects have perceptual and/or sensorimotor, that is nonconceptual, content and they further assume that the contents of representations of non-perceivable (e.g. abstract, mathematical or theoretical) objects are derivable (or constructible) from the contents of the representations of perceivable objects. The empiricist denial of the conceptual/nonconceptual content distinction faces four basic challenges. The first one is that, as Descartes famously noticed, one can perceive (or imagine) triangular objects and one can also apply to them the concept of a triangle. But a chiliagon (i.e. a geometrical object with 1 000 sides) is a discursively defined concept that can only be represented and distinguished from a closed geometrical figure with 999 sides conceptually, not by the imagination. If this is right, then concept-empiricism seems forced to posit a dual account according to which the contents of some, not all, concepts can be grounded in perceptual and/or motor processes. The second one is the perceptual binding problem. One can see a poodle, an alsatian, a dalmatian, a bulldog, and so on. Seeing one of them is clearly a different experience from seeing another. But so is hearing, smelling, or touching one of them. The challenge for concept-empiricism is to account for cross-modal binding between the properties of one and the same individual dog represented in different sensory modalities without positing amodal vehicles with cross-modal, i.e. conceptual, content. The third challenge is the belief/experience distinction. Among the reasons why we need to draw a distinction between perceptual beliefs and perceptual experiences are perceptual illusions. For example, when one is presented with two segments of equal length, with arrows pointing respectively in and out (as in the Müller-Lyer illusion), the former unavoidably looks longer than the latter. One’s visual experience represents the two segments as unequal even if one believes that they are equal. This shows that the content of one’s visual experience is encapsulated from the content of one’s beliefs in Fodor’s (1983) terms: unlike a belief, a visual experience is not revisable in light of further evidence. Given that the content of one’s visual experience of the two segments as unequal persists in the presence of one’s belief that the two segments are equal, it follows that the function, content, and character of one’s visual experiences cannot be identified to the function, content, and character of one’s beliefs. The fourth challenge faced by concept-empiricism is the logical form challenge. Advocates of the view that conceptual vehicles are amodal symbols in some language of thought, like Fodor (1975), argue that whereas the content of a primitive concept (e.g. the concept of a dog) is not further definable, the content of a complex concept (e.g. the concept of a blue dog) systematically depends on the contents of its constituents. Symbols with conceptual content have a logical form, or in Fodor’s (2007) terms, a ‘canonical decomposition’: not every part of a complex symbol with conceptual content is one of its proper

220   Pierre Jacob constituents. Some of its constituents stand for individuals, others for properties, still others for logical operations. By contrast, visual percepts, visual mental images and pictures are representations with iconic or non-conceptual content. The iconicity of percepts, images, and pictures enables them to represent by virtue of the fact that their internal structure preserves some of the spatial relations (e.g. part–whole relations, overlap, and occlusion) holding among the constituents of the states of affairs that they depict. But iconic representations with non-conceptual contents lack a logical form and a canonical decomposition. The challenge for advocates of concept-empiricism is to show how representations without a canonical decomposition could generate representations with a canonical decomposition.

The rejection of the computational approach to perception Marr’s (1982) book on vision is a stepping-stone of the computational approach to visual perception. It is based on a tripartite distinction between three levels of analysis. The highest ‘computational’ level involves a specification of the task which the visual system has been designed to perform, that is, the construction of a three-dimensional representation of distal stimuli on the basis of retinal inputs.2 The lowest (hardware) level involves a description of how the computations are physically or biologically ‘implemented’ or ‘realized’ (by the neurochemical properties underlying photo-transduction in retinal cells). The intermediate ‘algorithmic’ level involves the particular operations used by the system in performing its computations. In his book on vision, Marr (1982) also proposed a model of the computational level itself, which involves three complementary stages, the first of which takes retinal stimulations as input and generates a so-called primal sketch of reflectance changes on the surface outside the viewer. Secondly, from the primal sketch as input, the visual system computes a 2.5-D sketch of the surface, which encodes information about depth and contours of the surface relative to the perceiver. Finally, given the 2.5-D sketch as input, the visual system computes a 3-D representation of objects, which captures their shapes and orientations. Following Gibson’s (1979) ecological approach, advocates of embodied perception reject two complementary aspects of the computational approach to perception. First, they reject the thesis of multiple realizability, which follows from Marr’s (1982) tripartite distinction between the computational, the algorithmic, and the implementation levels, that is, the thesis that one and the same computational state can be realized by several distinct underlying physical, chemical, or biological states.3 Thus, Shapiro (2004: 175), who embraces the embodied perception research programme, argues against the thesis of multiple realizability on the grounds that it entails two related theses which he calls respectively body neutrality and the separability thesis (i.e. the separability between the computer program that runs visual perception and the body that implements the program), both of which are consistent with the idea ‘that the mind is a program that can be characterized in abstraction 2  Noë (2004: 22) rejects the view that ‘vision is a matter of generating a detailed internal representation of the visual world on the basis of information available at the retina alone’ (cf. section 3). 3  For well-known defences of the thesis of multiple realizability, see Putnam (1967) and Fodor (1974). For a critique, see Kim (1993).

Action-Based Accounts of Perception    221 from the kind of body/brain that realizes it’. For example, Shapiro (2004: 187) stresses the significance for visual perception of the fact that the human body involves two eyes: an object (or distal stimulus) projects two distinct retinal images onto each eye and the brain makes use of disparity information from both eyes to compute the relative depth of the object. Secondly, as Shapiro (2007) puts it, ‘embodied cognition  . . .  departs from traditional cognitive science in its reluctance to conceive of cognition as computational and in its emphasis on the significance of an organism’s body in how and what the organism thinks and perceives’. In other words, embodied perception theorists reject the assumption that perceptual processes are computations that could be performed by a disembodied machine on abstract symbols in virtue of their syntactic properties alone, as if the individual’s full bodily anatomy did not matter to the process. Instead, they stress the role of the human body in perceptual processes because they accept the ecological Gibsonian emphasis on the role of an agent’s movements in extracting information from the optic array and the detection of affordances (i.e. possibilities of action) for visual perception, both of which depend on the agent’s bodily anatomy. There are, however, two possible interpretations of the claim made by embodied perception theorists: one is the uncontroversial claim that human perception causally depends on the anatomical structure of the human body. The other is the more controversial claim that the character and content of human perceptual experiences are constitutively determined by the human anatomy (for a critical evaluation of the stronger constitutive claim, see Aizawa, 2007; Block, 2005). The distinction between the weaker and the stronger version of the dependency of perceptual processes on human anatomy is important for evaluating the tension between the embodied perception framework and the traditional action/perception dualism.

2  The two-visual systems model of human vision In the past forty years or so, a number of surprising dissociations between visual perception and visually guided actions have been at the basis of a novel approach to human vision: the two-visual systems model, first advocated by Milner and Goodale (1995). First of all, this model is based on the insight that the visual guidance of actions of prehension and the perceptual experience of objects make inconsistent demands on the visual processing of information about objects. Secondly, this model maps the functional distinction between visually guided actions and visual perception onto the anatomical segregation between the so-called ‘dorsal’ stream and the so-called ‘ventral’ stream of the human visual system. Finally, it assumes that activity in the so-called ‘ventral’ stream plays a distinctive role in visual awareness of the world. Humans have dexterous hands that enable them to grasp targets in their peripersonal space with either precision grip or full prehension. For the purpose of reaching a target of prehension, an agent must represent its spatial position in an egocentric coordinate system centred on her body. For the purpose of grasping it, the agent must represent its size, shape,

222   Pierre Jacob and orientation, not its colour. Arguably, a visuomotor representation of a target of prehension has, like Millikan’s (1996) pushmi-pullyu representations, a dual direction of fit: it describes some relevant features of the target in a format appropriate for the guidance of action. By contrast, the visual experience of objects serves two complementary functions: selection and recognition (or identification). Selecting an object consists in both segregating a complex visual array into several separable objects and in attributing to each item its own set of appropriate visual attributes (this is the so-called ‘binding’ problem). Usually, the colour and texture of an object will be highly relevant to its perceptual selection from a set of neighbouring objects. Segregation and binding require that the relative spatial locations of different objects in a visual array be coded by the perceptual system. Since perceptual recognition of an object must be achieved from many different spatial perspectives on many different times of the day in different seasons, it requires encoding visual information about an object’s enduring properties.

An anatomical bifurcation in the primate visual system Ungerleider and Mishkin (1982) were the first to report evidence for an anatomical bifurcation between two streams within the primate visual cortex itself: the ventral stream that projects the primary visual areas onto the inferotemporal cortex and the dorsal stream that projects the primary visual cortex onto the parietal lobe.4 They examined the selective effects of lesions in the brains of macaque monkeys on two kinds of behavioural tasks: a landmark task and an object-discrimination task. In the former task, the monkey had to discriminate between two covered wells—one empty and one containing a reward—according to whether they were located far away or near a landmark. In the latter task, the monkey had to discriminate two objects of different shapes, colours, and textures. Ungerleider and Mishkin found that a lesion in the inferotemporal cortex severely impaired the animal in the object-discrimination task, but not in the landmark task. Conversely, they found that a lesion in the posterior parietal cortex severely affected the animal’s performance in the landmark task, but not in the object-discrimination task. On the basis of these experiments, Ungerleider and Mishkin (1982) concluded that the ventral stream (which they called the ‘object-channel’) and the dorsal stream (which they called the ‘space-channel’) were each specialized in perceiving different aspects of the visual world. Indeed, their landmark task tested the animal’s ability to perceive spatial relations, not to act on a target.

Neuropsychological dissociations Further work based on the neuropsychological examination of brain-lesioned human patients gave rise to what is now known as the ‘two-visual systems model’ of human

4 For evidence that different neural networks underlie different visual capacities respectively in amphibians and in small rodents, without a visual cortex, cf. Ingle (1973) and Schneider (1969).

Action-Based Accounts of Perception    223 vision, the first step of which was the striking discovery of the phenomenon called ‘blindsight’. Weiskrantz et al. (1974) reported that patients who had lost conscious visual perception following a lesion in their primary visual cortex were still able to produce accurate visuomotor responses to visual stimuli presented in the part of their visual field impaired by the lesion, in forced choice conditions.5 The second major step was the discovery in the 1990s of a double dissociation between the visual capacities of patients with two kinds of visual impairments: visual form apperceptive agnosia and optic ataxia. On the one hand, Milner et al. (1991) and Goodale and Milner (1992) reported that examination of patient D.F. with apperceptive visual agnosia following bilateral damage to her ventral stream had become unable to recognize the sizes and shapes of three-dimensional objects (and drawings) visually presented to her, but she was still able to grasp these three-dimensional objects with precision grip. For example, her maximum grip aperture (i.e. her finger–thumb precision grip) turned out to be correlated with the physical sizes of targets in tasks of grasping. However, when asked to mime the width of a target in tasks of perceptual judgements, the distance between her thumb and index finger turned out to be at chance. Furthermore, while she was able to insert a card within a slot at different orientations, she was unable to match the different orientations of the slot by appropriately rotating her wrist in tasks of perceptual judgement. Finally, when asked to point to a peripheral target, her accuracy was like that of the controls. However, when a delay was interposed between the presentation of the stimulus and the signal to respond her performances decreased. On the other hand, Jeannerod et al. (1994) reported that patient A.T. with optic ataxia following bilateral damage to her dorsal stream had become unable to reach and grasp with accuracy objects whose shapes and sizes she was able to visually recognize. In visuomotor tasks, A.T. was able to reach for the target, but her grasp was systematically incorrect. However, when asked to match the size of objects with the distance between her thumb and index finger, she gave accurate estimates in positive correlation with the sizes of the objects.6 In contrast to patient D.F., when asked to point to a target, patient A.T.’s performances improved when a delay was interposed between the presentation of the stimulus and the signal to respond.

Psychophysical dissociations in healthy participants In addition to neuropsychological dissociations, numerous experiments have provided evidence of dissociations between perceptual judgements and visuomotor responses by healthy human adults. For example, Goodale et al. (1986) report that participants were able

5 

Rossetti (1998) describes a case of so-called ‘numbsense’ in which a patient exhibited a complete loss of all somatosensory processing on the left half of his whole body. When blindfolded and required to guess verbally the locus of tactile stimuli delivered to his forearm and hand, he performed at chance level. Nevertheless he showed a significant ability to point his finger at the accurate location of stimuli delivered to his arm. 6  For discussion of the scope and limits of the double dissociation between visual form agnosia and optic ataxia, see Pisella et al. (2006) and Rossetti et al. (2010), and for a reply, see Milner and Goodale (2010).

224   Pierre Jacob to accurately point their index finger to visual targets that changed position during a concurrent saccadic eye-movement, while they showed no conscious awareness of the motion of the target. Whereas activity in the dorsal stream enabled participants to unconsciously track the motion of the target and point, the conscious image of the target depends on slower computations performed by the ventral stream. This is consistent with the results of experiments by Pisella et al. (2000), in which healthy participants were instructed to perform a pointing task at a target that could change location at the onset of participants’ movements. In response to target perturbation, the instruction was to either correct or stop the ongoing movement (stop condition). Pisella et  al. (2000) report that participants in the stop condition irrepressibly produced a significant percentage of very fast non-intentional unwilled corrective movements, which they attributed to the ‘automatic pilot’. Furthermore, Rossetti and Pisella (2002) also report that, unlike the change of a target’s location, the change of the target’s colour did not elicit very fast, unwilled, corrective hand movements. Króliczak et al. (2006) report the following dissociation prompted by seeing the hollow mask illusion, in which a concave hollow face is perceived as being convex. Participants were presented with one of three distinct displays: they could see a convex mask that looked convex, a hollow mask that looked convex or a hollow mask that looked concave. On the forehead and cheek of each display, a small magnetic dot was attached. If asked to deliberately point to the small dot attached to the hollow mask, participants directed their finger movements to the illusory location of the target. However, if asked to quickly flick the target off the face (as if it were a small insect), they directed their finger movements to the actual or veridical location of the target, despite the presence of a strong hollow-face perceptual illusion. Many experiments have investigated dissociations between perceptual judgements and visuomotor responses to displays consisting of size-contrast visual illusions.7 For example, Ganel et al. (2008) report an experiment based on the Ponzo illusion in which participants saw two targets that were unequal in length, but due to the presence of the illusory background constituted by two converging lines, the shorter target (at the converging end) looked longer than the other one (at the diverging end). Participants were requested to either grasp one of the targets or use the distance between their index finger and thumb to estimate its length. Despite the fact that participants believed that the shorter object was the longer one, Ganel et al. (2008) report that, when requested to grasp it, their grip aperture in flight reflected the real not the illusory size of the target objects. But when participants were asked to estimate the size of the target objects rather than pick them up, their manual estimates reflected the apparent not the real size of the targets. In other words, on the same trials in which participants erroneously decided that one object was the longer (or shorter) of the two, the anticipatory opening between their fingers reflected the real direction and magnitude of size differences between the two objects.8

7  The investigation of the Ebbinghaus illusion has given rise to much controversy (cf. Franz et al., 2000, and replies by Goodale, 2011, and Milner and Goodale, 2010). 8  Further dissociations between perceptual overestimation of the slants of hills and the accuracy of the visuomotor guidance of locomotion have been reported by Proffitt et al. (1995).

Action-Based Accounts of Perception    225 The evidence (succinctly summarized in this section) for the two-visual systems model raises two complementary challenges for action-based approaches to visual perception: to what extent is action necessary and sufficient for visual perception? There is room for controversy and further investigation. Some scientists and philosophers (e.g. Jacob and Jeannerod, 2003; Block, 2005; Clark, 2010; Jacob and Vignemont, 2010; Milner and Goodale, 2010) have argued that the dissociations between visually-guided actions and conscious visual perception show that the former are neither necessary nor sufficient for the latter: on their interpretation, the residual visuomotor capacities of visual agnosic patients show that the ability to accurately reach and grasp an object is not sufficient for consciously perceiving its size and shape. The visuomotor impairment of optic ataxic patients shows that the ability to accurately reach and grasp an object is not necessary for consciously perceiving its size and shape. Conversely, other scientists and philosophers (e.g. O’Regan and Noë, 2001; Hurley, 2008; Noë, 2010) have responded that the above dissociations fail to demonstrate that while activity in the ventral stream supports conscious visual perception, activity in the dorsal stream underlies the automatic unconscious control of visually-guided actions. Instead, they have argued that all they show is that conscious visual perception can at best be dissociated from visual processing involved in the setting of low-level parameters for reaching and grasping. Furthermore, they have argued that since visual agnosic patients fail to recognize the function and significance of visually presented objects, they cannot act on them in ways appropriate to their use or function. Thus, they argue that so far, the evidence fails to show that the visual guidance of actions is independent from conscious visual perception.

3  The enactive approach to object-perception Whereas the embodied perception framework rejects the computational approach to perception, advocates of the so-called ‘enactive’ framework see an agent’s action as an alternative to classical representationalist approaches to perception. On the enactive conception (advocated by e.g. O’Regan and Noë, 2001; Noë, 2004), perception is intrinsically active and perceptual experiences are ways of acting. The content of an agent’s perceptual experience is determined by her practical or implicit knowledge of sensorimotor contingencies, that is the sensory consequences of her own movements. Noë (2004) has argued that enactivism solves the puzzle of perceptual presence: in perceiving an object whose back surface is occluded from my view (e.g. the back of a tomato in front of me), I am perceptually aware of the occluded part that does not reflect photons on my retina. On the enactive account, the reason I am perceptually aware of the occluded part is that I tacitly know that were I to pick up the tomato and turn it over, I would see the back of the tomato.9 The enactive account of perception gives rise to two important questions. 9 For an entirely different account of the feeling of the perceptual presence of graspable objects (as opposed to images) based on visual processing in the dorsal stream, cf. Matthen (2005, 2010).

226   Pierre Jacob

Can enactivism be a constitutive account of perceptual experience? The first question is:  to what extent is enactivism a constitutive account of the content of perceptual experience? There are two related reasons for caution. Consider a skilled driver’s visual experience of her red Porsche. On the one hand, her driver’s skill might underlie the content of her visual experience of her car as drivable. But she might also see her car as non-drivable (if e.g. the engine has been disassembled or after a bad accident), or on sale, or as a member of her collection of cars, and so on. It is not clear how her implicit knowledge of the sensory consequences of her skilled ability to drive it grounds the content of her visual experience of her car in these cases. On the other hand, she could only see her car as drivable if she had already identified it, that is, sorted it out from the background and from distracting cars in a visual scene. So the question arises: how could tacit knowledge of the sensory consequences of her driving actions onto her car enable her visual system to bind the colour, shape, texture of the body, and wheels of the car together. Unless she was antecedently able to sort it out from neighbouring distractors and from the background, how could she plan to act on it? For example, O’Regan (2010, ch. 9) has argued that the phenomenology of seeing redness is like the feel of driving a Porsche. But it remains unclear how one’s Porsche-driving skills, exercised while sitting at the wheel, could capture what it is like to see the redness of the body of the car while looking at it from the outside.

Enactivism and behaviourism The second question is whether enactivism is not a new version of psychological behaviourism, that is, the view that only observable inputs (i.e. stimuli that impinge on an organism’s body) and observable output (i.e. an organism’s behavioural responses) can be part of a scientific psychological explanation. To a large extent, the cognitive revolution that gave rise to the cognitive sciences was based on the rejection of behaviourism and the rehabilitation of mental representations in psychological explanation (see Chomsky, 1959). Arguably, the enactivist account can dissociate itself from behaviourism since its main thesis is not  that an agent’s perceptual experience arises from the lawful dependencies between her bodily movements and her sensory stimulations, but from the agent’s implicit sensorimotor knowledge of these lawful dependencies. Still the question arises to what extent the concept of action used by enactivists really differs from the behaviourist conception of a response to a stimulus. On behalf of action-based approaches to perception, Hurley (1998, 2001, 2008) has criticized ‘the classical sandwich conception of the mind . . . [that] regards perception as input from world to mind, action as output from mind to world, and cognition as sandwiched between’. The sandwich model of cognition makes two crucial assumptions: the first is the linearity assumption, according to which information flows from the outside world in, through sensory systems to perception, cognition, the motor system and then out again into the world. The second is the instrumentality assumption, according to which action and perception stand as a means to each other. While the ecological approach to perception rejects the

Action-Based Accounts of Perception    227 linearity assumption, behaviourism rejects the instrumentality assumption since it makes action constitutive, rather than a mere enabling condition, of perception. But as Hurley points out, behaviourism casts action as output, that is, as a mere response or reaction to a stimulus. One thing that psychological behaviourism missed is that an agent’s bodily movements cause re-afferent sensory changes in her own environment and the agent must be able to distinguish endogenously caused from externally caused sensory changes in her environment. Helmholtz was the first to raise the puzzle of how to explain the stability of the visual world during eye and head movements: the brain must be able to discriminate the visual signals on the retina respectively produced by externally moving objects and by the agent’s own eye and head movements. In answer to this puzzle, it has been hypothesized that when the motor system sends a motor instruction to contract the eye-muscles, it also produces a so-called ‘efference copy’ (or ‘corollary discharge’) of the motor instruction, which is the signature of the agent’s own agency. Furthermore, according to so-called ‘internal forward models of action’, from the efference copy of the motor command as input, the motor system is able to derive a prediction of the sensory consequences of the agent’s movements.10 This sensory prediction can further be compared with the actual sensory consequences (or re-afferences) of the agent’s movements and be used either for filtering incoming sensory information or for online correction of a failed action. Thus, the theory of internal forward models of action predicts that the more a sensory change is predictable by an agent, the more its phenomenological experience will be attenuated and conversely the less predictable a sensory change, the more its phenomenological experience will be highlighted (see Wolpert et al., 1998; Jeannerod, 2006). This prediction is supported by experiments by Blakemore et al. (2000) showing that healthy human adults are unable to tickle themselves because the sensory consequences of their own self-directed movements are too predictable. On the one hand, the problem for behaviourism is that a stimulus–response framework lacks the resources for distinguishing endogenously caused from exogenously caused sensory changes. On the other hand, it is not entirely clear whether the view that the experience of self-produced (as opposed to exogenously caused) sensory changes should be attenuated is compatible with the enactivist idea that the content and phenomenal character of an agent’s perceptual experience of an object arise from her tacit knowledge of the sensory consequences of her own actions onto the object.

An externalist account of perceptual experience Arguably, one important source of evidence for the enactive account of perception derives from the experimental discovery of so-called change blindness, that is, the surprising phenomenon that observers can fail for a surprisingly long time to notice large changes to visual scenes. To borrow an ordinary example from Dretske (2007), suppose that Sarah looks for a few seconds at a group of seven visible people gathered around a table, without

10  So-called ‘inverse models of action’ take the agent’s goal or expected sensory change as input and compute the best motor command suitable for achieving this goal.

228   Pierre Jacob attending to any in particular. Suppose that while she looks away, Sam joins the group. When Sarah looks back, there are now eight visible people gathered around the table, not seven. She fails to notice the difference made by Sam’s novel presence at the table: if asked whether she sees a difference between what she saw before and after she looked away, her answer is negative. If so, then she is said to be blind to the change created by adding one member to a group of seven individuals. In scientific experiments on change blindness conducted by O’Regan (1992) and others, participants are shown displays of natural scenes and asked to detect cyclically repeated changes, such as a large object shifting, changing colour, or appearing and disappearing. Under normal circumstances a change of this type creates a transient signal in the visual system: it is detected by low-level visual mechanisms and exogenously attracts attention to the location of the change, which is, therefore, immediately perceived. However, in change blindness experiments, the transient is prevented from playing its attention-grabbing role by different methods. In some cases, a global flicker is superimposed over the whole visual field at the moment of the change. In other cases, the change coincides with an eye saccade, an eye blink, or a film cut in a film sequence. Still in other cases, extraneous transients are distributed over the picture, somewhat like mud-splashes on a car windscreen. In all cases, participants’ attention is attracted away from the location of the change (see e.g. Simons and Levin, 1997, for review).11 On the enactivist interpretation, these findings are evidence for a failure to represent the details of visual stimuli. We do not see as much as we think we do: visual perception delivers impoverished and gappy representations of distal stimuli. As Dehaene et al. (2006) have put it, experiments on change blindness show that our over-confidence in the phenomenological richness of visual experience derives from the cognitive illusion that we see more than we do. On Noë’s (2004: 50) interpretation, change blindness shows that visual awareness of details is virtual awareness (i.e. awareness of the availability of details): there is no need to build up a detailed internal model of the visible world on one’s own internal memory drive. It is enough to know where in the world to go and what action to perform to retrieve the accessible details if needed. Advocates of enactivism endorse a version of what has been variously called ‘the extended mind thesis’, ‘vehicle externalism’, or ‘active externalism’, that is, the view that an individual’s mind extends beyond the limits of her brain and body and includes features of her environment. Clark and Chalmers (1998) have argued that the vehicles of an individual’s beliefs and other propositional attitudes should not be restricted to states of her brain on the grounds that the use of tools located outside an individual’s brain enhances her ability to solve problems. Advocates of enactivism (e.g. Noë, 2004, 2009; Wilson, 2010) similarly argue that the vehicles of an individual’s perceptual experiences are not restricted to the boundaries of her brain on the grounds that visual perception consists in sensorimotor skills that enable a perceiver to retrieve the details which lie in the outside world and are not fully represented in her brain.12 Clearly, the application of the extended mind thesis to perceptual experience relies on the enactivist premiss that the change blindness experiments are evidence of a

11 

A related phenomenon is inattentional blindness (see Simons and Chabris, 1999). Clark (2008) resists extending the extended mind thesis to visual experiences. For a survey of the links between embodied cognition and extended mind approaches, cf. Rowlands (2010). 12 

Action-Based Accounts of Perception    229 representational failure to see large changes to visual scenes. But this interpretation is questionable (see Dretske, 2004, 2007; Block, 2007) for two reasons. First, what people fail to notice in change blindness experiments is not the change itself (which is carefully concealed by the experimenters), but the difference between two scenes produced by a concealed change. Failure to see the change (an event) caused failure to notice the difference. In Dretske’s example, Sarah could not see Sam join the group (the change) since she was looking away. Thus, what she failed to notice was the difference in number. Secondly, from the fact that people fail to notice a difference between two stimuli, it does not follow that they failed to see something that was either present in the first stimulus and absent from the second stimulus or vice versa. It is one thing to see a pair of stimuli; it is another to make the appropriate comparison that enables an observer to tell the difference between them (cf. Simons and Rensink, 2005). For example, from the fact that Sarah failed to notice the difference between the groups of people before and after she looked away, it does not follow that she failed to see Sam the second time she looked at the group of people. Whether enactivism offers an alternative to the action/perception dualism depends in part on whether the failure to notice a difference between two stimuli counts as a perceptual or a cognitive failure.

4  Action-based accounts of social perception Humans do not perceive only objects that they can reach, grasp, and manipulate, but also gases, clouds, flames, smoke, rivers, liquids, holes, events, and especially actions, some of which are performed by other humans—as noticed long ago by the philosopher John Austin (1962). Broadly speaking, by ‘social perception’ cognitive scientists refer to the mechanisms involved in the perception of other people. To perceive other people is to perceive their bodies and actions. To perceive the bodies of other people is inter alia to perceive their faces. The perception of invariant aspects of a face helps us to recognize the identity of an individual. The perception of variable facial expressions helps in the understanding of an individual’s emotional state. To perceive others’ actions is inter alia to perceive their voice, their posture, and their gait, which, like all instances of biological motion, is governed by biomechanical constraints. Much research in cognitive neuroscience has been devoted to the mechanisms leading to the understanding of others’ psychological states (e.g. goals, emotions, intentions, desires, or beliefs) from the perception of others’ bodies, faces, and actions (for an overview, see Allison et al., 2000; Jacob and Jeannerod, 2003). Arguably, what makes social perception different from the perception of inanimate objects is that in social perception, while we observe the actions performed by others and perceive cues of others’ goals and emotions, we also have the capacity to execute similar actions, form similar goals, and experience similar emotions. Thus the question arises to what extent the mechanisms at work in social perception, which enable humans to understand the psychological states of others from the perception of their bodily actions and facial expressions, are processes of mental simulation or covert imitation. In fact, the discovery of mirror neurons has been interpreted as vindicating the assumption that

230   Pierre Jacob understanding others’ goals, intentions, and emotions depends on mentally rehearsing their bodily actions and/or their facial expressions.

Action-mirroring In the early 1990s a group of neuroscientists led by Giacomo Rizzolatti discovered by single-cell recording a class of sensorimotor neurons located in the ventral premotor cortex of macaque monkeys that fire both when an animal executes some particular action directed to a physical target and when the animal observes the same action being performed by another agent. These cells were called mirror neurons because their activity in an observer’s brain is taken to ‘mirror’ (simulate or resonate with) their activity in the agent’s brain, without giving rise to action execution. In accordance with the motor theory of speech perception (Liberman and Mattingly, 1985) and the theory of event coding (Hommel et  al., 2001), these findings have been widely interpreted as showing that the execution and the perception of actions share common representational resources in primates’ brain. Subsequent work based on brain imaging and transcranial magnetic stimulation confirmed the existence of a mirror system in humans. For example, brain-imaging studies conducted by Buccino et al. (2004) have revealed greater activations of motor and premotor areas when human observers see without hearing a video-clip displaying a human being produce silent speech (which the observers could covertly imitate) than when they see a video-clip displaying a dog bark (which the observers could not covertly imitate). In another brain-imaging study by Calvo-Merino et al. (2005), capoeira and ballet dancers saw short films displaying dance steps by either capoeira or classical dancers. They found enhanced activations in the mirror-neuron system of dancers observing movements belonging to their respective dancing motor repertoire. Mirror neuron activity in an observer’s brain has been interpreted as generating a neural similarity between the agent and the observer who comes to entertain and share the motor representation that guides the agent’s own action. This process of action-mirroring has been taken to enable the observer to understand an agent’s goal and even an agent’s intention by means of the so-called direct-matching model of action-understanding (cf. Rizzolatti et al., 2001; Rizzolatti and Craighero, 2004). According to this three-step model of action-understanding, the perception of the agent’s act first causes the observer to automatically map (or match) the agent’s act onto her own motor repertoire, that is to mentally rehearse or covertly imitate the agent’s act. Secondly, the mental rehearsal of the agent’s act is taken to enable the observer to share the agent’s goal. Finally, by sharing the agent’s goal, the observer is expected to understand it and to ascribe it to the agent. This model has led Fogassi et al. (2005: 662) to conclude that mirror neuron activity not only ‘codes the observed motor act but also allows the observer to understand the agent’s intentions’. Following the seminal ideas of Lipps (1900), a similar tripartite model has been extended from the perception of goal-directed actions and the understanding of an agent’s goal to the perception of expressive actions and the understanding of an agent’s emotions (cf. Gallese, 2001; Goldman, 2006; Rizzolatti and Sinigaglia, 2008). On the basis of the direct-matching model of action-understanding, Rizzolatti et al. (2001) and Rizzolatti and Craighero (2004) have drawn a contrast between two ways an action

Action-Based Accounts of Perception    231 might be understood: either through a purely perceptual analysis of the agent’s bodily movements or by mapping the agent’s perceived movements onto the observer’s motor repertoire. If and when an observer (e.g. a primate) cannot map a perceived action (e.g. a bird’s flight) onto his own motor repertoire, then, according to Rizzolatti et  al. (2001), the observer’s understanding of the perceived action cannot be grounded in ‘motor resonance’. As a result, there is something about the perceived action that the observer fails to understand: the action can only be categorized on the basis of its visual, not its motor, properties.

The scope and limits of mirroring-based accounts of social perception Assuming that mirror neuron activity results in an observer’s sharing an agent’s goal or intention, the direct-matching model of action-understanding raises at least the two further questions whether sharing an agent’s goal or intention is both necessary and sufficient for understanding it and ascribing it to the agent. The reason why sharing an agent’s goal might not be sufficient is that to share an agent’s goal is to have a goal. To have a goal is to be in a psychological state with a world-to-mind direction of fit that can be fulfilled or unfulfilled by the output of the agent’s action. But to ascribe a goal to an agent is to believe that the agent has a goal, that is, to be in a psychological state with a mind-to-world direction of fit that can be true or false, whether or not the agent performs her planned action. If so, then there is a gap between sharing a goal and ascribing a goal. Mere reflection suggests that mirroring might not be necessary for understanding an agent’s goal. For example, I may understand the goal of an agent who picks up a vanilla ice cream rather than a chocolate ice cream even though I myself would rather pick up a chocolate ice cream. Furthermore, some experimental evidence suggests that mirroring might not be necessary for ascribing a goal or intention to the agent. Much of this evidence comes from experiments in developmental psychology based on the violation-ofexpectation paradigm. This paradigm, which rests on the assumption corroborated by magic tricks that individuals look longer at an unexpected than an expected event, is applicable to preverbal human infants. Typical experiments involve two phases: in the course of habituation or familiarization trials, infants are induced to form an expectation by seeing an agent repeatedly execute one and the same action. In the pair of test trials, they see a novel action that is either consistent or inconsistent with their expectation. By measuring infants’ looking times respectively in the consistent and inconsistent test trials, psychologists get evidence bearing on the nature and content of an observer’s expectations about an agent’s goal-direction action. Unlike mirroring, expectations have the mind-to-world direction of fit, appropriate for ascribing a goal to the agent. In the familiarization trials of one experiment reported by Southgate et al. (2008), an experimental group of 6- to 8-month-olds saw a video showing a human hand perform a two-step goal-directed action during which the hand removed a box lying in its path before it retrieved a target that was located behind the box. A control group of infants of the same age saw a human hand inefficiently remove a box that did not lie in its path before retrieving a target. In the test trials, all infants either saw a human hand remove a box that was lying in its path before retrieving a target or they saw a human hand perform a biologically impossible but more efficient action whereby it retrieved the target by snaking

232   Pierre Jacob around the obstacles. Only infants in the experimental group (not infants in the control group) looked reliably longer at the biologically possible but less efficient hand-action than at the more efficient but biologically impossible hand-action. While it does not seem very likely that infants could mirror the biologically impossible snaking hand-movements, this experiment further suggests that goal-ascription by preverbal human infants is guided by the efficiency of the means selected by an agent for achieving a particular goal-state, in the presence of situational constraints (cf. Gergely and Csibra, 2003). At this stage, the contribution made by mirroring processes to social perception is very much an open question. Some scientists and philosophers (e.g. Gallese and Goldman, 1998; Goldman, 2006) assume that mirroring an agent’s action plays a causal role in the process whereby an agent’s goal or emotion is being ascribed to the agent. Others have argued for an alternative interpretation according to which the causal arrow might run in the reverse direction: mirroring an agent’s action might be the effect, not the cause, of the observer’s prior understanding of the agent’s goal or intention, which itself would be based on the perception of contextual cues such as whether a graspable target is edible or not (cf. Csibra, 2007; Jacob, 2008).

5  Concluding remarks Action-based accounts have yielded many new experimental findings and intriguing philosophical ideas about perception. Much of their appeal derives from their commitment to minimalist assumptions in the study of perception. This minimalism has two sides: first, it is based on the parsimonious assumption that the mechanisms underlying action also underlie perception. Secondly, the professed goal of action-based accounts is to minimize (if not to eliminate) the contribution of internal processing and mental representations to perception. One of the central challenges (if not the central challenge) for the two-fold minimalist commitment of action-based accounts of perception is that there is more to action than the execution of bodily movements. As philosophers of action have stressed for a long time, an agent’s bodily movements count as an action only if they are appropriately caused by the agent’s intentions. For example, an agent can execute one and the same hand gesture to frighten a fly or waive bye-bye to a departing host. Only by representing her intention can one action be distinguished from the other. Furthermore, there is much empirical evidence showing that an agent’s motor system is activated in at least two situations in which he or she fails to perform any overt action. In such situations, the agent’s motor system is activated off-line (see Jeannerod, 2006). On the one hand, some areas of an observer’s motor and premotor systems (e.g. mirror neurons) are active when he or she perceives an action performed by another agent (cf. section 4). On the other hand, parts of an agent’s motor system are active in tasks of motor imagery whereby the agent plans and/or imagines an action, which, for some reason or another, he or she fails to execute. In fact, we mentally represent and even plan many actions that we never carry out. The question is whether mental representations, which action-based accounts of perception mean to kick out, might not be surreptitiously coming in by the backdoor through the scientific study of motor cognition.13 13 

I am grateful to Mohan Matthen for his extended comments.

Action-Based Accounts of Perception    233

References Aizawa, K. (2007). ‘Understanding the embodiment of perception’. The Journal of Philosophy, 54(1), 6–25. Allison, T., Puce, A, and McCarthy, G. (2000). ‘Social perception from visual cues: role of the STS region’. Trends in Cognitive Sciences, 4(7), 267–78. Austin, J. L. (1962). Sense and Sensibilia, Oxford: Oxford University Press. Barsalou, L. W. (1999). ‘Perceptual symbol systems’. Behavioral and Brain Sciences, 22, 577–660. Barsalou, L. W., Simmons, W. K., Barbey, A. K., and Wilson, C. D. (2003). ‘Grounding conceptual knowledge in modality-specific systems’. Trends in Cognitive Sciences, 7(2), 84–91. Blakemore, S.-J., Wolpert, D., and Frith, C. D. (2000). ‘Why can’t you tickle yourself?’ NeuroReport, 11, R11–R16. Block, N (2005). ‘Review of Alva Noë’s Action in Perception’. The Journal of Philosophy, 52, 259–72. Block, N. (2007). ‘Consciousness, accessibility, and the mesh between psychology and neuroscience’. Behavioral and Brain Sciences, 30, 481–548. Buccino, G., Lui, F., Canessa, N., Patteri, I., Lagravinese, G., Benuzzi, F., Porro, C. A., and Rizzolatti, G. (2004). ‘Neural circuits involved in the recognition of actions performed by non-conspecifics: an fMRI study’. Journal of Cognitive Neuroscience, 16, 114–126. Calvo-Merino, B., Glaser, D. E., Grezes, J., Passingham, R. E., and Haggard, P. (2005). ‘Action observation and acquired motor skills: an fMRI study with expert dancers’. Cerebral Cortex, 15, 1243–49. Chomsky, N. (1959). ‘A Review of B. F. Skinner’s Verbal Behavior Language’. 35, 1, 26–58. Clark, A. (2008). Supersizing the Mind, Embodiment, Action, and Cognitive Extension. Oxford: Oxford University Press. Clark, A. (2010). ‘Perception, action and experience: unravelling the golden brain’. In N. Gangopadhyay, M. Madary, and F. Spicer (eds), Perception, Action and Consciousness. Oxford: Oxford University Press. Clark, A. and Chalmers, D. (1998). ‘The extended mind’. Analysis, 58(1), 7–19. Csibra, G. (2007). ‘Action mirroring and action understanding’. In P. Haggard, Y. Rossetti, and M. Kawato (eds), Sensorimotor Foundations of Higher Cognition, Attention and Performance XXII. Oxford: Oxford University Press. Davidson, D. (1980). Essays on Actions and Events. Oxford: Oxford University Press. Dehaene, S., Changeux, J.-P., Naccache, L., Sackur, J., and Sergent, C. (2006). ‘Conscious, preconscious, and subliminal processing: A testable taxonomy’. Trends in Cognitive Sciences, 10(5), 204–11. Dretske, F. (1995). Naturalizing the Mind. Cambridge, MA: MIT Press. Dretske, F. (2007). ‘What change blindness teaches about consciousness’. Philosophical Perspectives, 1, 215–20. Fodor, J. A. (1974). ‘Special Sciences’. Synthese, 28, 97–115. Fodor, J. A. (1975). The Language of Thought. New York: Crowell. Fodor, J. A. (1983). The Modularity of Mind. Cambridge, MA: MIT Press. Fodor, J. A. (2007). ‘The revenge of the given’. In B. P. McLaughlin and J. D. Cohen (eds), Contemporary Debates in Philosophy of Mind. Oxford: Oxford University Press. Fogassi, L., Ferrari, P. F., Gesierich, B., Rozzi, S., Chersi, F., and Rizzolatti, G. (2005). ‘Parietal lobe: from action organisation to intention understanding’. Science, 308, 662–67.

234   Pierre Jacob Franz, V. H., Gegenfurtner, K. R., Bulthoff, H. H., and Fahle, M. (2000). ‘Grasping visual illusions: No evidence for a dissociation between perception and action’. Psychological Science, 11, 20–25. Gallese, V. (2001). ‘The "Shared Manifold" hypothesis, from mirror neurons to empathy’. Journal of Consciousness Studies, 8, 5–7, 33–50. Gallese, V. and Goldman, A. I. (1998). ‘Mirror neurons and the simulation theory of mindreading’. Trends in Cognitive Sciences, 2, 493–501. Ganel, T., Tanzer, M., and Goodale, M. A. (2008). ‘A double dissociation between action and perception in the context of visual illusions: Opposite effects of real and illusory size’. Psychological Science, 19, 221–225. Gergely, G. and Csibra, G. (2003). ‘Teleological reasoning about actions: the naïve theory of rational actions’. Trends in Cognitive Sciences, 7, 287–292. Gibson, J. J. (1979). The Ecological Approach to Visual Perception. Boston: Houghton-Mifflin. Goldman, A. (2006). Simulating Minds. Oxford: Oxford University Press. Goodale, M. (2011). ‘Transforming vision into action’. Vision Research, 51(13), 1567–1587. Goodale, M. A. and Milner, A. D. (1992). ‘Separate visual pathways for perception and action’. Trends in Neuroscience, 15, 20–25. Hinton, J. M. (1973). Experiences. Oxford: Clarendon Press. Hommel, B., Müsseler, J., Aschersleben, G., and Prinz, W. (2001). ‘The theory of event coding (TEC): a framework for perception and action planning’. Behavioral and Brain Sciences, 24, 849–937. Hurley, S. (1998). Consciousness in Action. Cambridge, MA: Harvard University Press. Hurley, S. (2001). ‘Perception and action: alternative views’. Synthese, 129, 3–40. Hurley, S. (2008). ‘The shared circuits model: how control, mirroring and simulation can enable imitation and mindreading’. Behavioral and Brain Sciences, 31, 1–58. Ingle, D. J. (1973). ‘Two visual systems in the frog’. Science, 181, 1053–1055. Jacob, P. (2008). ‘What do mirror neurons contribute to human social cognition?’ Mind and Language, 23(2), 190–223. Jacob, P. and Jeannerod, M. (2003). Ways of Seeing, the Scope and Limits of Visual Cognition. Oxford: Oxford University Press. Jacob, P. and de Vignemont, F. (2010). ‘Spatial coordinates and phenomenology in the two visual systems model’. In N. Gangopadhyay, M. Madary, and F. Spicer (eds), Perception, Action and Consciousness. Oxford: Oxford University Press. Jeannerod, M. (2006). Motor Cognition. Oxford: Oxford University Press. Jeannerod, M., Decety, J., and Michel, F. (1994). ‘Impairment of grasping movements following a bilateral posterior parietal lesion’. Neuropsychologia, 32, 369–380. Kim, J. (1993). Supervenience and the Mind. Cambridge: Cambridge University Press. Króliczak, G., Heard, P., Goodale, M. A., and Gregory, R. L. (2006). ‘Dissociation of perception and action unmasked by the hollow-face illusion’. Brain Research, 1080, 9–16. Liberman, A. M. and Mattingly, I. G. (1985). ‘The motor theory of speech perception revised’. Cognition, 21, 1–36. Lipps, T. (1900). ‘Asthetische Einfuhlung’. Zeitschrift fur Psychologie und Physiologie der Sinnesorgane, 22, 415–450. McDowell, J. (1982). ‘Criteria, defeasibility and knowledge’. Proceedings of the British Academy, 68. McDowell, J. (1994). Mind and the World. Cambridge, MA: Harvard University Press. Machery, E. (2006). ‘Two dogmas of neo-empiricism’. Philosophy Compass, 1(4), 398–412.

Action-Based Accounts of Perception    235 Machery, E. (2007). ‘Concept empiricism: A methodological critique’. Cognition, 104, 19–46. Marr, D. (1982). Vision. San Francisco: Freeman. Matthen, M. (2005). Seeing, Doing, and Knowing: A Philosophical Theory of Sense Perception. Oxford: Oxford University Press. Matthen, M. (2010). ‘Two visual systems and the feeling of presence’. In N. Gangopadhyay, M. Madary, and F. Spicer (eds), Perception, Action and Consciousness. Oxford: Oxford University Press. Millikan, R. G. (1996). ‘Pushmi-pullyu representations’. In J. Tomberlin (ed.), Philosophical Perspectives, vol. IX. Atascadero CA: Ridgeview Publishing. Milner, A. D. and Goodale, M. A. (1995). The Visual Brain in Action. Oxford:  Oxford University Press. Milner, A. D. and Goodale, M. A. (2010). ‘Cortical visual systems for perception and action’. In N. Gangopadhyay, M. Madary, and F. Spicer (eds), Perception, Action and Consciousness. Oxford: Oxford University Press. Milner, A. D., Perrett, D. I., Johnston, R. S., Benson, P. J., Jordan, T. R., Heeley, D. W., Bettucci, D., Mortara, F., Mutani, R., Terazzi, E. and Davidson, D. L. W. (1991). ‘Perception and action in "visual form agnosia" ’. Brain, 114, 405–428. Noë, A. (2004). Action in Perception. Cambridge, MA: MIT Press. Noë, A. (2009). Out of our Heads. New York: Hill and Wang. Noë, A. (2010). ‘Vision without representation’. In N. Gangopadhyay, M. Madary, and F. Spicer (eds), Perception, Action and Consciousness. Oxford: Oxford University Press. O’Regan, J. K. (1992). ‘Solving the "real" mysteries of visual perception: the world as an outside memory’. Canadian Journal of Psychology, 46(3), 461–488. O’Regan, J. K. (2010). Why Red Doesn’t Sound Like a Bell, Understanding the Feel of Consciousness. Oxford: Oxford University Press. O’Regan, J. K. and Noë, A. (2001). ‘A sensorimotor account of vision and visual consciousness’. Behavioral and Brain Sciences, 24, 939–1031. Pisella, L., Gréa, H., Tilikete, C. A. Vighetto, A. M. Desmurget, G., Rode, G., Boisson, D., and Rossetti, Y. (2000). ‘An "automatic pilot" for the hand in human posterior parietal cortex: toward reinterpreting optic ataxia’. Nature Neuroscience, 3(7), 729–736. Pisella, L., Binkofski, F., Lasek, K., Toni, I., and Rossetti, Y. (2006). ‘No double-dissociation between optic ataxia and visual agnosia: multiple sub-streams for multiple visuo-manual integrations’. Neuropsychologia, 44, 2734–2748. Prinz, J. (2002). Furnishing the Mind:  Concepts and their Perceptual Basis. Cambridge, MA: MIT Press. Prinz, J. (2005). ‘The return of concept empiricism’. In H. Cohen and C. Leferbvre (eds), Categorization and Cognitive Science. Oxford: Elsevier. Proffitt, D. R., Bhalla, M., Gossweiler, R., and Midgett, J. (1995). ‘Perceiving geographical slant’. Psychonomic Bulletin & Review, 2, 409–428. Putnam, H. (1967). ‘The nature of mental states’. In H. Putnam (1975), Mind, Language and Reality, Philosophical Papers, vol. 2. Cambridge: Cambridge University Press. Rizzolatti, G. and Craighero, L. (2004). ‘The mirror-neuron system’. Annual Review of Neuroscience, 27, 169–192. Rizzolatti, G., Fogassi, L., and Gallese, V. (2001). ‘Neurophysiological mechanisms underlying the understanding and imitation of action’. Nature Review of Neuroscience, 2, 661–670. Rizzolatti, G. and Sinigaglia, C. (2008). Mirrors in the Brain: How Our Minds Share Actions and Emotions. Oxford: Oxford University Press.

236   Pierre Jacob Rossetti, Y. (1998). ‘Implicit short-Lived motor representations of space in brain damaged and healthy subjects’. Consciousness and Cognition, 7, 520–558. Rossetti, Y. and Pisella, L. (2002). ‘Several "vision for action" systems: a guide to dissociating and integrating dorsal and ventral functions (tutorial)’. In W. Prinz and B. Hommel (eds), Common Mechanisms in Perception and Action. Oxford: Oxford University Press. Rossetti, Y., Ota, H., Blangero, A., Vighetto, and Pisella, L. (2010). ‘Why does the perceptionaction functional dichotomy not match the ventral-dorsal streams anatomical segregation: optic ataxia and the function of the dorsal stream’. In N. Gangopadhyay, M. Madary, and F. Spicer (eds), Perception, Action and Consciousness. Oxford: Oxford University Press. Rowlands, M. (2010). The New Science of the Mind, from Extended Mind to Embodied Phenomenology. Cambridge, MA: MIT Press. Schneider, G. E. (1969). ‘Two visual systems’. Science, 163, 895–902. Searle, J. (1983). Intentionality: An Essay in the Philosophy of Mind. Cambridge: Cambridge University Press. Shapiro, L. (2004). The Mind Incarnate. Cambridge, MA: MIT Press. Shapiro, L. (2007). ‘The embodied cognition research programme’. Philosophy Compass, 2(2), 338–346. Simons, D. J. and Chabris, C. F. (1999). ‘Gorillas in our midst: sustained inattentional blindness for dynamic events’. Perception, 28(9), 1059–1074. Simons, D. J. and Levin, D. T. (1997). ‘Change blindness’. Trends in Cognitive Sciences, 1, 261–267. Simons, D. J. and Rensink, R. A. (2005). ‘Change blindness: past, present, and future’. Trends in Cognitive Sciences, 9(1), 16–20. Southgate, V., Johnson, M. H., and Csibra, G. (2008). ‘Infants attribute goals to even biologically impossible actions’. Cognition, 107, 1059–1069. Tye, M. (1995). Ten Problems of Consciousness. Cambridge, MA: MIT Press. Ungerleider, L. and Mishkin, M. (1982). ‘Two cortical visual systems’. In D. J. Ingle, M. A. Goodale, and R. J. W. Mansfield (eds), Analysis of Visual Behavior (pp. 549–586). Cambridge, MA: MIT Press. Weiskrantz, L., Warrington, E. R., Sanders, M. D., and Marshall, J. (1974). ‘Visual capacity in the hemianopic field following a restricted occipital ablation’. Brain, 97, 709–728. Wilson, R. (2010). ‘Extended vision’. In N. Gangopadhyay, M. Madary, and F. Spicer (eds), Perception, Action and Consciousness. Oxford: Oxford University Press. Wolpert, D. M., Miall, R. C., and Kawato, M. (1998). ‘Internal models in the cerebellum’. Trends in Cognitive Sciences, 2(9), 338–347.

Chapter 13

Perceptua l R eports Berit Brogaard

1 Perceptual verbs and their etymology Perceptual reports are utterances of sentences that contain a perceptual verb. Perceptual verbs include, among others, ‘look’, ‘sound’, ‘feel’, ‘taste’, ‘smell’, ‘see’, ‘hear’, and ‘perceive’. Consider:  (1)

(a) My chair looks red but it’s really white. (b) His voice sounded deep and earnest. (c) Vegemite tastes like spreadable beer. (d) Last night my house smelled like a Mexican restaurant. (e) The entrance is so white that it feels as if you’re walking into a huge iPod. (f) This fabric feels like velvet. (g) John saw Mary cry. (h) The witness heard a noise and found the victim on the ground.

Perceptual reports such as these purport to describe how objects in the world are perceived by subjects. It is natural to suppose that at least in many cases, these reports reflect aspects of the phenomenal character and representational content of a subject’s perceptual experiences. Whether perceptual reports actually reflect these things is a substantial question and one with which I will be partially concerned in this entry. Perceptual reports containing the verbs ‘look’, ‘seem’, ‘appear’, ‘taste’, ‘smell’, and ‘sound’ are likely to occur either with an implicit or explicit relativization to a perceiver. Syntactically, this can occur with a ‘to-X’ clause or other local constituents that make the relevant perceiver and perceiving conditions explicit, as in ‘Vegemite tastes like spreadable beer to me [in normal taste conditions]’ or ‘Vegemite tastes like spreadable beer to Americans the first time they taste it’. Perceptual verbs have an interesting etymology that has some bearing on their semantics in modern English. Starting with visual ‘appear’ verbs, ‘seem’ originates from the Old English ‘beseon’, which is a contraction of ‘be’ and ‘seon’ (literally: ‘to see’) (The Oxford

238   Berit Brogaard Dictionary of English Etymology and A Guide to Old English, p. 325). ‘Beseon’ was used in most of the grammatical constructions for which ‘look’ is used in modern English. Consider the following examples:  ‘Hwa don Willelm of Normandige beseon gelic’? What-does-William-of-Normandy-be-see-like? (What does William the Conqueror look like?) ‘Angelcynn beseon micel lytlian nu.’ England-be-see-very-different-now. (England looks so different now.)

‘Beseon’ functions as a subject-raising verb. ‘England looks so different now’ is a raising construction because ‘England’ is the surface-grammatical subject of ‘look’ but the semantic subject of ‘to be different now’. Verbs that take this position in a raising construction are called ‘(subject)-raising verbs’. ‘England be see so different now’ is the raised form of ‘It is seen that England is very different now’. ‘Look’ comes from the Old English verb ‘locian’, which means ‘to see, to gaze’. ‘Locian’, in turn, comes from from the West Germanic ‘lokjan’. ‘Locian’ in the sense of ‘having a certain appearance’ entered Old English around 1400, at which point it began to occur in the positions in which ‘beseon’ had previously occurred. As ‘look’ originated from ‘locian’ and ‘locian’ occurs in the same positions as the older ‘beseon’, it is very plausible to think that ‘look’ and ‘seem’ function in the same way. ‘Appear’ entered Old English from Old French ‘aparoir’ around the same time and acquired a meaning in Old English that is similar to ‘locian’ in the ‘appear’ sense. It is around the same time or later that ‘sound’, ‘taste’, ‘smell’, and ‘feel’ acquire the ‘appear’ meaning. ‘Taste’ entered Old English around 1300. It stems from the Old French ‘taster’ (‘to taste’), which originally meant ‘to touch, to handle’. Around 1550 it replaced the Old English ‘smack’. ‘Sound’ stems from the Latin ‘sonare’ and entered Old English in late 1300. It began to be used in the ‘appear’ sense some time in 1400. ‘Feel’ stems from the Old English ‘felan’ and slowly acquired its multiple senses between 1300 and 1829. ‘Smell’ was not found in Old English and is of unknown origin. The etymology of ‘appear’ words strongly suggests that ‘seem’, ‘appear’, and ‘look’ function in the same way. As ‘taste’, ‘sound’, and ‘feel’ took on the ‘appear’ sense by replacing the visual verbs in equivalent sentences, it is plausible that ‘appear’ words have a unified semantic theory. I return to this theory below.

2 Epistemic, comparative, and phenomenal uses of appearance verbs The first philosopher to offer a systematic account of visual ‘appear’ words was Roderick Chisholm (1957: ch. 4). Chisholm identified three uses of ‘look’: The epistemic, the

Perceptual Reports   239 comparative, and the non-comparative (non-epistemic) use. Frank Jackson (1977) calls the latter use the phenomenal use of ‘look’. Chisholm’s distinction plays a crucial role in Frank Jackson’s argument for the sense-datum theory of perception. According to Jackson, the hypothesis that there are accurate phenomenal ‘looks’ reports partially indicates that we do not perceive the world directly but perceive it via a veil of appearances. I will argue that this conclusion lends support, not to the sense-datum theory, but to the hypothesis that visual perception has representational content.

Epistemic uses If I hear on the radio that Bank of America has put the financial crisis behind it, I may say ‘It looks like I won’t need to find a new bank’. This use of ‘look’ is different from the ‘look’ that occurs in ‘It looks like the road is wet (but it’s not)’, said on the basis of how the road looks. In the former case, if I am presented with a defeater, it will no longer look to me as if I won’t need to find a new bank. For example, if the radio host later announces that Bank of America is going bankrupt, then it will no longer look to me as if I won’t need to find a new bank. In the latter case, a defeater is not going to change how things look. If I am told that the city has painted the roads to make them look wet as part of their drive-safe campaign, it will still look to me as if the roads are wet. Chisholm called the use of ‘look’ that is subject to defeat ‘the epistemic use’. Epistemic uses are less closely tied to perception than the other uses are. When ‘look’ is used epistemically, the description sentence containing it purports to describe a cognitive state concerning what is subjectively probable conditional on (total, total inner, total relevant, total relevant presented so far . . .) evidence. Furthermore, when ‘look’ is used epistemically, the cognitive state ceases to exist in the presence of a defeater (if the agent is rational). I will say of epistemic reports that they are evidence bearing for the speaker. When epistemic ‘looks’ reports are not evidence bearing, they are relativized to a third party. Suppose, for example, after stating the usual antidote and Fink problems, I say ‘It looks, prima facie, like one cannot analyze dispositions in terms of conditionals’. I then give a new more elaborate theory in which the proposal is in fact to analyze dispositions in terms of conditionals. Here, the ‘looks’ report is not evidence bearing for the speaker because it makes implicit reference to the evidence of a person who is not acquainted with the new theory. Like ‘look’, ‘seem’, ‘appear’, and ‘sound’ all have epistemic uses. Consider:  (2) (a) It seems that some companies do this on purpose. (b) This modelling school appears to be a scam. (c) That sounds like a personal problem for me. When so used ‘seem’, ‘appear’, and ‘sound’ reports also purport to describe a cognitive state concerning what is subjectively probable conditional on (total, total inner, total relevant, total relevant presented so far . . .) evidence. For example, in the epistemic sense, ‘This modelling school appears to be a scam’ can be interpreted as meaning ‘This modelling school probably is a scam’.

240   Berit Brogaard ‘Taste’ and ‘smell’ do not have any widespread epistemic uses. They are normally used epistemically only in idiomatic phrases and non-literal speech, as in:  (3) (a) [That attitude] smells like teen spirit (Nirvana). (b) This tastes like victory. 3(b) can but need not have an epistemic reading. In some contexts it signals an emotional response to something that might not actually be victory. For instance, you could say it after you came a close second to Usain Bolt.

Comparative uses Comparative perceptual reports tell us that two experiences have certain properties in common but they need not tell us what the properties are. For example, if I notice that Rose is visually similar to her sister, I can say ‘Rose looks like her sister’. ‘Look’, 'seem’, ‘appear’, ‘sound’, ‘taste’, and ‘smell’ all have non-epistemic comparative uses: (4) (a) The cliff looked like a dried-out body. (b) The cliff seemed like a dried-out body. (c) They appeared like the perfect couple in public. (d) The 95 Tiger 885 I just purchased sounds like a diesel truck. (e) The cookies smell like marzipan. (f) Homemade wine tastes like yeast. At least some comparative reports are epistemic reports. For example, suppose I say about war 1: ‘this looks like war 2’. That’s comparative but plausibly epistemic. As comparative reports have a distinctly comparative structure, it is natural to think that they are structurally related to more familiar comparative sentences. Consider:  (5) (a) John is taller than every girl. (b) John is taller than every girl is. (c) Ellen is as rich as her father. (d) It is warmer today than it might be tomorrow. (e) George is richer than his father was and his son will be. (f) John dances like Tom. (g) Mary eats like a bird. As Richard Larson (1988) argues, 5(a) can be dealt with by positing that (i) the quantified noun phrase (e.g. ‘every girl’ or ‘one of the girls’) moves to a wide-scope position, and (ii) the comparative expression ‘taller than’ combines with two type e expressions (i.e. variables or referring terms). On this view, 5(a) is of the form ‘[Every girl, x] tallerthan (John, x)’. However, ‘than’-clauses are syntactically akin to relative clauses such as ‘that every girl likes’ as it occurs in ‘John is a guy that every girl likes’. Quantified noun phrases cannot scope out of relative clauses. As ‘than’-clauses are syntactically akin to relative clauses, it is

Perceptual Reports   241 extremely implausible to think that quantified noun phrases (e.g. ‘every girl’) can move to a wide-scope position. Irene Heim (2006) argues that even if quantified noun phrases could scope out of ‘than’clauses, modal expressions, adverbs of quantification (e.g. ‘Mary typically eats breakfast’), and floating quantifiers (e.g. ‘the girls all went outside’) cannot possibly do that. So, Larson’s suggestion does not carry over to 5(b) to 5(c). Heim offers a new account of comparatives according to which comparatives ascribe relations between what she calls ‘degrees’ (i.e. abstract entities like heights, weights, ways, etc.). To account for quantifier scopes, Heim suggests that there are semantically vacuous ‘wh’-items in the sentence structure. 5(b) can be read as: ‘John is taller than every girl is wh’. To a first approximation, ‘every girl is wh’ is to be read as: ‘every girl x: x is this tall’. This item scopes out of the comparative clause, and the ‘wh’-item raises to a wide-scope position, yielding: [wh1[every girl is t1]]2 [John is taller than t2]

Likewise, 5(f), which is superficially similar to ‘looks’ reports, can be read as: ‘John dances like Tom does wh’, where ‘Tom does wh’ is to be read, roughly, as: ‘Tom dances this way’. This item scopes out of the comparative clause, yielding: [wh1[Tom dances t1]]2 [John dances t2]

We can assign the following truth-condition to 5(f): for some way w such that w is a way that Tom dances, John dances that way too. As comparative ‘looks’-reports are superficially similar to the comparatives in 5(f) and 5(g), it is very plausible that they have the same underlying structure (Brogaard 2010, 2012b, 2014). On this hypothesis, ‘X looks like Y’ contains the implicit wh-clause ‘wh1[Y looks t1]’. This item scopes out of the comparative clause, yielding:  [wh1[Y looks t1]]2 [X looks t2]

For example, ‘Rose looks like her sister’ is to be read as containing the implicit clause wh-clause:  ‘wh1[Rose’s sister looks t1]’. This item scopes out of the comparative clause, yielding:  [wh1[Rose’s sister looks t1]]2[Rose looks t2]

The comparative analysis of comparative ‘looks’ reports offers the full answer to the question of how to analyze comparative ‘looks’ reports linguistically, but it does not address the question of how to assign truth-conditions (see Byrne 2009; Martin 2010; Travis 2004, 2013). The reason is that the analysis makes unreduced appeal to a notion of ‘look x’. This notion needs further analysis in non-comparative terms.

Perceptual uses Non-comparative (non-epistemic) ‘looks’ reports purport to describe the properties of experience directly (rather than comparatively) (Glüer 2009; Byrne 2009; Brogaard 2012b).

242   Berit Brogaard For example, if I look at my red chair and say ‘the chair looks red’, what I said is plausibly a non-comparative report. ‘Look’, ‘seem’, ‘appear’, ‘sound’, ‘taste’, and ‘smell’ all have nonepistemic non-comparative uses. Consider: (6) (a) Her skin looked as smooth as silk. (b) Her skin seemed smooth as silk. (c) Her skin appeared smooth as silk. (d) The music sounded bracingly fresh and quite moving. (e) The strawberries taste sour. (f) The flowers smell sweet. Non-epistemic ‘looks’ reports fail to be evidence bearing. Even if I am told that the lines in the Müller-Lyer optical illusion have the same length, it will still look to me as if they have different lengths. Some apparently non-comparative ‘looks’ reports are implicitly comparative. For example, ‘John looks drunk’ is plausibly a contraction of ‘John looks like someone who is drunk’. Likewise, ‘John’s voice sounded earnest’ plausibly is a contraction of ‘John’s voice sounded like the voice of someone who is earnest’. There are, however, three reasons to think not all non-comparative reports are implicitly comparative. First, comparative ‘looks’ reports plausibly just are existentially quantified noncomparative ‘looks’ reports at the level of logical form. Second, it is evident that we cannot successfully reduce all non-comparative ‘looks’ reports to comparative reports. ‘That chair looks purely qualitatively red’ and ‘That chair looks the way a purely qualitatively red object would look’ plausibly have the same truth-conditions. Hence, the comparative report presupposes a non-comparative use of ‘looks’. Third, as Chisholm (1957:  51)  argues, if ‘look red’ is given a comparative reading, ‘red things look red’ is an analytic truth. It says ‘things that are red look the way things that are red look’, which is trivially true. If, on the other hand, ‘look red’ is given a non-comparative reading, then ‘red things look red’ is a synthetic truth. Even before she started studying neuroscience and physics, Frank Jackson’s Mary knew that red things look the way red things look. But she didn’t know that red things looked non-comparatively red.

3 Adverbial and raising accounts of ‘looks’ It remains to give a semantic analysis of the non-comparative ‘looks x’. The most important theoretical options for analyzing the semantics of non-comparative ‘looks x’ are treating ‘appear’ words as adverbs or treating them as subject-raising verbs. Let us consider the adverbial treatment first. It is consistent with a comparative linguistic analysis of comparative ‘looks’ reports that the truth-condition for ‘X dances like Y’ involves ‘X dances P-ly’. For example, ‘Amy dances like Eli’ is true if Amy and Eli both dance wildly with their eyes closed. When an

Perceptual Reports   243 adverb occurs in a final position it describes the manner of the activity picked out by the verb. Consider:  (7) (a) John spilled the beans clumsily. (b) John dances clumsily. 7(a) means ‘John spilled the beans in a clumsy manner’, and 7(b) means ‘John dances in a clumsy manner’. Adverbs that describe the manner of the activity picked out by the verb are also known as ‘manner adverbials’. 7(a) can be assigned the following truth-conditions using Davidsonian event semantics:  ‘e[spill(e, John, beans) & clumsily(e)]’. In English: There is an event e such that e is a spilling event that has John as an agent and the beans as a patient, and e was done clumsily. 7(b) can be assigned the following truth-condition: ‘e[dance(e, John) & clumsily(e)]’. In English: There is an event e such that e is a dance event that has John as an agent, and e was done clumsily. It is theoretically possible that the truth-condition for ‘X looks like Y’ likewise involves ‘X looks x-ly’, where ‘x-ly’ is a manner adverbial. ‘The tomato looks red to me’ would be assigned the truth-condition:  ‘e[look(e, tomato, me) & redly(e)]’. Or:  there is a looking event with the tomato and I as participants (agent and patient respectively), and the event takes place in a redly manner. This would be consistent with an adverbial theory of perception, according to which perceiving is an object acting upon a perceiver in a certain manner. I believe there are several reasons to resist an adverbial interpretation of ‘look x’. I offer some reasons below. In what follows I will briefly outline what I believe is the correct interpretation of ‘look’. A lot of what I am saying here carries over to ‘feel’, ‘taste’, ‘smell’, and ‘sound’. On my preferred account, when ‘look’ takes an adjective it functions semantically in the same way as the (subject)-raising verb (e.g. ‘seem’, ‘strike (me)’, ‘turn out’, ‘prove’) in the following sentences:  (8) (a) Tom was found missing. (b) Susan was proven guilty. (c) A laptop was reported stolen. (d) Patrick was assumed dead [after disappearing in South America in the 1970s]. (e) Some 67% of the students’ writing was deemed outstanding or good. The sentences in (8) are pleonastic paraphrases of ‘Tom was found to be missing’, ‘Susan was proven to be guilty’, ‘A laptop was reported to be stolen’, ‘Patrick was assumed to be dead’, and ‘Some 67% of the students’ writing was deemed to be outstanding or good’. They are thus structurally similar to sentences with raising verbs and infinitive predicates such as ‘John is expected to arrive on time’, ‘Mary is believed to have stolen two library books’, ‘The summer promises to be great’, ‘Tom was seen eating a sandwich’. One reason for thinking that ‘looks’ should be given the same semantics is that the etymology of ‘appear’ words strongly suggests that ‘look’ functions in the same way as ‘seem’. As ‘seem’ uncontroversially functions as a raising verb, so does ‘look’. The weakness of this argument, that etymology indicates a unified semantics for ‘appear’ words, is that even if there is strong evidence that the Old English ‘locian’ and

244   Berit Brogaard ‘besean’ function in the same way, ‘look’ and ‘seem’ could later have come to function differently. But it is more likely that changes and idiosyncrasies over the years explain why ‘look’ sometimes behaves differently from ‘seem’ and ‘appear’. Transformational grammar has taught us that sentences with raising verbs do not have the surface form that they appear to have. On the face of it, ‘John looked to be happy’ seems to have the same surface grammar as ‘John wanted to be happy’. However, this is not so. One of the big advances of transformational grammar was that it offered a way to distinguish between the different underlying forms of sentences like ‘John wants to be happy’ and ‘John looks to be happy’ (Partee 1975). The ‘want’ sentence has the underlying form ‘John wants [John to be happy]' and the ‘look’ sentence has the underlying form ‘[e looks[John to be happy]]’. ‘John looks to be happy’ is generated by applying the transformation rule Subject-to-Subject-Raising. When this rule is applied, ‘John’ is raised to become the surface-grammatical subject of ‘looks’. The subjects of raising verbs like ‘seem’, ‘proven’, and ‘look’ thus have no semantic relation to these experiential or epistemic verbs. Rather, the subjects are associated with the infinitive predicate or the verb of the embedded clause. For example, in ‘The apple looked to be red’ the subject ‘the apple’ is associated with ‘to be red’, and in ‘John seemed to prefer red wine’ the subject ‘John’ is associated with the verb ‘prefers’. In their unraised logical form, raising verbs often take an expletive, or dummy, subject, as in ‘It was proven that she was guilty’, ‘It seemed as if she was turning red’, or ‘It looks like she is done’ (Postal 1974; Chomsky 1981, 1986). Most verbs that function as raising verbs can also function as transitive verbs, as in ‘John looked (shy, shyly) at Mary’, ‘Tom (eagerly) expected the car crash’, and ‘Alice (enthusiastically) tasted the soup’. When they function as transitive verbs, they describe acts or actions of the referent of the semantic subject. When they function as intransitive raising verbs, they describe a passive experiential or epistemic state of an implicitly or explicitly mentioned perceiver. For example, ‘Lisa seemed angry to Paul’ describes a passive experiential or epistemic state of Paul, and ‘The tomato looks red to me’ describes a passive experiential or epistemic state of the speaker. If ‘appear’ words are raising verbs, then there are at least two reasons to think the adjectival treatment of ‘look’ is wrong. First, raising verbs are always followed by adjectives or infinitive clauses rather than adverbs. The ‘to be’ of infinitive clauses always takes an adjectival complement, not an adverbial one, as is apparent in ‘John was proven guilty’ and ‘Susan turned out to be a liar’. Hence, while complements of raising verbs can be modified by adverbs, as in ‘extremely beautiful’, they cannot themselves be adverbs or ‘to be’ plus adverbial clauses. Second, unlike adverbial modifiers, the adjectival complements of raising verbs and the corresponding infinitive clauses have the same meaning. The reason that they have the same meaning has to do with the underlying form of the sentences in which they occur. Raising verbs function as sentential operators. ‘John seemed worried’ has the underlying form ‘Seemed(John is worried)’. This underlying form is then transformed into the surface form ‘John seemed to be worried’. The latter can then undergo deletion of the predicate infinitive to become ‘John seemed worried’. The same goes for ‘looks’. ‘X looks red’ has the underlying form ‘Looks(X is red)’. This underlying form is transformed into the surface form ‘X looks to be red’, which through infinitive deletion becomes ‘X looks red’. The infinitive construction ‘X looks to be red’ is thus a pleonastic paraphrase of ‘X looks red’.

Perceptual Reports   245 In the case of ‘John dances clumsily’, on the other hand, there is no equivalent infinitive construction. For example, ‘John dances clumsily’ and ‘John dances to be clumsy’ have different truth-conditions.

4 Consequences for theories of perception How we use ‘appear’ words has some bearing on issues in the theories of perception. As Brogaard (2010) argues, from the hypothesis that ‘look’ is a raising verb, we can infer that some ‘looks’ reports reflect phenomenal properties of perception. From that we can infer that they reflect representational contents of perception and hence that perception has representational content. ‘S reflects property P’ and ‘S reflects content p’ can be defined as follows (Brogaard 2010):  Phenomenal Property Reflection A report that describes experience e reflects a phenomenal property P iff [necessarily, the report is true iff P is a phenomenal property of e] Content Reflection A report that describes experience e reflects a content p iff [necessarily, the report is true iff p is a content of e].

Phenomenal Property Reflection and Content Reflection are meant to be restricted to reports that are tokens of sentences that can have true tokens when uttered by us. As comparative ‘looks’ reports, if truly comparative, reduce to non-comparative reports, the question to be answered is that of whether some non-comparative ‘looks’ reports reflect phenomenal properties of perceptual experience. There are two components to this question: one is whether ‘looks’-reports mirror properties represented in perception. Another is whether these properties at least sometimes reflect distinctly phenomenal representation. As for the first question, we have already seen that ‘looks’, used as an intransitive verb, functions as a sentential operator at the level of logical form. ‘X looks red’ has the underlying structure ‘Looks(X is red)’. In the transformation of the underlying structure, ‘X’ raises to become a constituent of the higher clause ‘X looks to be red’. This then undergoes infinitive deletion to yield ‘X looks red’. ‘X looks red’ thus has the same underlying structure as ‘A laptop was reported stolen’ and ‘Patrick was assumed dead’. In all of these cases the underlying structure contains a subject-predicate subordinate clause with a predicate that expresses a property attributed to the referent of the semantic subject term. For example, ‘a laptop was reported stolen’ says that it was reported that a laptop was stolen. The subordinate clause thus attributes being stolen to some laptop. Likewise, the subordinate clause in ‘X looks red’ attributes being red to X. The subordinate clauses of ‘looks’ reports thus attribute properties expressed by the predicate term to the referent of the subject term of the subordinate clause. The sentential operator indicates how the properties got attributed. Being stolen was attributed to a laptop in an act of reporting. Being red is attributed to X in a perceptual act. It follows that the subordinate clauses of non-epistemic (or ‘perceptual) ‘looks’ reports mirror properties represented in perception.

246   Berit Brogaard The second question to be answered was whether at least some ‘looks’ reports reflect distinctly phenomenal properties. Here is an argument that they do. Let ‘X looks [ADJ] to O at t’ be a non-comparative ‘looks’ reports. Let the domain consist of properties that correspond to ‘[ADJ]’. Now, for some property P, if ‘X looks [ADJ] to O at t’ does not reflect P, then either the report is necessarily false, or it is not necessary that it is true iff P is a phenomenal property of O’s experience. As for the first horn of the dilemma: ‘X looks [ADJ] to O at t’ cannot plausibly be necessarily false for all values of ‘X’, ‘[ADJ]’, ‘O’, and ‘t’. If I look at a ripe tomato in good lighting conditions and say ‘That looks red to me now’, that obviously is true. As for the second horn of the dilemma: According to Jackson, ‘X looks red’ is true when X has an experience that is red. However, this is not the best way to interpret ‘X looks red’. As we have already established, ‘look’ expresses a perceiver’s experiential attitude relative to a represented property. When used non-epistemically, ‘look’ is a marker of experiential modality as opposed to epistemic modality. A non-epistemic ‘looks’ report states that the world as experienced is the way indicated by the subordinate clause. By making a ‘looks’ reports one thus seeks to eliminate the set of possible situations in which the subordinate clause is false. When used epistemically, ‘look’ is a marker of epistemic modality. An epistemic ‘looks’ report states that the world likely is the way indicated by the subordinate clause. Both experiential modality and epistemic modality relativize truth to individuals (perceivers or believers) by relating their current experiential state or their current state of belief to the content of their utterances. One difference between raising verbs (both epistemic and nonepistemic) and epistemic modals, such as ‘may’, ‘might’, ‘should’, and ‘must’, is that raising verbs often indicate the source of the perceiver’s experiential attitude or the believer’s belief. ‘The tomato looked red’ indicates that the perceiver was looking at the tomato. ‘The table felt hard’ indicates that the perceiver was feeling the table. ‘Tom was seen eating a sandwich’ indicates that the perceiver saw Tom. ‘John is expected to arrive on time’ indicates that a thinker was expecting something. Now, we need to show that:  For some ‘looks’ reports of the form ‘X looks [ADJ] to O’, the report is true iff a property corresponding to the adjectival phrase of the subordinate clause is a phenomenal property of O’s experience.

The right-to-left direction is obvious. Where P corresponds to ‘[ADJ]’, it is necessarily the case that if P is a phenomenal property of O’s experience, then ‘X looks [ADJ] to O’ is true. The left-to-right direction is less obvious. To see that it holds, consider a special case of blindsight. Blindsight is a kind of residual vision that some people with lesions to the primary visual cortex have. Blindsighters can make above-chance predictions about the attributes of visual stimuli presented to them in their blind field, without any distinctly visual awareness. In ordinary cases of blindsight, the stimulus does not seem or look any way to the blindsighter. Blindsighters feel that they are simply guessing. But consider the case of Ned Block’s super-blindsighter. A super-blindsighter has acquired the ability to guess correctly when to make a guess about a stimulus in her blind field. If someone were to ask a super-blindsighter ‘What colour does the stimulus in your blind field seem to be?’

Perceptual Reports   247 or ‘How does the stimulus look to you?’, she may just reply with ‘It seems red to me’ or ‘It looks red to me’. Consider another case, that of achromatopsia. When a person with achromatopsia looks at a red object, he has a phenomenally black experience of the object. Hence, he cannot tell on the basis of his perceptual experience whether the object is red or black. But suppose he is given a device that presents a black dot on a screen when it detects that an object is red. By means of this device an achromatopsic can discriminate between red and black objects. If shown a red object and asked ‘What colour does the stimulus seem to be?’ Or ‘How does the stimulus look to you?’, he may just reply with ‘It seems to be red to me’ or ‘It looks red to me’. These cases seem exceedingly plausible. However, as argued by Brogaard (2010), the occurrence of ‘look’ in these cases is epistemic. When a super-blindsighter detects the colour of a visual stimulus presented to her in her blind field, she has no distinctly visual awareness of the colour of the stimulus. So, when she reports on the colour of a stimulus presented to her in her blind field, she cannot make use of any visual phenomenology associated with the colour information. Rather, she must infer from her inclination to guess that the stimulus is red, that it is red. Were she to be presented with a defeater, she would no longer have the inclination to state that the stimulus looks read. So, when she says that the stimulus looks red, her report is evidence bearing and hence epistemic. Likewise, when a person with achromatopsia detects the colour of a visual stimulus by looking at a computer screen, he has no distinctly visual awareness of the colour of the stimulus. So, when he reports on the colour of the stimulus, he cannot make use of any visual phenomenology directly associated with the colour information. Rather, he must infer from the black dot on the screen that the stimulus must be red. Were he to be presented with a defeater, he would no longer have the inclination to state that the object looks red. So, when he says that the stimulus looks red, his report is evidence-bearing and hence epistemic. Generalizing:  (1) All and only epistemic ‘looks’ reports reflecting the speaker’s internal evidence state are evidence bearing. (2) When we use a ‘looks’ report to report on a visual stimulus without basing the report in any way on phenomenal properties of an experience of the stimulus, the report is always evidence bearing. (3) So, when we use a ‘looks’ report to report on a visual stimulus without basing the report in any way on phenomenal properties of an experience of the stimulus, the report is epistemic. A ‘looks’ report that reports on a visual stimulus can be true only if it is based on the visual phenomenology of an experience of the stimulus or is based on other information available about the stimulus. It follows that when we use a non-epistemic ‘looks’ report to report on a visual stimulus, the report is true just in case it is based on the phenomenal properties of an experience of the stimulus. So, for non-epistemic reports of the form ‘X looks [ADJ] to O’, it is necessary that if the report is true, then there is an experience with property P, where P corresponds to ‘[ADJ]’. Hence, for some ‘looks’ reports of the form ‘X looks [ADJ] to O’, the report is true iff a property corresponding to the adjectival phrase of the

248   Berit Brogaard subordinate clause is a phenomenal property of O’s experience. Hence, at least some ‘looks’ reports reflect distinctly phenomenal properties. Brogaard (2010) further argues that if ‘looks’ reports reflect phenomenal properties, they do not reflect non-phenomenal properties. Suppose I have a rough and imprecise experience of John, because I am not wearing my glasses. My experience then has the representational phenomenal property of representing a certain content in a rough and imprecise way (see Chalmers 2004). It also has the non-representational property of being rough and imprecise. But I cannot accurately report the roughness and imprecision of my experience using:  (9) John looks rough and imprecise. ‘My experience represents John in a rough and imprecise way’ and ‘My experience is rough and imprecise’ do not entail ‘my experience represents John as being rough and imprecise’. My rough and imprecise experience does not give rise to an appearance that attributes roughness and imprecision to John. Imprecision is similar to consciousness in this respect. A conscious visual experience of an apple does not represent the apple as conscious. The correct thing to say here is that not all of the phenomenal properties of perceptual experience can be accurately reported using a phenomenal ‘looks’ report. The argument from the claim that some ‘looks’ reports reflect representational phenomenal properties of perception to the claim that perception has content is straightforward (Brogaard 2010). The argument rests on the following principle:  Looks-Representation Bridge Principle (LRB) If X phenomenally looks to be P to O at t, then O’s experience at t has the representational phenomenal property of representing something as P.

LRB links phenomenal looks for a person at a time to a phenomenal property of that person’s experience at that time. Here is an argument for LRB. Suppose LRB is false. Then X phenomenally looks to be P to O at t, but O’s experience at t does not have the property of representing something as P. Then either P corresponds to a non-representational phenomenal property of O’s experience at t, or it does not correspond to any phenomenal property of O’s experience at t. P cannot correspond to a non-representational phenomenal property of O’s experience at t, for, as I argued earlier, things cannot phenomenally look to have a property that corresponds to a purely non-representational phenomenal property. So, P does not correspond to any phenomenal property of O’s experience at t. So, P does not contribute to what it is like for O to have the experience he has at t. But it is conceptually impossible for X to look to be P to O at t, despite the fact that P does not contribute in any way to what it is like for O to have the experience she has at t. So, LRB is true. By the LRB principle and the hypothesis that things cannot phenomenally look to have non-representational phenomenal properties, it follows that the properties which things can phenomenally look to have to observer O correspond to representational phenomenal properties of O’s experience. So, if a sentence of the form ‘X looks like Y to O’ is true, then there is a P such that the property of representing something as P is a representational phenomenal property of O’s experience, and P is how X and Y look to O. But a report that describes experience e reflects the representational phenomenal property of representing

Perceptual Reports   249 something as P iff [necessarily, the report is true iff representing something as P is a phenomenal property of e]. So, comparative phenomenal ‘looks’ reports reflect representational phenomenal properties of the perceptual experience they describe. What about non-comparative phenomenal ‘looks’ reports? Non-comparative phenomenal ‘looks’ reports typically have the form ‘X looks [ADJ] to O at t’. Sentences which have this surface form and which are not used epistemically are either implicitly comparative or non-comparative. If they are implicitly comparative, then, as we have just seen, they reflect representational phenomenal properties of the perceptual experience they describe. If they are non-comparative, and they are true, then it follows that X phenomenally looks [ADJ] to O at t. But by the LRB principle and the hypothesis that things cannot phenomenally look to have non-representational phenomenal properties, it follows that the property of representing something as [ADJ] is a representational phenomenal property of O’s experience at t. So, if ‘X looks [ADJ] to O at t’ is true, then O has an experience at t with the phenomenal property of representing something as [ADJ]. But a report that describes experience e, and is a token of a sentence S that can have true tokens, reflects the phenomenal property of representing something as P iff [necessarily, the report is true iff the property of representing something as P is a phenomenal property of e]. So, non-comparative phenomenal ‘looks’ reports reflect representational phenomenal properties of the perceptual experience they describe. Both comparative and non-comparative phenomenal reports reflect representational phenomenal properties of the experiences they describe. As Brogaard (2010) argues, there is a simple argument from the premise that phenomenal ‘looks’ reports reflect phenomenal properties of the experiences they describe to the conclusion that phenomenal ‘looks’ reports reflect contents of the experiences they describe. The argument runs as follows:  Look-Content Argument



(1) Phenomenal ‘looks’ reports reflect representational phenomenal properties of the perceptual experience they describe. (2) Any representational property of perceptual experience is the property of having a certain perceptual content. (3) Hence, phenomenal ‘looks’ reports reflect a content of the perceptual experience they describe.

We have already established that (1) is true. Here is an argument for premise (2). It is a priori that if an experience has the property of representing p, then the experience represents p. But if an experience has representational content, and it represents p, then p is a content of the experience. So, if an experience has the representational property of representing p, then p is a content of the experience. The thesis that perception has content rules out a number of theories of perception according to which perception does not have content. These include some versions of naive realism, disjunctivist versions of naive realism, sense-datum theories, and raw-feels theories. A standard objection to my arguments from perceptual reports to the metaphysics of perception is that even if the arguments are sound, the conclusion only settles that

250   Berit Brogaard perceptual experience has representational phenomenal properties and representational content in a weak sense of content. In the weak sense of content, disjunctivists, adverbialists, raw-feels theorists, and so on, may agree that perception has content. A stronger claim would be that part of what it is for an act to be a perceptual experience is for it to have phenomenal properties and/or content. Experiences can be said to have strong content in this sense (see Siegel 2009, 2010 for this distinction; see also Pautz 2009). This is not the place to address this issue at great length. But a few words about this distinction are in order. An argument can be made from ‘looks’ reports to the conclusion that some ‘looks’ reports reflect wide object-involving content. For example, ‘He looks like Dave’ reflects a wide content that contains Dave as a constituent (see Brogaard 2010). Most thinkers who deny that experience has content in the strong sense would disagree that experiences have this kind of wide object-involving content. It is thus possible to use arguments similar to those presented here to argue for the strong-content thesis. Another standard objection to arguments from language to metaphysics is that we cannot assume that the words of ordinary language are those that theories of perception are concerned with. It may be said, for example, that the ordinary language words ‘look’ and ‘see’ express concepts that are distinct from the concepts employed by philosophers of perception (cf. Brogaard 2012a). I grant that this is a genuine possibility. Philosophers of perception certainly use the word ‘experience’ differently from most people (Byrne 2009). In ordinary language, ‘experience’ has a variety of meanings none of which is exactly the one philosophers of perception have in mind. As a verb, it means ‘undergo’ (cf. ‘experience a great adventure’). As a noun, it can mean an event participated in (cf. ‘the trip was a fantastic experience’), knowledge that derives from participating in a given event or series of events (‘a lesson taught by experience’) or perceptual exposure to (cf. ‘my first experience of NP’). However, there is no similar difference between our ordinary and philosophical uses of ‘see’ and ‘look’. In fact, all of the examples philosophers normally use when addressing issues concerning events of seeing and looking come from ordinary language. So, I highly doubt that there are concepts of seeing and looking that are distinct from those of ordinary language. A third objection is this: The look-content argument only works if ordinary perceptual reports are true. If, however, their truth-conditions involve a false theory of perception they may be false. I grant that there is ordinary language discourse that is treated as true but is in fact false. A good example is ‘The sun rises’. This statement is strictly false. But the latter statement is about an object, viz. the sun, whereas perceptual reports involving ‘appear’ words are about how things appear to a particular perceiver. It is unlikely that all our expressions of how things appear to us are false. A fourth objection is this: The correct semantics for ‘looks’-sentences needs to be sensitive not just to armchair linguistic evidence, but also to philosophical reasoning about perception. For example, when science tells us that water is H2O, that tells us something about the semantics of ‘water’; if philosophy or science were to tell us, for example, that sense-datum theory or naive realism is true, that would tell us something about the semantics of ‘looks’. So, we can’t conclusively argue from semantics to metaphysics unless we have already done the metaphysics. By way of reply, it is doubtful that science can determine how things look to us. How things look to me seems to be something only I can determine. So, while science can help

Perceptual Reports   251 us discover the content of ‘water’, it cannot help us discover the content of perceptual reports. I think the same sort of point applies to the thought that science might discover that sense-datum theory or naïve realism is true. Because theories of content are intimately tied to how things look to us, as we have seen, science cannot confirm or falsify them. Of course, to the extent that linguistics is a science, science can indeed help us settle which theory of content is correct, but this is just the approach taken here.

5 Intensional uses and intentional content David Bourget (2010) has argued that the intensional nature of perceptual verbs causes trouble for Hinton’s argument (1967) for a disjunctive account of perception. Hinton argues that (10) should be analyzed in terms of (11):  (10) I am experiencing a flash of light. (11) Either (A) I see a flash of light, or (B) I have an illusion of a flash of light. On Hinton’s disjunctive account of perception, when I experience something I either stand in direct perceptual relation to an entity (in this case, a flash of light) or I have an illusion (or hallucination of that entity. So, if (11) is the correct analysis of (10), then that seems to provide evidence for a disjunctive analysis of perception. However, as Bourget points out, ‘see’ appears to have uses that introduce intensional contexts. Suppose the heartbroken Lois Lane takes a strong hallucinatory drug and then utters the following (these are modifications of Bourget’s examples):  (12) (a) Wow, I see Superman on my left. That’s a really strong drug. (b) I see Superman spinning in front of me, even though I know he isn’t even here. (c) I see Superman all over the place. Maybe I should stop taking the drug. The sentences in (12) seem intuitively true. But if they are true, then the result of substituting ‘Clark Kent’ for ‘Superman’ is false. So, ‘see’ introduces an intensional context, at least sometimes. If ‘see’ is used in this way in (10), then the hypothesis that (10) is to be analyzed as (11) does not lend evidence to a disjunctive analysis of perception. The occurrence of ‘see’ could be interpreted in the same way as the occurrence of ‘see’ in ‘Someone is poking my brain. I see stars everywhere’. Here is a further reason to think ‘see’ can have intensional uses (see Brogaard 2013). When we see an event, we needn’t see all of it. For example, ‘John saw the car accident’ does not imply ‘John saw every part of the car accident’. We see (or witness) complex events, and other high-level properties, in part by visually detecting other properties that typically are associated with the event in question. For example, ‘John witnessed the murder’ may be true if John heard a gunshot and saw a man fall to the ground and then ran away. Likewise, John can see a crying event by virtue of seeing various properties that typically are associated with crying, for instance, a shivering body, a handkerchief, runny mascara, and so on.

252   Berit Brogaard Hence, even if all crying events essentially involve shedding tears, it can be true that John saw someone cry even if he didn’t see them shed any tears. So, ‘John saw Mary cry’ and ‘John saw Mary shed tears’ need not be equivalent, even if we assume that crying events essentially involve shedding tears. The opacity of seeing reports of this type adds further support to the argument that ‘see’ has intensional uses.

References Bourget, D. (2010). ‘Intensional and Phenomenal Uses of Perceptual Verbs’, Chapter in ANU Dissertation. Brogaard, B. (2010). ‘Do “looks” Reports Reflect the Contents of Perception’, manuscript, University of Missouri, St. Louis. Brogaard, B. (2012a). Transient Truths: An Essay in the Metaphysics of Propositions. New York: Oxford University Press. Brogaard, B. (2012b). ‘What do We Say When We Say How or What We Feel?’ Philosophers’ Imprint, 12 (11), June 2012. Brogaard, B. (2013). ‘Seeing as a Non-Experiental Mental State: The Case from Synesthesia and Visual Imagery’. In R. Brown (ed.), Consciousness Inside and Out: Phenomenology, Neuroscience, and the Nature of Experience. Neuroscience Series, Synthese Library. Brogaard, B. (2014). ‘It’s Not What It Seems. A Semantic Account of "Seems" and Seemings’. In H. Cappelen and S. Shapiro (eds), Contextualism and Relativism, special issue of Inquiry. Byrne, A. (2009). ‘Experience and Content’. Philosophical Quarterly, 59, 429–451. Chalmers, D. (2004). ‘The Representational Character of Experience’. In B. Leiter (ed.), The Future for Philosophy (pp. 153–181). Oxford: Oxford University Press. Chisholm, R. M. (1957). Perceiving: A Philosophical Study. Ithaca, NY: Cornell University Press. Chomsky, N. (1981). Lectures on Government and Binding. Dordrecht: Foris. Chomsky, N. (1986). Knowledge of Language. New York: Praeger. Glüer, K. (2009). ‘In Defence of a Doxastic Account of Experience’. Mind and Language, 24, 297–327. Heim, I. (2006). ‘Remarks on Comparative Clauses as Generalized Quantifiers’, Manuscript, MIT. Hinton, J. M. (1967). ‘Visual Experiences’. Mind, 76, 217–227. Jackson, F. (1977). Perception: A Representative Theory. Cambridge: Cambridge University Press. Larson, R. (1988). ‘Scope and Comparatives’. Linguistics & and Philosophy, 11, 1–26 Martin, M. (2010). ‘What’s in a Look?’ In B. Nanay (ed.), Perceiving the World (pp. 160–226) Oxford: Oxford University Press. Matthen, M. (2010). ‘Colour Experience: A Semantic Theory’. In Jonathan Cohen and Mohan Matthen (eds), Colour Ontology and Colour Science (pp. 67–90). Cambridge, MA: MIT Press. Maund, B. J. (1986). ‘The Phenomenal and Other Uses of “Looks”’. Australasian Journal of Philosophy, 64, 170–180. Mitchell, B. and Robinson, F. C. (2001). A Guide to Old English, sixth edition. Oxford: Blackwell. Partee, B. (1975). ‘Deletion and Variable Binding’. In E. Klima (ed.), Formal Semantics of Natural Language (pp. 16–34). Cambridge: Cambridge University Press. Pautz, A. (2009). ‘What are the Contents of Experiences’. The Philosophical Quarterly 59, 483–507.

Perceptual Reports   253 Postal, P. (1974). On Raising. Cambridge, MA: MIT Press. Siegel, S. (2009). ‘Do Visual Experiences Have Contents?’ In B. Nanay (ed.), Perceiving the World (pp. 333–368). New York: Oxford University Press. Siegel, S. (2010). The Contents of Visual Experience. New York: Oxford University Press. Travis, C. (2004). ‘The Silence of the Senses’. Mind 113, 57–94. Travis, C. (2013). ‘The Preserve of Thinkers’. In Berit Brogaard (ed.), Does Perception Have Content? New York: Oxford University Press.

Pa rt  I I I

T H E  SE NSE S

Chapter 14

V ision David R. Hilbert

1  Vision in the philosophy of perception Discussions of vision dominate the philosophy of perception. Although the other senses are mentioned from time to time, the standard budget of philosophical examples that are used to explore issues in perception almost all involve seeing. From seeing sticks partially immersed in water to looking at pillars reflected in mirrors with a steady diet of ellipticallooking coins in between, discussion of most of the central issues in philosophy of perception is focused on visual examples. The philosophical obsession with vision has had two untoward effects. First, and most obvious, generalizations drawn from vision are inappropriately applied to the other senses (for discussion of this problem in the case of sound see, O’Callaghan, 2007, 2008). Second, and less commonly recognized, vision itself, with its own peculiarities and distinctive features has a tendency to fade from view and what we are left with is a generic sense, suitable for serving as the model for perception in general. Vision is, as philosophers intermittently recognize, quirky in the way characteristic of evolved capacities more generally. It combines great sophistication with surprising limitations and sometimes accomplishes its basic task of acquiring environmental information from ambient light in ways that are extremely counterintuitive.

2  Basics of vision Vision can be discussed from many different points of view: from philosophical reflections on the nature of the relation between mind and world involved in seeing to the quantum interactions that underlie the physics of reflection and the physiology of photon detection with the psychology of visual processing somewhere in between. For the purpose of philosophy of perception, many of these details can usually be safely ignored but there is a substrate of basic vision science that constrains the possibilities at higher levels in ways that are sometimes philosophically important. I will divide this material into two sections. First, a discussion of the optical process that includes the interaction of light with objects

258   David R. Hilbert in the environment and culminates in the formation of an image on the retina. Second, a discussion of some of the biological processes that begin with the absorption of photons by pigment molecules within the photoreceptors in the retina. Although this material will be familiar to many readers, not all of it will be familiar to all readers and there may be some surprises even for those who think they know this material. (A slightly dated but very useful overview of vision science is Palmer, 1999).

Visual optics As most of us were repeatedly taught, starting in grade school, the process of vision typically begins with a source of light that illuminates the objects in a scene.1 The light is reflected from the surfaces of objects and some of it enters the eye where the cornea and lens combine to focus the light and produce an image of the scene on the retina. This textbook scenario oversimplifies in numerous respects, most notably in ignoring the fact that we have two eyes with overlapping fields of view, but it captures two central truths about vision. First, it is light as modified by the surfaces and objects in the environment that enables vision. Differences between objects that don’t affect light (or aren’t correlated with differences that affect light) are not visible.2 Thus, it is the variable effects of different parts of the scene on the light falling on them that enables us to see the objects present and their qualities. Second, a crucial part of the process that allows us to make use of the light reaching the eye to see the objects in our surroundings is the formation of an image within the eye. Image formation is important because the formation of the retinal image separates the light coming from the different parts of the scene and thus enables spatial vision and with it the ability to visually attribute properties to different locations within the scene.

Light sources The light sources that initiate the process of vision can be described in terms of two sets of characteristics: spatial and spectral. First, light sources can be divided into those that are of significant spatial extent, like the sky on an overcast day or a bank of fluorescent tubes behind a diffusing panel, and those that approximate point sources, like the sun or a street lamp. The first can provide much more uniform illumination across the scene while the second illuminates objects in a way that depends much more strongly on their position and orientation with respect to the light source and the other objects in the scene. The character of the light emitted by a light source can be described using the combination of the overall intensity of the light (its total power across the spectrum) with the spectral power distribution of the light (how the power is distributed across the wavelengths of the visible spectrum). Since the sensitivity of human vision to light of different wavelengths varies significantly across the spectrum, two light sources of equal intensity can be very different in their effectiveness at producing a visual response if they differ in spectral 1  I will ignore the special case of the perception of self-luminous objects as well as transparent and translucent objects. 2  I will set aside, for now, the possibility of input from the other senses to vision. The simplification will be useful in spite of the fact that acoustic inputs affect what we see (and visual inputs affect what we hear).

Vision   259 power distribution. For example, the principal cause of the high efficiency of the familiar yellow street lights is not that they are particularly efficient at turning electricity into light, but rather that they emit almost all of their light at wavelengths to which the human eye is very sensitive.

Objects When light falls on an object some proportion of the light at each wavelength is reflected, some proportion is absorbed within the object, and, for transparent and translucent objects, some proportion is transmitted. Reflection can be quite complicated but for many purposes it is useful to separate the reflected light into two components. First, a diffuse component, in which the intensity of the reflected light displays relatively little dependence on the angle between the eye, the object’s surface, and the light source. Second, a specular component in which the reflection is mirror-like and highly directional. Typically, the diffuse component is much more influenced by characteristics of the object, while the specularly reflected light approximates the light source. A number of the characteristics of an object affect the way in which it modifies the light it reflects, most notably its chemical composition and the roughness of its surface. Since many objects are heterogeneous in their composition the reflecting characteristics of an object are typically variable and the variation often is found at several different spatial scales giving rise to both visible patterns and visible texture.

Scenes The result of the interaction between light sources and objects in a scene is that a complex pattern of light varying in both spectral composition and intensity permeates the environment. Most scenes have multiple light sources. The sky and the sun dominate in many outdoor scenes but there can be significant amounts of illumination deriving from partially opaque objects like leaves and light reflected from one object in the scene often is part of the illumination for other objects in the scene. Multiple light sources, including inter-reflection, are common in indoor scenes as well. The structure of environmental light is the joint product of the characteristics of the light sources and the reflecting characteristics of the objects in the scene. Disentangling these two factors is one of the principal problems facing the visual system in using the structured light to acquire information about the environment.

Eyes In order for an organism to make use of light to find out about the environment it must have some method of sampling and characterizing light. For us, this task is accomplished by the eye. Although essentially every theory of vision since antiquity recognizes the central role that light plays in vision, the exact nature of that role and its relation to the anatomy and physiology of the eye emerged surprisingly late. The most detailed and successful theories of vision in antiquity involved the emission of visual rays from the eye, rather than

260   David R. Hilbert the reception of light by the eye. Rather than sampling environmental light that interacts with the sensitive parts of the eye, these theories essentially extended the eye’s sensitivity to the surface of the seen objects. In these theories, light plays an important enabling role but is not itself the carrier of information about the world around us. The first relatively successful intromissionist theory of vision is that of ibn al-Haytham in the eleventh century and it was not until Kepler in the sixteenth century that the image-forming nature of the optics of the eye was correctly described.3 Extramissionist theories provided a solution to one of the central problems in vision and that fact helps to explain their persistence in the face of the severe physical and philosophical problems that they confronted from the very beginning. Suppose we grant—anachronistically but no harm is done—that the retina is the sensitive part of the eye. Each point on the retina will receive light from every part of the scene that is not blocked by some opaque object. There is no way that measuring the characteristics of the light at the retina can reveal the spatial structure of the scene that generated it.4 The eye is spatially extended and the opaque parts of the eye and body will block light from many parts of the scene but nevertheless each point on the sensitive part of the eye will receive light from every unoccluded point in the field of view. If vision is to deliver information about the spatial distribution of properties in the world then some method is needed for sampling light that preserves the spatial structure from which it is reflected. Extramissionist theories of vision solved this problem by postulating visual rays originating in the eye and travelling in straight lines. This ensures each point in the scene is contacted by a single visual ray which allows the eye to be sensitive to the spatial structure of the scene. The price paid is the postulation of the existence of visual rays capable of extending to the most distant visible objects. The key contribution of ibn al-Haytham’s theory of vision was to put forward an intromissionist theory that allowed the eye to capture spatial structure by rejecting all of the rays reaching it except those perpendicular to the surface of the lens. Although the mechanism for this rejection was somewhat obscure, it allowed a point-to-point correspondence between locations in the scene and locations in the sensitive part of the eye (the lens for ibn al-Haytham) and thus provided a mechanism for spatial vision. Kepler’s contribution was to recognize that refraction at the cornea and lens had the effect of focusing all of the light originating in a single location in the scene on a specific point on the retina, producing the necessary point-to-point correspondence via a plausible mechanism. In its outlines, Kepler’s theory matches our current understanding of the optics of the eye. In its optics, the vertebrate eye is essentially similar to a camera (or perhaps a camera is essentially similar to an eye). It’s a bad camera in some respects, particularly in its inability to bring light from both ends of the spectrum into focus on the retina at the same time (the eye has a high degree of chromatic aberration). It’s like a modern digital camera in that the image that is formed by the optics is then sampled by an array of discrete receptors that then produce signals that make information about light intensity available for further processing. The point of image formation in both eyes and digital cameras is not to make 3 

For a relatively accessible treatment of this history, see Lindberg (1976). There are alternatives here. One is to imagine an eye with a very small pupil scanning the scene one small region at a time. There are other more complicated possibilities as well. Since human vision (and vertebrate vision more generally) does not implement these possibilities they will be ignored here. 4 

Vision   261 an image available for viewing but rather to allow measurement of the spatial structure of the light reflected off a scene. Image formation in both cases is the end of the optical part of vision,5 but only the beginning of another set of processes that involve detecting and transforming the information made available by the image.

3  Vision in the brain The retina and LGN The retina The retina is a complicated structure that contains neurons of several different kinds as well as a great deal of machinery that supports and interacts with the neurons. The retina contains two fundamentally different types of photoreceptors that contribute to vision known as rods and cones.6 The rods are very sensitive to light but are unable to signal differences in intensity at higher light levels. They primarily serve night vision although they do contribute to vision at the intermediate light levels found at twilight and in many indoor artificially lit areas. Rods are sufficiently sensitive, as is the neural processing in the retina, that under the right conditions, absorption of a single photon by a rod can produce a noticeable visual response. The cones don’t contribute to vision at low light levels but, in contrast to the rods, continue to provide a differential response as light intensity increases up to the point at which they are damaged. The cones are the primary input to vision at higher light levels. In the human eye, the cones are further subdivided into three types that are characterized by their differences in spectral sensitivity, how responsive they are to light of different wavelengths. One cone type, the S-cones, has its peak sensitivity at the short wavelength end of the spectrum and its response is essentially zero in the middle and long wavelengths. The other two cone types (M-cones and L-cones) have closely spaced peak sensitivities near the middle of the spectrum. By sampling the retinal image with three spectrally different photoreceptor types, the visual system acquires information about the spectral power distribution of the light falling on it and not just its intensity. This information is used, not only for colour vision, but also for various other aspects of vision. The distribution of photoreceptor types across the retina varies in systematic ways that affect how we see. The centre of the retina (the fovea) contains several specializations for

5 

The retina itself is optically active and since light must transit the retina (including the bulk of the photoreceptor itself) before being absorbed and initiating the physiological process of vision, image formation is not really the last step in which the optics are relevant. In addition, many nocturnal animals have eyes that contain a reflecting layer behind the retina so that light passes through it twice to increase the chance of absorption, adding another optical step to the process. 6  The names derive from the characteristic shape of the outer segments of the two types of photore­ ceptors, the part that contains the pigments that interact with light to initiate visual processing. The human retina, as is characteristic of mammals, has a very simple structure as compared to the retinae of many non-mammalian vertebrates. Other vertebrates have retinae with a wider variety of photoreceptor types and other specialized features that are lacking in mammals. I will be focusing exclusively on the human eye, which is a pretty standard eye for an old world primate.

262   David R. Hilbert high-resolution spatial vision. There are no rods in the fovea and the cones are more tightly packed than in the peripheral retina.7 Blood vessels and the cell bodies of retinal neurons through which the light would otherwise have to pass to reach the photoreceptors are shunted to the side. In the very centre of the fovea, there are no S-cones, exclusively L- and M-cones. The effect on vision is that we can resolve much finer spatial detail centrally than peripherally and, although it is rarely evident, we are all yellow-blue colour blind to small, centrally presented stimuli. S-cones are very much less common than L- and M-cones throughout the retina, which contributes to poor spatial resolution for short-wavelength stimuli. This also has the effect of mitigating the eye’s substantial chromatic aberration since, although a sharp image in the middle to long wavelengths will be blurred in the short wavelengths, the eye has such coarse resolution at the short wavelengths that the blur at those wavelengths is less visually significant. The process of vision is initiated by the absorption of photons by the photopigments within the photoreceptors. Although the relative sensitivity of the photoreceptors to light of different wavelengths is fixed, the absolute sensitivity of the photoreceptors dynamically adjusts to the light level. This adaptation allows the cones to provide usable signals at the very wide range of light intensities that we encounter as we move about the environment. One consequence of this is that the cone outputs provide relatively little information about the absolute intensity of the light stimulating them. The darkest areas of a scene lit by direct daylight are comparable in absolute intensity to the brightest areas of a scene viewed under typical reading light even after correcting for the change in pupil size. In what will shortly become a central theme, the cone responses provide more information about differences or changes in light intensity than they do about absolute intensity. Processing of visual information begins within the retina itself and the output neurons of the retina, the ganglion cells, which communicate with other parts of the brain, have very different response properties from the photoreceptors themselves. Ganglion cells receive inputs (indirectly) from multiple photoreceptors and typically have centre-surround receptive fields in which they are excited (inhibited) by light in the centre of the receptive field and inhibited (excited) by light in the periphery of the receptive field. The centre and periphery can also differ in their sensitivity to light of different wavelengths. One result of this is that ganglion cells are more sensitive to contrast than to absolute intensity. They respond poorly to uniform light and much more strongly when there is an appropriate contrast between the light falling into different parts of the receptive field. Because of adaptation at both the level of the photoreceptors and at the ganglion cell level (and the neurons connecting the two), ganglion cells respond better to changes in the light reaching their receptive fields than to unvarying light. Like the photoreceptor density, the size of the receptive fields of retinal ganglion cells varies systematically from centre to periphery. Receptive field sizes are very small in the fovea, with the centre response driven by a single photoreceptor in many cases, and increase towards the periphery. These features combine with the photoreceptor spacing to result in much higher spatial resolution in the centre of the visual field than towards the edge.

7  One consequence of the lack of rods in the central fovea is that night vision is very poor in central vision. To see an object in the dark it is best to look to one side of it.

Vision   263 This arrangement has consequences for how we see. Since the highest resolution area of the retina covers only about 1º of visual angle (a little smaller than a typical thumbnail viewed at arm’s length) and the entire fovea covers only a little over 5º of visual angle, the part of the scene for which we have high spatial resolution at any given time is quite small. To acquire high-resolution information about our environment, we move our eyes in a series of jumps (called saccades) and we move our head and body with larger motions. Rather than acquiring a high-resolution image all at once in the manner of a camera, we sample the structured light in bits and pieces. The sequence of saccades is not random and especially interesting parts of scene are typically sampled repeatedly. Vision, in this sense, involves active exploration of our environment, rather than passive reception. It also means that for human beings the direction of gaze is very informative about what is being attended to, a fact that plays an important role in social interactions. Retinal processing also begins a tendency towards specialization that continues through later stages of the visual system. The most important is the subdivision of retinal ganglion cells into two separate processing streams known as the parvocellular and magnocellular streams. Very roughly, the parvocellular (P) stream carries information about sustained, high spatial resolution aspects of the retinal image and it is also the principal carrier of chromatic information. The magnocellular (M) system is responsive to rapidly changing stimuli, has lower spatial resolution, and is relatively insensitive to chromatic information. These two pathways are driven by the M- and L-cone outputs and the S-cone signal is carried by a separate pathway that primarily contributes to chromatic processing. The axons from the ganglion cells collectively make up the optic nerve and most of them project to a mid-brain area in the thalamus. They exit the eye at a single point which, consequently, contains no photoreceptors. Each eye, thus, contains a blind spot, and there is a corresponding region of space about which no visual information is obtained. The blind spots are both located peripherally from the centre of vision so the blind spots from the two eyes do not capture information from the same region of space and the entire binocular field of view has coverage from at least one eye. It is thus not surprising that the blind spots are not noticeable with binocular vision. More surprisingly, even when looking at a scene with one eye closed it is not apparent that there is an area of the scene the light from which is not sampled by the eye. The blind spot is roughly comparable in size to the fovea so an object as large as a thumb nail at arms length fits comfortably within it. In a static display it is very difficult to detect the blind spot and it is only the surprising disappearance of objects as they move in and out of the blind spot that reveals its existence. Our lack of awareness of the existence of the blind spot is often explained by appeal to the visual system filling in the missing information on the basis of the characteristics of the surrounding scene although this interpretation is controversial.

Stereoscopic vision Human beings, like other primates, have two forward-facing eyes with substantial overlap in their field of view. The motion of the two eyes is coordinated so that they both fixate on a single object or point in space. Since the eyes have slightly different points of view, the position in the two retinal images of objects that differ in depth from the fixated object will differ slightly. The human visual system is exceptionally sensitive to these

264   David R. Hilbert slight differences, known as disparities, and uses disparity in performance of a number of visual tasks.8 Famously, disparity is a source of information about the distance (relative to the fixation point) of objects. Although the basic idea can be found in Ptolemy (and, once again, was clearly worked out by Kepler) it has had difficulty attracting consistent attention from philosophers. The two-dimensional nature of visual experience and its associated elliptical-appearing coins and the like has been a main stay of philosophical discussions of perception throughout the modern era. Setting aside the 150  years of psychophysical investigation, there is now abundant evidence from invasive studies in monkeys and non-invasive imaging in both monkeys and humans that disparity plays an important role in early vision. Not only depth, but also slant and aspects of shape and motion make use of disparity inputs as does control of eye movements. The overlapping fields of view that enable stereoscopic vision limit the total field of view for human beings to about 200º (with the two eyes overlapping for about the central 120º). Many other mammals, e.g. horses and rabbits, have much more panoramic visual systems and their field of view can be nearly a full 360º. As a consequence, human vision is much more active and the limited field of view, like the specialization of the central retina, leads to eye, head, and body movements being an integral part of the process of vision. Although the difference between the images in the two eyes provides important information about the scene, it also leads to a problem. The differences in the two images would seem to imply that we see two different scenes simultaneously. The correct description of what we see in cases of binocular vision is not, however, completely clear. For objects that are at depths similar to the fixation point it is clear that we fuse the information from the two images and experience a single object that reflects information from both eyes. For objects that differ significantly in depth from the fixation point fusion is not possible. It seems that in some circumstances we are aware of a doubleness in vision but in other circumstances the spatial information from one of the eyes is suppressed and what we see reflects the location of the object as seen by one of the eyes. If you point to a distant object with a finger and then alternately close one eye and then the other you will typically discover that the pointing is accurate with one of the eyes and not the other. Precise description of the phenomenology of stereoscopic vision is complicated and much more uncertain than philosophical discussions typically assume. It is also important to keep in mind that binocular stereopsis is just one of many processes that contribute to experienced depth and the relative importance of the numerous cues to depth relied on by the visual system varies with the scene and the task.

Cortical processes From the retina, the main pathway supporting vision proceeds to the primary visual cortex (V1) by way of the dorsal lateral geniculate nucleus of the thalamus (LGN). Although about 90 per cent of the fibres in the optic nerve follow this pathway, the optic nerve is so 8  The visual system is capable of making use of disparity differences smaller than the spacing between individual receptors. As in its basic resolution, the retina is not uniform in its sensitivity to disparity with the human visual system being much more sensitive to small horizontal displacements than to small vertical ones.

Vision   265 massive (about a million fibres) that the minority that follow other paths are comparable in number to the entire acoustic nerve. Although we will focus on the main pathway it’s important to keep in mind that there is significant flow of information along other pathways. Although we will also follow tradition in focusing on the flow of information from periphery to more central areas it’s also important to keep in mind that the back projection from cortex to the thalamus is comparable in size to the forward projection (Farah 2000 provides a useful overview of the neuroscience of vision).

Early vision Neurons in V1 respond in ways that show that they are tuned to moderately complicated features of the visual stimulus. Most cells are sensitive to stimulus orientation in the sense that they respond better to elongated stimuli at some orientations as opposed to others. There are also cells that respond to binocular disparity and other cells that respond to wavelength differences across their receptive field. These cells are arranged in systematic ways with, for example, preferred orientation changing systematically in some regions. Although the response properties of the cells found in early visual areas provide the basis of all of our visual abilities, their properties do not align in any simple way with what we see. Although we can see the orientation of the objects in a scene and this ability derives from the orientation information extracted by cells in early vision, those cells invariably are sensitive not only to orientation but also to other stimulus features like contrast or luminance. Although there are cells that respond to the wavelength differences that underlie colour vision they are also sensitive to the spatial structure of the stimulus. The lack of correlation between the features of objects we see and the properties of neurons in early vision goes the other way as well. Cells that respond to wavelength differences contribute to spatial vision as well as colour vision. Cells that respond to motion contribute to the perception of spatial structure as well as to motion itself.

Two pathways After V1 (and two other early visual areas) the flow of visual information splits into two anatomically separate but interacting streams. One, known as the ventral stream, proceeds along the bottom of the cortex and ultimately involves areas that are, among other things, involved in tasks like object recognition and face recognition. The other, known as the dorsal stream, proceeds upward towards the top of the cortex and contains a number of areas involved in various spatial tasks. The two pathways are anatomically distinct and there are differences between them in the types of stimulus features the neurons in the two pathways respond to, but the functional characterization of the two streams is still a matter of controversy. The early proposal that the two streams represent a functional separation between what (colour, face, object category) and where (spatial location) has largely fallen out of favour. An interesting and influential proposal that has dominated discussion more recently involves a separation between processing visual information for conscious perception and processing for motor control. According to this hypothesis, activity in the ventral stream gives rise to conscious perceptual experience and with that accessibility of visual information to planning and memory. The dorsal stream is used instead for guidance of bodily motion and the information present in the dorsal stream is not ordinarily

266   David R. Hilbert available for conscious perception. This hypothesis has given rise to an enormous literature with the early papers laying out the hypothesis having thousands of citations each. In spite of the sizeable body of research, it’s still a matter of controversy whether the idea that vision for action is separate from vision for perception is correct and the hypothesis itself is vague in important details.

Neuropsychology of vision The empirical study of vision until very recently has primarily relied on three sources of data: psychophysical studies of human beings, primarily college students in the developed world; anatomical and invasive physiological studies of non-human animals, especially the macaque; and neuropsychological studies of humans that have impaired vision as a result of damage to the brain, primarily the cortex. Since psychophysical research involves analysing human behaviour in response to systematic variation in stimuli, it provides useful data about how human beings perform visual tasks and can provide evidence about the mechanisms that underlie that performance. Since the mechanisms of vision are complicated and psychophysics provides data only on how visual inputs are related to behavioural output, there are many questions about visual processing that are difficult to address using psychophysical techniques. Some of these gaps can be supplemented using the results of invasive studies on monkeys. The macaque visual system is similar to the human one in many respects and the possibility of inserting electrodes to measure neural activity in an awake, behaving animal has produced most of the data relevant to the response properties of individual neurons in the cortex. These physiological measurements can be supplemented with anatomical data to provide much of what is known about the features of the various visual areas, their connectivity, and their function. Macaques are not, however, human and it can be difficult to relate the macaque data to the human data. Neuropsychological studies have provided important and fascinating additional information about the way in which visual information is processed in human beings that helps to relate human behaviour and anatomy. More recently, the advent of non-invasive functional imaging techniques (primarily fMRI) has led to a new source of constraints on hypotheses about human visual processing. Damage to the visual areas of the cortex can produce an enormously wide variety of effects ranging from complete blindness to surprisingly narrow impairment in specific visual abilities. The field is much too broad to adequately survey here so I will focus my discussion on briefly characterizing a handful of neuropsychological results either because they have drawn philosophical attention or because they highlight interesting features of the organization of visual processing. The methodology involved in drawing conclusions about visual processing on the basis of the behaviour of individuals with brain damage is itself an interesting subject which I will largely ignore. Damage to V1 can result in blindness in an area of the visual field with the size and location of the blind area (scotoma) depending on the size and location of the lesion. For example, complete (or nearly complete) destruction of V1 on the left side of the cortex will result in a right homonymous hemianopsia in which visual sensitivity is lacking to the right side of space for both eyes. Less extensive damage will result in a less extensive scotoma, and because the centre of the visual field is overrepresented in V1 it’s most common for the

Vision   267 scotoma to affect peripheral vision with central vision spared. This relation between damage to V1 and (nearly) complete loss of visual function to a region of space helps to confirm the hypothesis that the main pathway for visual information passes through V1. V1 and its role in vision is probably best known to philosophers via its role in the much discussed phenomenon of blindsight. In blindsight, there is the spared ability to engage in visually guided behaviour in response to stimuli presented within a scotoma. From the physiological point of view this is interesting (and perhaps surprising) since it shows that visual information that does not follow the principal pathway from retina to V1 via the LGN can guide voluntary behaviour. The sometimes radical conclusions about the relation between visual information and consciousness that some philosophers have drawn are less clearly fully supported by the characteristics of actual people with blindsight. (A useful recent review can be found in Cowey 2010.) From the point of view of understanding normal visual processing, cases that present a pattern of sparing of some visual abilities accompanied with deficits in others are often the most informative. These kinds of cases have given rise to proposals to the effect that specific visual tasks are associated with specific anatomical areas and accompanying the anatomical specificity is some degree of functional modularity. For example, in prosopagnosia there can be profound impairment in the ability to visually recognize faces in spite of relatively normal performance on many other visual tasks. Strikingly, severe impairment in face recognition can be accompanied by much more normal performance in visually recognizing other types of objects. Often associated with prosopagnosia is central achromatopsia, in which there can be a complete loss in the ability to see colour without a comparable impairment in other visual abilities. In both of these cases, the evidence from the specific combinations of spared and impaired abilities in patients has given support to hypotheses that there are anatomically localized brain areas dedicated to the processing of colour and to facial recognition. How strong this support is has been a matter of considerable controversy, as has been the correct functional characterization of the type of processing involved. One notable feature of the literature on the neuropsychology of vision is the prominent role that a single subject can play in the literature. A man referred to in the literature as MS is the source of much of what is known about central achromatopsia. Even more famously, a woman known as DF, who suffered cortical damage as a result of carbon monoxide poisoning, provided most of the early data offered in support of the proposal that the dorsal and ventral streams are specialized for action and perception respectively. As a result of her injury DF suffered from a disorder known as visual form agnosia. Visual form agnosics display an almost complete inability to visually perceive spatial features of scenes including the size, shape, and orientation of objects. Perception of other simple visual attributes like brightness and colour is relatively spared. In the case of DF, even though she is unable to report or otherwise indicate the spatial features of the objects she is viewing, her bodily movements appear to rely on the availability of information about those spatial features she is unable to report on. DF’s vision has been intensively studied and she is the subject of a fascinating popular book (Goodale and Milner 2005). A different impairment, known as optic ataxia, provides a fascinating and informative contrast. In optic ataxia, relatively spared ability to describe spatial features of objects is combined with severe impairments in visually guided behaviour. The possibility of these two contrasting impairments resulting from different anatomical patterns of cortical damage is thought to provide important support for the two visual systems hypothesis.

268   David R. Hilbert

Modularity of vision The evidence is overwhelming that there is a great deal of specialized processing involved in how we see. Different cortical areas are specialized for processing different types of information and even within a single anatomical region there can be subpopulations of neurons that have response properties that suggest that they are involved in specific tasks. Results from neuropsychology show that what might have been thought to be tightly coupled visual abilities are dissociable, with the use of wavelength information for form vision spared in spite of the complete loss of colour vision and colour vision persisting in spite of severe impairment in form vision. Motion and position can be decoupled in normal subjects where illusions like the waterfall illusion show that it is possible to perceive motion without change in position. The upshot is a picture of vision in which various visual attributes are processed independently of each other and contribute independently to our ability to perform various tasks. Our visual experience and the visual information driving action and cognition will be assembled from these diverse sources in a way that can vary with the specific task being performed. Consistency of the outputs of the specialized areas would largely be a consequence of their all deriving their inputs from a common source. Although different visual areas do engage in specialized and partially independent processing of visual information, it is equally important to keep in mind that there are significant interactions between the specialized areas. Form plays an important role in colour processing, shading is used for form vision, and there are even intermodal interactions effects with auditory stimuli affecting vision and vice versa. There is also no reason to expect that the kinds of processing and outputs of specialized areas will necessarily coincide with folk or philosophical conceptions of the primitive components of vision. Various types of contrast seem to play a central role in visual processing but contrast as a primitive is not usually found in folk lists of visual attributes and is not a basic component of those philosophical theories that have been most important historically. Even when the processing seems to be devoted to a familiar category like faces, the evidence for exactly what stimulus features an area responds to is typically complicated and difficult to interpret. Vision is clearly modular in some sense but in ways that defy easy assimilation into any simple conception of modularity.

4  Philosophical issues of vision Until quite recently there was relatively little contact between philosophical discussions of vision and even the most basic facts about the visual system and visual processing.9 Consequently, discussions of vision were primarily focused on it as an example used to 9  I am setting aside, for current purposes, the surprisingly fertile interaction between the empirical study of vision and philosophy that can be found in a number of historical figures. Some notable examples include the influence of ibn al-Haytham’s theory of vision on Roger Bacon and the central role of a serious engagement with early modern theories of depth perception in the development of Berkeley’s philosophy. Descartes, like Berkeley, was a contributor both to the empirical study of vision and also to its philosophical understanding.

Vision   269 illuminate very general questions in the philosophy of perception. Questions such as the nature of the immediate objects of perception or the distinction between immediate and mediate perception itself have been discussed in large part using examples drawn from vision. Much of the language and many of the metaphors used in discussions of perception more generally are drawn from vision: e.g. the transparency of perception, the objects of perception. In spite of the centrality of vision in discussions of perception generally, I will focus in what follows on philosophical questions that have a more substantive connection with the nature of vision itself.

The objects of vision Perception has been taken by many philosophers to fundamentally involve awareness of objects in the world around us. This assumption is particularly natural in the case of vision since one fundamental feature of visual processing is the parsing of scenes into object and background. It’s much less clear that this is an apt description of the chemical senses and even in the case of hearing and touch it requires the introduction of objects that can be very different from the ‘moderate-sized specimens of dry goods’ that are the usual focus of philosophical discussion.10 But in the case of vision, it very often seems natural to describe what we see as familiar objects like apples or tables or human beings. Given this assumption, we can then ask as to the nature of the connection between these objects and us when they are (among) the objects that we see. One popular and plausible answer here is that that the seen object is among the causes of our visual experience. Theories of this type are most often motivated by appeals to common sense rather than vision science and the arguments typically rely on a steady diet of examples involving mirrors, hallucinogenic drugs, and science fiction. It may be that it is possible to motivate in this way the claim that a causal relationship is necessary if we are to see an object but it is much less clear that it is possible to state a sufficient condition for seeing in these terms.11 The problems here are manifold. Since vision is not local in the sense that only light reflected from an object influences perception of that object, the way in which an object is perceived (and whether it is perceived at all) can, and usually does, depend on the features of other parts of the scene. The object and its properties are causally relevant but the cause of any given visual experience is the entire visible area of the scene (perhaps extended in time as well as space). Any attempt to state a sufficient condition for seeing in purely causal terms will, of necessity, run into difficulty since the relations of causal dependency are so complex in vision. Changing the features of a part of the scene remote from the putative object can often cause significant changes in how that object is perceived. Although we see scenes as containing objects at locations, the perception of those objects at those locations depends on the entirety of the visual stimulus, not just the part that originates in the object of interest. Although there are potential solutions to these difficulties, often involving appeal to either content or action, it 10  Although as Austin (from whom the phrase is taken) points out many of the things that we take ourselves to see do not fit this description either (Austin 1962: 8). 11  Grice (1961) is often cited here although he is clearly far from the first to make such a claim. Grice pretty clearly only intended the causal condition as necessary. Jackson (1977) defends the causal thesis as both necessary and sufficient. Also useful here is Tye (1982) and for recent discussion, see Campbell (2002); Arstila and Pihlainen (2009).

270   David R. Hilbert is important to avoid overly simple pictures of the typical causal pathway involved in the genesis of a visual experience (or a visually guided response).

Visual experience When I look at the banana in the bowl there are many things that may be true of me. My eye is sampling the structured light at a particular location and there is resulting activity in the retina and the visual areas of the brain. It also may be the case that I see the banana and I may see that there is something yellow before me. I may believe that there is a banana in the bowl and I may believe that Rachel did not eat the last banana. Although these kinds of truths are often philosophically significant, there is an addition to this list that is often at the centre of philosophical discussions of perception in general and vision in particular. That extra element is that when I look at the banana I am undergoing a visual experience. Philosophical claims concerning vision are often framed as claims about the properties of visual experiences. Visual experiences are typically taken in such discussions as occupying a distinctive place in the process that begins with light sources and objects in the world and often ends with knowledge and belief. Sensory experience in general and visual experience in particular is usually taken to be the first conscious stage of the perceptual process. The phenomenology of vision is also usually taken to be determined by the properties of visual experience. For this reason, most philosophical discussions of vision take there to be an important distinction to be made between visual experience and its properties and other types of sensory experience on the one hand and non-sensory features like believing and hoping on the other. Exactly how to draw this distinction in either conceptual or operational terms is a matter of some controversy and more than a little obscurity (see Schwartz 1994: 84–124). The working assumption of many philosophical discussions is that introspection supplies a crucial and reliable source of information about some of the properties of visual experience but this assumption is often left largely unexamined. Given that what seems introspectively obvious to one philosopher often seems incompatible with the introspective evidence to another this is surely an assumption worth more attention than it often gets.12 Equally often neglected is the possibility that the philosophical methods for singling out visual experience fail to pick anything out at all. Although this assumption is often carried over, though differently formulated, into psychology it finds little support in what is known of the neurobiology of vision. Although the behaviour of neurons early in the visual pathway often depends pretty much exclusively on purely visual inputs, the later stages of visual processing are much less isolated from inputs from non-visual areas of the brain.

The structure and content of visual experience Assuming that there is a well-defined kind of mental state answering to the description of visual experience, we can raise the question of exactly what properties it has. Here I will focus on a particular subset of the properties of visual experience. One of the oldest 12  And the fact that psychologists found it useful to largely abandon introspective methods in the study of perception in the early twentieth century may contain a moral.

Vision   271 debates concerning vision is whether there is a relatively short list of properties that constitute the basic properties of visual experience. In its simplest form this holds that our visual experience is fully characterized using colour and position. Once the colour of each position in the visual field has been characterized there is nothing more to be said about the specifically visual component of our experience. Any other properties that we become aware of as a consequence involve the operation of other more cognitive, non-sensory, processes. Although there are good empirical reasons to enrich the characterization beyond this simple mosaic view to include other attributes (e.g. depth, shadow, texture, edges), the other side of this debate has been the claim that our visual experience requires for its characterization other less basic properties. This view is most often and most naturally developed as an attempt to characterize the content of visual experience (or alternatively the features of the objects of visual experience). For example, one common claim is that content of visual experience is better characterized in terms of possibilities for successful action than in terms of the simple, non-dispositional properties just discussed. This line of argument is inspired by the attractiveness of thinking of perception as designed for guiding action and the idea that the content of perceptual experience should be closely tied to this function. A separate line of thought involves the claim that visual experience can involve relations like that of cause and effect or can include as part of its content membership in abstract kinds like fruit or person.13 Here the motivation is less clear but the effect of these kinds of claims is to attempt to shift the line dividing what we know on the basis of vision alone and what requires inference. The arguments for this kind of view often revolve around claims concerning subtle phenomenology and are subject to the kinds of worries concerning the reliability of introspection already mentioned.

Vision and action We see in order to act. As long as our conception of action is broad enough it seems hardly possible to deny this truism. Recently some philosophers have argued that this is not merely a platitude but an important source of constraints on the content of visual experience. On this approach, the content of a type of visual experience is (partly) determined by what it enables us to do. Rather than vision representing an apple as having some physical size it represents it as graspable by the hand but not by the thumb and forefinger. Rather than representing an object as spherical, vision represents it as something that can be rolled. If the kind of action relevant to the content of visual experience is restricted to actions involving bodily movements then there are some obvious prima-facie problems with this line of thought. It’s not completely clear exactly what kind of action some visual attributes can be connected to in this way. There is no specific kind of action enabled by seeing that the cover of the book is predominantly blue. Even if we restrict ourselves to the colours of natural objects similar difficulties are apparent. There doesn’t seem to be any general type of action enabled by perceiving something as

13 

A very useful discussion of these issues can be found in Siegel (2010).

272   David R. Hilbert brown, for example. It may indicate that the grass will burn, that the apple is inedible, or that the meat is cooked. Depending on one’s desires there will be a variety of different actions to be taken with respect to these objects. Although there are potential replies to these worries it still seems that the tie between perception and behaviour is too tight on this kind of theory. Why can’t vision inform us about attributes of objects which are only indirectly relevant to behaviour and whose significance for behaviour is variable rather than fixed? This version of the idea that vision is tied to action also sits uncomfortably with the empirical hypothesis that there are separate visual pathways for conscious perception and motor control mentioned above. Graspability would be a property represented in the dorsal stream and not in the ventral stream tied to conscious perception, memory, and planning. A broader conception of action which focuses on epistemic action rather than overt behaviour allows for development of a theory of visual content that avoids most of the problems just mentioned. Here the idea that the content of vision should be tied to what vision enables us to do includes things we do like coming to know, reasoning, and inference. There is something odd about the idea that there are visual contents that play no role in our epistemic lives. This formulation also avoids the unmotivated assumption that every perceptual content must somehow be directly relevant to behaviour. There is still a remaining question about how to justify the claim that the epistemic effects of a visual state (partly) determine its content as opposed to the content being independently determined and then determining the epistemic effects. However these debates are concluded the relation between what we find out about the world visually and action and epistemology are interesting questions that are matters of continuing controversy.14

References Arstila, V. and Pihlainen, K. (2009). 'The causal theory of perception revisited'. Erkenntnis, 70, 397–417. Austin, J. L. (1962). Sense and Sensibilia. Oxford: Oxford University Press. Campbell, S. (2002). 'Causal analyses of seeing'. Erkenntnis, 56, 169–180. Cowey, A. (2010). 'The blindsight saga'. Experimental Brain Research, 200, 3–24. Farah, M. J. (2000). The Cognitive Neuroscience of Vision. Oxford: Blackwell. Goodale, M. A. and Milner, A. D. (2005). Sight Unseen. Oxford: Oxford University Press. Grice, H. P. (1961). 'The causal theory of perception'. Proceedings of the Aristotelian Society, Supplementary Volumes, 35, 121–152. Jackson, F. (1977). Perception: A representative theory. Cambridge: Cambridge University Press. Lindberg, D. C. (1976). Theories of Vision from al-Kindi to Kepler. Chicago:  University of Chicago Press. Matthen, M. (2007). Seeing, Doing, and Knowing. Oxford: Oxford University Press. Noë, A. (2004). Action in Perception. Cambridge, MA: MIT Press. O’Callaghan, C. (2007). Sounds: A Philosophical Theory. Oxford: Oxford University Press.

14  For an extended treatment, see Noë (2004). Matthen (2007) develops the extended conception of action.

Vision   273 O’Callaghan, C. (2008). 'Object perception: Vision and audition'. Philosophy Compass, 3(4), 803–829. Palmer, S. E. (1999). Vision Science: Photons to phenomenology. Cambridge, MA: MIT Press. Schwartz, R. (1994). Vision: Variations on some Berkeleian themes. Oxford: Blackwell. Siegel, S. (2010). The Contents of Visual Experience. New York: Oxford University Press. Tye, M. (1982). 'A causal analysis of seeing'. Philosophy and Phenomenological Research, 42, 311–325.

Chapter 15

Au dition Matthew Nudds

1  The problem of auditory perception The things we hear are varied. As well as hearing sounds of various kinds—music, the sound of a dog barking, or the squeaking of a rusty hinge—we can hear the things that make these sounds—a violin; the dog and its barking (something it does); the door opening (an event). In addition, we can hear speech: someone speaking, the sounds they make, and what they say. Despite this variety, there is a unity to auditory perception: all auditory perception involves the perception of sounds, and whatever we hear that is not a sound we hear by hearing the sound it makes. An account of auditory perception must give some account of what it is to hear sounds and explain how we hear the things that make these sounds. At any moment you might hear a number of different sounds. You can focus your attention on any one of them.1 When you do so you attend to something—a sound—that has a characteristic appearance. For example, the bird song outside might appear high-pitched and quiet; the buzzing of the computer lower in pitch and louder.2 The properties of sounds that determine their appearance are the acoustic properties of sounds; they are the properties in virtue of which sounds appear the same or different to each other. It is usually supposed that sounds can appear the same or different along three dimensions, corresponding to three kinds of acoustic property—loudness, timbre,3 and pitch. In switching your attention between different sounds—between the bird song and the buzzing—you switch your attention between individual things that instantiate different acoustic properties. Sounds 1  You can attend to the sources of sounds as well as to the sounds they make, and I discuss this further in section 4. In this section my focus is on the character of the things you attend to when you attend to sounds—rather than to the sources of sounds. 2  We do not always experience sounds in this way. We sometimes experience many sounds at the same time, in such a way that we cannot distinguish individual sounds. Such examples don’t undermine the claim that auditory experience is fundamentally of individual sounds, any more than the fact that in vision we cannot always distinguish individuals—as, for example, we can’t distinguish the individual bricks when we look at a brick wall from a distance—undermines the claim that our visual experience is fundamentally of individuals things. 3  Timbre is a rather poorly defined property—it is sometimes said to be the dimension along which sounds that are of the same pitch and loudness can appear different. For example, notes of the same pitch

Audition   275 appear to be individual things in which acoustic properties inhere.4 They do not appear to be properties of material objects in the way that, say, colours appear to be properties of material objects; nor do they appear to be parts of material objects.5 Sounds take time. Some may be very brief; others may go on for hours or days. Individual sounds change over time—the same sound may appear low in pitch at one time and high in pitch at another time—and the identity of a sound is not fixed by how it is at any time, but depends on the way it unfolds over time. So sounds occupy time in a way similar to the way events and processes occupy time. Sounds—the things that we can pick out and attend to in our auditory experience—appear to be individuals that instantiate acoustic properties and which unfold over time in an event- or process-like way.6 Hume claimed that, in reflecting on our perceptual experience, we ‘always suppose the very images presented by the senses, to be the external objects, and never entertain any suspicion, that the one are nothing but representations of the other’ (Hume 1751: 118). Hume was making a claim about how our perceptual experience introspectively seems to us. In reflecting on my visual experience of the cup on my desk, it is the cup and its properties (its shape and colour) that seem present in my experience. So reflection on our visual experience reveals it as seeming to present the mind-independent objects and properties that we take ourselves to see. It seems that my visual experience could not be as it actually is were the cup and its properties not present. The same is not true of auditory experience. In reflecting on my auditory experience of a barking dog, it is the sound of the barking and its acoustic properties that seem present in my experience, rather than the dog and its properties. It seems that my experience could not be as it actually is were the sound not present, but could be as it actually is were the dog that I take to be making the sound not making it. So reflection on our auditory experience reveals it as seeming to present sounds and the properties of sounds, rather than the mind-independent objects making those sounds that we take ourselves to hear.7 The problem of auditory perception is to explain, given the character of auditory experience, how we perceive the things that make the sounds we hear. Berkeley’s answer was that, strictly speaking, we don’t hear those things: ‘when I hear a coach driving along the streets, all I immediately perceive is the sound; but from my past experience that such and such a sound is connected with a coach, I am said to “hear the coach”. Still, it is obvious that in played at the same loudness on a clarinet and on a trumpet appear different; this difference is due to their having a different timbre. I use ‘timbre’ as a name for an aspect of the way sounds appear. It might be suggested that timbre can be identified with a property of a sound wave or a vibration, for example, some aspect of its spectral composition. Such an identification would result from the discovery that there is an underlying physical property in virtue of which sounds are the same or different in timbre. The claim that sounds have timbre is independent of any such further discoveries. 4 

For an extended defence of this claim, see O’Callaghan (2007: ch. 2). Of course to say that sounds do not appear to be properties of, or parts of, material objects doesn’t mean that they are not; just that they don’t appear to be. Similarly to say that the appearance of sounds is determined by acoustic properties doesn’t mean that they don’t also have non-acoustic properties. 6  This gives us a reason to reject Kulvicki’s ‘stable property’ view of sounds (2008) according to which sounds are dispositions of objects to vibrate in response to stimulation. 7  Heidegger says: ‘We never really first perceive a throng of sensations, e.g., tones and noises, in the appearance of things . . . rather we hear the storm whistling in the chimney, we hear the three-motored plane, we hear the Mercedes in immediate distinction from the Volkswagen. Much closer to us than all sensations are the things themselves. We hear the door shut in the house and never hear acoustical sensations or even mere sounds’ (Heidegger 1935: 151–152). He goes on to say that ‘in order to hear a bare sound we have to listen away from things, divert our ears from them, i.e. listen abstractly’ (152). We should 5 

276   Matthew Nudds truth and strictness nothing can be heard but the sound, and the coach in that example is not properly perceived by sense but only suggested from experience’ (Berkeley 1713/1996, First Dialogue). Was Berkeley right to claim this?

2  The epistemic account We can draw a distinction between perceiving a particular object or property, and perceiving a fact about that object. I can see the cat—a particular object—and I can see that the cat is black—a fact about the cat. To perceive a fact is to be in a position to know that fact on the basis of what is perceived. It is possible to perceive a fact about an object by perceiving some other object, as when I see that the battery in my phone is flat without seeing the battery, by seeing the flat battery indicator.8 Applied to auditory perception, we might say that hearing the sounds made by a dog barking puts me in a position to know facts about the dog. For example, I can recognize the sound as the kind of sound made by a dog barking, so can come to know that there is a dog barking, and so can be said to ‘hear the dog’, that is, hear that there is a dog, by hearing the sounds it makes. The dog itself is not an object of perception, so Berkeley is right that ‘in truth and strictness’ I don’t perceive the dog, I perceive that there is a dog. As a general account of hearing the sources of sounds this is inadequate. On occasions on which I can hear that the dog is barking, we can distinguish between those on which I can hear the dog and those on which I cannot hear the dog. Suppose I hear that the dog is barking in virtue of hearing the sound of the dog barking. In that case, I can hear the dog. Suppose I hear that the dog is barking in virtue of hearing the baby crying (and knowing that the baby cries only when the dog barks) without hearing any sound made by the dog. In that case I cannot hear the dog. In both cases hearing a sound puts me in a position to know that the dog is barking, but in the former case I hear that the dog is barking by hearing the dog’s barking. If that’s right, then hearing the sound of the dog’s barking puts me in a position to hear the dog in a way that doesn’t simply consist in being in a position to know some fact about the dog by hearing a sound.

3  The deferred demonstrative account We can refer in talk and thought to something that we cannot directly perceive by exploiting its relation to something that we can directly perceive. Pointing at a set of footprints,

understand Heidegger’s claim that we ‘never first perceive a throng of sensations’ as a claim about auditory attention—that in many cases we attend to the things that make sounds rather than the sounds themselves. That is consistent with the claim that it introspectively seems that my auditory experience could not be as it actually is were the sounds not present, but could be as it actually is were the things that I take to be making the sounds not making them. 8 

Dretske calls this a form of indirect perception: ‘Perception is indirect when one sees a fact by seeing objects other than those constitutive of the fact—objects that indicate, by their properties or behaviour, the fact in question’ (Dretske 2010: 56).

Audition   277 I might say ‘that man must be a giant’. What I say is true just in case the man who made those footprints is a giant. Hearing the sound made by a dog puts me in a position to think deferred demonstrative thoughts about the dog. It might be suggested that hearing the dog just is being in a position to think deferred demonstrative thoughts about it on the basis of hearing the sounds it makes. What this suggestion amounts to will depend on what account we give of deferred demonstrative reference. Evans suggested that ‘that man’, said whilst pointing at a footprint, functions as a descriptive phrase along the lines of ‘the man whose foot made that footprint’.9 Applied to hearing, the suggestion would be that hearing the sound made by a dog puts me in a position to think ‘that’s a dog’.10 My thought is equivalent to ‘the thing that is making that sound is a dog’. It is a thought about whatever is making the sound, and is true just in case that thing is a dog. There is an obvious problem with this suggestion. In hearing the sound made by the dog I might think ‘that dog might have been fierce’. Prima facie, my thought is about a particular dog, that it might have been fierce; but, according to the suggestion, my thought is about how the sound might have been, that it might have been made by a fierce dog. So my thought about the dog, on the basis of hearing its bark, is not equivalent to the descriptive phrase. This problem is a problem with deferred demonstrative reference generally.11 One solution would be to treat the descriptive component as rigid, along the lines of ‘the thing that is actually making that sound, whatever it is, is a dog’.12 There may be other better solutions,13 but there is a more serious problem with the deferred demonstrative account. There certainly are contexts in which we are prepared to say that someone hears what makes a sound simply in virtue of hearing a sound that it makes, in the way described by the deferred demonstrative account. We are prepared to say, for example, of a record being played on a stereo, that we hear ‘that record’. But hearing the things that make sounds cannot generally be explained in that way. One way to bring this out is to draw a contrast between two kinds of case. Suppose that I hear a dog in virtue of hearing the sound of its barking, that is, I hear the barking sound and so am in a position to demonstratively think about the thing making that sound. In contrast, suppose that the sound of a dog barking is the ringtone for my phone. When my phone rings I hear the phone in virtue of hearing the barking sound that it 9 

He argues that in such cases ‘the identification is by description’ (1982: 199). hearing the sound made by a dog puts me in a position to think ‘that’s a dog’, it is not itself sufficient for me to think that: further background capacities are required. In particular, I must be able to recognize the sound as the sound made by a dog. I think the fact that the deferred demonstrative account of hearing a sound source requires recognition is problematic, but there is not space here for further discussion. 11  See Borg (2002). 12  This kind of view seems to be what Martin has in mind when he says: ‘In the case of audition, the primary objects of demonstrative identification are sounds, associated with phrases such as “that barking” or “that noise”. One may pick out the source of the sound via picking out the sound itself—we might then understand the demonstrative expression “that dog” as involving deferred ostension, perhaps as the descriptive phrase, “the dog which is actually the source of this sound” ’ (Martin 1997: 93). 13  In fact a ‘rigidified description’ approach is not adequate as an account of deferred demonstrative reference in general (Borg 2009). Someone determined to defend this view of auditory perception might argue that, whatever the correct account of deferred demonstratives, it can be applied to the case of hearing. 10  Although

278   Matthew Nudds makes, that is, I hear the barking sound and so am in a position to demonstratively think about the thing making that sound. The demonstrative account I have been considering treats these two cases in the same way: I hear the phone in just the same way that I hear the dog. But our intuitions are that the cases are different. The sound made by a dog’s barking seems naturally connected to the dog’s barking, it seems to ‘match’ or be appropriate to the barking, in a way that the sound when made by the phone is not.14 Made by the phone, the sound seems only arbitrarily connected to the phone; in fact, there seems to be something misleading or wrong about hearing that sound as made by my phone. It might be suggested that this sense of the sound being misleading is simply a consequence of the fact that I associate the kind of sound made by a dog barking with a dog and not with a telephone, and so am disposed to judge that whatever made the sound is a dog. This disposition would lead me to judge something false when the sound is made by the telephone, and that explains why hearing the sound made by the telephone seems misleading. There are two reasons for rejecting this suggestion. The first is that the sound seems misleading even when I know it is made by my telephone and I have no disposition to judge that it is made by a dog. The second is that mere association between a sound and what makes it, no matter how strong, is insufficient to explain the sense in which there is something non-arbitrary about the connection between hearing a sound and what produced it.

4  The representational account Part of what is involved in hearing the dog’s barking is that hearing the sound of its barking puts you in a position to hear it as a dog’s barking. The sound is not simply an effect of what made it, it is connected to what made it in such a way that it reveals or tells us something about the thing that made it. The reason the barking ringtone seems misleading is that what it seems to reveal or tell us is false. In a discussion of olfaction, Lycan suggests that olfactory experience has two kinds of content: ‘that smells represent adaptively significant environmental entities, and they also represent odors. In fact, they represent the environmental entities by representing odors. By smelling a certain familiar odor I also smell—veridically or not—a dog’ (Lycan 1996: 148). Whether or not this is true of olfaction, we can explain the relation between our experience of sounds and their sources by appealing to this kind of representational structure. It is plausible that our experiences of sounds represent adaptively significant sound sources as well as sounds,15 and that they represent sound sources by representing the sounds that they produce. If that’s right, we should think of auditory experience as having two kinds

14  My claim here is not a claim about recognition, but about something more fundamental. It could be that we come to recognize the phone on the basis of hearing the barking sound as well as, or better than, we can recognize the dog, but even were that the case there would be a difference between the way hearing the sound enables us to hear the dog and the way it puts us in a position to hear the phone. 15  By ‘adaptively significant sound sources’ I mean the sound sources that the auditory system evolved to perceive, and not, e.g., things that produce sounds by means of loudspeakers.

Audition   279 of content—as representing sounds and as representing the sources of sounds. So hearing a sound doesn’t simply put the perceiver in a position to think about the source of the sound, it involves an experience that represents the source as being some way. When the experience is veridical the perceiver hears the source as well as the sound. That would allow us to explain what is misleading about the sound of a dog barking when made by my telephone: in hearing the sound I have an experience that represents the barking of a dog; since there is no barking dog, the experience is non-veridical. In the rest of this chapter I spell out in more detail this representational account of auditory perception. I begin with the problem of explaining how auditory perception could represent sound sources. To understand that requires understanding, in outline at least, how sound sources produce sounds.

How sounds are produced Sounds are produced by anything that produces a suitable compression wave16 in an appropriate medium. Most commonly, sounds are produced by objects that have been caused to vibrate and whose vibration disturbs the surrounding air to produce a compression wave. Other kinds of events produce sounds too: the turbulent flow of air around an obstruction, or the sudden heating of air by a spark can produce compression waves in air, and we are familiar with the sounds—of the wind, of thunder, or the crackle of a spark—produced by these kinds of event. For the sake of simplicity I am going to consider only sounds produced by vibrating objects.17 We tend to conceive of vibrations as simple—as depicted by sine waves—but naturally occurring vibrations are never simple. For example, when a taut string is plucked it vibrates along its entire length with the maximum displacement occurring in the middle of the string. The wavelength of this vibration is twice the length of the string; its frequency18—the lowest frequency component of the string’s complex vibration—is known as the fundamental frequency of the vibration. The string also vibrates at three times the fundamental frequency, with a wavelength that corresponds to two-thirds the length of the string—imagine the string divided in three with each part vibrating—at five times the fundamental frequency—divide the string into five—and so on. These higher frequencies—the odd integer multiples of the fundamental frequency—are known as harmonics or partials of that fundamental. They correspond to the modes of 16  Since

it is possible to produce very high and very low frequency compression waves that are not detected by the ears, it is tempting to say that a compression wave must be produced that has a frequency within an audible frequency range. But that’s not quite true: because of the non-linearity of the transmission of high-frequency compression waves in air, an ultrasonic compression wave may produce an audible sound (see Westervelt 1963; Yang et al. 2005). The possibility of producing sounds in this way is a further reason to doubt the adequacy of the event view of sounds (discussed in section 4.1.1). 17 Sounds produced by vibrating objects are, arguably, fundamental to understanding auditory perception, and our experience of sounds produced in other ways can be explained in terms of our experience of such sounds. 18 The frequency with which the string vibrates is inversely proportional to the wavelength, and proportional to the velocity of the wave. The velocity of the wave depends on the tension of the string and its mass; changing the tension therefore changes the fundamental frequency of the vibration and the pitch of the sound produced.

280   Matthew Nudds vibration of the string, and it vibrates in all these modes simultaneously.19 Any complex vibration can be analysed into a number of simple harmonic frequency components. The vibration of an object can be analysed into frequency components corresponding to the object’s modes of vibration. We can think of the overall vibration of the string—or of any object—as the complex vibration that results from superimposing all its different modes of vibration.20 When an object is struck, the impact causes it to vibrate. Whereas a string vibrates in one dimension, objects vibrate in two or three dimensions. Objects therefore have many more possible modes of vibration than a string. Many things determine the character of an object’s vibration—the number and proportion of different frequency components and the way they change over time—including its physical properties. The size of an object will determine the lowest frequency at which it vibrates; its shape will determine the spectral composition of its vibration.21 The time it takes for the object to return to equilibrium is determined by how quickly it decays or loses energy (due to the internal friction of the object, and air resistance). Heavily damped vibrations decay rapidly, whereas lightly damped vibrations are prolonged. The speed and manner in which a vibration decays—damping may alter the spectral composition of the vibration over time—varies with the composition of the object.22 The nature of the event or process that causes the vibration also affects the character of the vibration. Whether the object was struck once or repeatedly, whether it was scraped or scratched, affects the time-course of the vibration. The force with which an object is struck affects the spectral composition as well as the amplitude of the resulting vibration (the relative intensity of the higher-frequency

19 

A plucked string will vibrate at a number of different frequencies, but not all of these frequencies will be present in equal amounts. Where along its length it is plucked will alter the harmonic structure of the overall vibration. If it is plucked in the middle, the first harmonic is strongly excited; however, the second mode of vibration vibrates around the centre point of the string, and is not excited. To excite the second mode, the string must be plucked off-centre at a position a quarter of the way from the end. Plucking in the middle excites the odd modes, and off-centre the even modes. So the relative harmonic structure of the vibration of the string will vary according to how it is plucked (and the resultant sound will vary—being ‘brighter’ or ‘stronger’). Plucked strings behave slightly differently to strings caused to vibrate by other means. For a detailed discussion, see Fletcher and Rossing (1998: sections 2.7–2.11). 20  Two sine waves can be combined to produce a complex waveform which is simply the result of summing the amplitudes of the two waves at each moment of time. Complex waveforms can be combined in the same way. Conversely, any complex waveform can be analysed into a number of component sine waves of various frequencies and amplitudes which, when added together in the correct phase, produce the analysed wave (this process of analysis is known as Fourier analysis. Details can be found in any textbook on vibrations (e.g. French 1971: chs 2 and 6). The complex compression wave that is detected by our ears instantiates a vibration that is equivalent to a set of phase related sine wave components of differing frequency and amplitude. Many psychoacoustic theories suppose that the auditory system must perform some equivalent of a Fourier analysis. 21  Differences in geometrical properties of objects affect the way that they vibrate, but they do so only in virtue of being correlated with the mechanical properties of those objects; it is likely that the auditory system detects or tracks mechanical rather than geometrical properties of objects. Further work needs to be done to discover which mechanical properties the auditory system detects or tracks. See Carello et al. (2005: 14 ff). 22  How much damping different materials produce very clearly affects the character of the sound an object makes when it is struck—for example, wood, which is heavily damped, makes a thunking sound, whereas metal, which is less damped, rings.

Audition   281 harmonics of a vibration typically increases when an object is struck with greater force).23 Finally, what, if any, parts of the object are held fixed or are touching other objects and surfaces, affects the character of its vibration by changing which modes of vibration are excited and how they are damped. Because the character of the object’s vibration is determined by the nature of the object and its interactions with other objects, the vibration embodies or carries information about the object and its interaction with other objects. When a vibrating object is immersed in a suitable medium, its vibration produces a compression wave in that medium. As this compression wave propagates through the medium it interacts with and is reflected by objects and surfaces in the environment, and the reflections propagate in turn. These reflections carry information about objects and obstructions in the environment, so the compression waves that reach the ears carry information about the environment, in addition to information about the objects that produce sounds. All this information is, in principle at least, recoverable.24

The event view of sounds An obvious question to ask is what the relation is between sounds and objects’ vibrations. I have described objects and vibrations as producing sounds, but a number of writers have defended the view that sounds are identical to, or supervene on, objects’ vibrations. Casati and Dokic, for example, defend an event view of sounds, according to which ‘sounds are monadic events happening to material objects. This means that sounds are located at their sources, and are identical with, or at least supervene on, the relevant physical processes in them’ (Casati and Dokic 2011: 98).25 O’Callaghan defends a similar view: ‘particular sounds are events . . . [they] are the events in which a medium is disturbed or changed or set into motion in a wave-like way by the motions of bodies. Events such as collisions and vibrations of objects cause the sound events’ (O’Callaghan 2007: 36). Both think sounds supervene on a vibratory event or process occurring in an object, but they differ as to whether sounds should be identified with that event or process, or with the event that is that vibratory event disturbing the medium, and so over whether the existence of sounds depends on the presence of a medium.26 In what follows that difference won’t matter. If the event view of sounds is correct, then it would be reasonable to suppose that an account of auditory perception should begin with an explanation of how we perceive 23 

See Chowning (1999: 270). How much information the auditory system can recover is an unresolved empirical question. For a useful survey of the relation between an object’s properties and the way it vibrates, see Lufti (2008). The information is, in principle, recoverable and, although relatively little systematic research has been done, it is evident that our auditory system can recover at least some of it: it is evident, that is, that we can perceive the sources of sounds and their properties. This was first emphasized by Gaver (1993a, 1993b). For a recent survey of much of this evidence, see Carello et al. (2005). The mechanisms involved in the perception of sound source location are relatively well understood (for a survey, see, e.g., Schnupp et al. 2011: ch. 5; Blauert 1997). There have been a number of studies that show that we can perceive spatial properties of the environment in which sounds are produced (e.g. Rosenblum et al. 2000; Rosenblum et al. 2005, 2007). 25  See also Casati and Dokic (2004, 2011). 26  It is a consequence of O’Callaghan’s view that sounds do not exist in a vacuum: a tuning fork struck in a vacuum vibrates, but makes no sound because there is no medium to be set in motion; Casati and Dokic think that striking the tuning fork makes a sound, but that in a vacuum the sound cannot be heard. 24 

282   Matthew Nudds sound events, and that an explanation of our perception of the sources of sounds should be in terms of the perception of those sound events. In fact, the direction of explanation goes the other way. It is not possible to explain how we perceive sounds other than in terms of our perception of the sources of sounds.

Sound events and sound-producing events If you simultaneously scratch and tap a reverberant rough surface such as the aluminium cover of a laptop, you will hear two sounds: the scratching sound, which lasts as long as your scratching, and the almost instantaneous sound of the tap. If the tap occurs half way through the scratching, then you will hear the sound of the tap at the same time as you hear the sound of the scratching. The two sounds are produced simultaneously by a single object (the lid of the laptop). If you tap a reverberant object, such as a metal lampshade, twice in quick succession you will hear two successive sounds. Tapping the shade produces a ‘ringing’ sound that begins almost instantaneously and then fades away gradually (but still quite quickly). If the taps are close enough together, you will hear the second sound begin before the first sound has quite faded away. The two successive (but overlapping) sounds are produced by a single object (the lampshade). In reflecting on these examples we should be careful to distinguish the putative sound events (the events that the event view says are identical to sounds) from the events that cause them. The tap on the lampshade is a brief, almost instantaneous, event; that tap causes the lampshade to ‘ring’—to vibrate—for several seconds; the sound we hear begins with the tap, but lasts as long as the vibration—the ‘ringing’—lasts. The tap on the lampshade is not the putative sound event; it causes the sound event—the ‘ringing’ vibration of the shade. The scratching and the tapping on the laptop cause the lid to vibrate; that vibration dies away almost instantaneously, but the sounds we hear result from the vibration of the lid caused by the scratching and tapping. The tapping and scratching events are not the putative sound events; they are events that cause the putative sound events. We can call such causing events ‘sound-producing events’. In both of the examples we hear two sounds. According to the event view, we perceive two sound events occurring in the object (or setting the medium into motion). How might these events be individuated, that is, what makes it the case that the objects each instantiate two vibratory events, rather than a single event (or more than two events)? When an object makes a sound, the object goes from a state of equilibrium; it vibrates for a period of time; and eventually it returns to a state of equilibrium. So there is a vibratory event or process that begins when the object goes from a state of equilibrium and ends when the object returns to a state of equilibrium. This event can be individuated relative to the object to which it occurs: it is the event that consists in the object undergoing a vibratory process. The two examples show that putative sound events cannot be understood in this way. In the first example, the object goes from a state of equilibrium when it is scratched and returns to a state of equilibrium only after the scratching has stopped. The tap occurs during this time, when the object is already vibrating. If the sound-event is the event that begins when the object goes from a state of equilibrium and ends when the object returns to a state of equilibrium then the object undergoes a single vibratory event that comprises

Audition   283 both the scratching and the tapping. In the second example, the second tap occurs before the ringing produced by the first tap has died away. So the object goes from a state of equilibrium when it is first tapped and returns to a state of equilibrium only after the ringing produced by the second tap has died away. So, again, if the sound-event is the event that begins when the object goes from a state of equilibrium and ends when the object returns to a state of equilibrium, then the object undergoes a single vibratory event that comprises both taps. Since in both cases we hear two sounds, the sounds we hear are not identical with vibratory events individuated in this way. Can we say that the objects instantiate two vibratory events because they instantiate two vibrations—that we can individuate two vibrations qua vibrations? I described how a vibrating object has many modes of vibration. There is no intrinsic connection between these different modes other than that they are modes of vibration of the same object.27 That means that we can represent the object’s vibration as the complex vibration produced by the superimposition of the frequency components corresponding to all of its excited modes of vibration, or we can equally well represent it simply as a list of individual frequency components corresponding to each mode, or as some combination of components. The difference is just a difference in the way we represent the ways in which the object is vibrating. There is no physical basis for grouping frequency components, considered as such, in one way rather than another, and so no physical basis for individuating two vibrations instantiated by the object, rather than a single complex vibration or many individual modes of vibration. We can’t individuate sound events in terms of objects’ vibrations or in terms of vibrations as such; but we can individuate sound events in terms of what caused or excited the different modes of vibration of an object. In both examples the modes of vibration were caused or excited by one or other of the two sound-producing events. We can pick out modes of vibration according to which of the sound-producing events caused them, and can individuate two complex vibrations made up of the frequency components corresponding to the modes picked out in this way. That gives us a way—in fact, the only way—of individuating two vibrations, and hence two putative sound events, instantiated by the objects. If that’s right then two things follow. The first is that there can be no way of saying what sounds there are other than in terms of what produced them,28 and hence no account of our perception of sounds that is independent of an account of our perception of sound-producing events. The second is that an account of auditory perception needs to explain how the auditory system can track sound-producing events in a way that doesn’t depend on tracking sounds. In the following sections I provide such an explanation and show how it supports the representational account of auditory perception.

27  If a vibrating string is touched, the modes of vibration for which the point of touching is a node will continue unaffected whilst those for which it is an anti-node will be damped (French 1971: 167); in principle one can change the amplitude of any one mode without affecting the others; this demonstrates that modes of vibration are independent of one another (French 1971: 196). There may, however, be non-linearities in the movement of parts of an object so that the different modes of vibration are not strictly independent of each other, but that won’t help solve the individuation problem. 28  Or what would normally produce them: there may be cases of hearing a sound that was not in fact produced by a sound-producing event. I discuss such cases below.

284   Matthew Nudds

Transmission and detection If you are in proximity to a vibrating object the compression wave it produces will affect your ears. The ears function to capture and filter29 the compression wave, and to detect the different frequency components that compose it.30 The output of each ear is a state31 that encodes time-varying information about each of the frequency components that make up the vibration instantiated by the compression wave that reaches the cochlea or inner ear. The proximal state that encodes this signal serves as the input to subsequent auditory processing. Although the output of each ear is a proximal state that encodes the detected pattern of frequency components over time, what we hear are individual sounds and their sources. That means the auditory system must transform the proximal state into a representation of putative sound events and their causes. Doing that involves solving what is, in effect, an auditory under-determination problem. The compression wave that reaches our ears is normally produced by many different things—different sounds events—simultaneously producing sounds. The compression waves produced by these events interact with each other—in constructive or destructive ways, in ways that may obscure of mask frequency components—with surrounding objects, and with the medium, so that the compression wave that reaches and is detected by the ears is, at any moment, the result of the additive combination of the compression waves produced by all the putative sound events occurring in the immediate environment. Compression waves reflected (and re-reflected) by surrounding objects and surfaces reach the ears fractionally later in time than the original, and reflections of the early part of a compression wave may combine and interfere with later parts of that wave, so that the compression wave that reaches and is detected by the ears is temporally ‘smeared’. There is nothing intrinsic to a particular frequency component that marks it as having been produced by one sound event rather than another, and nothing intrinsic to each of a set of components that marks it as having been produced by a single sound event simultaneously with other components. There is no one-to-one mapping of the pattern of frequency components detected by the ears onto the sound events that produced them. That pattern is consistent with any number of different sound events: any number of different events could have produced the pattern of frequency components detected by the ears. How, then, does the auditory system work out what sound events actually produced the 29 

Parts of the ear function in a mechanical way to alter the sound wave. In particular, parts of the outer ear (or pinna) function to differentially reflect certain frequency components in such a way that the compression wave is selectively amplified and filtered. The filtering plays a role in sound source localization, particularly in determining the elevation of the source, and whether it came from in front of or behind the listener. The ear canal has characteristic resonances that, again, selectively filter the compression wave. See Rosowski (2010). 30  Having passed through the ear canal, the filtered compression wave passes into the cochlea, and causes the basilar membrane to vibrate. Different parts of the basilar membrane vibrate in response to different frequencies; its surface is covered with sensitive hairs, organized in such a way that different hairs move in response to different frequency components of the compression wave. The displacement of these hairs produces activity in associated neurons. For a survey, see Hackney and Furness (2010); a good introduction to the way the ear functions is Schnupp et al. (2011: ch. 3). 31  It would be more accurate, given the dynamic nature of this output, to characterize it as a process. It is common practice to refer to the stages in perceptual processing as states and for ease of exposition I will do the same.

Audition   285 frequency components it detects? It does so by working out what sound-producing events would best explain the pattern of frequency components detected by the ears.

Representing sound-producing events The auditory system transforms the pattern of frequency components encoded by the proximal state into a representation of sound-producing events and it does this by, in effect, making assumptions about the way the frequency components were produced and transmitted. For example, the vibration that is produced by a single sound-producing event—by the striking of an object, say—is made up of frequency components that will normally be related to each other in specific ways. The frequency components will share temporal properties, in particular they will start at the same time and be phase related; they will tend to change in similar ways over time, decaying steadily and at a similar rate; they will be harmonically related; new frequency components will not suddenly appear (and existing components will not suddenly disappear). It is unlikely (but not impossible) that frequency components produced by distinct sound-producing events will be related in these ways. Therefore, if frequency components detected by the auditory system are related in these ways, they are likely to result from a single sound-producing event. The auditory system can exploit this fact to determine what sound-producing events are occurring at any time: given the particular pattern of frequency components detected, the auditory system produces a representation of the sound-producing events that would best explain that pattern.32 If the frequency components were produced normally, then the best explanation will be correct, and the representation will be veridical. The frequency components of vibrations that result from a sequence of soundproducing events—from footsteps, for example—or from a temporally extended event or process—from a car engine, a dog barking, or a scratching—will be related to one another over time in characteristic ways. For example, a sequence of sound-producing events—such as footsteps—will produce frequency components with similar harmonic and temporal structures; and a temporally extended event or process—such as a scratching—will produce a vibration whose components will change in ways that are constrained by the nature of the event or process—they will not radically or suddenly change in their harmonic structure, for example. The auditory system can exploit these relationships amongst frequency components to determine how sound-producing events are related over time: given the pattern of frequency components detected over time, the auditory system produces a representation of the temporally extended sound-producing events, and sequences of sound-producing events, that would best explain that pattern. Again, if the frequency components were produced normally, the representation will be veridical. In some cases, the production of a representation of sound-producing events draws on knowledge of the properties of object or event that produced the sound. Such top-down processes are likely to be involved in the perception of many temporally structured events, 32  That the auditory system tends to treat components that share these features as having a single source has been experimentally demonstrated. My discussion here draws on Bregman (1990: esp. ch. 3), to which the reader should refer for details of the empirical support for the claims in the text. See also Schnupp et al. (2011: ch. 6).

286   Matthew Nudds including those involved in speech, and in the perception of familiar ‘meaningful’ sounds, such as the sound of a dog’s barking, of footsteps, of a car’s engine, and so on.33 The assumptions made by the auditory system about how the frequency components it detects were produced will, in normal circumstances, lead to veridical representation of the sound-producing events that in fact produced them. But sometimes the assumptions are false. In such cases, the representation of sound-producing events is non-veridical—a misrepresentation. For example, if two sound-producing events of the same kind involving the same kind of objects occur at exactly the same time, the resulting compression wave will have frequency components that stand in relationships to each other that normally only obtain between components that result from a single sound-producing event. The auditory system will assume that the frequency components were normally produced by a single sound-producing event, and so will (mis)represent the existence of a single event rather than two events. During the evolution of the auditory system, such errors are likely to have been rare. But with the advent of artificially reproduced and recorded sound, they have become commonplace. Stereo loudspeakers provide a familiar example of how the assumptions made by the auditory system can be false. Two loudspeakers—playing, for example, a recording of a drum being struck—produce a pattern of frequency components that stand in relationships to each other that would normally obtain only if they were produced by a sequence of sound-producing events involving a single object. The auditory system assumes that they were produced normally, and the resulting state (mis)represents the existence of a sequence of events—drum beats—involving a single object, when in fact the frequency components resulted from rather different events involving two distinct objects—two loudspeakers.34 The result is that we seem to hear a drum being struck. Stereo loudspeakers work because they produce a pattern of frequency components that would normally result from a number of different sound-producing events—from different musical instruments being struck, plucked, bowed, and so on. The auditory system assumes these frequency components were produced in the normal way and so (mis)represents the sound-producing events from which they would normally have resulted, rather than the events involving the loudspeakers that, in fact, produced them with the result that we seem to hear several different musical instruments being played.35

Representing sounds I have described how the auditory system transforms the pattern of frequency components it detects into a representation of sound-producing events. What is the connection 33  Bregman calls this kind of top-down process ‘schema-based segregation’ (1990: 395 ff., 665 ff.); it contrasts with what Bregman calls ‘primitive stream segregation’ (1990: 38 ff.) which is bottom-up and exploits invariable properties of the subject’s environment. 34  One of the most striking features of stereo loudspeakers is that the sounds they produce seem to come from locations in space at which there is no sound source. There is disagreement about how this phenomenon should be interpreted and what its significance is for the nature of sounds (for opposing views see, O’Callaghan 2007: ch. 3 and Nudds 2009). This disagreement turns on what account should be given of the spatial content of auditory experience. For a survey of empirical work relevant to such an account, see Blauert (1997). 35  Musical instruments playing together can produce similar kinds of misrepresentation or illusion. See Ball (2010) for examples of how composers have exploited this fact in producing musical works.

Audition   287 between this representation of sound-producing events and the sounds we experience? In transforming a detected pattern of frequency components into a representation of sound-producing events, the auditory system determines what sound producing events are likely to be responsible for those frequency components. In doing this it ‘allocates’ the frequency components it detects to these different events. If, for example, the auditory system produces a representation of two sound-producing events it allocates all the frequency components it detects to one or other of these events, and if it allocates a component to one event it cannot allocate it to the other. This makes good ecological sense. There must be some event that is responsible for each of the frequency components detected by the ears; so any representation of purported sound-producing events will be incorrect if the events represented cannot explain all the frequency components detected. One way to think about the transformation of the pattern of frequency components into a representation of sound-producing events is as a process that works out how best to allocate frequency components to sound-producing events: how many events are required in order to explain all the detected frequency components, where the explanation is constrained by the requirement that components allocated to the same event should be related to one another in ways that make it likely that they were produced by that event. Our experience of sounds is the result of the way frequency components are allocated to sound-producing events. How many sounds we experience is determined by how many sound-producing events the detected frequency components are allocated to; how those sounds appear is determined (at least in part) by the particular combination of frequency components allocated to each sound-producing event. For example, if all the frequency components detected by the auditory system are allocated to a single sound-producing event, then we experience a sound corresponding to that sound-producing event. The appearance of the sound is (at least partly) determined by the frequency components allocated to that source. If those same frequency components were allocated to two sound producing events,36 then we would experience two sounds corresponding to the two sound-producing events. The appearance of each sound would be different to the appearance of the single sound because each sound would result from a different combination of frequency components. The transformation that produces a representation of sound-producing events therefore results in an experience of sounds corresponding to the represented sound-producing events. The evidence for this is empirical. By manipulating the properties of frequency components we can manipulate the way they are transformed by the auditory system to produce representations of different sound-producing events, and so change the number and character of the sounds a listener experiences corresponding to those events.37 An illustration of some of the phenomena that I have been discussing is provided by the patterns of tone glides—sounds that change in pitch in a linear fashion—that result from hearing different groups’ frequency components. Suppose a rising tone glide is played at the same time as a falling tone glide. They can be experienced in one of two ways: either as 36 

This could be done by, e.g., introducing a spatial separation between the two groups of components and by introducing a temporal offset between them. 37  See Bregman (1990). Many of the same principles apply to grouping in music, and Diana Deutsch (1999) describes several examples of how changes in grouping change what musical sounds, and what sequences of musical sounds, are heard.

288   Matthew Nudds a sound that rises and then falls in pitch and a sound that falls and then rises, so that they seem to approach one another and then ‘bounce’ apart; or as a sound that steadily rises in pitch and a sound that steadily falls in pitch, so that they appear to ‘cross’ one another. In both cases the auditory system represents two sound-producing events. By altering the combinations of frequency components, it is possible to alter the way they are allocated to the sound-producing events. In the simplest case, the two tones are each made up of a single frequency component, one that increases in frequency whilst the other decreases. These result in an experience of sounds that ‘bounce’. The auditory system assumes that frequency components that are closer in frequency are likely to be the result of the same sound-producing event. However, if some harmonic structure is added to one of the tones, so that it comprises a fundamental and two higher harmonics, the result is an experience of sounds that cross. The auditory system assumes that frequency components with the same harmonic structure are likely to have been produced by a single event, and those with different harmonic structure are very unlikely to have been produced by a single event even when they are closer in pitch. There are many variations on this simple experiment,38 all of which can be explained in accordance with the same principles. The auditory system represents the sound producing events that best explain the frequency components detected by the ears, the frequency components are allocated to these events, and the sounds we experience are the result of this allocation. The result, when things go as they should, is that the auditory system veridically represents the sound-producing events in our environment and we experience sounds corresponding to these events: sounds that were in fact produced by them. This account of how auditory perception functions supports the representational view, according to which auditory perception represents sound sources by representing the sounds they produce. However, it suggests a significant revision to that view. What sounds we experience is a consequence of the way the auditory system transforms the frequency components it detects into a representation of sound-producing events. So the claim that auditory perception represents sound sources by representing the sounds they produce should not be understood as a claim about explanation, that is, that we can explain how we perceive sound sources by appeal to our perception of sounds.39 The auditory system does not represent sounds and then determine what produced those sounds; it determines what sound-producing events there are and that process results in an experience of sounds corresponding to those events. In fact, we saw that sound events cannot be individuated independently of sound-producing events, so there could be no explanation of sound perception other than in terms of the perception of sound-producing events. This reverses the 38  See, e.g., Tourgas and Bregman (1985); more complex examples (e.g. Tourgas and Bregman 1990; Kanafuka et al. 1996) can be explained in the same way. 39 Do sounds carry information about their sources? The pattern of frequency components that determine an experience of a sound carries information about the source of the sound. That pattern also determines the acoustic appearance of the sound. However, the processes that exploit the information carried by the frequency components that correspond to the sound don’t have as their input the acoustic appearance of the sound; they directly exploit the information in the pattern of frequency components. So we shouldn’t think of the sound and its appearance as playing any fundamental explanatory role. That’s not to deny that there may be cases in which the acoustic appearance of sounds does play an explanatory role—e.g. cases in which we recognize the source of the sound in virtue of recognizing a sound with a certain appearance as typically produced by a source of that kind—but such cases are not fundamental to understanding auditory perception.

Audition   289 order of explanation implicit in many accounts of auditory perception, and it makes clear that the function of auditory perception—what auditory perception is for—is not the perception of sounds, but the perception of the sources of sounds—the ecological significant events in our immediate environment that produce sounds.

Further questions I have given an outline of the representational view of auditory perception, but many further questions remain. One important question concerns the way sound-producing events are represented by the auditory system. What does it represent them as? Does it represent them merely as events, with no further capacity to keep track of the thing to which the event occurs, or does it represent them as events occurring to persisting material objects, that is, not just as a striking event, but as a striking of an object of a certain kind. Whether auditory perception represents objects as such will depend, first, on what conditions must be satisfied for something to be represented as an object (see the entry on Object Perception) and, second, on whether auditory perception satisfies those conditions. Both questions are difficult to answer. We might argue that material objects are spatio-temporally continuous, solid, bounded, and cohesive; so for something to be represented as an object is for it to be represented as having these properties. There are no acoustic equivalents of these properties, but the claim is not that sounds are represented as spatio-temporally continuous, solid, and so on, but that the things that produce sounds are. The auditory system cannot represent something as solid and bounded in the way the visual system does, by representing its surfaces or volumetric shape, but there appear to be auditory analogues of at least some of the properties that are constitutive of object representation. For example, we experience distinct sounds grouped together as sequences of sounds in virtue of having the same source. Only sounds that are likely to have been produced by the same object are grouped, so the auditory system must keep track of the particular object that makes the sounds. Similarly the auditory system appears sensitive to whether a sound source maintains its cohesiveness over time: think of the difference between hearing a bottle drop to the floor and bounce, and hearing it break into pieces; it is plausible that the difference is in how the source of the sound is represented, as a single object that bounces or as an object that breaks into smaller objects. These examples suggest that the auditory system could track properties that are constitutive of being an object, and hence that auditory perception could represent the sources of sounds as objects.40 If auditory perception does represent events as occurring to objects, what properties does it represent those objects as having? I described the range of information about sound

40 

Furthermore, there is some evidence that information about sound sources is bound together into a representation—an ‘auditory object’ file—that functions in auditory perception in a way that is similar to the way object files function in visual perception (Zmigrod and Hommel 2009). This is not an area that has been much investigated and in some discussions the ‘auditory objects’ that files are constitutive of or are taken to be sounds, rather than the sources of sounds, and so evidence is sought that there are object files that track sounds and their features whereas what we want is evidence that there are object files that track the sources of sounds and their features.

290   Matthew Nudds sources and the environment that is embodied in the pattern of frequency components detected by the ears, information about the object’s size, shape, material composition; about how it interacts with other objects; about the spatial properties of the environment; and so on. This information is recoverable in principle but, with the exception of some spatial properties, little is known about what information the auditory system in fact recovers and how it does so. These are all questions that need further empirical and philosophical investigation.41

5  The nature of sounds If the representational view is correct and our experiences of sounds and sound-producing events are related in the way that I have described, what follows about the nature of sounds? What are sounds, on the representational view? In producing a representation of sound-producing events, and allocating the frequency components it detects, the auditory system produces an experience of sounds corresponding to those sound-producing events. In the example of the laptop lid that was scratched and tapped, the auditory system represents two sound-producing events—the scratching and the tapping—and it allocates the frequency components it detects to one or other of these events, with the result that we experience a sound corresponding to each of them. It might be suggested that, in experiencing those sounds, what we experience is the vibratory events occurring in the laptop lid that are caused by the scratching and the tapping. Or, in other words, that our experience of the two sounds is an experience that represents two vibratory events caused by the scratching and the tapping. If that’s right, then the representational view is consistent with the event view of sounds; it might even be taken to support the event view. But what should we say about the sounds we apparently experience when we undergo non-veridical experiences of sound-producing events? For example, two loudspeakers playing a recording of a drum being struck produce an experience that misrepresents a sequence of sound-producing events—drum strikes—involving a single object—a drum; in fact, the frequency components detected by the ears were produced by different kinds of events involving two distinct objects—two loudspeakers. We seem to hear a drum being struck when there is no drum, and so experience a kind of illusion—an illusory experience of the source of the apparent sounds. What about our experience of the apparent sounds, is that illusory too? According to the event view, in experiencing the apparent sounds of the drum we experience what seems to be a sequence of vibratory events occurring in the drum. Since there are no such events, the experience is non-veridical: the apparent sound events we experience—the sounds—don’t exist. We seem to hear sounds when there are none. That is a very counter-intuitive consequence of the event view. It would mean that whenever we listen to music played on loudspeakers many of the sounds that we appear to hear don’t in fact exist. It would also mean that apparent sounds produced in ways 41 

For a good introduction to the perception of sound source location, see Schnupp et al. (2011: ch. 5). For evidence concerning the perception of non-spatial properties of sound sources, see the references in footnote 20. For a discussion of sound source recognition, see McAdams (1993); Peretz (1993).

Audition   291 that don’t involve a vibrating object, and so don’t involve a vibratory event of the right kind—sounds produced by sparks, by air or water turbulence, and so on—don’t exist. There is an alternative way to understand sounds that is consistent with the representational view. In claiming that auditory experiences represent sound sources by representing the sounds they produce, the representational view allows the possibility of experiences that veridically represent sounds, but misrepresent the sources of those sounds. That allows us to say that in loudspeaker cases our experience of the sounds is veridical—there really are sounds that we hear—but they seem to have been produced by something—the drum—that did not in fact produce them. Our auditory experience is therefore partly veridical:  it veridically represents the sounds, but misrepresents what produced them. Since the sounds we hear in such cases are not events occurring to objects, it follows that the event view of sounds is mistaken. On this alternative view, sounds are caused or produced by events occurring in objects, but are not identical to them.42 Which of these two different conceptions of sounds is correct depends, in part, on what we say about loudspeaker cases and other similar cases. If the apparent sounds we experience in those cases are veridical then the event view is false; if they are not veridical then the event view is plausibly true.43 But more needs to be said about both views of sounds, and about how each can accommodate a variety of auditory phenomena. For example, proponents of the event view point out that, since sounds seem to be spatially located at their sources, the alternative view is committed to claiming that sounds don’t spatially appear as they actually are.44 Conversely, proponents of the alternative view point out that the sounds we hear often appear different to the way they would have appeared had we heard them in close proximity to their sources—a sound heard at some distance from its source will be quieter, it may be distorted, and so on—the event view is therefore committed to claiming that many sounds we hear don’t acoustically appear as they actually are.45 Whether either view can satisfactorily deal with these and other problems remains to be seen. 42 

There is more than one way to develop the alternative view. One might argue that sounds should be identified with vibratory events instantiated by the compression wave, or one might reject physicalism about sounds, as Scruton (2009) does. Scruton’s paper contains many insights, and his view is generally amenable to the view of auditory perception set out in this chapter. 43  Arguably, my description of the function of auditory perception supports the claim that our experience of apparent sounds in loudspeaker cases is veridical. The sounds we experience are a consequence of the way detected frequency components are allocated to the events represented. When the events represented don’t exist—when they are misrepresented—the detected frequency components are allocated to misrepresented events. But that process of allocating detected frequency components functions in the same way it would have done had the events not been misrepresented. The detected frequency components actually exist, and there are patterns or relationships between them that are of the same kind as would have obtained had the events not been misrepresented. So the pattern of frequency components that is sufficient for the existence of a sound in the veridical case obtains in the non-veridical case too. Furthermore, there are no mechanisms that function to represent distal, rather than proximal, frequency components—i.e. those components actually instantiated by an object. The auditory system uses the frequency components it detects to construct a representation of the distal sound-producing events, not of the distal vibration that event causes. 44  Both O’Callaghan (2007) and Casati and Dokic (1994, 2009) appeal to the apparent spatial location of sounds to argue against the wave view; a defender of the wave view may reject the premiss that sounds seem to be located at their sources (see, e.g., Nudds 2009). 45  This kind of consideration is not decisive. Colours appear differently in different circumstances of viewing, but the way a colour appears can change consistently with the colour that appears remaining unchanged.

292   Matthew Nudds

References Ball, Philip (2010). The Music Instinct: How Music Works and Why We Can’t Do Without It. London: Bodley Head. Berkeley, George (1713/1996). Three Dialogues between Hylas and Philonous, ed. Howard Robinson. Oxford: Oxford University Press. Blauert, Jens (1997). Spatial Hearing: The psychophysics of human sound localization. Cambridge, MA: MIT Press. Borg, Emma (2002). 'Pointing at Jack, Talking About Jill: Understanding Deferred Uses of Demonstratives and Pronouns'. Mind & Language, 17, 489–512. Bregman, Albert S. (1990). Auditory Scene Analysis: The perceptual organization of sound. Cambridge, MA: MIT Press. Carello, C., Wagman, J. B., and Turvey, M. T. (2005). 'Acoustic specification of object properties'. In J. D. Anderson and B. Fisher (eds), Moving Image Theory: Ecological Considerations (pp. 79–104). Carbondale, IL: Southern Illinois University Press. Casati, Roberto and Dokic, Jérôme (1994). La philosophie du son. Nimes: Editions Jacqueline Chambon. Casati, Roberto and Dokic, Jérôme (2009). 'Some Varieties of Spatial Hearing'. In M. Nudds and C. O’Callaghan (eds), Sounds and Perception (pp. 97–110). Oxford: Oxford University Press. Casati, Roberto and Dokic, Jérôme (2011). 'Sounds'. In Edward N. Zalta (ed.), The Stanford Encyclopedia of Philosophy (Summer 2011 Edition). . Chowning, John (1999). 'Perceptual fusion and auditory perspective'. In P. R. Cook (ed.), Music, Cognition, and Computerised Sound (pp. 261–276). Cambridge, MA: MIT Press. Deutsch, D. (1999). 'Grouping mechanisms in music'. In D. Deutsch (ed.), The Psychology of Music (pp. 183–248). London: Academic Press. Dretske, Fred (2010). 'What we see'. In Bence Nenay (ed.), Perceiving the World (pp. 54–67). Oxford: Oxford University Press. Evans, Gareth (1982). The Varieties of Reference. Oxford: Clarendon Press. Fletcher, Neville H. and Rossing, Thomas D. (1998). The Physics of Musical Instruments. 2nd edn. New York: Springer. Freed, D. J. (1990). 'Auditory correlates of perceived mallet hardness for a set of recorded percussive events'. Journal of the Acoustical Society of America, 87, 311–322. French, A. P. (1971). Vibrations and Waves. New York: W. W. Norton & Company. Gaver, W. W. (1993a). 'How do we hear in the world? Explorations in ecological acoustics'. Ecological Psychology, 5, 285–313. Gaver, W. W. (1993b). 'What in the world do we hear? An ecological approach to auditory event perception'. Ecological Psychology, 5, 1–29. Hackney, Carole M. and Furness, David N. (2010). ‘Hair bundle structure and mechanotransduction’. In P. Fuchs (ed.), The Oxford Handbook of Auditory Science: The Ear (pp. 231–258). Oxford: Oxford University Press. Heidegger, Martin (1935). 'The Origin of the Work of Art'. In D. Farrell Krell (ed. and trans.), Martin Heidegger: Basic Writings. New York: Harper and Row, 1977. Hume, David. (1751). An Enquiry Concerning Human Understanding, ed. Tom L. Beauchamp. Oxford: Oxford University Press, 1999. Kanafuka, Kyoko, Tanaka, Shunsuke, Nakajima, Yoshitaka, and Sasaki, Takayuki (1996). 'An Illusory Transfer of Temporal Gaps Between Crossing Tones'. The Journal of the Acoustical Society of America, 100(4), 2683–2683.

Audition   293 Kulvicki, John (2008). 'The Nature of Noise'. Philosophers' Imprint, 8, 11. . Lufti, Robert (2008). 'Human Sound Source Identification'. In William Yost, Arthur Popper, and Richard Fay (eds), Auditory Perception of Sound Sources (pp. 13–42). New York, Springer. Lycan, William G. (1996). Consciousness and Experience. Cambridge, MA: MIT Press. McAdams, Stephen (1993). 'Recognition of sound sources and events'. In S. McAdams and E. Bigand (eds), Thinking in Sound (pp. 146–198). Oxford: Oxford University Press. Martin, M. G. F. (1997). The Shallows of the Mind. Proceedings of the Aristotelian Society Supplementary Volume 75, 75–98. Nudds, Matthew (2009). 'Sounds and Space'. In M. Nudds and C. O’Callaghan (eds), Sounds and Perception (pp. 69–96). Oxford: Oxford University Press. O’Callaghan, Casey (2007). Sounds: A Philosophical Theory. Oxford: Oxford University Press. Peretz, Isabelle (1993). 'Auditory agnosia: a functional analysis'. In S. McAdams and E. Bigand (eds), Thinking in Sound (pp. 199–230). Oxford: Oxford University Press. Rosenblum, Lawrence D. and Robart, Ryan L. (2005). 'Hearing Space: Identifying Rooms by Reflected Sound'. The Journal of the Acoustical Society of America, 117(4), 2562–2562. Rosenblum, Lawrence D. and Robart, Ryan L. (2007). 'Hearing Silent Shapes: Identifying the Shape of a Sound-Obstructing Surface'. Ecological Psychology, 19(4), 351–366. Rosenblum, Lawrence D., Gordon, Michael S., and Jarquin, Luis (2000). 'Echolocating Distance by Moving and Stationary Listeners'. Ecological Psychology, 12(3), 181–206. Rosowski, J. J. (2010). ‘External and middle ear function’. In P. Fuchs (ed.), The Oxford Handbook of Auditory Science: The Ear (pp. 49–92). Oxford: Oxford University Press. Schnupp, Jan, Nelken, Israel, and King, Andrew (2011). Auditory Neuroscience: Making Sense of Sound. Cambridge, MA: The MIT Press. Scruton, Roger (2009). 'Sounds as secondary objects and pure events'. In M. Nudds and C. O’Callaghan (eds), Sounds and Perception (pp. 50–68). Oxford: Oxford University Press. Tougas, Y. and Bregman, A. S. (1985). 'The crossing of auditory streams'. Journal of Experimental Psychology: Human Perception and Performance, 11, 788–798. Tougas, Y and Bregman, A. S. (1990). Auditory streaming and the continuity illusion. Perception and Psychophysics, 47, 121–126. Westervelt, Peter J. (1963). 'Parametric Acoustic Array'. The Journal of the Acoustical Society of America, 35(4), 535–537. Yang, Jun, Gan, Woon-Seng, Tan, Khim-Sia, and Er, Meng-Hwa (2005). 'Acoustic Beamforming of a Parametric Speaker Comprising Ultrasonic Transducers'. Sensors and Actuators A: Physical, 125(1), 91–99. Zmigrod, S. and Hommel, B. (2009). 'Auditory event files: Integrating auditory perception and action planning'. Attention, Perception & Psychophysics, 71(2), 352–362.

Chapter 16

Touch FrÉdÉrique de Vignemont and Olivier Massin

Since Aristotle, touch has been found especially hard to define. The idea of a privileged relation between touch and the body, however, has remained mostly unchallenged. This has led many physiologists, psychologists, and philosophers to subscribe to a bodily view of touch, according to which the mark of touch, by contrast to other senses, is to be related to the body in some specific way. Here we shall consider the relation between touch and bodily awareness from two different perspectives. On the one hand, we shall discuss the body template theory of touch according to which touch is defined by the fact that tactile content matches proprioceptive content. We shall contrast the body template theory with other theories of individuation of touch. We shall then expose in detail the difficulties and advantages of defining touch by its proper object, and more particularly pressure. On the other hand, we shall discuss the body map theory according to which tactile sensations are localized within the frame of reference provided by the mental representation of the space of the body. We shall oppose it to the Local Sign theory and review of series of putative counterexamples to the body map theory.

1 Defining touch Touch is more elusive than hearing, sight, smell, or taste. The problem is not that the biological underpinnings are less well known for touch than for other senses. The problem is rather conceptual:  what we mean by touch is not always sharply delineated. What is the explanandum of the various biological, psychological, and psychophysical theories of touch? There is little consensus. Here we shall examine several candidates, including intentional objects, organs, stimuli, and the relation to the body.

The impalpable nature of touch The problem was first noted by Aristotle (De Anima, 1961: 422b17–424a16). Sensory modalities, Aristotle argued, are to be individuated by their intentional object. The difficulty,

Touch   295 however, is that prima facie tactile objects do not constitute a natural class. Let us call the proper object of a sensory modality, the entity that we directly perceive through this modality only; and its primary object, the entity that we directly perceive through it as a matter of necessity (Sanford, 1976). It is classically assumed that the primary and proper objects of sight, hearing, taste, and smell are respectively colour, sound, taste, and smell. The proper objects of touch, however, are at first sight too heterogeneous to constitute a natural class and to be its primary objects. Hardness, solidity, impenetrability, texture, weight, mass, pressure, tension, contact, temperature, humidity, vibrations, painfulness, ticklishness, wetness, and so forth have all been claimed to be perceived by touch exclusively. Yet, it is highly unlikely that they all belong to the same natural class. Furthermore, it is dubious that each instance of tactile perception necessarily involves the perception of some hardness, vibration, temperature, texture, and so forth. Faced with the heterogeneity of the proper objects of touch, one may renounce defining touch by its intentional objects. Suppose one endorses a biological criterion instead. On this view, sensory modalities are individuated by their proper and primary organs. Touch is then defined by the biological apparatus (including organs, tissues, and receptors) that is specific and necessary to it. The skin is often put forward as the proper and primary organ of touch. But it is neither. First, the skin is not only a perceptual organ, it also accomplishes several other functions that we do not want to include in tactile perception (including protection, heat regulation, perspiration, and respiration). Second, tactile perception does not essentially involve skin stimulation. Touch on the eyes, on mucous membranes such as the mouth, on internal organs, and on the teeth can induce tactile sensations as well. One might even conceive that skinless creatures (such as arthropods) have tactile sensations. Looking for more specific organs, one faces an impressive anatomical and functional diversity of receptors involved in touch (Johnson, 2001). Some are dedicated to the perception of mechanical properties such as pressure, vibration, and texture, some to the perception of tissue damage, some to the perception of temperature. Each sub-group is itself heterogeneous. The mechanoreceptors include the Meissner and Pacini corpuscles, Ruffini organs, Merkel disks, and free nerve endings. All are anatomically and functionally heterogeneous: their location in the skin varies (some in the dermis, some in the epidermis, some—the Ruffini organs—are even also found in the joints); their activation threshold and adaptation rates differ (Vallbo and Johansson, 1984; Kandel et al., 2000: 438); they are innervated by different kinds of fibres, myelinated or not. They are classified as cutaneous mechanoreceptors only insofar as they allow us to be conscious of mechanical properties and/or they respond to mechanical stimuli.1 But if we need to appeal to intentional objects or stimuli to individuate organs, then the organ criterion for individuating the senses is not fundamental (Roxbee-Cox, 1970). Is there then any sui generis kind of physical stimuli that is necessarily and exclusively involved in tactile perception? Mechanical properties are sometimes put forward, but they may be also involved in other sensory modalities. Arguably, chemical property involved in olfactory and gustative perception, electromagnetic property involved in sight, and acoustic property involved in hearing are all kinds of mechanical properties. Besides, electromagnetic and chemical properties are also involved in tactile perception. It is actually unclear that physics—be it Newtonian, relativistic, or quantum mechanics—provides 1 

By contrast to intentional objects, one is not necessarily aware of physical stimuli in perception.

296    Frédérique de Vignemont and Olivier Massin us with any categorization of physical properties that matches the categorization of our senses. Whether one relies on intentional objects, organs, or stimuli, our ordinary concept of touch appears on closer inspection to be a rag-bag sense that scatters in many sub-senses.2 Such a multi-sensory view of touch, so to speak, is implicitly assumed in many textbooks, but it is hardly satisfying (Fulkerson, 2011). First, it is hardly a conception of touch since it boils down to dissolving its explandum into a disjunction of senses. Second, it is a strongly revisionary view, which is committed to dismissing ordinary intuitions about touch, intuitions about the privileged role of touch in our access to external reality, or intuitions about the particular relation between touch and bodily awareness. These intuitions, however, may help us to reach some definition of touch that preserves its unity, as illustrated by the bodily theories of touch.

Bodily theories of touch Even if sight and hearing are defined by their objects, organs, or stimuli, things may go differently for touch. The main proposal of this type relies on the common observation that tactile perception always involves some experience of our own body in combination with the experience of external objects. In other words, touch is bipolar (Katz, 1925). One can then take a second step and argue that the specific nature of touch consists in its relation to bodily awareness. This view can be tracked back at least to the Middle Age philosopher P. J. Olivi (see Yrjönsuuri, 2008) and more recent proponents include Armstrong (1962), O’Shaughnessy (1989, 2003), and Martin (1992, 1993). Touch, by contrast to other senses, would enter into a specific dependence relation with the awareness of our body. A difficulty for this approach is to specify the exact relation between touch and bodily awareness. For it has been proposed that proprioception plays a role in every sensory modality (O’Dea, 2011). In order for bodily theories of touch to work, other senses must not depend on bodily awareness in the same way. The most refined proposal to date is that bodily perception functions as a template for tactile perception. In a nutshell, our body is geometrically congruent with the external objects so that the awareness of our body gives us access to the spatial properties of the external object. For instance, Armstrong (1962: 18) argues that we perceive convex objects by feeling concavities of our flesh. The relation to bodily awareness is even more striking for so-called haptic touch. According to O’Shaughnessy (2003: 629, 656–680) and Martin (1992, 1993), we perceive the circularity of an object by feeling the circularity of the motion of our hand around it through proprioception. Template theories of touch, however, face three objections. First, they can account for the perception of spatial properties such as shape, but they fail for the perception of other properties such as weight, pressure, and solidity. One way to go, which is endorsed by Armstrong (1962: 21–32), is to claim that all tangible properties are reducible to spatial ones. Such a spatial reduction of tangible properties, however, clashes 2 

It is not clear that any other criterion (including qualia, sensori-motor contingencies, and associated beliefs) or combination of criteria for distinguishing the senses fares better as far as the unity of touch is concerned.

Touch   297 with the fact that felt weight or pressures might vary independently of any felt spatial variation (a fact later recognized by Armstrong, 1997: 97–98). Second, it is true that the content of tactile perception can be congruent with the content of bodily awareness. It is also true that this does not generally occur in other sensory modalities. But it can happen (Scott, 2001). For example, in tunnel vision (in which peripheral vision is completely lost, so that vision is restricted to a narrow, tunnel-like, central field), the visual content is congruent with the proprioceptive content: the motion of our eyes matches the shape of the explored object (e.g. Loomis et al., 1991). Third, even if one grants that tactile content, and only tactile content, is necessarily congruent with proprioceptive content, it is still doubtful that such congruence is essential to touch. As we shall explain in section 2, we sympathize with the general hypothesis according to which touch exhibits some specific relation with bodily awareness. However, we do not think that this is an essential property of touch. Our main worry with the bodily template theories of touch is that they entail that touch is not a sense in the same way as sight, hearing, smell, and taste. Touch is distinct from all the other senses because of its privileged relation to proprioception. But sight, hearing, smell, and taste are not distinct from each other because of their relation to proprioception. The criterion for distinguishing senses ends up being disjunctive. One version of it could be the following: x is a sensory modality if and only if either x has a natural class of proper and primary objects or x uses proprioception as a template. This threatens the very unity of the concept of sensory modality. Advocates of the template theories of touch save the unity of the concept of touch, but at the price of dismantling the concept of sense.3 The bullet seems to us hard to bite. A good definition of touch should not only explain the unity of touch; it should also account for its sensory character: touch is one single sensory modality like any other. To recap, once faces the following dilemma when defining touch:  either one gives up the intuition that touch is a single sense by claiming that it is a heterogeneous collection of senses; or one gives up the view that touch is a sensory modality by claiming that its unity-maker—for example its relation to the body—is distinct in kind from the unity-makers of other sensory modalities. Either touch is not a sense, or touch is not a sense. Hopefully, this dilemma is not inescapable.

The pressure theory of touch Following the intentionalist approach of the individuation of the senses, which distinguish sensory modalities by their proper and primary objects, we propose to revive the hypothesis that touch is the direct perception of pressure and tension. This view was introduced by Weber in 1846, right at the beginning of experimental psychology: [Touch] is essentially a sense of force. Our concepts of force would be very much less well developed were we unable to feel pressure, or to sense competing forces in which an equilibrium is established so that no movements are produced, yet in which the forces can still be felt. (Weber, 1846: 196) 3  They indeed declare themselves quite pessimistic about the concept of sensory modality (Martin, 1992: 215; O’Shaughnessy, 2003: 630).

298    Frédérique de Vignemont and Olivier Massin Pressure, we shall now argue, is the proper and primary object of touch. On the one hand, pressure cannot be directly seen or heard, although its causes and effects can. On the other hand, one can never perceive by touch without feeling some pressure or tension. Touch is by nature the direct perception of pressure and tension. In other words, pressure is the colour of touch. Weber’s proposal of a pressure sense was confirmed by the discovery of pressure spots on the skin (Blix, 1884; Goldscheider, 1884; Donaldson, 1885), which were soon associated with sui generis end-organs by Von Frey (see Boring, 1942; Norrsell et al., 1999; Pearce, 2005). On both phenomenological and physiological grounds, the existence of a pressure sense is hardly controversial. Such a pressure sense, however, has remained mostly unnoticed by philosophers (at the exception of Armstrong, 1993: 97–99; 1997: 213; Sanford, 1976; Perkins, 1983: 242ff; Fales, 1990: 16). One reason for this neglect might be the Cartesian and Humean prejudice against dynamic entities such as force, pressure, or tension. If touch is to be the perception of pressure and tension, then there must be such things. Here, we shall assume that macroscopic force and tension are real physical entities, distinct from their kinematic effects, rather than mere theoretical fictions (see Wilson, 2007 and Massin, 2009 for arguments in favour of realism about Newtonian forces). In order to know what touch is, one thus needs to venture into the metaphysics of pressure: what are they? Let us first consider forces. Forces are entities of dynamics. Dynamics is to be contrasted with kinematics (and not with statics). Kinematics describes motion; dynamics causally explains motion. Consequently, dynamics entities, including forces, mass, and energy, do not consist in motion but causally contribute to motion.. According to Newton’s second law of motion, forces cause accelerations of the bodies they exert on, when not counteracted by other forces. A body submitted to the influence of a single force, such as a body in free fall, is not subject to any pressure. The reason is that nothing prevents that single force from causing the acceleration of the body. Such a solitary force acting on a body is not perceptible by touch. In order for a force to be perceptible, it has to be counteracted by another force, which is often the one exerted by our body, such as when we weight an object. Only then do pressure and tension occur. Pressure, as noted by Weber, only arises when two forces act against each other and cancel each other. The same is true for tension. In order to put a stick under pressure, one needs to exert two inward forces on each of its extremities. In order to put a rope under tension, one has to exert two outward forces on each of its extremities. Pressure and tension are pairs of antagonist forces. Pace Humeans, the pressure exerted on a body might vary without it moving or undergoing any other spatial change. Likewise, one might feel an object steadily pressing on our body even when it no longer moves relative to the body (Perkins, 1983: 248). This remains true even when tactile adaptation occurs. To claim that touch is the sense of pressures and tensions amounts to a claim that the proper and primary object of touch is a pair of antagonist forces. If the pressure theory of touch is not to fall under the same criticism as the template theory, then one must assume that other sensory modalities are defined thanks to their proper and primary object as well. This assumption might be found a bit too bold in the context of the contemporary debate about sensory individuation (MacPherson, 2011). Surely more needs to be said in defence of the intentionalist criterion of individuation of the senses (see for example, Roxbee-Cox, 1970; Sanford, 1976; Brentano, 1979; Dretske, 1995; Ross, 2001, 2008). Suffice it to say here that the present pressure theory of touch dismisses one of

Touch   299 the main objections raised against the intentionalist criterion since Aristotle, namely the alleged lack of a proper object for touch. It has to be granted, however, that such a proposal entails that touch in the strict sense does not cover all the types of perception traditionally ascribed to it. In particular, the perception of temperatures and of pain needs to be excluded from touch proper. Temperature is indeed not pressure (though it might depend on pressure) and there is no promising way of grounding the perceptual awareness of temperature on the perceptual awareness of pressure. We feel that the temperature in the shade is cold without feeling the shadow or the air pressing on us. Temperature is not felt through pressure. Second, it might be that pain is a sensory quality, on a par with colour, sound, pressure, and temperature (as first clearly stated by Stumpf, 1928). Here again, pain might be felt independently of pressure. Plausibly, the main reason why the temperature sense, the pain sense, and the pressure sense have been fused into one single sense is that temperature, pain, and pressure are often felt in the same location, namely, in parts of our body (Weber, 1846:  69; Mill, 1869: vol. 1, 30; Brentano, 1995: 83). Despite this felt co-location, they have intentional objects of different categories, independent from each other. Touch in the generic sense therefore splits into the sense of pressure (touch in the strict sense), the sense of temperature and, possibly, the sense of pain. The pressure theory of touch, however, does not amount to a sheer giving up of the unity of touch, for the splitting of touch ends up there. There is no need to introduce further senses of texture, vibration, weight, contact, hardness, or solidity. The tactile perception of each of those actually depends on the perception of pressure and tension. There is no sui generis sense of texture distinct from the sense of pressure, for we feel the texture of a surface by feeling a spatio-temporal pattern of pressure when stroking it. There is no sui generis sense of vibration (pace Katz, 1925), for we feel vibration by feeling a temporal pattern of pressure on our body. There is no sui generis sense of weight for we perceive the weight of an object by perceiving the various pressure and tension that it exerts on our skin and muscles when wielding it. There is no sui generis contact sense for we feel that some object is in contact with our body by feeling that it presses, be it very slightly, on our skin. There is no sui generis sense of hardness for we feel that a body is hard by perceiving that pressing or pulling it does not change its shape. There is no sui generis sense of solidity or impenetrability for we feel that a body is impenetrable, by perceiving that pressing our body against it does not yield compenetration of our body with it. No texture, no vibration, no weight, no instance of contact, hardness, or impenetrability is ever tactually felt independently of the feeling of a pressure or tension. Such qualities are indirectly perceived on the basis of the direct perception of pressure. One might object that such a claim commits its proponents to some questionable atomistic approach to sensory psychology, according to which felt texture, weight, or vibration are either reducible to a summation of isolated sensation of pressure or inferred on the basis of the prior perception of such isolated pressure. This is not the case. On the one hand, pressures, on the basis of which other qualities are felt, are not necessarily spatially punctual or temporally instantaneous entities. There might be pressure-Gestalts that expand over space and time. On the other hand, the notion of indirect perception does not necessarily imply that what is indirectly perceived is reducible to, nor inferred from, what is directly perceived. Our claim is only that texture, vibration, weight, and solidity are perceived in virtue of the perception of pressure, and not the reverse (Jackson, 1977: 19–20). To

300    Frédérique de Vignemont and Olivier Massin say that only pressure and tension are directly perceived in touch is not even to claim that they are the most salient properties in tactile perception. Attention might go directly to the constant and distal properties of the felt bodies. But even if we focus on hardness, weight, or texture, these are tactually accessed thanks to the possibly sub-attentional consciousness of pressure and tension. Felt pressure and tension are the tactual matters that underlie all tactual perception (Katz, 1935: §5).

The objectivity of touch Defining touch as the sense of pressure and tension therefore allows defining touch in the same way as other sensory modalities while saving a substantial part of its unity. This is only one of the advantages of the pressure theory of touch. In addition, the theory can account for two specific features commonly ascribed to touch that cry for explanation. The first might be called the objectivity of touch. The second is the aforementioned bipolarity of touch. As echoed in the use ‘tangible’, which often means ‘real’, touch appears to enjoy some priority over the other senses when we wonder about the reality of external bodies. This priority should not be understood in terms of a higher reliability of touch with respect to other senses. This priority is rather phenomenological: only tangible objects can be presented to us as real, that is, as existing independently of us. This may seem controversial. For example, Siegel (2006) argues that ordinary visual experiences do represent the independence of their objects from the subject. When seeing an object, one expects that changing one’s perspective on it will not change its location.4 However, Siegel’s theory explicitly targets property-independence, that is, the fact that perceptual objects are presented as exemplifying properties (such as location) independently of us. By contrast, the present proposal targets existential-independence, that is, the fact that we are sometimes presented with the fact that the perceived object exists independently of us. It is only with respect to existential-independence that touch has some privilege over other senses. The point is not that other sensory modalities present us with their object as existentially dependent on us. Our claim is only that ordinary perception is mute with respect to the mind-independence of the existence of its objects. One reason for this is that independence from the subject is both a self-reflexive and modal (or essential) notion, and that it would be quite a heavy task for ordinary perception to present it on the top of its immediate objects. Touch, to that extent, plays an essential role in the phenomenology of self–world dualism. The pressure theory of touch paves the way for a neat explanation of tactile objectivity, as follows: P1 The feeling of physical effort (and of resistance) is the only experience that presents us with the existential independence of the physical world from us. P2 Touch is the only sensory modality essential to the feeling of physical effort. 4  But is it mind-independence or rather perspective-independence (independence from a point of view, which needs not be presented as a subject’s point of view) that is presented here? Alternatively, one might challenge Siegel’s claim that such expectations are parts of the perceptual content itself, rather than aroused by a simple perceptual content.

Touch   301 C Touch is the only sensory modality essential to the experience of the physical world as existing independently from us.

Following a long tradition, we shall assume the truth of P1.5 In order to ascertain P2, we need to clarify the nature of physical effort. An agent makes an effort on a body if and only if it exerts a force on that body so as to make it move (or stay at rest) and that this body exerts in return some opposite force on the agent (i.e. resistance). Given that pressure and tension are pairs of antagonist forces, this entails that there is no physical effort without pressure or tension. Likewise, there is no experience of physical effort without experience of physical pressure or tension. Since experiences of tension and pressure are, on our account, tactile experiences, then tactile experiences, and only them, are essential to the feeling of effort. In other words, in order to feel the resistance of the external world, we have to be aware that it exerts some force counteracting the force we are intentionally exerting on it. Such pairs of counteracting forces are the proper and primary objects of touch. If so, no being completely deprived of touch could ever experience the existential independence of physical bodies. This is not to say that tactile perception per se presents us with the mind-independence of its objects. It does not. Experience of effort does. It essentially involves tactile perception but it is not essential to tactile perception. This is neither to claim that ordinary perception, including tactile perception, does not present us with mind-independent objects. It does. But ordinary perception does not present us with their mind-independence per se. Our point is only that touch is the only sense required for the experience of the mind- independence of physical bodies. There lies its greater objectivity. On the present proposal, the objectivity of touch results from the nature of touch together with the nature of physical effort. As it stands, such an account of tactile objectivity is still a rough sketch. Though we are confident that more can be said in support of this approach, our present point is quite modest. Any good theory of touch should either account for tactile objectivity or explain it away. The pressure theory of touch, contrary to its rivals, suggests some fairly promising explanation of tactile objectivity. A second feature of tactile perception that needs to be accounted for by any theory of touch is its aforementioned bipolarity, that is, the fact that every instance of tactile perception presents us not only with external objects, but also with our body. Here again, the pressure theory of touch provides some natural explanation. On the pressure theory of touch, the bipolarity of touch follows from the nature of touch together with a likely hypothesis: pressure felt in touch is generally pressure between (parts of) our body and some external object. Pressure is a symmetrical relation: we feel something pressing on this part of our body or that this part of our body is pressing on something. The reason why touch is bipolar is that its proper and primary objects are relations, whose relata are generally parts of our body and external objects. This is not so with sound or colour: though their location might be perceived as a relation to our own location, their very sound- or colour-quality is not presented in terms of a relation between something and part of our 5  See e.g. Maine de Biran (1820); Müller (1842: 1080); Brown (1846: 151); Bain (1868: 82); Mill (1869); Baldwin (1906); Stout (1931: Bk IV, chs 1 and 6); Katz (1935: 8; 1925: 51); Garnett (1965); Scheler (1973a: 135–138; 1973b); Hamlyn (1990); Baldwin (1995); Russell (1995); Bermudez (1998: 164); Cassam (1999); Smith (2002); Williams (2002: 136); Matthen (2005: 8); Dilthey (2010).

302    Frédérique de Vignemont and Olivier Massin body. The bipolarity of touch results from the relational character of its proper and primary object. This explanation of tactile bipolarity relies on the hypothesis that pressure felt in touch consists in a relation between (parts of) our body and some object, and consequently, that felt pressure is always located on our body. This may come as a surprise: if pressure is to touch what sound is to hearing, how is it that contrary to sound, pressure is always felt to be located on our body?

2  Localizing tactile sensations When we have tactile sensations, we normally experience pressure being applied on a specific part of our body. We localize tactile sensations on our body, so to speak.6 We can then form a belief about the felt location of the sensations and report it (e.g. ‘I feel pressure on the shoulder’), attend to it (e.g. by looking at the shoulder), indicate it (e.g. by pointing to the shoulder), and act accordingly (e.g. by removing the object that is touching the shoulder). Furthermore, tactile sensations are bodily sensations. One way to interpret this claim is in spatial terms. At first sight, it seems indeed that the localization of tactile sensations is always within the boundaries of our body. However, in the same way as for other types of bodily sensations, the localization of tactile sensations displays puzzling features, which do not seem to be captured by typical treatments of spatiality. Most discussions in the literature have focused on the use of ‘in’ when one localizes pain. For instance, the pain that I feel in my leg is not felt in the refrigerator when my leg is in the refrigerator (Coburn, 1966). Nor is the pain that I feel in my thumb felt in my mouth when my thumb is in my mouth (Block, 1983). But one can raise similar questions if one considers the use of ‘on’ when one reports ‘I feel pressure on my hand’. For instance, I feel a light pressure on my hand; my hand is on my head; yet, I do not feel pressure on my head. The rule of spatial transitivity is not preserved. The peculiarities of the localization of tactile sensations might cast doubt on their intrinsic spatiality. But if tactile sensations are not essentially spatial, then their privileged relation to the body might disappear. They may be localized on the body most of the time, but there would be no necessity to it. The relation would be only contingent. Here, we shall determine whether tactile sensations are essentially localized and what constraints, if any, lay upon the localization of tactile sensations.

Are tactile sensations essentially spatial? How are tactile sensations individuated? Are they individuated exclusively by the type and intensity of the pressure exerted on the body? Or are they also individuated by their felt location? Let us consider, for example, that pressure of the same intensity is applied on two distinct parts of the body inducing two tactile sensations. Are the two sensations strictly

6  This is not to say that we localize the mental event of the tactile sensation in our body. Rather, we localize what we tactually feel, that is, the physical event of pressure.

Touch   303 similar or not? In other words, is the spatial component a constitutive part of tactile sensations or is it independently acquired and then associated to tactile sensations? The latter view has been defended by the Local Sign theory in the nineteenth century (Lotze, 1888; Wundt, 1897; James, 1890; Titchener, 1908), which can be summarized as follows: the theory which denies that there can be in a sensation any element of actual locality, any tone as it were which cries to us immediately and without further ado, ‘I am here’, or ‘I am there’. (James, 1890: 798)

On this view, the localization of tactile sensations is not grounded on spatial information carried by tactile signals. Rather, it is grounded on the sensation of a ‘peculiar qualitative colouring’ associated with each sensible nerve (Wundt, 1897) or of an ‘auxiliary impression’ of the specificity of the flesh that is touched (Lotze, 1888). However, this view has been widely criticized because of the lack of such local signs (Vesey, 1961; Coburn, 1966; O’Shaughnessy, 1980; Martin, 1995). It is true that the density of tactile receptors is not the same over the surface of the body. At the phenomenological level, there can be differences between specific parts of the body (such as the bony forehead and the plump cheek). But there is no non-spatial difference between two sites that are slightly apart on the same body part or between two similar sites on both sides of the body (left and right hands, for example). Furthermore, it is one thing to notice a difference—if there is any—between a sensation on the back of the hand and a sensation on the palm. It is another thing to localize this sensation on the palm. Specific qualitative colouring may be the natural sign of the body site of the sensation: its experience cannot but betray the location of the sensation in the same way that smoke cannot but betray the presence of a fire. Nonetheless, the presence of smoke needs to be interpreted on the basis of prior know­ ledge about the systematic association between smoke and fire. Likewise, the experience of a specific qualitative colouring needs to be interpreted on the basis of prior knowledge about the systematic association between the local sign and a specific body site. In particular, proponents of any theory that denies that tactile sensations are spatially individuated are committed to posit a purely contingent relation between tactile sensations and their spatial ascription, which seems unlikely (Brentano, 1979; Coburn, 1966; Holly, 1986). In addition, they are committed to accepting the possibility of floating tactile sensations, that is, tactile sensations with no apparent location. But can one feel tactile sensations without at least roughly ascribing the sensations to particular parts of the body? This might be possible in the case of bodily feelings, like thirst or hunger (Armstrong, 1962). However, in the case of touch, it seems difficult, if not impossible, to conceive feeling pressure nowhere in particular.7 As argued in section 1, pressure is a dynamical relation 7  Even

young infants, who cannot communicate the localization of their sensation, can orient their attention toward the location of pressure, as shown by a range of neonatal reflexes (such as automatically grasping unseen objects in contact with the hand) and voluntary behaviours (such as looking toward the bodily site that is touched, cf. Bremner et al., 2008). Furthermore, as far as we know, there has been no report of patients who experience sensations that are not at least approximately localized. Head and Holmes (1911: 139) describe the case of a patient who claims: ‘I feel you touch me, but I can’t tell where it is’. But the patient further reports: ‘The touch oozes all through my hand’. This indicates that tactile localization can be more or less indeterminate. It can also be more or less accurate. But this is not to be confused with tactile sensation with no localization whatsoever.

304    Frédérique de Vignemont and Olivier Massin between bodies, and bodies necessarily occupy regions of space. Consequently, experiencing pressure is experiencing pressure in a region of space. This seems to be confirmed by some recent empirical results that show that there is a systematic relationship between tactile sensations and localization. On the one hand, the more spatially determinate the sensation is, the less it takes time to experience pressure. For instance, it was found that we are faster to detect pressure when viewing the body part that is touched (Tipper et al., 2001). One possible explanation is that viewing the touched body part facilitates the localization of pressure, thus allowing pressure to be experienced more quickly. On the other hand, the less spatially determinate the sensation is, the more it takes time to experience pressure. For instance, it was shown that when we are uncertain about the location of pressure (because of a conflict between visual information and proprioceptive information, for instance), we take longer to detect pressure (Moseley et al., 2008; Folegatti et al., 2009). Roughly speaking, one has no sensation as long as one cannot localize where the pressure is applied. Taken all together, these results suggest that to be aware of pressure is to be aware of a location of pressure, though not necessarily correctly. The location of pressure is part of the individuation conditions of tactile sensations (Margolis, 1966; Brentano 1979). To conclude, the spatiality of tactile sensations may be peculiar compared to the spatiality of visual experiences for instance, but this does not suffice to show that tactile sensations are not essentially spatial. Rather, it invites us to deepen our understanding of tactile spatiality.

The body map theory Contrary to the Local Sign theory as described by James, tactile sensations immediately cry to us ‘I am here’. However, to be aware that one is touched here is one thing, and to locate where ‘here’ is is another thing. A cross indicating ‘you are here’ is of little interest if there is no reference point on the map. Likewise, tactile localization requires a spatial frame of reference, which is classically defined as ‘a locus or set of loci with respect to which spatial position is defined’ (Pick and Lockman, 1981: 40). In the case of tactile sensations, the set of loci is not provided by tactile signals themselves. If I am touched on the right index finger, the peripheral neural signal originating from the right index finger carries information about the location of the stimulation only. It does not carry the information that this is an index finger, or that it is located on the right hand. Thus, it does not suffice to fully account for the spatiality of tactile sensations. For tactile sensations to get a relatively rich and accurate spatial content, raw spatial tactile signals need to be interpreted with the help of a topological and geometric mental map of the body (hereafter body map). The notion of body map can be tracked back to Bonnier (1905), who first introduced the notion of schema to refer to the spatial organization of bodily sensations. Head and Holmes (1911) also posit the existence of what they call a superficial schema, which is the model of the skin surface of the body used for localizing bodily sensations. More recently, Schwoebel and Coslett (2005) argue in favour of what they call a body structural description, which is impaired in patients suffering from autotopagnosia. These patients are not able to correctly localize where they are touched and accurately identify the parts of their body. But it is O’Shaughnessy (1980, 1995) who best develops the body map theory. He

Touch   305 postulates the existence of a long-term body image in order to compensate for the intrinsic insufficiency of the body senses. The long-term body image explains how all bodily experiences share the same spatial content of the structural shape of the body over an extended period. It is thanks to the body map that tactile sensations are experienced as being at more than at an isolated body point. The body map plays a structural role in spatially shaping tactile sensations. In other words, it plays the role of a somatosensory field (Martin, 1992). In visual experiences, visual properties are localized relative to the visual field. In tactile experiences, pressure is localized relative to the body map. Although the body map represents long-term properties of the body such as the size of the limbs, it is flexible and can quickly adjust to changes. The body boundaries that we experience can actually stretch beyond the biological body boundaries to include either non-physical extension like phantom limbs or physical extensions like tools (Vesey, 1961; O’Shaughnessy, 1980; Martin, 1995). For example, the felt size of our limbs can be temporarily altered for the time of acting with a tool (Cardinali et al., 2009). Once the action achieved and the tool dropped off, the body map readjusts to the normal size of the body. We may think of it in terms of a rubber band: we can stretch it as much as we want but it always comes back to its default size. It is worth noting here that the body map does not necessarily represent the agentive body (space of the body used in action). Phantom limbs can be paralysed. Furthermore, the body map can be altered by the mere distorted vision of one’s static limbs (Taylor-Clarke et al., 2004).8 The body map is neither necessarily the representation of the affective body (space of the body where one can feel pleasure and pain). We indeed use a spoon to stir the pot of boiling soup, rightly unafraid to feel pain in the spoon. Nor does the body map necessarily include a self-referential component (space of one’s body qua one’s own). Indeed, unlike phantom limbs, we do not feel tools as our own body parts. As Botvinick (2004: 783) noted, ‘the feeling of ownership that we have for our bodies clearly does not extend to, for example, the fork we use at dinner’. If tactile localization were constrained by the felt boundaries of the body, as represented by the body map, rather than by the biological limits of the body, then one should be able to feel sensations in phantom limbs, in tools or in any other object represented within the body map. Amputees can experience sensations in their phantom limbs: ‘A draught of air on the stump produces the feeling of a draught on the [phantom] foot’ (James, 1887: 258). Likewise, they can experience sensations in their prosthesis: ‘That may sound strange, but to me, my prosthesis is an extension of my body. I can actually “feel” some things that come into contact with it, without having to see them’ (Murray, 2004: 970). It has been found that even individuals with a healthy and complete body can feel sensations in a prosthetic rubber hand (Botvinick and Cohen, 1998). In the Rubber Hand Illusion, healthy subjects sit with their left arm resting on a table, hidden behind a screen. They are asked to fixate on a rubber hand in front of them, and the experimenter simultaneously strokes the participant’s hand and the fake hand with two paintbrushes. After a short while, the majority of participants report that they feel the touch of the paintbrush at the location where they see the rubber hand being touched: ‘It seemed as if I were feeling the

8 

For a multimodal view of the body map (and of bodily experiences), see de Vignemont (2014).

306    Frédérique de Vignemont and Olivier Massin touch of the paintbrush in the location where I saw the rubber hand touched’ (item 3 in the questionnaire in Botvinick and Cohen, 1998: 756). Finally, one can experience sensations at the tip of a tool like Descartes’ blind man with his cane, if the body map has incorporated the cane. Sensations in tools are not a mere way of speaking (Lotze, 1888; Katz, 1925; Gibson, 1962; Martin, 1993; O’Shaughnessy, 2003). First, localization in tools appears as direct. One does not need any training before feelings sensations in tools. The first time blind people use their cane, they immediately feel the solidity of the floor in the cane. Moreover, one could not have a fine-grained control over the tool if one was aware only inferentially that the pressure and the forces were located in the tool: ‘Pen and brush would be clumsy instruments in the hand of the clerk or the painter, if we did not directly feel their contact with the paper’ (Lotze, 1888: Bk V, 588–589). Finally, it has been found that the localization of touch in tools follows the same principle as the localization of touch in hands (Yamamoto and Kitazawa, 2001b; for more details, see the next subsection). We have argued that the localization of tactile sensations uses the body map as a spatial reference frame. However, the body map theory can be true only if exosomesthesia is impossible, that is, if there is no case of extrapersonal sensations localized independently from the body. As argued, pressure is necessarily experienced as of being located. But is it necessarily located as of being located on the body? Only cases of exosomesthesia can show that there is no bodily constraint that lays upon tactile localization. We shall now analyse three cases that may cast doubt on the body map theory. Interestingly, these cases are almost the only cases of exosomesthesia reported in the literature in more than a century of research on the localization of bodily sensations. If there were no bodily constraint on tactile localization, then one might have expected more cases.

Is there no limit to where we can feel tactile sensations? But if the observer was permitted to see the movements of the loudspeaker in the room and coordinate them with the sensations on his arms, after some training he began to project the skin sensations out into the room. (Von Békésy, 1959: 14)

Von Békésy’s report seems to indicate that one can feel tactile sensations in external objects with no spatial contiguity and no spatial resemblance with the body. If this were true, then one should be able to feel sensations anywhere, maybe as far as the moon, as suggested by Armel and Ramachandran (2003): If you looked through a telescope at the moon and used an optical trick to stroke and touch it in synchrony with your hand, would you ‘project’ the sensations to the moon? (Armel and Ramachandran 2003: 1500)

One strategy to account for von Békésy’s surprising result is to compare sensations in the loudspeaker to sensations in phantom limbs and assume that the loudspeaker is felt within the extension of the body (Martin, 1993; Smith, 2002). But why would it be so? One cannot appeal to the fact that participants feel sensations in it for risk of circularity. And there seems to be no other plausible explanation for the embodiment of the loudspeaker. In

Touch   307 particular, it cannot be explained as a kind of Rubber Hand Illusion. It was indeed found that multisensory correlation could result in the embodiment of an external object if the object was bodily shaped such as a rubber hand, but not if it did not look like a body part, like a wooden spoon (Tsakiris and Haggard, 2005), or a loudspeaker. A different strategy to disqualify von Békésy’s results consists in denying that subjects localize sensations in the loudspeaker. More precisely, they localize sensations there, but only indirectly. For example, when talking on the phone, I  directly localize your voice close to my ear, and indirectly localize it in your office from which you are calling. Indirect localization is grounded on direct localization: with no knowledge on the functioning of the phone, I would localize your voice only close to my ear. Likewise, bodily sensations can be directly and indirectly localized.9 Following von Békésy’s own description, we suggest that participants do not immediately experience sensations in the loudspeaker, but rather learn to ‘project’ their sensations there. In our terminology, they indirectly localize tactile sensations in the loudspeaker. The rules that govern indirect localization are not necessarily the same as the rules that govern direct localization. Consequently, this result is not relevant for the study of the constraints that lay upon direct localization of tactile sensations, which is our primary concern here. If Martin’s strategy fails to account for von Békésy’s results, it is more successful in the case of a recent study that uses a well-known tactile illusion, namely, the cutaneous rabbit illusion: repeated rapid tactile stimulation at the wrist, then near the elbow, can create the illusion of touches at intervening locations along the arm, as if a rabbit hopped along it. In Miyazaki and colleague’s (2010) version of the study, participants lifted up a stick between their two fingers until it was in contact with the system that delivered mechanical pulses on the fingers via the stick. They received a series of tactile stimulations on their left index finger, then on their right index finger. Participants then reported feeling touches between the two fingers, that is, on the stick that they were holding. The authors concluded that tactile sensations could ‘hop out of the body’. Out of the biological body, yes, but do they hop out of the body map? This is less certain. Unlike von Békésy’s loudspeaker, one can suggest a plausible account of the embodiment of the stick within the body map, which does not fall into circularity. The stick can actually be easily conceived as a tool manipulated by the participants to interact with the stimulating device. As such, it can be experienced as an extension of the body. Tactile sensations experienced on the stick are then no more surprising than sensations in tools. The final series of putative cases of exosomesthesia comes from patient studies. An amputated patient reported feeling a sensation ‘in space distal to the [phantom]-finger- tips’ when his stump was stimulated (Cronholm, 1951: 190). Another patient ‘mislocalized the stimulus to the left hand into space near that hand’ (Shapiro et al., 1952: 484). Unfortunately, we have little information on these patients.10 Still, it is interesting to note that these reports are 9  For instance, when having a heart attack, I experience the pain primarily in my left arm, but I can also experience the pain derivatively in my heart if I am aware that the pain in my left arm indicates a heart attack. 10  For instance, does the amputated patient normally wear a prosthetic arm, which may be longer than his phantom arm? What other disorders does Shapiro’s patient suffer from? Both patients report feeling sensations beyond the end of their body part phantom or physical, but would they agree if asked whether they feel sensations outside their body?

308    Frédérique de Vignemont and Olivier Massin congruent with further results from von Békésy (1967). He reports having used vibrotactile stimulation to induce in healthy subjects sensations located in the region of empty space between two spread fingers and between the outspread thighs. In all those cases, sensations are localized in the space close to the body. It is now well-known that space immediately surrounding the body, which is known as peripersonal space, is represented in a special way (Làdavas and Farnè, 2004). For instance, when a threatening object enters a spatial margin of safety around their body, animals engage in a range of protective behaviours (Dosey and Meisels, 1969; Cooke and Graziano, 2003). In humans, it was found that viewing a light close to a part of one’s body interfered with simultaneous tactile sensations, if the location of the light was incongruent with the location of the tactile stimuli (Spence et al., 2004). The special significance of peripersonal space can be explained in various ways. It may correspond to a spatial overestimation of the boundaries of the body. Arguably, the brain computes the boundaries of the body, but taking into account some margin of spatial errors. From an evolutionary perspective, it is actually safer to overestimate the body boundaries than to underestimate them. Alternatively, the representation of peripersonal space may play an anticipatory role. Objects immediately surrounding the body are likely to be in contact with the body in a very close future. On both interpretations, peripersonal space is perceived as potentially the space of the body. Its spatial frame of reference is centred on body parts (if the body part moves, then peripersonal space follows, cf. Kennett et al., 2002). Consequently, tactile sensations that are localized in peripersonal space are not localized independently from the body. They are localized relative to a bodily frame of reference. To recap, the burden of proof is on the side of those who defend the view that tactile sensations can be felt in the outside world. We have analysed a series of apparent cases of exosomesthesia, but for each case, there is always an alternative interpretation, which respects our basic intuitions about the spatial relation between tactile sensations and the body. The body map theory then explains the constraints that lay upon the spatiality of tactile sensations. Tactile sensations are necessarily localized relative to the boundaries of the felt body because the frame of reference exploited by tactile sensations consists in the body map. Hence, when I feel a tactile sensation on my hand, I do not feel it on my hand as opposed to another hand. In contrast, when I see a red spot on my hand, I see it on my hand as opposed to many other hands. This is a fundamental difference between tactile experiences and visual experiences (Martin, 1992). A last precision needs to be added. The body map is not the only frame of reference of tactile sensations. If it were the only reference frame, then the posture of the body should not affect tactile sensations: within the bodily frame, a sensation on the right hand remains on the right hand and a sensation on the left hand remains on the left hand even when the hands are crossed. However, crossing hands matters. If you cross your hands over your body midline with your eyes closed and if your left hand is briefly touched, and then your right hand, you take more time and you are less accurate in judging where the first touch occurred (Yamamoto and Kitazawa, 2001a). Furthermore, you experience the same difficulties if you cross two sticks with your hands uncrossed (Yamamoto and Kitazawa, 2001b). Your difficulties could not be explained if tactile sensations were mapped only relatively to the body map. Rather, they result from the localization of tactile sensations in the external world (e.g. on the left), which comes into conflict with their localization on the body (e.g. on the right hand). Pressure is therefore encoded at being at two distinct locations, what BermÚdez (2005) calls A-location (i.e. within the bodily frame

Touch   309 independent of the posture of the body) and B-location (i.e. within an external frame relative to the posture of the body). In O’Shaughnessy’s terms, we experience sensations ‘at-a-part-of-body-at-a-point-in-body-relative-space’. This is not to say, however, that we localize sensations ‘relative to the fixed stars’ independently of bodily localization (O’Shaughnessy, 1980: vol. I, 184). This is neither to say that we experience pressure first in external space, and only secondarily on the body part that happens to be at this specific region in space. If that were the case, then one should be able to localize tactile sensations independently from the body. It is rather the reverse. The body map is the primary frame of reference. Tactile sensations are localized within the external frame only insofar they are localized within the body map. More precisely, B-location derives from the combined experience of A-location (which body part is touched) and of bodily posture (where the touched body part is in space). To conclude, the localization of tactile sensations requires a dual specification, both in terms of tactile signals and in terms of body map, which is used as a spatial reference frame. Tactile sensations therefore have a privileged relation to bodily awareness. This is not to say, however, that this relation is the ‘mark’ of tactile sensations. It is actually shared with other bodily sensations, including pain, thermal perception, tickles, and kinesthetic sensations. As argued, the mark of touch is that it is the only perceptual process that carries information about pressure. The body map theory is fully compatible with our earlier rejection of the body template theory. The body map theory and the body template theory must indeed be distinguished, although O’Shaughnessy (and to some extent Martin11) defend them both. They differ both in their explanandum and in their explanans. The body template theory aims to explain the spatial properties felt in tactile perception (e.g. I feel a circular shape) whereas the body map theory aims to explain the spatial location of tactile objects (e.g. I feel pressure at the tip of my finger). To this end, the body template theory appeals to proprioception (dynamic sense of bodily posture) whereas the body map theory appeals to a relatively stable mental representation of the structural bodily shape. One can therefore defend the body map theory without necessarily defending the body template theory. Tactile sensations consist in sensations of pressure localized within the reference frame of our body map.

References Aristotle (1961). De Anima, ed. W. D. Ross. Oxford: Clarendon Press. Armel, K. C. and Ramachandran, V. S. (2003). ‘Projecting sensations to external objects: evidence from skin conductance’. Proceedings of the Royal Society of London B Biol Sci 270(1523), 499–506. Armstrong, D. M. (1962). ‘Bodily Sensations’. Studies in philosophical psychology. London: Routledge & Paul Humanities Press. Armstrong, D. M. (1993). A Materialist Theory of the Mind. London: Routledge. Armstrong, D. M. (1997). ‘A World of States of Affairs’. Cambridge Studies in Philosophy. New York: Cambridge University Press. 11  Martin’s account is exclusively at the phenomenological level. He does not appeal to the notion of a mental representation of the body.

310    Frédérique de Vignemont and Olivier Massin Bain, A. (1868). The Senses and the Intellect. London: Longmans, Green, & Co. Baldwin, J. (1906). Thoughts and Things or Genetic Logic (Vols 1–3). London: Swann Sonnenschein and Co. Baldwin, T. (1995). ‘Objectivity, Causality and Agency’. In J. L. Bermudez, T. Marcel, and N. Eilan (eds), The Body and the Self (pp. 107–125). Cambridge MA: MIT Press. BermÚdez, J. (1998). The Paradox of Self-consciousnes. Cambridge, MA: MIT Press. BermÚdez, J. L. (2005). ‘The phenomenology of bodily awareness’. In D. Woodruff Smith, and A. Thomasson (eds), Phenomenology and Philosophy of Mind (pp. 295–316). Oxford: Clarendon Press. Blix, M. (1884). ‘Experimentelle Beitrage zur Losung der Frage iiber die specifische Energie der Hautnerven’. Z Biol, 20, 141–156. Block, N. (1983). ‘Mental pictures and cognitive science’. Philosophical Review, 92, 499–541. Bonnier, P. (1905). ‘L’Aschématie’. Revue Neurologique Paris, 13, 605–609. Boring, E. (1942). Sensation and Perception in the History of Experimental Psychology. Century Psychology Series. New York, London: D. Appleton-Century. Botvinick, M. (2004). ‘Neuroscience probing the neural basis of body ownership’. Science, 305, 782–783. Botvinick, M. and Cohen, J. (1998). ‘Rubber hands "feel" touch that eyes see’. Nature, 391, 756. Bremner, A. J., Mareschal, D., Lloyd-Fox, S., and Spence, C. (2008). ‘Spatial localization of touch in the first year of life: early influence of a visual spatial code and the development of remapping across changes in limb position’. Journal of Experimental Psychology: General, 1371, 149–162. Brentano, F. (1979). Untersuchungen zur Sinnespsychologie. Hamburg: Meiner Verlag. Brentano, F. (1995). Psychology from an Empirical Standpoint. London: Routledge. Brown, T. (1846). Lectures on the Philosophy of the Mind. Edinburgh: William Tait. Cardinali, L., Frassinetti, F., Brozzoli, C., Urquizar, C., Roy, A. C., and Farnè, A. (2009). ‘Tool-use induces morphological updating of the body schema’. Current Biology, 1912, R478–R479. Cassam, Q. (1999). Self and World. Oxford: Clarendon Press. Coburn, R. C. (1966). ‘Pain and Space’. Journal of Philosophy, 6313, 381–396. Cooke, D. F., and Graziano, M. S. A. (2003). ‘Defensive movements evoked by air puff in monkeys’. Journal of Neurophysiology, 90, 3317–3329. Cronholm, B. (1951). ‘Phantom limbs in amputees: a study of changes in the integration of centripetal impulses with special reference to referred sensations’. Acta Psychiatrica et Neurologica Scandinavica, Supplementum, 72, 1–310. Dilthey, W. (2010). ‘The Origin of Our Belief in the Reality of the External World and Its Justification’. In R. A. Makkreel and F. Rodi (eds), Wilhelm Dilthey: Selected Works, Volume II: Understanding the Human World. (pp. 8-57) Princeton, NJ: Princeton University Press. Donaldson, H. (1885). ‘On the temperature-sense’. Mind, 399–416. Dosey, M. A., and Meisels, M. (1969). ‘Personal space and self-protection’. Journal of Personality and Social Psychology, 11, 93–97. Dretske, F. (1995). Naturalizing the Mind. Cambridge, MA: MIT Press. Fales, E. (1990). Causation and Universals. London: Routledge. Folegatti, A., de Vignemont, F., Pavani, F., Rossetti, Y., and Farnè, A. (2009). ‘Losing one’s hand: Visual-proprioceptive conflict affects touch perception’. PLoS ONE, 49, e6920. Fulkerson, M. (2011). ‘The unity of haptic touch’. Philosophical Psychology, 24(4), 493–516. Garnett, A. (1965). The Perceptual Process. Madison, WI: University of Wisconsin Press. Gibson, J. (1962). ‘Observations on active touch’. Psychological Review, 69, 477–491.

Touch   311 Goldscheider, A. (1884). ‘Die spezifische Energie der gefühlsnerven der Haut’. Mh. Prakt. Derm, i, 283–300. Hamlyn, D. (1990). In and Out of the Black Box. Oxford: Blackwell. Head, H. and Holmes, G. (1911). ‘Sensory disturbances from cerebral lesions’. Brain, 34, 102–254. Holly, W. T. (1986). ‘The spatial coordinates of pain’. The Philosophical Quarterly, 36144, 343–356. Jackson, F. (1977). Perception: A Representative Theory. Cambridge, Cambridge University Press. James, W. (1887). ‘The consciousness of lost limbs’. Proceedings of the American Society for Psychical Research, 1, 249–258. James, W. (1890). The Principles of Psychology, 2 vols. New York: Henry Holt. Johnson, K. (2001). ‘The roles and functions of cutaneous mechanoreceptors’. Current Opinion in Neurobiology, 11, 455–461. Kandel, E., Schwartz, J., and Jessell, T. (2000). Principles of Neural Science. McGraw-Hill Medical. Katz, D. (1925). The World of Touch. Hillsdale, NJ: Erlbaum, translated by L.E. Krueger, 1989. Katz, D. (1935). The World of Colour. London: Kegan Paul. Kennett, S., Spence, C., and Driver, J. (2002). ‘Visuo-tactile links in covert exogenous spatial attention remap across changes in unseen hand posture’. Percept Psychophysics, 647, 1083–1094. Làdavas, E. and Farnè, A. (2004). ‘Visuo-tactile representation of near-the-body space’. Journal of Physiology, Paris, 981–983, 161–170. Loomis, J., Klatzky, R., and Lederman, S. (1991). ‘Similarity of tactual and visual picture recognition with limited field of view’. Perception, 20, 167–177. Lotze, H. (1888). Microcosmus:  An essay concerning man and his relation to the world. Scribner & Welford. MacPherson, F. (2011). The Senses, Classic an Contemporary Philosophical Perspectives. New York: Oxford University Press. Maine de Biran, P. (1820). Nouvelles considérations sur les rapports du physique et du moral Oeuvres, tome IX. Paris: Vrin, 2000. Margolis, J. (1966). ‘Awareness of sensations and of the location of sensations’. Analysis, 27, 29–32. Martin, M. G. F. (1992). ‘Sight and touch’. In T. Crane (ed.), The Content of Experience (pp. 199–201). Cambridge: Cambridge University Press. Martin, M. G. F. (1993). ‘Sense modalities and spatial properties’. In N. Eilan, R. McCarty, and B. Brewer (eds), Spatial Representations (pp. 206–218). Oxford: Oxford University Press. Martin, M. G. F. (1995). ‘Bodily awareness: a sense of ownership’. In J. L. BermÚdez, T. Marcel, and N. Eilan (eds), The Body and the Self (pp. 267–289). Cambridge MA: MIT Press. Massin, O. (2009). ‘The metaphysics of forces’. Dialectica, 63(4), 555–589. Matthen, M. (2005). Seeing, Doing, and Knowing: A philosophical theory of sense perception. Oxford: Oxford University Press. Miyazaki, M., Hirashima, M., and Nozaki, D. (2010). ‘The "cutaneous rabbit" hopping out of the body’. Journal of Neuroscience, 305, 1856–1860. Moseley, G. L., Olthof, N., Venema, A., Don, S., Wijers, M., Gallace, A., et al (2008). ‘Psychologically induced cooling of a specific body part caused by the illusory ownership of an artificial counterpart’. PNAS, 105, 13169–13173. Mill, J. (1869). Analysis of the Phenomena of the Human Mind, ed. A. Findlater, A. Bain, G. Grote, and J. S. Mill. Longmans, Green, Reader and Dyer. Müller, J. (1842). Elements of Physiology, Vol. II, trans. W. Baly. London: Taylor and Walton.

312    Frédérique de Vignemont and Olivier Massin Murray, C. D. (2004). ‘An interpretative phenomenological analysis of the embodiment of artificial limbs’. Disability and Rehabilitation, 26, 963–973. Norrsell, U., Finger, S., and Lajonchere, C. (1999). ‘Cutaneous sensory spots and the law of specific nerve energies: history and development of ideas’. Brain Research Bulletin, 48, 457–465. O’Dea, J. (2011). ‘A proprioceptive account of the sense modalities’. In F. Macpherson (ed.), The Senses: Classical and Contemporary Philosophical Perspectives (pp. 297–310). Oxford: Oxford University Press. O’Shaughnessy, B. (1980). The Will. Cambridge: Cambridge University Press. O’Shaughnessy, B. (1989). ‘The sense of touch’. Australasian Journal of Philosophy, 67, 37–58. O’Shaughnessy, B. (1995). ‘Proprioception and the body image’. In J. L. Bermudez, T. Marcel, and N. Eilan (eds), The Body and the Self (pp. 175–203). Cambridge, MA: MIT Press. O’Shaughnessy, B. (2003). Consciousness and the World. New York: Oxford University Press. Pearce, J. (2005). ‘The law of specific nerve energies and sensory spots’. European Neurology, 54, 115–117. Perkins, M. (1983). Sensing the World. Indianapolis: Hackett Pub. Co. Pick, H. L. and Lockman, J. J. (1981). ‘From frames of reference to spatial representation’. In L. S. Liben, A. H. Patterson, and N. Newcombe (eds), Spatial Representation and Behaviour across Life Span (pp. 39–61). New York: Academic Press. Ross, P. (2001). ‘Qualia and the senses’. The Philosophical Quarterly, 51, 495–511. Ross, P. (2008). ‘Common sense about qualities and senses’. Philosophical Studies, 138, 299–316. Roxbee-Cox, J. (1970). ‘Distinguishing the senses’. Mind, 79, 530–550. Russell, J. (1995). ‘At two with nature: agency and the development of self-world dualism’. In J.  L. Bermudez, T. Marcel, and N. Eilan (eds), The Body and the Self (pp. 127–152). Cambridge MA: MIT Press. Sanford, D. H. (1976). ‘The primary objects of perception’. Mind, 85, 189–208. Scheler, M. (1973a). Formalism in Ethics and Non-Formal Ethics of Value. Evanston: Northwestern University Press, trad. M. Frings and R.L. Funk. Scheler, M. (1973b). ‘Idealism and realism’. In Selected Philosophical Essays trans D.R. Lachterman (pp. 288–356). Evanston, IL: Northwestern University Press. Schwoebel, J. and Coslett, H. B. (2005). ‘Evidence for multiple, distinct representations of the human body’. Journal of Cognitive Neuroscience, 17, 543–553. Scott, M. (2001). ‘Tactual Perception’. Australasian Journal of Philosophy, 79, 149–160. Siegel, S. (2006). ‘Subject and object in the contents of visual experience’. Philosophical Review, 115(3), 355–388. Shapiro, M. F., Fink, M., and Bender, M. B. (1952). ‘Exosomesthesia or displacement of cutaneous sensation into extrapersonal space’. AMA Archives of Neurology and Psychiatry, 684, 481–490. Smith, A. (2002). The Problem of Perception. Cambridge, MA: Harvard University Press. Spence, C., Pavani, F., and Driver, J. (2004). ‘Spatial constraints on visual-tactile cross-modal distractor congruency effects’. Cognitive, Affective, and Behavioral Neuroscience, 42, 148–169. Stout, G. (1931). Mind and Matter. Cambridge: Cambridge University Press. Stumpf, C. (1928). Gefühl und Gefühlsempfindung. Leipzig: Verlag von Johann Ambrosis Barth. Taylor-Clarke, M., Jacobsen, P., and Haggard, P. (2004). ‘Keeping the world a constant size: object constancy in human touch’. Nature Neuroscience, 7(3), 219–220. Tipper, S. P., Phillips, N., Dancer, C., Lloyd, D., Howard, L. A., and McGlone, F. (2001). ‘Vision influences tactile perception at body sites that cannot be viewed directly’. Experimental Brain Research, 1392, 160–167.

Touch   313 Titchener, E. (1908). Lectures on the Elementary Psychology of Feeling and Attention. New York: The Macmillan Company. Tsakiris, M. and Haggard, P. (2005). ‘The rubber hand illusion revisited: visuotactile integration and self-attribution’. Journal of Experimental Psychology: Human Perception and Performance, 311, 80–91. Vallbo, A. and Johansson, R. (1984). ‘Properties of cutaneous mechanoreceptors in the human hand related to touch sensation’. Human Neurobiology, 3, 3–14. Vesey, G. N. A. (1961). ‘The location of bodily sensations’. Mind, LXX(277), 25–35. de Vignemont, F. (2014). ‘A Multimodal conception of bodily experiences’. Mind, 123(492), 989–1020. Von Békésy, G. (1959). ‘Similarities between hearing and skin sensations’. Psychology Review, 661, 1–22. Von Békésy, G. (1967). Sensory Inhibition. Princeton, NJ: Princeton University Press. Weber, E. H. (1846). E.H. Weber on the tactile senses, ed. and trans. H. E. Ross and D. J. Murray. Hove: Erlbaum UK Taylor & Francis, 1996. Williams, B. (2002). Truth and Truthfulness. Princeton, NJ. Princeton University Press. Wilson, J. (2007). ‘Newtonian forces’. The British Journal for the Philosophy of Science, 58, 173–205. Wundt, W.M. (1897). Outlines of Psychology. G.E. Stechert. Yamamoto, S. and Kitazawa, S. (2001a). ‘Reversal of subjective temporal order due to arm crossing’. Nature Neuroscience, 47, 759–765. Yamamoto, S. and Kitazawa, S. (2001b). ‘Sensation at the tips of invisible tool’. Nature Neuroscience, 4, 979–980. Yrjönsuuri, M. (2008). ‘Perceiving one’s own body’. In S. Knuuttila and P. Kärkkäinen (eds), Theories of Perception in Medieval and Early Modern Philosophy (pp. 101–116). Amsterdam: Springer Verlag.

Chapter 17

The Chemica l  Senses Barry C. Smith

1 Introduction The long-standing neglect of the chemical senses in the philosophy of perception is due, no doubt, to their being regarded as ‘lower’ senses, suggesting something simple and uninteresting in comparison with the reputedly ‘higher’ intellectual senses of vision and audition.1 This flawed view of the senses has obscured the relevance of the chemical senses and the objects to the philosophy of perception. But we are now beginning to understand their significance of the chemical senses in the light of empirical results in neuroscience and psychology, and the way these results have contributed to the radical overhaul of the traditional conception of the senses. The distinction between higher and lower senses was largely based on whether the associated stimuli were proximal or distal. Aquinas and Kant both thought that the lower, or bodily, senses could only provide us with information about ourselves, they produce mere bodily sensations, rather than enabling us to perceive the world around us. By contrast with the lower senses (touch, taste, and smell), vision and audition put us ‘in touch’, so to speak, with distal objects, revealing their perceiver-independent properties and so potentially supplying us with objective knowledge of our environment. This hierarchy of the senses, with its view of the lower senses as giving rise to mere bodily sensations, persists in

1 

Many people have helped with discussions of material for this chapter, including Dominic AlfordDuguid, Malika Auvray, Marco Azevedo, Tim Bayne, Colin Blakemore, Paul Boghossian, Adriano Naves de Britto, Denise Chen, Jonathan Cohen, Kevin Connolly, Ophelia Deroy, Chris Frith, Matthew Fulkerson, Jay Gottfried, Paul Horwich, Thomas Hummel, Ep Koster, Ron Kuypers, Juyun Lim, Per Moller, Bence Nanay, Ann Noble, Matthew Nudds, Casey O’Callaghan, John MacFarlane, Fiona MacPherson, Tony Marcel, Josef Parvizi, John Prescott, Maurice Ptito, Peter Railton, Louise Richardson, Jon Silas, Charles Spence, Marta Tafalla, Dominique Valentin, Keith Wilson, Chris Woolgar, Hong-Yu Wong, Martin Yeomans, Benjamin Young, and audiences in Aberdeen, Abu Dhabi, Antwerp, Austin, Barcelona, Beaune, Beijing, Bergen, Berlin, Bordeaux, Bogota, Cardiff, Copenhagen, Crete, Curitiba, Glasgow, East Anglia, Edinburgh, Hertfordshire, Houston, Istanbul, Kent, Lisbon, London, Milan, Oxford, Paris, Porto Alegre, Riga, Rio de Janeiro, San Diego, Toronto, Tubingen, Turin, and Venice. Thanks are due to them all. But the largest debt of gratitude is to Mohan Matthen for his patience, persistence, judicious editing, and critical insights.

The Chemical Senses   315 contemporary discussions of perception (Lycan, 2000; Smith, 2002). Yet despite that persistence, I shall offer reasons for thinking chemosensory perception puts us in touch with objective features of our environment just as much as the other senses do.

2  The scope of the chemical senses Like all other creatures, humans have receptors that respond to chemicals in their environment, though as humans we have just three chemical senses: taste, smell, and a certain kind of touch. Taste and smell are clearly chemical senses, involving contact between chemical stimuli and chemosensory receptors in the nose and in the mouth. Touch too is chemical when it involves chemesthesis, due to chemical irritation of the free nerve endings in soft tissue giving rise to characteristic sensations of tingling, stinging, burning, and cooling. Of course, not everything classified as touch involves chemesthesis (de Vignemont and Massin, Chapter 16, this volume). Though as we shall see, flavour experience is due to a complex interaction of the chemical and other somatosensory senses. It is sometimes claimed that the chemical senses in humans include signalling between people by means of pheromones. The term ‘pheromone’ was defined by Karlson and Luscher in 1959 to cover ‘substances which are secreted to the outside by an individual and received by a second individual of the same species, in which they release a specific reaction’.2 The definition makes reference to the type of information transmitted and the specific set of responses elicited. The analogy is with hormones, which carry signals between organs of the body. In vertebrates, processing is typically carried out by a dedicated set of receptors in the vomeronasal organ leading to a fixed behavioural reaction. Do humans signal by means of pheromones? There is little empirical evidence that they do. They certainly secrete chemicals that influence their own and other animals’ behaviour but these stimuli are processed by the olfactory system. Humans do have vestigial ducts of a vomeronasal organ in the nose, but the receptors there do not project to the olfactory bulb or any other brain region, so no known mechanism exists for our responding to pheromones, or for them shaping our behaviour. A single research group produced the evidence of pheromone detection in humans but their findings have been widely challenged and other explanations have been offered of the data. So perhaps the most charitable conclusion is that: Proponents of a human vomeronasal organ and of human pheromones . . . have stimulated much debate and raised interesting issues, but [they] do not at present have convincing evidence for the existence of pheromones in humans, functional vomeronasal organs, or an accessory olfactory system in the brain (Johnston, 2000: 120).

Humans are, nevertheless, capable of chemical signalling by means of odours and olfaction operating below our threshold of conscious awareness (Pause, 2012; Chen, 2006). Such chemical signals can have strong behavioural effects on mate selection and sexual behaviour, but the precise behavioural response is seldom fixed for all individuals of the species. 2 

Quoted by Johnston (2000: 120).

316   Barry C. Smith Unconscious cues and responses, including attraction and aversion, provide good examples of how the neural processing of sensory signals outside the sphere of consciousness can have a large impact on what takes place in consciousness, but all of these effects can be discussed without positing human pheromones. In what follows, we shall be concerned exclusively with the chemical senses of taste, touch, and smell—and their interactions.

3  Taste, smell, and touch Taste, smell, and touch are among the most basic senses with which we explore the world. We rely on them to assess our environment and to guide successful food choice. These responses are active in us at birth, and perhaps even earlier. Taste and smell serve as the gatekeepers to the environmental odours and substances that enter our bodies, and, as we shall see, the chemical senses play a key role in mating and feeding, avoiding danger, laying down memories, regulating mood, and maintaining quality of life. As such, they play a continuous role in everyday conscious experience. Deliverances of the chemical senses typically have hedonic value, giving us sensations we find appealing or aversive, and a key question for us will be whether their hedonic values are intrinsic parts of the chemical senses or just accompaniments. For example, sweet tasting foods are innately desirable, while bitter tasting ones are initially unpleasant. This helps to identify energy-rich nutritional sources and to protect us from toxins, many of which are bitter. In this way, taste promotes homeostasis. In addition to the above, saltiness is used in maintaining electrolyte balance, sourness to guard pH levels, and savouriness (umami) to motivate protein intake (Small et al., 2007). Smell functions in a similar hedonic way though there is no evidence that we have innate preferences for particular odours in the way we have innate preferences for tastes. Although we are not usually aware of it, the sense of smell is constantly at work. As J. J. Gibson pointed out, we smell because we breathe. The sense of smell guards the quality of the air we breathe and is connected to limbic areas of the brain responsible for memory and emotion. Thus it plays an important, though often unrecognized, role in our affective lives. Touch too has its hedonic side. Mild tingling sensations are pleasant when consuming spices; they work by stimulating the trigeminal nerve: the fifth cranial nerve that serves the eyes, the nose, and mouth. (This makes mustard feel ‘hot’ and peppermint feel ‘cool’ in the mouth, even though there is no change of temperature.) Such sensations enhance our enjoyment of food, but can become aversive when they sting or burn. Chemically mediated touch also plays a role in safeguarding us. Specific irritants to the trigeminal nerve endings in the nose can be dangerous, and need to be avoided, while others, such as the mild prickle on the tongue from CO2 in fizzy drinks can boost aroma perception and increase the pleasure of drinking. So far, we have been speaking about smell, taste, and touch, but care is needed when talking about the chemical senses because these terms are not being used in their colloquial sense. In colloquial use, terms like ‘taste’ and ‘smell’ usually pick out experiences that, as we shall see, combine sensory inputs from more than one chemical sense. So, from time to time, it will be necessary to use the more precise, scientific terms olfaction, gustation, and chemesthesis, especially when speaking about the interaction of the chemical senses.

The Chemical Senses   317

4  The interaction of the chemical senses Of particular interest in what follows, and distinctive of the chemical senses in humans, is the way they collectively conspire with other senses to give us experiences of tasting something. For what we ordinarily call ‘taste’ involves input not just from the tongue, but from touch and smell. Our failure to recognize the complexity in our tasting experiences has prevented progress not just in our understanding of what we call taste, but in our understanding of the senses more generally. For: Although the experience of the sensory qualities of a food are often described in terms of how it ‘tastes’; in practice, this experience of flavour is a complex interaction. (Yeomans et al., 2008)

The experience often described in unisensory terms as ‘taste’ depends on the multi-modal combining of inputs from different sense modalities into a unified flavour percept:  a percept produced by acts of tasting. Volatiles rising from the mouth pass over the olfactory receptors; this accounts for our ability to distinguish between the equivalently sweet essences of strawberry and cherry. Touch gives us information about the texture of food—whether something is creamy, oily, chewy, sticky, or crunchy; smell can affect what we taste on the tongue and fuse with those tastes to produce complex experiences of flavour. The term flavour picks out something perceived conjointly by taste, touch, and smell, and is used to avoid confusion with what is detected by taste proper (gustation). The resultant unity of our experience of a food or liquid’s flavour provides little clue that it is a complex interaction effect. This may be why people think of it in simple terms as an experience of ‘taste’. It may also have to do with the crucial component of smell going missing in the phenomenology of taste experiences. It takes experimental or clinical findings to reveal the indispensable role that smell plays in what we call ‘taste’. The real nature of these multi-modal experiences and the character of our awareness of them (or lack of it) is where the chemical senses hold greatest interest for philosophers of perception. The example of flavour perception as a kind of multi-modal, yet unified perceptual experience puts pressure on the traditional idea that perceiving is always done by means of either seeing, or hearing, touching, tasting, or smelling. Moreover, the study of flavour perception in sensory science has yielded significant insights into multisensory integration—something increasingly seen as the rule and not the exception in perceptual processing. Thus, the chemical senses far from being peripheral to the concerns of the philosophy of perception, may hold important clues to the multisensory nature of perception in general. But before we look at the mechanisms of multisensory integration underlying flavour perceptions let us review the traditional assumptions about perception that they challenge.

5  The senses working in isolation From the time of Aristotle to the present day, people have thought of themselves as having five distinct senses that enable them to see, hear, taste, touch, and smell; a classification that became further entrenched in the nineteenth and twentieth centuries (Matthen, Chapter 30, this

318   Barry C. Smith volume) Aristotle thought that each sense had a proper object, a kind of quality that could only be perceived with that sense: e.g. colour for seeing, sound for hearing, taste for tasting. There were, in addition, qualities or properties that could be perceived by more than one sense. For example, size and shape could be perceived by both sight and touch; these were called common sensibles. Some philosophers are tempted to think of common sensibles as having more objectivity than qualities which could be detected by one sense alone. These are the classic primary qualities of John Locke, whose natures, unlike colours, tastes, and smells, are independent of perceivers (Ross, Chapter 21, this volume). This picture of the senses and their proper objects encourages us to think of the senses as working in isolation. The experience of sounds comes to us via hearing, the experience of colour via seeing, and the experience of tastes via our sense of taste. In the science of perception, the largest focus was on seeing. The occipital cortex occupies about a third of the brain's processing capacity and this may have led to the dominance of vision science. But it may not only be science that explains this dominance. Seeing occupies so much of our attention that it is not surprising, perhaps, that it dominates discussions of perception. This may be why so many philosophers treat problems of perception as if they were exclusively questions about visual perception, assuming, not always explicitly, that the correct account of visual perception will just apply with minor modifications to perceiving by means of other senses. The prominence of the occipital cortex also encouraged a misplaced generalization. Surely, so the argument goes, if we could point to activity in specific brain regions as being responsible for processing information from other sensory receptors, then we would be entitled to study the senses in isolation. This strategy for studying the senses in isolation seemed at first to have anatomical support from the brain sciences. The advent of neural imaging led to better identification of separate cortical and sub-cortical regions responsible for different functions, and particular areas were identified as the visual cortex, the auditory cortex, the primary smell and taste cortices, and the somatosensory cortex. This encouraged the idea that significant areas of the brain were devoted to unimodal processing by analogy with early vision.

6  The merging of the senses The picture just described is no longer dominant in the sensory neurosciences, for it has been repeatedly demonstrated that perception relies on combining information from many sources (Bayne and Spence, Chapter 32, this volume): Our brains are continuously inundated with stimulation arriving through our various sensory pathways. The processes involved in synthesizing and organizing this multisensory deluge of inputs are fundamental to effective perception and cognitive functioning. (Talsma et al., 2010: 400)

This means that focusing solely on unisensory processes will continue to provide us only with an impoverished view of both brain and behavior. (Ghazanfar and Schroeder, 2006: 278)

The Chemical Senses   319 The field has been transformed through a recognition that the coordination and integration of information derived from different sensory systems is essential for providing a unified perception of our environment, and for directing attention and controlling movement within it. The capacity of the central nervous system to combine inputs across the senses can lead to marked improvements in the detection, localization and discrimination of external stimuli and to faster reactions to those stimuli. (King and Calvert, 2001: 322)

Many neuroscientists believe that ‘Perception is multisensory’ because ‘no single sensory signal can provide reliable information about the three-dimensional structure of the environment in all circumstances’ and ‘ . . . if a single modality is not enough to come up with a robust estimate, information from several modalities can be combined’ to provide a better fix on external events or objects (Ernst and Bulhoff, 2004: 162). In their important review paper Ernst and Bulhoff continue: To perceive the external environment our brain uses multiple sources of sensory information derived from several different modalities, including vision, touch and audition. All these different sources of information have to be efficiently merged to form a coherent and robust percept.

And: The key to robust perception is the combination and integration of multiple sources of sensory information. (Ernst and Bulhoff, 2004: 162)

For example, we sometimes just see, and sometimes just hear, a plastic bottle being crushed, but when we both see and hear it, this gives more neural activation than the sum of activation involved in the seeing and the hearing. This superadditivity in the neural pattern of responding to combined information is one sign of multisensory integration. It is as if the brain has learned to pay attention to, and mark out, congruent multisensory information to help it track environmental events. Philosophers have been, until recently, slow to recognize this important shift in the science of sensory perception, but one who has stressed its importance is Casey O’Callaghan: an adequate, complete understanding of perception requires comprehending the ways in which what goes on with one sense modality impacts what goes on with another. Theorizing about perception is not just a matter of assembling independently viable stories about vision, audition, olfaction, and the rest. Considering the relationships and interactions among perceptual modalities sheds light on what is most striking about perception: its capacity to furnish a sense of awareness of a world of things and happenings independent from oneself. (O’Callaghan, 2008: 316)

The prolonged failure to appreciate the significance of multisensory interactions is surprising. As DeGelder and Bertelson put it: Research on perceptual processing, whether behavioural or physiological, has generally considered one sense modality (sight, hearing, touch, smell, etc) at a time. Yet, most events in the natural environment generate stimulation to several modalities. An explosion simultaneously emits light, noise and heat, and experiencing all of these together make for a

320   Barry C. Smith richer percept than each individually; a speaker produces facial movements in a predictable temporal relationship to corresponding speech sounds and experiencing both together can provide a more adequate percept. (DeGelder and Bertleson, 2003: 460)

The two cases mentioned here are somewhat different, but the contrast between them provides a clue as to why it has taken so long to discover how sensory systems interact to shape our perceptions, not least our perceptions of flavour. The case of the explosion is one in which we are consciously aware of the light, the noise, and the heat of the event, though perhaps not aware of how the interactions of the sensory inputs intensify overall experience. The second case is one in which we are entirely unaware of how, and how much, visual information about a speaker’s lip movements contributes to the experience of the speech sounds we hear, and in particular to the way vision dominates audition when there is sensory conflict. It is clear, from the case of speech perception that the workings of one sense can affect not just the intensity of another sense, but the contents of the perceptions that rely on it. (See O’Callaghan, Chapter 25, this volume.) In the cinema, for example, we experience the voices as coming from the mouths of the actors on the screen, even though the sound comes from loudspeakers located at the sides of the movie theatre, under the seats, or even behind the viewer. When lip movements are seen in synchrony with the heard speech sounds, we get visual capture of auditory attention, making auditory perception present things as if the mouths on the screen were the sources of the speech sounds. Known as the ventriloquism effect, this illusion provides a clear case where vision dominates audition in specifying the (apparently) common source of sights and sounds (Allais and Burr, 2004). Often we are not even aware of sensory conflicts and simply assume that we are enjoying complimentary though distinct sensory experiences. A different case, though relevant to the interactions of the chemical senses, is where a new multisensory quality results from the joint upshot of different senses working together to produce something that could not be produced by any of the contributing senses alone. The McGurk Effect in speech perception is one such example, where one is looking at a film of a face making the speech gesture /ga/ while hearing a synchronized audio feed of the speech sound /ba/, resulting in the percept /da/, which is acoustically and production-wise intermediate between /ga/ and /ba/; something that one neither saw nor heard. The significance is that a new item in experience has been produced by cross-modal interaction between the senses, although it is not recognized as such by subjects of the experience. Many will say that the McGurk Effect is a carefully arranged illusion, and as such, it is not a natural phenomenon. Though we may want to ask if there are any natural cases of such new multisensory phenomena. Experiences of flavour from tasting foods or drinks provide just such cases from everyday life. The objects of perception in tasting are flavours, though we classify them as tastes. How should we understand such phenomena given how are they produced? We will examine that question in some detail. But at the outset it is worth noticing the considerable challenge flavour experiences pose to the traditional picture of perception. If it is possible to experience a quality like flavour only through the conjoint exercise of several senses, then we have a category of perceptual quality for which Aristotle’s classification made no room. Flavours are not common sensibles accessible by more than one sense; we need many senses—chemical and contact—working together to produce flavour perceptions. The only way to restore Aristotle’s idea of there being proper

The Chemical Senses   321 objects of the senses in the case of flavour perception is to claim that we have a single sense of flavour, albeit one that draws on the interactions or integration of other senses; a position some have proposed in the empirical literature (Auvray and Spence, 2008; Matthen, Chapter 30, this volume). Let us now examine the senses that contribute to the multi-modal experience of flavour, and try to say more about why we fail to recognize the complexity in the experiences of tasting food and drink.

7  What do we mean by taste? It’s difficult at first for us to focus on what happens when tasting a food or a liquid. We pop something in our mouths; we sip, or chew, then swallow, and the sequence of experiences we undergo is fleeting and ephemeral; gone in an instant and hard to focus on. The transient nature of these experiences often means the temporal dynamics of tasting go unnoticed until they are pointed out. Mostly, we are left with an impression of liking or disliking, and often people behave as if the whole point of tasting was to come up with a verdict about liking. Though as we shall see liking can be a distraction. Like seeing and listening, tasting is an activity that generates experiences of a distinctive kind: experiences of the flavours of the foods and drinks we consume. What is it for something to taste a certain way? Begin with the experiences themselves. They occur when eating or drinking, but we also think of tasting as the having of those experiences. In this sense, tasting is itself an experience, and many philosophers will say that it is a subjective experience in the individual taster. In one sense, of course, this is correct. Tasting experiences happen to individual subjects of experience. So do episodes of seeing and hearing. These, too, are going on in subjects of experience. It does not follow, however, that what one sees or what one hears is subjective; nor does it follow that tasting is all about the subject. What we taste—its taste—need not be purely subjective. For it to be purely subjective requires treating how something tastes as no more than a fact about the taster, about how something tastes to him or her. So there would be as many ways something tastes as there are tasters, or ways that tasters taste a given food. And yet, ordinarily, we don’t talk that way. We speak not about my tastes and your tastes, but about the taste of a dish or of a wine. The milk tastes sour, the anchovies taste salty, the wine tastes overly sweet. These are not ways of talking about me, but about the items we are tasting. The subjectivist is unmoved by the linguistic data, and will rightly point out how promiscuous our use of the word ‘taste’ is. It is variously used to mean: a quality of a food or liquid, like the sourness of a lemon; the characteristic experience we have when eating a lemon; the sense by which we detect that quality or generate that experience; and even, when used in aesthetics, a refined sensibility. Subjectivists treat all as aspects of subjective experience, deriving from mere sensations occurring in tasters. According to the subjectivist, we may talk of the taste of an apple or an onion, but they don’t really have tastes; rather, they give rise to tastes in us. Tastes are just sensations we undergo when chewing or sipping foods or drinks. (A more objectivist view about taste would distinguish between tastes as properties a food or wine has, and tasting as an experience that a subject has.)

322   Barry C. Smith

8  Tastes as sensations? There are simple and more sophisticated views of tastes and tasting as subjective.3 The simple view sees tastes as sensations on the tongue, inseparable from the subject who has those sensations, immediately available and knowable through and through for what they are. However, to think of the tastes of a wine as exhausted by sensations on the tongue precludes the idea that not every tasting experience is as good any another. When we taste a wine before and after eating a lemon, or brushing our teeth, it doesn’t taste the same way; yet we know that the wine hasn’t changed. We are just no longer able to taste it properly. The simple subjectivist has to say there are as many tastes to a wine as there are tasters and moments of tasting. But a wine’s taste is not exhausted by the sensations one has at a moment. Dispositionalists might say that what matters is how a wine or a dish tastes under ideal conditions. But independently of matters of taste how are we to spell out the ideal conditions, and why should they be the same for different individuals? As we’ll see, they are not. Individuals vary considerably in their responses to the same stimuli.4 Should we, then, regard tastes as properties of foods or liquids that we are able to perceive by tasting? This objectivist stance appears closer to common sense. For we seem to rely on tasting to give us knowledge of the taste of things we eat and drink, to tell us whether the strawberry is sweet and ripe, whether the coffee has sugar in it and whether the soup is salty. Taste predicates like these are attributed to food and not to the sensations we have when eating them. The subjectivist is unable to give a convincing account of these attributions. Can we even be sure of the experiences we have when tasting food and wines? And are these experiences just as they seem to us? To some philosophers, this last question will be unintelligible since they believe that how things appear to us in experience is how they are. There is no appearance–reality divide for conscious experience; experiences just are as they appear to be. This line of thought is part of a residual and unquestioning Cartesianism. That we undergo an experience is unmistakable, but that doesn’t guarantee that we know that experience for what it is. Being in a mental state is one thing: knowing what kind of mental state we are in is another. As we shall see, subjectivists who treat tastes as simple sensations take how their experiences appear to them, or how they take them to be, as the only way to individuate those experiences. Although, the modal signature of our flavour experiences is far from clear, nor is it clear what gives any of our experiences the modal signatures we take them to have.

9  The hidden complexity of tasting experiences It is a remarkable fact that we fail to recognize the complexity of our own tasting experiences, and remarkable that philosophers still readily assume that we have transparent

3 

For an objectivist view of tastes and tasting, see Smith (2007).

4 

See Jonathan Cohen (2013).

The Chemical Senses   323 access to those experiences, allowing us to know their natures through and through. It is this assumption of self-intimating experience that is supposed to enable us to know that what we ordinarily call ‘taste’ is a matter of simple sensations on the tongue, something clearly unisensory. And yet, it has taken psychologists and neuroscientists to reveal to us the full extent of the complexity of our tasting experiences.5 Brillat-Savarin understood this point early on. Writing in the nineteenth century, he was ‘tempted to believe that smell & taste are in fact but a single sense, whose laboratory is in the mouth & whose chimney is the nose’ (Brillat-Savarin, 1825: 41). These comments are much in line with the recent findings of neuroscience and psychology, which sees the experience of tasting as the product of multisensory integration: a fusion (confusion?) of inputs from different sensory modalities that gives rise to a unified percept (see Auvray and Spence, 2008; Prescott, 1999). Once taste and smell fuse into an experience of flavour, it is no longer possible to separate out the different components by phenomenological decomposition. Certain things go missing from our awareness of our tasting experiences because, at first, we don’t recognize that ‘flavour perception is not a single event but a dynamic process with a series of events’ (Piggott, 1994: 167). Tasting has a dynamic time course and slowing it down makes a difference to what we notice and what we can pick out. In this way, how we taste affects what we taste; and attending to each aspect of the dynamic time course changes the temporal scale of the tasting experiences we have. Through attention, we can focus on particular qualities of the taste, texture, and aroma of foods and liquids. (It does not, however, mean that we can separate taste and retronasal olfaction.) Despite such careful scrutiny, we tend to think of the unified experience of flavour as coming just from the tongue or the oral cavity. In fact, the tongue gives us very little. All it provides is awareness of the basic tastes: salt, sweet, sour, bitter, umami (savoury), and metallic. But think of all the flavours we can experience when tasting: ‘ripe mangoes, fresh figs, lemon, canteloupe melon, raspberries, coconut, green olives, ripe persimmon, onion, caraway, parsnip, peppermint, aniseed, cinnamon, fresh salmon’ (Sibley, 2006: 216). We don’t have taste receptors on the tongue for mango or raspberries or chicken or lamb. Nor are these recognizable flavours concocted from the basic tastes, as philosopher of aesthetics Frank Sibley perspicuously pointed out: how could one construct a blend of distinguishable tastes . . . to yield that of coconut, or lemon, or mint? Try to imagine a recipe: ‘To make the flavour of onion (or pepper, or raspberries or olives) add the following [basic tastes] in the following proportions . . . (Sibley, 2006: 216–217)

Clearly, there is no such procedure, and yet, the act of tasting gives us knowledge of all these recognizable flavours. The objects of perception in tasting are not tastes, but flavours, and as Sibley points out flavours are single, not simple. Tasting flavours involves much more than the tongue, and tastes proper make only a limited contribution to our experience of flavours. Properly speaking, taste exclusively concerns the gustatory inputs due to firings from taste receptors in the oral cavity, the soft palate, and the gastrointestinal tract. The

5 

Richardson (2013) tries to defend a non-naturalist view in this area by wisely discarding the idea that tastes come from the tongue on the grounds that philosophers don’t have to be committed to this. She does not, however, tell us where tastes are thought to occur, or what makes something an experience of taste.

324   Barry C. Smith sensations that the tongue produces—gustatory sensations—are hard to experience alone save in experimental settings, for example, when drops are put on parts of an anaesthetized tongue and we prevent any other sensations, of smell or touch, say, from contributing to the subject’s experience. What we call ‘taste’ is due not to sensations from the tongue alone but to the multisensory interactions that produce flavour experience. Smell plays a fundamental role in forming these flavour experiences. But not smell, as we ordinarily think of it: as a matter detecting the odours we inhale. Rather it is smelling by processing of odours that reach the nose from the mouth when they are released from the foods or liquids we chew, sip, and swallow. This is due to retronasal olfaction, mentioned above, where odours rise from the oral cavity through the nasopharynx to the olfactory epithelium in the nasal cleft. Should we treat this other route to the olfactory receptors as part of smell or a second sense of smell?

10  The duality of olfaction Olfaction takes place when odour molecules reach the olfactory epithelium in the nasal cleft. In humans (and rats) there are two different pathways by which molecules reach the olfactory epithelium: via the nose when inhaling, and via the mouth when eating or drinking. The orthonasal route is when chemical stimuli from the external environment travel to the nasal mucosa during inhaling or sniffing. This is what we commonly think of as ‘smell’. In addition, there is the retronasal route by which odour molecules released during the mastication of food reach the olfactory epithelium via the mouth. The act of swallowing pulses odours up to the olfactory epithelium via the nasopharynx. Should we take these two routes as just different directions in which stimuli are presented in olfaction; or should we treat them as giving rise to different modalities? Retronasal olfaction is fundamental to the formation of flavour experiences. Its inputs combine with inputs from taste to create unified flavour percepts. And yet as human tasters we don’t recognize this olfactory component. Although we smell or detect odours via retronasal olfaction, we think we taste them. That is, the qualities of fruitiness, or more precisely, pineapple or cherry, that olfaction contributes to the flavour we are tasting appear to occur in the mouth. The event of swallowing creates the most intense moment of flavour release due to odours being pulsed to receptors in the olfactory epithelium. In fact, when food-related odours reach the olfactory epithelium from the mouth, the qualities detected and the sensations generated are typically felt to be experiences of tasting in the mouth. This location illusion is due to oral referral, a phenomenon analogous to the case of pain referral or phantom limb, where a sensation is felt in a part of the body other than where the stimulus is being applied (Soler et al., 2010). Because the resultant flavour experiences are classed as tastes the dimension of smell goes missing (or unnoticed) phenomenologically. That smell doesn’t appear as a separable part of the complex experiences generated by the interaction of taste (and touch) and retronasal olfaction doesn’t mean that we can’t focus our attention on the dimension of flavour it contributes (e.g. fruitiness) to out experience. Perhaps the fact that we don’t recognize it as due to olfaction provides a prima-facie reason for thinking that retronasal olfaction should not be counted as part of our sense of smell. But this is too quick.

The Chemical Senses   325 We can experience the products of retronasal olfaction as smells under certain carefully controlled experimental conditions. In 2004, Heilman and Hummel developed a device for directly delivering odours above the soft palate to the olfactory epithelium by inserting nasal cannula endoscopically into their subjects’ noses with one outlet at the nares and the other at the epipharynx. This allows experimenters to pulse odours to the nasal cleft without interference from gustatory, trigeminal, or thermal mechanisms, thus allowing the study of retronasal olfaction in isolation (Bender et al., 2009a). Non-food odours such as lavender delivered via this retronasal route will be experienced as smells not tastes (Small, 2005). With this technique it is possible to show that despite having the same volume of odour in the olfactory cleft, thresholds for detection and intensity are different for orthonasally and retronasally delivered odour stimuli. Less invasively, participants can inhale the vapour state of an odorous liquid from a straw inserted into the headspace of an otherwise sealed glass jar. By means of the straw, participants inhale the odour in the headspace above the liquid and exhale the odour through the nose. In one study vanilla odour was presented alone and 54.7 per cent of participants reported the location of the vanilla experience as in the nose (Lim and Johnson, 2011). When there were incongruent tastants such as NaCl or caffeine in the mouth, this made little difference to the location of the vanilla odour. However, when the inhaled vanilla was presented with sucrose in the mouth, 69.4 per cent of participants localized the vanilla to the tongue or oral cavity. This indicates that if retronasally presented odours are combined with congruent tastes (i.e. taste-odour combinations they have encountered before), most people experience these stimuli not as separate smells and tastes but as flavours. The separately experienced smells from retronasally delivered odours are laboratory created, in contrast to the everyday cases where the phenomenologically individuated experiences of flavour give us no clue as to the influence of olfaction in generating these experiences. So we may want to draw a distinction between the sense of smell and olfactory processes in general, restricting the sense of smell to orthonasal olfaction alone.6 Another option is to treat olfaction with its orthonasal and retronasal pathways as a dual modality, and on that basis try to identify two separate senses of smell.

11  Two senses of smell? The first person to propose olfaction as a dual system was Paul Rozin (1982),7 who spoke of different evolutionary functions for orthonasal and retronasal olfaction:  to sense objects in the world and sense (or assess) objects in the mouth, respectively. He saw these in Gibsonian terms as two perceptual systems subserved by olfactory processes. One system is to perceive things in the environment such as food sources, predators, smoke, and 6 

This appears to be the strategy in Richardson (2013). his 1982 article, Rozin thought olfaction was the only dual sensory modality, although we can think of other candidates: the dual pathways of vision (Milner and Goodale) and active and passive touch (Marcel). 7  In

326   Barry C. Smith potential mates. The other system is to perceive and evaluate the quality of what we are eating or drinking in order to know whether we should continue to eat it or expunge it. Rozin hints at a reason why ‘the olfactory [retronasal] component loses its separate identity [when] combined with available oral inputs into an emergent percept’ (Rozin, 1982: 400). This, he thinks, is due to the fact that the olfactory stimulus may differ qualitatively in the in-mouth and out-there cases . . . It seems very likely that the olfactory component of flavour differs markedly from the olfactory consequences of the same substance in the external world. (Rozin, 1982: 400)

Neuroanatomy lends support to this idea and explanations have been offered as to why the same molecules, reaching the same olfactory receptor cells in the epithelium, are experienced differently depending on the route by which they got there. One explanation is that the brain distinguishes the direction of airflow and projects the pattern of receptor firings to slightly different cortical areas of the piriform, insular, and orbito-frontal cortices depending on whether the molecules come from the nose or from the mouth (see Small et al., 2005).8 A more psychologically convincing way to confirm Rozin’s idea is by pointing to our experiences of matches or mismatches between smells and flavours. We often have expectations about how something will taste from how it smells; at other times, we may be surprised by the mismatch between how something smells and ‘tastes’, as when a wine’s aroma gives little guidance as to its performance on the palate. There are two considerations here. Why should smells provide any expectations about how things taste? Smells are one thing, tastes are another. We can’t smell saltiness or bitterness; so smelling a dish or a wine may not give us enough information to discover its flavour. This would argue that whatever olfaction may contribute to flavour, it is sufficiently unlike things we sniff, to see these as different senses (and as the sceptic would say, to accept only one of these senses as involving smell). However, there are other cases too—we talk about sweet smells, although sweet is a taste. Compare this with our talk of an acidic wine as tasting sharp, although sharp is a feel. These cross-modal attributions indicate how easy it is for us to confuse taste and smell, and perhaps to see aspects of flavour as resembling what we can smell orthonasally. Smells tend to acquire the characteristic properties that belong to tastes—sweet, sour, etc.—because they are so closely associated with things which when put in the mouth have a sweet or sour taste. Why should a smell be so closely associated with something that tastes sweet or sour? It is because the smell is so closely associated with the retronasal smell that a sweet or sour smelling food or liquid has in the mouth which contributes to its flavour. Smell is a part of the flavour we ‘taste’. The wonderful perfume a ripe strawberry gives off is a pretty reliable guide to the experience we’ll have when we eat it. How is this to be explained except by supposing that the aroma detected by orthonasal olfaction is similar in quality to, though less intense than, some part of the flavour (detected by retronasal olfaction) we get when we put the strawberry in our mouths. 8 

fMRI studies reveal that retronasal presentation of chocolate odour produces preferential activation in the medial orbitofrontal cortex, the perigenual cingulate, the superior temporal gyrus, and the posterior cingulate cortex. Orthonasal presentation produces activation in the thalamus, right caudolateral orbitofrontal cortex, right hippocampus, frontal operculum bilaterally, temporal operculum, left anterodorsal insula and right anterior insula (Small, 2005; Negoias, 2008).

The Chemical Senses   327 The idea that the heady strawberry smell is sweet-smelling is intricately connected to the olfactory dimension of flavour. What is unpredictable on occasion about flavour from orthonasal olfaction shows the difference between the orthonasal and retronasal olfactory contributions, and what is predictable and shared suggests that they are both senses of smell. We can reconcile these claims by acknowledging that there are two senses of smell. The idea that the heady strawberry smell is sweet-smelling is a projection back onto the inhaled smell of some retronasally detected olfactory property intricately bound up with the gustatory element of flavour. This idea of matching, or sometimes mismatching, in terms of odours and smells, experienced by the orthonasal or retronasal route, helps to explain asymmetries between smell and flavour recognizable in our ordinary experience. When you smell a ripe Epoisse cheese, for instance, it can be pretty off-putting. (That’s the smell of isovaleric acid.) Yet, when you put it in your mouth, it can be surprisingly good. The cheese’s odour when processed externally and internally via the mouth lead to very different experiences. The reverse case is coffee, where the aroma of freshly brewed coffee always smells so wonderful though it is slightly disappointing in the mouth. Many of the volatile compounds are stripped off by saliva, and many are not experienced the same way as they are when inhaled through the nose.9 The connections between the experiences of odours and the experiences of food flavours play a key role in considering retronasal olfaction to be a part of smell. For instance, loss or deterioration of our ability to smell—identified phenomenologically by changes in our experience of odour—coincide with the loss of flavour perception. At first, the reduced experiences of anosmia sufferers are regarded as loss of taste not smell. For a clinical example, where patients lose olfaction, they often report that they cannot taste or smell. However, when questioned, patients acknowledge that they taste salty, sweet, sour and bitter, but ‘nothing else’. The ‘nothing else’ is the contribution of retronasal olfaction to flavour. (Bartoshuk and Duffy, 1998: 284)

More evidence of smell’s contribution to flavour experience comes from normally functioning subjects. Just about all normosmics have specific anosmias—that is, despite having a fully functioning sense of smell, there is some small set of odours which they cannot detect at all. When this is the case they will be unable to detect certain elements in airborne molecules, and also be unable to detect the contribution these volatile compounds make to flavour. For example, several normally functioning subjects are anosmic as far as Trichloranisole (TCA), or cork taint goes. If it is undetectable by the nose, drinkers will also fail to find any fault with a corked wine on the palate. The selective loss of smell is responsible for a corresponding deficit in flavour perception. Another similarity between everyday smell and retronasal olfaction as it contributes to the perceptions of a food’s flavour involves the Proust Phenomenon where the sudden whiff of a long forgotten scent seems to take us right back to a past time or a scene we didn’t know we had stored in our memories. Smell has a special capacity to trigger episodic memories—especially emotionally charged memories—because of the direct connections

9  Coffee is all about the aroma and advertisers know it. That’s why we are told to wake up and smell the coffee, not taste it.

328   Barry C. Smith olfaction has with parts of the limbic system involved in emotion and memory. Olfaction is unique in being the only sensory system that projects directly to the amygdala without going via the thalamus. As a result, odour memories are long-lasting and less susceptible to interference. This may account for the Proust Phenomenon, which takes its name from the famous episode where Proust’s narrator, Marcel, tastes a madeleine soaked in lime-blossom tea and is suddenly jolted by the pleasure, which gradually evokes a memory of his aunt on Sunday mornings dipping crumbs of madeleine into tisane, which she presented to him. Experimental attempts have been made to confirm the phenomenon (Chu and Downes, 2000; Hertz and Schooler, 2002). Are there any reasons to think the duality of olfaction just discussed gives us two senses of smell? There are number of things to be said here.



a. The cases mentioned above involving cheese, coffee, and chocolate show how orthonasally and retronasally presented odour stimuli lead to different experiential effects. Anecdotally, people often prefer the smell of cigars smoked by others to the experience of smoking them themselves. This is possibly due to the different qualities of the smoke when experienced orthonasally or retronasally. Retronasal olfaction also plays a crucial role in the perception of food flavours and when these are compared to the smell of foods, the comparisons are between, not smell and taste, but smell and a different olfactory dimension that contributes to flavour. Moreover, neurophysiologically, orthonasal and retronasal stimulation of the olfactory receptors project to slightly different cortical areas, and detection thresholds and intensity ratings are also different. b. There is also a non-clinical way to demonstrate how olfaction contributes to flavour, not by phenomenological decomposition but by subtraction. It is to hold one’s nose pinched tightly shut while popping a jelly bean into one’s mouth to chew. Keeping the nose and mouth tightly closed, one has no clue at this stage as to the fruit flavour of the jelly bean. All one experiences at first is some sweetness and perhaps some sourness. However, when one releases the nose there is a rush of odours to the nose from the mouth, and immediately one can tell which flavour—cherry, strawberry, or orange—the jelly bean has; and although the nose is evidently involved in supplying the additional flavour component, the fruity flavour now seems to be coming from the tongue. One knows this can’t be the case since just a moment ago when the nose was closed, one learned that all the tongue provided was sweet or sour. Only when the nose is unblocked can one have an experience of flavour. What we have here is a clash of appearances. Normally, it seems as if the fruit flavour is experienced in the mouth, on the tongue; and yet, with the nose blocked it appears that the tongue only provides awareness of sweetness, sourness, saltiness, etc. The way to resolve the tension is to recognize the contribution that smell makes to the experiences that seem to occur on the tongue. When people discover this for themselves they express surprise that the nose is required to have full flavour experiences, but they have no difficulty in updating their commonsense conception of tasting to incorporate the part smell plays in the experience of flavour. They do not—as those who resist the idea that tasting involves smell would have to suppose—conclude that these flavour experiences of tasting indicate that smell is

The Chemical Senses   329









not involved but that the single sense of taste is implemented at the neural level by among other things receptors that also serve in olfaction.10 c. Although retronasal smell is usually only experienced through its contribution to flavour, we can sometimes engage in phenomenological decomposition as well. The key case involves the flavour of menthol, normally experienced as a single percept. On reflection, we recognize that an experience of menthol comprises (i) a slightly bitter taste, (ii) a minty aroma, and (iii) a cool sensation in the mouth, and we can attend to each of these features in turn. Remove any one of them and you no are longer tasting menthol. The three chemical senses work together to produce a single unified flavour experience, and yet the reason why we can separately attend to the contributing sensory components in this case but not normally may be due to I-menthol’s exceptionally strong trigeminal stimultaion whose cooling sensations may keep one keenly aware of a minty aroma in one’s upper airways. But even in this case, one has to be prompted to attend to the parts separately and recognize the complexity of the experience. d. In a food context, we can think of the two senses of smell as associated with two different sorts of pleasure: the pleasure of anticipation when we smell foods orthonasally; and the pleasure of reward when we assess the flavour of the food as we chew and swallow. This hedonic distinction between the two senses of smell is closely related to Kent Berridge’s distinction between wanting and liking (Berridge, 1996). This distinction supports the idea that olfactory cues given orthonasally can trigger appetite and a salivary response, anticipating the reward of the food we can smell. If one continues to receive stimulation without having the food the salivary response diminishes as a result of habituation. But if the same stimulus odour is presented retronasally—controlling for the same intensity judgement—the salivary response returns, suggesting that different presentations of the same odour are perceived differently, just as different odours are, where repeated presentation of the first leads to habituation that is reversed with the new stimulus (Bender et al., 2009b). A distinction between the two pleasures associated with each sense of smell fits well with Rozin’s claim about the different evolutionary functions of the two senses of smell: to sense food sources (or danger) in the environment; and to assess the quality of what we’re consuming and are about to ingest. It also contributes to the explanation of the match and mismatch between orthonasally and retronasally experienced odours. e. There are other experiential differences between orthonasal and retronasal olfaction when we look at cross-modal interactions with further senses. A congruent colour cue enhances the intensity of our experience of odours orthonasally, but suppresses their intensity retronasally (Korza et al., 2005). Texture-odour interactions also showed differences in experience depending on route of odour delivery. Certain odours can increase the perception of thickness and creaminess but only when presented retronasally (Bult et al., 2007). f. Finally, there are dissociations between orthonasal and retronasal olfaction that show up in pathological smell experiences. There can be deficits in retronasal

10 

See Richardson (2013) for this philosophically mandated alternative to the commonsense response.

330   Barry C. Smith smelling without orthonasal deficits (Cowart and Halpern, 2003). Conversely, there can be intact retronasal smell shown by preserved flavour perception, but where ‘ordinary’ orthonasally derived smell is impaired because of nasal polyps. (Landis et al., 2003). And even without evidence of polyps, there is a population of patients who show event-related potentials for retronasally but not orthonasally presented stimuli who experience flavours but do not discriminate smells in the environment (Landis et al., 2005). Together, these considerations provide behavioural, neuroanatomical, phenomenological, hedonic, and pathological evidence in favour of counting olfaction from either route as belonging to the sense of smell, and provide motivating reasons for crediting humans with two senses of smell. It is worth noting that we are not, here, distinguishing between a unimodal experience of smell and smell as a component of a multisensory experience of a food’s flavour. Our ordinary sense of smell may seem intuitively to be unimodal; however, orthonasal olfaction produces experiences of odour mostly in concert with stimulations of the trigeminal nerve (Frasnelli and Hummel, 2005). The experiences we recognize as smelling are typically the joint upshot of olfactory and trigeminal processing; so are themselves multimodal experiences. Of course, we can smell pure odorants such as phenol ethyl alcohol that do not activate the trigeminal nerve, but we don’t consciously distinguish experiences of these odours from other odour experiences, although they can be distinguished behaviourally because we are less good at localizing odour sources when odours are nontrigeminal stimulants.

12  What does the tongue contribute to tasting flavours? So far we have acknowledged the hidden complexity of our tasting experiences and the fundamental contribution smell makes—especially that due to retronasal olfaction—to our experiences of flavour. Tastes rarely occur alone. Putting aside contributions from smell, there is still touch. We speak of foods as creamy, or chewy, crunchy, sticky, or oily. To experience tastes proper without other sensory additions one would need to anaesthetize the tongue not just for feel but from movement or pressure: something that cannot be achieved save in clinical settings. Nevertheless, pure taste plays a vital role in our tasting of flavours. So at this stage it is worth asking just what the tongue does contributes to the experience of flavour, and whether it contribute the same in each of us. Remember, taste proper concerns gustation alone where this covers a range of responses to chemical stimuli, including salts, sugars, acids, and toxins by different classes of taste receptors on the tongue, in the oral cavity, and in the gut. (Sweet taste receptors in the gut play an important part in insulin regulation.) Receptor firings on the tongue code for the so-called ‘basic tastes’ such as salt, sweet, sour, bitter, as well as the fifth taste, umami (savoury). To basic tastes we can add a sixth taste, metallic, and perhaps a fatty acid taste, although there is still controversy about fat as a basic taste (see Mattes, 2009). The list is

The Chemical Senses   331 determined in part phenomenologically—by characteristic experiences of human tasters associated with different kinds of stimuli—and in part neurobiologically—by the discovery of specific receptors for detecting each of these tastes. It is the task of psychophysics to align these two ways of picking out qualities like sweetness, bitterness, or saltiness. How are they aligned? It took discoveries in neurobiology to convince many Westerners to accept the positing of a fifth ‘basic’ taste, umami.11 Although tasting it everyday, the fifth taste remained unknown for some time, and even now it is still not easy for people to recognize it as a common element in the flavour of tomatoes, mushrooms, soy sauce, peas, parmesan cheese, and seafood, though it is a natural category for the Japanese. By contrast, English-speaking countries often confuse sour and bitter; leaving open the question of whether they cannot tell experiences of these two tastes apart, or whether they use the term ‘sour’ for instances of either taste. With training it is easier to distinguish sour and bitter by use of examples like lemon juice and quinine (or caffeine). Part of the problem may be that bitter-tasting substances like coffee can also exhibit sourness. The list of basic tastes has also changed through history. In medieval England, cooks recognized at least eight basic tastes with two categories for salt: salty and salty like the sea, as well as sour and vinegary (Woolgar, 2006). Even now, there are good grounds for saying there is no single taste of sour, or of bitter. We have several different types of receptors for detecting bitter chemicals. There are also reasons to worry about identifying the basic tastes by a subject’s experience of tasting, since gustatory tastes are modified by other influences. Mojet and her colleagues (Mojet et al., 2005) have shown that the basic tastes differ in perceived intensity when subjects taste them with and without a nose clip, thus showing that smell makes a contribution to the normal experience even of the so-called basic tastes. As we saw above, subjects who lose their smell tend to believe, falsely, that they cannot taste. This shows how unusual it is to experience salt, sweet, sour, and bitter just as tastes in the absence of smell, and how unrecognizable such experiences are at first to those who lose their sense of smell. The ordinary experience of a supposed basic taste is probably in fact the experience of a flavour: the result of combining taste and smell (see Spence et al., 2014). We are equipped physiologically to respond to the basic tastes and some researchers have suggested that labelled lines of neural relay from these distinct receptors carry information about particular groups of stimuli, such as salts, acids, toxins, to the primary taste cortex in the insula and from there to the orbitofrontal cortex. The separation of the basic tastes would seem to follow from the labelled line hypothesis: that receptors for salt, sweet, sour, bitter, etc., carry discrete information about each of these tastes to the brain. But how are they separately experienced phenomenologically? Philosophers often speak of experiencing a taste, such as the taste of honey, as if it were an irreducible item of our phenomenology. In fact we can dissect the tasting of honey, and even more basic flavours, into (i) the quality by which the experience is identified (here, its sweetness), (ii) the intensity of 11  First posited in 1908 by Ikeda, who wrote, ‘Those who pay careful attention to their tastebuds will discover in the complex flavour of asparagus, tomatoes, cheese and meat, a common and yet absolutely singular taste which cannot be called sweet, or sour, or salty, or bitter  . . .’ Dr Kikunae Ikeda, Eighth International Congress of Applied Chemistry, Washington 1912. It was confirmed as a basic taste with its own proprietary receptors in Chaudhari et al., (2000).

332   Barry C. Smith the experience, (iii) its temporal duration, including its onset and offset times, and (iv) its spatial location on the tongue. Let’s explore these aspects of tasting further. Just what are we perceiving by means of basic taste detection? Unlike some other components of flavour mixtures, salt seems to retain a recognizable identity. We can often discover whether foods contain salt just by tasting them, and on the basis of intensity determine whether foods are too salty. Do we therefore perceive salt when the response to the stimulus is above our individual detection threshold? Not necessarily: low levels of salt concentration just above threshold can appear sweet to a taster, and ‘concentrations of salt that are weaker than levels in saliva may give rise to bitter tastes’ (Breslin, 2000: 435). If we take the function of such taste sensations to generate perceptual experiences of salt, then these cases count as misperceptions. In these cases, the quality of the experience is modified by differences in the intensity of the stimulus. Moreover, saltiness is a response not only to sodium chloride but also to other salts, like potassium chloride. We can discriminate these by tasting, because potassium chloride has not just a salty taste but also a slightly bitter taste. Again, with sweetness, we can distinguish between sucrose and artificial sweeteners, because Aspertame has a later onset time and longer offset time; here it is the temporal difference in the onset and duration of receptor firing that allows us, phenomenologically and perceptually, to distinguish between the two. Hence, the different aspects of taste sensations identified above play a crucial role in determining the character and object of the resultant perceptual experience. Tastes also differ in their perceived location on the tongue. The receptive fields of the different taste receptors are not uniform in their sensitivity on all parts of the tongue (Collings, 1974). So we can localize our greatest sensitivity to bitterness at the back of the tongue, to sourness at the sides, and sweetness at the front, even though we have greater receptor capacity to detect bitterness at the front. Detecting bitterness at the back of the tongue gives us a last resort to reject before swallowing potentially toxic, bitter tasting compounds. Other changes to the experience of a taste result from increase in perceived intensity or concentration of the stimulus. When salt concentration is too high the perceived intensity creates a burning sensation, which is more accurately described as feel or touch, rather than a taste. There are other cross-modal effects involving taste. Cruz and Green (2000) have found that thermal stimulation of the tongue can evoke sensations of taste in about a quarter of the population. Warming an area of the tongue can give rise to a sweet taste, while cooling it can give rise to a salty or sour taste. Green calls these subjects thermal tasters and although the effects among this population are quite stable and affect their tasting of foods and drinks, we can think of these as illusions of taste. There are other ways of producing taste-like sensations through other modalities, including electrical stimulation of the tongue, odour-induced tastes, and colour-induced tastes. The latter occurs when subjects are given coloured drinks to rate for sweetness or sourness. The drinks do not contain any tastants. They are tasteless and odourless, and yet a significant proportion of subjects rate the lime green drink as sour and the cherry red drink as sweet (Spence et al., 2010). The case of odour-induced tastes is crucial for our understanding of the extent of flavour perception (see below): To experience odor-induced tastes, one need simply sniff odors such as strawberry, vanilla, mint, or chocolate, all of which are routinely described by Westerners as ‘sweet smelling’. (Stevenson and Tomiczeck, 2007: 295)

The Chemical Senses   333 This is a case of sweetness enhancement, where the orthonasal presentation of a congruent odor in the presence of a taste can increase the perceived intensity of the taste, and suppress sourness, thus functioning like taste proper. Some researchers have even reported that subjects can, under certain circumstances, experience sweet taste in a water solution with no sucrose when it is accompanied by a retronasally administered odour of isoamyl acetate (banana) (Hort and Hollowood, 2004). This phenomenon of sweetness enhancement is a cross-modal effect where the workings of a single modality, taste, is modified in that modality by stimulating another modality, olfaction, without the later being fused with taste into a single more complex flavour percept. The increase in perceived sweetness for the same item tasted and rated for sweetness before and after accompanying with a sweetness-enhancing odour (vanilla or strawberry) is superadditive. The result of combining a sucrose solution with the odour is greater in terms of neural activation and experiential effect than the summation of activation of the two components. This phenomenon of sweetness enhancement is a cross-modal effect but in this case we do not get the multisensory integration or fusion of ordinary (orthonasal) smell and taste into a single percept, but merely the influence of one modality on another. We still recognize the distinct components of orthonasal olfaction and gustation. Temperature has cross-modal effects on taste too: when coffee gets cold this increases its perceived bitterness. All of these taste-altering effects can be induced under special conditions, but in normal conditions of perception do tasters perceive the same things? Does the psychophysics give us grounds for intersubjective agreement about the basic tastes? As we saw earlier, there is cross-cultural variation in taste experience. In addition, there are complexities introduced by blending the basic tastes in mixtures. Sometimes, combined tastes don’t blend but keep their separate identity, as in sweet and sour sauces. At other time, sweetness suppresses sourness. Umami can enhance saltiness: so by adding monosodium glutamate to a dish it can taste saltier. Combining different types of umami creates synergistic reactions to intensify flavour. These produce greater intensity and are the basis for popular flavour pairings such as anchovy and tomato in pizza, and scallops and pea puree. Combining all the basic tastes usually leads to an overall suppression in taste intensity, unless one includes umami. This is maybe why tomato ketchup is so popular it combines all of the five basic tastes. Despite being able to recognize salt, sweet, sour, and bitter, people often disagree about the intensity of these tastes in a dish, arguing about whether the food has enough salt or too much. Does this show that tasting something as salty is wholly subjective? Not necessarily; tongues could have different numbers of fungiform papillae or taste buds. Those who have dense clustering of papillae on the tongue often show a quick and intense reaction to salt, sour, or bitter compounds. The explanation is clear in the case of salt. What makes something taste salty is the number of salt receptors that are simultaneously touched by NaCl molecules. So if an individual’s taste buds are densely packed they will need little salt, whereas those whose taste buds are more widely spread out will need lots more salt to get the same effect. Using this fact, nanotechnologists have been working on a salt molecule that will unwrap itself and touch more receptors simultaneously leading to the use of less salt for the same perceived effect. So, while individuals have different sensitivities, they are picking up on the same quality and just perceiving it differently.

334   Barry C. Smith Those who show an increased sensitivity to basic tastes are the so-called super-tasters. The term was coined by Linda Bartoshuk (see Bartoshuk et al., 1994) who fashioned a quick test to sort people into three groups: super-tasters, tasters, and non-tasters. The test is based on their sensitivity to tasting strips soaked in PROP (propthiouricil) or PTC (phenylthiocarbamide). When the strip is placed on the tongue people show one of three reactions—they find it intensely bitter, mildly bitter, or tasteless. There is a normal distribution with most people being (mild) tasters, 25 per cent being super-tasters and 25 per cent being non-tasters. General conclusions have been drawn from the test about how sensitive these groups will be to other basic tastes, but the conclusions have been overdrawn. Juyumn Lim and Barry Green (2008) have shown that while high sensitivity to PROP correlates with sensitivity to other tastes, there is no evidence that non-tasters have lower sensitivity to other tastes. Some PROP tasters do not even show increased sensitivity to bitter tasting quinine. Lim and Green suggest that we may need to distinguish between PROP tasters and general tasters, and that only the latter category is useful for predicting oral sensitivity and preferences. General super-tasters are often conservative eaters, finding some foods too bitter and white wines too sour; whereas low sensitivity (non-)tasters are more adventurous eaters and enjoy spicy foods. Super-tasters with more taste buds on their tongue will have more trigeminal nerve endings exposed to irritation by spices, which may explain why they tend to avoid spicy food and strong alcohol. Bartoshuk is right, though, that such sensitivity is a spectrum and at the lowest end we find people for whom no dish is too sweet or too salty. So before we try to draw conclusions about the subjectivity of taste from disagreements in judgements about the saltiness or sweetness of a dish we need to be sure we are comparing like with like and that the same phenomena are in dispute. Finally, we must not confuse the subjective experience of tasting that depends on detecting a basic taste, with the hedonic reaction to it. What we taste is one thing, and what we feel about it is another. The liking for sweetness and the disliking for bitterness are innate but they are not fixed and may be changed by experience or conditioning.12 Disagreements about taste are harder to establish than is commonly thought, and attempts to make philosophical hay from then have to be treated with caution (see MacFarlane, 2007; Smith, 2010).

13  Flavour as a psychological construct—the sophisticated sensation View As we saw earlier, the simple sensation view of tastes as the experiences we have when we eat or drink is hopelessly flawed. The items we consume strike us as fruity, creamy, spicy, sour, meaty, oily: and these properties, or our experiences of them, can only be had through the combined workings of the chemical senses and touch. So the simple sensation view of tasting is too simplified to cover the range of phenomena we include under experiences of tasting flavours. 12 

Things initially found bitter, such as coffee and alcohol, may come to be liked and desired as a result of the post-ingestive reward effects of alcohol and caffeine.

The Chemical Senses   335 The sophisticated sensation view denies that ‘taste’ (flavour) experiences are simply complex bundles of sensations. Flavours are experienced as unified wholes without a partwhole structure. Yet despite their unified appearance, flavour experiences arise through the multi-modal integration of taste proper with retronasal olfaction, to which we have to add somatosensory sensations of texture and temperature, mechanoreceptors triggered by chewing, and often the chemical irritation of the trigeminal nerve. The unity of flavour percepts seems to be created by the brain, so, on this view, there need be no flavours in nature: the flavours we experience only come into existence when our brains combine information from these different sensory systems. They are just psychological constructs, or brain products. This is psychologist John Prescott’s position (1999) and many others in psychology and neuroscience have followed suit. A fairly typical statement of the view is given by Dana Small, echoing remarks made by sensory scientist Gordon Sheppard: Flavour is in the brain, not the food. It is the brain that unites the discrete sensory inputs from the food and drinks we ingest to create flavour perceptions. (Small, 2012: 540)

In what follows, I will resist this position and make out a case for the objectivity of flavours and flavour perception. But first let us take a critical look at the view of flavours as constructs. There is a slip in the immediately preceding quote between the first and second sentence. Small is right that the brain binds discrete sensory inputs to create flavour perceptions, but this doesn’t support the claim that brains create flavours. Flavours are one thing, flavour perceptions are another. Many empirical scientists simply elide them, but without a distinction between the perceptions and what they are perceptions of, it becomes hard to draw a line between veridical and non-veridical perceptions, and harder still to know which aspects of the overall experience of eating and drinking contribute to flavour perception. What is it that makes an experience one of flavour? When it comes to defining flavour experiences, psychologists tend to be maximalists. Here is Martin Yeomans: arguably, multi-sensory integration may be at its most extreme in the case of flavour perception since few other experiences offer the opportunity for concomitant stimulation of all the major senses: gustation through the five primary tastes, olfaction through both orthoand retronasal stimulation of olfactory receptors by volatile compounds released from food, mechanoreception contributing to our perception of texture and providing information on temperature, pain arising from oral irritants and hearing that results from sounds and vibrations coming from the mouth contributing to our perception of aspects of texture. (Yeomans et al., 2008: 565)

Notice that we can selectively attend to some of the multisensory components of flavour experience, such as the contribution of touch: i.e. when a food or liquid is described as being creamy, oily, crunchy, or melting; these seem to be part of flavour experiences. But what of the look of a dish, or the sound of a crunchy food? Are they part of flavour? They can affect how we experience foods and drinks. For example, the colour of liquids can influence our perception of sweetness or sourness (Spence et al., 2010). So should we speak of a visual component of flavour? And in a classic experiment by Zampini and Spence (2004), participants who ate stale potato crisps while wearing headphones that amplified the high frequency sound of their own crunching found the crisps to taste fresh. So is sound part of flavour?

336   Barry C. Smith Often we don’t know that these visual or auditory inputs are changing our perceptions of flavour. We cannot phenomenologically separate them from the flavours we would experience anyway, nor can we attend to the contribution they make, as we can do with, say, the effect of the touch or texture of foods. Notice, though, that the contribution of touch is not always so easily phenomenologically separable. We say biscuits ‘taste’ stale, but these have the same taste and smell properties as fresh biscuits; but crumble differently in the mouth. It is this texture clue that leads to us say they ‘taste’ stale. And while, without input from sight and sound we still taste flavours, it would be hard to experience tastes or flavours without touch. But is touch part of flavour? To experience tastes without a sense of touch, pressure, or localization in the mouth requires one to have the chorda tympani nerve anesthetized. How would the free-floating tastes be experienced? Too little is known. So touch usually accompanies taste and smell, but is it part of flavour? Calling something creamy is attributing something to its flavour. Also, the creaminess of a food can affect its smell and therefore it’s flavour, and, conversely, certain odours can increase the perception of creaminess and richness of a food in the mouth (Bult and Hummel, 2007). An example of tactile inputs to tasting, though not recognized as such, is the experience of black pepper. The tongue doesn’t have taste receptors for black pepper. It activates nerve endings that belong not to taste but to touch, though they make a genuine contribution to our experience of the flavours of food and drinks. The chemical irritants in mustard oil and peppermint oil (or I-menthol) elicit different responses in the trigeminal nerve. Mustard feels hot and peppermint cool, although there is no change of temperature in the mouth. These effects are due to chemesthesis. The extension of the burning and stinging to other places on the skin shows that chemesthesis is not a single sense but overlaps with nociception and thermoreception and is part of somatosensation. The trigeminal nerve, though, is crucial to flavour experience and one of the hidden flavour senses. It governs our response to carbonation in fizzy drinks and sparkling wines. CO2 is a trigeminal stimulant that produces the prickle we feel on the tongue and around the mouth. The stimulation of trigeminal nerves in the nose accompanies most but not all odorants and boosts olfaction. Thus those who lose smell can sometimes continue to detect strong odours by means of trigeminal stimulation. (Should we think of this as smelling by feel?) Does the sense of touch play a role in flavour and flavour perception, and is it just a single sense of touch that is involved, including as it does tactile sensations, haptic explorations, thermal responses, pain, and chemical irritation? This just shows, perhaps, how difficult it is to think of touch as single sense, let alone flavour as a single sense.13

14  The nature of flavours and flavour experiences Taking all these factors into account, how should we understand our experiences of flavour? How should we react to the claim that there are non-phenomenologically distinct 13 

Many (though not all) would see touch as encompassing several senses (although, see Fulkerson, 2014.)

The Chemical Senses   337 components that contribute to the experience of tasting a single flavour? Should we let the phenomenology dictate the ultimate nature of flavour experiences, or should we individuate flavours via the interacting factors that give rise to our experiences of flavours? Should we view them as the complex products of multisensory integration? Or, as conscious experiences. Should we take them to be just as they seem to us—whole, unified, unimodal experiences? The trouble with both of these ways of proceeding is that they seem to let in all sorts of contributions from external touch, sound, and sight to the constitution of flavour. Let us consider the scientifically informed way of individuating flavours and flavour experiences as constructs of the brain by looking at the following definitions:

(1) Flavour is a ‘complex combination of the olfactory, gustatory and trigeminal sensations perceived during tasting. The flavour may be influenced by tactile, thermal, painful and/or kinaesthetic effects’ (AFNOR, 1992). (2) ‘Flavour perception should be used as the term for the combinations of taste, smell, the trigeminal system, touch, and so on, that we perceive when tasting food’ (Auvray and Spence, 2008, italics mine). (3) ‘Flavor perception arises from the central integration of peripherally distinct sensory inputs (taste, smell, texture, temperature, sight, and even sound of foods)’ (Small et al., 2004). (4) ‘Taste’ is often used as a synonym for ‘flavour’. This usage of ‘taste’ probably arose because the blend of true taste and retronasal olfaction is perceptually localized to the mouth via touch (Bartoshuk and Duffy, 2005: 27). Each of (1) to (4) acknowledges that the flavour experiences we have when tasting depend on sensory interactions, but they vary in mentioning different elements in the interactions. And despite being offered as definitions of flavour, what they actually describe are flavour experiences or flavour perceptions. (The definition of Auvray and Spence in (3) is more careful though elsewhere they collapse flavour and flavour perception.) If this conflation is deliberate then flavours should be inseparable from our experiences of flavours. However, the objectivist who seeks to distinguish flavours from flavour perceptions can appeal to cases where something can causally affect our perception of a food or liquid’s flavour without it being a factor in flavour perception. Certain elements can influence, without being part of, flavour perception. Here, we need a distinction between constitutive and causally affecting factors in flavour perception: i.e. between components that contribute to what a flavour perception is and those that merely have a causal effect on flavour perceptions. This is not an easy distinction to draw, but before we try to do so, let us look at the problems that arise for those who take flavours to be exhausted by flavour experiences. What can they tell us about the boundaries to flavour experiences? What does and doesn’t belong in the list of contributing factors? Researchers in psychology and neuroscience agree on what is going on when we eat and drink and which factors are present. What they disagree about is which factors belong to flavours and which don’t. For Auvray and Spence, taste, smell, touch, and the trigeminal systems are involved and maybe more; while for Bartoshuk, touch is not part of flavour. It merely plays a role in helping to combine taste and retronasal olfaction. For Yeomans, and maybe Small, sight and sound are also in there.

338   Barry C. Smith Why stop there? Is the hedonic value of the experience part of flavour? After all, when we eat something we immediately assess it for pleasantness or unpleasantness. Tasting is an evaluative activity, letting us judge whether something is delicious, bland, or disgusting. Some people take the whole point of tasting not to be about discovering the flavours of the food or drink, but about arriving at a hedonic rating: they like it or don’t, which will decide subsequent behaviour. It is also difficult to separate the unpleasantness of what we’re tasting—sea urchin, say—from the food itself, which is why we find it so hard to understand how others can overcome this flavour and enjoy it. So should we include the hedonic dimension of eating or drinking in our account of flavour making that notion even more subject to the varieties of subjectivities among tasters? For Yeomans, the issue of whether hedonic value is part of flavour is indeterminate since they go together and are tested together. Even expectations of the same food generated by different words can have an effect on its acceptability. Here is how it works. Yeomans et al. (2008) gave two groups of subjects smoked salmon ice cream to taste: one group was told that it was ice cream, while the other was told it was frozen savoury mousse. The first group liked it less than the second and reported it as tasting saltier and more savoury than the second group. So according to Yeomans: flavour perception is an integration of sensory information with past memory of similar stimuli predicated by the visual qualities, and accompanying written descriptor, for the rated food. (2008: 569, italics mine)

Verhagen (2007) argues that we should include hedonics in flavour since the processing in the food recognition network is paralleled by processing in a different but overlapping reward value system. The neural evidence is mixed, however, and does not entirely support this verdict, as we shall see below. But why stop here? The experience of eating and drinking responds to the posture of our bodies, the ambient temperature, sounds we are listening to, as well as other affective states. So why not consider the total sensory input to the overall experience of eating or drinking as part of flavour? And if not, what are the constraints on the combination of co-occurring sensory inputs? We know some of the rules of binding across sensory modalities. In the case of multi-modal flavour experiences, the key idea is congruence. Lim and Johnson (2011) show that it is congruent tastes and retronasally presented odours that lead to the referral of the olfactory component of flavour perception to the oral cavity. The subjectivist who thinks flavours are in the brain owes us an account in purely internal terms of what makes inputs from different sensory modalities congruent or incongruent. (This will be a more or less notion for the sophisticated subjectivist.) Perhaps, it will be said that it is a matter of their occurring together, but this would include all the cases of potential illusions we worried about above. The objectivist can better explain congruence in terms of the configurations of odorous and sapid properties in the foods that make up the naturally occurring flavours. Banana odour and sweetness are combined in bananas, and they are integrated when one is retronasally and the other is orally registered. So why not think of bananas as having a flavour which it is the job of the flavour perceptions produced by the brain to track? No such appeal is possible for the ‘flavour is in the brain’ theorist. She still hasn’t told us what flavours are. Do they include sounds and colours? As pointed out above, researchers

The Chemical Senses   339 agree about the nature of co-occurring inputs. What they disagree about is which inputs count as parts of flavour. Empirical inquiry will not settle this matter. So, if there is a line to be drawn between what constitutes flavour experience and what merely causally affects it, this will be drawn theoretically or philosophically by deciding on the best overall account to make sense of the phenomena in the light of the underlying facts about neuroanatomy, neural connections, behaviour, experience and stimuli. One might, at this point try to argue for a single flavour sense, whatever the underlying processing story, and try to pick out experiences generated by that sense on the basis of their characteristic phenomenology. But how sure are we that we can focus on our experiences for the purpose of picking out what is flavour and what isn’t, and how sure are we about what belongs exclusively within that category? Touch contributes to how things taste to us and certain aspects of touch, temperature, stinging, and burning sensations from trigeminal irritations can be attended to in isolation. So how does this sense relate to the single flavour sense? In one way, it seems to be part of it, and in another it is a separate but overlapping sense. There is also the experience of a menthol flavour that we can break down into a minty aroma, a bitter taste, and a cool sensation. So is flavour perception, or more accurately, the sense that generates flavour experiences, unimodal or multi-modal? Consider the four definitions of flavour (perceptions) just given. How are the components mentioned in each definition bound together, integrated, or blended? More importantly, what are the components that get bound together in flavour perceptions? The different definitions of flavour quoted above go different ways on the how and what questions. Answers given to the how and what questions constrain one another: what is the ultimate product of combining inputs from different sensory systems, and how do the different sensory inputs combine? We do not yet have a full explanation of multisensory integration or flavour binding, but various proposals, principles, and criteria have been offered. These include: (i) Spatio-temporal unity (ii) Superadditivity (iii) Semantic congruence (i) Spatio-temporal unity hypothesis treats the unity of flavour as arising from the fact that sensory information of various kinds, from various sources, are put together when presented close in space and time. This leaves open how we arrive at a unified percept. Perhaps it is no more than that what fires together wires together. But this is not sufficient. Interactions between multi-modal components go far beyond mere co-occurrence in consciousness. Is the hedonic component of eating and drinking a constitutive part of flavour or flavour perception (Smith, 2007, 2010)? It would be hard to exclude it on the spatio-temporal unity hypothesis. But we should exclude it. Even if the main purpose of eating and drinking is to determine whether particular foods are pleasurable, this is nonetheless different from how the food or drink tastes and what its qualities are. If force-fed the same food repetitively, even a food liked as much as chocolate, is disliked after excess consumption. The identity of the stimulus stays the same even when the hedonics vary, for if you were suddenly offered

340   Barry C. Smith a different type of chocolate you would notice the difference (Kringelbach and Stein, 2010; O’Docherty et al., 2000). (ii) Superadditivity occurs when the neural activation level of two or more sensory inputs is greater than the sum of their individual activation levels, as we saw above when both seeing and hearing a water bottle being crushed leads to greater neural activation in the perceiver than the sum of the separate activations for the visual and the auditory stimuli. Some take superadditivity to be a clue to multisensory integration, signalling how significant it is for the brain to link these pieces of information to track single objects or events. Perhaps, the integration that gives rise to unity of experience starts from a unity assumption about the different sensory inputs coming from a single object or event. The problem is that superadditivity can also occur without integration. The perceived creaminess of a food or liquid in the mouth can be affected by aroma. This does not yet create the kind of multisensory experience that results from the integration of information from different sense modalities.What of the perceived weight of a bottle or feel of a food in the hand? These factors can causally affect our perception of flavours (Piqueras-Fiszman, 2011), but they surely are not constitutive of flavour. This is the philosophical point that needs to be stressed. Not every component of our experience of flavour reveals a component of flavour itself. The difficultly is that although there may be agreement about all that takes place during eating and drinking, there is no empirical agreement about which parts of the overall experience constitute flavour. We need a philosophical account sensitive to the empirical facts. (iii) Semantic congruence. Does semantic congruency reveal a necessary feature of the relations that constitutes flavour? For example, strawberry odour + (congruent) sweet taste are combined into a flavour, but strawberry odour + (incongruent) salty taste are not, or less so. Congruency could also explain the role of expectation in flavour experiences. We get oral referral—by which flavour is experienced as located in the mouth, even though some components originate from the olfactory cleft and elsewhere (Lim and Johnson, 2011)— most for congruent taste-odour pairs. So congruency could help explain localization. Can the congruency hypothesis explain flavour perception by explaining the referral of retronasal olfactory components to the oral cavity?The full congruency hypothesis would be as follows: unified flavours are constituted by congruently related sensory cues. Congruency is not to be thought of as ‘all or nothing’—a sensory cue S1 is more or less congruent with sensory cue S2. Flavours would be constituted by these various ‘more or less’ congruency relations. Their role in determining flavours suggest that they intervene at the level of processing and (somehow) determine categorization resulting in determinate flavours (e.g. chicken); or they can be manifested as flavours with more or less resolution (e.g. chicken-like flavours). If the congruency of different sensory stimuli or inputs is supposed to explain the unity of flavour experiences, we need to ask what explains congruency? How is the notion to be understood by the subjectivist who thinks of flavour as in the brain: the result of combinings? It would be easy to explain congruency by saying that congruent features are considered to be attributes of the same kind of object. This is the idea offered by Lim and Johnson (2012), who argue that one needs an ecologically appropriate taste in the mouth,

The Chemical Senses   341 not just touch, to get oral referral of retronasally sensed olfactory components like fruitiness. This presupposes a unity to which these features belong as an ecologically valid part of the environment. This implies that flavour is not made in the brain, since the basis for congruence is ecological. Subjectivists can, however, fall back on sensory congruencies. They can say that two features ‘match’ when the estimation of the one affects the estimation of the other: the darker the colour, the more intense the flavour (ripeness), the heavier, the thicker the yogurt (density). Without resort to semantic congruencies, determined by naturally co-occurring configurations of texture, taste, odour, and irritant properties of foods and liquids, we just have these subjective, internal, and unexplained experienced sensory congruencies to determine which bundles of inputs are flavours. On such an account, flavours would reveal nothing beyond themselves.But rather than settle for this unexplanatory stopping point, with no way to prescribe precisely the extent of flavour experiences, there is the objectivist strategy of positing flavours as properties of foods and liquids. On this view, we could see flavours as affordances which our capacities for multisensory flavour perception track to guide successful food choice. By recognizing that flavours are external features of the environment we need not see flavours and flavour perceptions as always coinciding. The term flavour does not describe a construct of the brain, but it is a technical term used to describe properties of a solid or liquid. On this view we can distinguish the hedonics of eating from the perceptual experience of tasting.

15  Flavours and flavour perception Flavour is a configuration of the sapid and odorous properties of a substance, including its temperature and texture, as well as its power to irritate the trigeminal nerve. So when speaking about the 'taste' of a food, we are actually speaking about its flavour. This point is often missed because we fail to notice all the components of our tasting experiences and because we are unaware of the large role smell plays in sustaining them. Flavours are perceived when retronasal olfaction and gustation jointly give rise to a fused percept of flavour as a result of multisensory integration at the sub-personal level. We perceive flavours by the mechanisms of olfaction and gustation, under the right conditions, involving touch and sometimes irritation of the trigeminal nerve. What is flavour perception for? The flavour perception system guides successful food choice. It needs to pick out and track perceptible properties, or sensible qualities of foods and liquids. And to explain why we bind the sensory elements we do and not others we rely on inference to the best explanation. Multisensory integration tends to take place when there is a common environmental source or unity responded to by many modalities. These are flavours—configurations of properties—that can give rise to multisensory responses in creatures like us. Having dwelt a great deal on the role of smell in flavour perception, the next section will deal with remaining issues about our ordinary, orthonasal sense of smell.

342   Barry C. Smith

16  The sense of smell and its role in experience The role of the sense of smell in perception and conscious experience is subtle and elusive and this creates a puzzle for philosophy of perception about the nature of olfactory experience. Many say that they hardly notice their sense of smell and imagine it would be easy to give it up if they had to lose one of their senses. But smell is always with us: we smell because we breathe and we live in a world full of odours that subtly shapes our moods, influences our eating habits, our choice of sexual partner, our recognition of kin, and our response to one another’s fear or aggression. Odours and our responses to them play an important part in generating memories, particularly early memories; and these responses help to create feelings of familiarity and presence in certain surroundings. Given this wide role that olfactory system plays in the conscious experiences of daily life, it is surprising that we pay so little attention to smell. Often, it is only when people sniff at something that they become aware of their sense of smell. All the other ways odours contribute to, or condition, our experience are perhaps mediated by the unconscious workings of the olfactory system. Or is it that smell is so much part of the fabric of experience—so much a background to our conscious states—that we are unaware of it until we attend to it? There is no straightforward answer to this question, which is why smell creates a puzzle for the philosophy of perception. Consider, the connection between smell and memory. The highly emotive character of many odour memories is thought to be due to the direct connection between the primary olfactory cortex and the amygdala and entorhinal cortex that are involved in emotional and memory processing. When we get a whiff of scent that suddenly triggers a memory of a far distant time, it seems to take us right back there. The feeling of recognition is surely a way of tapping into a remembered episode of which the triggering odour was a part, whether noticed at the time or not. Importantly, the odour-memory does not simply conjure up a past experience of smelling, but rather a multisensory scene with sights and sounds. Reasons given for thinking we don’t experience smells all the time include our supposedly poor sense of smell. But how good or poor is the human sense of smell? It can be trained and shows improvement (until old age), unlike our senses of sight and hearing. The increased sensitivity and ability to discriminate and recognize odours shown by perfumers and wine tasters attests to the fact that humans may have a reasonably good sense of smell. As Sela and Sobel put it: ‘Paradoxically, although humans have a superb sense of smell, they don’t trust their nose.’ Navigation is an interesting case. As upright creatures, we rely more on our eyes and our ears. Nevertheless, we retain olfactory navigational capacities. Noam Sobel and colleges have demonstrated that when blindfolded and on all fours, humans can navigate by using odour (chocolate) trails (Porter et al., 2007). People will take longer than dogs, but they will succeed. Sobel’s research group also demonstrated a limited human ability to locate odour sources in space (Porter et al., 2005). Participants were fitted with a mask that had an artificial septum to exaggerate the separation of the nostrils. When using the mask, participants showed a rate of accuracy for the spatial location of odour sources of around 70 per cent, suggesting that the brain

The Chemical Senses   343 was using a mechanism akin to bi-aural hearing in audition. In commenting on these results Jay Gottfried asks the relevant question: does the apparatus endows participants with a new ability or just uncovers an old one (Gottfried 2007). The lack of training and the immediate improvement in olfactory spatial awareness, lost again when the mask is removed, strongly suggests the answer is the latter. Smell is constantly with us, and unlike other animals we don’t need continuous sniffing to smell odours in our environment (pace Richardson 2013, though see Sobel, 2006). Active sniffing may be useful when we are trying to locate the source of an odour, or making special efforts to take in, or attend to a particular smell. Normally, odours simply reach the nostrils through the dispersal of volatile molecules. Pleasant and unpleasant smells aside, above-threshold odours often go unnoticed because of sensory adaptation. You no longer notice the smell of your own home, though you register it well enough to notice change straightaway. (Is something burning?) Normally, it is only when you return after a period of time that you become consciously aware again of the smell of your home. How much of olfactory processing results in the conscious experiences of odours? Is ordinary olfactory experience just unattended to, or is there less to olfactory experience than, say, visual and auditory experience? To answer these questions let us look at the role smell plays in perception and conscious experience more generally. Jay Gottfried holds that the function of the olfactory system is to track behaviourally relevant odours in the environment and extract meaningful information from them. To do this, higher-order brain regions, such as the orbitofrontal cortex, are recruited to assemble patterns of odour qualities encoded at the level of olfactory receptors. The receptors activated by an odorant do not map onto the odour percept directly; the olfactory receptor neurons will code for thousands of different volatile compounds that get synthesized into a unified whole. Coffee has over 800 volatile compounds, chocolate over 600, and yet they are perceived as a whole. The route from odorant to odour quality coding and categorization to conscious percept is largely unknown for a variety of reasons. Over 1,000 different receptor types are attuned to aspects of particular odorants, but these do not determine the resulting percept: ‘the same olfactory input may generate different odor percepts depending on prior learning and experience’, and perceptual learning will continue to modify odour percepts: ‘neural representations of odor quality can be rapidly updated through mere perceptual experience’ and ‘[l]‌earning also changes odor quality. For instance, cherry odor becomes smokier in quality after being experienced together with a smoky odor’ (Stevenson, 2001 quoted by Li et al., 2006: 1097). Experience and familiarity significantly enhance odour quality discrimination, while exposure to odour mixtures alters the perceived quality of the individual components (Gottfried, 2007). Wilson and Stevenson (2006) also subscribe to the view that the brain’s job is to synthesize olfactory objects that correspond to collections of volatile molecules in the perceiver’s environment. They term this the Object Recognition Model. There is evidence that the brain’s ability to represent the wholes of olfactory processing is configurational, like face recognition, rather than analytical, in terms of their parts, which may be why we are poor at identifying an odour’s constituents. We can still recognize mixtures, though we seldom identify more than two or three constituents in a mixture of six or more elements (Laing et al., 2002). We can also recognize the simultaneous presence of many odours. For example, it is possible to smell fresh coffee and bacon frying, without these blending into a single odour percept. Should we think of these as distinguishable odour objects or as representing overlapping properties around us with no definite spatial location? The latter is Clare Batty’s

344   Barry C. Smith (2010) view. The lack of definite objects as sources is due, she thinks, to the smudginess of the odour trails we perceive. However, there are clear cases where we are aware of a source object for an odour, be it a flower or someone’s hair. Many sensory inputs may be involved in identifying the odour source, just as there is integration of vision and audition to identify the sound source of a human voice. Gottfried and Dolan (2003) showed, moreover, that subjects detected an odour more quickly and accurately when paired with a semantically congruent image: e.g. a picture of a bus with diesel odour, and of a lemon with citrus odour. Just as we can experience mixtures, and co-occurring odours, we can tell when an odour is complex (or one is more complex than another), even though we are unable to identify any of the constituent parts. What gives us this cue as to an odour’s complexity? It may have something to do with the temporal dynamics of smelling and the differential rates of processing different odorants. We may be unaware of the temporal sequence though it may leave a trace indicating its complexity. What we know, however, is that there is no obvious relation between perceived complexity and the chemical complexity of the compounds involved. Single molecules can be perceived as having parts; that is, a single compound like salicyladehyde can produce multiple smell percepts, appearing to smell both like aspirin and like almonds (Lawless, 1997). Perceptual constancy for odours is an important feature. Gottfried points out that odour objects are constantly maintained by processing in the piriform cortex despite variations in intensity. However, the perceived quality of a compound will change when the intensity changes greatly. Hexanoate at low intensity in wines can be perceived as pear, then at slightly higher intensity as grapefruit, while at higher intensity still it comes to be perceived as fecal. Notice that odour perception is maintained throughout gaps in perception. As Ophelia Deroy points out (p.c.), we breathe out as well as in, but we nonetheless recognize an ambient odour as present throughout. Batty’s view of the representational content of olfactory experience leads her to deny there can be olfactory illusions. On her view, olfactory experiences merely represent odour properties, not objects. Thus it cannot represent an object as having a property it doesn’t have. A problem with Batty’s view is its exclusive focus on unimodal olfactory perception. However, in the normal case, perceptual experience is multi-modal. We see, hear, feel, smell, and sometimes taste the objects we interact with, and in that context our overall perceptual experience of a smell can be illusory if it represents a familiar seen or felt object as having a surprising and perhaps disturbing smell. One patient reported (p.c) finding that cartons of orange juice she had bought smelled of fish. The smell was attributed to this multi-modal object; she was suffering from a vivid olfactory illusion. And as Thomas Hummel points out, in clinical settings there is a difference in smell disorders where patients present symptoms of parosmia or phantosmia. The latter is the vivid experience of odours in the absence of a relevant odour source, the former is where patients experience the wrong odour for a familiar odour source. For example, patients with parosmia perceive something after being presented with an odour of roses, which is not the expected odour of roses, but rather a distorted and often undefined odorous perception  . . .  these ‘other’ odour sensations are experienced as unpleasant. And they are generally only described in vague terms, for example as ‘chemical’. (Hummel et al., 2011: 2)

The Chemical Senses   345 By contrast to these representational views, Noam Sobel sees the function of the olfactory system as enabling the subject to arrive at hedonic ratings. The hedonic tone of an odour as pleasant or unpleasant can predict the reward potential of food as a result of learning. Humans can detect and discriminate countless odorants, but can identify few by name. The one thing humans can and do invariably say about an odor is whether it is pleasant or not. We argue that this hedonic determination is the key function of olfaction. Thus, the boundaries of an odor object are determined by its pleasantness, which—unlike something material and more like an emotion—remains poorly delineated with words. (Yeshurun and Sobel, 2010: 219)

Sobel believes sniffing both responds to and affects odorant intensity and therefore affects pleasantness and unpleasantness. Sobel also shows that although olfactory processing has significant effects on behaviour—much that happens above conscious threshold is simply unattended: ‘although human odorant detection thresholds are very low, only unusually high odorant concentrations spontaneously shift our attention to olfaction’. So: whereas vision and audition consist of nearly continuous input, olfactory input is discreet, made of sniffs widely separated in time. If similar temporal breaks are artificially introduced to vision and audition, they induce ‘change blindness’, a loss of attentional capture that results in a lack of awareness to change. Whereas ‘change blindness’ is an aberration of vision and audition, the long inter-sniff-interval renders ‘change anosmia’ the norm in human olfaction. (Sela and Sobel, 2010: 13)

‘All this, however, does not diminish the role of olfaction through sub-attentive mechanisms allowing subliminal smells a profound influence on human behavior and perception.’ For example, there are odours in women’s tears that lower men’s libido (Gelstein et al., 2011). A great deal of chemical signalling happens without explicit awareness. Betina Pause has shown that humans process body odours of kin differently from non-kin and are surprised to learn that they can tell kinship just by smelling tee-shirts that have been worn by family members. The emotional state of others is also communicated chemically, with effects on the perceiver. The role of chemical communication in humans might have been strongly underestimated as chemical communication between humans usually does not reach the level of conscious processing. (Pause, 2012: 56–57)

Pause shows that emotional states of others are communicated chemically, with effects on the perceiver. ‘Besides the effects on motor behavior, the [unconscious] perception of stress-related chemosignals significantly alters the perception of visual social signals in humans’ (Pause, 2012: 58). Such important environment signals about others’ emotions are processed in a rapid and automatic way with immediate impact on the perceiver.14 From all these cases, it is clear that a great deal of olfactory processing goes on 14  ‘Fertile individuals prefer the body odors of partners with a relatively dissimilar [immune system] to their own. This preference, in turn, seems to be one of the major reasons for mate selection in many vertebrates (Boehm and Zufall 2006; Restrepo et al. 2006)’ (Pause, 2012: 57).

346   Barry C. Smith unconsciously and with a very specific function. But what is the function of conscious olfactory experience? We can think of olfactory experience as a background to consciousness: part of the fabric that we no longer pay attention to until it changes dramatically. In that background role, smells can modulate our moods, and condition our responses. This idea has led to an even more radical suggestion put forward by Ep Köster, Per Møller, and Jozina Mojet (Köster et al., 2014) that the function of conscious olfaction is simply to detect change: that is what smell is for. Smelling the same things over a period of time leads to sensory adaption. Köster, Møller, and Mojet suggest that unlike vision there is no need for the brain to maintain a constant olfactory as well as visual scene. By not doing so, we free up resources for other cognitive activity. However, should something change in the background landscape—smoke, rotten food smells, garbage, the perfume from a dress—this will immediately switch on our conscious attention to smell. They call this the Misfit View of conscious odour perception. It has a number of things going for it: it can deal with the effects of sensory adaptation where smell is no longer experienced; but it can also account for why we can smell if someone has been in our room, or if there is a smell of a gas leak. It also suggest that our greatest sensitivity is to subtle changes, which may be just what works to capture our attention and provide us with pleasure in the blends created by perfumers and wine makers. Those small changes from a predictable to an unpredictable aroma may enthral us (see Smith, 2014).

17  The nature of olfactory experience This leads us to the vexed question of how we get at the nature of olfactory experience. Is the nature of olfactory experience ultimately answerable to how it appears to us: i.e. is the phenomenology of smell the ultimate guide to the nature of such sensory experiences? Some take this for granted, but it is far from clear either that how things appear in experience is the best guide to how they are, or that we can be entirely sure of bringing the phenomenology of smell sharply into view. Philosophers are wont to ask what it’s like to smell a rose, and to think that by this simple locution they have isolated a paradigm example of olfactory experience. But is there such a thing as a pure olfactory experience had in isolation from the rest of conscious experience? Surely, it is more likely that that we are attempting to attend to the olfactory element in experience rather than an olfactory experience alone. Smelling is more likely to be a matter of what we take it to be like. Most odours are also trigeminal stimulants and the experiences we have will be integrated products of trigeminal and olfactory stimulation (Hummel and Livermore, 2002). Anosmia sufferers may still be able to distinguish between different chemicals on the basis of the trigeminal sensations they produce, so they may think they retain some of their capacity to smell (Bryant and Silver, 2000). This can be demonstrated by testing them with Cognac in a black glass. When smelling they react as if to a foul smell because of the stinging of the alcohol. So, what normal subjects are experiencing is not a unimodal experience. What exactly is given than in an olfactory experience, or an experience with an olfactory component?

The Chemical Senses   347 All smells can be considered in terms of their particular quality (‘rose’, ‘mint’), their intensity (‘mild’, ‘pungent’), and their hedonic tone (‘pleasant’ or ‘unpleasant’). Each of these dimensions of olfactory experience can interact with the others. An odour can change its quality and hedonic tone with a change in concentration. It can flip from unpleasant to pleasant in a sexual context, say. The identification of the odour’s quality and hedonics may change given a different label or priming. For example, isovaleric acid can be experienced as Parmesan cheese or as vomit. So is the phenomenal quality or character the same when we flip from one to the other? It is possible, by reflection, to recognize something unchanging? Should we think about such ambiguous experiences as one or two? The point here is not to answer these questions but to cast doubt on the idea that we have a clear way to move from the phenomenal character of an experience—the supposed ‘what it’s like’—to any precise characterisation of such experiences. The three dimensions of smell experiences (quality, intensity, and hedonic tone) make it clear that we are not dealing with a simple, ineffable, and unanalysable experience of the sort philosophers usually allude to with simplistic talk of ‘what it’s like to smell a rose’. When we try to focus our attention on what we are smelling, we seem to get hold of something about which we can say next to nothing. Famously, we lack odour-specific terms for smells—except for aspects of smell, like acrid, pungent, floral; though some of these might describe effects of trigeminal stimulation rather than smell. Instead, we use the names of the sources of those smells: lemon, rose, tar, leather, cloves, mint; and this may explain our difficulty in being able to name the odorants we are smelling. There may be no special problem finding words for odours, as some have suggested.15 Rather, the problem may be in identifying the odour source just by smelling the odour itself. Jonsson et al. (2005) suggest that the difficulty may lie not with naming but with finding the object that is the source of this odour. If we were able to identify the odour source there may be no trouble finding a name for the odour, or describing it, but when presented with a colourless odour from a vial—a highly unnatural way to be presented with an odour object in the environment, which are usually encountered in combination with other ways of sensing the odour source—we may struggle to identify it. Presentation of an object with that characteristic odour, such as a pineapple, is usually a multisensory affair, of which its smell is an integral part. Reconstructing the odour source from a unimodal clue may be a very difficult and highly unnatural task that requires training. Perhaps the best way to understand the constant role that smell plays in our daily lives is to learn what would happen if one lost the sense of smell. Acquired anosmia can result from front-on head injury in which delicate fibres emanating from the olfactory receptor sheet are severed. Other causes include viruses, medication, and also neurodegenerative diseases like Parkinson’s disease and dementia. Patients who lose the sense of smell suddenly complain of life’s having lost its savour. Affect is flat and the quality of life diminishes. Food loses its flavour and as we saw patients often think they have lost their sense of taste. None of the places or people these patients know smell the same. Sufferers of anosmia often describe themselves as living behind glass, cut off from the world and feeling 15  Tyler Lorig has suggested that activating the olfactory system interferes with use of the language areas of the brain, which explains our difficulty in naming or describing odours.

348   Barry C. Smith alienated from familiar settings. This frightening loss of a dimension of our conscious waking lives shows how much smell was contributing to consciousness without our apparently noticing it. The dimension that smell contributes to experience is something we often don’t know about until it is gone. Old age also leads to reduction or loss of smell and may be a factor in the widespread depression among the elderly. In fact, people who lose their sense of smell remain depressed longer than those who lose their sight. Of course, for some people, the dimension that smell contributes to conscious experience was always missing. Those with congenital anosmia may not realize, at first, that anything is missing from their perception of the world. They may think their experience of the world is complete, and that they have the use of all of their senses. This is the position Marta Tafalla was in as a child who was not diagnosed with anosmia until the age of 11. As a philosopher, Tafalla describes her surprise at learning that some part of her access to the world and some perceptible dimension of the world was missing. She has since spent time and attention trying to find out what this invisible dimension people speak about consists in (Tafalla, 2013). One of her interesting speculations is that the sense of smell contributes to spatial awareness in normosmics. She considers our appreciation of gardens and suggests that those who can smell the flowers and plants may have a greater sense of space and immersion. In this way smell contributes to experiences created by the other senses.

18 Conclusions In contrast to the traditional view of the chemical senses as peripheral and marginal to the main questions in the philosophy of perception, we have seen how our understanding of the interactions among the chemical senses can reveal examples of cross-modal interaction and multisensory integration that we are coming to see as the rule not the exception in perceptual experience. The multisensory perception of flavour displays examples of: smell’s effect on taste and vice versa; touch’s effect on taste; smell’s effect on touch and vice versa, temperature’s effect on taste; audition’s effect on taste; sight’s effect on smell; and the way a multitude of sensory inputs from different modalities are integrated into a single, unified perception of flavour. What these senses demonstrate as well as any is how the brain relies on simultaneous input from many senses to create a coherent unified perception of the world around us and ourselves.

References AFNOR (1992). 'Sensory analysis'. Vocabulary. NF ISO 5492. AFNOR, Paris—la Défense. Allais, D. and Burr, D. (2004). 'The ventriloquist effect results from near-optimal bimodal integration'. Current Biology, 14, 257–262. Auvray, M. and Spence, C. (2008). 'The multisensory perception of flavour'. Consciousness and Cognition, 17, 1016–1031. Bartoshuk, L, Duffy, V., and Miller, I. (1994). 'PTC/PROP tasting: anatomy, psychophysics and sex effects'. Physiological Behaviour, 56, 1165–1171.

The Chemical Senses   349 Bartoshuk, L. and Duffy, V. (1998). 'Chemical Senses'. In G. Greenberg and M. M. Haraway (eds), Comparative Psychology: a handbook (pp. 25–33). New York: Garland Publishing. Batty, C. (2010). 'What the nose doesn’t know: non-veridicality and olfactory experience'. Journal of Consciousness Studies, 17, 10–27. Bender, G., Hummel, T., Negoias, S., and Small, D. M. (2009a). 'Separate signals for orthonasal vs. retronasal perception of food but not nonfood odors'. Behavioral Neuroscience, 123(3), 481–489. Bender, G., Meltzer, J. A., Gitelman, D., and Small, D. M. (2009b). 'Neural correlates of evaluative compared with passive tasting'. European Journal of Neuroscience, 30, 327–328. Berridge, K. C. (1996). 'Food reward: brain substrates of wanting and liking'. Neuroscience and Biobehavioral Reviews, 20, 1–25. Breslin, P. (2000). 'Human gustation'. In T. Finger, W. Silver, and D. Restrepo (eds), The Neurobiology of Taste and Smell (2nd edn) (pp. 423–461). New York: Wiley-Liss. Brillat-Savarin, J. A. (1825). La Physiologie du gout, trans. as The Physiology of Taste. London: Penguin, 1994. Bryant, B. and Silver, W. (2000). 'Chemesthesis: the common chemical senses'. In T. Finger, W. Silver, and D. Restrepo (eds), The Neurobiology of Taste and Smell (2nd edn) (pp. 73–100). New York: Wiley-Liss. Bult, J., Wijk, R., and Hummel, T. (2007). 'Investigations on multimodal sensory integration: texture, taste, and ortho- and retronasal olfactory stimuli in concert'. Neuroscience Letters, 411(1), 6–10. Chaudhari, N., Landin, A., and Roper, S. (2000). 'A metabotropic glutamate receptor variant functions as a taste receptor'. Nature Neuroscience, 3, 113–119. Chen, D., Katdare, A., Lucas, N. (2006). 'Chemosignals of Fear Enhance Cognitive Performamnce in Humans'. Chemical Senses, 31415–31423. Chu, S. and Downes, J. J. (2000). 'Odour-evoked autobiographical memories: psychological investigations of Proustian phenomena'. Chemical Senses, 25, 111–116. Cohen, J. (2013). 'A sensible subjectivism about flavour?' Talk given to the American Society for Aesthetics, San Diego October 2013. Collings, V. B. (1974). 'Human taste response as a function of locus of stimulation on the tongue and soft palate'. Perception & Psychophysics, 16, 169–174. Cowart, B. J., Halpern, B. P., et al. (2003). 'Differential loss of retronasal relative to orthonasal olfaction in a clinical population'. Chemical Senses, 28, A65. Cruz, A. and Green, B. G. (2000). 'Thermal stimulation of taste'. Nature, 403, 889–892. Dalton, P., Doolittle, N., Nagata, H., and Breslin, P. A. S. (2000). 'The merging of the senses: subthreshold integration of taste and smell'. Nature Neuroscience, 3, 431–432. DeGelder, B. and Bertleson, P. (2003). 'Multisensory integration, perception and ecological validity'. Trends in Cognitive Sciences, 7(10), 460–467. Ernst, M. O. and Bülthoff, H. H. (2004). 'Merging the senses into a robust percept'. Trends in Cognitive Sciences, 8(4), 162–169. Frasnelli, J., Heilmann, S., and Hummel, T. (2004). 'Responsiveness of human nasal mucosa to trigeminal stimuli depends on the site of stimulation'. Neuroscience Letters, 362, 65–69. Frasnelli, J. and Hummel, T. (2005). 'Intranasal trigeminal threshold in healthy subjects'. Environmental Toxicology and Pharmacology, 9, 575–580. Fulkerson, M. (2014). The First Sense: A Philosophical Study of Human Touch. MIT Press. Gallace, A. and Spence, C. (2008). 'The cognitive and neural correlates of "tactile consciousness": a multisensory perspective'. Consciousness and Cognition, 17, 370–407.

350   Barry C. Smith Gelstein, S., et al. (2011). 'Human tears contain a chemosignal'. Science, 331(6014), 226–230. Ghazanfar, A. A. and Schroeder, C. E. (2006). 'Is neocortex essentially multisensory?' Trends in Cognitive Sciences, 10, 278–285. Gottfried, J. (2007). 'What can an orbitofrontal cortex-endowed animal do with smells?' Annals of the New York Academy of Sciences, 1121, 102–120. Gottfried, J. A. and Dolan, R. J. (2003). 'The nose smells what the eye sees: crossmodal visual facilitation of human olfactory perception'. Neuron, 39, 375–396. Heilmann, S. and Hummel, T. (2004). 'A new method for comparing orthonasal and retronasal olfaction'. Behavioral Neuroscience, 118, 412–419. Herz, R. S. and Schooler, J. W. (2002). 'A naturalistic study of autobiographical memories evoked by olfactory and visual cues: testing the Proustian hypothesis'. American Journal of Psychology, 115, 21–32. Hort, J. and Hollowood, T. (2004). 'Controlled continuous flow delivery system for investigating taste−aroma interactions'. Journal of Agricultural and Food Chemistry, 52(15), 4834–4843. Hummel, T., Landis, B., and Huttenbrink, K. (2011). 'Smell and taste disorders, current topics in otorhinolaryngology'. Head and Neck Surgery, 10, ISSN 1865–1011. Hummel, T. and Livermore, A. (2002). 'Intranasal chemosensory function of the trigeminal nerve and aspects of its relation to olfaction'. International Archives of Occupational and Environmental Health, 75, 305–313. Ikeda, K. (1909). Journal of the Chemical Society of Tokyo, 30, 820–836, trans. as ‘New seasonings’, Chemical Senses, 27 (2002), 847–849. Jinks, A. and Laing, D. G. (1999). 'Temporal processing reveals a mechanism for limiting the capacity of humans to analyze odor mixtures'. Cognitive Brain Research, 8, 311–325. Johnston, R. (2000). 'Chemical communication and pheromones: the types of chemical signals and the role of the vomeronasal system'. In T. Finger, W. Silver, and D. Restrepo (eds), The Neurobiology of Taste and Smell (2nd edn) (pp. 101–127). New York: Wiley-Liss. Jonsson, F., Tchekhova, A., Lonner, P., and Olsson, M. J. (2005). 'A metamemory perspective on odor naming and identification'. Chemical Senses, 30, 353–365. Karlson, P. and Luscher, M. (1959). ' "Pheromones": a new term for a class of biologically active substances'. Nature, 185, 55–56. King, A. and Calvert, G. (2001). 'Multisensory integration: grouping by eye and ear'. Current Biology, 11, R322–R325. Köster, E., Møller, P., and Mojet, J. (2014). 'A "misfit" theory of spontaneous conscious odor perception (MITSCOP): reflections on the role and function of odor memory in everyday life'. Frontiers in Psychology, 5, 64. Koza, B. J., Cilmi, A., et al. (2005). 'Color enhances orthonasal olfactory intensity and reduces retronasal olfactory intensity'. Chemical Senses, 30(8), 643–649. Kringelbach, M. and Stein, A. (2010). 'Cortical mechanisms in human eating'. In W. Langhans and N. Geary (eds), Frontiers in Eating and Weight Regulation (pp. 164–175). Forum Nutr. vol. 63. Basel: Karger. Laing, D. G., Link, C., Jings, A. L., and Hutchinson, I. (2002). 'The limited capacity of humans to identify the components of taste mixtures and taste-odor mixtures'. Perception, 31, 617–635. Landis, B.N., Hummel, T., Hugentobler, M., Giger, R., and Lacroix, J.S. (2003). 'Ratings of Overall Olfactory Function'. Chemical Senses, 28, 691–694. Landis, B. N., Frasnelli, J., et al. (2005). 'Differences between orthonasal and retronasal olfactory functions in patients with loss of the sense of smell'. Archives Otolaryngol—Head and Neck Surgery, 131(11), 977–981.

The Chemical Senses   351 Lawless, H. (1997). 'Olfactory psychophysics'. In G. Beauchamp and L. Bartoshuk (eds), Tasting and Smelling (2nd edn) (pp. 125-174). New York: Academic Press. Li, W., Luxenburg, E., Parrish, T., and Gottfried, J. (2006). 'Learning to Smell the Roses: Experience-dependent Neural Plasticity in Human Piriform and Orbitofrontal Cortices'. Neuron, 52, 1097-1108. Lim, J. and Green, B. (2008). 'Tactile interaction with taste localization: influence of gustatory quality and intensity'. Chemical Senses, 33, 137–143. Lim, J. and Johnson, M. (2011). 'Potential mechanisms of retronasal odor referral to the mouth'. Chemical Senses, 36, 283–289. Lim, J. and Johnson, M. (2012). 'The role of congruency in retronasal odor referral to the mouth'. Chemical Senses, 37, 515–521. Lycan, W. (2000). 'The slighting of smell'. In N. Bhushan and S. Rosenfeld (eds), Of Minds and Molecules: new philosophical perspectives on chemistry (pp. 273–290). New York: Oxford University Press. MacFarlane, J. (2007). 'Relativism and disagreement'. Philosophical Studies, 132, 17–31. Marcel, A. (2003). 'Introspective report: trust, self-knowledge and science'. Journal of Consciousness Studies, 10(9–10), 167–186. Mattes, R. D. (2009). 'Is there a fatty acid taste?' Annual Review of Nutrition, 29, 305–327. Milner, R. and Goodale, M. (1992). 'Separate visual pathways for perception and action'. Trends in Neurosciences, 15(1), 20–25. Mojet, J., Köster, E. P., and Prinz, J. F. (2005). 'Do tastants have a smell?' Chemical Senses, 30, 9–21. O’Callaghan, C. (2008). 'Seeing what you hear: cross-modal illusions and perception'. Philosophical Issues, 18(1), 316–338. O’Doherty, J., Rolls, E., Francis, S., Bowtell, R., McGlone, F., Kobal, G., et al. (2000). 'Stimulusspecific satiety olfactory-related activation in the human orbitofrontal cortex'. Neuroreport, 11, 893–897. Pause, B. (2012). 'Processing of body odour signals in the human brain'. Chemosensory Perception, 5, 55–63. Piggot, J. (1994). 'Understanding flavour quality: difficult or impossible?' Food Quality and Preference, 5, 1–2, 167–171. Piqueras-Fiszman, B., Alcaide, J., Roura, E., and Spence, C. (2011). 'Does the weight of the dish influence our perception of food?' Food Quality and Preference, 22(8), 753–756. Porter, J., Anand, T., Johnson, B., Khan, R. M., and Sobel, N. (2005). 'Brain mechanisms for extracting spatial information from smell'. Neuron, 47, 581–592. Porter, J. et al. (2007). 'Mechanisms of scent-tracking in humans'. Nature Neuroscience, 10, 27–29. Prescott, J. (1999). 'Flavour as a psychological construct: implications for perceiving and measuring the sensory qualities of foods'. Food Quality and Preference, 10, 349–356. Richardson, L. (2013). 'Flavour, taste and smell'. Mind and Language, 28(3), 322–341. Rozin, P. (1982). ' "Taste-smell confusions" and the duality of the olfactory sense'. Perception and Psychophysics, 31(4), 397–401. Sela, L. and Sobel, N. (2010). 'Human olfaction: a constant state of change-blindness'. Experimental Brain Research, 205, 13–29. Sibley, F. (2006). Approaches to Aesthetics: Collected Papers on Philosophical Aesthetics, ed. J. Benson, B. Redfern, and J. Roxbee Cox. Oxford: Oxford University Press. Small, D. (2013). 'Flavour is in the brain'. Physiology and Behavior, 107, 540–552.

352   Barry C. Smith Small, D. Voss, J., Mak, E., Simmons, K., Parrish, T., and Gitelman, D. (2004). 'Experiencedependent neural integration of taste and smell in the human brain'. Journal of Neurophysiology, 92, 1892–1903. Small, D., Gerber, J., Mak, E., and Hummel, T. (2005). 'Differential neural responses evoked by orthonasal versus retronasal odorant perception in humans'. Neuron, 47, 593–605. Small, D., Bender, G., Veldhuizen, M., Rudenga, K., Natchigal, D., and Felsted, J. (2007). 'The role of the human orbitofrontal cortex in taste and flavor processing'. Annals of the New York Academy of Sciences, 1121, 136–151. Smith, A. D. (2002). The Problem of Perception. Princeton: Princeton University Press. Smith, B. (2007). 'The objectivity of tastes and tasting'. In B. C. Smith (ed.), Questions of Taste: the philosophy of wine. Oxford: Oxford University Press. Smith, B. (2010). 'Relativism, disagreement and predicates of personal taste'. In R. Recanati, I. Stojanovic, and N. Villanueva (eds), Context-Dependence, Perspective and Relativity. Berlin: DeGruyter. Smith, B. (2014). 'Complexity and blending in wine'. In R. Gougeon (ed.), Wine Active Compounds. Proceedings of Wine Active Compounds 2014 Dicilour Groupe. Spence, C., Levitan, C., Shankar, M., and Zampini, M. (2010). 'Does food color influence taste and flavor perception in humans?' Chemosensory Perception, 3, 68–84. Spence, C., Auvray, M., and Smith, B. (2014). 'Confusing tastes and flavours'. In D. Stokes, M. Matthen, and S. Briggs (eds), Perception and its Modalities. Oxford: Oxford University Press. Soler, M., Kumru, H., Vidal, J., Pelayo, R., Tormos, J., Fregni, F., Navarro, X., and PascualLeone, A. (2010). 'Referred sensations and neuropathic pain following spinal cord injury'. Pain, 150(1), 192–198. Stevenson, R. J. (2001). 'Is sweetness taste enhancement cognitively impenetrable?' Effects of exposure, training and knowledge. Appetite, 36, 241–242. Stevenson, R. J. and Tomiczek, C. (2007). 'Olfactory-induced synesthesias: a review and model'. Psychological Bulletin, 133, 294–309. Tafalla, M. (2013). 'A world without the olfactory dimension'. The Anatomical Record, 296, 1287–1296. Talsma, D., Senkowski, D., Soto-Faraco, S., and Woldorf, M. (2010). 'The multifaceted interplay between attention and multisensory integration'. Trends in Cognitive Sciences, 14, 400–410. Verhagen, J. (2007). 'The neurocognitive bases of human multimodal food perception: consciousness'. Brain Research Reviews, 53(2), 271–286. Woolgar, C. (2006). The Senses in Late Medieval England. New Haven: Yale University Press. Yeomans, R., Chambers, L., Blumenthal, H., and Blake, A. (2008). 'The role of expectancy in sensory and hedonic evaluation: the case of smoked salmon ice-cream'. Food Quality and Preference, 19, 565–573. Yeshurun, Y. and Sobel, N. (2010). 'An odor is not worth a thousand words: from multidimensional odors to unidimensional odor objects'. Annual Review of Psychology, 61, 219–224. Zampini, M. and Spence, C. (2004). 'Multisensory contribution to food perception: the role of auditory cues in modulating crispness and staleness in crisps'. Journal of Sensory Studies, 19(5), 347–363.

Chapter 18

The Bodily Senses J. Brendan Ritchie and Peter Carruthers

There is no single sense for perceiving our own bodies, any more than there is a single sense for perceiving the external world. Nonetheless, folk wisdom groups bodily sensations together—unlike the five external senses, which are intuitively distinct. In part this may be because the systems that produce an experience of arm position, an itch, or an upset stomach, lack obvious sensory organs. In addition, bodily sensations are private (you can see what I see, but you cannot feel my body as I do), and the properties of our bodies that they track are often obscure. In this chapter we focus on three broad systems of bodily perception: interoception, the vestibular system, and proprioception. Our main goal is to argue that they constitute (collections of) sense modalities, while discussing some of the philosophical issues they raise.

1  The interoceptive senses In order to function, the body must maintain physiological parameters within certain boundaries. Interoception is the collection of sensory systems that monitor the physiological state of the body in order to maintain this internal homeostasis. The term ‘interoceptor’ was first used by Sherrington (1906) to distinguish afferent pathways that carry information from the body to the brain, and was restricted to information produced by properties of the viscera: the respiratory, gastrointestinal, and cardiovascular organs. We take interoception to also embrace processes that originate in the skin, including those pathways that result in thermal sensations, cutaneous itch, and pain, rejecting the traditional view that pain and thermoception are aspects of touch. (For pain see Chapter 28; for touch, see Chapter 16, which defines touch as the perception of pressure, thus excluding pain and thermoception.) This might seem unintuitive, but excluding cutaneous systems of pain, itch and thermoception from the class of interoceptive systems would make a rather arbitrary cut, since these systems also monitor the viscera. Some go further (e.g. Cameron, 2001), and include proprioception and the vestibular system as part of interoception, but this taxonomy is too broad. What visceral and cutaneous monitoring systems have in common is that they constitute the afferent pathways used for regulating

354    J. Brendan Ritchie and Peter Carruthers and maintaining the homeostasis of our internal organs and body parts (Craig, 2002). Proprioception and the vestibular system, while ‘body-directed’, serve different functions, as we shall see. Indeed, there is a seeming disconnect between the functional and extensional definitions of ‘interoception’. In practice, researchers seem to restrict the extension of the term to internal monitoring systems of the viscera (e.g. Vaitl, 1996); but when it is defined functionally, the emphasis is on the internal monitoring and regulation of bodily organs for the purpose of maintaining homeostasis. Yet the skin, too, is an organ, one that shields internal structures, among many other functions. Thus we include detector systems in the skin as part of interoception, when functionally defined. Interoception is most familiar to philosophers through the conscious bodily sensations it produces. Itches, thermal sensations, sensations of orgasm, heart-beat, thirst, indigestion, shortness of breath, and any form of pain, along with aspects of moods, emotions, and affect more generally, are all forms of interoceptive experience. The label ‘bodily sensation’ is unfortunate, however, since it invites the grouping together of many different phenomena (including the vestibular system and proprioception) under one label, as if they could be treated uniformly. Most discussion of interoception in philosophy has focused on the introspectable qualities of bodily experiences. However, as we will see, focusing on the extent to which interoception consists of a set of distinct sense modalities can throw new light on philosophical disagreements about the nature of bodily experience. Later we turn to the connections between interoception, emotion, and cognition.

Multiple modalities Does interoception qualify as a sense modality at all? If so, should it be considered a collection of distinct sense modalities? The answers to these questions depend on how sense modalities are to be identified. One of us has defended a prototype account, which we adopt here (Picciuto and Carruthers, 2014). This account does not offer necessary and sufficient conditions for identifying sense modalities, but rather a list of characteristics that prototypical sense modalities possess. Candidate modalities are evaluated by the degree to which they have the following features:  (1) Sensitivity to a set or range of related physical properties. (2) Detector mechanisms that transduce these physical properties into an informational signal to the central nervous system. (3) The evolved function of detecting and representing the physical properties specified by (1). (4) Representations with nonconceptual (or ‘fine-grained’) content and a mind-toworld direction of fit. (5) Utilization of the informational signals by the organism (typically through integration with other sensory inputs) to guide intentional action. Modalities will be distinct to the extent that they differ from one another in these features (for example, by being sensitive to different ranges of properties, or by involving distinct kinds of transducer).

The Bodily Senses   355 Many would also wish to insist that a sense modality should produce mental states with distinctive phenomenal qualities or ‘feels’. But this criterion begs the question of the correct form of account of phenomenal consciousness. If first-order theories are on the right track (Tye, 1995), then phenomenal consciousness will be widespread in the animal kingdom. But if some form of higher-order account is correct (Carruthers, 2005), then phenomenal consciousness will be severely restricted in its distribution, perhaps to the primate clade. In the latter case it would not be appropriate to include possession of phenomenal qualities among the prototypical properties of a sensory modality. Since we do not wish to take a stand on this issue, we will include possession of phenomenal character in the discussions to come, while noting that its inclusion is controversial. (For a similar view, see chapter 30) Is interoception a collection of systems sensitive to a range of physical properties of the body? Many have claimed that bodily sensations are unique in representing nothing beyond themselves. Thus Armstrong (1962) initially suggests that while thermal sensations track secondary qualities of temperature, the same does not seem true of other bodily sensations: an itch in my arm does not track an itchiness quality in my arm. Moreover, there does not seem to be an appearance–reality distinction for bodily sensations in the same way as there is for our external senses (a claim we return to): if something feels itchy, it is itchy. Similarly, one might claim, bodily sensations cannot go unfelt. True, one might be distracted from the itch on one’s arm from a mosquito bite, but it is tempting to think that it does not make sense to talk of an unconscious itch. We think that such intuitions are only plausible if one ignores the nature of the interoceptive systems that produces our bodily experiences. The fact that our folk psychological theory finds no object of bodily sensation does not entail that there are no physical properties to which interoception is sensitive. In fact, quite the opposite is true, once we consider the different kinds of receptor cell that provide input to interoception. Since interoception tracks the functioning of many different internal organs, it relies on several kinds of receptor to transduce physical properties of the body into informational signals, which are sent to the brain through spinal pathways that are now well characterized (Craig, 2002). The classic work on species of interoceptors is Chernigovskiy (1967) who identified four main classes:  mechanoreceptors, chemoreceptors, thermoreceptors, and osmoreceptors. What follows is a (non-exhaustive) list of different kinds of receptor cell, which also provides a picture of the different physical properties of the body that interoceptive systems track: • Nociceptors are free nerve endings of small-diameter C-fibres and Aδ-fibres (most commonly associated with cutaneous pain sensation), which are found throughout the body’s tissue (and viscerally). They respond to mechanical stress as well as local metabolic properties, cutaneous parasite penetration, and immune system and hormonal activity. Importantly, this includes a class of histamine-responsive C-fibres found in the human skin, which provide the inputs for our sensation of cutaneous itch. • Thermoreceptors, found in the skin and viscerally, respond to temperature and temperature change of the body, with one set representing temperatures in the noxious range of cold or hot (thermal pain nociceptors), and another set representing changes within the innocuous range.

356    J. Brendan Ritchie and Peter Carruthers • Osmoreceptors track extracellular fluid osmotility relative to a homeostatic norm. Transduction is achieved via mechanoreceptors that respond to changes in cell volume. Osmoreceptors are found viscerally and in areas of the central nervous system that are outside the blood-brain barrier. • Glucoreceptors are neurons that use glucose as a signalling molecule, based on its concentration density. Glucosensing neurons can be found viscerally in the intestine, gut, and liver, among other places, as well as in the central nervous system. • Baroreceptors are nerve endings that respond to physical deformations and stress of the walls of the vessels in which they are located in order to track barometric pressure. These receptors are found throughout the blood vessels in the body. • Tension mechanoreceptors, found throughout the viscera, respond to changes in muscle tension due to stretch, stimulation of mucosa, or both. They can be found in the bladder, stomach, and oesophagus. Although the relationships between these transducers and our conscious interoceptive sensations are not transparent, it should nonetheless be clear that interoception tracks many different kinds of physical property, and that it involves a varied class of transducers that project to the central nervous system. At this point, however, one might resist our claim that the properties detected by all these systems are internal. For much of what we seem to sense via thermoception are properties of external things, such as a cool breeze, or the handle of a hot kettle; it is the air that is cold, and the metal that is hot, rather than one’s skin. Now, the same intuition does not seem to hold for itches, as we do localize histamine-responses to the skin. Nonetheless, it might seem we must exclude thermoception and perhaps other cutaneous senses from interoception, and include them as part of touch, for example. In reply, we would again emphasize our earlier point that this categorization scheme requires us to classify some instances of thermoception are part of interoception and some not. We would also argue that the exteroceptive use of thermoception does not make it a non-bodily sense, or part of touch, any more than the fact that spatial orientation is facilitated by combining touch with proprioception makes touch part of proprioception (cf. chapter 15). Held in isolation, thermoception might give us little in the way of a sense of an external object independent of haptic feedback and proprioceptive awareness. We might have little sense of a thermal object. For example, imagine that a deafferented patient (one whose afferent pathways for proprioception have become de-mylinated; see below) is touched on her back (out of her vision field) with an cold iron rod (thermoception is typically not altered in deafferented patients). We can imagine: does the subject feel a cold object, or feel some body part as being cold? Absent a tactile sensation of the contact of the cold rod, or an ability to spatially localize the sensation (from proprioception), it seems the sensation would be felt as a state of her body. So it seems likely that thermoception only aids in exteroception when combined with other inputs. And, of course, thermoception is not restricted to the skin. It also underlies the feeling that we ourselves are too hot or cold. Hence thermoception should be classified as an interoceptive modality. Given the existence of the above receptors and their accompanying pathways to the central nervous system, it should be clear that interoceptive systems evolved for the function of detecting many different physiological properties of internal and peripheral organs in order to maintain the internal homeostasis of the body, satisfying conditions (1)–(3) above.

The Bodily Senses   357 But what about (4): do they also represent such properties? This depends on whether interoceptive states are representational at all, which some have disputed. For example, Akins (1996) argues that thermoception is non-representational because it fails to track temperature (of the body) in the veridical fashion that philosophers often have in mind when discussing representation. Thermoreceptors have a non-linear response profile to temperature and temperature change, and an uneven distribution over the body, with greater concentration at the head, groin, and armpits. (This is part of the reason why when one enters a lake, the water feels colder at these areas.) In this respect, thermoception, like all forms of interoception, is ‘narcissitic’, tracking physical properties motivated by the needs of the organism (or its organs) rather than objectively reflecting those properties. Thus, if thermoception fails to represent, so do interoceptive systems generally. We think that the requirement of veridicality for representation is dubious, however. Unlike external modalities, the interoceptive senses are primarily sensitive to disturbances, becoming active when things in the body go wrong. When we eat some bad fish, catch the ’flu, or overly exert ourselves, the body is awash with interoceptive sensations. Under normal conditions we are barely aware of any feelings in the viscera, because most visceral afferents are typically ‘silent’, and respond only under very specific conditions. Thus afferents in the colon are active during bladder inflammation, but cease activity after the inflammation has ended. Likewise, interoception has not evolved to detect osmotic pressure as such, but to detect changes in osmotility that are significant deviations from the homeostatic norm. Thus lower osmotic pressure signals dehydration, causing thirst. Such event-like properties involving deviation from a norm are perfectly legitimate contents of mental representations, we believe. Another challenge to the representational status of interoceptive experiences is that it does not seem possible to misfeel some bodily experiences, like itches (Armstrong, 1962). Block (2006), too, argues that this apparent fact is a problem for representationalists about consciousness, who hold that the phenomenal character of consciousness is exhausted by its representational content. Armstrong (1962) eventually argues that there is enough of an appearance-gap between bodily sensations and physical stimuli to enable them to misrepresent. What Armstrong advocates is that we ‘translate’ between a bodily sensation and the physiological state of the body that causes the sensation. For example, one might feel nauseous, and it would seem that if one feels nauseous one is nauseous. However, nausea represents that one’s stomach is unsettled, and it is possible that one feels nauseous even when one’s stomach is not unsettled. Such a proposal is even more plausible when we keep in mind that there is a whole collection of systems that detect physical properties of the body and produce such sensations. Moreover, there are a number of features of interoception that serve to decrease the actual incidence of misfeeling. First, most forms of interoception detect stimuli that are proximate to the relevant transducers, since the latter are characteristically embedded within the organs that they monitor. This suggests that there is no proximal–distal distinction in such cases (in contrast with vision and hearing). Second, most forms of interoceptive awareness are comparatively coarse grained. Yet, that we rarely ‘misfeel’ does not imply that it is impossible, as Armstrong notes. While cases of robust interoceptive illusions are rare, there are cases of ‘phantom organs’ after surgery (Dorpat, 1971), and stroking exposed intestine can cause the illusion that defecation is needed (Airapetyantz and Bykov, 1945). Moreover, phantom limb patients often report sensations of phantom pain

358    J. Brendan Ritchie and Peter Carruthers or itch. In addition, research on the psychophysics of interoception, which has primarily been on heart-rate monitoring, supports the existence of a great deal of between-subject variability. (Similar results obtain for respiratory and gastrointestinal monitoring; see Vaitl, 1996.) For example, gender, weight, age, and fitness affect cardiac interoception (Cameron, 2002). Of course our bodily experiences often lack the sort of ‘transparency’ that is distinctive of visual experience. When one visually perceives something, one generally does not focus on the qualities of the experience itself, but rather on its objects, such as shape and colour. This transparency has been important to the case for representationalist theories of consciousness (Tye, 2002). When we focus on an itch, in contrast, we seem to be focusing on our experience itself. But this may result more from our folk-ignorance of the physical properties detected than the non-representational character of bodily experience. For notice that where folk-wisdom does provide an intuitive object—such as the gurgling in one’s stomach or the thumping of one’s heart after exercise—transparency seems to be present. When one focuses attention on one’s beating heart it is the beats of one’s heart that one focuses on, not one’s experience as such. The case for the non-representational character of bodily experience is not compelling, then. Furthermore, an important consideration supporting representationalism is the apparent localization of bodily sensations within the body. For example, feelings of constipation are localized to the colon, and such sensations are, of course, nonconceptual in character—or at least ‘fine-grained’ in comparison with the concepts that figure in our beliefs. Yet it seems implausible that our sensations should literally be located in a specific body part, as opposed to being represented as occurring there. This consideration is reinforced by the phenomenon of phantom-limb pain or phantom-limb itch, where the sensations are experienced as occurring in a limb that is no longer present. We suggest, then, that interoception has most of the qualities of a prototypical sense modality (or rather, of a set of such modalities). But what about guiding intentional action? Initially it might seem that interoception fails condition (5). For the primary function of most interoceptive modalities is to provide input for the automatic and unconscious homeostatic regulation of the physiological condition of the body—not for guiding action. Admittedly, interoception effects action indirectly, as when we stop running to catch our breath, or eat because we are hungry. But this does not so much guide, as give us cause for, intentional action. At this point it will be useful to recall the distinction drawn earlier between visceral and cutaneous interoceptive systems. For the latter forms of bodily experience can certainly guide action, and this is probably part of their function. Conscious experiences of pain and of heat are often used to guide actions, such as removing oneself (or a specific body part) from a source of pain, or removing a layer of clothing when hot. So cutaneous interoception certainly seems to consist of a set of prototypical sense modalities. It might be thought that matters are otherwise, however, with some forms of visceroception, whose outputs may never be centrally accessible and do not seem to guide intentional actions targeted on the world. Nevertheless, we do use visceroception to guide intentional actions directed at our own bodies. For localizing these sensations at different positions inside the body allows us to take actions (where possible) directed toward these areas.

The Bodily Senses   359 We suggest, then, that all forms of interoception should qualify as sense modalities to some important degree, and that most should be fully categorized as such. There seems to be a spectrum (both within and between interoceptive modalities), from bodily sensations offering a distinct phenomenology that can influence controlled action (e.g. those underlying cutaneous itch), to those states that can be made conscious (cardiac perception), and then those that are produced by low-level detector systems and never rise to the level of awareness (monitoring osmotic pressure). Interestingly, while it can be doubted whether some of these interoceptive systems play much of a role in guiding intentional action, many of the same systems have been co-opted by other processes that do clearly play a role in action planning, as we will see in the following section. This reflects the central role of interoception in the embodiment of the mind.

Interoception, emotion, and cognition The common-sense view of emotions is that they are feelings. James (1884) and Lange (1885) independently tried to develop this view, claiming that emotions are constituted by our awareness of interoceptive feelings, with different combinations of autonomic responses being distinctive of different emotions. For example, evaluating something in one’s environment as a threat causes a distinctive set of physiological changes, which interoception tracks, and it is one’s conscious interoceptive experience that constitutes one’s fear. As James famously argued, if you subtract away these interoceptive experiences, nothing is left of one’s fear. While the James–Lange theory has fallen out of favour, the current consensus in emotion theory is that our experience of emotion is constituted both by an affective component (partially reflecting interoceptive experience) and by a set of appraisal dimensions through which one conceptualizes and categorizes one’s affective experience, which form a cognitive component of emotion (Izard, 2007). Despite the widespread agreement in emotion theory that interoceptive sensations are partially constitutive of emotional states, the thesis that we can individuate emotions based on their felt bodily changes or a corresponding autonomic nervous system response (as the James–Lange theory claims), has remained controversial. Again, such a view seems to accord with common sense: being in a state of anger does not seem to feel the same as being in a state of joy. However, whether there are distinctive interoceptive sensations for distinct emotions partially depends on how one thinks about the ontology of emotions. Many theorists make a distinction between basic emotions such as fear, anger, sadness, disgust, surprise, and joy, and more complex, culturally dependent, emotions such as envy, guilt, and shame. But some argue that only the more evolutionarily ancient basic emotions have distinct autonomic responses and perhaps interoceptive experiences (cf. Griffths, 1997); or going further, some argue that even the basic emotions lack distinctive interoceptive qualities (Barrett, 2006). What is plausible is that a general sense of arousal, requiring contextual interpretation, is the primary contribution of interoception to emotion (Schachter and Singer, 1962). Moreover, Barrett (1998) has argued that individuals vary significantly in the extent to which they are sensitive to the arousal properties of their emotional states. A related finding is that the extent to which individuals rely on general arousal in categorizing their

360    J. Brendan Ritchie and Peter Carruthers emotional experiences correlates with their interoceptive sensitivity on a cardiac monitoring task (Barrett et al., 2004). In sum, evidence supporting the James–Lange view that distinct emotions are characterized by distinctive interoceptive sensations has proven hard to come by (Larsen et al., 2008; Barrett, 2006). A more recent suggestion is that interoception is the basis for one dimension of ‘core affect’ (Barrett and Russell, 1999). All forms of affective experience seem to vary along dimensions of valence and arousal. Valence can be either positive or negative, is tied to the reward system in the brain, and seems to be the main determinant of the choices that we make. Moreover, it is often identified with pleasure and displeasure. Whether it is a product of interoception, however, is doubtful; and one of us has argued that it should more properly be seen as a nonconceptual representation of value, rather than construed in hedonic terms (Carruthers, 2011). In contrast, feelings distinctive of arousal are certainly produced by interoception, though this arousal component of core affect comprises a heterogeneous collection of interoceptive sensations, rather than reflecting a single dimension of experience. It is plausible, then, that interoception underlies one dimension of core affect. At the same time, core affect is thought to play a role in cognition more generally. On the one hand, affective experiences seem to provide a constant background context to thought and action. On the other hand, it is also said to play a role in how we reason and make decisions, allowing us to evaluate the desirability of different choices or judgements. For example, according to the influential ‘somatic marker’ model proposed by Damasio (1994), we choose among options by imagining them, responding affectively to these imagined scenarios, and by monitoring our affective responses. In evaluating this proposal, however, much depends on the nature of the affective response. Much of the evidence of the role of affect in cognition suggests that our decisions are guided primarily by the valence dimension. But arousal, too, plays an important role. According to the affect-asinformation approach, arousal carries information about urgency (Storbeck and Clore, 2008). Intuitively, this makes sense: I may for example desire to view a crucial scene in a movie more than I do going to the bathroom, but under appropriate circumstances, satisfying the latter desire becomes far more pressing. Seemingly in support of this picture, Werner et al. (2009) found that subjects with better cardiac perception performed better in the Iowa Gambling Task, the same task that has been used to provide evidence for the somatic marker hypothesis. Moreover, arousal plays an important role in judgements about how we feel about something, rather than influencing our first-order judgements themselves. In this regard, arousal might make conscious our implicit reaction to a stimulus (Storbeck and Clore, 2008). This coheres well with the role that interoceptive outputs (including changes in arousal) have been thought to play in metacognition, or ‘thinking about thinking’ (Koriat, 2007). When we metacognize, we form representations of our own mental states in the service of controlling our own learning, remembering, or reasoning. The most familiar examples are long-term memory phenomena like the feeling of knowing (e.g. for a name you know, but cannot remember). Changes in affect can be caused by changes in how fluently we process information, and such affective changes are used as cues for inferring how successfully we are learning or thinking. In these cases interoceptive processes have been co-opted and are utilized to evaluate our own cognition, rather than the physiological state of the body.

The Bodily Senses   361

Conclusion We have argued that interoception should be seen as a set of distinct sense modalities for monitoring and maintaining the homeostasis of bodily organs. However, the role of interoception in emotion-individuation, as well as in reasoning and decision making, may be more limited than some have claimed.

2  The vestibular sense The vestibular system is perhaps one of the most interesting of the human sensory repertoire, and yet philosophers have paid it scant attention. It is crucial for our sense of balance, and receives its inputs from an elegant set of organs in the inner ear that respond to displacement of the head. This system is heavily integrated with ocular motor processes, as well as with vision and proprioception, and it is through this integration that the vestibular system contributes to spatial perception and navigation. Our discussion focuses on the prototypicality of the vestibular system as a sense modality.

A vestibular sense? The vestibular system is sensitive to acceleration and rotation of the head, as well as perturbations caused by contact with the ground. In fact it is plain that the vestibular system meets the first three conditions of being a prototypical sense, in that it detects a form of physical stimulus (inertial motion of the head) through a set of distinctive transducers, and has seemingly evolved for the purpose. Much is now known about the early processing of the vestibular system, and how and why it integrates with other modalities in non-cortical brain areas (Angelaki and Cullen, 2008). The primary peripheral component of the vestibular system is the ‘labyrinth’, a complex collection of chambers of the inner ear. This includes the two otolith organs, the utricle and saccule, which respond (respectively) to horizontal and vertical linear head displacement (due to tilting or translation), and the three semicircular canals, which each respond to head rotation along roughly orthogonal spatial axes (i.e. for roll, pitch, and yaw). This set of chambers is continuous with the cochlea, which is peripheral for audition. Like audition, vestibular transduction is via hair cells, a kind of mechanoreceptor. Thus the vestibular system clearly has a set of detector mechanisms that track the above mentioned physical properties. Does the vestibular system produce representations with nonconceptual content and a mind-to-world direction of fit? It certainly seems to. Indeed, the vestibular system would appear to provide both mind-to-world and mind-to-body direction of fit. For it provides information about the direction of gravity in addition to the movement of the body (even in the absence of input from external modalities). Moreover, the representations in question are plainly of a fine-grained or nonconceptual sort, and are provided as input for conceptual judgements about body movement and orientation.

362    J. Brendan Ritchie and Peter Carruthers Finally, that vestibular outputs are used in guiding intentional action seems clear, given their role in spatial navigation. Consider, for example, how difficult it is to navigate when experiencing vertigo or dizziness, both of which (especially in some chronic conditions) can be produced by vestibular disruption. Experimentally, the role of vestibular outputs has been shown using experiments that require subjects to orient spatially in the absence of vision. For example, in a common paradigm, subjects successfully reorient to an initial starting position after being passively rotated in the absence of visual input (e.g. the room is dark, or subjects are blindfolded). And as might be expected, individuals with peripheral vestibular pathologies are impaired relative to normal subjects when it comes to walking a linear path with their eyes closed, but not with eyes open (Cohen, 2000). We conclude that the vestibular system has all or most of the properties of a prototypical sense modality. Questions remain, however, about the nature of vestibular phenomenology. For some have claimed that by the time vestibular representations become conscious they have been tightly integrated with the outputs of the visual system and/or the proprioceptive system, in such a way that the resulting experiences are entirely visual or proprioceptive in nature.

A vestibular phenomenology? Angelaki and Cullen (2008) claim that there is no such thing as vestibular phenomenology. This is because of early integration of vestibular outputs with other modalities. While our sense of orientation and balance is largely determined by vestibular processing, it is only manifest in our awareness through visual and proprioceptive phenomenology. Put differently: if one were to subtract the visual and proprioceptive contents from our experience, then nothing distinctively vestibular would remain. If this claim were true, then the vestibular system might provide an example of a sense that does not contribute to conscious awareness through modality-specific phenomenology. Indeed, if the claim were true, then the status of the vestibular system as a prototypical sense modality could be challenged. For it would mean that vestibular outputs, as such, do not guide action. Rather, they provide input to vision and proprioception, and it is the latter that guide action. So the guidance function of the vestibular system would only be indirect. The claim of early integration seems insufficient to establish such a conclusion, however. For senses can be integrated and yet make distinctive contributions to our experience. (Think of taste and smell, for instance, or our earlier discussion of thermoception.) But another way of approaching the question is to consider what form of spatial reference frame vestibular processing utilizes. If it shares a reference frame with vision or proprioception, then that weakens the case for a distinctive contribution to phenomenology; whereas if it maintains its own reference frame then this would provide some support for the separateness of the contents produced. Early in processing the vestibular system operates with a head-centred reference frame, but since the vestibular system is also utilized for navigation, and hence action, vestibular representations must at least interact with body-centred (or somatotopic) and eye-centred (or retinotopic) reference frames. A  traditional view is that multi-modal integration depends on translation into a shared, ostensibly more abstract, reference frame, but Angelaki and Cullen (2008) themselves claim that evidence concerning vestibular

The Bodily Senses   363 integration with other modalities undermines this view. Inputs from the labyrinth and from neck proprioception allow the brain to relate the head-centred reference frame of the vestibular system to a somatotopic reference frame, which contains a neck component. This integration is provided by a mixture of modality specific and ‘intermediate’ reference frames. Another challenge to the idea of a common reference frame concerns visual–vestibular integration. Fetsch et  al. (2007) found that vestibular signals have a head-centred topography, but at the population level have a more retinotopic structure. Their data suggest that different reference frames remain distinct during integration. It remains possible, then, that the vestibular system makes a distinctive contribution to phenomenology. But this is hard to establish introspectively. In conditions like out-ofbody experience, for example, in which subjects feel as though they are experiencing the world and their body from a separate position, as well as in autoscopy, in which subjects still experience themselves as ‘at’ their body, but feel that their body is in extrapersonal space (it is not ‘part’ of them), subjects primarily report visual phenomenology. However, subjects may also report sensations of floating, lightness, or flying (Blanke et al., 2004). Such reports seem intuitive and plausible, and many of us will be familiar with similar experiences when dreaming (such as the sensation of falling). However, here we might again apply James’ thought experiment from earlier: when my head is actively or passively displaced, and I subtract the proprioceptive sensations of my neck, and of my feet on the ground, as well as any visual feedback, does a head-centred sense of inertia remain? This is by no means easy to answer. To assess whether there is such a phenomenology, one could perhaps put subjects in a sensory deprivation chamber to see what kinds of self-report are generated. Given the head-centred reference frame of vestibular outputs, perhaps one would predict a sense of the direction of gravity, and whether displacement is due to tilt or translation. Alternatively, one might probe the phenomenology of pilots, who rely heavily on the vestibular system during flight. However, while such phenomenology can be described, it is hard to imagine what it would feel like beyond a mere abstract description of this sort. The general question raised here is what it means for a sense modality to have a distinct phenomenology, and how one would go about searching for it, either introspectively or experimentally.

Conclusion While there are powerful reasons for recognizing the vestibular system as a distinct sense modality, it remains unclear whether it makes a direct and distinctive contribution to the phenomenology of our experience. (Indeed, this might partially explain why philosophers of perception have typically paid it so little attention.)

3  The proprioceptive senses The body-directed sense that has been of most interest to philosophers is proprioception, the sense of the body’s spatio-structural extension, which provides afferent feedback during movement for guiding action. Proprioception has not only been of interest to philosophers

364    J. Brendan Ritchie and Peter Carruthers in the analytic tradition, but also the phenomenological tradition—particularly in the work of Husserl (1989) and Merleau-Ponty (1962). (For phenomenological approaches to perception, see chapter 7.) Philosophers have offered markedly different views about the nature and possibility of a proprioceptive sense. Some claim that we do not have a sense modality for bodily position, because this implies the possibility of misrepresenting it (Anscombe, 1962). Others think that we have a special nonperceptual form of sensorimotor access to our own bodies (Merleau-Ponty, 1962). Thus there is no perceptual experience of one’s legs being crossed, only an awareness of the fact that one’s legs are crossed. We think, in contrast, that it is quite clear that proprioception is a prototypical sense modality.

A proprioceptive sense Proprioception is sensitive to (a range of) physical properties of the body, and receives input from detector mechanisms that transduce these properties into informational signals. Proprioceptive inputs come from the primary and secondary afferents of muscle spindles, cutaneous receptors that track skin elasticity, as well as mechanoreceptors in the joints. While it was previously believed that joint receptors play a major role in tracking position and movement, it is now known that muscle spindles are in fact the dominant input for proprioception (Proske and Gandevia, 2009). For example, vibrating the muscle spindles of blindfolded subjects produces illusions of arm movement and position. If this is paired with tactile contact between the vibrated limb and some other body part, stimulation induces illusions of body deformation. One example of this is the Pinocchio illusion, in which one experiences one’s nose growing outward while remaining in contact with the felt movement of one’s hand (Lackner, 1988). While information from muscle spindles provides the dominant afferent inputs for proprioception, when muscles span across multiple joints, muscle spindles can provide ambiguous inputs regarding joint contraction. In the case of fingers, the muscles are located in the hand and forearm, and the tendons extend across multiple finger joints. Thus for finger proprioception, cutaneous receptors that track skin elasticity seem to provide crucial information regarding finger position and movement, which cannot be extracted from the muscle spindles in the hand or forearm alone. Joint receptors, in contrast, are thought to be primarily limit detectors. That is, if one tries to straighten one’s arm beyond the limit of the joint, it is the sensation from the joint that informs us that the limit has been reached, not the experience of muscle stretch. It would seem that proprioception, or its constitutive input systems, satisfies conditions (1)–(3). But what about conditions (4) and (5)? It is clear that the constitutive systems of proprioception produce fine-grained or nonconceptual representational content with mind-to-world (or rather mind-to-body) direction of fit. Moreover, by providing afferent feedback about body position, some role for proprioception in guiding action seems clear. Matters appear somewhat more complicated, however, when we ask whether proprioception should properly be counted as a single sense, or multiple senses. On the one hand, the three main types of proprioceptive transducer track distinct physical properties. And it is possible for us to be aware of their outputs individually (e.g. the tensing of individual muscle groups when we exercise). So in some respects proprioception can be considered as

The Bodily Senses   365 three distinct sense modalities. (Contrast here the clear unity of the vestibular system.) On the other hand, it seems that part of the function of proprioception is to produce an overall representation of the position of the body for planning and guiding action, integrating together the outputs of the various contributing transducer mechanisms. (In this respect proprioception seems to differ from interoceptive systems, which are not obviously integrated into a single overall representation of body homeostasis.) Indeed, typically we are not aware of the components of proprioception. Rather, we are aware of the form and posture of our bodies. Furthermore, other sense modalities, such as touch, and aspects of interoception, require such form and posture representations to which they can be mapped. Deficits of different kinds put this fact in stark relief. Consider phantom limb patients, who still feel as though the amputated limb is present, or deafferented patients whose proprioceptive afferents have become de-mylinated, and so receive no proprioceptive (or tactile) inputs. In the former case, subjects report vividly the experience of a limb that they can see is not there, while in the latter, subjects have no awareness in the absence of vision of the bodies that they know themselves to possess. In such cases, subjects are clearly reporting deficits in body perception, which moves beyond sensations of stretched muscles or taught joints. This question of integrated representations for form and posture is related to how proprioception satisfies conditions (4) and (5). It seems plain that proprioception is used for guiding intentional action; indeed, this seems to be its primary function. It enables us to keep track of the spatial relationships between body parts and (along with other modalities) where the body is in relation to other objects in the environment. However, when one reaches for a cup of tea, one is not typically aware of how much one is stretching one’s muscles and aligning one’s joints. Rather it is the overall position of one’s arm and the form of one’s grip that one is aware of. In philosophy and cognitive science, holistic perception and representation of our own bodies has been associated with the notion of the body schema. Thus we now turn to the question of the relationship between proprioception and this construct. (For more on perceiving the body as an object, see chapter 27.)

Proprioception and the body schema Cognitive scientists have been increasingly interested in a distinction between the body schema and the body image, and it is in connection with these constructs that proprioceptive outputs have been thought to play an important role in conscious forms of perception and cognition, as well as in action. The origin of the distinction is in classical neurology (Head and Holmes, 1911–12), where it was thought to provide a way of explaining apparently contrasting deficits that somehow relate to the body. Unfortunately, use of the terminology has been confusing, with both terms having been used interchangeably or inconsistently by psychologists. (See Gallagher, 2005, for a useful review.) At first pass, the body schema is thought to be a long-term, regularly updated, unconscious representation of the body’s extension and posture, which is used to guide action. It is often claimed to be multi-modal, relying on proprioceptive, visual, haptic, vestibular, and motoric outputs. By contrast, the body image is a set of largely conscious states and processes, including our intermittent awareness of the body, our concepts and beliefs about the body, and our emotional responses to the body, and is sometimes claimed to be largely visual in nature.

366    J. Brendan Ritchie and Peter Carruthers While evidence for this distinction has traditionally come from neuropsychology, in recent years a great deal of research has been carried out with normal subjects. We cannot do justice here to the explosion of experimental and theoretical work in cognitive science on the body schema (and image). We will focus more on conceptual issues, while touching on recent empirical findings. We believe the distinction between body schema and body image as typically drawn is explanatorily inadequate. Clearly much of our conscious experience of our bodies is perceptual, while emotional or cognitive attitudes toward our own bodies are not. The heterogeneity of the body image suggests that it fails to constitute a natural kind. Moreover, it is much less closely related to proprioception. We therefore focus on the nature of the body schema and how it relates to proprioception as a sense modality. It should be noted, however, that there might be multiple body schemata, separately representing form and posture (Medina and Coslett, 2010). A first question is whether the body schema is genuinely multi-modal or primarily proprioceptive in nature. Is its frame of reference constitutively determined by proprioception along with vision and touch, or do the latter modalities simply provide inputs to what is in fact a proprioceptive representation? Note that if deafferented patients are thought to have a deficit in their body schema (Gallagher, 2005), this would seem to support the body schema being a proprioceptive representation; indeed, the point that subjects must compensate for the deficit using vision suggests that vision is insufficient by itself. If vision were constitutive of the body schema, then one might think it would still be providing some inputs. But if the body schema is a proprioceptive representation, then it makes sense that this deficit should be so utterly crippling for one’s awareness of one’s own body. In stark contrast, blind people are decidedly not blind to their own bodies, though they have a deficit in localizing them relative to objects in the environment. Furthermore, behavioural evidence suggests proprioceptive dominance. For example, during a motor imagery task proprioception, but not vision, influenced the coding of hand position during the task (Shenton et al., 2004). Also, in some situations where vision and proprioception provide conflicting information about body position (Hogendoorn et al., 2010), or size (Linkenauger et al., 2010), proprioception overrides vision. However, one might point out that there is also evidence that use of vision and proprioception to guide action is weighted based on which provides more reliable information about body position (van Beers et al., 1999). Does the body schema enter into consciousness? Traditionally many have defined the body schema as unconscious, hence distinguishing it from the body image. But given that we have set the body image aside we can ask whether there is a principled distinction to be drawn between the unconscious body schema and proprioceptive experience. Longo and Haggard (2010) found that when subjects are asked to point to positions on their occluded hand below a table, the body map of the hand is distorted with shortened fingers, which seems to reflect an unconscious representation with a similar distortion of form, namely the homunculus of somatosensory cortex. This result might be interpreted in terms of an unconscious representation determining our conscious experience. It suggests at least that form and posture representations are ‘built up’ out of proprioceptive components, creating distortions of form when not integrated with vision. Some have suggested that the body schema is part of the ‘how’-pathway for somatosensory processing, by analogy to the dual-pathways model of vision (Dijkerman and

The Bodily Senses   367 de Haan, 2007). As Milner and Goodale (1995) have shown, we can distinguish between the visual processes that contribute to action and those that contribute to object identification. Interpreted in this way, the body schema would be part of the action or ‘how’pathway, and be unconscious, while proprioceptive experience would be part of the perception or ‘what’-pathway, and be utilized in body recognition and localization. A complimentary addition to the dual-pathway view is that the body schema provides the afferent feedback for the forward model for motor control. The motor system generates two outputs. One, the ‘inverse model’, is a set of commands for motion given a prior representation of the body’s position. The other, the ‘forward model’, is a representation of the predicted position of the body after completion of the movement (Wolpert et al., 1995). This forward model is compared with the actual position of the body as the movement unfolds, based on afferent inputs from proprioception (and also vision), and enabling swift online corrections and adjustments. De Vignemont (2010) plausibly suggests that the body schema provides inputs for both the initial planning of action and its correction as action proceeds. Some evidence for this view, which ties the apparent unconscious nature of the body schema to its function of guiding action, comes from work on the rubber-hand illusion by Kammers et al. (2009). In their study the illusion was induced by synchronized stroking of a visible fake hand and the subject’s occluded hand; this stimulation has been shown to induce a sense of ownership over the visible fake hand (Botvinick and Cohen, 1998). Next, the fake hand was also occluded and subjects were asked to indicate the location of the stimulated limb in one of two ways: either by verbally reporting whether or not the experimenter was pointing to their limb location, or by reaching with their un-stimulated limb. If the body schema is influenced by the illusion, then it should influence subjects’ ballistic arm movements to bias them toward the fake hand, while if the illusion just influences their proprioceptive experience it would only have an effect on their verbal judgements. The finding was that ballistic arm-movements were not influenced by the illusion, which can be interpreted as supporting a dissociation between the body schema and proprioceptive experience. Despite this evidence, it is also likely that, as in the case of vision, these two pathways are not entirely encapsulated from one another (de Vignemont, 2010).

Conclusion Proprioception is a prototypical sense modality whose primary function is to inform motor planning and guide action. The primary output of proprioception can be identified with the body schema, under some characterizations of the construct, but it also issues in conscious experience of body position. While we have focused on the status of proprioception as a sense modality, there has been much interesting work on bodily self-awareness as a kind of nonconceptual primitive self-consciousness, and on how proprioception relates to our sense of ownership and agency over bodies and actions, both in philosophy (Bermudez et al., 1995) and cognitive science (Tsakiris and Haggard, 2005). A related topic is whether our knowledge of the body afforded by our bodily senses is immune to error through mis-identification (Evans, 1982; Legrand, 2006). In short, proprioception, and indeed the bodily senses in general, seem to be of central importance not just for our perception of our bodies, but of ourselves. (See de Vignemont, 2011; and chapter 27)

368    J. Brendan Ritchie and Peter Carruthers

4 Conclusion In this chapter we have discussed interoception, vestibular perception, and proprioception, arguing that each has a strong claim to be considered as (collections of) sense modalities, while also touching on a number of philosophical issues relating to these systems. In many cases these questions have not been widely discussed, since philosophers interested in perception have traditionally been focused on how we look out at the world. We believe that much interesting work remains to be done on how we look within.

Acknowledgements We are grateful to Frederike de Vignemont, Mohan Matthen, and Jeremy Pober for their helpful comments on an earlier draft of this chapter.

References Airapetyantz, E. and Bykov, K. (1945). 'Physiological experiments and the psychology of the subconscious'. Philosophy and Phenomenological Research, 5, 577–593. Akins, K. (1996). 'Of sensory systems and the "aboutness" of mental states'. Journal of Philosophy, 93, 337–372. Angelaki, D. and Cullen, K. (2008). 'Vestibular system: The many facets of a multimodal sense'. Annual Review of Neuroscience, 31, 125–150. Anscombe, E. (1962). 'On sensations of position'. Analysis, 22, 55–58. Armstrong, D. (1962). Bodily Sensations. London: Routledge & Kegan Paul. Barrett, L. F. (1998). 'Discrete emotions or dimensions? The role of valence focus and arousal focus'. Cognition & Emotion, 12, 579–599. Barrett, L. F. (2006). 'Are emotions natural kinds?' Perspectives on Psychological Science, 1, 28–58. Barrett, L. F. and Russell, J. (1999). 'Structure of current affect'. Current Directions in Psychological Science, 8, 10–14. Barrett, L. F., Quigley, K., Bliss-Moreau, E., and Aronson, K. (2004). 'Arousal focus and interoceptive sensitivity'. Journal of Personality and Social Psychology, 87, 684–697. Bermudez, J. L., Marcel, T., and Eilan, N. (eds) (1995). The Body and the Self. Cambridge, MA: MIT Press. Blanke, O., Landis, T., Spinelli, L., and Seeck, M. (2004). 'Out-of-body experience and autoscopy of neurological origin'. Brain, 127, 243–258. Block, N. (2006). 'Bodily sensations as an obstacle for representationalism'. In M. Aydede (ed.), Pain: New essays on its nature and the methodology of its study. Cambridge, MA: MIT Press. Botvinick, M. and Cohen, J. (1998). 'Rubber hands "feel" touch that eyes see'. Nature, 391, 756.

The Bodily Senses   369 Cameron, O. (2001). 'Interoception: The inside story—A model for psychosomatic processes'. Psychosomatic Medicine, 63, 697–710. Cameron, O. (2002). Visceral Sensory Neuroscience:  Interoception. New  York:  Oxford University Press. Carruthers, P. (2005). Consciousness. New York: Oxford. Carruthers, P. (2011). The opacity of mind. New York: Oxford. Chernigovskiy, V. (1967). Interoceptors. Washington, DC: American Psychological Association. Cohen, H. (2000). 'Vestibular disorders and impaired path integration along a linear trajectory'. Journal of Vestibular Research, 10, 7–15. Craig, A. (2002). 'How do you feel? Interoception: the sense of the physiological condition of the body'. Nature Reviews Neuroscience, 3, 655–666. Damasio, A. (1994). Descartes’ Error. New York: G.P. Putnam. Dijkerman, H. and de Haan, E. (2007). 'Somatosensory processes subserving perception and action'. Behavioral and Brain Sciences, 30, 189–201. Dorpat, T. (1971). 'Phantom sensations of internal organs'. Comprehensive Psychiatry, 12, 27–35. Evans. G. (1982). The Varieties of Reference. Oxford: Clarendon. Fetsch, C., Wang, S., Gu, Y., DeAngelis, G., and Angelaki, D. (2007). 'Spatial reference frames of visual, vestibular, and multimodal heading signals in the dorsal subdivision of the medial superior temporal area'. Journal of Neuroscience, 27, 700–712. Gallagher, S. (2005). How the Body Shapes the Mind. New York: Oxford. Griffiths, P. (1997). What Emotions Really Are. Chicago: Chicago University Press. Head, H. and Holmes, H. (1911–1912). 'Sensory disturbances from cerebral lesions'. Brain, 34, 102–254. Hogendoorn, H., Kammers, M., Carlson, T., and Verstraten F. (2010). 'Being in the dark about your hand: Resolution of visuo-proprioceptive conflict by disowning visible limbs'. Neuropsychologia, 47, 2698–2703. Husserl, E. (1989). Ideas Pertaining to a Pure Phenomenology and to a Phenomenological Philosophy, Second Book. Studies in the phenomenology of constitution, trans. R. Rojcewicz and A. Schuwer. Dordrecht: Kluwer. Izard, C. (2007). 'Basic emotions, natural kinds, emotion schemas, and a new paradigm'. Perspectives on Psychological Science, 2, 260–280. James, W. (1884). 'What is an emotion?' Mind, 9, 188–205. Kammers, M., de Vignemont, F., Verhagen, L., and Dijkerman, H. (2009). 'The rubber hand illusion in action'. Neuropsychologia, 47, 204–211. Koriat, A. (2007). 'Metacognition and consciousness'. In P. Zelazo, M. Moscovitch, and E. Thompson (eds), Cambridge Handbook of Consciousness (pp. 289–235). New York: Cambridge University Press. Lackner, J. (1988). 'Some proprioceptive influences on the perceptual representation of body shape and orientation'. Brain, 111, 281–297. Lange, C. (1885). Om sindsbevaegelser: et psyko-fysiologisk studie. Copenhagen: Jacob Lunds. Reprinted in C. Lange and W. James (eds), I. A. Haupt (trans.), The Emotions (1922). Baltimore: William & Wilkins. Larsen, J., Berntson, G., Poehlmann, K., Ito, T., and Cacioppo, J. (2008). 'The psychophysiology of emotion'. In M. Lewis, J. Haviland-Jones and L. Feldman Barrett (eds), Handbook of Emotions (3rd edn) (pp.180–195). New York: Guilford. Legrand, D. (2006). 'The bodily self: The sensori-motor roots of re-reflexive self-consciousness'. Phenomenology and the Cognitive Sciences, 5, 89–118.

370    J. Brendan Ritchie and Peter Carruthers Linkenauger, S., Ramenzoni, V., and Proffitt, D. (2010). 'Illusory shrinkage and growth: Bodybased rescaling affects the perception of size'. Psychological Science, 21, 1318–1325. Longo, M. and Haggard, P. (2010). 'An implicit body representation underlying human position sense'. Proceedings of the National Academy of Sciences, 107, 11727–11732. Medina, J. and Coslett, H. (2010). 'From maps to form to space: Touch and the body schema'. Neuropsychologia, 48, 645–654. Merleau-Ponty, M. (1962). The Phenomenology of Perception,. trans. C. Smith. London: Routledge & Kegan Paul. Milner, A. and Goodale, M. (1995). The Visual Brain in Action. Oxford: Oxford University Press. Picciuto, V. and Carruthers, P. (2014). 'Inner sense'. In M. Matthen, M. Biggs, and J. Stokes (eds), Perception and its Modalities. New York: Oxford University Press. Proske, U. and Gandevia, S. (2009). 'The kinaesthetic senses'. Journal of Physiology, 587, 4139–4146. Schachter, S. and Singer, J. (1962). 'Cognitive, social, and physiological determinants of emotional state'. Psychological Review, 69, 379–399. Shenton, J., Schwoebel, J., and Coslett, H. (2004). 'Mental motor imagery and the body schema: Evidence for prioceptive dominance'. Neuroscience Letters, 370, 19–24. Sherrington, C. (1906). Integrative Activity of the Nervous System. New Haven:  Yale University Press. Storbeck, J. and Clore, G. (2008). 'Affective arousal as information: How affective arousal influences judgments, learning, and memory'. Social and Personality Psychology Compass, 2/5, 1824–1843. Tsakiris, M. and Haggard, P. (2005). 'Experimenting with the active self'. Cognitive Neuropsychology, 22, 387–407. Tye, M. (1995). Ten Problems of Consciousness. Cambridge, MA: MIT Press. Tye, M. (2002). 'Representationalism and the transparency of experience'. Nous, 36, 137–151. Vaitl, D. (1996). 'Interoception'. Biological Psychology, 42, 1–27. van Beer, R., Sittig, A., and Denier van der Gon, J. (1999). 'Integration of proprioceptive and visual position-information: An experimentally supported model'. Journal of Neurophysiology, 81, 1355–1364. de Vignemont, F. (2010). 'Body schema and body image—Pros and cons'. Neuropsychologia, 48, 669–680. de Vignemont, F. (2011). 'Bodily Awareness'. In Edward N. Zalta (ed.), The Stanford Encyclopedia of Philosophy (Fall 2011 Edition). . Werner, N., Jung, K., Duschek, S., and Schandry, R. (2009). 'Enhancing cardiac perception is associated with benefits in decision-making'. Psychophysiology, 46, 1123–1129. Wolpert, D., Ghahramani, Z., and Jordan, M. (1995). An internal model for sensorimotor integration. Science, 269, 1880–1882.

Chapter 19

U nconscious Perception Jesse J. Prinz

Before the twentieth century, there was little appreciation of the fact that mental activity can occur without consciousness. Now unconscious mentality is widely accepted. This owes, in part, to Freud who emphasized unconscious motivations, but belief in the unconscious became more deeply entrenched with the rise of experimental psychology, and the discovery that human behaviour is often best explained by appeal to processes that take place without awareness. Unconscious perception is one of the most extensively studied phenomena of this kind. There is extensive evidence that we respond to stimuli presented to our senses without awareness. In this chapter, I will review some of that evidence and raise several questions:  what does it mean to say that there is unconscious perception? Is the evidence decisive? And how does unconscious perception differ from conscious perception?

1  Can perception be unconscious? The very idea of unconscious perception Perception is paradigmatically conscious, which is to say that the examples of perception that are most familiar take place consciously. Such examples include seeing a rose in full light, straight on, for a few seconds, or inhaling and enjoying its fragrance. When asked to provide examples of perception, we don’t think of cases where a stimulus is too brief or peripheral to be fully and clearly experienced. We learn the concept of perception from conscious cases, and this raises the question: what would it even mean to say that perception can take place unconsciously? The answer is that there are states and processes that are similar in certain important respects to conscious perception, but they are unconscious. To clarify this idea, we need to know which features of paradigm cases can be preserved when we are not conscious, and whether these are sufficient for applying the concept of perception. First, it is important to clarify what ‘conscious’ means in this context. There is no single use of that term, and, consequently, the concept of unconscious perception has several different meanings.

372   Jesse J. Prinz Conscious might mean full awareness, where that refers to our capacity to explicitly identify a stimulus that has been presented to our senses, that is, to classify it conceptually and make it available for executive processes such as reporting and deliberative reasoning. ‘Conscious’ might also mean minimal awareness, as when we notice that a stimulus has been presented, whether or not we can classify it or report its identity. And, finally, ‘conscious’ may refer to phenomenal character, in which case conscious perception is perception that feels like something to have. To postulate unconscious perception is to postulate perception that lacks one or more of these properties, which I’ve listed in order of controversy. Few would deny that we can perceive something without being able to classify it, but it is a bit more controversial to suppose that something can be perceived without being aware of it at all. Most controversial is the claim that we can perceive something without having any qualitative experience—the form of unconscious perception that will be my focus here. These controversies are empirical, but there is also a conceptual question. One might argue that the notion of unconscious perception is incoherent. If we learn the concept of perception by pointing to paradigm cases, and those cases are conscious, how can we be sure that that concept of perception does not include consciousness as a necessary feature? By analogy, some people suppose that pain is essentially conscious, because when we point to pain, we point to its painfulness, which is an experiential property (Kripke, 1980). Those who believe in unconscious pain must say that such feelings are contingent to pain and there is something else that is essential, such as the informational role. Settling the conceptual question of whether perception can be conscious depends on whether, in pointing to cases of conscious perception, we necessarily point to their consciousness. These is no easy way to settle this question without a semantics of ostension, but there is good reason to think that, when we point, the features we point to are treated as contingent unless we explicitly specify otherwise. For example, if someone has never seen a goat without fur, it doesn’t follow that fur is construed by that person as necessary. Fur will be part of the prototype that is used for goat classification, but prototypes are treated as sets of contingent features, which can come and go, as long as the majority are in place. In fact, with natural kind concepts, like goat, we are open to the possibility that every feature used to recognize goats could be absent and we could still have a goat. Likewise, when we point to cases of perception, there are many features that might suffice for application of that concept, and we may be agnostic about which, if any, are necessary. We may also recognize cases of perception by consciously noting certain essential features of perception that turn out to also occur unconsciously. The fact that we use consciousness to recognize certain features does not mean that those features depend on consciousness. What features might we be pointing to when we identify cases of perception? The most obvious and important might be called information transduction. ‘Transduction’ refers to the transformation from one magnitude into another, and in the case of perception, physical magnitudes, such as light, chemicals, and sound, are transduced into neural activity that carries information. Stated less reductively, perception involves a transition from stimulation to representation. Objects stimulate our senses, and we thereby represent those objects. Without getting bogged down in theories of perception (e.g. must perception always be causal? What causal chains are deviant?), we can note that prototypical cases of perception involve the transition from stimulation to representation, and we can also note

Unconscious Perception   373 that, conceptually at least, this is dissociable from consciousness. So consider a case where a stimulus gets represented but there is no experience of it and no awareness. Should we deny that this is perception? Such a move would treat perception as a concept that worked differently from the vast majority of other concepts by insisting that features found in all paradigm cases are necessary (though see Matthen, Chapter 30, this volume, on the insufficiency of representation for perception). More to the point, the thesis would be little more than a distraction. Suppose there is no fact of the matter which features in a folk concept are necessary, and hence no fact of the matter whether there can be unconscious perception. It would still follow that there is conceptual room for unconscious information transduction through the senses. And that possibility has much more theoretical and practical interest then question of how to label it. In what follows I will stipulatively use ‘unconscious perception’ to refer to unconscious transduction of information that is in someone useable by the organism that transduces it. Does unconscious perception so-defined exist? If so, what is its nature? I  will focus entirely on vision, because it has been most extensively investigated, but all the phenomena I describe have known analogues in other senses.

Evidence for unconscious perception The topic of unconscious perception came into popular awareness when advertisers began exploring the possibility of using subliminal messages to increase sales. In the 1950s, James Vicary claimed this was possible, but subsequent research found little support. That is, until recently. It has since been demonstrated, for example, that a subliminally presented smile can make a beverage seem more delicious. When presented with happy as opposed to angry faces below the conscious threshold, participants subsequently give higher ratings to a soft drink, pour more of it, and express willingness to pay a higher price (Winkielman et al., 2005). This, along with hundreds of other studies, suggests that unconscious perception is possible. A review of this vast empirical literature is impossible, but let us examine a few well-established and widely discussed phenomena. Recall that the term unconscious perception can mean three things: perception without awareness of what is seen, perception without awareness that anything was seen, and perception without any experience whatsoever. Different empirical phenomena help to establish each of these possibilities. Consider, first, change blindness. This term is usually used to refer to cases in which participants are presented with a complex stimulus that changes in some respect, but the changes go unreported. A typical experimental paradigm involves still images that flicker on and off, with a change occurring again and again with each flicker until it is noticed (Rensink et al., 1997). Alternatively a change may take place very slowly and go undetected, or it may occur when participants glance away (Simons et al., 2000). There is evidence that unreported changes are nevertheless perceived in such experiments. For example, when participants are presented with an array whose elements change, they can guess which elements were present above chance when given a forced choice (Mitroff et al., 2004). There is also evidence that change detection occurs even when it goes unreported (Fernandez-Duque and Thornton, 2000). What should we say about these features that are perceived unconsciously? Are they experienced at all? Do participants know that

374   Jesse J. Prinz something is experienced? Here I think we can safely answer in the affirmative. In some change blindness studies, participants will view the same flickering stimulus for many trials, lasting tens of minutes, before a change is reported. While this is going on, they presumably begin to attend to each portion of the stimulus hoping to catch the change. In some cases, the change is quite dramatic: the colour of a large focal object might change, a clearly discernable object may disappear or shrink dramatically, or the position of two objects may be switched. It is hard to believe that these objects are not consciously experienced when trials are repeated for long durations. A more likely explanation is that participants in many change blindness studies consciously see the features that change, but they fail to notice that they are changing. One might experience a red parrot, and then the same parrot may appear green, but one may fail to realize, from one moment to the next, that it has transformed. There are two possible explanations of why participants in change blindness studies fail to notice changes in features even when those features are experienced. The first is that the features are experienced but not categorized. On this view, we can fail to see a bird change from red to green because one doesn’t classify the colour on each viewing. Another possibility is that features are classified but not stored in short-term memory from one moment to the next. One may see the red bird as red, but not retain that information. It is not easy to decide which of these options is right, but there is some evidence for the second option: change blindness is a memory failure. The reason stems from the evidence that unnoticed changes are unconsciously perceived. As noted, unnoticed features prime subsequent judgements in forced choice tasks. That suggests that these features are being classified automatically when they are seen. Priming is usually explained by positing unconscious semantic networks. An item that does not get into short-term memory, may nevertheless activate a stored representation in any unconscious network, and that activation will then facilitate memory access during a subsequent retrieval task. If unnoticed features were not classified, such priming would be impossible to explain. It follows, then, that change blindness is, or at least can be, a case of perception without conscious access to what is seen. Participants in these studies are aware that they have seen the flickering images, and they may have even consciously experienced the items that change, but they have not retained those changing items in explicit memory. One might say they consciously experience that which changes but they lack awareness of the change. This can be explained by presuming that the changing items are phenomenally experienced but not encoded in memory. These items are accessible to memory, but not accessed or encoded. Equating such encoding with awareness, we can call this perception of change without awareness of change. I think we can even call it conscious experience of change without conscious awareness of change. A similar phenomenon occurs in Sperling’s (1960) classic studies of iconic memory. Here participants are briefly presented with arrays of alphanumeric characters and they have to report afterwards what was in the array. They typically retain only about three items, but it is quite clear to them that they saw an array and that the items in the array were alphanumeric. The unreported characters are definitely perceived, however, because they can be recalled if cued after stimulus offset. This suggests that every item gets into the visual system, but only a few enter awareness. The possibility of perception without awareness of what was seen can also be established in pathological cases. Consider associative agnosia. In this condition, people can discern

Unconscious Perception   375 and even draw the contours of visually presented objects, but they cannot recognize them. We might say that such individuals are conscious of seeing something, but they don’t know what. There is nevertheless evidence that, in some cases of associative agnosia, unconscious perception of object identity is possible. This is especially evident in some cases of prosopagnosia, or face blindness, which is a form of associative agnosia. Some people with prosopagnosia show signs of implicit recognition when viewing faces that they fail to recognize explicitly (Bauer, 1984). In the cases I have been considering, we might say that the stimuli in question are only partially outside awareness. For example, in a Sperling study, participants know that the undetected stimuli are alphanumeric. Partial awareness comes in degrees. For example, if a letter is presented more briefly than in Sperling’s studies, participants might see a brief flash and not know if it was a character or some other kind of stimulus. Even in this case—what I called minimal awareness above—viewers are aware of some aspect of the stimulus, such as its size and whether or not it was coloured. In this sense, minimal awareness is qualitatively like more complete awareness without awareness of what is seen. In both cases, object identity is perceived without awareness, but, in the more minimal cases, viewers are aware of hardly any stimulus features at all. To make a qualitative leap, we can ask whether perception can occur without any awareness of the stimulus. Let’s begin with cases where the stimulus is phenomenally experienced (qualitatively conscious), but without any awareness—this would be a case of experiencing something without realizing it. It is controversial whether this ever happens. Block (1995) argues for this possibility when he says that we can be phenomenally conscious of a stimulus while lacking access consciousness of it. His examples include things like hearing an air conditioner in the background; much of the time we are not aware of the dull sound, but when it pops into awareness, we have the impression that we were hearing it all along. Unfortunately, it is hard to confirm such reports. If a person claims the sound was there based on explicit memory, then presumably she had access to it, and if the report is not based on explicit memory, we cannot have confidence that it is accurate. Nevertheless, I agree with Block that we can have conscious experiences without awareness. One reason for this stems from the fact that the grain of awareness seems coarser than the grain of experience. To notice something, we have to classify it. But we can experience many things that we cannot classify. Colours, for instance, can only be classified in two ways: using categorical colour concepts, which treat very similar colours as alike, or by comparative categories, which can discriminate slightly different colours when, and only when, they are presented together but cannot imagine what happens when a specific shade of orange is presented in isolation. We cannot apply a concept that refers categorically to that exact shade, and nor can we compare it to another orange because none is present (I will also assume, for the example, that no other orange is imagined). In this case, we cannot be aware of the specific shade, because no exact classification, whether categorical or comparative is available. Nevertheless, it is phenomenologically plausible that we experience the exact shade. This may be a case of phenomenal perception without awareness; we are aware that the colour is orange, but not aware of its specific identity, even though it is experienced in all its specificity. Colour is a special case, because colour is a continuous magnitude, and such magnitudes may outstrip the representational resources that underlie conscious awareness. But the phenomenon in question may not be unusual. Awareness, I  have said, arises when

376   Jesse J. Prinz executive systems gain access to some aspect of a stimulus, but there is reason to think that many stimuli never get to this stage of processing. The visual world is extremely complex. At any given moment, we may be seeing dozens of objects, and a richly detailed field of colours and shapes. Vision is imperfect of course, unevenly sharp and changing with every saccade, but it is much richer in detail than awareness. Limits on short-term memory all but guarantee that we will be aware of less than we see. It could turn out that everything outside awareness is also unconscious phenomenologically, but the colour case is proof of the possibility that phenomenology outstrips awareness. Once we grant that possibility, there is little reason to deny that phenomenology is generally richer than awareness. One might push this idea to its limit and say that everything we perceive is also phenomenally experienced. But there may be reason to deny that. There may be stimuli that are perceived in the absence of phenomenology. This would be the most interesting kind of unconscious perception, because it would lack consciousness in every sense: awareness of what is seen, that something is seen, and phenomenal character. Is such perception possible? Many psychologists believe that perception can be unconscious in this strong sense. Four sources of evidence have been particularly well examined. One comes from the pathological condition called blindsight. Individuals with lesions in the primary visual cortex report blind spots (scotoma) corresponding to the receptive fields that have been damaged, but sometimes show residual visual capacity in those scotoma. For example, when asked the location of an object in the blind field, a person with blindsight will insist that he or she cannot see it, but, when forced to guess, answers will be highly accurate. Individuals with blindsight can even discriminate simple shapes (Weiskrantz, 1986), react to emotional expression (de Gelder et al., 2005), and to avoid obstacles while walking (de Gelder et al., 2008). With very intense stimuli or transcranial magnetic stimulation, these individuals sometimes report residual awareness, but, ordinarily they claim they lack experience in the affected region. Blindsight is an unusual syndrome brought on by brain injury, but there are also apparent cases of perception without experience in healthy individuals. The most extensively studied phenomenon is backwards visual masking. In this paradigm, people are presented with a brief visual stimulus, followed by a second stimulus (often a pattern of random shapes) that ‘masks’ the first. When the initial stimulus is presented for sufficiently short duration (usually under 25 milliseconds), people often have no idea that it was presented, and they perform at chance levels of accuracy when asked to guess which trials had a first stimulus and which didn’t in a block of mixed trials. Stimuli presented for longer intervals are often experienced but not discriminated, but when presentation drops below 25 milliseconds, subject experience drops off for most people (Szczepanowski and Pessoa, 2007). Nevertheless, there is evidence that such stimuli can be perceived. For example, Naccache and Dehaene (2001) presented participants with masked numbers for 16 milliseconds, and found that they primed performance on a subsequent task involving numerical quantities. A third phenomenon involves binocular rivalry. When people are simultaneously presented with two different stimuli, one in each of eye, they often report seeing only one. If these stimuli are equiluminous, the experience usually alternates between the two every three seconds or so, as if the two stimuli were equal rivals in the battle for consciousness. If one stimulus is considerably brighter, however, the other stimulus becomes completely inaccessible to consciousness, and people report that they have had no experience of it—a phenomenon called interocular suppression. The suppressed stimulus can nevertheless

Unconscious Perception   377 influence behaviour. For example, Jiang et al. (2006) presented participants with images of naked bodies in one eye while presenting bright colour patterns in the other. The colour patterns rendered the nudes invisible. They nevertheless found that participants showed improved performance in a subsequent perceptual discrimination task when the target to be discriminated was presented in the same location as the nude. The effect worked only for nudes that corresponded to a participant’s sexual preference, suggesting that the nude image was unconsciously recognized. Jiang et al. (2006) claim that the nude caused participants to attend to the unconscious nude, which would entail that attention can occur without consciousness. An alternative possibility is that the unconscious nude caused participants to generate a motor plan to shift their gaze to the location of the nude. This may be a more plausible explanation because ordinarily attention results in consciousness, as in the extensively studied phenomenon of visual pop-out where a contrasting stimulus captures attention and enters conscious awareness (for further discussion, see Prinz, 2012: ch. 3). There is also evidence that attention is necessary for conscious experience. The most discussed case of this is inattentional blindness (Mack and Rock, 1998). This phenomenon arises when people are presented with a task that consumes considerable attention to perform, such as discriminating which of two intersecting lines is longer. While this primary task is taking place, an unexpected stimulus is presented for a brief interval, and then participants are given a series of questions to probe whether it was seen. In many cases, participants claim not to have had any experience of the unexpected stimulus. That stimulus is presented for a long enough time to be readily seen in conditions where there is no attention-demanding task being performed. Mack and Rock presented their stimuli for 200 milliseconds. There is evidence that unseen stimuli can nevertheless influence behaviour in inattentional blindness. For example, Mack and Rock found that invisible words improved performance almost fivefold on a subsequent word-stem completion task. I will return to the theme of attention and consciousness below. For now, the important lesson is that withdrawal of attention can eliminate consciousness without eliminating information transduction. In both interocular suppression and masking, one stimulus renders another invisible. The effect is bottom-up in that there are features of the suppressing or masking stimulus that make it impossible for the other stimulus to be seen by the human visual system. Blindsight might also be described as a bottom-up phenomenon, in that it depends on an impairment of the visual system rather than a cognitive malfunction. In contrast, the phenomenon of inattentional blindness, the elimination of consciousness depends on top-down factors, namely the goal of the perceiver to focus on another task. Thus, there can be different causes of unconsciousness for stimuli presented in the visual field, but no matter the cause, unconsciousness presented stimuli exert an influence, suggesting that these are all cases of unconscious perception. Importantly, all four of these phenomena are similar in that the visually presented stimuli do not seem to cause any phenomenal experience. At least people in these studies do not report seeing the stimuli when explicitly asked. They do not even report that something unidentifiable was flashed. They are surprised if they are told that there was any stimulus presented. Nevertheless, the stimuli seem to have an impact on the visual system, and visual properties, such as shape, can be discerned. This is evidence for unconscious perception in the strongest sense—perception with no conscious experience. I will focus on this strong form in what follows.

378   Jesse J. Prinz

Alternative interpretations It is difficult to deny the claim that someone can perceive something without being aware of what was seen. To be aware of the identity of a stimulus is a cognitive state, and verbal reports are often a good measure of such states. If someone fails, when probed, to report the identity of a stimulus, while showing recognition on implicit measures, that indicates there was perception without awareness. But the evidence for perception without phenomenological experience is more controversial, because there is no uncontroversial measure of phenomenology. We have seen four lines of evidence for perception without experience, and each can be challenged. Here I will briefly consider some challenges. Let’s begin with blindsight, which has recently been subjected to an empirical challenge. On the textbook interpretation, individuals with blindsight are capable of perceiving things presented to their blind visual fields in the absence of phenomenology. But it is also known that stimuli presented in those blind fields can produce vague visual experiences under certain circumstances; for instance bright flashing stimuli can have a phenomenal impact. Overgaard et al. (2008) began to wonder whether successful visual discrimination in blindsight depends on such residual experience. Working with a blindsight patient, they replaced the standard binary scale, which had been used in other blindsight studies (Did you see the stimulus or not?), with a four-item scale (Did you see: a clear image, an almost clear image, a weak glimpse, or nothing at all?). In their patient, successful discrimination correlated with levels of clarity on this scale, and the correlation was comparable to what can be found in people with an intact visual system. Moreover, when the patient in their study reported having no experience, she showed no evidence of stimulus detection. Overgaard et al. (2008) also established that, when a binary scale is used, the patient does show successful discrimination without reported experience, as in previous blindsight studies; suggesting that the binary scale leads people to report cases of minimal experience as blindness. Overgaard et al. conclude that extant studies of blindsight provide no decisive evidence for perception without experience because they have relied on binary scales. These findings are certainly important, and they warrant reconsideration of methods used in blindsight research, but it is important to bear in mind that Overgaard et al. are reporting on one patient. Their patient may be have an unusually high level of spared phenomenology; she reports clearly seeing the stimulation in 7 out of 33 trials. Performance in blindsight is varied, and we cannot assume that every patient who has been successfully tested is relying on residual experience. In fact, there is reason to doubt this. Residual experience is normally limited in blindsight, even in the patient that Overgaard et al. tested, but accuracy can approach ceiling levels. For example, Weiskrantz (1986) reports nearly perfect performance in blindsight on discriminating Xs and Os, on reaching to a target, and on directed saccades after training. It is extremely unlikely that residual experience is available in every one of these successful trials. Why, then, do Overgaard et al. get such a high correlation between success and reported experience? One possibility is that the patient is unusual. Another possibility is that, once a stimulus has been unconsciously detected, the patient spontaneously generates mental imagery to enhance the percept. It is possible that successful performance derives from the unconscious representation, and that consciousness is a consequence of that unconscious representation; if so, consciousness isn’t the cause of success, but rather another effect of the same unconscious state.

Unconscious Perception   379 Overgaard et al. take subjective reports seriously in assessing consciousness, but it is sometimes pointed out that such reports are best seen as evidence for awareness rather than phenomenal experience (Block, 1995). This raises a challenge when considering the evidence that we examined in arguing for perception without phenomenology. Could it be that, in all these studies, there is experience that participants just can’t report? Let’s consider three possibilities. First, a sceptic might suggest that the stimuli used in masking, interocular suppression, and inattentional blindness studies are consciously experienced but merely too degraded to report. This seems unlikely, given that the stimuli are often unconsciously perceived to the point of recognition (Mack and Rock, 1998). All of these methods also use stimuli that are perfectly clear, and the studies are performed on individuals whose visual systems are intact. Models of visual processing do not normally suppose that, say, stimulus duration is essential for acuity. Furthermore, when visual representations are degraded, they are often still reportable, as when a person with myopia reports on a visible, but unintelligible sign in the distance. Second, the sceptic might suggest that the stimuli in these studies are consciously perceived but immediately forgotten. This possibility makes sense when considering masked priming studies, in which the stimulus lasts for less than a second, or Mack and Rock’s inattentional blindness studies (Wolfe, 1999). But it is an odd explanation for interocular suppression, where the unreported stimulus can be present for an indefinitely long period of time. In addition, Mack and Rock explicitly address the claim that inattentional blindness is really inattentional amnesia. In one study, they present participants with two consecutive stimuli that would ordinarily generate illusions of motion. When one of those stimuli is conjoined with a their high-attention task, the illusory motion disappears and participants report seeing just the other stimulus. This suggests that the unreported stimulus is processed in a different way than the one that is reported, and not simply experienced and forgotten. More strikingly, Most et  al. (2005) carried out an inattentional blindness study in which the unreported stimulus was present for five seconds, rather than 200 milliseconds. They used a primary task in which participants were asked to count how often animated polygons collided with the side of the screen, and while this was going on an unexpected stimulus that contrasted markedly from those polygons ambled across the centre of the screen. It seems unlikely that such an odd and slow stimulus would be forgotten if it were consciously seen. If unreported stimuli are not the result of degraded representations or fleeting memory, then the sceptic might adopt a more radical position: there can be conscious experiences that are present but unknown to us, despite the fact that they are long-lasting and perfectly clear. Admittedly, this is very hard to rule out, because reporting is not a perfect guide to phenomenology (as one can infer from cases of memory failure). But it is also hard to see why we should postulate phenomenal states that people insist they are not having. Imagine a participant in Jiang et al.’s (2006) interocular suppression study who insists that there is no nude visible while staring at a nude in the suppressed eye. The participant may be very disappointed that the nude is invisible and totally unable to discriminate trials with and without nude pictures. To assume that the nude is being experienced is a little bit like entertaining a sceptical hypothesis: we lack a positive reason for thinking that there is experience when that is emphatically denied.

380   Jesse J. Prinz The sceptic might backpedal and grant that there is no experience in interocular suppression cases, while insisting that there is experience in the cases of masked priming and inattentional blindness, since, in these cases, the unreported stimuli are not present when participants report on their phenomenology. But there is a phenomenological reason to reject this sceptical hypothesis. In both cases, there seems to be a dramatic subjective difference between conditions under which a stimulus is reported and when the same stimulus is not. Consider masking. When a stimulus is presented for less than 25 milliseconds, people report seeing nothing. When it is a little longer, they report seeing a flash. From a subjective point of view, these are very different states experientially. Participants in such studies will insist that this is not just a difference in what they can say about the stimulus, but in how it feels. The sceptic needs to explain that difference. Or take a phenomenon that is related to inattentional blindness: motion induced blindness. Here participants view a screen with three bright yellow dots against a black background with a dark blue grid. Nothing could be more visible. Then the grid starts to move and the dots seem to blink out of existence, because the motion captures attention. This is a startling effect, and viewers are astonished to learn that the dots are actually present the whole time. This seems to be part of phenomenology, not just reportability; we can experience the dots blinking away into oblivion. It would be bizarre to suppose that phenomenology remains constant while cognitive access fluctuates. Thus, the sceptic who insists that experience is present when masked or unattended stimuli go unreported is actually failing to acknowledge the phenomenological evidence. Like scepticism in epistemology, it is hard to silence those who insist on possibilities that are impossible to confirm and at odds with how things seem. But, when constructing theories of how the mind works, it’s not obvious that we should take such scepticism seriously. It’s not clear what predictions the sceptical view makes and how it can be falsified. I am inclined to conclude, therefore, that the evidence for perception without phenomenology is strong. It is also impressive that such evidence has been obtained using a variety of different methods, and that evidence for perception exists even in cases where participants insist with complete confidence that there was no experience. It is unlikely that such results stem from methodological flaws in the empirical studies. The best explanation of the empirical evidence is that we can perceive things in the absence of experience.

2  The nature of unconscious perception Having established that perception is possible without experience, we can now ask about the nature of there unconscious states. Are they like ordinary percepts except in that they lack phenomenology? What processes determine whether a percept will be conscious? These are the questions I turn to now.

Representations Earlier I  defined perception as the transduction of information, and I  have been arguing that this can occur without awareness or experience. Now a question arises about the information that gets transduced without consciousness. Is this information equivalent

Unconscious Perception   381 in content to what gets transduced consciously? Are there kinds of information that are essentially conscious? This can be understood as a question about perceptual representations. When we perceive unconsciously, are all kinds of perceptual representations used, or are some unavailable without consciousness? It is widely accepted that perceptual systems use multiple representational codes with different content. The visual system, for example, divides into as many as three dozen anatomical areas, which may be specialized for different purposes. Some of these areas are retinotopic, preserving the basic spatial configuration of the eyes; some are topographic but not retinotopic, preserving spatial relationships between objects regardless of eye position; and some are more abstract, giving object identities without information about relative position or size. It is plausible that some of these kinds of representations are never conscious. I have argued, for example, that we never have experiences corresponding to a representation of an object that abstracts away from the viewing position (Prinz, 2000, 2005). If this is right, some representations are essentially unconscious. My question here, however, is whether some representations are only available when we perceive consciously, regardless of whether the representations are themselves conscious. That is, I am asking whether unconscious perception limits the range of visual representations that can be utilized in some way. I will consider several possibilities, and conclude that unconscious vision is very much like conscious vision. Then I will propose one difference. To get our feet wet, consider the hypothesis that unconscious perception is always superficial—providing the shapes of objects but not their categorical identity. This is a tempting idea, since we don’t spontaneously make use of unconscious perception in deliberation; it often seems semantically idle. In cases of word-stem completion, unconscious primes clearly activate letter shapes, but that is a superficial feature. But priming effects are not restricted to shapes. Crepaldi et al. (2010) found that word priming involves semantic analysis: ‘fell’ primes ‘fall’ more than it primes ‘fill’. And recall that Naccache and Dehaene (2001) primed number concepts by presenting unconscious numerals. Therefore, it seems that words, at least, activate semantic information when presented unconsciously. This, however, does not establish that we have unconscious access to semantic visual information, because the semantic information associated with words may be stored outside of the visual system. But we have already seen evidence that images are processed semantically without consciousness. For example, Winkielman et al. (2005) primed evaluative judgements by presenting unconscious emotional expressions and de Gelder et al. (2005) got similar results with a blindsight patient. In an even more powerful demonstration of picture priming, Van den Bussche et al. (2009) presented participants with pictures of objects or animals for 13 milliseconds, followed by consciously presented words; on each trial, participants were then asked whether the word represents an animal or nonanimal, and they found that response times decreased when the category of the word was congruent with the picture. Because words and pictures have different shapes, this priming is clearly semantic, and it depend on extracting meaning from a picture, which implies that semantic representations in vision are available without consciousness. Let us turn now to another hypothesis. It is well-known that the visual system is hierarchical and represents objects from both a viewpoint-specific perspective and in a more abstract way that is invariant across a wide range of viewing positions. The invariant representations are known to be important for object recognition, but viewpoint-specific representations may play some role in object recognition as well (Tarr and Pinker, 1991). So the

382   Jesse J. Prinz question arises, are both viewpoint-specific and invariant representations active during unconscious perception? Here, again, the answer seems to be affirmative. Both priming experiments and electrophysiological studies confirm that these two levels of representation can be accessed when masked stimuli are presented (Eddy et al., 2009). Such findings rule out a third hypothesis about the distinctiveness of unconscious perception. Milner and Goodale (1995) proposed that vision divides into two streams: one that is dedicated to object recognition and one that is used for visually guided action. They speculate further that consciousness is located exclusively in the object-recognition stream, which is located in ventral areas of the visual system. The dorsal stream, which is used for action guidance is, on their view, entirely unconscious. This raises the possibility that unconscious perception activates the dorsal stream and not the ventral stream. One piece of evidence for that comes from blindsight. Individuals with blindsight have very limited object-recognition abilities, but show remarkably well preserved grasping behaviour. They rarely make mistakes when reaching for an object. For them, it is plausible that unconscious perception is intact in the ventral stream, but not the dorsal stream. This, however, does not generalize. We have just seen that there is evidence for object representation at every level of the visual hierarchy including the levels most involved in object recognition; these are located in the ventral stream. Moreover, the Milner and Goodale distinction has been empirically challenged on the ground that both dorsal and ventral streams seem to contribute to recognition and action, and there is considerable connectivity between the two (see Prinz, 2012, for review). The fact the blindsight patients are better at reaching than recognizing is probably owing to the fact that their lesions in the primary visual cortex degrade the inputs that are used for recognition, leaving subcortical paths that facilitate visually guided action intact. So far, we have considered three ways in which unconscious vision might be hypothesized to differ from conscious vision, and we found little support. Unconscious vision seems to involve every level and pathway within the visual system. But there is another hypothesis worth considering: it might be supposed that unconscious visual representations are not bound together. Binding can occur both locally and globally: the features of an object can be bound into a coherent whole, and different objects can be bound together into a coherent scene. Some researchers have suggested the binding requires attention (Treisman and Gelade, 1980) and consciousness (Crick and Koch, 1990). There is, however, evidence for unconscious binding. Consider words. These are collections of letters, which have multiple parts, bound together into ordered arrangements. Words can be perceived unconsciously, suggesting that both local and more global kinds of binding can occur without consciousness. There is also evidence that people can perceive the gist of a scene unconsciously, and gist information may depend on the integration of both local and global features (Loschky et al., 2007). That would suggest that binding can occur without consciousness (see Prinz, 2012, for a review of evidence separating binding from both attention and consciousness). Relatedly, one might think that conscious perception cannot integrate relational information between two objects or between foreground and background. This too, seems to be untrue. Unconscious priming with works shows integration of letters, and there there is also evidence for unconscious processing of context effects in which the brightness of a stimulus depends on the background (Persuh and Ro, 2011). This indicates that unconscious perception can integrate information across the visual field.

Unconscious Perception   383 All these findings suggest that unconscious perceptual representations can be very much like conscious perceptual representations. There is no compelling evidence that unconscious processing takes place in different sensory pathways than conscious processing nor that it carries different information. There is one obvious difference, however, between conscious and unconscious perceptual representation. The former feels like something and the latter does not. Notice that this is not just the platitude that conscious states are conscious. Rather, it is a substantive claim about conscious states, that they feel a certain way—they have qualitative character. The characteristic blueness of blue seems to be an aspect of conscious, but not of unconscious perceptual states. Some philosophers would reject this claim, saying that perceptual states always have their qualitative character whether conscious or not (e.g. Rosenthal, 1991). But I am disinclined to go that route. Philosophical puzzlement about consciousness often involves qualitative character. It is mysterious, for example, why brain states that are physically similar have different character, and elaborate thought experiments explore the question of whether qualitative character can be inverted. If qualitative character were the kind of thing that could be possessed in the absence of consciousness, these puzzles would be very difficult to motivate. I agree that qualitative character is something over and above consciousness, but I think it depends on consciousness, and it is what makes the questions about consciousness so challenging. If there could be conscious states without qualia (a possibility I doubt), they wouldn’t pose a particular hard challenge for materialism for such states wouldn’t feel like anything, and their being conscious might be cashed out in epistemic or informational terms. I think the mind–body problem arises only when we bring in how mental states feel, and consciousness is mysterious precisely because conscious states, unlike unconscious ones, feel like something. If this is right, then conscious perception and unconscious perception have at least one difference in addition to their status vis-à-vis consciousness. The former have and the latter lack qualitative character. This thesis places considerable constraints on what qualitative character could be. I have argued that conscious and unconscious perception carry the same information, which means that qualitative character cannot be a simple representational property, as some representationalists have argued (Dretske, 1995; Tye, 1995). Absent any reason to think that content changes when we become conscious, I think it is best to say consciousness states have a non-representation property that unconscious states lack. Much controversy surrounds the nature of that property, but we have informal ways of describing it in folk psychology, such as ‘the way things feel’. Blue character can be described as the way blue things feel when they are consciously perceived. When they are unconsciously perceived, they don’t feel like anything at all. But what are these ways of feeling? Can qualitative character be explained in more illuminating terms? Dualists will say that qualitative character is a non-physical property of perceptual states. This view often has the odd implication that perceptual states have non-physical properties when and only when they are conscious. It is hard to see why that would be so. Why would mental states acquire a radically new kind of property under certain circumstances, and why, in particular, would consciousness have that effect? Materialists think it is better to explain qualitative character in some other way. But what could it be if it is a property that arises only when we are conscious? Having ruled out representational properties on this ground, I am inclined to say that qualitative character is reducible to a physical property.

384   Jesse J. Prinz What physical properties arise when and only when perceptual representations are conscious? A complete answer to that question would depend on an account of the physical correlates of consciousness—a big topic, which I will touch on presently. Defending a theory of consciousness is far beyond the scope of this chapter, but, to illustrate what a theory of qualitative character might look like, I will offer a sketch of a theory I defend elsewhere (Prinz, 2012). I will begin the sketch here, by presenting a story about conscious representations, and then I will continue into the next section, with an account of conscious processes. From a physical perspective, what distinguishes conscious and unconscious representations? What is the difference in their neural realizers? There are many answers to this question in the literature, and the one I endorse is among the most popular: conscious representations are realized by neurons that oscillate in synchrony, perhaps in the gamma range. Let’s suppose, for illustration, that this is true. If so, it is natural to suppose that qualitative character is reducible to the property of synchronized neural activity. More accurately, it is reducible to patterns of synchronized activity that distinguish one neural population from another. When neurons that represent blue things oscillate, they form one pattern, and that pattern differs from the pattern associated with other colours. The blue population can be active without consciousness, but not in a synchronized way. On this view, unconscious perception of blue occurs when a particular population fires out of synchrony, and conscious perception of blue occurs when the same population fires in synchrony. The qualitative character of a perceptual state representing blue is reducible to synchrony in its neural correlates. This is one difference between conscious and unconscious states. I will not attempt to defend this view, but I will conclude by making one observation about it. Neural synchrony has been associated with the view that consciousness involves binding—a view that I have already rejected. I think there is good evidence to reject the claim that synchrony is the mechanism of binding (e.g. Lamme and Spekreijse, 1998), so in adopting a synchrony theory of conscious percepts, I do not mean to imply that consciousness and binding involve the same mechanisms. But if synchrony is not a mechanism for binding, what is it for, and why does it have anything to do with consciousness? I will suggest an answer in the next section. For now we can pause to take in the moral of the foregoing discussion: unconscious perceptual representations are just like conscious perceptual representations in terms of their content, but they differ in character, and that difference may result from the way neurons fire under conscious and unconscious conditions.

Processes I have just suggested the unconscious perception is representationally just like conscious perception, although different in qualitative character. In cognitive science, it is customary to distinguish representations and processes. So, after asking whether unconscious states differ representationally from their conscious analogues, we can now ask whether they differ in terms of processes. An answer to this question was already implied from the discussion in section 1. Perceptual states that are unconscious are not reportable or accessible to deliberation. When we have unconscious perceptions, we do not realize it, and we would emphatically deny that we have perceived anything when asked. This suggests that

Unconscious Perception   385 consciousness centrally involves availability to executive processing. Unconscious mental states are unavailable. This suggestion is extremely popular in consciousness studies, and may qualify as a majority view (e.g. Kirk, 1996; Tye, 1995; Carruthers, 2000; Dehaene and Naccache, 2001; Baars, 2002). Many researchers agree that consciousness involves availability. Some have argued that there can be conscious states that are unavailable (Block, 1995), but I expressed doubts about this possibility in the section entitled ‘Alternative interpretations’. Beyond that widespread consensus, there is disagreement about the psychological and neural processes that make perceptual representations available to executive centres. My own view is that availability is mediated by attention (Prinz, 2010, 2012). This is not the place to survey the evidence for the claim that attention is necessary for availability, and hence consciousness (see Prinz, 2012: ch. 3). But I do want to point out that this would help to account for the forms of unconscious perception that were surveyed above. Inattentional blindness is the most obvious case, since that phenomenon directly shows that attention renders stimuli both invisible and unavailable for reporting. Interocular suppression may also be understood as involving attention. Binocular rivalry studies have shown that competing perceptions can switch with shifts in attention (Mitchell et al., 2004). With interocular suppression that cannot occur, but that may be because the dominant stimulus always steals the spotlight. Attention often involves selection, and lateral competition between stimuli can determine which stimulus will be selected. This is how pop-out works, as when a red circle is presented in a sea of green squares, and pop-out is standardly interpreted as a kind of attention capture. Interocular suppression may be regarded in this way as well. Less obviously, masked priming may involve attention. When a stimulus is followed by a mask, the mask prevents the stimulus from become available. Why? We know that masked stimuli are represented, because they have priming effects. That suggests that the mask is not degrading the stimulus to a point where it can be recognized. Why, then, does masking prevent access to executive centres? One possibility is that the mask shortens the duration of the stimulus in a way that prevents modulation by attention. On this interpretation, masking is a kind of distraction rather than a kind of degradation. If so, masking should not depend on introducing a mask that overlaps with the invisible stimulus; it should be enough to have masks that occur in the area surrounding the stimulus, and hence draw attention away. This is precisely what happens (Enns and DiLollo, 2000). When a brief stimulus is followed by a mask in the surrounding area, it is rendered invisible. That suggests that masked priming is a form of inattentional blindness. Blindsight is the hardest case to interpret along these lines. Kentridge et al. (2008) argue that people with blindsight can attend to items in the blind field. They found that cueing locations in the blind field with an arrow increased stimulus detection. But the individual they studied still claimed that he couldn’t see the stimulus. This might suggest that attention is not the mechanism of availability or consciousness. But there are two other possible interpretations of this result. First, following Overgaard, it may turn out that the blindsight patient’s successful detection depends on very faint conscious awareness, which goes unreported because Kentridge et al. do not test for awareness using a multi-item scale. If so, cueing may increase stimulus detection precisely because attention increases consciousness. Alternatively, it might turn out that cueing simply causes a shift in the direction of gaze. It is well-established that people with blindsight can saccade to cued locations,

386   Jesse J. Prinz especially after training (the individual in the study was very well trained). If cueing caused a shift in eye position, or even an intention to shift, it might have led to an increase in performance when the target was presented, even if that shift in eye position did not occur with a shift in attention. Moreover, there is reason to think that individuals with blindsight cannot attend to stimuli in their blind fields. Damage to the primary visual cortex radically reduces the degree to which stimuli get cortically represented, and subcortical pathways seem to bypass brain structures that allow rich object representation, and engage only those cortical structures that encode spatial information. Thus, a person with blindsight might be able to attentionally modulate representations of spatial locations, not representations of objects in those locations. The inability to attend to object (e.g. to enhance an object representation by means of attention) explains the lack of experience. The link between consciousness, availability, and attention would also help to explain a conjecture that I endorsed in the previous section. There I said that conscious mental states are realized by synchronized neural populations. There is a growing body of empirical evidence linking attention to synchrony (e.g. Fries et al., 2001), and there is also evidence linking synchrony to information flow (e.g. Salinas and Sejnowski, 2001). Synchronized neural populations generate a strong coherent signal that can be propagated along long distances in the brain. If consciousness is availability, attention is a perfect mechanism. Attention creates neural changes that allow perception centres to communicate with executive centres. On this view, conscious states have qualitative character, because character results from the neural changes that implement attention, and attention allows for availability, which is the essence of consciousness. On this view, qualitative character is not an immaterial property, but a physical property that plays a functional role. The fact that conscious and unconscious states differ in their possession and lack of qualia, respectively, coincides with the fact that conscious and unconscious states have different roles in information processing. In summary, unconscious and conscious perceptual states may be alike in terms of content, but they implicate very different processes. Unconscious perceptual states are unavailable for deliberation and report. They are representationally analogous to conscious mental states, but functionally different. Thus functional difference may result from the fact that unconscious perceptual states are not modulated by attention, and, without attention, they can access semantic networks, but not executive centres. Semantic networks are local, nestled within the temporal cortices within which perception occurs. Executive centres are far off in frontal context. Thus, unconscious perception may be distinguished from conscious perception by distance travelled. It is perception without the benefit of a frontal cortex. Such perception is not useless. It might be what reptiles and insects rely on, but it is not available for the higher cognitive functions that undergird human uniqueness.

3 Conclusion In this chapter, I have argued that there are unconscious perceptual states. Indeed, such states are very much like conscious perceptual states in terms of the information they carry, but they present that information without qualitative character, and they don’t let it travel very far. Unconscious perception is no dumber and no smarter than reptilian

Unconscious Perception   387 perception. It is a sensory lizard that resides in each of us, and takes the reigns when we are not paying attention.

References Baars, B. J. (2002). 'The conscious access hypothesis: Origins and recent evidence'. Trends in Cognitive Science, 6, 47–52. Bauer, R. M. (1984). 'Autonomic recognition of names and faces in prosopagnosia: A neuropsychological application of the guilty knowledge test'. Neuropsychologia, 22, 457–469. Block, N. (1995). 'On a confusion about the function of consciousness'. Behavioral and Brain. Sciences, 18, 227–287. Carruthers, P. (2000). Phenomenal Consciousness: A naturalistic theory. Cambridge: Cambridge University Press. Crepaldi, D., Rastle, K., Coltheart, M., and Nickels, L. (2010). ' "Fell" primes "fall", but does "bell" prime "ball"?' Masked priming with irregularly-inflected primes. Journal of Memory and Language, 63, 83–99. Crick, F. and Koch, C. (1990). 'Towards a neurobiological theory of consciousness'. Seminars in Neuroscience, 2, 263–275. de Gelder, B., Morris, J. S., and Dolan, R. J. (2005). 'Unconscious fear influences emotional awareness of faces and voices'. Proceedings of the National Academy of Sciences, U.S.A., 102, 18682–18687. de Gelder, B., Tamietto, M., van Boxtel, G., Goebel, R., Sahraie, A., Van den Stock, J., Stienen, B. M. C., Weiskrantz, L., and Pegna, A. (2008). 'Intact navigation skills after bilateral loss of striate cortex'. Current Biology, 18, R1128–R1129. Dehaene, S. and Naccache, L. (2001). 'Towards a cognitive neuroscience of consciousness: Basic evidence and a workspace framework'. Cognition, 79, 1–37. Dretske, F. (1995). Naturalizing the Mind. Cambridge, MA: MIT Press. Eddy, M. D. and Holcomb, P. J. (2009). 'Electrophysiological evidence for size invariance in masked picture repetition priming'. Brain and Cognition, 71, 397–409. Enns, J. T. and DiLollo, V. (2000). 'What’s new in visual masking?' Trends in Cognitive Sciences, 4, 345–52. Fernandez-Duque, D. and Thornton, I. M. (2000). 'Change detection without awareness: Do explicit reports underestimate the representation of change in the visual system?' Visual Cognition, 7, 323–44. Fries, P., Reynolds, J. H., Rorie, A. E., and Desimone, R. (2001). 'Modulation of oscillatory neuronal synchronization by selective visual attention'. Science, 291, 1560–1563. Jiang, Y., Costello, P., Fang, F., Huang, M., and He, S. (2006). 'A gender- and sexual orientation-dependent spatial attentional effect of invisible images'. Proceedings of The National Academy of Science, 103, 17048–17052. Kentridge, R. W., Nijboer, T. C. W., and Heywood, C. A. (2008). 'Attended but unseen: Visual attention is not sufficient for visual awareness'. Neuropsychologia, 46, 864–869. Kirk, R. (1996). Raw Feeling:  A  philosophical account of the essence of consciousness. New York: Oxoford University Press. Kripke, S. (1980). Naming and Necessity. Cambridge, MA: Harvard University Press. Lamme, V. A. and Spekreijse, H. (1998). 'Neuronal synchrony does not represent texture segregation'. Nature, 362–366.

388   Jesse J. Prinz Loschky, L. C., Sethi, A., Simons, D. J., Pydimari, T., Ochs, D., and Corbeille, J. (2007). 'The importance of information localization in scene gist recognition'. Journal of Experimental Psychology: Human Perception and Performance, 33, 1431–1450. Mack, A. and Rock, I. (1998). Inattentional Blindness. Cambridge, MA: MIT Press. Milner, D. and Goodale, M. (1995). The visual brain in action. Oxford: Oxford University Press. Mitchell, J. F., Stoner, G. R., and Reynolds, J. H. (2004). 'Object-based attention determines dominance in binocular rivalry'. Nature, 429, 410–413. Mitroff, S. R., Simons, D. J., and Levin, D. T. (2004). 'Nothing compares two views: Change blindness can occur despite preserved access to the changed information'. Perception & Psychophysics, 66, 1268–1281. Most, S. B., Scholl, B. J., Clifford, E., and Simons, D. J. (2005). 'What you see is what you set: Sustained inattentional blindness and the capture of awareness'. Psychological Review, 112, 217–242. Naccache, L. and Dehaene, S. (2001). 'Unconscious semantic priming extends to novel unseen stimuli'. Cognition, 80, 215–229. Overgaard, M., Fehl, K., Mouridsen, K., Bergholt, B., and Cleeremans, A. (2008). 'Seeing without seeing? Degraded conscious vision in a blindsight patient'. PLoS One, 3, e3028. Persuh, M. and Ro, T. (2011). 'Context-dependent brightness priming occurs without visual awareness'. Consciousness and Cognition, 21, 177–185. Prinz, J. J. (2000). 'A neurofunctional theory of visual consciousness'. Consciousness and Cognition, 9, 243–259. Prinz, J. J. (2005). 'A neurofunctional theory of consciousness'. In Andy Brook and Kathleen Akins (eds), Cognition and the Brain: Philosophy and neuroscience movement (pp. 381–396). Cambridge: Cambridge University Press. Prinz, J. J. (2010). 'When is perception conscious?' In B. Nanay (ed.), Perceiving the World: New essays on perception. New York: Oxford University Press. Prinz, J. J. (2012). The Conscious Brain: How attention engenders experience. New York: Oxford University Press. Rensink, R. A., O’Regan, J. K., and Clark, J. J. (1997). 'To see or not to see: The need for attention to perceive changes in scenes'. Psychological Science, 8, 368–373. Rosenthal, D. M. (1991). 'The independence of consciousness and sensory quality'. Philosophical Issues, 1, 15–36. Salinas, E. and Sejnowski, T. J. (2001). 'Correlated neuronal activity and the flow of neural information'. Nature Reviews Neuroscience, 2, 539–550. Simons, D. J., Franconeri, S. L., and Reimer, R. L. (2000). 'Change blindness in the absence of a visual disruption'. Perception, 29, 1143–1154. Sperling, G. (1960). 'The information available in brief visual presentations'. Psychological Monographs, 74, 1–29. Szczepanowski, R. and Pessoa, L. (2007). 'Fear perception: Can objective and subjective awareness measures be dissociated?' Journal of Vision, 7(10), 1–17. Tarr, M. J. and Pinker, S. (1991). 'Orientation-dependent mechanisms in shape recognition: Further issues'. Psychological Science, 2, 207–209. Treisman, A. and Gelade, G. (1980). 'A feature integration theory of attention'. Cognitive Psychology, 12, 97–136. Tye, M. (1995). Ten Problems of Consciousness. Cambridge, MA: MIT Press. Van den Bussche, E., Notebaert, K., and Reynvoet, B. (2009). 'Masked primes can be genuinely semantically processed: A picture prime study'. Experimental Psychology, 56, 295–300.

Unconscious Perception   389 Weiskrantz, L. (1986). Blindsight: A case study and implications. Oxford: Oxford University Press. Winkielman, P., Berridge, K., and Wilbarger, J. (2005). ‘Unconscious affective reactions to masked happy versus angry faces influence consumption behavior and judgments of value.’ Personality and Social Psychology Bulletin, 1, 121–35. Wolfe, J. M. (1999). ‘Inattentional amnesia’. In V. Coltheart (ed.), Fleeting Memories (pp. 71–94). Cambridge, MA: MIT Press.

Pa rt  I V

W H AT W E PE RC E I V E

Chapter 20

Obj ect Perception Roberto Casati

The topic of object perception is different from that of the objects of perception. The latter deals with conceptual issues related to whether we perceive either extramental or mental items (sense-data); if we perceive extramental items, then whether we perceive them directly or indirectly, through perception of mental items; and the nature and variety of perceptual items. The topic of object perception is narrower:  in general, it presupposes that we are dealing with extramental items only, be they full three-dimensional objects or two-dimensional stimuli. Extramental items can belong to various categories—they can for instance be events, or properties, or property bundles, or spatial locations, or boundaries. The topic of object perception covers only a fraction of this fauna. Object perception is perception of stable three-dimensional entities, a category of items that as a first approximation includes ‘middle-sized dry goods’ (i.e. things like tables and pencils), but also rocks, trees, and animals. Such objects have well-defined, slowly and non-discontinuously changing boundaries; they continue to exist through time; they change and move (non-discontinuously). Thus, we can set aside controversies about the exclusiveness of ontological status or primacy of objects of perception (‘are all objects of perception events/properties/objects?’, ‘do we perceive events by perceiving objects or the other way around?’) and take for granted that there is an interesting type of objects of perception, namely concrete objects, and that we actually perceive these. The topic of object perception is then an account of what it is to perceive objects of this type.1 In the face of these preliminary clarifications, there is still ample room for conceptual work and many distinctions have to be drawn. By saying that we perceive tables and chairs, we say the obvious, but as usual the obvious can conceal interesting complexities. For instance, does object perception encompass perception of spatio-temporal but immaterial items such as shadows and holes? What are the relevant contrasts here? We shall first present an important problem with the visual resolution of what counts as an object. This will introduce the contrast classes against which object perception is to be understood and why certain items are privileged in perception as opposed to others. We

1  It

is thus also to be kept distinct from the vast topic of object recognition, although some basic mechanisms of object perception have been shown to be tributaries of mechanisms for recognition (Peterson and Gibson 1991).

394   Roberto Casati shall then review how the privilege accorded to certain type of objects is acknowledged in different research paradigms, each providing different and overlapping operationalizations of the theoretical notion of object. We shall finally see a likely mechanism that explains how the advantage is conferred precisely to objects. Consider a dramatic example from perceptual psychology. In the 1920s and 1930s a number of researchers in the Gestalt tradition uncovered rules of perceptual organization that control the parsing of the visual scene, including phenomena such as figure-ground articulation or perceptual masking and camouflage. The calculus of figure-ground relationships is an important mechanism of object perception. When an object F partly occludes another, G, (relative to a particular viewer) the outer boundary of F coincides, in the visual image, with the containing boundary of G. Vision allocates this shared boundary to the occluding object F: that is, it visually appears that this boundary belongs to F, not G. Figure-ground parsing consists in creating macro entities out of more elemental items, and in some cases to assigning the boundary to the occluder (the figure) in the interpretation of the scene (Hoffman 1998) according to certain preference rules. Among these is the rule of similarity. Consider the configuration in Figure 20.1:

Fig.  20.1  Gestalt law of similiarity dictates organization in  rows. The configuration appears organized in horizontal rows rather than columns because of a rule of similarity, stating that items of similar aspect (here sharing the feature of colour) tend to associate with each other, all else being equal, to create macro-structures. Black dots associate with the nearest black dots to form rows, and so do white dots. On top of the thirty-six individual dots of the picture, we also perceive six rows. Gestalt grouping explains part of what it is to perceive objects. Observe that we can conceptually single out many other macro-entities:  one constituted by the eighteen leftmost dots, one constituted by all dots minus the central four ones, and so on. That the six rows are somewhat forced upon us is not a trivial fact. Gestalt rules are thus relevant for object perception insofar as they both describe in great detail the perceptual performance and are an obvious bridge for any explanatory theory of perception. They provide the general architecture of the visual computations that deliver the six-row percept in the case of Figure 20.1. This unrivalled explanatory breadth accounts for the fact that Gestalt rules went unquestioned for about sixty years—indeed, most textbooks of perception listed the rules as rich and interesting consensual principles. However, Palmer and Rock (1994) asked the crucial question: Gestalt rules create macro structures out of items such as the blobs in Figure 20.1; but how come the individual blobs themselves are forced upon us, i.e. how are they computed as individuals by the visual system? That this question may appear at first blush surprising is an indication of the subtleties

Object Perception   395 of the challenges it sets. ‘Sure enough,’ one is tempted to answer, ‘no matter what the macro structures, there are 36 individuals, blobs, objects, in Figure 20.1. Each individual blob counts as one object.’ To better frame the issue, consider two such blobs (Figure 20.2).

Fig.  20.2 Two objects? Why are we inclined to say that there two objects here? The question makes sense when contrast classes are pointed out. Why two as opposed to one, first of all? We can consider the total black area in Figure 20.2. as a single object; but in general we don’t. Consider Figure 20.3.

Fig.  20.3  One object? Here the dual question is possible: why one as opposed to two? We may want to construe the left and the right halves of the blob in Figure 20.3 as individual objects, and indeed, we can conceptually single out those items. Why not? Consider further Figure 20.4, in which dashed lines are not actually part of the figure, they are just used for singling out a portion of it.

Fig.  20.4  An object? Why are we inclined not to count the two-colour spatial content of the dashed line as an object? The fact that we favour the two-object solution in Figure 20.2, the one-object solution in Figure 20.3, and the black blob solution in Figure 20.4, over various possible alternatives, indicates that the visual system is indeed computing individuality in its own, proprietary way. Palmer and Rock proposed that entry level objects are mainly computed according to two preference rules. First, they must be uniform in quality. This means, in our case, that each point has the same colour as any other point in the relevant area.2 This rules out any candidate that would straddle the boundaries of the blob in Figure 20.3. Second, they must occupy a connected area of the visual field. This means an area such that it is possible to link any two points within it through a continuous path that does not exit the area itself. This rules out the total black area in Figure 20.2 as a candidate for single objecthood: at least one path connecting a point on the left black portion to a point of the right black portion must pass through the non-black area. Applied together, the two conditions are meant to imply that precisely the blobs we intuitively consider as individuals in Figures 20.1–20.3 are indeed singled out as individuals. 2 

Uniformity need not be construed simply on the model of uniform colour. Gabor patches correspond to sinusoidal luminace patterns, hence their colour varies constantly, and are still perceived as unitary units. What is uniform in their case is the variation in colour.

396   Roberto Casati Here comes an interesting twist. As a matter of fact, the two conditions cannot succeed in singling out precisely the blobs unless a third proviso is added, namely that the area of uniform quality be maximal. This means that the area of uniform quality must not be a proper part of a connected area that is itself of uniform quality. The left half of the black blob in Figure 20.3 is connected and is uniformly black; but it does not constitute a perceptually legal individual. The fact that maximality was overlooked in the original Palmer and Rock account bears witness to the inherent difficulties posed by the notion of an object. Arguably, it is a very basic and fundamental notion, and it is easy to overlook what is basic. Observe that uniform and maximal connection delivers bottom-up responses. There is no top-down constraining of the visual system by higher cognitive abilities, such as conceptual organization. Indeed, such a top-down constraining of the solution would put the one-object solution on the same foot as any other solution. For instance, in Figure 20.2 we can invoke the concept of ‘black total’ to yield the ‘two-object’ solution. From this introductory example we can extract some elements. The visual system tends to favour some areas of the perceived scene over others, and labels the favoured areas as individuals or objects (the thirty-six dots in Figure 20.1.) These entry objects are such that they can enter in molecular combinations (the six rows in Figure 20.1.) They also are, in a relative sense, atomic: they may well have parts, but these parts are not entry objects themselves. We can provisionally state that entry objects are the atoms of perceptual content, where atomicity is operationalized as the result of the application of rules of maximal uniform connectedness in the visual field (in particular, for the time being, in stimuli that are flat images such as the ones of Figures 20.1–20.3). The main thrust of the example is that a discussion of object perception is in general tied to a discussion about the units of perception, i.e. about perceptual individuals, entities that can be counted as one. Object perception reflects the choices of cognition among a variety of options. Philosophical discussion has pointed out that alternative possibilities are plentiful. One’s realism about physical reality encourages one to claim that the content of any portion of space or of spacetime exists and must be counted as an individual. For instance, the item constituted by my undetached left hand and the content of a region that straddles the northern hemisphere of the Sun and part of the space around it exists.3 That we find such an individual a freak is just a symptom of the fact that our cognitive system is standardly unable to countenance it, as it is not a legitimate solution to the problem of parsing the perceptual scene. The fact, on the other hand, that we can create an ad hoc concept for singling out precisely the inhomogeneous content of that oddly shaped region of space (‘the total black area of Figure 20.1’) indicates that we have the conceptual resources for accepting any such content as legitimate in the conceptual sense. Thus it is all the more remarkable that most of these sparse items are ruled out in perception, and it is surely something that deserves an explanation. Why are precisely these choices made, as opposed to any other? What are the reasons for privileging the maximally uniform, connected items? If perceptual systems have evolved in such a way as to be able to track environmental regularities (for instance, the fact that most items are rigid and do not constantly undergo 3 

Palmer reports the introspective account of a subject with brain damage who claims ‘I do remember one case where there was what seemed to me to be one object which was partly motor car, partly tree and partly a man in a cricket shirt. They seemed somehow to belong together’ (1999: 255).

Object Perception   397 quick shape changes), then some putative advantages of maximal uniform connectedness can be put forward. Uniformity is privileged because of the likelihood that there is a common cause to a common visual manifestation. Most green in the visual field, to take an example, can be traced back to selective absorption of short and long wavelengths by magnesium-based processes in the vegetal realm. Connectedness is privileged because of the likelihood that spatial proximity is a cue to causal connectedness: the responsibility for something being what it is now at a given place is likely to be grounded on what went on at the same place at neighbouring times, or at neighbouring places, more than on what went and goes on at more distant or disconnected places (Bennett 1988). Maximality is privileged because all non-maximal solutions are tributary to the maximal solution. On the one hand, by acting on a part, we act on the whole of which it is a part. On the other hand, there are far too many parts to pay attention to if we singled them out non-maximally. The maximal solution is computationally stable. Our perceptual phenomenology is as of stable physical objects in a relatively stable environment. But the discussion so far has made it plain that we cannot take any list of candidate items for granted. To further this point, let us first present some contrast classes that will help us focus on the specificity of objects, and, second, present some alternative operationalizations of the object notion. (An operationalization tries to specify theoretical conditions for empirically measuring some inaccessible or hypothetical quantities. Operationalizing visual objects in terms of maximal uniform connection amounts to considering as an object whatever is such as to satisfy the principles mentioned above in experiments on visual perception.) When Quine (1974) announced the object privilege by stating that ‘humans are instinctively body-minded’, he pitted bodies, i.e. material objects, against a contrast class, ‘a more inclusive set of things that consists of any ‘arbitrary congeries of particle-stages, however spatio-temporally gerrymandered or dispersed’’ (Goldman 1992: 37, quoting Quine 1974: 54). Quine’s arbitrary congeries are ontologically of a kind with material objects; like objects, they are material occupants of spatial or spacetime regions; they just lack ‘wholeness’, being of apiece. In a sense, they are just material objects with very strange spatial and spatiotemporal ‘shapes’. An even larger ontological dispersion has been suggested in the literature. Among the contrast items we find localized properties (Strawson 1959), regions of space (relevant to attention research: Clark 2004; but see Cohen 2004; Matthen 2004), parts of objects (Hoffman and Richards 1984), collections of objects (Bloom 1996), causally unitary spatiotemporal stages that are shorter lived than standard objects (Hirsch 1993), causally non-unitary spatiotemporal stages that are spatiotemporally contiguous, events (Davidson 2001), property bundles, two-dimensional stimuli, and, because of their immateriality, shadows and holes. All these entities are spatial, material or quasimaterial, and localized (and thus differ from entities of more remote contrast classes, like dreams or numbers). A difficulty in analysis is due to the unstable status of these entities. For instance, Quine defended a monocategorial ontology that makes no place for the distinction between objects and events—events being construed as short-lived objects. His proposal may not be as revisionary as it first appears. We can accept short-lived objects, like the water sculptures by artist Shinichi Maruyama that only a slow motion camera can capture. (We may claim, in philosophical finesse, that we non-epistemically see them,4 but 4 

For a discussion of kinds of seeing, see Brogaard, Chapter 13, this volume.

398   Roberto Casati surely we cannot observe them at leisure with the naked eye the way we do with more stable objects.) The point is not that cognition is totally blind to any of the items belonging to the contrast classes. As Schwartz (2001) has pointed out, obviously we can attend to collections of objects, parts, the space between objects, etc. Atomic objects can have parts, but these are not objects themselves (in this particular operationalization of the notion of an object), and yet, as we can focus our attention onto them, we need to postulate subatomic object representations. The point is rather that there are some cognitive advantages to objects in many perceptual and cognitive tasks. Other cognitive performances may assign advantages to different items. An example is static peripheral vision, whose ontology appears to include (roughly) localized properties or loosely tied bundles of properties rather than whole objects. When we focus our attention on the peripheral content of our visual scene, we only see patches of indefinite colours in certain directions rather than full-blown whole objects. The advantages assigned to objects surface in various operationalizations of objects, of which the one discussed here, based on the visual properties of (maximal) uniform connectedness is but one. In discussing these operationalizations we need to ascertain whether they single out the same external entities, and whether these are the same that are intuitively contemplated by common sense. One prominent operationalization (Pylyshyn 2007; see also Scholl 2002) is provided by the theory of visual indexes. Visual attention can be simultaneously distributed over a limited number (four or five) of distinct items at diverse locations. Within certain limits, the items can be successfully followed through vision in their random motions. This performance is explained by hypothesizing a mechanism of visual tracking that links each of the attended objects to an ‘index’. The limitation in the number of indexes available explains the possibility of simultaneously attending to a limited number of objects only. Besides, indexes can be assigned to and can operate only on certain configurations, as is evident from the study of other visual performances. For instance, smooth change in colour during the movement of the targets does not affect the possibility of tracking (the change is not even noticed), nor does brief disappearence behind occluders, whereas dissolution in a cloudlike configuration followed by recollection negatively impacts tracking. It looks as if spatiotemporal features—continuity of movement, being all of a piece—are essential features of trackable items, whereas categorial features—e.g. colour—are less important. Objects are here operationalized as those items which can be successfully tracked in multiple object tracking tasks. Their constitutive properties are those that allow the hooking up of a visual index. Objects in this sense (called also ‘proto-objects’ or ‘visual objects’) constitute a relatively large category that includes 2d configurations. Relative to commonsense objects such as tables and chairs, the hypothesis may be non-specific. Moreover, trackable objects need not be uniform-connected objects. One can track multicoloured targets, such as blobs whose left half is red and whose right half is green (uniformity is violated), and even targets that are composed of parts that are visually disconnected, provided the parts move in unison (connectedness is violated). Higher order Gestalt preference rules control the parsing in such a way as to determine which items are tracked. Another important operationalizaton is related to the one just discussed. So-called ‘object-files’ (Kahneman and Treisman 1992) are mechanisms that have been hypothesized to explain reidentification, and in particular perceptual reidentification, of objects

Object Perception   399 through change of some of their locational and non-locational features. In experiments on visual attention, whole objects turn out to be advantaged over regions of space as the target of attention. The hypothesis is that a perceptually individuated and attended item gets assigned an object file which is then used by the visual (or other) system to store information about the item. Files are activated by items that present the features of wholeness, continuous motion, etc., that are usually associated with material objects. Object files are volatile, their survival is limited and subject to various spatiotemporal constraints. For instance, the reidentification of an object appearing from behind a screen as the same that disappeared behind the screen depends on the length of the time spent behind the screen, on the spatial and temporal relatability of the entry and exit trajectories, and so on. If the trajectories are not relatable, then a new object file is created for the appearing object. Interestingly, if the perceptual system is under pressure, for instance if the movements of the objects in a scene are very fast, then conflicts of reidentification have in the norm a spatiotemporal solution, which dominates categorial reidentification. A magician can make you see a piece of cloth ‘turn’ into a rabbit, if the speed of her movements is brisk enough. Object files can further merge or split according to the perceived or inferred trajectories of objects. Yet another operationalization derives from studies on cognitive development (Spelke 1990). A wealth of results have been obtained by studying infants’ responses to stimuli that mimic or violate some features of ordinary objects. The experimental paradigm uses surprise as the dependent variable. It is assumed that infants’ attention and interest decay upon repeated presentation of a given stimulus. If the presentation of a different stimulus arouses surprise (psychophysically measurable in terms of faster pulse, faster suction, for example) then it can be inferred, in controlled conditions, that the relevant difference was noticed by infants. Countless experiments converge on some interesting facts. Bounded, connected, impenetrable wholes that move only upon contact and move on continuous paths are significantly more advantaged than various contrast items that do not have one or more of these properties or behaviours. For instance, toddlers are surprised when they discover that two spatially disconnected screens conceal only one object—say, placed behind the left screen—when they previously were shown two indistinguishable objects, one behind each screen, taking turns in appearing and disappearing from behind each screen. The visual data are compatible with a theory of ‘metaphysical jumps’, according to which an object can move from one location to another, disconnected location, without passing through all the intermediate locations; but such a theory is not supported by young children’s preferences. The features of objects as operationalized in cognitive development define an interesting class that does not include object parts such as hands or tree branches (undetached items) or, for that matter, animals (self-propelled, can move without contact). Paradigmatically, they are exemplified by pebbles, logs, or small, compact artefacts. Other operationalizations refer to items that get counted by infants (Wynn 1992) or preferentially intended as referents of newly introduced names in children’s language (Bloom 2000). Now, the fact that spatiotemporal constraints dominate categorial constraints in all object operationalizations (developmental, movable object tracking, object files) in specific conditions (fast movement, attentional pressure) has elicited the hypothesis (Carey and Xu 2001) that two independent object reidentification mechanisms are present in adults, one of which, the spatiotemporal mechanism, already there at birth, is used

400   Roberto Casati in early infancy and later on in cases in which time constraints block the deployment of the later, categorial mechanism, based on object recognition. Indeed, it makes evolutionary sense that we are equipped with a coarse object-perception mechanism that provides quick solutions to tracking and reidentification problems; and in the norm the solutions provided by such a spatiotemporal mechanism are robust, given that in the environment most objects tend to move smoothly (for instance they do not stop or change pace when they are behind visual obtruders). Mechanisms for object perception centre upon some essential features of scenes including objects and on some requirements for treating the percept as including full, threedimensional items as opposed to flat, backless decoys. The main scene feature in a world populated with three-dimensional opaque objects is the presence of occlusion boundaries. Some objects or parts of objects are closer to the viewer than others, and some of the closer items block perception of parts of the faraway items. A general mechanism for figureground articulation is the main actor here. It allows disambiguation of a visual boundary, in a first analysis shared between the occluder and the occluded, by attributing the boundary to the occluder. As a visual portion of the occluder is now unbounded, its visually available boundaries get ‘completed’ behind the occluder. Thus, in Figure 20.5(a), the a–b portion of the boundary separating O1 from O2 is attributed to O1, and as now O2 is unbounded in the a–b portion (if it was bounded then its shape would be as in Figure 20.5(b)) its boundary gets completed behind O1, and O2’s shape is likely to be perceived as the one in 20.5(c). (a)

(b) a

(c)

O2 b

O2 a

b

O1

O2

a b

Fig.  20.5  Disambiguation of a visual boundary. Two questions arise here. First, what decides between the competing hypotheses 20.5(b) and 20.5(c), and second, why is the mentally reconstructed shape of O2 as in 20.5(c), as opposed to any other shape for the hidden portion? Both questions are in general dealt with in a Bayesian framework (Kersten, Mamassian, and Yuille 2004), that weights the interpretation of an image against some hypotheses about environmental statistics. In a Bayesian framework, the posterior probability P(s|i) that given a certain image (i), it corresponds to a scene (s), does not only depend on the likelihood P(i|s) that a given scene will produce the image, but also on the simple prior probability of the occurrence of the scene factored by a normalizing constant of the occurrence of the image:

P(s|i) = P(i|s)P(s) P (i) For instance, consider a line in the visual field. It may correspond to a large variety of three-dimensional structures. One of these is, say, a flat disc seen in cross-section. Now the posterior probability that the image of a line be an indication of the presence of a flat disc is

Object Perception   401 almost zero, given that the probability that a disc be seen in profile is almost zero—you are very likely to view any such disc at an angle. On the other hand, the probability that it indicates the presence of a crease or a fold separating two faces of a dihedral depends on the occurrence of those structures in the environment—and again, if there are few dihedrals around, the posterior probability shrinks, whereas if there are only dihedrals, it will move up. Finally, no matter how robust the likelihood is, and no matter how frequent the 3d structures in the scene are, if your visual system is only able to produce line-like percepts, i.e. if lines pollute the image, then encountering a line is not a good indication of anything anyway. (If, on the other hand, there are only dihedrals in the environment, and if they do massively generate lines in the image, then having only lines in the image is not particularly detrimental). A working hypothesis in perception theory is that the perceptual systems have learned the prior probabilities, or priors (Ps), in their phylogenetic history, given the stability of the environmental features: for example, that light comes from above, and that physical objects are in the norm rigid. They have also learned the likelihoods P(i|s), that are in general facts about the relationships of the system to the object: flat discs are seldom seen in cross-section, occlusions generate distinctive patterns relative to the viewer, approaching objects cover ever widening angles. As to the first question above (what decides between the competing hypotheses 20.5(b) and 20.5(c)), the prior probability of dovetail of the boundaries of two independent objects in the visual field is extremely low, whereby the sickle shape in Figure 20.5(b) for O2 is excluded. Some portion of the boundary of O2 is hidden, but—and this was the second question—what precisely is the shape in question? The shape of the completion is in the norm determined by relatability (Kellman and Shipley 1991) of hidden boundaries; if relatability is missing, then other default hypotheses may be confirmed (e.g. two objects are occluded instead of one). In these cases, it looks as if the visual system uses proprietary computations for determining solutions to the two questions of boundary attribution and boundary completion, computations that are impervious even to acquaintance to the actual hidden shape that in point of fact may be very different. Bayesian approaches account for a variety of phenomena in different sensory realms; not only for occlusion, but also for grouping of macro units out of atomic units. Bayesian considerations help explain the perceptual advantage of objects over the background visually bordering them, or over empty portions of space seen between objects. The shapes of the background and of regions of empty space are highly accidental, as they depend upon both the shapes of the objects and the viewpoint of the observer. Considering the observer’s viewpoint as generic disadvantages the shape of empty spaces between objects (Rock 1983; Palmer 1999). Grouping by similarity and common fate is also accounted for in terms of non-accidentalness. It would be an unlikely event if two unrelated objects moved in a coordinated way, whereby the visual system construes them as two parts of the same object. It is possible that storage of priors be not only phylogenetical but also partly ontogenetic. Kellman and Spelke’s (1983) study indicates that whereas 3–4-month-old children are sensitive to the non-coincidence of movement of the two extremes of a centrally occluded bar, they are not equally sensitive to the simple continuation of the two extremes’ profiles in static situation (whereas older children and adults are). It is as if static non-coincidence needed some form of learning.

402   Roberto Casati Shadow perception is subject to Bayesian inferences as well. Shadows can be completed behind other shadows and even behind objects. But surely these are illusions; real shadows do not exhibit these three-dimensional patterns. Arguably, these illusions are by-products of preference rules that are tuned to concrete objects, but which get applied to any visual scene in which boundaries are present. More generally, the fact that shadows (Sorensen 2008) and also holes (Casati and Varzi 1994; Giralt and Bloom 2000), which are localized, non-concrete spatiotemporal entities, can be perceived and tracked in more or less the same way that concrete objects are perceived and tracked suggests that their images are sufficiently close to those of objects as to elicit Bayesian mechanisms. Not only do we complete object parts that are occluded by other objects, we also are aware of objects’ back parts. We should certainly draw a distinction between perceiving and being aware of—a weak claim being that one is ‘just aware’ of the object’s back, which is not to say that one infers or accesses its actual shape.5 In ecological situations, movement provides disambiguation. The importance of the movements of the perceiver has been stressed by researchers in the ecological tradition, and much has been made of it by defenders of sensorimotor accounts of perception (O’Regan and Noë 2001). Consider again Figure 20.5. If O2, the occluded object, moves laterally relative to O1, whereas the observer is still, then more and more parts of its surface become visible or get occluded, whereby the visually shared boundary is interpreted as having physical significance for the occluder only. Likewise, the hidden part of an object can be revealed if we move around. An extreme line of thought here is that the subject’s movement are both necessary and sufficient for occlusion disambiguation and the feeling of presence of hidden parts. The perceptual complexity of objects would then be explained by the complexity of exploratory movements. However, this position is unnecessarily strong. First, it does not matter whether the mover is the observer or whether it is one of the two objects, the observer remaining still (subject’s movement is not necessary). Gombrich’s objection to J. J. Gibson is still valid: distant objects do not allow for disambiguating movements, but they are not perceived as devoid of a back side. Objects (foliage in the wind, running animals) do move relative to each other and to a still observer, and do spin. Second, an explanation for the performance in the static situation of Figure 20.5 is still necessary (movement not sufficient). As preference for whole objects is evident in perception and action from early infancy, it has been alleged to ground also conceptual and linguistic preferences (Bloom 2000). In the absence of clear contextual indications to the contrary, new names are by default assigned an object semantics rather than being taken to refer to any other type of entity. Is the concrete object advantage shared by sense modalities other than vision? The topic of object perception intersects the larger one of perceptual objects, which inherits from a historical discussion about the distinction between the senses. A classical criterion for distinguishing the senses relies on what was termed ‘proper sensibles’, i.e. items that are specific to, and thus identify, a given sensory modality: sounds for audition, colour for vision, smells for smell, and so on. 5  (One should also keep in mind the difference between first-person perceiving (or being aware of) and third-person concluding (as the result of an experiment) that the visual system calculates relatability, by excluding alternative hypotheses (‘you do not perceive the sickle in Figure 20.5(b); you do not perceive two independent objects if they are relatable’, which does not amount to say that you ‘see’ the hidden part or even that you ‘infer’ how it is configured.)

Object Perception   403 (See Matthen, Chapter 30, this volume.) The criterion’s ability to discriminate has been questioned on various grounds (for instance, it may allow for more senses than are usually acknowledged: vision-for-shape, vision-for-colour). Now, it is a fact that different items are highlighted in different sensory modalities, but does this amount to rejecting a concrete object privilege? Defenders of objects’ centrality have a harder life in some cases, like smell, which appears to rather target properties and places. Intermediate cases are interesting. (See Smith, Chapter 17, this volume.) Does auditory perception single out and track bells, tables and chairs, or possibly other entities, such as events? (See Nudds, Chapter 15, this volume.) Two questions must be kept apart here. The first is that of finding in audition the analogue of objects as described in vision science. The second is to ascertain whether concrete objects are auditorily perceived. As an answer to the first question, Kubovy and Van Valkenburg (2001) suggested that the senses target certain complexes of qualities, whereby a perceptual object may be a supramodal structure, which is instantiated in spatial complexes of shape and colour in vision and in temporal complexes of pitch and timber in audition. As an answer to the second question, object-centred Bayesian inferences have been alleged to account for some preference rules in the auditory realm, which turn out to centre once more on whole objects. For instance, the advantages given to notes that are in harmonic relationships reflects the prior probability that if thwacked, an object would resonate harmonically (i.e. release overtones of a principal tone). It is a priori implausible that two tones separated by a fifth come from two different, independently resonating, independently thwacked objects (Nudds 2010). François Jacob (1974) argued that object perception as we find in humans is phylogenetically recent. Other living beings need not represent objects and may just display sensitivity to properties—such as movement—rather than to whole objects. It is an open issue whether visual object perception is an autonomous province or whether it will be found to be an instance of a more general type of capability. The answer will crucially depend on multimodal explorations, and on the type of answer we are prepared to give to the question of the analogon of objects for other sense modalities.

References Bennett, J. (1988). Events and their Names. Oxford: Oxford University Press. Bloom, P. (1996). ‘Possible individuals in language and cognition’. Current Directions in Psychological Science, 5(3), 90–93. Bloom, P. (2000). How Children Learn the Meanings of Words. Cambridge, MA: MIT Press. Carey, S. and Xu, F. (2001). ‘Infants’ knowledge of objects: Beyond object files and object tracking’. Cognition, 80, 179–213. Casati, R. and Varzi, A. C. (1994). Holes and Other Superficialities. Cambridge, MA: MIT Press. 'Clark, A. (2004). Feature placing and proto-objects. Philosophical Psychology, 17(4), 443–469. Cohen, J. (2004). ‘Objects, places, and perception’. Philosophical Psychology, 17(4), 471–495. Davidson, D. (2001). Essays on Actions and Events (2nd edn). Oxford: Oxford University Press. Giralt, N. and Bloom, P. (2000). ‘How special are objects? Children’s reasoning about objects, parts, and holes’. Psychological Science, 11, 503–507. Goldman, A. (1992). Liaisons. Cambridge, MA: MIT Press. Hirsch, E. (1993). Dividing Reality. Oxford: Oxford University Press.

404   Roberto Casati Hoffman, D. (1998). Visual Intelligence: How we create what we see. New York: Norton. Hoffman, D. and Richards, W. (1984). ‘Parts of recognition’. Cognition, 18, 65–96. Jacob, F. (1974). La Logique du vivant. Eng. trans. The Logic of Life. Princeton:  Princeton University Press, 1993. Kahneman, D. and Treisman, A. (1992). ‘The reviewing of object files: Object-specific integration of information’. Cognitive Psychology, 24, 175–219. Kellman, P. J. and Shipley, T. F. (1991). ‘A theory of visual interpolation in object perception’. Cognitive Psychology, 23, 141–221. Kellman, P. J. and Spelke, E. (1983). ‘Perception of partly occluded objects in infancy’. Cognitive Psychology, 15, 483–524. Kersten, D., Mamassian, P., and Yuille, A. (2004). ‘Object perception as Bayesian inference’. Annual Review of Psychology, 55, 271–304. Kubovy, M. and Van Valkenburg, D. (2001). ‘Auditory and visual objects’. Cognition, 80(1–2), 97–126. Maruyama, S. Water Sculpture. . Matthen, M. P. (2004). ‘Features, objects, and places: Reflections on Austen Clark’s theory of sentience’. Philosophical Psychology, 17, 497–518. Nudds, M. (2010). ‘What are auditory objects?’ Review of Philosophy and Psychology, 1(1), 105–122. O’Regan, J. K. and Noë, A. (2001). ‘A sensorimotor account of vision and visual consciousness’. Behavioral and Brain Sciences, 24(5), 939–1031. Palmer, S. E. (1999). Vision Science. Cambridge, MA: MIT Press. Palmer, S. E. and Rock, I. (1994). ‘Rethinking perceptual organization: The role of uniform connectedness’. Psychonomic Bulletin and Review, 1(1), 29–55. Peterson, M. A. and Gibson, B. S. (1991). ‘The initial identification of figure–ground relationships: Contributions from shape recognition processes’. Bulletin of the Psychonomic Society, 29(3), 199–202. Pylyshyn, Z. W. (2007). Things and Places: How the mind connects with the world. Cambridge, MA: MIT Press. Quine, W. v. O. (1974). The Roots of Reference. La Salle, PA: Open Court. Rock, I. (1983). The Logic of Perception. Cambridge, MA: MIT Press. Scholl, B. (ed.) (2002). Objects and Attention. Cambridge, MA: MIT Press. Schwartz, R. (2001). The concept of an ‘object’ in perception and cognition. In Shipley and Kellman, 2001: 3–17. Shipley, T. F. and Kellman, J. (eds) (2001). From Fragments to Objects: Segmentation and grouping in vision. New York: Elsevier. Sorensen, R. (2008). Seeing Dark Things. Oxford: Oxford University Press. Spelke, E. S. (1990). ‘Principles of object perception’. Cognitive Science, 14, 29–56. Strawson, P. F. (1959). Individuals: An essay in descriptive metaphysics. London: Methuen. Wynn, K. (1992). ‘Addition and subtraction by human infants’. Nature, 358, 749–75.

Chapter 21

Pr im a ry a n d Secon da ry Qua lities Peter Ross

1 Introduction A well-known feature of early modern philosophy and science is the division of perceptible qualities into the so-called primary and secondary qualities. Galileo, Descartes, and Hobbes presented early versions of this distinction; Boyle and Locke were prominent among those who developed later versions. Later distinctions continue to structure discussion of perceptible qualities. Any proposed distinction between primary and secondary qualities is drawn on a theoretical basis. Founding theories are from science (for example, seventeenth-century mechanical philosophy), epistemology, and theories of mental representation. Because these theoretical commitments vary, there is not just one primary-secondary quality distinction, but a constellation of distinctions. In particular, because scientific theories have changed dramatically since the early modern period, any current distinction is significantly different from early modern ones. Nevertheless, lists of primary and secondary qualities have remained fairly stable. Since the seventeenth century, the primary qualities typically include extension, size, shape, and motion and rest, and the secondary qualities include colour, smell, taste, sound, and warmth and cold. Despite dramatic changes in scientific theory as well as epistemology, current proposals about the metaphysics of colour, for example, are sometimes labelled as descending from a particular early modern theorist’s view of colour as a secondary quality. This labelling in current philosophy expresses an attitude that, whatever a historical figure’s theoretical commitments were, that individual had located an important difference between shape and colour, where, broadly speaking, the difference is between qualities that are metaphysically perceiver independent and qualities that are metaphysically perceiver dependent. Indeed, understood in this broad way, the tenability of a distinction between primary and secondary qualities continues to be of importance, at least implicitly, in philosophical discussion of proposals about the metaphysics of colour, where an important issue is whether colour is perceiver dependent in a way that shape is not.

406   Peter Ross (This metaphysical distinction is typically combined with epistemic distinctions as well. However, epistemic differences between shape and colour by themselves aren’t sufficient to establish the primary-secondary quality distinction. For example, assuming, as seems true, that colour is perceptible by only one sensory modality whereas shape is perceptible by more than one modality, this would not be sufficient to establish the distinction.) Secondary qualities are sometimes used as a model for understanding qualities in areas outside of philosophy of perception. Thus, it is sometimes claimed that secondary qualities can provide a model for understanding moral qualities. (For this sort of proposal, see McDowell, 1985; for helpful discussions of the analogy between secondary qualities and moral qualities, see also McNaughton, 1988, 83–97, and McGinn, 1983, 145–155.) The general motivation for this analogy is that secondary qualities offer a model of subject-dependency which can serve to inform our understanding of moral qualities. To get this analogy off the ground, though, one must suppose that the primary-secondary quality distinction is on the right track. However, this is risky, since the tenability of the primary-secondary quality distinction is controversial. In what follows, I’ll (1)  give background on early modern proposals of the primary-secondary quality distinction, with attention devoted to the mechanical philosophers’ understanding of primary qualities; (2)  describe how understanding of the primary-secondary quality distinction has shifted focus from the mechanical philosophers’ proposal of primary qualities as explanatorily fundamental qualities to current theorists’ proposal of secondary qualities as metaphysically perceiver-dependent qualities; (3) describe some reasons current theorists give to uphold the primary-secondary quality distinction on the basis of the perceiver dependence of colour; (4) argue that these reasons for characterizing colour as a secondary quality are not convincing; and finally (5) argue that reasons for characterizing colour, smell, taste, sound, and warmth and colour as secondary qualities—in particular, those offered on the basis of our common-sense divisions among sensory modalities—also are not convincing.

2 Early modern background For mechanical philosophers, the distinction between mechanical and non-mechanical qualities was of greater importance than a distinction between colours and shapes. (Some terminological notes: I’ll use the term ‘mechanical philosophy’ for the early modern philosophical/scientific view in which a small number of qualities of matter explain all material change. This view encompasses Hobbesian materialist corpuscularism as well as dualist corpuscularism [Wilson, 1992, 227; Smith, 1990, 224; Anstey, 2000, 1–4 and 28]. Although Locke originated the terms ‘primary quality’ and ‘secondary quality’ in the Essay, I’ll use these terms for a distinction drawn by mechanical philosophers generally, including those whose work preceded the Essay. See also Simmons, Chapter 4, this volume, for a related discussion.) In early modern mechanical philosophy, mechanical qualities are fundamental qualities of matter; all other qualities of matter (thus the non-fundamental or non-mechanical qualities) can be explained in terms of these. For example, Boyle proposed that the size, shape, and motion and rest as mechanical qualities explain all other (micro or macro)

Primary and Secondary Qualities    407 qualities of matter—including magnetism and electricity, as well as colour, taste, and smell (Anstey, 2000, 28–30, and 45–47). And Margaret D. Wilson states that theorists from Galileo and Descartes through to Locke understood mechanical qualities—and in particular, size, shape, and motion and rest1—to be qualities of matter that play a fundamental explanatory role with respect to all other material qualities (1992, 227). Thus the early modern mechanical philosophers hark back to the ancient atomists who made similar claims about the explanatory role of qualities such as size, shape, and motion (for this connection see, for example, Smith, 1990, 226, and Hilbert, 1987, 2–3). Because of the fundamental explanatory role of mechanical qualities with respect to all other material qualities, the mechanical philosophers’ crucial distinction was between mechanical and non-mechanical qualities. Thus, for example, Peter Anstey claims that the distinction between mechanical and non-mechanical qualities was focal for Boyle (Anstey, 2000, 28–30); Lisa Downing notes that this distinction was focal for Locke (Downing, 1998, 389). However, according to the mechanical philosophers, primary qualities just are mechanical qualities. By contrast, though, the category of non-mechanical qualities is far broader than that of secondary qualities (that is, the list of colour, smell, taste, sound, and warmth and cold). Thus, not only did mechanical philosophy establish a common ground amongst early modern theorists with regard to a general understanding of primary qualities (Smith, 1990, 224–229), it also construed secondary qualities as just one category of non-fundamental quality among others, where non-fundamental qualities are a very large and heterogeneous group. (In fact, Boyle and Locke sometimes referred to the broader group of non-mechanical qualities, including magnetism and electricity, as secondary qualities, where ‘secondary’ means that the quality is the explanandum of primary-quality explanans. So, colour, smell, taste, sound, and warmth and colour are one group of qualities among this broader group of secondary qualities.)

Locke’s primary-secondary quality distinction Despite the common ground among mechanical philosophers with respect to a general understanding of primary qualities, there was also, of course, a great deal of disagreement among these theorists—a group, after all, including Galileo, Descartes, Hobbes, Boyle, and Locke. In order to make discussion of early modern proposals of the primary-secondary quality distinction manageable, I’ll sketch out one prominent proposal, namely, Locke’s. (Locke’s Essay alone suggests more than one way of characterizing the primary-secondary quality distinction; Downing, 2009, differentiates three ways of characterizing the distinction in the Essay, and Jacovides, 2007, finds six.) According to Locke, the only entities to which we have immediate epistemic access are mental entities, namely, ideas; thus, our epistemic access to material objects and their qualities is not immediate, but rather mediated by ideas. (This is a standard interpretation of Locke; however, there are dissenters. See Jacovides, 1999, 466, for a brief discussion of this issue.) As a result, Locke’s examination of the qualities of material objects begins with an examination of ideas of qualities of material objects.

1 

In what follows, I’ll leave out the ‘and rest’ qualification of motion.

408   Peter Ross Ideas of primary and secondary qualities are subgroups of so-called simple ideas. A simple idea ‘contains in it nothing but one uniform Appearance, or Conception in the mind, and is not distinguishable into different Ideas’ (II.ii.1, 119).2 Examples of ideas that Locke considered as simple include motion, the smell and whiteness of a lily, and the taste of sugar. Complex ideas ‘are ultimately resolvable into simple Ideas, of which they are compounded . . .’ (II.xxii.9, 292). Thus, a grasp of complex ideas requires a grasp of component ideas, but a grasp of simple ideas does not, since they have no component ideas. So, since our idea of the quality of being a yellow square is complex, this quality can’t be either a primary or secondary quality. Locke presents the distinction between ideas of primary and secondary qualities as being between ideas that resemble qualities of matter (these being ideas of primary qualities) and ideas that do not resemble qualities of matter (these being ideas of secondary qualities). Yet, what Locke meant by ‘resemblance’ is notoriously difficult to interpret. Michael Jacovides argues that Locke used the term ‘resemblance’ literally; a pattern formed by primary qualities of ideas resembles a pattern formed by primary qualities in objects (1999, 468–470). Also, Jacovides takes resemblance between primary qualities of ideas and primary qualities of matter to be crucial to Locke’s synthesis of his epistemology (in which the only entities to which we have immediate epistemic access are ideas) and corpuscular theory (which aims to explain changes in matter in terms of size, shape, and motion) (1999, 480–488). According to Jacovides, Locke thinks that despite our relation to the world beyond our ideas being mediate, we can accurately characterize the world in terms of these primary qualities because the size, shape, and motion3 of ideas resemble the size, shape, and motion of matter (Jacovides, 1999, 484–485; see also Jacovides, 2007, 109, and Downing, 1998, note 15, 388–389). also Samuel C. Rickless argues, however, that the resemblance thesis, rather than supporting the primary-secondary quality distinction, is a trivial consequence of the distinction. He claims that the distinction is founded on a distinction in perceiver dependence; this distinction is, in turn, established by a difference in perceiver relativity, which is illustrated by the example that colours are perceiver relative in ways that shapes aren’t (1997, 312–313). Rickless claims that Locke’s point regarding resemblance is that the idea of, for example, shape resembles a primary quality, namely shape, whereas the idea of, for example, colour does not resemble any primary quality (1997, 310). And, as he remarks, the latter lack of resemblance doesn’t depend on what’s meant by the term ‘resemblance’: ‘. . . on any conception of what resemblance might amount to, ideas of secondary qualities do not resemble any primary qualities. . . the resemblance thesis . . . turns out to be a fairly trivial point’ (1997, 310). Nevertheless, Rickless doesn’t explain why Locke held that there was a positive resemblance between ideas of primary qualities and material primary qualities. (Rickless notes a scholastic analogue to resemblance, descended from Aristotle [1997, 311]. However, this is not an adequate explanation, since Locke did not accept this scholastic view [Jacovides, 1999, 463].)

2 

Essay references have the syntax [Book.Chapter.Section]. I’ve also included the page number to the 1975 Nidditch edition. 3  Locke’s lists of primary qualities vary; following Downing, 2009, section 1.3, I’ll abbreviate the list to be size, shape, and motion.

Primary and Secondary Qualities    409 This disagreement about interpretation of the resemblance thesis relates to another disagreement, namely, about the evidential role played by Locke’s examples of perceiver relativity (II.viii.16–21, 137–139). Some commentators hold that Locke uses these examples as independent evidence for the primary-secondary quality distinction—that is, evidence independent of mechanical philosophy and its aim to explain changes in matter in terms of size, shape, and motion (Rickless, 1997, 313–315). Others, however, contend that these examples are mere applications of mechanical philosophy’s view that size, shape, and motion are explanatorily fundamental (Wilson, 1992, 219–226). These two disagreements give a sense of the disputes surrounding interpretation of Locke’s presentation of the primary-secondary quality distinction. However, these interpretive disputes occur along with consensus regarding the mechanical philosophers’ shared understanding of primary qualities. While there are some interpretive disputes about primary qualities, for example, whether primary qualities were taken to be essential to matter—Smith interprets Locke as holding primary qualities are essential (1990, 233–234), and Margaret Atherton interprets Locke as denying that they are essential (1991, 51, and 1992, 111–115)—there is consensus that Locke along with other mechanical philosophers held primary qualities to play a fundamental explanatory role in a science of matter (see, for example, Atherton, 1991, 54).

3 A shift in focus from primary to secondary qualities While the primary-secondary quality distinction is still referenced in philosophical debates about qualities such as colour, the mechanical philosophers’ characterization of primary qualities is obviously no longer a part of the debate. Whereas primary qualities, and their fundamental explanatory role in a science of matter, had been the mechanical philosopher’s focus, interest in the primary-secondary quality distinction no longer has to do with the foundations of physical science. In the current philosophical literature, primary qualities are characterized as qualities of physical objects which are perceiver independent, whether these qualities play a fundamental explanatory role in science or not. For example, the claim that colour is a reflectance property of objects currently counts as a primary-quality view of colour even though reflectance properties aren’t explanatorily fundamental but rather are dispositions to reflect light which are explained in terms of microphysical properties. Some historians of philosophy propose to revert back to having the term ‘primary quality’ refer only to those qualities which play a fundamental explanatory role in current science. Thus, A. D. Smith states that if we are to retain the term ‘primary quality’, we can ‘use the term to advert to the properties deemed fundamental by current science’ (1990, 253). Lisa Downing even suggests that Locke might accept as being primary ‘qualities unfamiliar from sense perception, say, spin or charm’ (2009, section 2.3). This reform of current usage of the term ‘primary quality’ would regain at least some of the scientific relevance of early modern primary qualities. But it would come at the expense of the philosophical relevance of the primary-secondary quality distinction. Current philosophical interest in the distinction does not have to do with whether colours, for example,

410   Peter Ross are explanatorily fundamental physical qualities—no one thinks that they are—but rather whether or not colours, unlike shapes, are perceiver dependent. Yet, as Rickless (1997) stresses, Locke was also concerned with perceiver dependence. So, setting aside opaque claims about resemblance, Locke’s proposal that secondary qualities such as colours are ‘in truth nothing in the Objects themselves, but Powers to produce various Sensations in us by their primary Qualities’ (II.iii.9, 135)  sounds quite contemporary; it sounds like the current characterization of colour offered by dispositionalism. (Dispositionalism about colour claims that colours are dispositions of physical objects to cause colour experiences in certain perceivers in certain viewing conditions.) So perhaps we can see eye to eye with Locke with respect to secondary qualities, if not with respect to primary qualities. It is a hope along these lines—a hope that early modern philosophers were onto something important about the metaphysics of qualities such as colours—that explains continued interest in the primary-secondary quality distinction, but with a shift in focus from primary qualities as fundamental to secondary qualities as perceiver dependent. Nevertheless, Locke was a mechanical philosopher and mind-body dualist; differences between Locke’s theoretical commitments and purposes and those of current theorists should make us cautious about whether agreement with Locke on secondary qualities is more than superficial. Wilson remarks on a very good indicator of the difference in commitments and purposes between Locke (and mechanical theorists in general) and current philosophers. She says:  common to most of [the mechanical philosophers] is a tendency to vacillate, just as Locke does, over whether terms like ‘color’ and ‘red’ denominate physical structures, or the “powers” that (partly) result from the structures to cause sensations, or (as Locke seems usually to suppose) the sensations themselves (1992, 228).

(See also Wilson’s other references to this point, 1992, 234 and 237.) The background for this remark is the mechanical philosophers’ account of perception in terms of a causal chain: a physical quality of an object affects a physical medium (such as light) between the object and perceiver; the physical medium affects a sensory organ (such as the eye) which includes links to the brain of the perceiver, and the perceiver’s neurophysiology produces a mental response (Anstey, 2000, 74). Once this causal chain is laid out, we can see that the vacillation that Wilson has in mind is with respect to what part of this causal chain should be referred to as colour. Is colour a physical quality of objects (which, according to mechanical philosophy, is a non-primary physical structure which can be explained in terms of primary qualities)? Or is it a power that this physical quality of objects has to produce mental responses in perceivers (by way of a physical medium, sense organs, and the brain)? Or is it a mental response, that is, a sensation? This vacillation can be explained in a way that highlights deep differences between Locke and current philosophers. Considering Locke’s theoretical commitments, he sharply distinguished between the physical production of colour sensation from the colour sensation itself. If we are talking about the qualitative aspect of colour of which we are conscious, he would take the reference of ‘colour’ to be a non-physical mental quality. Locke held that the only entities to which we have immediate epistemic access are ideas (which are mental entities), and he was a mind-body dualist (as most mechanical philosophers were). But Locke, being a mechanical philosopher, wasn’t always talking about the

Primary and Secondary Qualities    411 qualitative aspect of colour. So if we are talking about the physical production of colour sensation, he would take the reference of ‘colour’ to be a physical part of the causal chain. Consequently, the usage of ‘colour’ was flexible for Locke, but the metaphysics was not open to serious debate. Locke assumed that the qualitative aspect of colour, being a mental quality, was non-physical. He didn’t take an explanation of the qualitative aspect of colour in terms of primary qualities to be an option. Indeed, Locke didn’t think that the causal relation between the physical qualities of the brain and the qualitative aspect of colour—a relation involving mind-body causal interaction—is intelligible (Downing, 1998, 407; see also Curley, 1972, 450–454). But currently, it is common (but nevertheless controversial) for theorists to hold the claim that the mind is wholly material. Furthermore, Locke’s claim that the only entities to which we have immediate epistemic access are mental entities is widely rejected. Thus, an aim amongst many (but not all) theorists these days—and something which wasn’t even up for discussion for Locke—is to consider whether physical qualities of objects not only contribute to producing the qualitative aspect of colour of which we are conscious, but in fact are identified with the qualitative aspect of colour. Thus, while Rickless’s interpretation of Locke, in which Locke’s basis for the primary-secondary quality distinction is a difference in perceiver dependence, maps nicely onto our current interest in the distinction, we have to keep in mind important differences. In certain ways the issues are considerably narrower now, since our current interest is very specifically to do with the qualitative aspect of secondary qualities, and in certain ways the issues have become much broader, since we are open to contemplating a much wider scope of proposals of qualities to identify with the qualitative aspect of secondary qualities—including physical qualities of physical objects. The philosophical literature on colour has been the primary arena for examining the tenability of a primary-secondary quality distinction. Reflecting this role with respect to the distinction, theories of colour divide into those holding that colours are perceiver independent and those holding they are perceiver dependent. Among theories on the perceiverindependent side is physicalism (which claims that the qualitative aspect of colour of which we are conscious is a physical quality of physical objects). Among theories on the perceiver-dependent side are dispositionalism (which, as stated above, claims the qualitative aspect of colour is a disposition of physical objects to produce certain colour experiences), and subjectivism (which holds that the qualitative aspect of colour, if it exists as an instantiated quality, is a quality instantiated by perceivers and not a quality of physical objects at all). In the next section I’ll briefly survey some examples of support for secondary-quality theories of colour, as this support is given for dispositionalism and subjectivism.

4 Current support for the claim that colour is a secondary quality Speaking generally, the sort of perceiver dependence at issue is one where colour is at least in part constituted by a mental quality of a perceptual response. Thus, this perceiver

412   Peter Ross dependence is a metaphysical dependence, not simply an epistemic dependence with respect to perceptual conditions such as a particular viewing point. Taking the case of shape, obviously shapes of objects as seen are relative to a particular viewing point. But no matter what one’s viewing point is with respect to an object, it seems that the object’s shape exists independently of one’s perceptual access. Thus, it seems that shape is metaphysically, if not epistemically, perceiver independent. Is colour perceiver independent, as shape seems to be? According to theories that claim that colour is a secondary quality, perceptual responses don’t just provide access to colour, but are constitutive of colour. This characterization of the primary-secondary quality distinction needs support in two ways: first, it needs support for the claim that shapes exist independently of one’s perceptual access, and second that colours don’t. Current theorists tend to take the first claim for granted, though. So I will describe support for the second claim. Why think that colour is at least in part constituted by a mental quality? I’ll describe two standard approaches to answering this question. According to one approach, which I’ll call the qualia approach, the qualitative aspect of colour is necessarily conscious. According to the other approach, which I’ll call the reductive secondary-quality approach, even though colour is perceiver dependent, it is possible to reduce colour to non-chromatic qualities.

The qualia approach Taking up the first approach, the issue is the relation between consciousness and qualitativeness. Current proposals about colour are proposals about the qualitative aspect of colour of which we are conscious. And it might seem strange to deny that the qualitative aspect of colour of which we are conscious is at least partly mental. After all, something’s being conscious might seem sufficient for its being mental. But in talking about the qualitative aspect of colour, we might be talking about a qualitative aspect which is independent of our consciousness of it. One proposal for a characterization of qualitativeness which is independent of consciousness is in terms of what’s called a psychological quality space (Rosenthal, 1991, 144, and 2005b, 196–203). For example, the qualitative aspect of colour can be characterized in terms of the ordering of colours by relative similarity, an ordering which results in separate dimensions of hue, saturation, and lightness (Clark, 1993, ch. 4). An indication that qualitative orderings are independent of consciousness is that we can find orderings of qualities perceived by non-human creatures, through stimulus generalization and statistical techniques such as multidimensional scaling; for finding orderings in these cases does not require we assume that the creature is of a sort that has conscious experience (Clark, 1993, 117–118). So, according to this proposal, consciousness might be giving us access to the qualitative aspect without this aspect being necessarily conscious. Nevertheless, the qualitative aspect of colour is often taken to be necessarily conscious. For example, a standard characterization of the qualitative aspect of colour is as what it is like to be conscious of colour (a characterization originating in Nagel, 1974). Of course, there is no non-conscious way that it’s like to be conscious of colour, so this characterization takes the qualitative aspect of colour to be necessarily conscious. In turn, a standard

Primary and Secondary Qualities    413 way of accounting for what it is like to be conscious of colour is as a mental quality. Such mental qualities are often called qualia. Thus, according to the qualia approach, the qualitative aspect of colour is at least in part constituted by a mental quality that is necessarily conscious, namely, a quale. In further explaining this approach, I’ll describe reasons offered by Colin McGinn and Christopher Peacocke in support of dispositionalism about colour. McGinn’s dispositionalism assumes that colour experiences are characterized in terms of colour qualia (1983, 8–9). With this assumption stated, he offers the following thought experiment in connection with his dispositionalist proposal. Imagine a situation where a physical quality of objects is purported to be red because the physical quality looks red to ordinary human perceivers in ordinary viewing conditions. However, Martians land, and it turns out that this physical quality looks green to ordinary Martian perceivers in ordinary viewing conditions. Is the physical quality red or is it green? Intuitively, ‘. . . there will be no choosing between these groups of perceivers in respect of whose experience determines the colour of objects [with the physical quality] in question’ (1983, 10). McGinn presents this thought experiment as being merely illustrative of his dispositionalist proposal. Nevertheless, McGinn clearly thinks that the thought experiment, which is modelled after Jonathan Bennett’s widely cited phenol-thio-urea thought experiment (1965, 9–10), captures a common intuition that there really would be no choosing between human perceivers and Martian perceivers with respect to colour. (In Bennett’s thought experiment, phenol has a bitter taste to ordinary perceivers in ordinary tasting conditions, but, due to genetic control imposed by a world dictator, phenol becomes tasteless to everyone.) Thus, colours aren’t physical qualities of objects; for if they were, there would be a perceiver-independent way of choosing. Further, McGinn offers what he calls a variable realization argument against identification of colours with physical qualities of objects. If science discovered that the physical quality of objects perceived as red (by ordinary human perceivers in ordinary viewing conditions) is actually physically heterogeneous, ‘we would not then say that the objects varied in colour, contrary to what we had supposed on the basis of their appearance; we would say rather that the property of being red was correlated with no single underlying physical property’ (1983, 13). The possibility of variable physical realization indicates that physical qualities are red, not on the basis of physics, but because they produce colour experiences characterized by colour qualia. Thus the qualitative aspect of colour must, at least in part, be a mental quality. Peacocke, also assuming that colour experiences are characterized in terms of colour qualia, offers another thought experiment on behalf of dispositionalism. Imagine a congenitally blind person who detects a physical quality that looks red to ordinary sighted perceivers, but by way of an auditory signal. This person becomes as good at non-inferentially identifying this physical quality as a non-colour-blind person does. Even though this blind person non-inferentially identifies objects with this quality as red, a common intuition is that the person doesn’t have the concept of being red—where ‘being red’ is the qualitative aspect of red, which is after all what we are interested in. But then, having the concept of being red must require having the quale red, which the blind person lacks. Thus, red is not a physical quality, but must be at least in part constituted by a red quale (Peacocke, 1984, 54). McGinn and Peacocke use these thought experiments to support a version of dispositionalism about colour, according to which the qualitative aspect of colour is a

414   Peter Ross dispositional relation between physical qualities of objects and qualia of experiences—so, the qualitative aspect of colour is only in part a quale. (A challenge for dispositionalism has been to reconcile its claim that the qualitative aspect of colour is a disposition when, on the face of it, dispositions, being modal properties, aren’t straightforwardly seen. McGinn has by now rejected dispositionalism for this sort of reason [1996, 540]. However, as I understand Peacocke, his claim is that the qualitative aspect of colour is a physical quality of objects seen in virtue of a colour quale; the relation between the physical quality and the quale is dispositional, but we need not, and in fact don’t, see the qualitative aspect of colour as a disposition.) If a theory holds that the qualitative aspect of colour is nothing more than a quale, then the theory is a version of subjectivism. An important difference between dispositionalism and subjectivism is that dispositionalism claims that physical objects are coloured (since physical objects have the dispositions in question), whereas subjectivism, by contrast, claims that physical objects are not the proper bearers of colour. Despite their differences, however, subjectivists could support their view with McGinn’s and Peacocke’s arguments—since the target of these arguments is a shared opponent, namely physicalism about colour—and supplement them with arguments for the claim that physical objects are not the proper bearers of colour (for such arguments, see, for example, Boghossian and Velleman, 1989). Thus, the qualia approach can be employed to support different particular proposals for characterizing colour as a secondary quality, one according to which the qualitative aspect of colour is a disposition to produce colour experiences with qualia, and one according to which the qualitative aspect of colours is simply a quale. Either way, however, proponents of the qualia approach are sceptical that colour qualia, and thus colours that qualia at least in part constitute, will ever have a scientific explanation. Admittedly, we have no explanation of the qualitative aspect of colour in physical terms, an epistemic gap which Joseph Levine (1983) has named the explanatory gap. However, this scepticism implies that this gap will likely always exist, limiting how far science can explain our mental lives. While sceptics about eliminating the explanatory gap need not be dualists (since the gap is an epistemic gap, not a metaphysical one), this marking of a likely limit of science is, in a very general way, a successor of Locke’s view that the mechanical philosophy cannot explain the qualitative aspect of colour. Smith evokes this legacy of Locke by asserting that most of the contemporary philosophers who accept the primary-secondary quality distinction ‘do so because they believe that science tells them so’ (1990, 231). Smith, himself a proponent of the qualia approach, has in mind that most philosophers believe that science tells them it is at least unlikely that there will ever be a scientific explanation of qualia (Smith, 1990, 221, 231–232, 239–240). The reductive secondary-quality approach takes a wholly different attitude toward the explanatory gap. Proponents of this approach are optimistic about our prospects for eliminating the explanatory gap, at least eventually; this optimism is expressed, for example, by C. L. Hardin (1997). It might be that the primary-secondary quality distinction is often taken to imply the claim that secondary qualities are unlikely to be explained by science. If the distinction is taken this way, perceiver dependence is a necessary but not sufficient condition for being a

Primary and Secondary Qualities    415 secondary quality, another necessary condition being this claim that secondary qualities are unlikely to be explained. But there is no consensus about how to use the term ‘secondary quality’. I’ll briefly discuss the reductive secondary-quality approach because it claims that colour is perceiver dependent, and also because it is interesting to contrast the way in which proponents of the reductive approach and proponents of the qualia approach make use of science.

The reductive secondary-quality approach Proponents of the qualia approach, if they appeal to science at all, tend to use very general considerations about what qualities can play an explanatory role in science. (A case in point: McGinn states that secondary qualities are explanatorily idle. He claims that only primary qualities, not secondary qualities, explain the production of perceptual experiences; moreover, only primary qualities explain causal interactions among physical objects [1983, 14–15].) By contrast, proponents of the scientific reduction approach take specific findings from psychophysics and neurophysiology to argue about the metaphysics of colour. Over the past twenty-five years or so, Hardin has been a leader in demonstrating that science specifically relating to colour and colour vision is important to the philosophical discussion of colour. In addition to being optimistic about eliminating the explanatory gap, Hardin tells us that this science so strongly constrains positions on the metaphysics of colour that we can conclude that colour is not a physical quality of objects. Thus, Hardin uses findings from colour science to argue against physicalism. Against physicalism, Hardin explains that due to the operation of our visual systems, for a given perceiver in given viewing conditions, different reflectance properties match in colour; this matching despite physical difference is called metamerism. So, from the standpoint of reflectance properties, different reflectance properties can be seen as a determinate shade of red (Hardin, 1993, 26–29). The moral of this story is that the physical quality that looks red is not a physical kind (that is, a kind referred to in non-disjunctive terms of physics), but instead an indefinitely large physical disjunction. Thus, Hardin informed philosophers of the physical heterogeneity that McGinn had merely imagined. Assuming that colour can’t be a disjunctive quality of physical objects, a physicalist proposal about colour fails. Thus far, Hardin offers an argument that can be taken up by either reductive subjectivism or reductive dispositionalism against physicalism. And some proponents of reductive dispositionalism have taken up this sort of argument (see, for example, J. J. C. Smart’s description of his view before he held that colours are disjunctive physical qualities, 1975, 2–3). Hardin argues for a specific secondary-quality view, namely, the subjectivist view that holds that physical objects are not the proper bearers of colour. Against dispositionalist proposals—which characterize colour in terms of the visual experiences of certain perceivers in certain viewing conditions—Hardin argues that there are no specifications of perceivers and viewing conditions which are adequately principled. For example, ordinary viewing conditions, while providing a reasonable practical standard, are not scientifically

416   Peter Ross principled (1993, 67–76). Assuming that colour must be characterized in a way that is scientifically principled, dispositionalism fails. With this sampling of support for the claim that colour is a secondary quality, I’ll consider how one might reply on behalf of physicalism.

5 Why this current support is unconvincing According to the qualia approach, the qualitative aspect of colour is characterized as what it’s like to be conscious of colour; this renders the qualitative aspect of colour as being necessarily conscious. Even though the ‘what it’s like’ locution doesn’t sound theoretical, the idea that the qualitative aspect of colour is necessarily conscious isn’t just a matter of common sense, but rather is a theoretical claim about the relationship between consciousness and qualitativeness. And this theoretical claim might be mistaken (Rosenthal, 1991, 136–137). Alternatively, if the qualitative aspect of colour is only contingently conscious, consciousness gives us access to a qualitative aspect of colour that we can characterize without appeal to consciousness. This alternative opens up the possibility that the qualitative aspect of colour is a physical quality of objects to which our perceptual responses give us access. Thus, colour might be just like shape, in that for both, perception gives us access to the quality without constituting it (Rosenthal, 1991, 141). From the standpoint of this alternative to the qualia approach, the qualia approach simply ignores qualitative aspects of physical objects to which perceptual responses don’t give access. Thus, taking up McGinn’s question of whether human perceivers or Martian perceivers determine the colour of objects, McGinn simply ignores that such consciousness-independent qualitative aspects of physical objects might play a role in deciding the colours of objects. However, because both McGinn and Peacocke claim that acquiring the concept of red requires having visual experience with red qualia (McGinn, 1983, 8–9; Peacocke, 1984, 51 and 54), neither would be convinced of the possibility of a qualitative aspect of colour without consciousness. It is difficult to understand how the concept of red would have been acquired without conscious perception of red. However, to take this to show that red is metaphysically perceiver dependent is to conflate an epistemic point with a metaphysical one. If we take a description of colour as consciously perceived (such as ‘the quality attributed to physical objects in virtue of objects looking red to ordinary perceivers in ordinary viewing conditions’) as merely fixing the reference of colour terms, then the description serves to pick out (what turn out to be) physical qualities of objects. In this way the description specifies an epistemic role, and does not specify the metaphysics of colour. (For more about the reference fixing strategy, see Ross, 2010, which is indebted to Rosenthal, 1991, 138; Smart, 1975; and Armstrong, 1987.) Thus, a congenitally blind person might well have the concept of the qualitative aspect of red of which we are conscious, so long as we understand ‘the qualitative aspect of red of which we are conscious’ to mean the qualitative aspect of red to which visual consciousness can give us access. Since the qualitative aspect of red is independent of visual

Primary and Secondary Qualities    417 consciousness, and turns out to be a physical quality of objects, once this physical quality is picked out, it can be detected in other ways. The notion that the qualitative aspect of colour is what it’s like to be visually conscious of colour—and thus is necessarily visually conscious—would, of course, disallow that the blind person perceives the qualitative aspect of colour. But the notion that the qualitative aspect of colour is necessarily visually conscious might be mistaken. Yet, as McGinn speculated, and as Hardin confirms, the physical qualities of objects seen as red are physically heterogeneous. Thus, if colour is a physical quality of objects, it is a disjunctive quality. A serious worry about disjunctive qualities is whether they can have causal properties. And since physicalism holds that colours produce colour experiences, physicalism is stuck with holding that these physical qualities have causal properties. Fortunately, the plausibility of the idea that disjunctive qualities have causal properties has been supported (for this support, see Antony, 2003, and Clapp, 2001). In addition, since our sensory modalities have evolved to pick up qualities of biological relevance, it’s not surprising that our visual systems run roughshod over many distinctions of relevance to physics (Hatfield, 1992). Finally, the biological relevance of colour suggests that the ‘ordinary perceiver in ordinary viewing conditions’ way of picking out the colours of physical objects can be scientifically principled, if the science involved is biology, where, after all, such factors as perceivers’ capacities and perceptual conditions matter. Thus, considering the basis for the primary-secondary quality distinction as being a difference in perceiver dependence, and colour to be the testing ground for the tenability of the distinction, there’s reason to think the distinction fails. Of course, other arguments for colour being a secondary quality, apart from the representative ones given here, would have to be considered to reject the distinction. But colour might be every bit as perceiver independent as shape.

6 Distinguishing the senses and the primary-secondary quality distinction Theorists have been concerned that generalizing from colour and vision to other modalities and qualities is risky methodology, however (Heil, 1983, 136; O’Callaghan, 2008, 316; Batty, 2009). With this in mind, I’ll conclude by broadening the testing ground for the primary-secondary quality distinction to encompass other modalities and other qualities proposed as secondary. Indeed, H. Grice (1962) argues for the primary-secondary quality distinction on the basis of considering how we distinguish the sensory modalities. Grice argues that our common-sense distinctions among sight, smell, taste, hearing, and touch cannot be drawn on the basis of physical properties of objects; instead our common-sense divisions among senses require qualia. Thus, Grice distinguishes senses on the basis of secondary qualities where these are characterized as being at least in part constituted by qualia. In this case, our common-sense distinction among senses provides a reason for accepting the primarysecondary quality distinction (see also Smith, 1990, 239–240, for this contention).

418   Peter Ross But why believe that qualia are necessary for distinguishing senses? To set up his answer, Grice offers candidate criteria for distinguishing senses which correspond with the parts of the mechanical philosophers’ causal chain involved in perception. Grice’s four candidate criteria are: (1) distinctive perceptible qualities (such as size, shape, and colour) attributed to objects; (2)  a special introspectible character of perceptual states (which is a qualitative difference that Grice holds to be at least potentially separate from qualitative differences of perceptible qualities); (3) a distinctive physical stimulus involved in perception (such as light or pressure waves); and (4) a characteristic sensory organ including its connection to the brain (such as eyes or ears and their connections to the brain) (1962, 85). Grice then presents us with a thought experiment which is supposed to show that what he calls introspectible character—qualia—are necessary for distinguishing senses. Martians land. As it turns out, Martians have two sets of sensory organs, ‘each pair more or less like our eyes’, which are sensitive to light and perceptually attribute size, shape, and colour to objects; these sensory organs are for x-ing and y-ing. According to proposed criteria (1), (3), and (4)—the perceptible quality, physical stimulus, and sensory organ criteria—x-ing and y-ing are instances of seeing. But, Grice continues, if we ask a Martian whether x-ing blue is like y-ing blue, the Martian says ‘Oh, no, there’s all the difference in the world!’ Grice contends that x-ing and y-ing are found to be different senses due to the supposed difference in qualia (1962, 94). Thus, Grice provides an argument for distinguishing senses by way of qualities at least in part constituted by qualia. However, the question of distinguishing senses has been addressed in two ways, where both ways are variants on Aristotle’s proposal to distinguish the senses in terms of proper sensibles, which are qualities (such as colour, smell, taste, sound, and warmth and cold) that are perceived by only one sensory modality. (The historical relationship between the category of proper sensibles and the category of secondary qualities is not straightforward, since mechanical philosophers were aiming to revolutionize science by rejecting many Aristotelian theoretical commitments, including the claim that qualities such as warmth and cold are explanatorily fundamental; on this point, see Anstey, 2000, 21–22, 58–59.) According to one way of distinguishing senses, the divisions among the senses are drawn on the basis of Aristotelian proper sensibles, where these are characterized as perceiver-dependent qualities. But according to a second way of distinguishing senses, distinctions are drawn on the basis of Aristotelian proper sensibles, where these are characterized as physical qualities of objects. By denying the claim that the qualitative aspect of proper sensibles is necessarily conscious, we can account for the Martians’ ‘all the difference in the world’ remark in terms of a difference in qualitative aspects of perceptible qualities to which the Martians have conscious access but we don’t. In this case, Grice’s proposed criteria (1), (3), and (4) are sufficient for distinguishing senses, because criterion (1) captures ‘all the difference in the world’. Again, the qualia approach simply ignores consciousness-independent qualitative aspects of physical objects. Consequently, with respect to our common-sense distinction among senses, we can appeal to Aristotle’s proper sensibles understood as physical qualities of objects and avoid an appeal to qualia (for more discussion of this claim, see Ross, 2008 and 2011). The senses can be distinguished by the proper sensibles, but without a primary-secondary quality distinction. Taking into account a broader range of modalities and qualities, the conclusion is

Primary and Secondary Qualities    419 that colour, smell, taste, sound, and warmth and cold might well be as perceiver independent as shape.4

References Anstey, Peter R. (2000). The Philosophy of Robert Boyle. London: Routledge. Antony, Louise (2003). 'Who’s Afraid of Disjunctive Properties?' Philosophical Issues, 13(2003), 1–21. Armstrong, D. M. (1987). ‘Smart and the Secondary Qualities’. In Philip Pettit, Richard Sylvan, and Jean Norman (eds), Metaphysics and Morality: Essays in Honour of J. J. C. Smart (pp. 1–15). Oxford: Basil Blackwell. Reprinted in Byrne and Hilbert, 1997, 31–46. Atherton, Margaret (1991). 'Corpuscles, Mechanism, and Essentialism in Berkeley and Locke'. Journal of the History of Philosophy, 29(1), (January 1991), 47–67. Atherton, Margaret (1992). ' "Ideas in the Mind, Qualities in Bodies": Some Distinctive Features of Locke’s Account of Primary and Secondary Qualities'. In Phillip D. Cummins and Guenter Zoeller (eds), Minds, Ideas, and Objects: Essays on the Theory of Representation in Modern Philosophy (pp. 111–124). Atascadero, CA: Ridgeview Publishing Company. Batty, Clare (2009). 'What’s That Smell?' Southern Journal of Philosophy, 47, no. 4 (Winter 2009), 321–348. Bennett, Jonathan (1965). 'Substance, Reality, and Primary Qualities'. American Philosophical Quarterly, 2, no. 1 (January 1965), 1–17. Boghossian, Paul A. and Velleman, J. David (1989). 'Colour as a Secondary Quality'. Mind, 98, no. 389 (January 1989), 81–103. Reprinted in Byrne and Hilbert, 1997, 81–103. Byrne, Alex and Hilbert, David (eds.) (1997). Readings on Color, vol. 1, The Philosophy of Color. Cambridge, MA: MIT Press. Clapp, Lenny (2001). 'Disjunctive Properties: Multiple Realizations'. The Journal of Philosophy, 98, no. 3 (March 2001), 111–136. Clark, Austen (1993). Sensory Qualities. Oxford: Clarendon Press. Curley, E. M. (1972). 'Locke, Boyle, and the Distinction between Primary and Secondary Qualities'. The Philosophical Review, 81, no. 4 (October 1972): 438–464. Downing, Lisa (1998). 'The Status of Mechanism in Locke’s Essay'. The Philosophical Review, 107, no. 3 (July 1998), 381–414. Downing, Lisa (2009). 'Locke: The Primary and Secondary Quality Distinction'. In Robin Le Poidevin, Peter Simons, Andrew McGonigal, and Ross Cameron (eds), The Routledge Companion to Metaphysics (pp. 98–108). London: Routledge. Grice, H. (1962). 'Some Remarks about the Senses'. In R. J. Butler (ed.), Analytical Philosophy, First Series (pp. 133–153). Oxford: Blackwell. Reprinted in Macpherson, 2011, 83–100. Page numbers refer to reprint. Hardin, C. L. (1993). Color for Philosophers:  Unweaving the Rainbow, Expanded Edition. Indianapolis, IN: Hackett Publishing Company. Hardin, C. L. (1997). Reinverting the Spectrum. In Byrne and Hilbert, 1997, 289–301. Hatfield, Gary (1992). 'Color Perception and Neural Encoding: Does Metameric Matching Entail aLossofInformation?' InDavidHull,MickyForbes,andKathleenOkruhlik(eds),Proceedings 4  I am extremely grateful to Margaret Atherton, Todd Ganson, Ericka Tucker, Brian Keeley, Josh Weisberg, and Mohan Matthen for comments on a draft of this chapter.

420   Peter Ross of the 1992 Biennial Meeting of the Philosophy of Science Association, vol. 1, Contributed Papers (pp. 492–504). . Heil, John (1983). Perception and Cognition. Berkeley, CA:  University of California Press. Chapter 1 reprinted in Macpherson, 2011, 136–155. Page numbers refer to reprint. Hilbert, David R. (1987). Color and Color Perception: A Study in Anthropocentric Realism. Stanford, CA: Center for the Study of Language and Information. Jacovides, Michael (1999). 'Locke’s Resemblance Thesis'. The Philosophical Review, 108, no. 4 (October 1999), 461–496. Jacovides, Michael (2007). 'Locke’s Distinction between Primary and Secondary Qualities'. In Lex Newman (ed.), The Cambridge Companion to Locke’s Essay (pp. 101–129). Cambridge: Cambridge University Press . Levine, Joseph (1983). 'Materialism and Qualia: The Explanatory Gap'. Pacific Philosophical Quarterly, 64, no. 4 (October 1983), 354–361. Locke, John (1975). An Essay Concerning Human Understanding, ed. Peter H. Nidditch. Oxford: Clarendon Press. Macpherson, Fiona (ed.) (2011). The Senses:  Classic and Contemporary Philosophical Perspectives. Oxford: Oxford University Press. McDowell, John (1985). 'Values and Secondary Qualities'. In Ted Honderich (ed.), Morality and Objectivity: A Tribute to J. L. Mackie (pp. 110–129). London: Routledge & Kegan Paul. McGinn, Colin (1983). The Subjective View:  Secondary Qualities and Indexical Thoughts. Oxford: Clarendon Press. McGinn, Colin (1996). 'Another Look at Color'. The Journal of Philosophy, 93, no. 11 (November 1996), 537–553. McNaughton, David (1988). Moral Vision: An Introduction to Ethics. Oxford: Blackwell Publishers. Nagel, Thomas (1974). 'What Is It Like to Be a Bat?'. The Philosophical Review, 83, no. 4 (October 1974), 435–450. O’Callaghan, Casey (2008). 'Seeing What You Hear: Cross-Modal Illusions and Perception'. Philosophical Issues, 18 (2008), 316–338. Peacocke, Christopher (1984) 'Color Concepts and Color Experiences'. Synthese, 58, no. 3 (March 1984), 365–381. Reprinted in Byrne and Hilbert, 1997, 51–65. Page numbers refer to reprint. Rickless, Samuel C. (1997). 'Locke on Primary and Secondary Qualities'. Pacific Philosophical Quarterly, 78, no. 3 (September 1997), 297–319. Rosenthal, D. M. (1991). 'The Independence of Consciousness and Sensory Quality'. Philosophical Issues, 1 (1991), 15–36. Reprinted in Rosenthal 2005a, 135–148. Rosenthal, D. M. (2005a). Consciousness and Mind. Oxford: Clarendon Press. Rosenthal, D. M. (2005b). 'Sensory Qualities, Consciousness, and Perception'. in Rosenthal 2005a, 175–226. Ross, Peter W. (2008). 'Common Sense about Qualities and Senses'. Philosophical Studies, 138, no. 3 (April 2008), 299–316. Ross, Peter W. (2010). 'Fitting Color into the Physical World'. Philosophical Psychology, 23, no. 5 (October 2010), 575–599. Ross, Peter W. (2011). Review of The Senses:  Classic and Contemporary Philosophical Perspectives, ed. Fiona Macpherson, Notre Dame Philosophical Reviews, 2011.08.24 (August 2011).

Primary and Secondary Qualities    421 Smart, J. J. C. (1975). 'On Some Criticisms of a Physicalist Theory of Colors'. In Chung-ying Cheng (ed.), Philosophical Aspects of the Mind-Body Problem (pp. 54–63). Honolulu: University of Hawaii Press. Reprinted in Byrne and Hilbert, 1997, 1–10. Smith, A. D. (1990). 'Of Primary and Secondary Qualities'. The Philosophical Review, 99, no. 2 (April 1990): 221–254. Wilson, Margaret D. (1992). 'History of Philosophy in Philosophy Today; and the Case of the Sensible Qualities'. The Philosophical Review, 101, no. 1 (January 1992), 191–243.

Chapter 22

Colou r Perception Kathleen Akins and Martin Hahn

In this entry, we approach the topic of colour perception via a historical circle. We begin with a common view about colour perception, held today, that casts it as unique among our perceptual capacities. We then turn to the origins of our current view via a brief overview of the history of the philosophy of colour and colour science. Two philosophical theses, remnants of Artistole’s views, have driven the evolution of our understanding of colour perception. To a rough approximation, these are the views that (a) colour experiences are (the result of) sensory responses to the world, and; (b) that if we did not see colours (including the achromatic greyscale), we would not see anything at all. We hope to show how these two assumptions have shaped our views of colour perception but also how and why they meet with questions about the metaphysics of colour, of colour ontology. At the end of this entry, we take the liberty of presenting an alternative way of seeing colour perception, one that denies both assumptions and separates questions of colour ontology from questions of colour perception.

1  The peculiarity of colour perception Most contemporary philosophers see colour perception as unique among the perception of properties, objects, or facts. Imagine looking at a common (New York) scene, say a white goat grazing on a green lawn. Intuitively, our perceptions of this scene seem to form a continuum. You see the white of the goat and the green of the lawn along with myriad other colours. You also see the shape of the goat and its location and two objects, the goat and the lawn. And those of us with a bit of goat knowledge also see the goat as a goat, one that is grazing on the lawn. Finally, you see that your neighbour’s goat has once again escaped from his yard. At one end of the continuum, what we see is what any creature equipped with the roughly the same visual system would see, the colours of the visual world. At the other, there are conceptually rich perceptual beliefs. While there has been much controversy over where seeing-proper ends and inferred beliefs begin, whether there is seeing without seeing as, when (and whether) concepts become involved in perceptual content, etc., there has been broad agreement about where seeing colours lies: at the very start of the continuum.

Colour Perception   423 To see the whiteness of the goat, we only need to be in the right place at the right time and our sensory system will cause in us the response of seeing white, barring abnormalities. This is not to say that the causal mechanism that leads to this response is in any way simple or ‘direct’. As we will see, philosophers are only too aware these days of many of the complexities of colour processing. But no matter how complex the processing is, it is fundamentally different from seeing the shape or the size of the goat, or seeing the goat itself. We will be spelling out what this view of colour comes to and how it arose, but we also want to show how it came to be that facts about colour perception are taken to be relevant to issues of the metaphysics of colour in a way no other perceptual facts are relevant to corresponding ontological issues. Some examples of this unusual role of colour perception will help situate what we see as so peculiar about contemporary views of colour and its perception. When contemporary philosophers think about the nature of colours, they are prone to talk about what Frank Jackson (1996) has called the ‘Prime Intuition’ about colours: ‘Red’ denotes the property of an object putatively presented in visual experience when an object looks red. When philosophers think about the content of colour experience, they might well characterize it as does David Chalmers (2006): ‘the Fregean content of a phenomenally red experience will be satisfied when there is an object at the appropriate location relative to the perceiver that instantiates the property that normally causes phenomenally red experiences in the perceiver’ (p. 60). In the metaphysics of colour, the reality or independent existence of colour properties in the external world is often thought to hang on the sort of question posed by Jonathan Cohen (2004): is there any principled way to choose just one particular colour experience from among the multiple colour experiences caused by a single object in a given observer—or in different species or in different humans? How do we know—how can we know—which is the correct colour? These ways of speaking seem perfectly natural—indeed, Jackson, Chalmers, Cohen, and many other philosophers rarely find they need to argue for them. While these intuitions seem to fit colour perceptions, they do not apply very well to perceptions of other properties or objects. Consider the ‘truism’ that ‘being a goat’ (or ‘6 feet long’) denotes the property of an object putatively presented in visual experience when an object looks like a goat (or 6 feet long). No one would call these ‘prime intuitions’ about domestic animals or length. Insofar as the principles are true, they might express the important empirical fact that our perceptions are generally reliable: most goat-seeming objects are goats and we are reasonably good at estimating length, given the size-constancy mechanism of our perceptual system and lots of experience. Six-foot-long objects present us with different appearances, depending on distance, angle of viewing, surrounding objects, etc. But we do not suppose that the property of being 6 feet long depends on privileging one of these appearances as veridical (one might well think that most of them are veridical, even though they differ from one another). We will try to show how colour perception came to hold such a unique position by tracing its central assumptions through its history and showing how the unique role of colour perception in colour metaphysics arose from them. But first a word of caution. The usual way of articulating these assumptions is in terms of colour sensations: to perceive a colour is to have a sensation, usually caused by some external stimulus. Behind both Jackson’s and Chalmer’s view is the idea that colours are the normal causes of such sensations, while Cohen argues against the possibility of differentiating such standard or normal situations in which colour sensations occur from all the others. While this is a convenient way of

424    Kathleen Akins and Martin Hahn speaking, it is quite misleading. There is no agreement in current philosophy on the usage of the term ‘sensations’ and, consequently, when theorists affirm or deny their existence they often talk past one another.1 We will, therefore, avoid ‘sensation’ talk except in cases where the word was used by the authors we are discussing. Much of our historical overview will concern the science of colour perception. The distinction between sensation (in the specific sense to be explained herein) and perception has been widely accepted in psychology and colour has been thought to be, in the first place, sensed and not perceived. So, until fairly recently, psychologists and neuroscientists spent considerable energy looking for the colour area (Zeki, 1983):  an area (physical or functional) where sensory responses to colour stimuli, as determined by psycho-physics, would be mapped in neural space thereby instantiating the phenomenal space of colour. Again, there has been no similar quest for a domestic-animal-phenomenal-space nor indeed a systematic attempt to map shape-sensory response space using controlled psychophysical experiments and multi-dimensional scaling analysis. Colour has, until recently, held a unique place in vision science in ways we will explain more fully.

2 The Aristotelean legacy: Direct Realism Modern theorizing about colour begins with Aristotle. The fundamental assumption that Aristotle makes, remnants of which have been nearly irresistible to subsequent theorists, is that the senses provide a sort of direct conduit between the world and the subject. Whatever complexities our perceiving or thinking of the world involves, our relation to the world starts with the senses and that connection is a simple causal one. For Aristotle, the five senses are distinguished by their proper objects, the sensible species each transmits to the sensorium (Matthen, Chapter 30, this volume; Sorabji, 1971). The proper object of sight is colour: this is what objects cause the sensory organs to acquire. All other visual information about the world is derived from the distribution of colours with which the eyes provide the sensorium, the place where all sensory data congregate and await whatever processes turns them into perceptions. Vision has to start with colour, for what else does the world present to the eye but a two-dimensional distribution of colours (including the greyscale)? For Aristotle, if we did not see the colours, we could not see the other visual properties, the properties of shape, location, or anything else. Everything else we see by means of seeing the colours (Aristotle, 1956: II 6, 418a26–30). In vision, Aristotle says, the sensible species of colour travels through the transparent medium to the eyes; the eyes then take on colour. Redness is not represented; it is moved from object to perceiver. Thus colours, insofar as they inform objects (the rose itself) figure in the immediate causes of visual experience. It is hard to imagine a more simple or

1  One can begin to see the reason for this just in considering the diverse historical roots of the term. Quite apart from its heavily epistemology-invested antecedents in modern philosophy from Descartes through Hume, two of its highly influential introductions are by Reid to stand for the qualitative aspect of conscious experience on the one hand and by Helmholtz as a necessary stage between sensory transducers and perceptual processing on the other—one that need be neither conscious nor ‘qualitative’, whatever the latter might mean to Helmholtz.

Colour Perception   425 ‘direct’ relation between perceptual cause and effect than is hypothesized by this theory, i.e. a transfer of a property from one object to another. What is striking about Aristotle’s theory (as we said) is that the initial step in visual perception, is not a matter of representing the colours, but of having them transmitted through the senses. The causal results of this transmission are often called ‘sensations’, but to avoid confusion, we will call them ‘sensa’. The sensa give us a direct window onto the world—unless something goes wrong in the transmission. I have a red sensum if and only if there is a red object in front of me. Under standard conditions, and for normal observers, colour experiences are thus necessarily veridical.

3  New science: Secondary Quality Realism Aristotle’s theory did not find favour with the new science, beginning with Galileo, or with the philosophy of the Modern period, from Descartes onwards, that was inspired by it. On the Modern’s view, all causes are efficient causes and all efficient causes are explained in purely mechanistic terms. For Aristotle, every property exists in the world as such— not only the redness of the rose qua sensible species but also the rose-ness of the rose qua substantial form. For the Moderns, the only basic properties that exist in the world, primary properties, are measurable, mathematically describable properties: location, number, motion, mass, and (occasionally) solidity. Of course, there are many other items in the world in addition to primary qualities—properties such as sounds and flavours and fragility, objects such goats and thimbles. These facts must be explained in terms of the only non-reductive properties that the new science endorses, the primary properties. Colours have, since Galileo, been the paradigmatic examples of secondary qualities—i.e. properties that appear to be basic in our perception of the world, but need to be explained or explained away. (Ross, Chapter 21, this volume.) There were several aspects of Aristotle’s view, however, that the Moderns accepted. For one, colour sensa arise from a straightforward causal process, in which the perceiver is passive and there is a correlation between types of cause and of effect. Colours are not transferred from perceptual object to perceiver, nor need the effect resemble its cause, but colour sensa are fully determined by their causes in every particular circumstance. Second, colour sensa are the basic building blocks, the primary data, of vision. If there were no colour sensa, there would be no visual perception just as Aristotle thought. Although the Moderns disagreed on how sensa eventuated in perception, all agreed that vision begins with a visual field of colour sensa from which the perceiver must derive information about the distal world. The fundamental distinction between seeing colours and seeing anything else is preserved. While accepting these basic tenets, the Moderns transform the fundamental question to be asked about sensing in a way that affects almost everything that has been written about it since. Where Aristotle wants to know how colours are sensed, the fundamental question asked by, for example, Descartes in the 6th Meditation is how it is we come to have perceptual experiences of colour. And while Descartes agrees with Aristotle that the perceiver is passive in the production of these experiences since they are simply caused to appear, he emphasizes that it is far from clear what causes them. Famously, he comes to the conclusion that they are caused by objects in the world, or rather the geometric properties of the

426    Kathleen Akins and Martin Hahn world. But in framing the question in this way, the Moderns open up the possibility that there are no colours at all, that colour experiences are not caused by colours in the world. The Moderns’ new question about the nature and existence of colours, as it arises out of the new science of colour, is summarized by Locke. In his extended discussion of secondary properties (1690: Bk. II, Ch. VIII, 10), he gives us three possibilities for what colours are, apparently not quite sure himself which one to choose. Colours are ideas in the mind of the perceiver alone, the powers in things to cause those ideas, or the primary-quality grounds of those powers. The first option eliminates colours as anything but subjective phenomena, and by supposing they exist in the world common sense makes an error. On the second option, colours are irreducibly relational. They are dispositions to cause colour sensa in normal human perceivers. This view is commonly now called Dispositionalism or The Secondary Property View. The final option is that what we perceive as colours are ‘really’ in the objects that we perceive as coloured, and would be there whether or not there were any colour perceivers. Colours are to be identified with the primary properties of objects that cause us to perceive them. In contemporary literature, this view is called Realism or Primary Property Realism. But if there is no doubt that we have colour experiences, and if colours are whatever causes these experiences, how could they fail to exist? Every event, after all, is caused. The answer that they are not real because only primary properties are real will not do here because it is agreed that while the world ultimately consists of only primary properties, this does not mean that buildings, people, fragility, or harmony are not real—only that they are ultimately reducible to or grounded in primary properties. So what prevents us from saying that, since experiences have causes, colours are just those? The reason is that the causes of colour experiences do not resemble colours as they are appear to us in perception. As Descartes says: ‘there need be no resemblances between the ideas which the soul conceives and the movements which cause these ideas’ (Descartes (1988) I, 167, The Optics). Yellow, as it appears to us in perception, does not resemble any object or process defined by means of primary properties. But for all that, for scientists or for any philosopher with a naturalistic bent, not only the existence but also the systematic nature of colour experiences (what C. L. Hardin (1987) has called ‘the structure of phenomenal space’) requires a causal explanation. Colour experiences, as a group, are orderly: some colours are more similar to one another than to others (red to orange vs. red to blue); some colours appear to be ‘pure’ or ‘unique’ (red) while other colours seem to be ‘binary’ or mixtures of several hues (orange = red + yellow); some colours are possible (yellowish green) while others are not (greenish-red). There must, then, be an explanation of the existence and properties of colour experiences in terms of the properties of their causes. This requirement seems to rest on two intuitions. The first is that colour experiences have a function, namely to inform us of the state of the world, or at any rate provide the data from which it is possible to understand it. The experiences, as Descartes argued and modern evolutionary thought concurs, must have an informational function. The second intuition is one inherited from Aristotle: that the sensory organs simply transmit data from the world. Aristotle compares the process to that of a piece of wax receiving the impression of a signet ring (1956: 424a17) and Newton as well as other theorists at the time seem to subscribe to a similar model. One can give up the view that colour experiences resemble their causes but retain the view that sensory transduction is a straightforward case of causal interaction, with no contribution from the perceiver.

Colour Perception   427 But, given that colour experience do not resemble their causes, how can the senses provide the data from which knowledge of the world can be inferred? By second-order resemblance: the similarities and differences in colour experiences reflect the similarities and difference in their causes. If orange is ‘between’ yellow and red, for example, then this relation must be mirrored in the structure of the causes of yellow, orange, and red visual experiences. With the rejection of first-order resemblance, colour experiences must now bear a type-type relation with their causes, and the structure of the physical stimuli must explain the structure of the experiences with which they are causally paired. With this leap, we lose much of the impetus to choose among Locke’s three options. The question of whether the colours are really in the mind alone, or in the world as ‘mere dispositions’ or as the bases of those dispositions, no longer matters: if there is a relation of second-order resemblance, the explanatory task is complete. Whether the normal causes of orange visual experiences should be called ‘orange’ is a matter of terminology. Colours are, on this view, still paradigm cases of secondary properties. If we wish to, we could say that nothing in the world is really yellow—there are only powers to cause experiences of yellow or their primary property grounds. But unlike for other secondary properties, say pains, there is a strict type-type correlation between perceptual experiences and their causes so that we could extend the term ‘colours’ to the latter without incoherence. Thus colours can be said to be really in the world. Red, for example, is whatever property in the world causes, under standard conditions, the experience of redness in normal human observers. Standard conditions and normal observers not only guarantee veridicality, as they did for Aristotle, they define the nature of colours. Newton’s theory of colour, which was the pre-eminent theory of the time, offers a good example of a second-order resemblance account. Newton used two prisms to show that white light is composed of an array of lights of different colours. When white light is shone through a prism, the single beam separates into a rainbow of colours, light that is ordered according to its refrangibility from violet (most refrangible) through red (least refrangible). The differential refrangibility of light by transparent media is explained in turn in terms of the mass (and inertia) of light particles. Because red light consists of particles with the greatest mass, red light will have the greatest inertia on entering a denser medium, and thus it will be the most resistant to the attractive forces of that medium: red light will be the least refrangible. Newton is thus able to explain the colour ordering of the rainbow, from violet to red, purely in terms of the primary properties of light particles, their mass, inertia, velocity, and size. As to why we see the colours of the rainbow, ordered in just this way, Newton simply assumes that light transduction is relation preserving: the differently sized particles of light induce ‘vibrations’ in the nerves that preserve the relation of size—e.g. larger particles induce larger ‘vibrations’—and hence the structure of colours.

4  The decline of naive secondary Quality Realism Today we know that Newton’s Proposition 1 of The Optics (1718)—‘Lights which differ in colour differ also in degrees of refrangibility’—was correct. However, even at the time, it

428    Kathleen Akins and Martin Hahn was becoming apparent that many of the properties that colours appear to have are critically dependent upon the perceiver’s particular system of colour perception. Indeed, as Mollon (2003) suggests, the history of colour science is a history of retraction from Newton’s widely shared assumption that human colour perception can be fully explained in terms of the properties of light plus the laws of optics, a view that was possible only given the shared understanding of sensory transduction as a relation-preserving process. In effect, each new discovery eroded the view that phenomenal colour space was isomorphic with its external causes and forced science to look inwards, into the human visual system itself, for explanations of the origin and order of human colour phenomenology. Historically, there were many discoveries that fuelled this erosion, but for the sake of brevity, only four central examples are given here. First, in 1800, Herschel showed that white light, passed through a prism, produces not only a visible spectrum for violet to red, but also a temperature change on a surface ‘beyond’ the visible red light. This led to the speculation that heat and light were not distinct phenomena but rather form a continuum of refrangible light. If so, then according to Proposition 1, there must be colours—kinds of refrangible light—that we could not see. Second, by the end of the eighteenth century, it was known that people who are colour blind do not lack all colour vision, but rather fail to distinguish certain hues—a fact that begged for an explanation in terms of the individual loss of some aspects of colour reception (cf. Dalton, 1789). Third, there is the fact—and eventual understanding—of the trichromaticity of human colour. Suppose you are given three ‘test’ lights and allowed to adjust their intensity or brightness. If the three lights are ‘independent’—i.e. if you cannot match any one of the lights by mixing the other two while adjusting their intensity—then, with some complications we will omit, you can match any arbitrarily chosen fourth light by adjusting the intensity of the three test lights. Although trichromaticity was not fully understood in the eighteenth century, certain aspects of it were known and used as the basis for various systems of colour ordering and a variety of practical uses. For example, Le Blon distinguished between subtractive and additive colour mixing, and then developed three-colour printing (a subtractive process because each pigment subtracts from the light reflected from the surface) and three-colour (plus black and white) weaving, to achieve the full spectrum of colours (an additive process because composite colours are produced by threads of component colours). Many of the eighteenth-century Trichromacists were strongly anti-Newtonian. Everyone agreed that light with a given colour appearance could be produced by mixing three lights with different colour appearances. Newton held, however, that for every apparent colour, there was light of single refrangibility that could produce/ be identified with it. The problem that Newton never resolved was how the appearance of a single colour could be caused by a mixture of three different lights, given the simple physiology of transduction as he understood it. The Trichromatists believed that light itself was not coloured but rather the appearance of colour arose out of some kind of interaction between, for example, light with darkness or with the medium through which light passed. However, neither the Newtonians nor the Trichromacists could adequately explain all of the facts of trichromacy and of phenomenal colour space as they were beginning to be understood, which in retrospect we know to require the postulation of three different types of retinal receptors. It was not until 1802 that Thomas Young explained how light with a continuous spectrum of wavelengths could act upon a finite set of receptors to produce, by

Colour Perception   429 their interaction, the space of phenomenal colours and thus resolved many of these puzzles: there had to be different kinds of receptors with different spectral sensitivities. Our last example of a discovery that put pressure on the Newtonian account of colour are two related phenomena of colour perception, simultaneous contrast and colour constancy. Simultaneous contrast is the fact that the colour of an object’s background can profoundly change our perception of the object’s colour: a yellowish-grey cross looks yellow on a grey background, but grey on a yellow background. Colour constancy, on the traditional view, refers to the fact that while a single object of constant colour may produce a multiplicity of coloured images on the retina as a result of different illuminants (sunlight, tungsten light, lights filtered by colour), human colour perception is able to ‘discount the illuminant’ to perceive a constant colour. In a wonderful demonstration, Monge (1789) showed that a red cloth on white background does not appear to be of a more highly saturated red when viewed through a piece of red glass, as one might expect, but appears pale, almost white, instead. It will appear redder only if we restrict our viewing area to the cloth itself, without the background. Monge inferred from this (and other phenomena such as coloured shadows) that we see colours in the world not as a result of the absolute colours of the retinal image as Newton supposed, but rather based upon the ratios or contrast between image colours. Thus Monge, and later Young, inferred that simultaneous contrast and colour constancy arise in the same way: different light sources produce retinal images with constant ratios of colours across their surfaces (colour constancy) and altering the ratios through a change in background colour results in a change in colour perception (simultaneous contrast). By the beginning of the nineteenth century, then, there is a sea change in the understanding of the relation between our senses and the world, one articulated by Müller in his theory of specific nerve energies. Aristotle’s original idea, that the sensible species are transmitted from the object to the common sense via the sensory pathways had already been discarded by Galileo and his followers. But Newton, and many after him, retained the idea that there was a type-type correlation between the properties of colour experiences and the properties of their causes. The sensory organs were no longer windows through which qualities of the world were themselves admitted, but transducers that produced orderly arrays of experiences that mirrored the orderly relations between the properties of their causes (i.e. stimulus relations). By 1835, Müller is arguing that the differences between sensory experiences are the result of ‘nerve energies’, internal states of the perceptual system. Thus the difference between two modalities, say, vision and touch, is not explained by the sensory stimuli which typically evoke them, light versus mechanical stimulation, but by the peculiar ‘energy’ of visual versus somatosensory nerves (and/or the areas of cortex that receive them.) This same point applies to differences within the senses, say between the sensations of heat and cold in thermoreception. Says Müller: Sensation is not the conduction of a quality or state of external bodies to consciousness, but the conduction of a quality or a state of our nerves to consciousness, excited by an external cause. (Clarke and O’Malley, 1996: 205)

Enter Hermann von Helmholtz, a pivotal figure in the history of vision science. First, Helmholtz combined Young’s theory of three separately tuned receptors and Müller’s view of specific nerve energies to yield the Young–Helmholtz theory of trichromatic colour perception: the three separate receptors feed separate colour channels which, in combination,

430    Kathleen Akins and Martin Hahn yield the full array of colour experiences. The theory, originally proposed when there was no solid physiological evidence of three kinds of receptors, amounted to an attempt to reposition the facts of additive colour into the brain, an explanation of the production of colour experiences by a neural RGB (red-green-blue) projector. Second, Helmholtz distinguished three separate parts of perception in general, and hence three different realms of explanation: (a) the dioptrics of the eye which included the science of optics proper (i.e the nature of light and its interaction with various media) (b) the theory of sensations (given by the Young–Helmholtz theory for colour sensation)—or what we called sensa, and (c) the theory of the interpretation of the sensations or perception. Unlike his predecessors, Helmholtz did not believe that visual sensations are immediately and fully known to us or even perhaps available to consciousness. But whether the sensations themselves were or were not conscious, Helmholtz argued that the inferences, from the data given in sensation to the perceptual conclusions about the distal world, were both unconscious and learned. The Young–Helmholtz theory soon received empirical support from Clerk Maxwell’s work on colour mixing and subsequent estimates of the spectral sensitivities of the three receptors by Arthur König (König and Dieterici, 1892). The fact that there really are three, differently tuned, types of spectral transducers for daylight vision, the S, M, and L cones, has since been firmly established. But at the same time, Ewald Hering’s careful psychophysical observations lead him to the conclusion that colours, as we perceived them, were actually arranged along two opponent axes:  red/green and blue/yellow. This proposal explained facts such as the absence of reddish green or bluish yellow, but also the phenomenon of complementary colour after-images. Based on this observation, Hering proposed that there were actually six kinds of receptors, arranged in two opponent pairs: the two chromatic pairs plus black and white. As subsequent developments confirmed both Helmholtz’s receptor count and Hering’s view that phenomenal space has an opponent structure, the need to reconcile the two views became pressing. By the end of the nineteenth century, then, most of Aristotle’s account of colour perception was rejected. Colours, or anything resembling them, did not enter the perceiver’s mind as such. This much had been asserted by the Moderns already. But the latters’ assumption that the structure of our phenomenal colour space reflects the structure of its physical causes also turned out to be false. Instead, colours as we see them are filtered and structured by the properties of the transducers in our retina. Furthermore, Helmholtz denied that the sensations (or, more accurately, sensa, as we’ve been calling them) thus produced are necessarily available to the perceiver and argued that the processes that lead from sensa to conscious perception are in general not something of which the perceiver is aware. However, there are two central aspects of Aristotle’s view still to be found in both Helmholtz and Hering’s theories. One is that to see a colour is to have a sensory response to the environment, a sensum. The complications of trichromacy, or the opponent structure of our colour phenomenology, are explained by facts about retinal transducers. After the retina, in accordance with Müller’s theory of specific nerve energies, there is a straight pathway to the sensorium. On Helmholtz’s hugely influential way of making the distinction between sensation and perception, colours fall squarely on the sensation side. Indeed, most of volume ii (On The Sensations of Vision) of Helmhotz’s Treatise on Physiological Optics is devoted to colour sensations. The second point of agreement with Aristotle is that sensations, and thus in the case of vision, colours, form the basis on which all visual conclusions depend. If we did not see colours, we would not see at all.

Colour Perception   431 Helmholtz, although he did not see this with absolute clarity, was the first person to hint at the possibility that the two theses might be equivocating on the word ‘sensation’. There are, in fact, three theses here, not two. The empirical hypothesis, which Helmholtz defends extensively, is that colour sensa are a separable, early stage in visual perception. The hypothesis that perceptual processing is strictly hierarchical is somewhat controversial today; the idea that there is a theoretically fruitful way of pointing to a ‘sensum’ stage even more so. But if Helmholtz were right, then it would trivially follow that without sensa, there would be no perception—no later stages without earlier ones, whatever they are. On the other hand, it is supposed to be an intuitive fact about our visual experience that if we did not see colours, we would not see anything. In denying that sensa are necessarily experienced, Helmholtz is opening the possibility that the two theses are independent of one another. There is a third assumption in the traditional view: that sensa are colour experiences.

5  Modern colour science: Colour for colour There are many thematic routes into modern colour science, but here we will start with four major events in the history of visual neurophysiology between the mid-1950s and 60s that set the course of colour research until the mid-1990s. We will then return to the two remnants of Artisotelian theory. One important event in twentieth-century colour science was Hurvich and Jameson’s paper ‘An opponent-process theory of color vision’ (Hurvich and Jameson, 1957) in which the couple revived the central insights of Hering’s theory. Although the Young–Helmholtz theory had the advantages of parsimony and consequences that were easily tested, the theory could not explain how exactly the three outputs of the opponent processes could be mixed to yield the full range of colours that we experience. Hurvich and Jameson’s paper was both theoretical and practical: it suggested how a retina with three differently tuned cones, feeding into three opponent mechanisms (blue-yellow, red-green, and black-white) could predict and explain the complex properties of human colour psychophysics, e.g. colour matches, the colour dimensions of brightness, saturation, and hue, chromatic adaption, and colour blindness. Although Hurvich and Jameson identified their view with Hering’s, in some sense it adopted elements of the two rival theories. Like Helmholtz, the theory posited only three receptors, but ones that each responded across a broad spectrum of wavelengths. Unlike Helmholtz, who proposed the colour experience is additive, the result of a combination of outputs of the three receptors, Hurvich and Jameson posited that the three cones could have either inhibitory or excitatory effects upon a common cell. This explained how the opponent y-b, r-g, and w-b mechanisms—which pit two signals against each other—were possible. Notably, on Hurvich and Jameson’s view, the outputs of the three opponent mechanisms were to be associated with six sensory qualities. In explaining the diagram of the opponent channels in the 1957 paper, the authors note that ‘the convention of positive and negative signs is used to indicate that each neural system is capable of two modes of response that are physiologically opponent in nature, and that the paired sensory

432    Kathleen Akins and Martin Hahn qualities correlated with these opposed modes of response are also mutually opponent or exclusive’. In other words, green and red are mutually exclusive sensory qualities and we can explain this fact by reference to the opponent processing of cone inputs. A second important event, Kuffler’s discovery (1953) of centre-surround ganglion cells in the mammalian retina, directly addressed the second remnant of the Aristotelian view, that colour sensa (conscious and/or unconscious) are the data of vision per se. Exactly where this view qua scientific thesis began to falter is unclear, but Kuffler was central to this fact. In effect, Kuffler connected Cajal’s detailed anatomy of the mammalian retina with Hartline’s (Hartline, 1948; Hartline, Wagner, et al., 1952; Hartline and Ratliff, 1957; Hartline and Ratliff, 1958) and Granit’s (Granit and Riddell, 1934; Granit and Therman, 1935, 1937) research on the physiology of the non-mammalian retina and of the transmission of retinal signals within the optic nerve. The ganglion cell is the output cell of the mammalian retina, sending signals to the LGN (Lateral Geniculate Nucleus of the thalamus) and, from there, to primary visual cortex. As Kuffler was quick to understand, each ganglion cell encodes—indeed enhances—contrast by comparing the total photon catch of two areas of the retina, a circular ‘centre’ area, and the retinal area immediately around it, a ‘centre-surround’ configuration. Shine a light on the centre of a ‘positive’ centre-surround cell, leaving the surround in darkness, and the cell will respond with a maximal firing rate. Thus a ganglion cell signals a difference in photon catch across a spatial border, not the absolute value of light intensity at one place in the retinal image. About ten years later, De Valois (De Valois, 1965; Jacobs and De Valois, 1965) made the parallel discovery of a chromatic centre-surround cell in the LGN (the sensory ‘way station’ of geniculo-cortical visual system) of the squirrel monkey. This, on De Valois and Jacob’s view, was proof that signals from the retina to cortex had two basic forms: One class comprises those cells that give the same type of response, either excitation or inhibition, to all spectral stimulation; those cells seem to carry information about the brightness of the stimulus. The other class is made up of those that show the chromatic response pattern . . . a modulation of the spontaneous discharge rate, in the direction of excitation in response to some spectral stimuli and inhibition in response to others. (Jacobs and De Valois, 1965: 488)

In other words, neither absolute wavelength nor absolute intensity forms the basis of human vision. But Hurvich and Jameson’s thesis about the root of colour sensory qualities was put to rest only about twenty years later. Despite the opponent nature of the cells in the LGN and hence their prima-facie fit with the structure of human chromatic space, the responses of L-M opponent cells in the LGN do not align with the red-green dimensions of our psychophysical space (Derrington, Krauskopf, et al., 1984). Whatever neural mechanisms are responsible for our seeing red and green as opponent colours, these are not the mechanisms of neural opponency of retinal ganglion cells alone. Retinal output does not determine how we see opponent colours. Finally, perhaps the most important influence on our understanding of human colour perception at this time was Hubel and Wiesel’s theory of the neural architecture of mammalian vision (Hubel and Wiesel, 1959; 1962; 1963; 1977). Starting with the signals of centre-surround ganglion cells in the retina, Hubel and Wiesel presented evidence for a hierarchical system of vision, one that posited the convergence of signals from lower-level cells onto higher-level cells with the result of ever more complex response properties at each new level. At the level

Colour Perception   433 of retinal output and the LGN, each cell has the classic centre-surround organization and thus signals either chromatic or luminance contrast; ‘simple’ cells in V1 respond to edges of the correct orientation and position; ‘complex’ cells combine the signals of the simple cells to produce a response to moving oriented edges of light; the signals of these complex cells in V2 then converge in hypercomplex cells such that they respond to ‘end-stopped’ oriented moving edges across the retina (Hubel, 1963). Beginning with a ‘simple’ response to luminance contrast, then each new cortical level of cell responds to ever more complex spatial patterns of light in the retinal image in an orderly hierarchical arrangement. As with any hierarchically organized system, Hubel and Wiesel faced the thorny question of the visual system’s representational end point(s): what properties of the world do the cells at the top of the hierarchy signal? Certainly on Hubel’s view, this question could be answered in terms of the commonsense properties we attribute to distal objects. For example, in the opening of Livingstone and Hubel’s classic paper, ‘Segregation of form, colour, and stereopsis in primary area 18’ (Livingstone and Hubel, 1987), they simply begin with this statement: Visual perception is commonly thought of as subdivided into various categories, the most obvious of which are form, color, movement, depth, and texture. Although to introspection our vision does not seem particularly piecemeal or fragmented, psychophysics and clinical neurology both strongly suggest that these different components of perception may be carried by separate channels in the visual pathway. More direct evidence has lately come from neurophysiology and anatomy. (Livingstone and Hubel, 1987: 3416)

Thus the properties that we see—form, colour, movement, depth, and texture—are processed by separate components of the visual system. A significant consequence of this modular view is that colour processing receives a rather radical functional demotion: the primary use of chromatic information is just colour perception, not all of vision as had been supposed since Aristotle. For although Livingstone and Hubel reported that colour, as a property of the retinal image, might be useful for ‘linking’ tasks (for maintaining object unity across occlusion and so on) as well as for shape determination to a modest extent, the primary use of chromatic information was for seeing objects as having the property of colour, ‘colour for colouring’. This view is very much in keeping with an understanding of colour phenomenology as a ‘sensory response’ albeit a sensory response that comes about only after several levels of neural processing. At the end of the (experimental) day, at the ‘pinnacle’ of colour processing, one will find a neural area, the Colour Center, that represents the constant colours of objects, their true colours as opposed to their ‘image’ colours (the predominant wavelength of whatever parts of the retinal image are a result of light reflected from the relevant object). Indeed, Zeki reported finding such a Colour Area in area V4 of the human brain in Lueck et al. (1989), ‘The colour centre in the cerebral cortex of man’. To put this point another way, on this new architecture, luminance vision was now seen as responsible for the lion’s share of visual processing, and hence luminance contrast signals were the central data of visual processing. But colour perception itself was still seen as grounded in determinate colour experiences, in the physical instantiation of human phenomenal colour space by a multi-dimensional, systematic neural space of the appropriate kind. If the retina was no longer a logical location for the organization and origin of colour experiences, one needed to look inwards, further into

434    Kathleen Akins and Martin Hahn the brain, for a more congenial location. This was a central assumption of most colour research at this time. Let us return to the two theses of Aristotle that remained standing at the end of the nineteenth century. The first was that colour perception, unlike the perception of goats or even shapes or sizes, was a matter of having a sensory response to the environment, a sensum. The second thesis was that in order to see at all, one had to have perceptual experiences of colour. This second thesis was considered to be true on a priori grounds, not only by Aristotle, but also by later philosophers like Descartes and Berkeley. Even Helmholtz and his contemporaries did not think that the assumption required empirical proof, and one finds it still assumed by many contemporary philosophers, often defended in conversation, and occasionally in print: ‘if we do not see colors, then, intuitively, we do not see things at all. Intuitively, we see the facing surfaces of things by seeing their colors’ (Tye, 2000: 149). It is thus noteworthy that this view was deeply eroded by the empirical discovery of the centre-surround cells and the subsequent rich confirmation of what is one of the central tenets of contemporary visual neuroscience: the ‘data’ of biological vision is not absolute intensity or wavelength or any absolute value of the light stimulus but rather visual contrast of a variety of types, at a variety of spatial scales. Further, as we saw, the chromatic opponency as it is found in the LGN (a candidate for what we have been calling sensa) does not map onto the psychophysically documented facts of opponent structure of our phenomenal space. Hence it seems false that if we did not see the colours we could not see at all. Exactly how this second view became eroded—and what remains of it—is an interesting and subtle issue. Until the last century, the thesis that if we could not see colours we would not see at all was meant to be about all of the colours in phenomenal colour space, both the chromatic colours (red, green blue, etc.) and the achromatic colours from white to black. By the time of Wiesel and Hubel’s famous papers on the architecture of cortical vision, no one thought that luminance vision—and hence the properties of the world visible on the basis of luminance processing alone—required determinate greyscale experiences. No neuroscientist, in seeking to understand human vision, ever looked for the Grey Area, that area of the brain that encoded and gave rise to greyscale experiences, from white to black. Indeed, not even the perception of albedo, the perception of surfaces as light or dark, is thought to require, as its data, determinate achromatic colour experiences. Rather, we see surfaces as light or dark as a result of the complex processes that from the humble beginnings of luminance (and as it turns out, chromatic) contrast, manage to discount the various shadows that fall across objects, the level of ambient illumination within the scene, and so on (cf. Akins, 2014). In other words, the standard view for over fifty years now in neuroscience is that luminance vision uses luminance contrast to see the various properties of the world, including the property of lightness/darkness. Achromatic colour experiences are not a part of that causal chain. What about chromatic colour experiences? Are there visual tasks that we could not perform without seeing colours? Consider the Ishihara colour plates used to test for colour blindness. Each card consists of a field of coloured dots, some large, some small, some green, and some red. Embedded within each field of green dots is a number, say ‘47’, composed of red dots. Since the luminance values vary randomly across all the dots, the only property that differentiates the number from its background is the hue of the dots. Do you need to see the colour of the dots in order to see the number? There is a strong temptation

Colour Perception   435 to insist that Aristotle must be right about this case, for surely if you could not see the colours of the dots, you could not see the number ‘47’. It is the perception of the colours, red and green, that differentiates figure from ground. So, whatever one might say about the necessity of chromatic perception in general, isn’t it necessary at least for the segregation of the ‘47’? Finally, there is the issue of the first ‘remnant’ of Aristotle’s view, that colour perception is essentially a matter of a sensory response. We saw above that there are really two questions here. One is whether there is a point in the stages of visual processing which is the end point of something that might be fruitfully called sensory as opposed to perceptual processing and which supplies the material for all processing of the latter kind. The other is whether it is at (or by the time processing reaches) this point that colour experience is produced. The Colour Area proposed by Zeki (Lueck, Zeki, et al., 1989) satisfies both conditions. This Colour Area will instantiate the ‘colours as we see them’, which are different from the traditional sensations in that the effect of the illuminant will be discounted prior to the Colour Area to ensure that green grass looks the same colour to us at noon and at dusk. But whatever complications that occur prior to the Colour Area, the end result is the selection and instantiation of the right colour experience—the one which would occur under standard or white light. Indeed, that is what veridical colour perception is on this view. So even though the steps required prior to colour perception are far more complicated than what Aristotle, Helmholtz, or Hering envisioned, colour vision is understood primarily as a sensory response even in the later years of the twentieth century.

6  Colour perception and colour ontology A tremendous stroke of good fortune affected philosophy at about this point in the history of colour science: the publication of C. L. Hardin’s Color for Philosophers: Unweaving the Rainbow (1987). Philosophers have always been interested in colour, and since Locke, colour has played a central role as the prime example of a simple idea, sense datum, secondary property, the given, etc. It was Hardin’s book that brought colour science, in its contemporary form, to the attention of the philosophical community. And it was also Hardin’s book that brought to our attention the complexity of the relationship between colour experiences and their causes, a complexity that makes colour realism a much more difficult position to defend. We saw earlier that if one accepted Newton’s view that colour perception is based on second-order similarities between our sensations and their causes, then choosing between Locke’s three alternatives for colour ontology—primary properties, secondary properties, or the basis of secondary properties—becomes less pressing. Strictly speaking, only contents of experiences can be red. But we can legitimately call ‘red’ those surfaces in the world which, under standard conditions, cause red experiences in normal observers. What Hardin provided in his book was a long list of reasons why there is no straightforward correspondence between colour experiences and what we would like to call ‘colours’ in the world, e.g. surface spectral reflectances. The physical causes of colour perceptions are many and varied; the standard conditions that would need to be articulated in order to specify colours as dispositions to cause experiences are difficult to articulate in a

436    Kathleen Akins and Martin Hahn non-arbitrary way, as is the notion of a normal observer that would be needed; the facts of colour constancy and of simultaneous contrast are bewilderingly complex; colour categorization not only does not fit easily onto the facts about wavelength and surface properties, but is in fact culturally determined, and so on. There seems to be nothing in the world that could be called ‘red’ or ‘green’ in good conscience, once we know all the facts. With Hardin’s challenge, the logical space of the colour ontology debate seems to have been defined. One can accept the challenge to explain or explain away the prima-facie mismatch between colour perceptions and their causes. In other words, one needs to show that, despite all of Hardin’s evidence, our phenomenal colour space matches some aspect of the world well enough for the latter to constitute colours. Or one can accept there are no colours in the world but then needs to explain how it could be that colour vision, as a large and predominant component of human vision, came into existence although it does not provide systematic information about anything. The first is the Realist’s problem; the second is the Error Theorist’s task. There are other options: dispositionalism, primitivism, relationism, and more, but the challenge to all is the same: there are colour experiences on the one hand, their causes on the other, and we must account for the messy relationship between the two in a way at least minimally consistent with modern science. The framing of the problem ought to give us pause. There are goat experiences and size and shape experiences too. But it would be very odd to claim that whether or not goats or shapes or sizes are real features of the world is to be determined by whether we can find anything in the world which is related systemically enough to the content of our goat, shape, or size experiences. Odder still, to claim that what goats, or specific sizes and shapes are, is constituted by whatever appears to be goats or specific shapes or sizes in standard conditions. Put it another way, the average philosopher will expect to learn about colour ontology in a section on colour in this volume, but no one expects to find a section on goats in this book. And should there be a section on shape or size, one would expect it to be about shape and size perception and not expect to find anything there about whether shapes or sizes are real properties of the world.

7  An alternative path Over the last fifteen years a different account of colour perception has begun to emerge from psychophysics. This work starts with the very basic assumption that when light interacts with media of any kind—water, air, opaque, or transparent material—both its wavelength and intensity undergo changes in accordance with the laws of optics. Light is selectively absorbed, scattered, and refracted as a function of wavelength. The spatial distribution of both intensities and wavelengths of light in the retinal image provides information about the material composition, surface texture, shape, and depth properties of the distal scene. Add to this the fact that almost every object in the world differs from its background in both intensity and wavelength, while shadows cast by those objects differ in intensity alone. It would thus be quite easy to separate objects from the shadows they create, by looking for borders defined by both wavelength and intensity or simply by intensity in the retinal image. On the whole, then, if a visual system encoded both the wavelength

Colour Perception   437 and intensity of the reflected visual image, this would represent a huge boost to its informational resources. Although De Valois (1957) assumed that centre-surround cells encode ‘brightness contrast’ and ‘colour contrast’, we now realize that luminance and chromatic ganglion cells have far more complex informational properties. A  luminance cell does not respond to intensity contrast alone. Luminance cells are chromatically sensitive:  An L + M centre-surround luminance cell will respond more vigorously to a green light in its centre than to a blue light, even if both lights have the same intensity. Similarly chromatic cells do not require colour contrast per se to respond: an L-M chromatic cell will respond positively to a ‘white’ light in its centre. So the informational properties of chromatic and luminance cells are neither straightforward nor simple. What we now know, however, is that the luminance and chromatic systems of human vision, for all their complexity, access different aspects of the visual image. If you compared the edges the luminance system (M + L + S) can discern with the edges to which a Red/Green (L – M) cell responds, those two sets of edges would be both different and independent, one of the other. In other words, the two systems seem to be complementary, even if they do not abide by the neat divide of light into its natural dimensions of wavelength and amplitude. Moreover, by combining the luminance and chromatic signals within a common mechanism, or by deploying independent chromatic and luminance mechanisms combined with a ‘winner takes all’ strategy, a visual system stands to gain. Add a ‘colour’ system, and we can see more of the world, more quickly, and with greater reliability—more form, motion, depth, and texture both more accurately and quickly. And indeed, contra to Hubel and Livingstone, this is what the past decade or so of psychophysics have confirmed. In human vision, both luminance and chromatic contrast information are used for seeing, for the multitude of visual tasks we can perform. Of course, this does not mean that chromatic and luminance information are used in exactly parallel ways. After all, chromatic and luminance edges are independent. But it does suggest there are two parallel systems, the chromatic and luminance systems, in human vision and that the primary data of vision consist of both chromatic and luminance contrast. Let us return to the two remaining Aristotelian theses. Treating chromatic processing as a parallel system, similar to luminance processing, has the consequence that colour perception can now be treated as parallel to albedo (surface lightness) perception. There is no place in visual processing about which contemporary vision scientists would want to say: here is where sensory response eventuates, and all perceptual processing starts thereafter. Visual processing starts right at the retina, and continues in a highly complex, not strictly hierarchical, manner to eventuate in whatever conclusions are required for getting around the world and building representational models of it. If there is a sensorium, no one has found it, nor are they looking for it. A fortiori, then, the sensorium does not contain any experiences of surface properties such as colours. However it is that we are able to see the Ishihara plates, it may or may not be the case—and is certainly not a priori true—that we must first see the colours of the dots. After all, the only information that is needed to see the figure is that there are two groups of dots, which can be defined by their ‘signed’ (positive or negative) chromatic contrast with their white background. And information about the existence of signed contrast, unlike information about surface colour, is in the system from very early on.

438    Kathleen Akins and Martin Hahn Once we start thinking in this new way, we will come to the conclusion that the majority of really important visual tasks—scene segmentation, detection of movement, location of the agent and various objects in space, etc.—do not require us to determine the surface properties of objects. You don’t have to see that the berries are red for them to ‘pop out’ from the leaves—you only need to see that they contrast with the background. It is true that there could not be any chromatic contrast between the berries of the leaves if they did not have surface colours, nor could the visual system process for this contrast unless there were a special distribution of light of different wavelengths on the retina. But it is the contrast that is encoded at the retina, and neither the absolute values of the wavelengths in the retinal image nor the surface colours of objects need to be calculated in order to for the berries to pop out. Determining the surface properties of objects is a rather specialized task and of only rather specialized utility, largely to do with the identification and reidentification of objects, one would guess. It is a non-trivial computational task, certainly no less complex than the perception of shape or form. Because all perceptions are underdetermined by the data, all require hard-wired ‘assumptions’ and learned statistical regularities—whatever ‘tricks’ will add additional content to the encoding of the retinal image itself. In the case of colour perception, the visual system must ‘discount the illuminant’ or disambiguate the contribution of the light source and the distal surface to the wavelength composition of retinal image. But this is just the beginning of computing surface colour. The visual system must also take into account such scene properties as surface texture, shading, shadowing, transparency, spectral inter-reflection from nearby objects, specular reflection, and depth. Viewed this way, colour perception is a reasonably high level task requiring much information about the distal world. It is simply not the case that blue objects are all and only those that appear blue under normal conditions. On this alternative view, the perception of colour as an objective property of the world is no different from the perception of shape or even of a goat. There is no reason to conclude that colour ontology and colour categories are essentially tied to colour experiences, any more than the perceptions of shape or of animals are tied to shape or animal experiences. Contrary to the facts of our phenomenal space, we could find out that purple qua a colour of the distal world is not equidistant between blue and red, that brown and orange are extremely closely related to one another, that red is closer to heat than it is to violet (no matter how nonsensical that sounds to our ears). There are facts about colours, and then there are facts about how we perceive colours. And there is no a priori relation between them, although there are many empirical ones.

References Akins, K. (2014). ‘Black and white and colour’. In R. Brown (ed.), Conscousness Inside and Out: Phenomenology, Neuroscience, and the Nature of Consciousness (pp. 173–224). London: Springer. Aristotle (1956) De Anima, ed. W. D. Ross. Oxford: Clarendon Press. Chalmers, D. (2006). ‘Perception and the Fall from Eden’. In Gendler and Hawthorne (eds). Perceptual Experience. Oxford: Clarendon Press.

Colour Perception   439 Clarke, E. and O’Malley, C. D (1996). The Human Brain and Spinal Cord: A historical study illustrated by writings from antiquity to the twentieth century. San Francisco: Norman Publishing. Cohen, J. (2004) ‘Color Properties and Color Ascriptions: A Relationalist Manifesto’. The Philosphical Review, 113(4), 451–506. Dalton, J. (1789). ‘Extraordinary facts relating to the vision of colours’. Memoirs of the Literary and Philosophical Society of Manchester, 5, 28–43. Derrington, A. M., Krauskopf, J., et al (1984). Chromatic mechanisms in lateral geniculate nucleus of macaque. Journal of Physiology, 357, 241–265. Descartes, R. (1988). The Philosophical Writings of Descartes. Cambridge:  Cambridge University Press. De Valois, R. L. (1965). ‘Analysis and coding of color vision in the primate visual system’. Cold Spring Harbor Symposia on Quantitative Biology, 30, 567–579. Granit, R. and Riddell, L. (1934). ‘The electrical responses of light- and dark-adapted frogs’ eyes to rhythmic and continuous stimuli’. Journal of Physiology, 8(1), 1–28. Granit, R. and Therman, P. (1935). ‘Excitation and inhibition in the retina and in the optic nerve’. Journal of Physiology,83(3), 359–381. Granit, R. and Therman, P. (1937). ‘Excitation and inhibition in the off-effect of the retina’. Journal of Physiology, 91(2), 127–139. Hardin, L. (1987). Color for Philosophers:  Unweaving the rainbow. Indianapolis:  Hackett Publishing. Hartline, H. (1948). ‘Retinal action potentials of photoceptor cells and the discharge of nerve impulses in their axons’. American Journal of Medicine, 215(6), 714. Hartline, H. and Ratliff, F. (1957). ‘Inhibitory interaction of receptor units in the eye of the Limulus’. Journal of General Physiology, 40(3), 357–376. Hartline, H. and Ratliff, F. (1958). ‘Spatial summation of inhibitory influences in the eye of Limulus, and the mutual interaction of receptor units’. Journal of General Physiology, 41(5), 1049–1066. Hartline, H., Wagner, H., et al. (1952). ‘The peripheral origin of nervous activity in the visual system’. Cold Spring Harbor Symposia on Quantitative Biology 17, 125–141. Hubel, D. (1963). ‘The visual cortex of the brain’. Scientific American, 209(5), 54–63. Hubel, D. and Wiesel, T. (1959). ‘Receptive fields of single neurones in the cat’s striate cortex’. Journal of Physiology, 148, 574–591. Hubel, D. and Wiesel T. (1962). ‘Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex’. Journal of Physiology, 165, 106–154. Hubel, D. and Wiesel, T. (1963). ‘Shape and arrangement of columns in cat’s striate cortex’. Journal of Physiology, 1965, 559–568. Hubel, D. and Wiesel, T. (1977). Ferrier lecture: Functional architecture of macaque monkey visual cortex. Proc R Soc Lond B Biol Sci. 198, 1–59. Hurvich, L. M. and Jameson, D. (1957). An opponent-process theory of color vision. Psychological Review, 64, Part 1(6), 384–404. Jackson, F. (1996). ‘The Primary Quality View of Color’. Philosophical Perspectives, 10, 199–219. Jacobs, G. H. and De Valois, R. L. (1965). ‘Chromatic opponent cells in squirrel monkey lateral geniculate nucleus’. Nature, 206(4983), 487–489. König, A. and Dieterici, C. (1892). ‘Die Grundempfindungen in normalen und anomalen Farbensystemen und ihre Intensitätsverteilung im Spektrum’. Zeitschrift für Psychologie, 4, 241–347.

440    Kathleen Akins and Martin Hahn Kuffler, S. W. (1953). ‘Discharge patterns and functional organization of mammalian retina’. Journal of Neurophysiology, 16(1), 37–68. Livingstone, M. S. and Hubel, D. H. (1987). ‘Psychophysical evidence for separate channels for the perception of form, color, movement, and depth’. Journal of Neuroscience 7(11), 3416–3468. Locke, J. (1690). An Essay Concerning Human Understanding. London. Lueck, C. J., Zeki, S., et al (1989). ‘The colour centre in the cerebral cortex of man’. Nature, 340(6232), 386–389. Mollon, J. (2003). The origins of modern color science. The Science of Color, 2, 1–39. Monge, G. (1789). ‘Mémoire sur quelques phénomènes de la vision’. Annales de chimie, 3, 131–147. Müller, J. P. (1837–43). Elements of Physiology London: Printed for Taylor and Walton. Newton, I. (1718). Opticks, or, A treatise of the reflections, refractions, inflections and colours of light (2nd edn with additions). London: Printed for W. and J. Innys. Sorabji, R. (1971). ‘Aristotle on demarcating the five senses’. Philosophical Review, 80, 55–79. Tye., M. (2000). Consciousness, Colour, and Content. Cambridge, MA: MIT Press. Zeki, S. (1983). ‘Colour coding in the cerebral cortex: The reaction of cells in monkey visual cortex to wavelengths and colours’. Neuroscience, 9, 741–765.

Chapter 23

Perception a n d  Space * Jérôme Dokic

1 Introduction The relationship between perception and space is an extremely difficult topic, which has loomed large in philosophy as a whole, even before Kant put it at the centre stage of the discussion. Any reasonably detailed theory of spatial perception is bound to have far-reaching implications for our conception of the mind and its place in the world, the nature of the self, the possibility of objective thought, and the links between perception and action. It should also address the delicate methodological question of the division of labour between philosophy and psychology, and more generally cognitive science. A central problem in the study of perception is to determine the nature of the perceptual field and its relationship to physical space. There is a minimal, formal sense in which the perceptual field is a kind of space. The perceptual field imposes upon objects and positions a set of relations characteristic of space, and this set of relations corresponds to a mathematical structure that defines a perceptual space. However, the question arises as to whether the perceptual field should be conceived as being spatial in a more substantial sense, either as a portion of a much larger physical space or as a non-physical, purely phenomenal space. Alternatively, one might argue that the proper way of talking about the perceptual field is as a perceptual representation of physical space. In this chapter, I would like to relate these issues to the notion of a frame of reference, focusing on our ability to perceive positions, whether occupied or not by perceived objects.1 As Steven Pinker has observed, frames of reference are inextricable from the very idea of position (Pinker, 1997, 262). Basically, a frame of reference involves the selection of reference entities, namely the perceived object or position and some origin, and determines the way in which they are spatially related. An influential thesis has been that perceptual experience essentially involves an egocentric frame of reference, as it constitutively exploits spatial relations to the *  Thanks to Margherita Arcangeli, Mohan Matthen, and Hong Yu Wong for extended most helpful comments on earlier versions of this chapter. 1  Our ability to perceive positions is probably parasitic on our ability to perceive material objects, but it does not follow that we can perceive only occupied positions. In this chapter, I leave aside the problem of the exact relationship between these two perceptual abilities.

442   Jérôme Dokic perceiver’s body. The egocentricity of perception has been emphasized within both the phenomenological and the analytic traditions, but as we shall see, there are reasons to think that its importance has been overrated. Perceptual egocentricity might well be a myth worth abandoning, at least in some of its influential varieties. The chapter is structured as follows. The next section is about what I  shall call ‘the Orientation Challenge’, which is the challenge of accounting for the apparent fact that the perceptual field is an oriented space, with a right and left, an above and below, and a front and behind. Then I introduce a simple view about the nature of perceptual orientation, which has been endorsed (in different forms) by both Bertrand Russell and Ludwig Wittgenstein. After having argued that the simple orientation view cannot be motivated by considerations about perceptual phenomenology, I turn to a sophisticated version of the orientation view. On this version, the orientation of the perceptual field is a matter of the egocentric way positions are perceptually presented. It has been defended by Maurice Merleau-Ponty, Gareth Evans, Chris Peacocke, and many other contemporary philosophers. However, the sophisticated orientation view faces an objection of intellectualism; that is, it seems to demand too much of naïve perceivers. This objection leads to a more radical view, according to which the perceptual field is not really oriented. The apparent egocentric orientation of what we perceive might be a matter of post-perceptual processes, and more precisely the result of our spontaneous inclinations to memorize spatial configurations using egocentric notions. This yields a new way of considering the constitutive links between perception and action. Finally, I discuss the phenomenon of position constancy, which suggests that perceived positions are in fact relative to an allocentric frame of reference, however these positions are perceptually presented to the subject.

2 The Orientation Challenge According to what seems to be a plain phenomenological fact, the perceptual field is oriented, in two related senses.2 First, there is a phenomenal difference between seeing a figure such as ‘A’ and seeing the same figure in a different relative orientation, say as ‘∀’, even if all other things are perceptually equal. Second, our conscious experience enables us to differentiate between incongruent counterparts, such as a left hand and a right hand (considered as perfect 3D mirror images of each other).3 The same ‘hand’, considered in purely relational spatial terms, can be seen in two different ‘intrinsic’ orientations, and what it is like to see one (a left hand) is different from what it is like to see the other (a right hand), independently of their orientation relative to the rest of the perceptual scene. The orientation of the perceptual field imposes a constraint on theories of spatial perception. These theories should at least meet ‘the Orientation Challenge’, that is, they have to explain how it is that our perceptual field is oriented in both the relative and the intrinsic senses. For instance, a purely descriptivist account of spatial perception is unable to meet 2 

See Van Cleve and Frederick (1991). As Kant observed, to have an incongruent counterpart or be an ‘enantiomorph’, an object must not consist of ‘two halves which are symmetrically arranged relatively to a single intersecting plane’ (Kant, 1992, 370). Roughly, it has to be asymmetrical along each of the three perpendicular axes defining threedimensional space. 3 

Perception and Space   443 this challenge. Any correct general description of a left hand in terms of topological and metrical relations among its parts would suit a right hand as well. So if seeing a left hand consisted in applying such a description, there could not be any phenomenal difference between seeing a left hand and seeing its incongruent counterpart. In what follows, I shall present three views of spatial perception that aim at meeting the Orientation Challenge. Let us call ‘orientation terms’ expressions like ‘up/above’, ‘down/ below’, ‘left’, ‘right’, ‘front’, ‘behind’, and their compositions. According to the first two views, namely the simple orientation view and the sophisticated orientation view, orientation terms are necessary for a correct description of the phenomenology of perceptual experience. The perceptual field is necessarily oriented in the sense that we perceive objects as being up or down, to the left or to the right, in front or behind. Slightly more precisely, we perceive objects as being situated in a three-dimensional space where each of the three perpendicular axes has two phenomenally differentiated directions. For instance, we can see an object a as being to the left of another object b, where the perceived spatial situation is phenomenally different from a situation in which, ceteris paribus, a is now to the right of b. Moreover, a left hand can be phenomenally distinguished from a right hand, since when the palm is in front and the fingers point up, the thumb can be seen as pointing to the left rather than to the right. The main difference between the first two views is that the simple orientation view implies that the perceptual field is itself an oriented space, ontologically separate from real, physical space, whereas the sophisticated orientation view locates the orientation of the perceptual field in the manner in which locations in the physical world are perceptually presented to the subject. Finally, I shall introduce a third view, namely the no-orientation view, which offers an alternative explanation of the orientation of the perceptual field. On this view, orientation terms are not required to capture the phenomenal difference between a single perceived object in different relative orientations, or the phenomenal difference between two incongruent counterparts. The Orientation Challenge can be met, but by making more modest and less intellectualist assumptions about our ability to perceive positions.

3 The simple orientation view According to the simple orientation view, orientation terms denote what we may call ‘orientation properties’, namely phenomenal position properties of the perceptual scene. The simple orientation view has been defended, among others, by Wittgenstein (1975), Russell (1948), and Goodman (1951), although as Falkenstein (1989) notes, it goes back at least to Hermann Lotze and his theory of local signs. For example, Wittgenstein writes in Philosophical Remarks:  We can also say visual space is an oriented space, a space in which there is an above and below and a right and left. And this above and below, right and left have nothing to do with gravity or right and left hands. It would, e.g., still retain its sense even if we spent our whole lives gazing at the stars through a telescope. (1975, §206)

444   Jérôme Dokic And here is Russell in Human Knowledge: At every moment, what is in the centre of my visual field has a quality that may be called ‘centrality’; what is to the right is ‘dexter’, what to the left ‘sinister’, what above ‘superior’, what below ‘inferior’. These are qualities of the visual datum, not relations. (1948, 316)

Both Wittgenstein and Russell stress that orientation properties cannot be analysed in terms of relations to anything else, including the perceiver’s body. We can see an object to the left without seeing or perceiving it as being to the left relative to ourselves. As Wittgenstein makes clear, our body may not even be a visible object at all. Orientation properties are intrinsic properties of visual space (Wittgenstein) or visual objects (Russell).4 Proponents of the simple orientation view need not claim that the perception of position is achieved only through orientation qualities. For example, the question of which orientation quality is attached to a given object is constrained by its place in a certain order determined precisely by spatial relations. For example, it is not possible that an object a which is seen to the left of another object b be seen to the right of a third object c which is at the same time seen to the right of b. Slightly more formally, the phenomenal qualities that determine position in visual space are ordered in a similarity space that determines closeness in visual space. What the simple orientation theorist does claim is that the position of sensible objects cannot be fully determined by spatial relations.5 Indeed, the simple orientation view offers a straightforward description of what happens when the visual field, for example, is suddenly inverted, as when the subject puts on modified spectacles which transpose up and down, such as Stratton’s goggles in his famous experiments (Stratton, 1897). These goggles do not alter the spatial relations between visible objects: every visible object maintains its spatial relation to every other. What is changed is only the distribution of the orientation qualities over the visible objects: for instance, the objects which were (more or less) above a moment ago are now (more or less) below, and vice versa. Only the orientation properties of visible objects could explain the phenomenal difference between the visible figure ‘A’ and the same figure seen in a different orientation, say ‘∀’. The same holds for the phenomenal difference between seeing a left hand and its incongruent counterpart. Wittgenstein explicitly draws the connection between the orientation of the visual field and the fact that visual positions and motion are ‘absolute’:  Couldn’t we imagine a visual space in which we would only perceive spatial relations, but not absolute positions? [. . .] I don’t believe we could. [ . . . ] In visual space there is absolute position and hence also absolute motion. (1975, §206)

Independently of the study of perception, what is usually meant by ‘absolute position’ can be explicated, at least as a first approximation, in terms of the following supervenience claim:6 (A) The position of an object o in a space S does not supervene on the set of spatial relations that hold among o and other objects in S. 4 

For a contemporary defence of Russell’s position, see Casullo (1986). On this issue, see the debate between Casullo (1989) and Falkenstein (1989). 6  An analogous claim can be formulated in the case of motion, but here I focus on position. 5 

Perception and Space   445 According to (A), the position of an object in a given space can change while the set of its spatial relations to other objects in that space remains constant. Wittgenstein suggested that this is precisely what is going on in the visual field:  Think of the image of two stars in a pitch-black night, in which I can see nothing but these stars and they orbit around one another. (1975, §206)

Obviously, the only two things that are seen in this scenario, namely the stars, can orbit around one another in such a way that they maintain their visual spatial relationship to each other. Still, the stars are seen as moving, and thus as changing positions. So Wittgenstein’s example is supposed to illustrate a special case of (A), where S is the visual field:  (Av) The position of an object o in the visual field does not supervene on the set of spatial relations holding among o and other visual objects.

However, it is not clear that (Av) entails by itself that we can see absolute positions. In order to secure this entailment, and make (Av) a special case of (A), one has to assume that the visual field is a space of its own, rather than the visible part of a larger space. Indeed, Wittgenstein made this assumption in Philosophical Remarks because he was interested in a purely phenomenal description of the visual field, according to which visual objects can only bear spatial relations to other visual objects. To begin with, even if the visual position of an object does not supervene on the set of visual spatial relations, it might supervene on the wider set of spatial relations between the seen object and other perceived objects. Perhaps we should take into account cross-modally accessible spatial relations, such as spatial relations between visual objects and our body as we feel it through proprioception, conceived as a form of perception ‘from the inside’ of bodily posture. It might then be suggested that when we see a given star moving in a circle, we perceive it as changing its position relative to a network of spatial relations involving our body. For instance, while the star was perceived to be closer to our head a moment ago, it is now perceived to be closer to our feet. So Wittgenstein’s scenario alone cannot be used to motivate the claim that the positions we see the stars as having are absolute positions in the visual field. As far as visual phenomenology goes, these positions might be relative to objects outside the visual field, which would explain why we do not see these positions as being so relative. At this point, one might try to generalize (Av) to spatial relations holding among perceptually and not only visually accessible objects (i.e. S is now taken to be the perceptual field as a whole):  (Ap) The position of an object o in the perceptual field does not supervene on the set of spatial relations holding among o and other perceived objects.

It is not clear what kind of perceptual scenario would illustrate (Ap), i.e. what would be the analogue of Wittgenstein’s two stars scenario in this case, but even if (Ap) were independently justified, it would not by itself entail that we can perceive absolute positions in the perceptual field. Again, such an entailment would depend on the further assumption that the perceptual field is a space of its own, rather than the perceived part of a larger

446   Jérôme Dokic space. For instance, one might acknowledge that even if perceived position does not supervene on the set of all spatial relations between perceived objects, it supervenes on the much wider set of real, physical spatial relations between the perceived object and other objects, whether perceived or not. Then, the fact that we do not perceive the positions of the stars as being relative does not entail that they are absolute. Of course, this makes sense only if the perceptual field is conceived in a realist fashion, as a small portion of a larger space of mind-independent objects. To sum up, the simple orientation view vindicates a strong claim about the nature of perceptual orientation, but it is not sufficiently motivated by the phenomenology of perception. In particular, more premises are needed to motivate the conclusion that what orientations terms capture are absolute positions in the visual, or more generally perceptual, field.7

4 The sophisticated orientation view The sophisticated orientation view can be found, for example, in the writings of Merleau-Ponty, Evans, and Peacocke.8 Although these philosophers too start from the claim that the perceptual field is essentially oriented, their account is radically different from the simple orientation view, as it moves away from the notion of perceptual field as a genuine space to a view about perceptual presentation or representation of space.9 The sophisticated orientation view explains the apparent orientation of the perceptual field at the level of the perceptual modes of presentation rather than at the level of the perceptually

7 

One might insist that the visual positions of the stars are independent of their spatial relations to the body, on the grounds that the visual phenomenology would be the same as that of a disembodied person seeing two stars orbiting around one another. However, if it turns out that (Av) can only hold in a situation in which the seen stars can change their spatial relations to other, unseen or even unperceived objects or frames, the thought-experiment involving the disembodied person may involve some kind of cognitive illusion. Thanks to Mohan Matthen for raising this issue, which obviously needs more discussion than I can provide here. 8  See below, though, for some caveats about Merleau-Ponty. Following the standard interpretation (see, e.g., Briscoe, 2008), I take Peacocke to be a proponent of the view that perceptual experience represents physical locations as being relative to an egocentric frame of reference. However, Peacocke has developed over the years a complex theory of spatial perception which I cannot give justice to here. In Peacocke (1986, 1989), he argues for the existence of ‘manners of perception’. For instance, the same physical distance can be seen in two different manners, and thus fail to be experienced as the same. He also claims that the representational content of visual experience is given, at least in part, by what he calls a ‘scenario’, whose individuation involves specifying ways of ‘filling out’ space around the perceiver (Peacocke 1992). It follows that visual experience represents physical space using an egocentric frame of reference, but one of the main lessons of this chapter is that the phrase ‘using an egocentric frame of reference’ is ambiguous. In particular, visual experience can ‘use’ an egocentric frame of reference without representing objects and locations as having specific spatial relations to the perceiver’s body. 9  Phenomenologists such as Merleau-Ponty would balk at talk of perceptual representations, but would allow for different ways of presenting a given object or position in perceptual experience. This is all that we need for the present discussion, so I shall ignore the otherwise important distinction between perceptual presentation and representation.

Perception and Space   447 presented world itself. As a consequence, orientation terms do not denote intrinsic orientation properties of the perceived scene. Rather, they are used to show the special way in which our perceptual experience presents the world to us. For instance, Merleau-Ponty makes clear in his Phénoménologie de la perception that the world in itself is not oriented:  One cannot take the world and oriented space as given along with the contents of sense experience or the body in itself, since experience in fact shows that the same contents can be successively oriented in one direction or another. (1945, 285; page references are to the French edition)

Then, when the subject wears inverting spectacles, ‘all the objective relations of the body and of the environment are preserved in the new scene’ (1945, 286). In a similar spirit, Evans urges that the spatial information embodied in experience ‘is not information about a special kind of space’, but ‘a special information about space’ (1982, 157). Even though orientation terms have to be given a special, subjective sense (which, as we shall see later, is intimately linked to dispositions to action), their referents are impersonal positions in the objective world (1985, 386). The very same positions could be fully specified without the use of orientation terms, although of course such a specification would not highlight the specific way in which physical space is presented in perception. According to the sophisticated orientation view, the orientation of perception is phenomenally accessible to the perceiver—in the sense that it makes a difference to ‘what it is like’ to have the conscious experience—but it does not belong to what is perceived, i.e. the objective contents of experience. Thus, as far as perception of space is concerned, not all of what is phenomenally accessible to the subject through her experience is to be related to the perceived world itself. Now this last claim may seem odd in the case of Merleau-Ponty. Indeed, he draws a distinction between the ‘phenomenal world’ and the ‘world in itself’, i.e. the objective world available to the reflective subject, for instance the physicist. So it seems that he could accept a version of the accessibility claim: everything that is phenomenally accessible to the perceiver is to be related to the world as it is perceived, i.e. the phenomenal world. Nevertheless, I take it that Merleau-Ponty’s notions of ‘phenomenal world’ and ‘world in itself’ denote different points of view on the same world, and not ontologically separate worlds. Whatever the merits of this interpretation for the understanding of the Phénoménologie de la perception as a whole, it seems at least consistent with the quoted passages. It is also consistent with Merleau-Ponty’s claim that the phenomenal world is genetically and logically prior to the world in itself. Thus, unlike the simple orientation view, the sophisticated orientation view can acknowledge a sense in which the perceptual field is a proper part of real, physical space. One might say that the term ‘perceptual field’ exhibits something like a sense/reference ambiguity. It can be used to refer either to the perceiver’s point of view, i.e. her mode of presentation of physical space, or to a proper part of physical space as it is presented to the perceiver. The important point is that the issue of perceptual orientation arises only at the former level; the perceived space itself need not be really or objectively oriented.

448   Jérôme Dokic

5 Two kinds of subject-relativity On the sophisticated orientation view, orientation terms denote objective spatial relations rather than intrinsic phenomenal properties. For instance, if we are facing each other, your left and my right would denote the same objective direction. The difference between us concerns the way this direction is perceptually presented. Proponents of this view argue that spatial positions and relations are perceptually presented in a perceiver-relative way. In other words, they are presented according to an egocentric frame of reference, considered as a network of spatial relations whose origin is the perceiver or her body. Here is how Bill Brewer, for instance, puts the point:10 Perceptually presented locations are therefore uniquely determined only egocentrically, as up/down, left/right, in front/behind, where these determinations are essentially subject-relative. For the place perceptually presented as to the right a little up and in front, say, is that to the perceiver’s right, a little above and in front of him. Hence perceptual reference to particular places is essentially subject-relative, in the sense that displayed locations are identified relative to the subject making the reference himself. (1999, 189–190)

However, the claim that we perceive the position of an object using an egocentric frame of reference is not entirely clear. To begin with, does it entail that we see this position as being relative to ourselves? If it does, then Wittgenstein’s point surfaces again: we see the stars as moving relative to each other, and perhaps also to a fixed visual environment, but we do not see them as successively occupying different positions relative to ourselves. More recently, a similar worry has been expressed by Perry:  I see a cup of coffee in front of me. I reach out, pick it up, and drink from it. I must then have learned how far the cup was from me, and in what direction, for it is the position of the cup relative to me, and not its absolute position, that determines how I need to move my arm. But how can this be? I am not in the field of vision: no component of my visual experience is a perception of me. How then can this experience provide me with information about how objects are related to me? (1993, 205)

At this point, a distinction can be drawn between two versions of the sophisticated orientation view. On the first version, the canonical way of specifying the position of a visible object as it appears to us would be by using relational phrases such ‘x is above and to the left of me’, or perhaps ‘x is above and to the left of this body/body part’. It follows that we would always see objects and locations as involving various spatial relations to

10  Note that Brewer’s formulation is compatible with a different view, according to which what orientation terms denote is the network of spatial relations itself that defines the egocentric frame of reference. On this view, if we are facing each other, your left and my right would not strictly speaking denote the same spatial entity, since you would be involved in the reference of your use of the orientation term ‘left’ while I would be involved in the reference of my use of the orientation term ‘right’. Such a view is not very plausible. For instance, it would be hard to reconcile with the data about position constancy (see below).

Perception and Space   449 ourselves. However, this is implausible, for reasons that both Wittgenstein and Perry have pointed out. The second version of the sophisticated orientation view is expounded by Brewer as follows:11 There is a primitive use of [sentences involving orientation terms] in which this is effectively tied to the thinker himself, in his actual location at just that time, in such a way that he has no comprehension of what it would be for the object or place represented as standing in that ‘relation’ to him then to be differently spatially related to something else at some other time. So he is not really thinking relationally at all. Although it is a’s spatial relations with him then which determine its truth or falsity, his thought is more properly regimented as ‘R′a’: ‘a is to the right and a little in front’, say, as opposed to ‘a is to the right and a little in front of me now’. (1999, 193)

According to this version, the canonical way of specifying the position of an object in visual space is by using non-relational phrases such as ‘x is above and to the left’, where the subject is left unrepresented. When these phrases are used to specify perceived positions, they should be evaluated as true or false relative to the perceiver herself. Brewer’s suggestion can be seen as a specific application of Perry’s idea that we can, in certain circumstances, ‘use a n-place predicate or concept to deal with an n+1-ary relation’ (1993, 221). As an illustration, Perry considers the physical relation of simultaneity. Two physical events are not simultaneous tout court; they are simultaneous relative to a frame of reference. So physical simultaneity is at least a ternary relation; its relata are two or more events and a frame of reference. However, one can represent physical simultaneity by means of a representation of the form ‘Simul(e1,e2)’, which involves a 2-place predicate or concept. In Perry’s terminology, a token representation of the form ‘Simul(e1,e2)’ can concern a specific frame of reference, which is not itself explicitly represented.12 Of course, when simultaneity is represented in this way, the question arises as to what determines the relevant frame of reference. Perry’s answer to this question is broadly externalist: ‘the frame of reference in question is not determined by a representation in my thought, but by the broader situation in which my judgment takes place’ (1993, 221). One might want to give some kind of teleosemantic account at this point: our representation belongs to a system of representations whose function is to indicate physical simultaneity relative to the local frame of reference, i.e. the only one which is relevant to most of our thoughts and actions. Similarly, although perceived positions are presented only as being to the left or to the right, above or below, in front or behind, which origin these coordinates are relative to is determined by factors external to the contents of perception. More precisely, it is determined at the level of the psychological mode involved, i.e. by the fact that the relevant contents are perceptual, rather than imagistic or memorial. The perceptual mode guarantees, by way of our cognitive architecture, that the origin is always the bearer of the perceptual experience. Thus, only the second version of the sophisticated orientation view can deal with the phenomenological data, such as Wittgenstein’s two-stars scenario, without assuming that 11 

See Campbell (1994) and Eilan (1995) for similar considerations. maintains that in the most basic cases, the relevant frame of reference is not explicitly represented at all by the subject, not even indexically. Of course, it can also be represented indexically, for instance as the local frame of reference, but this would require more sophisticated representational abilities. 12  Perry

450   Jérôme Dokic the perceptual field is an absolute space. On this version, perceived positions can be presented in a subject-relative way even if the subject herself is neither seen nor more generally perceived, at least as the origin of the relevant egocentric frame of reference.

6 The no-orientation view According to both the simple and the sophisticated orientation views, the position of a visible object is presented as being oriented in space, for instance as being in front, up to the left. For the friend of the simple orientation view, the perceptual field itself is an intrinsically oriented space. The sophisticated orientation view rejects this claim, and maintains that the position of a visible object is presented as being relative to an egocentric frame of reference, even if the origin of the latter is left unarticulated. The no-orientation view rejects both the simple and the sophisticated views. In line with the sophisticated orientation view, it is not committed to the claim that the perceptual field is an absolute space. However, unlike the latter view, it denies that the position of a visible object must be presented as being relative to an egocentric frame of reference. Indeed, it denies that the position of a visible object must be presented as being relative to any frame of reference. In this sense, perceptual space may lack phenomenal orientation. The adherent of the no-orientation view claims that there is a way of specifying the perceived position of an object that is more primitive than the (even partially) explicit use of orientation terms. We can use non-egocentric demonstrative terms, such as ‘here’, ‘there’, or ‘this/that position’, which unlike orientation terms do not involve the notion of spatial relations between the demonstrated positions and other positions or regions in the perceptual field. The use of these terms reflects the fact that the position of a visible object need not be presented as being spatially related to any other position or region in the perceptual field, for instance as being to the left as opposed to the right.13 It is directly accessible as a position or region in three-dimensional space. In this sense, our ability to perceive positions is less cognitively demanding than proponents of either the simple or the sophisticated orientation views have assumed. An important caveat is in order. The no-orientation view is a negative claim about the way the position of an object is perceptually presented. What it says is that such a position is not presented as being relative to a frame of reference. This claim says nothing about the nature of the perceived position itself. It is compatible with the fact that this position is relative to an objective frame of reference, whether centred on the body or the environment. In other words, it is crucial to distinguish between the issue of whether the position of a visible object is perceptually presented as being relative to a frame of reference and the issue of what frame of reference the perceived position is in fact relative to. Of course the two issues are related. If the position of a visible object is veridically presented as being relative to, say, an egocentric frame of reference, then it must at least supervene 13  Perhaps demonstrative terms like ‘this/that position’ are more appropriate in this context, insofar as ‘here’ may convey a sense of relative proximity in contrast to ‘there’. Moreover, as Evans (1982) pointed out, uses of the terms ‘this/that position’, unlike that of terms such as ‘here’ and ‘there’, are typically information-based, in the sense that they require an actual perceptual experience.

Perception and Space   451 on complex spatial relations involving the body. However, it may be that even though the perceived position is relative to some frame of reference, the latter is not perceptually presented as such. This is exactly what the advocates of the no-orientation view claim. (The issue of what frame of reference perceived locations are relative to will be addressed in a later section.) An immediate objection to the no-orientation view is that it does not meet the Orientation Challenge, i.e. it does not deal with what the other two views were supposed to explain, namely the apparent orientation of the perceptual field. The perception of the same object in different relative orientations, as well as the perception of the difference between two incongruent counterparts, seems to require that each direction of the three perpendicular axes determining perceived space be phenomenally marked. Both the simple and the sophisticated orientation views argue that only egocentric terms can capture this requirement: the ability to see something to the left as opposed to the right, above as opposed to below, or in front as opposed to behind, is precisely what enables the perception of both relative and intrinsic orientation. In response to this objection, one might argue that the use of orientation terms is only necessary to recognize or re-identify the same direction across perceptual contexts. In order to memorize or encode an oriented spatial configuration such as a left hand, we have to use some frame of reference sufficiently articulated to mark the difference between left and right, up and down, and in front and behind. It is only relative to such a frame that we can memorize a sufficiently specific representation to the effect that, for instance, the palm is in front, the fingers point up, and the thumb is to the left. However, the use of non-egocentric demonstrative terms such as ‘here’, ‘there’, ‘this/that position’, which is tied to a single perceptual context, already suffices for specifying such a spatial configuration while it is perceived. The palm of the left hand is here, its fingers are there, its thumb is at this position in the perceptual field. This highly context-dependent representation simply cannot be about a right hand. However, since it does not involve the use of orientation terms, it cannot represent a left hand as such. Still, the objection goes on, there is a phenomenal difference between a left hand and a right hand even within a single perceptual context. When we face a left hand and a right hand next to each other, we can see that they are different in some way. However, the no-orientation view can provide an explanation of the relevant phenomenal difference without resorting to explicit frames of reference. What we see is that the hands do not match, in the sense that it is manifestly impossible to superimpose, even virtually, one exactly onto the other. What is phenomenally apparent is precisely the hands’ incongruence. According to the advocates of the no-orientation view, this is all that it takes to answer the Orientation Challenge. I have suggested that the use of orientation terms is required only at the encoding level, where it grounds the ability to recognize incongruent counterparts across perceptual contexts. Now, although this goes beyond the no-orientation view, even this suggestion can be resisted. Stephen Levinson and his team have described communities (for instance people of Tenejapa in highland Chiapas, Mexico) that are relatively indifferent to the distinction between objects and their incongruent counterparts, like left and right hands.14 Levinson

14 

See Brown and Levinson (1994) and Levinson (2003).

452   Jérôme Dokic himself links this inability (or rather, the absence of a cognitive ability or habit) to the fact that these communities do not use words like ‘left’ and ‘right’. His explanation is controversial, since the inability of the relevant subjects to encode the handedness of objects and structures is better explained by the fact that they spontaneously represent spatial configurations by way of an allocentric, environment-based frame of reference that is not fully articulated: they mark the distinction between the directions ‘uphill’ (roughly south) and ‘downhill’ (roughly north), but they have only one word (‘across’) for the orthogonal axis (either east or west). As a consequence, they cannot easily memorize the distinction between a left hand and a right hand. Two lessons can then be drawn from Levinson’s findings. First, we can at least conceive of perceivers who do not spontaneously encode the spatial positions of perceived objects egocentrically. Still, as far as spatial content is concerned, their visual experiences would be similar to ours: objects would be seen to be located here and there, but not to the left or to the right. Second, were these perceivers to use a fully articulated allocentric frame of reference, with the two directions corresponding to ‘across’ made explicit, they would be able to memorize the difference between two incongruent counterparts and recognize them across perceptual contexts. Thus, the use of an egocentric frame of reference is not even required to encode a left hand as such.

7 Perception and action The advocates of the sophisticated orientation view often stress the constitutive links between the orientation of the perceptual field and behavioural dispositions. As Taylor (1978–9, 154) puts the point: ‘[The] orientational structure marks our field as essentially that of an embodied agent’.15 Again, Merleau-Ponty makes a similar claim:  What is important for the orientation of the scene is not my body as it is in fact, like a thing in the objective space, but my body as a system of possible actions. (1945, 289)

According to Evans, the constitutive links between orientation and action are also apparent, although perhaps in a derivative way, in the meanings of the spatial expressions appropriate to the specification of the perceptual scene, i.e. what we have called ‘orientation terms’. As Evans explains, these expressions ‘derive their meaning from their (complicated) connections with the actions of the subject’ (Evans, 1985, 384).16 On the sophisticated orientation view, the intimate links between orientation and our primitive abilities to reach, grasp, or point to perceived objects motivate the claim that perception essentially involves an egocentric frame of reference. Consider for instance how Tyler Burge recently presents this claim:  Representation of space in perception, in perceptual thought, and in perception-guided action must be in an egocentric framework of reference. That is, spatial relations must be 15 

Quoted by Evans (1985), who also cites Pitcher (1971). also Pears (1988, vol. II, ch. 15), who acknowledges the connection between Evans and Merleau-Ponty. 16 See

Perception and Space   453 represented in a framework in which there is an origin or anchor point at the position of the individual perceiver, or some part of the individual. And the origin must be of direct relevance to the individual’s needs, motivation, or perspective. (2010, 199) Egocentric frameworks of some kind are a necessary feature of any perception. And egocentric spatial frameworks are necessary to spatial perceptual representation. Egocentric frameworks figure centrally in agency. (2010, 201)

Thus, the claim that perceptual space is egocentric should not be read merely as meaning that it is perspectivally centred on a body which is in fact the perceiver’s body. This reading would make the notion of egocentric space ‘extensional’, or purely geometric, whereas both Merleau-Ponty and Evans are concerned with the possibility of defining a more ‘intensional’ notion of egocentric space, intrinsically connected with action.17 There may still be important differences between Merleau-Ponty and Evans on these matters. The behavioural dispositions referred to by Evans are dispositions to move our physical body, whereas Merleau-Ponty insists that ‘we never move our objective body, but our phenomenal body’ (1945, 123). However, as a footnote (at the same page) makes clear, Merleau-Ponty thinks that the phenomenal body and the objective body correspond to different views on the same thing. So we only move our body as it appears to us, but this is consistent with the fact that when we move our phenomenal body, it is our objective body that moves. Insofar as the sophisticated orientation view acknowledges that perception constitutively depends on action, it is protected from a problem that is fatal to the simple orientation view. The problem is that perceptual orientation seems to generate necessarily private facts, such as the fact that a visible object is at the upper limit of the visual field (see Evans 1985, 396–397). If the simple orientation view cannot appeal to anything outside the visual field to identify a given visual position, then the phenomenal difference made by a sudden inversion of the visual field cannot be fully manifested to anyone but the perceiver herself. The simple orientation view would then raise familiar worries about the status of private objects, and of private concepts of these objects. (As is well known, some of these difficulties have been pointed out by Wittgenstein himself, of course after the time of Philosophical Remarks.) In contrast, the proponents of the sophisticated view urge that orientation facts can always, in principle, be fully manifested, since they are linked in complicated ways with public behaviour. For example, the movements appropriate to point to a visible object are very different according to whether the object is up or down in the visual field. Then, as this difference is internally linked with the orientation of the visual field itself, it can be considered as a full manifestation of what is experienced. Evans’s suggestion that the meaning of orientation terms is tied to behavioural dispositions is sensible. As a result, the correct use of egocentric representations requires a certain stability of one’s behavioural dispositions over time. For instance, when we are asked to point to the left half of our visual field, we can do so successfully whatever perceptual contents we are confronted with. This is because the representation ‘pointing to the left’ has been reliably associated with a general way of pointing, which can be instantiated by 17  For the distinction between intensional and extensional egocentric frames of reference, see Campbell (1994, ch. 1).

454   Jérôme Dokic various specific acts of pointing. Understanding what it means to point to the left is at least partly constituted by such association. The question is whether Evans’s suggestion implies that perception must involve an egocentric frame of reference. If the no-orientation view is a viable option, the answer should be negative. Evans’s suggestion is about detached egocentric representations, which we use in order to memorize and recognize egocentric directions across perceptual contexts. The proponents of the sophisticated orientation view have not shown that egocentric representations are constitutive of perception itself. Still, one of the functions of perception is to enable on-line, controlled actions toward perceived targets, such as reaching, grasping, or pointing to an object in front of us. One might then insist that the ability to engage in perception-based actions precisely requires an egocentric frame of reference. After all, as Perry observed, in order to grasp a visible object, we have to know where it is relative to ourselves. Otherwise, how could we even initiate the right bodily movement? However, this argument in favour of the sophisticated orientation view is flawed. Of course, in order to grasp a perceived object, our perceptual systems must ‘know’ where it is relative to the relevant bodily effectors, i.e. they must code the position of the perceived target egocentrically. According to recent empirical models, this is precisely the function of the visuo-motor system, which operates somewhat independently of the so-called visuo-semantic system underlying the conscious presentation of the scene.18 It does not follow, though, that the subject who intends to act on the perceived object must deploy explicit knowledge of where this object is relative to herself. It is only required that the content of her visual experience (‘This object is located at that position’) be commensurable with the content of her intention (‘I want my hand to go to that position’), i.e. involve the same demonstrative way of presenting the relevant position. The most that we can claim, then, is that our perceptual systems must involve egocentric frames of reference at some level of processing. This claim is independent from the main thrust of the sophisticated orientation view, namely that perceived positions are necessarily presented to the subject, at the conscious level, as being relative to an egocentric frame of reference.

8 Position constancy A pending question concerns the frame of reference that perceived positions are in fact relative to. What system of spatial relations do perceived positions supervene on? Is such a system egocentric or allocentric, i.e. centred on the environment? As we have seen, this question arises even if the no-orientation is true, and we do not need to perceive positions as being relative to any frame of reference. Of relevance to addressing this question is the phenomenon of position constancy. Our perceptual systems involve mechanisms which compensate for our own movements relative to the perceived positions of objects. For instance, when we move from left to right

18 

See Milner and Goodale (1995), Jacob and Jeannerod (2003), and Matthen (2005).

Perception and Space   455 while looking at a stationary object, there is a sense in which we do not see the object as changing its position, just because it was first seen to the right and then to the left. On the contrary, we may see it as maintaining its position. Like perceived motion, perceived position is relative to an allocentric (earth-fixed) frame of reference, at least by default. As Zenon Pylyshyn puts it, ‘our ability to sense or reach toward fixed points in space, independent of the position of our eyes, head, hands, or body, gives us potential access to points in allocentric space’ (Pylyshyn, 2007, 199). The phenomenon of position constancy is simply ignored by the simple orientation view, which does not even acknowledge that we can perceive ourselves moving in a cross-modally accessible space. What about the sophisticated orientation view? How can it account for position constancy? It seems that it faces two options. The first option is to insist that perceived position is in fact relative to an egocentric frame of reference, and claim that perceptual representations of allocentric positions are constructed out of more basic perceptual representations of egocentric positions. In other words, perception of allocentric positions would be somehow indirect; one would perceive them by perceiving egocentric positions. Evans claimed that a perceiver must have ‘the ability to locate his egocentric space in the framework of a cognitive map’ (1982, 163). If this ability is construed as a perceptual ability, then the claim is that there are two layers of perceptual content: although the primary layer is egocentric, there is a secondary layer which gives us access to allocentric positions. It is not clear that Evans subscribed to this claim, though. What he had in mind was probably that the imposition of allocentric representations on egocentric ones is a post-perceptual process, which is required in order to think objectively about the perceived world. The first option clearly over-intellectualizes position constancy, which is in fact a fairly primitive mechanism. It is empirically plausible that our conscious experience of allocentric positions is the result of operations over representations involving various egocentric frames of reference, but these operations take place at the subpersonal level, and need not have any correlates at the level of conscious experience. The second option available to the sophisticated orientation view is to acknowledge that perceived position is in fact allocentric but claim that it is perceived as being relative to an egocentric frame of reference. So when we move from the left to the right while looking at a stationary object, the latter does seem to maintain its position in allocentric space but at the same time it is seen as changing positions in egocentric space. Although this option is coherent, it is not very plausible for reasons explained above. It is unrealistic to require of naïve perceivers that they constantly keep track of complex spatial relations among different parts of their perceptual field. Although some cognitive tasks may require such sophistication, many others simply do not rely on the perception of positions as being relative to any frame of reference, as advocates of the no-orientation view claim. Eventually, the no-orientation view offers a simpler account of position constancy. Position constancy is an elementary mechanism which enables us to see objects as changing or maintaining positions, without presenting them as changing or maintaining positions relative to a frame of reference. In fact, such a frame is typically centred on the environment, although we should not exclude that some cognitive tasks require the perception of positions relative to an egocentric frame of reference. In other words, perceived positions are typically relative to an allocentric frame of reference, and position constancy may be seen as a partial manifestation of such relativity.

456   Jérôme Dokic

9 Conclusion: The myth of egocentricity Any realist theory of perception should acknowledge that spatial perception, and in particular the perception of positions, necessarily involves frames of reference. However, the sense in which this is so is not as straightforward as it might initially seem. Our discussion in this chapter has shown that the notion of a frame of reference is relevant to the correct characterization of at least three different levels: 

1. The ontological level of perceived positions themselves. 2. The conscious level of how perceived positions are presented to the subject. 3. The subpersonal level of the mechanisms underlying conscious perception.

First, perceived positions themselves must be relative to some frame of reference. Pace the simple orientation view, we do not perceive absolute positions. What I have suggested is that typically at least, perceived positions are relative to an allocentric, environmental frame of reference, as suggested by the phenomenon of position constancy. Second, it is less obvious that perceived positions must be presented as being relative to some frame of reference. The sophisticated orientation view, for instance, entails that perceived positions are presented as being relative to an egocentric frame of reference. It follows that they are presented as bearing complex spatial relations to other positions or regions in the perceptual field. This view seems too intellectualist, and betrays a contemplative attitude toward the perceptual field. What spatial relations are presented in perception is probably task-dependent. In the simplest cases, a given position is perceptually presented independently of a representation of its relationship to any other position, simply as ‘this/that position’. Third, there is the issue of what frames of reference are used by our perceptual systems. Cognitive science shows that there is a great variety of egocentric and allocentric frames of reference with different origins and coordinate systems (Gallistel, 2002). The relationship between these frames of reference and those that are relevant to the characterization of the personal level of spatial perception is not obvious. For instance, the claim that perceived positions are relative to an allocentric frame of reference does not entail that our perceptual systems involve a single overarching allocentric frame of reference, as Pylyshyn observes: Although one might in principle convert each of these frames of reference [i.e. the ones used by our visual system] to one global (e.g., allocentric) frame of reference, neurophysiological evidence appears to support pair-wise coordinate transformations among closely connected frameworks (e.g., eye- and head-centered frames of reference to a body-centered frame of reference, or a joint-angle frame of reference to a body-centered frame of reference). There is evidence that the many frames of reference are tied together by a web of coordinate transformation operations. (2007, 195)

In other words, the claim that the frame of reference relative to which perceived position is to be understood is allocentric does not entail that it plays any coordination role with respect to the many different frames of reference used by our perceptual and motor systems at the subpersonal level.

Perception and Space   457 Finally, the argument that perception-based action, such as reaching, grasping, or pointing to a perceived object, requires an egocentric frame of reference is best cast at the third level. Our abilities to act relative to a perceived object depend on the fact that our perceptual systems code the position of the object relative to the relevant action effectors. As we have suggested, it does not follow that we must exploit personal-level knowledge of where a perceived object is situated relative to ourselves in order to form intentions to act on it.

References Brewer, B. (1999). Perception and Reason. Oxford: Oxford University Press. Briscoe, R. (2008). ‘Egocentric Spatial Representation in Action and Perception’. Philosophy and Phenomenological Research, 79, 423–460. Brown, P. and Levinson, S. C. (1994). ‘Immanuel Kant among the Tenejapans: Anthropology as Empirical Philosophy’. Ethos, 22(1), 3–41. Burge, T. (2010). The Origins of Objectivity. Oxford: Oxford University Press. Campbell, J. (1994). Past, Space and Self. Cambridge, MA: MIT Press. Casullo, A. (1986). ‘The Spatial Structure of Perceptual Space’. Philosophy and Phenome­ nological Research, 46, 665–671. Casullo, A. (1989). ‘Perceptual Space is Monadic’. Philosophy and Phenomenological Research, 1, 131–134. Eilan, N., McCarthy, R., and Brewer, B. (1993). Spatial Representation: Problems in Philosophy and Psychology. Oxford: Blackwell. Evans, G. (1985). ‘Molyneux’s Question’. In Collected Papers (pp. 364–399). Oxford: Clarendon Press. Evans, G. (1982). The Varieties of Reference. Oxford: Clarendon Press. Falkenstein, L. (1989). ‘Is Perceptual Space Monadic?’ Philosophy and Phenomenological Research, 49, 709–713. Gallistel, C. R. (2002). ‘Language and Spatial Frames of Reference in Mind and Brain’. Trends in Cognitive Science, 6, 8, 321–322. Goodman, N. (1951). The Structure of Appearances. Cambridge, MA: Harvard University Press. Jacob, P. and Jeannerod, M. (2003). Ways of Seeing: The Scope and Limits of Visual Cognition. Oxford: Oxford University Press. Kant, I. (1992). ‘Concerning the Ultimate Ground of the Differentiation of Directions in Space’. In D. Walford and R. Meerbote (eds), The Cambridge Edition of the Works of Immanuel Kant: Theoretical Philosophy, 1755–1770 (pp. 365–372). Cambridge: Cambridge University Press. Levinson, S. C. (2003). Space in Language and Cognition: Exploration in Cognitive Diversity. Cambridge: Cambridge University Press. Matthen, M. (2005). Seeing, Doing, and Knowing. Oxford: Oxford University Press. Merleau-Ponty, J. (1945). Phénoménologie de la perception. Paris: Gallimard. Milner, D. and Goodale, M. (1995). The Visual Brain in Action. Oxford: Oxford University Press. Peacocke, C. (1986). ‘Analogue Content’. Proceedings of the Aristotelian Society, Supplementary Volume, 60, 1–17. Peacocke, C. (1992). A Study of Concepts. Cambridge, MA: MIT Press. Peacocke, C. (1989). ‘Perceptual Content’. In J. Almog, J. Perry, and H. Wettstein (eds), Themes from Kaplan (pp. 297–330). Oxford: Oxford University Press.

458   Jérôme Dokic Pears, D. (1988). The False Prison. Oxford: Oxford University Press. Perry, J. (1993). ‘Thought Without Representation’. In The Problem of the Essential Indexical, and Other Essays (pp. 205–225). Oxford: Oxford University Press. Pinker, S. (1997). How the Mind Works. New York: W. W. Norton & Company. Pitcher, G. (1971). A Theory of Perception. Princeton, NJ: Princeton University Press. Pylyshyn, Z. W. (2007). Things and Places: How the Mind Connects with the World. Cambridge, MA: MIT Press. Russell, B. (1948). Human Knowledge. London: Allen and Unwin. Stratton, G. M. (1897). ‘Vision without Inversion of the Retinal Image’. Psychological Review, 4(5), 463–481. Taylor, C. (1978–9). ‘The Validity of Transcendental Arguments’. Proceedings of the Aristotelian Society, 79, 151–165. Van Cleve, J. and Frederick, R. (eds) (1991). The Philosophy of Right and Left. Dordrecht: Kluwer, Academic Publishers. Wittgenstein, L. (1975). Philosophical Remarks, ed. R. Rhees. Oxford: Blackwell.

Chapter 24

Perception a n d  Ti m e Robin Le Poidevin

1 Questions about perception, time, and time perception Why should philosophers of perception be interested in time? Time perception is interesting in its own right, of course, as a special case study that raises distinctive puzzles. But time as an object of perception also disturbs certain assumptions that it otherwise seems natural to make about the perception of everyday objects around us. Moreover, perception is a temporal process. The act of perceiving something has a time dimension which, until it is made the subject of inquiry, we are otherwise liable to ignore. Time can thus inform and expand our understanding of perception. The influence can run the other way, too. Reflecting on the nature of perceptual experience can motivate, or make more problematic, theories of the nature of time itself. What this chapter aims to do, then, is to draw out the intriguing and complex connections between perception and time. It does not aim to give a comprehensive account of the nature of time perception, although certain phenomena and suggested mechanisms underlying those phenomena will have a bearing on the discussion. So let us begin by looking at some questions we can ask about perception, questions we can ask about time, and how they might be connected through a third set of questions about the perception of time. First, then, three questions about perception:  The demarcation question: What distinguishes perception from other forms of mental representation, such as memory? The object question: What is the direct object of perception the world itself, or a mental representation of the world? The realism question: How much of what we count as our perceptual experience reflects the state of the world and how much is a projection of our minds? Answering these questions is a vital part of the epistemological project of defining the mind’s cognitive relations with the world. They look as if they were empirical, scientific questions, and indeed the psychology of perception is certainly relevant to attempts to

460   Robin le Poidevin answer them as we shall see in the final section on time illusions but, for the most part, the approach taken here is to address them in connection with metaphysical issues. Now three questions about time:  The existence question: Does only the present exist? Or are all times equally real? The passage question: Does time really pass? That is, do events become present and then recede into the past, or is this simply a matter of our perspective? The direction question: What constitutes or explains the direction of time? What makes these metaphysical questions is that they are questions concerning the mind-independent nature of the world. There are other pertinent questions of this kind we can ask about time, for example about its topology its shape, as one might put it. Is it linear, or circular? Is there just one time series, or are there many? Is such a time series infinitely or only finitely divisible? And so on. But those questions I want to put on one side, partly because they cannot perhaps be adequately addressed without straying into the physics of time, and partly because the connections between them and issues of perception are less direct (though at the end, we will look briefly at what we might call the topology of phenomenal time). Another important question about time is how it is related to the events and states of affairs that take place in time: is it a construction from those, or is it an object in its own right, something that could exist in the absence of anything else? Again, I want to put that to one side, as whether or not time is a construction from its contents, it is plain that we can only perceive those contents. So, limiting ourselves to the two sets of questions above, one concerning perception, and the other concerning time, how might they be related to each other? One connection links the existence question about time and the demarcation question about perception. Suppose, in answer to the existence question, we say that only what is present is real. Then, given that genuine perception can only have a real item as its object (even if it sometimes misrepresents the nature of that object), perception can only have what is present as its object. But that seems inconsistent with what we might call the ‘time-lag problem’:  given the finite speeds of light and sound, and (more strikingly) of sensory information-processing systems in the brain, there is always a temporal gap between an external event such as a flash of lightning and the perception of that event. So the object of perception, the event itself, is not present. And that, according to the presentist, means that it is not real. Perhaps there is a way of avoiding this conflict, however, for the time-lag between external object and perceptual state has also been employed as an argument for the ‘representative realist’ account of perception: since the direct objection of perception cannot be at a temporal distance, which all external events are, it must be something closer to home, a present sense datum. (By ‘sense datum’ is to be understood, in this context, not simply the content of a perceptual experience, but an item, located in time and space, which itself has content.) Thus we have an intriguing connection between the existence question about time and the object question about perception. The realism question about perception and the passage question about time have a very direct connection with each other. It is often asserted that time’s passage is a pervasive and inescapable feature of perceptual experience, and that the natural explanation for that is that perception tracks objective passage in the world. So those philosophers who deny the objective passage of time must either offer an alternative explanation for the perceptual phenomenon, or argue that objective passage, if there were such a thing, is not something

Perception and Time   461 that could be perceived, and what is reported as the perception of passage is actually the perception of something else, which may or may not require passage to explain it. Another obstacle to the idea that perception only has what is present as its object is raised by the intuition that we perceive time order. We just hear, for example, one sound as following on from other. But a temporally ordered pair of events cannot possibly occupy the present moment only, for the (real) present has no duration, and so no temporal structure. So it would appear that the objects of perception are not confined to the present. This is a thesis about the way our perceptions are related to their external objects. We might consider, however, whether it has a phenomenological counterpart. What distinguishes perception from memory, we might think, is their very different phenomenology: perception presents things to us as present. But can something be presented to us both as wholly present and as temporally structured? Finally, one of the most striking features of time, one that distinguishes it from space, is the fact that we perceive it as having a direction. Does this reflect an objective feature of time, or is it a reflection only of our perspective on time, which we then project onto the world? (Here we have a special instance of the realism question.) If it is a real feature, what explains it? (The direction question.) And does that explanation encompass this very obvious feature of our experience? Can, that is, the perception of order be explained by the metaphysical basis of order, or does the direction of explanation go the other way? A bewildering variety of questions, to be sure. But one in which we begin to see some logical relationships that help to guide us through this tricky area. So let us now look at each of these relationships in more detail.

2 Is perception cinematic? When considering a paradigmatic instance of perception, philosophers are apt to think of the perception of an object such as a tree, and attend to relatively stable features of the object its shape or colour, for instance. But perception is a process that takes place in time, and sometimes in a constantly changing environment. So let us focus on the dynamic aspect of perception, and ask how it is possible for a perceptual state to take in an extended period, a period which may include change. Here is one answer to that question: The cinematic account of perception: perception of events consists of a series of ‘snapshots’ of the world, that is, a series of perceptual states each of which has as its object an instantaneous state of the world. The suggestion, and the cinematic analogy, is Bergson’s (Bergson 1911: 322–323; for variations on the theme, and rival accounts, see Grush 2007 and Dainton 2010b). On the cinematic account, perception has tightly defined temporal boundaries. Why should this seem entirely intuitive? Because it is linked to another intuition, that perception presents us with what how things are now, and that in turn is constrained by a feature of time itself. More formally, we can attempt to derive the cinematic account from one answer to the existence question: presentism. Presentism says that only what is present is real. Things cease to be real once they recede into the past, and the future is mere possibility. (This is a very brief and intuitive articulation of the position. For a more sophisticated statement, see Crisp 2004.) Now one of the defining characteristics of perception is that it can only take as its object what is real. A non-existent object may be an object of thought, or hallucination,

462   Robin le Poidevin but it cannot be an object of perception. Not all perception is veridical, of course, but even in non-veridical perception there is a real object that is perceived. Whether the perception is veridical or not depends on whether the object does in fact have the properties the perceptual state represents it as having, not whether or not the object exists. Combining this defining property of perception with presentism yields the result that perception can only have a present object. Perception, then, is confined to the present. But how long is the present? As Augustine showed, what is wholly present cannot have earlier and later parts, for anything earlier than a present part is not present but past. The present is indivisible, and so is not temporally extended, but instantaneous (Pinecoffin 1961, section 15). Any given perception is therefore no more than a snapshot of the world: it represents the state of affairs that is actually present. Thus we arrive at the cinematic account of perception. It would be helpful at this stage to clear out of the way one objection to the cinematic account, and that is that the ordinary objects of our perception trees, tables, and telegraph poles are things that persist through time, and so are not instantaneous. This, although true, is beside the point. For we perceive objects by perceiving them as how they are, or appear to be, at a particular time. We perceive objects because we perceive states of those objects, and those states do not persist. An object can continue to have a certain property for a period of time, to be sure, but the state of affairs that is a’s being F at time t is numerically distinct from a’s being F at a later time, t*. So the primary object of perception, according to the cinematic account, is a temporally specific state of that kind. There are, however, other, more serious objections to the cinematic account. The first objection may not touch all understandings of that account, but it is certainly devastating to the version of it we reached by the reasoning above. For that version confines the object of perception to the metaphysical present, and so to an indivisible moment, rather than a period, of time. But it is highly implausible that we are perceptually sensitive to durationless states. Even photographic snapshots take time, and what they record is the record of a period of time, however brief. It is unlikely that we are more sensitive than a camera. The second objection is similarly directed at this metaphysical understanding of the present. Given that it takes time for information about the environment first to reach us (for light and sound travel at finite speeds) and then to be processed by our brains, the external state of affairs that is the object of perception cannot still be present, but will have receded, if only slightly, into the past. It might be thought that presentist assumptions undermine this last objection. For if presentism is true and the present is durationless, then nothing in reality has duration. How then can we say that it ‘takes time’ for information to pass from the object into consciousness? Now if presentism really does outlaw such expressions as ‘It took me fifteen minutes to get to work today, the traffic was so slow’, then that would be bad news for presentism. Any reasonable presentist will of course offer an account (or recognize the obligation to offer an account) of how statements about times other than the present, including statements about duration, can be true. It may be, for instance, that present fact plus the laws of nature guarantee such truths. (For a discussion of this and other strategies, see Bourne 2006) At all events, we will not on these grounds be prevented from drawing attention to the finite speeds of light, sound, and information processing. It seems that this ontologically loaded conception of the present would be something of a millstone around the necks of proponents of the cinematic view, and they may choose to characterize it in more phenomenological terms. The defining feature of perception, they may

Perception and Time   463 say, is that everything it represents, it represents as happening now. That is, presentness is an essential part of the representational content of the perceptual state. This content is ‘instantaneous’ only in the sense that no temporal structure is discernible within it, not that what is perceived actually takes up no duration at all. Because it does not coincide with the metaphysical present, the present of experience is often described as the specious present (an expression made famous, but not coined, by William James, who has a distinctive view of what it means: see James 1890: 603; for a contemporary articulation, see, e.g., Kelly 2005 and Dainton 2010a, ch. 7). This may sidestep the two objections above, but it leads to a third objection. If every individual perceptual state is phenomenologically instantaneous, it follows that there can be no perception of either order or duration. What we call the experience of order and duration must involve the memory, which permits us to compare the content of different perceptual states and judge their difference or similarity. The worry is that this is not true to the phenomenological facts. Consider the experience of listening to a piece of music. It seems we hear each phrase in what appears to be a quite immediate way, one that does not involve attending just to one note, and remembering what came immediately before it. We just hear it. (As we might put it, perhaps somewhat more controversially, the phrase is a direct object of perception, in the sense that we do not hear the phrase by virtue of hearing its individual parts: see Matthen 2010 for a defence of this thesis.) But the phrase is temporally structured: the notes of the melody are not simultaneous. This temporal structure surely reveals itself in perception. And if that structure is apparent in a single act of perception, then what is presented cannot be phenomenologically instantaneous i.e. be such as not to appear to take up some time. Insofar as we hear musical phrases, then, the following must be true: (i) the object of perception is temporally extended; (ii) the object of perception can present itself as temporally extended. (i) is in tension with metaphysical presentism, and, we might be tempted to add, (ii) is in tension with presentism’s phenomenal counterpart: that what we perceive we always perceive as present. This very last point is resistable, however. Although it seems paradoxical to say that something can seem to be both present and temporally structured, perhaps we should accept that the content of some perceptions would generate an inconsistency if we took it as reflecting how things actually are.

3 Presentism and the time-lag problem Let us return to the second objection to the cinematic account. Perhaps this is better cast as an objection to presentism. To rehearse the problem: the following tetrad of propositions is inconsistent:  (1) We perceive things by perceiving the states they are in at specific times. (2) Only what is real can be perceived. (3) Only what is present is real. (4) There is a time lag between a given state and the perception of that state. (1) and (2) are among perception’s defining characteristics, (3) is the thesis of presentism, and (4) is an apparently unassailable empirical fact. At first sight, then, we appear to have a perceptual argument against a metaphysical thesis.

464   Robin le Poidevin But this is too quick. Let us distinguish the direct object of a perceptual state from its indirect object. Suppose you are looking at a photograph of some distant relative. Insofar as you see the relative, you do so only by seeing the photograph. The relative then is the indirect object of your perception. But (or so it seems) you do not see the photograph by seeing something else: the photograph is the direct object of perception. Suppose, however, that the direct object of perception is much closer to home. Let us say that it is a wholly mind-dependent entity, a ‘sense datum’, or collection of sense data. We perceive the external world by perceiving these sense data, which are constructions of the mind based partly on raw sensory input and partly on cognitive manipulation of that input. This is the representative realist theory of perception. This theory is developed in much greater detail, and to a higher level of sophistication, elsewhere in this volume, but this sketchy characterization will be enough for our immediate purposes. Now, a key argument for the representative theory appeals to the time lag between the external state and the perception of that state. Since the state is at a temporal distance (in the case of stars, a very considerable temporal distance) from the perception, it cannot, so the argument goes, be the direct object of that perception. We only ‘see’ the star in the derivative sense that it is the cause of something we do directly see, namely the sense datum. When the argument for representative realism is stated so baldly, the natural response is to ask why merely being at a temporal distance should prevent a state’s being the (direct) object of perception, when being at a spatial distance seems to raise no difficulties. Why should we not treat time and space as parallel in this respect? The answer must be because of an implicit assumption of presentism. A spatial distance does not impugn an event’s reality; a temporal distance does. Presentism, therefore, is the missing premise in the time-lag argument for representative realism. With a shift of perspective, we can also say that representative realism is needed to save presentism from the time-lag problem. So, armed with the distinction between direct and indirect objects of perception, we can qualify (2) in the tetrad of propositions above as follows: ‘Only what is real can be directly perceived.’ And (4) becomes: ‘There is a time lag between an event and the indirect perception of that event.’ The contradiction now vanishes. (The line of thought here is due to Power 2010.) Whether this is enough to save presentism from perceptual worries, however, is doubtful. For the temporal boundaries it sets for the objects of perception it will also set for the perceptual state itself. If what is real is confined to a durationless moment, then a perceptual state, insofar as it is real, is likewise durationless. But if perceptual states are to be conscious states, as some plainly are, this looks entirely false. How can an act of registering some state in the world take no time at all? Recalling the camera analogy, the taking of the snapshot itself takes time, and we are rather slower than cameras. Enough has been said to show that if presentism is not greatly to disrupt our understanding of perception, it must expand its conception of the present beyond the Augustinian instant to include a presumably short period. (This move is considered in Dainton 2010, 95–102. See ch. 6 for a discussion of the different versions of presentism.) The crucial question then is whether this period is temporally structured or not. If it is not structured (i.e. indivisible), then it will be too short to accommodate the perceptual state, for the minimum duration a mental state must have in order to be a conscious state surely exceeds that of the minimum physically possible duration. If it is temporally structured, then we have to understand temporal ordering in a way that is quite independent of the past-present-future distinction. And this sits oddly with a presentist outlook, as we shall now go on to show.

Perception and Time   465

4 Passage: Perception or projection? Time’s defining feature, anyone who has not already subscribed to a contrary philosophical theory would say, is that it passes. Things come into existence, are present, and then recede ever further into the past. We can think of things as passing out of reality when they cease to be present, which is the presentist view, or we can think of them as continuing to be real, and as contributing to an ever-increasing collection of real items, past and present, which is the ‘growing block’ view (the classic source for which is Broad 1923; for discussion, see Dainton 2010a, ch. 6). If time really does pass in this way, we have an explanation both for the temporal ordering of events and for time’s direction. All events in time exhibit an ordering: any given event will have a place in a series running from earlier times to later times. Intuitively, this ordering is not independent of the passage of time, for if, of two events, A is past and B is present, it immediately follows that A is earlier than B. Since we cannot make the reverse inference, from order facts to facts about passage, it is reasonable to regard passage facts as ontologically the more fundamental. This is why a presentist, who believes in the reality of temporal passage, would be ill-advised to introduce the conception of an objective present within which there is an earlier-later ordering. For within such a present, one part’s being earlier than another could not be explained in terms of one part’s being past and the other’s being present. An ordering by itself does not generate a direction. The series of integers has an ordering, in that 6 comes between 5 and 7, but it does not have a direction: to think of the series as running from the lower numbers to the higher is no more ‘correct’ than to think of it as running from the higher numbers to the lower. Engaging in a countdown (10, 9, 8 . . .) is not to fly in the face of nature. But time does have a direction, and it is natural to think of this as the direction of the passage of time: things recede into the past, the future arrives, etcetera. What has any of this to do with perception? It is ordinary perceptual experience, augmented by memory, which persuades us that time passes. What we perceive at one moment, we remember the next, when it is no longer being perceived. Moreover, perception presents us with a never-ceasing flow of images, sounds, and textures. Is this not direct experience of the passage of time? Two questions need to be asked, however: Is the passage of time something that can be perceived? And is it safe to argue from the nature of temporal experience to a feature of temporal reality? Assuming, what is likely to be challenged only be a minority, that we do not perceive the future, the genuine perception of time’s passage would involve the perception of a present item receding into the past. But an item is never perceived as past. That is, we do not perceive the pastness of an event as we perceive the roundness of a ball (though we may be aware of pastness in some non-perceptual sense, as when we contemplate an episodic memory). What is perceived is perceived as present. So even if time lag entails that we perceive the past, we do not perceive the pastness of the past. The star I see in the sky is as present to me as that ever-present intruder into my visual field, the end of my nose. So what might be somewhat misleadingly described as the perception of time’s passage is perhaps more truthfully characterized as the perception of something’s being replaced by something else: the perception of change, or at least a change in our perceptions. There is still an argument from this undoubted feature of our experience to the real passage of time, but

466   Robin le Poidevin it is less direct than an argument from the perception of a tree to the actual existence of a tree. The argument is that both our perception of change and our changing perceptions are best explained in terms of the metaphysical grounding of real change, namely the passage of time itself. But the force of this argument depends less on a feature of perceptual experience and more on the plausibility of the view that real change depends on real passage. Here now is a quite different metaphysical picture: time does not pass in reality. Events are not past or future in any absolute, non-relational, or mind-independent sense. The terms ‘past’, ‘present’, and ‘future’, and other tensed expressions (such as ‘now’, ‘yesterday’), reflect only our temporal perspective on reality, just as ‘here’ and ‘there’ reflect our spatial perspective. There is, quite independently of any mind, a temporal ordering events genuinely stand in relations of temporal precedence but that ordering does not depend on passage. This view of reality (fully articulated and defended in Mellor 1998) is sometimes described as the ‘static’ conception, since it holds that nothing moves through time. But the further implication that nothing changes is unwarranted. Change consists simply in the fact that one state of affairs obtains at one time and another state of affairs obtains at another. The view is also, and more neutrally, named the ‘B-theory’ of time. If, as the B-theorist must hold, the facts of perception are entirely compatible with this theory, why should perception incline us towards the view that the present moment is in some way privileged, that our temporal perspective is due to a feature of time itself? After all, our spatial experience is equally perspectival, but we seem fully aware of the fact: that is, we are aware that how we see the spatial orientation of things depends on our position in space. In contrast, we appear to be blind to our temporal perspective. What we perceive as happening now is not presented to us as a perspectival matter: we merely perceive what that perspective presents to us. That is a profound difference between the experience of time and the experience of space. What explains it? A vital feature of our perception of our environment is the spatial distances of external objects: we see, or hear, them as being near or far. We also perceive their orientation: as being to the right or to the left, up or down. Change in distance or orientation is something that, at least to some extent, is under our intentional control we can simply move towards an object and so decrease its perceived distance, or make it appear on the right rather than on the left. So awareness of the distance and orientation of objects is also awareness of our own spatial position, and as we can distinguish between two causes of the perceived position of objects, those that are due to a corresponding change in the relative position in the objects perceived, and those that are due merely to change in our position, we learn to see perceived distance as merely perspectival. There is no temporal equivalent to this. First, perception is much more temporally restricted than it is spatially restricted: we perceive things as being now, not as being now or then. We may be aware of the then, but only through memory. Second, whether an event is presented to us as past or present is not similarly under our intentional control. An event is first perceived, and so represented as present, and then, when no longer perceived, registered in the memory, and so registered as past. Nothing we can do can make that remembered event be once more the object of perception. So change in the perceived temporal proximity of events does not give us a sense of temporal distance as merely perspectival. It is therefore much more natural to think of the presentness or pastness of events as something objective, and not merely perspectival. Nevertheless, according to the B-theorist, they are features projected by the mind, not the causes of our perceptions. (See Le Poidevin

Perception and Time   467 2007 for an extended account of how the B-theory can be reconciled with what appear to be the facts of experience, including perception and memory.) Why should spatial proximity be controllable but temporal distance not? This points to an objective difference between time and space; the fact that time has a direction. What explains this? B-theorists cannot appeal to objective passage, so what else can they appeal to?

5 The psychological arrow of time Time is a one-way street. We have the sense of moving along that street. But if the B-theory is correct, nothing actually moves in time. Nevertheless, there is an asymmetry between earlier times and later times, and it is a causal asymmetry: what is earlier causally affects what is later, but not vice versa. We could attempt to explain this causal asymmetry between earlier and later by appealing to the direction of time, but a more fruitful strategy is to reverse the order of explanation and account for the arrow of time in terms of the arrow of causation: A is earlier than B because A is a cause of B. (We do not have to say that assertions of time order are equivalent in meaning to assertions of causal order, simply that facts about causal order make assertions about time order true.) There are a number of challenges a causal theory of time order faces: is there a viable analysis of the causal relation that explains the asymmetry of causation without appealing to temporal asymmetry? How does the causal analysis accommodate the intuition that time is more pervasive than causation, that there are temporal relations between events that are not causally connected? What should it say about apparent instances of simultaneous causation? (See Sklar 1974 for a critical discussion of the causal theory.) But the crucial challenge in the context of perception is whether it can account for what we might term the ‘psychological arrow of time’, under which heading we can group the following phenomena: that we can never perceive the future; that we perceive events as temporally ordered; and that we have a sense of time passing. First then, if we define time order in terms of causal order, it is a necessary (indeed, trivial) truth that causes occur before their effects. Perception is a causal process: what I perceive causes my perceptual state. So I only perceive those events that occur earlier than my perceptual state, never events that occur after it. But, from the B-theoretic view, a ‘past’ event is just one that is earlier than the time I perceive, and a ‘future’ event is one that is later than the time I perceive. From all of which it follows that I perceive the past, never the future. We cannot so readily derive an account of the perception of time order from the causal theory, but the following parallel naturally suggests itself: if the actual time order of events is grounded in the causal order of those events, then perhaps the perceived time order of events is grounded in the causal order of the perceptions of those events. The simplest causal story would go as follows: I perceive event A, sensory information from which is then stored in the short-term memory. When I subsequently perceive B, the memory trace of A colours that perception. It is that causal influence of the remembered perception of A on the perception of B that gives rise to the non-inferential awareness of B’s following A (Mellor 1998: 114–115; but see Dainton 2010a: 105–106 for objections).

468   Robin le Poidevin This causal story is compatible with one version of the cinematic account of perception. A mere temporally ordered series of perceptions of individual events does not, by itself, give rise to the perception of the time order of those events. But once we introduce causal connections of the right kind between the members of the series, we have, in addition to the ordered perceptions, a perception of time order. If the influence of remembered events on perception affects the phenomenology of that perception, there is no need for the cinematic theorist to insist that awareness of time order is inferential. The causal account of time-order perception highlights the difficulty of giving a simple answer to the demarcation question. If we say that memory’s colouring of perception gives rise to a further perception, that of time order, then it becomes harder to insist on a sharp distinction between perception and memory. But we can make some progress by recognizing that there is not one distinction here, but (at least) three, involving, respectively, phenomenology, content, and encoding mechanisms. Thus we can distinguish between the sense of being presented with a situation, without our willing it, and the sense of calling up an image at will. We can also distinguish between a psychological state that immediately gives rise to present-tensed beliefs (‘it has just started to rain’) and one that gives rise to past-tensed beliefs (‘it was raining earlier’). And we can distinguish between the processing of sensory information and the storing of that information. There is no reason to suppose that these distinctions map onto each other precisely. Thus, for example, storage mechanisms may influence sensory processing in a way that results in a phenomenology characteristic of perception. Returning to the main theme, and supposing the causal account of time-order perception to be largely correct, there is a further explanatory challenge to be met. Earlier, we distinguished between order and direction. We can order the integers by the asymmetric ‘greater than’ relation, for example, but that does not impose a direction on the series. Similarly, times are ordered by the ‘earlier than’ relation, and we can explain the asymmetry of that relation in terms of the asymmetry of causation. But that does not yet explain the sense of time as running from earlier to later. At this point, we might be tempted to appeal to the objective passage of time to explain our sense of direction. But, says the B-theorist, we should resist that temptation. As suggested in the previous section, the passage of time is not something that can be perceived directly. (For a sophisticated defence of that surprising thesis, see a series of papers by Simon Prosser: 2007, 2012, 2013.) Rather it is inferred from the undoubted fact that our perceptual point of view is rather tightly constrained temporally it appears to be confined almost to a single moment and what is presented to that point of view is constantly changing. If the B-theorist can explain that fact, then there is no further explanatory work (at least, as far as perception is concerned) for the passage of time to play. Now we have already encountered part of the explanation: we cannot perceive the future, for (given the causal theory of time order) later events can never cause earlier ones. So that explains one of the temporal boundaries of perception. But there is also an earlier boundary: the perception of one episode is quickly replaced by perception of another episode; we do not continue to perceive everything we have ever perceived. The reason for this is surely to do with the limits of our information-processing systems, and with the need for timely action. Once sensory input is registered, it needs to be quickly removed from our perceptual field to make room for new information, so that what dominates the perceptual field is what in the external world is more or less simultaneous with our experience. Our perceptual

Perception and Time   469 state must perforce be constantly changing. That is not the same as the perception of change, of course, but the perception of change is surely explained in the same terms as the perception of time order. In sum, if the direction of time is nothing more than the asymmetry of temporal precedence, then the B-theorist can allow that time has an objective direction, and it is explained by the asymmetry of causation. The perception of that order can be explained by the causal order of our perceptions. But if, by ‘the direction of time’ we mean more than mere asymmetry, then this is due to our sense of time’s passage, and this, it is plausible to suggest, is something we do not perceive, but rather project. The causal account of our perception of time order is an empirical thesis: it has not been derived a priori from more general theories concerning time or perception. So it is reasonable to ask whether any empirical finding falsifies time perception or any of the other accounts of it we have encountered during the course of this discussion. We therefore end by looking at some curious phenomena concerning time and perception.

6 Time illusions Relatively recently i.e., over the last ten years or so, psychologists have become increasingly interested in what we might call ‘time illusions’ misrepresentations or manipulations of some temporal aspect of a situation. These cast light on the brain’s mechanisms for representing time. But, in the context of this discussion, we should ask whether they also have philosophical implications: do they support, or disturb, any of the ideas raised in the previous sections? Do they raise any other difficulties? I shall group the illusions under three heads, corresponding to three kinds of temporal manipulation in which the brain appears to engage.

(i)  Filling in time It has become a commonplace that what we see is what the brain tells us we are seeing, but here is a rather surprising instance of that. The general phenomenon is known as chronostasis, but the most familiar example is the stopped-clock illusion. We glance at the clock on the wall, and for a moment it seems that the second hand has stopped. Even when the second hand rotates around the dial in a series of discrete movements, so that we are used to its staying still for a moment, we still have the peculiar impression that it stayed in one place for an unusually extended amount of time. Why is this? One explanation is that this phenomenon is a result of the brain’s compensating for the movement of our eyes towards the clock. When we switch our gaze from one position to another, we ought to see a blur as the scene rapidly sweeps before our eyes. Yet we do not see a blur, and the reason why is that the brain deletes or ignores the information arriving from the retina during the period in which the eyes were in motion information that would only confuse and disorient us and replaces it with the scene we behold once our eyes have ceased moving (Yarrow et al. 2001). It provides us retrospectively with an image of what we could have been seeing during that period. So, in the case of the clock, once our eyes rest on the second hand, the brain tells us that that was what we were seeing when our eyes were in fact in motion, hence

470   Robin le Poidevin the sense that the second hand has been in that position for a while. The brain ‘fills in’ the lost time for us. Another example of the tendency of the brain to fill in time is known as the phi phenomenon, though here the effect is the opposite: we have illusory motion rather than illusory absence of motion (first described by Wertheimer 1912). Here is a familiar example. At a level crossing, two red lights are flashing alternately, to warn you that a train is coming. It appears, particularly when the lights are off-centre in your visual field, that there is a single light moving back and forth. Here, although the eye is presented with two discrete stimuli, the brain fills in the gap between them, creating the illusion of continuous movement. These phenomena have a bearing on the object and realism questions concerning perception. Clearly, by virtue of being illusions, these experiences represent a certain temporal feature of the stimulus (its relative duration in the case of the stopped-clock illusion, and location in the case of the phi phenomenon) as being other than it is. Moreover, these are not just isolated phenomena. The mechanisms involved in chronostasis are called into play every time we move our eyes, and this includes the unnoticed saccadic eye movements that occur whenever we scan a scene. So, for a significant proportion of our perceptual lives, we are seeing something that is not there not there in the world, that is. And that provides further fuel for the representative theory of perception, not simply because we are dealing with an illusion here, but that the illusion is brought about by cognitive processing rather than a simple malfunctioning of the visual system. Between the stimulus and the experience is some cognitive manipulation of the data which produces an interpretation of what we are seeing.

Compensating for delays As noted above, it takes time to process input from the sense organs which is one reason why it is unwise to identify the present of experience with the metaphysical present and it seems that the brain is able to compensate for that delay, but also that such compensation can lead us into error. One instance of this is the flash-lag effect. A spot of light moves across a screen. When it reaches the halfway point, a similar spot of light appears very briefly below it before disappearing. Although the two spots are perfectly aligned, the moving spot is always perceived as being ahead of the flashed spot (Nijhawan 1994). The ‘prediction’ hypothesis is that the brain, as it follows the moving spot, is compensating for the fact that there is a delay between receiving sensory information and interpreting it, and anticipates the position of the spot: the spot is seen in a position that the brain thinks it should have reached by the time the information is processed, not where it was when the image of the spot first fell on the retina. But it cannot do this with the flashed spot, so the flash seems to lag behind the moving dot. When it comes to connecting stimuli with our own actions, it seems that the brain can be trained to take into account delays and then be made to trip up! By pressing a button you can make a light flash. The experimenter then introduces a delay between the button press and the flash. Once this happens a number of times, your brain adapts to the delay, and the flash is perceived as being closer to the press than previously (Haggard et al. 2002). Now the experimenter reduces the delay, so that the flash follows immediately after the

Perception and Time   471 press. The brain is taken by surprise the anticipated delay did not occur and you see the flash as happening before you pressed the button. It is as if the experimental set-up anticipated your action (Stetson et al. 2006). The first question we should ask about these results is whether they make difficulties for the cinematic account of perception: the idea, that is, that we perceive the world over a period by taking successive ‘snapshots’ of it. As we first presented that account, what is perceived at any moment is isolated from prior or subsequent perceptions: it presents only how things are at that particular time, as if the brain is simply passively registering the present state of affairs. Clearly, the two illusions above falsify the hypothesis that the content of each perception is isolated from previous experience. Now it might seem that the cinematic account can easily be modified to take account both of causal interference from earlier perceptions and also of the active role played by the brain. But there remains the problem that the cinematic account takes the fundamental objects of perceptual representation to be states what obtains at particular times. Whereas what is suggested by the flash-lag effect and the time-order illusion above is that what is represented by a perceptual state is something rather more dynamic, a process (Grush 2007). The brain has certain expectations of where the moving dot is going, and of how long it takes for the button press to bring about the flash, and these expectations are encoded in the resulting experience. A second question these two illusions raise is whether they support or conflict with the rather abstract account of the perception of time order offered as part of the B-theorist’s view that perception of order is quite independent of temporal passage (real or perceived). On that account, it is not enough for one perception to follow another to generate the perception of order: the earlier perception must have a causal influence on the other, Moreover, that causal influence has to be of a fairly direct kind. A remote memory may influence our experience of a current scene, but that hardly gives rise to the perception (as opposed to the inferential awareness) of time order. Now, although the flash-lag effect is not specifically to do with the perception of time order, it confirms the causal account of order perception to the extent that it provides a clear instance of earlier perceptions (the perception of the movement of the dot) influencing a later one (the perception of where the dot has reached when the flash occurred). But what of the other illusion? The perception of the button press presumably occurred earlier than, and so presumably was able to influence, the perception of the flash, and not the other way around. That is, the perception of the flash, being the later event, cannot have causally influenced the perception of the button press. Here, it seems, we could say one of two things. We could say that, given the actual time order of the perceptions (and the consequent causal order), the initial perception was actually that of the flash following the button. But because the flash followed too quickly on the button press, the brain overrode that perception as anomalous and replaced it with the reverse perception (cf. Dennett and Kinsbourne 1992). So here we have two mechanisms in play, not one. Or we could say that, because the perception of the button press and the perception of the flash were so close together, neither initially influenced the other, and so the time order was indeterminate, leaving the brain to impose an interpretation, based on previous experience. So here just one mechanism comes into play. But neither of these accounts actually falsifies the original causal model: they just augment it. Where stimuli are not so close apart, it may be that the causal model is sufficient. In any case, however one approaches the explanation of these two illusions, at no point do we find

472   Robin le Poidevin ourselves appealing to anything like the perception of time’s passage. So the B-theoretic view is left untouched. One observation to make about the illusions encountered so far is that they concern very small periods of time around 100 milliseconds. For longer periods, temporal relations seem to be less open to psychological manipulation, which suggests an operational characterization at least of the difference between perception and memory.

Dilation, contraction, and the unity of phenomenal time Perhaps no time illusion is more familiar than that of time speeding up or slowing down. It is tempting to say that this is illusory because it makes no sense to suppose that time itself is passing more or less slowly than it did. We measure the rate of change by time, so how can we measure the passage of time itself? (See, e.g., Olson 2009.) Or, if one rejects the notion of the objective passage of time, one might say that it is illusory precisely because time seems to be passing (more or less quickly). But one might be suspicious both of an objective passage of time and the idea that there is something irreducible in our experience that is the ‘experience as-of-time’s passage’. That just leaves us with the perception of change. And individual changes can speed up or slow down relative to others. So what is illusory is the fact that certain changes appear to have speeded up or slowed down when in fact they have not, or that a certain stimulus seems to last for a slower, or shorter, period of time than other stimuli when in fact it has not. Periods of emotional tension can seem to be dilated. You see the cat brush a priceless vase off the mantelpiece, and everything seems to go into slow motion, but although the vase takes an age to fall onto the floor and smash itself into fragments, you seem oddly powerless to step forward and catch it. Certain kinds of brain damage can also upset time judgements. In one instance of what has been dubbed the Zeitrafferphänomen, a patient suffering from damage to the left prefrontal cortex suddenly found that everything appeared to be happening at an accelerated rate (Binkofski and Block 1996). This made driving impossible, as the traffic appeared to be moving dangerously fast. Less dramatic instances of the distortion of the time sense are provided by cases where the duration of individual stimuli are contracted or dilated. Repeated presentation of a particular stimulus results in successive stimuli being judged to last for a shorter period of time than the initial stimulus (Rose and Summers 1995): familiarity breeds, if not contempt, then contraction. If an unexpected stimulus is introduced in this series, it is judged to have lasted longer than the other, familiar, stimuli, even though it was presented for the same amount of time (Pariyadath and Eagleman 2007). Is there a single mechanism here that is responsible for our duration and speed judgements, and which affects our time sense in general, or is the explanation atomistic, where the contraction or dilation of individual stimuli do not have consequences for judgements concerning other stimuli? Thinking about the vase example and the Zeitrafferphänomen, we may be tempted by the model of the internal pacemaker. Imagine a biological counter which emits ‘clicks’ or regular pulses. When a stimulus begins, these clicks are recorded by the short-time memory. When the stimulus ends, the number of clicks can be compared with the number of clicks associated with earlier stimuli, so as to form the basis of a judgement concerning relative duration (Treisman 1999). The rate at which the pacemaker

Perception and Time   473 emits its clicks can be altered in various ways: by emotional states, drugs, damage, unusual stimuli, or just ageing. The faster the rate of clicks, the slower things will appear to happen, so we can conjecture that emotion increases the rate of clicks. The slower the rate of clicks, the faster things will appear to happen, so evidently the pacemakers of older people are slowing down, explaining their common complaint that the years pass much more quickly than they used to. The pacemaker model, however, should affect all contemporaneous time judgements equally. But the contraction or dilation of the duration of certain stimuli can, it seems, be specific to the stimulus. So, for example, where an unfamiliar stimulus appeared in a series, resulting in a dilated duration estimate for that stimulus, estimates of other stimuli presented at the same time are unaffected (Pariyadath and Eagleman 2007). One possible hypothesis for this result is that perceived duration is correlated with the amount of neural energy required to represent a stimulus. Unfamiliar stimuli require more neural energy than familiar ones (ibid.). What this suggests is that the psychological representation of time is not a unified whole, but rather involves several parallel neural mechanisms (Eagleman 2008). What is the significance of this? If it means that phenomenal time time as it is experienced is not a unified whole, that it actually consists of fragmentary, disconnected conscious states, then this has some significance for the philosophical issues we have been discussing. For talk of ‘the experienced present’ implies that phenomenal time is unified: the contents of perception are experienced as co-present. So any experimental results supporting the disunity of phenomenal time would undermine the notion of ‘the’ experienced present, and so put a further nail in the coffin of the argument from experience to the objective passage of time. But whether those results really do bear that interpretation, and whether, if they do, that puts any pressure on the idea of an experienced present, is a matter for future debate. This is just a sample of time experiences that make us rethink our understanding of the way in which the brain’s registering of time corresponds, or fails to correspond, to time’s true nature. And psychological models of those experiences have much to contribute to the curious mixture of the empirical and the a priori that is characteristic of the philosophy of perception.1

References Bergson, Henri (1911). Creative Evolution, (trans.) Arthur Mitchell. London: Macmillan. Binkofski, F. and Block, R. A. (1996). ‘Accelerated Time after Left Frontal Cortex Lesion’. Neurocase, 2, 485–493. Bourne, Craig (2006). A Future for Presentism. Oxford: Clarendon Press. Broad, C. D. (1923). Scientific Thought. London: Kegan Paul. Crisp, Thomas M. (2004). ‘On Presentism and Triviality’. Oxford Studies in Metaphysics, 1, 15–20. Dainton, Barry (2010a). Time and Space, 2nd ed. Durham: Acumen. Dainton, Barry (2010b). ‘Temporal consciousness’. Stanford Encyclopedia of Philosophy.

Dennett, D. C. and Kinsbourne, M. (1992). ‘Time and the Observer: The Where and When of Consciousness in the Brain’. Behavioral and Brain Sciences, 15, 183–247. 1  Many thanks to Mohan Matthen for very helpful comments on previous versions of this chapter.

474   Robin le Poidevin Eagleman, David M. (2008). ‘Human Time Perception and its Illusions’. Current Opinion in Neurobiology, 18, 131–136. Grush, Rick (2007). ‘Time and Experience’. In Thomas Muller (ed.), The Philosophy of Time (pp. 27–44). Frankfurt: Klosterman. Haggard, P., Clarl, S., and Kalogeras, J. (2002). ‘Voluntary Action and Conscious Awareness’. Nature Neuroscience, 5, 382–385. James, William (1890). The Principles of Psychology, repr. in one volume. Cambridge, MA: Harvard University Press, 1983. Kelly, Sean D. (2005). ‘The Puzzle of Temporal Experience’. In Andy Brook and Kathleen Akins (eds), Philosophy and Neuroscience (pp. 208–239). Cambridge: Cambridge University Press. Le Poidevin, Robin (2007). The Images of Time:  An Essay on Temporal Representation. Oxford: Oxford University Press. Matthen, Mohan (2010). ‘On the Diversity of Auditory Objects’. Review of Philosophy and Psychology, 1, 63–89. Mellor, D. H. (1998). Real Time II. London: Routledge. Nijhawan, Romi (1994). ‘Motion Extrapolation in Catching’. Nature, 370, 256–257. Olson, Eric (2009). ‘The Rate of Time’s Passage’. Analysis, 69, 3–9. Pariyadath V. and Eagleman, D. M. (2007). ‘The Effect of Predictability on Subjective Duration’. PLoS ONE, 2, 1,264. Pinecoffin, R. S. (1961). Saint Augustine: Confessions. Harmondsworth: Penguin. Power, Sean (2010). ‘Perceiving External Things and the Time-Lag Argument’. European Journal of Philosophy. DOI: 10.1111/j.1468-0378.2010.00436.x. Prosser, Simon (2007). ‘Could we Experience the Passage of Time?’, Ratio, 20, 75–90. Prosser, Simon (2012). ‘Why does Time Seem to Pass?’ Philosophy and Phenomenological Research, 85, 92–119 Prosser, Simon (2013). ‘Passage and Perception’. Noûs, 47, 69–84. Rose, D. and Summers, J. (1995). ‘Duration Illusions in a Train of Visual Stimuli’. Perception, 24, 1,177–1,187. Sklar, Lawrence (1974). Space, Time and Spacetime. Berkeley, CA: University of California Press. Stetson, C., Cui X, Montague, P. R., and Eagleman, D. M. (2006). ‘Motor-Sensory Recalibration Leads to an Illusory Reversal of Action and Sensation’. Neuron, 51, 651–659. Treisman, Michel (1999). ‘The Perception of Time: Philosophical Views and Psychological Evidence’. In Jeremy Butterfield (ed.), The Arguments of Time (pp. 217–246). Oxford: Oxford University Press. Wertheimer, M. (1912). ‘Experimentelle Studien über Sehen von Bewegung’. Zeitschrift für Psychologie, 61, 161–265. Yarrow, K., Haggard, P., Heal, R., Brown, P., and Rothwell, J. C. (2001). ‘Illusory Perceptions of Space and Time Preserve Cross-Saccadic Perceptual Continuity’. Nature, 414, 302–305.

Chapter 25

Speech Perception Casey O’Callaghan

Philosophers have devoted tremendous effort to explicating what it takes to understand language. The answers focus on things such as possessing concepts, mastering grammar, and grasping meanings and truth conditions. The answers thereby focus on extra-perceptual cognition. Understanding spoken language, however, also involves perception—grasping a spoken utterance requires hearing or seeing it. Perception’s role in understanding spoken language has received far less philosophical attention. According to a simple view, understanding speech is just a matter of assigning meaning to the sounds you hear or to the gestures you see. If so, what perception contributes to understanding spoken language is not distinctive to the case of spoken utterances. Against this, however, is the prospect that speech is special. In this chapter, I present and evaluate the evidence that speech perception differs from non-linguistic auditory perception. In particular, I discuss the phenomenology, contents, objects, and mechanisms of speech perception. I make proposals about the ways in which speech is and is not perceptually special. According to the account I offer, the capacity to perceive speech in a manner that enables understanding is an acquired perceptual skill. It involves learning to hear language-specific types of ethologically significant sounds. According to this account, while the contents of perceptual experience when listening to familiar speech are of a variety that is distinctive to hearing spoken utterances, perceiving speech involves neither novel perceptual objects nor a unique perceptual modality. Much of what makes speech special stems from our fierce interest in it.

1  Is speech perceptually special? There is a thriving debate about whether the human capacity to use and understand language is special (see, e.g., Hauser et al., 2002; Pinker and Jackendoff, 2005). A key part of this wider debate is whether the capacity to speak and understand speech is special (see, e.g., Liberman, 1996; Trout, 2001; Mole, 2009). My concern here is with speech perception. Is the human capacity to perceive spoken language special?

476   Casey O’Callaghan To be special requires a difference. However, the debate about whether speech is special is not just about whether speech perception in some respect differs from other forms of perception. It concerns whether speech perception should be distinguished as a distinctive or a unique perceptual capacity. Put in this way, the question relies on a comparison. The most common contrast is with general audition. The question thus is whether speech perception differs or is a distinct perceptual capacity when compared with non-linguistic auditory perception. A  separate contrast is with the capacities of non-human animals. Is speech perception uniquely human? The contrast between human and non-human responses to spoken language is frequently used to illuminate the contrast between human speech perception and non-linguistic audition. A difference is a difference in some respect, and being distinctive or unique is being distinctive or unique in some way, for some reason. In what respects is speech special? It is helpful to divide the candidates into four broad classes. The first concerns the phenomenology of speech perception. Does what it is like to perceptually experience spoken utterances contrast with what it is like to perceptually experience non-linguistic sounds and events? One way to make progress on this question is to ask whether the perceptual experience of hearing speech in a language you know differs phenomenologically from that of hearing speech in an unfamiliar language. The second concerns the contents of speech perception. Does the perceptual experience of speech involve contents absent from non-linguistic auditory experience? Does understanding a language affect which properties perceptual experiences represent spoken utterances to have? The third concerns the objects of speech perception. Are the objects of speech perception distinct from the objects of non-linguistic audition? Does speech perception share objects with non-linguistic audition? The fourth concerns the mechanisms of speech perception. Does perceiving speech involve perceptual processes that differ from those involved in perceiving non-linguistic sounds and events? Does speech perception involve a special perceptual module? Is speech perception the work of a distinct perceptual modality? Answering the question, ‘Is speech special?’ thus means addressing a number of different questions. This essay focuses on the contrast between speech perception and human non-linguistic auditory perception. I  distinguish the various respects in which speech might be special when compared with non-linguistic audition. I assess the evidence and advance proposals about the respects in which speech perception is special.

2 Phenomenology Is perceiving speech phenomenologically special? Is what it’s like, for the subject, to perceptually experience speech different, distinctive, or unique when compared with non-linguistic audition? It is natural to think that the perceptual experience of listening to spoken language differs phenomenologically from the perceptual experience of listening to non-linguistic sounds, simply because speech sounds and non-linguistic sounds differ acoustically.

Speech Perception   477 Hearing the sound of a drop of water differs phenomenologically from hearing the sound of the spoken word ‘drop’ because the sounds differ in their basic audible qualities. However, the perceptual experience of spoken language may also involve distinctive phenomenological features that are absent from non-linguistic auditory experience. Start with the experiential contrast between listening to non-linguistic sounds and listening to spoken language. Begin with the case of a language you know. The experience of listening to speech in a language you know differs noticeably from the experience of listening to ordinary, non-linguistic environmental sounds, even once we eliminate acoustical differences. The phenomenological shifts associated with sinewave speech support this claim. Sinewave speech is an artificial signal in which an acoustically complex human voice is replaced by several sinewaves that vary in frequency and amplitude with the primary formants of the original speech signal, while removing acoustical energy at other frequencies (Remez et al., 1981). At first, it is difficult to recognize the sounds of sinewave speech as speech sounds. Instead, they just sound like computer-generated noises. However, after hearing the original human speech from which the sinewave speech is derived, it is easy to hear what the sinewave speech says. The same stimulus is first experienced as non-speech sounds, and then it is experienced as speech. And this change is accompanied by a dramatic phenomenological shift. In the case just described, you come to comprehend the speech. Thus, understanding might suffice to explain the phenomenological difference when you are listening to speech in a language you know. You grasp meanings, so the experiential difference could in principle be explained in terms of cognitive, rather than perceptual, phenomenology. (This explanation is unavailable if you reject that extra-perceptual cognition has proprietary phenomenology.) To control for any contribution from understanding, consider the experiential contrast between listening to non-speech sounds and listening to speech in a language you do not know. Is there any phenomenological difference? It is possible reliably to discriminate speech in a language you do not understand from ordinary environmental sounds. Neonates prefer speech sounds to non-speech sounds though they do not understand language. In addition, sinewave speech in a language you do not know may appear first as non-speech sounds and then as speech. Thus, we have evidence that perceptually experiencing a stimulus as speech rather than as non-speech sounds makes a phenomenological difference that does not depend on understanding. Understanding spoken utterances need not, however, contribute exclusively to the phenomenology of extra-perceptual cognition. Knowing a language may also impact the phenomenal character of perceptual experience. Consider the phenomenological contrast between the perceptual experience of listening to speech in a language you know and of listening to speech in an unfamiliar language. Of course, languages differ acoustically in ways that affect how they sound. For instance, whether or not you know Hindi, it sounds different from German. To control for acoustical differences that affect phenomenology, fix the language. Contrast the experience of a person who knows the language with that of a person who does not know the language when faced with the same spoken utterance. Or, consider a person’s experience prior to and after learning the language. Many philosophers agree that knowing the language affects the phenomenological character of perceptual experience, even while they disagree about the diagnosis (see O’Callaghan, 2011: 784-787).

478   Casey O’Callaghan What is the source of the difference? Speech in a language you know differs perceptually in several respects. Most obviously, your perceptual experience of its temporal characteristics differs. When you know the language, audible speech does not seem like an unbroken stream of sounds. It seems instead to include discernible gaps, pauses, and other boundaries between words, clauses, and sentences, and you are able perceptually to resolve qualitative features and contrasts at a much finer temporal grain. Familiar speech also appears in other respects to differ qualitatively from unfamiliar speech. For instance, when you have mastered a spoken language, you are able to detect subtle qualitative features and their contrasts, such as the difference between ‘s’ and ‘z’, or the dropped ‘g’ or ‘t’ of certain accents. The stimulus sounds different and more detailed when you recognize it as speech and you know the language. The argument of the last paragraph, unlike the argument from sinewave speech, requires comparing phenomenology across subjects or across long stretches of time. Thus, it is more controversial. An alternative way to establish the point is to compare the shift that occurs with sinewave speech in a language you know with the shift that occurs with sinewave speech in a language you do not know. In each case, recognizing the sounds as speech leads to a shift in phenomenal character. The change, however, is far more dramatic when you know the language. The difference between the two phenomenological contrasts is the difference that accrues thanks to knowing the language. These arguments indicate that one’s perceptual experiences may differ phenomenologically when listening to speech in a known language, when listening to speech in an unfamiliar language, and when listening to non-speech sounds. Moreover, such phenomenological differences can be evoked even when we have controlled for acoustical differences. This supports the following two claims: knowing a language impacts the phenomenal character of perceptual experience when listening to spoken utterances; and, speech perception has phenomenal features that are distinctive when compared with non-linguistic audition.

3 Contents Content concerns how things are represented to be. Content thus concerns things that are perceptually experienced and the features they are perceptually experienced to have. One way to characterize the contents of perceptual experiences appeals to their accuracy or veridicality conditions. Some prefer to speak of what a given perceptual experience purports to be facts about the world, or of how things seem or appear perceptually. Some philosophers hold that perceptual experiences differ phenomenologically only if they differ in how they represent things as being. Some also hold that there is a variety of content such that perceptual experiences differ in content only if they differ phenomenologically. In either case, a difference in content may help to explain the sorts of phenomenological differences mentioned in Section 2. What we perceive when we perceive speech may, in this sense, differ from what we perceive when we perceive non-speech sounds. Speech perception may involve contents that are special or distinctive when compared with non-linguistic audition. In what respects does the content of speech perception differ from that of non-linguistic audition? The characteristic sounds of human vocalization differ acoustically from the

Speech Perception   479 sounds of non-linguistic happenings such as blowing leaves, backfiring automobiles, and violins. The perceptual experience of speech reflects this. Such evident qualitative differences, which are underpinned by acoustical differences, are part of why sinewave speech at first sounds like meaningless computer noise, and why artificial speech often sounds inhuman. Perhaps, then, the perceptual experience of speech differs phenomenologically from the perceptual experience of non-linguistic sounds and happenings because its perceptually apparent features differ in a way that is recognizable and distinctive to spoken language. This is compatible with an austere view of the types of features that one perceptually experiences when listening to speech or to non-speech sounds. The phenomenological difference between perceptually experiencing speech and non-speech may just stem from a difference in the patterns of low-level properties that the perceptual experiences represent. For instance, it may just stem from a difference in the apparent pattern of pitch, timbre, and loudness of a sound stream over time. Any further experiential differences may result from extra-perceptual cognition, such as thought or imagination. This austere picture also suggests an attractive account of how perceptually experiencing speech in an unfamiliar language differs phenomenologically from perceptually experiencing speech in a language you know. As discussed in Section 2, the audibly apparent temporal and qualitative features of spoken utterances in a language you know generally differ from those of speech in a language that is unfamiliar to you. Foreign language may sound like a continuous stream of indistinct babble, but familiar speech perceptually appears to be chunked into units that correspond to words and phrases and to include discernible gaps, pauses, and boundaries that distinguish such units from each other. Hearing familiar language also involves the capacity to perceptually experience sublexical features at a finer temporal grain, and to discern linguistically significant qualitative details and contrasts that you could not make out before. Conversely, it also involves failing to discern other qualitative contrasts that are linguistically irrelevant. Thus, in these ways, differences in the perceptually apparent pattern of individual sounds and low-level audible qualities such as pitch, timbre, and loudness over time may explain the phenomenological difference that knowing a language makes. Nevertheless, such an austere account might not suffice. Some philosophers have claimed that grasping meanings or semantic properties contributes in a constitutive rather than merely causal manner to the phenomenal character of perceptual experience. They argue therefore that listening to spoken utterances when you know the language involves perceptually experiencing meanings or semantic properties (e.g., McDowell, 1998; Siegel, 2006; Bayne, 2009). According to such an account, perceptual experiences may represent or involve awareness not just as of low-level sensible features, such as pitch, timbre, loudness, and timing, but also as of high-level features, including semantic properties. Such an account supports a liberal view about what types of properties may be represented by episodes of perceptual experience (see, e.g., Siegel, 2006; Bayne, 2009). The liberal view of speech perception’s contents faces an objection if it also must explain the phenomenological difference between the perceptual experience of listening to speech in a familiar language and of listening to speech in an unfamiliar language. The account requires that, for an utterance you understand, there is something distinctive it is like for you to perceptually experience its specific meaning. That is because nothing suggests you could not hear foreign utterances as meaningful if that does not require hearing specific

480   Casey O’Callaghan meanings. Hearing meaningfulness, if not specific meanings, for instance, could help to explain the phenomenological difference between hearing speech in an unfamiliar language and hearing non-linguistic sounds. However, perceptually experiencing specific meanings can also account for the difference between hearing familiar and unfamiliar speech. Suppose, therefore, that you perceptually experience specific meanings, rather than just meaningfulness. Thus, differences in apparent meaning should impact the phenomenal character of perceptual experience for utterances in a known language. But consider homophonic utterances, which share pronunciation but not meaning. Homophonic utterances do not apparently cause perceptual experiences that differ in phenomenal character. For instance, even when they are embedded appropriately in meaningful linguistic contexts, the perceptual experience of hearing an utterance of ‘to’ does not clearly differ in phenomenal character from the perceptual experience of hearing an utterance of ‘too’ or ‘two’ (the same holds for homographic homophones). Complete sentences present a similar problem. Utterances of structurally ambiguous statements, such as, ‘Visiting relatives can be boring’, and those with scope ambiguities, such as, ‘Everyone chose someone’, may not, under their differing interpretations, lead to perceptual experiences that differ phenomenologically. The argument from homophones thus casts doubt on the claim that specific meanings make a distinctive difference to the phenomenal character of perceptual experience (O’Callaghan, 2011). A moderate account denies that the perceptual experience of speech includes awareness as of meanings or high-level semantic properties. It nevertheless explains the phenomenological difference that accrues thanks to knowing a language using resources beyond the austere account’s low-level acoustical features. According to one such account, listening to speech in a familiar language involves the perceptual experience of language-specific but non-semantic properties of spoken utterances. Phonological features, such as phones and phonemes, form the basis for recognizing and distinguishing spoken words. Phonological features in general are respects of discernible non-semantic similarity and difference among utterances that may make a semantic difference. Phonological features are like the basic perceptible vocabulary or ‘building blocks’ of spoken language.1 To illustrate, consider utterances of ‘bad’, ‘imbue’, and ‘glob’. In one respect, these utterances are perceptibly similar. Each apparently shares with the others the ‘b’ sound—[b]‌in phonological notation. Next consider utterances of ‘lab’ and ‘lash’. They perceptibly match, except that the former contains the ‘b’ sound and the latter contains the ‘sh’ sound—[∫] in phonological notion. The phones [b] and [∫] are examples of features which may be shared among distinct spoken utterances, which may differ among otherwise indistinguishable utterances, and which may make a semantic difference. Distinct phones are distinguished by a perceptible difference that is linguistically significant in some human language. A phone thus is usefully understood in terms of a type whose members make a common linguistic contribution to any given language. One phone is distinguished from another by some perceptually discernible difference that is or may be exploited by some spoken language to signal a semantically significant difference. 1  Here I am alluding to but not endorsing the notorious ‘beads on a string’ analogy. I do not accept that characterization of phonological attributes, because I believe neither that they are items or individuals nor that they occur in neat, discrete sequences. Instead, I believe they are properties whose instances overlap. Further discussion in Section 4.

Speech Perception   481 Since phones are the minimal perceptible features that make a linguistic difference in some world language, they are in this sense the perceptible ‘building blocks’ of spoken language. Specific spoken languages do not all make use of this basic stock of building blocks in the same manner. Some spoken languages, for instance, include clicks and buzzes, while others do not. Moreover, spoken languages may, even when they make use of the same basic stock, differ in which classes of utterances they treat as linguistically equivalent and in which classes of utterances they treat as distinct. For example, spoken English distinguishes [l]‌from [r],2 but Japanese does not. Thus, the phones [l] and [r] correspond to distinct English phonemes, /l/ and /r/, but are allophones or linguistically equivalent variations of a single Japanese phoneme. Another example is that [p] and [ph] are allophones of the English phoneme, /p/, but Mandarin Chinese treats them as distinct phonemes, /p/ and /ph/. The difference between [p] and [ph] suffices to distinguish Chinese but not English words. So, some languages treat [p] and [ph] as allophones of a single phoneme, while others treat them as distinct phonemes that may suffice for a semantic difference. Phonemes thus may usefully be understood in terms of language-specific classes whose members are treated as linguistically equivalent, or as allophonic, within the context of that spoken language, even if under certain conditions its members may be perceptually distinguishable. A language’s phonemes are distinguished from one another by perceptually discernible differences that are semantically significant. The lesson is that certain utterance pairs are treated as linguistically equivalent by some languages but as linguistically distinct by others. Thus, spoken languages yield differing families of equivalence classes of utterances that make a common semantic contribution. So, the way in which a basic stock of speech sounds, which have the potential to signal semantic difference, in fact is utilized by a particular language is specific to that language. A language’s stock of linguistically significant sound types is distinctive to that language. Since phonemes differ across languages, discerning a language’s phonemes requires substantial exposure and learning. That such features may be perceptually experienced nonetheless helps to explain patterns of similarity and difference among utterances that are apparent to users of a given language. The capacity perceptually to discern such similarities and differences is critical to understanding spoken language. It is not, however, explained by the perceptual experience of low-level audible attributes alone. What is noteworthy is that users of a given language commonly treat certain crucial pairs of sounds or utterances as perceptibly equivalent, while those who do not know that language treat them as perceptibly distinct. For example, auditory perceptual discrimination tasks in linguistic contexts reveal that the sounds corresponding to ‘t’ in utterances of ‘ton’ and ‘stun’ auditorily appear virtually the same to fluent monolingual English users, but appear noticeably to differ to fluent monolingual users of Chinese. Spoken utterances of ‘bed’ and ‘bad’ in linguistic contexts differ audibly to English speakers but not to Dutch speakers. Speakers of one language may discern a common linguistic sound across utterances that differ acoustically while speakers of another language do not. So, suppose we have two groups of language users. Suppose all are attentively listening, and that each is presented with two sounds uttered by the same talker in a linguistic context. Those in the

2  For readability, I use the upright rather than inverted ‘r’ for the alveolar approximant. The upright ‘r’ standardly (in the International Phonetic Alphabet) is used for the trill.

482   Casey O’Callaghan first group do not notice a difference between the speech sounds. They judge that they are audibly equivalent, and they behave as if the sounds are equivalent. Those in the other group do notice a difference between the speech sounds. They judge that they audibly differ, and they behave as if the sounds are not audibly equivalent. In this case, for at least one of the speech sounds, it is plausible to say that the perceptual experience of a language listener from the first group differs phenomenologically from the perceptual experience of a listener from the second group. If so, then for a large class of linguistic sounds, the perceptual experience of someone who knows a given language may differ from the perceptual experience of someone who does not. If only those who know a spoken language perceptually experience its language-specific phonological attributes, such as its phonemes, then this provides an attractive explanation for the difference. For instance, having a perceptual experience that represents the English phoneme /l/, rather than /r/, may explain why hearing an utterance of ‘law’ differs phenomenally from hearing an utterance of ‘raw’. Having perceptual experiences as of a single English phoneme explains a monolingual English speaker’s failure to perceptually distinguish utterances of distinct Chinese words. A central part of the phenomenological difference that accrues thanks to knowing a language thus stems from the perceptual experience of attributes whose linguistic significance is specific to that language. The perceptual experience of language-specific features explains apparent patterns of similarity and difference that to a noteworthy degree are independent from lower-level audible attributes, such as pitch, timbre, and loudness over time. For instance, the low-level audible qualities of an utterance of /p/ vary across phonological contexts, speakers, moods, and social contexts. The perceptual experience of a single phoneme explains this kind of perceptually apparent sameness in the face of differing lower-level audible qualities. On the other hand, the same acoustical signal may appear as a /p/ in some contexts and as a /b/ or /k/ in another. In different contexts, distinct apparent phonemes may accompany matching low-level audible qualities. A moderate account of this sort finds converging support from three sources of evidence. First, developmental evidence shows that young infants discern a wide variety of phonetic differences that are linguistically significant in various languages. However, between five and twelve months, infants cease to discern phonetic differences that are not linguistically significant in the languages to which they have been exposed. Babies in Pittsburgh stop distinguishing utterances that differ with respect to [p]‌and [ph], and babies in Madrid stop distinguishing utterances that differ with respect to [s] and [z]. Such pruning requires regular exposure to the language, and it is part of learning to become perceptually responsive to the features that are distinctive to a spoken language. Children thus learn to hear the sounds of their language (see, e.g., Eimas et al., 1971; Jusczyk, 1997). Second, adult perception of certain critical speech sounds, such as stop consonants, is categorical (see Chapter XX, this volume; Harnad, 1987). This means that, in critical cases, such as the perception of stop consonants, gradually varying the value of a diagnostic physical parameter leads to uneven perceptual variation. For example, suppose we start with a stimulus experienced as /ba/ and gradually increase its voice onset time. At first, this makes little difference. At some point, however, the stimulus abruptly appears to shift to a /pa/. In a dramatic case of categorical perception, the change seems perfectly abrupt. Thus, given a boundary that is diagnostic for a perceptual category, stimuli that differ by a certain physical magnitude may differ only slightly in perceptual appearance when each

Speech Perception   483 falls within that boundary; however, stimuli that differ by that same physical magnitude may differ greatly in perceptual appearance when one but not the other falls within the boundary. Patterns of categorical perception in fact vary accordingly. Adult categorical perception of speech sounds corresponds to language-specific phonological categories, generally those of the listener’s first language (though there is some flexibility). Perceptual awareness of phonological features thus helps to explain both perceptually apparent patterns of similarity and difference among utterances within a language and variation in patterns of apparent similarity and difference across speakers of different languages. Third, evidence from aphasias, language-related disorders, suggests that the capacity to understand spoken language normally requires the capacity to perceive language-specific attributes of speech that are not meanings. Moreover, the latter capacity affects the phenomenal character of auditory perceptual experience. Individuals with transcortical sensory aphasia (TSA) have a severely impaired capacity to grasp and to understand linguistic meanings, but they retain the capacities to hear, to generate, and to repeat spoken utterances. They commonly are unaware of their disorder. In contrast, individuals with pure word deafness (PWD) have intact semantic capacities but lack the capacity to perceive spoken language as such. Individuals with PWD are unable to hear sounds or utterances as spoken words or linguistic units. Their deficit is limited to auditory language perception. They may learn to use sign language or even read lips. And their hearing otherwise remains normal. They can hear and recognize barking dogs, cars, and even the sounds of familiar voices. Individuals with PWD say, however, that words fail to ‘come up’ and describe the auditory experience of spoken language as like hearing garbled sound or foreign language (see, especially, Poeppel 2001: 681). These descriptions of TSA and PWD suggest that there is an important phenomenological difference in perceptual experience that stems from being able to discern and to recognize language-specific features but that does not require the capacity to discern and to recognize the meanings of spoken utterances. Auditorily experiencing language-specific features other than meanings therefore plausibly captures this difference. Phonological and other structural features of spoken utterances are good candidates.3 Appealing to the content of perceptual experience thus helps to explain what is distinctive about the perceptual experience of listening to speech. In particular, two sorts of features help to account for the difference between the perceptual experience of listening to unfamiliar speech and of listening to speech in a language you know. When you know a language, the patterns of determinate low-level audible attributes you perceptually experience differ from when you do not know the language. This difference concerns the specific arrangement of low-level qualitative and temporal attributes, each of which you could, in principle, perceptually experience even non-linguistic sounds to bear. However, understanding speech also involves perceptually experiencing spoken utterances to bear language-specific attributes, including phonological properties such as phonemes. Developing the capacity to perceptually experience such language-specific features requires exposure and perceptual learning. Its exercise is part of any adequate explanation

3 Indeed, individuals with PWD perform poorly on tasks that require categorical perception for language-specific attributes. Thanks to Bob Slevc for discussion.

484   Casey O’Callaghan for the experiential difference that accrues thanks to knowing a language. While I have expressed doubt that meanings and high-level semantic properties are represented by perceptual experiences, I leave open whether and which additional language-specific features are among the contents of perceptual experience when listening to speech. For instance, you may perceptually experience morphemes, lexemes, or even grammatical properties when you listen to speech in a language you understand. Greater attention to the ways such features affect the phenomenal character of perceptual experience will inform broader debates about the richness of perceptual experience—that is, about the types of features awareness of which constitutively shapes the phenomenal character of perceptual experience. This, in turn, should impact how we understand the interface of perception with cognition.

4 Objects The previous section argued that the perceptual experience of speech differs in content from non-linguistic audition. This section concerns whether the objects of speech perception differ from those of non-linguistic audition. There are two ways to understand the objects of perception. Construed broadly, the objects of perception simply are targets of perception, and may include particular individuals, their attributes, happenings, or states of affairs. In this broad sense, to be an object of perception is just to be perceived. According to some accounts, objects of perception in the broad sense are the components of content. In Section 3, I proposed that the perceptual experience of speech involves awareness as of language-specific features. So, in the broad sense, the objects of speech perception are special when compared with those of non-linguistic audition. Construed more narrowly, however, the objects of perception are the individuals that bear perceptible attributes. In this narrow sense, vision’s objects might include ordinary material objects that look to have attributes such as shape and colour, and audition’s objects plausibly include individual sounds that have pitch, timbre, and loudness. Further philosophical debates concern the natures of the objects of perception, including whether they are public or private. The phenomenological differences between speech perception and non-linguistic audition, especially since they are dramatic, might be taken to suggest that the objects of speech perception in this sense differ from those of non-linguistic audition. This discussion concerns whether the objects of speech perception are special in the narrow sense that includes only individuals. In one respect, it is trivial that the objects of speech perception differ from those of non-linguistic audition. One case involves perceiving speech, and the other involves perceiving non-speech. At the very least, perceiving speech involves perceiving sounds of a kind to which non-linguistic sounds do not belong, and vice versa. Speech sounds and non-linguistic sounds differ in their causes, their sources, and their effects, as well as in their semantic and other linguistic properties. The claim that speech perception and general audition have different objects typically is not just the claim that they involve hearing different kinds of sounds or sounds with distinctive features. Speech perception researchers have claimed that the objects of speech perception are not sounds at all. This is a claim about the sorts of individuals perceived

Speech Perception   485 when one perceives speech. In particular, it is the claim that while the individuals you perceive in non-linguistic auditory perception are sounds, the individuals that you perceive when you listen to speech are not sounds. The objects of speech perception instead are individuals of a wholly different sort. Three main sorts of argument are offered. The first type of argument appeals to the mismatch between salient features of the objects of speech perception and features of the acoustic signal. We can reconstruct the argument in the following way. The objects of non-linguistic audition are sounds. The perceptible features of sounds correspond to aspects of the acoustic signal. But, the perceptible features of speech do not correspond to aspects of the acoustical signal. The perceptible features of speech are, thus, not perceptible features of sounds. So, the objects of speech perception differ from those of non-linguistic audition. This argument can be illustrated using the case of apparent phonological features, such as phones or phonemes. The acoustic attributes that correspond to a perceived phonological feature vary greatly depending upon setting and context. Not only do they vary in expected ways, with speaker, mood, and accent, but they also depend locally upon the surrounding linguistic context. For example, phonological features are not uttered in discrete, isolated units. Instead, they are articulated in a continuous stream that flows gradually from one to the next. This has two noteworthy consequences. First, information about one phoneme is blended with information about surrounding phonemes. Because distinct speech sounds are coarticulated, when I utter ‘imbue’, the fact that the /i/ is followed by /m/ shapes how I pronounce the /i/. This differs from how I pronounce the /i/ when it is followed by /d/, as in ‘idiom’. In fact, no clear invariant acoustic signature corresponds to an utterance of a given phoneme in all of its perceptible instances. And a given acoustical configuration might contribute to distinct apparent phonemes in different contexts. Second, some have been inclined to say that perceptible speech appears to be segmented into discrete phonemes. However, the acoustic information by which you discern the presence of a given phoneme is present during the utterance of surrounding phonemes. For instance, the acoustical information corresponding to /æ/ in an utterance of ‘dab’ is present during the articulation of both the /d/ and the /b/ (and vice versa). Thus, no clear acoustic boundaries correspond to any segmentation that is apparent between adjacent phonemes. Therefore, there exists no consistent, context-independent, homomorphic mapping between apparent phonemes and straightforward features of the acoustic signal (see, e.g., Appelbaum, 1999; Remez and Trout, 2009).4 This point should be evident to anyone who has laboured with speech recognition software. It leads some philosophers to anti-realism about phonological features. Rey (2012), for instance, holds that phonemes are intentional inexistents (see also Smith, 2009). In light of this, Liberman et al. (1967) and other early proponents of the Motor Theory famously proposed that the objects of speech perception are not sounds at all, but instead are something involved in the pronunciation of speech (see also the papers collected in Liberman, 1996). The core idea is that features of perceived speech do map in a homomorphic, invariant way onto types of gestures involved in the production of speech. For 4  Early

text-to-speech methods failed to appreciate this context dependence, and thus failed. Early attempts assigned each letter a sound and played the sounds assigned to specific letters in sequences that mirrored written texts. The results were unintelligible.

486   Casey O’Callaghan instance, pronouncing an instance of /d/ involves stopping airflow by placing the tongue at the front of the palate behind the teeth and then releasing it while activating the vocal folds. Pronouncing /b/ involves a voiced release of air from pursed lips. Such articulatory gestures, and the component configurations and movements they comprise, make the manner in which speech is perceptually experienced intelligible in a way that attention to the acoustic signal does not, since such gestures and their descriptions are less sensitive to context.5 The claim was that the acoustical signal encodes information about articulatory gestures and their features. If articulatory gestures and their features rather than sounds and their attributes are the best candidates for what we perceive when we are perceptually aware of instances of phonemes, then articulatory gestures are the objects of speech perception. Thus, the objects of speech perception and of non-linguistic audition differ in kind. The former are articulatory gestures with phonological characteristics, and the latter are sounds with audible attributes. These arguments do not establish that the bearers of phonological features are not bearers of non-linguistic audible attributes. Thus, they do not establish that the objects of speech perception include individuals of a wholly different kind from the objects of non-linguistic audition. On one hand, the mismatch argument relies on the presumption that ordinary auditory awareness does map in an invariant, homomorphic way onto features of the acoustic stimulus. However, even pitch, an apparently simple audible quality, has a complex relationship to frequency. In addition, context effects abound. For instance, varying the attack of a sound affects its timbre, and the apparent duration of a tone is affected by the duration of a tone presented earlier or even later. More generally, the apparent objects of auditory awareness in acoustically complex environments do not map clearly and in invariant ways onto straightforward features of the acoustic signal. Nothing obvious in an acoustical stream signals how to distinguish the sound of a guitar from the sound of a voice in a crowded bar. The central lesson of work on auditory scene analysis is that ordinary sounds are individuated—they are distinguished from each other at a time, and they are tracked and segmented over time—in the face of highly complex, interwoven acoustic information (Bregman, 1990). On the other hand, the argument also relies on the presumption that non-linguistic audition’s objects do not map in an illuminating way onto the events that produce acoustic information. However, audition’s vital function is to provide perceptual access to events in the environment. Accordingly, human audition carves up the acoustical scene in a way that is predicated upon an interest in identifying sound sources. In fact, the way in which sounds are individuated suggests that the objects of non-linguistic auditory perception include sound sources rather than mere acoustical events or sound streams. In the face of complex, entangled acoustical information, you distinguish the sound of the guitar from the sound of the voice because they have distinct sources. We attend to and identify sounds relative to sources, and this is reflected in our thought and talk about sounds, which concern, for instance, the sound of the car door, the sound of the dog, the sound of scratching. 5  One

complication is that due to coarticulation the gestures pronounced in normal speaking also exhibit some lack of invariance. Liberman and Mattingly (1985) revised the Motor Theory to claim that intended motor commands are the objects of speech perception. See Mole (2009) for a convincing critique of the revised account. Fowler’s (1986) Direct Realism maintains that articulatory gestures are the objects of speech perception but rejects that gestural events differ in kind from the objects of non-linguistic audition.

Speech Perception   487 Many descriptive sound words are source oriented: rattle, bang, crack. So, just as articulatory gestures illuminate the manner in which the objects of speech perception are individuated and classified (see Matthen, 2005), considering the environmental happenings that make sounds illuminates the manner in which the objects of non-linguistic auditory perception are individuated and classified (see, e.g., Nudds, 2010). Audition’s objects thus fail to map in an invariant, homomorphic manner onto simple physical properties of an acoustic stimulus, and sound sources help to explain the manner in which audition’s objects are individuated and classified. In these respects, non-linguistic audition does not differ from speech perception. The mismatch argument fails. The second type of argument is that cross-modal influences in the perception of speech reveal that the objects of speech perception differ in kind from the objects of non-linguistic audition (see, e.g., Trout, 2001, for discussion). The McGurk effect is one powerful example (McGurk and Macdonald, 1976). Subjects presented with audio of an utterance of the velar /ga/ along with video of a speaker uttering the bilabial /ba/ regularly report perceptually experiencing the alveolar /ga/. Seeing the speaker impacts which phoneme perceptually appears to be uttered. In fact, visual information systematically affects which phoneme you perceptually experience, so both vision and audition provide information about the objects of speech perception. Moreover, Gick and Derrick (2009) demonstrate tactile influences on speech perception. The objects of speech perception are multi-modally accessible. Sounds, however, are neither visible nor multi-modally accessible. Therefore, since sounds are the objects of ordinary non-linguistic audition, the argument concludes that the objects of speech perception and non-linguistic audition must differ. One objection stems from the reply to the first argument. If audition’s objects include sound sources, and sound sources are ordinary happenings like collisions and vibrations, then audition’s objects might include things that are visible. The other objection is that speech perception is not unique in being subject to influence from multiple senses. Crossmodal recalibrations and illusions are rampant. The ventriloquist illusion shows that vision impacts non-linguistic audition. The motion bounce effect and the sound-induced flash illusion show that non-linguistic audition alters visual experience. Visual capture and the rubber hand illusion show that vision affects touch and proprioception. And the touch-induced flash shows that touch alters vision. The examples multiply (for references and discussion, see, e.g., Spence and Driver, 2004; O’Callaghan, 2012; Bayne and Spence, Chapter 32, this volume). In many such cases, the best explanation for some cross-modal effect is that perceptual modalities share common objects (O’Callaghan, 2008, 2012). Consider the sound-induced flash illusion. When presented with a single flash accompanied by two beeps, many subjects illusorily visually experience two flashes instead of one as a result of the two sounds. This illusion occurs because an apparent conflict between vision and audition is resolved in audition’s favour. Since even apparent conflict requires the assumption of a common subject matter, perceptual processes unfold as if a common environmental source produces both the visual and the auditory stimulation. Since, under such conditions, audition is more reliable for temporal features, the overall perceptual experience that results is as of two events rather than one. If, therefore, cross-modal effects support the claim that multimodal speech perception targets common objects of perception, cross-modal effects may support the claim that there are common objects of perception in multi-modal cases that do not involve speech. Such cross-modal effects thus offer

488   Casey O’Callaghan additional support for the claim that non-linguistic audition reveals the sources of sounds, which also are visible. Multi-modality is not unique to speech. The third type of argument stems from the received view that speech perception is categorical. Some have argued that the categorical nature of phoneme perception (see Section 3) shows that its objects are not ordinary sounds, since ordinary sounds need not be perceived categorically (for discussion, see, e.g., Trout, 2001; Pinker and Jackendoff, 2005; for a critical perspective, see, e.g., Diehl et al., 2004). It is true that some attributes of sounds, such as loudness or pitch height (cf. pitch chroma), are not perceived categorically. Nevertheless, there are several lines of response to the argument from categorical perception. First, categorical perception may be limited to certain types of phonemes, such as stop consonants, so not all phoneme perception is categorical. Second, non-linguistic audition may involve categorical perception if speech perception does. Third, non-linguistic creatures, such as quail and monkeys, perceive some speech sounds categorically (see, e.g., Diehl et  al., 2004:  177). Finally, colour perception commonly is regarded as categorical, but this does not establish that the objects of colour vision differ from the objects of ordinary vision. Categorical perception for selected phonemes therefore does not show that the objects of speech perception and the objects of non-linguistic audition differ in kind. Arguments from mismatch, cross-modal influence, and categorical perception thus do not show that the objects of speech perception differ in nature from the objects of ordinary audition. Sounds are among the objects of auditory perception. But to deny that the objects of speech perception include sounds would require denying that spoken utterances may perceptually appear to have pitch, timbre, and loudness. Nonetheless, the considerations discussed above do support the claim that the objects of speech perception include events or happenings beyond sounds, such as the articulatory gestures of speakers. However, I  have maintained that environmental happenings that make or have sounds are also among the objects of non-linguistic auditory perception. For instance, while you hear the crashing sound, you also may hear the collision that makes it. Thus, in speech perception and in general audition, both sounds and sound sources plausibly are among the objects of perceptual awareness. Suppose one held that phonological features of perceptible speech, such as phones and phonemes, themselves were the objects of speech perception. Since phonological features are not individual sounds, one might be tempted to hold that the objects of speech perception differ from the objects of non-linguistic audition. This would be a mistake. It conflates the broad and the narrow ways to understand the objects of perception. I have been discussing the narrow understanding of the objects of perception as individuals that bear perceptible attributes. Phonological features as I have characterized them may be among the objects of perception in the broad sense, but they are not objects of perception in the narrow sense. The account I  have offered denies that phones and phonemes are novel perceptible objects, understood as items or individuals, wholly distinct from audible sounds and articulatory events. It maintains instead that phonological features, including specific phones and phonemes, are perceptible properties or attributes of audible and multi-modally perceptible objects, such as sounds and articulatory events. Thus, for instance, a stream of utterances may perceptually appear to have, to bear, or to instantiate phonological attributes, such as [d]‌ or /d/. Such perceptible linguistic features may be complex properties,

Speech Perception   489 and they may have complex relationships to simple acoustical, physical, or physiological properties. They may be common sensibles. One important virtue of this account is that it allows us to abandon the troublesome ‘beads on a string’ model of perceptible phonemes and to accommodate coarticulation. It does so because continuous sound streams or gestural events may perceptually appear at certain moments to instantiate multiple phonological attributes. Rather than perceptually appearing as discrete perceptible items or individuals arranged in a neatly segmented series (like typed letters in a written word), phonological properties of continuously unfolding spoken utterances may instead appear to be instantiated in connected, blended, or overlapping sequences by a common perceptible individual. The objects of speech perception thus need not be wholly distinct from the objects of non-linguistic audition. Each may include sounds and happenings in the environment that ordinarily are understood to be the sources of sounds. In the specific case of speech, the objects of perception may include sounds of speech and gestures used to articulate spoken language. In a broad sense, they also may include phonological features.

5 Processes What are the implications for questions about how humans perceive speech—about the means or mechanisms involved in speech perception? Does the perception of speech involve special processes, a special module, or perhaps even a special perceptual modality? There is evidence that perceiving speech sounds does involve distinctive perceptual processes beyond those involved in hearing non-linguistic sounds. Duplex perception for dichotic stimuli shows that a single stimulus presented to one ear can, in conjunction with information presented to the other ear, contribute simultaneously to the perceptual experience as of both a non-linguistic sound and an apparently distinct speech sound (Rand, 1974). The same acoustic cue is integrated into two distinct percepts. Duplex perception is thought by some to provide evidence for a special system or mode of listening for speech. That is because, under similar experimental conditions with only non-speech tones, masking rather than integration takes place. However, duplex perception does occur for complex non-linguistic sounds, such as slamming doors, so others have responded that speech perception does not involve dedicated perceptual processes distinct from general audition (Fowler and Rosenblum, 1990). Nevertheless, the capacity to perceive non-linguistic sounds does differ developmentally from the capacity to perceive speech. Notably, for instance, the timing of critical periods for the development of linguistic and non-linguistic perceptual capacities differs. In addition, functional neuroimaging establishes that the patterns of brain activity associated with the perception of speech sounds do not match those associated with the perception of non-linguistic sounds. Most tellingly, however, perceptual capacities and disorders related to speech may dissociate from those related to non-linguistic audition. The example of pure word deafness discussed above puts this into relief. Individuals with PWD have intact abilities to hear and to recognize ordinary sounds but are unable to hear and recognize speech sounds as such. In addition, auditory agnosia concerning environmental sounds may leave linguistic capacities intact (Saygin et  al., 2010). This shows that one could auditorily perceive speech while lacking   other

490   Casey O’Callaghan commonplace auditory capacities. Thus, there is evidence to support the claim that there exist perceptual resources and processes devoted to the perception of speech. Some have held on such grounds that, when compared with general, non-linguistic audition, speech perception is special in that it is modular (e.g., Fodor, 1983). Others even have claimed that it involves a special perceptual modality (Liberman, 1996). I am reluctant to accept the strong view that speech perception involves a dedicated perceptual modality that is distinct from general audition and vision. Audition and vision may treat speech sounds and spoken utterances in a manner that differs from non-linguistic sounds and events, but this does not show that speech perception is a novel perceptual modality. Vision, for instance, devotes special resources and deals in different ways with the perception of objects, colour, motion, and shape. Still, there is considerable debate concerning how to count and individuate perceptual modalities. We might identify modalities by their distinctive objects, stimuli, physiology, function, or phenomenology, or by some combination of these criteria. In the case of the classic sense modalities, at least, the criteria tend to align. Some have maintained that we should be pluralists when individuating and counting sense modalities (Macpherson, 2011). Maintaining that speech perception involves a novel perceptual modality nevertheless requires appealing to one or more of the criteria. None of these criteria, however, warrants positing a modality devoted to the perception of speech that is distinct from but on a par with the familiar examples of vision, hearing, smell, taste, and touch. For instance, speech perception does not involve awareness of novel perceptual objects, and it lacks proper sensibles inaccessible to other modalities. Speech perception lacks a distinguishing kind of proximal stimulus, and it lacks a dedicated sense organ and receptors. Its functional relations do not clearly mark it off as a wholly distinct way or manner of perceiving independent from audition or vision. And it is not apparent that its phenomenology has the type of proprietary, internally unified qualitative character that is distinctive to other perceptual modalities. For instance, while the phenomenology of other sensory modalities doubly dissociates, speech perception requires auditory or visual phenomenology and, thus, does not fully dissociate. Despite these indications, however, a more satisfactory theoretical understanding of the modalities of sensory perception will help to make progress on this question (see, e.g., Matthen). The weaker claim is that speech perception is modular. But good reasons also exist to doubt that a devoted perceptual module is responsible for the perception of speech. Appelbaum (1998), for instance, argues forcefully against Fodor that domain general, top-down influences impact the perception of speech sounds. If a process is modular only if it is informationally encapsulated, then speech perception is not modular. Perhaps it is possible to make do with a minimal story about the sense in which the processes associated with speech perception are special without appealing to a perceptual modality or even a perceptual module devoted to the perception of spoken language. Such a story may be framed in terms of our perceptual treatment of speech and speech sounds. Humans do have a special or differential selectivity or sensitivity for the sounds of speech. The striking evidence is that even neonates distinguish and prefer speech to non-speech sounds (Vouloumanos and Werker, 2007). The sounds of spoken utterances are of special interest to us, relative to other kinds of environmental sounds and events. Humans are not, however, born able to perceive all of the attributes that are distinctive to specific languages. Infants must prune and cease to perceive audible differences that are not linguistically significant in their own languages. They also must learn perceptually

Speech Perception   491 to discern linguistic sameness in the face of variation across speakers, moods, and contexts. This is learning perceptually to ignore irrelevant differences and to attend to crucial similarities, and it alters the language-specific perceptual similarity space involving speech sounds. Understanding a language, as it is spoken in a variety of contexts, demands such learning. In coming to know a spoken language, we begin to perceive the relevant language-specific features of sounds and utterances. Humans thus have a propensity for learning perceptually to discern the appropriate language-specific types to which spoken utterances belong.

6  What makes speech special? Perceiving the attributes that are distinctive to the speech sounds of a given language, I have argued, requires experience and learning. Learning a language thus is not simply a matter of learning a sound–meaning mapping. It involves acquiring the capacity perceptually to discern language-specific attributes of spoken utterances. In this sense, you learn to hear the sounds of your language. Learning a language is partly a matter of acquiring a perceptual skill. Humans have a special propensity to learn to perceive language-specific attributes of speech sounds from birth, but this capacity develops later than other familiar perceptual capacities. For instance, young infants perceive individual objects and events, persistence, and sensible qualities, including colour, pitch, and loudness, prior to perceptually discerning types of sounds that are specific to a particular language. Perceptual awareness of spoken language may therefore be more like perceptual awareness of clapping of hands, barking dogs, or fingernails scratching a chalkboard, each of which involves acquired perceptual skills. As with other auditory phenomena, the manner in which language-specific sounds are perceptually individuated and classified is illuminated by taking into account the environmental happenings that generate sounds. In particular, articulatory gestures and talking faces make sense of why users of a given language discern and treat various speech sounds as standing in relations of similarity and difference that do not stem in straightforward ways from acoustical characteristics. Considered as such, perceiving speech is a matter of detecting and discerning biologically significant kinds of sounds and happenings, rather than just detecting abstract features of an acoustic signal. How does perceiving speech differ from perceiving other biologically significant kinds of environmental sounds? Consider a family of perceptual capacities attuned to varieties of animacy. For instance, humans may sometimes perceptually experience a pattern of moving dots as running, or seem to be aware of one dot chasing another dot around a display (Heider and Simmel, 1944; see Scholl and Tremoulet, 2000; Gao et  al., 2009). Here we describe the perception of inanimate things and motion in terms applicable to animate creatures and activities. Since such effects require only very minimal cues, this suggests humans have a special propensity to perceive aspects of animate creatures and their activities. That is, we have differential sensitivity to certain kinds of activity that creatures engage in, in contrast to simple mechanical patterns of motion traced by inanimate things. Perceiving speech is similar to such perceptual capacities in that its concern is a

492   Casey O’Callaghan type of animacy exhibited by living things to which we have special sensitivity. In the case of speech (as in the case of faces) this perceptual capacity is directed predominantly at members of our own species. Speech perception belongs to an even more special subclass. Speech sounds are generated by communicative intentions of other humans. Like some facial expressions and non-linguistic vocalic sounds, the sounds of spoken utterances are caused by and thus have the potential to reveal the communicative intentions of their animate sources. Speech perception is among a class of ethologically significant perceptual phenomena that serve to disclose intentional activities involved in communication. Perceiving speech is detecting and discerning language-specific kinds of biologically significant events: those generated by communicative intentions of fellow human talkers. We hear people talking. We hear them as interlocutors.

Acknowledgements I have learned a great deal about the philosophical issues raised by speech perception from Matthen (2005), Mole (2009), Remez and Trout (2009), Rey (2012), and Smith (2009). These works, and conversations with their authors, drew me from my more general concern with sounds, audition, and multi-modality to the philosophically and empirically rich subject matter whose focus is the perception of spoken language. I gratefully acknowledge their influence upon my approach to this topic. Thanks to Mohan Matthen for helpful comments on this chapter.

References Appelbaum, I. (1998). 'Fodor, modularity, and speech perception'. Philosophical Psychology, 11(3), 317–330. Appelbaum, I. (1999). 'The dogma of isomorphism: A case study from speech perception'. Philosophy of Science, 66, S250–S259. Bayne, T. (2009). 'Perception and the reach of phenomenal content'. The Philosophical Quarterly, 59(236), 385–404. Bregman, A. S. (1990). Auditory Scene Analysis:  The Perceptual Organization of Sound. Cambridge, MA: MIT Press. Diehl, R. L., Lotto, A. J., and Holt, L. L. (2004). 'Speech perception'. Annual Review of Psychology, 55, 149–179. Eimas, P. D., Siqueland, E. R., Jusczyk, P., and Vigorito, J. (1971). 'Speech perception in infants'. Science, 171(3968), 303–306. Fodor, J. (1983). The Modularity of Mind. Cambridge, MA: MIT Press. Fowler, C. A. (1986). 'An event approach to the study of speech perception from a direct-realist perspective'. Journal of Phonetics, 14, 3–28. Fowler, C. A. and Rosenblum, L. D. (1990). 'Duplex perception: A comparison of monosyllables and slamming doors'. Journal of Experimental Psychology: Human Perception and Performance, 16(4), 742–754.

Speech Perception   493 Gao, T., Newman, G. E., and Scholl, B. J. (2009). 'The psychophysics of chasing: A case study in the perception of animacy'. Cognitive Psychology, 59, 154–179. Gick, B. and Derrick, D. (2009). 'Aero-tactile integration in speech perception'. Nature, 462(7272), 502–504. Harnad, S. (1987). Categorical Perception: The Groundwork of Cognition. Cambridge: Cambridge University Press. Hauser, M. D., Chomsky, N., and Fitch, W. T. (2002). 'The faculty of language: What is it, who has it, and how did it evolve?' Science, 298, 1569–1579. Heider, F. and Simmel, M. (1944). 'An experimental study of apparent behavior'. The American Journal of Psychology, 57(2), 243–259. Jusczyk, P. W. (1997). The Discovery of Spoken Language. Cambridge, MA: MIT Press. Liberman, A. M. (1996). Speech: A Special Code. Cambridge, MA: MIT Press. Liberman, A. M. and Mattingly, I. G. (1985). 'The motor theory of speech perception revised'. Cognition, 21, 1–36. Liberman, A. M., Cooper, F. S., Shankweiler, D. P., and Studdert-Kennedy, M. (1967). 'Perception of the speech code'. Psychological Review, 74(6), 431–461. McDowell, J. (1998). Meaning, Knowledge, and Reality. Cambridge, MA:  Harvard University Press. McGurk, H. and MacDonald, J. (1976). 'Hearing lips and seeing voices'. Nature, 264, 746–748. Macpherson, F. (2011). 'Taxonomising the senses'. Philosophical Studies, 153(1), 123–142. Matthen, M. (2005). Seeing, Doing, and Knowing: A Philosophical Theory of Sense Perception. Oxford: Oxford University Press. Mole, C. (2009). 'The Motor Theory of speech perception'. In M. Nudds and C. O’Callaghan (eds), Sounds and Perception: New Philosophical Essays (pp. 211–233). Oxford: Oxford University Press. Nudds, M. (2010). 'What are auditory objects?' Review of Philosophy and Psychology, 1(1), 105–122. O’Callaghan, C. (2008). 'Seeing what you hear: Cross-modal illusions and perception'. Philosophical Issues: A Supplement to Noýs, 18, 316–338. O’Callaghan, C. (2011). Against hearing meanings. Philosophical Quarterly, 61, 783–807. O’Callaghan, C. (2012). Perception and multimodality. In E. Margolis, R. Samuels, and S. Stich, S. (eds), Oxford Handbook of Philosophy and Cognitive Science. Oxford: Oxford University Press. Pinker, S. and Jackendoff, R. (2005). 'The faculty of language: What’s special about it?' Cognition, 95, 201–236. Poeppel, D. (2001). 'Pure word deafness and the bilateral processing of the speech code'. Cognitive Science, 25, 679–693. Rand, T. C. (1974). 'Dichotic release from masking for speech'. Journal of the Acoustical Society of America, 55, 678–680. Remez, R. E. and Trout, J. D. (2009). 'Philosophical messages in the medium of spoken language'. In M. Nudds, and C. O’Callaghan (eds), Sounds and Perception: New Philosophical Essays (pp. 234–264). Oxford: Oxford University Press. 'Remez, R. E., Rubin, P. E., Pisoni, D. B., and Carell, T. D. (1981). Speech perception without traditional speech cues'. Science, 212, 947–950. Rey, G. (2012). 'Externalism and inexistence in early content'. In R. Schantz (ed.), Prospects for Meaning (pp. 503–530). New York: de Gruyter. Saygin, A. P., Leech, R., and Dick, F. (2010). 'Nonverbal auditory agnosia with lesion to Wernicke’s area'. Neuropsychologia, 48, 107–113.

494   Casey O’Callaghan Scholl, B. and Tremoulet, P. (2000). 'Perceptual causality and animacy'. Trends in Cognitive Sciences, 4(8), 299–309. Siegel, S. (2006). 'Which properties are represented in perception?' In T. S. Gendler and J. Hawthorne (eds), Percepual Experience (pp. 481–503). New York: Oxford University Press. Smith, B. (2009). 'Speech sounds and the direct meeting of minds'. In M. Nudds and C. O’Callaghan (eds), Sounds and Perception: New Philosophical Essays (pp. 183–210). Oxford: Oxford University Press. Spence, C. and Driver, J. (eds) (2004). Crossmodal Space and Crossmodal Attention. Oxford: Oxford University Press. Trout, J. D. (2001). 'The biological basis of speech: What to infer from talking to the animals'. Psychological Review, 108(3), 523–549. Vouloumanos, A. and Werker, J. F. (2007). 'Listening to language at birth: evidence for a bias for speech in neonates'. Developmental Science, 10(2), 159–164.

Chapter 26

M usica l Perception Charles Nussbaum

1 Introduction: Acoustic and physiological preliminaries Music is sound art. This much is uncontroversial. But I will be adopting a narrower def­ inition, for sounds include noises, pitched sounds, and tones. Whereas a noise, acoustically speaking, is any longitudinal (compression and rarefaction) waveform disturbance in a conductive medium, a pitched sound is a noise that can be categorized according to frequency, and a tone is a pitched sound whose frequency profile is periodic and steady-state. A sound may be high frequency, like a seagull’s caw, or low frequency, like the rumble of thunder, without being either periodic or steady-state. Some sounds, like that of a shoe scraping on pavement, resist frequency categorization. Although there are forms of sound art (conceptual music, musique concrète) that include noises and non-steady-state pitched sounds among their elements, and although it may be possible to score a musical composition exclusively for percussion instruments (such as the tambourine, the woodblock, and the bass drum) that do not produce tones, for the purposes of this article, I will take music to be Tonkunst, as the Germans call it, the art of combining tones into melodic (sequential), harmonic (simultaneous), and rhythmic (temporally organized) patterns.1 Here we already run into one of the many remarkable properties of the art of music, so understood. On the one hand, because the human inner ear is an exquisitely sensitive acoustic spectral analyzer, the phenomenology of musical experience maintains certain robust correspondence relations with the (logarithmically expressed)2 frequency spectrum of physical sound. On the other hand, steady tones are quite rare in nature. Indeed, according to Albert Bregman, ‘steady pitch is an idealization that probably was discovered when humans invented musical instruments’ (1990, 104). Even the animal sounds of which Darwin made so much in his theory of sexual selection, 1  I will not, however, be defining music so narrowly as to exclude all music other than performance-art music. A musical work such as a motion-picture score may be composed just for recorded playback, and not for performance at all (cf. Davies 2001, 25ff). 2  We hear tones as separated by intervals such as octaves and fifths, which are ratios of frequencies rather than differences of frequency. Low-frequency octaves do not sound appreciably smaller than high-frequency ones. A straight frequency mapping would not suggest this.

496   Charles Nussbaum including those made by humans, Bregman claims, are not steady- state, but make frequent use of glides in pitch and rapid changes in timbre (loc. cit.). The phenomenology of steady tones and their combinations is as distinctive as their natural occurrence is unusual. It is on this distinctive phenomenology that music considered as tone art is built, an art, as we will see, that in at least one important way subverts the evolutionary purposes of the auditory system. In art, as that most aesthetically sensitive of philosophers has warned us, ‘the lie is sanctified and the will to deception has a good conscience’ (Nietzsche 1887/1967, 153). The human auditory system was designed by evolution not for musical processing, but for auditory scene analysis, which is the identification of ambient sounds and the location of their sources in physical space. Speech sounds produced by conspecifics, note well, constitute an important component of the auditory scene. Auditory scene analysis isolates momentary sounds, as well as temporally extended auditory ‘streams’ distinguished by pitch, timbre, rhythm, and/or direction, from the mass of ambient sound in order to detect the occurrence of events of importance. Direction is computed by the brain, based on such cues as asynchronous onset, transformations of the impinging sound waves by the structurally inverse pinnae or auricles of the outer ear, and Doppler effects. Sounds may directly indicate only events, but they indirectly indicate the objects responsible for them. Because of the physics of sound, the qualities and quality spaces that serve the ends of music, namely pitch, loudness, timbre, and rhythm, also (with one notable exception) serve the functions of auditory scene analysis. The exception is the source location function. In musical contexts, particularly in traditional Western musical contexts, this function is suppressed,3 albeit not eliminated, because the aim in Western music is to substitute as intentional objects ‘chimerical’ or virtual objects of experience located in a virtual musical space for the actual sources of sound located in physical space. Just what these virtual objects might be will be the main topic of discussion in the final section of this article. Stereophonic recording may seem to provide a counter-example to this claim, but it does not, for its purpose is not to enable the listener to ferret out the locations of the instruments in a performance, something already known by most listeners and in itself relatively inconsequential, but rather to provide musical lines, textures, and structures with greater clarity (Bregman 1990, 680). Pitch, loudness, timbre, and rhythm are all crucial for auditory scene analysis because of the physical nature of sound waves. These waves are transparent: their sources do not occlude one another in the way light-reflecting solid objects do, nor are sound waves blocked by objects that are neither transparent nor translucent, as transverse light waves are, nor (with certain exceptions to be noted) do combinations of components of the sound spectrum fuse in experience the way the combinations of the spectral components of light do. If the lens of the human eye were prismatic and functioned as a spectral analyzer in the manner of the organs of the inner ear, this would defeat the primary purpose of primate trichromatic vision, which is to differentiate specific classes of objects in the surround by means of their reflectance properties, specifically to distinguish relatively small red and orange objects, principally fruits and berries in surrounding green and greyish brown foliage (cf. Matthen 2005, 180–183).4 The human auditory system is tuned to detect sound producers and not, as in predating bats, sound reflectors, and it functions according to the heuristic principle 3 

4 

Antiphonally scored music may be an exception. See Changizi et al. (2006) for an alternative evolutionary account.

Musical Perception   497 that a sound with definite onset and discernible and relatively consistent pitch location and timbre is not likely to be a coincidental result of different sound producers acting in concert, but to emanate from one sound source. However, what would be an unlikely concert of sound producers in the natural auditory scene is, of course, the rule in musical venues.5

2 Musical quality spaces A quality is a worldly feature, like red, loud, or sweet, as accessed via sensation. A quality space is an abstract n-dimensional structure that represents degrees of similarity and difference between members of a specific range of qualities. Each member occupies a location in a quality space, with proximity representing degree of similarity. Taste space is a particularly clear example. Human taste reception generates a five-dimensional space defined by axes (intensity vectors) of sweet, salty, sour, bitter, and fatty, in which the range of determinate tastes can be located. The elements of music generate four fundamental quality spaces, those of pitch, timbre, rhythm, and loudness. Music also invariably displays metrical organization, a property to be distinguished from rhythmic patterning. The available evidence strongly suggests that these four quality spaces are musical universals. This should not be surprising, for they are all useful in auditory scene analysis. My plan here is to say something about each of them in turn, proceeding from the simplest and most straightforward (loudness) to the most complex and challenging (pitch).

Loudness Loudness is heard sound intensity and is measured in decibels, a logarithmic scale: each doubling of the intensity of a sound yields a three-decibel increase (cf. Levitin 2006, 67–69).6 Still, the relevant musical quality space may not be one-dimensional. Despite the fact that audio devices tend to have a loudness control marked ‘volume’, it is customary to distinguish between loudness and volume. Volume is the sense of the extent to which a sound, whatever its loudness, fills a physical space. A large symphony orchestra executing a fully scored pianissimo, say the concluding C major chord of Richard Strauss’s Death and Transfiguration (1889), produces a high volume sound. The pianissimo chord exhibits a lower amplitude than does the sound produced by the percussion instruments sounding at the opening of Aaron Copland’s Fanfare for the Common Man (1942), which can be deafening for players seated in front of them. 5  Cf. Bregman (1990, 649): ‘The coherence of continuous sequences can be interpreted in functional terms as a heuristic of the auditory system. This heuristic is equivalent to a bet that any sequence that exhibits acoustic continuity has probably come from a single environmental event.’ 6  It is important to distinguish sound power, measured in watts (or joules/second), sound intensity, measured in watts per square metre, and sound pressure, measured in micronewtons per square metre or micropascals. Since the factor of increase of sound intensity equals the square of the factor of increase of sound pressure, and since log x 2 = 2 log x, each doubling of sound pressure yields an increase of 2 × 3 or six decibels SPL (sound pressure level).

498   Charles Nussbaum

Metre and Rhythm Metre is the cyclical pattern of stress and temporal recurrence in any musical sequence. In traditional Western music, metre is marked by bar lines and tends to be duple, triple, or quadruple, with the first beat (in duple metres) or the first and third beats (in quadruple metres) played and recognized as strong (stressed), and the second or the second and the fourth or, in triple metres, the second and the third beats, played and recognized as weak. In classical Western music, metric values are divided with arithmetic exactitude and organized in a strictly hierarchical fashion within the bar. Aside from metre’s recurrent or cyclical structure, these Western metrical norms are not universal. For example, an Indian tala, or cycle of beats, can range from 3 to 108, though talas of 6 to 16 beats are most common (O’Brien 1977, 25). Some beats within the tala are more important than others, but they are not as regular as they are in Western music, nor are they precisely arithmetically divided and hierarchically ordered. Rhythm, in contradistinction from metre, is the temporal pattern of note values realized within the confines of the metre, though type-identical rhythms can be realized within different metres. In classical Western music, these note values are calibrated to the whole note/semibreve (equalling four quarter notes/crotchets), and range from double whole notes/semibreves (each equalling eight quarter notes/crotchets) to sixty-fourth notes/ hemidemisemiquavers (each equalling one-sixteenth of a quarter note/crotchet). Once again, this Western practice of strict arithmetic division is far from universal. In parts of Africa, for example, drumming rhythms are complex, generating polyrhythms resulting from the combination of different drums executing different patterns at different tempos without standard note values (O’Brien 1977, 99–103). Tempo, sometimes considered a third dimension of this quality space, concerns the pace and mood of a musical piece or episode. In the Western classical tradition, tempo is indicated in the score by verbal markings, usually in Italian, such as allegro (fast, lively), adagio (slow), presto (extremely fast), largo (very slow and calm), etcetera. Still, underneath all this diversity there is the unity of metrical cycle and stress pattern, which suggests the cyclical character of gait and the involvement of the motor system in processing musical patterns. The connection of metre with dance or other stylized or ritualized movements is almost too obvious to be worth mentioning. Indeed, one philosopher (Robinson 2005) has gone so far as to dub the involvement of the motor system in musical experience the ‘jazzercise effect’ (see also Levitin 2006, 174). There are empirical studies (Halpern 2003; Nakamura et al. 1999) indicating increased activity in the right supplementary motor area of the cerebral cortex, an area that supports motor planning, during musical listening. This will take on added importance when we come to discuss movement in musical space.

Timbre With timbre, or instrumental and vocal tone colour, the phenomenology of musical experience makes what is perhaps its most radical departure from acoustic physical reality and most resembles the phenomenology of colour vision. But unlike colour vision, musical timbre does this, to a significant degree, by subverting the evolutionary purposes of the

Musical Perception   499 auditory system: as we are about to see, music characteristically tricks the ear into accepting chimerical sound sources.7 Most periodically vibrating bodies do not vibrate at a single frequency (inharmonic overtones aside) in the manner of a tuning fork, which produces a compression wave heard as a pure tone that can be represented graphically as a simple sine curve. Rather, most such vibrating bodies vibrate simultaneously at multiple frequencies. These additional, higher frequencies produce waves heard as harmonic overtones or upper partials, which, without fairly rigorous training and some perceptual priming, are difficult for the ear to extract, but are heard, particularly by the untrained, as entirely fused with the fundamental. The overtones form a mathematically specified series. The first overtone is the octave, the second the fifth above that, the third the fourth above that, the fourth the major third above that, and the fifth the minor third above that. The series can be continued indefinitely, with members calculated arithmetically from the fundamental, the sound frequency that is generated by the body vibrating as a whole. When hearing a tone, we hear the frequency of the fundamental. Remarkably, if the ear is presented with an artificially contrived series of overtones lacking its fundamental, the brain will supply it, as if it to suggest that it ‘must’ be there. Just as remarkable is the following. The characteristic timbre of a musical instrument, say a trumpet or a clarinet, is a function of the selection of overtones present in its frequency spectrum. The set of overtones that configure the clarinet sound is more jagged or gappy than is the set of overtones that configure the trumpet sound (see Figure 26.1). The average (mean) of all the frequency components (each component weighted by amplitude) in a sampling of the sound of a brighter instrument like the oboe will be greater than the average in a sampling of the sound of a darker, more mellow instrument like the trombone (see Figure 26.2). This value is sometimes called the ‘spectral centroid’. Instrumental sounds can be positioned in timbral quality spaces that illustrate these relationships (see Figure 26.3). The dimensions of the version of timbral quality space given in Figure 26.3 are marked ‘rise time’, ‘spectral centroid’ (already discussed), and spectral flux. Rise time concerns the rate of evolution of a sound to steady-state after the initial attack. Notice the longer rise times for bowed stringed and wind instruments than for plucked instruments like the guitar and harp. Rise times are surprisingly determinative of the timbre of an instrument. If a recorded sound is reversed in time, its quality can change radically. Spectral flux measures degree of variation in the configuration of the frequency spectrum of a sound over time. Bowed stringed instruments rate high on this scale, woodwind and brass lower. Some of the timbres represented here are artificial syntheses of standard instrumental sonorities, like the ‘striano’ (‘sno’), which stands for a hybrid of the piano and a bowed, stringed instrument. Notice the difference in rise time between the piano and the bowed instrument, the latter being much longer than the former. It is to such artificial sound syntheses that the term ‘chimerical’ is often applied: there is no such instrument as a striano. When Bregman (1990, 508) claims that music deals in chimeras, however, he means something a little different, for Western composers have used combinations of actual instruments to create timbres or sonorities whose sources are, in a sense, virtual. There are countless

7  Cf. Bregman (1990, 457): ‘In order to get sounds to blend, the music must defeat the scene-analysis processes that are trying to uncover the individual physical sources of sound.’

500   Charles Nussbaum SPECTRAL IRREGULARITY Clarinet

70

70

60

60 Amplitude (dB)

Amplitude (dB)

Trumpet

50 40 30 20 10

50 40 30 20 10

936 1872 2806 3744 4880 5616 6552 7488 8424 Frequency (Hz)

936 1872 2806 3744 4880 5616 6552 7488 8424 Frequency (Hz)

Fig.  26.1 Spectral graphs of two instrumental sounds. The trumpet spectrum on the left has a lower spectral irregularity value than the clarinet spectrum on the right, which displays a more jagged profile with low energy overtones. Figure from Donnadieu (2007, 279), reprinted with kind permission from Springer Science+Business Media  B.  V. SPECTRAL CENTROID Trombone

Oboe 80

70

70 Amplitude (dB)

Amplitude (dB)

60 50 40 30

60 50 40 30

20

20 624

1248

1872 2496 3120 Frequency (Hz)

3744

4368

4992

624

1248 1872 2496 3120 3744 4368 4992 Frequency (Hz)

Fig. 26.2  Spectral graphs of two instrumental sounds. The trombone spectrum on the left has a lower spectral centroid value than the oboe spectrum on the right because of the lower average frequency (weighted by amplitude). Figure from Donnadieu (2007, 278), reprinted with kind permission from Springer Science+Business Media  B.  V. examples of this, but here is a simple yet notable one. The First Suite from the Caucasian Sketches (1894) by the Russian composer Mikhail Mikhailovitch Ippolitov-Ivanov features a movement entitled ‘The Procession of the Sardar’. Ippolitov-Ivanov introduces his catchy, exotic tune using an unusual instrumental combination: a piccolo doubled by a bassoon three octaves below. Now anyone with a modicum of training or experience can of course distinguish and identify the two cooperating instruments. Nonetheless, there is also a distinctive combined sonority to be heard as if deriving from a chimerical source, a piccoloon one might call it. Unisons, as opposed to octave doublings, are more difficult to

Musical Perception   501

Shorter

vbs

4

ols

hrp

3 gtr

2

vbn

obc

hcd

pno

1 Dimension 1 0 (rise time) –1 –2

gnt

lower

stg

3

ehn

hrn

sno

longer –3

tpr

tbn

cnt

bsn

tpt

2 1

Dimension 2 (spectral centroid)

0

–1 –2 higher –3 3

2 Greater

1

0

–1

–2

–3 lesser

Dimension 3 (spectral flux)

Fig. 26.3  A three-dimensional timbral quality space capturing the three parameters of rise time, spectral centroid, and spectral flux (see accompanying text for explanation). Hashed lines connect two of the hybrid timbres, the ‘vibrone’ (or vbn, a combination of vibraphone and trombone) and the ‘striano’ (or sno, a combination of piano and a stringed instrument). Figure from McAdams et al. (1995), cited by Donnadieu (2007, 280), reprinted with kind permission from Springer Science+Business Media B. V. distinguish, but pose no real challenge to the trained ear. Any experienced listener can tell that the chimerical instrumental sound used by Franz Schubert to introduce the principal theme of the first movement of his ‘Unfinished’ Symphony (1828) emanates from an ‘obinet’, i.e. an oboe and a clarinet playing in unison. In the context of natural auditory scene analysis, the chances that sounds produced by two distinct sound sources would coincide this perfectly are next to nil, so the auditory system tends to fuse them. Once again, the aim in auditory scene analysis is to isolate sounds and identify their sources. In the musical context this design feature is both defeated and exploited. As we will see, this is only the beginning of the tendency of musical art to subvert and then exploit the evolutionary design of the auditory system for its own ends. Traditionally, timbre has been used by Western composers for character and mood effects, but has not been employed to any appreciable extent as a structural element in its own right. Even Maurice Ravel’s Bolero (1928), dependent as it is on a deliberate sequence

502   Charles Nussbaum of instrumental colours from sources both actual and chimerical, indeed, even Anton Webern’s famous Klangfarben orchestration of J. S. Bach’s C minor Ricercar (1934–1935), do not use timbre as the sole source of musical structure. Fred Lerdahl (1987) has made a proposal along these lines. His idea is that to the extent timbres vary regarding brightness and rise time, it might be possible to organize timbral events hierarchically into analogues of the hierarchical prolongation structures he and Ray Jackendoff developed to represent increase and decrease in harmonic tension in Western tonal music (1983). Increased tension is correlated with increased brightness and with rise times that are either very fast or very slow. But Lerdahl’s proposals remain exploratory and programmatic. Still, for a genuine example on the hoof of a structural use of timbre, one could hardly do better than to look to Eliot Carter’s Eight Etudes and a Fantasy (1950) for woodwind quartet. The seventh of these etudes makes exclusive use of a single note, the G above middle C, which is sounded in turn (but never in combination) by members of the ensemble for various durations at different dynamic levels and with different modes of attack. The piece arguably succeeds in providing a musical experience lacking harmony or melody, but timbrally organized in accordance with principles of tension and release.

Pitch Periodic, steady-state sound-wave frequency is heard as musical pitch and musical pitch, like other qualities, can be organized in a quality space. But it is far from clear how this space should be constructed, for conflicting standards of similarity compete. Is middle C most similar to the B just below it or the C-sharp just above it because of scalar adjacency (heard frequency proximity)? Or is it most similar to the Cs just above and below it because of the low heard frequency ratio, the fact that octaves somehow sound the ‘same again’? By this second standard, octave similarity is exceeded only by the type identity of tones in unison. Any account of pitch space will have to accommodate both these standards and more. As a result, pitch space is the musical quality space that is most complex and challenging to analyze, but also the most intriguing. To begin with, because of the variety of scales and intervals employed at different times and in different places by various musical styles, there is no single pitch space that can usefully be applied to all music. With regard to scalar intervals, only the octave comes close to universal recognition and employment (and even it does not), with the perfect fifth a distant second. Here is an example. The gamelon music of Indonesia makes use of two scales, the five-tone slendro and the seven-tone pelog (O’Brien 1977, 91–92). If we identify middle C as the first tone of the slendro and then map it onto the Western tempered chromatic scale, the other pitches would be a slightly flat E-flat, a slightly flat F, a somewhat sharp G, and a slightly flat B-flat. Doing the same with the pelog would yield a flat D, a sharp E, a sharp F, a flat G, a sharp A, and a sharp B. While both scales incorporate the octave, neither contains any other intervals belonging to the Western chromatic scale. One characteristic that seems to be quite general in non-Western music is an emphasis on the importance of relative pitch, the intervallic relations between scalar pitches once the tonic or home pitch is fixed, over absolute-pitch placement. Specific keys and their distinctive colours seem to be less important than they are in traditional Western music and especially Western classical music since the eighteenth century, in which key decisions on

Musical Perception   503 the part of composers have taken on great importance. In my estimation, this is as much a function of the acoustic properties of Western wind and stringed instruments as it is of Western tonality. It is also highly unusual in any style of music for the octave to be divided into more than seven principal pitches. In musical styles that make use of microtones, i.e. intervals smaller than a minor second, these tend to circle around other, more central pitches. There does, however, seem to be one universal or near-universal aspect of pitch-space (though this is denied, albeit not very convincingly, by Lerdahl [2001, 191] and by Levitin [2006, 19]), namely the tendency to hear higher-frequency tones as located higher in pitch space. This is reflected in the musical terminology of many languages, even if our ‘high’ and ‘low’ are sometimes more dynamically (and, in anticipation of proposals to follow, quite appropriately) interpreted as ‘piercing’ and ‘heavy’, as in modern French (aigu/grave) and ancient Greek (oxy-/bary-). Explaining this psychological universal, or near-universal, poses something of a challenge. The wavelengths of high-pitched sounds are, of course, shorter, and shorter is smaller. But why should sounds with smaller wavelengths be heard as higher in pitch space? This remarkable phenomenon, I have proposed (2007, 55ff), is a product of the anatomical structure and function of the human inner ear. Movement of the stapes at the oval window of the cochlea creates waveform disturbances in the fluid contained in the scala vestibuli and the scala tympani, which cause displacements of the tectorial and basilar membranes (Figure 26.4). When these displacements occur, hair cells stretching between the two membranes (the organ of Corti) generate patterns of action potentials which transmit information to spiral ganglion cells in the auditory nerve (Figure 26.5). The hair cells, both inner (located near the central pillar or mediolus of the cochlea) and outer (those

Helicotrema

Malleus Incus

Reissner’s membrane Basilar membrane and organ of Corti

Stapes at oval window

Scala vestibuli Scala media Scala tympani

External auditory meatus Tympanic membrane

Bony wall of cochlea Round window Middle ear cavity

Fig. 26.4  The anatomy of the human middle and inner ear. The tectorial membrane is not shown. Figure from Kelly (1991, 483), reprinted with kind permission from McGraw-Hill Companies, Inc.

504   Charles Nussbaum (a)

Tectorial membrane

Basilar membrane

(b)

Deflection Hair cells

Fig. 26.5 A cross-section of the scala media showing the basilar and tectorial membranes and the organ of Corti, the hair cells stretching between them. Figure from Kelly (1991, 486), reprinted with kind permission from McGraw-Hill Companies, Inc. peripherally located), are tonotopically arranged, with different populations responding most strongly to different frequencies. Those sensitive to higher frequencies are located at the base of the coiled cochlea, those more sensitive to lower frequencies at the tip or helicotrema, where the basilar membrane widens and becomes more flexible (Figure 26.6). The human cochlea, like the cochleae of other mammals, is coiled, but if it were to be straightened, the base would be closest to the brainstem, the tip extended outward. This means that high-frequency sounds are felt to be proximal to the head, low-frequency sounds as distal to the head. Since the normal orientation of the human body is upright in the gravitational field, a body part felt to be distal to the head is felt to be lower in space. For this reason high- frequency tones are heard as located high in pitch space. It is true that to be distal relative to the head is not necessarily to be below the head. To counter this objection, I see no recourse other than to engage in a bit of informed evolutionary speculation. The ancestral homologue of the human cochlear nucleus in the

Musical Perception   505 33 mm

Base 100 μm high frequency stiff region near the oval window

Apex 500 μm low frequency flexible region

Fig. 26.6  The approximate shape and dimensions in micrometers (or microns) of the basilar membrane and the frequency distribution of the embedded hair cells. Figure from Kelly (1991, 486), reprinted with kind permission from McGraw-Hill Companies, Inc. brainstem, the first innervation point of the vestibulocochlear (eighth cranial) nerve, is the acousticolateral area of the medulla of bony aquatics. In addition, the ancestral homologue of the human cochlear hair cells are the motion-sensitive cells in the lateral line of fishes. As aquatic creatures adapted to life on land and became sensitive to atmospheric pressure waves, the lateral line and its neural connections to the medulla atrophied and were replaced in reptiles, birds, and mammals by ancestors of the cochlea. One bit of empirical evidence that supports this account is that this very process actually occurs in the ontogenesis of amphibians. The lateral line system, which is well developed in tadpoles, the larval stage of frogs, atrophies when metamorphosis occurs and the tadpole makes the transition from water to land. Just as the human cochlear nucleus is organized tonotopically to map frequencies high to low, the acousticolateral nuclei in the fish are organized somatotopically to map body locations from head to tail. In the case of the lateral line, what is distal is caudal. The same is true, I would argue, with regard to its descendant, the cochlea. Once again, we have a clash between the actual and the chimerical, for height in pitch space has nothing to do with elevation of sound sources in physical space. It is entirely an artefact of the anatomy of the human ear and, if I am correct, its evolutionary history. But even if this, or any account of why humans are inclined to hear high-frequency tones as elevated in pitch space, is found to be convincing, our troubles have only just begun. For, as already intimated, there is no one pitch-space structure that will apply to all musical styles, and there is in addition the spectre of Western atonal serial musical styles to contend with. The only viable option in the present context is limitation, and the only workable tactic available, to this author at least, is to limit discussion to the tonal pitch space of traditional Western music. Even with this retrenchment, there remains a good deal to discuss. There are available two basic models of tonal pitch space. Both have older roots, but the contemporary versions I will be considering derive, respectively, from Roger Shepard and Fred Lerdahl. Where the Shepard model is topological, Lerdahl’s is numerical. Shepard offers three topological models, each successive model a modification of a simpler predecessor. The simplest and best known of the three is configured as a cylinder encircled by a double helix, each helix representing a rising whole-tone scale (Figure 26.7). The circle of

HEIGHT

506   Charles Nussbaum

D´

C#´ C´

B A#

A

G# F#

G

E# F D#

D

C# C# G#

D#

F#

B

E

FIFT H S

A#

F

A C C

D G

Fig. 26.7  The double helix of musical pitch. Figure from Shepard (1982, 362), reprinted with kind permission from Elsevier. fifths8 is located around the base of the cylinder. If the cylinder were to be segmented at each semitone (i.e. C, C-sharp, D, D-sharp, etc.) located on the rising helices, adjacency of segmenting bands would represent scalar similarity, whereas band adjacency plus vertical pitch alignment on the surface of the cylinder would represent octave similarity. The model correctly represents tritones, the most distantly related intervals in pitch space, as diametrically opposed on the cylinder. Acoustically, the simpler the integer frequency ratio

8  The circle of fifths represents the most fundamental key relationships of traditional Western tonal music. Movement around the circle proceeds stepwise by intervals of a fifth. Each counterclockwise move adds a sharp to the signature of the key whose tonic or home note is the tone that has been reached, or it removes a flat. Each clockwise move adds a flat, or it removes a sharp. For example, C to F adds a flat, C to G adds a sharp. The two paths around the circle from C converge at the key signature of six sharps (F-sharp major) or six flats (G-flat major). In tempered tuning, these keys are enharmonically equivalent and the expressions ‘F-sharp’ and ‘G-flat’ denote the same chroma or pitch class. The relationships of the circle of fifths also hold between the minor keys.

Musical Perception   507 between two tones, the more consonant is the interval. The frequency ratio of the octave is 2:1, that of the fifth 3:2, and that of the fourth 4:3. The tritone ratio is 45:32 (on Levitin’s telling, 43:32). There is a well-known distinction to be made between psychoacoustic and musical consonance. The former concerns physics and physiology, the latter concerns culturally variable musical style. We, who have been familiar with Western tonal music since 1650, tend to hear the major third, whose frequency ratio is 5:4, as more consonant than the fourth, which is psychoacoustically more consonant. This has not always been the case, even in the West. Treating octaves as equivalent allows the cylinder to be wrapped around a torus (Figure  26.8). But with this, the dimension of height is lost. To reintroduce it, Shepard extends the cylinder into a helical shape (Figure 26.9), thereby preserving the virtues of both previous models. Shepard’s helical cylinder captures the relationships of the circle of fifths by locating them around the diameter of the cylinder, scalar pitches by locating them at consecutive segments of the cylinder, and pitch height by means of the helical rise of the cylinder. As with the original upright cylinder, octaves line up in straight lines above and below each other around the helix, an intuitively satisfying result. The cylindrical helix as pictured manages to project five dimensions (two for the inner diameter of the helix or the circle of fifths, two for the pitch circle around the helix, and one for pitch height) onto three-dimensional Euclidean space. I regard Shepard’s helical cylinder as the most successful model, phenomenologically speaking, of tonal pitch space, and will have more to say concerning why in a moment. But to say that it is phenomenologically most successful is not necessarily to say that it is the best in all respects. Lerdahl, who, as we saw, does not attach any very great significance to height in pitch space in Western tonal music, takes an entirely different approach. He is most concerned with developing a numerically expressible metric for representing distances in pitch space, chordal space, and regional or key space. These are different, if related, spaces he thinks Shepard’s model fails to distinguish. Each tone (understood as a pitch class: all the As, all the Bs, etc.) of the scale is given a ranking at a numerical level, with the tonic ranked first, the fifth ranked second, and the third ranked third (Figure 26.10). The tonic in the C major scale (represented as step 0 in the chromatic scale at the bottom level e) appears on all five levels, the dominant G (represented as step 7 in the chromatic scale) appears on levels e through b, and the major third E (represented as step 4) appears on levels e through c. These, of course, are the tones of the C major tonic triad. In the diagrams, a is the level of the octave. With each change of key, each pitch class is assigned a different number that reflects its scalar importance. In C major, the key at which the diagrams are set, the tonic C is ranked first with representation at all five levels. If we were to recalibrate to G major, G would be ranked first and C, now the subdominant, would be demoted to representation at only two levels, e and d. Still, the two scales are closely related because the new tonic tone, G, now represented at levels e through a, has been elevated only by one level of ‘embedding’ from its old location at levels e through b, and the scales share all pitch classes except one: in G major, F-sharp is substituted for F. Chordal distances and distances between regions or keys are calculated using similar formalisms. Unfortunately, length limitations make more detailed description here impossible. Lerdahl ties the experience of increasing tension and subsequent relaxation during musical listening to departure and return over distances ‘travelled’ in pitch, chordal, and regional spaces. Greater distances, especially those traversed via unexpected

508   Charles Nussbaum G A

F# G#

F E

B

FIFTH S

A#

C

D C#

D#

CHROMA

Fig.  26.8 The double helix wrapped around a torus in four dimensions (two for the inner diameter of the helix or the circle of fifths, and two for the pitch circle around the torus). Figure from Shepard (1982, 363), reprinted with kind permission from Elsevier. routes, are most productive of tension, while returns from such excursions are most conducive to relaxation. In defense of his numerical model, Lerdahl points out, quite rightly, that since we do not know how the brain performs its calculations of musical distance, there is no a priori reason to prefer topological models. Still, Lerdahl’s model has struck some as ‘ad hoc and complex’ and as possessing little psychological plausibility (Bigand and Pulon-Charronnat 2009, 65). Is musical experience, these critics ask, really a product of such unconscious numerical calculations? We cannot hope to adjudicate this issue here, but with a little tweaking, Shepard’s five-dimensional helical model exhibits considerable intuitive appeal and explanatory power. Imagine Shepard’s rising helix to be flexible so as to allow diametrical rotation around the axis running through the helical tube, to allow ‘vertical’ compression and extension (like a spring or a slinky), and to allow expansion and contraction of the helical tube in length and diameter. None of these modifications will compromise the basic topological structure. Nowhere is the helix cut, nowhere is it torn. These modifications can perhaps speak to Lerdahl’s complaint that such topological models are ‘too symmetrical’ (1988, 317). Recalling the evolutionary speculation just offered concerning heard pitch height and bodily orientation, let us also add a quasi-gravitational force that acts in a ‘downward’ direction, that is, in a direction opposite to the ‘upward’ spiralling of the helix. This dynamically

Musical Perception   509

Fig.  26.9 The double helix wrapped around a helical cylinder in five dimensions (two for the inner diameter of the helix or the circle of fifths, two for the pitch circle around the helix, and one for the height of the helix). Figure from Shepard (1982, 364), reprinted with kind permission from Elsevier. enhanced deformable pitch space, represented music-theoretically by the modified version of the Shepard helix just proposed, constitutes a musical action space, experienced by the comprehending, but music-theoretically unencumbered listener as a virtual egocentric space in which direction from ‘here’, comparative distance (farther/closer), and comparative size (bigger/smaller) are grasped non-conceptually without use of a metric, that is, as unit-free (see Peacocke 1992, 68–69; Nussbaum 2007, 5.5.2). I believe such a dynamic action space to be crucial for understanding musical experience, which I, like Robinson (2005), conceive as strongly embodied. With these modifications in place, Shepard’s model allows us to capture much that is central to the phenomenology of musical experience. Virtual listener position (obviously not the actual location of the physical listener in physical space) is always centred (minimally) at some tone, represented as a location on a segmentation of the helical coil.

510   Charles Nussbaum Total Pitch Space level a: level b: level c: level d: level e:

C C C C C

Db

level a: level b: level c: level d: level e:

D D

Eb

E E E

F F

F#

0 0 0 4 0 2 4 5 0 1 2 3 4 5 6

G G G G

Ab

A A

Bb

B B

(C) (C) (C) (C) (C)

7 7 7 9 b 7 8 9 a b

Fig.  26.10 Top:  The basic scalar space oriented to C major. Bottom:  A  numerical representation of the basic scalar space with the same key orientation. Figures from Lerdahl (1988, 321), reprinted with kind permission from the University of California Press and the author. Segmentations are shown as circular bands in the diagram. Each tone constitutes a possible listener position in an egocentric musical space. Suppose listener position is represented by the C’ shown at the top of the helix in Figure 26.6. Once this tone is taken to be the tonic or home tone, the helix may be imagined to rotate so as to place that C’ at the bottom of the inner helical tube, with all the Cs in all octaves, as before, lined up directly above and below it. Melodic motion is experienced as movement from one tone to another in this helically structured space, which can deform based on various musical cues. Here is an example. The humorously tentative scalar violin passages that occur in the Adagio introduction to the Finale of Beethoven’s First Symphony (1800) feel, because of dynamics, tempo, and other cues, like itty-bitty steps, very different in scope from the bounding strides of the upward-sweeping A major violin and viola scale preceding the fortissimo statement of the principal theme of the Vivace of the first movement of the Seventh Symphony (1812). For the First Symphony passage, a region of musical space contracts (unit-freely), which is represented in the model by bringing the relevant cross-sectional segmentations closer and reducing their diameters. Notice that the space itself must contract to achieve this effect, for all positions in virtual musical space are limited to tones (or tone types). The First Symphony scale starts out feinting G major. The helical model, therefore, may be imagined to rotate on its internal axis to place G at the bottom of the tube. Once the F natural is reached in bar six (played pianissimo, with very humorous effect), the feint is exposed for what it is. G major immediately gives way to C major and the helical model rotates again so as to place C at the bottom of the tube, until displaced by modulation. But what of the opening fortissimo full-orchestra multi-octave G chord? The listener in virtual musical space feels herself extended (I don’t say ‘standing’, for I have said nothing about the virtual ‘body shape’ assumed by the musical listener) like a perfectly balanced vertical beam in this helical space as if passing through the Gs on all turns of the coil, from the high G above the treble staff of the flute I to the G below the bass staff of the contrabass. I say extended ‘as if’ passing through all turns of the coil, because passage between and

Musical Perception   511 outside the turns is impossible in such a topological structure. But we are talking here of a musical chimera, of the phenomenology of musical spatial imagination unconstrained by topological possibility, something perhaps not unlike the topologically impossible pictorial representations of M. C. Escher. Notice, however, that by all rights, the helical model should not be a closed, continuous tube, but a discontinuous spiral of successive bands: there are no usable musical locations between the half-steps of the chromatic scale in traditional Western tonal music. Intermediate pitches are invariably classed as out-oftune neighbouring tones. But the phenomenology of musical movement is such that the intervening space seems continuous. The helical model may also be compressed or extended, depending on how large octaves are heard to be in context. Rapid octave oscillations on the piano, for example, the left-hand pattern in the opening of the first-movement Allegro di molto con brio of Beethoven’s Pathétique Sonata (1799), sound smaller than does the right-hand octave skip that begins the Sonata in G, Op. 14, No. 2 (1799). The former, consequently, calls for local compression, the latter for local extension. Octaves, recall, line up vertically between spiral turns. Expansions and contractions can also be localized. A melodic diminished minor third, for example, the pianissimo bassoon entrance that moves from G-flat to E above middle C in the introduction to the first movement of Beethoven’s Fourth Symphony (1807), sounds larger than does the enharmonically equivalent initial melodic major second (F-sharp to E) in the bassoon solo that opens ‘The Kalender Prince’ in Rimsky-Korsakov’s Scheherezade (1888). The former calls for an expansion between the relevant segmentations of the model. Such an expansion represents a heard local deformation. A ‘German’ augmented sixth chord is the enharmonic equivalent of a dominant seventh chord. But the augmented sixth in the former tends to sound smaller (expanding outward) than the minor seventh in the latter (contracting inward). As indicated, various melodic and harmonic events are heard as mandating compressions, extensions, expansions, and contractions. But none of these deformations will disturb the basic helical topology. Chords yield further interesting results. Although musical consonance and dissonance, as everyone agrees, vary with style, any acceptable chord in traditional Western tonal music will consist of tones fairly evenly distributed in helical space. The listener, therefore, will feel dynamically balanced in the musical gravitational field. Even Richard Wagner’s famously tensional Tristan chord, a four-note half-diminished seventh chord built on a tritone, the most unstable interval in Western classical tonal music, is fairly well distributed. On the other hand, a multi-note chord so dissonant as to be unusable in Western classical style before the twentieth century, say one constructed of four tones separated by half-steps, would be very poorly distributed. The structure of Shepard’s helical model graphically illustrates this. The tritone itself takes on a distinctive status in Shepard’s model, as it should. If a tone is assumed to be located at the bottom of some coil of the tube, its tritone complement will be located at the top of the tube on the opposite side, and vice-versa. This suggests the unstable symmetry of the tritone: its inversion yields another tritone. In this regard, the tritone is, with the exception of the octave, an interval unique in music using traditional Western scales. In the case of octave inversion, however, resultant tones remain aligned directly above and below one another. Melody may, then, be represented in this modified Shepard model as movement between segments of the helix, harmony as the balances achieved by combinations of tones in the vertical gravitational field.

512   Charles Nussbaum Finally, Shepard’s helical model also allows for the generation of more elaborate structures. Here we may think of ordered sequences of rotating and deforming helical space representations mandated by contrapuntally and/or harmonically complex works.

3 Chimerical individuals heard in music To this point, we have discussed musical qualities of loudness, rhythm, timbre, and pitch, along with their respective quality spaces. Rhythm, however, stands out from among these because the perception of rhythm, as opposed to the perception of loudness, timbre, or pitch, requires, not just allows, the recognition of pattern—and a pattern, ontologically speaking, is a type. Even to perceive a rhythm is to recognize a pattern. But the same applies, at least to some extent, to melody. A melody is a linear pattern of tones. Perception of type does not apply quite so obviously to harmony. A chord as a sound event is indeed a token of a type, but, because of the way the ear tends to fuse harmonies into a unified sonority, it is not clear that one must recognize its type to perceive it. One certainly need not recognize a tone as a token of its type to perceive it. Still, as in the case of melodies, a type-identical chord is recognizable as such when transposed to a different key. Chords are therefore easily perceived as tokens of types. To recognize any motif 9 in a musical work, then, is to recognize it as a type. But we tend not to experience motifs as types. Rather, we experience each tokening as the reappearance of an individual displaying a certain character. Compare Matthen (2010, 76): ‘melodies are analogues of three-dimensional objects in vision’. Such a musical individual is, of course, chimerical. But that detracts not at all from the compelling phenomenology of the experience. The comportments of these individuals, moreover, communicate an emotional tone by eliciting simulational responses in listeners. We may interpret each occurrence of a motif as a token gesture. But a gesture, strictly speaking, is a meaningful movement. It is not just an audible, and this requires the sense of something acting in musical space. As we hear musical sound, I propose, we simulate seemingly animate individuals in musical space. These chimerical musical individuals, moreover, are heard (or felt) to be in motion. How is this possible? Acoustically speaking, dynamic changes occur in music, but nothing literally moves: music consists entirely of series of evanescent tones. The initial temptation is to look to some analogy with the Phi phenomenon in vision, where successive flashing lights in darkness appear to move through the intervening space. But this analogy is weak. In order to fool the visual system, the spatial separation of the lights and the time interval between their flashing must be proportional and carefully calibrated so as to mimic the behaviour of moving objects in physical space. The visual system is fooled, but its application to physical space is preserved. An analogy with cinema also suggests itself. On the cinematic screen, nothing moves either. This comparison has something to recommend it, but it, too, has failings. The screen is a physical object we are viewing in physical space, and if the sequence of images on the screen is not quick enough, the illusion of motion will be lost. The illusion of musical motion suffers from no similar constraint.

9 

I conceive a motif as a pattern that is melodic, harmonic, and/or rhythmic.

Musical Perception   513 What is lacking in these analogies is recognition of the involvement in musical perception of the motor imagination.10 Melodic and harmonic musical structures, represented by the listener in the manner of hierarchically organized planning structures, entrain the listener’s motor system and encourage off-line simulation of actions in musical space. This enables the listener, even the music-theoretically untutored listener, to understand a musical sequence as an ensemble of gestures or pieces of communicative behaviour. ‘Covert imitation’, as Susan Hurley explains: may reflect a basic motivation of human beings, adults as well as children, to interact synchronously or entrain with one another, which is a mechanism of affiliation as well as of social perception and learning/ (2006, 211)

In most cultures, public musical performance has much in common with public ritual, especially with public religious ritual, and historically has had similar effects of affiliation, social perception, and learning. To the extent music subverts scene-analysis functions of the auditory system, it may not be an adaptation. But as a universal component of the socially constructed human extended phenotype, neither is it an evolutionary frill (cf. Patel 2008, 400ff).

References Bigand, E. and Poulin-Charronnat, B. (2009). ‘Tonal Cognition’. In S. Hallam, I. Cross, and M. Thaut (eds), The Oxford Handbook of Music Psychology (pp. 59–71). Oxford: Oxford University Press. Bregman, A. (1990). Auditory Scene Analysis. Cambridge, MA: Bradford/MIT Press. Changizi, M. A., Zhang, Q., and Shimojo, S. (2006). ‘Bare Skin, Blood and the Evolution of Primate Colour Vision’. Biological Letters. Doi: 10.1098/rsbl.2006.0440, 1–5. Davies, S. (2001). Musical Works & Performances: A Philosophical Exploration. Oxford: Oxford University Press. Donnadieu, S. (2007). ‘Mental Representation of the Timbre of Complex Sounds’. In J. Beauchamp (ed.), Analysis, Synthesis, and Perception of Musical Sounds: The Sounds of Music (pp. 272–319). New York: Springer. Halpern, A. (2003). ‘Cerebral Substrates of Musical Imagery’. In I. Peretz and R. Zarorre (eds), The Cognitive Neuroscience of Music (pp. 217–230). Oxford: Oxford University Press. I. Peretz and R. Zarorre Hurley, S. (2006). ‘Active Perception and Perceiving Action: the Shared Circuits Model.’ In T. Gendler and J. Hawthorne (eds), Perceptual Experience (pp. 205–159). Oxford: Clarendon Press. Kelly, J. (1991). ‘Hearing’. In E. Kandel, J. Schwartz, and T. Jessell (eds), Principles of Neural Science (Third Edition) (pp. 481–499). Norwalk, CT: Appleton & Lange. Lerdahl, F. (1987). ‘Timbral Hierarchies’. Contemporary Music Review, 2, 135–160. Lerdahl, F. (1988). ‘Tonal Pitch Space’. Music Perception, 5, 3, 315–350. Lerdahl, F. (2001). Tonal Pitch Space. Oxford: Oxford University Press.

10  Levitin (2006, 174) helpfully draws attention to the ‘similarities between music perception and motor action planning’. See also Nussbaum (2007, 2.3).

514   Charles Nussbaum Lerdahl, F. and R. Jackendoff (1983). A Generative Theory of Tonal Music. Cambridge, MA: Bradford/MIT Press. Levitin D. (2006). This Is Your Brain on Music:  The Science of a Human Obsession. New York: Dutton. Matthen, M. (2005). Seeing, Doing, and Knowing: A Philosophical Theory of Sense Perception. Oxford: Oxford University Press. Matthen, M. (2010). ‘On the Diversity of Auditory Objects’. Review of Philosophy and Psychology, 1, 63–89. Nakamura, S., Sadato, N., Oohashi, T., Nishina, E., Fuwamoto, Y., and Tonekura, Y. (1999). ‘Analysis of Music-Brain Interaction with Simultaneous Measurement of Regional Cerebral Blood Flow and Electroencephalogram Beta Rhythm in Human Subjects’. Neuroscience Letters, 275, 22–226. Nietzsche, F. (1887/1967). On the Genealogy of Morals, trans. W. Kaufmann. New  York: Vintage Books. Nussbaum, C. (2007). The Musical Representation:  Meaning, Ontology, and Emotion. Cambridge, MA: Bradford/MIT Press. O’Brien, J. (1977). Non-Western Music and the Western Listener. Dubuque, IA: Kendall/Hunt Publishing Company. Patel, A. (2008). Music, Language, and the Brain. Oxford: Oxford University Press. Peacocke, C. (1992). A Study of Concepts. Cambridge, MA: Bradford/MIT Press. Robinson, J. (2005). Deeper than Reason: Emotion and its Role in Literature, Music, and Art. Oxford: Clarendon Press. Shepard, R. (1982). ‘Structural Representations of Musical Pitch’. In D. Deutsch (ed.), The Psychology of Music (pp. 343–390). New York: Academic Press, Inc.

Chapter 27

Ow n-Body Perception Alisa Mandrigin and Evan Thompson

1 Introduction Our concern in this chapter is with four interrelated aspects of perception. First, perception is perspectival; you always perceive the world from a particular point of view. Second, the point of view from which you perceive the world is usually determined by the location of your body. Third, your body is one of the things you perceive, either from the outside, as when you look down at your hands, or from the inside, as when you experience your arms extended or your body as upright or inverted. Fourth, you do not merely experience your body as having these parts and properties; you also experience these body parts and properties as your own. In summary, one of the things you perceive from your first-person perspective is a particular body that you also perceive as your own. Psychologists and neuroscientists call this kind of perception ‘own-body perception’. Own-body perception is a type of bodily self-awareness. In perceiving your body as your own, you are aware of your bodily self. Besides own-body perception (‘this body is mine’), bodily self-experience includes agency (‘I’m the one making this movement’), self-location (‘I’m in my body’), and egocentric perception (‘I see the world from here’). Notice that these aspects of bodily self-awareness normally coincide—you feel that the body in which you are located (self-location) and through whose eyes you see the world (egocentric perception) is your own body (ownership) that you control from within (agency). Illusory own-body perceptions, however, disturb this unity. For example, in the socalled rubber hand illusion, subjects locate a sensation produced by something that is touching their own hand in a rubber hand, outside their bodily space (Botvinick and Cohen, 1998). Illusory own-body perceptions have also been created using virtual reality images (Lenggenhager et al., 2007) or real-time video images (Ehrsson, 2007). In these cases, subjects locate passively experienced touch sensations in a virtual body they see in front of them (Lenggenhager et al., 2007), or they experience themselves as displaced to a location behind where they are actually sitting (Ehrsson, 2007). Finally, illusory ownbody perceptions can also occur spontaneously without experimental induction, notably

516    Alisa Mandrigin and Evan Thompson in so-called autoscopic phenomena, in which an individual sees his or her body from an outside perspective (Blanke and Mohr, 2005). In this chapter, we first review these kinds of disruptions to bodily self-awareness, including what is known about their neural correlates, and we consider their significance for understanding own-body perception. We then argue that it is crucial to distinguish between the sense of ownership for one’s body as an object of perception—the body-asobject—and the sense of ownership for one’s body as that by which and through which one perceives the world—the body-as-subject (Thompson, 2005; Legrand, 2006; Legrand and Ruby, 2009; Christoff et al., 2011). Put another way, the body-as-subject can be described as the embodied and subjective perspective of perception, in contrast to the body perceived as one object among others from within that perspective. Despite the fact that illusory own-body perception provides an excellent case for illustrating this distinction, most discussions to date of own-body perception have been limited by a failure to make this distinction and apply it to the various clinical and experimental findings. Third, we summarize one recent model of the body-as-subject, according to which the body-as-subject is based on sensorimotor integration (Legrand and Ruby, 2009; Christoff et al., 2011). Finally, we use this model to clarify the phenomenon of illusory own-body perception.

2  The rubber hand illusion The rubber hand illusion, first reported by Botvinick and Cohen (1998), is the illusion of experiencing a rubber hand as being one’s own hand. To induce the rubber hand illusion, subjects see a rubber hand being stroked in synchrony with the stroking of their own hand, which is hidden from view. Subjects report seeming to feel the touch of the brush that is stroking the rubber hand, instead of the hidden brush stroking their hand, as if the rubber hand had sensed the touch. Subjects also report feeling ownership over the rubber hand, that is, that the rubber hand is their own hand (Tsakiris and Haggard, 2005: 80). This report might be interpreted in at least two ways: the rubber hand might replace the subject’s own hand in her experience of her body; alternatively, the rubber hand might simply be incorporated into the body and experienced as a third hand over which the subject feels ownership without any loss of experience of and ownership over her actual hand. Both behavioural measures and introspective reports made by subjects suggest that the former is the correct interpretation of the illusion. Subjects deny that they feel as if they have three hands when they experience the illusion and agree with the statement that they feel their own hand to have disappeared (Longo et al., 2008). There are changes to homeostatic regulation of the subject’s own hand when the subject experiences ownership over the rubber hand. Moseley et al. (2008) found that the skin temperature of the subject’s own hand dropped during the illusion. The drop in skin temperature correlated with the intensity of the illusion, and was limited to cases in which subjects reported feeling ownership over the rubber hand. By contrast, subjects who see a rubber hand being stroked asynchronously from the stroking of their own hidden hand do not report feeling any such ownership over the rubber hand. When asked to make a report on the location of their own hand post-illusion, and once both their own hand and the rubber hand are hidden from view, subjects mislocate

Own-Body Perception   517 their hand, deviating towards the location of the rubber hand. Subjects who receive asynchronous stroking on their own and on the rubber hand manifest a smaller drift effect—known as proprioceptive drift—towards the location of the rubber hand when they report on the location of their own hand. Both reports of felt ownership and measurements of proprioceptive drift as evidenced by reports on the location of one’s hand after synchronous stroking are used as measures of the illusion when contrasted with the asynchronous condition. Botvinick and Cohen (1998) explain the illusion in terms of multisensory integration (the integration of information from different sense modalities), specifically the integration of vision, touch, and proprioception. Temporally synchronous but spatially incongruent visual and tactile inputs are integrated together. The dominance of vision over touch results in the ‘mislocalization of the tactile percept towards the spatial location of the visual percept’ (Tsakiris and Haggard, 2005: 80). The dominant role of vision over proprioception results in the subject judging that her hand is located closer to the position of the rubber hand. Similarly, Tsakiris states that the rubber hand illusion involves ‘a three-way weighted interaction between vision, touch, and proprioception: vision of tactile stimulation on the rubber hand captures the tactile sensation on the participant’s own hand, and this visual capture results in a mislocalization of the felt location of one’s own hand towards the spatial location of the visual percept’ (Tsakiris, 2010: 705). Besides multisensory integration, however, representation of the body also seems to be involved in the rubber hand illusion, since the effect is diminished if the placement of the rubber hand is incongruent with that of the subject’s own hand. Tsakiris and Haggard found that placing the rubber hand at a 90° angle relative to the position of the subject’s own hand led to there being no significant difference in proprioceptive drift between synchronous and asynchronous conditions (Tsakiris and Haggard, 2005: 81–3). Moreover, after stimulation with a neutral, non-hand like object, in this case a wooden stick, in place of the rubber hand, the subjects’ judgements about the location of their hand were not significantly different from their judgements after asynchronous stimulation. Tsakiris and Haggard amend the multisensory integration view by suggesting that although ‘bottom-up processes of visuotactile correlation drive the illusion,’ it is also the case that long-term body-scheme representations can modulate the illusion (Tsakiris and Haggard, 2005: 91). The failure to induce the illusion when the rubber hand was placed at a posturally incongruent angle to the position of the actual hand, as well as when the rubber hand was replaced with a stick, suggests that when the visual stimuli in some way fails to cohere with a pre-existing representation of the body, then the illusion will be attenuated.

3  Seeing one’s body in extra-personal space In global versions of the rubber hand illusion, external representations of the body are used to induce a feeling in the subject that a tactile sensation is felt in an image of the body presented in extra-personal space (Ehrsson, 2007; Lenggenhager et al., 2007). As with the

518    Alisa Mandrigin and Evan Thompson rubber hand illusion, these experiments are used to investigate one’s feeling of body ownership. The aim is to expand the illusion in order to investigate one’s sense of ownership of the body as a whole rather than one’s sense of ownership of a particular limb (Blanke and Metzinger, 2009). Consistent with the rubber hand illusion, subjects report feeling that the genuine tactile sensations they receive are located on the perceived body, at some distance from the location of the stimulation. These body illusion experiments, described in more detail herein, seem to induce elements of a set of illusions collectively labelled ‘autoscopic phenomena’, in which subjects report the experience of seeing their own body from an outside perspective (Blanke and Mohr, 2005). Autoscopic phenomena are usually divided into the following three categories: (i) autoscopic hallucinations, in which the subject sees an illusory duplicate of her body from the perspective of her actual body but does not experience any ownership or control over this body; (ii) out-of-body experiences, in which the subject feels ownership for a body that she recognizes as her own body but that is seen from the outside; and (iii) heautoscopy, in which the subject reports that her visual perspective switches back and forth between her normal body and an illusory second body, for which she also reports feeling a sense of ownership. In the case of out-of-body experiences, subjects report seeing their own body from an outside perspective and at a distance from their own body, often from an elevated perspective. A similar outside perspective occurs in the ‘observer perspective’ of episodic memory, in which one sees oneself from the outside (typically from an elevated vantage point), in contrast to recalling the scene from the perspective of one’s own eyes (the ‘field perspective’) (Nigro and Neisser, 1983; McIssac and Eich, 2002). In the case of out-of-body experiences, however, the subject takes the experience to be an occurrent perceptual one, not a memory. Blanke and Mohr (2005) describe out-of-body experiences as generating a sense of disembodiment—a sense of oneself, as the subject of experience, being outside of one’s body. In contrast, for subjects who experience autoscopic hallucinations, there is no sense of disembodiment. Despite the experience of seeing a duplicate of one’s body, there is no shift in visuo-spatial perspective from the subject’s genuine body to the perspective of the illusory duplicate. Finally, in heautoscopy, subjects report both seeing a double of their own body and a sense of shifting between two possible perspectives—the perceptual perspective from the body’s actual location and the perspective from the double’s location. Nevertheless, they do not have a clear sense of disembodiment, but instead report confusion about their spatial location and perspective (Blanke and Mohr, 2005: 187). Autoscopic phenomena can also be described in terms of the differences in the visuo-spatial perspective that subjects report in each case (Blanke and Mohr, 2005: 187). In autoscopic hallucinations, the visuo-spatial perspective is body-centred; in out-ofbody experiences, it is extra-corporeal; and in heautoscopy, it is extracorporeal and body-centred. Thus autoscopic phenomena can be classified according to the degree to which the subject experiences an alteration in her visuo-spatial perspective on the world. The more sustained the experience of displacement toward the duplicate body, the stronger the subject’s sense of disembodiment. Blanke and Metzinger (2009) offer another system of classifying autoscopic phenomena. They suggest that we should understand the changes to bodily self-awareness in autoscopic phenomena as changes to one’s bodily sense of selfhood. According to Blanke and Metzinger, the simplest sense of self, which they call ‘minimal phenomenal selfhood’,

Own-Body Perception   519 comprises (i) having a perspectival model of reality; (ii) locating oneself in space and time; and (iii) identifying oneself with a representation of the body as a whole. A more robust sense of self or ‘strong first-person perspective’ occurs when one mentally represents oneself as a subject that is directed toward the objects of perception, including one’s own body. Using this framework, out-of-body experiences and heautoscopy can be understood as indicating a misrepresentation of one’s location in space together with changes in one’s experience of bodily possession, thereby interrupting one’s bodily sense of selfhood and altering one’s experience of having a perspective on the world. Blanke and Metzinger contend that, depending on which of the three contributing factors to minimal phenomenal selfhood is disturbed—the felt origin of one’s perspectival model of the world, self-location in space and time, or body identification—the subject will report experiences consistent with out-of-body experiences or heautoscopy. In the case of out-of-body experiences, subjects report feeling disembodied and having a displaced perspective on the world. According to Blanke and Metzinger’s model, this type of experience is caused by a disturbance in spatial self-location and a disturbance in the felt origin of the weak first-person perspective—both are located at a position outside the subject’s actual body. In addition, Blanke and Metzinger suggest that one identifies oneself with an illusory body that seems to be located at this position, as opposed to identifying oneself with one’s genuine body, which one perceives from the outside. Moreover, given that these aspects of minimal phenomenal selfhood are disturbed, so too is the strong first-person perspective. In contrast, in heautoscopy, one’s weak first-person perspective originates from either the real body or the illusory body, or alternates between them, and one locates oneself in and identifies with both bodies. Thus, all three aspects of minimal phenomenal selfhood are pathologically disrupted in heautoscopy. Finally, in autoscopic hallucinations, since subjects experience no shift in visuo-spatial perspective and no sense of disembodiment, all three aspects of minimal phenomenal selfhood are normal: one locates oneself at the position of the represented body, one identifies with this represented body, and one’s weak first-person perspective is represented as originating from the body. Presumably, then, according to Blanke and Metzinger’s model, this type of illusion is merely a visual hallucination, one in which one sees a double of one’s body in one’s extra-personal space, but for which one experiences no sense of ownership. Nevertheless, in Blanke and Metzinger’s model, both out-of-body experiences and heautoscopy can be understood as involving changes to one’s bodily sense of selfhood through the disturbance of features that constitute minimal phenomenal selfhood.

4  Neural correlates of out-of-body experiences Evidence from neuroscience suggests that out-of-body experiences depend on a particular area of the brain where the temporal and parietal lobes meet, known as the temporoparietal junction (TPJ). This area is known to be crucial for multisensory integration—for integrating signals from the different sensory modalities of sight, sound,

520    Alisa Mandrigin and Evan Thompson touch, and self-movement—and for being able to switch between taking a first-person perspective and a third-person perspective in mental imagery. The main evidence for the involvement of the TPJ in out-of-body experiences comes from studies of neurological patients. In 2002 Blanke and colleagues reported that they had repeatedly induced out-of-body experiences by electrically stimulating the brain of a patient being treated for a drug-resistant epilepsy (Blanke et al., 2002). During stimulation of the right angular gyrus—a structure belonging to the area of the TPJ—the patient reported having experiences resembling out-of-body experiences. The first stimulations produced vestibular feelings that the patient described as ‘sinking into the bed’ or ‘falling from a height’. When the doctors increased the amplitude of the electrical current, the patient reported, ‘I see myself lying in bed, from above, but I only see my legs and lower trunk.’ Two further stimulations produced the same experience, including an instantaneous feeling of ‘lightness’ and ‘floating’ about two metres above the bed, near the ceiling. In a subsequent study (Blanke et al., 2004), Blanke and colleagues recorded detailed information about spontaneous out-of-body experiences in this patient and five other neurological patients. The following is a description of one patient’s out-of-body experience just prior to surgery: The patient was lying in bed and awakened from sleep, and the first thing she remembered was ‘the feeling of being at the ceiling of the room.’ She ‘[. . .] had the impression that I was dreaming that I would float above [under the ceiling] of the room [. . .].’ The patient also saw herself in bed (in front view) and gave the description that ‘the bed was seen from above’ and that ‘there was a man and that she [the patient] was very frightened.’ The scene was in colour, and was visually clear and very realistic. (Blanke et al 2004: 247)

Blanke and colleagues found that five of these patients had brain damage in the TPJ of the right hemisphere. In a later study, the same brain region was found to be activated within half a second when healthy individuals were asked to imagine seeing things from an outof-body perspective (Blanke et al., 2005). Furthermore, interfering with this brain region by stimulating it magnetically impaired the ability to imagine this transformation of body position. These findings along with others have helped neuroscientists to build up a picture of the TPJ as a key neural site for the integration of information related to own-body perception, including how we recognize others on the basis of their bodies, as well as our sense of how our own body looks from the outside to other people (Arzy et al., 2006). The right TPJ includes the core region of the vestibular cortex, which is responsible for our sense of balance and spatial orientation. Other regions belonging to the TPJ coordinate proprioceptive, tactile, and visual information about the body. In addition, the TPJ is known to be involved in the perception of the human body, imagining one’s own body, switching between first-person and third-person perspectives, and being able to distinguish between oneself and others. In out-of-body experiences, vision, proprioception, and vestibular awareness come apart. One sees oneself as being at a location that does not coincide with the source of one’s egocentric visual perspective and with the source of one’s vestibular awareness. Since the TPJ is involved in integrating these different kinds of sensory information, it makes sense

Own-Body Perception   521 to suppose that out-of-body experiences depend on some kind of disruption to multisensory integration at the TPJ. Blanke proposes that out-of-body experiences happen when the TPJ suddenly fails to integrate sensory signals from the body in the normal way (Blanke and Arzy, 2005). Specifically, he proposes that two disturbances to multisensory integration combine to produce out-of-body experiences. On the one hand, proprioceptive, tactile, and visual signals about one’s own body are not properly matched to each other. On the other hand, the vestibular frame of reference for one’s personal space is not properly matched to the visual frame of reference for external space. These two sensory-integration disruptions combine to create the experience of seeing one’s body in a position that does not coincide with its felt location, together with the experience of floating and seeing things from an elevated visuo-spatial perspective.

5  Methods of inducing autoscopic illusions Although the following experiments differ in their details, in general subjects are presented with tactile stimuli on their body in conjunction with a visual representation of a virtual body, a mannequin, or a video image of their own body being touched in the same way. The visual stimuli are presented either in synchrony with the tactile stimulation or asynchronously. Subsequently, subjects are asked a series of questions based on those used in the rubber hand illusion experiment to investigate their sense of body ownership. In the Lenggenhager experiments (Lenggenhager et al., 2007), a video image of the subject’s back was filmed and presented to the subject through video-display goggles. The subject saw a video-enhanced image of his own back being stroked from the camera’s view so that he was presented with an image of his back located in a position in front of him. In a further experiment, subjects viewed video images of a fake body or a non-body shaped object. Ehrsson’s (2007) version of the illusion used a similar configuration but with two video cameras behind the subject to create an image of the subject’s back. Rather than stroking the subject’s back, as in the Lenggenhager experiment, Ehrsson had the experimenter perform a horizontal motion in space with a rod towards the subject, culminating in the rod touching the subject’s chest. A second rod performed a similar motion back and forth in front of the video cameras, towards a point just below them. Subjects were thus presented with an image of a rod performing a motion that coincided temporally with the touching of their own chest, but also with their own bodies, perceived from the back, located in front of them. Lastly, in Petkova and Ehrsson’s (2008) experiments, subjects were presented with video footage, through head-mounted video-display goggles, of a mannequin’s chest being stroked from the viewpoint of the mannequin’s head. An additional experiment, designed to induce what they describe as a ‘full-blown body-swap experience’ (p. 4), involved an experimenter shaking hands with the subject while the subject wore a head-mounted camera. The subject, adorned with the head-mounted display, is presented with the visual image of his own hand from the perspective of the experimenter, thereby inducing, it is suggested, the experience of shaking hands with himself.

522    Alisa Mandrigin and Evan Thompson Across the differences in these experimental set-ups, subjects report that they experience the tactile sensation as located in the body that they visually perceive, and some subjects report that they experience a sense of being seated behind their own bodies, able to perceive their own body from a separate location (Ehrsson, 2007: 1048). As in the rubber hand illusion, visual information seems to take precedence, inducing the sense that a tactile sensation one experiences is located on the body that one visually perceives at a distant location from the actual sensation and one’s actual position. The illusion induced by these experimental paradigms has been called a ‘full body illusion’ (Blanke and Metzinger, 2009). The adaptation of the rubber hand illusion experimental design to incorporate a larger image of the body gives some support to this way of describing the illusion. In addition, the feeling that subjects report of being located at a different position in space from their own bodies provides some reason to think that the illusion involves the whole body. Nevertheless, without further investigation of the extent to which the illusion is localized to the particular parts of the body that are being stimulated, it is difficult to know whether subjects experience a full body illusion rather than a partial one (Smith, 2010). Furthermore, it is worth considering whether subjects experience full body displacement, or rather whether they merely experience ownership over a body that is not their own. As with the rubber hand illusion, there is a question about whether the subject merely gains ownership over an extra body or body-part, or whether in addition the subject disowns their own body or body-part. In a recent addition to this experimental work, Ionta, Blanke, and colleagues used functional magnetic resonance imaging (fMRI) to investigate neural activity during autoscopic illusions (Ionta et al., 2011). The subjects lay on their back in the scanner and looked upward at a virtual body, and they received tactile stimulation to their back while viewing tactile stimulation being given to the virtual body on its back. To investigate shifts in the experience of self-location after synchronous versus asynchronous stimulation, participants were asked to judge how long a ball they held in their hand would take to reach the ground if dropped. The subjects also made free reports about their feelings of body ownership and their perspective on the world. Only during synchronous stroking—when the subjects felt their back being stroked at the same time as they saw the virtual back being stroked—did the subjects report feeling as though the virtual body was their own. The mental ball-dropping time estimates also showed that the subjects perceived their physical body drifting toward the virtual one during synchronous stroking. Yet the subjects also reported striking differences in the direction of their visual perspective. Half the subjects had the impression of being below the virtual body and looking up at it (the Up group); the other half had the impression of being above the virtual body and looking down at it (the Down group). In other words, the visuo-spatial perspective of the Up group was consistent with their actual supine physical position and first-person perspective in the scanner, whereas these were inconsistent for the Down group, who experienced an elevated visuo-spatial perspective and sensations of floating, as in out-of-body experiences. When Ionta and colleagues examined the brain activity, they found that the illusory changes in self-location correlated specifically with changes in activity at the TPJ. They also found there were different patterns of activity in this area for the Up and Down groups. Ionta and colleagues interpret these differences as showing that activity in the TPJ reflects not only how the integration of visual and tactile signals influences one’s

Own-Body Perception   523 sense of self-location, but also how the integration of visual and vestibular information about self-location and the orientation of one’s visuo-spatial perspective influences one’s first-person perspective. These results of combining brain imaging and virtual reality methods of manipulating bodily self-awareness support the idea that out-of-body experiences depend on how the TPJ deals with visual, tactile, and proprioceptive cues about one’s own body, on the one hand, and visual and vestibular cues about one’s bodily orientation in space, on the other hand. More generally, what these virtual reality experiments have done is to use the power of vision over other senses to manipulate bodily self-awareness in systematic ways that reveal different aspects of our bodily sense of self. We locate ourselves within our body, but we can be made to feel that we are located at places outside the borders of our body. We experience the world from the visuo-spatial perspective of our body, but we can be made to experience the world from perspectives outside our body with different ‘up’ or ‘down’ orientations. We feel that we own our bodies, but manipulating both our visuo-spatial perspective and the sensory cues we get from vision, touch, and proprioception can make us feel ownership for an artificial body or a purely virtual body.

6  Body-as-subject versus body-as-object Despite the experimental advances described in the previous sections, discussions of their significance for understanding own-body perception and bodily self-awareness have been hampered by a failure to distinguish clearly between two modes of bodily self-experience—the sense of ownership for one’s body as that through which and by which one perceives the world—the body-as-subject—and the sense of ownership for one’s body as a perceptually presented object (Legrand, 2010). We suggest that greater clarity can be brought to our understanding of the experimental findings, as well as of spontaneous autoscopic phenomena, by making use of the conceptual and phenomenological distinction between the body-as-subject, which structures perceptual experience and grounds higher levels of self-consciousness (Bermudez, 1998; Legrand, 2006), and the body as a perceived object presented within the perspective that the body-as-subject provides. The question we need to address is the extent to which the autoscopic experiments manipulate the body-as-subject versus the extent to which they manipulate the body-asobject. In other words, to what extent do the experimental manipulations alter how one experiences the body as that by which and through which one perceives—to what extent do they alter the embodied and subjective perspective of perception itself—versus to what extent do they alter how one’s body appears as a perceptual object from within that perspective? To answer this question we need to clarify exactly what the body-as-subject of perception is in contrast to the body-as-object of perception, and we need to provide a functional model of the body-as-subject that can be applied to the empirical investigations of own-body perception. We will then be in a better position to account for the feeling of disembodiment that subjects report in spontaneous out-of-body experiences as well as how this feeling relates to the manipulations of own-body perception in the experimentally induced cases.

524    Alisa Mandrigin and Evan Thompson

7  A sensorimotor model of the body-as-subject The concept of the body-as-subject of perception can be presented in terms of the systematic linkage of sensory and motor processes in the perception-action cycle (Legrand, 2006; Legrand and Ruby, 2009; Christoff et al., 2011). As a perceiver, one needs to be able to distinguish between sensory changes arising from one’s own motor actions (self) and sensory changes arising from the environment (nonself). The central nervous system (CNS) distinguishes the two by systematically relating the efferent signals (motor commands) for the production of an action (e.g. eye, head, or hand movements) to the afferent (sensory) signals arising from the execution of that action (e.g. the flow of visual or haptic sensory feedback). According to various models going back to Von Holst (1954), the basic mechanism of this integration is a comparator that compares a copy of the motor command (information about the action executed) with the sensory reafference (information about the sensory modifications due to the action) (Wolpert et al., 1995). Through such a mechanism, one can register that one has executed a given movement, and one can use this information to process the resulting sensory reafference. The crucial point for our purposes is that reafference is self-specific, for it is intrinsically related to one’s own action (there is no such thing as a nonself-specific reafference). Thus, by relating efferent signals to their afferent consequences, the CNS marks the difference between self-specific (reafferent) and nonself-specific (exafferent) information in the perception-action cycle. In this way, the CNS implements a functional self/nonself distinction that implicitly specifies one’s body as the perceiving subject and agent—as that through which and by which one perceives. It bears emphasizing that the reafferent-efferent processes just described specify one’s body not as a perceptual object, but instead as the perceptual subject and agent of action. For example, consider the visuo-motor act of looking up at the sky to track a bird and the resulting visual perception. This perceptual experience is characterized by (i) a specific content (the flying bird), (ii) a specific mode of presentation (seeing, not hearing), and (iii) a specific perspective (my experience of looking and seeing). The process of relating an efference (the head and eye movements of looking) to a reafference (the resulting sight of the bird) is what allows the perception to be characterized not only by a given content (the flying bird) but also by a self-specific perspective (I am the one looking at the bird) (Legrand, 2006; Legrand and Ruby, 2009). The perceiver’s embodied perspective is thus of central importance to this framework. Sensorimotor integration implements a unique embodied perspective on the world. Although the basic sensorimotor integration processes do not require any explicit representation of the bodily subject per se, the embodied perspective they implement is nonetheless self-specific in the strict sense of being both exclusive—it characterizes oneself and no one else—and noncontingent—changing or losing it entails changing or losing the distinction between self and nonself (Legrand and Ruby, 2009). In the general case, one perceives and acts from one’s self-specific perspective while implicitly experiencing oneself as a bodily subject. In some particular cases, what one perceives is one’s body-as-object, as when one visually recognizes oneself. Whereas the body-as-object consists in the perceptual features one recognizes or attributes to oneself, the body-as-subject consists in the

Own-Body Perception   525 self-specific, embodied perspective from which such perceptions occur. Hence to explain the body-as-subject we need to explain how such an embodied perspective is implemented. Our proposal is that reafferent-efferent processes of sensorimotor integration implement a self-specific, embodied perspective that functions as that through which and by which one perceives the world. In this way, sensorimotor integration produces the body as subject and agent of perception.

8  Out-of-body but not disembodied As we have seen, scientists and philosophers often describe certain types of spontaneous autoscopic phenomena as experiences in which one seems to oneself to be disembodied. The sensorimotor model of the body-as-subject, however, indicates that this description is inaccurate and in need of substantial revision. The crucial point is that the experience of perceiving one’s body as an object from the outside in out-of-body experiences and heautoscopy occurs within a phenomenal perspective that is embodied and sensorimotor in character, and thus implies a sense of oneself as a bodily subject, specifically a sense of ownership for one’s body-as-subject. Consider that subjects typically describe out-of-body experiences as happening in a bodily space that is perceived in an egocentric frame of reference and in relation to possible bodily movements. The first-person reports indicate that subjects experience themselves as having a visuo-spatial perspective (egocentric perception), as being located at the origin of that perspective (self-location), and sometimes as being able to move deliberately through space (agency). These features are present even in cases in which subjects report experiencing themselves as not having a body but as being a blob or a point of light (Blanke and Arzy, 2005: 22). No matter what physical or spatial form one takes, to have a visuo-spatial perspective and to be able to move through space or to direct one’s visual attention in space imply the experience of oneself as a bodily subject, at least in a minimal sensorimotor sense. In other words, even if one experiences oneself as a passive observational point of view, one still experiences oneself as able to look in this or that direction, that is, as being able to direct one’s visual attention within phenomenal space. Having this ability implies experiencing oneself as being situated in space and as possessing some kind of sensorimotor repertoire, not as a disembodied self that lacks all bodily properties. So far we have been describing the spatial perspective or frame of reference in out-ofbody experiences as an egocentric visual perspective, but it is also a gravity-centred or geocentric spatial reference frame with vertical ‘up’ and ‘down’ directions. In general in ordinary experience and in out-of-body experiences, we have a constant knowledge of the vertical orientation and of which way is up. Sensory receptors in the inner ear, the otolith vestibular receptors, are sensitive to gravity, linear acceleration (the rate of change in velocity as one moves in a given direction), and horizontal movement. In out-of-body experiences, subjects usually report that they seem to be above their physical bodies, and they typically describe feelings of rising rapidly, floating, and flying horizontally. Such sensations all involve the vestibular system of the inner ear (the semicircular canals indicating rotational movements, and the otoliths indicating linear accelerations). In short, the sense of self in out-of-body experiences includes not just having a visual egocentric perspective

526    Alisa Mandrigin and Evan Thompson but also a vestibular geocentric perspective with a bodily sense of balance, movement, and up-down orientation. For these reasons, the term ‘out-of-body experience’ is potentially misleading. A better term might be ‘out-of-own-body experience’, where this term is taken to mean the experience of seeing one’s body as an object perceived from a vantage point outside that body. To put the point another way, out-of-body experiences are not so much experiences of disembodiment as experiences of altered embodiment. In an out-of-body experience, one sees one’s body as perceptually presented from the outside and as being in a location that does not coincide with the felt location of one’s visual and vestibular awareness. In other words, one sees one’s body as an object at a place that does not coincide with the felt location of one’s awareness as a bodily subject. In this way, out-of-body experiences illustrate the crucial distinction between the body-as-subject and the body-as-object. In spontaneous out-of-body experiences, the body-as-object is one’s body seen from the outside, whereas the body-as-subject is oneself as perceiver. To put the distinction another way, the body-as-object is the external body image one identifies as one’s own body, thereby also feeling a sense of ownership for that body, whereas the body-as-subject is the felt origin of the visual (egocentric) and vestibular (geocentric) perspective from which one makes that identification and for which one feels a different kind of ownership, namely, ownership of one’s embodied perspective or body-as-subject. We can now say in more precise terms what makes out-of-body experiences (and heautoscopy) experiences of altered embodiment rather than of disembodiment. In out-of-body experiences, there is a dissociation between one’s body-as-object and one’s body-as-subject. Normally one experiences one’s body-as-object as being in the same place as one’s body-assubject. In out-of-body experiences, however, this unity comes apart, so that one’s bodyas-object and one’s body-as-subject have different locations. What these experiences also show is that when one’s body-as-subject and body-as-object do come apart this way, one’s sense of self adheres to one’s sense of attentional agency, and one’s sense of self-location adheres to one’s sense of visuo-spatial perspective. In other words, one’s sense of agency and location are determined by one’s body-as-subject and not one’s body-as-object.

9  The whole-body illusion experiments revisited We are now in position to clarify how the sense of ownership for the body-as-subject versus the body-as-object are related in the experiments on illusory whole-body perception. On the one hand, many of the principal experimental findings can be accounted for in terms of atypical perceptual experience of the body-as-object. The body is visually perceived as occupying a strange location relative to the usual position one sees it as occupying from one’s embodied first-person perspective. This perceptual change implies a change in the perceptual presentation of the body-as-object, but does not necessarily require any change in the embodied perspective itself. Thus subjects report that they experience tactile sensations as located in a body that they visually perceive at a distance, and they report

Own-Body Perception   527 feeling a sense of ownership for this visually presented body. In this case, the sense of ownership being expressed is for the perceptually presented and experimentally manipulated body-as-object. On the other hand, in some cases the experimental procedures do affect the body-assubject of perception or the embodied perspective itself. Thus, subjects sometimes report changes to their sense of spatial location, as when they report experiencing a sense of being seated behind their own bodies or floating above and looking down on a virtual body presented above them while they lie in the MRI scanner. In these cases, the subjects experience changes to their embodied perspective or body-as-subject, specifically to their sense of self-location as perceiving subjects and to their egocentric (visuo-spatial and vestibular) perspective. Notice that these changes to the body-as-subject make these altered own-body perceptions closer to spontaneous out-of-body experiences than do the experimental manipulations that affect only the sense of ownership for the body-as-object (as in the rubber hand illusion).

10  Conclusion and future directions A principal motivation for experimental research on own-body perception is to investigate how the sense of self is related to embodied experience. As we have seen, by changing the stimuli from a rubber hand to an image of the body as a whole, scientists aim to investigate ‘global, fully embodied self-consciousness . . . experienced as a single feature, namely a coherent representation of the whole, spatially situated body’ (Blanke and Metzinger, 2009:  9). Our discussion, however, reveals an important limitation in this approach to date. Researchers seem to have assumed that, by investigating the sense of ownership for the body as perceptually presented—the body-as-object—they are specifically targeting one’s bodily sense of selfhood. Yet, as we have seen, this assumption is unwarranted for two reasons. First, experimentally manipulating the sense of ownership for the body-asobject does not necessarily target the body-as-subject. To target the body-as-subject one needs to manipulate directly the reafferent-efferent loops that specify the body as subject of perception and action. Second, the body-as-object is not self-specific in the strict sense, that is, it is not exclusive to oneself and no one else (you and I can both be made to experience ownership for the same virtual reality image of the body); and it is not essential (noncontingent) to oneself (changing it does not entail changing one’s embodied perspective or body-as-subject). It follows that future research will need to investigate directly the body-as-subject and not simply the body-as-object in order to advance our understanding of the self-experience of being a bodily subject of perception and action.

References Arzy, S., Thut, G., Mohr, C., Michel, C. M., and Blanke, O. (2006). Neural basis of embodiment: distinct contributions of temporoparietal junction and extrastriate body area. Journal of Neuroscience, 26, 8074–8081. Bermúdez, J. L. (1998). The Paradox of Self-Consciousness. Cambridge, MA: MIT Press.

528    Alisa Mandrigin and Evan Thompson Blanke, O. and Arzy, S. (2005). ‘The out-of-body experience: Disturbed self-processing at the temporo-parietal junction;. The Neuroscientist, 11, 16–24. Blanke, O. and Metzinger, T. (2009). Full-body illusions and minimal phenomenal selfhood’. Trends in Cognitive Sciences, 13(1), 7–13. Blanke, O. and Mohr, C. (2005). ‘Out-of-body experience, heautoscopy, and autoscopic hallucination of neurological origin: Implications for neurocognitive mechanisms of corporeal awareness and self-consciousness’. Brain Research Reviews, 50, 184–199. Blanke, O., Landis, T., Spinelli, L., and Seeck, M. (2004). ‘Out-of-body experience and autoscopy of neurological origin’. Brain, 127, 243–258. Blanke, O., Mohr, C., Michel, C. M., Pascual-Leone, A., Brugger, P., Seeck, M., Landis, T., and Thut, G. (2005). ‘Linking out-of-body experience and self-processing to mental own-body imagery at the temporoparietal junction’. Journal of Neuroscience, 25, 550–557. Blanke, O., Oritgue, S., Landis, T., and Seeck, M. (2002). ‘Stimulating illusory own-body perceptions’. Nature, 419, 269–270. Botvinick, M. and Cohen, J. (1998). ‘Rubber hands "feel" touch that eyes see’. Nature, 391, 756. Christoff, K., Cosmelli, D., Legrand, D., and Thompson, E. (2011). ‘Specifying the self for cognitive neuroscience’. Trends in Cognitive Sciences, 15, 104–112. Ehrsson, H. H. (2007). ‘The experimental induction of out-of-body experiences’. Science, 317, 1048. Ionta, S., Heydrich, L., Lenggenhager, B., Mouthon, M., Fornari, E., Chapuis, D., Gassert, R., and Blanke, O. (2011). ‘Multisensory mechanisms in temporo-parietal cortex support selflocation and first-person perspective’. Neuron, 70, 363–374. Legrand, D. (2006). ‘The bodily self: The sensori-motor roots of pre-reflective self-consciousness’. Phenomenology and the Cognitive Sciences, 5, 89–118. Legrand, D. (2010). ‘Myself with no body? Body, bodily-consciousness, and self-consciousness’. In Daniel Schmicking and Shaun Gallagher (eds), Handbook of Phenomenology and Cognitive Science (pp. 181–200). New York: Springer. Legrand, D. and Ruby, P. (2009). ‘What is self-specific? Theoretical investigation and critical review of neuroimaging results’. Psychological Review, 116, 252–282. Lenggenhager, B., Tadi, T. Metzinger, T., and Blanke, O. (2007). ‘Video ergo sum: Manipulating bodily self-consciousness’. Science, 317, 1096–1099. Longo, M. R., Schüür, F., Kammers, M. P. M., Tsakiris, M., and Haggard, P. (2008). ‘What is embodiment? A psychometric approach’. Cognition, 107, 978–998. McIsaac, H. K. and Eich, E. (2002). ‘Vantage point in episodic memory’. Psychological Science, 9, 146–150. Moseley, G. L., Olthof, N., Wijers, M., Venema, A., Don, S., Gallace, A., and Spence, C. J. (2008). ‘Psychologically induced cooling of a specific body-part caused by the illusory ownership of an artificial counterpart’. Proceedings National Academy of Sciences, 105(35), 13169–13173. Nigro, G. and Neisser, U. (1983). ‘Point of view in personal memories’. Cognitive Psychology, 15, 467–482. Petkova, V. I. and Ehrsson, H. H. (2008). ‘If I were you: Perceptual illusion of body swapping’. PLoS One, 3(12), 1–9. Smith, A. J. T. (2010). ‘Comment: Minimal conditions on the simplest form of self-consciousness’. In T. Fuchs, H. Sattel, and P. Henningsen (eds), The Embodied Self: Dimensions, Coherence, Disorders (pp. 35–41). Stuttgart: Schattauer. Thompson, E. (2005). ‘Sensorimotor subjectivity and the enactive approach to experience’. Phenomenology and the Cognitive Sciences, 4, 407–427.

Own-Body Perception   529 Tsakiris, M. (2010). ‘My body in the brain: A neurocognitive model of body-ownership’. Neuropsychologia, 48(3), 703–712. Tsakiris, M. and Haggard, P. (2005). ‘The rubber hand illusion revisited: Visuotactile integration and self-attribution’. Journal of Experimental Psychology: Human Perception and Performance, 31, 80–91. Von Holst, E. (1954). ‘Relations between the central nervous system and the peripheral organs’. British Journal of Animal Behaviour, 2, 89–94. Wolpert, D. M., Ghahramani, Z., and Jordan, M. I. (1995). ‘An internal model for sensorimotor integration’. Science, 269, 1880–1882.

Chapter 28

Perception of Pa i n Valerie Gray Hardcastle

Pain occupies a curious space in philosophical discourse. On the one hand, it has historically been used as an example of a simple and readily understandable experience when discussing the nature of consciousness. We all, it would seem, grasp what it means to feel an acute pain, something like a toe-stub. On the other hand, it is becoming increasingly clear that the phenomenon of pain is neither simple nor readily understandable. It only takes a moment of consideration to appreciate the difficulties. Think about stubbing your toe. What is it you are feeling when you feel a pain in your toe? A hurting toe? Does that mean we have experiences in our feet? Or is the stubbing in the feet and the experience in the brain? If that is the case, then is feeling pain just a type of perception, like seeing red or hearing the doorbell ring? If so, can we misperceive being in pain, just as we can mistake a bird’s song for the doorbell ringing? Philosophers have struggled in answering the question of what a pain is: is it something like a perception, or is it something else entirely? One place to start in answering this question is by looking at how we think about and describe pains in our everyday lives.

1  Folk views of pain The challenge with our folk view of pain is that is pulls us in two radically different directions. When we say something like, ‘My toe hurts’, we are treating the pain as though it had a particular bodily location. There is a pain, and it is located in my toe. It has a certain spatiotemporal size and intensity. In this case, pain appears to be an attribute of some physical object, namely, our body (or perhaps an entity located in it), and we are standing in a perceptual relationship to that attribute (or entity). That is, we perceive pains in bodily locations, and we report on the objects of those perceptions via a perceptual report. ‘I feel a pain in my toe’, is structurally similar to, ‘I see a crumb on my toe’. In both cases, we are remarking on the objects of our perceptions. When one takes this common-sense approach to understanding pain, then one normally identifies pain with some sort of physical damage to a body part. Presumably, when we are in pain, we are perceiving that something has physically gone wrong with some part

Perception of Pain    531 of our body. That is, we are sensing directly the cause of our pain (normally something like tissue damage of some sort) in the Gricean sense of ‘perceive’, in which what we perceive are the causes of the sensations. And yet, the same common-sense intuitions that push us into thinking that pains are attributes of our body also push us into denying that pains are physical attributes at particular locations. We find it intuitively plausible that we can have tissue damage yet not be in pain (when we take pain-killers, for example.) We also accept that sometimes we have pain without any tissue damage (in cases of phantom limb pain, for example). These intuitions indicate that we do not believe that pains are veridical perceptions of something going on in particular physical locations. Perhaps a better way to understand our folk intuitions is to conceive of pain as the experience itself, instead of the object of experience. That is, we perceive something like the unpleasantness of pain, not the putative cause of the unpleasantness. In fact, this way of understanding pain underlies the International Association for the Study of Pain’s official 1980 definition:  Pain: An unpleasant and emotional experience associated with actual or potential tissue damage or described in terms of such damage. . . Pain is always subjective. . . Many people report pain in the absence of any tissue damage or any likely pathological cause; usually this happens for psychological reasons. There is no way to distinguish their experience from that due to tissue damage if we take the subjective report. If they regard their experience as pain and if they report it in the same ways as pain caused by tissue damage, it should be accepted as pain. Pain . . . is always a psychological state, even though we may well appreciate that pain most often has a proximate physical cause. (IASP, 1986: 250)

Pains are therefore assumed to be private, subjective, and incorrigible, just like other first-person, conscious, experiences. We cannot experience one another’s pains, any more than we have epistemic access to anyone else’s thoughts or desires. Unlike perceptions of our environment, where we (presumably) share the objects that give rise to our private experiences, pains are private through and through. That is, while we both can visually experience your toe, only you can experience its painfulness. Pains are subjective and incorrigible in the sense that their existence seems to be tied to our feeling them. When we sincerely say we feel a pain, then we are in fact in pain. If we do not feel a pain, then that pain does not exist. Unlike objects in the external world, whose existence does not depend on someone sensing them, pains exit only and as someone experiences them. Put another way, there is no appearance/reality distinction with pains (Kripke, 1980). Without this distinction, then we cannot be mistaken in our beliefs about our own pains. The appearance is the reality. Thus, from a folk conception, we end up with a very strange and probably inconsistent view of pain. On the one hand, folk-theoretically, we want to locate pains in the region of disturbed tissue. We can attend to the features of our bodily pains—they are sharp, dull, throbbing, diffuse, radiating. I have an inner perception, as it were, that indexes (roughly) bodily harm in some region. At the same time, this inner perception that is keyed to some event or series of events objectively taking place in our bodies is private, subjective, and incorrigible. The pain ends when we stop hurting. And only we can say when that is. Our common-sense intuitions about pain give us both a public and a private view of our pain experiences.

532   Valerie Gray Hardcastle At this point, philosophers normally take one of two tacks. They either work on refining our common-sense views of pain to eliminate any apparent contradictions or confusions (Aydede 2009a, 2009b), or they dismiss our folk intuitions as unhelpful and turn their attentions to developing a conception of pain from a more scientific point of view (Hardcastle, 1999; Hardcastle and Stewart, 2010). (There is actually a third, less common, tack, and that is developing a conception of pain that stems from other points of view, such as a phenomenological one (e.g. Oliver, 2008). (As a non-reductive approach, I will not be discussing it here, for it falls outside the purview of a review article on pain and perception.)

2  Saving folk approaches The primary ways of cleaning up our folk views of pain are to assimilate them to sense-datum, perceptual, or representational theories of mental phenomena. Each has its difficulties in approach. Sense-datum theories argue that we perceive the world only indirectly, if at all. (See Snowdon, Chapter 6, this volume.)Really, all we ever experience are sense-data, conscious phenomena that indicate some (external) situation which requires our interpretation (Moore, 1903; Russell, 1912). These sense-data are purely internal; they are subjective, private, and incorrigible. They tell us only obliquely, at best, about the world out there (if there is a world out there). Pains, then, are just paradigmatic sense-data; they are just conscious phenomena. They exist when and only when we perceive them. We think that they refer to external injury just as we would think that any sense perception refers to something external. But in both cases, at best we are getting oblique information, perhaps only indirectly referring to something outside our minds. There are two difficulties with the sense-data approach. The first is that it does not really explain anything about pain. All it does is endorse half of our rather odd common-sense view, that pain is private, subjective, and incorrigible, and we think it refers to something outside our mind. Saying that pain is a sense-datum does not advance our common-sense understanding; it merely repeats it using different words. Second, the sense-data approach commits its defenders to, at best, representative realism, or at worst, anti-realism of one stripe or another. If we have no direct access to the external world—if we can only learn about the external world indirectly, through interpreting our sensory experiences, or if our sensory experiences provide us with no verifiable information about the external world—then learning more about the connection between tissue damage and various mental experiences is rendered moot. The game is over before it starts. We know effectively as much as we are ever going to know, which is not very much at all. Perceptual theories of pain defend the view that pain and pain processing are merely a species of our sensory system, and that they work by the same functional principles as all our sensory modalities do. The guiding assumption of any perceptual theory of pain is that perceiving pain means that one is perceiving something that exists outside the mental sphere. That is, perceiving the pain of a stubbed toe is relevantly similar to seeing the redness of a stubbed toe—both are experiences that refer to an event in the toe. Pain perception is a form of somatosensory perception, which also includes our ability to sense touch, temperature, pressure, and bodily location (proprioception). Roughly

Perception of Pain    533 speaking, the perception of pain indexes tissue damage or some sort of similar bodily disruption or irritation. And the pain is located where the tissues have been damaged (Armstrong, 1962, 1964; Pitcher, 1970, 1971; see also Stephens and Graham, 1987; Newton, 1989; and Hill, 2004). Of course, the biggest challenge to this view is that we intuitively believe our experiences of pain to be private, subjective, and incorrigible. But if pains reflect some event in the external world (external to the brain, that is), then we could be mistaken about whether we are in fact perceiving a real pain. We could be hallucinating the pain, or, indeed, just flat wrong about whether any tissues are damaged in our bodies, and even if the pain does emanate from damage, we could be wrong about where this damage is. We could also damage our tissues without us having any experience of the pain, just as we can misperceive all sorts of things about our external world. In other words, there is an appearance/ reality distinction in a perceptual view of pain and pain processing. In this case, it does not appear to match up with our naïve folk intuitions. Part of the way out of this difficulty is to agree with the IASP’s definition of pain: pain is the subjective experience itself, which is only correlated (loosely) with tissue damage. This contrasts with how we describe other sorts of perceptual experiences. In those cases, we normally assume that when we name the percept, we are naming an object, state, or event in the external world. The ‘pain’ of a stubbed toe refers to the conscious phenomenon of a pain experience, while the ‘redness’ of a stubbed toe refers to the colour of the toe itself. The former refers to a subjective event; the latter to an objective one. Similarly, the former descriptor refers to a private and incorrigible experience, while the latter refers to a public event. Thus, even though pain is supposed to be a perceptual process, there is a clear asymmetry between pain language and the language associated with other perceptual processes. But now we have the difficulty of explaining our common-sense view that pains are in fact located in our body, outside of our mind. The pain is in the toe. We say: My toe hurts. We do not say: My pain has no physical location. However, it is also the case that pain experiences, just like other perceptual experiences, represent states of affairs in the world. Our pain experiences represent damage in our tissues. So when we say that we have a pain in our toe, that is just shorthand for saying that we have an experience that represents and points to something going wrong in our toe. On this view, then, the common-sense view of pain is slightly confused. Our expressions for pain refer to our experiences of pain, which are mental events. These mental events, in turn, represent a physical event, which is the real object of our experience. In addition, since these mental events represent, they have conditions for their veridicality. We can be mistaken about whether our tissues are in fact damaged, even if we cannot be mistaken about whether we are having the conscious experience of pain (Holly, 1986; Hyman, 2003; Wyller, 2005; Bain, 2007). Of course, perceptualists have to be very careful that they do not fall back into being proponents of sense-data theories. The mental experience that ‘pain’ refers to cannot be a mental object of any sort, else the perceptualist theory reduces to a sense-data one. Perceptualists generally take pain to refer to mental events of sorts or mental processes. The pain experience is just what we get when we are confronted with the prototypical sorts of tissue damage that normally cause experiences of pain. But if pain experiences are just prototypical mental reactions, then it is difficult to account for the very rich phenomenal experiences we have of pain (Everitt, 1988; Grahek,

534   Valerie Gray Hardcastle 1991). Pain experiences are multi-dimensional affairs, containing at a minimum affective reactions (e.g. ‘pain is awful’), cognitive reactions (e.g. ‘pain is to be avoided’), the pain phenomenology itself (e.g. dull and sharp), and location information (e.g. big toe) (Hardcastle, 1999). It is unclear why, if one were to point simply to tissue damage, all these facets need to be present. Why do we need all these reactions and information if pain indicates tissue damage, full stop? This challenge is especially acute for the early proponents of perceptual theories, because they tried to explain perceptions solely in terms of (uni-dimensional) beliefs. Again, this approach to understanding pain does not seem to help us understand much at all. We still have no way of accounting for the painfulness of pain, as it were. Why is our prototypical reaction like this, rather than something else? For example, in many cases, we can visually inspect tissue damage. We see torn and swollen flesh, perhaps. So, why does pain feel the way it does, as opposed to being something more akin to visual perception, the latter of which appears to indicate (at least some types of) tissue damage? It seems as though a mental index for pain could diverge from an experience of pain, without something better to anchor them together than a discussion of prototypical reactions. In response to this challenge, some have opted for a strong representationalist view of pain, which would allow for rich phenomenological experiences. Representationalists believe that the entire phenomenological experience of pain is identical to its representational content. If we can understand how pain represents tissue damage, then ipso facto we will understand why it is that pain feels the way it does (Harman, 1990; Tye, 1995, 2006a, 2006b; Seager, 2002; Bain, 2003; Dretske, 2006). Often these representations of pain and other perceptual experiences are described as being nonconceptual in nature. That is, they are not built from more fundamental discrete concepts (as one might construct the representation ‘bachelor’ from the concepts of ‘unmarried’ and ‘man’), as we sometimes assume our thoughts to be. The qualitative differences in pain experiences (e.g. throbbing versus stabbing) point to different types of tissue damage or different types of other bodily conditions. The location of the pain, then, is simply where the qualitative experience indicates that the disturbance is occurring. The phenomenology just is the representational content. While a representationalist account might be an improvement over the sense-data and perceptual theories of pain, it still runs into problems in covering all our folk intuitions successfully. It still cannot explain why we naturally report on pain as the experience itself, as opposed to the tissue damage. We say: My toe hurts, not: My toe is damaged, when reporting on our qualitative experience of pain (Aydede, 2009b). If our linguistic practices reflect our conceptual divisions in the world, then we need to explain how it is that the concept of pain appears to refer to the experience, but the concepts associated with other perceptual words (e.g. red) appear to refer to a property of an external object. At this point, philosophers diverge in the tack they take. Some continue to try to fiddle with their theories or our articulation of our folk intuitions to come up with an account that covers both (Conee, 1984; Guiguis, 1998). Others conclude from these exercises that something must be fundamentally wrong with our folk intuitions (e.g. Kaufman, 1985), and they turn to scientific data to help flesh out a philosophical analysis of pain. They give up on the project of trying to analyse our common-sense views of pain and instead adopt the project of analysing the data regarding pain in psychiatry, psychology, and neuroscience.

Perception of Pain    535

3  The science of pain What do we know about pain from a scientific point of view? The answer is at once:  a lot, and not very much. In 1911, Head and Holmes hypothesized that there are two different types of afferent projections in our pain-sensing system: an ‘epicritic’ one that processes information regarding intensity and precise location, and a ‘protopathic’ one that delivers the actual pain sensations. One hundred years later, we still believe that their proposal is essentially correct. Pain specialists typically divide our pain processing system into a ‘sensory discriminative’ or ‘fast pain’ subsystem, originating with the A∂ fibres, that computes the location, intensity, duration, and nature (stabbing, burning, prickling) of stimuli, and an ‘affective-motivational’ or ‘slow pain’ subsystem, beginning with the well-known C-fibres, that supports the unpleasant part of painful sensations. Each subsystem has a set of neurons that resides in the dorsal root ganglia of the spinal column. These neurons extend their axons to whatever tissue they innervate and receive input there; they also have a second axon which projects across to the dorsal horn. Then our basic pain system continues up through the brain. Very roughly speaking, once pain information exists in the dorsal horn, it divides into two streams, one travelling to the lateral nuclei in the thalamus and one to the medial nuclei (Craig et al., 1994). Each type of nuclei underwrites a different sort of information; the lateral nuclei process sensory-discriminative information, while the medial nuclei and associated reticular connections process affective-motivational information. The two thalamic streams remain separate on their trip to cortex as well. Pain neurons in the lateral nuclei synapse in the sensory cortex, which then can compute the location and characteristics of the pain; those in the medial nuclei synapse in the anterior cingulate gyrus in the frontal lobe, which figures in our emotional reactions to pain. The frontal lobe (and its connections) then processes the actual painfulness of pain. Based on these sorts of data, which intimate that pain processing is at least a dual-processing phenomenon, some philosophers have moved to a more mixed view of pain (Nelkin, 1994; Helm, 2002; Gustafson, 2006; Klein, 2007; Hall, 2008). They openly acknowledge that pain is a more complex phenomenon than originally assumed. It is both cognitive and affective, perhaps, or has both representational and non-representational elements. There are many different ways to describe the complexity of our perceptions of pain, but all the different ways have in common the idea that it would be impossible to distil what pain is down to a single component process or a simple event. Pain is actually a combination of different activities in the brain and psyche and any theory needs to reflect that fact.

4  More complexities about pain Current data regarding pain also show that the robust feedback loops in our brain affect the incoming sensory information for pain (so-called nociceptive information). There are additional ascending connections from the spinal cord to the brain stem, circular pathways from the spinal cord to other areas in the spinal cord itself and descending feedback

536   Valerie Gray Hardcastle loops from the cortex, hypothalamus, and brain stem back to the spinal cord. Some portions of these connections have been worked out in considerable detail. Many of the feedback loops in our ascending pain fibres are anatomically distinct from our pain processors and specialize in inhibiting incoming nociceptive information. Three areas are primarily responsible for this activity in the spinal column: the cortex, the thalamus, and the brain stem. The dorsal raphe is probably is heavily involved as well. In addition, neocortex and hypothalamus project to the periaqueductal gray region (PAG), which then sends projections back to the reticular formation. The reticular nuclei then work to inhibit activity in the dorsal horn. This processing stream works by preventing a central cortical representation of pain from ever forming. Endogenous opioids, like PAG activity, dampen incoming information in the dorsal horn. Hence, these pain inhibition streams do not merely disrupt the transmission of pain information, but they actively prevent it from occurring. It looks as though we actually have two separate systems that ultimately generate our experiences of pain, one effectively working to drown out the other. This sort of dual-system for pain actually makes good evolutionary sense. When we are under stress, it is often more adaptive to not feel pain than to be incapacitated by pain. If we are fighting or fleeing from an enemy, it would be preferable to do so unencumbered by the need to nurse or protect our limbs, even it this results in more nursing or protecting later (when we are presumably safe). It is important to know when damage is occurring in our bodies, but it is equally important to be able to shut that information out when circumstances demand. A dual system allows for just such contingencies; we inhibit our pains as needed, but then feel them again when the danger is gone. An ascending pain processing system and a descending pain inhibitory system could then serve two different goals: the pain processing system could keep us informed regarding the status of our bodies. It could monitor our tissues to help maintain their intactness whenever possible. In contrast, the pain inhibitory could shut down the pain processing system when flight or fleeing is immanent, and then enhance the pain processing system response again in moments of calm. These interactions between our pain processing and pain inhibitory systems mean that our experiences of pain are constructed on the fly. What a pain feels like, if it feels like anything at all, depends on how the nociceptive information is embedded in the environment in the brain (which in turn reflects the state of the organism—whether it is preparing for action, under stress, alone in a safe environment, and so forth). For example, some lesions to the thalamus and cortex can result in the cessation of pain experiences, even though the peripheral neurons continue to operate normally. Conversely, stimulating the medial periaqueductal gray region, tectum, or thalamus directly can make us experience pain, even though no tissue irritation has occurred at all (Davis et al., 1995; Keay et al., 1994). There is more. Placebos are notoriously helpful in relieving pain. (Interestingly enough, they relieve pain at half the rate of the putative pain-relieving drugs, regardless of the supposed strength of the drug (Evans, 1974).) Hypnosis allows some subjects to engage in what would otherwise be painful activities without being in pain. Stories of athletes and soldiers continuing to function without pain even though severely injured are legend. Indeed, about 40 per cent of all ER patients reported feeling no pain at the time of injury; 40 per cent more report greater pain that one would expect, leaving only 20 per cent of all ER visitors having pains appropriate to their injuries (Melzack et al., 1982). In fact, it turns out that there is a very poor correlation between nociceptive processing (or tissue damage) and the perception of pain (Wall and McMahon, 1985; Wall, 1989).

Perception of Pain    537 These facts are in part why the IASP concluded that pain is a subjective event. The simple truth is that it is not a good indicator of tissue damage, as many philosophers (and those with common sense) had previously assumed. Indeed, when we supplement these older data with newer information from imagining studies, we can see that the situation is much worse than philosophers ever dreamed.

5  Imaging studies of pain The first thing that becomes immediately clear when looking at imaging studies of pain is that it is a highly distributed process. The structures that are most consistently active during pain processing include the contralateral insula, anterior cingulate cortex, thalamus, premotor cortex, and the cerebellar vermis. This activation varies parametrically with perceived pain intensity (Apkarian, 1995; Casey, 1999). More specifically, we can see a central network for pain processing in imaging studies which runs from the thalamus to the primary and secondary sensory cortex, which probably code sensory-discriminative information and recognition, learning, and memory respectively; the insula, which is connected with autonomic reactions and the emotional aspects of learning and memory; and the anterior cingulate cortex, which is tied to the sensation of unpleasantness and appears to integrate intensity information, emotion, cognition, and response selection (Schnitzler and Ploner, 2000; Buchel et al., 2002; Chen et al., 2002; Ringeler et al., 2003). Then the information flows to prefrontal cortex (Treede et al., 1999). Obviously, the original simple dichotomy into sensory-discriminative and affectivemotivational processing streams was too simple—our pain networks are too interconnected to identify two or even three separate processing streams. In addition, the apparent simultaneous activation of contralateral primary and secondary sensory cortices suggests that thalamocortical nociceptive information is distributed in parallel. This anatomical arrangement contrasts with the serial cortical organization we find in tactile and visual processing in higher primates. Instead, it corresponds to the parallel cortical organization we find in more primative systems (Ploner et al., 1999; Price, 2000). But more than learning about the general anatomy and physiology of our pain processing systems, we can now better appreciate the relationship between pain processing, pain inhibition, and our beliefs, desires, and feelings—between pain and our doxastic surrounds. For example, the degree to which pain is perceived to be controllable affects pain tolerance, learning, motivation, and the ability to cope with intractable pain. In other words, the more we believe we are in control of our pains the less they bother us. It also affects our neural responses. It turns out that the more we believe we control our pain, the more we get attenuated activation in anterior cingulate, the insula, and secondary sensory cortex (Salomons et al., 2004). Cognitive factors mediate pain attenuation using both opioid and non-opioid mechanisms. The greater the sense of self-efficacy one has, the more one’s opioids are activated (Bandura et al., 1987). We can now literally see these effects in the level of activity in our pain processing networks. Attention and previous experience increase activation in the anterior cingulate and orbitofrontal cortex, but modulate pain activity in the thalamus, insula, and primary sensory cortex (Bantick et al., 2002; Bushnell et al., 1999; Dowman, 2001). It appears that

538   Valerie Gray Hardcastle attention works through the frontal cortical areas to damp down activation in other regions of the brain. How one directs attention affects reaction time, accuracy, and perceived pain intensity and unpleasantness (Miron et al., 1989; Longe et al., 2001). Distraction from painful stimuli decreases pain sensations. This explains why football players can continue playing with broken bones and soldiers can continue on despite severe injuries. We can see these attentional effects in anterior cingulate as well (Frankenstein et al., 2001). We also see increased activity in the PAG activity, which tells us that distraction or focusing one’s attention away from pain also activates our pain suppression system (Tracey et al., 2002). Finally, we can get the opposite effect: when subjects expect a stimulus to be painful, they typically experience more pain than if they expect a stimulus to be pleasant or neutral. Indeed, expecting a stimulus to be painful also increases the perceived unpleasantness of innocuous stimuli. Anticipation of pain modulates our cortical systems involved in pain processing even without actual noxious inputs (Porro et al., 2002; see also Carlsson et al., 2000). We can now correlate the expectation of a painful stimulus with enhanced brain responses to nonpainful stimuli in the anterior cingulate, the operculum and the posterior insula (Sawamoto et al., 2000). Similarly, a negative emotional context modulates neural responses and perceived discomfort with non-painful stimulation. We are more likely to read stimuli as painful, when they in fact may not be, when we are unhappy, depressed, sad, and the like (Phillips et al., 2003). At this point, some philosophers throw up their hands in despair and conclude that there is no such thing as pain, at least as traditionally conceived (Hardcastle, 1997, 1999; Dennett, 1978). The notion of pain has fragmented into a very complex process that involves most of the brain. The experience of pain is heavily modulated by everything else going on in the subject’s brain and in the environment as well. Conceiving of pain as a sense-datum, or a simple perception, or mere representation, or even as a dual-component process, is flatly wrong-headed. Furthermore, relying on our intuitions of pain appears only to lead us astray in theorizing about it. Nothing about these data suggest that pain is something located in our tissues or, conversely, something incorrigible. Philosophers who take this stance conclude that we need to eliminate our folk views of pain and replace them with what our best science is telling us about pain and pain processing, no matter where it seems to lead us. We can see the impact of the scientific studies of pain in official changes in the definitions of pain itself as well. The International Association for the Study of Pain definition of pain as being ‘always subjective. . . Pain is always a psychological state’ claims that we should identify pain by what people report and not by any physical condition they have. If someone claims to be in pain, then that person is in pain, regardless of how the nociceptors or the brain is activated. In contrast, in 2004, the National Research Council of Canada, Institute for Biodiagnostics, indicated that its ultimate goal in supporting imaging studies of the pain process ‘is to develop a clinical MR tool that will allow a clinician to assess . . . whether or not a patient is truly experiencing pain (either intermittent acute or chronic)’. Pain has switched from being purely subjective to something purely physical. This agency is suggesting that we should now determine whether people are in pain by what is happening in their brains and not by what they claim they are experiencing. This change in approach represents a complete transformation in how we are to think about pain, and it is a change due largely or solely in part to advances in our scientific

Perception of Pain    539 knowledge of pain. We have gone from a purely mentalistic description of pain—pain is a conscious experience—to one that is purely physical—pain is brain activity. We have also gone from a view that started with naïve folk intuitions about our own pain experiences and ended with how pain information is processed in the brain. What further impacts these sorts of changes will have on the philosophy of pain remains to be seen.

References Apkarian, A. V. (1995). ‘Functional imaging of pain, new insights regarding the role of the cerebral cortex in human pain perception’. Seminars in the Neurosciences, 7, 279–293. Armstrong, D. M. (1962). Bodily Sensations. London: Routledge and Kegan Paul. Armstrong, D. M. (1964). ‘Vesey on Bodily Sensations’. Australasian Journal of Philosophy, 42, 247–248. Aydede, M. (2009a). ‘ "Pain" Stanford Encyclopedia of Pain’. . Accessed on 14 May 2011. Aydede, M. (2009b). ‘Is feeling pain the perception of something?’ The Journal of Philosophy, cvi(10), 531–567. Bain, D. (2003). ‘Intentionalism and pain’. Philosophical Quarterly, 53, 502–522. Bain, D. (2007). ‘The location of pains’. Philosophical Papers, 36, 171–205. Bandura, A., O’Leary, A., Taylor, C.B., Gauthier, J., and Gossard, D. (1987). ‘Perceived Selfefficacy and Pain Control: Opioid and Nonopioid Mechanisms’. Journal of Personal and Social Psychology, 53, 563–571. Bantick, S. J., Wise, R. G., Ploghaus, A., Clare, S., Smith, S.M., and Tracey, I. (2002). ‘Imagining how attention modulates pain in humans using functional MRI’. Brain, 125, 310–319. Buchel, C., Bornhovd, K., Quante, M., Glauche, V., Bromm, B., and Weller, C. (2002). ‘Dissociable Neural responses related to pain intensity, stimulus intensity, and stimulus awareness within the anterior cingulated cortex: A parametric single-trial laser functional Magnetic Resonance Imaging study’. Journal of Neuroscience, 22, 970–976. Bushnell, M. C., Duncan, G. H., Hofbauer, R. K., Ha, B., Chen, J. I., and Carrier, B. (1999). ‘Pain perception: Is there a role for primary sensory cortex?’ Proceedings of the National Academy of Science, USA, 96, 7705–7709. Carlsson, K., Petrovic, P., Skare, S., Petersson, K. M., and Ingvar, M. (2000). ‘Tickling expectations: Neural processing in anticipation of a sensory stimulus’. Journal of Cognitive Neuroscience, 12, 691–703. Casey, K. L. (1999). ‘Forebrain mechanisms of nociception and pain: Analysis through imagining’. Proceedings of the National Academy of Science, USA, 96, 7668–7674. Chen, J. I., Ha, B., Bushnell, M. C., Pike, B., and Duncan, G. H. (2002). ‘Differentiating noxious- and innocuous-related activation of human sensory cortices using temporal analysis of FMRI’. Journal of Neurophysiology, 88, 464–474. Conee, E. (1984). ‘A defense of pain’. Philosophical Studies, 46, 239–248. Craig, A. D., Bushnell, M. C., Zhang, E.-T., and Blomqvist, A. (1994). ‘A thalamic nucleus specific for pain and temperature sensation’. Nature, 372, 770–773. Davis, K. D., Tasker, R. R., Kiss, Z. H. T., Hutchison, W. D., and Dostrovsky, J. O. (1995). ‘Visceral pain evoked by thalamic microstimulation in humans’. Neuroreport, 6, 369–374. Dennett, D. (1978). ‘Why you can’t make a computer that feels pain’. In D. Dennett (ed.), Brainstorms. Cambridge, MA: MIT Press.

540   Valerie Gray Hardcastle Dowman, R. (2001). ‘Attentional set effects on spinal and supraspinal spinal responses to pain’. Psychophyiology, 38, 451–464. Dretske, F. (2006). ‘The epistemology of pain’. In M. Aydede (ed.), Pain: New Essays on Its Nature and the Methodology of Its Study (pp. 59–74). Cambridge, MA: MIT Press. Evans, F. J. (1974). ‘The placebo response in pain reduction’. In J. J. Bonica (ed.), International Symposium on Pain, Advances in Neurology, Volume 4 (pp. 289–296). New York: Raven Press. Everitt, M. (1988). ‘Pain and perception’. Proceedings of the Aristotelian Society, 89, 113–124. Frankenstein, U. N, Richter, W., McIntyre, M. C., and Remy, F. (2001). ‘Distraction modulates anterior cingulate gyrus activations during the cold pressor test’. NeuroImage, 14, 827–836. Grahek, N. (1991). ‘Objective and subjective aspects of pain’. Philosophical Psychology, 4, 249–266. Gufstafson, D. (2006). ‘Categorizing Pain’. In M. Aydede (ed.), Pain: New Essays on Its Nature and the Methodology of Its Study (pp. 219–242). Cambridge, MA: MIT Press. Guirguis, M. M. (1998). ‘Robotoid Arthritis or How Humans Feel Pain’. Philosophical Writings, 7, 3–12. Hall, R. J. (2008). ‘If it itches, scratch!’ Australasian Journal of Philosophy, 86, 525–535. Hardcastle, V. G. (1997). ‘When a pain is not’. The Journal of Philosophy, 94, 381–409. Hardcastle, V. G. (1999). The Myth of Pain. Cambridge, MA: MIT Press. Hardcastle, V. G. and Stewart, C. M. (2010). ‘fMRI: A cerebrascope for the brain? The Case of Pain’. In J. Bickle (ed.), Oxford Companion to Philosophy and Neuroscience (pp. 179–199). New York: Oxford University Press. Harman, G. (1990). ‘The intrinsic quality of experience’. In J. Tomberlin (ed.), Philosophical Perspectives: Action Theory and Philosophy of Mind (pp. 31–52). Atascadero: Ridgeview. Head, H. and Holmes, G. (1911). ‘Sensory disturbances from cerebral lesions’. Brain, 34, 102–254. Helm, B. W. (2002). ‘Felt evaluations: A theory of pleasure and pain’. American Philosophical Quarterly, 39, 13–30. Hill, C. S. (2004). ‘Ouch! An essay with pain’. In J. Gennero (ed.), Higher Order Theories of Consciousness (pp. 339–362). Amsterdam: John Benjamins Press. Holly, W. J. (1986). ‘The spatial coordinates of pain’. The Philosophical Quarterly, 36, 343–356. Hyman, J. (2003). ‘Pain and places’. Philosophy, 73, 5–24. IASP (1986) ‘Pain terms: A list with definitions and notes on pain’. Pain, 3 (Suppl), 216–221. Kaufman, R. (1985). ‘Is the concept of pain incoherent?’ Southern Journal of Philosophy, 23, 279–284. Keay, K. A., Clement, C. I., Owler, B., Depaulis, A., and Bandelr, R. (1994). ‘Convergence of deep somatic and visceral nociceptive-information onto a discrete ventrolateral midbrain periqueductal gray region’. Neuroscience, 61, 727–732. Klein, C. (2007). ‘An imperative theory of pain’. Journal of Philosophy, 104, 517–532. Kripke, S. (1980). Naming and Necessity. Cambridge, MA: Harvard University Press. Longe, S. E., Wise, R., Bantik, S., Lloyd, D., Johansen-Berg, H., McGlone, F., and Tracey, I. (2001). ‘Counter-stimulatory effects on pain perception and processing are significantly altered by attention: An fMRI study’. Neuroreport, 12, 2021–2025. Melzack, R., Wall, P. D., and Ty, T. C. (1982). ‘Acute pain in an emergency clinic, latency of onset and descriptor patterns related to different injuries’. Pain, 14, 33–43. Miron, D., Duncan, G. H., and Bushnell, M. S. (1989). ‘Effects of attention on the intensity and unpleasantness of thermal pain’. Pain, 39, 345–352. Moore, G. E. (1903). ‘The refutation of idealism’. Mind, 12, 433–453. Nelkin, N. (1994). ‘Reconsidering Pain’. Philosophical Psychology, 7, 325–343. Newton, N. (1989). ‘On viewing pain as a secondary quality’. Nous, 23, 569–598.

Perception of Pain    541 Oliver, A. (2008). ‘The problem of defining pain’. Philosophy Today, 52, 1, 3–11. Phillips, M. L., Gregory, L. J., Cullen, S., Coen, S., Ng, V., Andrew, C., Giampietro, V., Bullmore, E., Zelaya, F., Amaro, E., Thompson, D. G., Hobson, A. R., Williwams, S. C., Brammer, M., and Aziz, Q. (2003). ‘The effect of negative emotional context on neural and behavioral responses to oesophageal stimulation’. Brain, 126, 669–684. Pitcher, G. (1970). ‘Pain perception’. The Philosophical Review, 79, 368–393. Pitcher, G. (1971). A Theory of Perception. Princeton, NJ: Princeton University Press. Ploner, M., Schmitz, F., Freund, H. J., and Schnitzler, A. (1999). ‘Parallel activation of primary and secondary sensory cortices in human pain processing’. Journal of Neurophysiology, 81, 3100–3104. Porro, C. A., Baraldi, P., Pagnoni, G., Serafini, M., facchin, P., Maieron, M., and Nichelli, P. (2002). ‘Does anticipation of pain affect cortical nociceptive systems?’ Journal of Neuroscience, 22, 3206–3214. Price, D. D. (2000). ‘Psychological and neural mechanisms of the affective dimension of pain’. Science, 288, 1769–1772. Ringler, R., Greiner, M., Kohlloeffel, L., Hnadwerker, H. O., and Forster, C. (2003). ‘BOLD Effects in Different Areas of the Cerebral Cortex during Painful Mechanical Stimulation’. Pain, 105, 445–553. Russel, B. (1912). The Problems of Philosophy. England: Home University Library. Salomons, T. V., Johnstone, T., Backonja, N. M., and Davidson, R. J. (2004). ‘Perceived controllability modulates the neural response to pain’. Journal of Neuroscience, 32, 7199–7203. Sawamoto, N., Honda, M., Okada, T., Hanakawa, T., Kanda, M., Fukuyama, H., and Konishi, J. (2000). ‘Expectation of pain enhances responses to nonpainful sensory stimulation in the anterior cingulated cortex and parietal operculum/posterior insula: An event-related functional magnetic resonance imagine study’. Journal of Neuroscience, 20, 7438–7445. Schnitzler, A. and Ploner, M. (2000). ‘Neurophysiology and functional neuroanatomy of pain perception’. Journal of Clinical Neurophysiology, 17, 592–603. Seager, W. (2002). ‘Emotional introspection’. Consciousness and Cognition, 11, 666–687. Stephens, G. L. and Graham, G. (1987). ‘Minding your P’s and Q’s: Pain and sensible qualities’. Nous, 21, 395–405. Tracey, I., Ploghaus, A., Gati,J.S., Clare, S., Smith, S., Menon, R. S., and Matthews, P. M. (2002). ‘Imaging attentional modulation of pain in the periaqueductal gray in humans’. Journal of Neuroscience, 22, 2748–2752. Treede, R. D., Kenshalo, D. R., Gracely, R. H., and Jones, A. K. (1999). ‘The cortical representation of pain’. Pain, 79, 105–111. Tye, M. (1995). Ten Problems of Consciousness: A Representational Theory of the Phenomenal Mind. Cambridge, MA: The MIT Press. Tye, M. (2006a). ‘Another look at representationalism about pain’. In M. Aydede (ed.), Pain: New Essays on Its Nature and the Methodology of Its Study (pp. 99–120). Cambridge, MA: MIT Press. Tye, M. (2006b). ‘In defense of representationalism: Reply to commentaries’. In M. Aydede (ed.), Pain: New Essays on Its Nature and the Methodology of Its Study (pp. 163–176). Cambridge, MA: MIT Press. Wall, P. D. (1989). ‘The dorsal horn’. In P. D. Wall and R. Melzack (eds), Textbook of Pain (pp. 102–111). New York: Churchill Livingstone. Wall, P. D. and McMahon, S. B. (1985). ‘Microneuronography and its relation to perceived sensation’. Pain, 21, 209–229. Wyller, T. (2005). ‘The place of pain in life’. Philosophy, 8, 385–393.

Chapter 29

Percei v i ng Nothi ngs Roy Sorensen

To perceive is to perceive something. Unless we are merely hallucinating, the object of perception must be an appropriate cause of the perceptual experience. So we cannot perceive nothing. Thus the perception of absence reduces to the absence of perception. Yet we see shadows (absences of light), hear silence (absence of sound), and feel holes (absences of matter). These perceptual reports are common in ordinary life and in science. The scientific significance of shadows has been chronicled by Roberto Casati’s The Shadow Club (2003). This chapter is organized around four twentieth-century resolutions of the inconsistency. These solutions come in dialectical sequence. They emerge, under French impetus, as a rejection of Georg Hegel’s metaphysical egalitarianism. Like the ancient Greek atomist Democritus, Hegel regards Being and Non-Being as equally real. The French took it as axiomatic that reality is fundamentally positive. Perceptions of absences had to be explained away. As prelude, I  shall first emphasize the range of perceived absences and the range of senses by which they are perceived. I shall begin with the sense most redolent of common sense—touch.

1  Touching the void Samuel Johnson tried to refute George Berkeley’s idealism by kicking a stone. A  key premiss of his enthymeme is ‘If the stone is a collection of ideas, then my foot should pass through it’. Johnson reasons that Berkeley’s idealism would make everything intangible. Since there is something tangible, there are some material objects. Are all tangible things material? When doubting Thomas cannot believe that Jesus has returned from the dead, Jesus takes Thomas’ finger and inserts it into the hole made by a soldier’s spear (Figure 29.1). The urge to touch percolates up into science. When Robert Hooke built vacuum chambers for the Royal Society his audience was initially content to see the vacuum and its effects on candles and hear its effect on ringing bells. But the logic of trusting only what you

Perceiving Nothings   543

Fig.  29.1  The Incredulity of Saint Thomas (Caravaggio).

witness firsthand eventually prompted Hooke to build a chamber large enough to accommodate an arm. Ever the empiricist, Hooke wished touch the void. Later, he built a vacuum chamber large enough to crawl into. (The apparatus malfunctioned half way through; Hooke escaped the fate of the mice and chicks that had entered smaller chambers.) Can empty space be felt? On the one hand, Michael Martin (1998: 271) denies that you have bodily awareness of empty space when simply holding out separated hands. As long as your eyes are closed, you are only aware of the relative positions of your hands. There could be a stack of books between them. Martin infers that only the visual field permits the sensing of empty space. On the other hand, if you actively run your fingers over the books on a bookshelf you can feel gaps in the sequence. Even in total darkness, there can be much tactile exploration of empty spaces. When you comb your hand in the dark, you feel the spaces between the teeth of the comb. But maybe Samuel Johnson can be content with the weaker point that we can only feel holes by feeling the material objects that host them (especially the lining of the hole). Remember, Johnson only wished to refute idealism, not to prove materialism.

2  Seeing independent absences Arguably, the eyes have a special advantage in seeing empty space that is not dependent on anything else. An astronomer can look up and be astounded to see nothing. He excitedly phones a colleague: ‘All the stars have vanished! It is just empty space up there!’. There is nothing to see. Yet the astronomer sees ‘it’. The seeing is transitive. The astronomer is not like a blind patient who is enjoying psychedelic sensations from electric brain stimulation. The astronomer is gaining perceptual knowledge of the external world. The astronomer is scrutinizing the scene with his eyes, operating in a normal way. Most absences are experienced in a figure–ground relation. The astronomer seeing empty space is a far-out exception. A more familiar exception to the figure–ground requirement is seeing the total absence of light. When the guide turns off the light, the

544   Roy Sorensen tourists marvel at how dark the cave looks. So they must be seeing without light. Indeed, Ludwig Wittgenstein claims that in one respect, we see best in the dark: ‘The limitlessness of the visual field is clearest when we are seeing nothing in complete darkness’ (Wittgenstein, 1967: 616). Wittgenstein repeats this paradoxical observation in his Philosophical Remarks, ‘The limitlessness of visual space stands out most clearly, when we see nothing, in pitch-darkness’ (Wittgenstein, 1975: 281). An architect might side with Wittgenstein when checking a large dome that is supposed to be light-tight. The architect turns out all the lights and looks for cracks of light. Seeing total darkness, the architect is satisfied. Wittgenstein might have been deliberately contradicting Ewald Hering’s who characterized total darkness as roomy but vaguely bounded. He might also be contradicting those who experience total darkness as a ‘fogged-in’ experience with very limited visibility. These divergent phenomenologies may betray the cognitive penetrability of vision. If you believe you are in a large space, you credit yourself with a range of vision commensurate with your sensitivity to light. The visual psychologist Leo Hurvich characterizes the sensation of blackness as the appropriate visual response to the absence of light (1981: 26). But the dominant tendency of common sense is to deny that we see in the dark. There may be metaphysical support for denial. If you see blackness in the dark, then you are experiencing a secondary quality that lacks a primary quality (any physical property that could be commingled with a perceiver’s psychology to yield a colour sensation). The vast majority of colour scientists distinguish sharply between darkness and blackness (which they restrict to surfaces that indiscriminately absorb light).

3  Higher level absences When the Mona Lisa was stolen in 1911, its absence passed unnoticed by officials at the Louvre for a day. Eventually, an artist who had made a special trip to copy the Mona Lisa complained to a guard. The police closed the Louvre museum for a week. When re-reopened, a long queue formed to view the absence of the Mona Lisa. More Frenchmen viewed the Mona Lisa’s absence than its presence (Figure 29.2). Le Figaro described the spectacle as ‘an enormous, horrific, gaping void’. According to the reporter, ‘the crowds didn’t look at the other paintings. They contemplated at length the dusty space where the divine Mona Lisa had smiled the week before’. The ‘wall of shame’ was kept vacant for weeks to accommodate demand. This permitted participation by foreign tourists, such as Franz Kafka and his friend Max Brod. The pair continued their pilgrimage the following evening by attending the cinema to see a parody: ‘Nick Winter and the Theft of the Mona Lisa’. Postcards were printed so that the pious could pray to St Anthony, the patron saint for things gone missing (Figure 29.3). Catholics pray to St Anthony to recover keys, wallets, and dogs. But they cannot pray to recover their lost faith. If you believe that God does not exist, then there is no one to pray to. Thus ‘Atheists worship nothing’ is properly read as characterizing these non-believers

Fig.  29.2  Mona Lisa’s Absence.

Fig.  29.3 St Anthony Postcard. When the postcard is held to candlelight, the heat makes the Mona Lisa  appear.

546   Roy Sorensen as non-worshippers rather than as worshippers of nothing. We might try the same response to those who claim that the curious Parisians came to perceive nothing. Under this hypothesis the Parisians came to the Louvre to not perceive the Mona Lisa.

4  Objectless seeing? Were the curious Parisians trying to enter a perceptual relationship that does not require anything to look at? Although ‘see’ is generally read as a transitive verb, there are certain kinds of visual experiences that do not require anything external to be seen. Leonardo Da Vinci asks, ‘Why does the eye see a thing more clearly in dreams than the imagination awake?’ The answer appears to be that some visual experiences proceed directly. The dreamer does not need to cope with distance, fog, or myopic eyes. However, the curious Parisians were interested in seeing something public. Dream seeing is private. Heraclitus writes, ‘When men dream, each has his own world. When they are awake, they have a common world’. The absence of the Mona Lisa was in the common world, more precisely, in the Salon Carré.

5 External things Was the absence of the Mona Lisa outside of the curious Parisians? Immanuel Kant distinguishes between things that are merely presented in space and things that can be met in space (Kant, 1965: A373). Fugitive objects such as rainbows, the sky, and the horizon might illustrate the distinction. They seem to retreat when approached. When G. E. Moore was setting the stage for his proof of the external world, he claims negative after-images illustrate Kant’s distinction. In addition to being presented in space, after-images involve objective visual processes such as recognition. For instance, after people stare at the dot in the inverse image in Figure 29.4 and redirect their gaze to a blank page, they experience a negative after-image. Two famous phenomenologists, Edmund Husserl and Maurice Merleau-Ponty, may have reservations about Moore’s phenomenological claim. If something is presented as being in space, then it should be presented as being independent of the viewer. But after-images follow movements of the eye. Merleau-Ponty athletically contends that mind-independent objects should be open to exploration and handling. Seeing is doing. His attempt to overcome the mind–body dichotomy has inspired contemporary work on ‘embodied cognition’ and ‘enactivism’. Husserl, somewhat more cerebrally, holds that when we conceive of an object we conceive of how its appearance would alter systematically with changes in perspective. No one seeks a side-view or a rear-view of an after-image. According to Husserl’s line of reasoning, after-images are frozen in a private space (along with aches and twinges). In Moore’s defence, consider floaters. Romans debated whether the black specks that drifted through their visual fields were hallucinations or the silhouettes of debris suspended in their eyeballs. The Romans did not use perspectival dependence to eliminate the

Perceiving Nothings   547

Fig.  29.4  Negative Inverse. second alternative. Their caution was vindicated by eye doctors who eventually developed the means of seeing the floaters of their patients. If a dark curtain appears to descend over your visual field, then you should not infer that this is an event in private space. You should instead infer that your retina has detached and is slowly folding on itself (and hurry to a hospital). If you are in equatorial Africa and appear to see a worm swimming through your eyeball, then this may be the Loa Loa eyeworm. All of these ‘entopic phenomena’ are perspective-dependent. You cannot manoeuvre around to see other sides of the physical objects in your eye (though they may rotate and you might manipulate the Loa Loa eyeworm after a surgeon fishes it out of your eyeball). After-images are caused by chemical changes in the retina. The perspectival dependence follows immediately from this general fact. The reason why the after-image is not a physical object is that its cause is not visually appropriate. It is like an electric charge that gives rise to visual effects. Moore’s phenomenology is also vindicated by a genre of magic trick that relies on substituting an after-image for a physical object such as a shiny coin or a red dress (Macknik et. al., 2010: 16, 24, 57, 108). If the after-image were not being presented as being in a public space, then it could not be mistaken as a physical object.

548   Roy Sorensen The absence of the Mona Lisa seems more like a shadow than a negative after-image. As Moore notes, shadows can be met in space. Shadows have precise shapes and dimensions. They robustly support an appearance–reality distinction. From the summit, the shadow of the mountain looks triangular. But this is a foreshortening illusion. The Parisians knew where to go to see the absence of the Mona Lisa. They were not on a vision quest in which hallucination is cultivated. Nor were they attempting to perceive spiritual beings as when they attended séances. The Parisians went to the Louvre sober and fit. They were engaged in the observation of a well-lit, public phenomenon. The Parisians regarded the absence of the Mona Lisa as something that could be photographed. Witness Figure 29.2.

6  Romantic defeatism? People who go blind continue to dream visually. While awake, many visualize the scene before them. They differ from congenitally blind people who miss a developmental opportunity for the visual system to develop. In this sense, completely blind people vary in their ability to see. Although they all fail to see, some meet more of the requirements than others. Those who visited the Louvre came closer to seeing the Mona Lisa than those who stayed home. Perhaps they merely wished to approximate seeing the Mona Lisa—without actually seeing. In Samuel Beckett’s phrase, they wished to ‘fail better’. Under this interpretation, the French were involved in a symbolic act. The crowd was protesting the fact that the Mona Lisa could no longer be viewed at the Louvre. The curious Parisians were merely going through the motions. They were not really trying to see. But many of the curious Parisians regarded their trip to the Louvre as a success rather than a noble failure. They jostled to get a good look. Short Parisians at the far end of the queue complained that they could not see anything. All had waited in line for the opportunity to view the absence of the Mona Lisa. When their turn came, they adopted the standard viewing position for a painting. In sum, the curious Parisians put us in a dilemma. We cannot find a perceptual object for their visit to Louvre. Yet we cannot charitably interpret them as seeking to not see the Mona Lisa.

7  Bergson’s metacognitivism Just three years prior to the theft of the Mona Lisa, the most famous French philosopher of the era, Henri Bergson, claimed to have refuted the possibility of perceiving absences. In the final chapter of Creative Evolution Bergson denies that the content of the perception can be negative: When I say, “This table is black,” I am speaking of the table; I have seen it black, and my judgment expresses what I have seen. But if I say, “This table is not white,” I surely do not express something I have perceived, for I have seen black, and not an absence of white. It is therefore,

Perceiving Nothings   549 at bottom, not on the table itself that I bring this judgment to bear, bur rather on the judgment that would declare the table white. I judge a judgment and not the table. (Bergson, 1911: 303–304)

Logically, negative ‘perceptual’ reports are a form of metacognition that gets confused as object level perception. Negation is never ‘a complete act of the mind’ (Bergson, 1911: 303). Instead of recording how things actually are, negative judgements detour into the realm of possibility. Idealist logicians had already refined Immanuel Kant’s doctrine that negative judgements are higher order judgements: ‘the task peculiar to negative judgments is to reject error’ (Kant, 1965: A709/B737). Bergson innovates by combining metacognitivism with non-cognitivism. Instead of focusing on the truth-value of negative perceptual reports, Bergson emphasizes their role in expressing and evincing emotions. According to Bergson, an admonitory speech act is performed with ‘The table is not white’. To deny is to do something—to go on the record as warning as against a potential error. Denials also express feelings. Bergson focuses on regret (1911: 311). A bride laments ‘This table is not white’ because she ordered a white table. This suggests that the perceivability of absences varies with the same factors that increase regret. Regret is most intense for unusual choices that could have easily been decided the other way. Regret correlates with suffering, intimations of mortality, leisure for reflection, imminence and irreversibility of outcome. Consider the careless explorer of a desert island who returns to shore to find that his boat has floated away with the tide. The absence of the boat pops out of the landscape. It is not just gone. It is very gone! Richard Gale (1976: 59) suspects that Bergson has confused negation with denial. Denial of p is a speech act that is logically independent of the truth-value of p. Only denial entails the existence of anyone. And the logical laws that are plausible for negation (double negation, excluded middle) are implausible for denial. Since Bergson is writing before J. L. Austin’s distinction between locutionary and illocutionary acts, the charge is not uncharitable. However, we can salvage more of what Bergson says by interpreting him as a proto-expressivist about absences. Although Bergson pre-dates C. L. Stevenson’s sharp distinction between subjectivism and emotivism, Bergson’s most intriguing remarks foreshadow an emotivist account of perceiving absences. Grammar is an incomplete guide when separating genuine perceptual reports from the pseudo-perceptual report of negative things. For sometimes the absence is hidden. Bergson’s own example of a positive judgement, ‘The table is black’, is ill-chosen because blackness is the colour characterized by an absence of hue and an absence of reflectance. Black is the colour of absence. Hooke’s claim to have produced a vacuum was challenged on the grounds that a genuine void would be black rather than transparent. (Wave theories of light required a medium.) Bergson stresses that negations are pedagogical. Whereas solitary creatures that live in the present can make positive judgements, negative judgements are for social creatures that remember the past and have expectations about the future. Negative ‘perceptual’ reports rely on memory and inference or social skills. They are not based solely on the sense organs. A total amnesiac could not make negative judgements (Bergson, 1911: 297). He would lack the premisses about the past needed to base an expectation.

550   Roy Sorensen Negative ‘perceptual’ reports also make essential use of our language faculty (Bergson, 1911: 308). For the function of negative judgement is communication. A total aphasic would not be capable of negative judgements. Genuine perception relies solely on the senses. That is why loss of memory, reasoning, language, and social skills are not visual handicaps. Thus our apparent perception of negative things cannot be genuine perceptions. The curious Parisians saw only a positive reality such as the pegs on the dusty wall. When they said ‘I see the absence of the Mona Lisa!’, they were expressing regret. A contemporary expressivist might allow that at least some of the Parisians were simply lamenting the Mona Lisa’s absence and were not aiming at any future action. But Bergson resembles his contemporary American pragmatists (especially William James with whom he corresponded). He looks forward to the practical upshot of what we say. Lamenting the Mona Lisa’s loss did pressure the police. In addition to the usual suspects, the detectives cast an inquisitive eye on the modern art movement. The modernists were supported by Bergson’s philosophy of creativity. Classical art, epitomized by the Mona Lisa, was static. Real life is dynamic, perspectival, and non-mechanical. Detectives hypothesized that the theft of the Louvre’s crown jewel was a vengeful prank intended to humiliate the curators who marginalized modern art. Bergson agrees with the American pragmatists in emphasizing action’s dominance over thought. However, Bergson does not associate truth with usefulness. Utility is instead a source of illusion. According to Bergson, we think to act. Action is designed to fill the gap between desire and reality. Our attention narrows to this gap. We neglect the positive background needed to host the gap. This void becomes a blank canvas upon which we must embroider a positive reality. This habit of concentrating on gaps makes nothingness seem like the default state. Positive reality becomes the thing that needs explanation. Bergson’s attack on the perception of nothings is intended to undermine any empirical ground for believing in negative things. But he also has metaphysical grounds for denying their very possibility. He is guided by the fundamental intuition, shared by all of humanity, that reality is positive. Any negative truth must be an indirect positive truth. For instance, seeing holes in a page must be interpreted as seeing a perforated page—where this is understood as a particular shape. Ancient atomists, such as Democritus, explicitly regard Being (atoms) and Non-Being (void) as equally real (to mark disagreement with Parmenides). Bergson sees the same misplaced egalitarianism in Georg Hegel’s writings. The expressivist explanation of the curious Parisians is that they are involved in a ceremony expressing loss. They go to the Louvre because they are entitled to see the Mona Lisa. This right is violated when they see a dusty wall instead of Da Vinci’s masterpiece. The Parisians say ‘I see nothing’ to vent disappointment just as they stamp their feet to express anger. There is a parallel between Bergson’s scepticism about the perception of absences and David Hume’s scepticism about the perception of causation. According to Hume, all we see are the succession of events. Causation is something inferred. Albert Michotte’s (1962) studied conditions under which one ball will appear to chase, lead, or force another ball to move. Subsequent psychologists made movies that elicit causal attributions. But these would be self-defeating as demonstrations that the attributions are

Perceiving Nothings   551 accurate. All the viewers see are frames of a movie, none of which bears a causal connection to a successor. All the psychologists have furnished is further evidence that causal attribution can be triggered without any underlying causation. Instead of justifying confidence in a future event, the attribution of causation expresses confidence—and attempts to elicit that confidence in others. In addition, it sets the groundwork for praise, blame, and remediation. Since we have different roles, search for ‘the cause’ will vary. The police, the weatherman, and the road engineer nominate different causes for the automobile accident. As evident from discussion of omissions, these factors will also influence negative judgements. David Hume, like Bergson, takes the further step of denying that there is any underlying causation. Correlations breed habitual response—in this case the prediction that the same type of event will occur as did occur. The perceiver projects this force of habit on to the external phenomena. Similarly, Bergson could argue that we project our negative feelings of frustration on to the external world. The exasperated golfer kicks the hole that has spoiled his game. According to Bergson, consciousness is designed to see the cup as half-empty rather than half-full. For consciousness serves desire and desire can only be served by fixing what is wrong, not savouring what is right.

8  Sartre’s subjectivism In 1940 Jean Paul Sartre believed Bergson’s affirmativism was supported by Edmund Husserl’s principle of the intentionality of consciousness: There cannot be an intuition of nothingness, precisely because nothingness is nothing and because all consciousness—intuitive or not—is consciousness of something. Nothingness can be given only as an infrastructure of something. The experience of nothingness is not, strictly speaking, an indirect experience, but is an experience that is, on principle, given ‘with’ and ‘in’. Bergson’s analyses remain valid here: an attempt to conceive death or the nothingness of existence directly is by nature doomed to fail. (Sartre, 2004: 187)

Sartre had second thoughts. The intentionality of consciousness involves neutrality about whether the object of thought exists. A physicist can think about monopoles while being completely agnostic about whether monopoles exist. Perception is more partisan than the ontologically neutral attitude of supposition. One can see only what exists. So even if Husserl shows that we can think about what does not exist, this would not be enough to show that we can perceive what does not exist. Thus Bergson could rest his rejection of negative perceptual reports on the principle that positive reality exhausts what is seen. Sartre always remains loyal to the principle that positive reality is the substrate for everything. One motive is his activist ethics. Folk morality and law agree that there is a morally relevant difference between acts and omissions. Letting someone die is less grave than killing him even though the consequences are the same. Sartre pre-empts this moral asymmetry by challenging the metaphysical presupposition that there are omissions. His

552   Roy Sorensen slogan is ‘To not act is to act’. An omission always involves some substitute action. Thus Sartre differs from a utilitarian who admits that omissions exist but affirms their moral equivalence to actions. (Existentialists have meagre resources for any direct answer to a moral question but they can challenge presuppositions of those questions and elaborate adequacy conditions for a direct answer.) Metaphysically, Sartre principally differs from Bergson in scale. Sartre believes that virtually everything people think about are institutional facts. Nearly everything is made-up—and it is the making up that Sartre admires. As a physical object, Sartre does not value the Mona Lisa because it is no longer provocative. In a 1972 interview with Esquire magazine, Sartre avers that he would not give a second thought to letting the Mona Lisa burn during a cultural revolution: But when I think of the Mona Lisa! There are some things that really are of no use at all, none at all! For a long time the Mona Lisa’s smile has only served as a cliché for bad writers! That smile used to be something, now it is nothing, it is hollow. The best thing that I’ve seen on the Mona Lisa was a photomontage: the only original use of the Mona Lisa since I was born! For me, it is absolutely typical of paintings that no longer have anything to say, while other paintings by da Vinci or Tintoretto can still mean something. (Esquire, 1972: 284, 286)

To see the Mona Lisa’s smile, one must organize the mosaic of colour patches into a coherent whole. The bits from the frame must be excluded. The human figure must be separated from the landscape. This negative difference making is a prelude to positive construction. Perceptual holism explains why there is an optimal viewing distance. It also explains the relevance of orientation. Distortions of the Mona Lisa pass unnoticed when the picture is presented upside down (Figure 29.5). In a Gestalt switch, the stimulus is constant but the perception varies. In addition to illustrating variation between observers, Gestalt switches reveal variation over time with the same observer. There are two intentional objects but just one stimulus. Gestalt switches substantiate the phenomenological difference between seeing a scene in a purely positive fashion and seeing it as having absences. In Being and Nothingness Sartre describes his tardy arrival for an appointment with Pierre. Sartre sees the absence of Pierre in a café. Nothingness pops out of Being like a parking space on a congested street in Montmartre. If Sartre’s companion Simone does not expect Pierre, she will be presented with exactly the same café scene. Yet she will see it differently. Why does Sartre not also see the Duke of Wellington? Because Sartre was not anticipating a rendezvous with the Duke. This psychological explanation gives hope of answering the proliferation challenge: If there is one absence, why not millions? The answer is that absences are fabricated by minds with limited attention. If Simone had expected the Duke of Wellington, then she would have seen the absence of the Duke. The relativist about absences would say that the Duke is absent for Simone but not absent for Sartre. For the relativist, all absences are absences for someone. Sartre is no more a relativist about absences than he is a relativist about pains. Headaches depend on sufferers but are not relative to sufferers. When Sartre visits an unfamiliar classroom, he knows that the teacher knows who is absent even though Sartre does not know who is absent. The students are not merely absent for the teacher.

Perceiving Nothings   553

Fig.  29.5  Mona Lisa upside-down distortion. Sartre writes like a dualist, strongly contrasting the inertness of objects (‘being in itself’) with consciousness (‘being for itself’). Sartre even sounds like an idealist when he says all potentiality and all destruction depends on consciousness. Yet Sartre’s explicit intent (as stated in his Critique of Dialectical Reason) is to be a materialistic monist. As a committed Marxist, Sartre regards himself as no more a dualist than the ancient atomists. Democritus says that Being (atoms) and non-Being (void) are equally real. But just as nothingness plays non-material roles in atomism (explaining unnatural motions such as water rising above air through a siphon), nothing plays non-material roles in Sartre’s metaphysics (unshackling Freudian causal chains that threaten freedom, distinguishing people from mere objects, and so on).

9  Immaterialism and mediated perception The ancient atomists thought of matter and space as mutually exclusive; space is space empty of matter. The holes in a sponge are not parts of the sponge. The holes are places where there is an absence of sponge. Roberto Casati and Achille Varzi allow holes to be part of the sponge and further allow that some things are comprised entirely of empty space. Just as a statue is made of clay, holes are made of space. Space is the matter of holes (Casati and Varzi, 1994: 32). Although constituted by pure space, a hole is not identical to any particular region of space. Holes can move. (When you take your doughnut out of your lunch bag, the hole goes where the doughnut goes.) Material objects may also be made of space but not space only.

554   Roy Sorensen A materialist about holes, who identifies holes with hole-linings, can easily explain how holes are perceived. But Casati and Varzi are immaterialists who concede that holes cannot transfer energy to a perceiver. ‘To perceive an immaterial entity would be to perceive it mediately, through perception of some material entity on which it depends (e.g. the host) or to which it is spatially linked in some relevant way (e.g. the filler)’ (Casati and Varzi, 1994: 157). Just as we can see objects by merely seeing attached parts of them, we can see immaterial things by seeing objects that are associated with them. Holes are like surfaces, corners, and edges (when these are construed geometrically rather than physically). To see the hole of a doughnut is to see an aspect of the doughnut or an aspect of the hole’s filler. To make holes visible, Casati and Varzi attempt to dilute the causal requirement to a counterfactual requirement: ‘if a certain fact in the world outside the perceiver were not the case, then a certain mental event would not be a perception’ (1994: 158). But in one crucial respect, this ‘weakening’ inadvertently strengthens the requirement beyond what the causal theory of perception demands. Suppose that a hologram of a doughnut will be activated if Homer fails to have the visual experience of a doughnut. Happily, there is a doughnut right in front of Homer and it is stimulating his eyes in a normal way. Intuitively, Homer sees the doughnut (and thereby, its hole). But Homer would have had the same experience if the doughnut had not been there. Causal pre-emption scenarios work as well for absences as they do for presences. Thus one cannot avoid the causal challenge by substituting a counterfactual requirement. Those committed to the perception of absences appear committed to causation by absences. Jonathan Schaffer denies this is a cost because he thinks the energy-flow model of causation has been decisively refuted and regards causation by absences as pervasive (Schaffer, 2000). The main burden of the mediation theory is to explain why we perceive the hole of a doughnut but fail to perceive other immaterial entities such as the doughnut’s centre of gravity or the equator. Casati and Varzi try to shoulder this burden with psychological generalizations about the conditions under we attribute the perception of holes: Something is perceived as a hole only if differentiated from its surroundings. Since black recedes and white emerges, black patches in uniform white fields are perceived as holes. Holes must be perceived as having a border. Holes are perceived when shading favours a concave interpretation (Casati and Varzi, 1994: 163). Since the visual system assumes light is coming from above, turning a picture upside-down can be relevant to whether it is perceived as a picture of a hole. An irregularity is perceived as a hole: a missing book from a stuffed bookshelf, a clearing in a forest, an unwritten section of a censored newspaper, etc. These generalizations are interesting enough to have been taken up by professional psychologists of perception. But they do not provide justification of the attribution or anything else that would show we have perceptual knowledge of holes.

10  The observer relativity of absences According to Pierre Duhem, theories entail predictions only when conjoined with auxiliary assumptions. This explains why the blame for a failed prediction can be shifted from a theory to background assumptions. If absences are refuted predictions, then we should

Perceiving Nothings   555 expect them to depend on both theory and background assumptions. People with different theories and background beliefs should vary in which absences they perceive. Art historians focus on the Mona Lisa’s complete absence of jewellery. She has no rings, no bracelets, no necklace. The historians know that the norm for the era was to present subjects with expensive decoration. Similarly, the convention in Caravaggio’s era is to endow the risen Christ with a halo. The absence of a halo underscores Christ’s corporality. Although the Mona Lisa also lacks any representations of eyelashes, the art historians deny that Da Vinci depicted her without eyelashes. Painting someone without including eyelashes is like drawing a stick man without adding shoes. The omission does not constitute a representation of shoelessness. This contrasts with hats; if you draw a stick man without a hat, he lacks a hat. Eyebrows are indeterminate for stick men but obligatory for formal portraits. Since there are no visible strokes for Mona Lisa eyebrows, some historians concluded she is represented as without eyebrows. But recent evidence supports the conservative theory that the strokes have merely become invisible with the passage of time (high resolution photographs show traces of slight eyebrow lines). Expectations can also be manipulated against our will. Marcel Duchamp vandalized Mona Lisa in the obvious way by drawing a moustaches and goatee (Figure 29.6). He then engaged in invisible graffiti by presenting an untouched postcard with the title ‘Mona Lisa shaved’. Duchamp’s motto: ‘The spectator makes the picture’. Expectations are built out of propositions. And propositions are built out of concepts. If pigeons lack the concept of a moustache, then Duchamp cannot make the pigeon see Mona Lisa as having a moustache. Therefore, Duchamp cannot make the pigeon see Mona Lisa’s absence of a moustache. Admittedly, pigeons would have trouble recognizing the absence of a moustache even if they had the concept of a moustache. Pigeons are very slow to learn to solve any problem involving recognition of an absence. Four year-olds are almost as bad (Sainsbury, 1973).

Fig.  29.6  Duchamp’s L.H.O.O.G.

556   Roy Sorensen Even adults exhibit the ‘feature-positive effect’ in the sense that it takes far more trials for them to recognize an absence of a feature as being a key difference. What goes for discrimination learning goes for memory and perception. This positive–negative asymmetry is predicted by Henri Bergson’s thesis that negative judgements are metacognitive. Human beings have far stronger metacognitive abilities than other animals. Metacognition matures gradually, exhibits considerable inter-personal variation, and is cognitively taxing.

11  Conceptually dependent absences Cicero once said that there is no proposition so absurd that some philosopher has not believed it. But none of Cicero’s predecessors believed that Leonardo’s Da Vinci’s Mona Lisa depicts a woman with a moustache. They could not believe the proposition because they lacked access to this proposition. To understand a name, one must have a causal connection with its bearer. This was not possible for Cicero’s predecessors because ‘Mona Lisa’ names an object that was yet to exist. The point can be stretched to predicates such as ‘helicopter’. The ancient Egyptians have inscriptions that look like helicopters but they cannot be depictions of helicopters; they lacked any appropriate connection to these artifacts. These connections need not be causal. Inventors of the helicopter had the concept of a helicopter before there were any helicopters. All they needed was a sufficiently detailed description of a helicopter. There is considerable vagueness as to what constitutes sufficient detail. Leonardo Da Vinci’s design for a Helical Air Screw is sometimes touted as the invention of the helicopter. His design embodies insights into principles relevant to helicopter flight. In the Codice Atlantico, Leonardo writes ‘. . . I have discovered that a screw-shaped device such as this, if it is well made from starched linen, will rise in the air if turned quickly . . .’. But many more insights and technological advances would be needed for a helicopter that could really fly. ‘Seeing as’ only requires conceptual competence, not belief. When you learn that some Romans believed that the stars were holes in the celestial sphere, you can see the stars the same way they did. But Romans cannot see the stars as you do—as giant bodies undergoing nuclear reactions. You are their conceptual superior. In turn, your grandchildren will be your conceptual superiors. They will see absences in your landscape, and perhaps on your desktop, that are necessarily invisible to you. These reflections on ‘seeing as’ can be a useful retreat for Sartre. He claims that the perception of absences requires frustrated belief. That is too strong. One can see while being agnostic about what one sees: Am I looking at a shadow or a dark hole or a circle of black paint? Raising the issue of whether there is an F is sufficient for seeing an absence of an F. Instead of being relative to a belief, Sartre should say that presences are relative to a question—which he comes close to doing at the beginning of Being and Nothingness.

Perceiving Nothings   557

12  Sartre’s compromise Sartre agrees with Bergson that being has priority over nothingness. Absences are always particular absences. Presences can be general. ‘In a word, we must recall here against Hegel that being is and that nothingness is not’ (Sartre, 1969: 15). Against Bergson, Sartre says that negative perceptual reports are not misleading. According to Sartre, ‘The table is not white’ is a genuine, perceptual report. Yet Sartre agrees with Bergson that negative perceptual reports are based on frustrated expectations. Positive reality is what gives rise to perceivers and so is more fundamental than negative reality. The slogan ‘To be is to be perceived’ is false. But its negative variation, ‘To be absent is to be perceived as absent’, is accepted by Sartre. Lastly, Sartre accepts Bergson’s anthropocentrism about absences:  ‘non-being always appears within the limits of a human expectation’ (Sartre, 1969: 7) This anthropocentricism is natural for Bergson because he takes denials to be complex, higher order, social acts that require language. Sartre has lowered the bar for the perception of absences. So his humanism is more embarrassed by our evolutionary continuity with animals.

13  Epistemic versus non-epistemic seeing Sartre’s compromise has a translation into analytic philosophy that suggests a principled way of segregating human perception from animal perception. The first draft of this friendly translation is written in the language of epistemic versus non-epistemic seeing (spoken with greatest fluency by Fred Dretske (1969) in Seeing and Knowing). Epistemic seeing, ‘seeing that’, depends on belief. This is the sense of ‘see’ that David Armstrong elaborates; seeing as a sensory type of belief pick up. There is always some full proposition that goes with epistemic seeing. Non-epistemic seeing does not involve belief in any distinctive way. A dog sees a fire hydrant without believing that it is a fire hydrant. Although we often puzzle about what beliefs to ascribe to animals, we confidently attribute vision to them. So there must be a less demanding sense of ‘see’ that does not commit us to belief attribution. Non-epistemic seeing is a matter of looking in the right direction with a well functioning visual system and with the right kind of uptake by the sensory system (as opposed to consciousness). Non-epistemic seeing is automatic. It is a necessary prelude for seeing that something is the case. The ancients saw objects undergoing nuclear reactions in the night sky. But they did not see that these objects were undergoing nuclear reactions. If observation were completely theory dependent, experiments would be circular and private. What you see would be too dependent on what you believe you see. Although the history of science contains many Kuhnian episodes in which theory has exerted excessive influence over what counts as an observation, it has also many episodes in which observation contradicts theory. In analytic terminology, Sartre’s compromise is that all seeing of absences is epistemic seeing. Whereas Bergson denies that there are any observation of absences, Sartre affirms

558   Roy Sorensen that there is limited seeing of absences. All perceptions of absences are at the level of epistemic seeing. This by no means demeans them. Sartre thinks that most of what is perceptually interesting is dependent on consciousness.

14  Objectivism about absences Richard Taylor (1952) tried to demonstrate that there is non-epistemic seeing of absences with the diagram should in Figure 29.7. You see the absence of a dot in the second circle. Taylor concludes that negative things exist. But critics brought up the Duke of Wellington problem. The right circle equally has an absence of dashes, ampersands, and tildes. Why don’t we see them along the absence of a dot? According to Sartre, we infer these absences from what we do see (which is entirely positive). We see absences like we see the wind. Just as we infer wind from seeing the branches move, we infer absences by seeing positive features. Absences are theoretical entities like electrons. The only way we can justifiably believe that there are absences is by their explanatory power. According to the analytic Sartreans, the curious Parisians wished to see that the Mona Lisa was absent from the Louvre. They had to travel to the Louvre for this experience. The objectivist counters that the absence of the Mona Lisa did not depend on expectation (Sorensen 2008). The Mona Lisa was already absent before anyone missed it. When the curious Parisians queued to see the absent Mona Lisa, they already knew the Mona Lisa was absent from the Louvre. They were seeking an antiquarian thrill of directly perceiving the absence—the same antiquarian thrill one gets by seeing the Mona Lisa when it is present. There was a rumour that the Mona Lisa had long ago disappeared from the Louvre and that a copy had actually been stolen. If that rumour were true, then the crowd would have seen the absence of the copy of the Mona Lisa—not the absence of the Mona Lisa. For the absence depends on the host object, not beliefs about the host object. The individuation conditions for absences go by causality rather than resemblance. When a single twin is missing from a class, the teacher sees only the absence of that missing twin. Just as the teacher need not be able to distinguish between the twins to see one twin rather than the other, the teacher need not be able to distinguish between their respective absences to see one absence rather than the other.

Fig.  29.7  Taylor’s absent  dot.

Perceiving Nothings   559 The subjectivist may reply that one can supplement testimonial knowledge with perceptual knowledge. The search for knowledge can involve the acquisition of new ways of knowing the same thing. Testimony suffices to provide knowledge that a friend is dead. But one may wish to see the corpse. This kind of experience yields better knowledge. Admittedly, this perceptual knowledge is redundant. But redundancy is sometimes virtuous rather than vicious. There was overwhelming circumstantial evidence that the far side of the moon had craters. Yet the photographs of the far side still constituted progress in astronomy. Proving the same point in different ways improves knowledge. The improvement need not be in terms of certainty. The Pythagorean Theorem was long ago proved beyond all doubt. But mathematicians continue to welcome new proofs. For the objectivist, seeing the Mona Lisa is like holding Da Vinci’s paintbrushes. Direct contact furnishes knowledge by acquaintance. No amount of propositional knowledge gives us this other form of knowledge. How does one have direct contact with an absence? The objectivist answers by appeal to ordinary standards of contact. Doubting Thomas feels the hole made in Jesus’ body by sticking his finger in the wound. Appropriate contact with the lining of a hole constitutes feeling the hole. Unlike shadows and silence, holes can be perceived by more than one sense. Holes can be both felt and seen. Arguably, they can also be heard. Holes in bicycle inner tubes are sometimes easier to hear than see. When Elizabethans were hunting for hidden Catholics some would hide priests in holes in the wall. Father John Gerard recalls that one searcher ‘gave the place a kick. I was afraid that he would hear the hollow sound of the hole where I was . . .’ (Morris, 1881: 168).

15 Startle reflex The objectivist’s strategy is to dumb down absence to such a depth that the subjectivist cannot follow. Thus the objectivist gravitates to the most primitive reaction to absences. The Moro reflex in infants is stimulated by the sudden loss of support. The baby’s arms stretch out, then contract, and lastly, the baby cries loudly. Like other reflexes, the Moro reflex requires no learning and is not extinguished by habituation. Prey fish have a startle reflex toward looming shadows (as when black cardboard is passed over the surface of an aquarium). Nile Tilapia undergo cardiac arrest. These fish are exposed to aerial predators and so the sudden absence of light is a sign of mortal danger. But the fish reacts too quickly for inference to play a role. Startle reactions to shadows are general to many animals at a larval state—especially those which need to frequent the surface such as mosquito larvae. A mature animal may preserve this reflexive reaction to shadows despite changes in which predators are casting them. Startle is an invariant, all or none response that happens too quickly to wait for consciousness (Robinson 1995). The speed makes them startle fast to accurately fake. So the absences associated with them cannot be judgement dependent.

560   Roy Sorensen

16  Externalist anomalies for Sartre The objective aspects of perception pose riddles that cannot be answered by the retreat from belief to seeing-as. First, perceptions of absences are corrigible. Pierre could be behind a pillar. No amount of psychic fizz will conjure the absence of Pierre into existence given that Pierre is present in the cafe. There can be no détente between Pierre and his absence. This corrigibility generalizes across the distinction between occurrent and dispositional experiences of absence. According to Maurice Merleau-Ponty, one does not feel the absence of a loved one until time comes to phone him. But the report of death could still be false. The feeling of absence, however vivid, is no guarantee of the absence. Second, absences are not self-intimating. Absences often pass unnoticed. If Sartre sees Pierre’s identical twin at the café then Sartre will feel no disappointment. Nevertheless Sartre’s expectation will not be fulfilled. Pierre will be absent but will not be experienced as absent. Amputees initially fail to detect their absent limbs because a phantom limb hallucination masks the absence. As typical for perceptual hallucination, conscious recognition of amputation does not suffice to end the sensations of presence. Therapy is therefore directed at perceptual modules rather than sufferer’s belief system. One might complicate Sartre’s theory by requiring that there be a match between the representation of an absence and reality. The worldly side of this match will be problematic. For the whole appeal of the psychological theory is that it avoids commitment to a negative state of affairs.

17  Absence blindness An absence of F can arise without anyone even raising a question about F. In the summer of 1911, Louis Béroud had been copying Mona Lisa in the Salon Carré. On 22 August he arrived in the morning to complete his copy. Béroud instantly spotted the absence of the Mona Lisa. He asked the guard where it was. The guard had not noticed the Mona Lisa’s absence. Regaining his aplomb, the guard surmised that the Mona Lisa was off being photographed. But the guard did not find it in the photography department. Nor was it found anywhere else in the Louvre. The Mona Lisa had actually been stolen the day before. The police were suspicious of the guard. The guard must have looked at the wall where the Mona Lisa was hung. How could the guard have missed it? Experiments on change blindness suggest that the guard was simply inattentive. Even if he looked directly at the spot, the change may just not have registered. In cases of inattentional blindness, people look at the object in an appropriate way but fail to register it. Motorcyclists who have been struck by cars often recall that the driver looked right at them but continued on the collision course. The problem is attention rather than visibility. If the driver is looking for other cars, then he is less apt to notice a motorcycle. This error is made more likely when there are competing demands for the driver’s attention.

Perceiving Nothings   561 Change blindness differs from inattentional blindness in being a failure to update. After laboratory subjects see a photograph of a plane with soldiers boarding, they presume the engine will be there when they next look at the photograph. Few of them noticed when a doctored photograph was substituted (a photograph that subtracted the plane’s engines). Another study focused on the sort of continuity errors familiar from the cinema. The researchers made their own short film in which the actress’ scarf disappears. Few viewers noticed the disappearance. The lesson psychologists draw is that vision demands attention. There is only a limited reservoir. So people are vulnerable to ‘blindness’ by distractions that deplete this resource. If their attention is taxed, their visual performance will suffer. Change blindness may be a symptom of the frame problem. When a situation changes one cannot afford to review every possible ramification. One must operate with a default principle that lets ‘sleeping dogs lie’. Even attentive search does not guarantee recognizing what one is looking at. A novice fossil hunter can look right at a fossil tooth and not distinguish it from the clutter of pebbles and sand. Adults are better at visual search than children—despite weakening eyes. Grandparents notice this when reading ‘Find it’ puzzle books with their grandchildren. These puzzles require the child to spot an object in a cluttered field of objects. Grandparents break camouflage faster than their grandchildren.

18  Disunified perceptions Another objectivist argument for the perception of absences is based on disunified perception. A person can see an absence with one eye and mis-perceive it as present with the other eye. If the perception of absence occurred at the level of consciousness, then the unity of consciousness would prevent bifurcated perceptions of absences. The neuro-scientist, Margaret Livingstone appeals to another disunity to explain the mystery of the Mona Lisa’s smile. When you look at Mona Lisa’s eyes, she seems to be smiling. When you check by looking at her lips, she seems to be not smiling. Livingston’s explanation is based on the distinction between foveal and peripheral vision. Foveal vision is proficient at discerning detail. It is less suited to detecting shadows. Peripheral vision has complementary strengths and weaknesses. It is adept at picking up shadows. When we look at Mona Lisa’s eyes, the shadows are detected by peripheral vision and are used to construct a smile. When we look directly at her lips the shadows cannot be detected by foveal vision and do not provide the cues necessary for the perception of a smile.

19  Imperceptible absences If seeing absences were a matter of epistemic seeing. ‘seeing that’, then one should be able to see the vanishing point behind Mona Lisa’s head. For a trained artist can see that her

562   Roy Sorensen head occludes the vanishing point. But he cannot see the vanishing point. Removing Mona Lisa’s head would not help. The vanishing point is a limit to what is represented. It is like the edges of the peripheral field. The viewer knows where the boundaries of what is visible are by what he sees. But the boundaries are not themselves seen. The perception of absences is conditioned by its positive counterpart. More specifically, the perception of absences is shaped by counterfactuals (which are in turn shaped by history, laws of nature, and resemblances). The particulars of the absence are what would have been perceived. To preserve the ‘wall of shame’, the Louvre’s curators did not move the paintings by Titian and Correggio that served as bookends for the Mona Lisa. When curiosity died down, they ended the spectacle by putting Raphael’s ‘Castiglione’ in its place and shuffling the order of the surrounding pictures. This counterfactual principle explains why we do not perceive the absence of bacteria in an autoclave. It also explains why one cannot see mathematical absences and psychological absences such as lexical gaps. The range of what is negatively perceivable is determined by what is positively perceivable.

20 Summary Our narrative began in Paris, during ‘La Belle Epoche’. The leading rival of Hegel, Henri Bergson, deployed the intuition that reality is fundamentally positive. According to Bergson, what appear to be perceptions of absences are sophisticated, indirect reports of what is positively perceptible. Jean Paul Sartre began his career as a disciple of Bergson’s metacognitive position. But a combination of Husserl’s phenomenology and Gestalt psychology led him to regard even positive perceptions as highly constructed. Negation is such a central feature in this construction that Sartre concluded that negative perceptions differ only modestly from positive perceptions. Although negative perceptions are filtered through human expectation, they are genuine perceptions. Even for Sartre, the fundamental substrate of reality is entirely positive. But human consciousness builds up a much more elaborate world of constructed entities. This sets up the possibility of perception of absences. In Dretske’s terminology, Sartre compromises by allowing epistemic perception of absences but not non-epistemic perception. The immaterialists Roberto Casati and Achille Varzi agree that absences supervene on material things but reject reductionism. They rely on this supervenience to explain how immaterial holes can be perceived. The fourth position, objectivism, is a reaction to Sartre’s subjectivism. The objectivist affirms the non-epistemic perception of absences. He uses empirical psychology to de-intellectualize the perception of absences. To anchor absences in the external world, the objectivist permits causation by absences. To stave off metaphysical objections, he refuses to lodge absences on either side of the Being versus Non-Being divide. He treats them as awkward characters, like space and time. It is better to say they exist than that they do not. But the upshot of affirming their reality seems particularly vaporous. The objectivist looks

Perceiving Nothings   563 to his feet in embarrassment and then casts his eyes into the distance, hoping for rescue by the young.

References Bergson, Henri (1911). Creative Evolution, trans. A. Mitchell. New York: The Modern Library, 1960. Casati, Roberto and Varzi, Achille (1994). Holes and Other Superficialities. Cambridge, MA: MIT Press. Casati, Roberto (2003). The Shadow Club, trans. Abigail Asher. New York: Knopf. Dretske, Fred (1969). Seeing and Knowing. Chicago: University of Chicago Press. Gale, Richard M. (1976). Negation and Non-Being. Monograph number 10 published by the American Philosophical Quarterly. London: William Clowes & Sons. Hurvich, Leo M. (1981). Color Vision. Sunderland, MA: Sinauer Associates, Inc. Husserl, Edmund, (1900–01). Logical Investigations, trans. J. N. Findlay, revised edition by D. Moran. London: Routledge 2001. Kant, Immanual (1965) Critique of Pure Reason, trans. N. K. Smith. London: MacMillan. Macknik, Stephen, Martinez-Conde, Susana, and Blakeslee, Sandra (2010). Sleights of Mind:  What the Neuroscience of Magic Reveals about Our Everyday Deceptions. New York: Henry Holt. Martin, M. (1998). ‘Bodily Awareness: A Sense of Ownership’. In J. Bermudez and N. Eilan (eds), The Body and the Self (pp. 267–290). Cambridge, MA: MIT Press. Merleau-Ponty, Maurice (1962). Phenomenology of Perception, trans. Colin Smith. New York: Humanities Press. Michotte, Albert (1962). The Perception of Causality. Andover, MA: Methuen. Moore, G. E. (1939). ‘A Proof of the External World’. Proceedings of the British Academy, 25, 273–300. Morris, John (1881) The Life of Father John Gerard, of the Society of Jesus. London:  Burs and Oates. Robinson, Jenefer (1995). 'Startle'. Journal of Philosophy, XCII, 53–74. Sainsbury, Robert (1973). 'Discrimination learning utilizing positive or negative cues'. Canadian Journal of Psychology, 27(1), 46–57. Sartre, Jean Paul (1969). Being and Nothingness, trans. H. E. Barnes. New York: Washington Square Press. Sartre, Jean Paul (2004). The Imaginary, trans. Jonathan Webber. London: Routledge. Originally published in 1940. Schaffer, Jonathan (2000). ‘Causation by Disconnection’. Philosophy of Science, 67, 285–300. Sorensen, Roy (2008). Seeing Dark Things. Oxford: Oxford University Press. Taylor, Richard (1952). ‘Negative Things’. The Journal of Philosophy, XIX/13, 433–449. Wittgnestein, Ludwig (1967) Zettel, ed. G. E. M. Anscombe and G. H. von Wright, trans. G. E. M. Anscombe. Oxford: Blackwell. Wittgenstein, Ludwig (1975) Philosophical Remarks, ed. Rush Rhees and trans. Raymond Hargreaves and Roger White. New York: Barnes and Noble Books.

Pa rt  V

I N T E GR AT I NG SE NS ORY I N FOR M AT ION

Chapter 30

The I n di v iduation of the Senses 1 Mohan Matthen

How many senses do humans possess? Five external senses—sight, hearing, touch, smell, and taste—as most cultures have it? Should proprioception, kinaesthesia, thirst, and pain be included, under the rubric bodily sense? (Ritchie and Carruthers, Chapter 18, this volume) What about the perception of time (Le Poidevin, Chapter 24, this volume) and the sense of number? And what makes the traditional five senses unitary anyway? Why are colour perception and shape perception both included under vision? Aristotle discussed such questions in the De Anima (Sorabji, 1971) as did Duns Scotus in his commentary on Aristotle’s work. H. P. Grice (1962) revived them. More recently, they have taken on fresh interest as a result of a collection of essays edited by Fiona Macpherson (2011a). Questions about counting the senses reduce to two more fundamental ones.

1 What kind of faculty counts as a sense? 2 By what principle do we distinguish the senses from one another?

This entry reviews some approaches to these questions. Inter alia, it advances two new ideas: • The senses constitute a group of information-gathering faculties within which content from each can be integrated with content from the others.2 This suggests a relational answer to question 1: the senses are different from other information-gathering faculties because together they form a system by virtue of the content-integrating relations they bear to one another.

1  I am grateful to Ophelia Deroy, Barry Dworkin, Matt Fulkerson, Randy Gallistel, Celia Heyes, Brian Keeley, Robin Le Poidevin, Casey O’Callaghan, Louise Richardson, Barry Smith, and Charles Spence for detailed and helpful comments. 2  I use the term ‘content’ to characterize the representational output of the senses—what they ‘tell’ us. Content is usefully defined as the set of circumstances in which the perceptual state bearing that content would be accurate. See Nanay, Chapter 8, this volume.

568   Mohan Matthen • Broadly speaking, there are two approaches to distinguishing the senses. One (the ‘sensory’ approach, as it will be called here) aligns more closely with how scientists tend to think about question 2. The other, introduced here as the ‘perceptual’ conception, aligns, at least extensionally, with everyday, or ‘folk’, conceptions. In the ‘perceptual’ conception, each modality is associated with a system of knowledge-gathering perceptual activities. These activities differentiate the modalities.

1  What is it to sense? In order to count the senses, one must know not just how they are distinguished from one another, but also how they are differentiated as a group from other capacities. Minimally, a sense is a faculty that monitors the current state of its external and bodily environments in order to mediate an organism’s response to these environments. But this is not a sufficient characterization, for it includes too much. The body monitors its environment for homeostatic regulation. For example, blood CO2-level is monitored by dedicated chemoreceptors; the output of these receptors is used to regulate breathing. Let’s call such mechanisms HR monitors. Yet, HR monitors, such as the chemoreceptors just mentioned, are not included among the senses, as conceived either by the tradition or by scientists. Why are they excluded? One commonly held intuition is that the senses provide the subject with content-bearing experience, while HR monitors do not. Thus, information (in the sense of Shannon, 1948) about blood CO2-level feeds directly into the body’s regulatory systems, and the subject does not directly experience them. Of course, you may come to believe that your blood CO2-level is high when you feel yourself breathing harder, but this is only because you know something about how the body controls breathing. Without such experience, you cannot come to know your blood CO2-level. The CO2 monitor provides us with conscious access to its content. Certain kinds of blindsight show a similar pattern. There are patients who, because of damage to a part of the visual system, cannot recognize visual objects. But, because other parts of their visual system are intact, they are able to execute visually guided action. These patients can become adept at knowing certain aspects of their environment—for example, the shape of things they handle—simply by monitoring their own bodily responses (in this case, grip shape). (Goodale and Milner (2004) describe the everyday life of one of these patients, DF.) What most people recognize holistically, directly, and non-inferentially, these patients identify by learned associations with other bodily responses and sensory states. We can come to know our blood CO2-levels in this second, but not in the first, way. Because sensory experience is connected with the formation and assessment of beliefs, direct availability via experience leads to another important characteristic. You can ‘withhold assent’ from putative information your senses provide: two lines might look equal, but you may refuse to believe that they are equal, because they are part of a Müller-Lyer diagram. But you cannot do the same with information provided by HR monitors. Subjects may, of course, voluntarily override homeostatic breathing control—for instance, they may deliberately breathe slowly and deeply after a run. When they do this, they are not

The Individuation of the Senses    569 contesting, or ‘withholding assent’ from, the output of the CO2 monitor. Having no access to this output, they exercise direct control of breathing independently of it. The senses do not control belief, then. Rather, they provide consciously available ‘perceptual content’, which the subject uses, with due attention to other aspects of her situation, as a reason for belief. By contrast, homeostatic systems directly influence the body’s reaction. They do not merely provide the organism with defeasible reasons for belief, which in turn is a condition for bodily control. This suggests: Subjective Experience Criterion A sense is an information-gathering faculty that provides a subject with informative experience that rationally bears on her beliefs, but from which she can withhold assent.

Though initially plausible, this characterization of the senses faces serious difficulties.





1 With regard to ‘withholding assent’, many philosophers hold, and there is good empirical reason to believe, that in humans, sensory appearance creates a causal disposition to believe. When evidence opposes the belief, the causal pressure can be overridden, but it still exists.3 So the case is structurally similar to the detection of elevated blood CO2-level. Both systems seem to create causal pressure to do something—to breathe more quickly or to form a belief. This causal pressure cannot be denied in either case, but the effect can deliberately be overridden in both. Subjects don’t neutralize the belief-creating force of sensory content; rather, they defeat the formation of belief. 2 With regard to content-bearing conscious experience, there may be unconscious perception. (Ross, 2008; Prinz, Chapter 19, this volume.) Moreover, signal detection theory (Winer and Snodgrass, Chapter 38, this volume) suggests that whether perceptual content is conscious, and whether it serves as a reason often depends not just on its own nature, but also on perceiver ‘bias’. Thus, consciousness and aptness for reasoning are not intrinsic characteristics of perceptual content and cannot serve as unequivocal markers of the senses. 3 With regard to the distinction between HR-monitors and sensory systems, lower animals such as honeybees and ants have sensory systems (Keeley, Chapter 44, this volume), but they lack rationality and perhaps even consciousness. Certainly, they do not treat sensory content as reasons for belief. So one might think that in these animals, the proposed mark of the senses breaks down.

One way to address these concerns is to lean more heavily on the idea of situation-dependent response—the idea that in some situations, sensory content, blue, will have one kind of response, and in others, another—putting aside considerations of rationality and consciousness of sensory output. Overemphasis of these characteristics of 3  The point is generally made in the context of action. William James wrote: ‘We may lay it down for certain that every representation of a movement awakens in some degree the actual movement which is its object’ (1890: 526). Generally speaking, acting contrary to an impulse is a matter of suppressing it, not of blocking its formation. For a review of ‘inhibitory control,’ see Munakata et al. (2011). This pattern extends to epistemic assessment, where response is slowed by the need to suppress unwanted causal influence. For recent evidence of this, see Kovács, Téglás, and Endress (2010).

570   Mohan Matthen response modulation is anthropocentric, it might be said: it seizes upon a human way of doing what many other kinds of animals do in different ways. Here is one model. The senses (as well as the emotions, etc.) post their output to a temporary store called ‘working memory’ where it is held for a short period. Various action-controlling and cognitive systems, including belief-formation systems, have access to content in working memory, and use it in conjunction with other information, including motivational and stored content. (Baars (1997) calls this temporary store the ‘global workspace’.) Response to a sensory state depends on the other contents of working memory: for example, if prior associations or memories available to working memory incline a subject to treat a particular object as green, sensory content that it is blue may be overridden. I started by noting that the senses enable an organism to respond appropriately to the current state of its external and bodily environments. I now add that the ‘traditional senses’ do so in the context of the organism’s goals and, importantly, in the light of other sensory information that feeds into working memory. The effect of the senses is, in other words, indirect. Sensory output is held in abeyance, so to speak, awaiting other information that might defeat it. In humans, the senses often work through consciousness and rationality. By consciously experiencing something as blue, I gain rational support for the belief that it is blue, but whether I actually believe that it is blue depends on other information, including other sensory information (Pollock, 1974). In lower animals, the assessment of sensory content is performed differently. A honeybee sips from flowers that bear one coloured pattern but not from those that bear another; its action is visually guided. But we have no idea whether it performs this act of discrimination by anything involving conscious awareness; certainly, it does not utilize the norms of rational assessment. Nonetheless, the bee’s action is also dependent on its goals and other information: the bee may detect a lurking predator and veer off in another direction, or it may be time to return to the hive, and so it may abstain. Thus, the bee’s visual system cannot be said to control action directly: it too posts output to a ‘workspace’ where it interacts with other content. A second important mark that sets the senses off from other information-gathering faculties, including HR monitors, is that they are the basis for learning. Consider, in particular, associative learning or conditioning. A response to a stimulus P can be transferred to a sensed stimulus, Q, because Q has been sensed together with P.4 • Suppose that Suzy habitually makes coffee while she prepares breakfast. Her body comes to associate the smell of coffee with food to come. One day, she makes coffee, but decides to skip breakfast. Nevertheless, her stomach growls at the smell of coffee. Her instinctive (or unconditioned) response to food has spread, by associative learning, from the food itself to the smell of coffee, because the smell of coffee is a learned (or conditioned) indication of food to come. 4 

Two important notes: (a) It is not necessary that both P and Q be sensed (though they are in Pavlov’s 1927 experiments). The conditioned stimulus Q needs to be sensed; the unconditioned stimulus need not be. On this point, see Dworkin and Dworkin (1995). (b) Gallistel and King (2010) argue that the transference of response model of associative learning cannot capture its complexities, and that a more computational model is required.

The Individuation of the Senses    571 Information capture for regulatory purposes does not feed into conditioning in this way. • Suppose that Barbara habitually undertakes a Bikram Yoga exercise (performed in a hot room) that increases her blood pressure and makes her sweat. Performing yoga is a reliable indicator of heat. One day, she shows a friend her yoga exercise, but does so in the cold outdoors. Her blood pressure rises, but she does not sweat. Though the yoga exercise and the elevated blood pressure are indicators of heat, they do not become triggers for the mechanisms that help her body adjust to heat. Associative learning does not occur. Organisms respond to the current environment in many different ways. One is sensory anticipation. A sensory signal can come to be treated as a sign of something to come, and may prompt an efferent system to take action that anticipates a future contingency. More importantly, organisms learn from the senses. Learning occurs both within and across the senses. In short, the senses are integrated into a learning system. HR monitors do not participate in this system. Their triggering function cannot be linked to any other signal. So, unless an autonomous bodily response has a sensory trigger, it simply has to wait for the HR monitor to activate it. To summarize, the senses trigger conditioned, as well as unconditioned, responses; HR monitors trigger only unconditioned responses.5 In virtue of the above considerations, I now propose: Integrated System Criterion There is a certain group of information-gathering faculties that together form a system within which goals, actions, beliefs, and learning are modulated by content provided by all the information-gathering faculties in the group, and do not depend on any one taken by itself. The information-gathering faculties in this group are the senses.

The Integrated System Criterion is relational. It concerns how sensory information can be used to anticipate unsensed things. Non-sensory information is excluded from these integrative connections. The thrust of the Criterion is that the senses have nothing intrinsic in common that distinguishes them from other information-gathering faculties—except that which is necessary for their output to be integrated in working memory and associative learning, and that which follows from such mutual integration. The conditions proposed thus far are not yet sufficient. The ‘number sense’ by which primates estimate the numerosity of (sufficiently small) collections (Gelman and Gallistel, 1978; Carey, 2009, c­ h. 8) seems to satisfy the above criteria—it contributes to working memory and associative learning, and seems to provide direct awareness of number. But it is not a sense modality. The reason is quite simple. Input to perceptual systems comes from transducers—cells that convert incident energy into a neural pulse that carries information about this energy. (For example, the rod cells of the eye convert light into neural pulses that carry information about this light.) Sensory transducers respond to the environment in physically determined ways: for instance, the basilar membrane in the ear 5  This may be a matter of degree, for there is some evidence that HR monitor outputs can become associated (Dworkin and Dworkin, 1995; Dworkin, 2007). However, this association is weak and limited to a few outputs at best. As Dworkin (2007) notes, however, the research on this was mostly conducted in Eastern Europe, and there has been little work on it since the 1960s.

572   Mohan Matthen is so constructed that different parts resonate, as a matter of physical law, to different auditory frequencies. The number sense lacks transducers. It operates equally well in different modalities—we can quickly estimate the number both of small collections of successive light flashes and of successive sounds (though possibly not of tactile stimuli—see Gallace, Tan, and Spence 2008). Collections of objects and events do not act on cells that emit a neural pulse that carries information about number. Rather, the number sense operates on the outputs of (some) other senses. It is thus a post-perceptual module, rather than a sense modality. A further complication is that in a genuine sense modality, the neural signal emitted by sensory transducers is, as Brian Keeley (2002) points out, processed in ways that are ‘historically’ (i.e. evolutionarily) dedicated to the recovery of information about external stimuli. Keeley notes that the weak electric current from a charged 9V battery creates a definite sensation on the human tongue. This is because touch and taste receptors respond to the electric current that the batteries emit. However, humans lack information-processing data-streams designed by natural selection to extract informational content about and respond to ambient electricity from this stimulation of touch receptors. Consequently, the sensation produced by a weak current feels like a tactile stimulus on the tongue, or sometimes like a flavour, not like an event of a distinct sensory type. By contrast, electric fish have systems dedicated to processing information about electrical fields, and they perceive quite specific features of these fields. Sharks, for example, detect prey by the disturbances in the electric field caused by their movement (Hughes, 1999). So though both humans and sharks sense electric current, sharks (but not humans) have a distinct sense modality for electric fields or currents. These ideas are important in determining whether pain and sexual arousal are sensory states; both were posited as such by some historical authors (Dallenbach, 1939) on the grounds that each is associated with a special kind of experience or quale, which is, moreover, informative about current circumstances. In the historical debate about pain, subjective experience proved inconclusive. Is the pain of being burned a particularly intense, and hence unpleasant, sensation of heat? Or is it a distinct sensation that accompanies intense sensations of heat? Introspection does not decide the question. The transducer/processor condition throws some light on the issue. Some pain relies on receptors activated by high threshold values of mechanical, thermal, and chemical stimuli, and processed by a dedicated system in the brain (Craig, 2002, 2009). One might hold that these kinds of pain are sensory, and assuming that there are dedicated data-streams for them, it may be appropriate to treat of them as constituting a single sense. (The same can be said, for instance, of thirst: it has dedicated transducers known as osmoreceptors and a dedicated computational system (McKinley and Johnson, 2004).) These pains do not belong in the same modality as the intense stimuli that are associated with them—sensations of warmth, etc. They are separate and are produced by a distinct system. On the other hand, certain sensations are painful just because they are very intense—loud noises and bright lights, for example. Here it is appropriate to say that an auditory sensation is made painful by its intensity. There is no separate sensation of pain here. In sum, some painful sensations are the products of a separate sense, while others are not. Sexual arousal is different from pain and thirst. First, it should be noted that sexual readiness, or arousal, arises in a context-sensitive way from other perceptions, some visual,

The Individuation of the Senses    573 some tactile, etc. Arousal is the state of the body that enables sexual performance. Now, there may well be processes that detect some of the bodily changes that constitute sexual arousal. This may account for the characteristic feeling of excitement associated with arousal. But this feeling of excitement should be distinguished from arousal itself. The feeling could be regarded as awareness of arousal—assuming that the conditions discussed above are met. Or it could be said to be a concomitant of arousal. Both options are different from saying that arousal is itself a perceptual state.6 The above considerations reveal two important aspects of the senses. They are subject, first, to a Source Condition A  sense depends on transducers and information processing systems dedicated to information-capture regarding specific qualities of the impinging stimulus.

Secondly, they are subject to an Output Condition—namely, the Integrated System Criterion formulated above: that is, they modulate each other with respect to the beliefs and actions that flow from their operations. Thus we have: Sense Definition A sense is an information-gathering faculty that is subject to the Source Condition and Integrated System Criterion above.

These conditions throw light on some difficult cases. One, discussed by Keeley (2002) is vomeronasal sex detection (VSD). Humans apparently possess nasal receptors for pheromones: these enable them reliably to detect the sex of another human simply by sniffing their breath. (This is empirically disputed—see Meredith (2001)—and my treatment is hypothetical.) Keeley classifies VSD as a perceptual system on the strength of the input condition alone. But one important point to consider is that most subjects are unaware that they are sensitive to sex in this way: the pheromonally mediated response is not, as far as we know, modulated by the other senses. (For example, the fact that a woman is disguised as a man will not slow or suppress the VSD response.) And there is no documented way that this information can assist in learning: you cannot, as far as anybody knows, associate (for example) an auditory signal with the pheromones detected by VSD. It is possible, therefore, that VSD operates autonomically; perhaps, it simply prepares the subject for the presence of a potential sexual partner. If so, it fails the Output (i.e. the Integrated System) Condition, and would not count as a sense. The sense of time is more difficult to judge. We possess a number of ‘endogenous oscillators’ (Gallistel, 1990) that govern bodily processes of different periodicities ranging from menstrual and circadian rhythms, down to the very short time periods involved in conditioning and the timing of our limbs while walking. The standard view is that these timing processes work by averaging the periods of the oscillators (since these are not precisely synchronous), and are thus ‘emergent properties of neural dynamics’ (Wittmann and van Wassenhove, 2009). Should the endogenous oscillators be regarded as transducers for a sense of time? That is, do periods of time cause them to emit a pulse that carries information about these periods of time? Both sides of the question can be argued. One 6 

Louise Richardson asks (in correspondence) whether the motivational import of sexual excitement disqualifies it from being sensory—the same question could be asked about pain. In my view, perceptions always have a motivational aspect: at the very least, they provide motivation for forming a belief. So no.

574   Mohan Matthen might argue that time is immaterial and therefore never a cause. Such a position would be a reason to exclude the sense of time: the oscillators are not causally influenced by time as such. This might lead one to say that what we experience as the passage of time is actually the periodicity of certain bodily processes, which stand proxy for the passage of time. As opposed to this, one might argue that the important point here is that the oscillators function as a proxy for time in much the same way as the retinal image is the proximate substitute for distal occurrences. These questions cannot be decided here. A methodological remark will serve as a transition to the next section. Matthew Nudds (2004) has argued that SENSE is an intuitive concept that we employ for certain everyday purposes. This sort of concept does not have objective scientific content, Nudds suggests; it relies simply on societal agreement, or ‘convention’. He rejects questions about such things as vomeronasal sex detection because ‘our conventions may have nothing to say about such cases’. This suggests that Nudds thinks of SENSE as extensionally defined. We know that the traditional five senses are senses because we have agreed to treat them as such, and not because we have an abstract idea under which these fall. I am proposing, by contrast, that SENSE is intensionally defined. It is not simply given to us in intuition that there are five external senses or that pain is or is not a sense. Rather, a certain concept of SENSE is given to us in intuition, and refined by science. Whether each of the traditional five senses falls under this concept depends on the facts about these—as we saw, pain turns out to be more diverse than intuition might have imagined. In principle, even vision could turn out this way (though, of course, there is no reason to think that this is going to happen). Other positions are possible. There may be an ur-concept that folk-psychology and science develop differently. Or there may be several different but overlapping concepts. Something might figure as a sense on some of these but not others (Macpherson, 2011b). Alternatively, the choice among divergent concepts could depend on the explanatory or conversational context.

2  Distinguishing the sensory modalities We turn now to the question of how to distinguish the senses. When does a group of perceptual processes count as belonging to a single sense modality? Philosophical approaches to this question generally appeal to one of two types of consideration—those immediately accessible to the perceiver and those based on the nature of things—the nature of the perceptual system, or of the kind of energy it detects, etc.—which the perceiver may not immediately know. (Some of them are recently discovered; some are clearly the state of science, but still possibly reversible.) In the literature, one perceiver-accessible criterion is paramount—the ‘special introspective character’ of experiences characteristic of a particular sense. The view is that vision is the sensory faculty that produces qualia that have the special introspective character of vision; audition those of audition, and so on. (Of course, visual qualia may be different in kind from one another: blue from red, for instance. The idea is that they are nonetheless marked by a higher commonality characteristic of vision.) H. P. Grice (1962), who holds a special place in the literature for reviving interest in the question of how to individuate the senses, is famous for arguing that the Special Introspective Character

The Individuation of the Senses    575 Criterion (SICC) is ineliminable. Grice tried to show that all other criteria either lean on it or are insufficient without it. Now, in the previous section, we encountered considerations that throw SICC into doubt. First, the honeybee—though we attribute vision to it, we do not know whether it has conscious visual experience at all, and if it does, whether this experience is anything at all like ours (cf. Heil, 1983).7 Secondly, the criterion is impossible to apply to certain cases. For instance, SICC does not help us decide whether the pain is a separate modality. If pain is a single modality, the painful experience of touching a hot stove belongs to the same modality as that of eating a very hot chilli, and thus to a different modality from the neutral sensation of heat that one experiences in a tepid bath. If, on the other hand, pain is simply a characteristic of other experiences (for example, that they are uncomfortably intense), then these two kinds of pain belong apart; one is tactile, the other gustatory. The quality of the experience does not seem to decide between these two accounts. Finally, there are information-rich experiences on which SICC delivers no verdict at all. Is sexual arousal a sensory state? If so, does it, or does it not, belong to the same modality as hunger? Is there perception of temporal passage separate from the perception of certain periodic bodily processes? SICC seems simply to be the wrong place to look for an answer to these questions. (It doesn’t seem, either, that we have an ‘everyday conception of the senses’, as invoked by Louise Richardson (2013), that would settle this issue about pain.) At this point, a philosopher might suspect that a world-based (as opposed to experience-based) criterion might be useful. This was Aristotle’s approach. Aristotle explored two criteria of differentiation between senses. The first, which he considered ontologically prior, concerns the special properties that each sense directly reveals as vision reveals colour, audition pitch and sound volume, etc. Let’s call this the Special Properties Criterion (SPC). On the plausible assumption that we know what properties are presented to us in perception, this is a perceiver-accessible criterion. There are two versions of SPC. In Hume’s version, each sense is characterized by any of the properties of which it provides direct perception. As Grice pointed out, Hume’s version falls to the problem of common sensibles—qualities, such as shape, which are sensed in more than one modality. But Hume actually discounts this problem, for he believed (following Locke) that visual shape is different from tactile shape (with the consequence that there are no common sensibles). But Gareth Evans (1985) convincingly argues that this is wrong:  since shape is a geometrical property, its defining characteristics are not defined by how they are sensed. A shape such as square could be defined by how proper sensibles are distributed, and such a distribution could be assessed in a manner that is not specific to any sense. Thus, Hume’s way with shape is inadmissible. He does have another defence: namely, that visual shape is perceived indirectly, through colour. (‘If we see it, it is colour.’) But this defence too is contentious. For in a display like the Kanisza triangle (Figure 30.1), the sensory detection of shape precedes that of colour. Here, the visual system falsely detects a triangular contour in the foreground, which it then highlights by filling it in with a bright appearance. (In fact, the colour of the illusory triangle is the same as that of the background.)

7  This difficulty would be ameliorated by the homology criterion, discussed in section 4—but this criterion is not perceiver accessible.

576   Mohan Matthen

Fig.  30.1  The Kanisza triangle:  Form detection precedes colour detection  here. Aristotle’s version of SPC is more cautious. It is that for each sense modality there is some property that is (a) directly perceived only in that modality, and (b) accompanies all perceptions in that modality. For vision, the distinctive property would be colour (or possibly brightness): everything that is visually perceived is perceived in colour. Audition has more than one such distinctive property: pitch and volume. Nudds (2004) argues that (b) is wrong. Anne Treisman and colleagues showed in the 1980s that colour is processed separately from shape and seem to be perceived together only because they are ‘bound’ together when a subject directs visual attention to some part of her visual field (Treisman and Gelade, 1996). Thus, shape might be sensed independently of colour. The same sort of thing could well be true of the other modalities as well. Aristotle’s version of SPC falls afoul, then, of the separateness of perceptual processes that fall under the same modality, though it could perhaps be modified to meet this objection. There is another consideration—generalized from an argument in Grice—that cuts decisively against both versions of the Special Properties Criterion. Properties cannot by themselves define boundaries between the senses. What makes colour a distinctively visual property? Surely the answer cannot lie in the intrinsic nature of colour; it has something to do with the relation colour bears to the senses. Perhaps the crucial fact is that information concerning colour reaches us through light and the visual transducers; perhaps it is that our experience of colour is distinctively visual. Either way, SPC is necessarily either incomplete or circular: it must lean on some other differentiating feature of the senses. It is worth noting that SPC and SICC actually cut against each other. Consider the McGurk effect. When you listen as well as watch somebody uttering the syllables /ga/, /ga/, /ga/, you may well feel that the look of the mouth is completely discrete and different from the sound of the phoneme: one is a bodily movement revealed by looking; the other is a sound that you hear. But McGurk and MacDonald (1976) performed the interesting experiment of filming a speaker saying /ga/, /ga/, /ga/, and substituting an audio track of the sound /ba/, /ba/, /ba/ (synchronized with the lip movements). Upon watching this mismatched audio-visual clip, subjects automatically and irresistibly hear /da/, /da/, /da/, a phoneme intermediate (in terms of articulation) between the visually presented /ga/ and the auditorily presented /ba/. In this experiment, subjects appear to hear /da/: thus, on SICC, phonemes are perceived by the auditory sense. However, the McGurk effect shows that vision contributes to the special introspective character of phonemes—thus indicating that this introspective character is not unique to audition (or speech perception). A further point regarding the McGurk effect is that when subjects look and listen to a speaker they are much faster and make far fewer errors of identification than when each

The Individuation of the Senses    577 modality is unassisted by the other. This shows that the two modalities act together in this domain (Green and Kuhl, 1991; Jones and Munhall, 1997). Phonemes are actually perceived by the operation of two kinds of perceptual process, visual and auditory. This seems to indicate that the detection of phonemes is multisensory; they are not, as shape is, merely perceived by two different modalities taken singly. This puts further pressure on SPC, which in Hume’s version doesn’t allow for shared properties, and even on Aristotle’s version does not consider the possibility of two modalities cooperating. Aristotle’s second world-based criterion is the medium through which received information is transmitted: in the case of vision, he says, this is ‘the transparent’; for audition, the compressible; for touch, flesh; for gustation, saliva; and for smell, something ‘analogous to the transparent’ (presumably an odourless medium capable of transmitting odour). Putting Aristotle’s outmoded science aside, this is a promising approach. Earlier, we saw that the input end of each sense modality is a set of transducers that convert ambient energy into a neural pulse. Different forms of energy require different transducers, and this may provide us not only with the evolutionary reason why there are different senses—environmental information is available in more than one form of energy—but also with the basis for differentiating them. Why do we have both vision and audition? Because both light (the sub-spectrum of electromagnetic radiation that is of the right wavelength to be reflected by atoms) and sonic energy (compression waves in air or water) carry information. Why are vision and audition different? Because the same transducers will not work for both—transducers are specialized for the type of energy incident upon them. This suggests: Hybrid Medium-Transducer Criterion (HMTC) A sense is a collection of perceptual processes that begin from transducers specialized for information capture from a particular kind of energy.

HMTC does not demand one type of transducer for each sense: rather, it groups transducers by their receptivity to the same kind of energy. Thus, vision has rod and cone cells, but both are sensitive to light. Similarly, touch has several kinds of receptors sensitive to mechanical energy. This criterion is inaccessible to the perceiver; it is world based and scientifically determined. It is, however, perceiver involving in that transducers are components of the perceiver’s sensory apparatus. One reason to group such perceptual processes together is that they will be separate and independent in their earlier stages. The early stages of auditory processes are concerned with extracting environmental information from sound; they will be discrete from the early stages of visual processes because the principles of extracting information from sound are different from those of getting information from light. This consideration is quite general: extracting information from one medium is different in principle from extracting it from another. (See, however, Kiverstein, Farina, and Clark (Chapter 35) on sensory substitution in this volume—particularly, on the ‘meta-modal brain’ concept.) HMTC captures quite well how scientists think about modality and multisensory perception. The early stages of sensory processing are about extracting properties of the signal received by the transducers. At these stages, processing is discrete. The later stages are concerned with the sources of the signal—i.e. the distal environmental objects that emit the signals received by the transducers. Since all modalities are ultimately concerned with the same (or at least with overlapping) environmental objects, the information extracted by

578   Mohan Matthen other perceptual processes becomes relevant. Thus in the late stages of sensory processing, the separate early data-streams flow together. This is well illustrated by the McGurk effect, where visual and auditory data are reconciled in order to get the best possible conclusion about the speaker’s utterances. Because HMTC is primarily concerned with a distinction between sensory data-streams in their early stages, it is appropriate to entitle it a sensory criterion, and the modalities distinguishes sensory modalities. Later, I shall introduce a partially overlapping notion of perceptual modalities. HMTC runs into perplexities, however, with touch and gustation. Touch has as many as six kinds of transducer: mechanoreceptors for pressure, weight, and stretching, thermoreceptors, chemoreceptors, pain and itch receptors, and (possibly) receptors specialized for gentle stroking. These are the ‘cutaneous’ receptors (McGlone and Spence, 2010). In addition, there are motion receptors embedded in the muscles and joints. Accordingly, some suggest that touch is multisensory (Lederman and Klatzky, 2009). Moreover, there is a difference between active and passive touch (Gibson, 1962). In active touch, which includes haptic exploration of objects (and thus employs bodily activity), one perceives the properties of external objects—shape, size, texture, temperature, etc. In momentary passive touch, one perceives the condition of one’s own body. (Temporally extended passive touch is intermediate: one can sense certain properties of external things when they are impressed upon or moved against the skin.) By the Special Properties Criterion, therefore, active and (momentary) passive touch are different modalities.8 Despite these complications, most people think of touch as a single modality. (Fulkerson, 2011, and Fulkerson 2014 have one account of why it should be so considered.) Flavour perception is even more complicated. Put a chocolate mint in your mouth, and you experience it as sweet, characteristically chocolatey, and cool. You probably feel little hesitation identifying these properties as delivered by a single modality (Richardson, 2013). In fact, you would probably say that the confection has a single, complex flavour in which the above are experienced as components. That is, the sweetness and the chocolatiness are not separate, as are the colour and the sound of a barking dog—they are merged together within a single complex whole. And it seems as if this complex property is the taste—or rather, to anticipate the next paragraph, the flavour—of the mint. How these components of flavour come together is philosophically quite puzzling (Auvray and Spence, 2008). The sweetness of the confection is detected by taste proper, which involves transducers on the tongue—sweet, sour, bitter, salty (and maybe umami and fat) are the qualities that come out of this set of transducers.9 Chocolate is detected by retronasal olfaction—that is, by vapours from the mouth rising into the nasal cavity and passing over olfactory transducers. (This is called ‘retronasal’ because the vapours pass over these transducers in the direction opposite to that when one sniffs something through the nostrils. The latter is known as orthonasal olfaction.) Interestingly, odours 8  Michael Martin (1992) claims that one’s awareness of something one touches and the sensation of touching it are ‘simply one state of mind, which can be attended to in different ways’ (204). The distinction between active and momentary passive touch argues against this; active exploration brings haptic awareness of things outside the body. There is, in fact, a perceptual deficit, astereognosia, in which patients are unable to feel the properties of external things, except through the sensation of being touched (as well as grip size, etc.) Martin would have to say that this is a deficit of attention, which it almost certainly is not. 9  Incidentally, taste cells are found in the stomach and gut. It’s not clear what they do there, but they probably do not function as sensory transducers.

The Individuation of the Senses    579 detected by sniffing are ‘referred to’ (i.e., sensed as located in) something external—the smell is experienced as emanating from the thing in front of the nose—while the properties detected by retronasal olfaction are referred to what is in the mouth, and are sensed as gustatory qualities. That the chocolate component of flavour is experienced as located in the chocolate (rather than in the nose) is (at least partially) the contribution of touch.10 Finally, the coolness of the mint is sensed by the trigeminal nerve, an important part of the tactile and pain system in the face. Because of the extra-taste components, the complex quality attributed to the chocolate mint is called flavour, not taste. (See Spence, Smith, and Auvray, 2014, for a systematic effort to straighten out this terminology.) Even more so than touch and phonetic perception, flavour is genuinely multisensory, at least when viewed through the HMTC lens—here it is not just that touch, smell, and taste cooperate to identify flavours; it is that flavour experiences have components contributed by touch, smell, and taste. From the point of view of HMTC, touch and flavour are multisensory. Consequently, most scientists show little hesitation in rejecting the intuitive view that they are single modalities (Smith, 2013). This surprises most non-scientists: there is a strong folk-psychological intuition that flavour is delivered by a single modality, as are tactile properties. This disagreement is not about the facts relevant to the application of the transducer criterion— the ordinary person does not contest scientific theories regarding how many kinds of transducers are involved in flavour perception. It appears, therefore, that ‘the folk’ employ a different criterion. In section 3, I try to figure out what this is.

3  Perceptual modalities There is a perceiver-accessible criterion that may be of use here, namely that of a perceptual activity. Such a criterion is hinted at by Grice, who talks not only about seeing and feeling (perceptual experiences) but also about looking and touching (things that one does in order to perceive). Thus, he denies that pain is a sense in part because ‘there is no standard procedure for getting a pain’, i.e. no analogue of an activity that stands to the experience of pain as ‘inhaling’ (I would prefer to say ‘sniffing’) does to the experience of smelling, and looking does to seeing. (As will emerge, I do not quite agree with Grice about pain.) He seems to suggest, in other words, that a sense is defined not just by an experience, but also by ‘procedures’ for bringing about sensory experience. Now, it is likely that the kind of ‘procedure’ that Grice had in mind was simply something like opening one’s eyes or unblocking one’s ears and thereby opening oneself up to visual or auditory experience— his point (with which I disagree) is that there is no analogous act of opening oneself up to pain. I want to follow up on Grice’s suggestion, but with a more expansive conception of ‘procedure’. Perception is not merely a matter of sensory experience; it is a matter of actively examining the world. One tastes something by putting it into one’s mouth, chewing, savouring, 10 

Murphy, Cain, and Bartoshuk (1977) describe this as a ‘confusion’ of smell and taste. If smell and taste are taken as components of flavour, this is wrong, since the location of the flavour is correctly attributed to the food.

580   Mohan Matthen attending to the components of flavour, etc. One visually examines things by looking at it from different angles and in different conditions of illumination, turning it over in one’s hands, bringing it closer to one’s eyes, etc. One listens to somebody by getting closer, cupping one’s ears, turning one’s head; one locates a sound or a smell by moving around and trying to get closer to its apparent location. One finds out the shape of something by touching and looking at it, and by manipulating it in one’s hands if it is small enough, or moving around it if not. These are examples of inter-connected purposeful activities that one uses perceptually to interrogate one’s surroundings. Let us call them modes of sensory exploration (Matthen, 2014). Sensory exploration is extremely important with regard to the externalization of perceptual content. In a momentary tactile event, if the subject is passive, she feels an event on her skin. If she actively explores something by moving her hand across it (etc.), she becomes aware of the thing’s properties (Gibson, 1962). In flavour perception, retronasal sensation is referred to food in the mouth: for example, if a little whiff of lemon is sprayed into the nasal cavity from a tube placed in the mouth, then food that is in the mouth will taste lemony. As remarked before, touch has something to do with this—active touch, that is. Similarly, as Susanna Siegel (2006) has observed, visually perceiving something as external is intimately connected with the awareness that the thing looks different when viewed at different angles—thus, the externality of visual content is tied up with one’s own motions when visually examining it. Pain is another example. Though philosophers such as Grice and Aydede (2009) treat pain simply as an experience, active exploration—stroking, prodding, palpating, etc.— endow it with objective features such as quality (burning, itching, throbbing, etc.), intensity, and location (‘It’s deep in my elbow, not high on my forearm as I initially thought’). When one has actively explored pain, one becomes aware of a disturbance in the body and the sensory properties of this entity. (Aydede does not notice that one can be mistaken about the objects of active pain perception.) Let us say that: Two types of perceptual activity A and A* are mutually certifying for property F if there is a way to use each to check up on and change the credibility of the other with regard to F.

Touching an object and looking at it are mutually certifying for shape because by looking, one can increase or decrease the credibility of determining by touch that something has such and such shape, and vice versa. Let us say further that: A set of activity-types S is a perceptual system if every member of S certifies every other for all properties detected by each.

Looking at and touching an object are mutually certifying for shape, but they are nonetheless not members of the same perceptual system because there are properties detected by each that cannot be certified by the other—colour and weight are examples. However, looking at an object while turning it over in your hands and looking at it in different conditions of illumination (for instance, by taking it over to brighter light by the window) are members of a system, because they mutually certify each other for colour, texture, shape, and every other property that either one detects. (Note that perceptual systems cannot overlap on this conception. For if activity A belonged to systems S and S*, then every

The Individuation of the Senses    581 activity in those two systems would have to reinforce A with respect to every property A detects. But this would mean that there is only one system here, not two.)11 With the idea of a perceptual system in hand, we define a perceptual modality as follows: The Perceptual System Criterion (PSC) A perceptual modality is a faculty that gathers environmental information by means of a perceptual system. Two perceptual modalities are different if their perceptual systems are different.

Let us look at flavour perception from this point of view. We saw earlier that viewed from the perspective of the transducer criterion (HTMC) it is multisensory. In other words, it is not a single sensory modality. From the Perceptual System perspective, however, it is single. It is one perceptual modality. What does one do when one puts something in one’s mouth? One chews, savours, moves it around, swallows, etc. These activities engage different transducers, and scientists are therefore inclined to say that flavour is multisensory. However, these activities constitute a system: each mutually certifies the others with regard to flavour. Applying PSC to flavour perception shows how it is at cross-purposes with HTMC, the hybrid transducer-medium criterion. PSC and HTMC play different roles in our thinking about the senses. HTMC is concerned with the different sources of information, the independent early stages of perceptual processing, and the merging of sensory information in later stages of perceptual processes. From the transducer perspective, flavour perception is not a single modality. PSC, on the other hand, offers an account of how agents perceptually explore their environments. PSC is more concordant with our intuitions because it appeals to perceiver-accessible factors.12 These criteria define different conceptions of how to differentiate the senses. I have acknowledged this by saying that PSC individuates perceptual modalities, and one could say that the transducer criterion (HMTC) is concerned with sensory modalities. This terminology is meant to acknowledge that there may be more than one way to look at modalities; it oversimplifies the matter to say simply that PSC is correct and HMTC mistaken (or vice versa). Barry Smith (2013) writes: [One] approach is to ask whether there is a single flavour sense, over and above its component senses. This approach treats the sense of flavour as a perceptual system that guides successful food selection by picking out flavours as multi-dimensional properties of things in our environment.

Smith implicitly suggests that there might be two ways of individuating the senses. I  am suggesting that one of these looks to the carriers and recipients of information—transducers and media, the other is a ‘perceptual system’. My suggestion is that perceptual systems are unified by the perceptual activity of savouring. PSC seems to imply that (momentary) passive touch and passive pain perception are not perceptual modalities. In neither does one undertake exploratory activity. In passive 11 

Matt Fulkerson observes that probing, stroking, palpating, and the like are shared by active touch and active pain perception. His remark reveals something about the individuation of perceptual activities, for in the first case these are directed to an unchanging external object, whereas the latter seeks to modify pain sensation in a controlled manner. 12  Here, I am indebted to Casey O’Callaghan for a very helpful conversation.

582   Mohan Matthen touch, one feels bodily contact, but not the properties of some external thing. (If one suddenly bumps up against something sharp, one feels a sensation on one’s skin, but not any external thing. If one runs one’s thumb across a knife-edge, one feels its sharpness.) In active pain exploration, one is able to report objective qualities of the pain. In passive pain, one suffers, but one cannot report objective properties of the pain. Passive pain and passive touch give rise to sensation (Aydede, 2009; Valerie Gray Hardcastle, Chapter 28, this volume), but they should not be regarded as perceptual modalities. PSC gives a properly nuanced account of tactile-visual sensory substitution (TVSS), and of why intuitions can diverge regarding this prosthetic modality. In TVSS, a low-resolution camera image is projected on the tongue by an array of electrotactile ‘pins’—the brighter a pixel, the greater the current applied by the corresponding pin. (The same result can be achieved by vibrotactile pins on the subject’s back.) This results in a matrix of touch experiences, which, in a remarkably short time, resolves itself into perception that is vision-like with respect to perspective and motion (Bach-y-Rita, 2004), though markedly low resolution. (See Deroy and Auvray, 2014, for a discussion of whether TVSS affords direct perception of visual properties.) TVSS activity is very much like looking, scanning, visually examining, etc. The camera that provides the image must be under the control of the subject, ‘zooming, aperture, and focus, and the correct interpretation of the effects of camera movement, such as occurs when the camera is moved from left to right and the image seems to move from right to left’ (Bach-y-Rita, 2004). What is vision-like in TVSS is the perceptual system, the suite of activities employed to take in a scene. Externalization of content too is visual: it needs perspectival changes. The TVSS subject has experiences as of visual objects and qualities by performing a visual tracking motion. This is the respect in which his experience is like vision. There is also a respect in which his experience is like touch—he can wiggle his tongue, for instance, and feel the buzz from the pins. So in this particular case, both the tactual and the visual perceptual systems can be brought to bear on the TVSS experience, giving information about quite different aspects of it. This explains why intuitions diverge.

4  Cross-species identifications One last problem. We noted earlier that honeybees and humans both have vision. But honeybees have very different ways of visually exploring their environments. Moreover, they have compound eyes and transducers that differ in kind from ours. How then can we say that both humans and honeybees have vision? I have argued elsewhere (Matthen, 2007) that homology is the right analytic instrument for judging sameness across biological taxa. In evolution, animals develop functions specialized for the niches they inhabit, and in so doing, various morphological differences develop among organs that are nonetheless adjudged the same. Thus, bird eyes might be morphologically different from human eyes, and avian visual exploration behaviours, such as scanning, may be different from the corresponding human behaviours. The reason why they are identified is that they have a common ‘ancestor’. (The ancestor-relation on organs and on perceptual exploration behaviours correlates with that relation on organisms, and on developmental pathways.) That is, there is some ancestral organ from which

The Individuation of the Senses    583 bird eyes and human eyes both descended with modifications for specialized function. Similarly, there are genetically programmed visual perceptual activities in each that are physiologically very different, but which originate in the behaviour of an ancestral organism (Ereshefsky, 2007). With regard to vision, the oldest relevant homology concerns the opsins that transduce light. These originated from proteins that facilitate photosynthesis in green algae (Deininger, Fuhrmann, and Hegemann, 2000). No other homology unites all visual systems. However, another important homology unites vertebrate eyes and distinguishes them from the compound eyes that invertebrates possess. (Vertebrate and invertebrate eyes originated independently, and have no common ancestor that is also an eye.) Presumably, perceptual activities such as scanning, circling etc., are also of relatively recent origin and divide vertebrates from invertebrates, and possibly (say) mammals from birds. Similarly, auditory systems are marked by a number of homologies, the most famous of which is not concerned with a transducer, but with the bones of the middle ear in tetrapods, which transmit energy from the eardrum. These descended from a jaw bone of fish. In general, as the example of vision shows, homologies are nested, and correspondingly there are kinds of vision—vertebrate vision as against invertebrate vision. The same holds true of the very notion of sense. The approach I  have taken here is to emphasize how the ‘traditional’ senses provide information that is held in working memory for situation-dependent action. But another approach would be to treat all information-gathering faculties together, on the grounds that they descend from very ancient systems such as bacterial phototaxis. These are simply different conceptions of sense, coextensive with different homologies. The advantage of the approach taken here is that it corresponds more closely to traditional treatments.

5 Conclusion The distinctions that are made between the sense modalities have scientific as well as everyday utility. There is, at best, partial overlap among the distinctions used for different purposes. Nevertheless, there appears to be a basic conception that scientists and ordinary folk agree upon: in humans, the senses are modes of picking up information about the world for the purposes of rational control of action and belief. The different senses correspond to differences in how information is picked up and used. Different conceptions of sense arise from emphasizing different aspects of the process.

References Auvray, M. and Spence, C. (2008). ‘The multisensory perception of flavour’. Consciousness and Cognition, 17, 1016–1031. Aydede, M. (2009). ‘Is feeling pain the perception of something?’ Journal of Philosophy, 106, 531–567. Baars, B. J. (1997). In the Theater of Consciousness: The workspace of the mind. New York: Oxford University Press.

584   Mohan Matthen Bach-y-Rita, P. (2004). ‘Tactile sensory substitution studies’. Annals of the New York Academy of Sciences, 1013, 83–91. Carey, Susan (2009). The Origin of Concepts. New York: Oxford University Press. Craig, A. D. (2002). ‘How do you feel? Interoception: The sense of the physiological condition of the body’. Nature Reviews Neuroscience, 3, 655–666. Craig, A. D. (2009). How do you feel—now? The anterior insula and human awareness. Nature Reviews Neuroscience, 10, 59–70. Dallenbach, K. M. (1939). Pain: History and present status. American Journal of Psychology, 52, 331–347. Deininger, W., Fuhrmann, M., and Hegemann, P. (2000). ‘Opsin evolution: Out of the wild green yonder?’ Trends in Genetics, 16, 158–159. Deroy, O. and Auvray, M. (2014). ‘Quasi-vision: The sensory substitution dilemma’. In D. Stokes, S. Biggs, and M. Matthen (eds), Perception and its Modalities. New York: Oxford University Press. Dworkin, B. R. (2007). ‘Interoception’. In J. T. Cacioppo, L. G. Tassinary, and G. G. Berntson (eds), Handbook of Psychophysiology (pp. 482–506) (3rd edn). New York: Cambridge University Press. Dworkin, B. R. and Dworkin, S. (1995). ‘Learning of physiological responses: II. Classical conditioning of the baroreflex’. Behavioral Neuroscience, 109, 1119–1136. Ereshefsky, Marc (2007). ‘Psychological categories as homologies: Lessons from ethology’. Biology and Philosophy, 22, 659–672. Evans, Gareth (1985). ‘Molyneux’s Question’. In Gareth Evans, Collected Papers. Oxford: Clarendon Press, 364–399. Fulkerson, M. (2011). ‘The unity of haptic touch’. Philosophical Psychology, 24, 493–516. Fulkerson, M. (2014). ‘What counts as touch’. In D. Stokes, S. Biggs, and M. Matthen (eds), Perception and its Modalities. New York: Oxford University Press. Gallace, A., Tan, H. Z., and Spence, C. (2008). ‘Can tactile stimuli be subitised? An unresolved controversy within the literature on numerosity judgments’. Perception, 37, 782–800. Gallistel, C. R. (1990). The Organization of Learning. Cambridge, MA:  Bradford Books, MIT Press. Gallistel, C. R. and King, A. P. (2010). Memory and the Computational Brain: Why cognitive neuroscience will transform neuroscience. Chichester: Wiley-Blackwell. Gelman, R. and Gallistel, C. R. (1978). The Child’s Understanding of Number. Cambridge, MA: Harvard University Press. Gibson, J. J. (1962). ‘Observations on active touch’. Psychological Review, 69, 477–491. Gibson, J. J. (1977). ‘The theory of affordances’. In R. Shaw and J. Bransford (eds), Perceiving, Acting, and Knowing: Toward an ecological psychology (pp. 67–82). Hillsdale, NJ: Lawrence Erlbaum. Goodale, M. and Milner, D. (2004). Sight Unseen: An exploration of conscious and unconscious vision. Oxford: Oxford University Press. Green, K. P. and Kuhl, P. K. (1991). ‘Integral processing of visual place and auditory voicing information during phonetic perception’. Journal of Experimental Psychology: Human Perception and Performance, 278–288. Grice, H. P. (1962). ‘Some remarks about the senses’. In R. J. Butler (ed.), Analytic Philosophy, First Series. Oxford: Basil Blackwell. Heil, J. (1983). Perception and Cognition. Berkeley and Los Angeles: University of California Press.

The Individuation of the Senses    585 Hughes, H. (1999). Sensory Exotica:  A  world beyond human experience. Cambridge, MA: MIT Press. James, W. (1890). Principles of Psychology. New York: Henry Holt. Jones, J. A. and Munhall, K. G. (1997). ‘The effects of separating auditory and visual sources on audiovisual integration of speech’. Canadian Acoustics, 25, 13–19. Keeley, B. L. (2002). ‘Making sense of the senses: Individuating modalities in humans and other animals’. Journal of Philosophy, 99, 5–28. Kovács, Á., Téglás, M. E., and Endress, D. A. (2010). ‘The social sense: Susceptibility of others’ beliefs in human infants and adults’. Science, 330, 1830–1834. Lederman, S. J. and Klatzky, R. L. (2009). ‘Haptic perception: A tutorial’. Attention, Perception, & Psychophysics, 71, 1439–1459. McGlone, F. P. and Spence, C. (2010). ‘The cutaneous senses: Touch, temperature, pain/itch, and pleasure’. Neuroscience & Biobehavioural Reviews, 34, 145–147. McGurk, H. and MacDonald, J. (1976). ‘Hearing lips and seeing voices’. Nature, 264, 746–748. McKinley, M. J. and Johnson, A. K. (2004). ‘The physiological regulation of thirst and fluid intake’. Physiology, 19, 1–6. Macpherson, F. (2011a). The Senses: Classical and contemporary philosophical perspectives. New York: Oxford University Press. Macpherson, F. (2011b). ‘Introduction: Individuating the senses’. In Macpherson, 2011a, 3–43. Martin, M. (1992). ‘Sight and touch’. In T. Crane (ed.), The Contents of Experience (pp. 196–215). New York: Cambridge University Press. Matthen, M. (2005). Seeing, Doing, and Knowing: A philosophical theory of sense perception. Oxford: Clarendon Press. Matthen, M. (2007). ‘Defining vision: What homology thinking contributes’. Biology and Philosophy, 22, 675–689. Matthen, M. (2014). ‘How to be sure: Sensory exploration and empirical certainty’. Philosophy and Phenomenological Research, 87, 38–69. DOI: 10.1111/j.1933-1592.2011.00548.x. Meredith, M. (2001). ‘Human vomeronasal organ function: A critical review of best and worst cases’. Chemical Senses, 26, 433–445. Munakata, Y. et al. (2011). ‘A unified framework for inhibitory control’. Trends in Cognitive Sciences, 15, 453–459. Murphy, C., Cain, W. S., and Bartoshuk, L. M. (1977). ‘Mutual action of taste and olfaction’. Sensory Processes, 1, 204–211. Nudds, M. (2004). ‘The significance of the senses’. Proceedings of the Aristotelian Society, 104, 31–51. Pavlov, I. P. (1927). Conditioned Reflexes: An investigation of the physiological activity of the cerebral cortex. London: Oxford University Press. (Republished by Dover in 1960.) Pollock, J. (1974). Knowledge and Justification. Princeton: Princeton University Press. Richardson, Louise (2013). ‘Flavour, taste, and smell’. Mind and Language, 28, 322–341. Ross, P. W. (2008). ‘Common sense about qualities and sense’. Philosophical Studies, 138, 299–316. Shannon, C. (1948). ‘The mathematical theory of communication’. Bell System Technical Journal, 27, 379–423, 623–656. Siegel, S. (2006). ‘Subject and object in the contents of visual experience’. Philosophical Review, 115, 355–388. Sorabji, R. (1971). ‘Aristotle on demarcating the five senses’. Philosophical Review, 80, 55–79.

586   Mohan Matthen Smith, B. C. (2013). ‘Taste, philosophical perspectives’. In H. Pashler (ed.), Encyclopedia of the Mind (pp. 731–734). Thousand Oaks, CA: Sage. Spence, C., Smith, B., and Auvray, M. (2014). ‘Confusing tastes and flavours’. In D. Stokes, S. Biggs, and M. Matthen (eds), Perception and its Modalities. New York: Oxford University Press. Treisman, A. and Gelade, G. (1996). ‘A feature integration theory of attention’. Cognitive Psychology, 12, 97–136. Wittmann, M. and van Wassenhove, V. (2009). ‘The experience of time: Neural mechanisms and the interplay of emotion, cognition and embodiment’. Philosophical Transactions of the Royal Society of London B, 1809–1813.

Chapter 31

Perceptua l At ten tion John Campbell

1  (Knowledge and action) vs. appearances We can make a broad contrast between two sets of philosophical questions relating to perceptual attention. One set has to do with the impact of attention on knowledge and action. The other has to do with the impact of attention on perceptual appearances: (1) How does conscious attention relate to our knowledge of our surroundings, and our ability to act on our surroundings? Is conscious attention implicated in our knowledge of what the objects and properties around us are? That is, is conscious attention implicated in our understanding of concepts of the objects and properties around us? Is conscious attention to a scene required for perception to provide one with propositional knowledge of the scene? Is conscious attention required for intentional action? (2) How are we to characterize the difference that conscious attention makes to perceptual experience? Should we, for example, think in terms of qualia distinctive of attention, a particular representational structure to the experience, or something else? Is attention a sui generis characteristic of perceptual experience, or is the difference it makes a matter of impacting characteristics such as perceived colour or contrast? The two questions are, of course, connected. James Stazicker (2011) develops the idea that an increase in the spatial resolution of vision can be constitutive of visual attention: ‘the selective “focalization, concentration of consciousness” [of which William James spoke] sometimes takes the form of an increase in the determinacy of conscious vision’ (180). On the face of it, this remark addresses question (2). But of course, it also has implications for the impact that attention has on knowledge. If attention is sometimes constitutively an increase in spatial resolution, that will enhance knowledge in cases where increased spatial resolution helps, and diminish it in cognitive tasks where an increase in spatial resolution is detrimental to performance. Yet even if there were no impact of attention on spatial resolution (or colour saturation and so on), there would still be a connection between conscious attention and

588   John Campbell the generation of knowledge on the basis of perception. It would still not be enough for knowledge that an object and its properties were there somewhere in your visual field, staring you in the face. You would still have to attend to that object and its characteristics. The connection I have in mind has to do with attention as a phenomenon of consciousness (and I will focus particularly on visual experience). It may be possible for blindsighted subjects to exhibit attention in the blind field (for discussion, see Kentridge, 2011; Mole, 2008). But unconscious attention does not have the same compelling link to knowledge and intentional action that ordinary conscious attention does. Consciously attending to the colour of an object in front of you can generate non-inferential perceptual knowledge that the thing is (say) red. And an ability to consciously attend to the colours of things seems partly constitutive of grasp of colour concepts. Finally, conscious attention to an object seems implicated in your ability to act intentionally on that thing, if you are going to pick it up, for example. None of these points seem to apply to the kind of attention that may be available to the blindsighted, in the blind field. Blindsighted subjects, no matter how many trials they have performed successfully, and how much knowledge they have of their trial record, are always said to be ‘only guessing’. They don’t have non-inferential knowledge of what is going on in the blind field. A blindsighted subject who knows their own flawless track record of correct guessing could infer that the thing is there; but even though this could yield knowledge, it’s different to the non-inferential knowledge that ordinary perception provides. Of course, you might argue that there are cases in which the environment impacts on you without your consciously attending, in such a way that you nonetheless form a reliably correct belief as to what is going on. Even if we are willing to call this kind of case ‘knowledge’, however, there is no particular reason to suppose there is any constitutive demand for non-conscious attention to play a role; all that is happening is that, one way or another, the environment is impacting on your belief formation. The distinctive role for attention in knowledge and action seems to be a role specifically for conscious attention (cf. also Roessler, 2009). These remarks relate to attention, as we ordinarily understand it:  a relation between a person and some aspect of the environment. You can attend to an ongoing process, as when you watch a game of soccer, to an individual object, as when you attend to a particular person, to a feature, as when you look out for the red targets, or to a location, as when you look at the doorway through which you expect your visitor to come. In contemporary psychology and philosophy much of the discussion of attention moves seamlessly between two levels: thinking of attention as a conscious relation between the person and the environment, and thinking of attention in terms of the structure of information processing in the brain. Attention at the personal level is presumably what James (1890/1950) had in mind when he said, ‘everyone knows what attention is’. In similar vein, you might say, ‘Everyone knows what colour vision is.’ Both claims, properly understood, leave it open that there is much more to be said about underlying brain processes related to attention, or to colour vision, and that many of the most philosophically arresting problems may have to do with the relations between the two levels. James thought of attention as fundamentally a matter of selection. From Donald Broadbent (1958) to the present day, psychologists have thought of this selection as reflecting some kind of resource limitation in the visual system in the brain, some kind of ceiling on the visual system’s processing capacity. On this view, the role of attention in knowledge or intentional action would presumably have to be understood in terms of the need for

Perceptual Attention   589 these limited resources to be allocated to the stimulus of which you have knowledge or on which you act. It’s possible to try to reverse this direction of explanation. It’s possible, for example, to argue that the generation of knowledge, or intentional action, in themselves require selection. The arm can only move in one direction at a time, for example, so in order for current visual perception to control what direction the arm moves in, there will have to be selection of one visual parameter rather than another to direct the movement of the arm. You might argue that this kind of consideration, rather than any processing limitation of the visual system, is what explains the role of selective attention in knowledge and action (see Allport (1989: 648) and Wu (2011) for exploration of the line of thought). The trouble with this line of thought is that (a) as Allport remarks, it is not obvious that intentional action as such really imposes much limitation on parameter selection for the control of action (for example, in moving your arm to reach the diamond protected by the beams of an alarm system, you may have to be monitoring your arm location with respect simultaneously to many spatial parameters), and (b) there seem to be quite definite limitations on selective attention that aren’t predicted merely by its role in the generation of knowledge and action. For example, consider the basic finding by Liqiang Huang and Harold Pashler (2007) that assessment of the symmetry of a variously coloured display has to proceed by selecting and assessing for symmetry, one at a time, the regions of each particular colour. This isn’t predicted merely by the demand that vision should generate knowledge and intentional action. It seems rather to reflect some deeper processing limitation on the brain. Notice, incidentally, that the distinction between personal-level and informationprocessing-level characterizations of attention also plays a significant role in the debate over the relative priority of objects and location in attention. Defined in personal-level terms, as an issue about whether people attend to objects or to places, there is of course no question but that people can attend to either. The more difficult issue is computational: does attention to objects, conceived as a sub-personal selection of information, depend on a prior selection of places? Or is selection of objects computationally more fundamental than selection of places? Often this question is simply postponed. For example, in John Duncan’s early work on the object as the unit of attention, the idea that the visual system represents objects is simply taken for granted, and no account is given of how it happens that the visual system is representing objects in the first place. Presumably Duncan would not have wanted thereby to be committed to the idea that no computational account can be given of how it comes about that objects are being represented by the visual system, and his experimental results do not of themselves show whether the representation of an object in the visual system depends on a prior selection of information from a particular location or locations.

2  Selection vs. Access: Where does consciousness fit in? In a trenchant remark, Huang and Pashler (2007) say: ‘Visual attention, in its most fundamental sense, is a selective visual process that governs access to consciousness’ (599). Huang (2010) makes explicit that the kind of ‘consciousness’ that is in question here is not

590   John Campbell what Block (1995) calls ‘phenomenal’ consciousness, the ‘what it is like’ of an experience, but rather ‘access’ consciousness, that is ‘being poised for the rational control of belief and action’. Where does phenomenal consciousness fit into this picture of attention? I shall suggest that the natural place for phenomenal visual awareness is as providing the field from which stimuli are selected so their properties can be accessed for further processing. Something like this is anyhow the commonsense picture. We ordinarily think of visual experience as providing a rich, stable background from which we select various stimuli to which to attend. Let us consider the two aspects to attention, selection and access. You could select a stimulus on the basis of some characteristic of it, such as colour. That does not of itself mean that you must access that very characteristic of the stimulus. Consider a tiger out hunting. It might discriminate its prey from its background by means of its colour (even using somewhat rudimentary colour vision). Colour, that is, might be involved in selection. It does not follow that the tiger must access the colour of the object. There may be nothing it can do with information specifically about the colour of the thing. The distinction between selection and access can be explained in terms of there being two different dimensions along which the difficulty of a task can be varied. For example, consider a case in which you are confronted by a display of variously coloured letters of the alphabet. Suppose your task is to report which letter or letters are coloured red. Here your task could be made more difficult in two ways. (1) Your task can be made more difficult by including many letters which have colours similar to red, such as pink or orange, rather than only letters that are either red or green. That makes it harder to select a target as opposed to a non-target letter (red vs. pink, for example). Alternatively, (2) your task can be made more difficult by including many different letters all coloured red, all of which have to be reported. In this case, your task has been made more difficult, not by making it harder to select a target, but by giving you more reporting to do. You have to access the shapes of more targets (cf. Huang, 2010). This way of explaining the distinction between selection and access characterizes it at the level of attention as a relation between the person and the stimulus. We can regard that distinction as grounded in a further distinction between selection and access at the level of data structures. Huang and Pashler (2007) articulate the basic points here in terms of their notion of a Boolean map. The ‘map’ here is a representation of the area of the visual field, and every location on the map is either selected or not. So we can talk about the (total) ‘selected region’ on a Boolean map, which may possibly be discontinuous. Selection, on this account, is a matter of selecting a particular (possibly discontinuous) region. Access is a matter of ‘labelling’ the selected region, making explicit which features are to be found at which locations. Both selection and access are subject to significant restrictions. For example, one strategy for selecting locations is to select on the basis of a feature: you might select all and only the red regions in your visual field, for example. You can’t select on the basis of more than one feature at a time, though having selected a region on the basis of one feature, you can proceed to generate another Boolean map on the basis of another feature, and then take the union or intersection of these two regions. Equally, there are significant limitations on access. You can’t access more than one feature of your selected region at a time (that is, you can access only one colour for your selected region at any time). The idea is that such

Perceptual Attention   591 characteristics of the underlying processing in the visual system explain our strengths and limitations in performing visual tasks, defined in terms of relations between the subject and the stimulus. To take a simple example, suppose subjects are presented with a multicoloured patterned display and asked if the display is symmetrical in the layout of colour. It turns out that the more colours in the display, the longer subjects need to perform the task. The proposal is that this is explained by the way in which the visual system in the brain constructs labelled Boolean maps. Phenomenal consciousness is the paradigmatic personal-level phenomenon: nothing is a personal-level phenomenon if phenomenal consciousness is not. How should we think of the relation between the kind of underlying data structure we have just been discussing and visual experience? One answer would be that we should think of the content of phenomenal experience being explained as the outcome of the process of selection and access. There would, of course, be a question about why phenomenal experience is generated at all by this process. But we could view this account as addressing a somewhat different problem. Given that we have a subject in whom phenomenal experience is being generated, why is one visual experience rather than another being brought about? It’s to that question that you might think this account provides an answer. The trouble is that it does not seem to be the right answer. Huang and Pashler make a convincing case that our access to colours is serial: only one colour at a time in a perceived array can be labelled. But our ordinary visual experience seems to be of a simultaneously given, richly detailed, stable array of objects and colours. This is one version of the form that the mind–body problem takes in the recent literature on visual attention. The difficulty is that as we study the visual system in the brain, we find only incomplete, transient representations of aspects of our surroundings. We find no assembly of detailed, stable representations that could ground our apparent visual experience. One, perhaps the most common, reaction to this point is to suppose that we are under a kind of persistent and pervasive illusion about the nature of our visual experience. It seems to be richly detailed and stable, but it is not (see, e.g., Cohen et al., 2012). To bring out the idea here, consider a thought experiment. Suppose you are fitted with an eye-tracker, and you are looking at a large, blank white wall. Suppose that a projector is linked to the eyetracker, and it projects onto the wall a small circle of letters around your fixation point. So what an observer sees when they walk into the room is you seated on a chair moving your head around while a small circle of letters moves around on the wall. To you, though, as the subject of this experiment, it seems as though you are facing a vast, stable array of letters on the wall, reaching from floor to ceiling and the full length of the wall, though of course you can only select and access—focus on—the characteristics of some small group of stimuli at a time. In this kind of case, it really seems that you are being subjected to some kind of illusion about the character of your visual experience. The suggestion is now that this is not a special or unusual case. In ordinary life we are continually subject to just that illusion about the character of our ordinary visual experience. There seem to be simultaneously present in our experience a rich, stable array of properties and objects when in fact there are not. This is a perplexing conclusion. Isn’t there any way we could rescue the idea that our ordinary experience is usually detailed and stable in the way it seems to be? The trouble is that this is impossible so long as we think that the nature of our visual experience must

592   John Campbell be entirely grounded in the characteristics of the brain. If we think of the relevant brain characteristics as representational, the trouble is that there aren’t the detailed, stable representations to be found. If we think in terms of a kind of ‘mental paint’ that presents characteristics of the brain to us in a ‘phenomenal’ way, the trouble is that there isn’t enough mental paint to go around, there isn’t enough mental paint to constitute that detailed, stable picture. The only way in which we can sustain our commonsense conception of visual experience is to be externalist about experience: to acknowledge that the physical grounding of experience is found not just in the brain, but in the relation of the brain to the external environment itself (cf. Noë, 2002). After all, the external array of objects and colours and so on is there, detailed and stable. If we think of the external objects and colours and so on as literally constituting the phenomenal content of visual experience, then we can take our ordinary visual experience at face value. If we think of visual experience in this way, we can view it as providing the basis for the possibility of selection and access, rather than as somehow being the sparse outcome of the process of selection and access. To see how on this picture, experience would relate to attention, consider the experience of someone looking at one of the standard Ishihara tests for colour vision: a display of variously dappled and brindled dots, in which a figure ‘5’ is differentiated from the rest of the display only by the colours of the dots. If you are able to report the presence of the ‘5’, then you must have selected the ‘5’ from the rest of the display, and the only way you can have done that is by way of the colour of the thing. But you may be quite unable to access the colour of the object. Perception of colour may be merely instrumental for perception of the object. This may indeed be the situation of young children and certain animals, who can use colour vision perfectly well to discriminate objects from their backgrounds, even if they are not able to give verbal reports of colour, engage in colour sorting or colour induction, and so on. Now consider whether someone who has visual experience of the ‘5’ must have visual experience of its colour. It seems to speak for itself that we can’t make anything of the idea that there could be visual experience of the ‘5’ without visual experience of its colour. The only thing that differentiates the ‘5’ from its background is colour, so how could there be visual experience of the ‘5’ unless there is also visual experience of the colour? Without experience of the colour, there might be blindsight-style guessing as to the presence of the ‘5’ in an apparently uniformly blank display. But that wouldn’t be a case in which there was experience of the object, the ‘5’ itself. As we have seen, though, experience of the ‘5’ seems to be possible even in the absence of any capacity to access the colours of things. The situation seems to be that if you are to have conscious experience of an object whose characteristics you can access, you must be able to select that object on the basis of properties that are making a difference to your visual experience. Experience of those characteristics is presupposed by your ability to attend to the object: to select that thing and access its characteristics. The detailed, stable world of visual experience is not a product of attentional selection and access: it is a background precondition of selection and access, if selection and access are to constitute attention to experienced objects. These points bear on the interpretation of inattentional blindness and change blindness (see, e.g., Simons and Chabris, 1999). In these cases, aspects of what we might take to be the detailed, stable world of visual experience are simply not noticed by subjects, and change

Perceptual Attention   593 in them is simply not noticed by subjects. This is really startling if you do not distinguish between: (a) the detailed, stable world of phenomenal visual experience, and (b) the subject’s access to properties of various regions and objects in the perceived scene. As I have said, access to colour seems to be serial. You can access only one colour property at a time of a region or object in your surroundings. As Huang and Pashler (2007) argue, this serial access to colour, and other observable properties of your surroundings, may be at the root of inattentional blindness and change blindness. Since spotting visible features of your environment, or identifying specific changes in them, can be done only on the basis of a serial scan, there is always the possibility of unnoticed features or changes inside your visual field that are not accessed. If you think of visual experience as the upshot of such a serial scan, then you have to explain what is going on when these features that seem so visible are present yet unnoticed right in front of your nose. These cases seem to write large the idea that our impressions about the detail and stability of our current visual experience are somehow entirely illusory. We can however give a straightforward resolution if we think of the detailed, stable content of visual experience in externalist terms, as simultaneously present, and available as the basis for selection and access.

3 Transparency I want now to look at the role of conscious attention in generating propositional knowledge and grasp of our concepts of the observable objects and properties around us. As I said, it seems immediately compelling that in order to grasp the concepts of various colours, for example, you have to be able to attend to the colours of things. And in order to be able to refer to the things you perceive, you have to be able to attend to them. But developing this point requires some further reflection on the notion of visual experience itself. G. E. Moore famously remarked that the visual experience of the blueness of an object seems to be ‘transparent’: ‘we look through it and see nothing but the blue’ (Moore, 1903). Gilbert Harman wrote: When you see a tree, you do not experience any features as intrinsic features of your experience. Look at a tree and try to turn your attention to intrinsic features of your visual experience. I predict that the only features there to turn your attention to will be features of the presented tree. (Harman, 1990: 39)

This is evidently a forceful idea, and the implication of it is that the notion of visual experience is not, as one might think, the concept of something immediately given to us. It is, rather, a theoretical concept, like ‘electron’, or ‘gluon’. You might compare the notion of ‘light’. It’s natural to suppose that light is the most immediately evident characteristic of the visual world. But of course, in general we do not experience light itself: all we see are the various objects around us and their properties, the blueness of the blue thing, the tree and its branches, and so on. Light is something postulated to explain why we can see sometimes but not always, why there is sometimes variation in how things look even though

594   John Campbell they haven’t changed, and so on. Similarly, visual experience is postulated to explain our epistemic access to one region rather than another; it is not something we encounter directly. In characterizing attention as a phenomenon of consciousness, then, we have to be guided by our interpretation of the explanatory role of conscious attention. What does it explain? So far I’ve suggested that it explains our capacity for non-inferential propositional perceptual knowledge, our grasp of concepts of perceptible properties and things, and our capacity for immediate intentional action on things we see. A simple way to see the role of conscious attention in the generation of propositional knowledge is to reflect on an example of Austin’s (1962). Suppose you are walking in the woods and come upon evidence of the imminence of a pig: droppings and so on. Then, in a clearing, you come right upon it. You walk all round it. You prod it with a stick. At this point, Austin says, it does not seem right to say that you have evidence of the presence of a pig. Your encounter is, and should be, completely decisive in making up your mind as to whether a pig is present. Now the point about decisiveness here has to do with both consciousness and attention. If you were blindsighted and had merely a blindsighter’s hunch that there is a pig in the blind field, that would not have the epistemic authority of your experience of the thing. And if the pig were merely somewhere in the area of your visual experience, but you didn’t attend to it, you might have a sense of the presence of a pig, but that again wouldn’t have the epistemic authority of your experience when you do consciously select the pig from its surroundings and visually access what kind of animal it is. One way of explaining the decisive force of conscious attention to the thing here is to appeal to the externalist picture of visual experience that, as we saw, seems needed in any case if we are to do justice to its detailed, stable nature. If we can think of conscious attention to the object as a relation between the subject and the external scene, we can understand how conscious attention to just this aspect of the external scene could constitute evidence that confers a probability of one on the hypothesis that there is a pig there. Of course, subjects might not have perfect insight into whether they do have such evidence, so the correct level of subjective confidence for the subject to have in this hypothesis might be a bit less than one. It may still be comfortably high enough for it to be right for one’s conscious attention to the thing to outweigh any other evidence one has (cf. Kennedy, 2010). As I said, another theoretical role that we need the notion of conscious attention to play is in explaining our grasp of concepts of the objects and properties around us (cf. Mole, 2011, for discussion). The quickest way to see this point is to recollect two classical problems for a functionalist analysis of the mind. One is that it has trouble explaining the normativity of propositional attitudes: the existence of standards of rightness and wrongness for the propositions we understand. The other is (what is usually called) the problem of qualia. Visual experience seems to have qualitative aspects that are hard to explain in internal functional terms. I am suggesting that these two problems are related. What provides us with our knowledge of the things and properties around us, the things to which we refer, is, in the first instance, our perceptual experience of our surroundings. But for knowledge of the references of concepts, it is not enough that these things or properties be idly there, somewhere in your visual field. Nor is it even enough, for knowledge of a property, that you be able to use it as a basis on which to select stimuli to attend to. As we saw, you may be able to use the colour of an object to differentiate it from its background,

Perceptual Attention   595 without having any capacity at all to make explicit what the colours of things are. What is required, for knowledge of which properties the colour concepts stand for, is the ability to visually access the colours of the objects around you. Similarly, to understand a demonstrative concept referring to an object you perceive, it is not enough merely that the object be there in your visual field: you must be able to access its various characteristics. If conscious attention is to play this explanatory role, in providing knowledge of the references of concepts referring to the things around us, it is hard to see how it can be right to conceive of conscious attention in internalist terms. If we were talking merely about ‘the conscious experience in the head’, as it’s sometimes put, it’s hard to see how that could do any work in explaining how we know what concepts refer to. But that shifts if we keep our hold on the way we ordinarily think of conscious attention, as a relation between the person and the scene observed. Notice, incidentally, that an immediate implication of transparency is that the ‘problem of hallucination’ that has gripped philosophers of perception since Descartes is not a real issue. The philosopher’s idea of a hallucination (as opposed to the analysis of e.g. phenomena relating to schizophrenia or drug use) is the idea of a mental state that is intrinsically just like seeing something, but without the external world being there. But the implication of transparency is that we do not have the conceptual materials even to formulate the idea of such a state. To describe the experience I have when I am seeing something, what I have to do is describe what I am seeing. The point about hallucination is this: you are asked to imagine a visual experience that is just like seeing an airport, only without the airport being there. But what is left of seeing the airport if the airport is not there? The philosopher’s answer has been: ‘the visual experience’. But the point about transparency is that your ordinary introspective knowledge of the experience of seeing an airport gives you no knowledge of any such internal state. Introspection of the experience of seeing an airport amounts merely to inspection of the airport itself. Subtract the airport, and there is nothing left to inspect. The natural idea is: well, I can imagine having an experience that is just like seeing an airport, only without the airport being there. But what exactly do you do in this imaginative exercise, when you imagine what it is like to see an airport? All there is for you to do is to imagine an airport. Within the context of your imagining the airport, you can suppose further that you are seeing this thing. But that exercise does not amount to an understanding of what it is to have the experience of seeing an airport, only without an airport being there. That makes no sense. It is like saying: imagine a state that is just like being a mile away from a hotel, only there is no hotel. Subtract the hotel, and there is not enough left of the original state for you to have a coherent imaginative exercise to perform. The natural contrast here is between visual experience and a sensation such as pain. You can attend to the features of your own headache—is it sharp or dull, pulsating or steady?— without attending to any aspect of the non-mental world. Now it is often thought that the nature of pain must be immediately given to one in introspection. (There is, therefore, a problem about how it can be that pain is a physical state (cf. Kripke, 1980).) Philosophers have usually thought of knowledge of your own visual experience on the model of knowledge of your own pain. That is, they assume that the nature of visual experience is immediately given to you in introspection, without the need to attend to anything non-mental. (And that there is, therefore, a similar problem about how visual experiences can be physical states.)

596   John Campbell Now in the case of pain, it seems evidently imaginable that there could be a state just like one’s current headache but which has a different distal cause. For example, suppose stress at work caused my current headache. It seems possible that there could be a state that is intrinsically just like my current headache but which had a different cause, being brought about by, say, dehydration. Just so, people suppose, I could have a state that is intrinsically just like my current visual experience of an apple, but brought about by quite different causes than my current visual experience. But the implication of transparency is that there is no such introspective knowledge of the nature of my current seeing of the apple, independent of my knowledge of the apple. Somehow it has seemed compelling to many philosophers to suppose that the nature of visual experience is immediately given to the subject in introspection. You think: ‘That visual experience, (pointing at the head), that I could have whatever was going on in the external world’. The nature of this visual experience is then thought to be evidently distinct from the nature of the environment, for after all the nature of the environment is not given immediately to the subject in introspection. But Moore’s point implies that there is no such thing as being given the intrinsic nature of one’s current visual experience of seeing, independently of one’s knowledge of one’s surroundings.

4  The philosophical interest of the binding problem Suppose we are interested in the role of vision in the generation of knowledge and intentional action. We have two kinds of reasonably well-understood vocabulary for characterizing the relevant working of vision. These are:

(1) Terms for the characteristics of the objects and properties around us: their colours and shapes, their sorts and behaviours, and so on. (2) Terms characterizing the information processing taking place in the visual system in the brain—edge-detection, symmetry analysis, and so on. Of course there are other aspects to vision—affective and aesthetic, for example—that have to come into a fuller description. But for a basic description of the role of vision in generating knowledge and action, these will do. Philosophers interested in the characterization of visual experience and its role in knowledge and action have often introduced proprietary technical terms. Early modern philosophers talked about ‘ideas’, sense-datum theorists talked about ‘sense-data’, of course, and many contemporary theorists talk about ‘qualia’ or ‘representational contents’, with the latter sometimes being modelled on the talk about information processing one finds in scientific psychology. I think it is not wildly controversial that each of these ways of going beyond type (1) and type (2) characterizations of visual experience has proven quite difficult to interpret. To have a vocabulary we are confident we understand, we should stick so far as we can to descriptions in terms of type (1) and type (2). When we are trying to specify the role of vision in generating knowledge and action, we naturally reach for type (1) terms, the terms you use to specify what it is that someone is

Perceptual Attention   597 seeing, what it is that they’re attending to, and so on. Type (2) terms have often been seen as somewhat irrelevant to purely philosophical projects. But I do want to suggest that we can enrich our ordinary understanding of visual experience and visual attention as relations between the person and the external scene, by looking at the relations between type (1) and type (2) characterizations. Consider, for example, the description of attention in terms of ‘selection’ and ‘access’ that I mentioned at the start of section 2. As we saw, that is explained by Huang and Pashler in terms of the underlying information processing in the brain. It is a matter of a particular feature being used to define a particular region. For example, the system might select all and only the red regions in the current visual field, by appealing to an underlying structure of Treismanian feature maps to identify the red regions—which let us suppose is the area of a figure ‘5’. Then, having made that selection of a particular region, the system might again use that same underlying architecture of Treismanian feature maps to explicitly label various properties of that region, such as shape, colour, orientation, and so on, one label per feature dimension. What I want to suggest is that finding this structure at the level of information processing in the brain has implications for the articulation of causal structure in visual experience, considered as a relation between the subject and the external object, the figure ‘5’. We can go further into the structure of the experience than merely saying that the subject sees the object and sees that it’s a ‘5’. We can specify that the experienced redness of the ‘5’ is what’s causally explaining the subject’s experience of the object, and that this in turn is what’s causally responsible for it being possible for the subject to see that it’s a ‘5’, to see that it’s tilted, and so on. The working assumption is that attention as a relation between the person and the scene, and attention at the information-processing level, are related. When someone is attending to a person giving a talk, to a doorway, or to an overheard conversation, there is, at the level of information processing in the brain, some selection of information for further processing, or some biasing of the information processing that is going on. Moreover, the causal structures one finds at the level of brain processing are mediating causal structures at the level of the person’s conscious attention to the scene. Obviously, incidentally, there is no question of simply being able to identify (a) a relation between the subject and the environment with (b) any particular property that the brain could have no matter what is going on in the environment (any more than you could identity the relation ‘being a yard away from’ with a brain state). The example of being able to consciously experience the ‘5’ because one is able to select it on the basis of its experienced colour, and consequently able to visually determine the properties of that object, provides the entry point to a way of thinking about the phenomenology of visual experience that is made possible by grasping its relation to the psychology of attention. One further example is provided by Shimon Ullman’s (2000) notion of a ‘visual routine’:  high-level computations that the visual system can wheel out on demand to find whether, for example, two points are located on the same line, whether one complex figure encloses an object, whether there is an object in between two others, and so on. The visual system does not compute these things automatically and in parallel for everything currently in the visual field. Rather, it applies these routines on demand to selected visual information. This point tells us more about the causal structure of visual experience of a scene. For example, if you see that your target has a clear path to the door, we can say that

598   John Campbell this depends on your having visual experience of some characteristic of the target that you are using to differentiate the target from the background, and your interrogating the scene to find whether there is a clear path from the target to the door. But we can go deeper into the example than this. ‘Having a clear path to the door’ is not an experienced characteristic on which basis you could select an object, because the visual system could not select an object on that basis; it is not a characteristic that could be used to delineate an object as figure from ground. Contrast the case of colour: the colour of an object can be used to select it as figure from ground, and it is further a characteristic that the visual system can access, so that the person can see that the object is one colour or another. This articulates the sense in which these geometrical properties of the scene are ‘high-level’ and colour properties are ‘low-level’: experience of the latter, but not of the former, can be used as a basis for attentional selection. This is, of course, only the beginning of a way of articulating the causal structure of the phenomenology of conscious attention, by seeing it as mediated by an underlying level of information processing. Consider a classical philosophical problem, the characterization of perceptual ‘modes of presentation’ of an object. Frege’s idea was that if you identify the same object in the same way twice, it is not informative to be told that it is the same thing; whereas if you identify the same object in two different ways, it is informative to be told that it is the same thing. But what a ‘way of identifying’ an object is remained tantalizingly elusive. Frege gave some perceptual metaphors and analogies, including the idea of a ‘mode of presentation’, but it’s not straightforward to see how they should be made explicit (Frege 1892/1952). We can see how to make progress with the idea of a perceptual ‘mode of presentation’, though, if we ask for the conditions under which the visual system can access multiple properties of a single stimulus. As a first shot—and it is only a first shot—we might say that if the stimulus is selected on the same basis, then different properties are accessed as properties of the same thing, and if stimulus A and stimulus B (possibly identitical) are selected on different bases, then different properties are accessed as properties of different things. At the level of attention as an experiential relation between the subject and the thing, this means that we have the same ‘mode of presentation’ if it is the same properties being used to delineate the object as figure from ground. If it is colour that is being used to delineate the figure ‘5’ in a scene, then that is what individuates the mode of presentation (even if the subject can’t access the colour of the thing). If you select what happens to be the same figure on a different basis, such as a noise it is making or a tactual characteristic of it, then you have a different mode of presentation of the thing. These are only preliminary remarks on how we can go about giving a fine-grained analysis of the content of visual experience, one that goes beyond a mere listing of the objects and properties involved. We can characterize the causal structure of your experience of the objects and properties in a scene. This causal structure can be quite rich. Consider the family of questions that are studied under that head of ‘the binding problem’, for example. In particular, suppose we focus on what Treisman called ‘feature binding’ (Treisman and Gelade, 1980). There are (at least) two sets of questions here. One has to do with how it is that we can access multiple properties of a single object. The other has to do with what happens when we select an object on the basis of more than one feature. Without going into the particular theories that have been proposed as to what happens here (theories in terms of ‘guided search’ and salience

Perceptual Attention   599 maps, for example, or ‘subset search’ theories (Wolfe, 1994; Egeth, Virzi, and Garbart, 1984). I want to point out that on the approach I am developing here, these theories will have definite implications for the causal structure of our conscious attention to the scene before us. They will give us an account of what is distinctive of the phenomenology of one kind of search rather than another. And this will be an account of the phenomenology whose correctness could not have been recognized merely by subjective or introspective first-person reflection.

5 Appearances I want finally to contrast a different approach to the phenomenology of attention. I said earlier that in the empirical literature, one question has been whether attention alters the appearance of a stimulus by changing apparent characteristics such as colour, contrast, or the spatial resolution of the perception. William James said that attention alters the appearance of a stimulus: ‘Every artist knows he can make a scene before his eyes appear warmer or colder in colour, according to the way he sets his attention. If for warm, he soon begins to see the colour red start out of everything; if for cold, blue’ (James, 1890/1950: 425). Ned Block (2010) argues that these differences in phenomenology, related to variations in attention, should, in some cases at least, be characterized in terms of a magnitude that he calls ‘mental paint’. This sounds like a difficult technical notion that may imply that we are making some progress with the subject. The metaphor is that it is something that stands to perception somewhat as ordinary paint stands to a portrait, for example. On the other hand, the metaphor cannot take much weight. We use the same vocabulary to characterize ordinary paint as we use to characterize the subject of a portrait—terms relating to colour, shape, and so on. But ‘mental paint’ is not literally red or round, and the fact is that we have no vocabulary at all to use in characterizing it. We literally do not know what we are talking about when we talk about ‘mental paint’. The implicit suggestion is that this may not matter, because we are after all describing the most immediate and evident characteristics of experience, so what need is there for explanation? The trouble with that idea, though, is the transparency of experience. In ordinary vision, we do not encounter mental paint. We encounter the things and properties around us. Block has recently highlighted one representative finding, on attention and appearance, by Carrasco et al. (2004). Subjects were asked to fixate on a central point flanked by two gratings, and to report on the orientation of the highest-contrast grating. The subject is not asked for an explicit report of the relative contrast of the gratings. Yet a judgement as to which grating is higher contrast is implicit in the subject choosing to report on the orientation of one rather than the other. Covert attention was drawn to one of the two gratings by a cue. The basic finding is that the subject’s implicit rating of the contrast of a grating is affected by attention. If the cue draws one’s attention covertly to one grating rather than another, one is more likely to report on the orientation of that grating. On Block’s interpretation of this result, there is here significant variation in the perceived contrast of a single grating between two perceptions of it (one with attention and one without), yet both perceptions are veridical.

600   John Campbell Block interprets the finding in terms of a difference in the subjective experience of the perceiver, depending on whether covert attention has been drawn to the stimulus. Since according to Block, the subject’s perception of the grating is veridical whether or not attention is allocated to it, so (he argues) the difference between subjective experience of contrast in the two cases (with and without attention) cannot be explained in terms of either a difference in the way the subject’s perceptual system is representing the grating, or in terms of a difference in what the subject is seeing. Rather, the difference is a difference in this magnitude that Block calls ‘mental paint’. He does not specify what difference between two types of mental paint is responsible for the difference in implicit judgements of contrast. What might it be? As a first shot, we might say that one variety of paint is more ‘contrasty’ than the other. The natural way to interpret ‘contrasty’ is on the model of the notion of ‘contrast’ that we ordinarily apply to gratings out there in the physical world. But of course, as Block acknowledges, that is not the notion we need here. Just as mental paint is not literally round or red, it is not literally more or less high in contrast. What then is the difference in mental paint that is responsible for the difference in implicit judgements of contrast? Block’s account peters out here, and it is not hard to see why. We simply have no way of introducing and explaining a technical vocabulary to describe the hypothesized differences in mental paint. On the one hand, ‘mental paint’ seems to be a theoretical notion, a postulate invoked to explain relatively subtle technical results such as Carrasco’s. If it were a theoretical notion, like ‘electron’ or ‘gluon’, one would expect some attempt to characterize its theoretical role: to specify the varieties of mental paint and their various causal powers. Block makes no such attempt, however. The idea actually seems to be that no such explanation of the notion is needed. After all, ‘mental paint’ is characterizing the most immediately evident aspects of everyday visual experience, so what further explanation of the notion is needed? But that idea does not recognize the transparency of vision. What is immediately evident in ordinary visual experience is the environment. Block explains what he is talking about entirely in negative terms: if visual experience cannot be characterized entirely in terms of representational or relational views of experience, then there is mental paint (2010: 23–24). But that doesn’t explain what ‘mental paint’ is. You might as well try to explain what an electron is by saying that it is neither a proton nor a neutron. The trouble is that there is a fundamental obscurity in the notion. It can’t be explained. It is supposed to be a technical theoretical notion. But on the other hand, no technical theoretical explanation of it can be given. Suppose you did manage to give an entirely theoretical characterization of the varieties of mental paint. That would be something that could be grasped by a Martian, innocent of the realities of human consciousness. But the whole point of the notion of ‘mental paint’ was to characterize the ‘what it is likeness’ of human experience. Someone who does grasp a description of perception in terms of mental paint ought to know what the perception is like. At this point you might revert to the idea that since we are dealing with a concept related to visual experience, it does not need to be explained. But the characterization of what one encounters in visual experience has to do with the objects and properties perceived. Of course, the analysis of this particular case is still of philosophical interest: such phenomena of attention are likely to take their place alongside puzzles such as double vision, blurry vision, the speckled hen, and so on, which are part of the traditional fare of theories of perception. These discussions, of course, do not address what is in some ways the

Perceptual Attention   601 most central and elusive question about the impact of attention on perceptual experience, namely, whether there is a sui generis contribution of attention itself to experience. Is there some particular characteristic of one’s sensory experience, the quale of attention? (For discussion of many related issues, see Watzl, 2013.) What I have been arguing is that there is indeed a phenomenology that is distinctive of conscious attention as such, but that it should not be characterized in terms of qualia. We should think of conscious attention to one’s surroundings as a matter of how one is experientially related to the external scene. And the ‘attentional structure’ of visual experience has to be explained in terms of the causal structure of the experience: which properties are allowing the object to be perceived at all, which properties the subject can access, and so on. It is this causal structure in visual experience that gives attention its distinctive role in the generation of knowledge, concept possession, and intentional action.

References Allport, A. (1989). ‘Visual attention’. In M. I. Posner (ed.), Foundations of Cognitive Science (pp. 631–682). Cambridge, MA: MIT Press. Austin, J. L. (1962). Sense and Sensibilia, ed. G. J. Warnock. Oxford: Oxford University Press. Block, N. (1995). ‘On a confusion about a function of consciousness’. In N. Block, O. Flanagan, and G. Guzelder (eds), The Nature of Consciousness. Cambridge, MA: MIT Press. Block, N. (2010). ‘Attention and mental paint’. Philosophical Issues, 20, 23–63. Broadbent, D. (1958). Perception and Communication. London: Pergamon Press. Carrasco, M., Ling, S., and Read, S. (2004). ‘Attention alters appearance’. Nature Neuroscience, 7, 308–313. Cohen, M. A., Cavanagh, P., Chun, M. M., and Nakayama, K. (2012). ‘The attentional requirements of consciousness’. Trends in Cognitive Sciences, 16, 411–417. Duncan, J. (1980a). ‘Demonstration of capacity limitation’. Cognitive Psychology, 12, 75–96. Duncan, J. (1980b). ‘The locus of interference in the perception of simultaneous stimuli’. Psychological Review, 87, 272–300. Egeth, H. E., Virzi, R. A., and Garbart, H. (1984). ‘Searching for conjunctively defined targets’. Journal of Experimental Psychology: Human Perception and Performance, 10, 32–39. Frege, G. (1892/1952). ‘On sense and reference’. In P. Geach and M. Black (eds), Translations from the Philosophical Writings of Gottlob Frege (pp. 56–78). Oxford: Blackwell. Harman, G. (1990). ‘The intrinsic qualities of experience’. Philosophical Perspectives, 4, 31–52. Huang, L. (2010). ‘What is the unit of visual attention? Object for selection, but Boolean map for access’. Journal of Experimental Psychology, 139, 162–179. Huang, L. and Pashler, H. (2007). ‘A Boolean map theory of visual attention’. Psychological Review, 114, 599–631. James, W. (1890/1950). The Principles of Psychology, ed. G. Miller. New York: Dover Publications. Kennedy, M. (2010). ‘Naïve Realism and perceptual experience’. Proceedings of the Aristotelian Society, 110, 77–109. Kentridge, R. (2011). ‘Attention without awareness: A brief review’. In C. Mole, D. Smithies, and W. Wu (eds), Attention: Philosophical and Psychological Essays (pp. 228–246). New York: Oxford University Press. Kripke, S. (1980). Naming and Necessity. Cambridge, MA: Harvard University Press. Mole, C. (2008). ‘Attention and consciousness’. Journal of Consciousness Studies, 15, 86–104.

602   John Campbell Mole, C. (2011). Attention is Cognitive Unison. New York: Oxford University Press. Moore, G. E. (1903). ‘The refutation of idealism’. Mind, 12, 433–453. Noë, A. (2002). ‘Is the visual world a grand illusion?’ Journal of Consciousness Studies, 9, 1–12. Roessler, J. (2009). ‘Perceptual experience and perceptual knowledge’. Mind, 118, 1013–1041. Simons, D. J. and Chabris, C. F. (1999). ‘Gorillas in our midst: Sustained inattentional blindness for dynamic events’. Perception, 28, 1059–1074. Stazicker, J. (2011). Attention, visual consciousness and indeterminacy. Mind and Language, 26, 156–184. Treisman, A. and Gelade, G. (1980). ‘A feature-integration theory of attention’. Cognitive Psychology, 12, 97–136. Ullman, S. (2000). High-Level Vision and Visual Cognition. Cambridge, MA: MIT Press. Watzl, S. (2013). Attention and the Structures of Consciousness. New York: Oxford University Press. Wolfe, J. (1994). ‘Guided search 2.0: A revised model of visual search’. Psychonomic Bulletin and Review, 1, 202–238. Wu, W. (2011). ‘Attention as selection for action’. In C. Mole, D. Smithies, and W. Wu (eds), Attention: Philosophical and Psychological Essays (pp. 97–116). New York: Oxford University Press.

Chapter 32

M u ltisensory Perception Tim Bayne and Charles Spence

The naïve would have expected evolution in its course to have supplied us with more various sense organs for ampler perception of the world. . . . The policy has rather been to bring by the nervous system the so-called ‘five’ into closer touch with one another. . . . A central clearing house of sense has grown up. . . . Not new senses, but better liaison between old senses is what the developing nervous system has in this respect stood for. (Sherrington, Man on His Nature, 1951)

1 Introduction It is New Year’s Eve, and you are watching the fireworks display. You see the pyrotechnics lighting-up the night sky. You hear the fireworks exploding overhead. And, if the wind is blowing in the right direction, you might even smell them, or at least their effects. Each of the senses has its own transducers, frames of reference, and each sense provides an observer with access to a unique range of properties. And yet, despite the differences between them, the senses conspire together to provide us with knowledge of a single unified world filled with objects and events. Any adequate account of perceptual experience must do justice to the respects in which the various senses are unique, and also to the various commonalities between them. This chapter aims to provide the reader with an introductory overview of some of the many facets of the multisensory nature of perception. We begin in the following section with a survey of three of its central manifestations. In section 3, we turn to some of the principles that underpin multisensory interaction. Finally, in section 4, we consider what philosophical lessons might be drawn from the growing body of evidence highlighting the multisensory nature of perception in humans (and, for that matter, other species).

604    Tim Bayne and Charles Spence

2  The varieties of multisensory perception By definition, multisensory interactions occur between two (or more) sensory modalities. Thus, it might appear that one cannot begin to explore the scope and nature of multisensory interactions without, in the first instance, having an account of how many senses there are, or where exactly the borders between them lie. Unfortunately, there is no such account—at least not one that commands widespread assent (see Matthen, Chapter 30, this volume; Macpherson, 2011a). Although some authors recognize only the five traditional Aristotelian senses of vision, audition, touch, taste, and smell, most theorists argue that human perception involves significantly more than just these five forms of sensory perception (Durie, 2005). There is, however, disagreement about how many additional senses ought to be recognized. For example, although common-sense tends to suggest that touch constitutes a single sensory modality, some theorists distinguish passive from active touch as separate species of perception. Others, meanwhile, hold that the detection of temperature, pressure, and vibration involve separate modalities (or, at the very least, sub-modalities). Perhaps even more problematic here are questions concerning the individuation of the chemical senses, given the close connection between the mechanisms that underpin the perception of taste, smell (both orthonasal and retronasal), and flavour (Auvray and Spence, 2008; Stevenson, 2009). There are also questions about whether the various forms of bodily and interoceptive awareness qualify as forms of perception strictly-speaking, or whether instead we should draw a sharp distinction between perception and bodily sensation (Craig, 2002; see Ritchie and Carruthers, Chapter 18, this volume). If interoception is a species of perception then interactions between (say) vision and interoception would qualify as multisensory, but if it doesn’t then such interactions would fall outside the scope of this category. Although uncertainty about how to individuate the senses ‘problematizes’ the discussion of multisensory perception, we believe that it is possible to make some progress here even without a full account of how the senses are to be individuated. For one thing, despite the on-going existence of ‘border disputes’ between the senses, it is often fairly clear whether a particular finding involves intrasensory rather than intersensory interactions. For example, interactions between vision and audition are clearly multisensory. Thus, we can make some progress in developing an account of multisensory perception without necessarily having a comprehensive taxonomy of the senses. Moreover, one might challenge the underlying assumption that a comprehensive taxonomy of the senses is conceptually prior to the study of multisensory interactions. Arguably, the task of providing a taxonomy of the senses ought to proceed in tandem with the task of giving an account of the multisensory nature of perception. And if that’s right, then one’s model of multisensory perception ought to constrain—and, in turn, be constrained by—one’s taxonomy of the senses. Having noted that there are important questions about how many senses there are, and where the divisions between them lie, we will leave such issues in the background in what follows. The multisensory interactions on which this chapter will focus characterize the typical human being (and indeed the members of many other species). As such, they can be contrasted with the atypical multisensory interactions that occur in several forms of synesthesia (Cytowic and Eagleman, 2009). In synesthesia, stimuli of a certain type (say, the digit ‘3’)

Multisensory Perception   605 automatically trigger experiences as of a property that the stimulus itself may not possess (say, being cobalt blue). In certain cases, the inducer and the induced (or ‘concurrent’) percept belong to the same sensory modality, but in other cases they belong to different sensory modalities. For example, experiencing a sound as having a particular frequency might invoke the experience of a specific colour (Day, 2005). It is unclear how much cross-modal forms of synesthesia might have in common with the various forms of multisensory interaction that can be found in the neurotypical adult population. Although the mechanisms underlying synesthesia might reflect more general forms of interaction between the senses (Maurer, 1997), it is perhaps more likely that synesthesia results from patterns of neural connectivity or inhibition that have no counterparts in the normal population (Grossenbacher and Lovelace, 2001). Considerations of space prevent us from exploring the relationship between synesthesia and other intersensory interactions in any depth here, and we refer the interested reader to the entry on synesthesia to be found elsewhere in this publication (see Auvray and Deroy, Chapter 18, this volume; see also Deroy and Spence, in press).

Molyneux’s question and cross-modal transfer The senses clearly differ from each other in a number of ways. For example, they differ in the nature of their input, and in the bodily organs and neural pathways that are involved in processing that input. The senses also differ from each other in the perspective that they provide us with on the world. A number of features of the world, such as colour and pitch, are directly perceptible by only a single sense. Aristotle called such features ‘proper sensibles’. Other features of the world, such as its spatial and temporal structure, he labelled ‘common sensibles’, since they are perceptible by more than one sensory modality. Many of the central questions surrounding the multisensory nature of perception concern the common sensibles. Some of these questions can be raised by considering a question that the lawyer William Molyneux put to John Locke in 1688 (Degenaar, 1996). Molyneux asked whether a blind man, who knew how to distinguish a sphere from a cube by touch could distinguish the two objects by sight alone if he were to acquire the power of sight. Molyneux himself suggested that this question should be answered in the negative, a verdict that was endorsed by Locke and his fellow empiricists such as Berkeley, who argued that the newlysighted man would need to learn how to match the objects of sight with those of touch. Other philosophers, such as Leibniz, for instance, have defended a positive answer to Molyneux’s question. Still others have argued that Molyneux’s question is importantly ambiguous. For example, Thomas Reid argued that we should distinguish the question of whether a newlysighted person would be able to match visually-perceived three-dimensional shapes to the objects of touch from the question of whether such a person would be able to match visuallyperceived two-dimensional shapes to the objects of touch. Reid defended a negative response to the first question but an affirmative response to the second. Although philosophers have attempted to answer Molyneux’s question on a priori grounds, on the face of things it would appear to be a straightforwardly empirical question that can be addressed by studying the perceptual capacities of the newly-sighted. Unfortunately, many of these studies are inconclusive on the ground that they have not demonstrated that the ‘newly-sighted’ individuals in question have indeed regained the

606    Tim Bayne and Charles Spence capacity to visually identify shapes. ‘Seeing’ is a skill that requires a significant period of time to acquire, and the fact that a person’s visual system is receiving input does not entail that that person has visual experience as of the shapes of objects. However, recent work does provide some grounds for thinking that the answer to Molyneux’s question is ‘no.’ Held et al. (2011) found that newly-sighted children who were able both to match one visually perceived shape with another and also to match one haptically perceived shape with another, were not immediately able to match haptically perceived shapes to visually perceived shapes, although they did acquire this ability in a matter of days. What about newly-sighted individuals of another kind—human infants? Are they able to match properties across modalities? Influential voices within the field of developmental psychology have long argued that they cannot. According to Piaget, young infants have ‘heterogeneous’ perceptual spaces, and these spaces become integrated into a single multisensory space only as the child begins to grasp what s/he sees and looks at what s/he touches (Piaget and Inhelder, 1969: 13–15). However, this picture has been undermined by research over the last three decades or so, and the balance of evidence currently indicates that even very young infants can match certain kinds of information cross-modally (see Bremner et al., 2012, for a recent review). For example, infants look longer at a previously unseen pacifier whose shape they had explored orally than at one that they had not previously explored, suggesting that at some level representations of visual and tactile shape share a common code (for reviews, see Meltzoff, 1993; Streri, 2012). In addition to being able to match information about shape and texture across vision and touch, infants also match information about visual and tactile numerosity (e.g. Féron et al., 2006). Although one cannot conclusively rule out the possibility that learned associations play a role in accounting for these findings, the fact that young infants do not typically engage in the simultaneous exploration of objects in different modalities renders this hypothesis somewhat unlikely, and suggests instead that infants are able to match information across modalities by exploiting various forms of ‘amodal’ representation. However, it is unclear what implications these forms of cross-modal matching might have for accounts of perceptual experience, for it is possible that the representations that underpin cross-modal matching are not conscious. We return to the question of amodal perceptual content in section 4.2.

Perceptual plasticity and adaptation A second aspect to the multisensory nature of perception concerns the long-term effects that changes in the input to one sensory modality can have on another sensory modality (e.g. Röder and Rösler, 2004). These effects demonstrate that the senses are not causally insulated from each other but are instead integrated with each other in various ways. One example of perceptual plasticity is observed in response to the loss of a sense. In an influential series of studies, Sur and colleagues rewired the brains of young ferrets so that retinal nerves projected directly into the auditory thalamus, which had been deprived of its normal auditory input (Merzenich, 2000; Von Melchner et al., 2000). As a result of this procedure, the ‘primary auditory cortex’ of the ferrets became topographically organized and was able to support visually guided behaviours. In other words, it took on some of the functionality of the primary visual cortex.

Multisensory Perception   607 Another example of neural plasticity involves congenitally blind Braille readers (e.g. Merabet and Pascual-Leone, 2010). The additional demands on tactile perception imposed by reading Braille has been shown to lead to the enlargement of the somatosensory finger areas and to the redeployment of the deafferented ‘visual cortex’ in haptic information processing. In fact, sighted individuals who have been blindfolded for no more than five days show some limited increase in activity in the visual cortex in response to tactile and auditory stimulation. Similar changes have also been seen following deafness, in which the auditory cortex is recruited for viewing sign-language and the processing of peripheral visual stimuli. The plasticity of the perceptual system has also been revealed by experiments in which people wear prismatic goggles that distort visual input by some fixed amount. The initial consequence of wearing such goggles is that the visually presented location and structure of objects is incongruent with that of their haptically presented location and structure. In the case of inverting goggles, for example, objects are seen to be ‘upside down’ relative to their felt orientation. However, after some period of time—usually weeks—subjects typically (but not universally) report that their perceptual fields have become realigned, such that objects are now seen to have the same orientation that they are felt to have (Kohler, 1951/1964). The need for perceptual plasticity is clear. Organisms must be able to accommodate changes in the structure of, and relations between, their sensory organs. For example, the distance between the ears, which is critical for the perception of azimuthal direction in audition, changes as the organism grows, and hence accurate perception requires taking such changes into account. However, sometimes the kind of recalibration required for optimal perception does not occur. An example of such a failure is provided by the phantom limb phenomenon (e.g. Cronholm, 1951). Here, individuals who have had a limb amputated often continue to locate nociceptive, tactile, and proprioceptive experiences in the missing limb. One might have expected the patients’ visual and tactile perception of the absence of the missing limb to ‘over-ride’ and recalibrate their proprioceptive experiences over time, but this often fails to occur. In considering long-term adaption, it is useful to draw a distinction here between cortical deference and cortical dominance (Hurley and Noë, 2003). In the case of cortical deference, cortical activity in an area gives rise to mental states normally associated with the new source of input. For example, activation of visual cortex in response to Braille reading in the congenitally blind involves tactile experiences. In cortical dominance, by contrast, cortical activation in an area from a new input source gives rise to mental states with a conscious character normally associated with that area. The phantom limb phenomenon provides an example of cortical dominance, for—as just noted—in such patients, activation in specific somatosensory areas often gives rise to somatosensory experiences despite the fact that these areas no longer receive significant direct somatosensory input. We refer the interested reader to Hurley and Noë’s (2003) discussion for some provocative speculations as to why certain syndromes exhibit cortical dominance whereas others exhibit cortical deference.

Multisensory integration We now turn from long-term interactions between the senses to short-term or ‘on-line’ multisensory interactions. What happens when two senses receive roughly simultaneous input? Sometimes nothing much over-and-above what would be expected as a result of

608    Tim Bayne and Charles Spence presenting stimuli to each of the senses individually. In other cases, however, some kind of multisensory interaction occurs, such that the response produced by one (or both) of the senses differs from that which would have occurred had the two stimuli not been presented together. It is useful to draw a rough distinction here between two kinds of intersensory interactions: cross-modal interactions and multisensory integration. The category of cross-modal interactions subsumes all those cases in which the presentation of a stimulus in one sensory modality exerts an influence on a person’s perception of, or ability to respond to, stimuli presented in another sensory modality. For example, the presentation of a visual stimulus can lead to a shift of spatial attention which will facilitate a subject’s ability to respond to a subsequently-presented auditory or tactile stimulus, say (see Spence and Driver, 2004, for reviews). Instances of multisensory integration form a proper subset of such cross-modal interactions. We would argue that what distinguishes multisensory integration from mere cross-modal interaction is that the interaction in question involves representational integration between the contents of different sensory modalities. In other words, in multisensory integration, the processing of input in one (or more) sensory modality (modalities) is sensitive in content-respecting ways to information about stimuli that have been registered in another sensory modality. We will focus on multisensory integration here, for it is the source of most philosophical interest. Here are a few of the many examples of multisensory integration that have been documented to date (see Calvert et al., 2004; Stein, 2012, for reviews). In the ventriloquism effect, the apparent source of a spatially discrepant sound source is mislocalized so that it more closely matches the seen source, be it the articulated lips of the ventriloquist’s dummy, the sight of a loudspeaker cone, or some other temporally synchronized visual event (Bertelson and de Gelder, 2004). In the McGurk effect, dubbing the phoneme /ba/ onto the lip movements for /ga/, normally produces a percept of the person uttering the phoneme /da/ (McGurk and McDonald, 1976). In the parchment skin illusion, people experience their hands as being drier when the sound produced by rubbing them together is appropriately modified (Guest et al., 2002). One of the most famous recent examples of multisensory integration is the rubber hand illusion, in which the experience of seeing a rubber hand being stroked whilst also feeling one’s own hand (which is hidden from view) being synchronously stroked generates a sense that the rubber hand is part of one’s own body (Botvinick and Cohen, 1998). Multisensory integration even extends to the experience of pain. Individuals who experience pain in one of their limbs report that it hurts less when they look at the affected body part through a pair of binoculars the ‘wrong way’, and hence see the limb looking smaller than it really is (Moseley et al., 2008). Under what conditions does multisensory integration occur? There is good evidence to suggest that spatial coincidence leads to enhanced multisensory integration when simple detection or spatially-based tasks/responses are required. However, spatial coincidence appears to play little, if any, role in modulating multisensory integration in tasks that require participants to identify the target stimuli, or to report on their temporal attributes (see Spence, 2012, for a review). Although multisensory integration is, in general, most pronounced when the contributing stimuli are presented at approximately the same point in time, there are content-specific exceptions to this rule (Chen and Spence, 2011; Vatakis and Spence, 2010). So, for example, when the auditory and visual stimuli consist of semantically meaningful stimuli—such as the sound and picture of a barking dog—then presenting the

Multisensory Perception   609 auditory stimulus some 100s of milliseconds before the visual stimulus will generate multisensory integration effects that are larger than those produced when the auditory and visual stimuli are presented simultaneously. Similarly, in his recent review, Spence (2013) has argued that spatial coincidence only enhances those multisensory integration effects in which space is relevant to the participant’s task. Let us turn from considering the conditions under which multisensory integration occurs to considering the kinds of perceptual effects that result from multisensory integration. In some cases, input in one modality leads to the extinction of perceptual experience in another modality in neurotypical participants. An example of this is provided by the Colavita visual dominance effect, which can be regarded as a form of cross-modal extinction (see Spence et al., 2011, for a review). Here, participants sometimes fail to respond to an auditory target when an auditory and a visual target are presented together, even though they exhibit no difficulty in responding to the auditory or visual targets when they are presented individually. In other cases, participants enjoy experiences in each of the contributing sensory modalities, but the contents of their perceptual experiences are modulated by the multisensory integration. The literature indicates that a wide range of perceptual contents can be influenced as a result of multisensory integration. The ventriloquism effect indicates that the spatial content of perceptual representation in one sensory modality can be modulated by input in another modality (Bertelson and de Gelder, 2004). In fact, input in one modality can alter not just the perceived location of a stimulus that is registered by another sensory modality but even the direction in which it is perceived to move. For example, the direction of visually-presented motion (either real or apparent) can reverse the experienced direction of an auditory or tactile motion stream (Soto-Faraco et al., 2004). An emerging body of research is also starting to reveal the various ways in which multisensory integration impacts on the temporal content of perception. In what is known as the ‘temporal ventriloquism’ effect, the subsequent presentation of an auditory stimulus changes the perceived timing of a previously-presented visual stimulus (MoreinZamir et al., 2003). Temporal recalibration effects have now been demonstrated between all possible combinations of auditory, visual, and tactile stimuli. Indeed, the results of multisensory integration are not restricted to relatively low-level perceptual features, but also extend to the perception of ‘categorical’ features, as demonstrated by the McGurk effect. Perhaps most surprisingly, input in one sensory modality can even modulate the number of objects that an individual perceives in another modality: that is, multiple tones can lead subjects to misperceive a single flash of light as two flashes and a single tap as two taps (Shams et al., 2000).

3  Accounting for multisensory integration How might one account for the nature and occurrence of multisensory integration? Clearly any explanatory model of multisensory integration will need to extend across multiple levels of analysis and draw on the resources of a number of disciplines. Hence, we will not attempt to provide an exhaustive survey of the various approaches that one might adopt in accounting for the multisensory nature of perception. Instead, we will restrict ourselves to

610    Tim Bayne and Charles Spence identifying some of the central ideas and principles that have been posited to explain multisensory integration. One historically influential approach has been the ‘modality appropriateness hypothesis’. According to Welch and Warren (1980, 1986), the modality that is ‘most appropriate’ for the task at hand dominates a particular multisensory interaction. This proposal provides an elegant (if rather post-hoc) explanation for why vision typically dominates audition in spatial tasks whereas audition normally dominates vision in temporal tasks: vision has a higher spatial resolution than audition, whereas audition has a higher temporal resolution than vision. Despite this, the ‘modality appropriateness hypothesis’ suffers from a number of shortcomings. For one thing, there are certain multisensory interactions, such as the McGurk effect, for which this account does not seem to have an obvious explanation. Furthermore, the account does not provide a rigorous treatment of what it is for a modality to be ‘appropriate’ in the first place. And without that, it is not altogether clear what the hypothesis predicts in any particular case (Alais and Burr, 2004). A more inclusive and robust model of multisensory interactions is provided by the maximum likelihood estimation (MLE) approach. The idea here is that the brain weights the relative contribution of each sensory input in accordance with its relative reliability (Ernst and Banks, 2002). This means of integrating multiple sensory inputs is statistically optimal in the sense that it reduces the variance associated with the final multisensory percept. The basic MLE approach has been extended by the introduction of priors to form what is now known as the Bayesian approach to multisensory integration (Ernst and Bülthoff, 2004; Trommershäuser et al., 2011). A prior probability distribution, typically referred to simply as ‘the prior’, of an uncertain stimulus is the probability distribution that expresses the uncertainty about that stimulus before the current sensory evidence has been taken into account. Bayes’ theorem is applied by multiplying the prior by the likelihood function and then normalizing the result in order to get the posterior probability distribution. More recently, coupling priors have been introduced in order to capture background knowledge that we all implicitly possess about the correlations between the sensory features present in different modalities, such as the fact that large objects typically make low-pitched noises (Parise and Spence, 2013; Spence, 2011). Perhaps the critical difference between the ‘modality appropriateness’ approach and the Bayesian approach lies in the fact that the former takes the weightings of the respective modalities to be fixed, whereas the latter allows them to be modified by the information that the organism registers about particular kinds of stimuli. The Bayesian approach has been successfully applied to a large number of multisensory effects, and it is currently the dominant model when it comes to thinking about multisensory interactions. In addition to the fact that the approach accounts for much of the behavioural data, we might also note that it makes sense from the perspective of cognitive ethology. There is a clear biological advantage to be gained from integrating the information available to different senses. Such an ability enhances a creature’s capacity to accurately identify objects in its natural environment and recognize the behaviour of conspecifics. In fact, at the present time, no species has been documented that has multiple senses but which does not integrate those senses for the coherent control of action. For example, dart-poison frogs will attack a conspecific only when they hear it produce an ‘advertisement call’ and see it produce vocal sac pulsations within a temporal window of half a second (Narins et al., 2005). Unless the visual and auditory cues are produced

Multisensory Perception   611 together in this way rival males will tend to engage in exploratory rather than aggressive behaviours (Partan and Marler, 1999). What light has contemporary neuroscience managed to shed on the mechanisms underlying multisensory integration? How and where does the brain go about integrating signals from distinct sensory modalities? Part of the answer to the ‘where’ question involves reference to parts of the brain that specialize in multisensory integration. (This proposal has echoes of Aristotle’s notion of a ‘common-sense’, an organ that was assigned the job of integrating the contents of the various sensory modalities.) There are a number of multisensory association areas (see Driver and Noesselt, 2008, for a review). For example, the superior colliculus, the parieto-insular vestibular cortex, the orbitofrontal cortex, and the temporoparietal junction, among others, have all been implicated in various aspects of multisensory integration. However, multisensory integration also involves areas that have traditionally been regarded as ‘unisensory’. It is important to note , though, that such areas may be unisensory only in terms of their initial afferent projections. Back-projections from multisensory convergence areas might result in activation in these areas caused by input in a secondary modality. Indeed, the ubiquity of multisensory interactions affecting brain activity in regions that have traditionally been thought of as unisensory has led some researchers to wonder whether the cortex might be ‘essentially multisensory’ (e.g. Ghazanfar and Schroeder, 2006). We are also starting to learn how neural activity implements multisensory integration. Angelaki and colleagues have found evidence that variations in the reliability of different kinds of cues are correlated with changes in the weights that multisensory neurons apply to cues of different types (Angelaki et al., 2009). However, there are still many unanswered questions about how multisensory integration is realized. For example, it is not yet known how populations of neurons might implement the kinds of probability distributions posited by the Bayesian treatment of multisensory integration.

4  Philosophical implications of multisensory perception We turn now to explore the implications of the research that we have briefly surveyed here for debates in the philosophy of mind and cognitive science. We will focus on three main issues: The nature and scope of modularity; the nature of perceptual content; and the structure of consciousness.

Modularity The modular approach to perception is a very broad church, and a number of quite different accounts can be appropriately described as modular in some sense or another (see Deroy, Chapter 40, this volume). Within philosophical psychology, the most influential conception of modularity is that developed by Fodor in his book The Modularity of Mind (1983). Fodor argued that the architecture of perception is largely modular,

612    Tim Bayne and Charles Spence where a Fodorian module is a mechanism that is domain-specific, informationallyencapsulated, and possesses a dedicated and innately determined neural basis. What implications does research on multisensory integration have for the Fodorian view? Certain aspects of the Fodorian picture are clearly unaffected by it. For example, the existence of multisensory integration does not undermine the claim that perceptual content cannot be penetrated by the contents of belief, desire, or intention. However, multisensory integration does put at least two aspects of the Fodorian picture under pressure (see Radeau, 1994, and the commentaries thereafter). First, the literature on long-term perceptual adaption that was surveyed in section 2.2 indicates that the relationship between neural architecture and psychological structure is far more dynamic and malleable than that suggested by Fodorian modularity, although certain brain regions may have a ‘predilection’ (Pascual-Leone and Hamilton, 2001) for information presented in a particular modality. A second respect in which multisensory integration puts pressure on Fodorian modularity concerns informational encapsulation. To the extent that input in different modalities is processed by informationally-encapsulated modules, then one should not expect that perceptual processing in one modality would be highly dependent on processing in other modalities, as the evidence indicates that it is. Although there is nothing in the modular approach to perception that rules out the possibility of multisensory modules—indeed, Fodor (1983: 132, n.13) himself cites the McGurk effect as evidence that modules can straddle the boundaries between sensory modalities—the ubiquity of multisensory integration is at odds with the spirit which animates modular conceptions of the mind. At a minimum, any account of the architecture of perception that posits informational encapsulation between sensory modalities must accommodate the fact that cross-modal interactions are ‘the rule and not the exception in perception’ (Shimojo and Shams, 2001: 505).

Perceptual content Some of the most interesting philosophical questions raised by multisensory perception concern the analysis of the intentional content and phenomenal character of perception. The intentional content of a perceptual experience is the way that it represents the world as being (cross-reference to see Nanay, Chapter 8, this volume). For example, a visual experience might represent the fact that there is a square of such-and-such a size directly in front of one. The phenomenal character of a perceptual experience is a matter of ‘what it is like’ to have it. Almost all contemporary theorists hold that there is a close relationship between perception’s intentional content and its phenomenal character, but there is disagreement about the exact nature of this relationship. There are two ways in which the literature on multisensory integration may bear on this issue: the first concerns the possibility of modality-specific differences in the representation of common sensibles; the second concerns the idea that every perceptual experience can be uniquely associated with a single sensory modality. Let us consider these topics in turn. Although it is clear that different sensory modalities can overlap in their contents insofar as they provide us with access to a common set of properties and relations (such as space, time, and size/shape), it is not clear whether such commonalities in content bring

Multisensory Perception   613 with them commonalities in phenomenal character. We can contrast two accounts here (Chalmers, 2004; Thompson, 2009). According to one account, the phenomenal character of an experience is determined purely by the properties and objects that it represents (e.g. Shoemaker, 2001; Tye, 2000). This account, known as the ‘Russellian’ or ‘pure informational’ account, entails that it is not possible for experiences with exactly the same intentional content to differ in phenomenal character. Another account, known as the ‘Fregean’ or ‘impure informational’ account, holds that the phenomenal character of a perceptual experience is determined not just by the objects and properties that it represents, but also by how those objects and properties are represented (e.g. Lopes, 2000; Thompson, 2010). Thus, the Fregean will allow that there are—or at least could be—modality-specific differences in the phenomenal character of experiences, such that even experiences with exactly the same intentional content could differ in phenomenal character. (Note that this contrast between the ‘Russellian’ and ‘Fregean’ accounts concerns the relationship between phenomenal character and intentional content, and is only indirectly related to the more familiar debate between Fregean and Russellian accounts of intentional content.) How might multisensory integration bear on the debate between these two accounts? Let us return to Molyneux’s question of whether a newly-sighted person would be able to immediately recognize the identity between certain visually perceived shapes and certain haptically perceived shapes. One might think that Russellians have reason to answer this question in the affirmative, for according to the Russellian view the phenomenal character of a perceptual experience of an object’s shape is fixed by the experience’s intentional content, so assuming that the subject’s visual and tactile experiences of the object’s shape have an intentional content in common, then they should also have a phenomenal element in common. And if that is so, then—one might argue—the identity between the visually presented shape and the haptically presented shape ought to be immediately apparent to the subject. By contrast, Fregeans will allow that vision and touch might be characterized by distinctive phenomenal characters, such that the newly-sighted person will need to learn how to map the properties perceived via one sensory modality onto those perceived by another (just as one must learn that the morning star is identical to the evening star). However, matters are more complicated than the foregoing might suggest. For one thing, the Fregean could defend an affirmative answer to Molyneux’s question by arguing that although visual and haptic representations of a single shape might differ in phenomenal character, the identity between these two representations is hard-wired, with the result that infants do not need to learn that this visually presented shape is identical to that haptically presented shape. In other words, evolution might have solved the matching problem for the infant (see Bremner et al., 2012). More controversially, one might argue that the Russellian could defend a ‘no’ answer to Molyneux’s question on the grounds that even if the subject’s visual and haptic perceptual experiences of an object’s shape have the same phenomenal character, the representations in question might be located in different regions of perceptual architecture, and thus some degree of learning might be required in order to recognize that this visually perceived shape is the same as that haptically perceived shape. In fact, this possibility can be motivated by noting that intermodal transfer in infants is often asymmetrical, in the sense that an infant might have the capacity to match modality A to modality B but not vice-versa (Streri, 2012). One would not expect this finding if inter-modal transfer were merely a function of the phenomenal character of the experiences in question. The upshot of all this is that although the literature

614    Tim Bayne and Charles Spence on Molyneux’s question and cross-modal transfer certainly has a bearing on the debate between Russellian and Fregean treatments of perception, the precise nature of that bearing is not yet settled. Let us turn now to the second question on our agenda here: can perceptual experience be exhaustively divided up into modality-specific chunks? Philosophers seem to have often assumed that the answer to this question is ‘yes’. As Nudds (2001: 224) puts it, philosophical treatments of perception often assume that what we perceive is the ‘sum total of what each sense alone provides’. Is this assumption—what we might call the modality-specificity thesis—plausible? Reflection on multisensory integration suggests not (O’Callaghan, 2008, 2011, 2014a, 2014b; Nudds, 2001). Consider first the experience of /da/ that one has in the context of the McGurk effect. Can such experiences be identified with just one sensory modality? Appealing to the sensory pathways that are responsible for this percept fails to assign it to a unique sense, for arguably both visual and auditory systems are involved in its production. Nor will an appeal to the properties that this experience represents assign it to a particular sense, for both vision and audition are able to represent this phoneme. What about the appeal to modality-specific phenomenal character? As we have just noted, it is controversial whether modality-specific phenomenal characters exist. Even if they do exist, it is not clear (to us, at least) whether the experience of /da/ that one has in the context of the McGurk effect is accompanied by visual phenomenology or auditory phenomenology. (Perhaps such experiences enjoy both visual and auditory phenomenal character!) So, there is reason to think that the McGurk effect puts pressure on the modality-specificity thesis (Macpherson, 2011b). Certain experiences of causation also put pressure on this thesis (Mitterer and Jesse, 2010; Nudds, 2001). Consider a situation in which one experiences a sound as having been produced by a visually perceived event, such as the hitting of a drum. In such a case, one might argue that one’s experience of causation is constitutively audio-visual, and cannot be factored into purely visual and purely auditory components. Experiences of embodiment also put pressure on the modality-specificity thesis. A number of paradigms indicate that multisensory integration—or the lack thereof—plays a crucial role in modulating various aspects of the experience of embodiment. Local aspects of this experience have been explored using the rubber hand illusion, which we have already mentioned. Global aspects of the experience of embodiment have been explored by studying out-of-body experiences and various autoscopic (literally, ‘self viewing’) phenomena (e.g. Slater et al., 2009). The fact that experiences of embodiment involve the integration of visual, tactile, proprioceptive, and vestibular cues indicates that embodiment is an aspect of one’s overall perceptual experience that cannot be ascribed to any one sensory modality (Moseley et al., 2012). Taking a step back, a plausible case can be made for thinking that one of the core principles that has been employed to explain multisensory integration is at odds with the modality-specificity thesis. There is a plausible argument to be made for the view that where there is multisensory integration between two modalities involving a certain set of properties and objects, then those properties and objects must be represented ‘amodally’ in some sense (Bedford, 2004). As Welch (1999: 373) has put it, ‘an intersensory conflict can be registered as such only if the two sensory modalities are providing information about a sensory situation that the observer has strong reasons to believe (not necessarily consciously) signifies a single (unitary) distal object or event’ (Welch, 1999: 373). The idea that

Multisensory Perception   615 multisensory integration is triggered by the representation of identity between the objects of one sense and the objects of another has come to be known as the unity assumption (Welch and Warren, 1980; Spence, 2007), although it should be noted the subject’s representation of unity (or better ‘identity’) need not take the form of a belief but might be purely sub-doxastic. For example, it is very plausible to explain ventriloquism effects by supposing that the perceptual system assumes that the event responsible for the auditory input is identical to the event that is responsible for the visual input. The unity assumption threatens the modality-specificity thesis, because it suggests that it may not be possible to provide a full specification of all perceptual experiences in modality-specific terms. If, in ventriloquism, it is internal to the content of one’s perception that what one sees is identical to what one hears, then it will not be possible to exhaustively capture the nature of one’s experience in purely visual and purely auditory terms (O’Callaghan, 2011).

The structure of consciousness A final area of debate within the philosophy of mind for which the multisensory character of perception is likely to have major implications concerns the structure of consciousness. Consider again the experience of enjoying a firework display. A  ten second segment of your perceptual experience during such an event would no doubt contain content drawn from several sensory modalities, but is your conscious perceptual experience multisensory during very short intervals of time? This question has an important bearing on how one approaches the various problems associated with the unity of consciousness. If the multisensory view is correct, then any adequate treatment of the unity of consciousness must accommodate that fact. By contrast, if the unisensory view is correct then accounts of the unity of consciousness need not explain how content related to distinct sensory modalities is bound together in order to form a global experiential state. What kind of evidence might we appeal to when adjudicating this issue? Although it is tempting to appeal to introspection, it is unclear whether the temporal grain of introspection is precise enough to be able to provide us with reliable information concerning this issue. Although many people claim that they are simultaneously conscious of stimuli in various sensory modalities, it is possible that these claims rest on an illusion of sorts, and that perceptual consciousness actually rapidly switches between different modalities. What about multisensory integration? Might the kinds of effects surveyed in section 2.3 provide evidence in favour of the multisensory view of consciousness? Perhaps not, for there is reason to suspect that multisensory integration might, in general at least, take place ‘outside’ of consciousness, in the sense that subjects will not typically be aware of the multisensory inputs that are integrated (Spence and Bayne, 2015; although, see also O'Callaghan, 2014b). But let us suppose, if only for the sake of argument, that human consciousness is at least occasionally multisensory. If so, how might we conceptualize the unity that obtains within a person’s overall conscious perspective? One approach to this question—dubbed by some ‘the received view’—holds that various perceptual experiences are bound together to form a total conscious state (e.g. Bayne and Chalmers, 2003; Dainton, 2006). Although the received view does not entail that experiences are modality-specific, presentations of the received view typically express some sympathy with the idea that there

616    Tim Bayne and Charles Spence are modality-specific experiences. Another approach, defended by Tye (2003) under the ‘one-experience’ label, holds that consciousness rarely (if ever) involves modality-specific experiences. According to Tye, although the subject’s total conscious state will contain content drawn from various sensory modalities, we should not identify these contents with particular experiences. Instead, Tye holds, the only experiences that human beings have are entire streams of consciousness. Exactly what the contrast between the received view and the one-experience view amounts to depends largely on how experiences ought to be individuated (Bayne, 2010). On some treatments of this issue the claim that a person might have modality-specific experiences at a time would not be at all plausible, whereas on other accounts it would. Perhaps more importantly, adopting a conception of experiences according to which subjects rarely (if ever) enjoy modality-specific experiences would not itself dissolve the various problems associated with the unity of consciousness, for one would still need to explain how content associated with different sensory modalities is brought together in the form of a single multisensory state.

5 Conclusions Although philosophers, psychologists, and neuroscientists have traditionally taken what can be characterized as a unisensory approach to the study of perception, it is increasingly clear that such an approach leaves us with a view of perception that is at best partial and at worst positively distorted. In this chapter, we have examined some of the many ways in which the neural and perceptual processing in one sensory modality is responsive to, and dependent on, processing in other sensory modalities. We have sketched out some of the philosophical implications of these interactions. Undoubtedly, the task of filling-in the details is going to require a significant investment of time and effort on the part of both philosophers and psychologists. However, we have no doubt that this investment will be a worthwhile one, for a comprehensive understanding of the multisensory character of perception will bring with it new ways of understanding the nature of perception itself.

References Alais, D. and Burr, D. (2004). ‘The ventriloquist effect results from near-optimal bimodal integration’. Current Biology, 14, 257–262. Angelaki, D., Gu, Y., and DeAngelis, G. (2009). ‘Multisensory integration: Psychophysics, neurophysiology, and computation’. Current Opinion in Neurobiology, 18, 452–458. Auvray, M. and Spence, C. (2008). ‘The multisensory perception of flavor’. Consciousness and Cognition, 17, 1016–1031. Bayne, T. (2010). The Unity of Consciousness. Oxford: Oxford University Press. Bayne, T. and Chalmers, D. (2003). ‘What is the unity of consciousness?’ In A. Cleeremans (ed.), The Unity of Consciousness: Binding, Integration, and Dissociation (pp. 23–58). Oxford: Oxford University Press.

Multisensory Perception   617 Bedford, F. L. (2004). ‘Analysis of a constraint on perception, cognition and development: One object, one place, one time’. Journal of Experimental Psychology: Human Perception and Performance, 30, 907–912. Bertelson, P. and de Gelder, B. (2004). ‘The psychology of multimodal perception’. In C. Spence and J. Driver (eds), Crossmodal Space and Crossmodal Attention (pp. 141–177). Oxford: Oxford University Press. Botvinick, M. and Cohen, J. (1998). ‘Rubber hands "feel" touch that eyes see’. Nature, 391, 756. Bremner, A. J., Holmes, N. P., and Spence, C. (2012). ‘Development of multisensory representations of the body and peripersonal space’. In A. J. Bremner, D. J. Lewkowicz, and C. Spence (eds), Multisensory Development (pp. 113–136). Oxford: Oxford University Press. Calvert, G. A., Stein, B. E., and Spence, C. (eds) (2004). The Handbook of Multisensory Processing. Cambridge, MA: MIT Press. Chalmers, D. (2004). ‘The representational character of experience’. In B. Leiter (ed.), The Future for Philosophy (pp. 153–181). Oxford: Oxford University Press. Chen, Y.-C. and Spence, C. (2011). ‘Crossmodal semantic priming by naturalistic sounds and spoken words enhances visual sensitivity’. Journal of Experimental Psychology: Human Perception and Performance, 37, 1554–1568. Craig, A. D. (2002). ‘How do you feel? Interoception: The sense of the physiological condition of the body’. Nature Reviews Neuroscience, 3, 655–666. Cronholm, B. (1951). ‘Phantom limb in amputees: Study of changes in integration of centripetal impulses with special reference to referred sensations’. Acta Psychiatrica et Neurologica Scandinavica (Supplementum), 72, 1–85. Cytowic, R. E. and Eagleman, D. M. (2009). Wednesday is Indigo Blue: Discovering the Brain of Synesthesia. Cambridge, MA: MIT Press. Dainton, B. (2006). Stream of Consciousness (2nd edn). London: Routledge. Day, S. A. (2005). ‘Some demographic and socio-cultural aspects of synesthesia’. In L. Robertson and N. Sagiv (eds), Synesthesia: Perspectives from Cognitive Neuroscience (pp. 11–33). Oxford: Oxford University Press. Degenaar, M. (1996). Molyneux’s Problem. Dordrecht: Kluwer. Deroy, O. and Spence, C. (in press). ‘Synaesthesia reclassified: Borderline cases of crossmodallyinduced experiences’. In O. Deroy (ed.), Sensory blending: New essays on synaesthesia. Oxford: Oxford University Press. Driver, J. and Noesselt, T. (2008). ‘Multisensory interplay reveals crossmodal influences on "sensory-specific" brain regions, neural responses, and judgments’. Neuron, 57, 11–23. Durie, B. (2005). ‘Future sense’. New Scientist, 2484, 33–36. Ernst, M. O. and Banks, M. S. (2002). ‘Humans integrate visual and haptic information in a statistically optimal fashion’. Nature, 415, 429–433. Ernst, M. O. and Bülthoff, H. H. (2004). ‘Merging the senses into a robust percept’. Trends in Cognitive Sciences, 8, 162–169. Féron, J., Gentaz, E., and Streri, A. (2006). ‘Evidence of amodal representation of small numbers across visual-tactile modalities in 5-month-old infants’. Cognitive Development, 21, 81–92. Fodor, J. (1983). The Modularity of Mind. Cambridge, MA: MIT Press. Ghazanfar, A. A. and Schroeder, C. E. (2006). ‘Is neocortex essentially multisensory?’ Trends in Cognitive Sciences, 10, 278–285. Grossenbacher, P. G. and Lovelace, C. T. (2001). ‘Mechanisms of synesthesia: Cognitive and physiological constraints’. Trends in Cognitive Sciences, 5, 36–41.

618    Tim Bayne and Charles Spence Guest, S., Catmur, C., Lloyd, D., and Spence, C. (2002). ‘Audiotactile interactions in roughness perception’. Experimental Brain Research, 146, 161–171. Held, R., Ostrovsky, Y., de Gelder, B., Gandhi, T., Ganesh, S., and Sinha, P. (2011). ‘The newly sighted fail to match seen with felt’. Nature Neuroscience, 14, 551–553. Hurley, S. and Noë, A. (2003). ‘Neural plasticity and consciousness’. Biology and Philosophy, 18, 131–168. Kohler, I. (1951/1964). ‘Formation and transformation of the perceptual world’, trans H. Fiss. Psychological Issues, 3(4), 1–173. Lopes, D. (2000). ‘What is it like to see with your ears? The representational theory of the mind’. Philosophy and Phenomenological Research, 90, 439–453. McGurk, H. and MacDonald, J. W. (1976). Hearing lips and seeing voices, Nature, 264, 746–748. Macpherson, F. (2011a). Individuating the senses. In F. Macpherson (ed.), The Senses: Classic and Contemporary Readings (pp. 3–45). New York: Oxford University Press. Macpherson, F. (2011b). ‘Cross-modal experiences’. Proceedings of the Aristotelian Society, CXI, 429–468. Maurer, D. (1997). ‘Neonatal synesthesia: Implications for the processing of speech and faces’. In S. Baron-Cohen and J. E. Harrison (eds), Synesthesia: Classic and Contemporary Readings (pp. 224–242). Malden, MA: Blackwell. Meltzoff, A. N. (1993). ‘Molyneux’s babies: Cross-modal perception, imitation, and the mind of the preverbal infant’. In N. Eilan, R. McCarthy, and B. Brewer (eds), Spatial Representation: Problems in Philosophy and Psychology (pp. 219–235). Oxford: Oxford University Press. Merabet, L. B. and Pascual-Leone, A. (2010). ‘Neural reorganization following sensory loss: The opportunity of change’. Nature Reviews Neuroscience, 11, 44–52. Meredith, M. A. and Stein, B. E. (1983). ‘Interactions among converging sensory inputs in the superior colliculus’. Science, 221, 389–391. Merzenich, M. (2000. Seeing in the sound zone. Nature, 404, 820–821. Mitterer, H. and Jesse, A. (2010). Correlation versus causation in multisensory perception. Psychonomic Bulletin and Review, 17, 329–334. Morein-Zamir, S., Soto-Faraco, S., and Kingstone, A. (2003). Auditory capture of vision: Examining temporal ventriloquism, Cognitive Brain Research, 17, 154–163. Moseley, G. L, Parsons, T., and Spence, C. (2008). Visual distortion modulates pain and swelling evoked by movement. Current Biology, 18, R1047–R1048. Moseley, G. L., Gallace, A., and Spence, C. (2012). Bodily illusions in health and disease—theoretical, physiological and clinical perspectives. Neuroscience and Biobehavioural Reviews, 36, 34–46. Narins, P. M., Grabul, D. S., Soma, K. K., Gaucher, P., and Hödl, W. (2005). ‘Cross-modal integration in a poison-dart frog’. Proceedings of the National Academy of Sciences USA, 102, 2425–2429. Nudds, M. (2001). ‘Experiencing the production of sounds’. European Journal of Philosophy, 9, 210–229. O’Callaghan, C. (2008). ‘Seeing what you hear: Cross-modal illusions and perception’. Philosophical Issues 18: Interdisciplinary Core Philosophy, 316–338. O’Callaghan, C. (2011). ‘Perception and multimodality’. In E. Margolis, R. Samuels, and S. Stich (eds), Oxford Handbook of Philosophy of Cognitive Science (pp. 92–117). Oxford: Oxford University Press. O’Callaghan, C. (2014a). ‘Not all perceptual experience is modality specific’. In D. Stokes, M. Matthen, and S. Biggs (eds), Perception and Its Modalities (pp. 133-65), New York: Oxford University Press.

Multisensory Perception   619 O’Callaghan, C. (2014b). ‘Intermodal binding awareness’. In D. Bennett and C. Hill (eds), Sensory Integration and the Unity of Consciousness (pp. 73-104), Cambridge, MA: MIT Press. Parise, C. V. and Spence, C. (2013). ‘Audiovisual crossmodal correspondences’. To appear in J. Simner and E. Hubbard (eds), The Oxford Handbook of Synaesthesia (pp. 790–815). Oxford: Oxford University Press. Partan, S. and Marler, P. (1999). ‘Communication goes multimodal’. Science, 283, 1272–1273. Pascual-Leone, A. and Hamilton, R. (2001). ‘The metamodal organization of the brain’. In C. Casanova and M. Ptito (eds), Progress in Brain Research, 134, 1–19. Piaget, J. and Inhelder, B. (1969). The Psychology of the Child. New York: Basic Books. Radeau, M. (1994). ‘Auditory-visual spatial interaction and modularity’. Current Psychology of Cognition, 13, 3–51. Röder, B. and Rösler, F. (2004). ‘Compensatory plasticity as a consequence of sensory loss’. In G. A. Calvert, C. Spence, and B. E. Stein (eds), The Handbook of Multisensory Processing (pp. 719–747). Cambridge, MA: MIT Press. Shams, L., Kamitani, Y., and Shimojo, S. (2000). ‘What you see is what you hear’. Nature, 408, 788. Shimojo, S. and Shams, L. (2001). ‘Sensory modalities are not separate modalities: Plasticity and interactions’. Current Opinion in Neurobiology, 11, 505–509. Shoemaker, S. (2001). ‘Introspection and phenomenal character’. Philosophical Topics, 28, 247–273. Slater, M., Perez-Marcos, D., Ehrsson, H. H., and Sanchez-Vives, M. V. (2009). Inducing illusory ownership of a virtual body. Frontiers in Neuroscience, 3, 214–220. Soto-Faraco, S., Spence, C., and Kingstone, A. (2004). Cross-modal dynamic capture: Congruency effects in the perception of motion across sensory modalities. Journal of Experimental Psychology: Human Perception and Performance, 30, 330–345. Spence, C. (2007). ‘Audiovisual multisensory integration’. Acoustical Science and Technology, 28, 61–70. Spence, C. (2011). ‘Crossmodal correspondences: A tutorial review’. Attention, Perception, & Psychophysics, 73, 971–995. Spence, C. (2012). ‘Multisensory perception, cognition, and behavior: Evaluating the factors modulating multisensory integration’. In B. E. Stein (ed.), The New Handbook of Multisensory Processing (pp. 241–264). Cambridge, MA: MIT Press. Spence, C. (2013). ‘Just how important is spatial coincidence to multisensory integration? Evaluating the spatial rule’. Annals of the New York Academy of Sciences. Spence, C., and Bayne, T. (2015). Is consciousness multisensory? In D. Stokes, S. Biggs, and M. Matthen (eds), Perception and its Modalities (pp. 95–132). New York: Oxford University Press. Spence, C. and Driver, J. (eds) (2004). Crossmodal Space and Crossmodal Attention. Oxford: Oxford University Press. Spence, C., Parise, C., and Chen, Y.-C. (2011). ‘The Colavita visual dominance effect’. In M. M. Murray and M. Wallace (eds), Frontiers in the Neural Bases of Multisensory Processes (pp. 523–550). Boca Raton, FL: CRC Press. Stein, B. E. (ed.) (2012). The New Handbook of Multisensory Processing. Cambridge, MA: MIT Press. Stevenson, R. J. (2009). The Psychology of Flavour. Oxford: Oxford University Press. Streri, A. (2012). ‘Cross-modal interactions in the human newborn’. In A. Bremner, D. Lewkowicz, and C. Spence (eds), Multisensory Development (pp. 88–112). Oxford: Oxford University Press. Thompson, B. (2009). ‘Senses for senses’. Australasian Journal of Philosophy, 87, 99–117.

620    Tim Bayne and Charles Spence Thompson, B. (2010). ‘The spatial content of experience’. Philosophy and Phenomenological Research, 81, 146–184. Trommershäuser, J., Landy, M. S., and Körding, K. P. (eds) (2011). Sensory Cue Integration. New York: Oxford University Press. Tye, M. (2000). Consciousness, Color, and Content. Cambridge, MA: MIT Press. Tye, M. (2003). Consciousness and Persons. Cambridge, MA: MIT Press. Vatakis, A. and Spence, C. (2010). ‘Audiovisual temporal integration for complex speech, object-action, animal call, and musical stimuli’. In M. J. Naumer and J. Kaiser (eds), Multisensory Object Perception in the Primate Brain (pp. 95–121). New York: Springer. Von Melchner, L., Pallas, S. L., and Sur, M. (2000). ‘Visual behaviour mediated by retinal projections directed to the auditory pathway’. Nature, 404, 871–876. Welch, R. B. (1999). ‘Meaning, attention, and the "Unity Assumption" in the intersensory bias of spatial and temporal perceptions’. In G. Ascherleben, T. Bachman, and J. Müsseler (eds), Cognitive Contributions to the Perception of Spatial and Temporal Events (pp. 371–387). Amsterdam: Elsevier. Welch, R. B. and Warren, D. H. (1980). ‘Immediate perceptual response to intersensory discrepancy’. Psychological Bulletin, 88, 638–667. Welch, R. B. and Warren, D. H. (1986). ‘Intersensory interactions’. In K. R. Boff, L. Kaufman, and J. P. Thomas (eds), Handbook of Perception and Performance: Vol. 1. Sensory Processes and Perception (pp. 25–36). New York: Wiley.

Chapter 33

Perceptua l Consta ncy Jonathan Cohen

Our eyes deceive us when we look down railway tracks, but our brains do not. The rails appear to converge in the distance, but we know that the rails are parallel. We know that they are the same distance apart a mile down the track as they are where we are standing, so the brain says, ‘The tracks only appear to converge because they are distant.’ But how does the brain know that the tracks are distant? The brain answers, ‘They must be distant because they appear to converge.’ (The flow of this logic must shock computer programmers, but they are accustomed to the limitations of inferior hardware.) (Hunter et al., 2007: 82)

1 Introduction Students of perception have long known that perceptual constancy is an important aspect of our perceptual interaction with the world. Here is a simple example of the phenomenon concerning colour perception: there is some ordinary sense in which an unpainted ceramic coffee cup made from a uniform material looks a uniform colour when it is viewed under uneven illumination, even though the light reflected by the shaded regions to our eyes is quite different from the light reflected by the unshaded regions to our eyes (see Figure 33.1). Or consider this example concerning size perception: there is some ordinary sense in which two telephone poles look the same size when the first is viewed from 100 metres and when the second is viewed from 1 metre, even though the visual angle subtended by the two poles on our retinae is very different (see Figure 33.2). Or consider this example concerning shape perception: there is some ordinary sense in which a penny looks round both when viewed head on and when viewed from an acute angle, even though the area projected by the penny onto our retinae under these two conditions is very different (see Figure 33.3). Or, finally, consider this example concerning auditory volume perception (which I cannot depict graphically): there is some ordinary sense in which a speaker’s voice sounds the same volume when heard from across the room and when heard from a distance of 1 metre, even though the energy striking our ears under these two conditions is very different.

622   Jonathan Cohen

Fig 33.1  There is some good sense in which the regions of the cup in shadow and the regions of the cup in direct sunlight look the same in colour. Photograph © Jonathan Cohen. The kind of perceptual constancy exemplified in these cases, and others like them, is ubiquitous, ordinary, and central to the way perception tells us about the world in which we live. Without this kind of constancy, we would experience the world as a Jamesian blooming, buzzing confusion—a constant flux of colours, shapes, and sounds with no apparent organization. For, unavoidably, the perceptual signals incident on our transducers are the results of not only the kinds of distal individuals there are and properties they exemplify, but also the constantly changing details of the circumstances under which we perceive (the angle and distance from the perceived object, the lighting conditions, the ambient noise, our own cognitive and perceptual histories and futures, our expectations, and so on). If perception were incapable of representing the world as in some ways constant despite various changes in our perceptual circumstances, it would radically misrepresent the distal world: it would fail to reveal ways in which the world is stable. And since these ways underpin our engagement with that world, this would (disastrously) undermine the possibility of effective action and empirical knowledge. However, despite its recognized ubiquity and importance, there are several respects in which the phenomenon of perceptual constancy is poorly understood. Aside from the independent interest in getting clear on these matters, perceptual constancy has figured prominently in recent debates about the ontology of colours and other sensible qualities, knowledge, attention, mental modularity, the contents of mental representation, and the objectivity of our representations of the world.1 Therefore, in this chapter I’ll review some 1  Recently a number of philosophers have returned to issues about constancy anew; for example, see Hilbert (2005); Thompson (2006); Cohen (2008); Bradley (2008); Hatfield (2009); Gert (2010); Matthen (2010); Wright (2013). Also see Burge (2010), for whom perceptual constancy is used as a touchstone for the objectivity of intentional representation quite generally.

Fig. 33.2  There is some good sense in which the telephone poles seen from different distances look the same size. Photograph © Jonathan Cohen.

Fig. 33.3  There is some good sense in which the penny looks the same in shape when seen from two different angles. Photograph © Jonathan Cohen.

624   Jonathan Cohen of what is and is not known about perceptual constancy with an eye to drawing connections with ongoing controversies in the philosophy of perception and elsewhere.2

2  Perceptual constancy as perceptual stability As both its name and the initial examples used to introduce the phenomenon above suggest, perceptual constancy is, in some sense yet to be explained, about the absence of change. Indeed, the textbook characterization has it that perceptual constancy is nothing more or less than a stability in perceptual response across a range of varying perceptual conditions.3 Thus, in the case of the unevenly illuminated coffeecup (Figure 33.1), the idea is that the perceptual system represents the distinct regions of the cup as bearing the same colour even though there is variation in the illumination incident on them (and, therefore, in the total amount of light energy they reflect to our retinal transducers). Or, again, in the case of volume perception, the thought is that perception represents the speaker’s voice as having the same volume even though there is significant variation in the distance from which it is heard (and, therefore, in the total amount of auditory energy absorbed by our aural transducers). While I will want to qualify the above characterization in what follows, one of the ways in which it is useful and interesting is that it presents perception as an active process of engagement with the world. It suggests that perception is not just a matter of passively registering the impinging energy array, but of somehow articulating or decomposing that array to arrive at a representation of a subset of the distal features that contribute to the configuration of the array. Unfortunately, the textbook characterization of perceptual constancy just presented can’t be quite right by itself. (Or, alternatively, we can retain that characterization by itself, but only at the cost of emptying the phenomenon of all of its instances.) For it is not true that our perceptual responses are entirely constant in the kinds of cases at issue. Returning once again to the unevenly illuminated coffeecup, we know there must be a difference in a subject’s perceptual response to the shaded and unshaded regions of the cup, or else she would be unable to discriminate the luminance boundary between them. Likewise in canonical cases of size constancy (subjects’ perceptual responses can clearly distinguish in some size-related way between the perception of the telephone pole at 100m and the perception of the telephone pole at 1m), shape constancy (there is clearly a discriminable difference between the subject’s perception of the penny seen head on and her perception of the penny seen at an acute angle), auditory volume constancy (there is clearly a discriminable difference between the subject’s perception of the speaker’s voice from across the room

2 Because there is vastly more research, by both philosophers and psychologists, on perceptual constancy in vision than in other modalities (and, even more particularly, on colour constancy), this entry is, regrettably, unavoidably visuocentric in its choice of examples and theories discussed. There remains much work to be done in this area. 3  See, for example, Byrne and Hilbert (1997: 445), Zaidi (1999: 339), Palmer (1999: 312–314, 723), Goldstein (1999: 567), Brainard et al. (2003: 308–309).

Perceptual Constancy   625

Fig. 33.4  An instance of simultaneous lightness contrast: the central patches are qualitatively identical, but perception represents them differently because of the contrast with surrounding  items. and her perception of the speaker’s voice from a distance of 1 meter), and all of the other canonical instances of perceptual constancy. Indeed, the non-constancy of our perceptual responses across variations in the perceptual circumstances is not only immediately apparent, but underlies another much-observed and much-discussed aspect of perception—the phenomenon of perceptual contrast.4 It is easy to find instances of perceptual contrast once one begins to look for them. For example, Figure 33.4 illustrates an instance of simultaneous lightness contrast: although the two central patches depicted here are qualitatively intrinsically identical, the perceptual system represents them as different in colour because of the different ways in which they contrast in lightness with surrounding items. Simultaneous lightness contrast plays a role in many classic visual illusions, such as the appearance of grey dots at the intersections of an achromatic grid (the Hermann grid illusion, Figure 33.5), the interpretation of a pair of opposed lightness gradients as two constant lightness regions separated by an edge (the Cornsweet illusion, Figure 33.6), and the appearance of light or dark bands next to the boundary between two different lightness gradients, even when the lightness on both sides of the boundary is the same (Mach bands, Figure 33.7).5 Perceptual contrast is by no means restricted to the perception of lightness/brightness; within vision there are also simultaneous contrast effects for chromatic colour, size, spatial frequency, orientation, motion, and speed, inter alia. For example, Figure 33.8 illustrates an instance of simultaneous size contrast: although the central circles are the same geometric size, the perceptual system represents them as different in size because of the contrast with the different elements surrounding them. Moreover, in addition to simultaneous contrast—contrast between simultaneously perceived items, there are also ubiquitous instances of successive contrast—effects of contrast between successively perceived items for each of these dimensions. And, of course, contrast occurs in non-visual modalities

4 

Whittle (2003) provides an excellent overview of the importance of perceptual contrast for colour vision. 5  For a discussion of the role of contrast in many lightness illusions, see Adelson (2000).

626   Jonathan Cohen

Fig.  33.5  The Hermann grid illusion.

Fig.  33.6  The Cornsweet illusion. as well (although there is much less systematic investigation of contrast outside vision). Thus, in gustation, we commonly observe that sweet wines strike us as markedly less sweet when consumed with dessert items (which contain much more sugar than the wines) than on their own. In audition, we find that it is much easier to detect variations in pitch (say, while tuning a guitar string) by contrasting the target against other (simultaneously or successively perceived) tones. Or, again, in kinaesthesia, Gibson (1933) reports that after

Perceptual Constancy   627

Fig.  33.7 Mach  bands.

Fig.  33.8 The Ebbinghaus illusion is an instance of perceptual simultaneous size contrast. blindfolded subjects run their fingers over a curved surface for three minutes, straight edges seem to them to be curved in the opposite direction. In each of these cases, the perceptual system reacts differently to objects depending on how they contrast with other perceived items. Perceptual contrast occurs because perceptual systems tend to be responsive to magnitude differences, as opposed to magnitudes themselves.6 For our purposes, the phenomenon of contrast is important because 6  The standard physiological explanation of this generalization turns on lateral inhibition between neurons carrying perceptual information (e.g. retinal ganglion cells, in the case of lightness perception).

628   Jonathan Cohen it makes for a vivid demonstration of the observation made above: contrary to the textbook characterization, our perceptual responses to an object/property are not constant, but instead change in interesting and systematic ways across variations in the perceptual circumstances.7

3  Psychophysics and measurement So far our discussion has been framed by questions of which qualitative discriminations are made by perceivers. However, for many purposes it is useful to have quantitative measures of similarity/dissimilarity in cases of perceptual constancy. The standard technique used for this purpose is to measure the dissimilarity between a subject’s reaction to two stimuli by measuring how much of a change she must make to one of them, holding the other fixed, before she regards the two as a perceptual match.8 Thus, for example, the main quantitative measure by which contemporary psychophysicists assess colour constancy, known as asymmetric colour matching (Wyszecki and Stiles, 1982: 281–293), involves asking subjects to change the chromaticity (or lightness, in lightness constancy experiments) of a test patch under one illuminant until it perceptually matches a standard patch under a different illuminant. The size of the chromaticity (/lightness) difference between the test and the standard patches required to achieve a perceptual match, then, is a quantitative measure of the effect of the illumination difference between test and standard patches on the subject’s total perceptual response to them—it is an operational measure of the extent to which perceptual responses are unchanging across variations in perceptual conditions. Such quantitative measures reinforce the assessment made above on the strength of qualitative reactions: in canonical instances of colour constancy, subjects’ perceptual responses are not simply unchanging—rather, they are in some respects similar or unchanging and in some other respects dissimilar or changing. Moreover, interestingly, (most) subjects can be made to switch between attending to the respects of similarity and the respects of dissimilarity in many canonical instances simply by changing the experimental instructions. For example, Arend and Reeves (1986) found that subjects in an asymmetric colour matching paradigm responded to instructions to ‘adjust the test patch Lateral inhibition results in the suppression of all but the most stimulated/least inhibited neurons; consequently, the overall firing pattern is highest in cells corresponding to parts of the stimulus where there is a steep spatial/temporal gradient—where a small population of most active cells is left relatively uninhibited by the firing of their neighbours. 7  Objection: The cases I have used to highlight contrast (the Hermann grid illusion, the Ebbinghaus illusion, etc.) are often put forward as textbook cases of perceptual illusion. They give no reason to suppose there is substantial non-constancy in veridical cases of perception. Response: Contrast is pretty clearly at work in ordinary perception; I have relied on textbook cases of perceptual illusion only because they make the results of perceptual contrast so vividly apparent. However, a theory of perception that set aside cases involving the operation of perceptual constancy would have little to say about the kinds of perceptual systems we happen to enjoy. 8  Note that perceptual matching is a statistical notion: two stimuli count as a perceptual match for a subject if the subject is unable to discriminate one from the other over several presentations at a rate higher than that attributable to chance.

Perceptual Constancy   629 to match its hue and saturation to those of the standard patch’ (1986: 1744) by making large chromaticity changes (suggesting that their perceptual systems initially represented the test and the standard patch as quite different), although the same subjects responded to instructions to ‘adjust the test patch to look as if it were “cut from the same piece of paper” as the standard, i.e. to match its surface color’ (1986: 1744) by making very small chromaticity changes (suggesting that their perceptual systems initially represented the test and the standard patch as quite similar).9

4  Stability and instability It seems, then, that the right thing to say is not, or not just, that the perceptual system responds in a constant or unchanging way in the face of variations in the perceptual conditions— either as a general matter or even in the cases that have been put forward as parade instances of perceptual constancy. On the other hand, neither does it seem that the perceptual system responds by treating objects as merely approximately the same in different perceptual conditions—the similarities and dissimilarities that perception recognizes are not collapsed into a single scalar value somewhere between the extremes of perfect qualitative match and perfect qualitative mismatch. Rather, what we should say is that perception represents both some aspects of similarity and some aspects of dissimilarity in its responses to objects across changing perceptual circumstances. Moreover, we should recognize that both the respects of similarity and the respects of dissimilarity are in many cases available to the perceiving subject for the purpose of making perceptual discriminations.10 This raises an important puzzle for the understanding of perceptual constancy. Given that there is clearly substantial variation in our 9  That the perceptual system displays this sort of bimodal behaviour has been understood for a long time; see Evans (1948; 163–164); Beck (1972: 66–67) for an overview of some of the earlier work. For more recent work (mostly on cases of simultaneous colour constancy), see Blackwell and Buchsbaum (1988); Valberg and Lange-Malecki (1990); Arend et al. (1991); Troost and deWeert (1991); Cornelissen and Brenner (1995); Bäuml (1999). While there has been far less systematic investigation of this effect with respect to cases of successive colour constancy, investigators have found the same sort of bimodal pattern of results here too (Delahunt (2001: 114–117); Delahunt and Brainard (2004: 71–74)). 10  Many philosophers and psychologists working in this area have tended to be so impressed by the constant aspects of our perceptual responses that they have played down, dismissed, or, more frequently, just ignored the inconstant aspects of our perceptual responses to the same scenarios. Thus, one sometimes sees assertions to the effect that the inconstant aspects of perception are ‘unnatural and sophisticated . . . [and] difficult to attain’ (Smith, 2002: 182, cf. 178). Whatever else we think of such claims, I suggest that an adequate theory of perception must account for all of the ways in which perceptual systems respond to the world rather than only some of them—whether these responses are natural or unnatural, naive or sophisticated, and easily attained or not. Emphasis on constant aspects of our perceptual responses at the expense of inconstant aspects also shows up in a prominent line of argument for the view that colours are illumination-independent features of objects (I discuss these arguments critically in Cohen, 2008). For example, Tye (2000: 147–148), Hilbert (1987: 65), and Byrne and Hilbert (2003: 9) explicitly appeal to constancy reactions in colour perception as cases where the very same feature can be extracted despite variation in the ambient illumination, and infer from this claim that colour (which they reasonably assume is indeed represented by colour perception) is itself illumination-independent. However, if it is reasonable to take constancy reactions to show that perception represents constant features, it is no less (and no more) reasonable to take inconstancy reactions to show that perception represents inconstant features. But if colour perception represents both

630   Jonathan Cohen perceptual responses to objects across changes in perceptual circumstances even in canonical cases of constancy (such as those used to introduce the topic in section 1), it won’t do to think of constancy simply in terms of stability of perceptual response. Rather, if we want to be able to say that there is perceptual constancy in such canonical cases, then we owe a characterization of just which kinds of perceptual similarity, in the context of just which kinds of variation in perceptual circumstances, are necessary for the exemplification of perceptual constancy. Moreover, we need a characterization that is applicable across the broad range of cases to which we want to apply the notion. Unfortunately, there is at present no adequate and fully general characterization of this sort, and therefore no general understanding of what perceptual constancy amounts to.

5  Computation and constancy While the problems just discussed should not be underplayed, neither should they make us lose sight of the initial observation that makes perceptual constancy so interesting: in canonical cases there is some interesting respect in which perception is unchanging in its treatment of an object despite differences in the conditions under which it is perceived, and despite the attendant differences in the total signals impinging on our sensory transducers, even if these must be characterized in a case by case way. This observation naturally invites the important question about how perception pulls off the feats of constant representation in the face of inconstant perceptual circumstances that it does. That is, given the complex total signal striking the transducers—a signal that is determined jointly by the features of perceived objects and perceptual circumstances, and therefore that changes as circumstances vary—how does the perceptual system arrive at a verdict about whether the perceived objects change? How, for example, does the perceptual system start with the varying array of light intensities reflected by the cup in Figure 33.1 and end with the information that the entire cup is uniform in colour (or, more cautiously, in some colour-related respect)? A burgeoning subfield of perceptual psychology has attempted to build empirically adequate computational models that would answer this question. Perhaps the dominant approach within this tradition is to think about perception as computing a solution to an ‘inverse problem’: the job is to find ways of factoring apart the complex resultant that is the impinging energy array to arrive at a representation of the distal features that contribute to the resultant. Thus, for example, consider colour constancy once again, since that is the area in which the most intense research on computational methods has been carried out.11 constant and inconstant features, there is no sound inference from the premiss that colour is represented by perception to the conclusion that colour is a constant (here, illumination-independent) feature. (Nor, for that matter, is there a sound inference from that premiss to the conclusion that colour is an inconstant/ illumination-dependent feature.) Consequently, the sort of appeal to perceptual constancy made by these authors does not successfully motivate the claim that colours are illumination-independent object features. 11  Much of the work in this tradition is restricted to the perception of surface colours (as opposed to the colours of lights, volumes, films, and so on). Moreover, many (but not all) of the models depend on the simplifying assumptions that surfaces are illuminated by constant or smoothly varying, and exclusively diffuse, illumination.

Perceptual Constancy   631 In colour perception the perceptual system begins with an array of light intensities on the retina which is the joint product of two factors—the features of the illumination incident on surfaces and those of the surfaces that reflect light to our eyes. The leading approach to computational colour constancy has involved finding methods of estimating the properties of the illuminant so that the system can, as it were, subtract off this factor from the total signal (in Helmholtz’s phrase, ‘discounting the illuminant’), leaving an illumination-independent characterization of the reflecting surface (Maloney and Wandell, 1986; Brainard et al., 1997; Brainard, 1998). Crucially, since this characterization is illumination-independent, the thought is that it will be shared by distinct regions of a uniform surface that happen to be illuminated differently (e.g. the regions of the cup in Figure 33.1). Therefore, a perceptual system that performed this sort of computation would be able to treat such regions as (in this one respect) perceptually similar, even though they are clearly discriminably different. Modellers have pursued a wide variety of strategies for estimating the separate contributions to the retinal array made by illuminants and surfaces. For example, Maloney (1986); Maloney and Wandell (1986) show how a system with more classes of receptors than there are degrees of freedom in (the system’s linear models of) surface reflection profiles can exploit its multiple receptoral signals to recover representations of surfaces. Other approaches solve the inverse problem by adding as constraints assumptions about the kinds of scenes perceptual systems will encounter. Thus, Buchsbaum (1980) proposes a model that rests on the assumption that the median lightness value in a scene corresponds to a middle grey surface, and computes from this assumption what the incident illumination would have to be to result in the observed intensity array. A related but distinct strategy proceeds from the assumption that anchors some part of the visual image (rather than a mean) to an extremal lightness value—for example, by treating the lightest visible surface as white (Land and McCann, 1971; Gilchrist et al., 1999). Others have proposed estimating illuminants from information about mutual reflections in the scene (Funt et al., 1991), the boundaries of regions known to be specular reflections (D’Zmura and Lennie, 1986; Lee, 1986), and shadows (D’Zmura, 1992). Still others propose to solve the inverse problem by appeal to higher-order scene statistics, such as the correlation between redness and luminance within the scene (Golz and MacLeod, 2002) or the statistical distribution of colours within the scene (MacLeod, 2003; Brainard et al., 2006). In recent years, many theorists have advocated ‘Bayesian’ probabilistic models as solutions to the illuminant estimation problem. According to Bayesians, the visual system first selects as its estimate that hypothesis about the illuminant with the highest probability conditional on the data received by the transducers, constrained by the prior probability of that illuminant hypothesis; then it goes on to select as its estimate about distal surfaces that hypothesis with the highest probability conditional on the transducer data and the illuminant estimate obtained at the first step, again constrained by prior probabilities assigned to the various hypotheses about surfaces (Brainard and Freeman, 1997).12 It is possible, of course, that human colour constancy involves a combination of these methods, or others. 12 In

such models, the kinds of substantive assumptions about the distal world that ground the deterministic models described above—e.g. about the way illuminants vary smoothly in ecological settings, about where the mean lightness values can be expected, and so on—show up as well, but here in the form of the prior probabilities about both illuminant and surfaces used to constrain the assignment of posterior probabilities.

632   Jonathan Cohen However, there is a different class of computational models for perceptual constancy—one that has received much less attention from philosophers—that rejects the assumption that constancy requires factoring out of the perceptual signal a representation of the distinctive contribution made by the perceived object and its features. Thus, Craven and Foster (1992); Foster and Nascimento (1994); Dannemiller (1993); Zaidi (1998, 2001); Amano et al. (2005) suggest that perceptual systems compute colour constancy not by deriving an illumination-independent representation of object surfaces, but by comparing total perceptual signals in light of what is known about the illumination or other properties of the total scene. Crudely, the idea is that the system can ask whether the difference between the two perceptual signals it gets from two perceptual episodes (simultaneous or not) can be accounted for by the behaviour of the illumination (rather than by a difference in the surfaces perceived on the two occasions). If, say, the system represents that the illumination profile includes a shadow cast over the scene (say, by a partially occluded light source) then this would have predictable effects on the perceptual signal: there would be higher intensities in the (portion of the) signal corresponding to the directly illuminated regions and lower intensities corresponding to the (portion of the) signal corresponding to the region in shadow. Therefore, the system can treat the image regions as being relevantly alike although they cause different perceptual signals (i.e. it can display perceptual constancy) if it can conclude that the two different perceptual signals lie in the graph of a transformation consistent with illumination variations. Here, as in more traditional computational models, the computation of colour constancy depends on deriving from the perceptual signal an estimate of the illumination. But unlike more traditional models, the suggestion is that the system can compute constancy directly from the perceptual signal and the illumination estimate, without going to the trouble of separately deriving a closed-form representation of object surfaces. Also unlike more traditional models, here there is no suggestion that the perceptual system discounts or discards the illuminant—on the contrary, the claim is that the system’s continuing to represent the illuminant is absolutely vital to the computation of constancy.13 And, though these are proposals about colour constancy in particular, the general lessons they teach may well be applicable for other visual and non-visual instances of perceptual constancy as well.

6  Is perceptual constancy perceptual? Perceptual constancy shows that perceivers are not passive receivers of the array of energy falling on their receptors—for if they were, they could not react in similar ways (in some respects), as they sometimes do, when there are large differences in that array. 13 

There are several further pieces of evidence that confirm the prediction of such models that perceptual systems maintain representations of the illumination rather than simply discarding them. Perhaps the most direct is just that subjects can, when asked, make matches of ambient illumination as opposed to surface lightness (Katz, 1935; Gilchrist, 1988; Hurlbert, 1989; Jameson and Hurvich, 1989; Zaidi, 1998). It is worth noting that the possibility of computing constancy without deriving specific object/ object-property representations undercuts the (oft-made) claim that object tracking and reidentification depend on representing condition-independent object properties.

Perceptual Constancy   633 Something more must be going on. But is that something more a perceptual process? Or is it a post-perceptual process that gets its start at the point where perception ends? It is clear that, for example, subjects will (under some experimental instructions) judge that the penny in Figure 33.3 is relevantly alike in shape when presented from two distinct viewpoints. But what is not clear is whether that judgement is informed by the output of perceptual systems by themselves, or by the integration of perceptual systems together with certain kinds of cognitive corrective factors (e.g. memories about the canonical colours, shapes, etc. of similar objects).14 An early instance of a post-perceptual/cognitive view about perceptual constancy is the proposal, defended by von Helmholtz (1962) and Hering (1964), that colour constancy is (at least in part) driven by our memory/knowledge about the colours of familiar objects.15 This ‘memory theory’ of colour constancy faces several difficulties. First, Katz (1911) showed that there is colour constancy for random and presumably newly encountered objects (for which there could not be colour memory), and thereby demonstrated that the sort of memory/ knowledge enlisted by the memory theory is not necessary for successful colour constancy. Second, it is doubtful that our memory for colour is sufficiently accurate to underwrite observed levels of constancy (Hurvich, 1981: 2; Halsey and Chapanis, 1951: 1058). A third line of concern for memory (and, more generally, cognitive) explanations of colour constancy is that one can dissociate the capacity for colour constancy from (what are generally taken to be) cognitive capacities in both directions. In one direction, there appears to be robust colour constancy in goldfish, honeybees, and several other non-human animals (see the review in Neumeyer (1998)) and human infants somewhere between 9 and 20 weeks old (Dannemiller and Hanko, 1987; Dannemiller, 1989), whose cognitive/conceptual resources are usually assumed to be pretty limited. In the other direction, there is (admittedly more limited) evidence from lesion studies where colour constancy is impaired but memory and other conceptual capacities are spared (Rüttiger et al., 1999). These reasons, among others, have led investigators to search for less obviously cognitive explanations of colour constancy. For example, contemporary explanations of colour constancy often cite several kinds of retinal adaptation (changes in the sensitivity of 14  Obviously, one’s approach to this last question will be shaped, in part, by how one understands the cognition/perception distinction. I won’t attempt to settle this vexed issue here, but will simply take for granted that, e.g., memory for the colours/shapes/sizes/etc. of objects and other apparent instances of concept deployment fall on the cognitive side of the divide, and that, e.g. receptoral adaptation effects are perceptual. What is at stake is (of course) not the labels, but instead what kinds of causal explanatory resources are invoked to explain observed instances of perceptual constancy. 15  An even earlier post-perceptual view of constancy emerges from Locke’s discussion of the role of judgement in sensation:

When we set before our eyes a round globe of any uniform colour  . . .  it is certain that the Idea thereby imprinted in our Mind, is of a flat Circle variously shadow’d, with several degrees of Light and Brightness coming to our Eyes. But we having by use been accustomed to perceive, what kind of appearance convex Bodies are wont to make in us; what alterations are made in the reflections of Light, by the difference of the sensible Figures of Bodies, the Judgment presently, by an habitual custom, alters the Appearances into their Causes: So that from that, which truly is variety of shadow or colour, collecting the Figure, it makes it pass for a mark of Figure, and frames to it self the perception of a convex Figure, and an uniform Colour; when the Idea we receive from thence, is only a Plain variously colour’d, as is evident in Painting. (Locke, 1975: II.ix.8)

634   Jonathan Cohen retinal receptors as a response to incident light) including adaptation over temporally and spatially local regions (so-called von Kries adaptation), adaptation to the spatial mean of the whole scene, and adaptation to the region of highest intensity in the scene (McCann, 2004). However, there is evidence suggesting that these factors are not always sufficient for colour constancy by themselves (Kraft and Brainard, 1999). Moreover, even if they are not by themselves sufficient for constancy, it appears that cognitive factors may make an important contribution to constancy after all: several investigators have found that familiarity for types of objects perceived (e.g. common fruits and vegetables) enhances colour constancy (Hurlbert and Ling, 2005; Olkkonen et al., 2008, 2012). A similarly complicated mix of findings seems to be the pattern for shape and size constancy. On the one hand, there is evidence that the visual system can in some conditions (e.g. at short distance ranges) compute constant size and shape from relatively low-level perceptual cues such as vergence (information about the relative ocular positions of the two eyes in their sockets) and disparities in the retinal projections from the two eyes. And, once again, there is double dissociation between constancy for size/shape and cognitive sophistication. Thus, for example, there appears to be size constancy (at least at short distance ranges) in comparatively cognitively unsophisticated creatures such as newborn human beings (Granrud, 2006, 2012; Slater et al., 1990) non-human primates (Fujita, 1997; Barbet and Fagot, 2002), goldfish (Douglas et al., 1988), and amphibians (Ingle, 1998). And, in the other direction, Cohen et al. (1994) give evidence of the selective impairment of certain kinds of size constancy that spare general cognitive abilities. All that said, it is also true that higher-level, cognitive cues—for example, memory for the canonical shape and size of recognized objects, comparison to other perceived items whose shape and form are established independently, the smoothed appearance of texture from a distance—enhance shape and size constancy substantially (for a useful overview, see Palmer, 1999: ch. 5, ch. 7). Cumulatively, these results strongly suggest that perceptual constancy is neither exclusively perceptual nor exclusively cognitive. Instead, it appears that ‘the’ phenomenon of perceptual constancy, even considered as constancy for a single dimension of a single quality within a single modality (e.g. just for lightness), is an interaction effect produced by several different mechanisms operating across different spatial and temporal scales—some possibly more and some possibly less cognitive than others, depending on how one chooses to mark the cognitive/non-cognitive distinction.16 Whether any one of these mechanisms contributes to perceptual constancy on any particular occasion will depend on the details of many features of the perceptual circumstance.

7 Conclusion While I have argued that the perceptual stabilities emphasized by traditional characterizations of perceptual constancy can only be part of the story, it remains true, indisputable, and important that some aspects of our perceptual responses are stable even through

16  Cf. Foster (2003), who points to the heterogeneity of the factors in operation as a reason to be sceptical about the very existence of colour constancy.

Perceptual Constancy   635 changes in perceptual circumstances that result in changes in transduced perceptual signals. It is no less indisputable that there are important lessons to be learned from the phenomenon of perceptual constancy, although many unresolved questions remain. As we have seen, there is no completely general account of which dimensions of perceptual response must remain fixed, and which may vary, across which kinds of variation in perceptual conditions, for a perceptual episode to count as an instance of perceptual constancy. Moreover, there is no general understanding of the relation between perceptual constancy and perceptual contrast. And, partly because so much less is understood about both constancy and contrast in non-visual modalities, it is so far unclear what (if any) systematic cross-modal generalizations hold for each. Finally, the range of computational strategies that perception uses to extract stabilities, of the mechanisms underlying their implementation, and of the ways these distinct strategies and mechanisms are combined with one another in real-time perception remain incompletely understood. Notwithstanding these substantial gaps in our knowledge, it seems clear that constancy is an absolutely fundamental aspect of perception, and therefore that it will figure centrally in our ultimate understanding of mind–world interaction.17

References Adelson, E. H. (2000). 'Lightness perception and lightness illusions'. In M. Gazzaniga (ed.), The New Cognitive Neurosciences, 2nd edn (pp. 339–351). Cambridge, MA: MIT Press. Amano, K. D., Foster, D. H., and Nascimento, S. M. C. (2005). 'Minimalist surface-colour matching'. Perception, 34, 1007–1011. Arend, L. and Reeves, A. (1986). 'Simultaneous color constancy'. Journal of the Optical Society of America A, 3(10), 1743–1751. Arend, L., Reeves, A., Schirillo, J., and Goldstein, R. (1991). 'Simultaneous color constancy: patterns with diverse Munsell values'. Journal of the Optical Society of America A, 8, 661–672. Barbet, I. and Fagot, J. (2002). 'Perception of the corridor illusion by baboons'. Behavioural Brain Research, 132(1), 111–115. Bäuml, K.-H. (1999). 'Simultaneous colour constancy: How surface color perception varies with the illuminant'. Vision Research, 39(8), 1531–1550. Beck, J. (ed.) (1972). Surface Color Perception. Ithaca, NY: Cornell University Press. Blackwell, K. T. and Buchsbaum, G. (1988). 'Quantitative studies in color constancy'. Journal of the Optical Society of America A, 5, 1772–1780. Bradley, P. (2008). 'Constancy, categories and Bayes: A new approach to representational theories of color constancy'. Philosophical Psychology, 21(5), 601–627. Brainard, D. H. (1998). 'Color constancy in the nearly natural image 2. achromatic loci'. Journal of the Optical Society of America A, 15(2), 307–325. Brainard, D. H. and Freeman, W. T. (1997). 'Bayesian color constancy'. Journal of the Optical Society of America A, 14(7), 1393–1411. Brainard, D. H., Brunt, W. A., and Speigle, J. M. (1997). 'Color constancy in the nearly natural image I. asymmetric matches'. Journal of the Optical Society of America, A, 14(9), 2091–2110.

17  Many thanks to Damon Crockett, Joshua Gert, Gary Hatfield, Don MacLeod, Mohan Matthen, and Sam Rickless for discussion and comments on earlier drafts.

636   Jonathan Cohen Brainard, D. H., Kraft, J. M., and Longere, P. (2003). 'Color constancy: Developing empirical tests of computational models'. In R. Mausfeld and D. Heyer (eds), Colour Perception: Mind and the Physical World (pp. 307–328). New York: Oxford University Press. Brainard, D. H., Delahunt, P. B., Freeman, W. T., Kraft, J. M., and Xiao, B. (2006). 'Bayesian model of human color constancy'. Journal of Vision, 6, 1267–1281. Buchsbaum, G. A. (1980). 'A spatial processor model for object colour perception'. Journal of the Franklin Institute, 310, 1–26. Burge, T. (2010). Origins of Objectivity. Oxford: Oxford University Press. Byrne, A. and Hilbert, D. R. (eds) (1997). Readings on Color, Volume 2: The Science of Color. Cambridge, MA: MIT Press. Byrne, A. and Hilbert, D. R. (2003). 'Color realism and color science'. Behavioral and Brain Sciences, 26(1), 3–64. Cohen, J. (2008). 'Color constancy as counterfactual'. Australasian Journal of Philosophy, 86(1), 61–92. Cohen, L., Gray, F., Meyrignac, C., Dehaene, S., and Degos, J.-D. (1994). 'Selective deficit of visual size perception: Two cases of hemimicropsia'. Journal of Neurology, Neurosurgery, and Psychiatry, 57, 73–78. Cornelissen, F. W. and Brenner, E. (1995). 'Simultaneous colour constancy revisited: An analysis of viewing strategies'. Vision Research, 35, 2431–2448. Craven, B. J. and Foster, D. H. (1992). 'An operational approach to colour constancy'. Vision Research, 32(7), 1359–1366. Dannemiller, J. L. (1989). 'A test of color constancy in 9- and 20-week-old human infants following simulated illuminant changes'. Developmental Psychology, 25, 171–184. Dannemiller, J. L. (1993). 'Rank orderings of photoreceptor photon catches from natural objects are nearly illuminant-invariant'. Vision Research, 33, 131–140. Dannemiller, J. L. and Hanko, S. A. (1987). 'A test of color constancy in 4-month-old human infants'. Journal of Experimental Child Psychology, 44, 255–267. Delahunt, P. B. (2001). 'An Evaluation of Color Constancy Across Illumination and Mutual Reflection Changes'. Ph.D. thesis, University of California, Santa Barbara. Delahunt, P. B. and Brainard, D. H. (2004). 'Does human color constancy incorporate the statistical regularity of natural daylight?' Journal of Vision, 4, 57–81. Douglas, R. H., Eva, J., and Guttridge, N. (1988). 'Size constancy in goldfish (carassius auratus).' Behavioural Brain Research, 30, 37–42. 'D’Zmura, M. (1992). Color constancy: surface color from changing illumination'. Journal of the Optical Society of America A, 9, 490–493. D’Zmura, M. and Lennie, P. (1986). 'Mechanisms of color constancy'. Journal of the Optical Society of America A, 3, 1662–1672. Evans, R. M. (1948). An Introduction to Color. New York: Wiley. Foster, D. H. (2003). 'Does colour constancy exist?' Trends in Cognitive Science, 7(10), 439–443. Foster, D. H. and Nascimento, S. M. C. (1994). Relational colour constancy from invariant cone-excitation ratios. Proceedings of the Royal Society of London B, 257, 115–121. Fujita, K. (1997). Perception of the Ponzo illusion by rhesus monkeys, chimpanzees and humans: similarity and difference in the three primate species. Perception & Psychophysics, 59, 284–292. Funt, B., Drew, M., and Ho, J. (1991). Color constancy from mutual reflection. International Journal of Computer Vision, 6, 5–24. Gert, J. (2010). 'Color constancy, complexity, and counterfactual'. Noûs, 44(4), 669–690.

Perceptual Constancy   637 Gibson, J. J. (1933). 'Adaptation, after-effect and contrast in the perception of curved lines'. Journal of Experimental Psychology, XVI(1), 1–31. Gilchrist, A. L. (1988). 'Lightness contrast and failures of constancy: A common explanation'. Perception & Psychophysics, 43, 415–424. Gilchrist, A. L., Kossifydis, C., Bonato, F., Agostini, T., Cataliotti, J., Li, X., Spehar, B., Annan, V., and Economou, E. (1999). 'An anchoring theory of lightness perception'. Psychological Review, 106(4), 795–834. Goldstein, E. B. (1999). Sensation & Perception 5th edn. Pacific Grove, CA:  Brooks/Cole Publishing. Golz, J. and MacLeod, D. I. A. (2002). 'Influence of scene statistics on colour constancy'. Nature, 415, 637–640. 'Granrud, C. E. (2006). Size constancy in infants: 4-month-olds’ responses to physical versus retinal image size'. Experimental Psychology: Human Perception and Performance, 32(6), 1398–1404. Granrud, C. E. (2012). 'Judging the size of a distant object: Strategy use by children and adults'. In G. Hatfield and S. Allred (eds), Visual Experience: Sensation, Cognition, and Constancy ­(ch. 1, pp. 13–34). Oxford: Oxford University Press. Halsey, R. M. and Chapanis, A. (1951). 'Number of absolutely identifiable hues'. Journal of the Optical Society of America, 41, 1057–1058. Hatfield, G. (2009). On perceptual constancy. In G. Hatfield (ed.), Perception and Cognition: Essays in the Philosophy of Psychology ­(ch. 6, pp. 178–211). Oxford: Clarendon Press. von Helmholtz, H. (1962). Helmholtz’s Treatise on Physiological Optics. New  York:  Dover. Originally published in 1867. Hering, E. (1964). Outlines of a Theory of the Light Sense. Cambridge, MA:  Harvard University Press. Originally published in 1878; translated by Leo M. Hurvich and Dorothea Jameson. Hilbert, D. R. (1987). Color and Color Perception: A Study in Anthropocentric Realism. Stanford, CA: CSLI. Hilbert, D. R. (2005). 'Color constancy and the complexity of color'. Philosophical Topics, 33, 141–158. Hunter, F., Biver, S., and Fuqua, P. (2007). Light Science and Magic: An Introduction to Photographic Lighting, 3rd edn. Oxford: Focal Press. Hurlbert, A. C. (1989). 'Cues to the color of the illuminant'. Ph.D. thesis, Massachusetts Institute of Technology, Cambridge, MA. Hurlbert, A. C. and Ling, Y. (2005). 'If it’s a banana, it must be yellow: The role of memory colors in color constancy'. Journal of Vision, 5(8), 787. Hurvich, L. M. (1981). Color Vision. Sunderland, MA: Sinauer Associates. Ingle, D. (1998). 'Perceptual constancies in lower vertebrates'. In V. Walsh and J. J. Kulikowski (eds), Perceptual Constancy: Why Things Look as They Do (pp. 173–191). Cambridge: Cambridge University Press. Jameson, D. and Hurvich, L. M. (1989). 'Essay concerning color constancy'. Annual Review of Psychology, 40, 1–22. Reprinted in Byrne and Hilbert (1997), 177–198. Katz, D. (1911). Die Erscheinungsweisen der Farben und ihre Beeinflussung durch die Individuele Erfahrung. Leipzig: Barth. Katz, D. (1935). The world of Color. London: Kegan, Paul, Trench Truber & Co. translated by R. B. MacLeod and C. W. Fox.

638   Jonathan Cohen Kraft, J. M. and Brainard, D. H. (1999). 'Mechanisms of color constancy under nearly natural viewing'. Proceedings of the National Academy of Sciences, U. S. A., 96, 307–312. Land, E. H. and McCann, J. J. (1971). 'Lightness and retinex theory'. Journal of the Optical Society of America, 1, 61, 1–11. Lee, H. C. (1986). 'Method for computing the scene-illuminant chromaticity from specular highlights'. Journal of the Optical Society of America A, 3, 1694–1699. Locke, J. (1975). An Essay Concerning Human Understanding (1689). New  York:  Oxford University Press. MacLeod, D. I. A. (2003). 'Colour discrimination, colour constancy, and natural scene statistics'. In J. D. Mollon, J. Pokorny, and K. Knoblauch (eds), Normal and Defective Colour Vision (pp. 189–217). Oxford: Oxford University Press. Maloney, L. T. (1986). 'Evaluation of linear models of surface spectral reflectance with small numbers of parameters'. Journal of the Optical Society of America A, 3, 1673–1683. Maloney, L. T. and Wandell, B. A. (1986). 'Color constancy: a method for recovering surface spectral reflectance'. Journal of the Optical Society of America A, 3(1), 29–33. Matthen, M. (2010). 'How things look (and what things look that way)'. In B. Nanay (ed.), Perceiving the World (pp. 226–252). New York: Oxford University Press. McCann, J. J. (2004). 'Mechanisms of color constancy'. In Proceedings of the IS&T/SID Color Imaging Conference, vol. 12 (pp. 29–36). Scottsdale, Arizona. Neumeyer, C. (1998). 'Comparative aspects of color constancy'. In V. Walsh and J. Kulikowski, (eds), Perceptual Constancy: Why Things Look as They Do (pp. 323–351). Cambridge: Cambridge University Press. Olkkonen, M., Hansen, T., and Gegenfurtner, K. R. (2008). 'Color appearance of familiar objects: Effects of object shape, texture, and illumination changes'. Journal of Vision, 8(5), 1–16. Olkkonen, M., Hansen, T., and Gegenfurtner, K. R. (2012). 'High-level perceptual influences on color appearance'. In G. Hatfield and S. Allred (eds), Visual Experience: Sensation, Cognition, and Constancy ­(ch. 9, pp. 179–198). Oxford: Oxford University Press. Palmer, S. E. (1999). Vision Science: Photons to Phenomenology. Cambridge, MA: MIT Press. Rüttiger, L., Braun, D. I., Gegenfurtner, K. R., Petersen, D., Schönle, P., and Sharpe, L. T. (1999). 'Selective color constancy deficits after circumscribed unilateral brain lesions'. The Journal of Neuroscience, 19(8), 3094–3106. Slater, A., Mattok, A., and Brown, E. (1990). 'Size constancy at birth: Newborn infants’ responses to retinal and real size'. Journal of Experimental Child Psychology, 49, 314–322. Smith, A. D. (2002). The Problem of Perception. Cambridge, MA: Harvard University Press. Thompson, B. (2006). 'Colour constancy and Russellian representationalism'. Australasian Journal of Philosophy, 84(1), 75–94. Troost, J. M. and deWeert, C. M. M. (1991). 'Naming versus matching in color constancy'. Perception & Psychophysics, 50, 591–602. Tye, M. (2000). Consciousness, Color, and Content. Cambridge, MA: MIT Press. Valberg, A. and Lange-Malecki, B. (1990). ' "Colour constancy" in Mondrian patterns: A partial cancellation of physical chromaticity shifts by simultaneous contrast'. Vision Research, 30(3), 371–380. Whittle, P. (2003). 'Contrast colours'. In R. Mausfeld and D. Heyer (eds), Colour Perception: Mind and the Physical World (pp. 115–138). New York: Oxford University Press. Wright, W. (2013). 'Color constancy reconsidered'. Acta Analytica, 28(4), 435–455.

Perceptual Constancy   639 Wyszecki, G. and Stiles, W. S. (1982). Color Science: Concepts and Methods, Quantitative Data and Formulae, 2nd edn. New York: Wiley. Zaidi, Q. (1998). 'Identification of illuminany and object colors: Heuristic-based algorithms'. Journal of the Optical Society of America A, 15, 1767–1776. Zaidi, Q. (1999). 'Color and brightness induction: From Mach bands to three-dimensional configurations'. In K. R. Gegenfurtner and L. T. Sharpe (eds), Color Vision: From Genes to Perception (pp. 317–343). Cambridge: Cambridge University Press. Zaidi, Q. (2001). 'Color constancy in a rough world'. Color Research and Application, 26, S192–S200.

Chapter 34

How Do Sy na esthetes Ex per ience the Wor ld? Malika Auvray and Ophelia Deroy

1

1 Introduction Synaesthesia corresponds to a condition in which ‘stimulation in one sensory or cognitive stream leads to associated experiences in a second, unstimulated stream’ (Hubbard, 2007: 193). It has also been characterized as a ‘startling sensory blending’ (Cytowic, 1997: 17) ‘not experienced by most people under comparable conditions’ (Grossenbacher and Lovelace, 2001: 36). The first medical account came from George Sachs (1812), an albino and synaesthete, who describes at the end of his medical dissertation how black letters appear to him in various shades (quoted by Ward, 2008). Previous mentions, more or less akin to thought experiments, are found in Locke (1690) and Leibniz (1765), who report of a blind man for whom thinking of the colour scarlet is like hearing ‘the sound of a trumpet’. Later, Fechner (1871; followed by Galton, 1880) published more general surveys of coloured letters, also known as ‘photisms’. Since these early studies, the initial source of puzzlement has not been clarified. What is particularly distinctive in synaesthetes is their ability to give consistent pairings of apparently unrelated sensory attributes, like letters and colours, over a very long stretch of time (BaronCohen et al., 1993; Rich et al., 2005).2 This ability seems to go beyond what normal memory can achieve and suggests that the associations are not random. This is what leads us to hypothesize the existence of a specific experiential basis for these associations. Accepting this turns out to be puzzling: what sort of experience can synaesthetes have when they report black letters being red, green, or yellow—or seeing them as such?3 How 1 

The two authors equally contributed to this work. is known as the ‘test of genuineness’. The same participants’ inducer-concurrent pairings are tested and retested after a long period of time. For example, in Baron-Cohen et al.’s (1993) study, synaesthetes gave 92% of consistent replies when given an unexpected retest after one year, compared to 38% of consistency for control participants retested with warning after one week. 3  The phenomenological reports vary importantly: some synaesthetes are happy to use the predicative form, and talk of coloured sounds and letters; others use more indirect expressions. However, in the absence of systematic studies investigating such differences in reports, or investigating correlation between the kind of reports given by synaesthetes and behaviourial evidence, it is impossible to make much of these linguistic variations. 2  This

How Do Synaesthetes Experience the World?    641 can we explain some individuals seeing colours when they hear sounds or musical notes? As we shall see, synaesthesia raises both broad questions about the exact nature of conscious experience as well as more technical challenges to philosophical models of perception and mental processes. Take the example of D. (mentioned in Ortmann, 1933) for whom the tone B2 looks green; that is, she has an experience of green whenever she hears B2. The problem here is not merely that D.’s experience is different from that of others but that it is at odds with reality: the green she experiences is not a sign that the note out there is really green. What can this experience be about then? Does it attribute an illusory colour property to the sound? Does it have any representational content at all? Synaesthetic experiences, as we begin to understand, do not easily fit in the dominant representationalist framework (Alter, 2006; Gray, 2001a; Wager, 1999, 2001). The same is true for functionalism: like other perceivers, D. also sees green when she looks at leaves, limes, wine bottles, or grass. She might perceive the exact same shade of green when she hears B2 and when she looks at a lime. What is puzzling is that her green experience sometimes seems to result from auditory stimulation and, at other times, from visual stimulation. Supposing that experiencing a certain shade of green constitutes a single kind of mental state, this leads to the conclusion that the same kind of state arises from two distinct streams and occupies two irreconcilable roles in D.’s cognitive system, which appears to contradict the functionalist principles (Gray et al., 1997, 2002; see also Macpherson, 2007, for a discussion). Finally, synaesthetic cases like D.’s also challenge models of mental architecture, that is the modular, impenetrable character of perception (Baron-Cohen et al., 1993; Gray, 2001b; Segal, 1997) and the neat individuation of the senses (Keeley, 2013). If visual experiences of colour can result from auditory stimulation, the senses might not be as separated—or easy to individuate—as usually posited. This quick overview is sufficient to highlight that synaesthesia is seen as a serious source of objections to standard theories of perception and mental processes. Most, if not all, of these philosophical objections are however advanced on the basis of three main general assumptions about synaesthesia, namely that it is: (1) a single, more or less unified, condition; whereby (2) some individuals have an additional experience, adjoined to normal perceptual experience; and that (3)  this condition teaches us something about non-synaesthetic perception. The aim of this chapter is to critically examine these three assumptions, which, as we shall see, appear more and more fragile once we consider the variety of cases and their more comprehensive studies. This has important consequences regarding the reality and scope of the philosophical challenges posed by synaesthesia, to which we shall return in conclusion.

2  Characterizing synaesthesia: A unified kind or a variety of phenomena? Cataloguing the cases Synaesthesia has long been seen as a natural kind, but it comes in a range of subspecies. Since Sachs (1812) and Fechner (1871), synaesthesia has been primarily studied through the induction of atypical colour experiences by musical notes and speech—be it read, heard,

642    Malika Auvray and Ophelia Deroy or both. But other forms of synaesthesia do not involve colours or linguistic and musical items: digits or months of the year can elicit certain locations in space (Galton, 1880; Sagiv et al., 2006), words can induce taste sensations in the mouth, and tastes can trigger tactile shape sensations: for instance, the word ‘jail’ tastes of cold, hard bacon (Ward and Simner, 2003) and chicken is pointy for some individuals (Cytowic, 1993). Other rare cases lead to proprioceptive experiences, which involve movement and bodily postures (Rothen et al., 2013). In cases of ordinal linguistic personification, numbers or days of the week are associated with personality traits, emotions, or gender (e.g. Smilek et al., 2007a). Furthermore, most often, synaesthetic experience comes with an emotional dimension, in addition to the sensory one (see Hochel et al., 2008, for a discussion). Some researchers wonder whether some of these variants would deserve to be recognized as distinct kinds on their own. For the various cases labelled as synaesthetic might, in fact, be the result of very distinct mechanisms/processes, some of which have little or nothing to do with sensory perception (see Simner, 2007, 2012; and Marks, 2011, for a review of recent debates). There is variation among individuals not only in the kinds of stimuli that elicit synaesthetic experience (often labelled ‘the inducer’, see Grossenbacher and Lovelace, 2001), or kinds of experience that is elicited (the ‘concurrent’ experience), but also in the concurrents of the same kind. That is, there are no universal mappings between inducers and concurrents that can be observed across synaesthetes. For instance, people do not report the same colours for all letters and numbers: J can be blue for some and orange for others. There are, however, some general trends (e.g. see Rich et al., 2005). These recurrent patterns across individuals reveal for instance that A is most often red across multiple colour-graphemes synaesthetes (Day, 2005; Simner et al., 2005) and vowels tend to elicit more luminous and intense colours than consonants (see Deroy and Spence, 2013; Smilek et al., 2007b; Spence and Deroy, 2013, for a discussion). Still, these trends do not exhaust the richness of individual experiences: the overall vividness and intensity of synaesthetic experiences vary from one person to the other. Large-scale studies (e.g. Day et al., 2005; Simner et al., 2006) reveal that the most common forms of synaesthetic experience are that of grapheme-colour and word-colour. Why? The question is not disconnected from the previous observation, as the frequency might be due to the fact that synaesthesia is a too-broad category, where further subspecies have to be recognized. Several other divisions have been introduced that could help explain away the dominance of linguistic-colour synaesthesia and cut down the category into more finely individuated types.

Individuating the various types of synaesthesia Ramachandran and Hubbard’s (2001a) distinction between ‘lower’ and ‘higher’ synaesthetes for instance might prove useful to breakdown synaesthesia into finer categories. Lower synaesthetes are sensitive to the basic sensory features of the inducer (e.g. contrast, lines, and orientation of the letter on the page) whereas higher synaesthetes are sensitive to the kind of object they recognize (As and Bs for instance).4 Dixon et al. (2004) have also 4 

Note that this is orthogonal to the problem of knowing whether synaesthesia is stimulus driven or internally driven. Even for lower synaesthetes, the basic sensory features might have to be recognized before they trigger the synaesthetic experience.

How Do Synaesthetes Experience the World?    643 proposed a subcategorization of synaesthetes into ‘associators’, who are the majority and experience the atypical colour in an internal space (in the ‘mind’s eye’, as they report) and the rare ‘projectors’ for whom the atypical colour appears to be ‘out there’, for instance overlaying the letter on the page. Resorting to such categorization matters, as projectors have often provided the core of philosophical challenges due to their perceptual-like spatial experiences (e.g. Macpherson, 2007; Wager, 1999). These divisions may not, however, be sharp, or real. The boundary between ‘higher’ versus ‘lower’ synaesthetes seems quite permeable, as it is unclear how independent higher synaesthetic experiences are from variation in the more basic features of the tokened type. Subtle colour changes in synaesthetic experiences arise for instance when small changes in font are introduced: although ‘R’ and ‘r’ are tokens of the same type and might both appear orange, the orange hue that one of them elicits may be brighter or more saturated than the other. Moreover, the transfer of synaesthetic experiences from one type of alphabet to another—which, at first sight, can be taken for a sign that higher synaesthesia is at stake—is shown to also depend on phonemic and visual similarities between the letters (Witthoft and Winawer, 2010). Finally, Hubbard et al. (2005) and Brang et al. (2011) have cast further doubt on the isolation of higher synaesthesia, by showing that the visual form of letters may have an impact on their colouring: letters that are closer in shape (e.g. KVWXY) may be closer in colour than those that differ in shape (e.g. CUDOQ). Similar questions arise for the projectors versus associators divide. Ward et al. (2007) for instance, demonstrated that the degree of projection varies as a function of the task at hand. In addition, Edquist et al.’s (2006) study revealed that some synaesthetes are left out by this categorization, as some grapheme-colour synaesthetes are neither associators, nor projectors. For them, the colours are extended but not in any particular location, that is, neither in the mind’s eye, nor out there. More importantly, there might not be such clear empirical cases of perfect projectors consistently having synaesthetic experiences of colour patches with a determined extension and localization (Hupe et al., 2011). If this turns out to be true, then the paradigmatic case of synaesthesia quoted by philosophers might simply vanish and turn into a mere thought experiment for long evenings in the armchair. Understanding real synaesthesia (and not an ideal one), by contrast, supposes that one has a clear sense of what its various types are. As research still struggles to encompass individual variations, finding the right subcategories to study them seems to us to constitute one of the most pressing challenges.

Is synaesthesia a case of ‘atypical perception’? An even more pressing issue is to define the nature of synaesthetic experiences; operating under the assumption that they can be clustered together as being similar. A frequent interpretation is that synaesthetic experiences are all instances of perception: synaesthesia is said to occur when ‘stimulation of one sensory modality automatically triggers a perception in a second modality, in the absence of any direct stimulation to this second modality’ (Harrison and Baron-Cohen, 1997: 3; italics are our emphasis). This definition is widely quoted and often endorsed in the literature (e.g. Baron-Cohen et al., 2007; Macpherson, 2007). Elsewhere, synaesthesia also gets defined as ‘the stimulation of one sensory modality reliably caus(ing) a perception in one or more different senses’ (Cytowic, 1989) or as an ‘anomalous sensory perception’ (Asher et al., 2009).

644    Malika Auvray and Ophelia Deroy Yet, one might immediately object that synaesthesia cannot really count as perceptual because it is not veridical (Gray, 2001a) or caused by the presence of the appropriate stimulus (Sagiv et al., 2011). Lycan (2006) and Fish (2010) for instance, both recommend synaesthetic experiences to be treated as cases of hallucination, whereas Simner (2007) insists that it is a higher cognitive-linguistic phenomenon. The characterization of synaesthesia as being perceptual certainly depends first on how one captures the synaesthetic experience, and second on how one defines perception, noticeably regarding its correlation with the external world. These are the issues which will be addressed in sections 2 and 3.

3  What is it like to have a synaesthetic experience? Are synaesthetic and non-synaesthetic experiences subjectively distinguishable? A first question to ask is whether synaesthetic experiences are subjectively indistinguishable from perceptual experiences. Note that from a philosophical point of view, the indistinguishability is not sufficient to claim that synaesthetic experiences should count as perceptual (as noticeably pressed by disjunctivist accounts, see Logue, Chapter 11, this volume). However, there is widespread agreement, coming from the argument of hallucination, that if an experience is subjectively distinguishable from a perceptual one, by being for instance not vivid or devoid of phenomenal presence, it is enough to declare that it is not perceptual (see Crane, 2011, for a discussion). The reports from synaesthetes do not deliver a straightforward answer:  synaesthetes often say that they are aware of a difference between their synaesthetic and non-synaesthetic experiences while they also experience the two to be comparable. The documented reports and various observations converge to stress that a synaesthetic experience can be as vivid and forceful as other experiences induced by other kinds of stimuli, and that it can also spontaneously lead to beliefs, for instance, the belief that B2 is genuinely green (Cytowic, 1993). Such synaesthetic experiences seem indistinguishable, or at least comparable to perceptual experiences. Synaesthetes might be ambivalent when reporting their synaesthetic experiences because they are also not shared by the majority of the population. This atypicality can explain why a synaesthete distinguishes between his idiosyncratic synaesthetic experience (and related beliefs) which nobody shares, and the other set of experiences which lead to commonly accepted beliefs. However, being rare and not being shared are not good reasons to consider a certain kind of experience as being non-genuinely perceptual. Take for instance what happens in taste perception: very few persons are super-tasters, and perceive or believe that Brussels sprouts are unbearably bitter. The rare character of the former experience does not rule out that super-tasters perceive certain tastes. Likewise, the idiosyncratic difference exhibited by synaesthetic experiences should not result in the conclusion that they are not perceptual.

How Do Synaesthetes Experience the World?    645 The atypical character of synaesthetic experiences must also be handled with care, especially because it is sometimes taken to be a defining feature of the condition (e.g. Grossenbacher and Lovelace, 2001). The fact that synaesthetic experiences do not seem to be so frequent, does not help determine their nature. To put it in general terms, the unusual character rather arises from having a certain ‘extra’ component than would normally be triggered by the same stimuli under the same conditions. Synaesthetic experiences therefore differ from what we assume to be the rules of normal perception, but may still be perceptual.

A frequent confusion about the inducer-concurrent pairings We have stressed that the focus on the distinguishability between the concurrent experience and ‘normal’ experiences does not deliver a clear verdict. Going one step further, we want to argue here that the fragile assumptions on which the verdict rest means that it is unlikely to become clearer. What the notion of concurrent is supposed to capture is the ‘extra’ component that constitutes the synaesthetic experience; this notion however proves highly problematic. A first thing to remember is that reports from synaesthetes highlight just how difficult it is to communicate on private experiences, especially when the experience is unusual. As one synaesthete describes: ‘I have trouble putting into words some of the things I experience. It is like explaining red to a blind person or middle-C to a deaf person’ (Cytowic, 1989). Thus, when D. reports having the experience of a ‘pretty yellow green’ when she hears a B2 and of a ‘dirty yellow-green’ when hearing a D# (Ortmann, 1933), we can only consider what she means here by the term ‘green’. Does it mean the same thing as when D. reports that a certain leaf looks green? The concern expressed here is not merely linguistic: it matters whether or not her synaesthetic sensations of green are identical to her non-synaesthetic ones, for which she uses the same descriptor. For this reason, synaesthetic reports are usually analysed in terms of two separate components: one corresponds to this extra experience, on which the synaesthete is asked to report, and the other corresponds to the inducer, which can be manipulated externally. In a very general way, experimental research consists of varying the inducer and measuring the consequent modifications in the concurrent. Thus what is misleading is certainly not the method, but the kind of interpretation it encourages: we would like to point out that there is a method-content confusion, starting with the testing of synaesthesia through associated pairs to the idea that synaesthetic experiences themselves are a pair of experiences. What we see is that the notion of an atypical synaesthetic concurrent often transfers to the idea that synaesthetic experience is a conjunction of two distinct experiences: the concurrent would be distinctively enjoyed by synaesthetes only, whereas the experience of the inducer would be common to synaesthetes and non-synaesthetes alike. An illustration of this transfer can be found in Macpherson (2007) when she recommends changing Cytowic’s description of synaesthesia as a ‘startling sensory blending whose quality seems difficult for most of us to imagine’ (Cytowic 1997: 17) to the claim that synaesthetes have an ‘additional experience’ joined to the otherwise normal experience common to us all. Synaesthesia, she admits, corresponds to cases where ‘an experience or element of experience is associated with some sensory modality and is distinct from’ the former (trigger)

646    Malika Auvray and Ophelia Deroy experience (Macpherson, 2007: 70). Keeley (2013) also notes that ‘in synaesthesia, it is not the case that the neurotypical sensations are replaced or eclipsed by the synaesthetic ones; instead they are experienced in a conjoint fashion’ (see also Wager, 1999). A synaesthetic experience where, for instance, a green concurrent is induced by a sound is understood as leading to the typical experience of the sound, plus the unusual experience of green. This view constitutes what we call a ‘dualistic model’ of synaesthesia. This model is largely assumed both in the scientific and philosophical literature. Yet, we argue, it is a disputable and possibly misleading way of capturing the atypical experiences documented in the empirical literature, as there are solid grounds on which to base non-dualistic theories of the synaesthetic experience.

Synaesthetic experiences and phenomenal enrichment Most of the reports do not imply that synaesthetes have an extra experience. They are perfectly compatible with the more minimal and cautious idea that synaesthetic experiences come with a richer sensing, that is, some extra-sensational aspect. Dualistic models infer too quickly that this richer sensing means that synaesthetes enjoy an additional experience with a distinct content and character, which can be unproblematically defined and detached from the other contents and character of the overall experience. In other words, dualistic models divide synaesthetes’ experiences into two otherwise normal experiences. This analysis is often taken for granted and yet, we argue, it is at odds with the evidence. The first problem with the idea of two joint experiences comes from the fact that the synaesthetic ‘extra’ in the content of the experience is dependent upon co-occurring internal attributes: the concurrent is never experienced independently, in the way the fuller experience which seems to be reported would be. It is noticeably never experienced by itself, but always as grafted onto the content of another experience. Why does this need to be stressed? Most of the time, researchers and philosophers seem to be misled by descriptions that make the synaesthetic ‘extra’ analogous to classical experiences—as if for instance, they were additional experiences of colour patches superimposed onto graphemes or floating after-images. But, on examining the reports more closely, the evidence suggests that the synaesthetic colour is often just an addition of some colour dimensions (such as hue) to a grapheme. In other words, the synaesthetic ‘extra’ is actually relatively poor—and seems to contain fewer dimensions than a typical experience of a coloured patch—which has hue, brightness, saturation but also extension, texture, orientation, etc. More crucially then, this ‘extra’ falls short of being the possible content of a distinct, normal experience. The fact that it is only experienced along with the inducer (for instance, the grapheme) goes against the idea of a separable concurrent experience, and on the contrary, supports the idea that it is intrinsically attached to the inducer: it could not be experienced in the absence of the experienced inducer. There is, after all, no non-synaesthetic experience of hue without spatial location, or without variations in other dimensions such as saturation or lightness, i.e. in the absence of precise localization and other spatial and qualitative properties. In synaesthetic experience, the extra colour can be conceived as integrated into the character of the normal experience, and not as a stand-alone independent experience which merely happens to be co-occurring with the experience of the inducer.

How Do Synaesthetes Experience the World?    647 A second argument against dualism comes from the strong connection that exists, within experience, between the synaesthetic extra and the content of perception elicited by the inducer. The available evidence actually goes against the sort of independence posited by most researchers. On the contrary, it suggests that the concurrent should be considered as interacting, or rather interfering, with the inducer at relatively early stages of the perceptual process. Interferences with the identification of perceived objects have been widely demonstrated by some variants of the Stroop task (Stroop, 1935). The original task revealed that people take longer to name the colour of a written word if the ink does not match the word than if the two match. For example, it takes longer to name the colour green when it is printed on the word ‘red’ than when it is printed on the word ‘green’. A  similar effect is observed when grapheme-colour synaesthetes are asked to name the actual colour of a grapheme while ignoring the  synaesthetic colour it elicits. If the letter R is synaesthetically experienced as orange, synaesthetes will be slower in naming the ink’s colour when the letter is printed in blue than when it is printed in orange (e.g. Mills et al., 1999).5 These results underscore what we have already suggested: the concurrent and the experienced inducer are two aspects of the same experience, intricately connected and difficult to disentangle when asked to attend selectively to one of them. Insisting that the concurrent is not distinct from and merely conjoined to the content of an otherwise perceptual experience provides a good reason to question the dominant dualistic models. At the same time, more needs to be done to account for the atypical insertion of this synaesthetic ‘extra’ which interacts with the other experienced contents. Drawing on Evans’s (1982) distinction between the richness (i.e. the number of distinct dimensions) and the fineness of grain (i.e. the number of distinct perceptible positions on each dimension), we contend that the concurrent only exists as an extra-dimension. That is, its presence is an enrichment of other ‘normally elicited’ contents. This preserves the idea of a supplement, while not granting any real or even theoretical independence to the concurrent as a separate or detachable experience. Another, closely related way to conceptualize its relation to the perceptual experience of the inducer is to consider the experience of parasite and host. This new definition not only represents the facts more closely; it also dissolves some intractable controversy. Recognizing that synaesthesia consists of the occurrence of atypical, enriched experiences—or experiences hosting phenomenal parasites—finally provides a way to understand the ambivalent reports and to put an end to the subjective indiscriminability question: the concurrent does not need to be granted with a perceptual status of its own. It is wrong to think about it being on its own in the first place, but reports show that it inherits the same subjective perceptual reality as the overall experience which it comes to enrich.

5 

These results, like in the classical task, remain ambivalent as to whether the interference occurs at the perceptual level (e.g. the letter E is automatically perceived by the synaesthete to be red, and this perceived redness interferes with the ‘true’ colour which is also perceived) or at the semantic level (the synaesthetic experience of red makes one automatically tag the concept red, and this concept slows down one’s ability to access the concept of the ‘true’ colour). Note that we are not saying that these results are sufficient to demonstrate a perceptual interference but that, in combination with other results, they provide cumulative evidence for the interference occurring, at least partly, at the level of perceptual experience.

648    Malika Auvray and Ophelia Deroy

4  Synaesthetic experiences and perception Are synaesthetically enriched experiences then fully perceptual? The question continues to make sense if one considers that the issue is not settled only at the phenomenological level that was described in section 2. Besides having a certain phenomenal character, perception is usually defined by three other key features: it co-varies with changes in the physical environment, it represents an object, and, finally, it recruits specific organs and brain areas. Addressing the perceptual status of synaesthesia makes it necessary to verify how well these requirements are satisfied by synaesthesia. Before addressing this, it is important to emphasize that all these requirements are the topic of vibrant disputes. The requirement that perception causally co-varies with external features might have to be relaxed to accommodate the perception of high-level properties or interoception. The idea that perception is representational is widely accepted by representationalist and intentionalist accounts, but enactivists and direct realists would reconsider it or give it up completely (see Jacob, Chapter 12, this volume, for elaboration of this point). Finally, accepting that perception is the result of activity within dedicated brain areas is in line with most contemporary physicalist sympathies, but it can be relaxed or given up by functionalist and non-physicalist accounts (for different reasons such as multiple realizability in the first case, and metaphysical dualism in the second). The goal of the last part of this chapter is not to argue which criteria should be included or prevail in the definition of perception, but to question the perceptual status of synaesthesia in light of each of these requirements.

The co-variation criterion What determines the occurrence and the character of synaesthetic experiences? Addressing this question is fundamental if one considers perception to be driven by environmental factors rather than mental ones. A perceptual state, such as hearing a sound, or a specific dimension of this state, such as loudness, is caused by physical stimulation and co-varies with changes in the environment. This requirement seems to be satisfied in certain cases of synaesthesia which are triggered by physical (mostly external) stimuli and vary with their physical properties. For instance D., the synaesthete described in Ortmann’s (1933) detailed single-case study, has synaesthetic experiences that vary with properties of the auditory stimuli, such as timbre. Difficulties arise from less straightforward cases for which synaesthetic experiences seem to be independent from certain variations in the external stimulus or seem to occur in the absence of an external stimulus. Does this mean that synaesthesia is mind dependent, or that some cases may well be? Let’s consider the two above-mentioned causes independently. Are there cases of synaesthesia that are purely mind, rather than environment dependent? As was said earlier, the dominant type of synaesthesia consists of colours elicited by graphemes. Such synaesthetes can be sensitive to the tokening of a certain type of letter or word, while being relatively indifferent to more minute changes in their physical instantiation (see section 1). These so-called higher synaesthetes respond in approximately similar

How Do Synaesthetes Experience the World?    649 ways to a wide range of physical stimuli as long as they fall in the appropriate kind. For instance some of them have the same synaesthetic experience with auditory or visual instantiations of the letter R and, in the latter case, for letters printed in different fonts. Recent evidence additionally reveals that such synaesthetic experience can rapidly transfer to new fonts not previously experienced or identified as members of an existing or known kind of letter (Mroczko et al., 2009). From there, it is tempting to conclude that the true determinant of synaesthesia is the recognition of the physical stimulus as an instance of a certain kind (e.g. a letter) and therefore that the determinant is likely to be ‘conceptual’ rather than physical (see Simner, 2007). However, the existence of such feature-independent synaesthetes does not necessarily challenge the perceptual nature of synaesthesia. The fact that a state responds to high-level, and not merely physical, properties does not necessarily mean that it cannot be perceptual. Some have argued (see Bayne, 2009; Siegel, 2006) that being an A, like being a tiger, a pine-tree, or an R can be part of the perceptual content. This, in turn, needs to be separated from the question of whether this kind of property needs to be conceptually recognized (as accepted for instance in Simner, 2007, but see Deroy, 2013a). What about cases where a conceptual, emotional, or mental state suffices to trigger a synaesthetic experience? The threat of mind dependency would be strengthened by such cases. Dixon et al. (2000) have reported the case of C, a digit-colour synaesthete who has a coloured experience of numbers not only when viewing the external stimuli (e.g. the number 7) but also when conceptually entertaining them (just being presented with 5 + 2, and mentally arriving at the result of the addition). The immediate cause of the synaesthetic experience, evidenced in this case, cannot be the physical sensory stimulation, given that the number 7 was not physically presented. This rare case requires cautious analysis:  the experiment shows the possibility of involuntary synaesthetic experiences to be both mind driven (here by a mental calculation) and environment driven (by a physically present numeral). However, cases of purely mind-driven synaesthetic experiences—i.e. cases which would only be triggered by mental representation of an object and not by its physical presentation—remain undocumented. In the meantime, mind-driven cases seem to exist only as an interesting extension, but extension nonetheless, of the externally driven ones. It is therefore premature to say whether mental determinants such as concepts are in themselves sufficient to elicit synaesthetic experiences (see Spiller and Jansaria, 2008, for more data) and what role they play alongside necessary physical stimuli, in explaining some variations and occurrences of synaesthetic experiences. It is fair to say that for the vast majority of cases, including that of higher synaesthesia, the presence of an external physical stimulus remains a necessary condition. In addition the object properties explain the variations and distribution of the synaesthetic experiences obtained.

The representational criterion Going beyond the covariation criterion, it is possible to claim that perceptual states represent a mind-independent object (or property). Bypassing the discussions surrounding the formulation of this definitional requirement, let’s only examine the idea that perceptual states have a certain representational (or intentional) content. This content is what

650    Malika Auvray and Ophelia Deroy corresponds to their phenomenal character. For instance, the perceptual state of ‘experiencing blue’ represents the property of being blue. Another way to state this relation is to say that the phenomenal character supervenes on the representational content: as Tye puts it, ‘necessarily, experiences that are alike in their representational contents are alike in their phenomenal character’ (Tye, 2002: 137). At first glance, synaesthetic cases seem to contradict this form of representationalism. This can be most clearly illustrated by Wager’s (1999) theoretical case of Cynthia. Cynthia is a coloured-hearing projector synaesthete who experiences a determinate shade of red at a precise location when she hears middle-C. What Cynthia’s overall synaesthetic experience of a red middle-C represents is—arguably—no more than middle-C. Middle-C is also the only thing that is represented by the normal auditory experience of Norma, which does not have any red addition or enrichment. There is therefore a difference in phenomenal character without any difference in intentional content (see also Rosenberg, 2004, for a similar objection). Does this difference in character but not intentional content pose a problem for representationalism? The first thing to note is that Cynthia is an ideal case: as was mentioned earlier, pure projectors may not exist. It could be problematic to the classification of synaesthesia if such a case, and its further variations (Wager, 1999, 2001; Gray, 2001a), are given too much weight, as such tokens bring the debate away from empirically valid objections, and conform to the ‘thought-experiment’ challenges to representationalism, such as the inverted spectrum case (Block, 1978). That being said, let’s consider what valid insights might be taken from these ‘ideal cases’. As was emphasized by Lycan (2006) and by Wager (1999) himself, the objection is directed against one class of representationalist theories:  externalist representationalist theories, for example Dretske’s (1995, 2000) or Tye’s (2000) which require that the intentional object represented in perception is identical to the physical object. An internalist account has no problem in acknowledging that Cynthia’s experience represents both middle-C and redness. In that case, as Lycan (2006) puts it, ‘vision is telling her that there is redness dead ahead, just as in the cases of after-images and hallucinated rodents’. However, this implicitly leads him to endorse the (otherwise misleading) dualist view. What the internalist account also needs to explain is what makes one aspect of the experience true and the other one false, given that the two are subjectively indistinguishable (unlike the case of after-images). Going back to the externalist perspective though, the objection does not seem fatal. As stressed by Alter (2006), an externalist does not need to claim that every phenomenal difference introduces a difference in representational content. Both a tactile sensation and a visual sensation can represent an external object although the two feel different (see Tye, 2000, for a defence of this view on behalf of representationalism). The same kind of phenomenal character can take multiple forms: representationalists who accept this claim have only to interpret Cynthia’s synaesthetic experiences as being a declination within the general kind of ‘experiencing middle-C’. Her experience represents middle-C and so does a non-synaesthetic experience, as much as the look of a cube and the tactile sensation of a cube can represent the property of being cubic. The counter-argument turns out to be problematic though in the synaesthetic case, at least in the sense that it conflicts with another frequent representationalist credo: the transparent character of experience. The transparency claim goes further than the basic representationalist supervenience claim. The latter merely posits that states which have similar kinds of intentional content are alike in kinds of phenomenal characters; whereas

How Do Synaesthetes Experience the World?    651 transparency means that the phenomenal character is identical to the intentional content. Obviously, if the synaesthetic experience of middle-C is phenomenally richer, then the intentional content of that experience will inevitably also be richer. But in which sense does an experience of red middle-C represent more of the external object than the non-synaesthetic experience of middle-C? No further attribute of the object seems to be captured in the synaesthetic experience. The fact that the same attributes can be represented faster does not introduce a difference in content, despite the suggestion offered by Sollberger (2013) to rescue representationalism from synaesthesia’s challenge. The challenge raised by synaesthesia for representationalism certainly goes beyond this chapter. It seems fair to conclude that the challenge is serious and that synaesthesia is difficult to square with the representationalist’s requirements for perception. Each solution gives rise to further challenges: first, if one claims that synaesthetic experiences represent a non-existent intentional object—or misrepresent the existing one—then synaesthetic experiences end up amounting to hallucinations or illusions. This is at odds with the idea that they follow systematic co-variations with the external environment (as described in the previous subsection). To date the empirical evidence favours this co-variation hypothesis. If synaesthesia is an illusion, it is a very specific one, which would therefore require a redefinition of illusion. Alternatively, the representationalist can choose to accommodate synaesthetic experiences within the range of possible variations in the way a certain object is represented in perception. But the stretch might be difficult: very different phenomenal characters have to represent the same external object; up to the extent that a synaesthetic experience of an ‘orange sound’ represents the properties of a sound in the same way as a strictly auditory experience. The final option available to the representationalist is to show that the enriched synaesthetic experiences have a richer representational content than the non-synaesthetic ones. This is an interesting, but difficult route to explore if synaesthesia were to count as perceptual.

The sensory processing criterion Synaesthesia raises further concerns with respect to the definition of perception. Certain physicalist accounts would suggest that each kind of mental state correspond to its own kind of neurological state/states. Perceptual states, in this sense, can be defined as those resulting from the activation of specific brain areas (for example sensory ones, to keep it simple). One question to emerge is how easy is it then to apply this criterion to synaesthetic states? As this field of research is developing rapidly, we will offer a succinct review of the key data, while focusing on their consequences for physicalist requirements. Brain imaging studies have revealed that synaesthetic experiences often involve increased activation in primary sensory brain areas (e.g. Nunn et al., 2002; Rouw, 2011; see also Deroy, 2013a, 2013b, for discussion). For instance, when synaesthetes hear an auditory inducer that triggers colours for them, there is an increased activity in the area in the fusiform gyrus known as V4 or V8, that is, the brain areas involved when non-synaesthetes perceive colours. Interestingly, such increased activation does not occur in non-synaesthetes trained to associate sounds with colours and who are subsequently asked to visually imagine the corresponding colour when hearing the sound (Nunn et al., 2002). Other— not strictly sensory—areas of the brain (i.e. parietal and frontal regions) also show specific

652    Malika Auvray and Ophelia Deroy patterns of activation in some synaesthetes (see Rouw and Scholte, 2007). Synaesthesia is likely to be characterized more broadly as an increased cortical connectivity between various sensory brain regions, either directly or indirectly (via the mediation of non-sensory processing). Whether indirect activation and top-down influences mean that synaesthesia cannot count as sensory is controversial (see Simner, 2007, for a discussion). Further questions then arise as to whether the increased connectivity comes from a lack of inhibition or from an abnormal increase in connectivity (see Bargary and Mitchell, 2008, for a review). In their review Bargary and Mitchell (2008) concluded that the localization of the activity plays a less crucial role than finer structural or functional differences in terms of the neural correlates of synaesthesia. More recently, doubts have also arisen as to whether shared localization in V4/V8 from bold activity means that the same kinds of neurons are active in synaesthetic colour-enriched auditory experience and non-synaesthetic colour experiences. A study by Van Leeuwen et al. (2010) revealed that the neurons involved in synaesthetic experiences do not show the same patterns of suppression through repetition as the ones involved in typical perception, concluding that ‘the neural correlates of synaesthetic colour experience and real colour experience are not fully shared’ (Van Leeuwen et al., 2010; see also Hong and Blake, 2008, for further discussion). For the purposes of the current discussion of synaesthesia, the study raises the question of how specific and well defined or broad and global the neural processes involved have to be in order to be informative; in other words, is shared localization sufficient, and which additional aspects have to be included? Further neuroscientific investigation of synaesthesia is certainly needed, and as such speculation will not provide further support for nor undermine a representationalist view. It is however worth noting that resorting to a neurological or physicalist definition to defend the fact that synaesthesia is a form of perception might turn out to lead to a Pyrrhic victory: in that it finally challenges our conceptual understanding of the nature and individuation of sensory areas in the first place. This brings us to the concluding remarks.

5  Conclusions and further challenges Every paper focusing on synaesthesia must end on a cautious note, given the obvious need for further exploration and empirical research. Substantial steps can yet be taken, and are worth debating, on the basis of revised assumptions and a better look at the empirical cases. First, as was highlighted in the first part of the chapter, it is not clear that there is a unified condition, called ‘synaesthesia’, which is the same in coloured-grapheme, coloured-hearing, and for example, taste-words synaesthetes, and which also includes mirror-touch phenomena or spatial-number cases. Real synaesthesia, in this sense, might be much more varied and complex than the ideal case of colour-grapheme synaesthetes who project additional colours onto the letters out there, which is often found in the philosophical literature. Yet there is still room, or so we argue, for some important philosophical questions regarding the proper individuation of the various types of synaesthesia and for an overall definition of synaesthesia as a certain kind of mental state. The variety of cases might noticeably be a legitimate concern for a neurological definition of synaesthesia

How Do Synaesthetes Experience the World?    653 (Simner, 2012) and therefore for the idea that kinds of mental states correspond to kinds of neurological states. The second part of the chapter is our attempt to provide a definition of synaesthesia as a kind of mental state, more precisely as a kind of conscious state. This, in itself, is not novel as most definitions of synaesthesia consider it to be a kind of atypical experience. However, as we pointed out, one needs to avoid the temptation to analyse this atypical character as a conjunction of two typical components: this temptation comes from a method-content confusion and we should certainly not take dual models of synaesthetic experiences for granted. Alternatively, synaesthetic experiences can be defined as richer, unified experiences, where an additional sensory attribute (or qualia) gets hosted in the content of perception. The definition makes synaesthetic experiences something that neurotypical perceivers cannot enjoy, nor perhaps easily understand—but it takes the idea seriously that synaesthesia leads to atypical experiences, which are difficult to communicate. This new definition certainly bears heavily on the third point that most researchers have assumed, that synaesthesia teaches us something more general about the mind, and more specifically about perception. On the one hand, the rejection of the dual model is particularly important, as it pushes away the idea that synaesthesia challenges functionalism. As Gray has put it on multiple occasions (Gray et al., 1997, 2002, 2006) synaesthesia might seem to refute the claim that experiences with different functional properties necessarily have different qualitative properties—a claim which holds at least for strong versions of functionalism (see Macpherson, 2007; see also Block, 1980, and Shoemaker, 1975, for more general discussions). The objection only works when one assumes that a coloured-hearing synaesthete experiences the same kind of conscious mental state (e.g. experiencing red) when she sees a cherry and when she hears a certain sound. This understanding is ruled out once synaesthesia is interpreted, as we suggest it should be, in a non-dualistic way. Gray’s premiss that synaesthetic and non-synaesthetic experiences (say of the colour red) have a shared content rests on the dual experience model we have rejected. Once synaesthetic experiences are understood as phenomenally enriched experiences, there is no such thing as a single kind of content (red) that could be detached and compared across synaesthetic and nonsynaesthetic experiences. The background premiss is blocked and the challenge therefore does not hold.6 If synaesthesia does not in itself provide good reasons to give up functionalism, why is it so challenging? Does the enrichment account not make synaesthesia trivial? Certainly not, as the main challenge remains as we outlined in section 3. The important point is to see whether enriched synaesthetic experiences deserve to be classified as perceptual. As was stressed, several conditions need to be considered besides the fact that synaesthetic experiences enjoy the same subjective status as perceptual experiences before they can be granted a full verifiable perceptual status. These conditions involve the extent to which these experiences co-vary with the external environment, the kind of representational content they come to have and their neurophysiological correlates. Given the data currently available, it is certainly difficult to confirm if all these requirements are met, or that 6  Our goal here is not to defend functionalism, be it its strong or weak versions, but to stress that synaesthesia does not constitute an empirical objection to this model.

654    Malika Auvray and Ophelia Deroy they can be observed across all cases actually considered synaesthetic (see Auvray and Farina, in press, for a discussion on the boundaries of synesthesia). For the sake of the argument, let us postulate that some cases of synaesthesia fulfil all these conditions. Granting that there are cases of synaesthetic perception, what do they teach us about typical, i.e. non-synaesthetic, perception? Two stances can be adopted here:  one granting that synaesthesia is continuous or analogous to non-synaesthetic perception and the other, that it remains an idiopathic, isolated condition. This dilemma is what we expect to be synaesthesia’s core challenges. Saying that synaesthetes and non-synaesthetes are on a continuum leads to the problem of theorizing the ‘more or less’ synaesthetic aspect of all perception, whereas saying that it is perceptual but distinct from typical perception will involve consideration of perception as containing multiple distinct kinds. These two very distinct routes leave us to explore perception as a more varied or more eclectic phenomenon than what was, and often still is, assumed.

Acknowledgements We wish to thank the Editor Mohan Matthen, Brian Keeley, Lawrence Marks, Charles Spence, David Rosenthal, Jessica Hartcher O’Brian, the audience of the CUNY cognitive science seminar and at the Riga Summer School on Multisensory Perception, for their helpful comments on this material.

References Alter, T. (2006). ‘Does synaesthesia undermine representationalism?’ Psyche, 12(5), 1–11. Asher, J. E., Lamb, J. A., Brocklebank, D., Cazier, J. B., Maestrini, E., Addis, L., Sen, M., BaronCohen, S., and Monaco, A. P. (2009). ‘A whole-genome scan and fine-mapping linkage study of auditory-visual synesthesia reveals evidence of linkage to chromosomes 2q24, 5q33, 6p12, and 12p12’. American Journal of Human Genetic, 84(2), 279–285. Auvray, M. and Farina, M. (both authors are first authors) (in press). ‘Patrolling the boundaries of synaesthesia: A critical appraisal of transient and artificially-acquired forms of synaesthetic experience’. In O. Deroy (ed.), Sensory Blending: New Essays on Synaesthesia. Oxford: Oxford University Press. Bargary, G. and Mitchell, K. J. (2008). ‘Synaesthesia and cortical connectivity’. Trends in Neuroscience, 31(7), 335–342. Baron-Cohen, S., Bor, D., Billington, J., Asher, J., Wheelwright, S., and Ashwin, C. (2007). Savant memory in a man with colour form-number synaesthesia and asperger syndrome. Journal of Consciousness Studies, 14(9–10), 237–251. Baron-Cohen, S., Harrison, J. E., Goldstein, L. H., and Wyke, M. (1993). ‘Coloured speech perception: Is synaesthesia what happens when modularity breaks down?’ Perception, 50(4), 419–426. Bayne, T. (2009). Perception and the reach of phenomenal content. Philosophical Quarterly, 59(236), 385–404.

How Do Synaesthetes Experience the World?    655 Block, N. (1978). ‘Troubles with functionalism’. In C. W. (ed.), Minnesota Studies in the Philosophy of Science, vol. IX. Minneapolis: University of Minnesota Press. Repr. in N. Block (ed.), Readings in Philosophy of Psychology (pp. 268–305). Cambridge, MA: Harvard University Press. Block, N. (1980). ‘Introduction: What is functionalism?’ In N. Block (ed.), Readings in Philosophy of Psychology (pp. 171–184). Cambridge, MA: Harvard University Press. Brang, D., Coulson, S., and Ramachandran, V. S. (2011). ‘Similarly shaped letters evoke similar colors in grapheme-color synesthesia’. Neuropsychologia, 49(9), 1355–1358. Crane, T. (2011). ‘The problem of perception’. In E. Zalta (ed.), The Stanford Encyclopedia of Philosophy (Spring 2011 Edition), . Cytowic, R. E. (1989). Synaesthesia: a union of the senses. New York: Springer. Cytowic, R. E. (1993). The Man who Tasted Shapes. London: Abacus. Cytowic, R. E. (1997). ‘Synaesthesia: Phenomenology and neuropsychology. A review of current knowledge’. In S. Baron-Cohen and J. E. Harrison (eds), Synaesthesia: Classic and contemporary readings (pp. 17–39). Oxford: Basil Blackwell. Day, S. A. (2005). ‘Some demographic and socio-cultural aspects of synesthesia’. In L. C. Robertson and N. Sagiv (eds), Synesthesia: Perspectives from cognitive neuroscience (pp. 11–33). Oxford: Oxford University Press. Deroy, O. (2013a). ‘Phenomenal contrast without cognitive penetrability of perception’. Philosophical Studies, 162(1), 87–107. Deroy, O. (2013b). ‘Synaesthesia: An experience of the third kind?’ In R. Brown (ed.), Consciousness Inside and Out: Phenomenology, Neuroscience, and the Nature of Experience. Studies in Brain and Mind (pp. 395–407). Amsterdam: Springer Press. Deroy, O. and Spence, C. (2013). ‘Why we are not all synesthetes (not even weakly so)’. Psychonomic Bulletin & Review, 20(1), 1–22. Dixon, M. J., Smilek, D., Cudahy, C., and Merikle, P. M. (2000). ‘Five plus two equals yellow’. Nature, 406(6794), 365. Dixon, M. J., Smilek, D., and Merikle, P. M. (2004). ‘Not all synaesthetes are created equal: Projector versus associator synesthetes’. Cognitive, Affective and Behavioral Neuroscience, 4(3), 335–343. Dretske, F. (1995). Naturalizing the Mind. Cambridge, MA: The MIT Press. Dretske, F. (2000). Perception, Knowledge and Belief. Cambridge: Cambridge University Press. Edquist, J., Rich, A. N., Brinkman, C., and Mattingley, J. B. (2006). ‘Do synaesthetic colours act as unique features in visual search?’ Cortex, 42(2), 221–231. Evans, G. (1982). The Varieties of Reference. Oxford: Oxford University Press. Fechner, T. (1871). Vorschule der Aesthetik. Leipzig: Breitkopf und Hartel. Fish, W. (2010). Philosophy of Perception: A contemporary introduction. New York: Routledge. Galton, F. (1880). ‘Visualised numerals’. Nature, 21, 252–256. Gray, J. A., Chopping, S., Nunn, J., Parslow, D., Gregory, L., Williams, S. C. R, Brammer, M. J., and Baron-Cohen, S. (2002). ‘Implications of synaesthesia for functionalism: Theory and experiments’. Journal of Consciousness Studies, 9(12), 5–31. Gray, J. A., Parslow, D. M., Brammer, M. J., Chopping, S., Vythelingum, G. N., and Fytche, D. H. (2006). ‘Evidence against functionalism from neuroimaging of the alien color effect in synesthesia’. Cortex, 42(2), 309–318. Gray, J. A., Williams, S. C. R., Nunn, J., and Baron-Cohen, S. (1997). ‘Possible implications of synaesthesia for the hard question of consciousness’. In S. Baron-Cohen and J. E.

656    Malika Auvray and Ophelia Deroy Harrison (eds), Synaesthesia: Classic and contemporary readings (pp. 173–181). Oxford: Blackwell. Gray, R. (2001a). ‘Synaesthesia and misrepresentation: A reply to Wager’. Philosophical Psychology, 14(3), 339–346. Gray, R. (2001b). ‘Cognitive modules, synaesthesia and the constitution of psychological natural kinds’. Philosophical Psychology, 14(1), 65–82. Grossenbacher, P. G. and Lovelace, C. T. (2001). ‘Mechanisms of synesthesia: Cognitive and physiological constraints’. Trends in Cognitive Sciences, 5(1), 36–41. Harrison, J. and Baron-Cohen, S. (1997). ‘Synesthesia: A review of psychological theory’. In S. Baron-Cohen and J. E. Harrison (eds), Synaesthesia: Classic and contemporary readings (pp. 109–122). Oxford: Basil Blackwell. Hochel, M., Milan, E. G., Martin, J. L. M., Gonzalez, A. E., Garcia, D. F., Tornay, F., and Vila, J. (2008). ‘Congruence or coherence? Emotional and physiological responses to colours in synaesthesia’. The European Journal of Cognitive Psychology, 21(5), 703–723. Hong, S. W. and Blake, R. (2008). ‘Early visual mechanisms do not contribute to synesthetic color experience’. Vision Research, 48(8), 1018–1026. Hubbard, E. M. (2007). ‘Neurophysiology of synesthesia’. Current Psychiatry Reports, 9(3), 193–199. Hubbard, E. M., Arman, A. C., Ramachandran, V. S., and Boynton, G. M. (2005). ‘Individual differences among grapheme-color synesthetes: Brain-behavior correlations’. Neuron, 45(6), 975–985. Hupe, J. M., Bordier, C., and Dojat, M. (2011). ‘The neural bases of grapheme-color synesthesia are not localized in real color-sensitive areas’. Cerebral Cortex, 22(7), 1622–1633. Keeley, B. L. (2013). ‘What exactly is a sense?’ In J. Simner and E. Hubbard (eds), The Oxford Handbook of Synaesthesia. Oxford: Oxford University Press. Leeuwen (van), T. M., Petersson, K. M., and Hagoort, P. (2010). ‘Synaesthetic colour in the brain: Beyond colour areas. A functional magnetic resonance imaging study of synaesthetes and matched controls’. PLoS ONE, 5(8), e12074. Leibniz, G. (1765/1982). New Essays on Human Understanding, trans. and ed. Peter Remnant and Jonathan Bennett. Cambridge: Cambridge University Press. Locke, J. (1690/2008). Essay Concerning Human Understanding. Oxford:  Oxford University Press. Lycan, W. (2006). Representational theories of consciousness. In E. N. Zalta (ed.), The Stanford Encyclopedia of Philosophy, URL = . Macpherson, F. (2007). Synaesthesia, functionalism and phenomenology. In M. de Caro, F. Ferretti, and M. Marraffa (eds), Cartographies of the Mind: Philosophy and psychology in intersection series. Studies in brain and mind (pp. 65–80). Dordrecht: Springer. Marks, L. (2011). ‘Synaesthesia, now and then’. Intellectica, 55, 47–80. Mills, C., Botteler, E. H., and Oliver, G. K. (1999). ‘Digit synaesthesia: a case study using a Stroop type test’. Cognitive Neuropsychology, 16(2), 181–191. Mroczko, A., Metzinger, T., Singer, W., and Nikolić, D. (2009). ‘Immediate transfer of synesthesia to a novel inducer’. Journal of Vision, 9(12), 1–8. Nunn, J. A., Gregory, L. J., Brammer, M., Williams, S. C. R., Parslow, D. M., Morgan, M. J., Morris, R. G., Bullmore, E. T., Baron-Cohen, S., and Gray, J. A. (2002). ‘Functional magnetic resonance imaging of synesthesia: Activation of V4/V8 by spoken words’. Nature Neuroscience, 5(4), 371–375.

How Do Synaesthetes Experience the World?    657 Ortmann, O. (1933). ‘Theories of synesthesia in the light of a case of colored hearing’. Human Biology, 5, 155–211. Palmeri, T. J., Blake, R., Marois, R., Flanery, M. A., and Whetsell, W. (2002). ‘The perceptual reality of synesthetic colors’. Proceedings of the National Academy of Science of the United States of America, 99(6), 4127–4131. Ramachandran, V. S. and Hubbard, E. M. (2001a). ‘Synaesthesia: A window into perception thought and language’. Journal of Consciousness Studies, 8(12), 3–34. Ramachandran, V. S. and Hubbard, E. M. (2001b). ‘Psychophysical investigations into the neural basis of synaesthesia’. Proceedings of the Royal Society of London B, 268(1470), 979–983. Rich, A. N., Bradshaw, J. L., and Mattingley, J. B. (2005). ‘A systematic, large-scale study of synaesthesia: Implications for the role of early experience in lexical-colour associations’. Cognition, 98(1), 53–84. Rosenberg, G. (2004). A Place for Consciousness: Probing the deep structure of the natural world. New York: Oxford University Press. Rothen, N., Nikolić, D., Jurgens, U. M., Mroczko-Wąsowicz, A., Cock, J., and Meier, B. (2013). ‘Psychophysiological evidence for the genuineness of swimming-style coloursynaesthesia’. Consciousness and Cognition, 22(1), 35–46. Rouw, R. (2011). ‘ "Special cases": Neural mechanisms and individual differences in synaesthesia’. Journal of Neuropsychology, 5, 145–151. Rouw, R. and Scholte, H. S. (2007). ‘Increased structural connectivity in grapheme-color synesthesia’. Nature Neuroscience, 10(6), 792–797. Sachs, G. T. L. (1812). Historiae naturalis duorum leucaetiopum: Auctoris ipsius et sororis ei us. Solisbaci. Germany: Sumptibus Bibliopolii Seideliani. Sagiv, N., Simner, J., Collins, J., Butterworth, B., and Ward, J. (2006). ‘What is the relationship between synaesthesia and visuo-spatial number forms?’ Cognition, 101(1), 114–128. Sagiv, N., Ilbeigi, I., and Ben-Tal, O. (2011). Reflections on synaesthesia, perception, and cognition. Intellectica, 55, 81–94. Segal, G. (1997). ‘Synaesthesia: Implications for the modularity of mind’. In S. Baron-Cohen and J. E. Harrison (eds), Synaesthesia: Classic and contemporary reading (pp. 211–224). Oxford: Basil Blackwell. Shoemaker, S. (1975). ‘Functionalism and qualia’. Philosophical Studies, 27(5), 291–315. Siegel, S. (2006). ‘Which properties are represented in perception?’ In T. Gendler and J. Hawthorne (eds), Perceptual Experience (pp. 481–503). Oxford: Oxford University Press. Simner, J. (2007). ‘Beyond perception: Synaesthesia as a psycholinguistic phenomenon’. Trends in Cognitive Sciences, 11(1), 23–29. Simner, J. (2012). ‘Defining synaesthesia’. British Journal of Psychology, 103(1), 1–15. Simner, J., Mulvenna, C., Sagiv, N., Tsakanikos, E., Witherby, S. A., Fraser, C., Scott, K., and Ward, J. (2006). ‘Synaesthesia: The prevalence of atypical cross-modal experiences’. Perception, 35(8), 1024–1033. Simner, J., Ward, J., Lanz, M., Jansari, A., Noonan, K., and Glover, L. (2005). ‘Non-random associations of graphemes to colours in synaesthetic and non-synaesthetic populations’. Cognitive Neuropsychology, 22(8), 1069–1085. Smilek, D., Dixon, M. J., and Merikle, P. M. (2001). ‘Synaesthetic photisms guide attention’. Brain and Cognition, 53(2), 364–367. Smilek, D., Malcolmson, K. A., Carriere, J. S. A., Eller, M., Kwan, D., and Reynolds, M. (2007a). ‘When "3" is a jerk and "E" is a king: Personifying inanimate objects in synesthesia’. Journal of Cognitive Neuroscience, 19(6), 981–992.

658    Malika Auvray and Ophelia Deroy Smilek, D., Carriere, J. S., Dixon, M. J., and Merikle, P. M. (2007b). ‘Grapheme frequency and color luminance in grapheme–color synaesthesia’. Psychological Science, 18(9), 793. Sollberger, M. (2013). ‘Rethinking synaesthesia’. Philosophical Psychology, 26, 171–187. Spence, C. and Deroy, O. (2013). ‘Crossmodal mental imagery’. In S. Lacey and R. Lawson (eds), Multisensory Imagery: Theory and applications (pp. 157–183). Philadelphia: Springer. Spiller, M. J. and Jansaria, A. S. (2008). ‘Mental imagery and synaesthesia: Is synaesthesia from internally-generated stimuli possible?’ Cognition, 109(1), 143–151. Stroop, J. R. (1935). ‘Studies of interference in serial verbal reactions’. Journal of Experimental Psychology, 18(6), 643–662. Tye, M. (2000). Consciousness, Color, and Content. Cambridge, MA: MIT Press. Tye, M. (2002). ‘Representationalism and the transparency of experience’. Nous, 36(1), 137–151. Wager, A. (1999). ‘The extra qualia problem: synaesthesia and representationalism’. Philosophical Psychology, 12(3), 263–281. Wager, A. (2001). ‘Synaesthesia misrepresented’. Philosophical Psychology, 14(3), 347–351. Ward, J. (2008). The Frog who Croaked Blue: Synesthesia and the mixing of the senses. New York: Routledge. Ward, J., Li, R., Salih, S., and Sagiv, N. (2007). ‘Varieties of grapheme-colour synaesthesia: A new theory of phenomenological and behavioural differences’. Consciousness and Cognition, 16(4), 913–931. Ward, J. and Simner, J. (2003). ‘Lexical-gustatory synaesthesia: Linguistic and conceptual factors’. Cognition, 89(3), 237–261. Witthoft, N. and Winawer, J. (2010). ‘10 Color-grapheme synesthetes with highly similar learned associations’. Journal of Vision, 10(7), 871.

Chapter 35

Su bstitu ti ng the  Senses Julian Kiverstein, Mirko Farina, and Andy Clark

1  Introduction Are the eyes necessary for seeing or could a person or animal with impaired vision see by means of a sensory prosthesis?1 How one answers this question will depend on the criteria by which one makes a distinction between the senses. Roxbee-Cox (1970) discusses what he calls ‘the sense organ account’ according to which ‘certain parts of the body with which we are familiar, make an instance of perception sight; the functioning of other processes, involving other parts of the body, make it a case of hearing, etc’. (1970: 530, quoted by Nudds, 2004: 3) Sensory substitution devices (or SSDs) are potential counterexamples to this ‘sense organ’ account of the senses.2 SSDs are wearable computing devices engineered to function as prostheses that compensate for the loss of vision in the visually impaired. Consider by way of analogy how a blind person uses a cane to find their way about in the world. Each tap of the cane is felt in the hand, yet as Descartes noticed in his Physiological Optics the blind person can use these tactile sensations to feel the space around them. Descartes even goes so far as to suggest that a blind person that has used the stick for all their lives might be said to ‘see with their hands’ (1653/1985: 153). The first sensory substitution devices were attempts at harnessing tactile inputs more globally, converting visual images transduced by a mobile camera into vibrotactile stimulation. Paul Bach-y-Rita was the first to attempt such an experiment in the 1970s. Through his work on neural rehabilitation Bach-y-Rita was among the first neurologists to recognize the brain’s plasticity and the enormous potential for recovery it offered to patients 1 

When we talk of ‘seeing’ we will normally not distinguish between conscious and unconscious vision. We wish to largely bracket questions about consciousness in this chapter. When this is not possible we will make this explicit. 2  This might well be denied: Morgan (1977) has argued for instance that the difference between sensory pick-up by means of a substitution device and sensory pick-up by means of the eyes is quantitative, not qualitative. He argues that these structural differences are not sufficient to warrant the conclusion that perception with an SSD is not visual. We briefly return to this line of argument in section 2.

660    Julian Kiverstein, Mirko Farina, and Andy Clark suffering from brain damage. This work led him to wonder whether it might be possible to exploit this plasticity to restore sight in the blind, using the sense of touch to substitute for their impaired visual sense. Vision is an enormously rich source of knowledge about the external world; Bach-y-Rita hypothesized that the brain could learn to work together with a prosthesis so as to acquire the very same types of knowledge through touch. This was a hypothesis he was later to confirm, building what, by today’s standards, look to be incredibly cumbersome devices. Despite the apparent user unfriendliness of the devices, blind users nevertheless succeeded in mastering their use to report on a wide variety of different stimuli. The most striking feature of the initial results with the systems is that subjects, blind and sighted, are able, after only relatively short training periods, to identify familiar objects and to describe their arrangement in depth. . . . Evidently even a crude 400-tactor system is capable of providing sufficient information to permit construction of what is usually called the visual world. (White et al., 1970: 25)

The potential of these devices to restore some degree of visual function is of course of tremendous practical relevance for those suffering from visual impairments. Still we should resist jumping to any conclusions about the type of sense perception the skilled user enjoys. Does the device restore visual function or does it simply extend the range of objects and properties that can be discriminated using touch or hearing? We can make this question a little more precise by introducing some terminology we will occasionally employ below. We will use the expression ‘substituting modality’ to refer to the transducing sensory channel delivering the information to the brain (e.g. touch or hearing). We will also refer to the ‘substituted modality’ by which we will mean the sense modality that the SSD is being used to replace3. A key question we wish to address is whether the perceptual states the device helps to implement remain in the substituting modality, or do these states switch from the substituting modality to the substituted modality once the user has learned how to use the device?4

2  Sensory substitution and the meta-modal brain Sensory substitution devices (SSDs) have most commonly been researched as visual prostheses for the blind.5 A  human machine interface (HMI) converts images from a camera attached to the user’s body into tactile or auditory inputs. In the early tactile-visual devices, a fixed camera delivered real-time images to an HMI, which transformed these 3  For useful reviews, see Bach-y-Rita and Kercel (2003) and the opening sections of Auvray and Myin (2009). 4  Matthen takes up the question of how to individuate the senses. 5  Touch has also been used to substitute for other senses. For example, Schürmann et al. (2006) have built an SSD that uses touch to replace hearing, and Tyler et al. (2003) have done the same for the same vestibular sense (if this counts as sense). Bach-y-Rita and Kercel (2003) describes a tactile–tactile substitution device that restores peripheral sensation. In what follows we will concentrate entirely on devices that aim to function as substitutes for vision.

Substituting the Senses   661 images pixel by pixel into electro- or vibro-tactile stimulation delivered to the skin of the user. After a period of training, the user learned how to interpret the tactile stimulation so as to recognize the orientation, distance, and spatial arrangement of simple stimuli, and to successfully perform discrimination tasks. Some subjects also reported undergoing such distinctively visual experiences as motion parallax and looming (Bach-y-Rita et al., 1969). The early devices suffered from a number of problems, not least of all the user’s lack of mobility. The first devices built by Bach-y-Rita consisted of a dentist’s chair that would produce a pattern of vibration on the back of the person sitting in the chair. The devices in use today are much more user-friendly and can be readily deployed by the visually impaired in their everyday dealings with the world. Below we discuss two of the mostly widely researched devices: a portable version of the original Bach-y-Rita visual-to-tactile system called Brain-Port and a visual-audio system known as the vOICe.6 Brain-Port is built from three core components: (1) A 3cm × 3cm tongue array (containing 100 to 600+ electrodes); (2) a hand-held controller (slightly larger than a cell phone) used for zoom and contrast inversion; and (3) a head-mounted camera that maps the environment into high-resolution pictures. The camera captures high-resolution images at a rate of 30 frames per second. The hand-held controller contains a central processing unit (CPU) that converts these images into retinotopically organized electrical pulses delivered to an array of stimulators worn on the tongue.7 Each electrode on the tongue array corresponds to a specific set of pixels. When the user actively controls the camera, she gradually begins to acquire a feel for the things around her, and she does so on the basis of sensations on the surface of her tongue. Long-term, proficient users of Brain-Port report the sensation of ‘seeing pictures painted on their tongue with champagne bubbles’ (Danilov et al., 2005). They can routinely recognize high-contrast objects in a scene, read letters or numbers, individuate shapes, perform perceptual judgements using perspective, parallax, looming and zooming, and successfully estimate depth (Arnoldussen and Fletcher, 2012).8 The vOICe9 is an visual-auditory substitution device that works by transforming images from a digital camera embedded in a pair of sunglasses into auditory frequencies (‘soundscapes’), which the user hears through headphones. The mapping of images onto sounds follows three basic rules:

1) The vertical position of objects in the video image is encoded by tone frequency; high pitch tones correspond to objects located in the upper regions of the image and low pitch tones to objects located in lower regions. 2) Once a second, the camera scans from left to right and objects to the user’s left or right are encoded as sounds delivered to the left or right ear respectively. 3) Finally, the luminosity or brightness of an object is encoded as loudness, the brighter the object the louder the sound it produces.

6  Perhaps the most well-known of this class of devices is the vOICe which we will discuss in more detail in what follows (see e.g. Meijer, 1992). Other devices that work according to the same principles are the PSVA (Prosthesis Substituting Vision with Audition, Capelle et al., 1998) and the Vibe (Auvray et al., 2005). 7  Besides housing the central processing unit (CPU), the hand-held controller is also in charge of other crucial functions. Through its specific configuration it in fact allows the impaired user to: (1) zoom in on items of interest; (2) control light settings and shock intensity levels; and (3) adjust contrast at need. 8  For an impressive demonstration of what a blind user of Brain-Port can do, see: . 9  ‘OIC’ stands for ‘Oh I see!’

662    Julian Kiverstein, Mirko Farina, and Andy Clark Soundscapes preserve the visual information contained in the camera images to a high degree of spatial resolution.10 Visually impaired users of the vOICe can effectively determine the spatial location of objects (Auvray et  al., 2007); discriminate size, shape and orientation (Cronly-Dillon et al., 2000; Amedi et al., 2007); identify and classify types of objects (Auvray et al., 2007) and discriminate between patterns of dots (Arno et al., 2001) and between images of objects and scenes (Cronly-Dillon et al., 2000). Over the last ten years or so, cognitive neuroscientists have begun to study the changes that take place in the user’s brain as they learn to use these devices. There is growing evidence that SSDs work by exploiting the brain’s cross-modal plasticity: the ‘ability of a cortical area normally specified for one type of sensory input . . . to take on some of the functions of another sensory system’ (Huttenlocher 2002: 81). In this light, consider the classic work by Sadato and colleagues, who used positron emission tomography (PET) to study the behaviour of the visual cortex in congenitally and early blind users of Braille (Sadato et al., 1996). They found that tactile contact with Braille activates areas of occipital cortex that normally process visual inputs. Visual cortex in early blind readers of Braille reorganizes so as to support the processing of tactile inputs.11 This kind of plasticity has also been observed in cases of auditory deprivation. Sign language, for instance, has been consistently reported to trigger the auditory cortex of deaf individuals (Sadato et al., 2004). Sensory substitution also works by exploiting the cross-modal plasticity of sensory cortex. Ptito et al. (2005) trained a group of early blind patients and a blindfolded control group to use a tactile-visual substitution system (a Tongue Display Unit or TDU).12 Following training, they were given the task of determining the orientation of a letter (T) presented on a screen while undergoing a PET scan of their brains. Ptito et al report the activation of ‘large areas of occipital (cuneus, inferior, medial, and lateral occipital cortex), occipito-parietal and occipito-temporal (fusiform gyrus) cortices’ in blind subjects, but not in the control group (Ptito et al., 2005: 486). In the blind users, occipital cortex is recruited for tactile discrimination, and the patients’ tongues come to act ‘as portals to convey somatosensory information to visual cortex’ (Ptito et al., 2005: 606). We suggest that SSDs unmask a potential in the sensory cortex to process stimuli regardless of sensory modality. This contrasts with a view of sensory processing that goes back to the great neurophysiologist of the nineteenth century, Johannes Müller, and his law of specific nerve energies (Müller 1843/2003). Müller argued that the nerve fibres connecting the different sense organs (the eyes, ears, tongue, etc.) to the brain behave in fundamentally the same ways as the nerve fibres connecting the brain to the muscle system in the body. Thus he was led to ask what it is about these nerve fibres that makes for differences in sense perception. The answer he returned was that the sensory differences are determined by 10  Up to about 60×60 pixels for a 1s sound scan and a 5kHz audio bandwidth (Bubic et al., 2010). Further technical details can be found at where it is also possible to download a demo of the vOICe and get a real sense of how the device works. 11 This finding has been confirmed in countless studies. Cohen et  al. (1997), for instance. used transcranial magnetic stimulation (TMS) over primary visual cortex in early-onset blind subjects during Braille reading. This interfered and disrupted the subject’s ability to discriminate the Braille characters again demonstrating the necessary role of visual cortex in supporting tactile discrimination in these subjects. 12  Also see the earlier study by Arno et al. (2001), a PET study in which activation of occipital cortex was found in blind users of an visual-auditory substitution device.

Substituting the Senses   663 the specific anatomical regions in the brain to which the nerve fibres lead. Visual perception is the result of nerve fibres that connect the retina to the visual areas of the brain. Auditory perception by contrast is the result of wiring that runs from the ears to the auditory cortex. The discovery of cross-modal plasticity is quite consistent with the fully developed sensory cortex having specialized functions, some areas of sensory cortex favouring the processing of visual inputs, other areas preferring to process auditory inputs and so on. However cross-modal plasticity strongly suggests that every sensory area also has the potential to process inputs from other modalities. Just as Müller argued that the nerve fibres that connect the sense organs to the brain do not distinguish between the nature of the signals they transmit, so it seems we should say the same about the terminus of those signals. Sensory cortex has a latent potential to process inputs from other sense modalities that is unmasked in cases of cross-modal plasticity. Pascual-Leone and Hamilton (2001) suggest that we should understand sensory cortex as ‘meta-modal’: One might envision a cortical region with a functional role in spatial discrimination that might therefore be predisposed to perform the kinds of processes that vision requires. In this setting, from early development onward, sight would be progressively selected as the input signal for such an ‘operator’. Eventually such an operator might appear to be ‘visual cortex’ by virtue of its dominant input, when in fact, under certain conditions, the presence of meta-modal inputs could be unmasked. (Pascual-Leone and Hamilton, 2001: 6)

During development, regions of the cortex become increasingly specialized in function because regions differ in their computational properties with some brain regions being better at carrying out particular types of computation than others. Occipital cortex, for instance, might be particularly good at the high-acuity processing of spatial information required for vision. Over time, processing of visual inputs in the occipital cortex is reinforced and processing of inputs from other sensory modalities is inhibited. In the early blind, occipital cortex never takes on this function but its computational properties make it best suited to processing tasks that require high acuity. This is perhaps why we find it activated in tasks that might be thought to require highly accurate spatial processing such as Braille reading13 and SSD perception.14 On a meta-modal view of sensory cortex

13 

There is some evidence that Braille reading may not require particularly precise spatial processing, see e.g. Loomis et al. (2012). Loomis and colleague’s work strongly suggests that touch lacks the spatial resolution of vision presenting an upper bound on the spatial information the visual cortex can extract from tactile inputs. This doesn’t impact on the ability of a blind person to read Braille since the latter doesn’t require high acuity spatial processing (Loomis, 1990). It may, however, provide a serious constraint on the types of visual information we can access through touch, and hence on the types of visual information a perceiver can access through this type of sensory substitution. Loomis argues that tactile substitution works best in ecologically unrealistic settings in which subjects are presented with distinctive objects in simplified contexts against high-contrast backgrounds. In this way work on tactile substitution avoids much of the challenge of segregating objects and features ordinarily taken care of in early visual processing. Loomis’s work suggests this is a challenge touch is ill-equipped to meet. It is an important question for empirical research to what extent this problem also carries over to auditory substitution. We thank Jack Loomis for discussion of this and other issues relating to the limits of sensory substitution. 14  There is further support for such a claim in the finding that auditory and tactile processing of motion in both blind and sighted subjects recruits area MT/V5, a visual area specialized for processing motion in

664    Julian Kiverstein, Mirko Farina, and Andy Clark we should think of functional specialization in terms of computation of features such as shape, motion, and spatial location. A given brain region can exhibit this specialization irrespective of the nature of the inputs that are projecting to this area.15

3  Keeley on the senses Brian Keeley (2002, 2009) defends a neurobiological criterion for counting the senses, according to which token perceptions belong to different sense modalities because of the ‘character of the putative sense organs and their modes of connection with the brain’ (2002: 13). Thus he is defending a revised version of what we’ve called above the ‘sense organ account’ of the senses. Instead of differentiating the senses just on the basis of whether experience begins in the eyes, ears, nose, tongue, or skin, Keeley argues we must also take into account whether the sense organ is, as he puts it, ‘appropriately wired up’. The view of sensory processing we’ve outlined here suggests that the sense organ account may be something of a red herring for a theory of the senses. The brain can model the ways in which sensory input varies as we interact with the environment in ways that are independent of the bodily transducer. Keeley discusses a view like the one we’ve been defending in a part of his paper dealing with the ecological psychologist J. J. Gibson’s theory of sensory systems. He represents Gibsonians as holding that ‘it makes sense to attribute a sensory modality to any organism that can act on structured stimuli of a particular physical type regardless of how that information is obtained by the organism’ (Keeley, 2002: 18). This accords well with the findings we described in the previous section. Keeley argues, however, that this Gibsonian position overlooks an important distinction between ‘detection’ and ‘reception’. He invites us to suppose that a blind user of an SSD could ‘come to have every propositional attitude a sighted person has’. Still, he suggests, such a person would continue to count as blind because they would lack the modality of vision. They would be equipped with a tool for reliably detecting information carried by electromagnetic forms of energy. Their perceptual mode of interaction with the world would remain tactile because they lack a perceptual system dedicated to the reception of electromagnetic stimuli. If a blind person using a substitution device could come to share every propositional attitude with a sighted person would we really continue to count them blind? Keeley says we would, but we might equally well conclude that this points to the implausibility of his account of the senses. Suppose we adopt a functionalist definition of the senses for instance normally sighted subjects (Ricciardi et al., 2007). Amedi et al. (2003) have likewise shown that the lateral occipital cortex is recruited for tactile processing of shapes and of Braille in sighted and blind subjects. Wolbers et al. (2011) found activation of the parahippocampal place area involved with the processing of visually presented scenes in blind subjects haptically exploring lego-block models of rooms. These studies and many other strongly point to the conclusion that sensory brain areas construct representations of properties like shape or motion independently of the format of the transducing sense. 15 This view of the sensory cortex as made up of meta-modal operators fits well with a view of perception that is growing in popularity within cognitive neuroscience according to which perception is fundamentally a matter of prediction (see Rao and Ballard, 1999; Lee and Mumford, 2003; Friston, 2005, 2010; Clark, 2012).

Substituting the Senses   665 and we say the differences between the senses can be accounted for in terms of the causal role sensory states typically play in mediating between inputs and outputs. blind user of an SSD who formed, in the very same circumstances, the same propositional attitudes as a sighted person, would have perceptual states that were functionally equivalent with the visual states of sighted person. We would say of such a user that the device provided him with a visual mode of access to the world. In reality, substitution devices fall a long way short of achieving this level of performance (Loomis et al., 2012). At best, users can succeed in simplified experimental settings at localizing and recognizing objects to a level of competence that approximates that of a sighted person. Furthermore many questions remain about how to fill out a functionalist theory of the senses. How are we to determine the nature of the inputs and outputs that the sensory states in question mediate between? How we answer this question will decide whether we count sensory substitution as functionally equivalent with vision. If we take the relevant inputs to be proximal, vibro-tactile stimulation for instance, we will count tactile substitution and vision as distinct functional states. The proximal stimulation will be vibro-tactile stimulation in the one case and light striking the retina in the other. If we start from the distal stimulus, however, this might support a conclusion that sensory substitution and vision are functionally equivalent. The distal stimulus that is the source of sensory inputs is the same in both cases even though there are differences in proximal stimulation. This is of course a well-known general problem for functionalist theories of mind (Block, 1978). Supposing, however, these and other questions can be finessed, a broad commitment to functionalism gives us some grounds for questioning Keeley’s distinction between detection and reception. Let us consider then how he goes about justifying this distinction. Keeley invites us to consider the human ability to behaviourally distinguish the electrical charge of two batteries by applying them to the tongue. The tongue will certainly allow us to distinguish a live from a dead battery, but it is not a sense organ dedicated to the reception of electrical forms of energy. Now contrast the human tongue with sharks that can allegedly detect the presence of prey using electric fields. Sharks have a sense that we lack because they have a sense that has, says Keeley ‘evolved specifically to process biologically meaningful electrical stimuli in their environment’ (Keeley, 2002: 26). Sharks have a dedicated anatomical structure that is ‘electroreceptive’. The human tongue, and the neural circuitry to which it is wired up, is by contrast dedicated to the reception of pain, taste, and touch. It can be used to detect electricity but only by means of receptors dedicated to picking up other forms of physical energy. The appeal to evolutionary history plays a central role in Keeley’s reasoning: What makes the eye part of a visual system, but not part of a mechanosensory or stimulating-electrode-receptive system, is the evolutionary history of those verterbates which have eyes. It is this history which determines to what sense a putative sense-organ is dedicated. (Keeley, 2002: 18)

However, his account of the senses as dedicated sense organs hooked up to functionally specialized anatomical structures doesn’t fit at all well with the idea of the meta-modal brain we discussed above. This should come as no surprise since we’ve already seen how he rejects Gibsonian views which share a commitment to meta-modality. Compare Keeley’s

666    Julian Kiverstein, Mirko Farina, and Andy Clark account of the senses with Müller’s law of specific nerve energies discussed earlier (in section 1). In more recent work, Keeley discusses with measured approval, Müller’s claim that ‘sensation consists in the sensorium receiving through the medium of the nerves, and as the result of the action of an external cause, a knowledge of certain qualities or conditions . . . of the nerves of sense themselves . . . the nerve of each sense having its own peculiar quality or energy’ (Keeley, 2009: 241). The ‘sensorium’ is Müller’s term for the areas of the brain ‘responsible for receiving and analysing information from the sense organs’ (2009: 241). Keeley doesn’t accept Müller’s claim that these brain areas each have their own peculiar quality or energy that cannot be given a further analysis. However, he does endorse the claim that for each sense, there are neural pathway(s) dedicated to the reception of particular forms of energy in the environment. In discussing sensory substitution he quotes Müller again: ‘Among the well-attested facts of physiology, again, there is not one to support the belief that one nerve of sense can assume the functions of another’ (2009: 242). This last claim looks to be seriously challenged by the evidence for meta-modality. At least it does if we follow Keeley in interpreting the ‘nerves of sense’ as the ‘sensorium’ or ‘the brain areas responsible for the receiving and analysis of information from the sense organs’. What this work shows is that one and the same brain area (e.g. the lateral occipital cortex) can be involved in receiving and analysing information from very different senses. We don’t have anatomical structures that are dedicated (because of our evolutionary history) to the reception and analysis of particular forms of physical energy such as electromagnetic stimuli received by the eyes or chemical gradients picked up the tongue or nose. Consider as a further example, Thaler et  al.’s (2011) finding that when blind persons echolocate, there is time-locked activation of the visual cortex. Blind individuals echolocate by emitting high frequency burst clicks generated by tickling the palate of the mouth with the tip of the tongue. These high frequency clicks bounce off the objects located in the external environment in a way that allows the adept echolocator to map the environment surrounding them perceiving the distance, size, position, shape, and texture of objects (Teng et al., 2012).16 Proficient echolocators (such as Daniel Kish and the late Ben Underwood) have been reported to use echolocation to hike, roller-skate, mountain bike, and to play fussball, basketball, and video games.17 Thaler and colleagues used fMRI to compare the brain responses of two blind subjects (EB, short for ‘Early Blind’ and LB ‘Late Blind’) who have been echolocating for many years with two control subjects who cannot echolocate.18 The subjects were played clicks and echoes recorded under three scenarios while they were scanned. In all three of the conditions, they found increased activation in the calcarine area of the visual cortex in 16  The

cognitive anthropologist Greg Downey has a rich discussion of echo-location, including the Thaler study we discuss posted on his Neuroanthropology blog accessible here: . 17  Daniel Kish (subject EB in the study discussed below) runs a school in California in which he trains other blind people in the use of echolocation and in what he calls ‘Perceptual Mobility’. More information can be found here: . For a sample of the outstanding skill displayed by echolocators, see the following: (last accessed August 2011). 18  Many thanks to Lore Thaler and Mel Goodale for discussion of this experiment.

Substituting the Senses   667 the blind echolocators, but not in the controls. Primary visual cortex (V1) is located in and around the calcarine sulcus, so it seems that the primary visual cortex is being used in these subjects for the analysis of auditory signals. Particularly interesting for us was a condition in which a BOLD response19 was compared for recordings made outdoors, the first containing both clicks and echoes and a second, control recording containing only the clicks and the background sounds with the echoes removed. The two echolocating subjects showed an increased BOLD signal in the calcarine sulcus, and in areas of the cerebellum, in contrast to the control subjects who showed no differential response in these areas. Interestingly, however, no difference in the BOLD signal was found for activity in the auditory cortex in any of the four subjects. Auditory cortex was certainly active in both conditions, as shown by a comparison of BOLD response in a silence condition with a condition in which the recording was played. There was, however, no difference in BOLD signal response in auditory cortex when the echo was part of the outdoor recording, and when it was not. Thus, it seems that the brain responses (as reflected in the BOLD signal) that seem to make the difference for successful echolocation are not to be found in the auditory cortex, but are instead found in the visual cortex, and in the cerebellum. The experimenters interpret this as further evidence for the meta-modal properties of the primary visual cortex: ‘calcarine cortex’, they write, ‘performs some sort of spatial computation that uses input from the processing of echolocation sounds that was carried out elsewhere, most likely in brain areas devoted to auditory processing’ (Thaler et al., 2011: 8). It seems then that visual cortex is not dedicated to the reception of forms of energy fixed by our evolutionary history. Visual cortex processes information carried by light in the normally sighted, but it can also process information carried by clicks and echoes. Visual cortex (and sensory cortex more generally20) is plastic, and its functioning is in large part experience-dependent. In this case it is dependent on the acquisition of a skill. The two subjects in this experiment have repeatedly engaged in echolocation, EB from a very early age and LB a little later in life. There are also interesting individual differences in BOLD response in the two echolocation experts studied in this experiment which the experimenters suggest may reflect level of expertise or practice. For instance, echo-related activity in the calcarine sulcus showed a contralateral bias in subject EB, but not in subject LB. (This same contralateral bias is found in sighted subjects for light, so this is an example of the visual cortex taking on structural properties independently of the character of inputs.) EB also showed greater visual cortex response in some of the conditions as compared with LB, which the authors suggest ‘could reflect EB’s much longer use of echolocation and/or his more reliable performance in passive echolocation task’ (Thaler et al., 2011: 3). There is then a match between the level of skill of the echolocator and the acuity and richness of the spatial information the visual cortex can extract from sound. Returning to Keeley, it is clear that he will want to say our echolocators use hearing to detect spatial properties we normally perceive through vision such as the distance, size, 19  The BOLD (Blood Oxygen Level Dependent) signal is the indirect measure of brain activity relied on in fMRI studies. The nature of the correlation between levels of oxygen in blood flow and neural activity remains a matter of controversy in cognitive neuroscience. 20  For evidence of cross-modal plasticity in auditory cortex, see Lomber et al. (2010).

668    Julian Kiverstein, Mirko Farina, and Andy Clark and position of objects. However, they are not doing so by means of auditory cortex which is what his theory requires. They are doing this using primary visual cortex which is precisely not dedicated to processing information carried by sounds. Moreover, this is a function the visual cortex acquires not as a result of our evolutionary history but through repeated engagement in a skilful behaviour. This is exactly the same kind of neural reorganization we find in readers of Braille whose brains devote a larger area of somatosensory cortex to the processing of sensory input from the fingertips (Pascual-Leone et  al., 1993), or in London taxi-drivers whose hippocampus is enlarged as a consequence of their learning to navigate around London (Maguire et al., 2000). Keeley’s defence of the distinction between detection and reception is unsuccessful, but doesn’t his point about sensory substitution still stand? Keeley argued that blind users of sensory substitution devices don’t miraculously regain their sight. To the extent that they are able to detect spatial properties of objects by means of touch or hearing, they do so in much the same way as we can detect the electrical properties of batteries using our tongues. Are blind users of substitution devices simply detecting properties we normally perceive visually by a non-standard sense?

4  Does the substituting sense dominate in sensory substitution? When a blind person masters the use of an SSD do they learn to see or does their experience continue to have the character of the substituting modality? Central to the idea of sensory substitution is that an intact, normally functioning sense can act as a stand in or substitute for an impaired sense. We’ve seen how substitution devices can provide some of the same information as the impaired sense, and can enable the user to act and respond in some of the same ways as vision. Thus there is a functional similarity between perception with a substitution device and perception in the substituted modality. Is this functional similarity a sufficient ground for saying that a blind person can learn to see through the use of an SSD? We can distinguish between three possible answers which we must decide between in order to settle this question:



1) Through training, the visually impaired user learns to acquire new types of sensory information by means of a substituting modality. The mode of perception of the user’s experience, however, remains in the substituting modality no matter how experienced they are with the device. It remains tactile if the system is a visual-tactile substitution system, or auditory if the system is a visual-auditory substitution system. 2) Once the subject has learned how to use a substitution device, the character of her experience changes. The subject has learned how to use the device to see. Seeing is thus something we can learn to do by learning how to make use of a prosthetic device. It is a skill in much the same sense as echolocation (i.e. it is something that subjects can learn to do by learning how to use a visual prosthesis). Once the relevant skills have been learned, the subject’s experience switches

Substituting the Senses   669



from the substituting modality (touch or hearing) to the substituted modality (vision).21 3) After training with the device, the subject’s mode of perception changes—it doesn’t continue to have the character of the substituting modality. Now the subject perceives the world in part by means of the prosthesis. However, this way of perceiving the world cannot properly be described as visual. The mode of perception the device enables isn’t that of the substituting modality, but nor is it that of the substituted modality. It is something new.

How can we decide between these three possibilities? There is significant evidence that during training subjects undergo some sort of change in the character of their experience, and possibly in their mode of perception. From the earliest days of experiments with sensory substitution, subjects have reported that after training they cease to notice the proximal stimulation at the skin (in TVSS) and they become ‘directly aware’ of objects and their properties that are detected by the camera (Bach-y-Rita, 1972). Their attention is taken up with the external objects that are the causes of this stimulation. Might this change in the character of experience be understood as a change in mode of perception, in a way that supports either the second or third option above? (See Farina, 2013, for an in-depth analysis of this point.) In a much quoted paper from the 1970s on the experience of tactile vision, Gerard Guarniero describes his own experience of using a TVSS as follows: Very soon after I had learned how to scan, the sensations no longer felt as if they were on my back, and I became less and less aware that vibrating pins were making contact with my skin. By this time objects had come to have a top and a bottom; a right side and a left, but no depth—they existed in an ordered two-dimensional space. (Guarniero, 1974: 104, quoted by Deroy and Auvray, 2014: 6)

Ward and Meijer (2010) document the reports of two long-term users of the vOICe that repeatedly describe experiences that closely resemble vision much more than they do audition. The type of visual experience subjects PF and CC report is indeterminate and lacking in acuity and fine detail, but precise enough to be able to make out edges and contours. Both subjects report that the phenomenal character of their experience has developed over time: Initially, both report their visual experiences induced from the vOICe to be flat, jerky, lacking in detail and monochrome. The next major transitions are smooth movement perception and the perception of depth. Both involve going beyond the information explicitly given in the soundscape, analogous to interpreting depth in a photograph based on perspective, shading and occlusion. (Ward and Meijer, 2010: 499)

We will use the term ‘distal attribution’ to characterize the change in the character of experience that users of substitution devices report over time. By ‘distal attribution’ we mean the ability to attribute the cause of proximal stimulation to some external object. Does distal attribution provide us with grounds for saying that following training, the mode of perception switches from the substituting to the substituted modality? 21  See O’Regan and Noë (2001); Hurley and Noë (2003); and O’Regan (2011) for an account of the senses in terms of sensorimotor skills.

670    Julian Kiverstein, Mirko Farina, and Andy Clark

5  Explaining distal attribution Distal attribution is characterized by a lack of awareness of proximal stimulation. The subject’s attention is instead absorbed by the external objects that are the distal causes of this proximal stimulation. The substitution device comes to function as transparent equipment for the skilled user. When all goes smoothly, the user ceases to notice the ways in which her perception is mediated by the device, she relies on the device to inform her about the world in much the same way as we ordinarily rely on our other senses. Can we adequately account for this aspect of the phenomenology of sensory substitution if we suppose that the user’s perceptual experience remains in the substituting modality? The subject no longer notices the stimulation in the substituting modality. If they did, the device would cease to function as transparent equipment for them. Should we say that although the subject doesn’t notice she is perceiving the world by means of touch or sound, her experience nevertheless remains tactile or auditory in character? Jesse Prinz agrees that when the user is capable of distal attribution the substituting sense has begun to perform a function for them a bit like vision. Nevertheless, Prinz insists that the perceptual experiences of the user don’t feel like seeing (Prinz, 2006). Prinz denies that one sense ‘begins to cause perceptual states that are qualitatively like another sense simply by conveying the same information’ (2006: 4). He suggests that distal attribution should be understood in terms of the substituting modality coming to supply some of the same kinds of information as vision. We often have tactile experiences of objects that we are not in contact with: when we drive we feel the surface of the road beneath the car, when we move towards a fan or a flame we feel the change in temperature without coming into contact with the fan or flame. Prinz suggests that something analogous happens in sensory substitution.22 Prinz’s suggestion is, however, hard to square with the evidence for cross-modal plasticity. When the occipital cortex is activated by tactile or auditory inputs why label this processing tactile rather than amodal? At most Prinz establishes that distal attribution doesn’t entail a change in mode of perception to vision. It doesn’t follow that the mode of perception remains in the substituting modality. It could, for instance, result in a new mode of perception (option 3 above). Moreover, although Prinz asserts that distal attribution could be understood in terms of accessing properties and objects normally perceived through vision by means of touch, he doesn’t explain how this happens. If we are to attempt to settle our question about which mode of perception best characterizes perception with a substitution device, we need to know a little more about the process of learning that leads to distal attribution. Fortunately there is some recent experimental work that may help us to address this question. In an important study from 2005, Malika Auvray and colleagues gave a group of blindfolded subjects the task of using a visual-auditory substitution device without informing them of how the device functioned. They were interested in determining whether subjects would settle on an interpretation of what the device was doing in terms of distal attribution if they knew nothing about how the device was functioning. Subjects were told only that they would hear sounds produced 22 

Block (2003) makes a similar point.

Substituting the Senses   671 by the device and they would be asked to rate various explanations of the sounds. Subjects were allowed to play with the device for fifteen minutes and then were asked to assign scores to seven possible scenarios. In a second part of the study an obstacle was introduced that interrupted the camera feed. They found that subjects rated scenarios involving distal attribution highly when they were able to control either the camera, or were able to indirectly influence their relation to the external object. This allowed subjects to track the systematic ways in which proximal stimulation changed as the subjects varied their relationship to the external object. However, Auvray and colleagues also argue that this tracking ability is not sufficient for distal attribution. Subjects tended to rate more highly the correct scenario and downgrade their rating of the other scenarios in the second session after the obstacle had been introduced. They argue that the experience of the obstacle was also necessary to give the perceiver an understanding that variations in proximal stimulation are causally linked to one and the same distal object. Introducing the obstacle breaks this causal link in a way that seemed to enable the perceiver to grasp that the variations in stimulation brought about by their actions are causally dependent on their relation to a distal object. Distal attribution seems to depend on a subject’s grasp of sensorimotor contingencies or the patterns of dependence that hold between sensory inputs generated by the device and movement. Subjects begin to recognize the relationship between proximal stimulation and its distal causes as they move and discover patterns in sensory stimulation contingent on their movement. The substitution device is a tool the user has to learn how to use, which means learning the effects the tool has on the world, and on the perceiver’s relation to the world. Changes in input are brought about through the perceiver’s own movements of the camera and what changes is the perceiver’s relation to a distal object. As the user becomes more at home with the ways in which her own movements generate changes in proximal stimulation, so the use of the device becomes more intuitive, and distal attribution more automatic (Auvray et al., 2007). Now we know more about how distal attribution works, we will conclude by returning to the question with which we started this section. When the blind person has learned how to use a substitution device, does he also learn how to see, or does his perceptual experience remain in the modality of the substituting sense?

6 Conclusions: Skills, senses, and substitution Sensorimotor theories of perception claim that perception with a substitution device is more like vision than it is like perception in the substituting modality. Thus Hurley and Noë (2003) write: What it is like to see is similar to what it is like to perceive by TVSS because seeing and TVSS-perception are similar ways of exploring the environment: they are governed by similar sensorimotor constraints, draw on similar sensorimotor skills, and are directed toward similar visual properties . . . (Hurley and Noë, 2003: 144–5)

672    Julian Kiverstein, Mirko Farina, and Andy Clark The sensorimotor contingencies that govern perception with a substitution device share much in common with vision. Should we use sensorimotor contingencies as our criteria for distinguishing the senses and conclude that perception with a substitution device is ‘vision-like’? There are currently too many functional differences between perception with a substitution device and vision for such a description to be warranted. The visual acuity of blind users of a visual-tactile device is extremely low for instance averaging 40/860 in a standard ophthalmological study (Sampaio et al., 2001). Carter Collins tested a visual-tactile device in a real world situation and found that the system was quickly overloaded by ‘the wrong kind of tactile information’. He writes ‘the bandwidth of the skin as an information-carrying medium is limited and apparently cannot handle the vast amount of data in a raw television image as complex as a sidewalk scene’ (Collins, 1985: 37).23 Finally there is the worry that substitution devices, so far at least, do not deliver the perception of colour, and reports of depth perception are controversial and rare (but for the latter see Renier et al., 2005). Vision is normally associated with representation of colour, depth, shape, and motion at a distance from our bodies. Taken together these points suggest to us that a description of sensory substitution as being like vision cannot be sustained.24 Should we conclude then that following distal attribution perception remains in the substituting modality? Consider again our blind echolocators: they have learned to navigate using clicks and echoes. Is this hearing or is it something new? We suggest that it is a mode of perception best described in terms of skills we ordinary perceivers lack: a new mode of perception made available by the meta-modality of sensory processing. Existing and near-future technologies of ‘sensory substitution’ are, we conclude, best understood in just this manner. Such technologies deliver new modes of perception that exploit the brain’s plasticity and its rich capacity for meta-modal processing. SSDs are thus mind-enhancing tools (Clark, 2003; Auvray and Myin, 2009) that function as sensory prostheses enabling the visually impaired to acquire skills and capacities they would otherwise lack. What have we learned about the senses through our reflections on sensory substitution? Many things, but we wish to highlight just two. First we’ve shown that the meta-modality of sensory processing has implications for theories that would attempt to distinguish the senses based on neurobiology. The brain has a ‘no discrimination policy’ when it comes to answering questions about what in the world might be causing the changes in its internal states. This is a problem the brain can solve irrespective of the format of the incoming sensory signals. In addition to the question of how to distinguish the senses, philosophers have also been interested in the question of how to count the senses. Are we restricted, as Aristotle claimed, to five senses, corresponding to the gross varieties of peripheral sense-organ? The answer is no. Sensory substitution, we have argued, gives the subject a brand new mode of perception, not fully reducible to that of any existing sense or any combinations of existing senses. 23  Loomis et al. (2012) discuss this problem for what they call ‘general purpose sensory substitution’ in great detail. 24  We set aside the question of whether sensorimotor contingencies provide a basis for distinguishing between the senses. O’Regan (2011) defends a sensorimotor account of the senses, but he agrees with us that the sensorimotor contingencies in perception with a substitution device are currently not sufficiently like those in vision for us to count perception with a substitution device as visual. Whether he is right that sensorimotor contingencies provide us with the basis for distinguishing the senses is a question we must leave for another occasion.

Substituting the Senses   673

References Amedi, A., Raz, N., Pianka, P., Malach, R., and Zohary, E. (2003). ‘Early "visual" cortex activation correlates with superior verbal memory performance in the blind. Nature Neuroscience, 6, 758–766. Amedi, A., Stern, W. M., Camprodon, J. A., Bermpohl, F., Merabet, L., and Rotman, S. (2007). ‘Shape conveyed by visual-to-auditory sensory substitution activates the lateral occipital complex’. Nature Neuroscience, 10(6), 687–689. Arno, P., Vanlierde, A., Streel, E., Wanet-Defalque, M. C., Sanabria-Bohorquez, S., and Veraart, C. (2001). ‘Auditory substitution of vision: pattern recognition by the blind’. Applied Cognitive Psychology, 15, 509–519. Arnoldussen, A. and Fletcher, D. C. (2012). ‘Visual perception for the blind: The BrainPort vision device’. Retinal Physician, 9(1), 32–34. Auvray, M. and Myin, E. (2009). ‘Perception with compensatory devices. From sensory substitution to sensorimotor extension’. Cognitive Science, 33, 1036–1058. Auvray M., Hanneton S., Lenay C., and O’Regan J. K. (2005). ‘There is something out there: Distal attribution in sensory substitution, twenty years later’. Journal of Integrative Neuroscience, 4, 505–21. Auvray, M., Hanneton, S., and O’Regan, J. K. (2007). ‘Learning to perceive with a visuo-auditory substitution system: Localization and object recognition with The Voice’. Perception, 36, 416–430. Bach-y-Rita, P. (1972). Brain Mechanisms in Sensory Substitution. New York: Academic Press. Bach-y-Rita, P., Collins, C. C., Saunders, F. A., White, B., and Scadden, L. (1969). ‘Vision substitution by tactile image projection’. Nature, 221, 963–964. Bach-y-Rita, P. and Kercel, S. W. (2003). ‘Sensory substitution and the human-machine interface’. Trends in Cognitive Sciences, 7, 541–546. Block, N. (1978). ‘Troubles with functionalism’. Minnesota Studies in the Philosophy of Science, 9, 261–325. Block, N. (2003). ‘Tactile sensation via spatial perception’. Trends in Cognitive Sciences, 7, 285–286. Bubic A., Striem-Amit E., and Amedi, A. (2010). ‘Large-scale brain plasticity following blindness and the use of sensory substitution devices’. In M. J. Naumer and J, Kaiser (eds), Multisensory Object Perception in the Primate Brain (pp. 351–380). Heidelberg: Springer. Capelle, C., Trullemans, C., Arno, P., and Veraart, C., (1998). ‘A real-time experimental prototype for enhancement of vision rehabilitation using auditory substitution’. IEEE Transaction on Biomedical Engineering, 45, 1279–1293. Clark, A. (2003). Natural Born Cyborgs: Minds, technologies, and the future of human intelligence. New York: Oxford University Press. Clark, A. (2012). ‘Whatever next?: Predictive brains, situated agents, and the future of cognitive science’. Behavioural and Brain Sciences, 36(3), 181–204. Cohen, L. G., Celnik, P., Pascual-Leone, A., Corwell, B., Falz, L., Dambrosia, J., et al. (1997). ‘Functional relevance of cross-modal plasticity in blind humans’. Nature, 389, 180–183. Collins, C. C. (1985). ‘On mobility aids for the blind’. In D. Warren and E. R. Strelow (eds), Electronic Spatial Sensing for the Blind: Contributions from perception, rehabilitation, and computer vision (pp. 35–64). Dordrecht: Martinus Nijhoff Publications.

674    Julian Kiverstein, Mirko Farina, and Andy Clark Cronly-Dillon, J., Persaud, K., and Blore, F. (2000). ‘Blind subjects construct conscious mental images of visual scenes encoded in musical form’. Proceedings of Royal Society of London, B 267, 2231–2238. Danilov, Y. and Tyler, M. (2005). ‘BrainPort: An alternative input to the brain’. Journal of Integrative Neuroscience 4, 537–550. Deroy, O. and Auvray, M. (2014). ‘Quasi-vision: the sensory substitution dilemma’. In S. Biggs, M. Matthen, and D. Stokes (eds), The Senses Volume. Oxford: Oxford University Press. Descartes, R. (1637/1985). Philosophical Writings, trans. J. Cottingham, R. Stoothoff, and D. Murdoch. Cambridge: Cambridge University Press. Farina, M. (2013). ‘Neither Touch nor Vision: sensory substitution as artificial synaesthesia?’ Biology and Philosophy, 28(4), 639–655. Friston, K. (2005). ‘A theory of cortical responses’. Philosophical Transactions of the Royal Society, London B Biological Sciences, 29, 360(1456), 815–836. Friston, K. (2010). ‘The free-energy principle: a unified brain theory?’ Nature Reviews Neurosciences, 11(2), 127–138. Guarniero, G. (1974). ‘Experience of tactile vision’. Perception, 3, 101–104. Hurley, S. and Noë, A. (2003). ‘Neural plasticity and consciousness’. Biology and Philosophy, 18, 131–168. Huttenlocher, P. R. (2002). Neural Plasticity: The effects of the environment on the development of the cerebral cortex. Cambridge, MA: Harvard University Press. Keeley, B. (2002). ‘Making sense of the senses: Individuating modalities in humans and other animals’. The Journal of Philosophy, 99, 5–28. Keeley, B. (2009). ‘The role of neurobiology in differentiating the senses’. In J. Bickle (ed.), The Oxford Handbook of Philosophy and Neuroscience (pp. 226–250). Oxford: Oxford University Press. Lee, T. S. and Mumford, D. (2003). ‘Hierarchical Bayesian inference in the visual cortex’. Journal of Optical Society of America, A. 20(7), 1434–1448. Loomis, J. (1990). ‘A model of character recognition and legibility’. Journal of Experimental Psychology: Human Perception and Performance, 16(1), 106–120. Loomis, J., Klatzky, R. L., and Giudice, N. A. (2012). ‘Sensory substitution of vision: importance of perceptual and cognitive processing’. In R. Manduchi and S. Kurniawan (eds), Assistive Technology for Blindness and Low Vision. Boca-Raton, FL: Taylor & Francis/CRC Press. Lomber, S. G., Meredith, A. M., and Kral, A. (2010). ‘Cross-modal plasticity in specific auditory cortices underlies visual compensations in the deaf’. Nature Neuroscience, 13, 1421–1427. Maguire, E. A., Gadian, D. G., Johnsrude, I. S., Good, C. D., Ashburner, J., Frackowiak, R. S. J., and Frith, C. D. (2000). ‘Navigation-related structural change in the hippocampi of taxi drivers’. Proceedings of the National Academy of Science USA, 97, 4398–4403. Meijer, P. B. L. (1992). ‘An experimental system for auditory image representations’. IEEE Transactions on Biomedical Engineering, 39, 112–121. Morgan, M. J. (1977). Molyneux’s Question. Vision, touch and the philosophy of Perception. Cambridge: Cambridge University Press. Müller, J. (1843/2003). Müller’s Elements of Physiology (trans. W. Baly, 4 vols). Bristol: Thoemmes Press. Nudds, M. (2004). ‘The significance of the senses’. Proceedings of the Aristotelian Society, 104(1), 31–51.

Substituting the Senses   675 O’Regan, J. K. (2011). Why Red Doesn’t Sound Like a Bell: Explaining the Feel of Consciousness. Oxford: Oxford University Press. O’Regan, J. K. and Noë, A. (2001). ‘A sensorimotor account of vision and visual consciousness’. Behavioral and Brain Sciences, 24, 939–973. Pascual-Leone, A. and Hamilton, R. (2001). ‘The metamodal organization of the brain’. Progress in Brain Research, 134, 427–445. Pascual-Leone, A., Cammarota, A., Wassermann, E. M., Brasil-Neto, J. P., Cohen, L. G., and Hallett, M. (1993). ‘Modulation of motor cortical outputs to the reading hand of braille readers’. Annals of Neurology, 34, 33–37. Prinz, J. (2006). ‘Putting the brakes on enactive perception’. Psyche, 12, 1–19. Ptito, M., Moesgaard, S. M., Gjedde, A., and Kupers, R. (2005). ‘Cross-modal plasticity revealed by electrotactile stimulation of the tongue in the congenitally blind’. Brain, 128, 606–614. Rao, R. and Ballard, D. (1999). ‘Predictive coding in the visual cortex: A functional interpretation of some extra-classical receptive-field effects’. Nature Neuroscience, 2(1), 79. Renier, L., Collignon, O., Poirier, C., Tranduy, D., Vanlierde, A., Bol, A., Veraart, C., and De Volder, A.G. (2005). ‘Cross-modal activation of visual cortex during depth perception using auditory substitution of vision’. Neuroimage, 26, 573–580. Ricciardi, E., Vanello, N., Sani, L., Gentili, C., Scilingo, E. P., Landini, L., Guazzelli, M., Bicchi, A., Haxby, J. V., and Pietrini, P., (2007). ‘The effect of visual experience on the development of functional architecture in hMT+’. Cerebral Cortex, 17, 2933–2939. Roxbee-Cox, J. M. (1970). ‘Distinguishing the Senses’. Mind, New Series, 79(316), 530–550. Sadato, N., Pascual-Leone, A., Grafman, J., Ibanez, V., Deiber, M. P., Dold, G., and Hallett, M. (1996). ‘Activation of the primary visual cortex by Braille reading in blind subjects’. Nature, 380, 526–528. Sadato, N., Okada, T., Kubota, K., and Yonekura, Y. (2004). ‘Tactile discrimination activates the visual cortex of the recently blind naive to Braille: a functional magnetic resonance imaging study in humans’. Neuroscience Letters, 359, 49–52. Sampaio, E., Maris, S., and Bach-y-Rita, P. (2001). ‘Brain plasticity: "visual" acuity of blind persons via the tongue’. Brain Research, 908(2), 204–207. Schürmann, M., Caetano, G., Hlushchuk, Y., Jousmaki, V., and Hari, R. (2006). ‘Touch activates human auditory cortex’. Neuroimage, 30, 1325–1331. Teng, S., Puri, A., and Whitney, D. (2012). ‘Ultrafine spatial acuity of blind expert human echolocators’. Experimental Brain Research, 216(4), 483–488. Thaler, L., et al. (2011). ‘Neural correlates of natural human echolocation in early and late blind echolocation experts’. PLoS ONE 6.5, e20162. doi:10.1371/journal.pone.0020162. Tyler, M. E., Danilov, Y., and Bach-y-Rita, P. (2003). ‘Closing an open-loop control system: Vestibular substitution through the tongue’. Journal of Integrative Neuroscience, 2, 1–6. Ward, J. and Meijer, P. (2010). ‘Visual experiences in the blind induced by an auditory sensory substitution device’. Consciousness and Cognition, 19, 492–500. White, B. W., Saunders, F. A., Scadden, L., Bach-y-Rita, P., and Collins, C. C. (1970). ‘Seeing with the skin’. Perception and Psychophysic, 7, 23–27. Wolbers, T., Zahorik, P., and Giudice, N. A. (2011). ‘Decoding the direction of auditory motion in blind humans’. Current Biology, 2111, 984–989.

Pa rt  V I

F R A M E WOR K S FOR  PE RC E P T ION

Chapter 36

Simil a r it y  Spaces Diana Raffman

1 Introduction No relationship is of greater significance in perceptual processing than that of perceptual similarity. Visual stimuli appear (‘look’) the same or different in hue or shape or size, auditory stimuli appear (‘sound’) the same or different in pitch or loudness or distance away, gustatory stimuli appear (‘taste’) the same or different in sweetness or hotness or richness, and so forth. Virtually all philosophers and scientists studying the mind take perceptual similarity to be, in one way or another, the foundation of the perceptual types—blue, round, loud, loudest, less than ten feet away—into which we classify the stimuli we perceive. To that extent, apprehension of perceptual similarity determines how we experience the stimuli in our environment, how we explain and predict their behaviour, and how we behave toward them. Mobilization of perceptual categories enables us to reason inductively about the world around us and, in general, to learn.1 Hence our perception of similarity must be key to our survival. Mohan Matthen observes: The senses are essentially classificatory systems. They assign stimuli to classes, order these classes in similarity relations, and provide sensory consciousness with awareness of the results of this activity. Thus, sensory consciousness is articulated in terms of class membership, in the subsumption of individual stimuli under classes, and similarity relations. (2005: 150)

Central to perceptual similarity are a variety of ordering relations: one object will look bluer or larger, or darker or brighter, than another, and one tone will be louder than another, or sound more stable or more tonally centred in a given musical key. Often these orderings are conceived as delineating an abstract geometrical space in which distance represents degree of similarity on a given dimension (hue, stability, relative importance, etc.). In other words, a perceptual space is an abstract space in which relative similarity

1  Indeed,

some theorists contend that perceptual representations play a greater role in conceptual processing than is commonly appreciated; see for example Smith and Heise (1992) and Goldstone and Barsalou (1996) for discussion.

680   Diana Raffman and difference among perceived stimulus properties are represented by relative distances. Representation of perceptual values in a similarity space enables us to remember and hence recognize new instances of them, and also to associate them with useful non-perceptual information. Some such associations are probably wired in (e.g. that of increasing loudness of a sound with increasing proximity of the sound source), while others are doubtless learned (e.g. that of the orange of carrots and cantaloupe with beta carotene). As we’ll see, perceptual similarity spaces are often organized hierarchically: for example, blue and green are determinates of the determinable hue, while indigo and ultramarine are in turn determinates of the determinable blue. Thus hues can be represented as a hierarchy in which each level is more inclusive, hence more abstract, than the ones below it. In particular, within each level, values (hues) within the same determinate category are—perceived as—more similar than values in different determinate categories. Thus different shades of indigo are more similar than any indigo is to any ultramarine. Similarity is sometimes conceived in a broader way. For example, certain perceptual relationships among the pitches, chords, and keys of tonal music can be represented in a hierarchical space defined by relative importance or ‘stability’. The tonic pitch (‘do’) and tonic triad are heard as the most important in a key, the dominant (‘sol’) and dominant triad as the second most important, the mediant (‘mi’) and median triad third, and so forth. Pitches and chords that are relatively closer together in this hierarchy are perceived as more closely related, hence in a broad sense more similar in a given key, than pitches and chords that are farther apart. So, for example, the tonic and dominant pitches and chords in a key are more closely related, and to that extent more similar, than either is to the subdominant (‘Fa’). Tillman et al. (2000) write that an important feature of Western musical grammar is that tones and chords have different structural functions within a key. According to Meyer (1956), ‘In the major mode in Western music the tonic tone is the tone of ultimate rest toward which all other tones tend to move. On the next higher level the third and fifth of the scale, though active melodic tones relative to the tonic, join the tonic as structural tones; and all the other tones, whether diatonic or chromatic, tend toward one of these’ (pp. 214–215). These differences in musical functions create within-key hierarchies (2000: 886).

Also, keys themselves stand in relations of similarity and difference, which can be represented geometrically. Again, Tillman et al.: Some keys share numerous chords and tones. For example, the C-major key shares four chords and six tones with the G-major key, two chords and five tones with the D-major key, and only one tone with the F-major key. Keys sharing chords or tones are said to be harmonically related. The strength of these harmonic relationships depends on the number of shared chords or tones. In music theory, keys are conceived spatially as a circle, referred to as the [circle] of fifths. The number of steps separating two keys on this circle (whatever the direction of the rotation) defines their harmonic distance (2000: 886).

Hierarchical organization is thought to facilitate learning and memory of perceptual information. Similarity spaces are often structured hierarchically. The following discussion is divided into three parts. The first part describes several specific models of visual and auditory similarity spaces—specifically, colour and pitch spaces. The second part discusses some of the interesting and often surprising ways in

Similarity Spaces   681 which perceptual systems distort or ‘warp’ their similarity spaces. It turns out that, sometimes, in order to serve their adaptive functions, perceptual systems must in effect misrepresent the relationships among stimuli in a certain domain. We will see several vivid examples of this. Lastly, I will consider the implications of some psychological studies of hue perception for a philosophical problem about the individuation of determinate shades of colour.

2  Models of similarity spaces: Colour The structure of colour space has received a great deal of attention from both scientists and philosophers.2 The overarching goal of the part of this research that interests us here is to correctly represent perceived similarities and differences among all of the colours we can see. Models of colour space typically have at least these three desiderata: (1) to isolate a set of fundamental magnitudes or dimensions (e.g. hue, brightness, and saturation, or ‘primary’ hues red, blue, and yellow) in terms of which any humanly perceivable colour can be analysed and identified, (2) to assign to every colour a location in a geometrical space defined by those dimensions, and (3) to determine the physical stimulus values or ranges of values associated with each colour in the space and, thereby, to discover what relationships exist between physical stimulus properties and our perceptual responses to them.3 Stimuli and responses are not always related systematically, but often some more or less reliable correlations can be established in terms of the responses of an idealized ‘standard observer’ under ‘standard conditions’—viz., a normally sighted person viewing stimuli of a certain size against a dark achromatic background under North Daylight, illuminated at 90 degrees, viewed at 45 degrees, etc. (I will say more about the lack of systematic correlation between stimulus properties and perceptual responses in section 4 of this chapter.) Many models of colour space have been developed, but no single one has emerged as superior. To some extent, different models are useful for different purposes. Here I can discuss only a few of the most interesting and important ones. The effort to construct a geometric map of colour space has a venerable history tracing back at least to Newton’s colour circle in his Opticks (1704). The visible spectrum itself is simply a series of single wavelengths of light that can be seen by the unaided human eye, extending from about 400 nanometres (seen as blue violet) to about 700 nm (seen as orange red). Newton realized that the two ends of the spectrum, blue violet and orange red, could be connected perceptually in a circle by the addition of three colours—red violet, magenta, and pure or unique red; but these do not occur in the visible spectrum (Figure 36.1).4 Interestingly, he discovered that these extraspectral hues could be produced by overlapping (‘mixing’) the blue violet and orange red spectral lights. This circular 2 

For sustained discussion of colour perception, see Akins and Hahn, Chapter 22, this volume. model is sought for commercial purposes as well as purely intellectual ones. Shared, stable standards for identifying colours are needed for gemstone evaluation, dye matching, and paint manufacturing, among other things. 4  The spectrum contains only monochromatic light, viz., light of a single wavelength. Newton’s three extra colours are produced by adding two or more wavelengths together. In particular, all spectral reds 3  A  reliable

682   Diana Raffman

Fig. 36.1 Newton’s colour circle (1704). The pulled-out section contains three extra, non-spectral colours: red violet, magenta, and unique red. The seven upper-case letters around the outer rim of the circle refer to the seven pitches of the C Major scale. Newton’s belief in an important relationship between hue and pitch led him to posit the seven primary hues whose names appear in the diagram: violet, red, orange, yellow, green, blue, and indigo. (Bruce MacEvoy, 2009, )

geometry enabled him to represent the fact that red (~700 nm) looks more similar to blue (~400 nm) than to green (~500 nm), even though the wavelengths of red and green light are closer together in the spectrum. Intuitively, there is nothing circular about the spectrum; the circularity is strictly perceptual. In 1874, with the new science of psychophysics on solid ground in the work of Weber, Fechner, and Helmholtz among others, Ewald Hering (1892) postulated, correctly, that human colour perception was structured by three opponencies: red vs. green, blue vs. yellow, and light vs. dark.5 (This structure was later confirmed in a famous experiment by Hurvich and Jameson (1957), using a hue cancellation task; see section 4.) contain a small amount of yellow, thus requiring the addition of a small amount of yellow-cancelling blue light to obtain unique red. (Unique red is pure red containing no blue or yellow, unique blue is pure blue containing no green or red, and similarly for yellow and green.) 5  Hering mistakenly conceived of lightness and darkness as the colours white and black, which are not opponents; black is only a surface or pigment colour, not a colour of light, and white and black pigments mixed together produce shades of grey. In fact, the light vs. dark (or brilliant vs. dim) parameter is a genuine, though achromatic, opponency.

Similarity Spaces   683 Interestingly, Hering based his theory on largely phenomenological considerations, such as the fact that we can experience a reddish yellow or reddish blue, but not a reddish green, and the fact that yellow seems (looks) as basic as red, green, or blue. However, using the technique of multi-dimensional scaling, Roger Shepard (1962) confirmed experimentally both the perceptual ordering of spectral hues around a circle and the red/green and blue/yellow opponencies. Shepard had subjects rate the similarity between members of each pair among fourteen colour chips matching spectral hues. (He excluded the extraspectral red violet, magenta, and unique red so as not to bias his subjects toward a circular geometry.) When fed into a multi-dimensional scaling program, his data produced the map in Figure 36.2, in which similarity is represented by spatial distance. The hue opponencies are evinced in the locations of opponent hues more or less diametrically opposite one another. In 1857, Grassmann had proposed that an adequate description of a colour (as opposed to just a hue) must specify its hue, brightness (lightness or value), and saturation (chroma). This view remains standard today, and most modern colour maps plot the locations of colours along axes assigned to these psychophysical attributes in a three-dimensional space (Figure 36.3). Since the 1920s, one of the most widely used models of colour space has been the one developed by the artist and educator Albert Munsell (1929). Munsell’s goal was to provide a ‘rational way to describe colour’ that would employ decimal notation instead of colour names, which he thought were misleading. To this end, he constructed a space in which the colours were meant to be perceptually equally spaced along the dimensions of hue, brightness, and saturation. In earlier work, Munsell had proposed a spherical colour space, but he discovered that if the colours were to be separated by perceptually equal steps, the space would have to be geometrically irregular. Just for example, there are more perceptually equal hue steps between unique red and unique blue than between unique red and unique yellow. Figure 36.4 shows an approximation of Munsell colour space after several revisions by the Optical Society of America (OSA) (Newhall et al., 1943).

Gr

Yell

537

ow

555

584

504

600

II 490

610

Blue

I 472 465

628 651 674

445

Red

n ee

434 Violet

Fig.  36.2 Shepard’s hue map (1962:  236), based on pairwise similarities among fourteen Munsell  chips.

684   Diana Raffman

Fig.  36.3  The standard three-dimensional colour  space. © Michael Horvath, 2009

Commons:  GNU_Free_Documentation_License_1.2

Fig. 36.4 Munsell colour solid. Munsell colour space is defined in terms of three dimensions: hue, value, and chroma. © Color Planning Center Inc.

Another well-known framework for representing colour space, the Swedish Natural Colour System (NCS, introduced officially in 1979) is based on Hering’s opponency theory. This model (Figure 36.5) is intended primarily to enable easy verbal communication about colours. Here the locations of colours are determined at least in part by averaging over the judgements of many test subjects in a magnitude estimation task. These judgements are subjects’ estimates of the percentages of the four unique hues (pure red, yellow,

Similarity Spaces   685

Fig.  36.5  A  portion of the green region of NCS colour space. NCS space is defined in terms of six elementary colours:  white (top), black (bottom), and from left, yellow, red, blue, and  green. (NCS—Natural Colour System®© property of and used on licence from NCS Colour AB, Stockholm 2014. References to NCS®© in this publication are used with permission from NCS Colour  AB.)

green, blue), and also black and white, that are contained in non-unique test stimuli. Other approaches, such as the series of models developed by the Commission Internationale de l’Éclairage (CIE) starting in 1976, are based at least in part on the cone excitation ratios produced by the corresponding stimuli. Different ways of measuring similarities among colours (magnitude estimation, similarity ratings, cone excitation ratios, etc.) yield different representations of colour space. Every colour model proposed thus far suffers from certain shortcomings as a representation of perceptual space. For example, the increments between neighbouring values in the Munsell map turn out not to be perceptually uniform, and to the degree that the increments are uniform, they are uniform only within single dimensions (hue, brightness, saturation); hence it doesn’t provide an ordering of colours. (Subsequent efforts by the OSA to construct a genuinely uniform colour space led to the conclusion that it couldn’t be done in three dimensions.) The NCS model misrepresents the brightness (brilliance, dimness) of colours: whereas spectral colours vary widely in brightness—for example, monochromatic yellows are brighter than monochromatic blues—the NCS model locates all of them at the same level on its vertical white/black dimension. These are just some examples. In general, any three-dimensional model must involve an extreme, distorting simplification of the phenomena it represents. First of all, probably no single geometry can accommodate all of the different ways of measuring colour similarity and difference. But in addition, the three-dimensional models must abstract from many nonlinear relationships obtaining between physical stimulus properties and perceptual responses. Among other things, just noticeable differences (jnds) at the extremes of a perceptual dimension are larger than jnds in the middle (cf. section 4); the value perceived on one dimension may be affected by values perceived on other dimensions (e.g. the hue of a stimulus may

686   Diana Raffman vary with its brightness and saturation), and the ways in which values on the different dimensions influence each other may not be uniform across all hue categories. Also, contextual factors such as stimulus size, visual surround, lighting conditions, contrast effects, and the state of the subject’s visual system play a role in determining how a stimulus looks.6 Cognitive factors may also enter in; for instance, a picture of a banana may be rated as looking more yellow than an abstract shape cut from the same uniformly coloured piece of yellow paper.

3  Models of similarity spaces: Pitch With the emergence of the psychology of music as a discipline starting around 1970, considerable attention has been paid to mapping the pitch space of tonal music, viz., music that has a tonal centre or key. The quasi-syntactic character of tonal music is unusual (perhaps unique) among non-linguistic perceptual stimuli, and as such it provides a rich source of insight into the structure of human perception and cognition. Here the perceptual space is defined primarily by hierarchies of relative importance or stability in a scale or key or in a specific musical work. Our mental representations of these structures are thought to underlie, or constitute, what could be called our understanding of tonal music—for example, our recognition of wrong notes, our surprise at deceptive cadences, our amusement at a dominant chord left unresolved, our recognition of a return to the tonic.7 In speaking just now of pitch perception I meant ‘pitch’ in the broad sense in which the dimension of pitch is opposed to (e.g.) the temporal dimensions of metre and rhythm or the dimension of loudness. Within the domain of pitch broadly conceived are the three defining elements of tonal pitch space: pitches, chords, and keys, where ‘pitch’ is now construed more narrowly as a count noun, as when we say that intervals contain two pitches; a chord contains three or more pitches, and a key is a region of pitch space delineated by the seven pitches of a diatonic scale (do, re, mi, fa, sol, la, ti). Like hue space, pitch space is at bottom just a straight line from the lowest frequency we can hear (about 20 hertz) to the highest (about 20,000 hertz). To model the pitch space activated in perception of tonal music, however, we must incorporate the relation of octave equivalence, viz., the apparent sameness (type-identity) of pitch or chroma obtaining between pitches that are an octave apart. In that case pitch space is modelled as a circle consisting of twelve pitch classes (Figure 36.6). (We can think of the pitch circle as constructed in something like the way Newton constructed his hue circle, viz., by taking a straight line representing the twelve pitches in an octave and bringing its opposite ends together so that they overlap. The overlap is the octave equivalence.) Of course this pitch circle contains very little information about the myriad relationships among the twelve pitches in an octave. Roger Shepard (1982), among others, sought to capture more of this relational information by modelling pitch space as a double helix based on the circle of fifths; however, this structure was built on a pair of whole-tone scales, which 6  The series of models developed by the CIE have begun to try to take a greater range of these contextual factors into account; see, for example, Hunt (2004) and Fairchild (2005). 7  For further discussion, see Charles Nussbaum, Chapter 26, this volume, on music perception.

Similarity Spaces   687

B

C

C# D

A#

D#

A

E

G# G

F#

F

Fig. 36.6  Circle of twelve pitch classes (Deutsch, 2013: 299). play little if any role in tonal music. (Indeed, whole-tone scales tend to weaken the sense of a tonal centre.) Subsequently a variety of approaches, including statistical learning theories, music-theoretic analyses, and probe-tone and multi-dimensional scaling techniques, among others, have produced richer and more plausible models. Just for example, Carol Krumhansl applied multi-dimensional scaling techniques to data from an experimental probe-tone task8 and achieved the results shown in Figure 36.7: pitch class distance relationships within a scale or key (here, C Major) in Figure 36.7(a), chordal relationships within a key in Figure 36.7(b), and relationships among the twelve major and twelve minor keys in Figure 36.7(c). The key relationships, structured both by the circle of fifths and the relative and parallel relationships between the major and minor keys, can be represented equivalently as points on the surface of a torus or doughnut shape. (Notice how much more information about pitch relations is captured by Krumhansl’s conical space than by the simple pitch circle. Among other things, the pitches of the tonic triad—C, E, G—cluster near the vertex of the cone, indicating that they are the most important, most stable pitches in the key of C Major.)

4  Warpings of perceptual space The geometric nature of the models described above may give the mistaken impression that perceptual spaces are more or less regular—evenly spaced, uniformly divided, etc.— and that perceptual values are more or less systematically related to physical stimulus characteristics. Stevan Harnad writes: The default assumption is that [psychological similarity] spaces have a fixed dimensional structure and that each dimension has a fixed and generally linear metric. Implicit in this 8  In a probe-tone task, subjects hear a stimulus (e.g. a chord or fragment of a scale or melody) followed by a short silence and then a single tone. They are asked to judge how well the probe tone ‘fits’ with the preceding stimulus.

688   Diana Raffman (a)

(b) F#

D#

C#

III

A# C# F

VI

A

D E C

IV I

B

G

V

VII

C'

II

(c) f#

db/c#

d a

e

g#

f

a

c#

g

f#

d

ab

c

eb

g

b

f#

db/c#

f

c

e

b

eb/d#

bb

eb/d#

bb d

bb

Fig. 36.7  (a) Krumhansl (1979: 357) pitch class space for key of C Major. (b) Krumhansl et al. chord space within a key. (1982: 32). (c) Two-dimensional representation of relationships among the twenty-four major and minor keys (Krumhansl and Kessler, 1982). If the top and bottom edges of the rectangle are brought together, and the left- and right-hand edges, the result is a doughnut shape or torus. view is the idea that any given object has a determinate location in this space and that its proximity to other items, including prototypes or other summary central tendencies, can be calculated in a straightforward fashion. The salience of various dimensions may shift as a function of category training, but the space itself is assumed to be an unchanging constant (1998: 732).

In fact our similarity spaces are somewhat irregular and unpredictable, but as we’ll see, the irregularity often serves an adaptive purpose, and it must be accommodated by any adequate philosophical or psychological theory of perception. ‘Warpings’ or distortions of perceptual space are effected in both discrimination (same/ different judgements) and categorization (type-identification); in other words, both our jnd spaces and our category spaces are warped at various places. Perhaps the most familiar warping of jnd space occurs in response compression, cited briefly above, in which jnds are larger at extremes of a perceptual stimulus range—for example, at the high and low ends of the pitch and brightness ranges—than in the middle. We have more difficulty hearing

Similarity Spaces   689 the difference in pitch between two very high frequencies or two very low frequencies than between two frequencies in the middle of the audible range; and more difficulty seeing the difference in brightness between two very brilliant lights or two very dim lights than between lights in the middle of our brightness range. These asymmetries may be biologically adaptive in that our discriminations needn’t be sharpest ‘at the edges’, as one might put it; rather, they must be at their most sensitive in the regions of perceptual space ‘where . . . stimuli are most densely concentrated’ (MacLeod, 2003). An especially interesting type of warping occurs in categorical perception. In the ideal, perception is categorical when discrimination is limited by, i.e. is no finer-grained than, categorization—that is, when we cannot discriminate different values within a category. A well-known instance of this phenomenon occurs in phoneme discrimination: in a series of stimuli progressing gradually (acoustically) from ‘pa’ to ‘ba’, speakers cannot distinguish different ‘pa’ sounds or different ‘ba’ sounds. They cannot hear a gradual progression from the one phoneme to the other, even though an acoustic progression is presented; they hear only ‘pa’ and then ‘ba’, separated by a ‘pop’ rather than a gradual change (Liberman et al., 1957). This is categorical perception in the ideal, but the phenomenon is usually thought of as admitting of degrees; we can say that perception is categorical to the extent that it sustains within-category compression and between-category expansion of perceived stimulus difference. For example, hue discrimination is said to be categorical to the extent that subjects are faster and more accurate at discriminating between hues belonging to different hue categories (e.g. red and blue) than they are at discriminating between hues from the same category (e.g. two shades of red), even when the ‘within-category’ stimuli differ physically by the same amount as, or even by more than, the between-category stimuli. In short, discrimination is better between categories than within them. In at least some cases, categorical perception too is thought to be adaptive. Pevtzow and Harnad (1997) argue that it occurs in the service of category learning:  to reliably resolve confusion at the category boundary, where uncertainty is maximal, internal representations of stimuli that are near to or on the wrong side of the category boundary must be ‘moved.’ The movement is manifested as within-category compression and/or between-category separation—whatever is needed to partition similarity space and generate reliable, all-or-none categorization. (190)

Pevtzow and Harnad note that most studies of categorical perception have focused on categories that are either innate or acquired more or less automatically by exposure. But they have found evidence that the phenomenon emerges even where the categories in question are acquired with explicit training—provided they are sufficiently difficult to learn. To show this, Pevtzow and Harnad had subjects make same/different judgements of pairs of visual texture stimuli, some easy and some difficult (Figure 36.8), and found that categorical perception occurred in the difficult cases, where stimuli were easily confused. This result supports the exciting hypothesis that categorical perception occurs in the service of categorisation when the categorisation is neither trivially easy nor impossibly difficult; the magnitude of the separation/compression effect depends on how much the internal representations of the ‘shadows’ cast by the members and nonmembers of categories have to be ‘moved’ in order to get them on the right side of the category

690   Diana Raffman

0%n (50%m)

EASY

20%n (30%m)

10%n (40%m)

HARD

30%n (20%m)

Fig.  36.8  Pevtzow and Harnad (1997:  190)  discrimination stimuli. boundary  . . .  The output of perceptual categorization is a similarity space that has been deformed in various ways to carve out the parts of the world that we need to act upon differentially and call by different names. Once the names are grounded in perceptual ‘chunks’ which have been learned the hard way, through trial and error feedback, those names become available for another form of representation and another means of learning new categories:  Names can be strung together in the form of propositions that define further categories (Harnad 1996, Cangelosi and Harnad, in preparation). This unique way of acquiring categories is what sets us apart from other species (Pevtzow and Harnad, 1997: 194).

A different kind of warping is evident in a striking result obtained by Shepard and Jordan (1984). They constructed an equalized octatonic scale, i.e. a sequence of eight pitches separated by logarithmically equal increments of frequency, spanning an octave. Then they instructed subjects to judge of each successive pair of these pitches (1–2, 2–3, 3–4, etc.) whether the increment between the two pitches was smaller than, larger than, or the same as, the increment between the preceding two. Logarithmic (ratio) equality normally ensures perceptual equality, but in this case, subjects heard the physically equal ratios as varying in size;9 specifically, the 3–4 and 7–8 increments in the equalized scale were heard as being larger than the others. Shepard and Jordan explain this result by proposing that listeners have a stored mental schema or template of a diatonic musical scale, in which the sequence of steps is whole tone (1–2), whole tone (2–3), semitone (3–4), whole tone (4–5), whole tone (5–6), whole tone (6–7), semitone (7–8). Their thought is that the 3–4 and 7–8 steps in the octatonic stimulus, ‘though physically equal to the others, [were] judged larger

9 

In this connection, see Gary Hatfield, Chapter 5, this volume, on nineteenth-century psychology.

Similarity Spaces   691 because [they were] wide in relation to the narrower [semitone] gaps “expected” by the input template’ (1984: 1333).10 What’s especially fascinating about this result is that the diatonic scale is a mere cultural artefact. Among other things, it was created, presumably for artistic purposes, by compressing (tuning slightly flat) the perfect fifth intervals so that they equal 3½ tones. This enables the fifths to fit properly into the octaves,11 thereby permitting modulation from any of the twenty-four major and minor keys to any other. Evidently, although it is an artefact, the structure of the diatonic scale is so deeply etched into the acculturated mind that Shepard and Jordan’s subjects could not hear the physically equal steps as equal. Their pitch space had been warped by it.

5  A philosophical implication: Identity conditions for determinate shades The idea of determinate hues or shades of colour, viz., the finest shades we can discriminate, is generally taken by philosophers to be both uncontroversial, indeed commonsensical, and incoherent. It is taken to be commonsensical because it is commonsensical: every red object is some particular shade of red, indeed every magenta object some particular shade of magenta. It is taken to be incoherent because identity conditions for determinate shades would have to be given in terms of a relation of indiscriminability, so the story goes, and indiscriminability is non-transitive. In a classic discussion, Christopher Peacocke says: It is pretheoretically tempting to suppose that . . . perceived shades s and s’ are identical if and only if s is not discriminably different from s’. The nontransitivity of nondiscriminable difference (“matching”) entails that there is no way of dividing the spectrum into shades that meets that condition. Take an example in which, in respect of colour, x matches y, y matches z, but x does not match z. To conform to the above principle about shades, the shade of y would have to be identical to shades that are distinct from one another (1992: 83).

And Matthen writes: Strictly speaking, the notion of a fully determinate shade is logically defective since, as Peacocke 1987 points out, the relation of sensory indiscriminability is intransitive . . . (2005: 101).

Let me make several points here. Even setting aside doubts about the supposition that indiscriminability is non-transitive,12 why think that determinate shades (hues) can be defined only in terms of indiscriminability? Results of some psychophysical experiments suggest at least one other feasible way to define them—viz., in terms of some form of hue cancellation or magnitude estimation. In a hue cancellation task, the subject adjusts the amounts of coloured light in a target light until she arrives at a unique hue. For example, if the target is originally blue-green, she will add red light to obtain a unique blue, or yellow 10 

The inequality of step sizes is essential to the establishment of a tonal centre. When in tune, i.e. not compressed, a perfect fifth is slightly larger than 3½ tones. 12  For some of the doubts, see e.g., Raffman (2000, 2011, 2012); Graff (2001). 11 

692   Diana Raffman light to obtain a unique green. By measuring the amount of red or yellow she adds, the experimenter then learns the percentages of green or blue that she perceived in the original stimulus; for instance, she might have seen the original blue-green light as containing 70 per cent blue and 30 per cent green. Hue cancellation can be contrasted with the more difficult task of magnitude estimation, in which the subject has to estimate those percentages, simply by inspection of a presented stimulus. It seems plausible, and I suggest, that determinate shades can be identified by their percentages of the four chromatic components red, blue, yellow, and green. In fact we know already that the four unique hues can be identified quite reliably in this way, as 100 per cent red, 100 per cent blue, 100 per cent yellow, and 100 per cent green. The perfectly balanced binary hues—orange, violet, blue-green, yellow-green—contain 50 per cent of each component; red-orange might be 75 per cent red and 25 per cent yellow, and so on. Of course the extent of our ability to identify or categorize colour stimuli in this way cannot be unlimited: for example, probably we cannot recognize as such a shade that is 43 per cent blue and 57 per cent green, as opposed to one that is 44 per cent blue and 56 per cent green—much less 43.5 per cent blue and 56.5 per cent green, etc. But absent independent reason to think that our experiences of non-unique, non-balanced determinate hues are relevantly different from our experiences of the unique hues and balanced binaries, we can plausibly take our experiences of the former to be of a kind with our experiences of the latter, and regard them as similarly type-identifiable at least in principle. (Of course there will be, and need be, no perfect correlation between perceived values and physical stimulus properties. What I’m proposing is a way to individuate and recognize perceived values—phenomenal appearances, if you like—not to discover any such correlation.) If the foregoing proposal is adequate, then as far as the individuation of determinate shades is concerned, it doesn’t matter whether indiscriminability is non-transitive. The behaviour, indeed the coherence, of the indiscriminability relation remains a puzzle, but contrary to received wisdom, that fact needn’t threaten the notion of determinate shades.

References Deutsch, D. (2013). The Psychology of Music (3rd edn). San Diego: Elsevier. Fairchild, M. D. (2005). Color Appearance Models (2nd edn). Chichester: Wiley-IS&T. Goldstone, R. and Barsalou, L. W. (1996). ‘Reuniting perception and conception’. Cognition, 65, 231–262. Graff, D. (2001). ‘Phenomenal continua and the sorites’. Mind, 110(440), 905–935. Grassmann, H. C. (1853). ‘Zur theorie der farbenmischung’. Annals of Physics, 89, 69–84. Hatfield, G. (2015). ‘Perception in philosophy and psychology in the 19th and early 20th centuries’. In M. Matthen (ed.), The Oxford Handbook of the Philosophy of Perception. Oxford: Oxford University Press. Hering, E. (1977). The Theory of Binocular Vision. New York: Plenum Press. Hunt, R. W. G. (2004). The Reproduction of Colour. Chichester: Wiley-IS&T. Hurvich, L. M. and Jameson, D. (1957). ‘An opponent-process theory of color vision’. Psychological Review, 64(6) (Part I), 384–404. Krumhansl, C. L. (1979). ‘The psychological representation of musical pitch in a tonal context’. Cognitive Psychology, 11, 346–374.

Similarity Spaces   693 Krumhansl, C. L., Bharucha, J. J., and Kessler, E. J. (1982). ‘Perceived harmonic structure of chords in three related musical keys’. Journal of Experimental Psychology: Human Perception and Performance, 8(1), 24–36. Krumhansl, C. L. and Kessler, E. J. (1982). ‘Tracing the dynamic changes in perceived tonal organization in a spatial representation of musical keys’. Psychological Review, 89(4), 334–368. Liberman, A. M., Harris, K. S., Hoffman, H. S., and Griffith, B. C. (1957). ‘The discrimination of speech sounds within and across phoneme boundaries’. Journal of Experimental Psychology, 54(5), 358–368. Livingston, K. R., Andrews, J. K., and Harnad, S. (1998). ‘Categorical perceptions effects induced by category learning’. Journal of Experimental Psychology: Learning, Memory, and Cognition, 24(3), 732–753. MacLeod, D. (2003). ‘Colour discrimination, colour constancy and natural scene statistics’. In J. D. Mollon, J. Pokorny, and K. Knoblauch (eds), Normal and Defective Colour Vision. London: Oxford University Press. Matthen, M. (2005). Seeing, Doing, and Knowing: A Philosophical Theory of Sense Perception. Oxford: Oxford University Press. Meyer, L. B. (1956). Emotion and Meaning in Music. Chicago: University of Chicago Press. Munsell, A. (1929). Munsell Book of Color. Boston: Munsell Color Company. Newhall, S. M., Nickerson, D., and Judd, D. B. (1943). ‘Final report of the O.S.A. Committee on the spacing of the Munsell colors’. Journal of the Optical Society of America, 33(7), 385–411. Newton, I. (1721). Opticks. London: William Innys at the West-End of St Paul’s. Notman, L., Sowden, P. T., and Özgen, E. (2005). ‘The nature of learned categorical perception effects: a psychophysical approach’. Cognition, 95, B1–B14. Nussbaum, C. (2015). ‘Music perception’. In M. Matthen (ed.), The Oxford Handbook of Philosophy of Perception. Oxford: Oxford University Press. Peacocke, C. (1992). A Study of Concepts. Cambridge, MA: MIT Press. Pevtzow, R. and Harnad, S. (1997). ‘Warping similarity space in category learning by human subjects: the role of task difficulty’. In M. Ramscar, U. Hahn, E. Cambouropolos, and H. Pain (eds), Proceedings of SimCat 1997: Interdisciplinary Workshop on Similarity and Categorization (pp. 189–195). Department of Artificial Intelligence, Edinburgh University. Raffman, D. (2000). ‘Is perceptual indiscriminability nontransitive?’ Philosophical Topics, 28(1), 153–175. Vagueness, ed. C. Hill. Raffman, D. (2011). ‘Vagueness and cognitive science’. In G. Ronzitti (ed.), Vagueness: A Guide. Berlin: Springer. Raffman, D. (2012). ‘Indiscriminability and phenomenal continua’. In J. Hawthorne (ed.), Philosophical Perspectives 26: Philosophy of Mind. New York: Wiley-Blackwell. Shepard, R. (1962). ‘The analysis of proximities: multidimensional scaling with an unknown distance function, II’. Psychometrika, 27(3), 236. Shepard, R. (1982). ‘Geometrical approximations to the structure of musical pitch’. Psychological Review, 89(4), 305–333. Shepard, R. and Jordan, D. (1984). ‘Auditory illusions demonstrating that tones are assimilated to an internalized musical scale’. Science, 226, 1333–1334. Smith, L. and Heise, D. (1992). ‘Perceptual similarity and conceptual structure’. In B. Burns (ed.), Percepts, Concepts, and Categories (pp. 233–272). Amsterdam: Elsevier. Tillman, B., Bharucha, J., and Bigand, E. (2000). ‘Implicit Learning of tonality: a selforganizing approach’. Psychological Review, 107(4), 885–913.

Chapter 37

Bay esi a n Perceptua l Psychol ogy Michael Rescorla

Bayesian decision theory is a mathematical framework that models reasoning and decision-making under uncertainty. Around 1990, perceptual psychologists began constructing detailed Bayesian models of perception.1 This research program has proved enormously fruitful. As two leading perceptual psychologists put it, ‘Bayesian concepts are transforming perception research by providing a rigorous mathematical framework for representing the physical and statistical properties of the environment, describing the tasks that perceptual systems are trying to perform, and deriving appropriate computational theories of how to perform those tasks’ (Geisler and Kersten, 2002, 508). To understand perception, one must acquire detailed knowledge of Bayesian perceptual psychology. Or so I hope to convince you.

1 Perception as unconscious inference Perception solves an underdetermination problem. The perceptual system estimates environmental conditions, such as the shapes, sizes, colours, and locations of distal objects. It does so based upon proximal stimulations of sensory organs. The proximal stimulations underdetermine their environmental causes. For instance, a convex object under normal lighting generates retinal stimulations ambiguous between at least two possibilities: that the object is convex and that light comes from overhead; or that the object is concave and that light comes from below. Similarly, light reflected from a surface generates retinal stimulations consistent with various colours (e.g. the surface may be red and bathed in daylight, or the surface may be white and bathed in red light). In general, then, retinal input underdetermines possible states of the distal environment. We cannot yet programme a

1  Bayesian perceptual psychology generalizes signal detection theory, which was developed in the 1950s. For comparison of the two frameworks, see Kersten and Schrater (2002, 193–199).

Bayesian Perceptual Psychology    695 computer that solves this underdetermination problem. The perceptual system solves it quickly, effortlessly, automatically, and reliably. How? Helmholtz (1867) proposed that the perceptual system executes an ‘unconscious inference’ from sensory stimulations to hypotheses about the environment. The inference reflects ‘implicit assumptions’ concerning the environment or the interaction between environment and perceiver. For instance, the visual system deploys an ‘implicit assumption’ that light comes from overhead. Helmholtz’s approach, now called constructivism, helps explain two notable phenomena: perceptual constancies and illusions. Perceptual constancies are capacities to represent properties or entities as the same despite large variation in proximal stimulation. To varying degrees, human vision displays constancies for numerous properties, including size, shape, location, colour, depth, and motion. How does the perceptual system achieve constancies? By using ‘implicit assumptions’ to discount variations in proximal stimulation. Colour constancy provides a good illustration. This is the capacity to perceive surface colour as constant despite large variation in viewing conditions, including background illumination. To estimate surface colour, the perceptual system first deploys various ‘implicit assumptions’ (such as that the light source is fairly uniform, or that certain surface colours are likelier than others) to estimate background illumination based upon overall retinal stimulation. The perceptual system then deploys this background illumination estimate so as to estimate a surface’s colour based upon retinal stimulation caused by that surface. As Helmholtz famously put it, the perceptual system ‘discounts the illuminant’. Perceptual constancies are reliable but fallible, as demonstrated by illusions. Consider again the assumption that light comes from overhead. The assumption is correct in normal cases, so it usually supports an inference to an accurate percept. When the assumption fails, the resulting percept is inaccurate. For instance, lighting a concave object from below generates an illusory percept as of a convex object. Constructivists explain the mistaken shape-estimate by isolating its source: the implicit assumption that light comes from overhead. Similarly, a red spotlight directed upon a single object violates the implicit assumption of a fairly uniform illuminant, thereby inducing an illusory colour percept. These examples illustrate constructivism’s template for explaining illusions: isolate an implicit assumption deployed during perceptual inference; show how failure of the assumption can induce an inaccurate percept. Perceptual processes are subpersonal and inaccessible to the thinker. There is no good sense in which the thinker herself, as opposed to her perceptual system, executes perceptual inferences. For instance, a normal perceiver simply sees a surface as having a certain colour. Even if she notices the light spectrum reaching her eye, as a painter might, she cannot access the perceptual system’s inference from retinal stimulations to surface colour.2 The twentieth century produced various rivals to constructivism, including Gibson’s direct perception framework. Gibson (1979) denied that perception involves complex psychological activity, inferential or otherwise. He held that the perceptual system directly ‘picks up’ certain distal properties by ‘resonating’ to them. Gibson’s work yielded many invaluable insights, such as the importance of optic flow, which can be incorporated into constructivism. Viewed as an alternative to constructivism, Gibson’s direct perception

2  On the distinction between the perceiver and her perceptual system, see Burge (2010, 23–24; 2011, 68–69).

696   Michael Rescorla framework has difficulty explaining the vast bulk of constancies and illusions (Fodor and Pylyshyn, 1981). That is why the direct perception framework remains marginal within perceptual psychology. A satisfactory development of constructivism must answer three questions:

a. In what sense does the perceptual system execute ‘inferences’? b. In what sense do the inferences ‘reflect’ various ‘implicit assumptions’? c. In what sense does perceptual inference yield the ‘best’ hypothesis?

Different versions of constructivism answer these questions differently. For instance, some constructivists regard ‘implicit assumptions’ as stored premises fit to participate in unconscious deductive, inductive, or abductive inferences (Rock, 1983, 272–282). Bayesian perceptual psychology develops constructivism in a different direction, as I will now explain.

2 Perception as unconscious statistical inference The perceptual system operates under conditions of uncertainty, stemming from at least three sources: 1. Ambiguity of sensory input, as described above. 2. Noisiness of perceptual organs and neural mechanisms: that is, their vulnerability to corruption by random errors. 3. Potential conflict between sensory modalities (e.g. visual versus auditory cues to an object’s location) or between cues within a modality (e.g. binocular disparity cues to depth versus monocular linear perspective cues to depth). It therefore seems natural to formalize constructivism through Bayesian decision theory, which models decision-making under uncertainty. The core notion underlying Bayesian decision theory is subjective probability. Subjective probabilities reflect psychological facets of the individual or her subsystems, rather than ‘objective’ features of reality. To formalize probabilities, we introduce a hypothesis space H containing various hypotheses h. Each hypothesis h reflects a possible state of the world (e.g. a possible shape of some distal object; or a possible colour of some distal surface; or a possible assignment of distal objects to spatial locations). A probability function p maps each hypothesis h to a real number p(h), reflecting the agent’s subjective probabilities.3 Bayesian decision theory dictates how to update subjective probabilities based on new evidence. Bayes’s Theorem states that: 

p(h | e) ∝ p(e | h) p(h) 3  When the hypothesis space is continuous, p(h) is a probability density function. See below for details. For ease of exposition, I often blur the distinction between probability and probability density.

Bayesian Perceptual Psychology    697 meaning that the left-hand side is proportional to the right-hand side. p(h | e) and p(e | h) are conditional probabilities. For instance, p(e | h) is the probability of e, conditional on h. Bayes’s Rule states that, when one receives evidence e, one should update p(h) by replacing it with p(h | e). To execute Bayes’s Rule, one multiplies the prior probability p(h) by the prior likelihood p(e | h). One then normalizes so that all probabilities sum to 1. Finally, one adopts the resulting posterior probability p(h | e) as a revised probability assignment for h. Thus, the new probability of h is proportional to its original probability, multiplied by the likelihood of evidence e given h.4 Bayesian perceptual psychologists use this framework to model perceptual inference (Knill and Richards, 1996). On a Bayesian approach, the perceptual system entertains hypotheses drawn from a hypothesis space H. The perceptual system assigns prior probabilities to hypotheses h and prior likelihoods to (e, h) pairs, where each e corresponds to some possible sensory input. After receiving input e, the perceptual system reallocates probabilities across the hypothesis space, in rough accord with Bayes’s Rule. To illustrate, consider the extraction of shape from shading (Mamassian, Landy, and Maloney, 2002). Let s reflect possible shapes, θ reflect possible lighting directions, and e reflect possible patterns of retinal illumination. The visual system encodes:  A prior probability p(s), which assigns higher probability to certain distal shapes than others (e.g. it may assign higher probability to convex shapes). A prior probability p(θ), which assigns higher probability to an overhead lighting direction than to alternative lighting directions. A prior likelihood p(e | s, θ), which assigns a higher probability to an (e, s, θ) triplet if distal shape s and lighting direction θ are likely to cause retinal illumination e.

Upon receiving retinal illumination e, the perceptual system redistributes probabilities over shape-estimates, yielding a posterior p(s | e). Depending on the case, the posterior might assign a much higher probability to convexity than concavity. For details, see Stone (2011). Perception normally yields a determinate percept. For instance, one sees an object as having a determinate shape, not a spectrum of more or less probable shapes. Accordingly, Bayesian models explain how the perceptual system selects a single hypothesis h based on the posterior p(h | e). Typical models invoke expected utility maximization. The ‘action’ is selection of h. The utility function, which is task-dependent, reflects the penalty for an incorrect answer. If the utility function has a suitable shape, then expected utility maximization reduces to a much simpler decision rule, such as selecting the mean or the mode of the posterior probability. As another example, Bayesian models of surface colour perception proceed roughly as follows. A  surface has reflectance R(λ), specifying the fraction of incident light that the surface reflects at each wavelength λ.5 The illuminant has spectral power 4  There is an unfortunate tendency among scientists and even some philosophers to conflate Bayes’s Theorem and Bayes’s Rule. The former is an easily provable mathematical theorem. The latter is a prescriptive norm that dictates how to reallocate probabilities in light of new evidence. 5  The models described in this paragraph assume diffusely illuminated flat matte surfaces. To handle other viewing conditions, we must replace R(λ) with a more complicated surface reflectance property, such as a bidirectional reflectance distribution.

698   Michael Rescorla distribution I(λ): the light power at each wavelength. The retina receives light spectrum C(λ) = I(λ) R(λ) from the surface. The visual system seeks to estimate surface reflectance R(λ). This estimation problem is underdetermined, since C(λ) is consistent with numerous I(λ)-R(λ) pairs. Typical Bayesian models posit that two surfaces have the same colour appearance for a perceiver when her perceptual system estimates the same reflectance for each surface. To estimate R(λ), the visual system estimates I(λ). It does so through a Bayesian inference, based upon overall retinal stimulation, that deploys a prior probability over possible illuminants and possible surface reflectances. To a first approximation, the illuminant prior assigns higher probability to illuminants that resemble natural daylight, while the surface prior assigns higher probability to surface reflectances that occur more commonly in the natural environment. This framework can explain both the success and the failure of human colour constancy under various conditions. For details, see Brainard (2009).6 We can schematize a typical Bayesian model through the template depicted in Figure  37.1. Note that this template does not require perception to represent Bayesian norms. There is no evidence that the perceptual system explicitly represents Bayes’s Rule or expected utility maximization. The perceptual system simply proceeds in rough accord with Bayesian norms. A typical Bayesian model dictates a unique outcome given four factors: prior probabilities, prior likelihoods, sensory input, and the utility function.7 In that sense, the model is deterministic. Of course, the model’s generalizations are ceteris paribus. Perceptual malfunction, external interference, or corruption by internal noise can induce exceptions. Most Bayesian models conform roughly to the foregoing template. But some models vary the template. For instance, some models augment the template by incorporating motor efference copy.8 Other models replace expected utility maximization with probability matching, a non-deterministic process whereby the probability that the perceptual system selects some hypothesis matches the posterior probability assigned to that hypothesis (Mamassian, Landy, and Maloney, 2002). One phenomenon sometimes analyzed through non-deterministic Bayesian modelling is multistable perception (such as the Necker cube). During multistable perception, experience fluctuates between distinct percepts, rather than yielding a unique percept. One can construe Bayesian models of perception in two different ways (Kersten and Mamassian, 2009). On the first construal, a Bayesian model describes how an ‘ideal 6  Current models describe perception of surface colour. As Matthen (2005, 176) emphasizes, colour perception also responds to transmitted colour (e.g. stained-glass windows) and coloured light sources. Thus, we should not identify colours with surface reflectance properties. Should we identify colours with other, possibly disjunctive, physical properties? Maybe. But the Bayesian models I am describing do not presuppose a physicalist reduction of colour. One might combine those models with various metaphysical views of colour, such as that colours are dispositions to cause sensations in normal human perceivers, or such as Matthen’s (2005) pluralistic realism. Current Bayesian models assume no particular metaphysics of colour. They simply assume that human surface colour perception involves estimation of surface reflectance, as informed by an estimate of background illumination. 7  Cf. Burge’s ‘Proximality Principle’ (2005). 8 Motor efference copy figures most prominently in Bayesian models of sensorimotor control (Wolpert, 2007).

Bayesian Perceptual Psychology    699 Prior probability

Prior likelihood

Sensory input

p(h)

p(e | h)

e

Bayes’s Rule

Posterior probability p(h | e)

Utility function

Expected Utility Maximization

Hypothesis h

Fig.  37.1  A  template for Bayesian models of perception. observer’ would estimate environmental conditions based upon sensory input. We construct the model only so as compare human performance with an ideal benchmark. On the second construal, a Bayesian model approximately describes actual mental processes. The model seeks to describe, perhaps in an idealized way, how the perceptual system actually transits from sensory input to perceptual estimates. Both construals figure in the scientific literature. I emphasize the second construal. I am discussing Bayesian models as empirical theories of actual human perception. Many Bayesian models are fairly unrealistic. For example, the hypothesis space is often uncountable. In general, Bayesian inference over an uncountable hypothesis space is computationally intractable. So I think that we should regard most Bayesian perceptual models as idealizations, akin to models from physics that postulate frictionless surfaces or infinite wires. Of course, we eventually want less idealized descriptions. However, I see no principled problem here. Artificial Intelligence (AI) offers numerous tools for constructing computationally tractable approximations to idealized Bayesian computation. No doubt we will eventually supplement or replace current perceptual models with computationally tractable approximations. Bayesian perceptual psychology provides detailed answers to the three questions (a), (b), and (c) posed at the end of the previous section:

a. Transitions among perceptual states approximately conform to norms of Bayesian inference. In that sense, the transitions are statistical inferences. b. Bayesian models replace talk about ‘implicit assumptions’ with talk about prior probabilities and likelihoods. The models thereby depart substantially from many earlier versions of constructivism. On Rock’s approach, for example, an ‘implicit

700   Michael Rescorla assumption’ that light comes from overhead corresponds to a single stored premise whose content is that light comes from overhead. Bayesians instead posit a prior assignment of probabilities to possible lighting directions. This prior figures not as a premise but rather as input to Bayesian reallocation of subjective probabilities over shape-estimates. c. The perceptual system produces an estimate that is ‘best’ or ‘optimal’ insofar as it conforms to rational norms of Bayesian decision theory. In this manner, Bayesian models depict numerous perceptual illusions as natural by-products of a near-optimal process that infers environmental conditions from ambiguous sensory stimulations.



Hence, the Bayesian framework converts talk about ‘implicit assumptions’ and ‘unconscious inferences’ into mathematically rigorous, quantitatively precise psychological models. Where do the prior probabilities and prior likelihoods come from? The human visual system evolved over millennia in a fairly stable environment. Accordingly, one might expect certain lawlike or statistical environmental regularities to be ‘encoded in the genes’. Nevertheless, Bayesian perceptual priors do not simply reflect innate programming. For instance, even the ‘light-from-overhead’ prior reflects a complex interplay between nature and nurture. It gathers considerable strength during early childhood (Stone, 2011), and it changes rapidly upon adult exposure to deviant environments (Adams, Graf, and Ernst, 2004). At present, we do not know how genetic endowment and individual experience jointly determine Bayesian priors. Current research mainly tries to identify the priors, not to explain the aetiology of the priors.9 Ultimately, we want detailed theories explaining how Bayesian priors originate and develop. Even lacking such theories, we can cite the priors to explain constancies and illusions. In this connection, I stress that the priors postulated by Bayesian perceptual psychology are not ad hoc. Admittedly, a precise quantitative match usually requires some ‘curve fitting’. Qualitatively, though, the priors typically reflect antecedently motivated claims about lawlike or statistical properties of our environment. It is plausible that the perceptual system acquires these priors through some combination of nature and nurture, even if we do not yet know how.10 How can we legitimately postulate Bayesian priors, lacking a developed theory of their aetiology? Because Bayesian priors generate the unifying predictive power characteristic of good explanation. To illustrate, consider motion perception.11 The visual system can 9 

There are exceptions, such as Knill (2007). In some cases, the priors reflect non-obvious statistical regularities about the environment (Geisler, 2008). In other cases, a satisfying explanation awaits discovery. An example: somewhat mysteriously, the perceptual system assumes that the light source is located overhead and slightly to the left (Mamassian, Landy, and Maloney, 2002). One question in this area concerns informational encapsulation:  to what extent can cognition influence the priors? 11  Cue combination provides another good illustration. The perceptual system typically receives multiple cues, often through different sensory modalities, regarding a single environmental variable. Bayesian perceptual psychology offers a unified framework for explaining diverse cases of intermodal and intramodal sensory fusion: visual and auditory cues to location; visual and proprioceptive cues to limb position; conflicting visual cues to depth; and so on. See Trommershäuser, Körding, and Landy (2011) for an overview. 10 

Bayesian Perceptual Psychology    701 directly measure local retinal image velocities, which underdetermine the distal motions that cause them. The visual system must estimate distal motion based upon local retinal image velocities. It does so fairly well but not perfectly, as illustrated by the fact that low-contrast stimuli appear to move more slowly than high-contrast stimuli. (This may explain why drivers accelerate in the fog—they underestimate relative velocities.) Weiss, Simoncelli, and Adelson (2002) offer a Bayesian motion perception model with two features: The prior probability favours slow distal motions. The visual system treats low-contrast retinal images as less reliable.12 This model explains why vision underestimates velocity under low-contrast conditions: namely, because the slow-motion prior exerts more influence over the velocity-estimate. The model also explains other motion illusions, including the following: a fat rhombus moving horizontally appears to move horizontally, but a thin rhombus seems to move diagonally at low contrasts and horizontally at high contrasts. (Readers can experience this effect at .) Thus, a single Bayesian model explains diverse illusions that otherwise resist unified treatment. Subsequent models have elaborated the Bayesian approach to motion perception in increasingly sophisticated ways (Ernst, 2010). Bayesian perceptual psychology offers illuminating, rigorous explanations for numerous constancies and illusions. It is our best current science of perception. We should carefully consider how it bears upon contemporary philosophy of mind—a task to which I now turn.

3 Estimation and representation A natural view holds that perceptual states are evaluable as accurate or inaccurate. For instance, suppose I perceive a concave object that appears convex due to misleading lighting. It seems natural to say that my percept is inaccurate. To say this, we must ascribe truth, accuracy, or veridicality conditions to the percept. Some philosophers distinguish among ‘truth’, ‘accuracy’, and ‘veridicality’ (Burge, 2010), but I remain neutral on this issue. Call the view that perceptual states have veridicality-conditions representationalism. Burge (2005, 2010, 2011) argues that current perceptual psychology supports representationalism. I will now defend the same conclusion by examining Bayesian models of perception.13 On the Bayesian approach, perceptual inference reallocates probabilities over a hypothesis space and then selects a favoured hypothesis. This favoured hypothesis is incorporated into the final percept, whose accuracy depends upon whether the hypothesis is accurate. 12 

More technically: the prior likelihood p(e | h), considered as a function of h for fixed e, has higher variance when the retinal image e has lower contrast. 13 Burge discusses several Bayesian perceptual models, but he does not discuss their specifically Bayesian features. Bradley (2008) defends representationalism by citing Bayesian models of colour perception.

702   Michael Rescorla To illustrate, consider Bayesian models of shape perception. The perceptual system assigns prior probabilities to estimates of specific distal shapes. After receiving sensory input, perceptual inference revises the probability assignment and selects a favoured estimate of a specific distal shape. The resulting percept incorporates this favoured shape-estimate. The percept may also incorporate various size-estimates, motion-estimates, and so on. Accuracy of the percept depends upon accuracy of the individual estimates. By describing perceptual inference in this way, we type-identify perceptual states representationally. We individuate perceptual states partly through environmental conditions that must obtain for the states to be accurate. What exactly are the accuracy-conditions of percepts? According to Davies (1992), a percept involves something like existential quantification. The percept is accurate when there exist objects with properties represented by the percept. An opposing view, espoused by Burge (2005), holds that perceptual accuracy-conditions are object-dependent. A percept represents environmental particulars, such as physical bodies or events. The percept attributes properties to those particulars. It is accurate only if those particulars have the represented properties. I remain neutral between these two views. I emphasize a shared presupposition underlying both views:  that perceptual states have accuracy-conditions. This presupposition is integral to perceptual psychology. The science seeks to explain how the perceptual system generates a percept that estimates specific environmental conditions. Estimates can be either accurate or inaccurate. Following standard philosophical usage, I say that a mental state has representational content when it has a veridicality-condition. On this usage, perceptual states have representational content. I do not assume a specific theory of representational content. One might gloss perceptual contents as sets of possible worlds, or Russellian propositions, or Fregean senses. There are many other options.14 The key point for us is that the science routinely individuates perceptual states through their representational import. Bayesian models individuate both explananda and explanantia in representational terms. The science explains perceptual states under representational descriptions, and it does so by citing other mental states under representational descriptions. For instance, Bayesian models of shape from shading assume prior probabilities over hypotheses about specific distal shapes and about specific lighting directions. The models articulate generalizations describing how retinal input, combined with these priors, causes a revised probability assignment to hypotheses about specific distal shapes, subsequently inducing a unique estimate of a specific distal shape. The generalizations type-identify perceptual states as estimates of specific distal shapes. Similarly, Bayesian models of surface colour perception type-identify perceptual states as estimates of specific surface reflectances. Thus, the science assigns representation a central role within its explanatory generalizations. The generalizations describe how mental states that bear certain representational relations to the environment combine with sensory input to cause mental states that bear certain representational relations to the environment. In what follows, I  develop my analysis by examining various philosophical theories that either reject representationalism or else downplay the importance of representational content. 14 

For a survey of philosophical approaches to perceptual content, see Siegel (2011).

Bayesian Perceptual Psychology    703

4 The relational view of perception Brewer (2007), Campbell (2010), Martin (2004), and Travis (2004) espouse a relational view of perception. Relationalists eschew all talk about perceptual representation. They treat perceptual states as relations not to representational contents but rather to objects or properties in the perceived environment. For instance, Campbell (2010, 202) holds that ‘the content of visual experience is constituted by the objects and properties in the scene perceived’, rather than by anything resembling an accuracy-condition. He cautions that we should not ‘think of experience itself as already a representational state’ (ibid.). The relational approach is sometimes allied with Gibsonian direct perception, sometimes not. To illustrate, consider two counterfactual situations A and B in which I perceive the same object O, yielding qualitatively indistinguishable percepts PA and PB: In situation A, O is convex and looks convex. In situation B, O is concave but looks convex through misleading lighting. Representational taxonomization type-identifies PA and PB by correlating them with the same accuracy-condition. In particular, both percepts estimate the same distal shape: convexity. In situation A, the estimate is correct. In situation B, the estimate is incorrect. By contrast, Campbell’s relational taxonomization treats PA and PB as type-distinct. Campbell type-identifies the first percept through its relation to a distal property (convexity) to which the second percept is not appropriately related. Bayesian perceptual psychology supports representationalism over relationalism. A core postulate underlying the science is that perception produces an estimate of environmental conditions, where the estimate may be either accurate or inaccurate. Consider Figure 37.1. If we neglect noise, malfunction, and external interference, then a unique percept-type is determined by four factors:  the prior probability, the prior likelihood, proximal sensory input, and the utility function. We may stipulate that all four factors are the same in situations A and B. It follows that percepts PA and PB are type-identical from the perspective of the Bayesian model. In both cases, the final percept incorporates a convexity-estimate. The perceptual system produces a convexity-estimate whether or not the perceived object is convex. (Cf. Burge, 2005, 22–25; 2010, 362–364.) An appropriately modified diagnosis applies to non-deterministic Bayesian models, such as models that replace expected utility maximization with probability matching. For such models, the probability that situation A yields a convexity-estimate equals the probability that situation B yields a convexity-estimate. Thus, explanatory generalizations of Bayesian perceptual psychology enshrine a representational, non-relational taxonomic scheme. The generalizations type-identify percepts by specifying environmental conditions that must obtain for a given percept to be accurate. Campbell (2010) suggests that we can interpret perceptual science in relational terms. This suggestion seems unpromising, because the Bayesian explanation of illusion relies essentially upon non-relational taxonomization. The central idea is that the perceptual system estimates some environmental state, which may or may not obtain. Bayesian modelling seeks to explain the environmental state estimate, regardless of whether the

704   Michael Rescorla estimate is veridical. Contrary to Campbell’s relationalist strictures, the science routinely type-identifies veridical and non-veridical percepts. Of course, there is a difference between the veridical and the non-veridical percept. Perceptual psychologists acknow­ ledge this difference. Yet they also emphasize fundamental representational commonalities between the two percepts. Those commonalities play a key individuative role within Bayesian explanatory generalizations. So a relational, non-representational taxonomic scheme flouts explanatory practice within perceptual psychology. Brewer (2007, 173)  seeks to accommodate illusions inside a relational framework. He concedes that there can be a ‘visually relevant similarity’ between a veridical and a non-veridical percept. He compares: (i) a red surface in daylight; and (ii) a white surface surreptitiously bathed in red light. He acknowledges that the surface in scenario (ii) looks red. He says that ‘this consists in the fact that [the surface] has visually relevant similarities with paradigm red objects: the light reflected from it is like that reflected from such paradigms in normal viewing conditions’ (ibid.). Naturally, I agree that (i) and (ii) emit similar light spectra. However, merely noting this commonality does not capture the fact that both surfaces look red. A surface that emits the same light spectrum under different viewing conditions may not look red. A surface that emits a radically different light spectrum under different viewing conditions can still look red. Thus, we must reject Brewer’s proposed analysis of looks red. In contrast, representationalists can say that a surface looks red when one’s percept represents the surface as red. Brewer’s account omits crucial scientifically relevant commonalities between the two percepts. A key scientifically relevant commonality is that both percepts result from perceptual estimation of a single surface reflectance R(λ). The estimate is correct in (i), incorrect in (ii). We do not capture this key commonality between the percepts simply by noting that (i)  and (ii) emit similar light. The perceptual system can estimate reflectance R(λ) despite large variation in the light spectrum C(λ) emitted by a surface. Moreover, depending on the perceptual system’s estimate of illumination I(λ), it may not estimate R(λ) even when the surface emits the same light spectrum C(λ). Capturing the scientifically relevant commonalities between (i) and (ii) requires us to cite perceptual estimation (and hence perceptual representation) of surface reflectance. Yet relationalists eschew all talk about perceptual representation. There are delicate issues here surrounding the relation between colours and surface reflectances. According to current science, a percept that represents a surface as red is caused by perceptual activity that represents reflectance. But does the final percept itself represent reflectance? There are at least three salient options:

a. The percept represents colour but not reflectance. b. The percept represents reflectance and separately represents colour. c. The percept represents reflectance and thereby represents colour.

The choice between (a), (b), and (c) depends upon other matters, including the metaphysics of colour (cf. note 6). We need not choose among (a)–(c) here. The crucial point is that relationalists must reject all three options. Relationalists do not countenance perceptual representation of colour, reflectance, or any other distal property. In summary, relationalism cannot accommodate a core postulate underlying contemporary perceptual psychology:  that perception produces an estimate of environmental conditions, where the estimate may be either accurate or inaccurate.

Bayesian Perceptual Psychology    705

5 Eliminativism, instrumentalism, and realism Beginning with Quine (1960), various philosophers have argued that intentionality (or representationality) deserves no place in serious scientific discourse. They have argued that we should replace intentional psychology with some alternative framework, such as Skinnerian behaviourism (Quine, 1960) or neuroscience (Churchland, 1981). This eliminativist position concedes that representational locutions are instrumentally useful in everyday life. It denies that they offer literally true descriptions. Dennett (1987) advocates a broadly instrumentalist position intermediate between intentional realism and eliminativism. He acknowledges that the ‘intentional stance’ is instrumentally useful for scientific psychology, but he questions whether mental states really have representational content. I assume a broadly scientific realist perspective: explanatory success is a prima facie guide to truth. From a scientific realist perspective, the explanatory success of Bayesian perceptual psychology provides prima facie reason to attribute representational content to perceptual states. The science is empirically successful and mathematically rigorous. It routinely individuates perceptual states through representational relations to the environment. We have no clue how to preserve the resulting explanatory benefits without employing representational locutions. Thus, current perceptual psychology strongly supports intentional realism over eliminativism and instrumentalism. We should no more adopt an eliminativist or instrumentalist posture towards intentionality than we should adopt an eliminativist or instrumentalist posture towards electrons. The famous Quinean criticisms of intentional psychology are notably less rigorous and compelling than the science they purport to undermine. Philosophers who reject intentionality as spooky, obscure, or otherwise unscientific are in fact opposing our current best science of perception. One might greet my argument by proposing an instrumentalist interpretation of perceptual psychology. In this vein, McDowell insists that appeals to representational content within perceptual psychology are ‘metaphorical’ (2010, 250). On his analysis, perceptual psychologists do not literally claim that perception represents. They claim only that perception proceeds as if it represents. Representational talk is mere heuristic. McDowell’s proposal misinterprets perceptual psychology. (Cf. Burge, 2011, 67–68.) A fundamental idea underlying how the science treats illusion is that a perceptual estimate can be inaccurate. An estimate is accurate only if the environmental conditions that it estimates actually obtain. Thus, intentional attribution is embedded within the foundations of the science. Representational locutions do not play a metaphorical role within Bayesian perceptual psychology. They are not heuristic chitchat. They reflect the central, explicit goal of the science: to describe how the perceptual system estimates environmental conditions. Instrumentalism is no more justified toward Bayesian perceptual psychology than toward any other science. Even readers who reject full-blown instrumentalism may contemplate a moderate instrumentalist agenda:  construe representational description literally when applied to explananda but metaphorically when applied to explanantia. Consider again Figure 37.1. Moderate instrumentalism adopts a realist stance towards sensory input e and the output hypothesis h but an instrumentalist stance towards the priors, posterior, and utility function. On this approach, the priors, posterior, and utility function are simply useful tools

706   Michael Rescorla for predicting how certain sensory inputs cause certain perceptual states. The perceptual system transits from retinal input to perceptual estimates as if it encodes Bayesian priors. Moderate instrumentalism concedes that the perceptual system implements a mapping from sensory inputs to perceptual estimates, but it remains neutral regarding how the perceptual system implements that mapping. For defence of moderate instrumentalism regarding Bayesian perceptual psychology, see Colombo and Seriès (2012). Moderate instrumentalism does not flout the science as blatantly as full-blown instrumentalism. Nevertheless, it strikes me as unsatisfactory. A key point here is that experience can alter the mapping from proximal input to perceptual estimates. For example, Adams, Graf, and Ernst (2004) manipulated the light-from-overhead prior by exposing subjects to deviant haptic feedback regarding shape. The new prior caused altered shapeestimates. Moreover, the new prior transferred to a different task that required subjects to estimate which side of an oriented bar was lighter than the other. Realists can offer a principled,unified explanation for the altered shape-estimates and lightness-estimates: namely, that there is a change in the prior over lighting directions. Moderate instrumentalists seem unable to offer a comparably satisfying explanation. Moderate instrumentalists must simply say that the mapping from retinal input to shape-estimates changes and that the mapping from retinal input to lightness-estimates changes, without offering any underlying explanation for why the mappings change as they do. In this case, at least, realism seems more explanatorily fruitful than moderate instrumentalism.15 We must exercise care in stating the realist position. As already noted, current Bayesian models are highly idealized. When the hypothesis space is large enough, the perceptual system may only approximately encode the priors and the posterior. What does it mean to ‘approximately encode’ a probability assignment? What is the difference between saying that the mind approximately implements Bayesian inference and saying that the mind merely behaves as if it implements Bayesian inference?16 These questions—which lie at the intersection of philosophy, AI, and empirical psychology—merit extensive further study.

6 Phenomenal content Relatively few philosophers reject representationalism. However, many popular philosophical theories downplay perceptual representation of the distal environment. Most of these theories are consistent with but unsupported by contemporary science. I will now illustrate by considering phenomenal content, as postulated by Chalmers (2006), Horgan and Tienson (2002), Thompson (2010), and various other philosophers. A distinguishing feature of phenomenal content is that it supervenes upon phenomenal aspects of experience. For example, suppose that a normal perceiver Nonvert observes

15  There are additional phenomena in a similar vein that favour realism towards prior probabilities and likelihoods (Seydell, Knill, and Trommershäuser, 2011; Beierholm, Quartz, and Shams, 2009). Realism towards the utility function seems well-supported for Bayesian models of bodily motion (Maloney and Mamassian, 2009). I am less sure about the utility functions that figure in Bayesian models of perception. Moderate instrumentalism may be more promising for that case. 16  Clark (2013) raises the same worry.

Bayesian Perceptual Psychology    707 a red object and experiences a perceptual state with a certain phenomenological character. Suppose that a spectrally inverted perceiver Invert observes a green object and experiences a phenomenally indistinguishable perceptual state. Chalmers and Thompson hold that, in both cases, the resulting percept is veridical. Nonvert’s percept correctly attributes redness, while Invert’s percept correctly attributes greenness. Chalmers and Thompson also hold that the two percepts share a uniform phenomenal content. The content represents red as used by Nonvert and green as used by Invert. Similarly, Chalmers and Thompson hold that a single phenomenal content might represent circularity as used by one perceiver and non-circular ellipticality as used by a phenomenological twin suitably embedded in a sufficiently different environment. There may be many good reasons for positing phenomenal contents. However, Bayesian perceptual psychology makes no use of such contents. The science delineates explanatory generalizations dictating how mental states that represent certain environmental properties induce other mental states that represent certain environmental properties. Bayesian models describe how the perceiver, exercising standing capacities to represent specific environmental properties, executes perceptual inferences yielding estimates of specific environmental properties. To illustrate, let us follow Thompson (2010) by considering phenomenological twins embedded in such different environments that one twin’s percept P represents circularity while the other twin’s qualitatively indistinguishable percept P* represents noncircular ellipticality. There may be many worthy explanatory projects that type-identify P and P*. But Bayesian perceptual psychology does not type-identify the two percepts. The science studies perceptual estimation of environmental conditions. P and P* estimate radically different environmental conditions: P estimates circularity, while P* estimates noncircular ellipticality. The science features no explanatory generalizations that assimilate these two percepts, because the relevant generalizations are tailored to specific shapes. Phenomenological overlap per se is irrelevant to the current science. What matters is representational overlap. Similarly, suppose that Nonvert observes a red object, while spectrally inverted Invert observes a green object. Chalmers and Thompson associate the resulting qualitatively indistinguishable percepts with a shared phenomenal content. In contrast, Bayesian perceptual psychology does not type-identify the percepts. Bayesian models treat surface colour perception as involving estimation of reflectance. Explanatory generalizations cite representational relations to specific reflectances. Current Bayesian models of Nonvert describe how retinal illumination C(λ) induces an estimate of illuminant I(λ), subsequently inducing an estimate of reflectance R(λ). Current Bayesian models of Invert describe how different retinal illumination C*(λ) induces an estimate of a different reflectance R*(λ). Reflectance-estimate R(λ) as used by Nonvert and reflectance-estimate R*(λ) as used by Invert may be associated with the same phenomenology. But this phenomenological overlap is irrelevant to the science. No explanatory generalizations type-identify the relevant perceptual processes. At no level of description does current science assimilate Nonvert’s colour perception and Invert’s colour perception.17 17  As noted above, one might hold that the final percept represents colour but not reflectance. However, this suggestion provides no support for phenomenal content. If one perceives a surface as a specific colour, then one’s percept is veridical only if the surface has that colour. Since Invert’s percept is veridical, and since the perceived surface is green, Invert does not perceive the surface as red. So Nonvert perceives a

708   Michael Rescorla Current perceptual psychology individuates perceptual states by citing representational relations to specific environmental properties.18 Taxonomization through phenomenal content ignores these representational relations. I conclude that phenomenal content is an armchair construct with no grounding inside contemporary science. Readers must judge for themselves whether philosophical energy is better expended studying this armchair construct or analyzing our current best science of perception.

7 The computational theory of mind I now want to consider the relation between Bayesian perceptual psychology and the popular philosophical view that mental activity involves computation over formal syntactic types in a language of thought (Field, 2001), (Fodor, 2008), (Stich, 1983). The paradigm here is a Turing machine manipulating formal syntactic items, such as stroke marks, inscribed in memory locations. A formal syntactic type may have a meaning. But it could have had a different meaning, just as the English word ‘cat’ could have denoted dogs. Depending on the perceiver’s causal or evolutionary history, a formal syntactic type that represents some distal property could just as easily have represented some other distal property. Formal syntactic manipulation is not sensitive to such changes in meaning. Transition rules governing mental computation allude solely to ‘local’ syntactic properties of mental states, without citing representational relations to the external environment. Field (2001) and Stich (1983) combine the formal syntactic picture with eliminativism. They urge scientific psychology to eschew any talk about representational content. Fodor (2008) combines the formal syntactic picture with intentional realism. In particular, he urges scientific psychology to delineate causal laws that cite representational content. He holds that intentional laws are implemented by syntactic mechanisms. So Fodor assigns a central role to representational content in addition to formal syntactic manipulation. Egan (1992) argues that perceptual psychology postulates formal syntactic manipulation. She defends her conclusion by analyzing the writings of Marr (1982). I  set aside whether Egan correctly describes Marr’s work, which was historically important but is now outdated.19 I claim that the formal syntactic picture finds no support within current perceptual psychology, as epitomized by Bayesian modelling. Current perceptual psych­ ology individuates mental computations in representational rather than formal syntactic

surface as red, while Invert does not perceive a surface as red. There is no basis here for type-identifying the relevant percepts. 18  One can individuate perceptual states through the environmental properties they represent without individuating them through the environmental particulars they represent. Burge (2010) introduces an individuative scheme for perceptual content along these lines. To illustrate, suppose that a percept attributes convexity to object O. According to Burge, any percept expressing the same content must also represent convexity. But a percept might express that same content while attributing convexity to a distinct object O*. Or a percept expressing that same content may involve a referential illusion, in which case it does not successfully attribute convexity to any object. 19  Silverberg (2006) argues that Egan misinterprets Marr. Egan (2009) discusses Bayesian models of perception but does not discuss how they bear upon her views regarding non-intentional computational modelling.

Bayesian Perceptual Psychology    709 terms (Burge, 2010, 95–101). For instance, Bayesian models of shape perception describe a computation whereby the visual system reallocates probabilities over hypotheses about distal shape. Each hypothesis is individuated partly by its representational relation to a specific distal shape. Transition rules governing the computation derive from Bayesian norms. Of course, the transition rules characterize initial sensory inputs (such as retinal inputs) physiologically rather than representationally. Crucially, though, the rules use representational vocabulary to characterize the perceptual states caused by initial sensory inputs. The rules do not cite formal syntax when characterizing sensory inputs (which are described physiologically) or ensuing perceptual states (which are described representationally). Bayesian models do not cite formal syntactic items divested of representational import.20 A complete science of perception must illuminate the neural mechanisms that implement Bayesian computation.21 Thus, a complete theory should include non-representational neural descriptions. But should it include non-representational syntactic descriptions? Syntax is supposed to be multiply realizable, in the sense that systems with wildly different intrinsic physical constitutions can satisfy the same syntactic description (Fodor, 2008, 91). Systems may be homogeneous under syntactic description but heterogeneous under neural description. Should a good theory posit formal syntactic types that are multiply realizable and that underdetermine representational content? There may be many good reasons for positing formal syntactic types with these features. Yet no such types figure in current perceptual psychology. The science does not employ computational descriptions that prescind from both representational and neural details. Eliminativist versions of the formal syntactic picture conflict with current perceptual psychology. Intentional realist versions of the formal syntactic picture are consistent with but unsupported by current perceptual psychology. A common rejoinder is that we can reinterpret intentional explanations in formal syntactic terms, without explanatory loss. In this vein, Field (2001, 72–82, 153–156) proposes a version of Bayesian modelling on which subjective probabilities attach to formal syntactic items individuated without regard to meaning or content. He claims that this framework can preserve any alleged explanatory benefits offered by intentional explanation. Field’s proposal is revisionary regarding contemporary psychology. Current science individuates perceptual states representationally. Field proposes an alternative scientific framework that individuates perceptual states in formal syntactic terms. Whether an alternative hypothesis subserves equally good explanations is not a question to be settled a priori. Proponents must first develop the alternative hypothesis in rigorous mathematical and empirical detail. Field must reconstruct current science, expunging any apparent reference to representation. Yet he does not indicate how to execute the needed reconstruction for a single real case study. He does not demonstrate through a single real example that his approach can replicate the explanatory benefits offered by intentional explanation within Bayesian psychology. Thus, Field’s proposal amounts to an unsupported conjecture that we can gut perceptual psychology of a central theoretical construct without explanatory loss. We have no reason to believe this conjecture, absent detailed confirmation.22 20 

Rescorla (2012) relates these points to the computational models employed within CS and AI. For discussion of possible neural mechanisms, see Clark (2013) and Knill and Pouget (2004). 22  The details of Field’s discussion raise further doubts about the conjecture. He claims that there is no viable interpersonal notion of type-identity for mental representation tokens (2001, 75, fn. 3). In other words, Field’s favoured taxonomic scheme cannot type-identify the mental states of distinct 21 

710   Michael Rescorla Generally speaking, we cannot radically alter how a science individuates its subject matter while preserving the science’s explanatory shape. We should not expect that we can transfigure the taxonomic scheme employed by current Bayesian models while retaining the explanatory benefits provided by those models. In her later writings, Egan (2010) avoids talk about formal syntactic manipulation. Instead, she claims that computational models of perception offer “abstract mathematical descriptions” that ignore representational properties of perceptual states. This new account shares a crucial feature with the formal syntactic picture. Both accounts prioritize non-intentional, non-neural computational descriptions. As I have argued, no such descriptions figure in Bayesian perceptual psychology. Philosophers motivate non-intentional computational modelling through various arguments. One popular argument emphasizes explanatory generality (Egan, 2010; Stich, 1983, 160–170). Following Egan (2010), consider a creature Visua whose perceptual states represent some environmental property (such as depth). Imagine a neurophysiological duplicate Twin Visua embedded in such a radically different environment that its corresponding perceptual states do not represent the same property.23 A non-intentional computational description can type-identify the doppelgangers. We cannot type-identify the doppelgangers if we classify perceptual states through representational relations to the environment. Shouldn’t we prefer the more general theory? Assessing the merits of this argument is a large task that lies beyond our main focus. The key point for present purposes is that Bayesian perceptual psychology does not type-identify Egan’s putative neurophysiological twins. The science explains how perceptual systems of terrestrial animals transit from sensory input to hypotheses that represent specific environmental properties. It studies terrestrial animals endowed with standing capacities to represent specific environmental properties. Its scope is not intergalactic. It does not seek to accommodate chimerical creatures imagined by philosophers. Whatever the putative explanatory benefits of non-intentional computational modelling, our actual best science of perception individuates perceptual states partly through representational relations to specific environmental properties.

8 An abstract mathematical description? To bolster my assessment, I  will now examine more carefully the role that probability theory plays within Bayesian modelling. Interested readers can consult any standard probability-theory textbook for the technical background to my discussion. Probability theory, as axiomatized by Kolmogorov, posits a sample space Ω whose elements are possible ‘outcomes’. Kolmogorov’s axioms place no restrictions on elements of Ω. creatures. This result is incompatible with current perceptual psychology, which routinely type-identifies the perceptual states of distinct creatures. How could any serious science of perception do otherwise? 23 

Not everyone accepts that there exist creatures Visua and Twin-Visua satisfying these assumptions. In particular, Segal (1991) denies that perceptual states of neurophysiological twins can represent different environmental properties. For the sake of argument, I grant Egan’s description of the thought experiment.

Bayesian Perceptual Psychology    711

Fig.  37.2  The probability density function for a Normal distribution. If Ω is discrete, then we can assign probabilities directly to its elements. If Ω is continuous, then we instead assign probabilities to privileged subsets of Ω. We introduce a σ-algebra over Ω (i.e. a set of subsets of Ω that contains Ω and is closed under countable union and complementation in Ω). A  probability measure assigns a probability (a real number) to each element of the σ-algebra. A random variable is a measurable function from Ω to the real numbers ℝ.24 A probability measure and a random variable jointly induce a probability distribution: an assignment of probabilities to privileged subsets of ℝ. Intuitively, the random variable lets us transform a probability assignment involving Ω into a probability assignment involving ℝ.25 The probability distribution exists entirely within the realm of abstract mathematical entities. By citing the random variable and the probability distribution, we vastly increase the elegance and utility of our mathematical formalism. In particular, we can now apply real analysis to probabilistic modelling. When Ω is continuous, we can often introduce a probability density function (pdf), which carries each element of ℝ to a probability density (also drawn from ℝ). A famous example is the Normal (or Gaussian) distribution, whose associated probability density function is depicted in Figure 37.2. The probability that a random variable attains a value within some region is found by integrating the pdf over that region. In other words, the probability assigned by the probability distribution to a region equals the integral of the pdf over that region.26 A pdf is a purely mathematical entity, just like a probability distribution. To apply probability theory to psychological modelling, we must specify the nature of the underlying sample space Ω. When we seek to model perception, we should construe Ω’s elements as perceptual estimates or hypotheses. For instance, if we are modelling depth perception, then we should construe each element of Ω as a perceptual estimate of some particular depth. One might gloss ‘perceptual estimates’ as mental representations, or Russellian propositions, or Fregean senses, or sets of possible worlds, and so on. The key point is that we individuate perceptual estimates at least partly through the environmental properties that the estimates represent. As I have argued, this is how the science 24 A function X: Ω→ℝ is measurable just in case, for every Borel set B⊆ℝ, X–1(B) belongs to the σ-algebra.

One can generalize the definition of random variable to include functions from Ω to mathematical structures besides the real numbers. For ease of exposition, I  focus on real-valued random variables. Consideration of generalized random variables would not alter my main conclusions. 25 Let P be a probability measure, let X: Ω→ℝ be a random variable, and let B⊆ℝ be a Borel set. Then we define a probability distribution PX by PX(B) = P(X–1(B)). b 26 If P is a probability distribution, and if ρ(x) is an associated pdf, then P ([a, b]) = ρ( x )dx . ∫ a

712   Michael Rescorla typically individuates perceptual estimates. Once we have introduced an underlying sample space, we can also introduce appropriate random variables. To illustrate, suppose that Ω contains depth-estimates. Then we can introduce a random variable D that maps each depth-estimate h to a real number D(h). Depending on our choice of D, the real number D(h) might be the depth estimated by h as measured in metres, or as measured in feet, and so on. In practice, Bayesian perceptual psychologists rarely highlight the underlying sample space Ω. Typical models, including all the models described in this chapter, instead emphasize probability distributions or pdfs. For instance, Jacobs (1999) posits a pdf for a random variable corresponding to depth. A pdf is a purely mathematical entity. By specifying it, we do not specify a unique sample space Ω. The pdf is consistent with numerous sample spaces. At first blush, the scientific emphasis on probability distributions and pdfs may seem to undermine my representationalist interpretation of Bayesian perceptual psychology. Consider once again Visua, whose perceptual states represent depth, and doppelganger Twin Visua, whose corresponding states do not represent depth. According to Egan, explanatory generalizations of perceptual psychology should and do apply uniformly to Visua and Twin Visua. We can supplement the generalizations by specifying the environmental properties represented by Visua or Twin Visua. But the generalizations themselves ignore environmental representata. The generalizations constitute an ‘abstract mathematical description’ equally consistent with diverse distal interpretations (Egan, 2010, 256). Initially, Bayesian models may seem to offer precisely what Egan demands: ‘abstract mathematical descriptions’ that prescind from environmental representata. After all, Bayesian models emphasize pdfs, and a pdf is a purely mathematical entity: a function from real numbers to real numbers. Shouldn’t we conclude that Bayesian models of depth perception describe Twin Visua just as well as Visua? Any such conclusion would be mistaken. I concede that a Bayesian perceptual model has an abstract mathematical form. I concede that, in principle, this abstract form encompasses diverse chimerical creatures. Nevertheless, the model describes statistical inferences over perceptual hypotheses, which it individuates partly through representational relations to specific environmental properties. Bayesian perceptual psychology does not pursue explanatory generalizations framed at an abstract mathematical level. Just as physics uses abstract mathematics to articulate generalizations over physical state-types, perceptual psychology uses abstract mathematics to articulate generalizations over representational mental state-types. The central issue here is the notion of random variable. A random variable is a function from a sample space Ω to the real numbers ℝ. Thus, a random variable is defined only given a sample space. Ultimately, any Bayesian perceptual model featuring a random variable presupposes an appropriate sample space Ω. Perceptual models cite random variables only so as to illuminate probability assignments to environmental state estimates. The goal is to describe a statistical inference over estimates about the perceiver’s environment. The random variable is a valuable device for describing this statistical inference. But it is simply a tool for formulating rigorous, elegant explanatory generalizations concerning perceptual estimates. As evidence for my position, I cite alternative measurement units. Our mapping from depth-estimates to real numbers depends upon our choice of units. The metric system

Bayesian Perceptual Psychology    713 yields one random variable. The British imperial system yields another. Our choice of random variable reflects our measurement units. Thus, the specific mathematical parameters enshrined by a random variable are mere artefacts of our measurement system. The parameters lack any explanatory significance for scientific psychology. We may use metric units to measure depth, but the perceptual system almost certainly does not. Psychological significance resides in the state estimate, not the mathematical entities through which we parameterize state estimates. Our ultimate concern is the probability measure over environmental state estimates, not the probability distribution over mathematical parameters. To privilege the latter over the former is to read our own idiosyncratic measurement system into the psychological phenomena. We must not conflate our measurement units with the environmental states that we use the units to measure. I conclude that Bayesian perceptual psychology offers intentional generalizations governing probability assignments to environmental state estimates. We articulate the generalizations by citing probability distributions and pdfs over mathematical entities. But these purely mathematical functions are artefacts of our measurement units. They reflect our idiosyncratic measurement conventions, not the underlying psychological reality. They do not yield any explanatorily significant level of non-representational psychological description. They are tools for describing how the perceptual system allocates probabilities over a hypothesis space whose elements are individuated representationally. A Bayesian perceptual model has an abstract mathematical form, but this form does not secure explanatorily significant non-representational descriptions of perceptual states. What if we identify the privileged measurement units used by the perceptual system? Can’t we assign explanatory priority to a pdf defined over those units? And won’t the resulting theory be non-representational? One problem with this suggestion is that the perceptual system may not employ measurement units. In Peacocke’s (1992) terminology, perceptual representation may be ‘unit-free’. As far as we know, for example, the visual system may form a depth-estimate without denominating that estimate in feet, metres, or any other measurement units (although we use units to describe the estimate’s accuracy-condition). Admittedly, we may eventually discover that the perceptual system employs measurement units. It is difficult to anticipate how such a discovery might impact perceptual psychology. At present, the matter is speculative. All we can say for sure is that current Bayesian models do not attribute measurement units to the perceptual system. Current science posits probabilistic updating over perceptual hypotheses. It individuates the hypotheses partly through the specific environmental properties they represent.

9 Open questions Bayesian perceptual psychology raises numerous further questions, many on the border between philosophy and science. A few examples: What neural mechanisms implement, or approximately implement, the computations posited by Bayesian models? Does the Bayesian paradigm generalize from perception to cognition?

714   Michael Rescorla Can Bayesian models illuminate the relation between normativity and intentionality? Can Bayesian models illuminate what it is to represent the external world? Philosophers who pursue these questions will discover an imposing scientific literature that rewards intensive foundational analysis.27

References Adams, W., Graf, E., and Ernst, M. (2004). ‘Experience Can Change the “Light-From-Above” Prior’. Nature Neuroscience, 7, 1,057–1,058. Beierholm, U., Quartz, S., and Shams, L. (2009). ‘Bayesian Priors Are Encoded Independently from Likelihoods in Human Multisensory Perception’. Journal of Vision, 9, 1–9. Bradley, P. (2008). ‘Constancy, Categories, and Bayes’. Philosophical Psychology, 21, 601–627. Brainard, D. (2009). ‘Bayesian Approaches to Color Vision’. In M. Gazzaniga (ed.), The Visual Neurosciences, 4th edn (pp. 395–408). Cambridge, MA: MIT Press. Brewer, B. (2007). ‘How to Account for Illusion’. In F. Macpherson and A. Haddock (eds), Disjunctivism: Perception, Action, Knowledge (pp. 168–180). Oxford: Oxford University Press. Burge, T. (2011). ‘Disjunctivism Again’. Philosophical Explorations, 14, 43–80. Burge, T. (2005). ‘Disjunctivism and Perceptual Psychology’. Philosophical Topics, 33, 1–78. Burge, T. (2010). Origins of Objectivity. Oxford: Oxford University Press. Campbell, J. (2010). ‘Demonstrative Reference, the Relational View of Experience, and the Proximality Principle’. In R. Jeshion (ed.), New Essays on Singular Thought (pp. 193–212). Oxford: Oxford University Press. Chalmers, D. (2006). ‘Perception and the Fall from Eden’. In T. Gendler and J. Hawthorne (eds), Perceptual Experience (pp. 49–125). Oxford: Oxford University Press. Churchland, P. (1981). ‘Eliminative Materialism and the Propositional Attitudes’. Journal of Philosophy, 78, 67–90. Clark, A. (2013). ‘Whatever Next? Predictive Brains, Situated Agents, and the Future of Cognitive Science’. Behavioral and Brain Sciences, 36, 181–204. Colombo, M. and Seriès, P. (2012). ‘Bayes in the Brain—On Bayesian Modeling in Neuroscience’. British Journal for the Philosophy of Science, 63, 697–723 Davies, M. (1992). ‘Perceptual Content and Local Supervenience’. Proceedings of the Aristotelian Society, 92, 21–45. Dennett, D. (1987). The Intentional Stance. Cambridge, MA: MIT Press. Egan, F. (2010). ‘Computational Models: A Modest Role for Content’. Studies in History and Philosophy of Science, 41, 253–259. Egan, F. (1992). ‘Individualism, Computation, and Perceptual Content’. Mind, 101, 443–459. Egan, F. (2009). ‘Vision’. In E, . Craig (ed.), Routledge Encyclopedia of Philosophy Online. . Ernst, M. (2010). ‘Eye Movements: Illusions in Slow Motion’. Current Biology, 20, R357–R359. Field, H. (2001). Truth and the Absence of Fact. Oxford: Oxford University Press. 27  I am indebted to Mohan Matthen and Susanna Siegel for comments that vastly improved this entry. I have also benefited from discussion of these issues with Tyler Burge, John Campbell, Kevin Falvey, Ian Nance, Christopher Peacocke, and Tamar Weber.

Bayesian Perceptual Psychology    715 Fodor, J. (2008). LOT 2. Oxford: Oxford University Press. Fodor, J. and Pylyshyn, Z. (1981). ‘How Direct is Visual Perception? Some Reflections on Gibson’s “Ecological Approach” ’. Cognition, 9, 139–196. Geisler, W. (2008). ‘Visual Perception and the Statistical Properties of Natural Scenes’. Annual Review of Psychology, 59, 167–192. Geisler, W. and Kersten, D. (2002). ‘Illusions, Perception, and Bayes’. Nature Neuroscience, 5, 508–510. Gibson, J. J. (1979). The Ecological Approach to Visual Perception. Boston, MA: Houghton Mifflin. Helmholtz, H. von. (1867). Handbuch der Physiologischen Optik. Leipzig: Voss. Horgan, T. and J. Tienson (2002). ‘The Intentionality of Phenomenology and the Phenomenology of Intentionality’. In D. Chalmers (ed.), Philosophy of Mind: Classical and Contemporary Readings (pp. 520–533). Oxford: Oxford University Press. Jacobs, R. (1999). ‘Optimal Integration of Texture and Motion Cues to Depth’. Vision Research 39, 3,621–3,629. Kersten, D. and Mamassian, P. (2009). ‘Ideal Observer Theory’. iIn L. Squire (ed.), Encyclopedia of Neuroscience, vol. 5 (pp. 89–95). Oxford: Oxford University Press. Kersten, D. and Schrater, P. (2002). ‘Pattern Inference Theory: A Probabilistic Approach to Vision’. In D. Heyer and R. Mausfeld (eds), Perception and the Physical World (pp. 191–228). New York: Wiley and Sons. Knill, D. (2007). ‘Learning Bayesian Priors for Depth Perception’. Journal of Vision, 7, 1–20. Knill, D. and Pouget, A. (2004). ‘The Bayesian Brain: The Role of Uncertainty in Neural Coding and Computation’. Trends in Neuroscience, 27, 712–719. Knill, D. and Richards, W. (eds) (1996). Perception as Bayesian Inference. Cambridge: Cambridge University Press. Maloney, L. and Mamassian, P. (2009). ‘Bayesian Decision Theory as a Model of Human Visual Perception: Testing Bayesian Transfer’. Visual Neuroscience, 26, 147–155. Mamassian, P., Landy, M., and Maloney, L. (2002). ‘Bayesian Modeling of Visual Perception’. In R. Rao, B. Olshausen, and M. Lewicki (eds), Probabilistic Models of the Brain (pp. 13–36). Cambridge, MA: MIT Press. Marr, D. (1982). Vision. New York: Freeman. Martin, M. (2004). ‘The Limits of Self-Awareness’. Philosophical Studies, 120, 37–89. Matthen, M. (2005). Seeing, Doing, and Knowing. Oxford: Clarendon Press. McDowell, J. (2010). ‘Tyler Burge on Disjunctivism’. Philosophical Explorations, 13, 243–255. Peacocke, C. (1992). ‘Scenarios, Concepts and Perception’. In T. Crane (ed.), The Contents of Experience (pp. 105–135). Cambridge: Cambridge University Press. Quine, W. V. (1960). Word and Object. Cambridge, MA: MIT Press. Rescorla, M. (2012). ‘How to Integrate Representation into Computational Modeling, and Why We Should’. Journal of Cognitive Science, 13, 1–38. Rock, I. (1983). The Logic of Perception. Cambridge, MA: MIT Press. Segal, G. (1991). ‘In Defense of a Reasonable Individualism’. Mind, 100, 485–494. Seydell, A., Knill, D., and Trommershäuser, J. (2011). ‘Prior and Learning in Cue Integration’. In J. Trommershäuser, K. Körding, and M. Landy (eds), Sensory Cue Integration (pp. 155– 172). Oxford: Oxford University Press. Siegel, S. (2011). ‘The Contents of Perception’. In E. Zalta (ed.), The Stanford Encyclopedia of Philosophy. . Silverberg, A. (2006). ‘Chomsky and Egan on Computational Theories of Vision’. Minds and Machines, 16, 495–524.

716   Michael Rescorla Stich, S. (1983). From Folk Psychology to Cognitive Science. Cambridge, MA: MIT Press. Stone, J. (2011). ‘Footprints Sticking Out of the Sand, Part 2: Children’s Bayesian Priors for Shape and Lighting Direction’. Perception, 40, 175–190. Thompson, B. (2010). ‘The Spatial Content of Experience’. Philosophy and Phenomenological Research, 81, 146–184. Travis, C. (2004). ‘The Silence of the Senses’. Mind, 113, 57–94. Trommershäuser, J., Körding, K., and Landy, M. (eds) (2011). Sensory Cue Integration. Oxford: Oxford University Press. Weiss, Y., Simoncelli, E., and Adelson, E. (2002). ‘Motion Illusions as Optimal Percepts’. Nature Neuroscience, 5, 598–604. Wolpert, D. (2007). ‘Probabilistic Models in Human Sensorimotor Control’. Human Movement Science, 26, 511–524.

Chapter 38

Signa l Detection  Theory E. Samuel Winer and Michael Snodgrass

In the scientific study of perception, we often wish to assess observers’ perceptual abilities given various stimuli and under various conditions. In such experimental paradigms, individuals make discriminative responses (e.g. ‘I saw/did not see X’) on many trials, allowing estimation of such abilities. Nonintuitively, however, it turns out that such responses do not simply reflect unmediated perceptual experience, but are rather the joint product of independent, separable perceptual and decision processes. Signal detection theory (SDT; Green and Swets, 1966) is the theoretical and empirical framework that established this surprising fact, thereby enabling the decomposition of observers’ discrimination responses into the underlying processes of perceptual sensitivity (d′) and the decision criterion (c). Although SDT may be profitably used for any domain involving difficult decisions (e.g. medical diagnosis), it was first primarily applied to psychophysics—that is, the relationship between various experimentally manipulated stimulus characteristics and/or other factors known to affect perception or decision processes and resultant changes in d′ or c. In this chapter, we first detail the predecessors and origins of SDT. We then provide a basic introduction to SDT, highlighting its foundational concepts and seminal implications for understanding perception. Then, because perceptual processes likely occur both consciously and unconsciously, we explore SDT’s central role in demarcating both varieties, as well as perhaps illustrating certain properties of these processes themselves. Lastly, we conclude by discussing some of the philosophical implications for understanding perception that may follow from SDT. The aim of this entry is to provide the reader with a sound understanding of the merits and implications of the framework that permanently altered the way investigators examine, conceptualize, and analyse decision-making processes.

1  Antecedents of SDT: Classical psychophysics Classical psychophysics emerged as an attempt to answer questions wrought from philosophical interrogations of perceptual experience—in particular, the relationship between

718    E. Samuel Winer and Michael Snodgrass the physical properties of external stimuli and internal sensations. The forefathers of psychophysics, Ernst Weber and Gustav Fechner, were some of the first to attempt to assess the psychological experience of sensations in a scientific manner by manipulating the intensities of various physical properties (e.g. size, weight, brightness, and loudness) and determining their relationship to the subjective magnitudes of resultant sensory experiences. For example, Weber (1834/1978) investigated humans’ ability to distinguish between two stimuli varying across dimensions such as size. He concluded, through lengthy experimentation, that there seemed to be a constant proportion for determining a just noticeable difference—that is, the smallest difference that could be reliably discerned. Indeed, the point at which a person could differentiate a second stimulus from a first was proportional to the size or intensity of the initial stimulus (Weber’s law: ∆Stim / Stim = Kw, where Kw is Weber’s constant). Fechner (1860/1966) expanded upon Weber’s work to focus more specifically on perceptual experience. Some of the main methods used in classical psychophysics include:

1) the method of limits (e.g. the intensity of a light in a dark box is gradually increased, and the observer reports when the light becomes visible); 2) the method of constant stimuli (e.g. the light is repeatedly presented at several intensities, and again the observer indicates when it is visible or invisible); and 3) the method of adjustment (e.g. an observer turns a dial and adjusts the light until it is just visible, or until it is just noticeably brighter than before).

Fechner noted that psychological experience could be associated with a logarithmic function relating it to its physical intensity (Fechner’s law:  Ψ  =  (k)log(Χ), where Ψ is psychological experience, k is constant, and Χ is stimulus intensity). In other words, his law introduced the possibility that the relationship between physical and mental correlates is logarithmic, and that this relationship can be charted by finding the constant (k) connecting them. These initial investigations established perceptual psychophysics as a legitimate method of scientific inquiry, and were seen by some to usher in a new type of science (particularly in response to Fechner’s work). Wilhelm Wundt (1874/1904), for example, widely considered to be the father of experimental psychology, based much of his and his students’ early experimental work on expanding upon Fechner’s investigations. Despite Wundt’s experimentation advancing the science of psychology, however, classical psychophysics could not solve the harder problems of perceptual experience. The associations between sensations and perceptions noted by Weber, Fechner, and Wundt provided relatively accurate heuristics, but certain stimuli intensities did not evoke psychological responses as predicted by the laws. In particular, stimuli of limited intensity (i.e. those experienced only ephemerally, or apparently outside of observers’ awareness) do not commonly follow these laws. Most importantly, it became increasingly evident to psychophysicists that observers’ reports at least sometimes did not straightforwardly reflect their perceptual experience, thus undermining the entire enterprise. In particular, various lines of evidence suggested that observers differed not only in their perceptual ability, but in how willing they were to affirm the existence of various stimuli—that is, they also differed in what came to be known as the criterion or response bias. In spite of these equivocal results, paradigms in the emergent field of experimental psychology continued to employ the methods of classical psychophysics until the advent of

Signal Detection Theory   719 SDT in the mid-twentieth century. The ‘New Look’ in perception was one of the last preSDT movements in perceptual research to use these classic paradigms. New Look studies attempted to assess the extent to which consciousness was affected by nonperceptual states such as belief and desire. Taken as purported evidence for the effect of emotionally relevant stimuli on subliminal (i.e. unconscious) perception, they found that observers would exhibit either perceptual vigilance or defence. In the former, apparently enhanced perception would occur for stimuli reflecting particularly positive or desirable objects or qualities, while in the latter (e.g. the classic ‘taboo word’ studies), observers would deny awareness for disagreeable stimuli at intensities where they would affirm perception of less objectionable stimuli. These studies represented a widespread attempt to assess ‘subception’ or subliminal perception of information. As noted by Erdelyi (1974), this movement resulted in many important advances and interesting hypotheses which still inform current research. However, New Look (and classical psychophysical studies generally) suffered from a major flaw. That is, they did not disentangle observers’ response biases from their actual perceptual experience or ability. They simply assumed that observers’ reports directly and accurately reflected their internal experience. Critically, SDT found ways to pry these apart. As we shall see, SDT revealed that all subjective reports of experience are the joint product of independent perceptual and decision processes, and hence do not straightforwardly reflect perceptual experience as hitherto assumed. Consequently, the introduction of SDT, as well as multiple methodological critiques, not only ended the New Look era but largely supplanted classical psychophysics in general. Today, SDT is the dominant framework for analysing discriminative behaviour of virtually any sort (e.g. perception, memory), and is arguably one of the most successful theories in all psychology or even science in general.

2  The signal detection theory framework SDT was developed (Green and Swets, 1966) in an attempt to solve the interpretability issues that plague classical psychophysics (Gescheider, 1997). Although here we largely focus on SDT in relation to perception, we also note that SDT’s framework is applicable to effectively all investigations involving decision-making. Indeed, a major text summarizing and updating the framework chose to drop ‘signal’ from the name and instead merely referred to SDT as detection theory, emphasizing its broad applicability (Macmillan and Creelman, 1991). At the core of classical psychophysics’ inability to solve its central problems was the assumption that perception is essentially threshold-based—that is, that a stimulus was either perceived or not, in a binary, all-or-none fashion. This assumption led naturally to models in which observers could then simply report this perception (or its lack). It became increasingly clear, however, that even stimuli presented repeatedly under identical conditions (intensity, etc.) actually yielded a continuously varying range of perceived stimulus intensities, usually following a normal distribution (the familiar ‘bell curve’)—presumably due to various fluctuations in internal and external factors such as random variation in neural efficiency, attention, stimulus equipment, and so on (i.e. ‘noise’). Moreover, this occurs even when no stimulus at all is presented, with ‘no-stimulus’ presentations

720    E. Samuel Winer and Michael Snodgrass Detection criterion (c) d Noise (blank) distribution

Signal-plus-noise (word) distribution

Less word-like More word-like Detection decision axis

Fig.  38.1  SDT signal and noise distributions. yielding perceptions that resemble the target stimulus to varying degrees, again producing a smoothly graded normal distribution. As such, perception becomes understood as a detection problem—essentially attempting to discern whether a stimulus (intrinsically embedded in noise—hence, ‘signal + noise’) or noise only has been presented. Finally, given that the situations in question usually involve at least somewhat difficult perceptual discriminations (else they are trivially easy, yielding little informative data), the two distributions (signal + noise and noise only) overlap to substantial degrees. Figure 38.1 depicts this general situation for the simplest perceptual discrimination task—detection. Here, for example, the detection task requires observers to determine whether a word or a blank has been visually presented (briefly).1 Many trials are conducted under identical (e.g. the same stimulus intensity) conditions, producing the signal + noise and noise only distributions (here, in Figure 38.1, the ‘word’ and ‘blank’ distributions) internal to the observers for those conditions. The x-axis represents how ‘wordlike’ the perceptions are—that is, how strongly the internal perceptual experience resembles a word, varying continuously from very weak to very strong. The y-axis represents how frequently (i.e. on what proportion of the trials) the noise and stimulus + noise stimuli produce these various degrees of perceptual experiences as reflected by their magnitude (position) on the x-axis. The normalized distance between the means of the two distributions (i.e. the distance on the x-axis between the average internal ‘wordlikeness’ of the two distributions) represents actual perceptual sensitivity (d′), usually thought to vary between a lower bound of d′ = 0 (if the distributions overlap completely), indicating no perceptual sensitivity at all, and progressively larger d′s (as the separation between the distributions increases). Independently, however, observers must moreover decide just how ‘wordlike’ their perception must be (i.e. at what point on the x-axis, usually referred to as the ‘decision’ axis) to warrant a ‘yes’ response. This reflects their response criterion (c; sometimes called ‘bias’ or ‘response bias’); individual perceptions exceeding this criterion will yield ‘yes’ (or ‘word’, etc.) responses, while those below this criterion will yield ‘no’ (or ‘blank’, etc.) responses. c = 0 if the criterion is placed exactly where the distributions cross (i.e. exactly halfway between the distributions; such a criterion is called ‘unbiased’ and will produce the best 1  One could do the same thing with auditory stimuli, etc. In true detection discriminations, the ‘noise’ condition really is blank; if the discrimination were between words and nonwords, that is actually a categorization (not detection) task. In this chapter, by and large such more fine-grained distinctions are avoided.

Signal Detection Theory   721 Response

Yes

Yes

No

Hit (H)

Miss (M)

False alarm (FA)

Correct rejection (CR)

Stimulus Presence No

Fig.  38.2  The standard SDT  2  ×  2. performance in terms of percentage correct). Criteria to the right of c = 0 reflect relatively conservative criterion placement (i.e. more total ‘noes’ than ‘yeses’, as in Figure 38.1), while those to the left of c = 0 reflect relatively liberal criterion placement (i.e. more total ‘yeses’ than ‘noes’). All this becomes clearer when we consider the classic 2 × 2 stimulus/response table afforded by SDT. See Figure 38.2. In it, the four possible response outcomes are depicted. If an observer responds ‘yes’ when the target stimulus was in fact presented, this is a ‘hit’ (H). If the observer instead responds ‘no’, this is a ‘miss’ (M). If the observer responds ‘yes’ when in fact no signal (i.e. noise only) was presented this is a ‘false alarm’ (FA), and finally if the observer instead responds ‘no,’ this is a ‘correct rejection’ (CR). Actual data from multiple presentations of the signal plus noise and noise alone are entered in this table as percentages, such that each row adds to unity (i.e. 1). Since M and CR are completely dependent upon H and FA (i.e. M = 1 – H; CR = 1 – FA), only the latter two are independent parameters (equivalently, one could analyse M and CR; then H and FA would be redundant). Finally, usually signal plus noise and noise alone trials are presented with equal frequency, but this is not necessary. Specific entries in such 2 × 2 tables follow directly from the relevant underlying d′ and c (e.g. see again Figure 38.1). In particular, H is the proportion of the entire signal plus noise distribution that lies equal to or beyond the vertical line representing c, while FA is the analogous proportion of the noise-only distribution. With this in mind, it becomes readily apparent that as c changes, so do both H and FA, even when d′ remains constant—once again illustrating that d′ and c are independent processes, and that they jointly determine observers’ responses. More conservative cs (i.e. as the criterion moves to the right) will reduce both H and FA, while more liberal cs (as it moves to the left) will increase both H and FA. D′ is calculated by taking the normal transform (z-score) of the percentage of false alarms and subtracting from it the z-score of the percentage of hits: d′ = [z(H) – z(FA)]. The criterion is calculated as c = –0.5* [z(H) + z(FA)]; the negative sign means that positive scores reflect conservative criteria, whereas negative scores reflect liberal criteria. The individual z-scores represent how far, expressed in standard deviations, H and FA vary from the respective means of the signal plus noise and noise only distributions, with each normal distribution’s mean set to zero. Relatedly, for calculation, when H or FA are exactly zero, z = 0; when H or FA exceed .5, use positive zs for the individual value(s) in question; finally, when either is less than .5, use negative zs for the relevant values.

722    E. Samuel Winer and Michael Snodgrass Critically, d′ and c are independent. In particular, regardless of d′, observers can place c wherever they wish. Indeed, much experimentation has shown that criterion placement is under voluntary control and is responsive to various experimental manipulations that encourage more versus less conservative responses. Further, it is important to realize that c is, in itself, nonperceptual—rather, it reflects metacognitive response criteria that are applied to perceptions in order to select a response given observers’ priorities relating to the costs and benefits of more versus less conservative responding. At the same time, observers frequently do not realize that this is what they are doing when they make discriminative judgements, instead feeling that they are simply reporting their perceptual experience. Nonetheless, criteria vary quite substantially among individual observers, with substantive consequences. In particular, given the independence of d′ and c, it becomes clear that, given a particular d′, it is impossible to increase H without also increasing FA; relatedly, one cannot reduce FA without reducing H. For example, if one wants to get ‘tough on crime’, this will catch more guilty individuals (increase H) but will necessarily also increase arrests, convictions, etc. of innocent ones (increase FA). In contrast, to increase H and reduce FA, d′ itself must be improved (e.g. through training, feedback, and practice). To illustrate, consider a situation in which a person is asked to evaluate how accurate two judges have been in rendering verdicts. (Again, SDT applies in virtually all situations requiring decisions, not just perceptual ones.) For this example, imagine that both judges have been presented with 200 cases for which DNA data documenting clear evidence of true innocence or guilt (100 each) of the defendants has been made available after judgment was decreed. One might first examine the judges’ hits. Say Judge 1 has correctly rendered verdicts on 92 per cent of the guilty defendants, whereas Judge 2 has correctly rendered verdicts on only 54 per cent of the guilty defendants. Here, it seems that the first judge is more effective. Is this the case? SDT demonstrates definitively that one cannot determine either judge’s effectiveness with the information thus far provided. Their effectiveness is dependent on not only their hits, but also on their false alarms. Judge 1 has correctly convicted almost all of the guilty defendants, but how many innocent defendants has he correctly vindicated? And while Judge 2 only correctly convicted about half of the guilty defendants, how does she fare with the innocent defendants? To flesh out the big picture, imagine that Judge 1 has only reached a correct verdict for 54 per cent of his innocent defendants, whereas Judge 2 has rendered correct verdicts for 92 per cent of her innocent defendants. See Figure 38.3. Incorporating this additional information (and realizing that FA = 1 – CR) shows that the judges have identical sensitivity to actual guilt versus innocence: Judge 1 d′ = [z(.92) – z(.46)] = 1.51; Judge 2 d′ = [z(.54) – z(.08)] = 1.51. But how, then, can we explain their apparently very different performance patterns, wherein Judge 1 appears to be a wizard at nailing actually guilty defendants, while Judge 2 seems to be a master at exonerating actually innocent defendants? Application of SDT resolves this mystery: The two judges actually differ markedly in their response criteria, not in their ability to discriminate guilt versus innocence. To illustrate, if correctly judging guilt is a hit, then judging someone guilty when they are actually innocent is a false alarm. Judge 1 has elevated hits, but also substantial false alarms—thus exhibiting a liberal criterion (c = – .65). In contrast, Judge 2 has fewer hits, but very few false alarms, indicating a conservative criterion (c = + .65). Thus, Judge 1 is more likely to convict, while Judge 2 is less likely to do so—even given identical evidence and the same ability to discriminate guilt versus innocence—because Judge 1 is willing to accept relatively less evidence to say ‘yes’ (i.e. convict) than Judge 2.

Signal Detection Theory   723 Verdict (Judge 1)

Guilty

Guilty

Innocent

.92

.08

.46

.54

DNA evidence Innocent

Verdict (Judge 2)

Guilty

Guilty

Innocent

.54

.46

.08

.92

DNA evidence Innocent

Fig.  38.3  Hits, misses, false alarms, and correct rejections by judges 1  and  2. But why do the judges differ so markedly in their response criteria? Much evidence suggests that observers choose their criteria in line with their evaluation of the benefits and costs of H and FA. Broadly speaking, if hits are more important, more liberal criteria will be employed; if (reducing) false alarms is more important, more conservative criteria will be employed. Here, Judge 1’s main priority is convicting the guilty, while Judge 2’s main priority is protecting the innocent. Analogous criterion shifts have been demonstrated experimentally with differing monetary rewards for H versus FA, or even simply instructing observers to prioritize one versus the other. At the same time, observers have their own, initially preferred ‘natural’ criteria given the task and conditions, again reflecting their (often implicit) assessment of the relevant costs versus benefits, and typically resist changing these criteria to various degrees. Indeed, as noted above, observers often feel that they are simply directly reporting their perceptual experience, without realizing that criteria are intrinsically involved at all. At the same time, many observers seem to naturally prefer keeping FA to a fairly low level (the Neyman–Pearson objective), apparently reflecting an aversion to committing errors (FAs)—in this situation, ‘seeing things that aren’t there’. When discrimination is fairly easy (i.e. the signal + noise and noise distributions overlap little, yielding high d′s), this can be accomplished with relatively unbiased criteria. On the other hand, when discrimination is difficult (i.e. the distributions overlap substantially), keeping FAs low requires more conservative criteria.2 2 

It has been asked whether this suggests that (for instance) as night falls and visual discrimination becomes hard, or as conversation in a cocktail party becomes louder and your perception of your companion’s speech becomes less distinct, the criterion will move steadily rightwards. In other words, as evidence becomes more and more difficult to get, observers will actually demand more of it. Why do observers not settle for less? Is it that moving c to the right maximizes their hit to false alarm ratio? In this

724    E. Samuel Winer and Michael Snodgrass Further insight into the meaning of c follows when we consider an important variant of the detection task—the rating task. Here, rather than simply replying ‘yes’ or ‘no’, observers rate their confidence that the target stimulus has been presented on each trial (e.g. 1 = ‘no confidence’; 2 = ‘slightly confident’; 3 = ‘somewhat confident’; 4 = ‘highly confident’; 5 = ‘certain’). Critically, much experimentation shows that d′s obtained using the binary yes/no and rating versions (given the same stimuli, conditions, etc.) are equal, and that the single criterion c derived from the yes/no task corresponds to observers’ preferred level of confidence for the binary yes/no decision. As such, c reflects the subjective strength of evidence (that the target event has just occurred) required by the observer corresponding to their desired level of confidence to respond affirmatively. Using the rating task further enables the construction of receiver operating curves (ROCs), in which H and FA points corresponding to the various confidence ratings can be graphed. ROCs are useful because they enable determination of the actual (i.e. empirical) ratio of the signal plus noise versus noise alone variances, which sometimes differ from the standard SDT assumption of equal variances, and further allow calculation of A′ (i.e. the proportion of the total area under the graphed curve), a nonparametric analogue of d′. Obtaining ROCs can be cumbersome, however, and are often unnecessary for many applications. Finally, one might wonder how d′ fares compared to other common indexes of sensitivity such as the familiar percentage correct (PC), which is simply the mean of H and CR. PC is certainly an intuitively appealing index, as it appears to straightforwardly reflect perceptual (memorial, etc.) accuracy. SDT has shown, however, that using PC as an accuracy index implies an underlying threshold model (i.e. one either sees/remembers/knows something or not), and as such is now known to be flawed. In particular, PC confounds sensitivity and response bias. Notably, it turns out that if either liberal or conservative biases manifest, PC systematically underestimates true sensitivity in proportion to the magnitude of the bias. These distorting effects can be quite substantive. For example, in ‘blindsight,’ a theoretically important neuropsychological syndrome involving damage to the primary visual cortex, discrimination performance on certain visual tasks is quite substantial despite essentially chance PC levels on detection tasks, apparently indicating that the former is driven by unconscious perceptual processes. It turns out, however, that some (but not all) blindsight phenomena can be explained as resulting from markedly conservative response criteria on the detection task, thus artefactually depressing detection PC. On the other hand, using PC is not problematic if c = 0 (i.e. with unbiased responding), and causes little distortion if biases are present but small. If there are strong reasons to believe biases are minimal, then, PC can be used to index accuracy reasonably safely. When chapter we are trying to avoid these subtleties, but a note here may be of help to readers. Actually, relative to the noise distribution, the criterion stays in the same place (which keeps FA constant). What’s changing in this situation is the signal + noise distribution, which gets ‘farther away’ to the right as d′ increases, and closer to the noise distribution as d′ decreases (i.e. when discrimination is more difficult). Also, when we measure c, it is always relative to c = 0, defined as the point where the distributions overlap. Relative to the x-axis, this ‘zero point’ will move as d′ changes. So, if you compute c in the two situations with someone who wants to keep FA low, this will be close to zero in the high-d′ situation, but markedly positive in the low-d′ situation—even though it cuts off the same amount of the noise-only distribution in both cases. And finally, as now noted, people like to keep FA low because they don’t want to make certain kinds of errors, even though this would result in poorer performance (as reflected in percentage correct, not d′).

Signal Detection Theory   725 possible, however, SDT approaches are generally preferable because they do not require these strong assumptions.

3 Extensions of SDT Thus far, we have primarily considered the classic yes/no detection and rating tasks. Besides being the first SDT tasks to be developed, they share another key characteristic—namely, they are single-dimensional. That is, the two possible stimulus classes (signal plus noise, noise-alone) differ only in whether a single signal is present, and as such the relevant decision/strength of evidence axis can be seen as reflecting a single dimension along which the observers’ perceptions can be ordered—in our primary example, how ‘wordlike’ the perception is. Many discrimination tasks, however, involve attempting to discern whether two (or more) signals can be discriminated from each other. We briefly mention some of these now; a fuller description is beyond the scope of this introduction to SDT. As a calculation/interpretive note, in all such non-detection (i.e. multiple signal) tasks, one stimulus (or stimulus class) is arbitrarily selected to be the signal (i.e. where correctly affirming its presentation is a hit), and the other the noise (where incorrectly claiming that this second signal is the first is a false alarm). Which is which makes no difference as long as the chosen mapping is clear. The simplest such case is the two-alternative identification task (e.g. ‘Was the stimulus word A or word B?’). Here, observers are being asked to discriminate which of two signals have been presented—not whether a particular signal has been presented at all, as in detection tasks. SDT models such tasks in a two-dimensional decision space, where each signal dimension reflects its own signal plus noise versus noise alone detection representation as above, and the standardized distance between the means of the signal A and signal B distributions is the A versus B identification d′. Identification d′ depends not only on the individual detection d′s for signal A and B, but also the correlation (i.e. similarity or overlap) between the respective signal A and B dimensions. To illustrate, imagine detection d′ A = detection, d′ B = 1. If, as is often assumed (but frequently without justification), the A and B dimensions are uncorrelated (graphically represented as a 90° angle), by the Pythagorean theorem, identification d′ = √2. Indeed, standard texts typically recommend reducing empirically obtained identification d′s by this factor to facilitate comparison with detection d′s. However, if the A & B dimensions are instead correlated, identification d′ < √2, declining to a minimum of 0 if the correlation is perfect. Empirically, then, it is advisable to actually obtain identification d′ directly along with the individual A and B detection d′s, which then allows calculation of the actual dimensional overlap and enables precise adjustment of identification d′ if desired. Frequently, however, in this situation one simply cares about identification d′; in this case, using its unadjusted (original) value likely makes more sense. Another very common situation is classification discrimination, where multiple stimuli belonging to category A (e.g. nouns) and B (e.g. adjectives) are presented, and observers must determine which category a given stimulus belongs to. The guilt versus innocence task discussed above is another example of a classification discrimination. Memory paradigms (e.g. source discrimination—‘Was this word previously presented in List A or List B?’) also extensively utilize classification tasks. Classification is more

726    E. Samuel Winer and Michael Snodgrass difficult than either identification or true detection, essentially because more information is necessary to successfully perform the former. Despite the differences between detection, identification, and classification (and other discrimination tasks modelled by SDT), however, many researchers imprecisely refer to whichever discrimination task is at hand as ‘detection’, which can sometimes obscure significant consequences of such task differences.

4 SDT and conscious versus unconscious perception Current cognitive theory and research suggests that perception occurs not only consciously, but unconsciously as well. Given this, we might wonder how both conscious and unconscious perceptual processes contribute to obtained d′s. Further, to isolate and study unconscious perceptual processes, it is necessary to somehow ascertain whether the relevant stimuli are consciously perceivable. Commonly, this is attempted by rendering the stimuli difficult to see (or hear, etc.) by utilizing very brief presentations combined with various masking techniques. Then, one must somehow assess observers’ conscious perceptual abilities, so that the success or failure of such manipulations can be assessed. Here again, we might wonder what lessons or applicability SDT might afford in this respect.

Traditional view: SDT is agnostic—or is it? As first proposed, SDT did not explicitly address implications for conscious versus unconscious perceptual processes (Green and Swets, 1966). Rather, its architects suggested that d′ simply reflected observers’ available perceptual evidence, which might be conscious, unconscious, or both. Therefore, in the traditional SDT framework, consciousness is simply irrelevant. Individuals may or may not report detecting some signal depending on their criterion, but their perceptual sensitivity indicates that it is nonetheless available to them. This agnostic view contrasted sharply with classical psychophysics, which was very interested in determining perceptual thresholds—above which observers were conscious of the relevant stimuli, and below which they were not. Notwithstanding its ostensible agnosticism, however, in virtually the same breath SDT theorists often spoke as if SDT indeed indexed only conscious perceptual processes. For example, responding to the ‘New Look’ studies of the time, Green and Swets suggested that studies that had purportedly demonstrated subliminal perception essentially assumed, without justification, that below-criterion stimuli (i.e. where observers will say ‘no’ despite a signal being presented) were unconscious. Along these lines, they concluded that there was ‘no need to postulate a physiologically determined threshold, or a separate psychological or physiological process such as subliminal perception’ (Green and Swets, 1966: 336)— clearly implying that d′ reflects only conscious perception, with observers’ affirmations

Signal Detection Theory   727 versus denials simply reflecting criterion placement. If indeed SDT indexes only conscious perceptual processes, a possible way forward begins to emerge.

Objective threshold approaches The SDT criterion artefact critique of New Look experimentation and its classical psychophysical methods suggested a powerful reinterpretation of such ostensibly unconscious influences on observers’ thresholds for awareness—namely, that such influences were not unconscious but simply below criterion. This SDT critique essentially halted investigations into unconscious processes for some 25 years (roughly from the late 1950s to the early 1980s). Eventually, the development of priming paradigms seemed to offer an alternative approach. These paradigms, which assessed effects of immediately preceding stimuli on rapidly following clearly visible ones (e.g. participants could more quickly determine that the latter, ‘target’ stimulus was a word if the immediately preceding prime was related in meaning or structure than if not) were thought to index automatic, unintended, and possibly intrinsically unconscious mental processes which might not be available for ‘direct’, voluntarily controlled tasks such as SDT discriminations, which are sensitive to any consciously perceived information. Accordingly, perhaps unconscious perceptual effects could manifest on ‘indirect’ tasks such as priming, even when the very same observers exhibit d′ = 0 on a separate SDT discrimination task—thus additionally showing that the primes are not consciously perceivable. In this way, these are ‘dissociation’ paradigms—that is, they seek to show that at least some degree of priming is possible in the complete absence of relevant conscious perception, thereby demonstrating unconscious perceptual processes (else d′ would exceed zero). These paradigms are called ‘objective threshold’ approaches because they seek to demarcate conscious versus unconscious perceptual processes via ostensibly objectively observable (i.e. by third party) behavioural measures such as d′, rather than asking observers to report their subjective experience per se. Further, here, ‘threshold’ refers to when d′ = 0—that is, when the signal plus noise and noise-only distributions overlap completely. In contrast, with the benefit of SDT-based hindsight, ‘thresholds’ in classical psychophysics actually refer to observers’ response criteria.3 Although early objective threshold investigations yielded mixed results and suffered from various methodological shortcomings, the current consensus is that such paradigms do produce reliable positive findings on indirect tasks such as priming. Given further their greater ability to rule out alternative criterion artefact explanations (versus subjective approaches), they are the dominant unconscious perception paradigm in current use. Additionally, their empirical success apparently strengthens the possibility reinforcing

3 

Priming does not increase sensitivity. ‘Sensitivity’ only applies to SDT tasks (priming is not an SDT task, but rather investigates the influence of an immediately preceding stimulus on some subsequent stimulus). The force of the objective threshold approaches is that they are not subject to the SDT criterion artefact critique, since they identify ‘threshold’ as where d′ = 0, not with the response criterion as in classical psychophysics. So, if d′ = 0 on some appropriate SDT task, we can rule out conscious perception of the relevant stimuli. If in an additional priming task under identical stimulus conditions we nonetheless find priming effects, such effects must be due to unconscious, not conscious, perception.

728    E. Samuel Winer and Michael Snodgrass that direct discriminations such as SDT tasks do indeed selectively index only conscious perceptual processes.

Subjective threshold approaches Instead, however, if unconscious perceptual influences do contribute to d′, attaining objective threshold conditions would either eliminate or at least reduce such unconscious processes along with any concurrent conscious ones. Plausibly, then, objective threshold approaches are only valid if d′ indeed exclusively indexes conscious perceptual processes only (Reingold and Merikle, 1990), which seems a strong assumption requiring considerable substantiation. Further, some argue that objective threshold approaches are fundamentally wrongheaded because they apparently do not assess subjective experience itself—the essential feature of consciousness—but rather simply assess discriminative ability (d′). Relatedly, some researchers argue that we should take observers at their word when they deny awareness, as in classical psychophysical experiments—that is, utilize ‘subjective threshold’ approaches instead. Notably, this typically occurs with stimulus intensities that yield d′s that substantially exceed zero (e.g. 5 – 1); indeed, such effects were the original evidence for unconscious perception, going all the way back to the 1890s. If such findings are valid, they clearly suggest that d′ is indeed influenced by unconscious perception. These reservations and criticisms of objective threshold approaches, however, may not fully appreciate the power of the original SDT criterion artefact critique. It is important to emphasize that, perhaps nonintuitively, the SDT critique does not somehow contradict or deny observers’ subjective reports. Rather, because SDT demonstrates that all such reports are influenced by both perceptual sensitivity and response criteria, alternative explanations of subjective experience in terms of the latter become straightforwardly possible. For example, very weak conscious perception will plausibly resemble no stimulus at all much more closely than it does some positive response option, yielding denials of awareness. After all, very weak stimuli could easily produce remarkably faint, indistinct, and unclear conscious perceptual experiences that, not surprisingly, individuals are quite uncertain about. In other words, denying awareness might simply indicate very low confidence, not the complete absence of conscious perceptual experience. Reinforcing this conclusion, recall that the rating task shows that d′, rather than being some purely behavioural index as sometimes alleged, actually assesses varying degrees of subjective confidence that the target stimulus has been presented (Dulany, 1997). In this way, regarding below-criterion (i.e. subjective threshold) perceptions as weakly conscious is more plausible than it might initially appear—particularly given that SDT further shows that perception is smoothly and continuously graded, ranging from exceedingly weak to very strong. At least partially recognizing the power of the SDT criterion artefact critique, however, modern subjective threshold approaches (unlike their classical psychophysics forbearers) generally recognize the need to demonstrate that simple denials of awareness may not be enough to guarantee true lack of awareness. Accordingly, they typically stress obtaining convergent qualitative differences between ostensibly unconscious and conscious effects (Merikle et al., 2001). One of the most compelling of such qualitative differences derives from Jacoby’s (1991) process-dissociation paradigm, which seeks to derive estimates of

Signal Detection Theory   729 conscious versus unconscious influences when d′ > 0. In particular, observers are required to either respond with a previously presented stimulus (as in typical discrimination tasks), or instead with a nonpresented alternative. In one common example, observers are asked to complete word stems (e.g. rea___) which could be completed to make several different words (e.g. reader, reason, and reaper). Just prior to presenting the clearly perceivable word stem, another word is presented, which on critical trials is a word that begins with the same stem (e.g. reason) and could thus be used as a response on the word stem. Under inclusion instructions observers are told to use the previously presented word, if possible, to complete the stem, whereas under exclusion instructions they are asked to complete the stem with a word different from the previously presented one. Critically, when the initial word stimuli are difficult to perceive (e.g. when presented at subjective threshold intensities), observers given exclusion instructions will often nonetheless give the previously presented word to complete the stem above chance (i.e. baseline) levels despite being told not to do so (i.e. exclusion failure). This is often taken to demonstrate unconscious perception (or memory, etc.), relying on the intuitively compelling reasoning that if the initial word stimuli were consciously perceived, observers would exclude them from responding as instructed. Indeed, when clearly consciously perceivable stimuli are used, observers do exactly this, apparently supporting exclusion failure’s unconscious status. Here again, however, an extension of the SDT criterion artefact critique suggests that exclusion failure may be weakly conscious after all. After all, to exclude or not is a decision; hence, response criteria are likely applied in these situations as well, such that high confidence perceptions are excluded, while low-confidence ones are not, yielding exclusion failure if enough stimuli are below-criterion (Snodgrass, 2002). If this extended SDT critique is correct, exclusion failure should vary with shifts in exclusion criteria, rather than being some fixed phenomenon. Indeed, Visser and Merikle (1999) found that exclusion failure disappeared when observers were rewarded for successful exclusion (a classic SDT manipulation to induce criterion shifts, although this was not these authors’ intention), which should not occur if stimuli below observers’ original, more conservative exclusion criteria were indeed unconscious—but is completely consistent with the possibility that nonexcluded stimuli are weakly conscious. Despite these difficulties, however, exclusion paradigms remain popular, perhaps in part owing to their strong intuitive appeal. Further, process-dissociation models have been revised to incorporate both a continuous, SDT-like unconscious process and a threshold-based conscious process (e.g. Yonelinas, 1994), but single-process SDT alternatives, either incorporating unequal variances for the signal plus noise versus noise distributions (Glanzer et al., 1999) or multidimensional representations (Banks, 2000) appear capable of accounting for the same phenomena without requiring separate unconscious processes.

Type II tasks and consciousness Notably, all the above SDT tasks, despite their differences, nonetheless share a deep connection—namely, they ask observers to discriminate stimulus classes, and are referred to as ‘Type 1’ tasks. Importantly, however, a fundamentally distinct variety of SDT tasks exists, which instead requires observers to discriminate the correctness of their own

730    E. Samuel Winer and Michael Snodgrass responses—that is, an explicitly metacognitive, introspective judgement. Such ‘Type II’ tasks proceed by first asking observers to make a conventional binary Type I response (e.g. ‘I think word A was just presented’), and then rate their confidence in the correctness of this response (e.g. more versus less confident). After many such trials, Type II d′ and c are then calculated, with the ‘stimulus classes’ now being the actually correct versus incorrect trials. Although the Type II approach has been known since shortly after SDT’s inception, it was rarely used until its formal properties were described in depth by Galvin, Podd, Drga, and Whitmore (2003), and even now is relatively uncommon. Nonetheless, it is clearly important where metacognitive questions are concerned such as observers’ (meta) knowledge about their own abilities (perceptual, memorial, etc.), and is increasingly employed in various recent investigations seeking to differentiate conscious versus unconscious processing. Here, the Type II task’s appeal largely stems from its apparent ability to truly index subjective experience, in that observers are asked to introspectively assess their confidence in the correctness of their responses—whereas some feel (perhaps erroneously; see above) that Type I tasks do not refer to subjective experience at all. In this way, Type II tasks are viewed by some as perhaps the Holy Grail—an objective way to measure subjective consciousness. Unfortunately, however, Type II tasks have serious intrinsic drawbacks. For example, Type II d′ is not independent of either Type I or Type II c (or, less surprisingly, Type I d′). Further, Type II d′ is almost always, for mathematical reasons not germane to perceptual processes, less than Type I d′. These intrinsic properties may place serious limitations on the usefulness of Type II approaches. Further, it is important to realize that Type II approaches are actually intended to index a form of ‘metaconfidence’, that is, confidence in the accuracy of one’s (Type I) discrimination responses—which, as discussed above, themselves reflect confidence that the target stimulus is present versus absent (or that word A versus. B was just presented, etc.) ‘out there’ in the real world. With this in mind, observers can simply use their Type I confidence to make Type II responses (e.g. ‘Hmm . . . I’m pretty confident that word A was presented, so I’m pretty sure I answered correctly, so I’ll say I’m more rather than less confident that I was right’). If this occurs, Type II tasks may sometimes simply reflect a particular transformation of Type I  information, rather than providing genuinely new information as intended. At the same time, empirically, Type II d′ is often less than what should obtain if observers optimally employed this approach, suggesting that it may indeed index a form of metaconfidence as intended. Alternatively, observers may simply find Type II tasks more difficult, as they likely require not simply judging if their perceptual experience exceeds their Type I criterion, but by how much it falls above or below this criterion—a considerably more demanding task.

5  Summary and philosophical implications As stressed repeatedly above, SDT forces the critical and perhaps nonintuitive conclusion that observers’ reports do not transparently reflect perceptual sensitivity/experience, but are rather the joint product of two processes:  (1)  Perceptual sensitivity/experience

Signal Detection Theory   731 itself  (d′); and (2)  Separate, independent, and voluntarily controllable flexible response criteria (c) applied to this perceptual evidence for the discrimination purposes at hand. Moreover, the SDT framework suggests that we never simply ‘perceive’ anything. Rather, we form probabilistic/statistical perceptual hypotheses about various experiences, and some degree of error is intrinsically inescapable. One way to see this follows from an alternative representation of the response criterion—ß (beta). ß is the ratio between the percentage of the signal plus noise distribution to the right of a given c and the percentage of the noise alone distribution to the right of the same c (we describe ß in this way for expository convenience; in reality, selecting a desired ß does not require first selecting some c). In this way, ß reflects the likelihood (expressed as an odds ratio) that a particular perception at that exact criterion actually comes from the signal plus noise (versus noise alone) distribution. The statistically informed reader will realize that SDT thus bears deep connections with conventional statistics, which relatedly provide the probability of the likelihood of the ‘null hypothesis’ (i.e. that the data actually come from the ‘noise’ distribution). From this perspective, d′ is essentially equivalent to the experimental ‘effect size’, while selecting differing criteria corresponds to selecting the desired probability value for rejection of the null hypothesis (i.e. affirming that a signal exists). In these ways and more, SDT has fundamentally altered our understanding of perceptual and memorial processes, as well as decision processes generally. Despite the overwhelming evidence favouring the SDT framework over classical psychophysics/subjective threshold models, however, many continue to question d′s relevance to consciousness. Such reservations appear to stem largely from the probably erroneous notion that d′ is brutely behavioural, with no apparent direct connection to subjective experience. In contrast, however, analysis of the SDT rating task suggests that d′ actually indexes subjective confidence itself—but in a framework that disentangles our confidence-based ability to discriminate the stimulus classes at hand from our criteria for response selection, rather than confounding these separate processes.

A possible synthesis Given the continuing power of the SDT criterion artefact critique, one possible conclusion is that subjective threshold approaches are simply invalid, and relatedly that d′ indeed indexes purely conscious perceptual processes—hence suggesting that only objective threshold approaches are appropriate to study unconscious perceptual influences. At the same time, it seems unsatisfying to simply reject subjective approaches outright, because they seem to capture something important about our conscious experience. But what, exactly? A valuable clue may lie in further consideration of the qualitative differences in the effects of subjectively unconscious versus clearly conscious stimuli. Importantly, all such differences to date (e.g. exclusion failure versus success) share a key common feature— namely, observers typically do not attempt to further process such stimuli, but rather ignore them because they believe they do not exist (or are at least too weak and low-confidence to merit further processing). As such, criterion-based subjective threshold methods may indeed index a fundamental form of consciousness, at least in humans—namely, an important variety of Block’s (e.g. 2005) ‘access consciousness,’ which intrinsically involves

732    E. Samuel Winer and Michael Snodgrass metacognitive, second-order conscious processes relevant to ‘ . . . reasoning, planning, evaluation of alternatives, decision-making, voluntary direction of attention, and more generally, rational control of action’ (Block, 2005: 48). In contrast, d′ may index first-order phenomenal consciousness itself, the contents of which may or may not be accessed for additional second-order processing that takes the relevant phenomenally conscious contents as objects for further analysis. While we cannot pursue this here (see Snodgrass et al., 2009, for more details), this framework may be able to reconcile seemingly intractable controversies over objective versus subjective approaches by clarifying their respective foci on first-order (phenomenal) versus second-order (access) consciousness, with each having their own valid and important domains. Distinguishing first- versus second-order consciousness may also shed light on the appeal of Type II approaches—namely, these methods clearly attempt to assess the latter, which some believe captures the primary meaning of consciousness. Relatedly, making this distinction could help resolve other longstanding controversies, such as whether animals are conscious. It may be, for example, that many animals possess phenomenal consciousness (and hence have subjective experiences of seeing, hearing, smelling, emotions, etc.), but that relatively few may additionally possess second-order access consciousness. Along these lines, it is perhaps not coincidental that many experimental tests of animal consciousness clearly attempt to assess second-order conscious processes (e.g. recognizing one’s self in a mirror). In any case, while important unresolved issues and controversies remain, it seems increasingly apparent that SDT may have much to tell us about not just perception but the nature of conscious and unconscious processes themselves.

References Banks, W. A. (2000). 'Recognition and source memory as multivariate decision processes'. Psychological Science, 11, 267–273. Block, N. (1995). 'On a confusion about a function of consciousness'. Behavioral & Brain Sciences, 18, 227–287. Block, N. (2005). 'Two neutral correlates of consciousness'. Trends in Cognitive Sciences, 9, 46–52. Dulany, D. E. (1997). 'Consciousness in the explicit (deliberative) and implicit (evocative)'. In J. D. Cohen and J. W. Schooler (eds), Scientific Approaches to Consciousness (pp. 179–212). Mahwah, NJ: Erlbaum. Erdelyi, M. H. (1974). 'A new look at the New Look: Perceptual defense and vigilance'. Psychological Review, 81, 1–25. Fechner, G. T. (1966). Elements of Psychophysics (Vol. 1), ed. E. G. Boring and D. H. Howes and trans. H. E. Adler. New York: Holt, Rinehart & Winston (original work published 1860). Galvin, S. J., Podd, J. V., Drga, V., and Whitmore, J. (2003). 'Type 2 tasks in the theory of signal detectability: Discrimination between correct and incorrect decisions'. Psychonomic Bulletin & Review, 10, 843–876. Gescheider, G. (1997). Psychophysics:  The fundamentals (3rd edn). Mahwah, NJ:  Lawrence Erlbaum Associates. Glanzer, M., Kim, K., Hilford, A., and Adams, J. K. (1999). 'Slope of the receiver operating characteristic in recognition memory'. Journal of Experimental Psychology: Learning, Memory, & Cognition, 25, 500–513.

Signal Detection Theory   733 Green, D. M. and Swets, J. A. (1966). Signal Detection Theory and Psychophysics. New York: John Wiley and Sons, Inc. Jacoby, L. L. (1991). 'A process dissociation framework: Separating automatic from intentional uses of memory'. Journal of Memory & Language, 30, 513–541. Macmillan, N. A. and C. D. Creelman. (1991). Detection theory: A user’s guide. New York: Cambridge University Press. Merikle, P. M., Smilek, D., and Eastwood, J. D. (2001). Perception without awareness: Perspectives from cognitive psychology. Cognition, 79, 115–134. Reingold, E. M. and Merikle, P. M. (1990). 'On the inter-relatedness of theory and measurement in the study of unconscious processes'. Mind & Language, 5, 9–28. Snodgrass, M. (2002). 'Disambiguating conscious and unconscious influences: Do exclusion paradigms demonstrate unconscious perception?' American Journal of Psychology, 115, 545–580. Snodgrass, M., Kalaida, N., and Winer, E. S. (2009). 'Access is mainly a second-order process: SDT models whether phenomenally (first-order) conscious states are accessed by reflectively (second-order) conscious processes'. Consciousness & Cognition, 18, 561–564. Visser, T. A. W. and Merikle, P. M. (1999). 'Conscious and unconscious processes: The effects of motivation'. Consciousness & Cognition, 8, 94–113. Weber, E. H. (1978). 'De subtilitate tactus'. In H. E. Ross and D. H. Murray (trans. H. E. Ross), E. H. Weber: The sense of touch. London: Academic Press (original work published 1834). Wundt, W. M. (1904). Principles of Physiological Psychology. New York: Macmillan (original work published 1874). Yonelinas, A. P. (1994). 'Receiver-operating characteristics in recognition memory: Evidence of a dual-process model'. Journal of Experimental Psychology: Learning, Memory, & Cognition, 20, 1341–1354.

Chapter 39

I n for m ation Theory John Kulvicki

The weather report carries information about the weather, the telegram carries information about the crime, and the angle of the sunflowers carries information about the location of the sun. A plume of black smoke carries information about the burning forest, in one context, or the Cardinals’ failed vote, in another. The ordinary notion of information is at home when one state of affairs tells us about another. The flowers’ angles depend on the position of the sun, and the pillar of smoke depends on fire, in a way that naturally allows us to learn about one on the basis of the other. They have a ‘natural meaning’ (Grice, 1957) that we can exploit if suitably tuned to the dependence. Telegrams, weather reports, and Sistine Chapel chimneys constitute the heights of artifice, but it’s hard to resist the view that, as informants, these deeply artificial signals share something important with tilting flowers and smoking forests. Ordinary representational practices serve as a model for the mind—they reflect it—and information theory is a way of trying to make that general model into a respectable theory. Information theorists suggest that perceptual states detect features of the environment by exploiting regularities like those that constitute instances of natural meaning. In an ambitious mood, they also suggest that such detectors constitute the ‘aboriginal instance of intentionality’ (Fodor, 1990b:  81)  from which the rest of the mind’s most impressive features can be derived. Perhaps, suitably adorned, Gricean natural meaning can figure centrally in naturalistic accounts of sensation, intentionality, belief, knowledge, and consciousness. More modestly, information theory might constitute a partial explanation or model of some mental phenomena, with the rest of the work being done some other way. Information-theoretic approaches to perception are also inspired by communication theory (Shannon and Weaver, 1949), which is widely considered to be a useful, objective way of understanding many of today’s most prominent technologies: the internet, computers, telephones. Communication theory focuses on amounts of information, noise, and equivocation, but it has little to say about what information a signal carries. A theory of ‘semantic information’ (Bar-Hillel and Carnap, 1953) uses the structure of communication theory as a model for our deeply semantic, ordinary talk of information. In Fred Dretske’s hands (1981), for example, the theory is developed with a keen eye on its potential to naturalize the mind and understand knowledge, but the end result bears a limited resemblance to communication theory. Further inspiration for information theory comes from observing the scientists who investigate the mind. Hubel and Wiesel’s (1962; Hubel, 1981) groundbreaking research

Information Theory   735 sought external features to which small regions of cats’ brains respond with enthusiasm, as a way of mapping out the functional architecture of their visual systems. Once, when trying to determine the sensitivity of a cell, a chance meeting of a researcher’s hand with the projector’s light produced a linear shadow at a specific orientation, and a surprise party in the cat’s brain. They had found a region of the brain that was preferentially sensitive to a specific orientation of a lightness gradient. Though they did not characterize it this way, the experiment gave good reason for believing that under ecologically ordinary circumstances parts of the cat’s brain naturally mean that there’s such a contrast boundary in the vicinity.1 Similarly, models of perception by David Marr (1982), Richard Gregory (1997), Irving Rock (1983), and others all start with the idea that something like indication is central to understanding how perceivers interact with the world.2 Recently, Luciano Floridi (2010) has suggested that we can develop a general philosophy of information, in terms of which many philosophical problems can be framed.3 This is an important and exciting development in the general theory of information, but little has been done in this vein specifically related to perception, and for that reason it will not be the focus of what follows. The next section looks at a number of attempts to articulate a coherent and naturalizable notion of information. Section 2 details some fundamental worries about the naturalizability and usefulness of these notions. Sections 3 and 4 review how one might deal with those worries, with particular focus on Dretske’s work, because it is the most articulate attempt to do so in the context of the philosophy of perception. Sections 5–7 consider a number of issues in philosophy of perception to which information theory has been applied: sensation and perception, perceptible qualities and consciousness, and the objects of perception. The final section considers the foundational question of how to work norms into the information-theoretic picture.

1  Conceptions of information Dretske, the most influential proponent of information theory, especially in relation to perception, suggests that we think about information in terms of conditional probabilities.4 Informational content: A signal r carries the information that s is F = the conditional probability of s being F, given r (and k) is 1 (but, given k alone, less than 1). (Dretske, 1981: 65) 1  I say ‘good reason’ because there is nothing even faintly ecologically ordinary about the circumstances under which such experiments take place. See specifically Hubel and Wiesel (1962:  114–15) and Hubel (1981: 27) 2  Gary Hatfield has done more than anyone else in philosophy to sort out how these theories relate to one another, how they are sourced historically, and what their prospects are. See especially the important collection of his work: Hatfield (2009). Nicoletta Orlandi (2011) looks at similar material from the perspective of embedded cognition, suggesting that the embedded view is superior. 3  See, e.g., Adams (2010) and, more generally, the issue of Metaphilosophy (vol. 41, no. 3) in which that article appears, for discussion of Floridi’s work. 4  Jon Barwise, John Perry, and David Israel (Barwise and Perry, 1983; Israel and Perry, 1990) develop a notion of information that is related to Dretske’s but not as explicitly geared toward problems in philosophy of perception, so it will not be the focus in what follows. See Adams (2003: 492–3) for discussion.

736   John Kulvicki The signal, r, is meant to be some state of affairs: an object instantiating some property, or some objects instantiating a relation. So, the sunflower’s pointing in a certain direction carries information that the sun is located in that direction just in case the probability that the sun is there, given the pointing of the sunflower, is 1. This definition also appeals to k, or what a perceiver already knows. Since the point of information theory is to show how a phenomenon that exists prior to perceivers is recruited as a constitutive part of perceivers, this is unhelpful (see Dennett, 1969: 186–7). Dretske realizes this and suggests that the definition is recursive: ‘Eventually, we reach a point where the information received does not depend on any prior knowledge about the source, and it is this fact that enables our formulation to avoid circularity’. (Dretske, 1981: 87) So, the pared-down version of the definition, which characterizes sunflowers and smoky fires, is something more like: r being R carries the information that s is F = the conditional probability of s being F, given r’s being R, is 1 (but, unconditionally, less than 1).5

The right hand side of this definition seems coherent and naturalistic, and it’s doubtless that some events satisfy the schema. Competing accounts of information are variations on this theme, designed to remedy the original’s perceived inadequacies. There is no consensus on the best formulation, or even on how all of the formulations relate to one another, so it pays to present a range of them. Dretske himself suggested that his schema can be interpreted as saying that there is ‘a regularity which nomically precludes [r’s being R] when s is not F’. (Dretske, 1981: 245; cf. Jacob, 1997: 52–3) That is to say, we think of the individual events in question as falling under kinds that are related in a lawlike manner. Barry Loewer unpacks this as: it’s a law that whenever an event of type R occurs, s is F.

(Loewer, 1983: 76)

Loewer rejects this proposal because ‘it’s quite implausible that there are laws connecting [for example] brain states and dogs (described as such)’. Avoiding laws, he proposes instead: r is R, and if r is R then s must have been (or must be) F.

(Loewer, 1983: 76)

Alternatively, r is R, and necessarily, if r is R then s is F.

Millikan suggests that for Dretske the signal and source must be related as a matter of ‘strict, natural necessity’ (Millikan, 2001: 106), and Jonathan Cohen and Aaron Meskin (2006: 338) agree that this is the way to explicate Loewer. This strict conditional implies two subjunctive conditionals, one of them a counterfactual: if r were R then s would be (or would have been) F, and if s were not F then r would not have been R. 5  In what follows I change others’ examples to conform to the rule that the information-carrying state of affairs is given by ‘r’s being R’ and the state of affairs about which information is carried is given by ‘s’s being F’, as per Loewer (1983). Savitt (1987: 188) calls this ‘absolute informational content’, while Millikan calls it ‘context-free information’. See also Millikan (2001: 106) and Taylor (1987: 101).

Information Theory   737 The first of these is relatively uninteresting because its antecedent is true, but a number of people focus on the second, counterfactual conditional as a way of understanding the information-carrying relation. So, r being R carries the information that s if F if and only if: r is R only because s is F, and r wouldn’t be R, were it not for the fact that s is F. (Stampe, 1977: 47) the generalization ‘s’s being F causes r to be R’ is true and counterfactual supporting. (Fodor, 1990a: 176; cf. Stampe, 1986: 130) the counterfactual conditional ‘if s were not F then r would not have been R’ is non-vacuously true. (Cohen and Meskin, 2006: 335)

Non-vacuous truth is just truth for some reason other than the antecedent being necessarily false (see Lewis, 1973: 560), and this stipulation does the same work as Dretske’s insistence that the probability of s’s being F not be 1 absent r’s being R. Controversy attends any suggestion for how probabilities, laws, and counterfactuals relate to one another, and any number of these proposals can collapse into one another depending on such views. Most proposals that respond to Dretske move away from his formulation in terms of conditional probabilities because, as Loewer (1983) pointed out, it is unclear that there is a notion of probability that can fill the shoes Dretske fashions for it. Cohen and Meskin (2006: 338), for example, suggest that their account comes quite close to Loewer’s, especially if one thinks that nomic necessities and counterfactuals can be understood in terms of one another, as Goodman (1983) suggested.6 Communication theory was an important inspiration for information-theoretic approaches to perception. Most of the foregoing proposals, however, bear limited relation to what inspired them.7 That is perhaps for the best, as Bar-Hillel and Carnap warned that simply transferring the notion of information from communication theory to a semantic context ‘may at best have some heuristic stimulating value but at worst be absolutely misleading’ (Bar-Hillel and Carnap, 1953: 148). In contrast to the trend just mentioned, Kenneth Sayre (1976, 1987) uses a notion of information that cleaves closely to communication theory, as does Austen Clark (1993: 50–2). Dretske thought that an information channel is only useful for doing explanatory work in philosophy of perception if it is without noise or equivocation. A  signal is noisy to the extent that it carries information about objects between the source and receiver, and it is equivocal to the extent that it fails to carry as much information as is generated at the source. When you hear features of your poor stereo system, as well as features of the recorded concert, the channel is noisy. If the recording is not accurate enough to tell you whether the singer said ‘Kick it!’ or ‘Chicken!’ the channel is equivocal. In everyday life we use noisy and equivocal channels to great effect. Sayre and Clark bet that it can be explanatorily helpful to think about perceptual systems in this way as well. More recently, Andrea Scarantino and Gualtiero Piccinini (2010) suggested that it can be helpful to consider information about s’s being F, as distinct from the information that s is F. The earlier kind 6  See (2008) for a response to Cohen and Meskin. Scarantino (2008) and Meskin and Cohen (2008) helpfully continue the discussion of the Cohen and Meskin proposal. 7  See Foley (1987: 166–167), Sayre (1983: 78), and even Dretske (1983).

738   John Kulvicki of information is what you get when r’s being R raises the probability that s is F, but doesn’t necessarily raise it to unity: P(s is F|r is R) > P(s is F|-(r is R))

This is what happens for communication channels that are noisy, equivocal, or both. You get the information that s is F when noise and equivocation disappear. Finally, Olimpia Lombardi (2005) has shown how one can (1)  fix technical problems with Dretske’s explication of information in terms of communication theory and (2) end up with a notion that is both close to communication theory and useful in the philosophy of perception. These conceptions of information are plausibly semantic in part because they exhibit some of the intentional features of bona fide meaningful states.8 For example, chance co-extensiveness is not counterfactual supporting, so a state that carries the information that x is a bird, does not carry the information that x weighs less than a pound, even if all of the birds around happen to weigh less than a pound. There is no counterfactual-supporting regularity that connects weighing less than a pound to being a bird. We will see later on how some more interesting features of intentionality emerge once we distinguish between primary and secondary information.

2  The frame problems The previous proposals identify naturalistic relations between states of affairs as being information-carrying relations. The notion is unhelpful, however, absent good reasons for thinking that events of interest stand in such relations. Sayre wondered whether ‘regularities of this sort are ever found in nature (apart perhaps from the isotropic laws of classical mechanics and particle physics)’ (Sayre, 1983: 79). Sunflowers track the sun through the sky, but fallibly. Pillars of smoke are excellent evidence for forest fires, but are they any more than that? Shadows stop sunflowers in their tracks and smoldering hibachis can mislead us about the state of the forest. Is the probability one that the sun is over there, given the sunflowers’ pointing? Do such events really satisfy a strict conditional, or the relevant counterfactual? In a sense, they do. For example, fix the ordinary environment in which one finds sunflowers, assume we’ve got some healthy flowers on our hands, with access to the sun throughout the day, and it becomes quite plausible that the probability they point towards the sun is one. Similarly, it’s likely that there is a nomic or counterfactual-supporting dependence of the flowers’ positions on the sun’s location Remarks about information are only plausible when framed appropriately, fixing certain things while allowing others to vary. Frames make trouble, however, by raising worries about the naturalization and usefulness of information. It’s easy enough to see how a conversation can fix a frame within which counterfactuals and claims about conditional probabilities are evaluated. Dretske himself defined 8 

See Dretske (1981: 75–76, 172f) and Enҫ (1982: 176–177).

Information Theory   739 information, in the first instance, relative to what perceivers already know. If information is to be a vehicle for naturalizing mental states, however, it cannot essentially be dependent on those mental states. It must be ‘an objective, mind-independent commodity’. (Adams, 2003: 476). The information theorist is ‘committed to the existence of non-doxastic facts of the matter about the distribution of probabilities and possibilities’.9 One kind of response is to say that there are many, many facts about counterfactual dependencies. They are all objective, in the sense that every frame fixes some fact about what carries information about what. What we do when we articulate and evaluate claims is select one or some from among the myriad frames available. Selecting a frame does not speak against its objectivity, any more than selecting a topic of conversation does. A serious worry about this kind of response, however, is that it trivializes the claim that some event carries information about some other event, thus rendering the notion unhelpful. Gilbert Harman made this worry quite salient, focusing on the tie Dretske articulates between information and knowledge: Since any cause r of a true belief that s is F carries the information that s is F relative to some framework (e.g. that consisting in the set of actual and possible situations in which either s is F or r does not occur), Dretske’s account of knowledge reduces at best to the trivial point that knowledge is true belief plus something else. (Harman, 1983: 72)

Even if you don’t accept that one can arbitrarily choose possible situations in the way Harman suggests, the fact remains that there are many frames to choose from. Ubiquity of frames suggests that information is ubiquitous as well, and thus not very useful.10 In addition, for the notion to be of use in understanding perception, information must flow. A perceptual state or sensation must be able to carry information about distal objects, even though they are at some causal remove from the perceiver. We see the man reflected in the mirror, so somehow, through the light reflecting off the man, then off the mirror and into our eyes, information about the man is preserved. Information flow requires chains of signals and sources, each of which is understood with respect to a certain frame. Can frames change at different points along the chain? Must they stay the same? Without some account of information flow, the notion might prove unhelpful, even though it is coherent.

3  Channel conditions Dretske realized that while his definition of the information-carrying relation only referred to states of the signal, r, and what a perceiver knows, k, typically ‘the relevant probability is conditional on a framework that includes more than r and k’ (Harman, 1983: 9 

Cohen and Meskin (2006: 344). See also Dretske (1981: vii), Scarantino (2008: 631), and Morris (1990: 378–379). 10  John Heintz (1990: 22) raises a similar worry about Israel and Perry’s (1990) development of their notion of information. Millikan (2004: 38–39) presses the point against Dretske. In a related vein, Ernest Sosa (1983: 80) suggested that there would be no non-circular way to fix the frames and establish Dretske’s goals, a point echoed more recently by Robert Cummins (1996: 54). Fodor (1984: 245f) presses the point against Stampe (1977: 50), who responds in Stampe (1986: 128f).

740   John Kulvicki 72). We also need a set of ‘channel conditions’ in place before states of r can carry information about states of s.11 Explaining perception, or perceptual content, by appeal to information is a complicated empirical enterprise. Candidate information carriers must be identified, along with the information they are supposed to carry. And then we have to find a channel. Let’s say we decide that certain states of the eye or primary visual cortex carry information about the orientations of contrast boundaries in the visual field. Instances of these neural states carry that information just in case there is a channel that renders the conditional probabilities as Dretske says they must be (Dretske 1981: 131), or that renders the relevant counterfactuals true, and so on. This is a helpful, and potentially explanatory, thing to say about the eye if those channel conditions obtain fairly stably across many situations in which perceivers find themselves.12 Channel conditions can fail, and bring about a perceptual failure. For example, the eye goes into the contrast-boundary-indicating state absent the contrast boundary, or, though there is such a boundary, the state in question does not carry that information. If it turns out that states of the eye do not generally carry any information of interest, perhaps because there are no channel conditions under which that information can flow, then the hypothesis that information is a useful tool for explaining perception fails. Information must flow farther than the eye, of course, and it is likely that the channel conditions for good information flow within the nervous system also have to be found. These considerations block the worry that such conditions are arbitrary, despite the fact that information remains ubiquitous. It’s hardly a trivial matter whether the states we single out as information carriers do so against a background of conditions like the ones in which a perceiver will typically find herself. A number of responses to information theory take the form that, even given considerations of channel conditions, information cannot play its designated role. For example, Dretske claims that ‘An information-gathering function, essential in most cases to the satisfaction of a biological need, can only be successfully realized in a system capable of occupying states that serve as natural signs of external (and sometimes other internal) conditions’ (Dretske, 1986:  25). Some doubt whether such a high bar need be set for satisfying an organism’s biological needs. Mechanisms that put perceivers in touch with their environments might be imperfect, even at the best of times, under the most ideal conditions. Millikan is the most prominent critic of these uses of information theory. She stresses the fact that many states that play the role of representing aspects of the environment can typically misfire (Millikan, 1984, 1989a). By focusing on how organisms use representations, Millikan suggests we can ‘cleanly bypass the whole genre of causal/informational accounts of mental content’ (Millikan, 1989a: 290) because their use is more helpful for determining content than any information such states might carry is. Kenneth Taylor (1987: 111–12) also worries that under many circumstances ‘hair-trigger’ mechanisms can

11 

Stampe (1977: 49) calls these ‘fidelity conditions’; and see Israel and Perry (1990: 4). (1981: 119); Stampe (1977: 52); cf. Millikan (2004: 38–39). Austen Clark, in line with Sayre (1976), suggests that information is a useful tool for understanding perception ‘only if the conditional probabilities linking neural events to stimuli depart markedly from a random distribution’ (Clark, 1993: 52 and see Clark, 2000, 112–113), but he does not insist that we need to find a channel without noise or equivocation, as Dretske does. 12  Dretske

Information Theory   741 prove quite beneficial to an organism, even if they are often incorrect.13 The obvious example of this is a mechanism that is supposed to represent danger. All things considered, it’s better to have a mechanism in place that tends toward false positives than one that allows for false negatives. Mohan Matthen claims that many if not most perceptual states ‘are imperfect but the best available, where [their] use confers an over-all advantage upon the organism, despite the occasional occurrence of error’ (Matthen, 1988: 13) even under normal circumstances. Kathleen Akins (1996) suggests that focusing on what sensory systems do for perceivers undermines one’s confidence that the function of the senses is to detect distal features in the first place.14

4  Nesting and primary information Isolating perceptual states and the states about which they carry information, and finding a channel that allows for information flow, are two important steps in an informationtheoretic account of perception. Such facts alone do not yet give us the contents of representational mental states, however, or anything like what Grice called ‘non-natural meaning’ (Grice, 1957: 378). What’s more, they do not allow us to identify the objects we perceive in information theoretic terms. Any state that carries the information that we would like to identify with its representational content also carries information that we are reluctant to identify with its content. Notice that any state carrying the information that s is F also carries the information that s is F-or-G, for any choice of G.15 This is easy to see because being F entails being F-or-G. So, if the conditional probability is one that s is F given r’s being R then so is the conditional probability of s’s being F-or-G. This is a problem because if it’s plausible to identify a mental state as being about greenness, it’s likely implausible to identify it as being about greenness-or-squareness. Since both are accurate characterizations of information carried by the state, we need a way to choose between them.16 In addition, we ought to expect that given the channel conditions, any candidate perceptual state carries information about many other states of affairs. A leaf’s being green carries the information that it contains chlorophyll, as well as the information that there is a water-source nearby. If there’s a dandelion in the garden, there will be more, and there have been others. Any reasonable statement of natural laws and standard conditions will

13 

Godfrey-Smith (1989: 547), following Millikan, makes a similar point. See also Dretske (1988: 90n). In more recent work, Matthen (2005) argues for a kind of ‘pluralistic realism’ that does not, as Akins suggests, undermine the claim that the function of the senses is to detect distal features. Rather, his view suggests, plausibly, that the diversity of creatures and ecological niches will yield an impressive diversity of ways in which such features are tracked and used to motivate action. 15  Aside from anything that entails being not-F. In that case, we have a condition that necessarily obtains, and as shown in section 1, most information theorists suggest that you cannot carry information about necessarily obtaining states of affairs. 16  It would be wrong to say that the state carries more information than we are interested in because it carries the information that s is F and the information that s is F-or-G. The latter adds nothing to the original information: it’s merely an incomplete characterization of the signal’s information. (Cf. Adams, 2003: 478.) The next point suggests that a signal typically carries too much information. 14 

742   John Kulvicki Table 39.1 Two ways of carrying the same information about two different objects Case 1:

r is R

carries the information that

s is F and t is G

Case 2:

r is R s is F

carries the information that carries the information that

s is F t is G

require that green leaves have chlorophyll, they can’t stay green and lively without access to water, dandelions procreate, and so on. ‘In general, if there is a natural law to the effect that whenever s is F, t is G, . . . then no signal can bear the message that s is F without also conveying the information that t is G’ (Dretske, 1981: 71). Under these circumstances, Dretske says that the information that s is F nests the information that t is G.17 So, any signal that carries some information that we think can play an important role in helping to explain perception also carries less determinate information and, most likely, more determinate information, and neither of these is terribly explanatorily useful. Nesting is supposed to help single out the interesting pieces of information, but it’s a somewhat subtle affair how this happens. Not much has been written about problems with understanding nesting, despite its significance to whether information theory is a helpful tool for understanding perception. Fundamentally, nesting allows one to distinguish two ways in which r’s being R can carry the information that s is F and t is G. Table 39.1 helps what will be unavoidably awkward prose. In both cases, r’s being R carries the information that s is F and t is G. In both cases, therefore, the information that t is G is nested in the information r carries. In Case 1, however, there is no information-carrying relation of interest between s and t. So, r’s being R is sensitive to s and t independently. In Case 2, the information that t is G is itself nested in the information that s is F. So, r’s being R carries the information that t is G only insofar as it carries information about s, and it follows too that r carries information about both s and t only insofar as it carries information about s. For these reasons, we can say that, in Case 2, r carries the information that s is F in a primary way, while it carries the information that t is G, and even the conjunctive information that s if F and t is G, in a secondary way. In Case 1, r’s being R carries the information that s is F and t is G in a primary way, while it carries information about each conjunct in a secondary way (see Dretske, 1981: 160f). 17 

Well, that’s not quite the way Dretske unpacks nesting. He says: ‘The information that t is G is nested in s’s being F = s’s being F carries the information that t is G’ (Dretske, 1981: 71). It’s true that, if there is a natural law connecting s’s being F and t’s being G, then the former can carry information about the latter and any state that carries the information that s is F also carries the information that t is G. But if we adopt Stampe’s or Cohen and Meskin’s suggestion that information is a matter of counterfactual dependence, then we cannot be assured of the latter point, since counterfactuals are not transitive. Perhaps if r were not R, then s would not be F, and if s were not F, then t would not be G, but we cannot conclude on that basis that if r were not R, then t would not be G. Cohen and Meskin’s account of information denies Dretske’s ‘Xerox principle’, to the effect that ‘If A carries the information that B, and B carries the information that C, then A carries the information that C’ (Detske, 1981: 57), They agree, however, that nesting can be an important tool for understanding information flow, so the presentation above is fashioned to be consistent with their proposal (see Millikan, 2004: ch. 4). It’s probably not true that ‘any account of the information carried by a signal must preserve the validity of the Xerox principle’ (Dretske, 1981: 58). Kistler (2000) suggests that we stick with Dretske’s original conception of information in terms of probabilities in order to preserve the Xerox principle.

Information Theory   743 Primacy, notice, is not a matter of whether one piece of information is more determinate than another. That s is F and t is G says more about the world than the claim that s is F, but that it not enough to render the conjunctive information that which the signal carries in a primary way. What matters are whether there are dependencies, articulated along the lines of nesting, among the many pieces of information a signal carries. Facts about primacy and nesting allow one to single out some pieces of information as special, and the information theorist’s bet is that those special pieces of information play an important explanatory role in understanding perception. Though Dretske does not do this, we can extend this reasoning about primary and secondary information to cases involving a single object. To see this, consider an example of Dretske’s, evocative of Lenin and Danto: r’s being R carries the information that x is a red square (Table 39.2).18 Now consider two cases, structurally similar to the ones discussed above. In both cases, r’s being R carries the same information about x—x is a red square—but these signals differ in the information they carry in a primary way. In Case 3, the signal only carries the information that x is square in virtue of carrying the information that x is a red square, so the latter piece of information is carried in a primary way, while the information that x is square is carried in a secondary way. In Case 4, r’s being R is a more complicated state of affairs. There are qualities, P and Q, which are constitutive of R: R = P&Q. r’s being P carries the information that x is red while r’s being Q carries the information that x is square. In this case, the signal does not carry the information that x is square in virtue of carrying the information that x is a red square: r’s being Q carries the information that x is square, but not the information that x is a red square. Are there pieces of information that the Case 4 signal carries in a secondary way? None of interest. As we saw earlier, all signals that carry the information that s is F also carry the information that s is F-or-G, so in that sense the Case 4 signal carries many pieces of information in a secondary way. More generally: A signal, r, carries the information that s is F in a primary way just in case there is some R such that





(1) r being R carries the information that s is F, (2) there is no G (distinct from F) such that r being R carries the information that s is G and s being G carries the information that s is F, and (3) there is no t, such that t being H carries the information that s is F and r being R carries the information that t is H.19

In addition: The total primary information carried by r being R is the collection of every piece of information that r being R carries in a primary way.

18 

Dretske (1981: 180). See also Enҫ (1982: 178). Kulvicki (2004, 2005) unpacks a condition like this in terms of whether some piece of information is ‘extractable’ from a signal, and then suggests that the primary information carried by a signal shares a structure with the properties about which information is carried in a primary way. Stampe seems to have had something like this in mind too, when he claimed: 19 

The causal relation we have in mind is one that holds between a set of properties F(f 1 . . . f n) of the thing (s) represented and a set of properties R(r 1 . . . rn) of the representation (r). The specification of sets

744   John Kulvicki Table 39.2 Two ways of carrying the same information about the same object Case 3:

r is R

carries the information that

Primary information:

x is a red square

Secondary information:

x is red

x is a red square

x is square Case 4:

r is P

carries the information that

x is red

r is Q

carries the information that

x is square

R = P&Q Primary information:

x is red x is square x is red and square (i.e. x is a red square)

Secondary information:

None of interest, but, e.g.: x is red or a gira_e

Cases 1–4 highlight different ways signals can code information. These coding strategies are significant for understanding the usefulness of information theory for understanding perception. As a start, notice that the distinction between primary information and secondary information allows some states to exhibit a rather interesting kind of intentionality. At the end of section 1, we saw that it is possible to carry the information that x is a bird, but not the information that x weighs less than one pound, even if, by chance, those qualities are coextensive. But the formulations in section 1 suggest that any signal carrying the information that x is a bird also carries the information that x has wings, because there is a nomological relation between these two states. But we can distinguish a signal that carries the information that x is a bird and the information that x has wings in a primary way from one that merely carries the information that x has wings in a secondary way. This gives us the room to say that some state represents being a bird, but not having wings, even though the earlier piece of information nests the latter. This even applies to more demanding necessities. It is analytically necessary that squares have corners, but a state can carry the information that x is square in a primary way without carrying the information that x has corners in a primary way.20 F and R will consist of predicates that identify certain constituents, and certain relations among these constituents, of s and r respectively. The relevant causal relationship between s’s being F and r’s being R will be one that normally, if not necessarily, preserves an isomorphism between the structures thus defined. (Stampe, 1977: 46)

It’s often suggested that there is no interesting work for resemblance or shared structure to do once one thinks about representation in causal or informational terms. It’s the causal/informational relationships that do the work identifying what is represented, not resemblances or isomorphisms. (See, e.g., Lewis (1980: 239) and Mackie’s (1976: 14) interpretation of Locke’s resemblance talk.) It’s an open question, however, whether such isomorphisms can do some explanatory work in the philosophy of perception (see Kulvicki, 2004, 2005, and, for different takes on the matter, see Cummins, 1996 and Millikan, 2004: ch. 4). 20  See Dretske (1981: 177–178).

Information Theory   745

5  Sensation and perception Look at the office and you see the books, the stereo set, the lamp, window, prints, and so on. Information about their colours, shapes, and locations is readily available. Dretske’s picture is that a sensation carries information about all of these objects in a primary way. Sensations, as understood here, have impressively rich representational contents, in that they represent a great many features of a great many things. And these contents are quite detailed, in that the most specific piece of information about any given object is typically quite specific. Sensations present us with specific shapes, colours, illumination profiles, relative locations, and orientations of the objects we see.21 What’s more, they make less detailed information about these objects readily available as well. It’s just as easy to learn that something is red, just by looking, as it is to learn that something is scarlet, just by looking.22 Sensations also carry information about a great many other objects in a secondary way, but that information is not a candidate for being the content of the sensation. Dretske bets that a total primary characterization of a sensation’s information will yield something corresponding to its representational content. Perceivers, as Jerry Fodor put it, ‘so represent the world as to make it available to thought’ (Fodor, 1983: 40, emphasis removed). Aspects of sensations’ rich, detailed information are used by perceivers pursuing their goals. For Dretske, perceivers can use aspects of this information to the extent that they can form new states based on sensations, which carry only part of the sensations’ information. The sensation carries the information that s is a red square, and much else besides, but one forms a belief about s to the effect that it is red. This perceptual belief carries the information that s is red in a primary way, but not the information that s is square, to the left of the blue triangle, or anything else. Dretske calls the process by which one extracts some part of the primary information a signal carries digitization, the idea being that these parts are carried by the sensation in nested, or analogue, form, while the belief carries the information in digital form (Dretske, 1981: 136–9 and 178f).23 How do perceivers manage this feat of abstraction? Recall that carrying information in a primary way requires there to be some feature of the signal, in this case the sensation, that carries the information that s is red to the exclusion of the information that s 21  See,

e.g., Evans (1982: 227), Dretske (1995), Heck (2000), and Tye (1995, 2000, 2006a) for ways of articulating the claim that sensation states have contents that are non-conceptual, rich, and specific. This line of thought runs contrary to suggetions by Armstrong (1961), Dennett (1969), and Pitcher (1971) to the effect that there is no need to talk about perceptual content distinct from the contents of perceptually formed beliefs. See also Wayne Wright, Chapter 10, this volume. 22  Kulvicki (2007a) suggests that one virtue of information theory is that it readily handles this ‘vertical articulateness’ of the contents of sensations. A total, primary characterization of a sensation’s content will typically involve qualities across levels of abstraction—x is coloured and x is red and x is scarlet, etc.— since it’s possible for each of those qualities to be their own parts of the primary information carried by a perceptual state. 23 Élisabeth Pacherie (2000) argues convincingly that information theorists can, and ought to, articulate more levels of content than sensations and cognitive states like beliefs. She draws some evidence for her claim from discussions of high-level vision, expecially Ullman (1996). See also Pacherie (1995). Prinz (2006, 2007, 2012) suggests that such intermediate levels of representation are key to understanding consciousness.

746   John Kulvicki is a red square. If the signal carries the information that s is red in a primary way, then it carries that information, but not solely in virtue of carrying some other piece of information. Some state subsequent to the sensation can be sensitive to the information that s is red because it can be sensitive to whatever feature of the sensation is responsible for carrying that information, and nothing else. ‘Hence, of all the information carried by an incoming signal, a semantic structure is causally sensitive to a unique component of this incoming information’ (Dretske, 1981: 180, emphasis added, and see 138; cf. Yourgrau, 1987: 240f). And this, incidentally, shows why the total primary information carried by a sensation is a good candidate for being its representational content: it amounts precisely to the information that the sensation makes available to thought.24

6  Perceptible qualities and consciousness Information-theoretic approaches to representation do not in and of themselves entail accounts of consciousness or perceptible qualities. They do, as the previous sections show, articulate fairly rich conceptions of mental representation, which can be deployed in the service of understanding these issues. What’s more, there is a close link between understanding phenomenal consciousness and understanding the qualities we perceive. Colours, sounds, smells, and the other traditional secondary qualities are precisely those cited in debates about the nature of conscious experience.25 For the information theorist, these problems concern the information perceptual states carry, and how they make such information available to thought. They concern, that is, what we have been calling the total primary information carried by sensations and how that information is taken up in the formation of beliefs.26 Information theorists tend to favour two approaches to these issues. First, they conceive of perceptible qualities as the most straightforward features about which information can be carried: physical qualities of the objects that seem to have them. This is controversial when it involves the paradigmatically secondary qualities, like colours, smells, and audible qualities.27 Second, they hope to explain phenomenological aspects of perceptual experiences in terms of what they represent. Dretske (1995, 2003), Tye (1995, 2000), 24 

Tye (1995, 2000) defends an information-theoretic account of the contents of sensations, and insists that such contents be ‘poised’ or, ‘ready and available to make a direct impact on beliefs and/or desires’ (Tye, 2000: 62). While Tye describes what poise amounts to, he doesn’t use notions of primary and secondary information to explain how a part of a state’s content might be poised. Prinz (2006) expands the Dretskean approach to the sensation/perception distinction. 25  Alex Byrne (2006) suggests that there is just one problem here, which he identifies broadly as reconciling the manifest and scientific images (Sellars, 1962). 26  This specific way of understanding the problem is, unsurprisingly, most obvious in Kulvicki (2005, 2007b, 2010). This section will be fairly summary in character, as the other entries in this companion cover similar ground, especially those on consciousness (Hellie), secondary qualities (Ross) and all of those on perceptible qualities (sections III and IV). 27  Hilbert (1987) and Byrne and Hilbert (2003) defend an account of colours according to which they are dispositions to reflect or transmit light, and thus ordinary physical qualities of the objects that seem to have them. Casey O’Callaghan (2007) and Kulvicki (2008) suggest competing approaches to understanding audible qualities, which are both sympathetic to Byrne and Hilbert’s physicalism.

Information Theory   747 Aydede and Güzeldere (2005), and Prinz (2012) all offer detailed, representationalist, information-theoretic accounts of perceptual consciousness. They all also endorse physicalist accounts of the colours, with Tye (2000) developing his own version of colour physicalism. By contrast, Austen Clark (1993) develops a view of sensory qualities and consciousness (Clark, 2000) indebted to a more communication-theoretic version of information theory, Nelson Goodman’s (1972, 1977) work on the structure of appearance, and Peter Strawson’s (1963) work on feature placing. He explains phenomenal consciousness locally, in terms of relations between perceptual states, rather than in terms of the contents of such states, and he is not committed to colour physicalism. Tye (1990, 2006b) also offers a representationalist, information-theoretic account of pains, while Aydede (2006, 2009) offers an account of pains that is sympathetic with information theory while not being fully representationalist. Prinz (2004) offers a Jamesian account of the emotions within an information-theoretic framework. And Clare Batty (2010, 2011) offers an account of smelling in representationalist terms that is sympathetic to information theory. Increased interest in perceptual and phenomenal states beyond vision promise more opportunities to articulate information-theoretic approaches to perception.

7  Primacy and perceived objects Primacy accomplishes two feats at once. It says something about the range of features about which information is carried in an available form, ready for extraction in belief formation. And it says something about the objects about which one readily forms perceptual beliefs. Sensations carry information about the ordinary objects in our environment, the ones we usually take ourselves to perceive: Austin’s ‘moderate-sized specimens of dry goods’ (Austin, 1962: 8). If primary information is to accomplish this aim, it had better be those ordinary objects about which perceptual states carry information.28 Section 4 suggests that any perceptual state carrying information about the dandelion also carries information to the effect that there are other dandelions around, that the dandelion’s ancestors lived and procreated in the area, and so on. The perceptual state carries primary information about the dandelion seen, however, because it only carries the other information in virtue of carrying information about that dandelion. Such asymmetries are key to isolating the information plausibly identified with representational content. But such a proposal only works if it is also true that perceptual states do not carry information about dandelions solely in virtue of carrying information about more proximal states of affairs. If the total primary information carried by a perceptual state amounts to information about the array of light reaching the eye, for example, or a pattern of depolarization 28  The

problem of identifying the objects of perception is given a solution in terms of standing and differential causes by Price (1950: 70), which is undermined in good measure by Grice (1961: 142–3) and Kim (1977: 612–613). Dretske sees information as a way of doing what mere causation cannot (Dretske, 1981: 156–60). This problem is also discussed in information-theoretic terms by Stampe (1977: 44), as the problem of ‘diachronic selectivity’, and by Lloyd (1987: 35–41).

748   John Kulvicki and hyperpolarization in the retinal photoreceptors, then primary information fails to identify what we take the contents of perceptual states to be. Identifying the objects of perception is a delicate balancing act. Either the reason for which one moves in toward the perceiver gives out precisely at the perceived object, or that reason is trumped by another principled consideration that moves one out from the proximal ‘irritations of our sensory surfaces’ (Quine, 1976: 253). Dretske (1981: 162–8) thinks the reason for moving out gives out. While a perceptual state carries information about the dandelion, it does not carry any information about intervening states of the light, ocular neurons, or what have you. These states support the flow of information from dandelion to sensation, but they support it in a way that keeps them out of the informational picture. Perceptual constancies are key to this proposal. We recognize the grass as being a certain shade of green, for example, across a range of lighting conditions. These lighting conditions mean, among other things, that the spectral composition of light reaching the eye is different across the range of circumstances in which that colour of green is recognized. So, the belief that results is insensitive to the varied routes that lead to it, and as such the belief carries no information about which route was taken. The most proximal object about which information is carried turns out to be the object we intuitively thought was the one we see. An objection, articulated by Carl Ginet (1983: 69), Loewer (1987: 298), Dan Lloyd (1987: 40–1), and John Haugeland (1996: 271–2), suggests that the primary information carried by a sensation does not concern the ordinary objects Dretske (1981) hoped it did. Even if the end state does not carry information about which of a number of routes were taken, it does carry the information that one or another of them was. If we can’t avoid the conclusion that perceptual states carry information about states more proximal than the perceived object, then we can’t avoid the conclusion that we perceive those more proximal objects. Dretske (1983) sketched a response, but never fully addressed the problem as it applies to the representational contents of sensations.29 In a later work (Dretske, 1986) he suggested a way of handling the problem for the objects of beliefs, but not for sensations. Abandoning constancy mechanisms, presumably because they are easily captured by the objection, Dretske suggests that sufficiently complicated creatures can set up information-carrying relations that have indefinitely varied intermediaries. He imagines, in other words, that some creatures can engage in associative learning: ‘we have a cognitive mechanism that not only transforms a variety of different sensory inputs . . . into one output determining state . . ., but is capable of modifying the character of this many-one mapping over time’. (Dretske, 1986: 35) Because, over time, the intermediaries change, the only stable state about which information is carried in a consistent manner is the distal one. This response threatens to undercut many of Dretske’s claims to the effect that our sensory states represent distal objects. On this view, they do not, absent a response to the Ginet–Loewer objection, and only more sophisticated, cognitive states do. Any 29  Fred

Adams (2003: 490–491) suggests that the perceived object is the first one about which ‘nondisjunctive’ information is carried, but any piece of information can be characterized in disjunctive and non-disjunctive terms. Lloyd (1987: 44–48) suggests that multiple channels of information, say from different eyes, from the eyes and the ears, and so on, might allow triangulation on an object which happens to be the object perceived.

Information Theory   749 animals lacking these more sophisticated abilities likewise fail to represent distal objects perceptually.30

8  Norms and content The total, primary information carried by a sensation is a good candidate for being its representational content, and it is at least plausible that the same can be said for perceptually generated beliefs. That is to say, total primary information matches our sense of perceptual states’ contents fairly well, and it exhibits aspects of intentionality characteristic of representational states. Unadorned, however, total primary information does not account for what makes such states representations in the first place. Encounters with red squares, under good conditions, plausibly result in states whose primary information involves a red square, but mistaken encounters with red squares and imaginings, considerings, and other kinds of entertaining of red squares do not. All such situations involve states that represent red squares, but mistakes involve the violation of some norm, while imaginings involve satisfaction of some other norm (see Fodor, 1990a: 180). The frames for understanding information flow—channel conditions—do not explain how a state tokened afoul of the frames represents what it does. Also, idle thoughts about red squares can be tokened even while the channel conditions are in place, but there are no red squares around. In all such cases, the red square state fails to carry the information that determines its representational content. These worries are foundational, in the sense that they concern the link between the candidate for determining representational content and representation itself. The obvious thing to do at this point: punt. Suggest that sensations, perceptual beliefs, and thoughts are norm-governed in such a way that facts about the information some of them carry determine the representational contents of the rest of them. Stampe (1977) was clear at the outset that information by itself is not a solution to the problems of perception, absent some accompanying thoughts about norms, or the functions of the systems that use information. Perhaps, for example, some sensations and beliefs have the function of carrying information, but in some circumstances they fail to do so. Dretske claims that ‘the occurrence of misrepresentation depends on there being some principled, non-arbitrary way of saying what the indicator function of a system is’ (Dretske, 1988: 70). Matthen, though more on Millikan’s side than Dretske’s, suggests that ‘a perceptual state with internal characteristic S is a presentation of property F if and only if its function is to detect the presence of a thing with property F’.31 These responses naturally require an account of what it is for a state of some creature to have the function of indicating something else. 30 

See also Aydede and Güzeldere (2005: 213–14). Jacob (1997: 101–3) suggests that this problem does not admit of an information-theoretic solution; he thinks that this is where some appeal to teleology must be made. Cf. Haugeland (1996) and Cummins (1989: 74–5) for different responses. 31  Matthen (1988: 20). See also Enҫ (1982: 181). Hardcastle (1994) suggests ways of filling in the story about functions, informed by learning theory and neuroscience. More generally, the literature on fitting functions into the natural world is vast. For discussion specifically related to the issues discussed here, see Millikan (1984, 1989b, 1990) and Neander (1991, 1995).

750   John Kulvicki Such responses to perceptual error, even if articulated in convincing detail, do not carry over neatly to cases in which states are tokened, as it were, off-line: idly in thought or conversation. Imaginings of pink elephants are to be encouraged, even though hallucinations of them are not. Imaginings do not have the function of carrying information, but they do have representational content. There is nothing wrong with thinking about things in their absence. This worry led Fodor to claim ‘It wouldn’t be very misleading to say that [information theory] gives us at best a naturalistic theory of representation in perception, but no theory at all of representation in thought’ (Fodor, 1990a: 181). Fodor’s well-known suggestion about asymmetric dependence is fashioned to remedy this problem: ‘ “Cow” means cow because but that “cow” tokens carry information about cows, they wouldn’t carry information about anything’ (Fodor, 1990b: 91, original emphasis). A mental state can be caused in a number of ways, but only some of them amount to it being caused by what we take to be its content. Only sometimes do actual cows cause them to think about cows. But those kinds of causes enjoy a kind of explanatory advantage over the others. It is in virtue of cows causing ‘cow’ tokens that other things, like water buffalos and trompe l’œil images of cows, cause them. Dretske (1994) endorsed a related idea, by appeal to channel conditions. When channel conditions obtain, and someone is perceiving the world, cows cause ‘cow’ tokens. And tokens of that type inherit their representational content from that fact. Other contexts in which such tokens are caused, whether in error or in imagination, inherit their contents from those privileged circumstances. Jesse Prinz (2002: 249–51) appeals to a variant of this strategy. Whether these approaches to such foundational issues work is still a matter of controversy. It is notable, and has been noted by Godfrey-Smith (2006), that the literature suggests a waning of interest in these foundational issues, at least for now.

References Adams, F. (2003). 'The informational turn in philosophy'. Minds and Machines, 13, 471–501. Adams, F. (2010). 'Information and knowledge á la Floridi'. Metaphilosophy, 41(3), 331–344. Akins, K. (1996). 'Of sensory systems and the "aboutness" of mental states'. Journal of Philosophy, 93(7), 337–372. Armstrong, D. (1961). Perception and the Physical World. London: Routledge and Kegan Paul. Austin, J. L. (1962). Sense and Sensibilia, ed. G. J. Warnock. London: Oxford University Press. Aydede, M. (ed.) (2006). Pain: New Essays on its Nature and the Methodology of its Study. Cambridge, MA: MIT Press. Aydede, M. (2009). 'Is feeling pain the perception of something?' Journal of Philosophy, 106(10), 531–567. Aydede, M. and Guzeldere, G. (2005). 'Cognitive architecture, concepts, and introspection: An information-theoretic solution to the problem of phenomenal consciousness'. Noûs, 39(2), 197–255. Bar-Hillel, Y. and Carnap, R. (1953). 'Semantic information'. British Journal for the Philosophy of Science, 4(14), 147–157. Barwise, J. and J. Perry. (1983). Situations and Attitudes. Cambridge, MA: MIT Press. Batty, C. (2010). 'A representational account of olfactory experience'. Canadian Journal of Philosophy, 40(4), 511–538.

Information Theory   751 Batty, C. (2011). 'Smelling lessons'. Philosophical Studies, 153(1), 161–174. Byrne, A. (2006). 'Color and the mind–body problem'. Dialectica, 60(3), 223–244. Byrne, A. and Hilbert, D. (2003). 'Color realism and color science'. Behavioral and Brain Sciences, 26, 3–21. Clark, Austen (1993). Sensory Qualities. Oxford: Clarendon Press. Clark, Austen (2000). A Theory of Sentience. Oxford: Oxford University Press. Cohen, J. (2008). 'Color constancy as counterfactual'. Australasian Journal of Philosophy, 86, 61–92. Cohen, J. and Meskin, A. (2006). 'An objective counterfactual theory of information'. Australasian Journal of Philosophy, 84(3), 333–352. Cummins, R. (1989) Meaning and Mental Representation. Cambridge, MA: MIT Press. Cummins, R. (1996). Representations, Targets, and Attitudes. Cambridge, MA: MIT Press. Cummins, R. and Poirer, P. (2004). 'Representation and indication'. In P. Staines and P. Slezak (eds), Representation in Mind (pp. 21–40). Oxford: Elsevier. Demir, H. (2008). 'Counterfactuals vs. conditional probabilities: a critical analysis of the counterfactual theory of information'. Australasian Journal of Philosophy, 86(1), 45–60. Dennett, D. (1969). Content and Consciousness. London: Routledge and Kegan Paul. Dretske, F. (1972). 'Contrastive statements'. Philosophical Review, 81, 411–437. Dretske, F. (1981). Knowledge and the Flow of Information. Cambridge, MA: MIT Press. Dretske, F. (1983). 'Responses to Critics'. Behavioral and Brain Sciences, 6, 82–89. Dretske, F. (1986). 'Misrepresentation'. In R. Bogdan (ed.), Belief (pp. 17–36). Oxford: Oxford University Press. Dretske, F. (1988). Explaining Behavior. Cambridge, MA: MIT Press. Dretske, F. (1994). 'If you can’t make one you don’t know how it works'. Midwest Studies in Philosophy, 19, 468–482. Dretske, F. (1995). Naturalizing the Mind. Cambridge, MA: MIT Press. Dretske, F. (2003). 'Experience as representation'. Philosophical Issues, 13(1), 67–82. Enҫ, B. (1982). 'Intentional states of mechanical devices'. Mind, 91, 161–182. Evans, G. (1982). Varieties of Reference, ed. J. McDowell. Oxford: Clarendon Press. Floridi, L. (2010). The Philosophy of Information. Oxford: Oxford University Press. Floridi, L. (2011). 'Semantic conceptions of information'. In E. Zalta (ed.), Stanford Encyclopedia of Philosophy. Stanford, CA: Stanford University Press. Fodor, J. (1980). 'Methodological solipsism considered as a research strategy in cognitive psychology'. Behavioral and Brain Sciences, 3(1), 63–73. Fodor, J. (1983). The Modularity of Mind. Cambridge, MA: MIT Press. Fodor, J. (1984). 'Semantics, Wisconsin style'. Synthese, 59(3), 231–250. Fodor, J. (1990a). 'Information and representation'. In P. Hanson (ed.), Information, Language, and Cognition (pp. 175–190). Oxford: Oxford University Press. Fodor, J. (1990b). A Theory of Content and Other Essays. Cambridge, MA: MIT Press. Foley, R. (1987). 'Dretske’s "information-theoretic" account of knowledge'. Synthese, 70, 159–184. Ginet, C. (1983). 'Four difficulties with Dretske’s theory of knowledge'. Behavioral and Brain Sciences, 6, 69–70. Godfrey-Smith, P. (1989). 'Misinformation'. Canadian Journal of Philosophy, 19(4), 533–550. Godfrey-Smith, P. (2006). 'Mental representation, naturalism, and teleosemantics'. In G. Macdonald and D. Papineau (eds), Teleosemantics (pp. 42–68). New York: Oxford University Press. Goodman, N. (1972). Problems and Projects. Indianapolis: Hackett. Goodman, N. (1977). The Structure of Appearance, 3rd edn. Boston: Dordrecht Reidel.

752   John Kulvicki Goodman, N. (1983). Fact, Fiction, and Forecast, 4th edn. Cambridge, MA:  Harvard University Press. Gregory, R. L. (1997). Eye and Brain:  The Psychology of Seeing, 5th edn. Princeton, NJ: Princeton University Press. Grice, H. P. (1957). 'Meaning'. Philosophical Review, 66(3), 377–388. Grice, H. P. (1961). 'The causal theory of perception'. Proceedings of the Aristotelian Society, Supp. 35, 121–152. Hardcastle, V. G. (1994). 'Indicator semantics and Dretske’s function'. Philosophical Psychology, 7(3), 367–382. Harman, G. (1983). 'Knowledge and the relativity of information'. Behavioral and Brain Sciences, 6, 72. Hatfield, G. (2009). Perception & Cognition. Oxford: Clarendon Press. Haugeland, J. (1996). 'Objective perception'. In K. Akins (ed.), Perception (pp. 268–289). New York: Oxford University Press. Heck, R. (2000). 'Non-conceptual content and the "space of reasons" '. The Philosophical Review, 109(4), 483–523. Heintz, J. (1990). 'Comment: What can a pencil sharpener know?' In P. Hanson (ed.), Information, Language, and Cognition (pp. 20–28). Oxford: Oxford University Press. Hilbert, D. (1987). Color and Color Perception. Stanford, CA: CSLI. Hubel, D. (1981). 'Evolution of ideas on the primary visual cortex, 1955–1978: A biased historical account'. Nobel Lecture accessed 20 May 12. Hubel, D. and Wiesel, T. (1962). 'Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex'. Journal of Physiology, 160, 106–154. Israel, D. and Perry, J. (1990). 'What is information?' In P. Hanson (ed.), Information, Language, and Cognition (pp. 1–19). Oxford: Oxford University Press. Jacob, P. (1997). What Minds can Do. Cambridge: Cambridge University Press. Kim, J. (1977). 'Perception and reference without causality'. Journal of Philosophy, 74(10), 606–620. Kistler, M. (2000). 'Source and channel in the informational theory of mental content'. Facta Philosophica, 2, 213–235. Kulvicki, J. (2004). 'Isomorphism in information-carrying systems'. Pacific Philosophical Quarterly, 85(4), 380–395. Kulvicki, J. (2005). 'Perceptual content, information, and the primary/secondary quality distinction'. Philosophical Studies, 122(2), 103–132. Kulvicki, J. (2007a). 'Perceptual content is vertically articulate'. American Philosophical Quarterly, 44(4), 357–369. Kulvicki, J. (2007b). 'What is what it’s like? Introducing perceptual modes of presentation'. Synthese, 156, 205–229. Kulvicki, J. (2008) 'The nature of noise'. Philosophers’ Imprint, 8(11), 1–16. Kulvicki, J. (2010). 'Introspective availability'. Philosophy and Phenomenological Research, 80(1), 208–228. Lewis, D. (1973). 'Causation'. Journal of Philosophy, 70(17), 556–567. Lewis, D. (1979). 'Counterfactual dependence and time’s arrow'. Noûs, 13(4), 455–476. Lewis, D. (1980). 'Veridical hallucination and prosthetic vision'. Australasian Journal of Philosophy, 58(3), 239–249. Lloyd, D. (1987). 'Mental representation from the bottom up'. Synthese, 70(1), 23–78.

Information Theory   753 Loewer, B. (1983). 'Information and belief'. Behavioral and Brain Sciences, 6, 75–76. Loewer, B. (1987). 'From information to intentionality'. Synthese, 70, 287–317. Lombardi, O. (2005). 'Dretske, Shannon’s theory and the interpretation of information'. Synthese, 144(1), 23–39. Mackie, J. L. (1976). Problems from Locke. Oxford: Oxford University Press. Marr, D. (1982). Vision. New York: W.H. Freeman. Matthen, M. (1988). 'Biological functions and perceptual content'. Journal of Philosophy, 85(1), 5–27. Matthen, M. (2005). Seeing, Doing, and Knowing. Oxford: Clarendon Press. Meskin, A. and Cohen, J. (2008). 'Counterfactuals, probabilities, and information: response to critics'. Australasian Journal of Philosophy, 86(4), 635–642. Millikan, R. (1984). Language, Thought, and other Biological Categories. Cambridge, MA: MIT Press. Millikan, R. (1989a). 'Biosemantics'. Journal of Philosophy, 86(6), 281–297. Millikan, R. (1989b). 'In defense of proper functions'. Philosophy of Science, 56(2), 288–302. Millikan, R. (1990). 'Compare and contrast Dretske, Fodor, and Millikan on teleosemantics'. Philosophical Topics, 18(2), 151–161. Millikan, R. (2001). 'What has natural information to do with intentional representation?' In D. Walsh (ed.), Evolution, Naturalism, and the Mind (pp. 105–125). Cambridge: Cambridge University Press. Millikan, R. (2004). Varieties of Meaning. Cambridge, MA: MIT Press. Morris, W. E. (1990). 'Knowledge and the regularity theory of information'. Synthese, 82(3), 375–398. Neander, K. (1991). 'The teleological notion of a function'. Australasian Journal of Philosophy, 69, 454–468. Neander, K. (1995). 'Misrepresenting and malfunctioning'. Philosophical Studies, 79, 109–141. O’Callaghan, C. (2007). Sounds. Oxford: Oxford University Press. Orlandi, N. (2011). 'Embedded seeing: vision in the natural world'. Noûs, published online 29 December 2011, doi: 10.1111/j.1468-0068.2011.00845.x. Pacherie, É. (1995). 'Do we see with microscopes?' The Monist, 78(2), 171–188. Pacherie, É. (2000). 'Levels of perceptual content'. Philosophical Studies, 100(3), 237–254. Pitcher, G. (1971). A Theory of Perception. Princeton, NJ: Princeton University Press. Price, H.H. (1950). Perception, 2nd edn. London: Methuen. Prinz, J. (2002). Furnishing the Mind. Cambridge, MA: MIT Press. Prinz, J. (2004). Gut Reactions. New York: Oxford University Press. Prinz, J. (2006). 'Beyond appearances: The content of sensation and perception'. In T. Gendler and J. Hawthorne (eds), Perceptual Experience. Oxford: Oxford University Press. Prinz, J. (2007). 'A ll consciousness is perceptual'. In J. Cohen and B. McLaughlin (eds), Fundamental Debates in Philosophy of Mind. Oxford: Blackwell. Prinz, J. (2012). The Conscious Brain. New York: Oxford University Press. Quine, W. (1976). The Ways of Paradox, and Other Essays, revised and enlarged edn. Cambridge, MA: Harvard University Press. Rock, I. (1983). The Logic of Perception. Cambridge, MA: MIT Press. Savitt, S. (1987). 'Absolute informational content'. Synthese, 70(2), 185–190. Sayre, K. (1976). Cybernetics and the Philosophy of Mind. London: Routledge and Kegan Paul. Sayre, K. (1983). 'Some untoward consequences of Dretske’s "causal theory" of information'. Behavioral and Brain Sciences, 6, 78–79.

754   John Kulvicki Sayre, K. (1986). 'Cognitive science and the problem of semantic content'. Synthese, 70(2), 247–269. Sayre, K. (1987). 'Cognitive science and the problem of semantic content'. Synthese, 70, 247–269. Scarantino, A. (2008). 'Shell games, information, and counterfactuals'. Australasian Journal of Philosophy, 86(4), 629–634. Scarantino, A. and Piccinini, G. (2010). 'Information without truth'. Metaphilosophy, 41(3), 313–330. Sellars, W. (1962). 'Philosophy and the scientific image of man'. In R. Colodny (ed.), Frontiers of Science and Philosophy. Pittsburgh: University of Pittsburgh Press. Shannon, C. and Weaver, W. (1949) A Mathematical Model of Communication. Urbana, IL: University of Illinois Press. Sosa, E. (1983). 'On the "content" and "relevance" of information-theoretic epistemology'. Behavioral and Brain Sciences, 6, 79–81. Stampe, D. (1977). 'Towards a causal theory of linguistic representation'. Midwest Studies in Philosophy, 2, 42–63. Stampe, D. (1986). 'Verificationism and a causal account of meaning'. Synthese, 69(1), 107–137. Strawson, P. F. (1963). Individuals. New York: Anchor Books. Taylor, K. (1987). 'Belief, information and semantic content: a naturalist’s lament'. Synthese, 71(1), 97–124. Tye, M. (1990). 'A representational theory of pains and their phenomenal character'. Philo­ sophical Perspectives, 9, 223–239. Tye, M. (1995). Ten Problems of Consciousness. Cambridge, MA: MIT Press. Tye, M. (2000). Consciousness, Color, and Content. Cambridge, MA: MIT Press. Tye, M. (2006a). 'Non-conceptual content, richness, and fineness of grain'. In T. Gendler and J. Hawthorne (eds), Perceptual Experience (pp. 504–530). Oxford: Oxford University Press. Tye, M. (2006b). 'Another look at representationalism about pain'. In M. Aydede (ed.), Pain: New Essays on its Nature and the Methodology of its Study (pp. 99–120). Cambridge, MA: MIT Press. Ullman, S. (1996). High-level Vision. Cambridge, MA: MIT Press. Yourgrau, P. (1987). 'Information retrieval and cognitive accessibility'. Synthese, 70(2), 229–246.

Chapter 40

Modu l a r it y of Perception Ophelia Deroy

Adaptive specialization of mechanisms is so ubiquitous and so obvious in biology, at every level of analysis, and for every kind of function, that no one thinks it necessary to call attention to it as a general principle about biological mechanisms. [  . . .  ] From a biological perspective, (forgetting) this assumption is equivalent to assuming that there is a general-purpose sensory organ that solves the problem of sensing. (Gallistel, 1999: 1179)

As noted by Gallistel, and underscored all through this volume (see Matthen [Chapter 30], Smith [Chapter 17], Bayne and Spence [Chapter 32]), an obvious fact about perception is its functional specialization: we do not just have one ‘general-purpose sensory organ’ but are equipped with different senses which deliver various kinds of representations. From a functional point of view, each sense can be viewed as a specific system in charge of tracking a certain kind of change or property in the environment. Each sense then fulfils this role by transforming a set of characteristic stimuli (or ‘inputs’) into characteristic representations (or ‘outputs’). In philosophy and cognitive sciences, modularity can be seen as a way of turning this description of sensory systems into a more precise and well-argued architectural model. The notion of modularity took off as a mainstream thesis in cognitive science and philosophy of mind with Fodor’s 1983 book The Modularity of Mind.1 Refining Marr’s computational model of vision,2 Fodor put forward the idea that the mind contained, among other things, some input systems or modules capable of transforming the signals resulting 1  As underscored by Fodor in his book, the idea of modularity has much older roots in the traditional faculty psychology, especially in Legall’s phrenology or Wernicke’s seminal work on aphasia. I will not be concerned here with these historical aspects, but see Fodor (1983: Introduction) or Bergeron (2007). 2  Marr (1983).

756   Ophelia Deroy from receptors into representations , and this through a series of encapsulated and dedicated inferences. Modules, which for Fodor include vision, audition, speech processing, face processing, etc., were therefore thought as intermediate between transducers in charge of automatically converting stimuli into signals and a central system which worked under a more subject-controlled and holistic way and could compare and process the various representations delivered by the peripheral modules. As only these intermediate systems were thought to be modular, one could, with Prinz (2006), note that Fodor’s book ‘would have been more aptly, if less provocatively, called The Modularity of low-level peripheral systems’.3 It is precisely with the modularity of low-level peripheral systems that the present chapter will be concerned. This means that it will not overtly be concerned with the most recent debates on modularity, which have examined the possibility or relevance of generalizing modularity to the whole of cognition. Whether the mind is entirely or ‘massively’ modular is a question that goes beyond the scope of the present discussion.4 The goal is rather to examine the challenges that have been raised for modular models of perceptual systems by recent empirical and fundamental research on the senses and have been somehow kept in the background while massive modularity models were making the front page.

1  Arguments for the modular architecture of the mind and perception in particular The notion of modules is intimately linked to computational models of the mind where mental processes are defined as a series of formal manipulations over syntactically structured information.5 Computational theories actually provide one of the key justifications for thinking of the mind as modular, that is as a larger system decomposed in specific subsystems working (relatively) independently of one another. If computations were to access all or at least a large quantity of the background information present in the system, they would be overloaded and ultimately ‘paralyzed’.6 To avoid this ‘computational explosion’,7 it makes sense for computational processes to operate only on a tractable, restricted store of information or background knowledge. As stated by Carruthers (2003) and confirmed by Fodor (2000), computational and modular models are like two sides of the same coin: 3 

Prinz (2006: 22). See Barrett and Kurzban (2006), Samuels (2006), and Sperber (2002) for recent statements and reviews. This said, the issues of massive modularity and modularity of perception are not totally disconnected. First, if it turns out that sensory systems are not in any interesting sense modular, then the claim that all of our cognitive system is modular should be challenged. Second, it is an interesting question to see whether the same or different definitions of modules apply across perceptual and non-perceptual skills (with most people agreeing that different definitions apply, see for instance Carruthers, 2006). 5  For initial expositions, see Fodor (1975, 1981, 1987); Marr (1983); Putman (1963); Pylyshyn (1984). See Piccinini (2009) for a recent review. 6  Cosmides and Tooby (1994: 91). 7  Carruthers (2003). 4 

Modularity of Perception    757 Viewer centred Input image

Perceived intensities

Object centred

Primal sketch

2½-D sketch

3-D Model representation

Zero crossings, blobs, edges, bars, ends, virtual lines, groups, curves boundaries

Local surface orientation and discontinuities in depth and in surface orientation

3-D models hierarchically organised in terms of surface and volumetric primitives

Fig. 40.1  Marr’s computational model of vision (1983). The only known way of realizing (fast and non-explosive computations), is to make such processes modular in nature. So if computational psychology is to be possible at all, we should expect the mind to contain a range of modular processing systems.8

A related argument for modular architecture usually concerns processing speed: again, if computations were too widely comprehensive, they would take a very long time. If members of a species took five minutes to recognize a predator or a dangerous cliff, that species would certainly not be around for long. The argument here is that sensory systems must have evolved and been selected in the most optimal way. Creatures which developed encapsulated, faster sensory systems (optimally constrained by conditions of accuracy) had better chances to survive. One can think here of at least three other benefits of modular systems, which are less often talked about but also stress why modules seem to provide an optimal evolutionary solution for sensory systems. First, modularity allows simultaneous processing to be performed without the processes competing with each other for shared resources.9 Second, as modules do not share the results of all their computational steps with the central system, they lighten up the informational load from the central system. They deliver outputs ready to be used by the next system. For instance, in Marr’s classical model of vision, our visual system arguably computes a mere 2D representation, which is however not shared with other modules or central systems. Only what Marr called 2½ D representation, which can be used to build up the 3D representation is accessed by other systems (See Figure 40.1). Last but not least, it is also easier for a modular system to readapt in evolution—i.e. to take something that is already working in one role and repurpose it with small changes to another. It is also easier for the system as a whole to be able to recover from damage—as one can fix only one part, without disturbing the whole system. This feature makes modular

8  Carruthers (2003: 71); see also Cosmides and Tooby (1994). Fodor (2000) presents a reverse argument, stressing that, as the mind does not contain only modules, we need to reform or give up the claim that the mind is computational. On the debates generated by Fodor’s statement that The mind doesn’t work that way (i.e. computationally through and through), see for instance Collins (2005). 9  Note though that this does not mean that there is no competition between the outputs at the level of central systems. Nor does it mean that modules cannot work to the same purpose in a coordinated way; but contrary to players in an orchestra, they do not need to be coordinated from above, nor, like in a jazz band, to exchange information between them. The distribution of labour has been, so to say, negotiated before.

758   Ophelia Deroy systems especially attractive to account for the way evolution might have affected different aspects or parts of the mind and taken them in different directions. It has been nicely illustrated by Simon (1962) who, as an early exponent of modularity, proposed the following parable comparing two watchmakers, Hora and Tempus: The watches the men made consisted of about 1000 parts each. Tempus had so constructed his that if he had one partially assembled and had to put it down—to answer the phone, say—it immediately fell to pieces and had to be reassembled from the elements . . . The watches Hora handled were no less complex . . . but he had designed them so that he could put together sub-assemblies of about ten elements each. Ten of these sub-assemblies, again, could be put together into a larger sub-assembly and a system of ten of the latter constituted the whole watch. Hence, when Hora had to put down a partly assembled watch in order to answer the phone, he lost only a small part of his work, and he assembled his watches in only a fraction of the man-hours it took Tempus.10

Modular architecture, such as the one adopted by Hora, looks like the best natural match not only for watches but also for sensory systems since these systems need to provide fast information about changes in the environment (or bodily states), should be able to process information in parallel, and need to adapt to environmental changes over the long course of evolution.

2  Modules 1.0: Fodorian modules Modular systems are more than just the assembled systems envisaged by Simon. Fodor (1983) claims that modules are a natural psychological kind and that their ‘informational encapsulation’ plays a key role in the functioning of the mind, especially for perception. While agreeing later11 that encapsulation is a key definitional feature of modules, Fodor’s initial statement famously introduced eight other features which were supposed to be characteristic of modular systems and to help individuating them. Accordingly, a module would likely present:

1. Domain specificity, that is only operate on a limited range of inputs (for instance, vision only operates on retinal signal, face recognition on inputs which satisfy a certain face-template); 2. Mandatory operation, that is, operate in an automatic, largely involuntary way, like reflexes; 3. Limited central accessibility or informational opacity, meaning that other systems, and mainly the central systems, do not have access to everything which is computed in the module; 4. Fast processing; 5. Informational encapsulation, which means, as stated previously, that the module accesses only a limited subset of the information present in the system, and noticeably (but not only) in memory; 10 

Simon (1962); quoted in Samuels (2006: 43)..

11 

Fodor (2000).

Modularity of Perception    759

6. ‘Shallow’ outputs, meaning that they are likely to be further processed and enriched; 7. Fixed neural architecture; 8. Characteristic and specific breakdown patterns, for instance in brain damages; 9. Characteristic ontogenetic trajectory.

This (relatively) long list inspires some questions. First, how independent or related are these features? Some characteristics like 7, 8, and 9 seem to cluster around the same idea that modules have a precise neuroanatomical localization, and the characteristic of being automatic, fast, and informationally encapsulated seem to go together. Several attempts have been made at providing a more rational reconstruction of what a module is and how these features fit together,12 and the present section will in some sense be no different. However, the goal of the reconstruction is here a bit different: I don’t want to insist on turning Fodor’s initial gesturing at a set of characteristics into a definite set of necessary conditions; on the contrary, I want to suggest how some natural articulations and hierarchies of characteristics in Fodor’s definition planted the seed for future challenges to modularity. Another question which naturally arises in front of Fodor’s list is to wonder whether these features provide a definition of modules, or merely intend to be diagnostic. Partially addressing this worry, Fodor reckoned that a cognitive system could count as modular if it presented most of the nine features mentioned above to a certain degree.13 This might be problematic if one thinks of modules as a classical natural kind, and consequently searches for a strict distinction between modular and non-modular systems. Notice however that there is nothing intrinsically wrong in definitions of modularity resulting in a continuum from more modular to less modular processes—and this is the view that seems to be endorsed by several proponents of massive modularity14 and, arguably, is present in Marr (1983). It is quite useful to realize that the debates around modularity in the philosophy of perception have forgotten about this initial continuum idea: they have taken modularity only as an all-or-nothing way and not a graded one. Contrary to what it seems, and as detailed later, the graded understanding of modularity does not necessarily damage the value of modular models, as there are still interesting debates to be had as to what explains the degree of modularity in a certain system. There is however one real worry with the graded understanding when it comes to making sense of a system possessing certain features in Fodor’s list to a certain degree. What could it mean for outputs to be shallow to a certain extent, or for modules to present characteristic breakdowns to a certain degree? The difficulty in providing a graded understanding of these modular characteristics might be overcome, but it becomes particularly acute when we turn to informational encapsulation, which occupies, as previously said, a central role in the definition of modules. The idea of information-proof, or ‘cognitive impenetrability’ as Pylyshyn (1999) puts it, does not easily receive a graded interpretation:15 any sign that the function computed by the system is sensitive to the organism’s

12  13 

Or not, see for instance Bergeron (2007), Prinz (2006), Samuels (2006). 14  For example, Sperber (2002). Fodor (1983: 37). 15  The fundamental character of informational encapsulation is stressed in Fodor (2000) but also Currie and Sterelny (2000), Scholl and Leslie (1999), and Pylyshyn (1999). This said, others like Coltheart (1999) consider that domain specificity is the key feature of modules—see Hirschfeld and Gelman (1994) for a discussion.

760   Ophelia Deroy

Fig. 40.2 The Müller-Lyer illusion (1889). Knowing that the two lines are of equal length does not stop them appearing of different lengths. goals and beliefs is sufficient for it to count as non-modular. There seems to be no room, in this interpretation, for the system to be encapsulated to some significant degree and count as modular. With these questions in mind, let’s start a more detailed examination of the features of the initial Fodorian modules (or modules 1.0, as I suggest to call them) starting with their informational encapsulation.

Informational encapsulation and informational opacity Modular processes cannot be penetrated or influenced by propositional or external information, and their intermediate stages of processing are also opaque to the other systems. Being two-ways hermetic, modules can be compared to tubes: nothing escapes before the output, and nothing gets in except at the level of inputs. Now, if encapsulation makes sense as an optimal design for fast complex computational systems, which reason do we have to think that perceptual systems follow such a model? Most of the arguments here come from reflecting on examples. Take Figure 40.2 above, which you might already be familiar with: the two arrowed lines look of different lengths, and yet, when measured with a ruler, they turn out to be of the same size. However, knowing that the lines are of the same size does not dissipate the faulty appearance:  the two lines stubbornly look of different lengths, however hard you try to see them as equal. The persistence of visual illusions like Müller-Lyer is only one illustration among many of the lack of power that background knowledge or expectations have over our experience. Size, weight and colour constancy, etc.—all these well-documented phenomena 16 are also beyond our correction. The way we perceive the world seems in this sense most of the time unaffected by our knowledge: we cannot just see what we want or think we should see. Does perception always work in such an autonomous way? Think of another famous case like the duck-rabbit drawing in Figure 40.3. Here, the reverse seems to apply: what you expect to see changes the nature your experience. Thinking ‘this is a picture of a rabbit’ or ‘this is a picture of a duck’ seems to modify what you experience.17 Children are for 16 

On perceptual constancy, see Cohen, Chapter 33, this volume. Starting with Jastrow (1899, 1900) this example has for a long time been taken to illustrate the role of expectations and world knowledge in perception (see also Wittgenstein, 1953/1958/1958), although it is likely in this case that this influence is mediated by attention. On the related claim that observation is expectation driven or ‘theory-laden’, see Feyerabend (1962); Kuhn (1962); and Raftopoulos (2000) for discussion. 17 

Modularity of Perception    761

Fig. 40.3  A version of the duck-rabbit ambiguous figure (Jastrow, 1899, 1900). Expecting to see a rabbit makes it more likely that one sees a rabbit in this figure. instance more likely to see the figure as a rabbit on Easter Sunday than on a Sunday in October.18 However, the encapsulation of perceptual processing is not threatened by most examples of the duck-rabbit sort as they are likely to receive an explanation at the pre- or post-modular level.19 On the one hand, some cases are easily explained by differences in attention.20 If two persons are confronted by the same visual object in the same illumination conditions, but do not attend to this object it in the same way, they may not perceive the same thing—not because the modular processing is affected by their knowledge, but because the order in which they attended to the different elements of the stimulus affects the input that the modular processing operates on. This is certainly the case with the duck-rabbit example, but even more convincingly with the Necker cube, presented in Figure 40.4: if you attend to the dotted corners, you will perceive the picture of the cube as having different orientations.21 The same Necker cube on the page ends up generating two different visual outputs, but attention—not differences in processing caused by beliefs about the orientation—is here responsible. Other cases where background knowledge or expectations seem to affect how things are perceived by us include perceptual expertise. If you are a wine expert, what you know might make your tasting experience richer than the novice’s experience of the very same wine; if you are an expert botanist, you are likely to see many different trees where the layman only sees an indistinct bushy forest.22 However, these cases are likely to be explained by post-modular differences—that is, by differences in recognition or discriminative capacities after the ‘raw’ representation of the wine or the scene has been generated by modular perceptual processes. According to the modularity model, the role of knowledge is limited to the orientation of endogenous or top-down attention and to late or high-level perception. It is important here to notice that our experience of the world most often corresponds to an interpreted 18 

Brugger and Brugger (1993). See Deroy (2013), Macpherson (2012), and Stokes (2012) for discussion of more precise cases. 20  See Long and Toppino (2004) for a review of various explanations of ambiguous and reversible figures. 21  On the label assignment underlying this effect, see Waltz (1975) and Pylyshyn (2003: ch. 3). 22  See Smith (2007) and Siegel (2011) respectively for these two examples. 19 

762   Ophelia Deroy

Fig.  40.4 The Necker cube demonstrates the role of attention, and not thought, in determining the perceived orientation of the drawing. Attending to the dark or the grey corner will switch the perceived orientation of the cube (as illustrated on the right). perception of recognized objects, events, and scenes; it is trivially true that this level is sensitive to what we know. Low-level perception refers to an earlier stage of representation determined only by the nature of the stimulus and bottom-up rules of processing. For instance, the visual system transforms the retinal signal that results from the stimuli received on photoreceptive cells in the eye, into visual representations. These first steps constitute the ‘modular’ part of the visual system, being hermetic to cognitive influences and simply following their own computational rules. As long as information simply feeds forward from the transducers into the modular processing and from there into higher-level perception, the tube-like model is safe. The partition between low- and high-level perception is still widespread in philosophy and cognitive science, but far from unproblematic. Methodologically, the boundary might be difficult to use as a support for modular models of perception if it can simply be moved in an ad hoc way and identify as ‘low level’ whatever part conforms to the definition of a modular processing. Theoretically, it is far from clear how the distinction extends to other senses than vision where it has been mostly developed.23 Is there something like low-level non-interpreted olfaction, or is all olfactory perception a perception of interpreted or recognized odors?24 Even restricted to the visual modality, new difficulties arise, as one can wonder for instance if higher-level vision is part of vision or judgement25 or if lower-level vision really deserves to be called ‘vision’ is if it does not deliver full-blown perceptual representations. What needs to be clarified here is the status of this non-interpreted perceptual representation or, as Fodor’s put it, the ‘shallow’ nature of modular outputs.

Shallow outputs The ‘shallow’ character of the output is perhaps the most metaphorical of Fodor’s list. What is meant here is that the representations generated by modular, and noticeably perceptual 23 

See Matthen (2010) for an extension to audition. See Gottfried (2010) for a defence of this view. 25  See Raftopoulos (2011) on this issue. 24 

Modularity of Perception    763 systems, are somewhat primitive, pre-semantic, non-interpreted, and not yet the content of an attitude. This negative characterization is supposed to remind that the outputs of a module do not necessarily correspond to the conceptually interpreted and structured perceptual experience (otherwise known as ‘epistemic’ perception) that we enjoy when we see that there is a table in front of us. From this, it is tempting to relate the idea of ‘shallow’ output to the idea of a non-conceptual content.26 However, the discussions about the nature of modular outputs and about non-conceptual perceptual content should not be related too quickly, especially to avoid some pre-conceptions or theories of non-conceptual content to contaminate the understanding of modules. First, if one thinks of non-conceptual content as an analogue content, being continuous or all of one block, then the shallowness of perceptual outputs does not necessarily meet the corresponding requirement of being unstructured:  for instance, the output of visual modular processing can be seen as a structured representation, where certain colours and shapes are bound together and ‘attributed’ to the same location.27 The same visual features could be differently structured and the overall representational content of the experience changed, as with propositional-conceptual contents. One could argue here that looking at the level at which colours and shapes are bound together is not really looking at a shallow enough level, but at the later result of the binding of distinct modular outputs—i.e. colour and shape.28 Looking more specifically at colour processing, the output of modular processing can still be at odds with some of the requirements for being non-conceptual, at least in the sense that it is also cleverly structured by certain constancies. For instance, the output of colour processing can represent that the various sides of a cup look of different shades and also represent the cup as of one and the same colour all over. It is important to stress that the representation of this unique-yet-different colour is not added later by recognition that it is a cup—and therefore likely to be of the same colour; it is on the contrary what precedes our recognition that it is a single object—and then a cup. Quite importantly, the same isolated bit of retinal signal can lead to different categorical perceptions or visual outputs, by being represented as red when surrounded by other red shades, and as orange when surrounded by more orange shades. Some differences in the output can depend on the rest of the information or inputs with which a certain bit of signal finds itself encoded and therefore ‘jointly-processed’ in the module.29 Nothing excludes that information is shared within the module itself:  there are intra-modular ‘holistic’ and top-down effects, and not just sequential mechanical bit-by-bit processing. If this was the case, then speech could never have been thought of as modular—as it was by Fodor (1983): for instance, a speech perception module can deliver a full word as output even if some sounds are missing in the middle of the initial utterance. Hearing speech is not just hearing a series of sounds, one after the other, but relies on a fast and optimal series of

26 

27  Treisman (1993); see also Tacca (2010, 2012). For example, Heck (2007); Fodor (2008). (2012) for instance considers that colour processing is a good candidate for being a distinct modular processing, and assumes that it does not mix with shape processing. This said, the current functional segregation of shape and colour processing is debated (see Sincich and Horton, 2005). 29 This is for instance the most straightforward explanation of some effects like biased racial categorization of faces (e.g. Rossion and Michel, 2011) which some want to see as effects of background concepts. 28  Macpherson

764   Ophelia Deroy inferences performed on the set of information present in the module. The same holistic characteristic is also found in face perception. To conclude on this point, the conceptual–non-conceptual or pure bottom-up– top-down distinctions might not be helpful when it comes to fine-grained distinctions of levels of visual representation and the intra-modular aspects of processing. One of the interesting challenges here for philosophers is to find a better terminology to understand what sort of ‘shallow’ representations are produced by low-level, automatic, encapsulated perceptual processing.

The biological basis of modules Modules are also defined by three final characters, which are intimately related. These are, so to speak, the biological aspects of modules: modules come with a specific neuroanatomical localization (either a single locus or a distributed network); they present some characteristic breakdowns and developmental trajectories. There is an obvious link between the idea that modules are implemented by some specific part of the brain, and the fact that brain-damage in this area will result in a specific functional loss or damage.30 Evidence from brain-injuries in this sense has played an important role in the individuation of modules, for instance in speech perception. This role has been seen as coming, although not uncontroversially, from evidence of double dissociations showing that breakdowns in the processing of A would not impair the processing of B, and vice versa.31 This supports, but certainly doesn’t demonstrate, the claim that A and B correspond to two independent modules, as double dissociations are compatible with other explanations. What’s more, as far as the empirical evidence is concerned, double dissociations are never so clean cut. In the domain of perception, specific breakdowns and anatomical distinction have, however, generated an important series of debates, leading noticeably to the hypothesis of two independent visual systems or modules. The idea that one independent channel would have the function of localizing objects in space and the other of representing their appearance to consciousness relied heavily on observed effects of brain-damage to the dorsal stream.32 Interestingly, this hypothesis is facing pressure from two fronts: one, which focuses on evidence of neuroanatomical connections between the two streams33 and suggests that the criterion of segregated and dedicated neuro-architecture is not met in this visual case, and the other which stresses the difficulty of isolating a distinction between functional roles for the two pathways, when modules are supposed to have such distinct roles.34 The link between fixed neural architecture and the idea of characteristic ontogeny raises the question of the innateness of modules. According to Fodor, modules are fixed at birth 30 

See Bergeron (2007) and Bergeron and Matthen (2008) for further discussion. See Juola and Plunkett (2000); Plaut (1995). 32  Milner and Goodale (2008); see Jacob (Chapter 12, this volume) and Prinz (Chapter 19, this volume) for further discussion. 33  See Nassi and Callaway (2009) for a review. 34 See Jeannerod and Jacob (2005). A more convincing argument against the non-functional distinction might come from the (discussed) evidence that certain specific features could be processed by the two streams (e.g. binocular disparity; see Nassi and Callaway (2009) for discussion). 31 

Modularity of Perception    765 or shortly thereafter.35 Most commonsense models of the senses would also assume that the distinction of vision, audition, olfaction, etc. (or whatever the list of senses is) as well as their basic rules of processing are given at birth. However, the functional specification of perception can also be seen as a largely acquired trait, where modules would develop (and vary) through exposure and learning.36 Now this flexibility might be problematic for the definition of modules, as will become clearer below.

3  Challenges to modularity Despite their separate treatment in the literature, three challenges to the modular architecture of perceptual systems are intimately connected in that they all stress the difficulty of reconciling the features attributed to modules and a form of flexibility observed in perception. As just said, the same bit of signal does not necessarily find itself represented in the same way depending on what else is processed or perceived at the same time; the same function can be differently realized in young infants and adults, or functional specificity can evolve through time. The anti-modularity line in all these cases consists in showing that modularity is not descriptively accurate and does not capture these crucial aspects of perception. It should be noticed as a preliminary remark that this judgement most often comes with a certain understanding or reconstruction of what Fodorian (or 1.0.) modules are—which end up making them more inflexible than they can be understood to be. Even if modules 1.0 are indeed too rigid to accommodate the flexible characters of perception, these objections should not miss the fact that modularity is a general explanatory project,37 beside the descriptive role it has been used to serve, and that, as such, it can be adjusted to meet these challenges.

Cognitive penetrability of perception Many everyday and empirical examples can be listed where what one expects or desires seems to impact what one sees or perceives:  besides the example of expert perception, where knowledge seems to make perception richer, it has been shown that the same odorant presented under the label ‘parmesan cheese’ is experienced as a nice food smell, and, presented as vomit smell, experienced as a disgusting non-food smell; the same liquid tastes sweeter if it is delivered with a strawberry smell that makes one expect a sweet-flavoured drink; and, arguably, desires can make the same object look closer or further away.38 These claims are difficult to reconcile with the key features of modules 1.0, that is, their informational encapsulation (Fodor, 2000) or to put it in Pylyshyn’s terms, their cognitive impenetrability. According to this classical understanding of modularity, the kind of 35 

36  See Karmiloff-Smith (1992). Fodor (1983, 1992). I agree here with Butterfill (2007) that the focus should go back to the key question: what are modules supposed to explain? 38  For example, Balcetis and Dunning (2010); Herz and von Clef (2001); Shankar et al. (2010); van Ulzen et al. (2008); (see also Deroy (2013) and Stokes (2012) for a review of earlier studies). 37 

766   Ophelia Deroy input-output function or algorithm which characterizes a perceptual system cannot be changed in virtue of the content of other states of the cognitive system—such as thoughts, beliefs, desires, and other propositional attitudes. Now both the previous examples and evolutionary arguments might be advanced against the fact that perceptual systems should be cognitively impenetrable, stressing both the need and some evidence for them being sensitive to background knowledge or desires of the organism. As said earlier, the notion of informational encapsulation is not threatened by all effects of context sensitivity or flexibility in perception. Many effects of this kind (and perceptual expertise might be one of them) can be explained by a difference in attention, or in perceptual decision or judgement. Establishing that a perceptual variation is actually due to cognitive penetration is a difficult matter.39 The real challenge here might come from content that everyone accepts as being a shallow output of perceptual processing. This content should also not controversially depend on higher-level recognition, as in the case of seeing kind-properties like the one of being an elm or a sparrow. Both ‘austere’ and ‘liberal’ views of perceptual content can accept that colour is such a candidate.40 What’s more, it is generally agreed that we directly experience the shallow output of perceptual processing when seeing colour: this is captured by various claims, according to which our experience of seeing red only depends, by being caused or by supervening, on facts about the colour signal and colour processing. Problematic evidence is supposed to arise from various studies demonstrating that the experience of colour can be affected by background beliefs, knowledge, or expectations about the perceived object. Going back to the 1950s, the demonstration of such effects became popular within a movement known as the ‘New Look’. In one experiment, resurrected by Macpherson (2012), John Delk and Samuel Fillenbaum showed that participants would match different figures, cut from an identical red-orange material with different background shades depending on the colour associated with each figure. In the experiment, the three figures (heart, apple, and lips) of objects known to be red were matched with a more saturated red background than the geometrical ones, and the brown-associated figures (horse, bell, and mushroom) were matched to a more grey-brown shade of red than geometrical ones (oval, circle, and ellipse) to which no particular colour was believed to attach. The difference in the chosen background colour was ‘not only significant statistically, but of substantial magnitude perceptually’.41 The conclusion of the authors, shared by Macpherson (2012), is that participants saw hearts, lips, and apples being ‘more red’ than the other figures. The same kind of effects, this time on the ‘shallow’ output of size, was found in another experiment by Bruner and Goodman (1947), suggesting that children would

39 

For example, Deroy (2013); Macpherson (2012).. The distinction between the ‘austere’ and ‘liberal’ view is usefully proposed by Bayne (2009). The proponents of austerity mostly take their list of admissible content of perceptual experience from the list of features/representations generated by lower-level perceptual processing, when liberals are happy to include contents of higher-level processing. 41  Delk and Fillenbaum (1965: 293). To be more specific, the initial figures’ colour corresponded to the Munsell chip R/5/12 (where R indicates the hue, the first number the saturation, and the second the ‘chroma’). The chosen backgrounds were R/4/12 for ‘red-associated’, R/4/13 for geometrical figures, and R/5/13 for ‘brown-associated’ ones, making the perceptual difference more subtle than concluded—that is there is no hue or saturation difference between geometrical and red-associated figures, but a chromatic one only (see Deroy (2013) for discussion). 40 

Modularity of Perception    767 significantly overestimate the size of a circular object in their hand if it was a coin, rather than a piece of cardboard, and that the effect was even stronger when the desirability of the coin was stronger—for instance if children were from a poor background. All these studies would be worrying for the informational encapsulation for visual processing if there were not so many intrinsic problems with their methods. Most of them can be ruled out or criticized for their failure to be replicated within the same conditions, the lack of several important controls, or a biased treatment of statistics.42 More recent studies43 have certainly shown intriguing effects in colour processing, correlated with the kind of object that is perceived. When presented with shapes of fruits and vegetables (banana, strawberry, orange, zucchini, etc.) coloured in their natural colour, and instructed to adjust the colour on the screen until the figure looked grey, participants adjust most fruit shapes further toward the point of their opponent colour. For instance, the final colour pointed toward bluish for the yellow banana, suggesting that, at the point where the circle looked achromatic (grey), the banana still looked slightly yellow and in need of further adjustment. As I have argued elsewhere, these later results minimally show that performance in the proposed task is different when a colour is embedded in a shape associated with an object with a typical colour, and when it is not. It is the actual or the perceived bearer of the colour that makes a difference to the finally experienced colour—what I call a form of object sensitivity of perception.44 Although object sensitivity can be explained as mediated by conceptual thinking, this explanation is neither mandatory nor well grounded. True, the banana shape was adjusted toward yellow’s opponent colour. True, this is consistent with the belief according to which bananas are yellow. But does this mean that: the simplest and best explanation of the case is that cognitive penetration is occurring . . . the subject’s belief that some of the cutouts have shapes of objects that are characteristically (yellow) penetrates the relevant color experiences of the subject.45

Focusing on a single case study is certainly distorting here and other cases show that beliefs per se are not really causally responsible for colour effects: believing or knowing that, for instance, French letter boxes are yellow does not bias a British perception of these yellow letter boxes.46 Propositional representations of typical colours of objects are not sufficient to observe a change in colour perception, but experience is. What is on the contrary necessary is exposure. The most important challenge is not to go back to old debates about thought penetrating perception, but to understand sensory adaptation through exposure, and at 42  See

Francis (2012) for a recent discussion of Balcetis’s work suggesting that desired objects look ‘closer’ than others. 43  Hansen and Gegenfurtner (2006); Hansen et al. (2006). 44  Notice that it is not clear that this effect makes any difference to the final judgement someone is ready to make on the basis of his/her experience—for instance, one is still disposed to judge that the banana is yellow under normal or weak illumination conditions (thus blocking the kind of epistemic threats that some, like Siegel (2011) want to attribute to such examples). Object sensitivity, on the contrary, should result in easier and faster accurate object recognition. 45  Macpherson (2012: 49) initially commenting on the effect for red about Delk and Fillenbaum’s results. 46  See Mitterer and de Ruiter (2008) and Mitterer et al. (2009) for cross-cultural comparisons on these colour effects obtained with artefacts.

768   Ophelia Deroy (a)

(b)

Fig.  40.5  Evidence of the ‘light from above’ prior. The same drawing is perceived to curve out (a)  or to curve in (b)  depending on the vertical distribution of the grey shades. which level it occurs. Sensory adaptation has classically been understood as perceptual processing automatically adjusting to prevailing circumstances, independently of beliefs formed about those circumstances. Key examples occur across modalities: for instance, when wearing prismatic goggles that shift the optical scene upside down, individuals can recalibrate with the new tactile-visual contingencies and manage to get around normally and reach for objects correctly; 47 when exposed to temporally asynchronous speech sounds and lip movements, individuals progressively re-establish the perception of synchrony. 48 Arguably, this adaptation affects perceptual experience and is not just to be explained by a correction of judgements. Going back to the present concern, similar adaptation has been shown to occur intra-modally and affect the prior information that is classically considered to be part of the visual module. Take for instance the processing of shape, noticeably convexity, which is considered as a key example of modular processing. The shape module has been shown to operate on a ‘light from above’ prior assumption, as demonstrated in Figure 40.5. The same figure is seen as convex or concave depending on its vertical orientation because of the (noncognitive, intra-modular) ‘assumption’ that lights come from above.49 Intra-modular assumptions like this are not necessarily fixed once for all, especially innately, but can be updated, including intra-modally by visual cues alone.50 Now certainly in such cases, perceivers do not stop believing that the light comes from the sun and the sun is above, nor does their concept of natural light or sun change, even if their perceptual experience starts working on a different assumption. Understood along similar lines, the effects of object sensitivity of colour perception do no longer threaten modularity: perceptual processing can change with exposure to pairings of colours and shapes. This explanation in terms of sensory adaptation can work either by assuming that shape and colour are jointly processed at lower levels, or by attributing the changes to a further, 47 

48  See Spence and Squire (2003) for a review. Held (1965). Notice that natural constraints are not to be thought of as being ‘represented’ by or in the module (see Collins (2007) and Smith (2006) for a discussion in speech perception): talk about prior knowledge, or rules, is just a way to describe an inner mechanism. 50  Adams et al. (2010). 49 

Modularity of Perception    769 still encapsulated level where shape and colour are (automatically) integrated in a way that adapts with past exposure.51 Let me consider one response to the idea that it is sensory adaptation, more than cognitive penetration, which constitutes the most important challenge for modularity. One can try and rescue the threat of cognitive penetration by considering the top-down influence of concepts on perceptual processing, and not beliefs or knowledge. One temptation here, favoured by more recent work on concepts as abilities or skills52 is to equate the fact that one shows sensory adaptation by responding differently to various kinds of objects (or sets of co-occuring features) and the possession of the related concepts for the related kinds of objects. It is however doubtful that showing a discriminated response to a certain kind of object or co-occurring features (like here, to the co-occurrence of a yellow colour and a certain crescent shape) corresponds to the canonical notion of concepts used by philosophers when thinking of constituents of thoughts, generality constraints, etc. More importantly, it is not sure than anything is won by accepting the equivalence between being sensitive to the co-ocurrence of features and having the concept of a banana: if considering that one has the concept of banana is a way of describing a low-level horizontal interaction between shape and colour processing, then it is not sufficient to show that the concepts exert a top-down influence on colour processing.

Multisensory perception Once vertical effects of cognitive penetrability are pushed aside, awaiting better empirical evidence,53 an important question remains: do horizontal effects―cases in which what is going on in different perceptual systems influences the outcome of certain perceptual processes―threaten modularity? In recent years, the existence of multisensory interactions has been taken as a reason to move ‘beyond modularity’, as suggested in Driver and Spence (2000), but the terminological confusions require us to be clear about what sort of interactions are really threatening. On the one hand, our perception can be said to be ‘multimodal’ in the sense that our conscious experience of the world seems to comprehend auditory, visual, tactile, olfactory, etc. components all at once,54 in a single unified field. This in itself might be a source of philosophical questions, and perhaps challenge for computational 51 

There is both an evolutionary and an empirical argument for thinking that integrating shape and colour information into a single percept (that is, by losing access to independent estimates of these properties, see Hillis et al., 2002) occurs automatically within the same modality: combinations of shapes and colours are likely to co-occur and in the same locus of attention/object, which is not the case for instance for combinations of tactile size and visual size, where one can be looking at something and touching something else. 52  See Machery (2009) for a review. 53  It should be clear that the issue cannot be solved a priori and that new or better evidence can be brought for cognitive penetration of early visual processing. As the most important question is how to best explain the effects noticed in perception, my argument here is to say that the modular explanatory framework is not obviously unable to explain or dismiss effects which are elsewhere explained as instances of cognitive penetration. This does not mean that alternative models, which give up on the cognitive penetration debate altogether, like predictive coding do not have an overall explanatory advantage over modular models. 54  See Bayne (2010) for an overview.

770   Ophelia Deroy

Processing 1

Processing 2

Processing 1

Processing 2

Fig. 40.6  Two models for multisensory interactions (adapted from Driver and Spence, 2000). models (Fodor, 2000) but it does not raise difficulties for 1.0 modularity, which relegates the problem of unification to the central system. On the other hand, perception is multisensory in the sense that one kind of sensory processing can be influenced by the sensory processing in another (stimulated) stream at an early level. For instance, so called ventriloquist effects show that auditory localization is influenced by visual localization when the two stimuli occur close in time. Tactile localization can also be influence by visual localization, also at an early level. These multisensory influences have been shown to be the rule rather than the exception in everyday perception (see O’Callaghan (2011); see also Bayne and Spence, Chapter 32, this volume). At first sight, they are compatible with a traditional modular model, where outputs of visual and auditory processing are independent and which then feeds in another module or system which uses them as joint inputs to generate representations of audio-visual localization or speech sounds (see Figure 40.6). More generally, it seems (at least theoretically) possible to systematically reinterpret the evidence of multisensory effects, either by attributing them to a second layer of processing where a first series of modular outputs are used as new inputs, or by extending the range of admissible inputs for one of the first modules.55 What’s more, Radeau (1994) has argued for instance that audiovisual parings respect the four main criteria of Fodorian modules, i.e. cognitive impenetrability, computational autonomy, innate specification, and dedicated neural architecture. Arguing whether these processes can really fit with the description of modules 1.0. is however not here the main challenge. What matters is to see whether these modularityfriendly interpretations of multisensory interactions remain, in an interesting sense, explanatory. Two limits can be envisaged: first, the idea of outputs being fed forward into other modules does not illuminate the early occurrence of multisensory interactions in the brain, which closely coincides with unisensory processing; second, a series of important top-down (but not necessarily counting as cognitive penetration) effects are missed in such a model, leaving unaddressed the role and status of various multisensory assumptions in the fusion or integration of multiple sensory cues.56 As such, multisensory interactions represent a key challenge to modular models 1.0.

55  Radeau (1994) argues for instance that audio-visual parings respect the four main criteria of Fodorian modules, i.e. cognitive impenetrability, computational autonomy, innate specification, and dedicated neural architecture. 56  See Deroy, (2013); Driver and Spence, (2001); Spence and Deroy, (2012).

Modularity of Perception    771

Synaesthesia Synaesthesia57 is often presented as yet another case of sensory interactions where the barriers that are supposed to keep modules and their processing separate break down.58 Synaesthesia seems at first to repeat the previous challenges to informational encapsulation raised by multisensory perception:  it shows that information is shared across sensory ‘modules’ (for instance auditory processing and colour or visual processing in coloured-hearing synaesthesia). So why consider it a specific challenge? The reply comes here from the ambivalence revealed by synaesthesia in the interpretation of modularity. The very same phenomenon is seen as a confirmation or a disconfirmation of modular models of the mind, showing how easily the debates around modules 1.0 become a matter of describing things in the right way. In this sense, the debates seem coloured by a form of confirmation bias in favour or against modularity. What matters however is not whether synaesthesia satisfies enough of the Fodorian criteria to be compatible with modularity, but to see what is gained (or lost) in thinking of synaesthesia (or any other perceptual phenomena) in these architectural terms. Let’s consider for a moment the hypothesis, defended by Segal (1997) that synaesthesia instead of showing a failure of modularity, shows the presence of an extra module. Synaesthetes would possess a supplementary module whose function is to transform a certain kind of input, for instance sound, into an unusual kind of output, like coloured sound. This supplementary module might be an innate oddity, or acquired through a tractable ontogenetic pattern. Synaesthetic processing is also fast and mandatory, even if it is not automatic in the strong sense of operating pre-attentionally like certain reflexes59 and as seems to be expected of Fodorian modules. Most importantly, the ‘extra-processing’ is also informationally encapsulated and opaque. Clearly, the fact that synaesthetes know that sounds don’t have colours has no effect on their experience and, therefore, on what could be the colour-hearing module. In addition, the synaesthete does not know why a certain mapping between sounds and colours occurs and this information remains opaque. The mapping is also fairly rigid—in the sense that it remains constant through life in spite of many contradictory experiences. Even if presented a hundred times with new sound-colour pairing, synaesthetes will not learn this new association. What might have started as a malleable state of association becomes stable, involuntary, and fast. Two things are put in balance here: on the one hand, calling synaesthesia an extra-module requires some adjustment to the notion of automaticity—as some synaesthetes seem to be able to exert some control over the intensity of their conscious concurrent, and synaesthesia is not pre-attentional (see Auvray and Deroy, Chapter 34, this volume). What’s more, as neurological investigation grows, it seems less and less easy to consider that synaesthesia is like a precise module localized in a specific area in the brain, as it is likely to involve a complex network and patterns of connectivity. On the other hand, the label seems to capture a crucial feature of synaesthetic associations, by contrast with other cross-modal associations present in everyone, which is precisely their rigidity, and their failure to adapt and disappear despite a large amount of exposure or voluntary training.60 57 

See Auvray and Deroy, Chapter 34, this volume. Ward et al., (2010). 60  See Deroy and Spence (2013) for a discussion. 59 

58 

Baron-Cohen et al., (1993).

772   Ophelia Deroy

4  A chance for revision? The most important challenges to modularity, as envisaged in Fodor’s initial model, don’t come from the threat of cognitive penetrability, that is, from the influence of propositional thinking on perceptual processing. It is of course possible to wonder what our minds would be like if cognitive penetration occurred, but I take it that this is deserting the concern for the actual working of the mind, for which modularity was theorized in the first place. Other challenges, originating in a growing series of studies of actual minds and brains, press us toward the best way to delineate and explain perceptual adaptation and flexibility, especially at the multisensory level. As these challenges become clearer, and alternative ways of thinking about perception or mental architecture are getting developed, it is fair to give modular models a chance to address these challenges. Can’t modular models be revised to accommodate them, and if so, in what ways? Two ways out are offered to modular models, which should not be seen as making the notion of modularity trivial but as updating the initial 1983 version of modularity.

Micro-modularity—the shallower, the better One way to introduce flexibility into modular architecture is to break each module into finer components. Each of them can more easily respect the Fodorian criteria of informational encapsulation: it is certainly, in principle, possible to find a small stage of processing which is both autonomous, fast, and mandatory. This process is also more likely to be neurologically localized, and to rescue most features of modules 1.0 albeit at the micro (or why not nano?) scale.61 Micro-modules might be hierarchically organized into integrative modules, corresponding to the more classical sensory modules posited by Fodor or Pylyshyn. The cost of this ‘micro-modularity’ line however concerns the nature of outputs: what micro-modules will produce will be all the more basic and different from what finally appears in conscious perceptual experience. Micro-modularity can explain perceptual processing, but it seems no longer adequate to explain our perceptual experience. This said, it is fair to underscore that the conscious requirement is not part of the definition of Fodorian modules. Outputs are shallow—not necessarily conscious. Some might be, and Fodor indeed uses the permanence of conscious visual illusions, like Muller-Lyer, to convince others that visual perception of size is modular. But this does not mean that outputs of modular processing are always or necessarily phenomenally conscious. Can we really perceptually experience the elementary features or dimensions which result from micro-modular processes, or is our conscious experience necessarily of bundles of features or fused features—and therefore not that shallow? How far toward the shallow end of representations our consciousness reaches is a fascinating issue, but orthogonal to modularity. 61  Bergeron (2007) suggests something along similar lines when he considers that the right way to think about (at least some) modules is as them doing some relatively narrow functional work, which then contributes to a larger functional role, and not trying to attribute each functional role to a single module.

Modularity of Perception    773

Fig.  40.7  The local inversion of the eyes and mouth in a face is not noticed when the face is upside down. When the face is upright, these inverted features make it look distorted, and even uncanny. This ‘inverted face illusion’ suggests that inputs fitting in the domain of the face recognition module are automatically processed as faces (adapted from Thompson,  1980). The real concern here seems to be that modular models stop being interesting if they grant that we might not be consciously experiencing the output of perceptual modular processing. The case of face perception is here interesting. Pushing aside some controversies, let’s consider that face perception is a fairly good candidate for being modular. For instance, it computes inputs in a way that is fast, automatic, encapsulated, as shown by the persistence of the inverted face illusion (Figure 40.7). It shows specific breakdowns, independent from visual processing (i.e. prosopagnosia). Thus defined, the face perception module takes as input the outputs of visual processing—including representations of colour, shape, texture, etc.—and generates a conscious perception of a face. Now are the initial visual representations independently experienced? In other words, do we also have the conscious experience of the outputs of visual processing that were fed into the face perception module? Several effects (also interpreted as examples of cognitive penetrability) have interpreted the difference in racial categorization of ambivalent faces coloured in the same way but presenting different morphological and hair features as the sign that the same visual output (e.g. the shade of skin colour) is processed as lighter or darker.62 There is no contradiction here in considering that we are only conscious of the face in general, in a holistic way, after it has been processed by the face perception module. What it means is that we are not independently conscious of the colour output of visual processing, but only conscious of a coloured-face where the colour and morphological elements have been made optimally consistent—like in the McGurk case where we are conscious of a phoneme which is optimally consistent with discrepant visual and auditory inputs. When and where the output of modular processing shows in consciousness is, again, an interesting question, but not one which in itself rules out (micro) modular models of the mind. 62 

Levin and Banaji, (2006).

774   Ophelia Deroy

Modularity 2.0 Modules can also be made more flexible in another way, which leads back to a remark which was present in Fodor (1983). Fodor’s remark that systems could count as modular as long as they satisfied most of the criteria to a certain degree suggests that modularity can be a graded matter. Perceptual systems in this sense can be thought of on a scale of automaticity or degree of innate specification, and as sharing more or less information, and more or less automatically so, as suggested by multisensory interactions. Although modularity 1.0 seems to be capable of updating into a more flexible model, a major revision is needed when it comes to cognitive impenetrability or encapsulation: this defining criteria requires that there should be no influence at all from conceptual cognitive states on the function computed by the module. Both the quantity of information exchanged by modules and the frequency at which it exerts an influence could be used here to help with this revision and to define a continuum from perfect encapsulation to a total exchange. This more flexible solution does not make modularity trivial, as what remains to be explained is the relative difference in modularity (or modularization) between various systems or various stages of life. Most of the current research questions in perception seem to break down into questions about degrees of automaticity, speed, domain specificity, fixed neuroanatomical localization, etc. All these questions remain perfectly sound, and can fit in a revised model where modules have limited interactions.

5 Conclusions Modular models made their entry in philosophy when computational models of the mind were dominant and the mind was seen as a computer. There is a hint of irony in the computer-related labels I gave here to the two possible (and non-exhaustive) ways to rescue or improve modular models, moving away from modules 1.0, accused of being too rigid. These two ways forward for modularity are to opt for micro (why not nano?) architecture, or to accept to be more interactive. However, what has mainly occurred in the field of modularity in the last decades is a resurgence of biological and evolutionary interest in the notion of modules. Many evolutionary psychologists have argued that modularity should be grounded in the notion of functional specialization rather than any of the Fodorian criteria.63 What comes first is the functional investigation, and it is from there that we can then look for the kind of architecture or implementation which realizes these cognitive functions—an area where revised modular architectures can still have their say. To conclude, the model of the mind/brain as an evolving biological entity might be ultimately signing a change from digital modular explanations to biological ones, as the explanatory project of understanding the mind as a computer is progressively fading. In that sense, what recent debates around Fodor (2000) suggest is that the real threat to modular models of perception is more likely to arise from 63 

For example, Barrett (2005); Sperber (1994); Tooby and Cosmides (1992).

Modularity of Perception    775 the rise of powerful, biologically inspired alternative models, like embodied accounts or predictive coding.

References Adams, W. J., Kerrigan, I. S., and Graf, E. W. (2010). ‘Efficient visual recalibration from either visual or haptic feedback: The importance of being wrong’. Journal of Neuroscience, 30, 14745–14749. Balcetis, E. and Dunning, D. (2010). ‘Wishful seeing: Desired objects are seen as closer’. Psychological Science, 21, 147–152. Baron-Cohen, S., Harrison, J., Goldstein, L., and Wyke, M. (1993). ‘Coloured speech perception: Is synaesthesia what happens when modularity breaks down?’ Perception, 22, 419–426. Barrett, H. C. (2005). ‘Enzymatic computation and cognitive modularity’. Mind & Language, 20, 259–287. Barrett, H. C. and Kurzban, R. (2006). ‘Modularity in cognition: Framing the debate’. Psychological Review, 113, 628–647. Bayne, T. (2009). ‘Perceptual experience and the reach of phenomenal content’. Philosophical Quarterly, 59, 385–404. Bayne, T. (2010). The Unity of Consciousness. Oxford: Oxford University Press. Bergeron, V. (2007). ‘Anatomical and functional modularity in cognitive science: Shifting the focus’. Philosophical Psychology, 20, 175–195. Brugger, P. and Brugger, S. (1993). ‘The Easter Bunny in October: Is it disguised as a duck?’ Perceptual & Motor Skills, 76, 577–578. Bruner, J. S. and Goodman, C. C. (1947). ‘Value and need as organizing factors in perception’. Journal of Abnormal and Social Psychology, 42, 33–44. Butterfill, S. (2007). ‘What are modules and what is their role in development?’ Mind & Language, 22, 450–473. Carruthers, P. (2003). ‘Moderately massive modularity’. In A. O’Hear (ed.), Mind and Persons (pp. 69–91). Cambridge: Cambridge University Press. Collins, J. M. (2005). ‘On the input problem for massive modularity’. Minds and Machines, 15, 1–22. Collins, J. M. (2007). ‘Linguistic competence without knowledge’. Philosophy Compass, 2, 880–895. Coltheart, M. (1999). ‘Modularity and cognition’. Trends in Cognitive Sciences, 3, 115–120. Cosmides, L. and Tooby, J. (1992). ‘Cognitive adaptations for social exchange’. In J. Barkow, L. Cosmides, and J. Tooby (eds), The Adapted Mind. Oxford: Oxford University Press. Cosmides, L. and Tooby, J. (1994). ‘Origins of domain specificity: The evolution of functional organization’. In L. Hirschfeld and S. Gelman (eds), Mapping the Mind: Domain specificity in cognition and culture (pp. 85–116). New York: Cambridge University Press. Delk, J. L. and Fillenbaum, S. (1965). ‘Differences in perceived colour as a function of characteristic color’. American Journal of Psychology, 78, 290–293. Deroy, O. (2013). ‘Phenomenal contrast without cognitive penetrability of perception’. Philosophical Studies, 162, 87–107. Deroy, O. and Spence, C. (2013). ‘Why we are not all synesthetes (not even weakly so)’. Psychonomic Bulletin & Review, 20, 1–22.

776   Ophelia Deroy Driver, J. and Spence, C. (2000). ‘Multisensory perception: Beyond modularity and convergence’. Current Biology, 10, 731–735. Feyerabend, P. (1962). ‘Explanation, reduction and empiricism’. In H. Feigl and G. Maxwell (eds), Minnesota Studies in Philosophy of Science, vol. iii. Minneapolis: University of Minnesota Press. Fodor, J. (1975). The Language of Thought. New York: Thomas Crowell. Fodor, J. (1981). Representations. Cambridge, MA: Bradford Books/MIT Press. Fodor, J. (1983). Modularity of Mind:  An essay on faculty psychology. Cambridge, MA: MIT Press. Fodor, J. (1987). Psychosemantics. Cambridge, MA: Bradford Books. Fodor, J. (2000). The Mind doesn’t Work that Way: The scope and limits of computational psychology. Cambridge, MA: MIT Press. Francis, G. (2012). ‘The same old New Look: Publication bias in a study of wishful seeing’. I-Perception, 3, 176–178. Gallistel, C. R. (1999). ‘The replacement of general-purpose learning models with adaptively specialized learning modules’. In M. S. Gazzaniga (ed.), The Cognitive Neurosciences (2nd edn) (pp. 1179–1191). Cambridge, MA: MIT Press. Gottfried, J. A. (2010). ‘Central mechanisms of odour object perception’. Nature Reviews Neuroscience, 11, 628–641. Hansen, T. and Gegenfurtner, K. R. (2006). ‘Color scaling of discs and natural objects at different luminance levels’. Visual Neuroscience, 23, 603–610. Hansen, T., Olkkonen, M., Walter, S., and Gegenfurtner, K. R. (2006). ‘Memory modulates color appearance’. Nature Neuroscience, 9, 1367–1368. Heck, R. G. (2007). ‘Are there different kinds of content?’ In B. P. Mclaughlin and J. Cohen (eds), Contemporary Debates in Philosophy of Mind (pp. 117–138). New York: Blackwell Publishing. Herz, R. S. and von Clef, J. (2001). ‘The influence of verbal labeling on the perception of odors: Evidence for olfactory illusions?’ Perception, 30, 381–391. Hillis, J. M., Ernst, M. O., Banks, M. S., and Landy, M. S. (2002). ‘Combining sensory information: Mandatory fusion within, but not between senses’. Science, 298, 1627–1630. Hirschfeld, L. A. and Gelman, S. A. (eds.) (1994). Mapping the mind: Domain specificity in cognition and culture. Cambridge: Cambridge University Press. Jastrow, J. (1899). ‘The mind’s eye’. Popular Science Monthly, 54, 299–312. Jastrow, J. (1900). Fact and Fable in Psychology. Boston: Houghton Mifflin. Jeannerod, M. and Jacob, P. (2005). ‘Visual cognition: A new look at the two-visual systems model’. Neuropsychologia, 43, 301–312. Juola, P. and Plunkett, K. (2000). ‘Why double dissociations don’t mean much’. In G. Cohen, R. A. Johnston, and K. Plunkett (eds), Exploring Cognition: Damaged brains and neural networks (pp. 319–327). Hove: Psychology Press. Kuhn, T. (1962). The Structure of Scientific Revolutions. Chicago: University of Chicago Press. Levin, D. T. and Banaji, M. R. (2006). ‘Distortions in the perceived lightness of faces: The role of race categories’. Journal of Experimental Psychology: General, 135, 501–512. Long, G. M. and Toppino, T. C. (2004). ‘Enduring interest in perceptual ambiguity: Alternating views of reversible figures’. Psychological Bulletin, 130, 748–768. Machery, E. (2009). Doing without Concepts. Oxford: Oxford University Press. Macpherson, F. (2012). ‘Cognitive penetration of colour experience: Rethinking the issue in light of an indirect mechanism’. Philosophy and Phenomenological Research, 84, 24–62. Marr, D. (1983). Vision: A Computational Investigation into the Human Representation and Processing of Visual Information. New York: W. H. Freeman and Company.

Modularity of Perception    777 Matthen, M. (2010). ‘On the diversity of auditory objects’. Review of Philosophy and Psychology, 63–89. Milner, A. D. and Goodale, M. A. (2008). ‘Two visual systems re-viewed’. Neuropsychologia, 46, 774–785. Mitterer, H. and de Ruiter, J. P. (2008). ‘Recalibrating color categories using world-knowledge’. Psychological Science, 19, 629–634. Mitterer, H., Horschig, J. M., Musseler, J., and Majid, A. (2009). ‘The influence of memory on perception: It’s not what things look like, it’s what you call them’. Journal of Experimental Psychology: Learning, Memory, and Cognition, 35, 1557–1562. Müller-Lyer, F. C. (1889). ‘Optische Urteilstäuschungen’. Archiv für Physiologie Suppl., 263–270. Nassi, J. J. and Callaway, E. M. (2009). ‘Parallel processing strategies of the primate visual system’. Nature Reviews in Neuroscience, 10, 360–372. O’Callaghan, C. (2011). ‘Perception and multimodality’. In E. Margolis, R. Samuels, and S. Stich (eds), The Oxford Handbook of Philosophy of Cognitive Science (pp. 92–117). Oxford: Oxford University Press. Piccinini, G. (2009). ‘Computationalism in the philosophy of mind’. Philosophy Compass, 4, 515–532. Plaut, D. C. (1995). ‘Double dissociation without modularity: Evidence from connectionist neuropsychology’. Journal of Clinical & Experimental Neuropsychology, 17, 291–321. Prinz, J. (2006). ‘Is the mind really modular?’ In R. Stainton (ed.), Contemporary Debates in Cognitive Science (pp. 22–36). London: Blackwell. Putnam, H. (1963). ‘Brains and behavior’. In R. J. Butler (ed.), Analytical Philosophy, Second Series (pp. 211–235). Oxford: Basil Blackwell. Pylyshyn, Z. (1984). Computation and Cognition: Toward a foundation for cognitive science. Cambridge, MA: Bradford Books/MIT Press. Pylyshyn, Z. (1999). ‘Is vision continuous with cognition? The case for cognitive impenetrability of visual perception’. Behavioral and Brain Sciences, 22, 341–423. Pylyshyn, Z. W. (2003). Seeing and Visualizing: It’s not what you think. Cambridge, MA: MIT Press. Raftopoulos, A. (2001). ‘Is perception informationally encapsulated? The issue of the theoryladenness of perception’. Cognitive Science, 25, 423–451. Raftopoulos, A. (2011). ‘Late vision: Processes and epistemic status’. Frontiers in Psychology, 2, 382. Rossion, B. and Michel, C. (2011). ‘An experience-based holistic account’. In A. Calder, G. Rhodes, M. Johnson, and J. Haxby (eds), Oxford Handbook of Face Perception (p. 215). Oxford: Oxford University Press. Samuels, R. (2006). ‘Is the mind massively modular?’ In R. Stainton (ed.), Contemporary Debates in Cognitive Science (pp. 37–56). London: Wiley-Blackwell. Scholl, B. J. and Leslie, A. (1999). ‘Modularity, development and "theory of mind" ‘. Mind & Language, 14, 131–153. Segal, G. (1997). ‘Synaesthesia: Implications for modularity of mind’. In S. Baron-Cohen and J. Harrison (eds), Synaesthesia: Classic and contemporary readings (pp. 211–223). Cambridge, MA: Blackwell. Shankar, M. U., Levitan, C. A., and Spence, C. (2010). ‘Grape expectations: The role of cognitive influence on color-flavor interactions’. Consciousness and Cognition, 19, 380–390. Siegel, S. (2011). ‘Cognitive penetrability and perceptual justification’. Noûs, 46, 201–222. Sincich, L. C. and Horton, J. C. (2005). ‘The circuitry of V1 and V2: Integration of color, form, and motion’. Annual Review Neuroscience, 28, 303–326.

778   Ophelia Deroy Smith, B. C. (2006). ‘What I know when I know a language’. In E. Lepore and B. C. Smith (eds), The Oxford Handbook of Philosophy of Language (pp. 941–982). Oxford: Oxford University Press. Smith, B. C. (2007). ‘The objectivity of taste and flavours’. In B. C. Smith (ed.), Questions of Taste. New York: Oxford University Press. Spence, C. and Deroy, O. (2012). ‘Crossmodal correspondences: Innate or learned?’ I-Perception, 3, 316–318. Spence, C. and Squire, S. (2003). ‘Multisensory integration: maintaining the perception of synchrony’. Current Biology, 13, R519–R521. Sperber, D. (1994). ‘The modularity of thought and the epidemiology of representations’. In L. A. Hirschfeld and S. A. Gelman (eds), Mapping the Mind (pp. 39–67). Cambridge: Cambridge University Press. Sperber, D. (2002). ‘In defense of massive modularity’. In E. Dupoux (ed.), Language, Brain, and Cognitive Development (pp. 47–57). Cambridge, MA: MIT Press. Stokes, D. (2012). ‘Perceiving and desiring: A New Look at the cognitive penetrability of experience’. Philosophical Studies, 158, 477–492. Tacca, M. C. (2010). Seeing Objects: The structure of visual representations. Paderborn: Mentis. Thompson, P. (1980). ‘Margaret Thatcher: A new illusion’. Perception, 9, 483–484. Treisman, A. M. (1993). ‘The perception of features and objects’. In A. Baddeley and L.  Weiskrantz (eds), Attention: Selection, awareness, and control (pp. 1–36). Oxford: Clarendon Press. van Ulzen, N. R., Semin, G. R., Oudejans, R., and Beek, P. (2008). ‘Affective stimulus properties influence size perception and the Ebbinghaus illusion’. Psychological Research, 72, 304–310. Waltz, D. (1975). ‘Understanding line drawings of scenes with shadows’. In P. H. Wilson (ed.), The Psychology of Computer Vision (pp. 19–91). New York: McGraw-Hill. Ward, J., Jonas, C., Dienes, Z., and Seth, A. (2010). ‘Grapheme-colour synaesthesia improves detection of embedded shapes, but without pre-attentive "pop-out" of synaesthetic colour’. Proceedings Biological Sciences, 277, 1021–1026. Wittgenstein, L. (1953/1958). Philosophical Investigations, trans. G. E. M. Anscombe (2nd edn). Oxford: Blackwell.

Pa rt  V i i

BROA DE R PH I L O S OPH IC A L IS SU E S

Chapter 41

The Epistemology of Perception Susanna Siegel and Nicholas Silins

1 Introduction Seeing a jar of mustard in the refrigerator can give you reason to believe that the fridge contains mustard. Or so it seems natural to suppose. When you see a jar of mustard, you have a perceptual experience, or experience for short, and we’ll say that when experiences provide reason for beliefs, they justify them.1 Some philosophers have denied that experiences can justify beliefs. Donald Davidson (1986) held that the transition from experience to belief is merely causal, rather than rational, on the grounds that experiences are not beliefs, and that only beliefs can justify other beliefs. Some sceptics hold that no external-world beliefs are justified, a fortiori that none are justified by experience. Other philosophers assume that experiences justify only introspective beliefs, and that perceptual justification, and more generally empirical knowledge, has to be reconstructed as an inference from an introspective belief to an external-world belief.2 In this entry, we begin from the assumption that experiences (such as the one you have when you see the mustard) can justify external world beliefs about the things you see, such as beliefs that the mustard jar is in the fridge. From now on, we often let it remain implicit that we are talking about external world beliefs, when we talk about the kind of beliefs that experiences justify.3 Our main question is this: what features of experiences explain how they justify external world beliefs?

1 

We focus on the visual case, leaving it to the reader to consider how the discussion generalizes to other modalities. 2  This structure is the hallmark of indirect realist theories of empirical knowledge. Different versions of this structure are found in Chisholm (1966), Russell (1912), and Ayer (1973). On the relationship between indirect realism and early modern theories of perception and its epistemic role, see Caston (Chapter 1), Perler (Chapter 2), and Simmons (Chapter 4) in this volume. 3 The distinction between external world beliefs and beliefs about one’s mental states can seem oversimplified when one considers the perspectival characteristics of visual experience. For discussion, see Peacocke’s entry on vision and the first person.

782    Susanna Siegel and Nicholas Silins We clarify our question further in Part I, where we explain why we have chosen this point of departure, and highlight a range of theses about the role of experience in providing different types of justification. In Parts II and III, we consider the role of two categories of features of experience. Part II considers constitutive features of experience, including its phenomenal character, its contents, and its status as attentive or inattentive (sections 3–7). Part III considers causal features of experience, such as its reliability, and the impact of other mental states on its formation (sections 8–10). Along the way, we discuss the relationships between visual experience and seeing (sections 1 and 8), and we contrast perceptual justification and perceptual knowledge (section 9).

2 Part i  Our point of departure For many philosophers, the topic of perceptual justification takes its shape from the idea that experience differs fundamentally from belief. For instance, in his classic discussion of perceptual justification, Sellars (1956) considers whether experiences could be foundations of knowledge, if they were acts of sensing particular mental objects, and not states with contents that can be correct or incorrect depending on what’s in the subject’s surroundings. If experiences do not represent or refer to how things are in the external world, part of the problem of perceptual justification will be to explain how to rationally bridge the divide between states that do not represent the external world, and states that do. In contrast, if experiences already represent or refer to things in the external world, then a theory of how the transition from experience to belief can be rational need not also explain how to bridge that divide. The philosophical problems that give shape to the topic of perceptual justification look very different, depending on whether it’s assumed that experiences represent or refer to how things are in the external world.4 Rather than departing from Sellars’s traditional starting point, in which experiences don’t represent or refer to the external world, we begin from two assumptions about the nature of visual experiences that have become entrenched in many contemporary discussions. First, we assume experiences have externally directed contents, ones that determine at least some of the propositions that are good candidates for being justified by the experience. Second, we assume that perceivers need not believe that things are as experiences present them, even though often they do believe this. Because these assumptions have become widespread, we want to outline the epistemological problems they give shape to, and the options they open up for solving those problems. Even readers who reject these assumptions may want to see what the problems of perceptual justification look like, once they are made.

Experiences Our central question asks about the rational role of conscious visual experiences in justifying beliefs about what you see.5 So far, we’ve referred to a conscious state or episode of 4 

For an overview of responses to this problem that reject the assumption, see BonJour (2009). Any rational role for unconscious perception is beyond the scope of our discussion, although we will touch on related issues at the beginning of sections 3 and 5. 5 

The Epistemology of Perception    783 seeing as an experience.6 Since both ‘experience’ and ‘seeing’ have multiple uses in ordinary language and philosophy, we pause to explain how we use these terms. In our usage, an experience is a phenomenal state, individuated by what it is like to be in that state (or equivalently, by its phenomenal character). Some phenomenal states are distinct from any states of seeing, which are in turn individuated by relations to one’s surroundings. If you were hallucinating when you opened the fridge, for example, you would be having a visual experience, but wouldn’t be seeing anything. It is controversial how phenomenal states are related to states of seeing. It is thus also controversial whether any experiences in our sense are identical with any states of ­seeing.7 When we ask about the epistemic role of experiences, we are asking about the role of phenomenal states, whatever their relation to states of seeing turns out to be. For the sake of fixing ideas, however, it is easiest to use phrases such as ‘mustard-experience’ and ‘hand-experience’ to denote experiences, whether they are hallucinations or not, in which you seem to see mustard (or hands) and it looks to you as if there is some mustard (or there are some hands) in front of you.8 In section 8, we discuss epistemic roles that may be ­specific to states of seeing.

Justification Justification is a normative notion, tied to what it is rational for a subject to believe.9 Within the basic normative notion of justification, we can distinguish between two rational roles experiences can play. These roles can be elucidated using the notions of propositional justification, which concerns (roughly) what reasons we have, and doxastic justification, which concerns (roughly) how we respond to reasons we have. Suppose you suspect that there is mustard in the fridge, and open the door to check. There’s the mustard, in plain view. You see it, and notice it, and don’t suffer any illusion. Whether or not you actually increase your confidence that there’s mustard in the fridge, it would be rational for you to do so. We’ll say that an experience of a subject provides propositional justification if and only if it provides justification for a proposition, whether or not the subject believes the proposition or adjusts her confidence in it on the basis of the experience. The notion of propositional justification arises from the idea that we can ask what kind of rational support a mental state provides for believing a proposition, while abstracting away from the role it actually plays (if any) in the subject’s forming or maintaining a belief in that proposition. 6 Since

the differences between states and episodes are largely irrelevant to our discussion, we ignore them. 7 Some disjunctivists about phenomenal character identify some phenomenal states with certain states of seeing, such as the state of seeing the mustard when it looks yellow. For discussion, see Soteriou (2010). 8 Of course these characterizations of the phenomenal character of experience are exceedingly simplified. 9  This notion of justification leaves several substantive issues unsettled. First, it is not tied by definition to being able to produce explicit reasons, or to being blameless in forming or maintaining a belief. For discussion, see Pryor (2001). Second, it is an open question exactly what normative notion justification or epistemic rationality is. Standardly it is taken as the form of permissibility, so that justified beliefs are those it is permissible to form. In some cases, obligation rather than permissibility seems to be at issue. For instance, in some visual cases it is arguably irrational not to believe your eyes, and in other cases it is arguably irrational not to believe obvious logical consequences of what you already rationally believe.

784    Susanna Siegel and Nicholas Silins In contrast, the notion of doxastic justification arises from the idea that there are rationally better and rationally worse ways to form and maintain beliefs. For instance, normally, looking in the fridge is an epistemically good way to form beliefs about whether the fridge contains mustard. The idea that experiences can lead to doxastically justified beliefs is closely related to the more general idea that some beliefs are based on experience, just as they can be based on other beliefs. Very roughly, a belief that is based on a mental state M is a response to M. Paradigmatically, your belief that you are hungry will be based on your feeling of hunger, and your belief that tomorrow is Wednesday will be based on your belief that it is Tuesday. Satisfactory analyses of the basing relation have proven elusive. But such a notion is needed if there are rationally better and worse ways in which beliefs can be formed or maintained.10 We’ll say that a belief is doxastically justified by an experience if and only if it is rationally formed, adjusted, or maintained on the basis of experience. (We can think of adjusting beliefs as special cases of forming them.) In principle, one could approach the topic of perceptual justification by starting with justified beliefs that are formed as the result of perception, and then ask: • What kind of process gave rise to that belief? • Which aspects of the process, if any, made it a rational process by which to form the belief? • What role did the perceptual experience play in that process? Analogous questions could be asked for adjustments of beliefs. These questions look backward at the aetiology of the belief, and ask about the relationship between the belief’s aetiology and its epistemic status. Our starting point is different. We focus mainly on propositional justification. Rather than start with beliefs, we start with experiences and ask: • Given an experience, which propositions, if any, does this experience provide rational support for believing? • Which features of the experience make it the case that it can provide rational support for those propositions? These questions approach the topic by looking forward from experiences to the propositions they rationally support. We can divide the features of experience that potentially explain what makes them support propositions into two broad categories: features related to the constitutive nature of experience, and features related to the aetiology of experience. Both categories are examined in Parts II and III.11 In the rest of Part I, we draw more distinctions within the basic normative notion of justification, to highlight different aspects 10  One construal of basing allows for ‘bad basing’: a belief B can be based on a mental state M, where M gives propositional justification to hold B, even if B is not thereby doxastically justified. On a different construal of basing, basing B on a mental state which supplies propositional justification for B’s content is sufficient for B to be doxastically justified. For further discussion of the basing relation, see Lehrer (1971), Swain (1979), Audi (1986), Korcz (1997), and Turri (2010). 11  A feature of an experience could be both constitutive and causal. For instance, according to a standard externalist theory of content determination inspired by Putnam’s theory of natural kind terms (Putnam, 1975) and Burge’s theory of deference (Burge, 1982), a mental state has the content it does by virtue

The Epistemology of Perception    785 of normative support that experiences could in principle provide for beliefs. We begin with the ways in which experiences and prior beliefs rationally interact. Suppose you know that it is unlikely for there to be money in the fridge, but when you open the fridge door, you see some money (and it looks like money). In many cases, it seems plain that you can rationally believe that there is money in the fridge, on the strength of your experience. But is it always rational for experiences to override prior beliefs in this way? Suppose you know there is unlikely to be water in the distance in the desert—even if you seem to see some—and when you look ahead in the desert you seem to see a pool. In such a case, the rational thing to do is presumably to raise your confidence that you are seeing a mirage, rather than to revise your antecedent expectation. A theory of perceptual justification should allow prior beliefs to influence the epistemic status of experiences. When prior beliefs have a negative influence on the rational support provided by experiences, they act as defeaters.12 In a simple form of defeat, they remove all justificatory force from experience. In more complex forms of influence, prior beliefs reduce the justificatory force that experiences provide without completely eliminating it. If the epistemic status of experiences is sensitive to prior beliefs and their epistemic status, then a theory of perceptual justification needs a way to describe this kind of sensitivity. For instance, one could frame a theory around the question of which transitions to a new doxastic state are licensed by an experience, given one’s initial overall doxastic state.13 This framework employs the basic normative notion of justification to describe changes in overall doxastic states. It is one thing for prior beliefs to defeat an experience as a source of rational support for a belief. It is another for prior beliefs always to be needed, for experiences to provide rational support at all. The idea that prior beliefs are not always needed can be sharpened by the notion of immediate justification.14 Intuitively, when you know you are in pain, the only source of justification you are relying on is the pain itself. You are not relying on separate grounds for believing that your pain beliefs are reliable, or on background beliefs that identify your sensation as a pain. Your pain instead gives you justification to believe you are in pain in a way which does not rely on your having reason to hold any other beliefs.15 When applied to perceptual experiences, the notion of immediate justification figures in defences of foundationalism, the view that the justification of all beliefs ultimately depends

of the state’s standing in certain causal relations (roughly, a mental state represents redness if it tends to be tokened by red things). If a mental state is partly constituted by having the contents it does, then according to these externalist theories, having those contents is both a constitutive feature of the state and a causal feature of it. Externalist theories are applied to the contents of visual experiences by Dretske (1997), Tye (1995), Lycan (2001), and Burge (2003, 2010). Lycan and Dretske identify the phenomenal character of experiences with the property of having specific content. Once that move is made, phenomenal character is another example of a feature of experience that is both constitutive and causal. 12  One might say that prior beliefs as such are never defeaters, instead ascribing all negative effects of defeat to one’s justification to have to those beliefs, so that an unjustified belief would never have a defeating effect. For discussion of this issue, see Pryor (2004). 13  For approaches of this sort, see Gupta (2006). 14  For defences of the thesis that our experiences immediately justify some external world beliefs, see Pollock (1974), Pryor (2000, 2005), and Huemer (2001). 15  Immediate justification can be defined in terms of mediate justification. Your experience E gives you mediate justification to believe that P just in case E gives you justification to believe that P, in a way which depends on your having justification to believe some proposition, from some source other than E.

786    Susanna Siegel and Nicholas Silins on a special class of beliefs, which need not themselves be justified by relations to any other beliefs.16 The notion also figures in responses to scepticism about knowledge and justification along the lines of G. E. Moore, who claimed to refute the sceptic by looking at his hands. (We discuss Moore’s reasoning below in this section.) Even if experiences sometimes provide immediate justification, having an experience need not suffice to provide any kind of justification. Suppose that your hand-experience (call it E) provides immediate justification for believing that you have hands. Some contingent factor might still need to be added to E, in order for E to immediately justify this proposition (or any other). For example, E might need to be part of a process that reliably produces true beliefs, or it might need to be a case of seeing. We can thus distinguish between two theses concerning immediate justification by experience. Immediacy: For some external-world proposition P, there is an experience E which provides immediate justification for P. Sufficiency-for-IJ: Necessarily, if you have an experience with content P, then the experience gives you immediate justification for P.

The Sufficiency-for-IJ thesis bears on a central question in epistemology. If we want to carve perceptual justification at its joints, will experience turn out to be a basic element in the story? Or will the most basic elements be combinations of experiences with other mental factors (such as beliefs, inferential dispositions, or other mental states) or other non-mental factors (such as causes of the experience)? Immediacy leaves both options open, whereas Sufficiency-for-IJ entails that experiences are joints in the basic structure of justification. Just what else besides a phenomenal state constitutes the joint will depend on the ultimate explanation for what makes the experience provide immediate justification. As stated, the Sufficiency-for-IJ entails that an experience provides justification, even if you know that you’re hallucinating. But you should lower your confidence in such a case. To avoid the result that you shouldn’t lower your confidence, the Sufficiency-for-IJ thesis could be modified using the notion of prima-facie justification. A subject’s knowledge that she is hallucinating is a paradigm of a defeater for the experience.17 An experience provides prima-facie justification if and only if it provides justification, in the absence of defeaters. Here’s the thesis modified: Sufficiency-for-pf-IJ: Necessarily, if you have an experience E with content P, then E gives you immediate prima facie justification for P. For example, your experience gives you justification to believe that it will rain, in a way which depends on your having justification from memory to believe that, if there are dark clouds, then it will rain. Immediate justification can now be defined as follows: E gives you immediate justification to believe that P just in case E gives you justification to believe that P that is not mediate justification to believe that P. This definition allows that an experience can immediately justify a subject in believing more than one proposition, such as the proposition that you have hands, and the proposition that you are seeing your hands, so long as E is the sole source on which you are relying for justification in believing both propositions. For further clarification of the notion of immediate justification, see Audi (1993) and Pryor (2000, 2005). 16  17 

For discussion of foundationalism, see BonJour (1985) and Audi (1993: chs. 1–4). For more on defeat, see Pollock (1986) and Pollock and Cruz (1999).

The Epistemology of Perception    787 Other sufficiency theses can be defined independently of the notion of immediate justification. These too entail that experiences form a joint in the overall structure of justification. We now turn from exposition of theses concerning immediate justification to their evaluation. The idea that experiences can provide immediate justification at all has been challenged on the grounds that it attributes to experiences more justificatory power than they have. We describe two of the most influential challenges of this sort.18 First, Immediacy seems to allow that we have justification from experience to believe the following Moorean premiss, without having to already have justification to believe the Moorean conclusion. Moorean Premiss: I have hands. Link: If I have hands, then I am not a handless brain in a vat.

Therefore: Moorean Conclusion: I am not a handless brain in vat.

But if one has immediate justification to believe the premiss of the argument, nothing would seem to bar one from acquiring justification to believe the conclusion simply by deducing it from the premiss. According to the easy justification objection, Immediacy allows one to gain justification to reject sceptical hypotheses too easily.19After all, if one were a handless brain in a vat, one’s hands-experience would be inaccurate. According to the objector, we cannot rely on experience itself to answer questions about its own accuracy. (Compare the following observation often attributed to Kripke: we cannot rely on witnesses to testify to their own accuracy). If so, then we cannot become justified in rejecting sceptical hypotheses by the inference corresponding to the argument above. And if we cannot gain justification to reject sceptical hypotheses by performing such inferences, the objector says, we do not gain immediate justification from our experiences for external world beliefs either.20 Proponents of Immediacy have several lines of response to this argument. A first response embraces the Moorean reasoning, on the grounds that the inference can indeed be successful, and the sense that it can’t provide justification can be explained away. For instance, the inference might merely seem defective, because of its dialectical impotence to persuade an interlocutor who doubts the conclusion, leaving open that it provides justification for the subject who performs the inference (Pryor, 2004). Compare: when I reason that I must exist given that I am thinking, I acquire justification to believe that I exist in a perfectly legitimate way, despite the fact that the reasoning will probably not rationally 18  Other challenges for the Sufficiency theses are surveyed in sections 8–10. A further challenge comes from the example of the ‘speckled hen’, which goes back at least as far as Chisholm (1942)—if you see a speckled hen in good conditions, and the side facing you has say 39 speckles, does your experience both represent that there are 39 speckles and give you justification to believe that there are 39 speckles? For discussion of how much detail our experiences represent, and of whether they give us justification to believe their most specific contents, see Sosa (2003) and Feldman (2004). 19  We use the term ‘easy justification’ to echo Cohen (2002)’s discussion of ‘easy knowledge’. See also Wright (1985, 2000) and White (2006). 20  The argument can be expanded into one for scepticism, when combined with the further claim that nothing other than an experience could justify one in rejecting sceptical hypotheses, and with the claim that we must have justification to reject sceptical hypotheses to have justification from our experiences. For discussion, see Pryor (2000) and Wright (2004).

788    Susanna Siegel and Nicholas Silins persuade someone who doubts that I exist. Alternatively, perhaps the inference seems defective because we underestimate what justifies the subject in believing the Moorean premiss. If the experiences that justify one in believing the Moorean premiss are also states of seeing hands (as we’ll discuss in section 7), then the state which justifies one in believing the Moorean premiss guarantees the truth of the Moorean conclusion. According to this line of thought, an experience that can be had, only if the Moorean conclusion is true, is a good candidate for providing justification to believe the Moorean conclusion (for criticism, see Wright, 2002). A second response to the easy-justification objection denies that Immediacy legitimates Moorean reasoning. According to this response, the hand-experience gives one immediate justification to believe that one has hands, without providing justification (via inference) to deny that one is a handless brain in a vat. One possibility here is that an experience could give one immediate justification to believe an ordinary proposition about the external world, while one fails to have any justification to reject the sceptical hypothesis.21 Another possibility is that when an experience gives one immediate justification to believe an ordinary proposition about the external world, one has an independent source of justification to reject the sceptical hypothesis, even though that independent source is not part of what gives one perceptual justification to believe the ordinary proposition in the first place (Silins, 2008). Compare: whenever you have perceptual justification to believe you have hands, you have independent justification to believe the triviality that all hands are hands, but you do not have perceptual justification to believe that you have hands in virtue of having independent reason to believe that all hands are hands. Like the easy-justification objection to Immediacy, the bootstrapping objection develops the idea that Immediacy makes justification too easy. According to the Bootstrapping objection, Immediacy implies that one’s experiences can give one justification to believe that they themselves are reliable, where experiences have no power to do any such thing (Cohen, 2002). Suppose one forms a series of justified beliefs of this form: Something at location L has property F, and it visually seems to me that something at L has F.

According to the objection, one could then deduce that one’s experiences were accurate on all the occasions surveyed, and one could then rationally conclude by induction that one’s experiences are reliable—why else would they have been accurate on all those occasions? Just as the proponent of Immediacy could embrace the Moorean reasoning above, here too, a first response to the bootstrapping objection is to maintain that we can have justification, via experience, to believe our experiences are reliable, on the grounds that this seems to be the only possible source of justification to believe they are reliable, barring a great expansion of the domain of a priori justification (see Van Cleve (2003) for discussion). Alternatively, one might block the bootstrapping inference by proposing that an experience justifies one in believing that P, only if one already has justification to believe that experience is a reliable source. In particular, we might have have a priori justification to believe that one’s experiences are reliable thanks to the availability of what Wedgwood 21  This move holds that you can have justification for P, know that P entails Q, yet lack justification for Q. It is defended by Dretske (1970).

The Epistemology of Perception    789 calls an ‘a priori bootstrapping’ argument. When reasoning through such an argument, one would suppose that one has an experience with the content that P, and then infer, on the supposition one has the experience with the content that P, that it is the case that P. One could then conclude that, if one has the experience with the content that P, P. Thanks to the possibility of putting such inferences together, one might have a priori justification to believe that experience is a reliable source whenever one has reason from an experience for a belief about the external world.22 A third response is that bootstrapping reasoning uses a defective form of induction, where the defect has nothing specific to do with whether experiences ever immediately justify beliefs. This response entails that Immediacy makes no prediction about the legitimacy of bootstrapping reasoning (Weisberg, 2010). If these objections can be answered, then a further question is: in virtue of what do experiences provide immediate justification? If the objections stand, then our starting question remains: in virtue of what do experiences provide justification at all, whether it is immediate or not? In principle, the same answers may apply to both questions. We now turn to two types of answers: those that invoke constitutive features of experience, and those that invoke causal features of experience.

3 Part ii  Constitutive features of experience The Phenomenal Approach According to the Phenomenal Approach, experiences provide justification at least partly in virtue of their phenomenal character.23 Some proponents of the Phenomenal Approach motivate it by contrasting the epistemic situation of sighted and blindsighted subjects. For instance, consider a sighted subject who enjoys a visual experience of a basketball, while a blindsighted subject has no experience of the ball but nevertheless registers its presence in unconscious perceptual processing. Across a range of cases, both subjects reliably form accurate judgements about whether a basketball is present. If the sighted subject has more justification for believing that a ball is there, or a different kind of justification, then one might think that the justificatory difference is due to the conscious character of her experience, since the conscious and the unconscious perceptions are so similar in their other features.24 Smithies (2011) draws on principles to give a direct argument for the Phenomenal Approach by appealing to a version of access internalism about justification, the idea that the factors that determine whether a subject is justified in believing a proposition are both internal and accessible to the subject.25 According to Smithies, the introspective

22  For

further discussion, see Cohen (2010) or Wedgwood (2013). Wedgwood (2013) addresses the question of whether the move is compatible with Immediacy. 23  Campbell (2002), Pryor (2000), Huemer (2001), Johnston (2006), Smithies (2011). 24  Not all theorists will agree that the sighted subject has more justification to believe that an orange sphere is present, or even be disposed to make the intuitive judgement that the sighted subject has more justification to believe that an orange sphere is present. See Lyons (2009) and Burge (2003). 25  On access internalism, see Conee and Feldman (2001).

790    Susanna Siegel and Nicholas Silins accessibility of the sighted subject’s visual experience enables it to provide justification, whereas the introspective inaccessibility of the blindsighter’s subpersonal state makes it unable to provide justification.26 Other philosophers defend the Phenomenal Approach indirectly, by first arguing that phenomenal character of experience is directly implicated in other features of experiences, and that these features in turn help explain how experiences justify external world beliefs. Some features of experience potentially tied to phenomenal character in this way include attention, states of seeing, and being a state of seeming with accuracy conditions. The Phenomenal Approach could also be bolstered by a conception of phenomenal character of perceptual experiences that fits naturally with the idea that experiences have accuracy conditions. This conception of phenomenal character has two strands. The first strand is that phenomenal character conveys information about external objects. The idea that experiences provide justification for external world beliefs in virtue of their phenomenal character might well seem less compelling, against the background assumption that it is a raw feel or mere sensory affect that does not present any properties as being instantiated in the space around the perceiver. For instance, according to Laurence BonJour (2001), in virtue of their phenomenal character, experiences immediately justify self-ascriptions of experiences, but not external world beliefs. Perhaps BonJour was drawn to this position by his assumption that one could only describe the phenomenal character of experience ‘in terms of patches of color arranged in two-dimensional visual space’ (2001: 32). Likewise, if Davidson (1986) had thought that experiences were belief-like in ways that allowed their contents to stand in the same kinds of relations (such as entailment or probabilification) that the contents of beliefs stand in to one another, perhaps he would not have excluded experiences from the states that he thought could justify beliefs. The second strand zooms in on how phenomenal character takes a stand on how things are in the space around the perceiver. This putative aspect of phenomenal character, or something like it, has been discussed under various labels, including assertoric, phenomenal, or coercive force, in parallel with Frege’s idea that assertoric sentences have forces in addition to senses.27 We discuss phenomenal force in section 4, and its potential link to accuracy conditions in section 5. In section 6 we discuss the idea that the specific objects and properties that figure in the contents of experience constrain the propositions that the experiences can justify.

Phenomenal force We can fix on the phenomenal force of perceptual experiences by contrasting it with other kinds of phenomenal character. There seems to be an aspect of the phenomenal character of perceptual experience that is distinct from the phenomenal character of imagery, episodes of wondering, and pangs of desire—even when these states are all directed toward the same thing. Our perceptual experience purports to reveal how the world is, whereas visualizing the dot, wondering whether there is such a black dot in front of you, or feeling a pang of desire for a black dot does not. 26  27 

For criticism of access internalism, see Goldman (1999). Heck (2000), Martin (2002), Matthen (2005: ch. 13).

The Epistemology of Perception    791 Phenomenal force is analogous to assertoric force, in its role for the speaker. Making an assertion is a way to express how you believe things to be. Similarly, perceptual experience is a way to take in how things seem to you to be. Phenomenal force is analogous to assertoric force to the extent that both attach to belief-like states. Do perceptual experiences really enjoy a distinctive kind of phenomenal force, a kind that imagery lacks? According to a Humean line of objection to this idea, there is no deep difference in kind between visualizing and visual experience, only a difference of degree. Visual experience is not distinguished from visualizing by its phenomenal force, but instead only by the greater determinacy of its content. The epistemic role of visual experience, according to the Humean we have in mind, is due to the greater determinacy of its content. This objection fails if there are perceptual experiences with less determinate content than imagination, but which still provide better justification. For instance, a degraded visual experience of a tomato in poor lighting might still provide justification for believing that a round thing is present, whereas one might think that your imagining a tomato, no matter how vividly, does not give you any justification at all to believe this.28 A classic experiment done by Perky (1910) suggests that phenomenal force may not be pervasive among visual perceptual experiences. Subjects were asked to look at a white screen and to imagine a red tomato (for example). A faint red tomato was projected onto the screen. Most subjects ended up claiming that they were imagining a red tomato, not that they were seeing a red tomato.29 Some made comments such as the following: “I am imagining it all; it’s all imagination” or “Feels as if I was making them up in my mind” (1910: 432). On the basis of her result, one might claim that the subject of the experiment has a visual perceptual experience, although the experience lacks phenomenal force—if they did have an experience with phenomenal force, why would they only say they are imagining? By itself, this verdict does not directly challenge the idea that the phenomenal force of experiences helps explain how they provide justification, since the Perky subject may well lack justification from her experience for believing that a red tomato is in front of her. According to a different interpretation of the Perky result, the Perky subject is having a visual experience with phenomenal force, and simply is mistaken insofar as she thinks she does not. On this interpretation, even if subjects were to mistakenly deny that they were seeing a red tomato, this does not undermine claims about the phenomenal differences between visualizing and visual experience. Analogously, a subject might falsely believe she is in pain, but this does not in any way undermine the claim that there is a phenomenal ­difference between pain and non-pain. The idea that experiences have phenomenal force has been thought to help explain apparent epistemological differences between perceptual experience and other kinds

28 

For discussion, see McGinn (2004). A separate question concerns the scope of phenomenal force. Consider your experience of an object partly occluded by a fence, or your experience of a triangle vs. your experience of a Kanisza triangle. There is a difference between the way the whole object is presented to you, and the way that its visible parts between the bars of the fence are presented to you. Does your experience give you more justification to believe that the visible parts are present, than it does to believe that the whole object is present? 29  For attempts to replicate this result, see Segal (1972). For discussion, see Nigel (2010).

792    Susanna Siegel and Nicholas Silins of mental states.30 For instance, phenomenal force might be thought to answer ‘Sellars’s Dilemma’ (Sellars, 1956). On one horn of the dilemma, if experiences assertorically represent propositions, then they would have to be justified in order to justify belief in those propositions, and thus could no longer serve as stoppers of regresses regarding the justification of beliefs. Here, experiences are allegedly too similar to beliefs. On the other horn of the dilemma, if experiences do not assertively represent propositions, then it is no longer clear how they are capable of justifying beliefs at all. Here, experiences are allegedly too dissimilar from beliefs. In response, one could hold that phenomenal force of experiences make them belief-like enough to justify because they assertively represent propositions, but they are not belief-like enough to require justification themselves. In effect, this response questions the reasoning in the first horn, holding that a state can assertorically represent a proposition and provide justification for believing it, without needing justification itself.31

Accuracy conditions and the Phenomenal Approach In section 4, we discussed the idea that perceptual experiences take a stand on how things are in the space around the perceiver. This conception of experiences can be made more precise by the thesis that experiences have accuracy conditions. Beliefs have contents, and the contents of beliefs have conditions under which the belief (i.e. the state of believing) is true. According to the conception of experience assumed here, experiences can be accurate, and the contents of experience have conditions under which experiences have this status.32 In this section and section 6, we discuss potential rational roles for contents of experiences. Although we do so under the rubric of the Phenomenal Approach, our discussion could be adapted to theories that focus simply on the contents of experiences, or on non-phenomenal features of experiences such as their reliability, without assigning any rational role to phenomenology. Some such theories allow that unconscious perception provides as much justification for external world beliefs as conscious perception. How might having contents enable experiences to justify beliefs? This claim is sometimes motivated by the idea that the kind of relation that premisses of an argument stand in to a conclusion provides a model for justification in general. Pryor (2005) calls this idea the ‘Premise Principle’: Premise Principle: The only things that can justify a belief that P are other states that assertively represent propositions, and those propositions have to be ones that could be used as premises in an argument for P. They have to stand in some kind of inferential relation to P: they have to imply it or inductively support it. (2005: 189) 30  A  further question concerns whether the phenomenology of waking visual experiences is ever present when one dreams. For discussion of this issue, as well as of its significance for sceptical arguments involving considerations about dreaming, see Sosa (2005). 31  For further discussion of Sellars’s dilemma, see BonJour (1985) or Pryor (2005). 32  This assumption is compatible with a wide range of theories of the nature of experience (see Siegel, 2010, for dispute see Travis, 2013).

The Epistemology of Perception    793 The Premiss Principle faces a number of challenges. First, the principle is motivated by the idea that when one’s state S1 gives one justification to be in state S2, one can give a justifying argument in favour of the content of S2 by affirming the content of S1. But consider a case where an experience of something red justifies believing that something is red. Here one cannot give a justifying argument in favour of the claim that something is red just by affirming that something is red. The motivation for the Premiss Principle suggests that to justify my belief that something is red, my experience would instead need to have the self-representational content that I see something is red, or some other content which could be marshalled in a defence of the claim that something is red (Searle, 1983). However, a visual experience with the content that something is red is presumably a good candidate to justify believing that something is red, whether or not it has such further contents. A different pair of challenges relates to introspection. First, suppose that pains do not have contents. Even if one grants this assumption, it would seem that pains could still justify self-ascriptions of pains. Second, even if (contrary to the assumption), pains do have contents, these contents typically provide no obvious inferential support for the contents of self-ascriptions, or at least not enough support for our self-ascriptions of pain to be as justified as they are. This point applies equally to the self-ascription of bodily sensations and perceptual experiences. For instance, the proposition that a red roller skate is in front of you does not entail that you are seeing a red roller skate, and intuitively, there need be no inductive generalization linking the presence of red roller skates in your vicinity to your seeing red roller skates. These challenges could be avoided by limiting the Premiss Principle to perceptual experience (as opposed to bodily sensations, to avoid controversy about the status of pains as contentful), and to external world beliefs (as opposed to self-ascriptions). According to the limited thesis that results, perceptual experiences justify external world beliefs, only if the perceptual experiences have accuracy conditions. The limited thesis suggests that no unified account of justification by experiences is available, and that the justification of any beliefs by bodily sensations, as well as the justification of self-ascriptions of any sort of experiences, is explained by different features from those that explain the justification of external world beliefs by perceptual experiences. It is an open question whether the joints of epistemology fall in the way the doubly limited thesis suggests, with both non-perceptual experiences and self-ascriptions needing special treatment.33 Even if, contrary to the original Premiss Principle, having content is not necessary for experiences to justify belief, the specific contents an experience has may help explain which propositions they provide justification for believing. We turn to this idea next.

Contents and the Phenomenal Approach It is plausible that when experiences justify beliefs, there is a non-arbitrary relationship between the contents of the experience and the contents of beliefs they justify. For instance, by looking in the fridge, you get justification to believe that it contains mustard, 33  Goldman (2008) assumes that there should be no such hiving off, treating unified accounts as an explanatory virtue. In contrast, Moran (2001) argues for hiving off self-ascriptions of bodily sensations for special treatment.

794    Susanna Siegel and Nicholas Silins but not justification to believe that the sunset is streaked with orange. The objections to the Premiss Principle suggest that this non-arbitrary relationship cannot be shoehorned into the structure of the relation between a premiss and a conclusion in a dialectically effective argument. Given the assumption that experiences have contents and provide justification for beliefs, it is natural to think that the specific content of an experience helps explain which propositions it can justify believing. According to a simple version of this idea, experiences can justify beliefs whose contents are among the contents of experience. This idea presupposes that beliefs can have exactly the same kind of contents as experience. Different forms of this presupposition are defended by McDowell (1994), Brewer (1999), and Stalnaker (2003) (but note that Brewer (2006, 2011) and McDowell (2009) revise their earlier views). The presupposition has come under attack from philosophers who argue that it is not possible to believe exactly what you experience, because experiences form part of a system of perceptual representation that is so different from belief that the states of each system have fundamentally different contents. Often the specifically perceptual contents are called ‘nonconceptual’, here in the sense that they cannot be believed.34 However, even proponents of nonconceptual content can agree that some belief-contents are closer to some exclusively perceptual contents than others.35 In some cases, the notion of closeness might be cashed out in terms of similarity between properties. For instance, suppose an experience represents a determinate colour such as red39 and attributes it to an apple. Now compare a belief that attributes a more determinable property (such as darkish red) to the apple, with a belief that attributes a completely different colour property (such as green) or a different kind of property altogether (such as being an elephant). The content of the experience is closer to the content of the belief attributing darkish red, than it is to the content of the belief attributing the property of being an elephant. In general, one might think that an experience presenting a red square on the left provides justification for believing a proposition closely related to these contents. Feldman (2003: 75) endorses this idea, claiming that ‘when the contents of the belief are closer to the direct contents of experience, they are more apt to be properly based on experience’. The contents of experience might plausibly be thought to delimit the contents for which experiences provide immediate justification.36 According to a proposal along these lines, an experience can provide immediate justification for believing P, only if P is a content of the experience, or is suitably close to such a content. Call this the Content Constraint on immediate justification. The closer the content of experience is to a proposition P, the less the experience might seem to need to be supplemented to provide justification for P. For instance, if the contents of experiences were limited to colour, shape, and illumination properties, it might seem that it could justify believing that mustard is in the fridge, only

34 

Peacocke (1995), but see Stalnaker (2003). For useful discussion of various notions of ‘nonconceptual content’, see Speaks (2005). 35  Compare Peacocke’s notion of ‘canonical correspondence’ between nonconceptual and conceptual contents in his 2004. 36  A more ambitious project is to explain how experiences justify—rather than simply identify what they justify—in terms of what makes experiences have the contents they do. For instances of such a programme, see Burge (2003) or Peacocke (2004), and for evaluation, see Martin (2001) or Silins (2012).

The Epistemology of Perception    795 when supplemented with justification for believing that the layout of coloured shapes you see is a mustard jar. One might challenge the Content Constraint, on the grounds that in a range of common cases, experiences provide immediate justification for believing propositions that are not included among their contents, and are not even entailed by those contents. For example, perhaps perceptual experiences can immediately justify self-ascriptions of those experiences, even though they differ in contents, roughly in the way that the sentences ‘there is a red cube in front of me’ and ‘I see a red cube in front of me’ differ in their contents. Much will depend here on how the qualifications of ‘suitably close’ contents are cashed out.37 Even if the Content Constraint on immediate justification fails, there may still be a non-arbitrary relationship between the contents of experience and the propositions a subject’s experience provides justification for believing. And if the range of propositions an experience justifies depends on which contents it has, then it becomes important to settle which contents can be contents of experience. Are the contents of perceptual experience limited to ‘low-level’ properties such as colour, shape, texture, illumination, motion, or can they represent more complex properties such as being a lemon, being familiar, or being a cause of an event? One might try to gain traction on this question via epistemological considerations, rather than trying to gain traction on the epistemological questions via considerations about which contents experiences have. For instance, the Content Constraint could be reversed, as follows: Reverse Content Constraint: If an experience E immediately justifies believing P, then P is a content of E.

If there are independent reasons to think that experiences can immediately justify believing contents as complex as ‘Fiona is carrying a dog’ or ‘My kite is teetering on the edge of a cliff’, then according to the Reverse Content Constraint, these contents are contents of experience.

Attention and the Phenomenal Approach Earlier we floated the intuition that a blindsighted subject who unconsciously registers the presence of a red ball would have less justification for believing that a red ball is present, compared with a sighted subject who saw the red ball (and had an experience in which it looked red and spherical). The blindsight intuition does not tell us whether attending to the red ball is necessary for the subject to have justification from her experience. For all the blindsight scenario has specified so far, the sighted subject may be attending to the ball. Likewise, standard cases of perceptual justification are also cases in which the subject is attending to what she sees—as when you look in the fridge and find the mustard you were searching for, or when Moore, while giving his Proof of the external world attends to his hands (‘I hold up a hand and gesture . . .’) (Moore 1939). If the Phenomenal Approach is 37 

See Silins (2011) for further discussion.

796    Susanna Siegel and Nicholas Silins correct, is it only attentive experiences that provide justification, or do inattentive experiences provide justification as well? This question will not arise if, necessarily, all experiences are attentive, since consciousness requires attention. But if there are inattentive experiences—phenomenal states in which one has an experience of representing a red ball but does not attend to it—then the Phenomenal Approach faces a basic question about its scope: is it phenomenal character per se that provides justification, or is it phenomenal character of the sort one has when attending to what one sees?38 We can distinguish between two answers to this question. According to the Attention Needed view, only attentive experiences provide propositional justification. According to the Attention Optional view, inattentive experiences can provide propositional justification. We can illustrate their different predictions by considering some classic experiments about inattentional blindness. In a typical experiment (Most et al, 2001, 2005) subjects are asked to perform the attentionally demanding task of counting how many white boxes bounce off the side of a display. Many of the subjects do not report seeing a red cross that passes slowly (over five seconds) through the fixation point. Assuming that these subjects experience the red cross (as a red cross) without attending it, their inattentive experience does not influence what they report. According to the Attention Needed view, the inattentive experience of the red cross cannot provide propositional justification for a red-cross belief. According to the Attention Optional view, it can. Prima facie, both positions seem defensible. It seems plain that attention is not epistemically idle. Typically, if you look closely in the fridge, you’ll be in a better epistemic position with respect to whether there’s mustard inside, compared to your epistemic position if you look quickly or carelessly. Perhaps at the limit, if attention runs out but consciousness persists, there’s no justification provided by the experience. If so, this conclusion would favour the Attention Needed view. On the other hand, evidence can survive unnoticed, and this may seem to favour the Attention Optional view. Attentive experiences tend to correlate with experiences that are accessible to the rest of the cognitive system (for instance, by feeding into belief and action). Suppose you know that you have an appointment with x alone at noon, that you have an appointment with y alone at noon, and that x ≠ y. You could have propositional justification to believe that you have conflicting appointments, even if you haven’t noticed the conflict. Once you notice it, you’ll have based your belief on the previously unnoticed evidence. So long as the evidence is unnoticed, it is in that sense inaccessible. If beliefs can provide unnoticed evidence, then there seems no obvious reason to deny that experiences could do the same. If they can, this conclusion would favour the Attention Optional view. Which position, Attention Needed or Attention Optional, is true? This question matters for debates about ‘internalism’ and ‘externalism’ in epistemology. According to internalism about justification, whether a subject is justified in believing a proposition depends exclusively on factors internal to the subject’s mind. It is natural to think that, if one privileges the role of consciousness in epistemology, one will be as ‘internalist’ as one could be. But which factors internal to the mind matter? Does everything given in consciousness deserve the privilege, or only what the subject is given in attention, and thereby made

38 

For further discussion of the role of inattentive experiences in epistemology, see Silins and Siegel (2014).

The Epistemology of Perception    797 cognitively accessible? If consciousness outside attention is rationally idle, as it is on the Attention Needed view, that suggests that perceptual justification depends on the kind of cognitive accessibility bestowed by attention. The Attention Needed view could be developed as an accessibilist form of internalism that filters out inattentive experiences from the grounds for justification, leaving in only attentive experiences.

States of seeing Which features of experiences explain how they provide justification? In discussing this question, we’ve divided constitutive features of experience from causal features. On which side of this distinction does the status of an experience as a state of seeing belong? The answer depends on whether the relationship between visual experiences and the things you see is causal or constitutive. For simplicity, we assume that if this relation is causal, it is also non-constitutive and contingent. If the relation is causal and non-constitutive, then you could have the same experience (a state with the same phenomenal character), even if you were hallucinating. Whether an experience is a state of seeing as opposed to hallucination depends on its aetiology. Hallucinations have endogenous causes, whereas states of seeing are caused in part by the things seen (scenes, objects, events, etc.). In contrast, if the relation between visual experiences and states of seeing is constitutive, then which experiences you have (or equivalently, which phenomenal state you are in) depends on whether you are seeing or not. Whether you are seeing or hallucinating is thus not merely a matter of aetiology.39 On this view, the thesis that states of seeing help explain how experiences provide justification falls under the Phenomenal Approach. 40 There is a range of states of seeing which might be privileged in epistemology. So far we have ignored the differences between seeing objects, facts, events, and other entities. Let us focus on states of seeing ordinary objects such as basketballs, and ignore the differences between seeing the basketball bouncing, which is arguably a relation to an event or a state of affairs, and seeing that the basketball is bouncing, which is arguably a relation to a proposition. Either way, to see that the basketball is bouncing, or to see the basketball bouncing, it needs to be the case that the basketball is bouncing. Moreover, such factive states of seeing are absent from cases of illusion (when the things you see look to have properties they in fact lack) and hallucination.41 Non-factive states of seeing can be present in cases of illusion, when you do see the basketball, but misperceive its colour or motion. Non-factive states of seeing are still not present in cases of hallucination. Let us begin with both factive and non-factive states of seeing, and focus on how much justification such states provide for beliefs, leaving open the corresponding issues about

39  Individual experiential episodes may be constitutively linked to states of seeing. But our discussion should be understood to be at the level of types. 40  See Campbell (2002: ch. 6). 41  Sometimes it is also assumed to entail that the subject knows that P (Dretske, 1969; Williamson, 2000), and other times merely that the subject believes that P. We will not assume that either of these additional entailments holds.

798    Susanna Siegel and Nicholas Silins knowledge, or still other epistemic states. Does the status of an experience as a state of seeing of either sort contribute to its justificatory force? A first potential asymmetry concerns the propositions for which states of seeing provide justification. Are there propositions that are justified by states of seeing, but not by hallucinations? Candidates include the propositions that figure in de re beliefs about the things you see. De re beliefs about objects are beliefs whose truth or falsity, relative to any world w, depend on how things are with that object in world w.42 The perceptual beliefs we form about the objects we see are typically de re beliefs, such as when we form the belief that would be natural to express by saying that is an orange ball.43 Since in (pure) hallucinations, there is nothing that you see, the hallucination does not put you in a position to form such de re beliefs, let alone provide justification for them.44 A second potential asymmetry concerns the degree of justification provided by factive states of seeing, as opposed to all other experiences (both hallucinations and nonfactive states of seeing). Normally, if you see a pig in a pigpen, and it is the way it looks, your factive state of seeing settles the question for you of whether there is a pig in the pigpen. And aside from justifying the de re belief (that is a pig in a pigpen), your state of seeing the pig also justifies a belief with existentially quantified contents (there is a pig in the pigpen). Unlike the de re belief with singular contents, the belief with general content can be formed on the basis of experience in cases of hallucination. With respect to propositions that are available to be believed in cases of factive seeing, non-factive seeing, and hallucination, do factive states of seeing provide more justification than hallucinations? If so, then these states of seeing have a privileged status as providers of justification, compared with hallucinations. Both McDowell (1996, 2008) and Johnston (2006) could be seen as defending asymmetries in the degree of justification provided by factive states of seeing, as opposed to hallucinations and non-factive states of seeing. The idea that factive states of seeing, or a limited subclass of them, enjoy a privileged epistemic status is motivated by the idea that such states are constitutively connected to the facts that make true the very beliefs that those perceptual states justify. The factive state of seeing is constitutively infallible with respect to the proposition that Wilbur (the pig you see) is a pig in the pigpen. You won’t count as seeing that that is a pig in a pigpen, unless that is a pig in a pigpen. Constitutive infallibility involves modal features that may attach equally to states with necessarily true contents, yet which intuitively do not provide justification for believing those contents. For instance, if you guess that P (for some necessarily true proposition P), then you cannot make that guess without P being true.45 An analogous point holds 42 

The need to specify truth relative to a world arises from the fact that experiences provide justification for beliefs that can be expressed using sentences containing demonstratives such as ‘that is a red ball’, where the demonstratives are rigid designators. For discussion, see Kaplan (1989). 43  Some philosophers invoke this role to argue either that phenomenal states themselves have singular contents, or are sometimes partly constituted by relations of seeing and the objects seen. On this motivation for taking relations of seeing to be partly constitutive of experiences, see Campbell (2002). 44  We set aside putative hallucinations with de re contents, such as hallucinations of your father. Arguably even these do not put the subject in a position to initiate the kind of connection to an object that makes a mental state de re with respect to that object, though once such a connection is established, a de re hallucination may put one in a position to form new de re mental states about it. For discussion, see Johnston (2004). 45 An important difference between factive seeing and factive guessing is that the constitutive infallibility stems from the content of the guess, but from the state in the case of factive seeing.

The Epistemology of Perception    799 for forgetting or overlooking necessary truths. But a proponent of the idea that factive states of seeing provide more justification than non-factive experiential states (including non-factive states of seeing) might invoke additional features besides constitutive infallibility to ground the epistemic privilege of factive states of seeing, such as the phenomenal force found in states of seeing, and not found in guesses or cases of forgetting necessary truths. An opponent of the idea that factive states of seeing enjoy privileged epistemic status with respect to justification might focus on the rational responses to seamless transitions between such states and hallucinations.46 (This point of focus is also found in the ‘new evil demon’ problem to be discussed in section 9.) If factive states of seeing provide some additional boost of justification, then it will be irrational to maintain confidence at the same level through seamless transition from seeing to hallucination, when one starts out with the highest degree of confidence justified by the state of seeing. Contraposing, if it would be rational to maintain the same confidence level across such transitions, then this undermines the idea that factive states of seeing enjoy privileged epistemic status with respect to justification. We have examined the pros and cons of the idea that factive states of seeing are epistemically privileged with respect to justification, regardless of whether such states are identified with phenomenal states (experiences in our official sense). The epistemic advantages of factive states of seeing could arguably be enjoyed whether or not any phenomenal state is identical with a factive state of seeing, as would be maintained by a metaphysical disjunctivist. The epistemological upshots of metaphysical disjunctivism are highly controversial.47

4 Part iii  Causal features How might the aetiology of an experience affect its ability to provide justification? A first idea, explored in section 8, is that an experience might be caused by an object or scene that you see, and its status as a state of seeing helps explain some of its justificatory powers. A second idea, examined in section 9, is that experiences confer justification, when they do, in virtue of being parts of processes that reliably give rise to true beliefs. A third idea, examined in section 10, is that causal influences on experiences from subject’s prior mental states can affect the epistemic status of the experience.

Reliability We begin with a simple version of reliabilism, according to which your experience of type E gives you justification to believe that P, just in case E is reliably correlated with its being the case that P.  This simple version of reliabilism draws a straightforward connection

46  47 

For some discussion of seamless transition cases, see Johnston (2004). See Logue, Chapter 11, this volume.

800    Susanna Siegel and Nicholas Silins between justification and truth. Since this idea is so powerful and reliabilism in its many forms is so influential, we won’t say much else by way of motivation. Reliabilism is compatible with Immediacy, as we mentioned in section 2. In general, Immediacy is compatible with the idea that as phenomenal types, experiences do not suffice to provide immediate justification, because further etiological constraints must be met. For instance, Goldman (2008) argues that experiences can and do provide immediate justification, but only if they are part of a process that generates reliably true beliefs. Reliabilism is versatile enough to be compatible with the Phenomenal Approach (which denies that the blindsighter’s perceptions provide justification), as well as the opposing position. Since the perceptual states of the blindsighter are as reliable as the experiences of the sighted subject, simple reliabilists will say that the blindsighted subject has just as much justification from his perceptual states as the sighted subject gains from his ­experience. The phenomenology enjoyed by the sighted subject will not contribute to justification. But in principle, a more refined version of reliabilism could be combined with the Phenomenal Approach, resulting in the position that experience provides justification for believing certain contents, in virtue of both its phenomenal character and the type of process to which it belongs, because the appropriate process has to be individuated in part by a phenomenal state. Simple reliabilism has been attacked from a number of directions. First, as a sufficient condition for perceptual justification, reliable correlation is often held to be insufficiently demanding, on the grounds that an agent might be endowed with a reliable faculty of clairvoyance, while still failing to gain justification from it, if the subject has no inkling that she has such a faculty, or if she has what are intuitively good reasons to think her perception is unreliable (BonJour, 1980). A related objection is exactly analogous to the bootstrapping objection discussed in section 2. According to the bootstrapping objection, if reliabilism is true, then one’s experiences can end up themselves giving one a justified belief that they are reliable. In response, the same options listed in section 2 are available here as well. Second, as a necessary condition for perceptual justification, reliable correlation is often held to be too demanding. Suppose an evil demon makes someone’s experiences misleading most of the time. When it visually seems to the person that P, it tends to not be the case that P. Suppose further that these misleading experiences could not easily have been accurate, so that they are robustly unreliable. According to the classic objection, contrary to what reliabilism about perceptual justification predicts, the victim’s experiences still give her justification for ordinary beliefs (Lehrer and Cohen, 1983). Simple reliabilism might be refined as a causal theory, as in process reliabilism, according to which an experience justifies a proposition P if and only if it results from a process that reliably produces true beliefs that P.48 When one forms a perceptual belief on the basis of a given process, that process falls under many process types, and these types of process may differ in how reliable they are. A reliabilist theory will make different predictions about which beliefs are justified, depending on which process type figures in the theory. A third objection to reliabilism, known as the Generality Problem, specifically targets process reliabilism. A  given process that generates a belief presumably can fall under many types, such as being a process generated by veridical perception, being a process which occurs on Tuesday, and so on. 48  Goldman (1976, 2008). For further refinements of reliabilist approaches, see Sosa (1991, 2007, 2009), Plantinga (1993, 1996), or Comesaña (2002, 2010).

The Epistemology of Perception    801 The challenge is to specify which process type is relevant (Conee and Feldman, 1998).49 A related challenge is to specify the relevant type in such a way that a reliability requirement is not too easily satisfied by every true perceptual belief. For instance, being formed on the basis of a veridical hallucination is a reliable process, but presumably not all beliefs formed on the basis of veridical hallucinations are justified. Responses to the Generality Problem that specify the relevant type of process may address the earlier objections as well. In contrast to the controversy over reliabilism about justification, the reliability of processes by which beliefs are formed has been less controversial as a necessary condition for perceptual knowledge. Reliability has been advanced as a condition that rules out the sort of ‘epistemic luck’ present in classic Gettier cases, in which one has a justified true belief without yet having knowledge. For a potential example of such a case, suppose that, at noon, you happen to check the time on a clock with a dial frozen at noon (Russell (1948) has a similar case). Several different anti-luck conditions have been proposed to explain why knowledge is absent from such cases, appealing to different kinds of co-variation between one’s beliefs and the facts one putatively knows. First, according to ‘sensitivity’ requirements for knowledge (to a first approximation), one’s knowing that P requires that if it weren’t the case that P, one would not believe that P (Nozick, 1981). For instance, suppose that someone is looking at a wolf which looks like a dog, where the person forms a perceptual belief that there is a dog in front of her. Suppose further that there is indeed a dog in front of her—namely a chihauhau that the wolf happens to have just gulped down. If the person doesn’t know of the wolf’s recent dog consumption, she presumably does not know that there is a dog in front of her. According to the proponent of a sensitivity requirement for knowledge, the best diagnosis for the person’s failure to know is that she would still have believed that there is a dog in front of her, even if there hadn’t been a dog in front of her. Providing an adequate formulation of a sensitivity requirement for perceptual knowledge is challenging. Suppose that someone is looking at a chihauhau in good conditions, and let us stipulate that if there hadn’t been a dog in front of her, there would have been a dog-resembling wolf in front of her instead. Here she still seems to be in a perfectly good position to know that there is a dog in front of her, given that she is looking at a chihauhau in good conditions. However, if there hadn’t been a dog in front of her, she would still have believed that there’s a dog in front of her, due to the presence of the dog-looking wolf (see Williamson 2000 for an overview of such cases). Or consider the everyday sort of change blindness discussed by Dretske (2004). If your friend shaved off his moustache, you would fail to notice, and so would retain your belief that he has a moustache. Still, such facts do not seem to damage your ability to know that he has a moustache when he is right in front of you.50 To avoid the counterexamples, one might focus on the specific method used by the person to form her belief. But this raises the question, reminiscent of the Generality Problem for reliabilism about justification, about how to individuate the method. For example, one question is whether the perceptual states essential to the method are states of seeing, or phenomenal states that a subject can be in, regardless of whether they are seeing.51 49 

For responses to the generality problem, see Beebe (2004) or Comesaña (2006). Discussion with Max Kwon was helpful here. 51  For discussion of the challenges to specifying a sensitivity account, see DeRose (1995, 2010) or Williamson (2000). 50 

802    Susanna Siegel and Nicholas Silins Setting aside exactly how to articulate a sensitivity requirement for perceptual knowledge, many reject the sensitivity approach on the grounds that it is too demanding. Consider any belief you have to the effect that you do not falsely believe that P. If you were to falsely believe that P, you presumably would still believe that you didn’t falsely believe that P. Knowledge that you don’t falsely believe that P thus seems out of reach on the sensitivity approach, and a restriction of the approach just to perceptual knowledge of the environment is presumably ad hoc (Vogel, 2000). Indeed, if scepticism is to be avoided by the proponent of the sensitivity approach, they will have to allow that you might have perceptual knowledge that P, as well as logical knowledge that P only if you don’t falsely believe that P, yet still be unable to know that you don’t falsely believe that P. On pain of accepting scepticism, the sensitivity approach would seem to have to violate a ‘closure’ principle to the effect that knowledge is ‘closed’ under known entailment.52 According to reliabilist critics of sensitivity conditions on knowledge, we should prefer a different reliability requirement for perceptual knowledge called ‘safety’. Here the key idea is that if one has perceptual knowledge that P, then one could not easily have been mistaken about whether P (Sosa, 1999; Williamson, 2000). Proponents of the safety requirement say that it avoids the over-demanding character of the sensitivity requirement for perceptual knowledge, while providing a good diagnosis of the absence of knowledge in classic Gettier cases. If the dog-looking wolf could easily have failed to come by the chihauhau to eat, one could easily have been mistaken in believing that there is a dog in front of one. Whether the safety requirement indeed avoids the challenges facing the sensitivity requirement is unclear. First, consider the quantum mechanical hypothesis that the matter in my car disperses so as to leave behind a mere car façade (Hawthorne, 2004). As improbable as the hypothesis is, it still arguably could ‘easily’ have been true, in the sense that it is true in some worlds only slightly different from the actual world. In such worlds, however, I make a mistake about whether my car is outside, and thus fail to have perceptual knowledge in the actual world, at least on one understanding of the safety proposal.53 Second, some philosophers have argued that a safety requirement for knowledge cannot be combined with acceptance of a suitable closure principle for knowledge, without accepting scepticism.54 Debates surrounding reliabilism raise the question whether aetiological facts about experiences can affect whether they provide justification, even when the subject is not aware of those aetiological features. These debates focus on ‘aetiology from without’— causal chains that originate outside the subject’s mind and terminate in experience or belief. We now consider a range of ways in which the aetiology of experiences from within the subject’s mind might affect the epistemic status of experiences—even when the subject is unaware of it.

52  Nozick

(1981) and Dretske (2005) embrace the conclusion and reject closure. Vogel (1990b) and Hawthorne (2005) defend the closure principle. Roush (2006) argues that the sensitivity approach can avoid rejecting closure. 53  For further discussion, see Pritchard (2005) and Greco (2007). 54  The worry traces to Kripke’s lectures on Nozick’s theory of knowledge. For discussion, see e.g. Cohen (2008). For further criticisms of safety requirements for knowledge, see Brueckner and Fiocco (2002) or Neta and Rohrbaugh (2004).

The Epistemology of Perception    803

Cognitive penetration There is no doubt that a subject’s background beliefs affect how she responds to what she sees, and how it is reasonable for her to respond. Consider an experienced birdwatcher who identifies a bird she sees as a flycatcher, on the basis of her expert background beliefs about how flycatchers look. In contrast, if someone unaccustomed to observing birds saw the same bird from exactly the same position, she would normally not form any belief about what specific kind of bird it is, because she cannot discriminate kinds of birds from one another. If she did form the belief that it’s a flycatcher, that would be a lucky guess, unjustified by any expertise or reasoning (Feldman, 2003). It is one thing for expert and novice to form different beliefs on the basis of what they both see. It is another for the expert and novice to have phenomenally different experiences as the result of their difference in what else they know or believe. In principle, a phenomenal difference could result from attention, as when the expert pays attention to different features of the same bird from the novice, or perhaps it could arise even when expert and novice attend to exactly the same features. In either case, the contents of the experience could differ, along with its phenomenal character, as the result of the differences in expertise. In this example, it is expertise that influences the experiences, but we could imagine examples in which the influence comes from moods, desires, suspicions, fears, or other mental states. We can call influences on the phenomenal character of experience by any of these kinds of states ‘cognitive penetration’.55 Some forms of cognitive penetration, such as those that may be found in expertise, seem to improve a subject’s epistemological situation. For instance, suppose that expertise in radiology changes what one sees when looking at an x-ray by perceptual reorganization, creating new perceptual cues that are unavailable to the non-expert. This kind of cognitive penetration would help the radiologist spot the tumours when looking for them on x-rays. Other forms of cognitive penetration seem to put pressure on a traditional conception of the rational roles played by experiences. In science, experiments play a central role in confirming scientific theories, because they allow for controlled observation through which experimenters test hypotheses against one another. In everyday contexts, we treat perception as a means of finding out mundane facts, such as whether there is mustard in the fridge, or whether the dog is inside. At the level of abstraction found in philosophy, experience and reason are traditionally taken to be the two ultimate sources of rational support for beliefs. But if what you see is determined by what you already fear, suspect, or believe to be the case, then these penetrating psychological states seem to stack the tribunal of experience in their favour, preventing us from using experience to rationally assess our beliefs, fears, or suspicions. How widespread is cognitive penetration? The idea that perception and scientific observation can be free of such influences underlies the idea that perceptual systems are modular, taking in information without systematic influence from other parts of the cognitive 55  This use of ‘cognitive penetration’ is broader than the kind targeted by Pylyshyn in his (1998). Pylyshyn argues that early vision is exclusively the output of a module and as such is not the product of other cognitive states (though its outputs maybe influenced by perceptual learning. See Goldstone and Byrge, Chapter 42, this volume). In contrast, we are concerned with whether visual experience can be influenced by prior mental states. It can be, even if experience is partly the output of a module. For discussion of cognitive penetration, see Siegel (2013).

804    Susanna Siegel and Nicholas Silins system.56 A host of experimental results suggest that non-perceptual states of all sorts can influence perception, and on the face of it, many of these seem to threaten modularism about perception.57 All of these results are controversial, and further interpretation and experimentation is needed to discover the exact nature of the impact on perception that prior mental states have. From the perspective of some internalist theories of justification, such as those which say that an experience with the content that P is sufficient to give one prima-facie justification to believe that P, cognitive penetration can have no direct impact on justification. According to these theories, cognitive penetration may lead to illusory (falsidical) experiences, but the rational role of these experiences is not compromised any more than it is in standard visual illusions (such as seeing the Müller-Lyer lines), or in scenarios where experiences are systematically in error (think of the Matrix-like brain-in-vat scenarios, where brains of subjects are systematically manipulated to produce illusory experiences). Some reliabilist theories might also hold that cognitive penetration has little epistemological significance, if what the theory privileges is the reliability of perception at a sufficiently general level. Other versions of internalism, however, can grant that cognitive penetration can compromise the status of experiences as providers of justification, without allowing that experiences in standard visual illusions or brain-in-a-vat scenarios are compromised. For instance, suppose someone’s unjustified suspicion that there is a gun in her fridge influences the contents of her experience, so that when she opens the fridge to look inside, she has an experience as of a gun. It is open to internalists to hold that this case of ‘fearful seeing’ is not any less irrational than a structurally similar case of fearful belief, where fear influences belief directly (Siegel, 2013). The idea that experiences can have irrational aetiologies is at odds with the traditional idea that experience, like reason itself, is an ultimate arbiter of belief. On the traditional picture, the epistemic goodness or badness of belief derives from the way it is grounded in reason and experience, but reason and experience themselves never have any further grounds. As some foundationalists would put it, experiences justify beliefs without themselves being justified, or otherwise susceptible to rational evaluation. On the revised picture, experiences of wishful seeing, like wishful thinking, are conduits of irrationality, and in that sense those experiences are susceptible to rational evaluation.

5 Conclusion Traditionally, discussions of perceptual justification have focused on whether the transition from perception to belief can be rational. In this chapter, we began from the 56  Fodor (1983) holds that modular processes form only one part of perception, leaving it as an open question whether he thinks conscious perceptual experience is exclusively or even mainly the output of modules. Some of Fodor’s examples of outputs of modules are conscious experiences, such as the experience of seeing the Müller-Lyer lines as different in length even when one knows they have the same length. But other experiences may arise so close to the end of the process of belief fixation as he construes it that they are partly the output of central processing. 57  Some of the results claimed to challenge modularism include Levin and Banaji (2006), the papers collected in Bar (2011), and many of the papers cited in Proffitt and Linkenauger (2013).

The Epistemology of Perception    805 substantive assumption that the transition is sometimes rational, and explored a range of potential features of experiences that make experiences suited to stand in such rational relations, when they do. This approach brings into focus the complex interrelationship between the philosophy of perception and epistemology.58

References Audi, R. (1986). ‘Belief, reason, and inference’. Philosophical Topics, 14(1), 27–65. Audi, R. (1993). The Structure of Justification. Cambridge: Cambridge University Press. Ayer, A. J. (1973) The Central Questions of Philosophy. London: Weidenfeld. Bar, M. (ed.) (2011). Prediction and the Brain. Oxford: Oxford University Press. Beebe, J. (2004). ‘The generality problem, statistical relevance and the tri-level hypothesis’. Noûs, 38(1), 177–95. Bergmann, M. (2006). Justification without Awareness: A defense of epistemic externalism. Oxford: Oxford University Press. Block, N. (1990). ‘Inverted earth’. Philosophical Perspectives, 4, 53–79. Block, N. (2008). ‘Consciousness and cognitive access’. Proceedings of the Aristotelian Society, 108(1/3), 289–317. BonJour, L. (1980). ‘Externalist theories of empirical knowledge’. Midwest Studies in Philosophy, 5, 53–73. BonJour, L. (1985). The Structure of Empirical Knowledge. Cambridge, MA: Harvard University Press. BonJour, L. (2001). ‘Back to foundationalism’. In M. DePaul (ed.), Resurrecting Old-Fashioned Foundationalism. Lanham, MD: Rowman and Littlefield. Brewer, B. (1999). Perception and Reason. Oxford: Oxford University Press. Brewer, B. (2006). ‘Perception and content’. European Journal of Philosophy, 14(2), 165–181. Brewer, B. (2011). Perception and its Objects. Oxford: Oxford University Press. Brueckner, A. (1992). ‘Semantic answers to skepticism’. Pacific Philosophical Quarterly, 73(3), 200–19. Brueckner, A. and Oreste Fiocco, M. (2002). ‘Williamson’s anti-Luminosity argument’. Philosophical Studies, 110(3), 285–93. Burge, T. (1982). ‘Other bodies’. In Andrew Woodfield (ed.), Thought and Object. Oxford: Oxford University Press. Burge, T. (2003). ‘Perceptual entitlement’. Philosophy and Phenomenological Research, 67(3), 503–48. Burge, T. (2010). Origins of Objectivity. New York: Oxford University Press. Campbell, J. (2002). Reference and Consciousness. New York: Oxford University Press. Chalmers, D. (2004). ‘The representational character of experience’. In B. Leiter (ed.), The Future for Philosophy. Oxford: Oxford University Press. Chisholm, R. (1966). Theory of Knowledge. Englewood Cliffs, NJ: Prentice-Hall. Cohen, S. (1984). ‘Justification and truth’. Philosophical Studies, 46(3), 279–95.

58 

For helpful discussion and criticism, thanks to Selim Berker, Alex Byrne, Dan Greco, Sophie Horowitz, Mohan Matthen, James Pryor, Declan Smithies, Zeynep Soysal, Scott Sturgeon, Jonathan Vogel, and members of the MIT Epistemology Reading Group.

806    Susanna Siegel and Nicholas Silins Cohen, S. (2002). ‘Basic knowledge and the problem of easy knowledge’. Philosophy and Phenomenological Research, 65(2), 309–29. Cohen, S. (2008). ‘Structure and connection: Comments on Sosa’s epistemology’. In J. Greco (ed.), Sosa and his Critics. Oxford: Blackwell. Cohen, S. (2010). ‘Bootstrapping, defeasible reasoning, and a priori justification’. Philosophical Perspectives, 24(1), 141–59. Cohen, S. and Lehrer, K. (1983). ‘Justification, truth, and coherence’. Synthese, 55(2), 191–207. Comesaña, J. (2002). The diagonal and the demon. Philosophical Studies, 110(3), 249–66. Comesaña, J. (2006). A well-founded solution to the generality problem. Philosophical Studies, 129(1), 27–47. Comesaña, J. (2010). ‘Evidentialist reliabilism’. Noûs, 44(4), 571–600. Conee, E. and Feldman, R. (1998). ‘The generality problem for reliabilism’. Philosophical Studies, 89(1), 1–29. Conee, E. and Feldman, R. (2001). ‘Internalism defended’. Reprinted in E. Conee and R. Feldman (eds), Evidentialism. Oxford: Oxford University Press, 2004. David, M. and Warfield, T. A. (2008). ‘Knowledge-closure and skepticism’. In Q. Smith (ed.), Epistemology: New Essays. Oxford: Oxford University Press. Davidson, D. (1986). ‘A coherence theory of truth and knowledge’. In E. LePore (ed.), Truth and Interpretation: Perspectives on the philosophy of Donald Davidson (pp. 307–319). Oxford: Basil Blackwell. DeRose, K. (1995). ‘Solving the skeptical problem’. Philosophical Review, 104(1), 1–52. DeRose, K. (2010). ‘Insensitivity is back, baby!’ Philosophical Perspectives, 24(1), 161–87. Dickie, I. (2011). ‘Visual attention fixes demonstrative reference by eliminating referential luck’. In C. Mole, D. Smithies, and W. Wu (eds), Attention: Philosophical and psychological essays. Oxford: Oxford University Press. Dretske, F. (1969). Seeing and Knowing. Chicago: University of Chicago Press. Dretske, F. (1970). ‘Epistemic operators’. Journal of Philosophy, 67, 1007–23. Dretske, F. (1971). ‘Conclusive reasons’. Australasian Journal of Philosophy, 49, 1–22. Dretske, F. (1997). Naturalizing the Mind. Cambridge, MA: MIT Press. Dretske, F. (2004). ‘Change blindness’. Philosophical Studies, 120(1/3), 1–18. Dretske, F. (2005). ‘Is knowledge closed under known entailment?’ In M. Steup (ed.), Contemporay Debates in Epistemology. Oxford: Blackwell. Dretske, F. (2006). ‘Information and closure’. Erkenntnis, 64(3), 409–13. Feldman, R. (2003). Epistemology. Foundations of Philosophy series. Upper Saddle River, NJ: Prentice Hall. Feldman, R. (2004). Foundational justification. In J. Greco (ed.), Ernest Sosa and his Critics. Oxford: Blackwell. Fodor, J. (1983). The Modularity of Mind. Cambridge, MA: MIT Press. Fumerton, R. (1995). Metaepistemology and Skepticism. Lanham, MD: Rowman & Littlefield. Fumerton, R. (2005). Speckled hens and objects of acquaintance. Philosophical Perspectives, 19, 121–38. Fumerton, R. (2006). Epistemic internalism, philosophical assurance, and the skeptical predicament. In T. M. Crisp, M. Davidson, and D. Vander Laan (eds), Knowledge and Reality: Essays in honor of Alvin Plantinga. Dordrecht: Kluwer. Fumerton, R. (2009). ‘Markie, Speckles, and classical foundationalism’. Philosophy and Phenomenological Research, 79(1), 207–12.

The Epistemology of Perception    807 Goldman, A. (1976). ‘Discrimination and perceptual knowledge’. Journal of Philosophy, 73, 771–91. Goldman, A. (1986). Epistemology and Cognition. Cambridge, MA: Harvard University Press. Goldman, A. (1999). ‘Internalism exposed’. Journal of Philosophy, 96(6), 271–93. Goldman, A. (2008). ‘Reliability and immediate justification’. In Q. Smith (ed.), Epistemology: New Essays. Oxford: Oxford University Press. Greco, J. (2005). ‘Justification is not internal’. In M. Steup and E. Sosa (eds), Contemporary Debates in Epistemology. Oxford: Blackwell. Greco, J. (2007). ‘Worries about Pritchard’s safety’. Synthese, 158, 299–302. Gupta, A. (2006). Empiricism and Experience. Oxford: Oxford University Press. Hawthorne, J. (2004). Knowledge and Lotteries. Oxford: Oxford University Press. Hawthorne, J. (2005). ‘The case for closure’. In M. Steup and E. Sosa (eds), Contemporary Debates in Epistemology (pp. 26–42). Malden, MA: Blackwell. Heck, R. (2000). ‘Nonconceptual content and the "space of reasons" ‘. Philosophical Review, 109(4), 483–523. Huemer, M. (2001). Skepticism and the Veil of Perception. Lanham, MD:  Rowman and Littlefield. Johnston, M. (2004). The obscure object of hallucination. Philosophical Studies, 120(1), 113–83. Johnston, M. (2006). ‘Better than mere knowledge? The function of sensory awareness’. In T. S. Gendler and J. Hawthorne (eds), Perceptual Experience. Oxford: Oxford University Press. Kaplan, D. (1989). ‘Demonstratives’. In J. Almog et al (eds), Themes from Kaplan. Oxford: Oxford University Press. Korcz, K. (1997). ‘Recent work on the basing relation’. American Philosophical Quarterly, 34(2), 171–92. Kuhn, T. S. (1962). The Structure of Scientific Revolutions. Chicago: University of Chicago Press, reprinted 1996. Lawlor, K. (2005). ‘Living without closure’. Grazer Philosophische Studien, 69(1), 25–50. Goldman, Alvin (1976). ‘Discrimination and perceptual knowledge’. Journal of Philosophy, 73 (November), 771–91. Hawthorne, J. and Lasonen-Aarnio, M. (2009). ‘Knowledge and objective chance’. In P. Greenough and D. Pritchard (eds), Williamson on Knowledge. Oxford: Oxford University Press. Lehrer, K. (1971). ‘How reasons give us knowledge, or the case of the gypsy lawyer’. Journal of Philosophy, 68(10), 311–13. Levin, R. and Banaji, M. (2006). ‘Distortions in the perceived lightness of faces: The role of race categories’. Journal of Experimental Psychology: General, 135, 501–12. Lycan, W. (2001). ‘The case for phenomenal externalism’. Philosophical Perspectives, 15(s15), 17–35. Lyons, J. (2009). Perception and Basic Beliefs. New York: Oxford University Press. Matthen, M. (2005). Seeing, Doing, and Knowing. New York: Oxford University Press. McDowell, J. (1982). ‘Criteria, defeasibility, and knowledge’. Proceedings of the British Academy, 68, 455–79. Also in J. Dancy (ed.), Perceptual Knowledge. Oxford: Oxford University Press, 1988. McDowell, J. (1994). Mind and World. Cambridge, MA: Harvard University Press.

808    Susanna Siegel and Nicholas Silins McDowell, J. (1995). ‘Knowledge and the internal’. Philosophy and Phenomenological Research, 55(4), 877–93. McDowell, J. (2008). ‘The disjunctive conception of experience as material for a transcendental argument’. In F. Macpherson and A. Haddock (eds), Disjunctivism: Perception, Action, Knowledge. Oxford: Oxford University Press. McDowell, J. (2009). Avoiding the Myth of the Given, in Having the World in View: Essays on Kant, Hegel, and Sellars. Harvard University Press. McGinn, C. (2004). Consciousness and its Objects. Oxford: Oxford University Press. Manley, D. (2007). ‘Safety, content, apriority, self-knowledge’. Journal of Philosophy, 104(8), 403–23. Martin, M. G. F. (2001). ‘Epistemic openness and perceptual defeasibility’. Philosophy and Phenomenological Research, 63(2), 441–8. Martin, M. G. F. (2002). ‘The transparency of experience’. Mind and Language, 4(4), 376–425. Moore, G. E. (1939). ‘Proof of an external world’. Proceedings of the British Academy, 25, 273–300. Moran, R. (2001). Authority and Estrangement:  An Essay on Self-Knowledge. Princeton: Princeton University Press. Most, S. B., Simons, D. J., Scholl, B. J., Jimenez, R., Clifford, E., and Chabris, C. F. (2001). ‘How not to be seen: The contribution of similarity and selective ignoring to sustained inattentional blindness’. Psychological Science, 12, 9–17. Most, S. B., Scholl, B. J., Clifford, E., and Simons, D. J. (2005). ‘What you see is what you set: Sustained inattentional blindness and the capture of awareness’. Psychological Review, 112(1), 217–42. Neta, R. and Rohrbaugh, G. (2004). ‘Luminosity and the safety of knowledge’. Pacific Philosophical Quarterly, 85(4), 396–406. Nigel, J. T. (2010). ‘Mental imagery’. In Edward N. Zalta (ed.), The Stanford Encyclopedia of Philosophy (Fall 2010 edn), . Nozick, R. (1981). Philosophical Explanations. Cambridge, MA: Harvard University Press. Peacocke, C. (1995). A Study of Concepts. Cambridge, MA: MIT Press. Peacocke, C. (2004). The Realm of Reason. New York: Oxford University Press. Perky, C. W. (1910). ‘An experimental study of imagination’. American Journal of Psychology, 21, 422–52. Plantinga, A. (1993). Warrant and Proper Function. Oxford: Oxford University Press. Plantinga, A. (1996). ‘Respondeo’. In J. Kvanvig (ed.), Warrant in Contemporary Epistemology. Lanham, MD: Rowman and Littlefield. Pollock, J. (1974). Knowledge and Justification. Princeton: Princeton University Press. Pollock, J. (1986). Contemporary Theories of Knowledge. Lanham, MD: Rowman and Littlefield. Pollock, J. L. and Cruz, Joseph (1999). Contemporary Theories of Knowledge. Lanham, MD: Rowman and Littlefield. Pritchard, D. (2005). Epistemic Luck. Oxford: Clarendon Press. Proffitt, D. and Linkenauger, S. (2013). ‘Perception viewed as a phenotypic expression’. In W. Prinz, M. Beisert, and A. Herwig (eds), Tutorials in Action Science. Cambridge, MA: MIT Press. Pryor, J. (2000). ‘The skeptic and the dogmatist’. Noûs, 34(4), 517–49.

The Epistemology of Perception    809 Pryor, J. (2001). ‘Highlights of recent epistemology’. British Journal for the Philosophy of Science, 52(1), 95–124. Pryor, J. (2004). ‘What’s wrong with Moore’s argument?’ Philosophical Issues, 14, 349–78. Pryor, J. (2005). Is there immediate justification? In M. Steup (ed.), Contemporary Debates in Epistemology. Oxford: Blackwell. Putnam, H. (1975). ‘The meaning of "meaning". Minnesota Studies in the Philosophy of Science, 7, 131–93. Pylyshyn, Z. W. (1998). Is vision continuous with cognition? The case for cognitive impenetrability of visual perception. Behavioral and Brain Sciences, 1999(22), 341–65. Roush, S. (2006). Tracking Truth:  Knowledge, evidence, and science. New  York:  Oxford University Press. Russell, B. (1912). The Problems of Philosophy. New York: Henry Holt and Company. Russell, B. (1948). Human Knowledge:  Its Scope and Limits. London:  George Allen and Unwin. Sainsbury, R. M. (1997). ‘Easy possibilities’. Philosophy and Phenomenological Research, 57(4), 907–19. Schechter, J. (2013). ‘Rational self-doubt and the failure of closure’. Philosophical Studies, 163, 428–52. Searle, J. (1983): Intentionality: An Essay in the Philosophy of Mind. Cambridge: Cambridge University Press. Segal, S. J. (1972). ‘Assimilation of a stimulus in the construction of an image: The Perky effect revisited’. In P. W. Sheehan (ed.), The Function and Nature of Imagery (pp. 203–30). New York: Academic Press. Sellars, W. (1956). ‘Empiricism and the philosophy of mind’. In H. Feigl and M. Scriven (eds), Minnesota Studies in the Philosophy of Science, i: The Foundations of Science and the Concepts of Psychology and Psychoanalysis (pp. 253–329). Minneapolis: University of Minnesota Press. Siegel, S. (2010). The Contents of Visual Experience. New York: Oxford University Press. Siegel, S. (2013). ‘The epistemic impact of the etiology of experience’. Philosophical Studies, 162(2), 697–722. Silins, N. (2005). ‘Deception and evidence’. Philosophical Perspectives, 19(1), 375–404. Silins, N. (2008). ‘Basic justification and the Moorean response to the skeptic’. In T. Gendler and J. Hawthorne (eds), Oxford Studies in Epistemology, vol. ii (pp. 108–142). Oxford: Oxford University Press. Silins, N. (2011). ‘Seeing through the "veil of perception" ‘. Mind, 120(478), 329–67. Silins, N. (2012). ‘Explaining perceptual entitlement’. Erkenntnis, 76(2), 243–61. Silins, N. and Siegel, S. (2014). ‘Consciousness, attention and justification’. In D. Dodd and E. Zardini (eds), Contemporary Perspectives on Scepticism and Perceptual Justification. Oxford: Oxford University Press. Smithies, D. (2011). ‘What is the role of consciousness in demonstrative thought?’ Journal of Philosophy, 108(1), 5–34. Smithies, D. (2012). ‘Mentalism and epistemic transparency’. Australasian Journal of Philosophy, 4, 723–41. Sosa, E. (1991). Knowledge in Perspective: Selected Essays in Epistemology. Cambridge: Cambridge University Press.

810    Susanna Siegel and Nicholas Silins Sosa, E. (1997). ‘Reflective knowledge in the best circles’. Journal of Philosophy, 94(8), 410–30. Sosa, E. (1999). ‘How to defeat opposition to Moore’. Philosophical Perspectives, 13(s13), 137–49. Sosa, E. (2003). ‘Beyond internal foundations to external virtues’. In L. BonJour and E. Sosa (eds), Epistemic Justification: Internalism vs. Externalism, Foundations vs. Virtues (pp. 3–96), Malden, MA: Blackwell. Sosa, E. (2005). ‘Dreams and philosophy’. Proceedings and Addresses of the American Philosophical Association, 79(2), 7–18. Sosa, E. (2007). A Virtue Epistemology: Apt Belief and Reflective Knowledge, vol. i. Oxford: Oxford University Press. Sosa, E. (2009). Reflective Knowledge: Apt Belief and Reflective Knowledge, vol. ii. Oxford: Oxford University Press. Soteriou, M. (2010). ‘The disjunctive theory of perception’. In Edward N. Zalta (ed.), The Stanford Encyclopedia of Philosophy (Winter 2010 edn), . Speaks, J. (2005). ‘Is there a problem about non-conceptual content?’3 Philosophical Review, 114, 359–98. Sperling, G. (1960). ‘The information available in brief visual presentations’. Psychological Monographs, 74, 1–29. Stalnaker, R. (2003). ‘What might nonconceptual content be?’ In York H. Gunther (ed.), Essays on Nonconceptual Content. Cambridge, MA: MIT Press. Swain, M. (1979). ‘Justification and the basis of belief’. In G. Pappas (ed.), Justification and Knowledge. Boston: D. Reidel. Travis. C. (2004). ‘The silence of the senses’. Mind, 113, 57–94. Travis, C. (2013). ‘Comment on Siegel, The Contents of Visual Experience’. Philosophical Studies, 163(3), 837–46. Turri, J. (2010). On the Relationship between Propositional and Doxastic Justification. Philosophy and Phenomenological Research, 80(1), 312–26. Tye, M. (1995). Ten Problems of Consciousness: A Representational Theory of the Phenomenal Mind. Cambridge, MA: MIT Press. Van Cleve, J. (2003). ‘Is knowledge easy or impossible?’ In S. Luper (ed.), The Skeptics: Contemporary Essays. Aldershot: Ashgate Publishing Limited. Vogel, J. (1990a). ‘Cartesian skepticism and inference to the best explanation’. Journal of Philosophy, 87(11), Eighty-Seventh Annual Meeting American Philosophical Association, Eastern Division (Nov. 1990), 658–66. Vogel, J. (1990b). ‘Are there counterexamples to closure?’ In M. D. Roth and G. Ross (eds), Doubting: Contemporary Perspectives on Skepiticism. Dordrecht: Kluwer. Vogel, J. (2000). ‘Reliabilism leveled’. Journal of Philosophy, 97(11), 602–23. Wedgwood, R. (2013). ‘A priori bootstrapping’. In Albert Casullo and Joshua Thurow (eds), The A Priori in Philosophy. Oxford: Oxford University Press. Weisberg, J. (2010). ‘Bootstrapping in general’. Philosophy and Phenomenological Research, 81(3), 525–48. White, R. (2006). ‘Problems for dogmatism’. Philosophical Studies, 131(3), 525–57. Williamson, T. (2000). Knowledge and its Limits. Oxford: Oxford University Press. Wright, C. (1985). ‘Facts and certainty’. Proceedings of the British Academy, 71, 429–72.

The Epistemology of Perception    811 Wright, C. (2002). ‘(Anti-)sceptics simple and subtle: G. E. Moore and John McDowell’. Philosophy and Phenomenological Research, 65(2), 330–48. Wright, C. (2004). ‘Warrant for nothing (and foundations for free?)’. Proceedings of the Aristotelian Society, 78, 167–212. Wright, C. (2007). ‘The perils of dogmatism’. In S. Nuccetelli and G. Seay (eds), Themes from G. E. Moore: New essays in epistemology. Oxford: Oxford University Press.

Chapter 42

Perceptua l L e a r n i ng Robert L. Goldstone and Lisa A. Byrge

Perception can be learned. Experience shapes the way people see and hear. In one sense, these are barely interesting claims. After all, experience provides the sensory input to our perceptions as well as knowledge about the identities and functions of the objects that make up our physical environment. Perceptual learning, however, speaks to a much deeper relationship between experience and perception, in which fundamentally different perceptions of the same sensory input may arise in individuals with differing experiences or training. This raises important issues about the ontology of sensory experience, the relationship between cognition and perception, and the possibility of a theory-neutral perceptual ground for science. Given the importance of responding efficiently and effectively to the environment, one might expect that ‘hard-wired’ perceptual circuitry would be an optimal design, especially given that the optic properties of the environment are largely stable. Plastic perceptual circuitry might be thought to be risky, given that changes made to the early representation of information will affect all subsequent processing. ‘Early’ in the previous sentence has two meanings, and both are relevant to our discussion. Developmentally early perceptual changes are those that occur during infancy. Operationally early perceptual changes are those that occur during the first few processing stages of transducing external signals to brain events. Although developmentally and operationally early perceptual changes are risky, ubiquitous variation in every individual’s environment—in local and regional attributes of objects, flora, fauna, even communication systems—highlight the inherent limitations of ‘built-in’ perceptual mechanisms and the importance of mechanisms for perceptual learning. One might expect that developmentally early perceptual learning would occur rarely, because evolution should have already tuned our perceptual systems to be sensitive to the most important elements of our shared environment. In fact, developmental research shows that some human perceptual constraints appear to be learned rather than innate (Quinn and Bhatt, 2006). This research indicates that the observed perceptual plasticity is often times highly specific to the trained environment rather than the environment acting as a general trigger for the maturation of the perceptual system. For example, 4-month old infants develop perceptual representations for specific configurations of visual elements that co-occur (Needham et al., 2005).

Perceptual Learning   813 Furthermore, work in animals (Blakemore and Cooper, 1970) has shown that the absence of environmental regularities (such as horizontally orientated features) early in development can result in the lack of neurons dedicated to these features in the primary visual cortex, and insensitivity to these regularities in adulthood—demonstrating the importance of perceptual learning in establishing even the core structures on which perception is built. The importance of a mechanism for integrating past visual experience with ‘hard-wired’ circuitry is underscored by what we might consider to be the functional utility of perception, ‘to produce the best current interpretation of the visual scene in light of past experience either of ourselves or of our ancestors’ (Crick and Koch, 1995). In what follows, we will consider changes to the organization of a sensory object into elements or relatively long-lasting changes to one’s sensitivity to elements to be perceptual changes. While some researchers argue that changes of attention to stimulus elements should be considered pre- or post-perceptual (Pylyshyn, 1999), habitual attention to task relevant features leads to their perceptual sensitization, and affects how the objects are subjectively perceived (see MacPherson, 2012, for a theoretical analysis of some of the evidence for this) as well as perceptual discriminations that one can make (Goldstone, 1994).

1  Individual differences, individual similarities, and the (im) penetrability of perception The role of past experience in shaping the perceptual system suggests the possibility of substantial individual differences in perception. The tailoring of perceptual systems to individuals’ situations raises the concern of unconstrained cognitive penetration. Cognitive penetration of central system beliefs and goals on perception occurs if perceptions can differ by virtue of people having different beliefs or goals. Some researchers argue that cognitive penetrability does not in fact occur (Pylyshyn, 1999, 2003). Such cognitive penetrability of perception might entail that there is no neutral perceptual ground upon which scientific theory can be verified (Fodor, 1984; see also Susanna Siegel and Nicholas Silins, Chapter 41, this volume), raising the spectre of relativistic theories of meaning. If what we see depends on what we believe, then how can scientists from different paradigms use perceptual evidence to adjudicate in favour of one theory or another? One answer to these concerns has been to claim that perceptual learning is ‘data-driven or task-driven’, and not ‘theory-driven’, meaning that it is the stimulus properties that determine perceptual learning not antecedent beliefs regarding the ‘intentional stance’ of these stimulus properties. This therefore constitutes only an indirect cognitive penetrability of perception (via allocation of focal attention to spatial locations in the stimulus). Because in principle, scientists with unequal perceptual learning (but similar perceptual circuitry and scientific training) can receive instructions in how to allocate their attention and therefore obtain an equal perceptual basis, this indirect cognitive penetrability of perception is not problematic for the existence of a theory-neutral perceptual ground, and this dispels relativistic concerns (Raftopoulos, 2001).

814    Robert L. Goldstone and Lisa A. Byrge However, there is reason to think that perceptual learning is sometimes theory-driven. There is an important sense in which certain early visual algorithms (e.g. edge detection) are not directly trainable, and it is certainly not the case that all information influences all processing at all levels. Nevertheless, there are so many ways of guiding perception according to one’s goals, that to say that top-down information cannot influence a small stage of early vision seems overly scholastic—because the functional consequence is of cognitive, theory-driven, influence on perception. For instance, one might deform the stimulus input by pushing on one’s eye, which changes the character of the stimulus data—a theory-laden influence on the stimulus array, which then produces changes in ‘data-driven’ selective attention. In subsequent sections of this chapter, we consider other evidence on how tasks and goals influence how perceptual systems adapt. Furthermore, this argument for an across-the-board denial of theory-laden influences on perception seems to presuppose that perceptual systems are essentially identical apart from perceptual learning. However, habitually executed tasks cause not only particular sensory features to be selectively attended to, but also feature creation (Schyns et al., 1998). Feature creation involves creating new perceptual organizations of sensations. For example, brightness and saturation, two components of colour, are psychologically fused for most people in that it is difficult to attend to one of these components without attending to the other. However, sufficient experience with isolating out colour components can lead colour experts to create psychological dimensions that can be selectively attended to (Burns and Shepp, 1988). Perceptual learning that requires allocating attention to previously created features can count as theory-driven perceptual learning given the importance of goals and tasks on the original construction of perceptual features. It is important to distinguish between the role of goals in shaping perception over a long time course and the role of goals in shaping one’s moment-to-moment perception without permanently changing a perceptual system. Here we are focusing on the former kind of goal-dependence. Given the developmental evidence, it seems likely that much of perceptual learning builds upon previously created perceptual building blocks. Pylyshyn (1999, 2003) grants that cognition can shape the operation of vision into certain ‘compiled transducers’, computational routines that become automatized through practice and eventually become encapsulated as part of the early vision system itself. Although this diachronic change does not count as direct cognitive penetrability of perception, it entails that people with different experiences could have fundamentally different perceptual systems. This, in turn, is problematic for claims that equivalent training can equalize perceptual differences among scientists, and consequently problematic for the existence of a theory-neutral perceptual ground. Given the ubiquity of top-down influences on perception (even on ostensibly ‘early’ visual processes, e.g. Chen and Zhou, 2011), some have questioned the value of drawing a firm boundary between perception and cognition at all (Goldstone and Barsalou, 1998; Barsalou, 1999). While we will not defend the radical claim of continuity between perception and cognition here, gross anatomical considerations do indicate that when one brain region X sends feed-forward information to region Y, there are typically dense recurrent connections from Y to X. A theory-neutral foundation for perceptual processes is unlikely to exist in practice, either at a low level or a high one because, for example, novices and bird experts perceive the same birds differently, even given the same- retinal input (Gauthier et al., 2010).

Perceptual Learning   815 However, this does not preclude the lack of such a shared basis in principle, given common training experiences. While there are striking differences in individuals’ perceptual processes due to their experiences, the process by which perceptual systems change with experience is largely shared across individuals. In other words, providing equivalent perceptual learning opportunities generally produces equivalent perceptual changes. (See Olshausen and Field, 1996, for one model, though on an evolutionary time scale.) This shift from a first-order to second-order invariant across people grants that equivalent perceptual learning opportunities involve more than just data-driven perceptual learning. By questioning a strict boundary between perception and conception, we can suppose that with shared communication, a shared familiar world of physical objects, shared scientific and perceptual training, and shared developmental trajectories, the shared perceptual-cognitive basis is probably ‘neutral enough’ for successful scientific communication (Swoyer, 2003).

2  Evidence and rationale for early loci of perceptual change During learning, changes to the human perceptual system occur at multiple stages in the information processing stream. There is general consensus that changes to earlier, more peripheral stages of processing are more unambiguously identifiable as ‘genuine’ cases of perceptual learning compared to later, more central stages. This belief hinges on the assumption that later stages of information processing in the brain may not reflect perceptual processes that are uncontaminated by context and experience. There is evidence, however, that learning and context influence even relatively early stages of perceptual processing. This evidence is based on both neuroscience and functional behaviour.

Neurological evidence for early changes to perception Neurologically speaking, changes to early-to-middle stages of visual processing have been implicated in the development of expertise. Electrophysiological recordings of dog and bird experts show enhanced electrical activity at 164 milliseconds after the presentation of dog or bird pictures (Gauthier et al., 2010). Practice in discriminating small motions in different directions significantly alters electrical brain potentials that occur within 100 milli­ seconds of the stimulus onset (Fahle, 1994). These electrical changes are centred over the part of the visual cortex primarily responsible for motion perception (the medial temporal visual area, MT), suggesting plasticity in early visual processing. Furmanski et al. (2004) used functional magnetic resonance imaging (fMRI) to measure brain activity before and after one month of practice detecting hard-to-see oriented line gratings. Training increased V1 response for the practised orientation relative to the other orientations, and the magnitude of V1 changes were correlated with detection performance. Bao et al. (2010) trained human subjects for one month to detect a diagonal grating, and found EEG differences in V1 for trained versus untrained orientations within 50–70 milliseconds after the onset of the stimulus. The rapidity of the EEG difference, combined with the demanding

816    Robert L. Goldstone and Lisa A. Byrge nature of the primary behavioural task during testing make it unlikely that the earliest EEG differences were mediated by top-down feedback from higher cortical levels. In the somewhat later visual area V4, single-cell recording studies in monkeys have shown activity changes of cells in early visual cortex (Yang and Maunsell, 2004). Individual neurons with receptive fields overlapping the trained location of a line orientation discrimination developed stronger responses, and more narrow tuning, to the particular trained orientation, compared with neurons with receptive fields that did not fall on the trained location. In the auditory modality, Weinberger (1993) describes evidence that cells in the primary auditory cortex become tuned to the frequency of often-repeated tones. Training in a selective attention task produces differential responses as early as the cochlea (Puel et al., 1988). This amazing degree of top-down modulation of a peripheral neural system is mediated by descending pathways of neurons that project from the auditory cortex all the way back to olivocochlear neurons, which directly project to outer hair cells within the cochlea—an impressively peripheral locus of modulation. More generally, we do not find it particularly productive to ask the question of ‘How early does perception change due to learning?’ because it is clear that prior learning affects sensory processing even before sensory processing begins. In particular, learning influences how objects will impinge upon our sensory organs. In many cases, perceptual learning involves acquiring new procedures for actively probing one’s environment (Gibson, 1969), such as learning procedures for efficiently scanning the edges of an object (Salapatek and Kessen, 1974). The result is that adults look at objects differently than children, and experts look at objects differently than novices; and since each fixates objects differently, the visual patterns that fall on an observer’s retina vary with experience. Perceptual changes are found at many different neural loci and a general rule seems to be that earlier brain regions are implicated in finer, more detailed perceptual training tasks (Ahissar and Hochstein, 1997). The claim for widespread neural plasticity in brain regions related to perception should not be interpreted as an argument for the equipotentiality of brain regions for implementing modifications to perception. Evidence for plasticity at the earliest visual processing area of the cortex, V1, remains controversial (Crist et al., 2001; Kourtzi and DiCarlo, 2006). Some of the observed activity pattern differences in V1 may be attributable to top-down influences after a first forward sweep of activity has passed. However, the very presence of large recurrent connections from more central to more peripheral brain regions attests to the evolutionary importance of tailoring input representations to one’s tasks. Properties of V1 cells depend on the perceptual task being performed and experience, in the sense that neurons respond differently to identical visual patterns under different discrimination tasks and with different experiences. Moreover, these top-down influences are seen from the onset of neural response to a stimulus (Li et al., 2004). The perceptual change, thus, is early both in the information processing stream of the brain and chronometrically. One common source of evidence for an early neural locus for perceptual learning has been observations of surprisingly limited transfer of learning. Training on simple visual discriminations often does not transfer to different eyes, to different tasks, to different spatial locations (Shiu and Pashler, 1992), or to different viewing distances (Huang et al., 2011). The customary interpretation of these narrow degrees of generalization is that early perceptual detectors tend to have narrow and small receptive fields, and that downstream detectors have larger receptive fields. So, if narrow generalization is found, it is likely to

Perceptual Learning   817 be driven by the earlier detectors. In truth, changes to perceptual systems are found at multiple stages during perceptual processing (Ahissar and Hochstein, 1997), with some kinds of perceptual learning being mediated by more general and strategic changes. Accordingly, perceptual learning is not as restricted as one might suspect according to a ‘data-driven’ account in which perceptual change is accomplished by a perceptual process being passively imprinted upon by environmental stimuli. A role for strategic adaptation is also suggested by the finding that limited generalization of perceptual learning can be greatly reduced by giving observers a small amount of training at the transferred direction or location (Xiao et al., 2008).

Functional evidence for early changes to perception Parallel to neurological evidence for loci of perceptual changes that accompany experience, there are functional, behavioural sources of evidence indicating changes to early stages of perceptual processing. Experience often exerts an influence before other putatively early perceptual processes have been completed. For example, subjective experience exerts an influence on colour perception before the perceptual stage that creates colour after-images has completed its processing (Moscovici and Personnaz, 1991). As a second example, Peterson and Gibson (1994) found that the organization of a scene into figure and ground is influenced by the visual familiarity of the contours. Their participants made judgements about whether a visual form was a figure or ground. Familiar forms were more likely to be judged to be figures than unfamiliar forms. This effect was not found when the familiarity of the objects was eliminated by flipping the scenes upside down. Interpreting the familiar region as a figure was found even when the unfamiliar regions had the strong Gestalt organization cue of symmetry. Peterson and Lampignano (2003) found direct evidence that the acquired familiarity of a shape successfully competes against Gestalt cues such as partial closure to determine the organization of a scene into figure and ground. Perceptual organizations that are natural according to Gestalt laws of perception can be overcome in favour of perceptual organizations that involve familiarized materials. This indicates that training can influence relatively early stages of the information processing stream. In work on object-based attention, Behrmann, Zemel, and Mozer (1998) found that judgements about whether two parts had the same number of humps were faster when the two parts belonged to the same object rather than different objects, with objecthood being based on standard Gestalt laws of organization. However, follow-up work found an influence of experience on subsequent part comparisons. Two fragments were interpreted as belonging to the same object if they had co-occurred many times in a single shape (Zemel et al., 2002). Object fragments that are not naturally grouped together because they do not follow the Gestalt law of good continuation can nonetheless be perceptually joined if ­participants are familiarized with an object that unifies the fragments. Another approach to identifying the functional locus of perceptual changes is to observe the time course of the use of particular types of information. For example, on the basis of priming evidence, Sekuler, Palmer, and Flynn (1994) argue that knowledge about what an occluded object would look like if it were completed influences processing after as little as 150 milliseconds. Dog and bird experts reveal significantly enhanced

818    Robert L. Goldstone and Lisa A. Byrge N1701 electrophysiological responses when categorizing objects within their domain of expertise relative to objects outside of this domain (Tanaka and Curran, 2001). These influences are sufficiently early to typically be counted as perceptual processing.

Reasons why early perceptual learning occurs The plasticity of early perceptual processes may seem counterproductive. Don’t we want perception to act as a source of information that is uncontaminated by a perceiver’s beliefs and history? To the extent that we all live in the same physical world, shouldn’t we all be equipped with the same perceptual apparatus? There is something right about this intuition. Our early perceptual processes should change slowly and conservatively, because they are the bedrock for all subsequent processes. A change early in the brain will have ripples of influence downstream. The answer, though, to why it may still be a good idea to adapt early perceptual processes to experiences is that flexibility is beneficial when the world is variable. If everyone were confronted with the same environment, and this environment remained unchanged millennium after millennium, then perceptual systems could become hard-wired for this particular environment. These perceptual systems would be efficient because they are specifically tuned to the unchanging environment. Some environmental factors, such as colour characteristics of sunlight, the position of the horizon, and the change in appearance that an approaching object makes, have all been mostly stable over the time that the human visual system has developed. However, if we look more closely, there is an important sense in which different people face different environments. To a large extent, a person’s environment consists of animals, people, and things made by people. Animals and people have been designed by evolution to show variability, and artefacts vary widely across cultures. Evolutionary pressures may have been able to build a perceptual system that is generally adept at processing faces (Gauthier et al., 2010), but they could not have hardwired a neural system that was adept at processing a particular face, such as Barack Obama’s, for the simple reason that there is too much generational variability among faces. Individual faces do not last from generation to generation, and so people’s ability to recognize specific, highly familiar faces cannot be hardwired. Rather, what is hardwired is the ability to develop perceptual systems tuned to particular faces. Variability is apparent over only slightly longer periods for artefacts, words, ecological environments, and animal appearances. Thus, we can be virtually positive that tools show too much variability over time for there to be a hardwired detector for hammers. Words and languages vary too much for there to be a hardwired detector for the written letter ‘A’. Biological organisms are too geographically diverse for people to have formed a hardwired cow-detector. When environmental variability is high, the best strategy for an organism is to develop a general perceptual system that can adapt to its local conditions. There is an even deeper sense in which people face different environments. People find themselves in different worlds because they choose to specialize. English-speaking people

1 

The 170 in N170 means that the response occurs 170 milliseconds after stimulus onset.

Perceptual Learning   819 become specialized at hearing and seeing English words. People familiar with a particular race become specialized at recognizing faces from that race (Levin, 2000). Experts at wine tasting, chick sexing, X-ray diagnosing, identical twin identifying, and baseball pitch judging all have unique perceptual capabilities because of the tasks they perform. Experts typically have highly specialized skills, many of which are perceptual (Sowden et al., 2000). Moreover, the above examples of word and face recognition suggest that every person has domains in which they show expertise. Even if all people confronted the same world initially, they would create distinctive communities with unique languages, artefacts, and objects of importance. One’s social niche will depend on many factors including proclivity, community needs, local support, random accidents, and positive feedback loops. Thus, it is again advisable to build a perceptual system with the flexibility needed to support any one of a large number of niches. It is worth emphasizing that people do not have to strategically, consciously tune their perceptual systems to support these tasks that are specific to an individual’s needs and niches. Instead, the perceptual machinery itself, and the brain areas it projects to, can often automatically accomplish the necessary tuning. Likewise, perceptual tuning can occur independently of a perceiver’s antecedent beliefs. In fact, once a tailored perceptual system is firmly in place, it is impossible to revert it to an untailored state even if one is motivated to do so by momentary beliefs or desires.

3  Mechanisms of perceptual change The above neurophysiological and functional evidence makes a good case for relatively early changes to perception due to experience, particularly if one includes changes to eye fixations that change perceptual differences even before the retina. This case having been briefly made, the discussion that follows focuses not on the loci of changes, but rather on the mechanisms that underlie some of these changes. These mechanisms will be described at a functional rather than physiological level. Even though neurological details are known in some cases, a functional level of description is appropriate for unifying accounts of human and computational learning (Goldstone, 2003), and for understanding the kinds of changes that may benefit a perceiver.

Tuned attention A person can dynamically shift their attention to different stimulus features depending on their perceived importance. One way that perception becomes adapted to tasks and environments is by increasing the attention paid to perceptual features that are important, and/or by decreasing attention to irrelevant dimensions and features. This mechanism can be distinguished from a more passive ‘imprinting’ process in which functional perceptual detectors are developed that are specialized for stimuli or parts of a stimuli. In computational learning terms, imprinting is an unsupervised learning mechanism in that there is no need for a parent, teacher, or programmer to tell the learner what a stimulus is called or what parts are important. However, in attentional tuning, supervision, including

820    Robert L. Goldstone and Lisa A. Byrge self-supervision, is key to learning. In distinguishing ripe from unripe mangoes, colour must be attended to, but to distinguish books from magazines, colour is not useful. An observer equipped with the ability to tune their attention to different object dimensions would be able to learn both of these categorizations. The world is structured such that different dimensions are important for different life-relevant categorizations. As such, it comes as little surprise that most successful theories of categorization and learning incorporate selective attention. To take Nosofsky’s (1986) Generalized Context Model of categorization as an example, the categorization of an object depends on its similarity to previously stored category members in a multidimensional space. Critically, psychological distances between objects are compressed and expanded along dimensions in this space depending on the categorization required. Distances between objects on relevant dimensions are expanded while irrelevant dimensions are compressed. For example, Nosofsky finds that if participants are given a categorization where the angle of a line embedded in a circular form is relevant while the size of the circular form is irrelevant, then distances between objects on the angle dimension are increased and distances on the size dimension are decreased. In the Generalized Context Model, entire perceptual dimensions like length, brightness, and orientation are psychologically stretched or shrunk. However, specific regions within a perceptual dimension can also be selectively attended. This capacity is important for driving the phenomenon of Categorical Perception (CP) (Goldstone and Hendrickson, 2010; see also Raffman, Chapter 36, this volume). In CP, our perceptions are adapted such that differences between objects that belong to different categories are accentuated, and differences between objects that fall into the same category are deemphasized. That is, our perceptual systems transform relatively linear sensory signals into relatively non-linear internal representations. The extreme case of this kind of non-linear transformation is a step function by which increases to a sensory signal have no effect on perception until the signal reaches a certain threshold. At that threshold, perception qualitatively and suddenly changes. During the flat portion of the staircase function, different input signals have equivalent effects. This flat response profile for a range of stimuli provides a mechanism that grounds equivalence classes—for treating distinguishable stimuli as equivalent. Equivalence classes, in turn, provide us with the beginnings of symbolic thought—quasi-discrete responses that are reliably generated when stimuli within a range are presented or contemplated. The underpinning for the non-linear perceptions of CP is region-specific attention tuning. For example, in Liberman, Harris, Hoffman, and Griffith’s (1957) seminal research on speech perception, a continuum of equally spaced consonant–vowel syllables with endpoints reliably identified as /be/ and /ge/ was generated. At a specific point along this continuum observers rapidly shift from hearing the sound as a /be/ to hearing it as /de/. In addition to giving participants an identification task, participants were also given an ABX discrimination task. In this task, observers listened to three sounds—A followed by B followed by X—and indicated whether X was identical to A or B. Observers performed the task more accurately when syllables A and B belonged to different phonemic categories, as indicated by their identification probabilities, than when they were variants of the same category, even when physical differences were equated. Although some cases of CP may be innate, there is also strong evidence that at least some cases involve learning. Using laboratory-created, speech-like stimuli that were assigned to different categories based on their labels, Lane (1964) found CP effects despite

Perceptual Learning   821 a lack of correspondence between the trained categories and naturally occurring language categories. In general, a sound difference that crosses the boundary between phonemes in a language will be more discriminable to speakers of that language than to speakers of a language in which the sound difference does not cross phonemic boundaries (Repp, 1984). Goldstone (1994) showed that when arbitrary new visual categorizations are made, they alter participants’ same/different judgements. Discriminations along categorization-relevant dimensions were improved, and this improvement was greatest at the boundary between the categories. The attention tuning phenomena reviewed above indicate an important interplay between humans’ higher-level conceptual systems and their lower-level perceptual systems. Traditional information flow diagrams in cognitive science typically draw a clean division between perceptual and conceptual systems, with information moving only from perception to the conceptual system. The common occurrence of attentional tuning indicates permeability and bidirectional influence between these systems. We humans do not simply base our categories on the outputs of perceptual systems independent of feedback. Instead, our perceptual systems become customized to the task-useful categories that we acquire. We are not optimistic that a clean dividing line between perception and attention can be drawn because (1) with training, the sensory encoding for attended objects becomes richer and more differentiated than for unattended objects; (2) with training, attention to objects becomes automatically deployed and difficult to strategically control; and (3) fast and widely prevalent recurrent connections from higher to lower cortical regions makes it difficult, sometimes impossible, to identify a ‘forward-volley’ stage of sensory processing that is uninfluenced by attention.

Unitization Beyond simply tuning attention to existing perceptual dimensions or regions of a dimension, there are two perceptual learning mechanisms that create new ‘building blocks’, new perceptual units that can be attended to, searched for, and combined together to form new percepts. One of these mechanisms is unitization, according to which single functional units are constructed that are triggered when a familiar complex configuration arises. Unitization is the perceptual equivalent of the memory-based phenomenon of chunking. In memory, even if we can only store 7 +/– 2 items in our short-term memory, we can learn to store increasing amounts of information by increasing the size of each of the items (chunks) in memory. We can easily remember the 27 letters ‘M O N T U E W E D F B I C I A K G B C B S N B C A B C’ even though there are far more than 7 letters by combining the letters to create acronyms such as MON, FBI, and CBS. Chunking at a still higher level, we can remember that the acronyms form three categories: days of the week, secretive government organizations, and television broadcast channels. Similarly, our perceptual systems can build new perceptual chunks to encode as single units what would otherwise be complex visual patterns. Whereas for chunks in memory elements are combined together because they make semantic sense, perceptual units are formed because they can be seen as coherent perceptual objects, ­obeying, for example, the Gestalt laws of proximity, continuity, and closure. Cattell (1886) invoked the notion of perceptual unitization to account for the advantage that he found for tachistoscopically presented words relative to non-words. Unitization has

822    Robert L. Goldstone and Lisa A. Byrge also been posited in the field of attention, where researchers have claimed that shape components of often-presented stimuli become processed as a single functional unit with practice. Shiffrin and Lightfoot (1997) report evidence from the slopes relating the number of distracter elements to response time in a feature search task. When participants learned a conjunctive search task in which three line segments were needed to distinguish the target from distracters, impressive and prolonged decreases in search slopes were observed over 20 hour-long sessions. These prolonged decreases were not observed for a simple search task requiring attention to only one component. The authors concluded that conjunctive training leads to the unitization of the set of diagnostic line segments, resulting in fewer required comparisons. Gauthier and Tarr (1997; see also Gauthier et al., 1998) found that prolonged experience with a novel object leads to a configural representation of it that combines all of its parts into a single, viewpoint specific, functional unit. Their evidence for such a representation is that recognition of these familiarized objects improved considerably with practice, and was much more efficient when the object was in its customary upright form rather than inverted. Two more recently developed methods for detecting processing based upon single functional units for an entire complex object, also called ‘holistic’ processing, involve ‘whole-part’ and ‘composite’ tasks. The logic of the whole–part paradigm is that if a stimulus element is being processed as part of a larger perceptual unit, then recognition of the part should be less efficient than recognition of the entire unit. For example, Tanaka and Farah (1993) show subjects a whole face named ‘John’. After a brief delay, they are either asked to identify which of two noses in isolation, say John’s or Kevin’s, was the previously displayed nose. On other trials, they are asked to say which of two faces was John’s face; one of the faces is indeed John’s face, and the other face is exactly the same as John’s face except that John’s nose has been replaced by Kevin’s nose. Even though both tasks ostensibly require selecting the face with John’s nose rather than Kevin’s nose, subjects are better in the whole face than the isolated part condition. This result is reminiscent of earlier demonstrated word-superiority and object-superiority effects showing that perception of parts in the context of larger familiar objects is better than perception of isolated parts (Wheeler, 1970), and has been explained in terms of the whole face being a functional unit of perception. In the composite task, a composite face is formed by combining the top and bottom parts of possibly different individual faces. Subjects are tasked with responding to only the top or bottom part of the face while trying to ignore the other half (Carey and Diamond, 1977). In fact, subjects find it difficult to ignore the irrelevant half. For example, if the top part of a composite belongs to John but the bottom belongs to Kevin, then it is difficult for subjects to respond ‘John’ to the top part, compared to a situation in which both halves belong to John. The distracting influence of the irrelevant part is reduced or eliminated if the halves are misaligned—if they no longer meet to create to form an apparently coherent face (Gauthier and Bukach, 2007). From the evidence considered thus far from both the whole–part and composite paradigms, it might simply be concluded that humans have been evolutionarily wired such that whole faces are the unit of perception. It is, indeed, rare to stumble across isolated noses, and when one does, they are not usually social objects of importance. However, the units implicated by these tasks are not only faces, but are more generally objects with which an observer has had prolonged experience (Gauthier et al., 2010). Empirical evidence for

Perceptual Learning   823 perceptual units has been found for objects as diverse as birds, words, grids of lines, random wire structures, fingerprints, artificial blobs, and three-dimensional creatures made from simple geometric components, as long as these objects have been familiarized over at least somewhat protracted time courses. Units like these may have been created from more elemental parts, but once they have been formed, they renounce their origins. Once created, the units operate such that perceptions are no longer experienced in terms of the parts, but rather in terms of the whole.

Attribute differentiation If unitization creates large, complex units out of elemental parts, then attribute differentiation begins with a complex perception that fuses together multiple components, and then develops separate percepts for the individual components. In this sense, unitization and attribute differentiation are complementary mechanisms of perceptual learning. However, in another sense they are flip sides of the same coin—they both involve creating perceptual units that are tailored to one’s tasks and experiences. To understand the role of attribute differentiation, it is useful to return to the first described mechanism—tuned attention. Attention tuning requires the ability to selectively attend to perceptual attributes that have already been psychologically isolated. That is, it is only possible for an observer to attend to the brightness of a shape and ignore its size if the attribute of brightness has been isolated and separated from size. What happens if two attributes have not yet been isolated? Saturation (a psychological dimension related to the amount of white/black added to a colour) and brightness (a psychological dimension related to the amount of luminance energy emitted by a colour) are two such attributes. They are fused together to form an overall impression of colour. Ordinarily, people cannot selectively attend to just the saturation of a colour, ignoring its brightness, or vice versa (Garner, 1976). However, there is evidence that people can learn to selectively attend to attributes. If subjects are given training in which saturation is relevant for a categorization and brightness is irrelevant, they can learn to form this categorization, and when they do, subjects subsequently find it easier to distinguish between objects on the basis of saturation rather than brightness differences (Goldstone, 1994). A particularly efficient way to separate the attributes of saturation from brightness is to repeatedly alternate training where saturation is task-relevant with training where brightness is task-relevant, with the end result that either of the attributes can be selectively attended while ignoring the other (Goldstone and Steyvers, 2001). Colour experts such as artists or vision scientists are better able to selectively attend to the component dimensions of colour than are novices (Burns and Shepp, 1988). Even arbitrary dimensions such as those created by morphing between randomly selected faces can be isolated with training that repeatedly makes one, then the other, relevant for a categorization (Goldstone and Steyvers, 2001). The arbitrary dimensions shown in Figure 42.1 do not start off being perceptually isolated for an observer, but can come to be selectively attended with practice that requires their isolation. There is developmental evidence that attributes that are easily isolated by adults, such as the brightness and size of a square, are treated as fused together for 4-year old children (Smith and Kemler, 1978). It is relatively difficult for children to decide whether two objects

824    Robert L. Goldstone and Lisa A. Byrge

Dimension A 1

2

Dimension B

3

4

Fig.  42.1  Stimuli from Goldstone and Steyvers (2001). Every face in the 4  × 4 grid is formed by combining in equal parts its value on Dimensions A  and B.  These dimensions, in turn, are formed by varying the proportion of two randomly chosen faces. Increasing values of Dimension A  correspond to simultaneously decreasing the amount of Face 1 and increasing the amount of Face 2.  Similarly, increasing values of Dimension B correspond to decreasing the amount of Face 3 as the amount of Face 4 increases. are identical on a particular attribute, but relatively easy for them to decide whether they are similar across many attributes. For example, children seem to be distracted by shape differences when they are instructed to make comparisons based on colour. From a functional perspective, we interpret this intrusion of one dimension when trying to respond to another dimension as indicating relatively fused perceptual encodings in children. This interpretation is supported by neurophysiological evidence that children’s cortical sensory areas are not as specialized for specific modalities as are adults’ (Maurer and Mondloch, 2004). Children’s brains have cross-modal connections between auditory, visual, and tactile sensory areas that are subsequently pruned with experience (Spector and Maurer, 2009). Attribute differentiation is not equivalent to tuned attention because prior to differentiation training that separates two attributes, it might have been impossible for an observer to selectively attend to either of these attributes while ignoring the other. After differentiation training, there is a longer-term ability of the observer to switch their attention to either of the two attributes. If selective attention to an attribute can be understood as

Perceptual Learning   825 learning to weight that attribute heavily for a judgement, then attribute differentiation can be understood as learning to learn how to weight an attribute. For example, once an individual has perceptually differentiated the saturation of a colour from its brightness, then s/he can quickly learn a discrimination based on either attending saturation or brightness, quickly learning to switch attention to whichever dimension is relevant at a given time. The extent of dimension differentiation determines the efficiency with which newly relevant ­dimensions can be selectively attended. Attribute differentiation is of theoretical interest because even the possibility of it may be denied. One might suppose that if two attributes are fused together at some point in perceptual processing, then they can never be split apart later. Once yellow and blue watercolour paint have been mixed, they cannot be unmixed. Fortunately, there are computational models that explain how attribute differentiation mechanisms might operate. Competitive learning neural networks differentiate inputs into categories by specializing detectors to respond to classes of inputs. Random detectors that are slightly more similar to an input than other detectors will learn to adapt themselves toward the input and will inhibit other detectors from doing so (Rumelhart and Zipser, 1985). A model that extended this mechanism to sorting object parts into detectors, when presented with an original set of training objects, was able to discover part-based building blocks that could be recombined to recreate the original training objects (Goldstone, 2003). Another learning system shows similar functional behaviour by using Bayesian methods (Austerweil and Griffiths, 2011). In short, advances in machine learning provide existence proofs of mechanisms for dimension differentiation. Computationally speaking, green paint can be separated into its yellow and blue components if one has not only a single sample of green, but several colour samples with different proportions of yellow and blue.

Summary of mechanisms of perceptual learning The perceptual representations that result from distal objects are influenced by experiences with the objects. By attentional tuning, pre-existing perceptual attributes are sensitized or desensitized. By unitization, originally separated parts of an object are combined into unified and coherent perceptual wholes. Once constructed, the unit can be efficiently recognized and has properties similar to an image-like template. The opposite process, attribute differentiation, can also occur, separating originally integrated percepts into psychologically differentiated dimensions or parts. Rather than viewing unitization and differentiation as contradictory, they are best viewed as aspects of the same process that bundles stimulus components together if they diagnostically co-occur, and separates these bundles from other statistically independent bundles. Under this conception, learning a perceptual organization consists in learning how to carve a stimulus into useful components. These empirical phenomena, and their associated computational models, strongly suggest that perceptual learning is affected by our concepts. To be sure, our perceptions also ground our concepts, but interestingly, they provide a better grounding for our concepts because they are flexibly altered by these concepts. Like a mattress that provides support by conforming to the sleeping body that lies on it, our perceptions support our concepts by conforming to them.

826    Robert L. Goldstone and Lisa A. Byrge

4  Modifying perception to get what one wants There is some evidence that we change our perceptual systems even when this is not what we intend to do. For example, mere exposure to perceptual information that is irrelevant to a task can suffice for improving an observer’s perceptual sensitivity to this information (Watanabe et al., 2001). Moreover, this kind of perceptual learning can occur even when the information is in the visual periphery and subthreshold, that is, below the threshold for conscious detection. However, complimenting these results are other results showing that what is learned and how efficiently it is learned depends on the observer’s task and goal. Even when sensitivity to a line orientation appears to have a relatively early locus of change, in that it does not transfer strongly across eyes or visual regions, it nonetheless depends on the observer’s goals (Shiu and Pashler, 1992). Perceptual sensitization to a line orientation is much more robust when it is relevant for the task than when it is irrelevant. When observers are given the same stimuli in two conditions, but are required to make fine, subordinate-level categorizations in one condition but coarser, basic-level categorizations in the other, then greater selectivity of cortical regions implicated in object processing were found in the former condition (Gillebert et al., 2008). Finally, much of the evidence for categorical perception (see Diana Raffman, Chapter 36, this volume) indicates that perceptual discriminations are easier to make at boundaries between important categories for an observer, such as between a /p/ and /b/ phoneme that would be important for distinguishing ‘pats’ from ‘bats’. Evidence from training studies and cross-linguistic comparisons indicates that it is not just perceptual sensitivities that are driving the categories, but rather the important categories are also driving perceptual sensitivities (Goldstone and Hendrickson, 2010). All of these studies show that we get selectively better at making perceptual discriminations just where we should get better in order to help us do what we want and need to do. This observation is particularly relevant to the question of cognitive penetrability (see Susanna Siegel and Nicholas Silins, Chapter 41, this volume). What we see is influenced by repeatedly possessed goals that are close to our biologically determined needs and drives. This is different from the claim that our in-the-moment goals influence our perceptions, which we also believe happens but requires a different assembly of empirical justifications. Our current claim is that the habitual discriminations and categorizations that we make influence our perceptual abilities in a particular functional direction—we selectively improve our perceptual abilities so that the tasks that we need to perform are performed better. In the previous sentence, there are two interpretations of ‘so that’. By one interpretation, ‘so that’ means ‘with the intention that’, implying that we strategically and explicitly alter our perceptual abilities. By the second interpretation, ‘so that’ means ‘with the end result that’, implying that our perceptual abilities are altered naturally through an automatic, non-conscious process. We will return later to the former interpretation, but our main interest lies with the latter interpretation, which has a strong evidentiary basis and is theoretically important as well. The alterability of our perceptual systems in a personally useful direction without our explicit intention to do so is simple but powerful. It is simple in that it requires no more sophisticated a mechanism than random variation plus selection. The effective strengths

Perceptual Learning   827 of neuronal connections are constantly varying. If a change causes important discriminations to be made with increasing efficiency, then the change tends to be preserved and extended. If not, the change will not be made permanent. Dopamine plays a key role during reinforcement learning in which external or internal rewards drive a learning consolidation process. For example, in a motion discrimination task, incorrectly predicting a reward guides changes in connections involving the primary brain area dedicated for motion processing (MT), by selectively strengthening the connections from the most sensitive neurons in the sensory population (Law and Gold, 2009). There may be other more goal-directed processes of neuronal change, but simple random variation with reinforcement that is potentially internally generated suffices to systematically improve perceptual systems at a longer time scale. This systematicity is the key to the power of these changes. Even without opening up the black box of a perceptual module (assuming such a beast to exist), it is possible to make these modules reliably improve their performance much more often than they degenerate, as long as the observer, the environment, or a teacher of some sort can provide feedback on whether the observer is doing better or worse after a change than before. Returning to the question of strategic alterations to perceptual systems, there do appear to be cases in which people purposefully ‘hack’ their perceptual systems in order to facilitate performance. Through trial and error we learn that we can create a sharper image of an object by arranging our fingers so as to create a small aperture near our eye. We learn that cupping our hands behind our ears allows us to hear better, whereas clamping our jaw tight makes our ears less sensitive to noise. Altering one’s physical interactions with the world is a major part of Gibson’s (1979) theory of ‘active perception’. At a second-order level, we learn over time how to improve our own learning. If we are trying to become an expert at identifying birds, we have learned to facilitate this by studying photos of birds repeatedly. Depending on our sophistication and knowledge of the learning process, we may strategically present birds in more caricatured forms such as idealized drawings rather than photographs, in different viewpoints, with time in between presentations, and with different species intermixed. These are all strategic actions that improve perceptual learning, and they can all be accomplished without requiring cognitive penetration that involves our direct alteration of neuronal processes underlying perception. There are many intermediate cases in which it is difficult to tell whether we intend to change our perceptual systems or they just naturally change. For example, work in our laboratory has shown that we change our visual processing system so that it allows us to reason in a formally sanctioned way more efficiently than we would have been able to without the change. In particular, we have studied the ways in which we ‘rig up’ our visual processing to facilitate mathematical reasoning (Landy and Goldstone, 2007; Goldstone et al., 2010). In algebra, multiplication has a higher order of precedence, than addition with ‘2 + 4 × 5’ equalling 22, not 40. Experiments show that one way that we come to produce the correct solution efficiently is by automatically directing our own attention to the ‘×’ operator instead of the ‘+’ operator. Our first eye movements are toward the ‘×’ operator, and the ‘×’ operator distracts our attention from a task requiring attention to the ‘+’ operator more so than vice versa (Goldstone et al., 2010). We learn to solve algebra problems by creating visual operations like moving a notation element from the left to the right side, marking notational elements as processed, cancelling matching notational elements, and directing attention to mathematical groups. Although these operations are ‘merely’ perceptions and

828    Robert L. Goldstone and Lisa A. Byrge actions, these physical operations can be adjusted and tailored so that they conform to formally sanctionable operations. If evolution has done a good job of designing perceptual learning processes, then it should be the case that naturally occurring changes to perception will often mimic the changes that would have been produced had we strategically orchestrated the changes. Thus, for the case of developing perceptual routines to help us solve formal math problems, we may consciously be training our perceptual systems to ‘do the right thing’, or we may just be relying on automatic processes of fluency, habit, and trainable perceptual grouping to help us. General-purpose neural processes of reinforcement learning, associative learning, and hierarchical action chunking help us to perform increasingly sophisticated activities by shunting activities that originally required strategic planning and central executive processes over to fast perception-action processes. Whereas some philosophers have argued that a hallmark of an advanced science is that it no longer requires notions of perceptual resemblance as the basis for its categories (Quine, 1977), the argument from perceptual learning is that our perceptual systems are not doomed to use untutored and

X *4 X+4 may resemble but to 3*4 3+ 4 the perceptually trained student of math, only the 4s in the second case look like they can be cancelled. At first sight, the marsupial Tasmanian wolf may resemble its quite distant evolutionary cousin, the placental grey wolf, but to a trained biologist, their teeth, jaws, ears, and nasal cavities are quite obviously distinct. If one wishes to become cognitively more sophisticated, an alternative to trumping perception is to train one’s perceptual systems instead. potential misleading perceptual systems. At first sight,

Acknowledgements The authors wish to express thanks to David Landy, Mohan Matthen, Susanna Siegel, and Linda Smith for helpful suggestions on this work. This work was funded by National Science Foundation REESE grant 0910218 and Department of Education IES grant R305A1100060. More information about the laboratory can be found at . Correspondence concerning this article should be addressed to Robert Goldstone, Psychological and Brain Sciences, Indiana University, Bloomington, IN, 47405. Email: [email protected].

References Ahissar, M. and Hochstein, S. (1997). ‘Task difficulty and the specificity of perceptual learning’. Nature, 387, 401–6. Austerweil, J. and Griffiths, T. L. (2011). ‘A rational model of the effects of distributional information of feature learning’. Cognitive Psychology, 63, 173–209.

Perceptual Learning   829 Bao, M., Yang, L., Rios, C., and Engel, S. A. (2010). ‘Perceptual learning increases the strength of the earliest signals in visual cortex’. Journal of Neuroscience, 30, 15080–4. Barsalou, L. W. (1999). ‘Perceptual symbol systems’. Behavioral and Brain Science, 22, 577–660. Behrmann, M., Zemel, R. S., and Mozer, M. C. (1998). ‘Object-based attention and occlusion: Evidence from normal participants and a computational model’. Journal of Experimental Psychology: Human Perception and Performance, 24, 1011–36. Blakemore, C. and Cooper, G. F. (1970). ‘Development of the brain depends on the visual environment’. Nature, 228, 477–8. Burns, B. and Shepp, B. E. (1988). ‘Dimensional interactions and the structure of psychological space: The representation of hue, saturation, and brightness’. Perception and Psychophysics, 43, 494–507. Carey, S. and Diamond, R. (1977). ‘From piecemeal to configurational representation of faces’. Science, 195(4275), 312–14. Cattell, J. M. (1886). ‘The time it takes to see and name objects’. Mind, 11, 63–65. Chen, L. and Zhou, X. (2011). ‘Visual apparent motion can be modulated by task-irrelevant lexical information’. Attention, Perception, & Psychophysics, 73, 1010–5. Churchland, P. (1989). ‘The anti-realist epistemology of Van-Fraassen’s The Scientific Image’. In B. A. Brody and R. E. Grandy (eds), Readings in the Philosophy of Science, Englewood Cliffs, NJ: Prentice Hall. Crick, F. and Koch, C. (1995). ‘Are we aware of neural activity in primary visual cortex?’ Nature 375, 121–3. Crist, R. E., Li, W., and Gilbert, C. D. (2001). ‘Learning to see: experience and attention in primary visual cortex’. Nature Neuroscience, 4, 519–25. Fahle, M. (1994). ‘Human pattern recognition: Parallel processing and perceptual learning’. Perception, 23, 411–27. Fodor, J. (1984). ‘Observation reconsidered’. Philosophy of Science, 51, 23–43. Furmanski, C. S., Schluppeck, D., and Engel, S. A. (2004). ‘Learning strengthens the response of primary visual cortex to simple patterns’. Current Biology, 14, 573–8. Garner, W. R. (1976). ‘Interaction of stimulus dimensions in concept and choice processes’. Cognitive Psychology, 8, 98–123. Gauthier, I. and Bukach, C. (2007). ‘Should we reject the expertise hypothesis?’ Cognition, 103, 322–30. Gauthier, I. and Tarr, M. J. (1997). ‘Becoming a “Greeble” expert: exploring mechanisms for face recognition.’ Vision Research, 37.12, 1673–82. Gauthier, I. Tarr, M. J., and Bubb, D. (eds) (2010). Perceptual Expertise: Bridging brain and behavior. Oxford: Oxford University Press. Gauthier, I., Williams, P., Tarr, M. J., and Tanaka, J. (1998). ‘Training "greeble" experts: a framework for studying expert object recognition processes.’ Vision research, 38(15), 2401–28. Ghahramani, Z. (1995). Factorial learning and the EM algorithm. In G. Tesauro, D. S. Touretzky, and T. K. Leen (eds), Advances in Neural Information Processing Systems 7 (pp. 617–24). Cambridge, MA: MIT Press. Gibson, E. J. (1969). Principles of Perceptual Learning and Development. East Norwalk, CT: Appleton-Century-Crofts. Gibson, J. J. (1979). The Ecological Theory of Perception. Boston: Haughton and Mifflin. Gillebert, C. R., Op de Beeck, H. P., Panis, S., and Wagemans, J. (2008). ‘Subordinate categorization enhances the neural selectivity in human object-selective cortex for fine shape discriminations’. Journal of Cognitive Neuroscience, 21, 1054–64.

830    Robert L. Goldstone and Lisa A. Byrge Goldstone, R. L. (1994). ‘Influences of categorization on perceptual discrimination’. Journal of Experimental Psychology: General, 123, 178–200. Goldstone, R. L. (2003). ‘Learning to perceive while perceiving to learn’. In R. Kimchi, M. Behrmann, and C. Olson (eds), Perceptual Organization in Vision: Behavioral and Neural Perspectives (pp. 233–78). Mahwah, NJ: Lawrence Erlbaum Associates. Goldstone, R. and Barsalou, L. W. (1998). ‘Reuniting cognition and perception: The perceptual bases of rules and similarity’. Cognition, 65, 231–62. Goldstone, R. L. and Hendrickson, A. T. (2010). Categorical Perception. Interdisciplinary Reviews: Cognitive Science, 1, 65–78. Goldstone, R. L. and Steyvers, M. (2001). ‘The sensitization and differentiation of dimensions during category learning’. Journal of Experimental Psychology: General, 130, 116–39. Goldstone, R. L., Landy, D. H., and Son, J. Y. (2010). ‘The education of perception’. Topics in Cognitive Science, 2, 265–84. Huang, X., Lu, H., Zhou, Y., and Liu, Z. (2011). ‘General and specific perceptual learning in radial speed discrimination’. Journal of Vision, 11, 1–11. Kourtzi, Z. and DiCarlo, J. J. (2006). ‘Learning and neural plasticity in visual object recognition’. Current Opinion in Neurobiology, 16, 152–8. Landy, D. and Goldstone, R. L. (2007). ‘How abstract is symbolic thought?’ Journal of Experimental Psychology: Learning, Memory, & Cognition, 33, 720–33. Lane, H. (1964). ‘Motor theory of speech perception: A critical review’. Psychological Review, 72, 275–309. Law, C. T. and Gold, J. I. (2009). A reinforcement learning rule can account for both the associative and perceptual learning on a visual discrimination task, Nature Neuroscience, 12, 655–63. Levin, D. T. (2000). Race as a visual feature: Using visual search and perceptual discrimination tasks to understand face categories and the cross-race recognition deficit. Journal of Experimental Psychology: General, 129, 559–74. Li, W., Piëch, V., and Gilbert, C. D. (2004). Perceptual learning and top-down influences in primary visual cortex. Nature Neuroscience, 7, 651–57. Liberman, A. M., Harris, K. S., Hoffman, H. S., and Griffith, B. C., (1957). The discrimination of speech sounds within and across phoneme boundaries. Journal of experimental psychology, 54(5), 358–68. MacPherson, F. (2012). ‘Cognitive penetration of colour experience: Rethinking the issue in light of an indirect mechanism’. Philosophy and Phenomenological Research, 84, 24–62. MacPherson, F. (2012). ‘Cognitive penetration of colour experience: Rethinking the issue in light of an indirect mechanism’. Philosophy and Phenomenological Research, 84, 24–62. Moscovici, S. and Personnaz, B. (1991). ‘Studies in social influence: VI. Is Lenin orange or red? Imagery and social influence’. European Journal of Social Psychology, 21, 101–18. Needham, A., Dueker, G.L., and Lockhead, G. (2005). ‘Infants’ formation and use of categories to segregate objects’. Cognition, 94, 215–40. Nosofsky, R. M. (1986). ‘Attention, similarity, and the identification–categorization relationship’. Journal of Experimental Psychology: General, 115, 39–57. Olshausen, B. A. and Field, D. J. (1996). ‘Emergence of simple-cell receptive field properties by learning a sparse code for natural images’. Nature, 381, 607–9. Peterson, M. A. and Gibson, B. S. (1994). ‘Must figure–ground organization precede object recognition? An assumption in peril’. Psychological Science, 5, 253–9.

Perceptual Learning   831 Peterson, M. A. and Lampignano, D. W. (2003). ‘Implicit memory of novel figure–ground displays includes a history of cross-border competition’. Journal of Experimental Psychology: Human Perception and Performance, 29, 808–22. Puel, J. L., Bonfils, P., and Pujol, R. (1988). ‘Selective attention modifies the active micromechanical properties of the cochlea’. Brain Research, 447, 380–3. Pylyshyn, Z. W. (1999). ‘Is vision continuous with cognition? The case for cognitive impenetrability of visual perception’. Behavioral and Brain Sciences, 22, 341–423. Pylyshyn, Z. (2003). Seeing and Visualizing: It’s not what you Think. Cambridge, MA: The MIT Press. Quine, W. V. (1977). ‘Natural kinds’. In W. V. Quine (ed.), Ontological Relativity and Other Essays (pp. 114–138). New York: Columbia University Press. Quinn, P. C., and Bhatt, R. S. (2006). ‘Are some Gestalt principles deployed more readily than others during early development? The case of lightness versus form similarity’. Journal of Experimental Psychology: Human Perception and Performance, 32, 1221–30. Raftopoulos, A. (2001). ‘Perceptual learning meets philosophy: cognitive penetrability of perception and its philosophical implications’. In J. D. Moore and K. Stenning (eds), Proceedings of the 23nd Annual Conference of the Cognitive Science Society (pp. 802–8). Mahwah, NJ: Lawrence Erlbaum. Repp, B. H. (1984). ‘Categorical perception: Issues, methods, findings’. Speech and Language: Advances in basic research and practice, 10, 243–335. Rumelhart, D. E. and Zipser, D. (1985). ‘Feature discovery by competitive learning’. Cognitive Science, 9, 75–112. Salapatek, P. and Kessen, W. (1973). ‘Prolonged investigation of a plane geometric triangle by the human newborn’. Journal of Experimental Child Psychology, 15, 22–9. Schyns, P. G., Goldstone, R. L., and Thibaut, J. P. (1998). ‘The development of features in object concepts’. Behavioral and Brain Sciences, 21, 1–54. Sekuler, A. B., Palmer, S. E., and Flynn, C. (1994). ‘Local and global processes in visual completion’. Psychological Science, 5, 260–7. Shiffrin, R. M. and Lightfoot, N. (1997). ‘Perceptual learning of alphanumeric-like characters’. In R. L. Goldstone, P. G. Schyns, and D. L. Medin (eds), The Psychology of Learning and Motivation, Volume 36 (pp. 45–82). San Diego: Academic Press. Shiu, L. P. and Pashler, H. (1992). ‘Improvement in line orientation discrimination is retinally local but dependent on cognitive set’. Perception & Psychophysics, 52, 582–8. Smith, L. B. and Kemler, D. G. (1978). ‘Levels of experienced dimensionality in children and adults’. Cognitive Psychology, 10, 502–32. Sowden, P. T., Davies, I. R. L., and Roling, P. (2000). ‘Perceptual learning of the detection of features in X-ray images: A functional role for improvements in adults’ visual sensitivity?’ Journal of Experimental Psychology: Human Perception and Performance, 26, 379–90. Spector, F. and Maurer, D. (2009). ‘Synesthesia: A new approach to understanding the development of perception’. Developmental Psychology, 45, 175–89. Swoyer, C. (2003). ‘Relativism and the Constructive Aspects of Perception, Supplement to Relativism’. In E. Zalta, (ed.), The Stanford Encyclopedia of Philosophy. Retrieved April 2011 from . Tanaka, J. W. and Curran, T. (2001). ‘A neural basis for expert object recognition’. Psychological Science, 12, 43–7. Tanaka, J. W. and Farah, M. J. (1993). ‘Parts and wholes in face recognition’. The Quarterly Journal of Experimental Psychology A: Human Experimental Psychology, 46A, 225–45.

832    Robert L. Goldstone and Lisa A. Byrge Watanabe, T., Náñez, J. E., and Sasaki, Y. (2001). ‘Perceptual learning without perception’. Nature, 413, 844–8. Weinberger, N. M. (1993). ‘Learning-induced changes of auditory receptive fields’. Current Opinion in Neurobiology, 3, 570–7. Wheeler, D. D. (1970). ‘Processes in word recognition’. Cognitive Psychology, 1, 59–85. Xiao, L.-Q., Zhang, J.-Y., Wang, R., Klein, S. A., Levi, D. M., and Yu, C. (2008). ‘Complete transfer of perceptual learning across retinal locations enabled by double training’. Current Biology, 18, 1922–6. Yang, T. and Maunsell, J. H. (2004). ‘The effect of perceptual learning on neuronal responses in monkey visual area V4’. Journal of Neuroscience, 24, 1617–26. Zemel, R., Behrmann, M., Mozer, M. C., and Bavelier, D. (2002). ‘Experience-dependent perceptual grouping and object-based attention’. Journal of Experimental Psychology: Human Perception and Performance, 28, 202–17.

Chapter 43

Perception a n d Demonstr ati v es Imogen Dickie

This chapter is about the relationship between perception and perceptual demonstrative thought. A ‘perceptual demonstrative thought’ is a thought standardly made available by a perceptual link with an object in the external world, and standardly expressed using the demonstratives ‘this’ or ‘that’. For example, the thought you express when you say ‘That is rolling’, looking at an orange rolling along the table in front of you in an ordinary situation, is a central case of perceptual demonstrative thought. The class of perceptual demonstrative thoughts includes this kind of central case and other thoughts relevantly similar to it. (What counts as a ‘relevant’ similarity will emerge as we go on.) The definition of ‘perceptual demonstrative thought’ supposes that, in central cases, your perceptual link with an external-world object does enable you to think about it. (This sets aside various extreme views from the history of discussion of the relationship between perception and thought. For example it sets aside ‘idealist’ views, according to which perception enables us to think only about mind-dependent entities.1 And it sets aside ‘inscrutability’ views, according to which there is no fact of the matter as to which object a thought is about, or whether it is about an object at all.2) But to insist that, in this kind of case, a perceptual link does enable thought about an external-world object is to say nothing about how it does. This question—the question of how a perceptual link with an object enables perceptual demonstrative thought about it—is the fundamental question that an account of the relation between perception and perceptual demonstrative thought must address. To keep to handbook-entry scale, I shall restrict the discussion to the question as it arises for visual perception: How does a visual perceptual link with an object enable perceptual demonstrative thought about it? (The philosophical literature on perceptual demonstrative 1  See relevant entries in this volume—Simmons, Chapter 4, on early modern philosophy, and Snowdon, Chapter 6, on sense-data. 2  Views of this kind are an extension of Quine’s thesis of the inscrutability of reference. The canonical source for Quine’s original arguments for this thesis is Quine, 1960: ch 2. Quine gives a more accessible introduction in Quine, 1969. For a discussion of the inscrutability challenge as it arises for the case of perceptual demonstrative thought, see Campbell, 2002: 216–34.

834   Imogen Dickie thought has developed around vision.3 There are hard questions about how closely perceptual demonstrative thought in other modalities parallels the visual case.) The chapter has three sections. Section 1 elucidates the fundamental question by sketching some of the traditional and contemporary puzzles that accompany it. Section 2 lays out the contrast between ‘acquaintance theoretic’ and ‘description theoretic’ responses to the fundamental question and the associated puzzles. Section 3 is about how a traditional view of the nature of perception has skewed the acquaintance vs. description discussion. Section 4 shows how recent advances in our understanding of perception—advances away from the traditional view—shift the ground on which the debate about perception and demonstratives must take place.

1  Some puzzles about perceptual demonstrative thought Recall the example of perceptual demonstrative thought with which we began. You are looking at an orange rolling along a table in front of you in a situation devoid of perceptual or cognitive perversity—no mirrors, distorting lenses, or unusual lighting; no worries about whether you are a brain in a vat—and thinking the thought you would express by saying ‘That is rolling’. We are supposing that in this case your perceptual link with the orange does enable you to think about it. The fundamental question for an account of the relation between perception and demonstratives is the question of how your perceptual link is doing its aboutness-fixing and thought-enabling work. But many philosophers have thought it best to approach the fundamental question against the background of a range of puzzles or subsidiary questions which bring out the kinds of commitment that an answer to the central question might carry. This section sketches seven of these puzzles.

First puzzle—directness4 The first puzzle concerns how direct in space, time, and causal pathway a perceptual link must be if it is to enable perceptual demonstrative thought. Let us say that a ‘perceptual link’ with an object is a perceptual information feed whose deliverances are sensitive to the object’s properties. Then all of the following count as cases where subject S has a perceptual link with object o: (1) a central case situation of the kind we started with (S is looking at o as o rolls along a table a couple of feet away). (2) S is looking at o in a mirror.

3  There are exceptions. See in particular Evans, 1982: ch. 6 (pp. 143–91). For discussion of perception in nonvisual modalities, see chapters 15–18 in Part III of this volume. For discussion of multisensory perception, see Bayne and Spence, Chapter 32, this volume. 4  For discussions of this puzzle, see Evans, 1982: 144, 149–50; Campbell, 2002: 111.

Perception and Demonstratives    835 (3) S is looking at o’s shadow. (4) S is looking at o on live television. (5) S is looking at a now long-dead person on film; the film shows the person still in her prime (6) S is looking at a bright, reddish, star which in fact ceased to exist 500 years ago, but is still visible to S because it was 1,000 light years away. (7) S is looking at some o* many of whose properties depend causally on o’s (for example, o’s footprint in the sand). (8) S is looking at some o* only a few of whose properties depend causally on o’s (for example, S might be looking at o’s car, inexpertly parked by o, and now festooned with parking tickets, and thinking ‘He’ll be sorry’). Let us say that in (1)—the central case situation—S’s perceptual link with o is ‘maximally direct’: it is as direct in space, time, and causal pathway as our perceptual links get. Then (2)–(8) describe various kinds and degrees of progression away from maximal directness. Most participants in the debate about perception and demonstratives agree that the perceptual links at (7) and (8) cannot sustain perceptual demonstrative thought. These perceptual links are too indirect to put S in a position to think about o in the same way as the perceptual link at (1) does (they enable S to think about o, but only under the descriptions ‘the maker of this footprint’ or ‘the driver of that car’). But (2)–(6) describe a continuum of situations in which S’s perceptual link with o is less and less direct. Where along this continuum does a perceptual link with an object become too indirect to enable perceptual demonstrative thought about it?

Second puzzle—comprehensiveness5 The second puzzle is about how much of a thing a perceptual link must put you in contact with to enable perceptual demonstrative thought about it. On the face of things, it seems that a central-case perceptual link does not put you in touch with an entire ordinary object. Ordinary objects extend in three dimensions. But your eyes pick up only light reflected by the surfaces of the object that face you. So (it seems) there are reasonable grounds for saying that there are parts of the object (its interior; its back surfaces) with which your perceptual link is not putting you in contact.6 But it is also plausible that the thought you express when you say ‘That is rolling’ in the central case we have described is about the whole orange, not just the part of it that faces you. So it seems that in central-case perceptual demonstrative thought, perceptual contact with part of a thing enables perceptual demonstrative thought about all of it. And in that case we should be able to say how much of a thing a perceptual link must put you in contact with—how much of the thing the perceptual link must comprehend—if it is to sustain perceptual demonstrative thought. Can S have a perceptual demonstrative thought about a whole fence-post on the basis of a 5  The contemporary discussion of this puzzle starts with Moore, 1918 and 1962: 177. For an influential discussion, see Evans, 1982: 144–45. 6  There is a debate about whether this claim should be accepted. See Crane, 2011 for an introduction and references.

836   Imogen Dickie perceptual link that comprehends only the part of the post that sticks up above the ground? Does a perceptual link that comprehends only the tip of a fog-bound peninsula put you in a position to have a perceptual demonstrative thought about a whole continent? Note that this is another continuum puzzle. There is a continuum running from the central case to something like the peninsula/continent case. We should be able to say how far along this continuum the class of perceptual links that can sustain perceptual demonstrative thought extends.

Third puzzle—classification The third puzzle concerns the relation between perceptual demonstrative reference and sortal classification. In intuitive terms, ‘sortal’ classification is classification of objects according to the kinds of objects they are. For example, ‘X is an animal’, ‘X is a tree’, and ‘X is an item of furniture’ are all statements of sortal classification because ‘animal’, ‘tree’, and ‘item of furniture’ are all kinds in some intuitive sense of ‘kind’. ‘X is red’ is not a statement of sortal classification because it does not carry genuine kind information: to tell you that X is red is not yet to tell you anything about how X might have come into existence; how it might change over time; or the kind of event that would destroy it. (There are very hard questions about how the intuitive notion of ‘sortal’ classification is to be made precise.7) The puzzle about perceptual demonstrative thought and sortal classification arises because there are apparently strong grounds for both the (‘sortalist’) claim that a perceptual link enables perceptual demonstrative thought only if it enables sortal classification and the competing (‘anti-sortalist’) claim that a perceptual link can enable perceptual demonstrative thought about an object without even in principle putting the subject in a position to know what kind of thing it is. One argument for sortalism comes from the old contention that any scene perception might present can be divided up into objects in different ways, and it is only by supplying a sortal concept that you can isolate one of the objects in a scene as the thing you are going to go on and think about. For example, looking at a stand of trees you might think each of ‘Here is a copse’, ‘Here are five trees’, or ‘Here are thousands of leaves, dozens of branches, and five trunks’. According to the sortalist, your perceptual feed alone cannot do the job of determining whether you are thinking about the copse, the trees, or the leaves and branches. Rather, you must supply a sortal concept (‘copse’, ‘tree’, ‘branch’) to determine which of the things in the perceived scene is the object of your perceptual demonstrative thought.8 The main grounds for anti-sortalism come from apparent counterexamples to sortalism: cases where a perceptual link seems to enable perceptual demonstrative thought without enabling sortal classification. Here are some examples of this kind.9 CASE 1—You stumble upon a thing of a kind you have not come across before. You have no idea where it might fit in your system of sortal classification—whether it is animal, 7  Grandy, 2008 provides an informative introduction and references. The contemporary debate on this question traces to Frege, 1953: §53 (pp. 65–7). There is a related debate about whether sortal properties can be presented in perception. For a ‘yes’ answer, see Siegel, 2011. 8  For sortalist views, see Frege, 1953: esp. 28, 59, 62, 66; Quine, 1953; Wiggins, 2001; Evans, 1982: esp. 105–12, 178–9; Dummett, 1973: 73–80, 179–80. There are important differences in motivation between these varieties of sortalism. 9  For examples like this in the anti-sortalist literature, see Ayers, 1974, 1997; Campbell, 2006.

Perception and Demonstratives    837 vegetable, or mineral; whether it is an artefact or something that came to exist without human intervention. But surely (it seems) your perceptual link is enabling you to think about it (there it is, sitting on your desk, and you are thinking ‘I wonder what this thing is’). CASE 2—You think (looking at o and noting o’s tallness-for-a-man) ‘That is a tall man’. In fact, o is a waxwork. But surely (it seems) your perceptual link is securing o as the object your thought is about. CASE 3—You can see a thing moving around in the distance. It is too far away for you to tell what kind of thing it is. But surely (it seems) you can think about it. (You and your friend might be having a dispute about whether it is a bird, a plane, or Superman.) The standard sortalist response to cases like these10 is to say that perceptual demonstrative thought does not require a right sortal classification of the object the thought is about. A perceptual link that enables perceptual demonstrative thought need only enable the subject to make an approximately right classification. Or, alternatively, it need only supply the subject with the in principle capacity to home in on a right classification. But anyone making this move faces difficult questions. How is the notion of ‘approximating to’ or ‘homing in on’ a right sortal classification to be made precise? Does the move work against all of the examples the anti-sortalist might raise? Does the weakened sortalist view respect the arguments that motivate sortalism in the first place? A right account of how perception enables perceptual demonstrative thought should enable us to settle the sortalist/ anti-sortalist dispute.

Fourth puzzle—category11 The fourth puzzle concerns the category of particular about which perceptual demonstrative thought is possible. In the central-case example we started with, the perceptual demonstrative thought is about an ordinary material thing. But does the class of things about which we can have this kind of thought include non-ordinary physical things like ripples, clouds, and shadows? Does it include mereological gryphons built from parts of distinct material objects (so that a suitable perceptual link could enable me to have ‘that’ thoughts about the mereological sum of my dog’s left ear and a tree in the middle distance)? What about places, times, and events—can a perceptual feed enable a subject to think about these particulars in the same way as it enables us to think about ordinary material things in central cases?

Fifth puzzle—focus12 The fifth puzzle is (roughly) the puzzle of whether there can be perceptual demonstrative thought on the basis of merely peripheral vision (strictly speaking this is a puzzle 10  See Wiggins, 2001: 7, 55–61; Evans, 1982: 178–9. For a discussion of the viability of the standard move, see Dickie, 2011b: §4. An alternative move is to say that the reference requires being right about sortal classification, but that the only referentially relevant sortal is something very general like ‘physical object’. See Xu, 1997 for a psychologist making this proposal; Wiggins, 1997 and Ayers, 1997 for sortalist and antisortalist replies. 11  For discussion of this puzzle, see Dickie, 2011a: 315–8. 12  The most thorough discussion of this puzzle to date is in Campbell, 2002.

838   Imogen Dickie about unattended peripheral vision).13 There is significant intuitive appeal to the claim that peripheral vision cannot sustain perceptual demonstrative thought. For (it seems), a perceptual link that can sustain perceptual demonstrative thought about o must at least present o as distinct from the objects around it. And objects with which we have peripheral vision are not presented as sharply distinct from one another. (Consider how you will do if you try to describe what you can see in peripheral vision at the same time as you read these words. Your peripheral perceptual feed will let you do quite well at statements of gross macroscopic property instantiation: ‘There is something pale coloured over there’; ‘There is something with sharp edges’; ‘There is something moving’. And you will be better than chance at putting gross macroscopic properties together (‘The sharp-edged thing is moving’; ‘The pale thing is curved’). But, if you are relying on peripheral vision alone, you will also make many mistakes about which properties go together as properties of the same object.14 And any fine detail you add to your account will have to come from either guesswork or prior knowledge.) However, we need a more-than-just-intuitive story about why peripheral vision cannot sustain perceptual demonstrative thought. And the case of peripheral vision also raises yet another continuum problem. On the face of things, there are degrees of perceptual focus and peripherality.15 What degree of centrality to perceptual focus does perceptual demonstrative thought require?

Sixth puzzle—awareness16 The sixth puzzle is about the relation between perceptual demonstrative thought and conscious awareness. It is a widespread (though comparatively recent) observation that the perceptual system can generate a quite rich information feed even though the subject has no awareness of the object from which the information is derived. For example, subjects with a specific type of brain damage can give, on the basis of visual contact alone, very reliable forced-choice answers to questions about the properties of objects they are not aware of seeing. (These are ‘blindsight’ patients. Blindsight is commonly explained as involving information delivery without consciousness.17) There are also non-pathological cases of this kind. These are cases of ‘subliminal’ information delivery, where perceptual contact with a thing is too brief to yield awareness, but the fact that the subject has received information from the brief perceptual contact shows up in other aspects of his or her behaviour. (For example, subliminal

13 

For an introduction to discussion of perceptual attention see Campbell, Chapter 31, this volume. an introduction to empirical results about the relationship between perceptual attention and reliable ‘binding’ of features as belonging to the same thing, see Palmer, 1999: 556–563. 15  More accurately, there are ‘degrees of perceptual attention’. See Campbell, Chapter 31, this volume, on attention. 16  This puzzle was brought into the debate on perception and demonstratives by Campbell (2002). For subsequent discussions, see Campbell, 2011; Dickie, 2011a; Smithies, 2011. The puzzle is connected with general issues about the relationship between conscious awareness and intentionality of the kind most famously raised by Searle (1980). 17  See Prinz, Chapter 19, this volume. 14  For

Perception and Demonstratives    839 exposure to a word or to a familiar face has measureable impact how the subject behaves next.)18 But the possibility of perceptual information delivery without perceptual awareness raises immediate questions that a right account of how perception enables perceptual demonstrative thought should enable us to answer. In central cases, the perceptual link that enables perceptual demonstrative thought delivers both information and awareness. But is the ‘awareness’ part playing a necessary aboutness-securing role, or could there be cases (for us, or for subjects like us, or perhaps for subjects radically different from us) where perceptual demonstrative thought is enabled by a perceptual link that delivers information alone?

Seventh puzzle—emptiness The seventh puzzle concerns what to say about cases which seem, at first sight, to involve perceptual demonstrative aboutness failure. Here are two cases of this kind: CASE 4—Your experience is indistinguishable by you from the experience you would have if you were looking at an orange rolling along the table in front of you. You form the belief you would express by saying ‘That is an orange’. In fact you are hallucinating—there is no orange there for you to experience. CASE 5—It seems to you that you are looking at a rectangular thing in the middle distance. You form the belief you would express by saying ‘That is rectangular’. In fact, your experience as of a rectangular thing in the middle distance is caused by a freak combination of a speck on your glasses and a tree on the horizon. In each of these cases, you have an experience indistinguishable by you from the experience you would have in a case where a perceptual link with an object does sustain perceptual demonstrative thought. In each case you form a thought you would express using ‘that’, and which is indistinguishable by you from a perceptual demonstrative thought. And in each case it seems plausible that there is nothing your thought is about. But does it follow that there can be perceptual demonstrative thoughts about nothing? What are the implications of a decision on this issue for the question of how perceptual demonstrative aboutness-fixing works?

2  Descriptivist theories vs. Acquaintance theories Accounts of how perception enables perceptual demonstrative thought divide into descriptivist theories and acquaintance theories. This section introduces the two types of view, and consolidates the contrast between them by saying a little about the responses to the puzzles from section 1 that each generates. 18 

See Dehaene et al., 2006.

840   Imogen Dickie

Descriptivist theories According to a ‘descriptivist theory’ of perceptual demonstrative thought, a perceptual link with o enables perceptual demonstrative thought about o by putting the subject in a position to grasp a descriptive condition that o satisfies. (In general, a ‘descriptivist theory’ of thoughts of a kind is a theory according to which being in a position to have a thought of the kind about o involves grasping, or being in a position to come to grasp, a descriptive condition that o satisfies. Descriptivism about perceptual demonstrative thought is a special case of this kind of view. A  descriptivist about perceptual demonstratives might be motivated either by ‘global descriptivism’—the view that all aboutness-fixing is descriptive—or by specific arguments for descriptivism in the perceptual demonstrative case.) Different descriptivist views are generated by different accounts of the aboutness-fixing description. For example, philosophers have upheld each of the following (this is not an exhaustive list). Traditional descriptivism about perceptual demonstrative thought—In a central case, S’s perceptual link enables perceptual demonstrative thought about an object by providing S with a specification of macroscopic observable properties. The specification will be of form ‘the thing of shape F1, colour F2, position F3, degree of motion F4, size F5 . . .’ (with the list completed by specifications of other kinds of property, and no kind of property having to appear on the list). ‘That’-beliefs formed on the basis of the perceptual link are about o iff o is the unique object that has all (or most of the most important) properties in the specification.19 Sortal descriptivism about perceptual demonstrative thought—In a central case, S’s perceptual link with o enables perceptual demonstrative thought about o by putting S in a position to grasp a descriptive condition of form ‘the thing of sort σ at position p’ which o satisfies.20 Reflexive descriptivism about perceptual demonstrative thought—In a central case, S’s perceptual link with o enables perceptual demonstrative thought about o by providing S with grasp of a descriptive condition which o satisfies, where this condition mentions either the perceptual link or the experience it delivers, for example, the condition might be ‘the object at the end of this perceptual link’ or ‘the object causing this experience’.21 One way to bring out the important similarities and differences between these descriptivist views is in terms of the responses they generate to the puzzles from section 1. Let us start with the emptiness puzzle. According to all three descriptivist views, thinking a perceptual demonstrative thought involves (actual or potential) grasp of an aboutness-fixing description, and the thought is about o iff o is the description’s satisfier. But it takes some squirming to formulate a descriptive condition grasp of which entails the

19  For an approximation to this kind of view, see Russell, 1912: ch. 2, pp. 46–59. Kaplan’s 1969 notion of a ‘vivid’ name combines traditional descriptivism with a causal link requirement. 20  Dummett is a sortal descriptivist about perceptual demonstratives. See for example his 1973: 233. Evans’s view in his 1982, ch. 6, is that a perceptual link with an object enables perceptual demonstrative reference to it iff the link gives the subject the ‘practical and in principle’ capacity to locate an object and discover its kind. Evans denies that this is a descriptivist view. But it is a form of ‘sortal descriptivism’ in the sense defined here. 21  Compare Searle, 1983: 225–7.

Perception and Demonstratives    841 existence of an object that satisfies it. So the most obvious forms of descriptivism about perceptual demonstrative thought (including the three I have sketched) all entail that a perceptual feed might enable grasp of an aboutness-fixing description with no satisfier. For descriptivist views that have this consequence, there is at least an intuitive sense in which a perceptual demonstrative thought may be ‘available’ even though there is nothing it is about. (There are still hard questions about whether the thoughts a subject forms in cases like Case 4 and Case 5 are false (the alternative is to say that they have no truth values) but addressing these questions requires engagement with issues about the relationship between aboutness-fixing and truth-conditions that I cannot discuss here.22) I shall give a brief indication of how the other puzzles come out under each descriptivist view in turn. A traditional descriptivist’s response to the puzzles about directness and comprehensiveness will be to say that a perceptual link is direct and comprehensive enough to sustain perceptual demonstrative reference as long as it provides the subject with a (sufficiently rich) range of observable macroscopic property information that the object matches. Traditional descriptivism also generates a solution to the puzzle about category: there can be perceptual demonstrative thought about any particular for which a perceptual link can deliver grasp of the traditional descriptivist’s favoured model of aboutness-fixing description. So if perceptual links with times, places, and mereological gryphons (‘things’ that are the mereological sums of parts of distinct objects) can deliver grasp of this kind of description, there can be perceptual demonstrative thought about them. Similarly, a traditional descriptivist’s response to the puzzle about focus will be to say that peripheral vision does not sustain perceptual demonstrative thought because a peripheral perceptual link with o does not enable the subject to formulate a sufficiently rich descriptive condition. With respect to the puzzle about classification, traditional descriptivism is an anti-sortalist view—a traditional descriptivist maintains that aboutness for a perceptual demonstrative thought can be determined just by a suitably wide match between the subject’s beliefs and the object’s properties, with no one property of the object’s having to feature in the list of its properties that the subject gets right.. Finally, with respect to the awareness puzzle, a traditional descriptivist’s view will depend on whether he or she thinks a perceptual link can provide grasp of a suitably rich description without also delivering awareness. In contrast, a sortal descriptivist will say that a perceptual link with o is direct and comprehensive enough to sustain perceptual demonstrative thought about o iff it puts the subject in a position to locate o and discover its kind. So, for example, the sortal descriptivist has a reason to say that there can be demonstrative reference to things seen in mirrors but not to things seen on television: a mirror-mediated perceptual link with a thing does, while a television-mediated link does not put you in a position (without any information not transmitted through the link) to discover the object’s location.23 And for the case of comprehension, a sortal descriptivist will say that a perceptual link that might involve immediate contact with only the facing surface of a three-dimensional object puts you in a position to refer to the whole object because it is the whole object that your sortal concept classifies.

22  23 

For two important first moves in this debate, see Strawson, 1950 and Dummett, 1978. Evans argues like this at 1982: 147–50. See also Dennett, 1978.

842   Imogen Dickie The sortal descriptivist’s responses to the puzzles of focus and awareness will depend on his or her view of whether an unfocused (non-attentional) perceptual link, on the one hand, and a perceptual link that delivers information without awareness, on the other, can enable sortal classification. On the face of things, it is implausible that a merely peripheral perceptual link with an object can deliver precise or detailed enough information to enable sortal classification. So the sortal descriptivist has grounds to deny that an unfocused perceptual link with an object can put a subject in a postion to have perceptual demonstrative thoughts about it. The question of whether a perceptual link that does not deliver awareness can enable sortal classification is more complex, so let us rest with noting that the answer to this question will determine the sortal descriptivists’s response to the awareness puzzle. Finally, there can be no surprises about the sortal descriptivist’s response to the puzzle about classification: perceptual demonstrative thought, on this view, does rest on the capacity for sortal classification. (What exactly a sortal descriptivist will say about the category puzzle depends on how his or her sortalism is motivated. This is too complicated an issue to dabble in here.) For a reflexive descriptivist, the solutions to the puzzles will depend on the nature of the proposed reflexive descriptive condition. For example, if the descriptive condition is ‘the object this perceptual link is focused on’, perceptual demonstrative thought on the basis of peripheral vision will be ruled out automatically because a peripheral perceptual link is not focused. In contrast, if the favoured descriptive condition is ‘the cause of this experience’, there seems to be no barrier to perceptual demonstrative thought on the basis of a merely peripheral perceptual link. Again, a reflexive descriptivist favouring the first kind of descriptive condition will say that an indirect perceptual link can sustain perceptual demonstrative thought iff it is focused (so that the burden in saying whether there can be perceptual demonstrative thought about an object seen in a mirror is shifted to the question of whether the perceptual link in this case is focused on the object or its reflection). In contrast, for a reflexive descriptivist favouring the second kind of descriptive condition, the burden will shift to the question of whether the object or its reflection is the cause of the experience. For the kinds of reflexive descriptivism I have mentioned so far, grasp of the descriptive condition requires that the perceptual link be delivering awareness. But a reflexive descriptivist might step back from this result by formulating a different aboutness-fixing description, for example, ‘the object that is causing me to hold these beliefs and respond in these ways’.

Acquaintance-theoretic views The notion of ‘acquaintance’ has both a negative and a positive characterization in the philosophical tradition: Negative characterization of ‘acquaintance’—S is ‘acquainted’ with o iff S is in a position to think about o in virtue of a perceptual link with o, and independently of actual or potential grasp of any descriptive condition that o satisfies. Positive characterization of ‘acquaintance’—S is ‘acquainted’ with o iff S is in a position to think about o because S has a perceptual link with o that enables S to have thoughts that are directly about o.

Perception and Demonstratives    843 One of the problems in developing an acquaintance-theoretic view is to explain how the notion of ‘directness’ that figures in the positive characterization is to be understood.24 An explanation that collapses the positive characterization into the negative characterization is to say that S is thinking ‘directly’ about o iff S has a thought about o whose aboutness is not secured by S’s actual or potential grasp of an aboutness-fixing description that o satisfies.25 Another possibility is to explain acquaintance-theoretic ‘directness’ in terms of the claim that acquaintance-based thoughts are formed in immediate response to perception. An acquaintance theorist making this move will say that your ‘That is rolling’ thought is formed in cognitively unmediated response to your perceptual information feed (it is formed by taking what this feed delivers at face value), and that the aboutness-fixing story about this kind of thought is a story about why a thought formed in this way should count as ‘about’ the thing at the end of the perceptual link that gives rise to it.26 A third possible move is to try to explain the relevant notion of directness in terms of the acquaintance theorist’s characteristic response to the emptiness puzzle. If (as the acquaintance theorist maintains) a central-case perceptual link does its thought-enabling work by establishing a direct ‘aboutness’ relation between the subject and the thing at the end of the perceptual link, if there is no thing at the end of the perceptual link there will be no thought enabling and (therefore) no thought. Philosophers trying to elucidate the acquaintance-theoretic ‘directness’ in this way have said that thoughts made available by acquaintance with objects are ‘object-dependent’ thoughts—thoughts which (in an intuitive sense of ‘available’) are not available if there are no objects they are about.27 (One interesting question for philosophers working on acquaintance-theoretic views is that of the extent to which these three accounts of ‘directness’ coincide.) In the absence of more detail about what an acquaintance-theoretic view might look like, we are not in a position to see how acquaintance theorists will respond to the other puzzles from section 1. I shall say more about this in section 4. Note that an acquaintance theorist should allow that a perceptual link with o that puts S in a position to think about o might also enable S to grasp a range of descriptive conditions that o satisfies, for example, ‘the object I am now thinking about’; ‘the object that is the cause of this experience’; or (in a central case, where the viewing conditions are good) ‘the object having such-and-such macroscopic properties’. The acquaintance theorist’s point is just that it is not by enabling grasp of these descriptive conditions that a perceptual link that enables perceptual demonstrative thought does its aboutness-fixing work.

24 

For an introduction to the history of attempts to answer this question, see Recanati, 2012: 3–14. See, for example, Russell, 1912: 46; Matthen, 2005: ch. 13. 26  This kind of view forms one strand in Evans’s account of the ‘directness’ of perceptual demonstrative aboutnesss-fixing. See for example Evans, 1982: 146 (but remember that Evans’s view also incorporates a descriptivist element – see n. 20). The proposals in Campbell, 2002 and Dickie, 2010, 2011a also involve this kind of view of the ‘directness’ of perceptual demonstrative thought. 27  For infuential discussions of this idea, see Evans, 1982: esp. 10–33; Evans, 1985; McDowell, 1984. 25 

844   Imogen Dickie

3  The traditional view of what cognitively unaided perception delivers, and how it skews the traditional debate about perception and demonstratives Let us return to our example of a central-case perceptual demonstrative thought. You are looking at an orange as it rolls along the table in front of you in a situation devoid of cognitive or perceptual funny-business. You form the belief you would express by saying ‘That is rolling’. The fundamental question for accounts of how perception enables perceptual demonstrative thought is the question of how your perceptual link with the object secures its status as the thing your belief is about. Any answer to this question will need three components: First component—an account of how belief formation on the basis of a perceptual link works. Second component—a general account of what it takes for a belief to be ‘about’ a particular thing. Third component—an explanation of how, given the first component, a central case of belief formation on the basis of a perceptual link with o meets the requirements for the belief to be about o entailed by the second component. The second and third components here are irredeemably philosophical. But at least part of what is required of the first component is an empirical story: a story about the causal pathway by which our perceptual systems deliver perceptual experiences to which our cognitive systems respond by forming beliefs. So an account of how perception enables perceptual demonstrative thought must incorporate an empirical element. The mainstream traditional debate about perception and demonstratives got its empirical element from a less than reputable source. Philosophers participating in this debate28 took it that the path to formation of central-case perceptual demonstrative beliefs involves introspectively discernible stages:  perception makes its contribution; then cognition goes to work on what perception delivers; the boundary between what is given in perception and what must be worked out by cognition is a boundary that can be discerned by each of us from within. And these philosophers claimed that what is delivered by the first (perception-only) stage of this process, though phenomenally rich, is structurally poor. The suggestion was that cognitively unaided perception delivers a succession of qualitative seemings. These seemings have spatial structure in that they fill the visual field: each point in the visual field has a colour; same-coloured adjacent points result in the appearance of a patchwork of coloured shapes. And the succession of qualitative seemings has temporal structure: coloured patchworks succeed one another. But this is all the structure 28  See

Simmons, Chapter 4, this volume, on early modern views of perception and Casati, Chapter 20, this volume, on object perception. For a recent account of the transition away from the traditional view see Matthen, 2005: ch. 1.

Perception and Demonstratives    845 that cognitively unaided perceptual experience has. On this traditional view, the kind of account that would come to mind most readily if you were asked to say what you see (‘It looks like an orange round thing is rolling from half-shade into full light’) does not capture what perception delivers. It captures the output of cognitive processing brought to bear on the raw, comparatively unstructured, perceptual feed. This traditional view has had an impact on the debate about perceptual demonstratives that is too various and pervasive to survey here. But I  think it is fair to say that its most important result has been the skewing of the debate about the viability of acquaintance-theoretic views. For the traditional view makes it very hard to see how an acquaintance-theoretic view could be made to work. To see why, consider the following three claims (A) Cognitively unaided perception delivers an information feed that is too unstructured to sustain the claim that our perceptual experiences ‘represent’ or are ‘of’ ordinary three-dimensional objects and their macroscopic observable properties. (Consequence of the traditional view) (B) We form perceptual demonstrative thoughts by taking what perception delivers at face value. (Acquaintance-theoretic commitment) (C) Central case perceptual demonstrative thoughts are about ordinary threedimensional material objects. (Consequence of the definition of ‘perceptual demonstrative thought’) (A), (B), and (C) are inconsistent. But (C) is a consequence of the definition of ‘perceptual demonstrative thought’. So if (A) is assumed, (B) will have to go. The inconsistency between (A), (B), and (C) brings out why the traditional view presents a barrier to acquaintance-theoretic views that explain ‘directness’ in terms of direct uptake of what is delivered by perception. Other ways of explaining acquaintance-theoretic ‘directness’ will bring the tension between the traditional view and acquaintance theories to the surface in other ways.

4  The new debate The traditional view took as empirical fact something that its proponents found plausible in their own cases, and expected each of us to find plausible in ours. But claims about matters of empirical fact are subject to empirical falsification. And one of the morals philosophers have been forced to draw from the growing flood of experimental findings about perception is that the traditional view is false. In particular, findings about object-directed attention29 and the ways that the perceptual system binds features together as belonging to the same object establish that the perceptual system does not deliver a patchwork which must then be parsed into particular things by thought-level processes. And findings about perceptual constancy30 establish that the property information the perceptual

29 

See Campbell, Chapter 31, this volume, on attention.

30 

See Cohen, Chapter 33, this volume.

846   Imogen Dickie

Input to basic cognitive processing

Input to

belief

Input to the rest of the subject’s belief system

• The purely perceptual information feed delivers a succession of patchwork-like seemings

• Basic cognitive process convert the information delivered by the purely perceptual feed into representations of the presence of material objects instantiating observable macroscopic properties

• beliefs are formed as a result of taking the results of the most basic level of cognitive processing at face value

Fig.  43.1  The traditional  view. system delivers is not mere two-dimensional patchwork-level information. Rather, the step to representation of observable macroscopic properties (properties like three-dimensional shape; size; colour; and motion or rest) is made by perceptual processing that is already complete before cognition comes to the table.31 The two charts (Figures 43.1 and 43.2) summarize the difference this step beyond the traditional view makes from the point of view of an account of the causal pathway from perceptual contact with an object to formation of perceptual demonstrative beliefs. I do not think it is an exaggeration to say that the step to the post-traditional view opens up the possibility of a revolution in the debate about how perception enables perceptual demonstrative thought.32 Though the impact of this step redounds through the full range of questions considered in this chapter, it is most obvious with respect to the point about skewing of the description-theoretic vs. acquaintance-theoretic debate from the end of section 3. For with the traditional view gone, (A) in the inconsistent triad laid out above is gone too, and with (A) gone, the acquaintance theorist can look anew for a view that upholds both (B) and (C). So the move away from the traditional view redefines the terrain on which the discussion about perception and demonstratives must take place. I shall give a brief indication of what I take to be the central issues in the new debate. I suggest that the first step in the new debate lies with an empirical claim about the pathway to belief described by the post-traditional view. To motivate this claim, suppose you are looking at a strange object whose shape properties and reflectance-of-light properties

31 

For a recent book-length discussion of which properties are represented in perception, see Siegel, 2010. Many others working on these and related issues are of the same mind. Evans’s talk of ‘perceptual information channels’ (1982:  ch. 6)  is already a step in this direction. See also Campbell, 2002; Matthen, 2005. 32 

Perception and Demonstratives    847

Input to further perceptual processing

Input to formation of beliefs

Input to the subject’s overall belief system

• The most basic perceptual processes deliver patchwork-level information

• Further perceptual processes deliver experiences as of three dimensional objects instantiating macroscopic observable properties

• beliefs are formed by taking the results of the further perceptual processing at face value

Fig.  43.2  The post-traditional  view. are radically anomalous. This object starts out round, then, in the space of a few seconds, compresses into an ellipse, then into a skinnier ellipse, then into a flat rectangle, before expanding through shades of ellipticality back into a circle. Meanwhile, the object absorbs and reflects light in a non-standard way. The overall result is that the shape and orientation cues picked up by your visual system are the same as those it picks up when you look at a constantly shaped disc rotating on an axis that passes through its centre parallel to its surface (a coin rotating on a vertical spit). In this case, the ‘further perceptual processing’ mentioned in the middle box of Figure 43.2 makes the same calculations as it does in a case where you actually are seeing a rotating disc. (That is why you end up with an experience indistinguishable by you from a ‘rotating disc’ experience.) But, given the category of object you are looking at, the beliefs you form based on these property calculations will not tend to get the object’s properties right. This conclusion generalizes. The visual system uses a series of algorithms to calculate the perceptual property information that serves as input for formation of belief. These algorithms result in reliable belief formation only given perceptual encounters with objects that behave the way ordinary objects behave in ordinary circumstances. So the post-traditional view generates an empirical claim about the relation between beliefs and the object at the end of the perceptual link that gives rise to them: An empirical claim about the formation of perceptual demonstrative beliefs—The pathway by which perceptual demonstrative beliefs are formed produces beliefs that tend to match the properties of the object at the other end of the perceptual link iff it is an ordinary material object. I suggest that perceptual links with ordinary objects enable us to have perceptual demonstrative thoughts about them because they deliver beliefs that are reliable in this

848   Imogen Dickie sense: beliefs that tend to match what the object at the end of the perceptual link is like. And I suggest that, in a post-traditional framework, the fundamental question about perception and demonstratives—the question of how perceptual contact with objects enables perceptual demonstrative thoughts about them—becomes the question of how to build an account of perceptual demonstrative aboutness-fixing around this central claim about reliability. To see why this suggestion is plausible, let us step back for a moment from our close focus on aboutness-fixing for perceptual demonstrative thoughts to consider aboutness-fixing for thoughts in general. (Recall that one element in an account of perceptual demonstrative aboutness-fixing is going to be a general background view of how aboutness-fixing works: this is the second component identified at the start of section 3.) Here are two widely accepted elements of this more general discussion: A claim connecting truth and justification—In general, justification homes in on truth, so that if you have sufficient justification to make it rational for you to form a belief, you will be unlucky if the belief is not true and not merely lucky if it is. A claim connecting what a belief is about with what it would take for it to be true— A body of beliefs is about an object iff their truth or falsity depends on what the object is like. The first claim coordinates aboutness and truth. The second coordinates truth and justification. Putting the two coordinating claims together we get an argument for a third claim coordinating aboutness and justification (aboutness and truth + truth and justification → aboutness and justification): A claim connecting aboutness and justification—A body of beliefs is about an object iff justification for holding them homes in on getting the object’s properties right. (For example, this claim says that the beliefs you would express using the proper name ‘Barack Obama’ are about Obama iff, in general, if it is rational for you to hold the belief you would express by saying ┌Barack Obama is Φ┐, it will not just be a matter of luck whether Obama is Φ—given your justification for the belief, you will be unlucky if Obama is not Φ and not merely lucky if he is.33 I suppress both the details of how the third claim follows from the first two, and any discussion of how the intuitive notions of ‘homing in on truth’ and being ‘lucky’ or ‘unlucky’ in the relevant sense are to be made precise.34) The claim connecting aboutness and justification can be restated as a claim about the kind of relation that suffices to fix aboutness. For the special case of perceptual demonstrative aboutness-fixing, the account can be summarized like this: A claim about perceptual demonstrative aboutness-fixing—A perceptual link with o enables perceptual demonstrative thought about o by securing the result that beliefs justified on the basis of this link will tend to get o’s properties right. 33 

I am using ‘Φ’ as a schematic variable ranging over conceptual representations of properties when it occurs inside pointy brackets and the associated predicates when it occurs in corner quotes. ‘Φ’ ranges over properties, and is braced with ‘Φ’:┌ That is Φ ┐ expresses the thought , and both are true if the referent of ‘that’ is Φ. 34  I provide some of the relevant details in Dickie, 2011a.

Perception and Demonstratives    849 Now let us place the empirical claim and the claim about perceptual demonstrative aboutness-fixing side by side:

The empirical claim

The claim about perceptual demonstrative aboutness—fixing

As long as the object at the other end of a perceptual link is an ordinary object, the ordinary route to formation of beliefs will tend to deliver beliefs that match the object’s properties.

A perceptual link with an object enables perceptual demonstrative thought about it by securing the result that beliefs justified on the basis of the link tend to match what the object is like.

The left-hand claim here is a ‘descriptive’ or ‘is’ claim (it describes a result that a causal process usually generates). The right-hand claim is a ‘normative’ or ‘ought’ claim (it describes a result that will tend to be generated if subjects form only beliefs that conform to a specific normative condition (beliefs that the subject ought to form given justification on the basis of perception)). To get from the first (‘is’) claim to the second (‘ought’) claim, we need a reason to say that the ordinary route to formation of beliefs is justification conferring. So, given the left-hand claim, the problem in explaining how a perceptual link with an object enables a perceptual demonstrative thought about it reduces to the problem of explaining why perceptual demonstrative beliefs formed by ordinary means are justified: different views of how perceptual demonstrative beliefs are justified will coordinate with different views of how perceptual demonstrative aboutness is secured. The question of how perception justifies perception-based beliefs has its own entries elsewhere in this volume.35 So, rather than embarking on a discussion of this topic here, I shall close by saying a little about the more immediate consequences of the post-traditional approach I have suggested for the puzzles from section 1. I should stress that the following remarks are not intended as comprehensive solutions to the puzzles. They are intended as a first indication of what a solution to each puzzle will look like in the framework I have proposed. Let us consider the category puzzle first. This is the puzzle of whether there can be perceptual demonstrative thought about particulars other than ordinary material objects, for example, events, times, places, shadows, ripples, or mereological gryphons. According to the post-traditional approach, the mechanism of perceptual demonstrative aboutness-fixing is keyed to ordinary objects. For it is if, but only if, the object at the end of the perceptual link is an ordinary object that the ordinary route to formation of beliefs will tend to produce beliefs that match the object’s properties. So it is if, but only if, the object at the end of the perceptual link is an ordinary object that it will be the object the beliefs formed by these means are about. If you are having an experience indistinguishable by you from an experience of looking at an ordinary thing when (unbeknownst to you) the ‘visual object’36 you are looking at is a shadow or a ripple or a

35 

36 

See Siegel and Silins Chapter 41, and Brogaard, Chapter 13, this volume. This is a psychologists’ term used to describe whatever is treated as an object by the visual system.

850   Imogen Dickie mereological gryphon, and if you go along with appearances and form a belief you would express by saying you are not having a perceptual demonstrative belief about the visual object from which your perceptual information is derived; you are having a belief about nothing. (Note that the point is not that a perceptual link cannot put you in a position to think about a particular that is not an ordinary object. It is just that the mechanism of aboutness-fixing in this kind of case is not the mechanism of perceptual demonstrative aboutness fixing. Note also that the post-traditional approach generates a way to refine our grasp of the boundaries of the category of ordinary objects: an ‘ordinary object’ (given this approach) is just a potential object of perceptual demonstrative thought; an object whose properties the ordinary mechanisms for formation of perceptual demonstrative belief will tend to get right.) Now consider the directness puzzle. This is the puzzle of how causally serpentine a perceptual link with an object may be while still sustaining perceptual demonstrative reference to it. For example, one standard case discussed by philosophers interested in perception and demonstratives is the case of an object seen in a mirror: is a thought formed on the basis of this kind of perceptual feed about the reflected object or its reflection or neither? The approach I have recommended generates an answer to this question, and a general rubric for addressing similar ones. Your thought is about the object, not the reflection, because it is the object’s properties, not the reflection’s, that you will tend to get right if you form beliefs by taking what the perceptual feed delivers at face value. If your are watching an orange roll in a mirror, you will form ; ; ; and beliefs. Given the way they are formed, these beliefs will almost all match what the orange, not the reflection, is like. (Your beliefs about the direction in which the thing is located and its orientation will tend to match the reflection, but these are exceptions to the rule). And this pattern holds in general: a perceptual link with an object is direct enough to sustain perceptual demonstrative reference to it iff beliefs formed by direct uptake from what the link delivers will tend to produce beliefs that match the object’s properties. The comprehensiveness puzzle is the puzzle of how much of an object you have to be in perceptual contact with if you are to form perceptual demonstrative thoughts about it. The approach I have suggested explains why perceptual contact with an ordinary object’s facing surfaces enables perceptual demonstrative reference to all of it: the path to formation of beliefs produces beliefs that tend to match what the whole object, not its facing surface, is like. And, again, this rubric generalizes to other comprehensiveness puzzle cases: perceptual contact with a part of X will sustain perceptual demonstrative thought about all of X wherever this same pattern ( beliefs formed by uptake of what the perceptual link delivers tend to match X’s properties) holds. The classification puzzle concerns whether the capacity for perceptual demonstrative thought depends on the capacity for sortal classification. The post-traditional approach generates an (at least initial) ‘No’ answer to this question. According to this approach, a perceptual link with an ordinary object enables perceptual demonstrative thought about it because the ordinary route to formation of beliefs on the basis of the link tends to get the object’s properties right. But on the face of things this route to formation of beliefs does not involve sortal classification. So, on the face of things, perceptual demonstrative aboutness-fixing does not involve sortal classification either.

Perception and Demonstratives    851 The focus puzzle is the puzzle of whether (and if not why not) we can have perceptual demonstrative thoughts about objects in peripheral vision. Recall that one of the difficulties involved in addressing this puzzle is that there are degrees of peripherality, so we seem to have to ask how much focus perceptual demonstrative aboutness-fixing requires. But according to the post-traditional approach, a perceptual link that secures perceptual demonstrative aboutness does so in virtue of the fact that beliefs formed on the basis of the link will reliably match what the object at the end of the link is like. And the reliability of a perceptual link depends on the degree to which it is focused. So we have a principle to use in addressing the focus puzzle. A perceptual link will enable perceptual demonstrative thought iff it is sufficiently focused on an object to put the information it delivers above an appropriate reliability threshold. Peripheral vision cannot sustain perceptual demonstrative thought because merely peripheral perceptual feeds do not generate beliefs that reliably match what the perceived objects are like. Finally, consider the awareness puzzle: what role (if any) does perceptual awareness play in enabling perceptual demonstrative thought. The post-traditional view does not generate an immediate response to this puzzle. But it does tell us where to look for one. Perceptual awareness plays an essential role in perceptual demonstrative aboutness-fixing iff it plays an essential role in the justification of beliefs formed by ordinary means.

References Ayers, M. (1974). ‘Individuals without sortals’. Canadian Journal of Philosophy, 4(1), 113–48. Ayers, M. (1997). ‘Is “physical object” a sortal concept? A reply to Xu’. Mind and Language, 12(3–4), 393–405. Campbell, J. (2002). Reference and Consciousness. Oxford: Oxford University Press. Campbell, J. (2006). ‘Does visual reference depend on sortal classification?’ Philosophical Studies, 127(2), 221–37. Campbell, J. (2011). ‘Visual attention and the epistemic role of consciousness’. In Mole, Smithies, and Wu, (eds), Attention: Philosophical and Psychological Essays (pp. 323–41). Oxford: Oxford University Press. Crane, T. (2011). ‘The problem of perception’. In Edwards N. Zalta (ed.), The Stanford Encyclopedia of Philosophy (Spring 2011 edn). Dehaene, S., Changeux, J.-P., Naccachel, L., Sackur, J., and Sergent, C. (2006). ‘Conscious, preconscious, and subliminal processing: a testable taxonomy’. Trends in Cognitive Science, 10, 204–11. Dennett, D. (1978). ‘Where am I?’ In Dennett, Brainstorms: Philosophical Essays on Mind and Psychology. Cambridge, MA: MIT Press. Dickie, I. (2010). ‘We are acquainted with ordinary things’. In Robin Jeshion (ed.), New Essays on Singular Thought (pp. 213–45). Oxford: Oxford Unversity Press. Dickie, I. (2011a). ‘Visual attention fixes demonstrative reference by eliminating referential luck’. In C. Mole, D. Smithies, and W. Wu (eds), Attention: Philosophical and psychological essays (pp. 293–322). Oxford: Oxford University Press. Dickie, I. (2011b). ‘The sortal dependence of demonstrative reference’. European Journal of Philosophy. Dummett, M. (1973). Frege: Philosophy of language. London: Duckworth.

852   Imogen Dickie Dummett, M. (1978). ‘Truth’. In Michael Dummett, Truth and Other Enigmas. London: Duckworth. Evans, G. (1985). ‘Understanding demonstratives’. In Gareth Evans (ed.), Collected Papers (pp. 291–321). Oxford: Oxford University Press. Evans, G. (1982). The Varieties of Reference. Oxford: Oxford University Press. Frege, G. (1884/1953). The Foundations of Arithmetic, trans. J. L. Austin. Oxford: Blackwell. Grandy, R. E. (2008). ‘Sortals’. In Edward N. Zalta (ed.), The Stanford Encyclopedia of Philosophy (Fall 2008 edn). Kaplan, D. (1968–9). ‘Quantifying in’. Synthese, 19, 178–214. McDowell, J. (1984). ‘De re senses’. Philosophical Quarterly, 34, 283–94. Matthen, M. (2005). Seeing, Doing, and Knowing. Oxford: Oxford University Press. Moore, G. E. (1918). ‘Some judgements of perception’. Proceedings of the Aristotelian Society, New Series, 19, 1–29. Moore, G. E. (1962). Commonplace Book 1919–1953. London: Allen and Unwin. Palmer, S. E. (1999). Vision Science—Photons to Phenomenology. Cambridge, MA: MIT Press. Quine, W. v. O. (1953). ‘Identity, ostension, and hypostasis’. In W. v. O. Quine (ed.), From a Logical Point of View (pp. 65–79). Cambridge, MA: Harvard University Press. Quine, W. v. O. (1960). Word and Object. Cambridge, MA: MIT Press. Quine, W. v. O. (1969). ‘Speaking of objects’. In W. v. O. Quine, Ontological Relativity and Other Essays (pp. 1–25). New York: Columbia University Press Recanati, F. (2012). Mental Files. Oxford: Oxford University Press. Russell, B. (1912). The Problems of Philosophy. Oxford: Oxford University Press. Searle, J. (1980). ‘Minds, brains, and programs’. Behavioral and Brain Sciences, 3, 417–57. Searle, J.(1983). Intentionality. Cambridge: Cambridge University Press. Siegel, S. (2010). The Contents of Visual Experience. Oxford: Oxford University Press. Smithies, D. (2011). ‘What is the role of consciousness in demonstrative thought?’ Journal of Philosophy, 108, 5–34. Strawson, P. F. (1950). ‘On referring’. Mind, 59, 320–44. Wiggins, D. (1997). ‘Sortal concepts: A reply to Fei Xu’. Mind and Language, 12, 413–21. Wiggins, D. (2001). Sameness and Substance Renewed. Cambridge: Cambridge University Press. Xu, F. (1997). ‘From Lot’s wife to a pillar of salt: Evidence that physical object is a sortal concept’. Mind and Language, 12, 365–392.

Chapter 44

Non h um a n A n i m a l  Senses Brian L. Keeley

1 Introduction: A star-nosed mole problem redux Aristotle is given credit for developing the earliest systematic account of the nature of the senses in the Western world. Famously, in works such as De Anima and De Sensu, he identifies five senses: sight, hearing, smell, taste, and touch (for more, see Sorabji, 1971, 1992; Keeley, 2009). What is less well appreciated is that Aristotle apparently thought these five senses were not only the complete list of senses in humans; he also thought they were the only possible senses—and hence the senses of all animals. (Although he discusses many other animals, he never discusses any different perceptual abilities in them; only suggesting that some animals have more powerful versions of the senses humans have.) It is now commonly (but not universally1) proposed that Aristotle undercounted when it comes to humans. Now sensory scientists speak of a vestibular (balance) sense and a sense of the position of one’s limbs (proprioception). Additionally, the sense of touch might better be thought of as separate senses, due to the different physiological mechanisms carrying out different neural processes (pressure, temperature, pain). However, there is no disagreement when it comes to Aristotle’s count of the animal senses. We now believe that nonhuman animals have many more senses than Aristotle thought: electrical senses in a variety of fish, sharks, and perhaps even the platypus (Scheich et al., 1986; Keeley, 1999; Pettigrew, 1999),2 magnetic senses in elasmobranch fishes (sharks, skates, and rays (Kalmijn, 1982)), and perhaps turtles (Putman et al., 2011) and migratory birds (Mouritsen and Ritz, 2005).3 Moreover, a number of animals can perceive stimuli well beyond what humans can, as in the famed olfactory abilities of bloodhounds, ultrasound perception in shad (Mann et al., 1998), ultraviolet perception in bees, and infrared perception in vampire bats, snakes, and 1 

See Nudds (2004, 2011) for a recent discussion that takes a five-sense view of humans as its starting point. And just recently, evidence has been published that a placental mammal, a dolphin, has an electric sense (Czech-Damal et al., 2011). 3  However, also see Johnsen and Lohmann (2005) for a discussion of the difficulties scientists have had in definitively identifying the nature of magnetoreceptive mechanisms in many organisms. 2 

854   Brian L. Keeley boas. There is a strong scientific and lay consensus that nonhuman animals exhibit much more sensory variety than Aristotle dreamed.4

How ought we to think about such nonhuman animal senses? A number of years ago, I published a paper (Keeley, 2002) that, in part, I hoped would rekindle interest among philosophers in this question: the question of how to individuate the sensory modalities. Related to this question are a number of others: what makes one sense (hearing, say) different from another (e.g. smell)? How many senses should neurotypical humans be said to have? How should we think of the senses of neuroatypical individuals, for instance as in synaesthesia or blindsight? What evidence would be needed to establish the existence of a previously unknown sense—a ‘sixth sense’? Are senses natural kinds? If not, what are they? And so on. In that paper, I identified two broad classes of questions in this area of interest. The first I called Aristotle’s Problem: ‘How many modalities do humans have and how ought we decide the issue?’ (Keeley, 2002: 10.) There has indeed been a resurgence of philosophical interest in the nature of human senses recently.5 On the strength of that interest, as well as the litany of reasons I discuss in Keeley (2002: 7–9), I will here be assuming the philosophical importance of Aristotle’s problem. Concerning the second class of questions I identified in my earlier paper, there has been considerably less uptake. Immediately after discussing Aristotle’s problem, and describing a particular late twentieth-century scientific debate concerning what sense to assign to the striking, fleshy nose of one species of mole, I then observed that, The same problem arises again, in a slightly different guise, in animal sensory biology. . . . What I shall call the star-nosed mole problem is the general philosophical problem raised by this type of scientific controversy. On what philosophical grounds should we decide which organisms possess which modalities? When scientists claim to have discovered a new sensory modality, what is the theoretical content of this claim? (10–11)

I called this the star-nosed mole problem in celebration of the work of sensory biologists (most notably, Ken Catania—who has since been awarded a MacArthur Fellowship for his work) at determining that the nose of these moles is a sensory organ and that, further, the sensory modality of this organ is tactile in nature (and not electrical or chemical, as had previously been hypothesized).6 Star-nosed moles are striking but not unique and can be taken to represent the more general class of concerns about the nature of nonhuman animal senses. When confronted with a nonhuman animal, how ought we to decide what sensory capacities it has, how those senses relate 4  A good, general overview of animal senses can be found in Hughes (1999). There is even reason to speak of the senses of plants, for example, in the case of the vibrisa of venus flytraps or the light and chemical sensitivity of vines (see Milius, 2009). However, for reasons of space, plants won’t be discussed here. 5  Here’s just a small sampling: Noë (2002, 2004); Nudds (2004);; Ross (2008); Batty (2010); Fish (2010: ch. 9); Gray and Tanesini (2010). Fiona Macpherson’s (2011a) edited collection of papers both new and old represents an important contribution to these concerns. See also the Editor’s contribution to this volume. 6  See Keeley (2002) for details and citations.

Nonhuman Animal Senses   855 to other animals (including humans), and how any given nonhuman animal sense relates to its other senses and other non-sensory abilities? At the time, I noted that this set of problems had been almost completely ignored by philosophers and not much has changed since.7 What might be motivating this ongoing oversight? Is it mere philosophical prejudice? I think not. Although it is rarely explicitly discussed, there are prima facie reasons for ignoring nonhuman animal senses in the philosophy of perception. However, on deeper consideration, I believe those reasons do not hold up. This is an issue that has not received sufficient consideration and in this chapter I plan to explore the relationship between the human and nonhuman cases with an eye towards explaining why I believe philosophers should not ignore the nonhuman cases. In section 2 I will propose an explanation for this lacuna concerning nonhuman cases in the philosophical discussion. In section 3 I will present my reasons for embracing the philosophical study of nonhuman animal senses, despite those unvoiced concerns. Before wrapping up this chapter, in section 4, I will have a look at some nonhuman animal senses—infrared reception, but also a bit at magnetoreception—to leave us with examples to think about.

2 A meta-question If my proposal is correct concerning the existence of Aristotle’s and the Star-Nose Mole Problems, then there are two important sets of questions here, but if so, another question—a meta-question—arises: what is the relationship between these two questions? One’s answer to this meta-question may have important consequences for the relationship between the answers to these two sets of questions. For example, it might be argued that because of a fundamental continuity of humans and other animals, these really ought to be treated as the same larger set of questions, but applied to different segments of a single domain: the world of animals.8 On the other hand, one might argue that with respect to the senses, at least, the nonhuman animal and the human cases are discontinuous to such a degree that we can expect their answers to be discontinuous as well. On this view, the answers given to one problem may have little to do with answers to the other. Another way of characterizing these two answers to the meta-question is in terms of the strength of the proposals being made.9 The latter approach to the question is a weaker proposition. On the discontinuity reading, there are two sets of questions—one concerning the senses of humans, the other concerning the senses of nonhuman animals—but the answers one develops for one set might nonetheless help in answering the questions of the 7  Tom Nagel’s famous 1974 paper, ‘What is it like to be a bat?’ was clearly inspired by the then recent discoveries concerning bat echolocation, but it wasn’t really about the sense of echolocation, so it isn’t a clear counterexample to the point I’m making here. However, one philosopher that clearly has written about the issue of nonhuman senses is Richard Gray (2005, 2011) and I will be looking at his work in section 4. Macpherson (2011b) also discusses senses outside the human case. Finally, I applaud Mohan Matthen for including a chapter on this topic in the current volume. 8  To be honest, in my 2002 paper, I simply assumed this kind of connection between the human and nonhuman cases, in part relying on arguments I had presented earlier (Keeley, 2000), although this earlier paper did not explicitly talk about the senses. Part of my goal here is to make that case more explicit. 9  My thanks to Richard Gray for this way of characterizing the issue here.

856   Brian L. Keeley other set by providing us with innovative ideas and hypotheses. On the continuous reading, we have a stronger thesis: answering one of these questions just is an answer to the other question. You cannot answer the questions of one set without answering the other. That strong connection is the alleged upshot of the biological continuity of human and nonhuman animals.10 Let’s look at these two answers a little more closely. If one believes that there is a deep connection between the sensory abilities of humans and other animals, then this would imply that there ought to be a similar connection between the explanations of the senses in both cases. This view relates to why I am using the somewhat cumbersome phrase ‘nonhuman animal’ instead of just ‘animal’. In our post-Darwinian world, we now recognize (or should recognize) that humans are animals, too. Hence, in the case of traits that we share with our nonhuman evolutionary cousins, we should be wary of discontinuous explanations. If this is the case, then one test of a philosophical theory of the senses ought to be not just how well it captures human senses, but also how well it captures those of nonhuman animals, whose senses were formed by the same processes of natural selection and ontogenetic development as were human senses. On such an approach, it is reasonable to conclude that of course the understandings of human and nonhuman senses ought to develop hand-in-hand with one another. On this view, Aristotle’s problem and the star-nosed mole problems are just opposite sides of the same coin. Let’s call this the continuity thesis. If, on the other hand, one takes the two cases to be importantly discontinuous—that is, if one denies the continuity thesis—then one will not be bothered that the account of human senses relies upon features that are missing or inaccessible in the case of nonhuman animals. In fact, the importance of such a feature might be a big part of what motivates the belief in the discontinuity in the first place. Let’s call this the discontinuity thesis. Among philosophers of perception who discuss the senses, I am aware of no significant discussion of this relationship or arguments concerning what answer to the meta-question we should endorse. I would like to suggest that the fact philosophers have largely ignored the star-nosed mole problem indicates an implicit belief in the discontinuity of the human and nonhuman cases. They have not addressed the issue of nonhuman senses, I suggest, because when they speak of the senses, they take themselves to be tackling a different set of questions. Alternatively, they may simply be assuming a strong version of the continuity thesis and therefore believe that because of the deep connection between human and other species that any account of the senses in humans just is an account of the senses in other animals. But there is a problem with both assumptions, in that on either account more needs to be said to give a full account. Historically, the notion of a discontinuity between humans and nonhuman animals with respect to the senses is a modern idea. As Heller-Roazen (2009: ch. 5) describes it, 10 

There is a third possibility here—arguably, the position that Aristotle himself takes—namely, that the two cases are discontinuous, but nonetheless the explanation in one domain is built on the answer in the other domain. On this account, although humans are importantly different from nonhuman animals, a full understanding of the human case will require an understanding of those nonhuman animals. In other words, humans are a special kind of animal, so that in an important sense humans are both continuous and discontinuous with these other creatures. I will not address this case specifically, but if I am correct in what I propose about both the continuous and discontinuous cases, I believe my conclusions will apply to this third, middle option, mutatis mutandis. (Thanks to Mohan Matthen for pointing out this third option to me.)

Nonhuman Animal Senses   857 prior to Descartes and the Modern turn in philosophy, sentience was a trait that humans and nonhuman animals were taken to share. Further, with that sentience came a shared sense of self-awareness (a view that Heller-Roazen dubs ‘Sentio ergo sum’). What separated humans and nonhumans, instead, is our rationality, our capacity to reason. But this close connection of humans and animals was lost in the move to Cartesianism, which reserved ‘real’ sensation, along with self-awareness and consciousness, to humans (and at the same time stripping the animals of their sentient status, or at least putting humans and nonhuman animals into importantly different categories when it came to the senses). In the background here is what might be called the ‘specialness’ of human conscious experience, of which perceptual experience is clearly a significant part. If human consciousness has an importantly different status from that of other animals, then it should only be expected that the senses of humans and nonhuman animals would be treated differently. Why think of human consciousness as especially different from that of other animals? Well, for starters, due to our capacity for language and categorization, we are able to report on the nature of our phenomenology in a way that other animals seem incapable of. It is, for example, open to humans and arguably humans alone to undertake to refine our introspective reports on the basis of careful practice and training, as psychologist Edward Titchener (1901–1905) proposed.11 Second, in addition to the ability to report verbally on them, there is also the obvious fact that we possess human senses and as a result, stand in a relationship to them that is importantly different from our relationship to the senses of other animals. The intuition here is that the peculiar phenomenal experiences associated with the operation of humans senses can be understood to be ‘obvious’ to a neurotypical human in a way that the senses of animals are not—especially those senses that are not shared by humans. I believe that this intuition plays an important role in the most famous philosophical discussion of animal senses: Thomas Nagel’s (1974) ‘What is it like to be a bat?’ That paper raises the titular question about the senses of bats as a question distinct from the hoary problem of other minds; it is not the same question of asking what it is like to be another human being. This is presumably because, by virtue of being human and possessing human senses, you know what it is like to sense the world as I do, in a way importantly different from how either of us can know what it’s like to sense the world via echolocation. On both of these accounts, we should expect the understanding of human senses to be importantly different from our understanding of nonhuman animal ones. This is because they purport to understand the senses in ways that aren’t applicable in the nonhuman cases: the former deals with senses-as-introspectively-reportable, the latter with senses-aspossessed-and-experienced-by-humans. If it is the senses understood in these ways that motivates and interests you, then it might seem that nonhuman animal senses—which share neither of these defining features—are just irrelevant. At least, if I  am correct in proposing that something like this line of reasoning is cogent, it would give us a plausible hypothesis to explain why, to date, philosophers have simply not been all that interested in thinking about nonhuman animal senses. 11  For a contemporary critical discussion of introspective techniques, see ­Schwitzgebel (2011: ch.5). Also, in stressing the essential importance of the ‘special introspective quality’ of the different senses, I suggest that the view presented in Grice (1962/1989) is representative of the position I describe in this paragraph.

858   Brian L. Keeley

3  The importance of nonhuman cases With the potential answers to the meta-question on the table—answers stressing either the continuity or discontinuity of human and nonhuman sensory abilities—I now want to turn my attention to reasons why there is philosophical and theoretical value to understanding nonhuman animal senses in light of those answers. I hope to show that no matter how one answers the meta-question, there are nonetheless good reasons for philosophers to pay attention to the nonhuman senses. If the continuity thesis is correct, then as noted above, it is hard to see how nonhuman animal senses will not be relevant to addressing Aristotle’s problem. After all, the continuity thesis supporter will note, we have no sensory organs not shared by other organisms, which is why we routinely do things such as use the macaque (a genus of old world monkey) as the workhorse of visual neurophysiology. The reason we’ve been able to learn so much about human vision from studying monkeys is that, in a real sense, much of what we think of as ‘human vision’ is not correctly thought of as human at all, but rather primate or mammalian or even vertebrate. Indeed, when we don’t recognize this, we are committing a form of anthropocentrism; that is, we are taking as specifically human a trait that is more correctly thought as belonging more broadly to other organisms, additional to us.12 The role of nonhuman senses for the understanding of human senses is not merely theoretical; it has historical precedent. Consider the case of what some argue is the first sense besides Aristotle’s five to be scientifically established:  the sense of balance. As psychologist-turned-historian-of-neuroscience, Nicholas Wade (2003) describes, a sense of balance was a phenomenon in search of an organ by the nineteenth century. Even the ancients were familiar with the experience of vertigo and wondered what underlay it. At the same time, the semicircular canals of the inner ear were organs in search of a function. There was an idea that they were part of our auditory system, specifically responsible for the perception of noise; after all, these organs are located adjacent to our cochlea and are even physically connected to the ear canal. They are all part of the bony labyrinth. How did we come to our contemporary view that there is a vestibular sense and that the semicircular canals are the organs that mediate that sense? Largely through comparative evidence. After all, humans are not unique in having these structures, so to figure out the role of these organs in humans, physiologists probed their role in these other animals. For example, in the 1820s Flourens lesioned these organs in doves, pigeons, and rabbits and observed the effects on their movements (‘ . . . but which [Flourens] preferred to interpret, from his bias to the acoustic theory of the labyrinth, as the expression of painful auditive disturbances’ (Mach, 1898: 290). Later in the century, Ernst Mach did ingenious work on the vestibular sense in humans, using a specially built spinning chair. However, according to Mach, the ‘most remarkable, most beautiful, and most convincing experiment’ was conducted by Austrian physiologist Alois Kreidl on crustaceans: According to Hansen, certain Crustacea on sloughing spontaneously introduce fine grains of sand as auditive stones into their otolith vesicle. At the ingenious suggestion of S. Exner,

12 

I explore this idea more fully in Keeley (2004). See also Matthen (2007).

Nonhuman Animal Senses   859 Dr. Kreidl constrained some of these animals to put up with iron filings (ferrum limatum). If the pole of an electromagnet be brought near the animal, it will at once turn its back away from the pole accompanying the movement with appropriate reflex motions of the eye the moment the current is closed, exactly as if gravity had been brought to bear upon the animal in the same direction as the magnetic force. This, in fact, is what should be expected from the function ascribed to the otoliths. If the eyes be covered with asphalt varnish, and the auditive sacs removed, the crustaceans lose their sense of direction utterly, tumble head over heels, lie on their side or back indifferently. This does not happen when the eyes only are covered. (302–3)

According to Mach then, it was work on crustaceans along with work like his own on humans that established the sensory function of the semicircular canals and convinced sensory scientists that there was a sixth sense of balance.13 This is not a unique case. As I  discuss in Keeley (2002), the decisive evidence in Catania’s case that the nose of the star-nosed mole was primarily a tactile sensory organ—and not an olfactory or electrosensory one, as had been proposed by others—was the comparative evidence he was able to present showing that the sensory end organs and cortical wiring of the star-nosed mole were homologous to the organs in other species of mole; organs commonly agreed to be tactile in nature. Comparative evidence often lets us answer questions related to one species by comparing that species to others; humans are no exception. Yet another issue that points to the importance of understanding nonhuman animal senses has to do with the evolution of sensory systems. Today we are confronted with the sensory skills of extant species, including ourselves. But all of these extant species, including us, are the product of evolution from many, many extinct species. The further back in history we go, the less like us these organisms were in terms of the sensory systems they had. Our sensory systems have evolved from other systems just as much as any of our other traits. There was a time in the history of this planet when the first creatures evolved the ability to sense light and we are likely the descendants of those creatures (or some others that convergently evolved the same ability) (Parker, 2003). How our current sensory systems have come to have the form they currently have—how they process information, how sensory representations have changed over time, what the nature of our evolutionary niche is—informs our understanding of our current senses. For example, the proposal that primate colour vision originally evolved to enhance our ancestors’ ability to spot ripe fruit within a cluttered visual environment can inform our understanding of the nature of colour perception (Regan et al., 2001). To sum up the argument of the last few pages, a proponent of the continuity thesis will argue that the senses of humans are senses that we share with other, nonhuman animals. As such, they are better thought of as traits of some larger taxonomic grouping than simply humans—they are, for example, vertebrate or mammalian traits. This being the case, then humans are just one instance of the trait in this larger set. Regardless, when one learns about a shared sense in one of these organisms one ipso facto learns about that sense in the human case.

13 

Something that is just common sense to the Anlo-Ewe of West Africa, who count a sense of balance among their set of basic senses, alongside vision, hearing, etc. (see Geurts, 2002). This only goes to show that what common sense purportedly tells us may not be so ‘common’ after all.

860   Brian L. Keeley But if one denies the continuity thesis, what then? To deny this is to deny the claim that humans share important features with nonhumans. Or rather, it is to deny that humans and nonhumans fully share what it is to have a sense—there are uniquely human aspects to the human possession of senses, such as reportability or the fact that we humans possess those senses. On the second point, we already can see reasons for calling it into question. The charge of anthropocentrism—that it is incorrect to speak of human senses as purely human senses and to do so is to assign a trait to humans that properly belongs to a broader taxonomic category—takes on that claim directly. Yet, setting that aside, what if there are unique aspects to human senses? Would not that imply that it is misguided to pay too much attention to nonhuman animal senses? Not necessarily. To see a reason why not, consider a concept drawn from the science of neuroethology—the biological science that studies the neural basis of naturally occurring behaviour: champion species. We begin with the observation that humans are unique in many ways, perhaps even in ways related to our senses. Arguably, we have a unique capacity to report on the nature of our sensory experiences. OK, but also observe that all evolved organisms are unique (Foley, 1987). That is to say that while humans are unique, they are not uniquely unique. Neuroethology has traditionally taken as its paradigm cases, unique and extraordinary animal systems that are arguably the best at some capacity. So, because bats and owls have the most developed auditory systems, neuroethologists have chosen to study these animals as a way to understand the nature of audition. Neuroethologist Walter Heiligenberg explicitly endorses this approach and enumerates its rationale, noting that some animal species are champions in particular aspects of sensory or motor performance and, . . .  such superior capabilities are linked to highly specialized neuronal structures. Such structures incorporate and optimize particular neuronal designs that may be less conspicuous in organisms lacking these superior capabilities. Moreover, the behavioral repertoire of such ‘champion’ species readily offers paradigms for testing the performance of their special designs at the level of the intact animal. Electric fish and echolocating bats, for example, are masters in the processing of temporal information and show an abundance of mechanisms devoted to the analysis of temporal signal characteristics. Therefore, these animals provide powerful model systems for behavioral as well as cellular studies of a wide scope of neural mechanisms dedicated to temporal information processing. Their exploration will reveal the diversity and limitations of these mechanisms and should ultimately facilitate our understanding of temporal information processing in other systems, for example, speech perception in humans. (Heiligenberg, 1991: 2; see also Carr, 1993; Keeley, 2000)

Seen in this light, humans can be seen as champion species when it comes to reportability.14 We can (and have) done psychophysics on nonhuman animals. For example, we can present sensory discrimination tasks to dogs to determine the nature of canine colour vision (Coile, Pollitz, and Smith, 1989; Neitz, Geist, and Jacobs, 1989). But due to our capacity for language we can report a lot about what we experience through our senses, more so than other species. By studying humans we can learn about the ability of other animals to

14  Perhaps, as well as what we humans can do with those reports (see Mohan Matthen’s contribution to this volume).

Nonhuman Animal Senses   861 report on their senses; similarly, as Heiligenberg notes, we can learn about the senses of humans by studying champion species who share our senses. The argument of this section so far has a straightforward structure. I begin by taking it for granted that there is philosophical and theoretical interest in understanding the nature of human senses, i.e. Aristotle’s problem. I then present a number of reasons and lines of argument to show how the study of nonhuman animal senses is important for a full understanding of human senses. This is to treat nonhuman senses instrumentally, as a tool for understanding their human counterpart. I now want to end this section by arguing that, all that aside, nonhuman animal senses are intrinsically interesting. Independent of what they can tell us about our own senses, they are philosophically and theoretically interesting all on their own. One reason is perhaps the most banal and the least likely to sway the unconvinced: philosophers ought to be interested in nonhuman animal senses for one of the same reasons scientists are, namely that these organisms are part of the same world that we occupy. Hence, to the extent that we wish to understand our world—a goal that scientists and philosophers share, one hopes—then the senses of animals, especially the list of senses not shared by humans, ought to be something that philosophers ought to help us understand. Part of the motivation here is the simultaneous recognition that (1) one goal of philosophy is to understand the world as it is experienced and that (2) nonhuman animals inhabit different worlds from us, in part as a result of their different senses. In claiming that animals occupy ‘different worlds’ from humans, I am invoking Jakob Johann von Uexküll’s (1934/2010) venerable notion of the Ümwelt here. Translated literally, Ümwelt just means ‘environment’ or ‘surroundings’, but in the technical sense von Uexküll employs it is closer to the environment of the organism as it senses and interacts with that world, such that two different species of organisms that inhabit the same physical environment would occupy different (but, perhaps overlapping) Ümwelten as a function of their respective sensory systems and means of interacting with that physical environment. In a famous example, von Uexküll describes the role of butyric acid in the Ümwelt of the tick. (Butyric acid, which is released from the sebaceous follicles of mammals, is the sensory cue that a tick uses to determine when to drop from its perch on a branch in the hopes of landing on a mammal below.) By way of analogy, he compares butyric-acid-for-the-tick with raisins-for-a-human-gourmet: Just as a gourmet picks only the raisins out of the cake, the tick only distinguishes butyric acid from among the things in its surroundings. We are not interested in what taste sensations the raisins produce in the gourmet but only in the fact that they become perception marks of his environment [Ümwelt] because they are of special biological significance for him; we also do not ask how the butyric acid tastes or smells to the tick, but rather, we only register the fact that butyric acid, as biologically significant, becomes the perception mark for the tick. We content ourselves with the observation that perception cells must be present in the perception organ of the tick that send out their perception signs, just as we assume the same for the perception organs of the gourmet. The only difference is that the tick’s perception signs transform the butyric acid stimulus into a perception mark of its environment, whereas the gourmet’s perception signs in his environment transform the raisin stimulus into a perception mark. Every subject spins out, like the spider’s threads, its relations to certain qualities of things and weaves them into a solid web, which carries its existence. (von Uexküll 1934/2010: 52)

862   Brian L. Keeley The ‘solid web’ of this metaphor can be thought of as the phenomenology of the tick.15 Although I  will not argue for it here, I  contend that philosophers should not be exclusively interested in the phenomenological worlds of neurotypical humans; we should also explore and seek to understand those of the autism spectrum disorder, synaesthesia, schizophrenia, ‘human echolocators’ (such as Daniel Kish, 2009), and so on. By the same token, what I am arguing in this chapter is that we should be similarly interested in the phenomenological worlds—the Ümwelten—of bats, electric fish, and star-nosed moles. The upshot of this section should be that there are two answers to the meta-question and on both answers, there is still philosophical and theoretical value in the question of nonhuman animal senses. If the two questions are indeed deeply related due to the continuity of biology, then the understanding of nonhuman animal senses is necessary to understand human senses. If there is a significant discontinuity between nonhuman and human senses, the understanding of nonhuman senses is still philosophically and theoretically important because (1) not everything about human senses is discontinuous, (2) the champion species argument has merit, and (3) nonhuman senses are intrinsically interesting.

4  The cases of infrared reception and magnetoreception I want to end by taking a look at an example of a nonhuman animal sense in order to leave the reader with a feel for how they might be interesting examples to explore. It has the property of challenging us to think about the sensory possibilities in ways that commonsense reflection on our own human sense perhaps does not. It represents a sense that Aristotle, with his Ancient Greek physics and philosophy of mind, would have had a hard time getting his head around. He was, for example, familiar with the existence of strongly electric fish, such as the Mediterranean Torpedo, but, lacking any concept of electricity, he was at a loss what to make of these creatures beyond that they were extremely odd in their ability to make a fisher’s arms feel leaden as a net was pulled in.16 While we now have a much firmer grasp on the phenomenon of electricity, the perception of the world via electroreception is still rather mysterious. Something similar can be said of the infrared case to which I will now turn. I propose that these cases are interesting both in themselves and for what they can tell us about our own senses. The ability of some animals to perceive infrared wavelength light creates some interesting difficulties in thinking about the senses. Some animals can directly perceive electromagnetic radiation in the infrared portion of the light spectrum. The modifier ‘directly’ is important here, because if one thinks about it, in a sense humans can come to know about the existence of infrared radiation in a variety of indirect ways. For example, it makes some 15 Buchanan (2008) explores the connections between von Uexküll’s work and the philosophical movement of phenomenology. 16  Hence, our modern English words ‘torpor’ and ‘torpid’ share a common root with the Torpedo. Plato, too, was familiar with this electric fish. There is a humorous passage in the Meno where the effect of Socrates’ speech on his listeners is likened to the paralysing effects of the Torpedo (and his flat forehead also unfavourably compared to the head of this unattractive fish).

Nonhuman Animal Senses   863 sense to say that, even from a good distance, you can see that the red, glowing iron bar that the blacksmith has just pulled from the furnace with her tongs is hot. As another example, most of us are familiar with the night vision goggles that are now commonly used by advanced militaries; these electronic optical devices rely, in part, on infrared sensitivity to enable their users to operate visually in low-light conditions. However, in both of these cases, perception of the thermal energy properties of the world are brought about by the eyes operating in the normal—non-infrared—range of the electromagnetic spectrum. In the case of the iron bar, what we are seeing is a particular red coloured glow in a specific context and are calling on our learned knowledge to infer or judge that it must be hot. In the case of the night vision goggles, this equipment operates by taking infrared information and systematically converting it into light within the range that human eyes can see. To distinguish such cases of direct from indirect cases of perception, neuroethologists find it important to distinguish between cases of reception from detection.17 The suffix -detection is applied in cases where an organism is capable of responding, by any means, to the presence of a particular type of stimulation in the environment. The suffix -reception is reserved for those organisms that carry out such sensory discriminations through the use of a dedicated anatomical system of structures.18 So, as has been described, humans are infrared-detecting their environment. This is different from the cases I am about to describe where organisms have specially evolved organs for direct perception of light in the infrared range.19 A number of animals have been described that are commonly believed to be infrared receptors, most famously pit vipers which, along with some other snakes, boas, and pythons, possess a morphologically distinct organ near the eyes known as a pit organ.20 As established by neuroethologist Theodore Bullock and biologist Raymond Cowles in the 1950s (see Bullock and Cowles, 1952; Bullock and Diecke, 1956), these pit organs enable their possessors to thermally image their environment, a useful ability for an animal that preys upon warm-blooded animals often in low-to-no-(visible)-light situations, such as in underground dens. These animals also possess separate organs (what we would naturally think of as ‘eyes’) that operate in visible ranges.21 The importance of the pit organs is illustrated by experiments in which snakes with intact pit organs but occluded or destroyed eyes can nonetheless hunt and strike accurately (De Cock Buning, 1983). While the photoreceptors of human and snake retinae reliably respond to the impact of photons in the visible range of electromagnetic radiation, the thermoreceptors of the pit organ are different. Richard Gray (2005) aptly summarizes it: Although the mechanisms underlying thermal imaging are still not fully known, behavioural and physiological studies have provided a good deal of information. What is apparent 17 

I discuss this distinction further in Keeley (1999: 404) (textbox). The concept of evolutionary and developmental ‘dedication’ is discussed in Keeley (2002: 17–19). 19  Although, as I will discuss, it might make sense to speak of an infrared receptive sense in humans mediated by thermoreceptors in the skin, as when one feels the warmth of the sun on a bright and sunny day. But even so, this would be a different sensory activity than the two examples of infrared detection described here. 20  Infrared reception has also been investigated in a number of insect species, such as in ‘fire-beetles’ that seek out recent forest fire areas for breeding (Schmitz, Schmitz, and Bleckmann, 2000). See Campbell et al. (2002) for a review. 21 It is difficult to talk about infrared perception in non-question-begging ways because of the anthropocentrism that is built into our ordinary language about light. After all infrared, as well as 18 

864   Brian L. Keeley is that the sensory endings of the receptors are receptive to specific wavelengths of electromagnetic radiation (photons of specific energy level). Sensitivity is maximized at the middle region of the infra-red range (about 10 μm). Significantly, the sensory endings are insensitive under conditions of stimulation similar to their natural environments to other ranges of electromagnetic radiation, for example visible light, to other forms of heat transference, for example conduction (making them independent of fluctuations in the body temperature of the snake), and to other forms of physical stimuli, for example sounds, odours and vibrations. Their sensitivity to radiant energy enables pit vipers to distinguish objects within their perceptual range from the differences in their temperatures from a background level which can be as small as 0.01° C. (466)

This case is interesting because two commonly used and commonsense criteria for dividing up the senses pull us in different directions in the case of the perception of infrared stimulation, as discussed (in more detail than I can here) by Gray (2005). For example, one very important criterion when talking of the senses is the presence of an identifiable organ. This was clearly important to Aristotle, for example, and it seems plausible to point to the importance of organs to our commonsense way of talking about the senses. Using this criterion, when confronted with pit vipers, we ought to talk about two separate senses. After all, there are many anatomical differences between the organs for perceiving visible and infrared light. Additional to the physically distinct locations of the two organs, one of them (the eye) has a lens for focusing its photonic stimuli whereas the pit organ lacks a lens. However, as I (2002) and others have argued, another important criterion is ‘the differing general features of the external physical conditions on which the various modes of perceiving depend[; . . . ] differences in the “stimuli” connected with different senses,’ as Paul Grice (1962/1989: 250) puts it. As I have already noted, there is no non-arbitrary difference between the range of physical stimuli perceived by these two different organs. In terms of the physics, there are no grounds for distinguishing between two senses here. There is but one sense distributed over two anatomical structures. Therefore two criteria here—organs and physics—contradict one another. Further, this is not simply a difficulty for pit vipers. After all, humans also have thermoreceptors. Ours are in our skin as part of our capacity to detect temperature. As Gray notes, there is evidence that these cells respond to stimulation in the infrared range (Greffrath et al., 2002). Perhaps, as I suggest in footnote 19, these thermoreceptors are responsible for the feeling we have as the sun beats down upon us or when we react to the radiant energy of a fire from a distance. If so, then relying on the physics criterion would lump this skintemperature sense in with the eyes, both as part of the same sense. On the organs criterion, we clearly are presented with two senses. There is a further complication. In this discussion, I have sometimes slipped between speaking of ‘infrared perception’ and ‘heat perception’, as if they were synonymous. ultraviolet, are defined entirely in terms of our normal human capacities. Beyond what our species-typical eyes are capable of, there is nothing else that separates the wavelengths we call ‘red’ and ‘violet’ from ‘infrared’ and ‘ultraviolet’. Yes, they differ by wavelength, but so do green and yellow. As Gray (2005: 466) puts it ‘According to physics, electromagnetic radiation forms a continuous spectrum with no intrinsic divisions.’ Further, ‘Photons have discrete energy values; nevertheless, there are no more general ways of distinguishing them by means of their intrinsic properties. Those divisions that are made in the electromagnetic spectrum arise from the relational properties of electromagnetic radiation, viz. the way it interacts with objects’ (474).

Nonhuman Animal Senses   865 They’re not. Our commonsense notion of ‘heat’, in fact, covers two different, but related, physical situations. Some of what we commonly think of as heat is radiant heat. This is electromagnetic energy à la infrared radiation. However, part of what we think of as heat is kinetic energy; it is the vibration of particles that make up a substance. So, heat can be transferred in two ways: (1) it can be radiated as infrared (and other frequencies) of light and (2) it can be conducted by contact as the kinetic energy of a hotter source is conducted to a cooler target. Further, the thermoreceptors in the skin are normally thought of as detecting conductive heat. You submerge your hand into water warmer than the skin of your hand. The kinetic energy of the water is conducted into your skin, stimulating your thermoreceptors and you perceive the warmth. Gray uses this distinction between radiant and conductive heat energy to propose another problem for thinking about the temperature senses. He calls it the Vampire Bat Problem22 and it is this: ‘Vampire bats are so-called because of their exclusive diet of blood. What is less well-known is that they locate the capillaries that are close to the skin surface of their prey by means of heat sensors in their noseleafs’ (469). He continues: The possibility thus arises of the heat sensors of the vampire bat being receptive, i.e. sensitive as part of the bat’s natural environment, to both radiant energy (when it is at a distance from its prey) and kinetic energy (when it is in contact with its prey). So the following question naturally arises: would it possess two senses or just one sense? The vampire bat problem is the converse of the pit viper problem: the vampire bat problem arises where one sensory organ is an appropriately wired up sensory organ that is historically dedicated to facilitating behaviour with respect [to] two identifiable physical classes of energy. (469–70)

One response to this conflict is to bring in additional criteria to arbitrate the conflict between these first two criteria. As Gray (2005) observes . . . [W]‌hy should we expect distinct forms of energy, as characterized by physics, to correspond neatly (one to one) with distinct sensory systems, as characterized by neurobiology? . . . Nature and the evolutionary process which acts on what nature supplies are more complex’ (472). We can call upon a criterion based upon the behaviours used in association with putative senses to help understand how they ought to be grouped or separated.23 For example, in the case of the pit viper, that the two organs are used differently in separate situations as part of the normal life of the animal (hunting outdoors in daylight versus underground and at night, say) help us to see these as separate senses, regardless of the similarity implied by the physics criterion. Something related can be said in the human case—I am not aware of species-typical behaviour in humans that use the infrared perception of the skin in conjunction with the eyes in normal lighting, so it does not make sense to speak of skin thermoreception and vision as parts of the same electromagnetic sense. The vampire bat problem poses a more intriguing problem. If it is correct to think of kinetic heat and radiant heat as two different identifiable physical classes of energy, as Gray

22  Gray notes that while he bases his account on work in neuroethology (Altringham, 1996), many of the details of this sensory system are still unknown. Therefore, he stresses that his account should be taken as a ‘thought experiment’ (469) or a proposal for how this system might turn out to function. It hits me as a plausible account. 23  Compare the similar role of behaviour in differentiating the senses proposed in Mohan Matthen’s contribution to this volume.

866   Brian L. Keeley argues, it is unclear how one can avoid the conflict between the physics criterion and the organ criterion (which points to two modalities in a single organ). Given the close association between these two forms of energy—radiant energy hitting a substance is converted into kinetic energy and perhaps both can be better thought of as forms of a unitary concept of thermal energy—perhaps the physics criterion can be thought of as not calling for a division here. However, even if something along these lines could be made to work, a very similar problem arises in another case: magnetoreception. The perception of magnetic fields is currently a very exciting area in the study of the senses. As mentioned in my introduction, a variety of animals have been discovered with behaviour that certainly seems to be magnetic in nature, but in many of these animals, the mechanism that mediates this behaviour is currently a hot topic of debate. Identifying the mechanism is relevant in order to determine which animals have genuine magnetoreception and which are carrying out magnetodetection via some other sense. Therefore, what needs to be discovered are magnetoreceptive organs and this poses special challenges, as Johnsen and Lohmann (2005) observe: Several factors have made locating magnetoreceptors unusually difficult. One is that magnetic fields pass freely through biological tissue. So, whereas receptors for sensory modalities such as vision and olfaction must contact the external environment to detect stimuli, this restriction does not apply to magnetoreceptors, which might plausibly be located almost anywhere in an animal’s body. In addition, magnetoreceptors might be tiny and dispersed throughout a large volume of tissue, or the transduction process might occur as a set of chemical reactions, so that there is not necessarily any obvious organ or structure devoted to magnetoreception. Finally, humans either lack magnetoreception or are not consciously aware of it, so our own sensory experiences provide little intuitive insight into where magnetoreceptors might be found.24 (703)

Yet another possibility is that magnetoreception is occurring in a sensory organ we normally think of as dedicated to another modality, hence invoking the Vampire Bat Problem. One currently intriguing avenue of research into a possible magnetoreceptor mechanism derives from a 1977 Nature paper by M. J. R. Leask. Leask proposes that magnetic energy could effect the way that photons are absorbed by photoreceptors. That is, as an organism visually senses the world, information about the ambient magnetic environment is revealed. The details of this cryptochrome hypothesis—as it has come to be known, referring to the blue light photoprotein implicated by the proposal—involve quantum mechanics and would take us too far astray to get into here.25 (However, I will note that, intriguingly, a relevant cryptochrome has been identified in the human retina (Foley, Gegear, and Reppert, 2011).) Regardless, the point here is that if this mechanism is established as mediating magnetoreception in some organism, then we will again be faced with Gray’s vampire bat problem, as in these creatures it could be said that they ‘see’ magnetic fields. Does this mean that as they turn from looking westward to look to the north that the 24  Hughes (1999) echoes this point:  ‘It is worth noting that the bodies of animals are completely transparent to magnetic fields (if that were not the case, magnetic resonance images of the inside of the body would not be possible). This means that the magnetoreceptor would work no matter where in the body it was located—the search has focused in the head, but the magnetoreceptor could be anywhere’ (158, emphasis in original). 25  See Hughes (1999: 162–70); Phillips, Jorge, and Muheim (2010); and Ritz et al. (2010) for more information.

Nonhuman Animal Senses   867 overall colour of the scene that they are peering at changes in a systematic way? This is an intriguing possibility and one that would cause us to rethink the way we understand the relationship between sensory modalities and their organs. The upshot of this section and this chapter as a whole has been to demonstrate current areas of interest and philosophical controversy within the study of nonhuman animal senses. There are a number of open questions here. Whether we are talking about electroreception, infrared reception, or magnetoreception, we have active areas of scientific investigation that pose a number of conceptual difficulties that can reward deeper philosophical exploration. The scientists working on such nonhuman animal systems are regularly grasping with conceptual, theoretical, and philosophical difficulties involving cases that force us to rethink the nature of our own senses. Further, these researchers daily grapple with trying to move beyond the human realm of sensory experience to grasp other ways of interacting with the physical world.

Acknowledgements I would like to thank Richard Gray for insightful discussions of this chapter and to Mohan Matthen for comments. I also thank the members of the Champalimaud Centre for the Unknown in Lisbon for providing me an excellent environment during the writing of this chapter. Finally, I thank Pitzer College for a Research and Awards Grant for financial support.

References Altringham, J. D. (1996). Bats, Biology and Behaviour. Oxford: Oxford University Press. Batty, C. (2010). ‘What’s that smell?’ Southern Journal of Philosophy, 47(4), 321–48. Buchanan, B. (2008). Onto-ethologies:  The animal environments of Uexküll, Heidegger, Merleau-Ponty, and Deleuze. Suny series in environmental philosophy and ethics. New York: SUNY Press. Bullock, T. H. and Cowles, R. B. (1952). ‘Physiology of an infrared receptor: The facial pit of pit vipers’. Science, 115(2994), 541–3. Reprinted in T. H. Bullock, How do Brains Work? Papers of a comparative neurophysiologist. Boston: Birkhauser, 1993, 138–9. Bullock, T. H. and Diecke, F. P. J. (1956). Properties of an infra-red receptor. J Physiol, 134, 47–87. Campbell, A. L., Naik, R. R., Sowards, L., and Stone, M. O. (2002). ‘Biological infrared imaging and sensing’. Micron, 33(2), 211–25. Carr, C. E. (1993). Processing of temporal information in the brain. In W. M. Cowan (ed.), Annual Review of Neuroscience. Palo Alto, CA: Annual Reviews, Inc. 16, 223–43. Coile, D. C., Pollitz, C. H., and Smith, J. C. (1989). ‘Behavioral determination of critical flicker fusion in dogs’. Physiology & Behavior, 45, 1087–92. Czech-Damal, N. U., Liebschner, A., Miersch, L., Klauer, G., Hanke, F. D., Marshall, C., Dehnhardt, G., and Hanke, W. (2011). ‘Electroreception in the guiana dolphin (Sotalia guianensis)’. Proceedings of the Royal Society B. De Cock Buning, T. (1983). ‘Thermal sensitivity as a specialization for prey capture and feeding in snakes’. American Zoologist, 23, 363–75.

868   Brian L. Keeley Fish, W. (2010). The Philosophy of Perception: A contemporary introduction. Routledge contemporary introductions to philosophy. London: Routledge. Foley, L. E., Gegear, R. J., and Reppert, S. M. (2011). ‘Human cryptochrome exhibits lightdependent magnetosensitivity’. Nature Communications, 2, 356. Foley, R. (1987). Another Unique Species:  Patterns in human evolutionary ecology. New York: Longman Scientific. Geurts, K. L. (2002). Culture and the Senses:  Bodily ways of knowing in an African community. Ethnographic studies in subjectivity. Berkeley and Los Angeles:  University of California Press. Gray, R. (2005). ‘On the concept of a sense’. Synthese, 147, 461–75. Gray, R. (2011). ‘On the nature of the senses’. In F. Macpherson (ed.), The Senses: Classical and contemporary readings (pp. 242–60). Oxford: Oxford University Press. Gray, R. and A. Tanesini (2010). ‘Perception and action: The taste test’. Philosophical Quarterly, 60(241), 718–34. Greffrath, W., Nemenov, M. I., Schwartz, S., Baumgartner, U., Vogel, H., Arendt-Nielsen, L., and R. D. Trede (2002). ‘Inward currents in primary nociceptive neurons of the rat and pain sensations in humans elicited by infrared diode laser pulses’. Pain, 99, 145–55. Grice, H. P. (1962/1989). ‘Some remarks about the senses’. In R. J. Butler (ed.), Analytical Philosophy, series I. Oxford: Oxford University Press. Reprinted in Paul Grice, Studies in the Way of Words (pp. 248–68). Cambridge, MA: Harvard University Press, 1989 and F. Macpherson (ed.), The Senses: Classic and contemporary philosophical perspectives. Oxford: Oxford University Press, 2011. Heiligenberg, W. (1991). Neural Nets and Electric Fish. Cambridge, MA: MIT Press (A Bradford Book). Heller-Roazen, D. (2009). The Inner Touch: Archaeology of a sensation. New York: Z one Books. Hughes, H. C. (1999). Sensory Exotica: A world beyond human experience. Cambridge, MA: MIT Press. Johnsen, S. and Lohmann, K. J. (2005). ‘The physics and neurobiology of magnetoreception’. Nature Reviews Neuroscience, 6(9), 703–12. Kalmijn, A. J. (1982). ‘Electric and magnetic field detection in elasmobranch fishes’. Science, 218, 916–18. Keeley, B. L. (1999). ‘Fixing content and function in neurobiological systems: The neuroethology of electroreception’. Biology & Philosophy, 14, 395–430. Keeley, B. L. (2000). ‘Neuroethology and the philosophy of cognitive science’. Philosophy of Science, 67(Proceedings), S404–S417. Keeley, B. L. (2002). ‘Making sense of the senses: Individuating modalities in humans and other animals’. Journal of Philosophy, 99(1), 5–28. Reprinted in F. Macpherson (ed.), The Senses: Classic and contemporary philosophical perspectives (pp. 220–40). Oxford: Oxford University Press, 2011. Keeley, B. L. (2004). Anthropomorphism, primatomorphism, mammalomorphism: Understanding cross-species comparisons. Biology & Philosophy, 19, 521–40. Keeley, B. L. (2009). ‘The role of neurobiology in differentiating the senses’. In J. Bickle (ed.), The Oxford Handbook of Philosophy and Neuroscience. Oxford: Oxford University Press. Kish, D. (2009). ‘Human echolocation: How to "see" like a bat’. New Scientist, 201(2703), 31–3. Leask, M. J. M. (1977). ‘A physicochemical mechanism for magnetic field detection by migratory birds and homing pigeons’. Nature, 267, 144–5. Mach, E. (1898). Popular Scientific Lectures (3rd edn). Chicago: Open Court.

Nonhuman Animal Senses   869 Macpherson, F. (ed.) (2011a). The Senses: Classic and contemporary philosophical perspectives. Philosophy of mind. Oxford: Oxford University Press. Macpherson, F. (2011b). ‘Taxonomising the senses’. Philosophical Studies, 153(1), 123–42. Mann, D. A., Lu, Z., Hastings, M. C., and Popper, A. N. (1998). ‘Detection of ultrasonic tones and simulated dolphin echolocation clicks by a teleost fish, the American shad (Alosa sapidissima)’. The Journal of the Acoustical Society of America, 104, 562–8. Matthen, M. (2007). ‘Defining vision: What homology thinking contributes’. Biology & Philosophy, 22(5), 675–89. Milius, S. (2009). ‘No brainer behavior: Messages, memory, maybe even intelligence— botanists wrangle over how far plants can go’. Science News, 175(13), 16. Mouritsen, H. and Ritz, T. (2005). ‘Magnetoreception and its use in bird navigation’. Current Opinion in Neurobiology, 15(4), 406–14. Nagel, T. (1974). ‘What is it like to be a bat?’ Philosophical Review, 83(4), 435–50. Neitz, J., Geist, T. and Jacobs, G. H. (1989). ‘Color vision in the dog’. Visual Neuroscience, 3, 119–25. Noë, A. (2002). ‘On what we see’. Pacific Philosophical Quarterly, 83(1), 57–80. Noë, A. (2004). Action in Perception. Representation and mind. Cambridge, MA: MIT Press. Nudds, M. (2004). ‘The significance of the senses’. Proceedings of the Aristotelian Society, 104(1), 31–51. Nudds, M. (2011). ‘The senses as psychological kinds’. In F. Macpherson (ed.), The Senses: Classical and contemporary philosophical perspectives (pp. 311–40). Oxford: Oxford University Press. Parker, A. (2003). In the Blink of an Eye. Cambridge, MA: Perseus Publishing. Pettigrew, J. D. (1999). ‘Electroreception in monotremes’. Journal of Experimental Biology, 202, 1447–54. Phillips, J. B., Jorge, P. E., and Muheim, R. (2010). Light-dependent magnetic compass orientation in amphibians and insects: Candidate receptors and candidate molecular mechanisms. Journal of the Royal Society Interface, 7, S241–S256. Putman, N. F., Endres, C. S., Lohmann, C. M. F., and Lohmann, K. J. (2011). ‘Longitude perception and bicoordinate magnetic maps in sea turtles’. Current Biology, 21(4). Regan, B. C., Julliot, C., Simmen, B., Viénot, F., and Charles-Dominique, P. (2001). ‘Fruits, foliage and the evolution of colour vision’. Philosophical Transactions of the Royal Society of London B, 356, 229–83. Ritz, T., Ahmad, M., Mouritsen, H., Wiltschko, R., and Wiltschko, W. (2010). ‘Photoreceptorbased magnetoreception: Optimal design of receptor molecules, cells, and neuronal processing’. Journal of the Royal Society Interface, 7 Suppl 2, S135–46. Ross, P. W. (2008). ‘Common sense about qualities and senses’. Philosophical Studies, 138(3), 299–316. Scheich, H. G., Langmer, C., Tidemann, C., Coles, R. B., and Guppy, A. (1986). ‘Electroreception and electrolocation in the platypus’. Nature, 319, 401–2. Schmitz, H., Schmitz, A., and Bleckmann, H. (2000). ‘A new type of infrared organ in the Australian "fire-beetle" Merimna atrata (Coleoptera: Buprestidae)’. Naturwissenschaften, 87(12), 542–45. Schwitzgebel, E. (2011). Perplexities of Consciousness. Life and mind: Philosophical issues in biology and psychology. Cambridge, MA: MIT Press (A Bradford Book).

870   Brian L. Keeley Sorabji, R. (1971). ‘Aristotle on demarcating the five senses’. Philosophical Review, 80, 55–79. Reprinted in F. Macpherson (ed.) (2011). The Senses: Classic and contemporary philosophical perspectives. Oxford: Oxford University Press. Sorabji, R. (1992). Intentionality and physiological processes:  Aristotle’s theory of sense-perception. In M. C. Nussbaum and A. O. Rorty (eds.), Essays on Aristotle’s De anima (pp. 195–225). Oxford: Clarendon Press. Titchener, E. B. (1901–5). Experimental Psychology:  A  manual of laboratory practice. London: Macmillan. von Uexküll, J. (1934/2010). Foray into the Worlds of Animals and Humans: With a theory of meaning. Posthumanities. Minneapolis: University of Minnesota Press. Wade, N. J. (2003). The search for a sixth sense: The cases for vestibular, muscle, and temperature senses. J Hist Neurosci, 12(2), 175–202. Reprinted in D. Howes, The Sixth Sense Reader. (pp. 55–86) Oxford: Berg.

Chapter 45

Perception a n d  A rt Dominic McIver Lopes

Dispatches from the public relations division of contemporary neuroscience sometimes give the impression that the search is on for a module of the brain dedicated to making and appreciating works of art. Of course the brain houses no ‘art module’. The aptitude for making and appreciating works of art spins off cognitive capacities that principally serve non-artistic functions. Storytelling takes advantage of the capacity to entertain counterfactual states of affairs (see Nichols, 2006), dramatic acting repurposes capacities for language and expressive gesture (Hamilton, 2007), and pictures harness the powers of perception and perceptual memory. And yet pictures are not just straightforward applications of perceptual processes, deserving nothing more than a footnote in the story of perception, for they implicate perception in unique ways.

1  Art, value, and perception Conjoining ‘perception’ to ‘art’, as in the title of this chapter, immediately brings in questions of value. A picture is presumably valuable as a work of pictorial art only when its value is due, at least in part, to the fact that it offers a perceptual experience—presumably one of a specifically pictorial kind. What kind of experience is that? Before turning to what philosophers have had to say in answer to this question, it would be prudent to zero in on the explanandum, the artistic value of pictures. In what follows, let ‘art’ be a term of art—that is, a term whose meaning is determined by its theoretical role: it refers to those images that are made to realize value by engaging our perceptual faculties. We value Rembrandt’s self-portraits for the honest self-assessments they make visible and we value Henri Cartier-Bresson’s Behind the Gare Saint-Lazare for magically freezing the motion of the visual world. By stipulation, what makes these and other images works of art is that they are made to realize value by depicting scenes and hence by providing us with second-hand experiences of those scenes. Stipulating to this technical conception of art is a way to sidestep a roaring debate about the nature of art that was sparked by the creation of works of art that are perceptually indiscernible from non-art objects (see Shelley, 2003). The stock example is Andy Warhol’s

872   Dominic McIver Lopes Brillo Boxes, an assemblage of plywood boxes painted to look like the packaging for Brillo scouring pads. Having stumbled upon the Brillo Boxes at the Stable Gallery in New York, Arthur Danto (1964) came to think that the features that make an item a work of art do not supervene on its perceptible features. As Danto reasoned, the features that make an item a work of art supervene on its perceptible features only if a work of art is perceptually discernible from every item that is not a work of art, but some works of art (e.g. Brillo Boxes) are not perceptually discernible from items that are not works of art. Many philosophers regard this argument as decisive, but others object that Brillo Boxes is not a genuine art work (e.g. Beardsley, 1983). In the technical sense of art stipulated above, Brillo Boxes is not a work of art. The stipulation is not a ploy for avoiding the thorny questions thrown up by works like Warhol’s. These questions are certainly worth asking (Lopes, 2008). However, an accelerating pace of scholarship on art in neuroscience and psychology as well as in the humanities intensifies the problem of how to integrate these research programmes, so as to enable scholars in different disciplines to take advantage of each others’ results (Bergeron and Lopes, 2011). Integration only requires converging conceptions of what is to be explained—not agreement on how to explain it—but there is little consensus on explananda. On one hand, scientists who offer hypotheses about art are often surprised to find them rejected by humanists as inadequate to explaining contemporary art practices, like that of Warhol. On the other hand, humanists rarely consider the heuristic benefits of focusing on works whose value is essentially connected with the fact that they offer specifically pictorial kinds of perceptual experiences. These works allow us to bring our relatively advanced understanding of perception to bear on our relatively rudimentary understanding of artistic value. Moreover, an understanding of the kinds of features that realize the value of Behind the Gare Saint-Lazare is likely to be a good step towards an understanding of the kinds of features that realize the value of Brillo Boxes. One more preliminary. Art images are works whose value is essentially connected with the fact that they offer certain kinds of perceptual experiences, but those experiences need not be visual. Some argue that sound recordings may be auditory images (Kulvicki, 2006). Tactile pictures used by blind people have many of the same features as visual pictures (Kennedy, 1993; Lopes, 1997). This complication may be set aside for now since only visual pictures have been studied in enough detail to yield material for an explanation of their perceptually-realized value.

2  Theories and priorities The philosophical literature on the nature of pictures and their association with perception is large, diverse, and rapidly expanding. However, it grows out of the pioneering work of E. H. Gombrich in Art and Illusion (1961) and Nelson Goodman in Languages of Art (1976) and it continues to reflect their starkly contrasting approaches. Each approach prioritizes different questions and different theoretical resources. One question is what makes some representations pictures—and hence what distinguishes them from other image-like representations and from wholly non-imagistic representations. Traditionally, the main concern was to compare pictures to mental images and to contrast them with linguistic utterances, but it is equally important to situate pictures

Perception and Art   873 within the sprawling family of representational genera whose members also include models, equations, diagrams, charts, and tables. An answer to this first question is a theory of pictures. Such a theory differentiates the genus of pictures from other genera or kinds within the family of representation, bringing out relevant similarities and differences. A second question is what makes it the case that a picture depicts what it does. For example, why does a given photograph depict a street in the rain rather than a stand of cacti? What kinds of facts about the history of its making and the properties of its surface determine its content? An answer to this question is to be sought in a theory of depiction (sometimes called a theory of pictorial representation). Such a theory explains the distinctive way in which the content of a picture is determined. Pictures are objects of experience and those experiences seem to have a distinct phenomenal character. They are not identical to experiences of mere abstract patterns—wallpaper patterns, for example. Nor are they identical to experiences of the scenes they depict—unlike Zeuxis’s birds, we are rarely in any mood to confuse seeing a picture of a bunch grapes with seeing some grapes face-to-face. Putting the point very roughly, our experiences of pictures is somehow a blend of experiences like wallpaper patterns and experiences like seeing bunches of grapes. A theory of pictorial experience states what it is for an experience to be a pictorial blend of these elements. A fourth question engages some writers, who think that pictures have a distinctive type of content. According to Gombrich’s ‘eyewitness principle’, for example, pictures always represent scenes from perceptual points of view (1982:  253–6). This principle imposes a structure on the accuracy conditions of a picture. Representing a brick wall precludes representing what is behind the wall. Noting that the eyewitness principle only applies to pictures in standard perspective, Lopes (1996) prefers a more general ‘selectivity principle’. For every image, representing some determinables precludes the representation of some other determinables. Thus a cubist image might represent a brick wall and also what is behind it, but only by sacrificing the representation of another determinable, such as the wall’s overall shape. A theory of pictorial content states what, if anything, is distinctive of pictorial content (see also Haugeland, 1991 and Kulvicki, 2006). An answer to any one of these questions may potentially deliver answers to all or some of the others, but philosophers have reached no consensus on priorities, and the liveliest and least tractable disagreements about the theories offered in answer to any of the four questions often stem from different assignments of explanatory priority. Here is a sample of the currently most influential theories representing three main approaches. One approach follows a strand in Gombrich that gives priority to theories of pictorial experience (Wollheim, 1980, 1987; Peacocke, 1987; Budd, 1992; Hopkins, 1998; Abell, 2009). Richard Wollheim proposed that our experiences of pictures are ‘twofold’. In his earlier work (1980), he analysed twofoldness as the simultaneous occurrence of two experiences, one representing the picture surface and the other representing the depicted scene. He later (1987) rejected this analysis, proposing instead that a twofold experience is a single experience that simultaneously combines an awareness of the picture surface with an awareness of the depicted scene (see Nanay, 2005). As Wollheim saw it, not only pictures evoke twofold experiences—an experience of a water stain on a wall might simultaneously combine an awareness of the shape of the stain with, for example, an awareness of a woman’s figure. Accordingly, he argued that a picture depicts a scene just in case it prompts a twofold experience of the picture and the scene and it was intended to do so (or caused to do so by a camera). A picture is then a representation that depicts.

874   Dominic McIver Lopes Commentators complained that Wollheim’s analysis of twofoldness is more redescription than explanation; Robert Hopkins (1998) repairs this defect by reducing twofold experience to experienced resemblance in ‘outline shape’. The outline shape of an item is its solid angle from a point of view, and a two-dimensional projection of the three-dimensional object may share the same outline shape. Hopkins argues that pictorial experience is twofold in the sense that it is an experience of a resemblance between a two-dimensional surface and a three-dimensional scene in respect of outline shape. This theory of pictorial experience outputs a theory of depiction: a picture depicts a scene just in case we experience it as resembling the scene in the way just described and when this experience is intended (or caused by a camera). Again, a picture is a representation that depicts. Campaigning against experience-first approaches, Goodman (1976) urged that the special character of our experience of a picture of a scene is a consequence of its depicting the scene. What gets priority for Goodman is a theory of pictures, which is a component of a general theory of representation. Systems of representation differ in whether or not they are syntactically disjoint (each mark belongs to no more than one character), syntactically differentiated (it is determinate which character a mark belongs to), semantically disjoint (no character has more than one reference), and semantically differentiated (it is determinate what a character refers to). Pitch markings in modern musical notation are syntactically and semantically disjoint and differentiated. Spoken languages like English are syntactically but not semantically disjoint and differentiated. Pictures are analogue representations: they are not syntactically or semantically disjoint or differentiated. This explains why, in a picture, the smallest difference makes a difference both to the character of the picture and to what it represents. Since pictures are not uniquely analogue, Goodman adds that they are relatively replete—more of their determinable features are representationally relevant. Given this theory of pictures, he adds that a picture depicts a scene just in case it represents it in a pictorial system and it evokes an experience that is something like an experience of the scene because it represents in a system that is ‘familiar’ or ‘entrenched’. While Goodman’s characterization of the structural properties of pictorial systems is extremely powerful, his theories of depiction and pictorial experience are not convincing. There is ample empirical evidence that perceptual mechanisms are implicated in our engagement with pictures and some have argued that representations in analogue, relatively replete, and familiar systems that do not implicate these mechanism are not pictures and do not evoke the experiences that pictures typically evoke. For example, the three representations reproduced in Figure 45.1 carry the same information, all are analogue, and all are equally replete, but no amount of familiarity with the drawing system used in (c) makes it like (a) or (b). These doubts about Goodman’s theories of depiction and pictorial experience give one reason to doubt the value of his theory of pictures. John Kulvicki (2006) ingeniously mitigates these objections, principally by adding a condition to Goodman’s theory of pictures. This condition attributes to pictures a distinctive type of content. Pictures are transparent in the special sense that a picture in a system and a picture of that same picture made in the same system are syntactically identical and represent the same colour and spatial properties—they have the same ‘bare bones’ content. Redness is part of the bare bones content of a colour photograph of a lady wearing a red hat because a colour photograph of the colour photograph also represents redness. This theory of pictures provides materials for theories of depiction and pictorial experience. Define mimetic representations as those that instantiate their own content. It follows

Perception and Art   875

Fig.  45.1 Three  faces. that pictures are mimetic with respect to their bare bones content—a colour photograph of a lady wearing a red hat represents the hat as red by being red. The fact that pictures are mimetic explains what is special about our experiences of them: we experience them as having some of the properties they depict scenes as having. In addition, Kulvicki suggests that bare bones contents trigger recognition abilities, and that is why pictures depict not just colour and spatial properties but also cacti and rainy streets. The third approach, whose roots may be traced to a strong naturalistic strand in Gombrich, assigns priority to theories of depiction. Gombrich speculated that artists have learned to make pictures by noticing how marks on a picture surface might trigger perceptual effects like those that might be triggered by the depicted scene, were it encountered face-to-face. These learned tricks are valuable and likely to spread. According to recognition theories of depiction, a picture depicts a scene because it recruits the same perceptual resources that subserve recognizing elements of the scene face to face (Schier, 1986; Lopes, 1996; Newall, 2011). Pictures are simply two-dimensional representations that depict, and pictorial experience is an experience implicating both the triggering of recognition abilities for two-dimensional surfaces and the triggering of recognition abilities for the three-dimensional constituents of the depicted scene. The main advantage of this approach is that it imposes no a priori restrictions on depiction or pictorial experience. An image—a cubist composition, for example—might depict a scene that it does not resemble in outline shape. Or it might not evoke an experience of simultaneous awareness of picture and depicted scene—it might evoke, for example, an experience oscillating between one and the other. Depiction will turn out to be as unsystematic as the jury-rigged assemblage of mechanisms that make up human perception. Of course, this feature of the approach may be viewed as a disadvantage by its detractors.

3  Pictorial experience Art images are images whose value is realized through specifically pictorial kinds of perceptual experiences. Those who set up a theory of pictorial experience to deliver a theory of depiction or a theory of pictures are likely to think that pictures evoke a single kind of

876   Dominic McIver Lopes experience. Those who view pictorial experience as non-unitary tend to expect that a theory of pictures or a theory of depiction will explain why we experience pictures in a variety of ways roughly reflecting the patchwork of mechanisms that underlie visual processing. The question of the unity of pictorial experience polarizes the theoretical options. Since the character of pictorial experience is controversial, it is useful to have a neutral vocabulary in which the disagreement can be expressed (Hopkins, 2010: 151–2). It is safe to say that experiences of pictures have dual contents. An experience of a picture typically represents a flat, marked surface (typically a surface on a wall, bounded by a frame). When there is uptake on the part of the viewer, the experience also represents a depicted scene—a three-dimensional arrangement of shapes and colours, or a building, or the Eiffel Tower. The disagreement turns on how these dual contents are represented in pictorial experience. Approaches that prioritize theories of pictorial experience typically take it to be unitary. The claim that pictorial experience is twofold, a simultaneous awareness of picture surface and depicted scene, is bedrock for Wollheim’s (1987) philosophy of visual art. Hopkins (1998) argues that pictorial experience has a single, complex content which marries the dual contents: it represents a similarity between the picture surface and the depicted scene. Malcolm Budd (1992) and Kendall Walton (1990) also treat pictorial experience as unitary. According to Walton, for example, a pictorial experience is a rich imagining of one’s seeing the picture surface that it is a case of seeing the depicted scene. (The imagining is rich to the extent that seeing determinate features of the surface is imagined to be seeing determinate features of the depicted scene.) How are these strong claims to be adjudicated? The literature exhibits an abductive inference pattern that happens to fit the more general pattern set out by Susanna Siegel (2011). First a fact about the phenomenology of our experiences of pictures is identified. Then a claim about the content of the experience is offered as the best explanation of that fact. (In running this inference, nobody has seriously argued that the fact about the phenomenology is explained by the experience’s non-representational raw feel.) With that in mind, what is needed is a look at the putative facts about the phenomenology of pictorial experience. A famous example is an anecdote told by Gombrich about Kenneth Clark. Gombrich write that, Looking at a great Velázquez, [Clark] wanted to observe what went on when the brush strokes and dabs of pigment on the canvas transformed themselves into a vision of transfigured reality as he stepped back. But try as he might, stepping backward and forward, he could never hold both visions at the same time. (Gombrich, 1961: 6)

For Gombrich, pictures work an ‘elusive magic of transformation which is so hard to put into words’ (1961: 6). Taking this as his datum, Gombrich went on to argue that experience never represents a picture’s surface and the scene it depicts simultaneously. Consequently, an overall experience of a picture must involve switching from one experience to the other over a span of time. Wollheim disagreed, arguing that pictorial experience has a ‘distinctive phenomenology’ because it involves a representation of surface and scene together, so that ‘it is not just permitted to, but required of, me that I attend simultaneously to object and medium’ (Wollheim, 1980: 213; see also Nanay, 2005).

Perception and Art   877 When faced with such head-butting over descriptions of the phenomenology of pictorial experience, it seems reasonable to split the difference and accept that some pictorial experiences are as each of these gentlemen describe, albeit neither describes the full gamut of pictorial experiences. Given this option, we should predict a spectrum of cases. At one extreme are trompe l’œil images, which evoke experiences of depicted scenes and normally block experiences of their own surfaces, thereby conjuring up strong illusions. However, Clark’s Velázquez is neither an example of trompe l’œil nor does it fit Wollheim’s description. If he is to be taken at his word, Clark is never unaware of the surface but he cannot bring parts of that surface into focus as ones which manage to represent the depicted scene. To make sense of this, we might distinguish between the surface properties of a picture and its design properties. A picture’s design properties are a subset of its surface properties, namely those in virtue of which it depicts a scene (Lopes, 2005). Seeing the former does not imply seeing the latter. After all, it might be considerably harder to see what features of a surface are responsible for depicting a scene than it is to see a surface that is doing some depicting. So perhaps Clark switches back and forth between an experience representing the scene and another representing the design and perhaps he also has a single experience simultaneously representing scene and surface (hence no trompe l’œil). Call images like the Velázquez ‘naturalist’. They inhibit experiences simultaneously representing design and scene but promote experiences simultaneously representing scene and surface. At the opposite end of the spectrum are images within Wollheim’s paradigm—they evoke experiences simultaneously representing scene and design. Call them ‘twofold’ images. The three-way distinction between representing scene, surface, and design makes space for some further intricacies. One is that an experience may misrepresent the properties in virtue of which a scene is depicted: this is an experience of ‘pseudo-design’ (Lopes, 2005). An example is subjective contour. If, when you are looking at Figure 45.2, you have an experience representing a Dalmatian, then you experience a contour on the picture surface as one in virtue of which the dog is depicted—the contour seems to match the outline of the dog’s body. However, this contour is not one in virtue of which the dog is depicted. On the contrary, it is represented in experience only as long as experience represents the depicted dog, and a consequence of the picture’s depicting a dog cannot explain its depicting a dog. The outline is not actually a design feature. Figure 45.2 is ‘pseudo-naturalist’. Another intricacy is the phenomenon of what has come to be called inflection (Hopkins, 2010; Nanay, 2010). In many cases, the surface, design, and scene properties that are represented in a pictorial experience can be characterized independently of each other. However, an experience representing a depicted scene may be inflected by design properties, so that a full characterization of the scene-representing content of the experience must make reference to some design properties (Hopkins, 2010: 158). The result is that the scene is represented as having properties that could not be represented in a face-to-face experience of the same scene. Georges Seurat’s 1889 Eiffel Tower (Figure 45.3) depicts the tower as made up of pure radiation. It does not show how the tower would look were it made of pure radiation; it depicts it as it does by depicting it by means of the touches of Seurat’s brush. Inflected pictorial experiences simultaneously represent scene and design, but it is an open question whether every experience of a ‘twofold’ image is also inflected. Examples of pseudo-naturalist and inflected pictures should not be dismissed as curiosities. In general, if an imagistic effect is at all possible, it is very likely to be exploited

878   Dominic McIver Lopes

Fig.  45.2 Dalmatian. by artists, since it has long been part of the practice of the image arts to discover and then control optical and phenomenological effects, often making them a major theme of some artistic practice or another. Impressed by this, psychologists and neuroscientists are apt to remark that artists have long been vision scientists avant la lettre (e.g. Zeki, 2000; Livingstone, 2002). There is no reason for artists to have shied away from making pseudo-naturalist and inflected images in particular. The challenge for those who take pictorial experience to be unitary is its apparent phenomenological diversity. To take this challenge seriously, unitarians must characterize the content of pictorial experience in a way that accommodates the phenomenological differences between trompe l’œil, naturalist, quasi-naturalist, twofold, and inflected images. Alternatively, unitarians might downplay the challenge and deny that all of these pictures evoke genuine pictorial experiences. Downplaying has its virtues. A picture cannot trompe l’œil as long as we experience it as representing a similarity between its surface and the depicted scene. Nevertheless, there is room to doubt that trompe l’œil representations do evoke genuine pictorial experiences (e.g. Wollheim, 1987; see also Feagin, 1998). One might think that the benefits of a unified account of pictorial experience outweigh the cost of denying that the relatively small number of genuine trompe l’œil representations are pictures. The more serious challenge to the downplaying strategy comes from naturalist images. Experiences of naturalist images like Clark’s Velázquez do not represent design and scene at the same time, so they do not evoke experiences that represent a surface–scene similarity (since an experience that represents that similarity thereby represents the design). It turns out that Clark’s Velázquez does not evoke a genuine pictorial experience. That is a rather hard bullet to bite: naturalism is central to art.

Perception and Art   879

Fig. 45.3  Georges Seurat, Eiffel Tower (1889). Oil on panel 24.1 cm × 15.2 cm. Taking the challenge seriously is not plain sailing either. The task for unitarians would be to find a characterization of the content of pictorial experience that is abstract enough to embrace its phenomenological diversity and yet specific enough to capture what is pictorial in pictorial experience. The view that pictorial experience represents a surface–scene similarity will not do. What about Walton’s (1990) proposal that a pictorial experience is a rich imagining of one’s seeing the picture surface that it is an instance of seeing the depicted scene? Presumably, this imagining explains why we are not fooled by the illusion. By the same token, trompe l’œil does not involve imagining when, as sometimes happens, we really are fooled by the illusion (we reach for the grapes or cluck in admiration at the engineering feat of the apparent dome above us). The problem is that whether we imagine or do not imagine makes no difference to the phenomenology of the experience. Moreover, one may ask why it should be that for twofold images the content of the relevant imagining includes design features whereas for naturalist images it does not. And why is it that in quasi-naturalist images the relevant imagining represents design features that are not surface features? Perhaps these questions can be answered in a way that construes

880   Dominic McIver Lopes pictorial experience as unitary. If not, then the proposal to give a theory of pictorial experience pride of place begins to look less promising. Describing pictorial experience as non-unitary is not without its own challenges. One is to make sense of inflected images, wherein a full characterization of a picture’s scene-representing content must include properties of the design. Theories of unitary pictorial experience like those of Hopkins and Walton handily accommodate inflected images, since they imply that pictorial experience only represents the depicted scene as standing in some relation to the surface or design of the picture. Accommodating inflected images is less straightforward for those who take pictorial experience to be non-unitary. The problem is to say what it is for an experience to represent a scene in a way that cannot be fully characterized without reference to the design of a picture. One option is to hold that pictorial experiences of inflected images (but not all images) represent a surface–scene similarity. Another option explored by Bence Nanay (2010) leverages some ideas of Mohan Matthen’s (2005: ch. 13) to argue that inflected images recruit visual processing through the so-called dorsal and ventral pathways of vision in a special way. Historically, philosophers writing on pictorial art were interested in its value and paid scant attention to the details of what pictures are, how they represent, and how they figure in experience. The balance shifted radically under the influence of Gombrich and Goodman. For the past several decades, theories of pictures, depiction, and pictorial experience have hogged the limelight. Only recently has there been a return to work on the value of pictures (e.g. Lopes, 2005).

4 Pictorial art The task has been to gather materials for an account of how pictures realize value as works of pictorial art—that is, as vehicles for pictorial experiences. Abundant materials now lie at hand. Conceptions of pictorial art are flawed when they attempt to make do with pickings that are too slim. Many pictures are valued above all because they afford experiences that substitute for face-to-face experiences of the scenes they represent. They are, for example, mementos of family members, recordings of historical events, or portable practical guides. Their effectively serving in these roles depends on their imitative power, and this fact generalizes as the mimetic theory of the value of art pictures (known to its detractors as the ‘pretty girl theory’). According to this theory, the value of a picture is either identical to or determined by the value of what it depicts, as would be accessed in a face-to-face experience of what it depicts. Pretty pictures are ones that depict pretty looking things, informative history paintings are ones that reflect the facts that would have been taken in by an eye witness, practically useful images are those that depict opportunities for action that items afford and make visible to their users. Despite its popular appeal, the mimetic theory will not do. While the theory accounts for the value of some pictures, the value of a picture may not track the value accessible in a face-to-face experience of the depicted scene. Some scenes are not good to see except through pictures—they may be as banal as the pair of muddy work boots so evocatively portrayed by van Gogh. Some pictures are valuable because they depict opportunities for

Perception and Art   881 action that it would otherwise be hard to see an item affording (Lopes, 2004). Finally, the value of a picture—indeed, its perceptually-realized value—may consist in its having an edge on a face-to-face experience of its subject. We can, after all, discover things about the world through pictures. The mimetic theory goes wrong at least in part because it overlooks the value that may accrue to a picture in virtue of its surface features, and this diagnosis inspires the family of theories known as ‘formalism’ (e.g. Bell, 1914). A principal motivation for formalism was to accommodate abstract art—both pure or non-figurative abstraction (e.g. Mondrian’s grids) and figurative abstraction (e.g. cubism). The standard story is that abstract art met with a hostile reaction in the early twentieth century because audiences mistakenly judged abstract works by the standards of the mimetic theory. Acceptance of formalism may explain why Mondrians and Rothkos are now crowd-pleasers. Unsurprisingly, formalism modelled itself on theories of musical value designed with purely instrumental music in mind (e.g. Hanslick, 1980). Another motivation for formalism was to model a conception of artistic value as nothing more than aesthetic value and as excluding, in particular, the cognitive value that comes from a picture’s representing a scene in a way that conveys knowledge or insight. This marks a departure from the pluralist view that artistic values include cognitive values. Note that the stipulated conception of pictorial art as realizing value through perceptual experience does not imply formalist anti-pluralism. What is on the agenda is any kind of value realized by pictures as vehicles for perceptual experiences. Given its dual motivations, classical formalism conjoins two claims. First, the only artistic values realized by a picture are its aesthetic values. Second, the aesthetic values of a picture wholly supervene on its formal properties. What it depicts is irrelevant. Bell defined formal properties as ‘lines and colours combined in a particular way’ (1914: 17) though other conceptions of formal properties are possible (see Budd, 1995: 49–54). Even if it was once useful in counteracting the strong pull of the mimetic theory, formalism is no longer a live option. The restrictive identification of artistic value with aesthetic value has been challenged (see Gaut, 2005 and John, 2005). More importantly, powerful arguments would be needed to establish the thesis that the aesthetic value of a picture has nothing whosoever to do with the fact that it depicts a scene. The only argument that has been given is contentious at best. According to this argument, pure abstraction shows that depiction is not essential to imaging, so only the formal properties of pictures’ surfaces are essential to them. But the artistic values of a picture are those that are realized by its essential features. Therefore, the artistic values of a picture supervene on the formal properties of its surface. One problem with this argument is the assumption that the artistic values of a picture may be realized only by its essential features. Spice adds goodness to a dish even (and sometimes especially) when it is optional. Another problem is that non-abstract pictures form a kind whose members essentially evoke pictorial experiences. Why not reason that depiction is essential to figurative images so their artistic values supervene on their non-formal, representational features? Appearances to the contrary, the mimetic and formalist theories of the artistic value of pictures are not merely straw men, historical curiosities. Contemporary work either conjoins or disjoins them. Conjunctivists take a picture to realize its value as a vehicle for an experience of the depicted scene together with the picture’s formal properties. Disjunctivists take a picture to realize its value either as a vehicle for an experience of the depicted scene or as a vehicle for an experience of the picture’s formal properties.

882   Dominic McIver Lopes An example of the conjunctive strategy is Nanay’s (2014) proposal that the aesthetic value of a picture supervenes on properties that cannot be fully characterized without reference to the picture’s surface properties. The proposal nicely fits twofold and inflected images, which evoke experiences which cannot be fully characterized without reference to their surface properties. These pictures realize value as vehicles for experiences of something like a similarity between the scene on one hand and the ‘lines and colours’ of the surface on the other hand. Here again trompe l’œil throws a spanner in the works. Presumably their value as works of pictorial art can only be mimetic. Granted, an effective trompe l’œil image may be good partly precisely because we are aware that it controls our attention so as to divert it away from any awareness of the picture surface that would give away the illusion. That awareness may be an awareness of a surface, but its content is minimal. Rich awareness of the formal properties of that surface is imcompatible with the illusion. So, again, the options are to go disjunctive and allow that the artistic value of a trompe l’œil is mimetic or to deny that trompe l’œil images are valuable as works of pictorial art. Any temptation to prefer the latter option should be tempered by a look at naturalist images. In one limiting case of a naturalist image, only the experience of the depicted scene carries any interest. It is possible to switch from an experience of that scene to an experience of the design by means of which the scene is depicted, but making the switch is pointless. The design is humdrum and neutral. The value of this picture can be characterized without reference to its surface properties. All the same, naturalist images have been extolled as an artistic ideal. The disjunctive strategy need not come with a loss of explanatory power. The strategy views artists as equipped with a toolkit which gives them assorted ways to make pictures that are valuable as pictures. Their goal is to create a picture that is a vehicle for an experience and, as such, realizes some value. However, the experiences artists make pictures to evoke are not cut from a single cloth. They are eclectic: they include the kinds of experiences that characterize twofold images, but also naturalist and trompe l’œil images as well as pure abstractions, where there is nothing but the surface to see, nothing analogous to seeing a depicted scene. Goodman headlined his critique of the theory that pictures are copies of reality with Virginia Woolf’s wisecrack that there is no point in creating copies of reality: ‘one of the damn things is enough’. The lesson for philosophers interested in perception and art is that pictorial perception is not a redundant duplication of ordinary perceptual processes. It is a special application and extension of those processes and one way to get a fix on how it applies and extends perception is to ask about the value of making pictures. Answering that question means putting in time not only in the armchair and the laboratory but also the gallery.

References Abell, Catharine (2009). ‘Canny Resemblance’. Philosophical Review, 118, 183–223. Beardsley, Monroe C. (1983). ‘An Aesthetic Definition of Art’. In Hugh Curtler (ed.), What Is Art? (pp. 15–29). New Haven, CT: Yale University Press. Bell, Clive (1914). Art. London: Chatto and Windus.

Perception and Art   883 Bergeron, Vincent and Lopes, Dominic McIver (2011). ‘Aesthetic Theory and Aesthetic Science: Prospects for Integration’. In Steven Palmer and Arthur Shimamura (eds), Aesthetic Science: Connecting Minds, Brains, and Experience (pp. 63–79). Oxford: Oxford University Press. Budd, Malcolm (1992). ‘On Looking at a Picture’. In John Hopkins and Anthony Savile (eds), Psychoanalysis, Mind and Art (pp. 259–80). Oxford: Blackwell. Budd, Malcolm (1995). Values of Art: Pictures, Poetry, and Music. London: Penguin. Danto, Arthur C. (1964). ‘The Artworld’. Journal of Philosophy, 61, 571–84. Feagin, Susan (1998). ‘Presentation and Representation’. Journal of Aesthetics and Art Criticism, 56, 234–40. Gaut, Berys (2005). ‘Art and Ethics’. In Berys Gaut and Dominic McIver Lopes (eds), Routledge Companion to Aesthetics, 2nd edn (pp. 431–43). London: Routledge. Gombrich, E. H. (1961). Art and Illusion. Princeton, NJ: Princeton University Press. Gombrich, E. H. (1982). The Image and the Eye: Further Studies in the Psychology of Pictorial Representation. Oxford: Phaidon. Goodman, Nelson (1976). Languages of Art, 2nd edn. Indianapolis: Hackett. Hamilton, James R. (2007). The Art of Theater. Oxford: Blackwell. Hanslick, Eduard (1980). On the Musically Beautiful, trans. Geoffrey Payzant. Indianpolis: Hackett. Haugeland, John (1991). ‘Representational Genera’. In William Ramsey, Stephen Stich, and David Rumelhart (eds), Philosophy and Connectionist Theory. Hillsdale: Lawrence Erlbaum Associates. Hopkins, Robert (1998). Picture, Image, and Experience. Cambridge:  Cambridge University Press. Hopkins, Robert (2010). ‘Inflected Pictorial Experience: Its Treatment and Significance’. In Catharine Abell and Katerina Bantinaki (eds), Philosophical Perspectives on Depiction (pp. 151–80). Oxford: Oxford University Press. John, Eileen (2005). ‘Art and Knowledge’. In Berys Gaut and Dominic McIver Lopes (eds), Routledge Companion to Aesthetics, 2nd edn (pp. 417–29). London: Routledge. Kennedy, John M. (1993). Drawing and the Blind: Pictures to Touch. Yale: Yale University Press. Kulvicki, John (2006). On Images:  Their Structure and Content. Oxford:  Oxford University Press. Livingstone, Margaret (2002). Vision and Art: The Biology of Seeing. New York: Abrams. Lopes, Dominic McIver (1996). Understanding Pictures. Oxford: Oxford University Press. Lopes, Dominic McIver (1997). ‘Art Media and the Sense Modalities: Tactile Pictures’. Philosophical Quarterly, 47, 425–40. Lopes, Dominic McIver (2004). ‘Directive Pictures’. Journal of Aesthetics and Art Criticism, 62, 189–96. Lopes, Dominic McIver (2005). Sight and Sensibility: Evaluating Pictures. Oxford: Oxford University Press. Lopes, Dominic McIver (2008). ‘Nobody Needs a Theory of Art’. Journal of Philosophy, 105, 109–27. Matthen, Mohan (2005). Seeing, Doing, and Knowing:  A  Philosophical Theory of Sense Perception. Oxford: Oxford University Press. Nanay, Bence (2005). ‘Is Twofoldness Necessary for Representational Seeing?’ British Journal of Aesthetics, 45, 262–73.

884   Dominic McIver Lopes Nanay, Bence (2010). ‘Inflected and Uninflected Experiences of Pictures’. In Catharine Abell and Katerina Bantinaki (eds), Philosophical Perspectives on Depiction (pp. 181–207). Oxford: Oxford University Press. Nanay, Bence (2014). Aesthetics as Philosophy of Perception. Oxford: Oxford University Press. Newall, Michael (2011). What Is a Picture? Depiction, Realism, Abstraction. London: Palgrave Macmillan. Nichols, Shaun (ed.) (2006). The Architecture of the Imagination:  New Essays on Pretense, Possibility, and Fiction. Oxford: Oxford University Press. Peacocke, Christopher (1987). ‘Depiction’. Philosophical Review, 96, 383–410. Podro, Michael (1998). Depiction. Cambridge, MA: Harvard University Press. Schier, Flint (1986). Deeper into Pictures. Cambridge: Cambridge University Press. Shelley, James (2003). ‘The Problem of Non-Perceptual Art’. British Journal of Aesthetics, 43, 363–78. Siegel, Susanna (2011). The Contents of Visual Experience. Oxford: Oxford University Press. Walton, Kendall (1990). Mimesis as Make Believe: On the Foundations of the Representational Arts. Cambridge, MA: Harvard University Press. Wollheim, Richard (1980). ‘Seeing-as, Seeing-in, and Pictorial Representation’. In Richard Wollheim, Art and Its Objects, 2nd edn. Cambridge: Cambridge University Press. Wollheim, Richard (1987). Painting as an Art. New York and London: Thames and Hudson. Zeki, Semir (2000). Inner Vision:  An Exploration of Art and the Brain. Oxford:  Oxford University Press.

Author Index

The index was prepared by Steven Coyne Abbott, T.  111 Abell, C.  873 Adams, F.  290, 501, 700, 706, 735, 739, 741, 748, 768 Adams, J.K.  729 Adams, W.  688, 700, 706 Addis, L.  643 Adelson, E.  625, 701 Agostini, T.  631 Ahissar, M.  816–17, 828 Ahmad, M.  866 Airapetyantz, E.  357, 368 Aizawa, K.  221 Akins, K.  96, 109, 357, 368, 422, 434, 681, 741 Alais, D.  610, 616 Alexander, S.  69, 111–12 Allais, D.  320, 348 Allais, D.  320 Allison, T.  229 Allport, A.  589 Alter, T.  77, 128, 300, 308, 363, 373, 378, 385, 416, 441, 574, 590, 641, 650–1, 653, 730, 736, 788, 827, 856, 878 Altringham, J.  865 Amano, K.D.  632 Amaro, E.  538 Amedi, A.  662 Anand, T.  342 Andrew, C.  538 Angelaki, D.  361–2, 363, 368, 611, 616 Annan, V.  631 Annas, J.  69–70 Anscombe, E.  364, 368 Anstey, P.  406–7, 410, 418 Antony, L.  210, 417 Apkarian, A.  537 Appelbaum, I.  485, 490 Aquinas, T.  45, 51, 53–4, 56–8, 60–3, 314

Arbini, R.  92 Arend, L.  628, 629 Arendt-Nielsen, L.  864 Arman, A.C.  643 Armel, K.  306 Armstrong, D.  76, 119, 296, 298, 303, 355, 357, 368, 416, 533, 557, 745 Arnauld, A.  91–3 Arno, P.  661, 662 Arnoldussen, A.  661 Aronson, K.  360 Arstila, V.  269 Arzy, S.  520–1, 525, 527 Aschersleben, G.  230 Ashburner, J.  668 Asher, J.  643 Ashwin, C.  643 Atherton, M.  89, 409 Audi, R.  274, 483, 487, 490, 496, 576, 661, 663, 667, 784, 786 Austerweil, J.  825, 828 Austin, J.  119, 123, 127, 129, 134, 202, 229, 269, 314, 549, 594, 747 Auvray, M.  314, 321, 323, 337, 348, 578–9, 582–3, 604–5, 616, 640, 654, 660–2, 669–72, 771 Avotins, I.  34 Aydede, M.  368, 532, 534, 580, 582–3, 747, 749 Ayer, A.  71, 118–19, 123–7, 201, 781, 836–7 Ayers, M.  836–7 Aziz, Q.  538 Baars, B.  385, 570, 583 Bach, K.  156, 161, 502, 582, 659–61, 669 Bach-y-Rita, P.  672 Backonja, N.M.  437 Bain, A.  111, 301 Bain, D.  533–4

886   author INDEX Balcetis, E.  765, 767 Baldwin, T.  301 Ball, P.  158, 286, 664 Ballard, D.  158, 664 Baltussen, H.  32 Banaji, M.  773, 804 Bandelr, R.  538 Bandura, A.  537 Banks, W.  610, 729 Bantik, S.  538 Bao, M.  815 Baraldi, P.  540 Barbet, I.  634 Bargary, G.  652 Bar-Hillel, Y.  734, 737 Barnes, J.  69–70 Baron-Cohen, S.  640–1, 643, 771 Barrett, H.  756, 774 Barrett, L.  359–60, 368 Barsalou, L.  218, 679, 814 Bartoshuk, L.  327, 334, 337, 348, 579 Barwise, J.  735 Batty, C.  162, 343–4, 417, 747, 854 Bauer, R.  375 Baumgartner, U.  864 Bavelier, D.  827 Bayle, P.  69–70, 73, 77 Bayne, T.  314, 318, 479, 603, 615–16, 649, 755, 766, 769–70, 834 Beardsley, M.  872 Beare, J.  32 Beck, J.  548, 629 Bedford, F.  614 Beebe, J.  801 Behrmann, M.  817 Beierholm. U.  706 Bell, C.  562, 881 Bender, M.  307, 316 Bennett, J.  91, 397, 413 Benson, P.J.  223 Bergeron, V.  755, 759, 764, 772, 872 Bergholt, B.   378–9 Bergson, H.  461, 548–52, 556–7, 562 Berkeley, G.  70, 84–5, 87–96, 102–3, 111, 120–1, 131–2, 268, 275–6, 434, 542, 605 Bermpohl, F.  662 Bermudez, J.  301, 308, 367–8, 523

Berntson, G.  360 Berridge, K.  329, 373, 381 Bertelson, P.  157, 319, 320, 608–9 Bett, R.  66 Bettucci, D.  223 Bhalla, M.  224 Bharucha, J.  680, 688 Bhatt, R.  812 Bicchi, A.  664 Bigand, E.  508, 680 Billington, J.   643 Binkofski, F.  223, 472 Biver, S.  621 Blackwell, K.T.  629 Blake, A.  317, 335, 337, 338 Blake, R.  227, 314, 652, 813 Blakemore, C.   813, 827 Blakemore, S.-J.   227, 233 Blangero, A.  224 Blanke, O.  363, 368, 516, 518, 519, 520, 521, 522, 525, 527 Blauert, J.  281, 286 Bliss-Moreau, E.  360 Bleckmann, H.  869 Blix, M.  298 Block, N.  192–3, 203, 208, 221, 225, 229, 246, 302, 357, 368, 375, 379, 385, 590, 599–600, 650, 653, 665, 670, 731–2 Block, R.  472 Blomqvist, A.  535 Bloom, P.  397, 399, 402, 828 Bloom, P.  402 Blore, F.  662 Blumenthal, H.  317, 335, 337, 338 Boghossian, P.  314, 414 Boisson, D.  224 Bol, A.  672 Bolton, M.  92–3 Bonato, F.  631 Bonfils, P.  816 BonJour, L.  77, 782, 786, 790, 792, 800 Bonnier, P.  304 Bor, D.  643 Bordier, C.  643 Borg, E.  277 Boring, E.  102, 107, 109, 298 Bornhovd, K.  537

author INDEx   887 Bottelier, E.H.  647 Botvinick, M.  173, 305–6, 367–8, 515–17, 608 Bouguer, P.  109 Bourget, D.  251 Bourne, C.  462 Boyle, R.  82–3, 101, 405–7 Boynton, G.M.  643 Boynton, R.  109–10 Bradley, P.  622, 701 Bradshaw, J.L.  640, 642 Brainard, D.  624, 629, 631, 634, 698 Brammer, M.  538, 651 Brandl, J.  137 Brandom, R.  177 Brang, D.  643 Brasil-Neto, J.P.  668 Braun, D.I.  633 Bregman, A.  285–8, 486, 495–7, 499 Bremner, A.J.  606, 613 Brenner, E.  629 Brentano, F.  47, 110–11, 136–40, 142–4, 149, 298–9, 303–4 Breslin, P.  332 Brewer, B.  154–5, 181–5, 188–91, 193, 199, 204, 210, 212, 448–9, 703–4, 794 Brillat-Savarin, J.  323 Brinkman, C.  643 Briscoe, R.  446 Brisson, L.  40 Broad, C.  119, 201, 229, 465, 568, 588, 723 Broadbent, D.  588 Brocklebank, D.  643 Brogaard, B.  153, 156, 237, 241, 245, 247–51, 397, 849 Bromm, B.  537 Brooks, R.  155 Brown, E.  634 Brown, P.  469 Brown, T.  103, 111, 301, 451 Brueckner, A.  802 Brugger, P.  521, 525, 761 Brugger, S.  761 Bruner, J.  766 Bruno, N.  156 Brunt, W.A.  631 Bryant, B.  346 Bubb, D.  822

Bubic, A.  662 Buchanan, B.  862 Buchel, C.  537 Buchsbaum, G.  629, 631 Budd, M.  873, 876, 881 Bukach, C.  822 Bullmore, E.  538, 651 Bullock, T.  863 Bulthoff, H.H.  224 Burge, T.  154, 174, 210, 452–3, 622, 695, 698, 701–3, 705, 708–9, 784–5, 789, 794 Buridan, J.  52 Burns, B.  814, 823 Burnyeat, M.  41, 43, 57 Burr, D.  320, 348, 610, 616 Bushnell, M.C.  535, 537 Butterfill, S.  765 Butterworth, B.  642 Bykov, K.  357, 368 Byrne, A.  154, 183, 198, 202–4, 206–7, 210–11, 241, 250, 624, 629, 746 Cacioppo, J.  360 Caetano, G.  660 Cain, W.S.  579 Callaway, E.  764 Calvert, G.  319, 608 Cameron, O.  353, 358 Cammmarota, A.  668 Campbell, A.L.   863n20, 867 Campbell, J.   24, 154, 170, 179, 210n36, 214, 449, 453, 457, 703, 714, 789, 797n40, 798n43, 805, 833n2, 836n9, 837n12, 838nn13, 15, 16, 845n29, 846n32 Campbell, K.   156, 163 Camprodon, J.A.  662 Capelle, C.  661 Carell, T.D.   477, 485, 492 Carey, S.  399, 571, 822 Carlson, T.  366 Carlsson, K.  538 Carman, T.  149 Carnap, R.  734, 737 Carr, C.  353–5, 360, 385, 546, 560, 567, 599–600, 756–7, 860 Carrasco, M.  599–600 Carrier, B.  537

888   author INDEX Carriere, J.S.A.  642 Carruthers, P.  353–5, 360, 385, 567, 756–7 Casati, R.  162, 281, 291, 393, 402, 542, 553–4, 562, 844 Casati, R.  402, 553–4, 562 Case, T.  111, 314, 319, 475, 537, 567, 581, 742–4, 746, 841 Casey, K.  314, 319, 475, 537, 567, 581, 746 Cassam, Q.  301 Caston, V.  29–30, 33, 43, 46–7, 781 Casullo, A.  444 Cataliotti, J.  631 Catmur, C.  608 Cattell, J.  821 Cavanagh, P.  191, 591 Cazier, J.B.  643 Celnik, P.  662 Chabris, C.  228, 592, 796 Chalmers, D.  156, 160, 204, 228, 248, 423, 613, 615–16, 706–7 Chambers, L.  317, 335, 337, 338 Changeux, J.-P.  228 Chapanis, A.  633 Chappell, V.  92–3 Chapuis, D.  522 Charles-Dominique, P.  859 Chatton, W.  54–5 Chen, D.   315, 349 Chen, J.I.   537, 539 Chen, L.   814, 819 Chen, Y.-C.   608, 617, 619 Chernigovskiy, V.  355 Chersi, F.  230 Chisholm, R.  70–1, 238–9, 242, 781, 787 Chomsky, N.  226, 244, 475 Chopping, S.  653 Chowning, J.  281 Christoff, K.  516, 524 Chu, S.  328, 705, 821 Chun, M.M.  591 Churchland, P.  705 Clapp, L.  417 Clare, S.  537, 538, 539, 540 Clark, A.  155, 161, 225, 228, 403, 404, 412, 664, 672, 705, 709, 714, 737, 740, 747 Clark, J.  388 Clarl, S.  470

Clement, C.I.  538 Cleeremans, A.  378–9 Clifford, E.   379, 796 Clore, G.  360 Coburn, R.  302–3 Cock, J.  642 Coen, S.  538 Cohen, H.  362 Cohen, J.  104, 161, 173, 305–6, 314, 322, 367–8, 397, 423, 515–17, 608, 621–3, 629, 634, 640–1, 643, 736–7, 739, 742, 760, 771, 845 Cohen, L.G.  636, 662, 668 Cohen, M.A.  591 Cohen, S.  787, 788, 789, 800, 802 Coile, D.C.  860 Coles, R.B.  853 Collignon, O.  672 Collings, V.  332 Collins, C.  659, 660, 672, 757, 768 Collins, J.  642, 672, 757, 768 Coltheart, M.  381, 759 Conee, E.  534, 789, 801 Cooke, D.  308 Cooper, F.S.  485, 493 Cooper, G.F.   813, 827, 829 Cooper, J.   41, 49 Corbeille, J.  382 Cornelissen, F.  629 Corwell, B.  662 Coslett, H.  304, 366 Cosmelli, D.  516, 524 Cosmides, L.  756–7, 774 Costantini, M.  173 Costello, P.  377, 379 Coulson, S.  643 Cowey, A.  267 Cowles, R.  863 Craig, A.  230, 354–5, 535, 572, 604 Craighero, L.  230 Crane, T.  154, 160, 205, 644, 835 Crathorn, W.  58–9 Craven, B.  632 Crepaldi, D.  381 Crick, F.  382, 813 Crisp, T.  461 Crist, R.E.  816 Cronholm, B.  307, 607

author INDEx   889 Cronly-Dillon, J.  662 Crowther, T.  182 Cruz, A.  332, 786 Cruz, J.  332, 786 Csibra, G.  231 Cudahy, C.  642 Cui, X.  471 Cullen, K.  361–2, 368 Cullen, S.  538 Cummins, R.  739, 744, 749 Curley, E.  411 Curran, T.  818 Cytowic, R.  604, 640, 642–5 Czech-Damal, N.U.  853 Dainton, B.  461, 463–5, 467, 615 Dallenbach, K.  572 Dalton, J.  428 Damasio, A.  360 Dambrosia, J.  662 Dancer, C.  304 Danilov, Y.  660, 661 Dannemiller, J.  632–3 Danto, A.  743, 872 Daprati, E.  156 David, M.  160, 181, 184–7, 217, 243, 251, 257, 397, 423, 550–1, 557, 735, 781, 790 Davidson, D.  184–6, 217, 223, 243, 397, 781, 790 Davidson, R.J.  437 Davies, I.R.L.  819 Davies, M.  204, 495, 702 Davies, S.  204, 495, 702 Davis, K.D.  536 Day, J.  33, 39 Day, S.  605, 642, 681 de Gelder, B.  606 De Volder, A.G.  672 DeAngelis, G.  363, 611 Decety, J.  223 DeGelder, B.  319–20 Degenaar, M.  89, 605 Degos, J.-D.  636 Dehaene, S.  228, 376, 381, 385, 636, 839 Deiber, M.P.  662 Delahunt, P.  629, 631 Delk, J.  766–7

Demir, H.  737 Denery, D.  52, 55 Dennett, D.  161, 170, 187, 471, 538, 705, 736, 745, 841 Depaulis, A.  538 DeRose, K.  801 Deroy, O.  314, 344, 567, 582, 605, 611, 640, 642, 649, 651, 669, 755, 761, 765–6, 770–1 Derrick, D.  487 Descartes, R.  67, 69, 71–4, 77, 81–9, 91–6, 101–3, 108, 219, 268, 306, 405, 407, 424–6, 434, 595, 659, 857 Desimone, R.  386 Desmurget, G.   224 Deutsch, D.  287, 687 deWeert, C. M. M.  629 Diamond, R.  822 DiCarlo, J.  816 Dick, F.  489 Dickie, I.  833, 837–8, 843, 848 Dienes, Z.  771 DiLollo, V.  385 Dilthey, W.  301 Dixon, M.J.  642 Dojat, M.  643 Dokic, J.  162, 184, 189, 281, 291, 441 Dolan, R.  344 Dold, G.  662 Don, S.  304, 516, 608 Donaldson, H.  298 Donnadieu, S.  500–1 Dorpat, T.  357 Dosey, M.  308 Dostrovsky, J.O.  536 Dowman, R.  537 Downes, J.  328 Downing, L.  407–9, 411 Dretske, F.  203, 217, 227, 229, 276, 298, 383, 534, 557, 562, 650, 734–50, 785, 788, 797, 801–2 Dreyfus, H.  149 Drga, V.  730 Driver, J.  157, 487, 608, 611, 769–70 Dueker, G.L.  812 Duffy, V.  327, 337, 348 Dulany, D.  728

890   author INDEX Dummett, M.  836, 840–1 Duncan, G.H.  537 Duncan, J.  589 Dunning, D.  765 Durie, B.  604 Dworkin, B.  567, 570, 571 Dworkin, S.  567, 570, 571 Eagleman, D.  471, 472, 473 604 Ebert, T.  47 Economou, E.  631 Eddy, M.  382 Edgar, S.  104 Edquist, J.  643 Egan, F.  708, 710, 712 Egeth, H.E.  599 Ehrsson, H.  515, 517, 521–2 Ehrsson, H.H.  614 Eich, E.  518 Eilan, N.  347 Eller, M.  642 Endres, C.S.  853 Engel, S.A.  815 Enns, J.  385 Epstein, W.  101, 108 Erdelyi, M.  719 Ereshefsky, M.  583 Ernst, M.  100, 319, 610, 700–1, 706, 718, 858 Evans, F.  536 Evans, G.  170, 176–7, 181–3, 185, 189–90, 194, 277, 367, 442, 446–7, 450, 452–5, 575, 647, 745, 834–7, 840–1, 843, 846 Evans, R.  629 Everitt, M.  533 Facchin, P.  540 Fagot, J.  634 Fahle, F.  224 Fahle, M.  815 Fairchild, M.  686 Fales, E.  298 Falkenstein, L.  103, 443–4 Falz, L.  662 Fang, F.  377, 379 Farah, M.  265, 822 Farina, M.  577, 654, 659, 669

Feagin, S.  878 Fechner, G.  104, 106–7, 640–1, 682, 718 Felh, K.   378–9 Feldman, R.  787, 789, 794, 801, 803 Felsted, J.  337 Fernandez-Duque, D.  373 Ferrari, P.F.  230 Fetsch, C.  363 Feyerabend, P.  760 Field, D.  708–9, 815 Field, H.  708–9, 815 Fillenbaum, S.  766–7 Findlay, J.  162 Fine, G.  39 Fink, M.  307 Fish, W.  209, 211–12, 644, 854 Fitch, W.T.  475 Fletcher, D.  280, 661 Flew, A.  124 Floridi, L.  735 Flynn, C.  817 Fodor, J.  219–20, 490, 611–12, 696, 708–9, 734, 737, 739, 745, 749–50, 755–60, 762–5, 770–2, 774, 804, 813 Fogassi, L.  230, 231 Foley, L.E.   860, 866, 868 Foley, R.  737, 751 Fornari, E.  522 Foster, D.  203, 632, 634 Foster, J.  203, 632, 634 Fowler, C.  486, 489 Frackowiak, R.S.J.  668 Francis, G.  767 Frankenstein, U.N.  538 Franz, V.H.  224 Frasnelli, J.  330 Frasser, C.  642 Frede, M.  30, 41, 140, 294, 368, 442 Freeman, W.  631 Frege, G.  156, 423, 598, 613–14, 702, 711, 790, 836 Fregni, F.  324 French, A.  145, 238, 280, 283, 447, 503, 542, 544, 548, 767 Freund, H.J.  537 Fries, P.  386 Friston, K.  664

author INDEx   891 Frith, C.D.  668 Fritz, K.  35 Fujita, K.  634 Fukuyama, H.  538 Fulkerson, M.  296, 314, 336, 567, 578, 581 Fumerton, R.  70–1 Funkhouser, E.  161 Fuqua, P.  621 Furmanski, C.S.  815 Furness, D.  284 Fytche, D.H.  653 Gadian, D.G.  668 Gale, R.  36, 549 Gallace, A.  311, 516, 572, 614 Gallagher, S.  149, 173, 365–6 Gallese, V.  230, 231, 232 Gallistel, C.  456, 567, 570–1, 573, 755 Galton, F.  640, 642 Galvin, S.J.  730 Gandevia, S.  364 Gandhi, T.  606 Ganel, T.  224 Ganesh, S.  606 Ganson, T.  38 Gao, T.  491 Garbart, H.  599 Garcia, D.F.  642 Garner, W.  823 Garnett, A.  301 Gassert, R.  522 Gati, J.S.   538–40 Gaucher, P.  610 Gaut, B.  814–15, 818, 822, 881 Gauthier, I.  814, 815, 818, 822, 829 Gauthier, J.  537 Gaver, W.  281 Gegear, R.J.  866 Gegenfurtner, K.R.  224, 633, 634, 767 Gehler, J.  105 Geisler, W.  694, 700 Gelade, G.  382, 576, 598 Gelman, S.  571, 759 Gelstein, S.  345 Gentili, C.  664 Gentilucci, M.  156 Gergely, G.  232

Gert, J.  622 Gescheider, G.  719 Gesierich, B.  230 Geurts, K.  859 Ghahramani, Z.  367, 524 Ghazanfar, A.  318, 611 Giampietro, V.  538 Gibson, B.  393n1, 817 Gibson, B.S.   306, 310, 316 Gibson, E.J.   816, 829 Gibson, J.J.   24, 101, 113, 114, 133, 135, 164, 169, 179, 187, 196, 218, 234, 306, 310, 316, 402, 637, 695, 715, 816, 817 Gick, B.  487 Giger, R.  330 Gilbert, C.D.  816 Gilchrist, A.  631–2 Gilchrist, I.  112, 162 Gillam, B.  156 Gillebert, C.R.  826 Ginet, C.  748 Giralt, N.  402 Gitelman, D.  337 Giudice, N.A.   663, 665, 672 Gjedde, A.  662 Glanzer, M.  729 Glauche, V.  537 Glover, L.  642 Goebel, R.  376 Godfrey-Smith, P.  741, 750 Goethe, J.  105 Gold, J.  76–7, 230, 232, 298, 397, 624, 679, 790, 793, 800, 803, 812–14, 819–21, 823–8 Goldman, A.  76–7, 230, 232, 397, 790, 793, 800 Goldscheider, A.  298 Goldstein, E.  624 Goldstein, L.  771 Goldstein, L.H.  640, 641 Goldstein, R.  629 Goldstone, R.  679, 803, 812–14, 819–21, 823–8 Golz, J.  631 Gombrich, E.  402, 872–3, 875–6, 880 Gonzalez, A.E.  642 Good, C.D.  668

892   author INDEX Goodale, M.  156, 221, 223–5, 267, 325, 367, 382, 454, 568, 666, 764 Goodman, C.  443, 737, 747, 766, 872, 874, 880, 882 Goodman, N.  443, 737, 747, 766, 872, 874, 880, 882 Gossard, D.  537 Gottfried, J.  314, 343–4, 762 Grabul, D.S.  610 Gracely, R.H.   537 Graf, E.W.  668, 700, 706 Graff, D.  691 Grafman, J.  662 Graham, G.  533 Graham, P.  533 Grahek, N.  534 Grandy, R.  836 Granit, R.  432 Granrud, C.  634 Grassmann, H.  109, 683 Gray, F.  636 Gray, J.A.  641, 655 Gray, R.   641, 644, 650, 653, 855n7, 863, 864, 865, 868 Graziano, M.  308 Grea, H.  224 Greco, J.  802 Green, B.  332, 334 Green, D.  437, 577 Green, K.  717, 719, 726 Greffrath, W.  864 Gregoric, P.  47 Gregory, L.J.  538, 653, 735 Gregory, R.L.  224, 735 Grellard, C.  62 Grice, H.  269, 417–18, 531, 567, 574–6, 579–80, 734, 741, 747, 857, 864 Griffith, B.C.  689, 820 Griffiths, P.  825, 828 Griffiths, T.  825, 828 Grossenbacher, P.  605, 640, 642, 645 Grossweiler, R.  224 Grush, R.  461, 471 Gu, Y.  363, 611 Guarniero, G.  669 Guazzelli, M.  664

Guest, S.  608 Guppy, A.  853 Gupta, A.  785 Ha, B.  537 Hackney, C.  284n30 Haggard, P.  173, 307, 366, 376, 469, 470, 516, 517 Hagoort, P.  652 Hall, R.  89, 200, 535, 797 Hallett, M.  662, 668 Halpern, A.  330, 498 Halsey, R.  633 Hamilton, J.R.  871 Hamilton, R.   619, 663, 675 Hamilton, W.  100, 103, 111, 114 Hamlyn, D.  301 Hanakawa, T.  538 Hanke, F.D.  853 Hanko, S.  633 Hanna, R.  186 Hanneton, S.  661 Hansen, T.  634, 676, 858 Hanslick, E.  881 Hardcastle, V.  530, 532, 534, 538, 749 Hardin, C.  414–15, 417, 426, 435–6 Hardin, L.  414–15, 417, 426, 435–6 Hari, R.  660 Harman, G.  203, 534, 593, 739 Harnad, S.  482, 687, 689–90 Harris, K.S.  689, 820 Harrison, G.E.  640, 641 Harrison, J.  643, 771 Hartline, H.  432 Hastings, M.C.  853 Hatfield, G.  86, 89, 100–6, 108–12, 417, 622, 690, 735 Haugeland, J.  748–9, 873 Hauser, M.D.  475 Hawthorne, J.  802 Haxby, J.V.  664 He, S.  377, 379 Head, H.  303, 304, 365, 535, 552 Heal, R.  469 Heard, P.  224 Heck, R.  182, 185–7, 189–90, 193–4, 745, 763, 790

author INDEx   893 Heeley, D.W.  223 Heidegger, M.  146, 149, 275–6 Heidelberger, M.  104, 107 Heider, F.  491 Heil, J.  325, 417, 575, 860–1 Heiligenberg, W.  860–1 Heim, I.  241 Heintz, J.  739 Heise, D.  679 Held, R.  606 Heller-Roazen, D.  52, 856–7 Hellie, B.  154, 746 Helm, B.  100, 104, 106, 109–12, 227, 424, 429–31, 434–5, 535, 631, 633, 682, 695 Helmholtz, H.  100, 104, 106, 109–12, 227, 424, 429–31, 434–5, 631, 633, 682, 695 Hendrickson, A.  820, 826 Hering, E.  100, 104, 106, 110, 112, 430–1, 435, 544, 633, 682–4 Herz, R.  765 Heydrich, L.  522 Heywood, C.A.  385 Hilbert, D.  257, 407, 622, 624, 629, 746 Hilford, A.  729 Hill, C.  108, 503–5, 533, 734, 737, 769 Hillebrand, F.  108 Hinton, J.  210, 217, 251 Hirashima, M.  307 Hirsch, E.  397, 759 Hirschfeld, L.  759 Hlushchuk, Y.  660 Hobbes, T.  72, 82–6, 92, 405–7 Hobhouse, L.  111 Hobson, A.R.  538 Hochel, M.  642 Hochstein, S.  816–17, 828 Hodl, W.  610 Hofbauer, R.K.  537 Hoffman, D.  58–9, 102, 394, 397, 820 Hoffman, H.S.  689, 820 Hoffman, P.  58–9, 102, 394, 397, 820 Holcomb, P.  382 Hollowood, T.  333 Holmes, G.  303–4, 365, 535 Holmes, N.P.  606, 613 Hommel, B.  289n40

Honda, M.  538 Hong, S.  314, 441, 652 Hogendoorn, H.  366 Hopkins, R.  873–4, 876–7, 880 Hopp, W.  149 Horgan, T.  706 Horschig, J.M.  767 Hort, J.  333, 763 Horton, J.  763 Howard, L.A.  304 Huang, L.  589–91, 593, 597, 816 Huang, M.  377, 379 Hubbard, E.  640, 642–3 Hubel, D.  432–4, 437, 734–5 Huemer, M.  201, 785, 789 Hugentobler, M.  330 Hughes, H.  572, 854, 866 Hume, D.  93, 103, 111–12, 120–1, 130–1, 168, 174–5, 209, 275, 298, 424, 550–1, 575, 577, 791 Hummel, T.  314, 325, 330, 336, 344, 346 Humphreys, G.  161 Hunt, R.  621, 686 Hunter, F.  621 Hupe, J.M.  643 Hurlbert, A.  632, 634 Hurley, S.  225–7, 513, 607, 669, 671 Hurvich, L.  110, 431–2, 544, 632–3, 682 Husserl, E.  136, 138–44, 148–9, 364, 546, 551, 562 Hussey, E.  30 Hutchinson, I.  343 Hutchison, W.D.  536 Huttenlocher, P.  662 Hyman, J.  533 Ibanez, V.  662 Ierodiakonou, K.  33 Ikeda, K.  331 Ingle, D.  222, 634 Ingvar, M.  538 Inhelder, B.  606 Ionta, S.  522 Israel, D.  735, 739–40 Ito, T.  360 Izard, C.  359

894   author INDEX Jackendoff, R.  475, 488, 502 Jackson, F.  119, 132, 203, 239, 242, 246, 269, 299, 305, 423 Jacob, F.  403 Jacob, P.  156, 161, 217, 225, 229, 232, 432, 454, 648, 712, 728, 736, 749, 764, 860 Jacobs, R.  432, 712, 860 Jacoby, L.  728 Jacovides, M.  407–8 James, W.  100, 109–10, 112, 118, 303–5, 359–60, 363, 373, 431–2, 463, 550, 569, 587–8, 599, 622, 632, 682, 747 Jameson, D.  110, 431–2, 632, 682 Jansari, A.  642 Jansaria, A.  649 Jastrow, J.  760–1 Jeannerod, M.  156, 161, 223, 225, 227, 229, 232, 454, 764 Jesse, A.  371, 614, 670, 750 Jiang, Y.  377, 379 Jimenez, R.  796 Jings, A.L.  343 Johansen, T.  44 Johansen-Berg, H.  538 Johansson, R.  295 Johnsen, S.  853, 866 Johnson, A.  572 Johnson, B.  342 Johnson, K.  295 Johnson, M.  231, 325, 338, 340, 542–3 Johnson, M.  231, 325, 338, 340 Johnsrude, I.S.  668 Johnston, M.  161, 200, 202–3, 207, 211, 315, 789, 798–9 Johnston, R.  161, 200, 202–3, 207, 211, 315, 789, 798–9 Johnston, R.S.  223 Johnston, W.  161, 200, 202–3, 207, 211, 315, 789, 798–9 Johnstone, T.  537 Jolley, N.  90, 92 Jonas, C.  771 Jones, A.K.   537 Jones, J.  577 Jonsson, F.  347

Jousmaki, V.  660 Jordan, D.  690–1 Jordan, M.I.  367, 524 Jordan, T.R.  223 Jorge, P.E.  866 Judd, D.B.  683 Julliot, C.  859 Juola, P.  764 Jurgens, U.M.  642 Jusczyk, P.  482 Kahn, C.  31, 192, 398 Kahneman, D.  192, 398 Kaiser, P.  109–10, 140 Kalaida, N.  732 Kalderon, M.  210 Kalmijn, A.  853 Kalogeras, J.  470 Kamitani, Y.  609, 612 Kammers, M.P.M.  366, 516 Kanafuka, K.  288 Kanda, M.  538 Kant, I.  100, 103–4, 144, 172, 176–7, 181, 185–6, 314, 441–2, 546, 549 Kaplan, D.  156, 798, 840 Karlson, P.  315 Katz, D.  296, 299–301, 306, 632–3 Kaufman, R.  534 Kawatto, M.  227 Keay, K.A.  536 Keeley, B.  567, 569, 572–3, 641, 646, 664–8, 853–5, 858–60, 863 Kellman, J.  401 Kellman, P.  401 Kelly, J.  463, 503–5 Kelly, S.  149, 161, 182, 189 Kemler, D.  823 Kennedy, J.M.  872, 883 Kennedy, M.   207, 214 Kenshalo, D.R.  537 Kentridge, R.  385, 588 Kepler, J.  85, 101, 260, 264 Kercel, S.  660 Kerrigan, I.S.  668 Kersten, D.  400, 694, 698 Kessen, W.  816

author INDEx   895 Kessler, E.  688 Khan, R.M.  342 Kim, J.  220, 747 Kim, K.  729 King, A.  281, 284–5, 290, 319, 570 King, P.  59 Kingstone, A.  619 Kinsbourne, M.  471 Kirk, R.  385 Kish, D.  666, 862 Kiss, Z.H.T.  536 Kistler, M.  742 Kitazawa, S.  306, 308 Kitcher, P.  103 Klatzky, R.  578, 663, 665, 672 Klauer, G.  853 Klein, C.  535 Klein, S.A.  826 Knill, D.  697, 700, 706, 709 Koch, C.  382, 813 Koffka, K.  112 Kohler, I.  607 Konishi, J.  538 Korcz, K.  784 Kording, K.  619, 700 Koriat, A.  360 Kossifydis, C.  631 Koster, E.P.  331 Kourtzi, Z.  816 Kraft, J.  629, 631, 634 Kral, A.  667 Kretzmann, N.  62 Kriegel, U.  161 Kringelbach, M.  340 Kripke, S.  372, 531, 595, 787, 802 Kroliczak, G.  224 Krumhansl, C.  687–8 Kubota, K.  662 Kuffler, S.  432 Kuhl, P.  577 Kuhn, T.  557, 760 Kulvicki, J.  162, 275, 734, 743–6, 872–5 Kumru, H.  324 Kupers, R.  662 Kurzban, R.  756 Kwan, D.  642

Lackey, J.  77 Lackner, J.  364 Lacroix, J.S.  330 Lagerlund, H.  61, 68 Laing, D.  343 Lamb, J.A.  643 Lamme, V.  191–2, 384 Lampignano, D.  817 Land, E.  330, 368, 631, 697–8, 700, 827 Landini, L.  664 Landis, B.  330 Landis, T.  363, 521, 525 Landy, D.  697–8, 700, 827 Landy, M.  619, 697, 698, 700 Lane, H.  251, 820 Lange, C.  359–60, 629 Lange-Malecki, B.  629 Langmer, C.  853 Lanz, M.  642 Larsen, J.  360 Larson, R.  240–1 Lasek, K.  223 Laurence, S.  193, 195, 790 Law, C.  344, 827 Lawless, H.  344 Leask, M.  866 Lederman, S.  578 Lee, H.  631, 652 Lee, T.  664 Leech, R.  489 Legrand, D.  367, 516, 523, 524 Lehrer, K.  784, 800 Leibniz, G.  87, 90–1, 102–3, 605, 640 Lenay, C.  661 Lenggenhager, B.  173, 515, 517, 521, 522 Lennon, T.  92–3 Lerdahl, F.  502–3, 505, 507–8, 510 Lesher, J.  30 Leslie, A.  759 Levi, D.M.  826 Levin, D.  189, 228, 373, 414, 451–2, 773, 819 Levin, R.   804 Levine, J.  189, 414 Levinson, S.  451–2 Levithan, C.A.  765

896   author INDEX Lewis, C.  70–1 Lewis, D.  737, 744 Li, W.  343, 816 Li, X.  631 Liberman, A.  230, 475, 485–6, 490, 689, 820 Liebschner, A.  853 Lightfoot, N.  822 Lim, J.  325, 334, 338, 340 Lindberg, D.  52, 55, 101, 260 Ling, Y.  634 Link, C.  343 Linkenauger, S.  366, 804 Lipps, T.  230 Livermore, A.  346 Livingstone, M.  433, 437, 561, 878 Lloyd, D.  43, 304, 538, 608, 747–8 Lloyd, G.  43, 747–8 Loar, B.  156 Locke, J.  69, 82–5, 87, 90, 93–6, 101–3, 111, 120–1, 131–3, 318, 405–11, 414, 426–7, 435, 575, 605, 633, 640, 744 Lockhead, G.  812 Lockman, J.  304 Loewer, B.  736–7, 748 Logue, H.  148, 154, 198, 202, 204–7, 210–12, 644, 799 Lohmann, C.M.F.  853 Lohmann, K.J.   853n3, 866, 868 Lombardi, O.  738 Lomber, S.  667 Long, A.  68, 366, 516, 538, 661 Long, G.  761 Longe, S.E.  538 Longere, P.  629, 631 Longo, M.  366, 516 Longo, M.R.  516 Lonner, P.  347 Loomis, J.  297, 663, 665, 672 Lopes, D.  613, 871–3, 875, 877, 880–1 Loschky, L.C.  382 Lotze, H.  106, 303, 306, 443 Lovelace, C.  605, 640, 642, 645 Lu, Z.  853 Luft, S.  149, 281 Lufti, R.  281 Luscher, M.  315 Luxenburg, E.  343

Lycan, W.  75, 162, 278, 315, 644, 650, 785 Lyons, J.  193, 789 MacDonald, J.  576 MacFarlane, J.  314, 334 Mach, E.  100, 104, 110, 112, 219, 625, 627, 769, 858–9 Machery, E.  219, 769 Mack, A.  377, 379, 547, 744 Mackie, J.  744 Macknik, S.  547 MacLeod, D.  631, 689 Macpherson, F.  298, 314, 490, 567, 574, 604, 614, 641, 643, 645–6, 653, 761, 763, 766–7, 813, 854–5 Maestrini, E.  643 Maguire, E.A.  668 Maieron, M.  540 Majid, A.  767 Mak, E.  337 Malach, R.  664 Malcolmson, K.A.  642 Malebranche, N.  81, 83, 85, 87–90, 92–6 Maloney, L.  697, 698, 700, 706 Mamassian, P.  400, 697, 698, 700, 706 Mann, D.A.  853 Marcel, A.  314, 325, 328, 368, 555 Marcel, T.  367 Margolis, J.  193, 195, 304 Maris, S.  672 Marks, L.  642 Marler, P.  611 Marr, D.  113, 187, 192, 217, 220, 708, 735, 755–7, 759 Marshall, C.  853 Marshall, J.  223 Martin, J.L.M.  642 Martin, M.  154, 156, 159, 173, 202, 209–12, 241, 277, 296–7, 303, 305–9, 314, 335, 422, 543, 578, 703, 790, 794 Massin, O.  294, 298, 315 Mates, B.  69–70 Mattes, R.  330 Matthen, M.  68, 132, 134, 159, 161, 181, 191, 193–5, 198, 225, 232, 301, 314, 317, 321, Matthen, M. (Cont.) 368, 373, 397, 403, 424, 441, 446, 454, 463, 487, 490, 496, 512,

author INDEx   897 567, 580, 582, 604, 622, 660, 679, 691, 698, 741, 749, 755, 762, 764, 790, 843–4, 846, 855–6, 858, 860, 865, 880 Matthews, P.M.  538–40 Mattingley, J.B.  640, 642, 643 Mattingly, I.  230, 486 Mattok, A.  634 Maunsell, J.  816 Maurer, D.  605, 824 Mausfeld, R.  194 McAdams, S.  290, 501 McCann, J.  631, 634 McCarthy, G.  229 McDowell, J.  41, 68, 149, 158, 178, 181–91, 193–4, 206–7, 217, 219, 406, 479, 705, 794, 798, 843 McGinn, C.  204, 406, 413–17, 791 McGlone, F.  304, 538 McGurk, H.  320, 487, 576, 578, 608–10, 612, 614, 773 McIntyre, M.C.  538 McKinley, M.  572 McMahon, S.  536 McNaughton, D.  406 Medina, J.  366 Meier, B.  642 Meijer, P.  669 Meisels, M.  308 Mellor, D.  466–7 Meltzoff, A.  606 Melzack, R.  536 Menon, R.S.  538–40 Merabet, L.  607, 662 Meredith, M.  573, 608, 667 Merikle, P.  642, 728, 729 Merleau-Ponty, M.  136, 143–9, 364, 442, 446–7, 452–3, 546, 560 Merzenich, M.  606 Meskin, A.  736–7, 739, 742 Metzinger, T.  173, 515, 517, 518, 519, 521, 522, 527, 649 Meyer, L.  680 Meyrignac, C.   636 Miall, R.C.  227 Michael, C.M.  521, 525 Michel, C.  520, 527, 763 Michel, F.  23

Michotte, A.  550 Midgett, J.  224 Miersch, L.  853 Milan, E.G.  642 Milius, S.  854 Mill, J.  100, 103, 111, 130, 207, 222, 299, 301, 348, 647, 736, 739–42, 744, 749 Millar, A.  207 Millikan, R.  130, 222, 736, 739–42, 744, 749 Mills, C.  647 Milner, A.  156, 221, 223–5, 267, 325, 367, 382, 454, 568, 764 Mishkin, M.  222 Mitchell, B.  385, 652 Mitchell, J.F.  385 Mitchell, K.J.  652 Mitroff, S.R.  373 Mitterer, H.  614, 767 Miyazaki, M.  307 Moesgaard, S.M.  662 Mohr, C.  516, 518, 520, 521, 525, 527 Mojet, J.  331, 346 Mole, C.  35, 475, 486, 588, 594, 855 Mollon, J.  428 Monaco, A.P.  643 Mondloch, C.  824 Monge, G.  429 Montague, P.R.  471 Moore, G.  74–8, 111, 118–19, 121–3, 201, 206–7, 532, 546–8, 593, 596, 786–8, 795, 835 Moran, R.  793 Morein-Zamir, S.  619 Morgan, M.  651, 659 Morris, J.  559, 739 Morris, R.G.  651 Morris, W.  559, 739 Mortara, F.  223 Moscovici, S.  817 Moseley, G.L.  304, 516, 608, 614 Most, S.B.  379, 796 Mouridsen, K.  378–9 Mouritsen, H.  853, 856 Mouthon, M.  522 Mozer, M.C.  817, 827 Mroczko, A.  649 Mroczko-Wasowicz, A.  642 Muheim, R.  866

898   author INDEX Mulligan, K.  149, 156 Mulvenna, C.  642 Mumford, D.  664 Munakata, Y.  569 Munhall, K.  577 Munsell, A.  182, 683–5, 766 Murphy, C.  579 Murray, C.  305 Musseler, J.  230, 676 Mutani, R.   223 Myin, E.  660, 672 Naccache, L.  228, 376, 381, 385 Nadler, S.  90, 92 Nagel, T.  412, 855, 857 Naik, R.R.  863 Nakajima, Y.  288n38 Nakayama, K.  591 Nanay, B.  153–6, 158–9, 161–2, 314, 567, 612, 873, 876–7, 880–2 Nanez, J.E.  826 Narins, P.M.  610 Nascimento, S.  632 Nassi, J.  764 Natchigal, D.  337 Navarro, X.  324 Neander, K.  749 Needham, A.  812 Neisser, U.  518 Nelken, I.  281, 284–5, 290 Nelkin, N.  535 Nemenov, M.I.  864 Neta, R.  206, 802 Neumeyer, C.  633 Newall, M.  875 Newhall, S.M.  683 Newman, G.E.  491 Newton, I.  101, 105, 108, 295, 298, 426–9, 435, 681–2, 686 Newton, N.  533 Ng, V.  538 Nichelli, P.  540 Nichols, R.   91, 99 Nichols, S.  871, 884 Nickels, L.  381 Nickerson, D.  683 Nietzsche, F.  496

Nigel, J.  791 Nigro, G.  518 Nijhawan, R.  470 Nikolic, D.  642, 649 Njiboer, T.C.W.  385 Noë, A.   150, 154, 155, 158, 165, 220, 225, 235, 272, 402, 404, 592, 60–2, 607, 618, 669, 671, 674, 854, 869 Noesselt, T.  611 Nolan, L.  101 Noonan, K.  642 Nosofsky, R.  820 Notebaert, K.  381 Nozaki, D.  311 Nozick, R.  76–7, 801–2 Nudds, M.  162, 274, 286, 291, 314, 403, 487, 574, 576, 614, 659, 853–4 Nunn, J.  651, 653 Nussbaum, C.  495, 509, 686 Ochs, D.  382 Ockham, W.  52, 54–5, 59, 63 Okada, T.  538, 662 O'Leary, A.  537 Oliver, A.  532 Oliver, G.K.  647 Olkkonen, M.  634, 767 Olshausen, B.  815 Olson, E.  472 Olsson, M.J.  347 Olthof, N.  304, 516, 608 Op de Beeck, H.P.  826 O'Regan, J.K.  373, 661 Orlandi, N.  735 Ortmann, O.  641, 645, 648 Ostrovsky, Y.  606 Ota, H.  224 Overgaard, S.  149, 378–9, 385 Owler, B.  538 Pacherie, E.  184, 189, 745 Pagnoni, G.  540 Pallas, S.L.  606 Palmer, S.  258, 394–6, 401, 624, 634, 817, 838 Panis, S.  826 Parise, C.  610 Park, J.  108, 347, 859

author INDEx   899 Parker, A.  859 Parrish, T.  337, 343 Parslow, D.  651, 653 Parsons, T.  618 Partan, S.  611 Partee, B.  244 Pascual-Leone, A.  352, 607, 612, 663, 668 Pashler, H.  589–91, 593, 597, 816, 826 Pasnau, R.  51, 53, 55, 57–8, 62, 162 Pastore, N.  102 Patel, A.  513 Paul, G.  118, 124, 244, 314, 325, 368, 551, 562, 659, 864 Pause, B.  315, 345 Pautz, A.  154, 250 Pavlov, I.  570 Peacocke, C.  162, 168, 170–1, 177–9, 182–3, 186, 189–90, 203, 413–14, 416, 442, 446, 509, 691, 713, 781, 794, 873 Pearce, J.  298 Pears, D.  452, 723 Pegna, A.  376 Pelayo, R.  324 Peretz, I.  290 Perez-Marcos, D.  614 Perkins, M.  298 Perky, C.  791 Perler, D.  51, 57–8, 61–3, 102, 781 Perrett, D.I.  223 Perry, J.  448–9, 454, 735, 739–40 Persaud, K.  662 Personnaz, B.  817 Persuh, M.  382 Pessoa, L.  376 Petersen, D.  633 Peterson, M.  393, 817 Petersson, K.M.  538, 652 Petkova, V.  521 Petrovic, P.  538 Pettigrew, J.  853 Pevtzow, R.  689–90 Phillips, J.B.  866, 869 Phillips, M.L.   538 Phillips, N.  304, 312 Piaget, J.  606 Pianka, P.  664 Piccinini, G.  737, 756

Picciuto, V.  354 Pick, H.  195, 304 Pietrini, P.  664 Piggot, J.  323 Pihlainen, K.  269 Pike, B.  537 Pinecoffin, R.  462 Pinker, S.  381, 441, 475, 488 Pisella, L.  223, 224 Pisoni, D.B.   477, 485, 492 Pitcher, G.  119, 452, 533, 745 Plantinga, A.  77, 800 Plaut, D.  764 Ploghaus, A.  537, 538, 539, 540 Ploner, M.  537 Plug, C.  108 Plunkett, K.  764 Podd, J.V.  730 Poehlmann, K.  360 Poeppel, D.  483 Poirier, C.  672 Pollitz, C.H.  860 Pollock, J.  570, 785–6 Popkin, R.  81 Popper, A.N.  853 Porro, C.A.   233, 540 Porter, J.  342 Postal, P.  244 Pouget, A.  709 Power, S.  410, 464 Prescott, J.  314, 335 Price, D.  537 Price, H.  70, 112, 119, 201, 747 Prinz, J.  162, 218, 371, 377, 381–2, 384–5, 569, 670, 745–7, 750, 756, 759, 764, 838 Prinz, W.  230 Pritchard, D.  206–7, 802 Proffitt, D.  224, 364, 804 Proske, U.  364 Prosser, S.  468 Pryor, J.  76–7, 783, 785–7, 789, 792 Ptito, M.  662 Puce, A.  229 Puel, J.L.  816 Pujol, R.  816 Puri, A.  666 Putman, N.F.  853

900   author INDEX Putnam, H.  220, 784 Pydimari, T.  382 Pylyshyn, Z.  161, 171, 192, 194, 398, 455–6, 696, 756, 759, 761, 765, 772, 803, 813–14 Quante, M.  537 Quartz, S.  706 Quigley, K.  360 Quine, W.  397, 705, 748, 828, 833, 836 Quinn, P.  812 Radeau, M.  612, 770 Raffman, D.  679, 691, 820, 826 Raftopoulos, A.  182, 184, 191–3, 760, 762, 813 Ramachandran, V.  173, 306, 642, 643 Ramenzoni, V.  366 Ramsey, W.  155 Rand, T.  489, 567, 825 Rao, R.  664 Rastle, K.  381 Ratliff, F.  432 Raz, N.  664 Recanati, F.  843 Reed, B.  66–8, 72, 77 Reeves, A.  628, 629 Regan, B.C.  859 Reid, T.  74, 83, 85–7, 89–91, 93–7, 102–3, 111, 424, 605 Reingold, E.  728 Remez, R.  477, 485, 492 Remy, F.  538 Renier, L.  672 Rensink, R.  192, 229, 373 Repp, B.  821, 866 Reppert, S.M.  866 Rescorla, M.  694, 709 Rey, G.  485 Reynolds, J.H.  385, 386 Reynolds, M.  642 Reynvoet, B.  381 Ricciardi, E.  664 Rich, A.N.  640, 642, 643 Richards, W.  314, 323, 325, 329, 343, 397, 567, 573, 575, 578, 697 Richardson, L.  314, 323, 325, 329, 343, 567, 573, 575, 578

Richter, W.  538 Rickless, S.  408–11 Riddell, L.  432 Riddoch, M.  161 Rios, C.  815 Ritz, T.  853, 866 Rizzolatti, G.  230, 231 Ro, T.  382 Robart, R.  281n24 Robinson, F.  202–3 Robinson, H.  119, 128, 129, 135 Robinson, J.  498, 509, 559, 563 Rock, I.  377, 379, 394–6, 401, 696, 699, 735 Rode, G.  224 Roessler, J.  588 Rogers, G.  102 Rohrbaugh, G.  802 Roling, P.  819 Romdehn-Romluc, K.  149 Rorie, A.E.  386 Rorty, R.  91 Rose, D.  240–1, 281, 383, 412, 416, 472, 489, 650, 801 Rosenberg, G.  650 Rosenblum, L.  281, 489 Rosenthal, D.  383, 412, 416 Roskies, A.  182, 189–91 Rosowski, J.  284 Ross, H.  108–9, 318 Ross, P.  82, 223–4, 280, 298, 405, 416, 418, 425, 569, 746, 763, 854 Rossetti, Y.  223, 224 Rossing, T.  280 Rossion, B.  763 Rothen, N.  642 Rothwell, J.C.  469 Rotman, S.  662 Roush, S.  802 Rouw, R.  651–2 Rowlands, M.  228 Roxbee-Cox, J.  295, 298, 659 Rozin, P.  325–6, 329 Rozzi, S.  230 Rubin, P.E.   477, 485, 492 Ruby, P.  516, 524 Rudenga, K.  337 Rudolph, K.  34

author INDEx   901 Rumelhart, D.  825 Russell, B.  70, 74–5, 111–12, 118–21, 154, 156, 160, 162, 183, 201, 360, 368, 442–4, 532, 613–14, 702, 711, 781, 801, 840, 843 Russell, J.  301 Russell, J.  360 Ruttiger, L.  633 Sachs, G.  640–1 Sackur, J.  228 Sadato, N.  662 Sagiv, N.  642 Sainsbury, R.  555 Sahraie, A.  376 Salapatek, P.  816 Salinas, E.  386 Salomons, T.V.  537 Sampaio, E.  672 Samuels, R.  756, 758–9 Sanabria-Mohorquez, S.  662 Sanchez-Vives, M.V.  614 Sanders, M.D.  223 Sanford, D.  295, 298 Sani, L.  664 Sartre, J.  551–3, 556–8, 560, 562 Sasaki, T.  288 Sasaki, Y.  826 Saunders, F.A.  659, 660 Savitt, S.  736 Sawamoto, N.   538 Saygin, A.P.  489 Sayre, K.  737–8, 740 Scadden, L.  659 Scarantino, A.  737, 739 Schachter, S.  359 Schaffer, J.  554 Schear, J.  149 Scheerer, E.  111 Scheich, H.G.  853 Scheler, M.  301 Schellenberg, S.  154, 156, 162, 204–5 Schier, F.  875 Schirillo, J.  629 Schirren, T.  30 Schluppeck, D.  815 Schmid, S.  61 Schmitz, A.   863, 869

Schmitz, F.   537, 541 Schmitz, H.   863, 869 Schneider, G.  52, 222 Schnitzler, A.  537 Schnupp, J.  281, 284–5, 290 Schnupp, J.  281n24, 284–5, 290n41 Scholl, B.  161, 379, 398, 491, 759, 796 Scholte, H.  652 Schonle, P.  633 Schooler, J.  328 Schrater, P.  694 Schroeder, C.  318, 611 Schurmann, M.  660 Schuur, F.  516 Schwartz, R.  270, 398 Schwartz, S.  864 Schwitzgebel, E.  857 Schwoebel, J.  304, 366 Schyns, P.G.  814 Scilingo, E.P.  664 Scott, K.  642 Scott, M.  103, 297 Scruton, R.  291 Seager, W.  534 Searle, J.  217, 793, 838, 840 Sedley, D.  33, 68 Seeck, M.  363, 520, 521, 525 Segal, G.  641, 710, 771 Segal, S.  791 Sejnowski, T.  386 Sekuler, A.B.  817 Sela, L.  342, 345 Sellars, W.  112, 121, 185, 746, 782, 792 Sen, M.  643 Senkowski, D.  324 Serafini, M.  540 Sergent, C.  228 Seth, A.  771 Sethi, A.  382 Seydell, A.  706 Shams, L.  706 Shankar, M.U.  765 Shankweiler, D.P.  485 Shannon, C.  568, 734 Shapiro, L.  220–1, 307 Sharpe, L.T.  633 Shelley, J.  871

902   author INDEX Shenton, J.  366 Shepard, R.  505–9, 511–12, 683, 686, 690–1 Shepp, B.  335, 814, 823 Sherrington, C.  353, 603 Shiffrin, R.  822 Shimojo, S.  612 Shipley, T.  401 Shiu, L.  816, 826 Shoemaker, S.  171, 175, 613, 653 Sibley, F.  323 Siegel, S.  156, 160–1, 200, 204–5, 250, 271, 300, 479, 580, 649, 702, 761, 767, 781, 792, 796, 803–4, 813, 826, 836, 846, 849, 876 Siewert, C.  136, 149 Silins, N.  781, 788, 794–6, 813, 826, 849 Silver, W.  346, 708 Silverberg, A.  708 Simmel, M.  491 Simmen, B.  859 Simmons, A.  81, 84, 89–90, 92, 96, 102–3, 406, 781, 833, 844 Simmons, K.  337 Simner, J.  642, 644, 649, 652–3 Simoncelli, E.  701 Simons, D.  228–9, 373, 382, 592, 796 Sincich, L.  763 Singer, J.  359 Singer, W.  649 Sinha, P.  606 Sinigaglia, C.  230 Skare, S.  538 Sklar, L.  467 Slater, A.  634 Slater, M.  614 Small, D.  316, 325–6, 335, 337 Smart, J.  415–16 Smilek, D.  642 Smith, A.  35, 56, 143, 202, 407, 409, 414, 629 Smith, B.  149, 156, 322, 334, 339, 346, 403, 492, 579, 581, 778 Smith, D.  143 Smith, J.C.  860 Smith, L.  679, 823 Smith, S.  537, 538, 539, 540 Smithies, D.  789, 838 Snell, B.  30 Snodgrass, M.  569, 717, 729, 732

Snowdon, P.  118, 128–9, 201, 206, 210, 532, 833 Sobel, N.  342, 345 Sollberger, M.  651 Solmsen, F.  32 Soler, M.  324 Soma, K.K.  610 Sorabji, R.  424, 567, 853 Sorensen, R.  402, 542, 558 Sosa, E.  75, 179, 739, 787, 792, 800, 802 Soteriou, M.  154, 156, 783 Soto-Faraco, S.  318, 609 Southgate, V.  231 Sowards, L.  863 Sowden, P.T.  819 Speaks, J.  182–3, 188, 794 Spector, F.  824 Spehar, B.  631 Speigle, J.M.  631 Spekreijse, H.  384 Spelke, E.  399, 401 Spelke, E.  401 Spence, C.  157, 308, 314, 318, 321, 323, 331–2, 335, 337, 348, 487, 567, 572, 578–9, 583, 603–5, 608–10, 615–16, 642, 755, 768–71, 834 Sperber, D.  756, 759, 774 Sperling, G.  192, 374–5 Spiller, M.  649 Spinelli, K.L.  363, 520 Spinoza, B.  87 Squire, S.  768 Stalnaker, R.  183, 794 Stampe, D.  737, 739–40, 742–4, 747, 749 Stazicker, J.  587 Stein, A.  340 Stein, B.  608 Steinbuch, J.  104–6, 111–12 Stephens, G.  533 Sterm, W.M.  662 Stetson, C.  471 Stevenson, R.  332, 343, 549, 604 Stewart, C.  532 Steyvers, M.  823–4 Stich, S.  708, 710 Stienen, B.M.C.  376 Stiles, W.  628

author INDEx   903 Stokes, D.  761, 765 Stone, J.  697, 700 Stone, M.O.  863 Stoner, G.R.  385 Storbeck, J.  360 Stout, G.  111, 301 Stratton, G.  32, 147, 444 Strawson, P.  170, 176, 397, 747, 841 Streel, E.  662 Streim-Amit. E.  662 Streri, A.  606, 613 Stroop, J.  647 Stroud, B.  206 Studdert-Kennedy, M.  485 Stumpf, C.  299 Summers, J.  472 Sur, M.  606 Swain, M.  784 Swets, J.  717, 719, 726 Swoyer, C.  815 Szczepanowski, R.  376 Tacca, M.  763 Tachau, K.  54–5 Tadi, T.  173, 515, 517, 521 Tafalla, M.  314, 348 Talsma, D.  318 Tamietto, M.  376 Tan, H.Z.  572 Tanaka, J.  818, 822 Tanaka, S.  288 Tanesini, A.  854 Tarr, M.  381, 822 Tasker, R.R.  536 Taylor-Clarke, M.  305 Taylor, C.  452 Taylor, K.  736, 740 Taylor, R.  558 Tazner, M.  224 Tchekhova, A.  347 Teng, S.  666 Terazzi, E.  223 Thau, M.  201 Therman, P.  432 Thibaut, J.P.  814 Thompson, B.  706, 707 Thompson, D.G.  538

Thompson, E.  149, 160, 515–16, 613, 622 Thompson, P.  773 Thornton, I.  373 Thorsrud, H.  66 Thouless, R.  108 Thut, G.  520 Thut, G.  521, 525 Tiedmann, C.  853 Tienson, J.  706 Tilikete, C.A.  224 Tillman, B.  680 Tipper, S.P.  304 Titchener, E.  107–8, 303, 857 Tomiczek, C.  332 Toni, I.  223 Tooby, J.  756–7, 774 Toppino, T.  761 Toribio, J.  183–4 Tormos, J.  324 Tornay, F.  642 Tourtual, C.  104–6 Tracey, I.  537 Tranduy, D.  672 Travis, C.  154–5, 199, 205, 241, 703, 792 Trede, R.D.  864 Treede, R.D.  537 Treisman, A.  192, 382, 398, 472, 576, 597–8, 763 Treisman, M.  382, 398, 472, 576, 597–8, 763 Tremoulet, P.  491 Trommershauser, J.  619, 700, 706 Troost, J.  629 Trout, J.  475, 485, 487–8 Trullemans, C.  661 Tsakanikos, E.   642 Tsakiris, M.  307, 367, 516–17 Turnbull, R.  41, 44 Turner, R.  104–6, 110 Turri, J.  784 Ty, T.C.  536 Tye, M.  154, 156, 162, 181–4, 188–9, 194, 203–5, 217, 269, 355, 358, 383, 385, 434, 534, 613, 616, 629, 650, 745–7, 785 Tyler, M.  210, 347, 452, 660 Ullman, S.  597, 745 Ungerleider, L.  222

904   author INDEX Vaitl, D.  354, 358 Valberg, A.  629 Vallbo, A.  295 van Boxtel, G.  376 Van den Bussche, E.  381 Van den Stock, J.  376 van Leeuwan, T.M.  652 Vanello, N.  664 Vanlierde, A.  672 Varzi, A.  402, 553–4, 562 Vatakis, A.  608 Veinot, F.  859 Veldhuizen, M.  337 Velleman, J.  414 Venema, A.  304, 516, 608 Veraart, C.  661, 662, 672 Verhagen, J.  338 Verstraten, F.  366 Vesey, G.  303, 305 Videl, J.  324 Vighetto, A.M.  224 Vila, J.  642 Virzi, R.A.  599 Vision, G.  89, 181, 184, 186, 194–5, 257, 261, 263, 268, 271, 376, 394, 424, 430, 490, 660–1, 672 Visser, T.  729 Vogel, H.  864 Vogel, J.  802 Von Melchner, L.  606 Voss, J.  337 Vouloumanos, A.  490 Vythelingum, G.N.  653 Wade, N.  105, 858 Wager, A.  641, 643, 646, 650 Wagermans, J.  826 Wagner, M.  108, 432, 511 Wahl, R.  92 Walk, R.  133 Wall, P.  536 Walter, S.  767 Walton, K.  876, 879–80 Waltz, D.  761 Wandell, B.  631 Wanet-Defalque, M.C.  662 Wang, R.  826

Wang, S.  363 Ward, J.  640, 642–3, 669, 771 Warren, D.  33, 610, 615 Warrington, E.R.  223 Wassermann, E.M.  668 Watanabe, T.  826 Watzl, S.  601 Weaver, W.  734 Weber, E.  104, 106–7, 109, 297–9, 502, 682, 718 Wedgwood, R.  788–9 Weinberger, N.  816 Weisberg, J.  789 Weiskrantz, L.  223, 376, 378 Weiss, Y.  701 Welch, R.  610, 614–15 Weller, C.  537 Werker, J.  490 Wertheimer, M.  470 Westervelt, P.  279 Wheeler, D.  822 Wheelright, S.  643 White, B.  659, 660 White, R.  120, 660, 787 Whitmore, J.  730 Whitney, D.  666 Whittle, P.  625 Wiesel, T.  432–4, 734–5 Wiggins, D.  836–7 Wijers, M.  304, 516, 608 Wilbarger, J.  373, 381 Williams, B.   313 Williams, P.  822, 828, 829 Williams, S.C.R.   641, 651, 655 Williamson, T.  206, 797, 801–2 Williwams, S.C.  538 Wilson, C.D.  112 Wilson, J.  298 Wilson, M.  84, 86, 88, 92, 409, 410 Wilson, R.  228 Wiltschko, R.  866 Wiltschko, W.  866 Winawer, J.  643 Winer, E.S.  732 Winkielman, P.  373, 381 Wise, R.  537 Witherby, S.A.  642

author INDEx   905 Wittgenstein, L.  119, 122, 124, 129, 134, 174–5, 442–5, 448–9, 453, 544, 760 Witthoft, N.  643 Wolbders, T.  664 Woldorf, M.  324 Wolf-Devine, C.  89, 102 Wolfe, J.  379, 599 Wollheim, R.  873–4, 876–8 Wolpert, D.  227, 367, 524, 698 Woodworth, R.  107 Woolgar, C.  314, 331 Wrathall, M.  149 Wright, C.  787, 788 Wright, W.  622 Wu, W.  100, 104, 108, 303, 589, 718 Wundt, W.  100, 104, 108, 303, 718 Wyke, M.  640, 641, 771 Wyller, T.  533 Wynn, K.  399 Wyszecki, G.  628 Xiao, B.  631 Xiao, L.-Q.  826 Xu, F.  399, 837 Yamamoto, S.  306, 308 Yang, L.  815 Yang, T.  279, 816

Yarrow, K.  469 Yeomans, R.  317, 335, 337, 338 Yeshurun, Y.  162, 345 Yolton, J.  92, 103 Yonekura, Y.  662 Yonelinas, A.  729 Yourgrau, P.  746 Yu, C.  826 Yuille, A.  400 Zahavi, D.  143, 149 Zahorik, P.  664 Zaidi, Q.  624, 632 Zampini, M.  335 Zeki, S.  424, 433, 435, 878 Zelaya, F.  538 Zemel, R.  827 Zhang, E.-T.  535 Zhang, J.-Y.  826 Zhou, X.  814 Zipser, D.  825 Zmigrod, S.  289 Zohary, E.  664

Subject Index

The index was prepared by Steven Coyne absences 542–62 examples of  542 materialism and  553–4 modelled as after-images  546–8 negative judgments and  548–52 as objective  558–9 as observer-relative  554–7 phenomenology of  552 achromatopsia 247 acquaintance  120, 559, 834, 839, 842–6 action  39, 44–5, 55–6, 59, 63, 88–9, 130–2, 140, 217, 225–6, 231–2, 244–5, 454, 668 attention and  587–9 perception and  96, 442, 524 perception of absences and  550 proprioception and  366–7 vision and  271–2 action-based accounts of perception, see enactivism active perception  579–80, 827 of pain  579–80 adaptation, see evolution affective experience  359–60 affordances  24, 146, 221, 341 definition of  218 after-images  47, 121, 546–7, 650 agnosia  7, 8, 17, 223, 235, 267, 374–5, 489, 493, 773 animacy 491–4 animal perception  17–18, 24–5, 188, 222, 381, 556–7, 569, 818, 836, 853–70 animal cognition and  570 animal conscious experience and  575, 732 Aquinas on  53–4 of balance  859 compared with human perception 582–3, 853

continuity of, with human perception 855–61   de se thought and  177 early Greeks on  31–3, 43 individuation of senses in  855 infrared reception as uniquely belonging to 862–6 magnetoreception as uniquely belonging to 866–7 pre-adult 815 anti-representationalism  154–5, 157–8; see also relationalism appearances  36–7, 137, 139–42, 144, 147–8, 482–3, 587, 599 etymology of  238 Kant’s definition of  103 sense-data and  122, 128, 130–1 apotemnophilia 173 Aquinas  53–64, 314 on the bodily senses (e.g. smell)  314 on the distinction between sensibles  per se and sensibles per accidens 53 on the process of perception, especially of colour 55–7 on the proof of God’s existence  51 on the reliability of perception  60–4 on the relationship between perception and reason  53 Aristotle  10–11, 13, 15, 17, 30, 33–5, 41, 43–52, 64, 294, 424–7, 429–30, 433–5, 853–6, 870 on colour  424–5 and criticism of Plato  44n50 on the criterion for individuation of the senses  575–7, 853 influence of, on medieval philosophers of perception 50

908   subject index Aristotle (Cont.) on perceptible properties  46 on perceptual content  47 on the causal theory of perception  44, 424–5 on the proper object of each sense  318, 424 art  147, 872, 880–2 mimetic theory of  880 attention  20–1, 379–80, 396–9, 587–99, 761–2 access vs. selective aspects of  385, 589–93, 597, 814, 820, 824 as altering the qualitative character of experience 587 as making attended properties more determinate 161–2 binding and  382, 596–9 blindsight and  588, 594 consciousness and  377 demonstrative thought and  837–8, 845–8 dispersed across multiple locations, in vision 398 divided into personal and informationprocessing streams  588–9, 591, 597 object-based  192, 817, 829, 832 and olfaction or smell  345–7 pain and  537–40 perceptual learning and  819–27 phenomenology of  144, 598–601 as providing knowledge  587–8, 593–6, 795–7 attentional blink  380 audition  10–11, 274–91; see also sound compared with vestibular transduction 361 compared with vision  134n30, 275, 314, 345, 614 dominated by vision  32, 320, 610 introspective character of  575–7 location in  105, 281, 290n41, 291 non-linguistic  476, 478, 484–90 objects of  165–6, 274–5, 289, 484, 487, 777 physiological process of  284–5, 503–12, 614, 858, 860 processing of  286–7 representationality of  278–9, 286–9 auditory experience  274–6, 278, 286, 291, 343, 483, 579, 651

Austin, J.L.  23, 119, 123, 127, 129, 134, 202, 229, 269, 549, 594, 747 autoscopic illusions  518, 521–3 B-theory of time  466–9, 471 Bayesian models of perception  402, 694–714 as evidence against relationalism  703–4 as evidence for intentional psychology 705–6 inference in  701–2 mathematical description of  696–8, 710–3 as non-representational  698, 709 and phenomenal content  706–7 source of prior probabilities in  700–1 belief  5–6, 22–5, 70–1, 76–8,182–6, 246, 270, 550, 556–8, 568–71, 735, 767, 781–804, 844–51 basic  71, 807 cognitive penetration and  803–4 contents of  36, 156, 159–60, 790, 792–3, 798   de re 798 demonstrative  844, 846–7, 849–50 experience and  782 formation of  185, 569, 588, 747, 844 in information theory  745–6 tensed 468 Bergson, H.  461, 473, 548–52, 557, 562–3 against perceiving absences  548–52 Berkeley, G.  3, 70, 84–5, 87–94, 102–3, 111, 121, 131–2, 268, 276, 542 against the primary-secondary quality distinction 84 idealism of  91, 103, 542 puzzle of, about distance perception  89 binding problem  18, 20, 219, 382, 384, 596–9, 763, 838 in perceptual demonstrative thought 845–7 unconscious 382 blindness  413, 416–17, 645, 659, 663–4, 668, 671, 675 blindsight  223, 246, 267, 376–8, 382, 385–6, 568, 724, 800, 838, 854   588, 789, 795, 800

Subject index   909 bodily movement  171, 221, 224–7, 232, 367, 402, 525; see also enactivism, perceptual constancy, proprioception relation to perception in Merleau-Ponty 146–7 bodily perception, see interoceptive senses; vestibular system; proprioception body image  173, 365–6, 370 definition of  365–6 body map theory of touch  294, 304–9, 366 definition of  294 body ownership  179, 518, 521–2 body schema  146, 149, 173, 365–7, 370 as multi-modal or proprioceptive  366 definition of  365–6 body template theory  294, 309 BOLD response  667 Boolean Map Theory  590 brain  84–9, 105, 227–30, 318–19, 535–41, 596–7, 662–4, 814–19 early theories of  32, 40, 53, 84–7 in a vat  200, 205, 787, 804 nineteenth century theories of  101, 105 role of, in vision  261–3, 266–7, 433–4, 838 of synaesthestes  652 Brain-Port 661 Brentano  23, 47, 111, 136–9, 142–3, 149, 298–9, 303–4, 310 against sense-data theories  137 distinction between judgment and presentation 137 brightness  17, 20–1, 267, 382, 428, 431–2, 681, 683, 685–6, 688–9, 814, 820, 823, 825, 829 Categorical Perception (CP)  20, 482–3, 488, 689, 763, 820, 826, 830 definition of  820 categorization of the senses, see classification of the senses cats 735 causal theory of perception  34, 38–9, 272, 554, 800 in ancient writers  34–9 partial rejection by early modern writers 86–7

central nervous system (CNS)  156, 319, 354, 356, 524, 529 change blindness  228, 345, 373–4, 560–1, 592–3, 801, 806 and attention  592–3 as evidence for enactivism  227–9 as evidence for unconscious perception 374 in the perception of absences  560–1 chemical senses  269, 314–17, 319–49, 604, 666; see also olfaction; taste and food  316 including some aspects of touch  315 cinematic account of perception  461–3, 468, 471 classification of the senses  296, 314, 356, 372, 375 Aristotle’s  317, 320 in Aquinas  56 CNS, see central nervous system cognition  30–1, 226, 398, 713; see also judgment distinguished from perception  10, 30, 59, 93n80, 484, 633, 814, 844–6 extra-perceptual  475, 477, 479 cognitive encapsulation  612, 700, 758–61, 765–7, 771–4 cognitive penetration  544, 759, 766–7, 769–70, 772–3, 775, 777–8, 803–4, 813, 826–7, 831 arguments against  813 direct vs. indirect  813 of vision  777, 809, 831 rational belief and  803–4 cognitive processing  470, 845–6 colour  10–11, 20–2, 44–7, 55–60, 82–8, 209, 246–7, 374–7, 405–19, 575–6, 590–9, 629–31,794; see also colour vision early modern views of  83 early physics of  108–9, 427–9 Newton’s view of  428, 680–2 ontology of  83, 422, 435–6, 622 as perceiver-dependent  411–16, 423, 434 as perceiver-independent  416–17, 438 qualitative aspect of  410–14, 416–17, 682–3 and secondary qualities  405–19

910   subject index colour constancy  109, 635–9, 751 colour space  681–6, 691–2 Munsell’s model of  683 Natural Colour System model of  684–5 Newton’s model of  681–3 opponency model of  682 shades in  691–2 colour vision  16–17, 33, 61, 94–5, 100–1, 261, 265, 422–36, 488, 498, 588, 641, 752, 766, 869 as necessary for vision in general  425, 434, 592 medieval views of  55–6 physiology of  110, 428–35 processing of  267–8, 423, 432–3, 437–8, 763, 766–7, 769 as serial  592–3 common sensibles  52–4, 61, 82, 318, 320, 489, 575, 605, 612 definition of  52 computational model of perception  708–12, 755 arguments against  220–1 concepts  25, 53–4, 61–2, 181–96, 534, 555–6, 593–5, 768–9, 825 acquisition of  188, 190 in ancient Greek thinkers  29–30, 47 capacity to use  181, 184–6, 188–9, 191, 194, 633 colour  181, 375, 588, 595 demonstrative  188–90, 595 empiricism about  218–20 first person  169, 177–9 individuation of  177–8 sortal  836, 841, 851–2 conceptualism  149, 181–6, 188–91, 193; see also nonconceptualism and the argument from illusion  187–8 definition of  181–2 empirical argument against  186 epistemic argument for  184–5 Kant on  186 congruence, between different taste experiences 338–9 consciousness  9, 136–9, 142–4, 172–6, 193n3, 355–8, 372–86, 412, 416–17, 653, 719, 731–3, 746–7; see also unconscious perception

attention and  587–96 epistemology of  796–7 first-order vs. higher-order theories of  355, 731–2 olfactory background to  346 structure of  611, 615–16 unity of  41, 176, 561, 615–16 constancy of perception, see perceptual constancy content of perception  33, 47, 140, 153–4, 156, 159–66, 170, 186, 188, 219, 569, 612, 647, 653, 752–3, 786, 793–5 according to Aristotle  46–7 first person  168–9, 171, 175–6 inferential 3 according to medieval philosophy  51 multisensory 612–15 non-conceptual  47n66, 149, 183, 189, 220, 752, 754, 763, 810 object-dependent 204–5 according to Plato  40–1 as propositional    11–13, 29, 41, 160, 795–7 as restricted by conceptual capacity of the subject 181 visual  273, 809–10, 852, 884 CP, see Categorical Perception cross-modal interaction  21, 320, 326, 329, 332–3, 605–9, 612, 614 between audition and number sense  609 distinguished from multisensory integration 608 between temperature and taste  333 between vision and audition  487–8, 604, 606, 608 between vision and proprioception  445 between vision and touch  604, 606–7, 608 defeasibility of perceptual experience  77 Democritus  14, 31, 34–7, 39, 47–9, 550, 553 demonstrative reference  186, 189–90, 806, 836, 840–1, 850–1 Husserl on  141 in audition  276–7 demonstrative thought, see perceptual demonstrative thought demonstratives  17, 171, 189, 450–1, 798, 807, 833–5, 837–9, 841, 843–51

Subject index   911 depth perception  3, 104, 111, 268, 669, 672, 712, 715 Descartes  1, 3, 67, 69, 71–4, 81–9, 91–9, 101–3, 405, 407, 424–6, 659 clear and distinct perception according to 72–3 on colour  425–6 compared with the Academics  73 on innate judgments  89 the limitations of sensory knowledge according to  102 on three grades of sensory response  88 on visual processing  86 three skeptical arguments by  72–3 descriptivism 839–40   de se content  169–72, 175–9 possibility of perception without  170 determinates  147, 160–2, 202, 304, 483, 680, 743, 874 direct perception  53, 57, 60, 82, 112, 269, 459, 463–4 discrimination tasks  21, 47n65, 377, 720, 813, 826, 830 disjunctivism  23–4, 148, 198–207, 209–11, 213–15, 217, 714–15, 808 arguments against  212 about the contents of experience  204–5 definition of  198 epistemological  187, 206–7 metaphysical  209–10, 213, 799 negative  202, 211, 213 about objects of experience  201–3 about perceptual evidence  206–7 positive 211–13 dispositionalism  410–11, 413–16, 426, 436 dispositions  83, 134, 194, 239, 275, 278, 409–11, 414, 426–7, 435, 447, 452–3, 698, 746 distal attribution  669–71, 673 distance  34, 60, 84–6, 88–9, 92–4, 105–8, 111–13, 174–5, 223–4, 507, 518, 621–5, 666–7, 679, 820 perception of  34, 89–90 dogmatism 76 domain specificity  758–9, 774–6 dorsal stream  156–7, 221–5, 265, 272, 382, 764

ears  279, 281, 290, 292–3, 307, 489, 499, 579–80, 662–4, 827–8; see also audition Empedocles on the structure of  33 physiology of  284–5, 503–12, 614, 858, 860 embodied cognition  221, 546 echolocation  666–8, 855, 857 egocentric space  453, 455 Empedocles  31–4, 48 emotions  30, 229–30, 232, 316, 328, 345, 354, 359, 368–9, 473, 537, 570, 586 empiricism in Locke and Berkeley  90 enactivism  24, 154–5, 158, 218, 225–6, 228–9, 546; see also affordances contrasted with behaviourism  226–7 contrasted with computational or representational approaches  218, 225–6 contrasted with relationalism and representationalism 154–5 empirical evidence for  222–5 epistemic access, immediate  407–8, 410–11 epistemology, see justification; belief; evidence Evans, G.  176–7, 179, 182, 194, 367, 446–7, 450, 452–5, 834–7, 840–1, 843, 852 on individuating the senses  575 events in perception, see listings for individual senses (e.g. sounds, sights, smells) evidence  51, 239, 289–90, 569, 594, 764 in Bayesian theory  610, 696–8 conditional 63 in signal detection theory  722–6 in skepticism  77, 205–7 evolution  308, 325, 329, 577, 582, 766, 826, 856–9, 865 in animals and humans, compared  557, 859 of audition  15, 286, 496, 501, 504–6, 508 in Empedocles  33 of information detection in organisms 740–1 of modularity of perceptual functions 757–8 of multimodal perception  607, 613

912   subject index evolution (cont.) of object recognition  400, 403 of pain perception  536 perceptual learning and  816, 818–19, 827–8 of sensory substitution  665, 667–8 of vision  15, 109, 426 of warpings of perceptual spaces  689 experience  6, 132, 139–48; see also consciousness content of  143, 183–9, 191–4, 204–5, 226, 271, 483, 794 Husserl’s definition of  139 Intentionalism about  208–9 Kant on the limits of  103 objects of  201–4, 873 structure of  82, 88–9, 119–20, 141, 202, 208, 210 temporal extension of  4 transparency of  599, 808 exosomesthesia 306–8 externalism  77, 785 eyes  630–4, 863–5; see also vision animal vs. human  582–3, 863 Aquinas’ theory of  56–7 Descartes on the physiology of  85–90 Kepler’s observations about  85 movement of  105, 179, 297, 469–71 nineteenth century developments on the physiology of  104–5 physiology of  258–64 Plato’s theory of  39–40, 42 Presocratic theories of  30–4 factive states  797–9 feed-forward sweep, see FFS FFS (feed-forward sweep)  191–2 figure and ground segregation  394–5 first person  71, 171–5, 177, 179, 325, 431, 781 and embodiment  171 and illusionary experience  176 and intention  184–5 and perception  168–9 flavour perception  4, 18, 317, 320–1, 323, 327, 332, 335–41, 578–81 flavour  4, 18, 44–5, 81–2, 87–8, 320–1, 323–31, 333–41, 347–9, 351–2, 585–6 components of  578–80

contrasted with the object of taste or gustation alone  317 contribution of pleasure to  338–41 divided into constitutive and causally affecting 337 empirical definitions of  337 in Democritus  35 as psychological  340–8 multisensory perception of  348, 581–3 Fodorian modules  612, 758, 770–2 foundationalism  70–1, 785, 786n16 frame of reference  441–2, 446, 448–57, 521, 525 and proprioception  366 and touch  294, 304, 308–9 functionalism  641, 653–7, 665, 673 Galileo  69, 82–4, 98, 101, 405, 407, 425, 429 generality problem, for reliabilism  800–1, 805–6 geometric properties  35–6, 39, 425 Gestalt psychology  8, 101, 112–13, 144, 394, 398, 562, 817, 821 Gestalt switches  552 Gorgias  32, 34, 48 gustation, see taste hallucinations  4, 24, 127–9, 148, 154, 198–202, 206, 208, 210–15, 546, 548, 595, 644, 651, 797–9 as having sense-data as their objects 200–1 autoscopic  518–19, 528 definition of  200 demonstrative thought and  839 justification and  797–9 relation to representationalism  154–5 partial  200–4, 209–10 total, see total hallucinations Hamilton, W.  100, 103, 111 haptic touch  296 harmony  20, 502, 511–12 harmonic structure  280, 285, 288 hearing, see audition heautoscopy 517–19 hedonic tones  345, 347 helical model  510–11

Subject index   913 Helmholtz  7, 104, 106, 109–12, 114–15, 227, 424, 429–31, 435, 633, 682, 695, 715 on unconscious inference and perception 695 Heraclitus  31, 546 hierarchy of the senses, see classification of the senses HMI (Human Machine Interface)  660 HMTC (Hybrid Medium Transducer Criterion)  577–9, 581 Hobbes, T.  criticism by, of Descartes  72n24 on the mind-dependence of colours and sounds 83 on the primary-secondary quality distinction 405–7 human machine interface, see HMI Hume, D.  174–5 distinction of, between ideas and impressions 103 on attending to oneself  168, 174–5 on individuating the senses  575 on the necessity attending visual experience 275 on representationalism  275 as a sense-data theorist  130 and skepticism  103, 121, 550–1 Husserl, E.  23–4, 136, 138–44, 148–50, 364, 546, 551, 563 arguments of, against Brentano  142 and demonstrative thought  140 as an externalist  143 general approach of  139 and the noema of an action  140 on own-body experience  142 on perceiving after-images  546 phenomenology of  562 Hybrid Medium Transducer Criterion, see HMTC hylomorphism 59 iconic memory  374 ideas  14, 645–9 contrasted with impressions, in Hume  103 divided into acts and representations by Descartes 91 Locke on  407–10

as objects of perception  82, 91–4, 407–8 relationship of, to objects  87, 91 relationship of, to sense data  22, 120, 128–33, 201, 596 illumination  12, 14, 20, 259, 429, 435, 580, 628, 630–2, 697–8, 704, 707, 795 illusion  155, 199–204, 209–10, 251, 797 in art  882–3 in audition  290, 487 autoscopic  518, 521–3 argument from  22, 127–9 in Bayesian theory  695–6, 700–5 definition of  199 in Aureol  54–5 interoceptive  357, 364 olfactory 344 and representationalism  154–5 rubber band, see rubber band illusion tactile  305, 307 taste  332, 344 of time  460, 469–72 in vision  156, 209, 219, 224, 268, 625–8, 772–3, 804 images  2–3, 16, 84–6, 113, 219–20, 258, 260–1, 264, 400–2, 468–70, 521, 660–2, 875, 877, 883 formation of  258, 260–1 inflected  878, 880, 882 optical 2–3 receptoral 2–4 two-dimensional  86, 105–6, 111 imagination 791 inattentional blindness  228, 377, 379–80, 385, 560–1, 592–3, 796 indeterminacy of experience  130–1, 144–5 individuation of emotions  359–61 of objects  192 of perceptual states  159, 304 of shades of colour  681, 692 individuation of the senses  294, 297–8, 567–83, 604, 641, 648, 664, 666 Aristotle’s criterion for  575–7 on the basis of experience  568, 572, 574 on the basis of sense organs  659, 664 as conventional  574 from homeostatic systems  569–72

914   subject index individuation of the senses (Cont.) Hume’s criterion for  575 sensory vs. perceptual  578–82 infants  111, 188, 231–2, 399, 482, 490, 559, 606, 613, 812, 830 inference, in perception  5, 695–6, 702 in Agrippa  70 infrared light perception of, in animals  862–6 processing of  157, 318, 386, 462, 572, 588, 596–8, 668, 815 information Dretske’s definition of  735 nesting of  742 objectivity of a definition of  738–9 primary and secondary  742–4, 747–8 semantic theory of  734, 736–7 information-theoretic account of perception 734–51 consciousness and  746–7 Millikan’s criticisms of  740 objects of perception and  747–9 representationalism and  745–6, 749–50 role of perceptual constancy in  748 inner perception  19, 52, 63, 142, 531 contrasted with outer perception in Brentano 138 intentionality  145–7, 177, 217, 229–32, 386, 454, 612, 729 definition of  136 intentional content  169, 174–8, 251, 612–13, 650–1 intentional objects  294–6, 299, 496, 552, 650 internal perception, see inner perception internalism  796–7, 804 access 789 Husserl’s view of  149 interoception  8, 353–61, 365, 369–70, 604, 648 as tracking many kinds of properties 355–6 considered as one or multiple sense modalities 354 definition of  353 representationality of  357–8 introspection  83, 108, 162, 198, 205, 209n33, 211, 433, 572, 595–6, 615, 793

phenomenology of  574, 576 reliability of  270–1 itches  19, 121, 353–8, 540 Ishihara test  592 James, W.  100, 112, 303–5, 359–60, 363, 550, 569, 587–8, 599 on attention  588 JND (just noticeable difference)  106, 685, 718 judgement  12, 63, 88–9, 131–2, 137–42, 177–9, 223–4, 765–6 Aristotle on  56–7 learned vs. innate  89 negative  549–51, 556 positive  63, 549 and perception in Brentano  138 and perception in Descartes  88 and perception in Husserl  141 vs. presentation  137, 142 and sensation in Reid  89 justification  5, 70–1, 75–6, 126, 154, 182, 184, 187, 725–6, 777, 781–3, 785–802, 804–10, 848, 851 a priori  788–9, 806 attention and  795–7 basic normative notion of  783–5 bootstrapping and  788–9 defeaters to  785 experience as sufficient for  786 immediate  76, 785–9, 794–5, 800, 807, 809 mediate 785–6 phenomenal character and  789 propositional vs. doxastic  783–4, 796 structure of  70, 787 just noticeable difference, see JND Kant  103, 172, 314, 546 Husserl and  144 on conceptualism  186 on congruence  442n3 on intuitions  185 on negative judgments  549 Kepler  2–3, 85, 101, 260, 264 knowledge    3, 5, 13, 51–2, 74–7, 93–5, 176–80, 559, 801–2 action and  587–9, 593–6 closure and  206n23, 207

Subject index   915 Descartes on  100–3 first-personal  169, 176–80 information and  739 Plato on  38, 41–3 propositional  559, 587, 593–4 role of justification in  801 role of sensitivity in  801 safety condition on  179 language  218–19, 250, 475–84, 490–1, 557, 819, 821, 851–2; see also phonemes lateral geniculate nucleus, see LGN learning, see perceptual learning Leibniz  87, 90–1, 98, 102–3, 605, 640, 656 on the similarity between touch and vision 90 LGN (lateral geniculate nucleus)  261, 264, 267, 432–4 light spectral composition of  108–9 listening, see audition Local Sign Theory  106, 294, 303–4, 443 location  260, 266, 269, 398–9, 422–5, 448–51, 496–7, 534–5, 589–90, 816–17 A vs. B types of, in touch  308–9 Locke  69, 82–5, 87, 90, 93–8, 101–3, 116–17, 120–1, 131–3, 405–11, 414, 419–20, 426–7, 435, 633 on ideas  103 on resemblance  94, 121 on the reliability of the senses  95 as a sense-datum theorist  120 loudness  274–5, 479, 482, 484, 488, 491, 496–7, 512, 648, 661, 679, 686, 718 definition of  497 Malebranche  81, 83, 85, 87–90, 92–9 on the distinction between natural and free judgments  88 material change  43n49, 45, 59, 406 Maximum Likelihood Estimation, see MLE McGurk effect  320, 487, 576, 578, 608–10, 612, 614 mechanical philosophy  82–7, 91, 404–7, 409–10, 414, 418 melody  6, 146, 463, 502, 511–12 memory  192, 570–1, 583

as an intermediary between the senses and cognition 570–1 mental paint  592, 599–600 Merleau-Ponty  23, 136, 143–50, 364, 446–7, 452–3, 546, 563 on attention  144 compared with Husserl  143 on orientation in visual perception  446–7, 452–3 on own-body perception  145, 364 on perceiving after-images  546 on the visual field  144–5 metacognition  360, 549, 556 metamerism  35, 415 micro-modularity 772 microstructures 34–7 mind–body problem  86, 383, 591, 751 misperception, see non-veridical perception; illusion; hallucination MLE (Maximum Likelihood Estimation) 610 models of perception  6–7, 119–22, 127, 129–30, 132–4, 406, 505–7, 630–2, 680–1, 684–7, 696–9, 701–3, 712, 734–5, 770; see also Bayesian models of perception; signal detection theory; information-theoretic account of perception modularity  268, 490–2, 560, 611, 614–15, 755– 9, 761–3, 765, 768–74, 774–8, 827 arguments against, from cognitive penetration  761–2, 765–9 arguments against, from multisensory integration  612, 769–70 arguments against, from synaesthesia  771 computational model and  757–8 Fodor’s model of  758–9 micro- 772–3 non-conceptual content and  763 physiology of  764–5 Molyneux’s Problem  90, 132, 605–6, 613–14 Moore  78–9, 118–21, 201, 206–7, 546, 548, 593, 795, 808, 810–11, 852 on common sense  74–5 on sense-data  122–3  motion  5–8, 94, 224, 263–5, 379–80, 490–1, 512, 827 absolute 444

916   subject index motion (Cont.) as a primary quality  69–70, 82–8 laws of  296–8 modular perception of  8 perception of  5, 700–1, 815 sound and  281–2 motor system  219, 226–7, 367, 456, 498 mouth  4, 18, 168, 295, 302, 315–16, 320–1, 323–9, 335–7, 340, 576, 578–81, 642, 666 Muller-Lyer illusion  128, 219, 760 multidimensional scaling  412, 693 multiple realizability  220, 648 multisensory integration  8, 21, 157, 317, 319, 323, 333, 337, 339–41, 348–50, 517, 519, 521, 577, 603–5, 607–16, 769–71, 776, 778, 834 arguments against  318 between vision and audition  608–9, 614–15 between vision and touch  608, 614 distinguished from cross-modal interaction 608 dominance of one sense over another in 610–1 as evidence against modularity of the senses 612 and the representationalist/antirepresentationalist debate  157 in taste  317, 336–41, 578–9 visuomotor  222, 517 multisensory interaction, see multisensory integration; cross-modal interaction multisensory perception, see multisensory integration; cross-modal interaction multimodal perception, see multisensory integration; cross-modal interaction music  4, 6, 287, 495–513, 686, 693 definition of  495 phenomenology of  495–6, 498–9, 509–12 qualities of, see loudness; timbre; pitch; rhythm and vision  512–13 Myth of the Given  185–6 naïve realism  199, 203, 207–14, 251, 601 definition of  112 natural geometry  89 natural kinds  366, 368–9, 641, 831, 854

nesting 742–3 Newton  101, 298 on colour  426–9, 435, 681–2 non-veridical experiences  198–201, 206–7; see also illusions; hallucinations and synaesthesia  643–4 definition of  199 in audition  290–1 nonconceptual content  178–9, 181–3, 185–7, 189–91, 193–7, 219, 361, 794, 807, 810 as a Russellian proposition  183 as a set of possible worlds  183 as structured  194 nonconceptualism  181–5, 188–91, 193; see also conceptualism arguments for  188–91 concept vs. state  182 definition of  181–2 nonhuman senses, see animal senses number sense  571–2 object files  192, 194, 289, 399, 403–4; see also binding objective threshold approaches  727–8, 731 objectivity of perception  109, 179, 301, 318, 321–2, 335, 337–8, 375, 381, 559, 562, 622, 739, 778, 805 and first person thought  175–7 and subjecthood  175–7 object perception  393–403, 747–9 evolution of  403 individuation of objects in  395–6 recognition of  194, 265, 381–2, 393, 400, 830 relation to perception of scene  394–5, 400 stability of  393 and visual processing  400–2 objects  3, 86, 88, 191, 223, 512, 841, 874; see also object perception definitions of  393, 297 mind-independent  217, 275, 301, 446, 546, 649 odour  343–4, 347 relation of, to properties in perception  53, 160–3 as relations in Aureol  54 proto-  192, 398, 403

Subject index   917 objects of perception  19, 22, 44, 54, 320, 323, 393, 461, 464, 484, 488–9, 747–8 contrasted with object perception  393 in Aristotle  44 in early modern philosophy  82–4, 91–3 in Husserl  140 as ideas  91 own bodies as  516, 523 in speech  484–9 occasionalism 87 octaves  495, 499–500, 502–3, 507, 510–11, 686, 690–1 odours  40, 44–5, 47, 52, 82–3, 94, 162, 315, 324–5, 327–30, 333, 336, 338, 341–5, 347; see also olfaction as the object of olfaction or smell  344 olfaction  18, 82, 133, 162, 278, 315, 319, 324–9, 333, 335–6, 341–3, 345, 347, 765 active vs. passive  342–3 as a dual process across orthonasal and retronasal pathways  324–30 function of  343 location in  342 processing in the brain of  343–6 role of attention in  345–6 and spatial awareness  343 opponency, colour  434, 682, 684 orientation  220–3, 265, 361–2, 466, 513, 599, 607, 735, 761, 815 as an intrinsic property in perception 443–6 as linked to recognition rather than initial perception 451 as linked to behaviour disposition  453–4 as a mode of presentation in perception 446–9 of the visual field  19, 147, 442–4, 448–53 out-of-body experiences  363, 518–27, 614 and failure to synthesize experience from different modalities  521 neurophysiology of  519–21 own-body perception  515–27 contrasted with bodily self-experience  515 as distinct from perception of external objects 172–4 in Husserl  142 and self-location  515–23, 526

and sensorimotor integration  524–5 pain  19, 32–3, 121–2, 133, 299, 307, 352–4, 372, 530–41, 572–5, 578–80, 595–6, 754, 785, 793 folk theory of  530–2 individuation of, within the senses  572 introspection of  595–6 localization of  133n29, 530–2 physiology of  535–8 as partly unconscious  372 as representational  533–4 as sense-data  532 Study of Pain definition of  531 subjectivity of  531, 533, 537–8 perceiver-accessible criterion  574–5, 579 perceptible properties  34–6, 39, 44–8, 405, 418, 735, 746 Aristotle’s theory of  44–5 perception, embodied 169, 171–2, 177, 179– 80, 306–7, 2, 359, 524–7, 614 perception of space, see spatial perception; orientation perception of time, see time perceptual constancy  14, 20, 108, 136, 148, 179, 344, 622, 625, 630, 632–5, 637–8, 695–6, 700–1, 714, 748, 760, 845 cognition and  632–4 of colour  429, 433, 621, 625, 628–34, 695, 760 computational models of  630–2 in Husserl  139 illusions and  695 in Merleau-Ponty  143 models of, in information theory  747–8 of odour  344 vs. perceptual contrast  624–8 of position  442, 448, 454–6 as a relation between stimulus and sensation  112, 117 of size  106–8, 624, 634 of shape  634 of sound  621, 624 perceptual demonstrative thought  188, 196, 204, 809, 833–51 acquaintance theory of  842–3 audition and  276–8 attention and  837–9 belief formation and  844 comprehensiveness of link underlying  835

918   subject index perceptual demonstrative thought (Cont.) definition of  833 descriptivist theory of  840–2 directness of link underlying  834–5 non-veridical cases of  839 role of reliability in  847–51 sortals and  836–7, 840 perceptual learning  21, 111, 133, 343, 668–9, 803n55, 812–32 as driven by attention  819–21, 824 as driven by cognition  813–15, 826–7 associative  570–1, 748, 828 in audition  816 in colour vision  823–5 composition and chunking as an example of 821–3 differentiation of complexes as an example of 823–5 definition of  813 examples of  814–15 functional evidence for  815–7 intentional 826–8 neurological evidence for  815–16 and speech  475, 477, 481–3, 491 in taste  6 in vision  813, 816–20, 822–3, 826 perceptual organization  394, 817, 825 Perceptual System Criterion, see PSC perceptual verbs  24, 237, 251–2 three kinds of  238–41 peripheral vision  161, 267, 297, 561, 837–8, 841–2, 851 phenomenal consciousness  355, 590–1, 732, 747, 750 phenomenal experience  40, 59, 193, 377, 379, 591, 857 non-representational 248–9 phenomenology  21–3, 136–49, 300, 378–80 disjunctivism and  210 as encompassing more than experience 376 phenomenal character of experience  208, 477–80, 484, 783, 790, 803 of attention  599, 601 of colour  428–30, 434, 498 decomposition  323, 328–9 explanation of approach in  136

of flavour and taste  317, 331, 337 of music  495–8, 509, 511–12 of pain  534 reflected in perceptual reports  245–51 representational   248–50 of smell  346–8 of speech perception  475–8 of time  460, 472–3 and unity of experience  139 visual  226, 247, 270–1, 445–6, 490, 614 phonemes  6, 9, 20, 480–3, 485–8, 576–7, 608, 614, 689, 773, 821, 826 definition of  6, 480 photoreceptors  258, 261–3, 863, 866 pictures  134, 871–8, 880–4 absences and  552–4 formal syntactic  708–10 Goodman’s theory of  874 as objects of experience  873–80 representationality of  873, 877 pit organs  863–4 pitch  11–12, 20, 105–6, 274–5, 287–8, 479, 482, 484, 488, 496–7, 502–3, 507–9, 512, 679–80, 682, 686–90 definition of  502 numerical vs. topological models of 506–12 relation to ear structures  503–6 spaces 686–7 plasticity of perception 606–7 Plato  11–12, 29, 37–48, 50, 81 as a causal theorist  38–40 on the content of perception  40–2 on meta-perception  42 presentism 460–4 definition of  461 and time lag  460–4 Presocratics  30–7, 47 pressure  11, 14, 17, 162, 158, 294–306, 308–9, 317, 330, 336, 399, 429, 577–8, 604, 612, 614 direct perception of 297–9 location of 303–4 perception of  17, 298–9, 353 primary colours  33, 35, 110 Democritus on  33, 35

Subject index   919 primary qualities  69, 82–4, 94, 405–11, 415, 425–7, 435, 544 as resembling their objects  94 compared with common sensibles  82 defined in terms of perceiver-independence 409 examples of  405 Locke’s definition of  83, 407–9 compared with mechanical properties 407 prior probabilities  401, 631, 697–700, 702–3, 706 priority  55, 300, 557, 722–3, 872–5 time-lag  460, 463–4 vampire bat  865–6 proper sensibles  11, 17, 52, 60–1, 82, 402, 418, 490, 575, 605 definition 52 properties  245–9, 405–19; see also objects; flavour; size; colour; odour; position; primary qualities disjunctive  415, 417 explanation of perceptibility of  58, 60–2, 82, 84 hallucinations and  207–9 mechanical vs. non-mechanical  406–7 in medieval philosophy  51–5 as part of sense-data  120–2 perceptible  39, 318, 341, 488, 594 relation of, to objects in perception  36, 88, 140, 160–3 sortal 841 propositional attitudes  159, 204, 228, 594, 664–5, 766 propositional content, see content of perception propositions  11, 125, 140, 203–4, 206–7, 428, 463–4, 555–6, 594, 690, 782–6, 789–90, 792–8, 800 proprioception  19, 82, 169, 176, 296–7, 309, 353–6, 361–8, 445, 517, 520, 523, 532, 607, 642 relation to touch  296–7 sources of  364 prototypical sense modalities  354, 358, 362, 364, 367 PSC (Perceptual System Criterion)  581–2

psychophysics  106, 109, 266, 331, 333, 358, 415, 431, 433, 436–7, 628, 682 Fechner’s three methods of  107 Pure word deafness, see PWD PWD (pure word deafness)  483, 489, 493 qualia  36, 48, 383–4, 386, 412–16, 418 qualities, see properties rationality  783, 804, 857 in animals  10, 569–70 reason, see cognition Receptoral Image Model, see RIM reception  863, 866 vs. detection  863 receptors  3–4, 15–16, 18, 32, 295, 315, 329, 331, 333, 355–6, 428–31, 577–8, 631–2, 864, 866 kinds of, within the body  355–6 recognition  141, 143, 265, 267, 277–8, 342–3, 378–9, 382, 512–13, 686, 761, 763, 822, 829, 831 recurrent processing, see RP reference  23–4, 169, 172–8, 189–90, 277, 594–5, 836–7, 840–1, 850 in Brentano  137, 140 of colour terms  411, 416 egocentric frame of  441, 446–57, 521, 525 frame of  304, 308–9, 362–3, 366, 603 inscrutablity of  833n2 reflectance properties  409, 415, 496 Reid, T.  74, 83, 87, 89–91, 94–5, 102–3, 111, 424, 605 arguments of, against Descartes  86 relationalism  154–5, 158, 703–4 Bayesian arguments against  703–4 contrasted with representationalism and enactivism 154–5 definition of  154 relativism about perception  69–70, 77 reliabilism  76–7, 207, 799–802, 806, 810 arguments against  800–2 definition of  799 reliability of perception  11–13, 52, 60, 62, 71, 76–7, 82, 271, 300, 782, 792, 799, 801, 804, 807, 848, 851 in early modern philosophy  94–7

920   subject index reliability of perception (Cont.) in medieval philosophy  60–4 in Plato  38 role of, in demonstrative thought  847–51 representationalism  153–6, 158–9, 217, 706–12 about audition  287–91 arguments in favour of  155–9, 245–51 contrasted with relationalism and enactivism 154–5 definition of  153 Descartes on  153–6 about interoception  357–8 about pain  534 and synaesthesia  649–51 representations  23, 102, 171, 703, 705, 743–6, 749–51, 872–4 Bayesian model of  702, 704 compared with art  872–4 perceptual and non-perceptual  159–60 iconic 220 in Plato  42–3 in Spinoza  87 unconscious 381–4 and veridical experience  208, 213 content of  160, 173–9, 181–95, 245, 249–50, 534, 650–1 representative realism definition of  112 resemblance  306, 558, 562, 744n19, 828, 874, between ideas and objects  87, 90, 94, 120–1, 408–10, 426 between ideas of primary qualities and those qualities  408 response bias  718–20, 724 retina  2, 7, 260–4, 429–33, 437–9, 621, 694–5, 698, 702, 706, 709 Kepler’s model of  85–6 nineteenth century empirical evidence about 103–5 physiology of  191–2 retinal image  2–3, 6–7, 20, 85–6, 89, 101–2, 104–5, 111, 258, 261, 263, 429, 432–3, 436, 438 retronasal olfaction  18, 323–30, 335, 337, 341, 578–9

rhythm  5–6, 495–8, 512 and pattern  512–3 definition of  498 RIM (Receptoral Image Model)  2–5, 9, 12 RP (recurrent processing)  192 rubber hand illusion  8, 172–3, 179, 305–7, 487, 515–18, 522, 527, 529, 608, 614 definition of  516 Russell, B.  74–5, 111–12, 120–1, 442–4, 532, 613–4, 638 on the content of propositions  154, 156, 160, 162, 183, 702, 711 on the orientation of spatial perception 444 secondary qualities  14, 69–70, 82–4, 94–6, 101, 109, 355, 405–21, 426–7, 435, 544, 746 arguments for colour being one of the 411–16 compared with proper sensibles  82–3 early account of, in Sextus Empiricus  69 examples of  405 extended to apply to non-visual modalities 417–19 Locke’s definition of  83, 407, 410 selection  222, 589–93, 597–8 sexual  315, 345, 496 self-consciousness  367, 515–16, 518, 523, 528 self-knowledge  42, 808 self-location  171–2, 515, 519, 522–3, 525–7 self-ownership 172–4 sense-data  22, 119–24, 127–9, 132, 134–5, 137, 201–2, 239, 393, 532–4, 538, 596, 833 compared with the notion of an idea  119 compared with the notion of an appearance 130 and pain  532–4 problems with theory of  129–34 as relative to the observer  120–4 three views of  119 sense organs  44n46; see also ear; eyes; nose sense organ account  659, 664 sensible species  56–9, 84, 424–5, 429 definition of  57 sensorimotor integration  516, 524–5

Subject index   921 in Merleau-Ponty  147 sensory processing  84, 96, 577–8, 662, 664, 770, 816, 821 sensory substitution  21, 577, 660, 662–3, 665–6, 668–70, 672 as allowing detection but not reception 664–5 brain responses in  666–8 and echolocation  666 as replacing the substituted sense  668–72 sensory substitution devices, see SSDs Sextus Empiricus  14, 69–70 shadows  393, 397, 402, 429, 434, 436, 542, 548, 559, 561, 631 shape  606, 613 signal detection theory, 717–32 consciousness explained within  726–32 distinction between noise and signal in 720 early developments in  717–19 outline of calculations in  720–5 as non-binary  720, 724–5 similarity as a property detected by the senses  41 relations  20, 174, 491, 679–80 spaces  444, 679–81, 683, 685–91, 693 surface–scene 878–80 similarity spaces  444, 679–93 colour, see colour space defined qualitatively  683–4 defined on the basis of physical characteristics 682–3 defined on the basis of averaged judgments 684–5 hierarchical definition of  680 pitch 686–7 warpings of  687–91 skepticism  4–5, 13–14, 21–2, 64–5, 78–9, 94, 99, 121, 199, 205, 379–80, 414, 786–8, 802, 809 Descartes’s response to  71–3 in Academic thought  67–8 dogmatic response to  76 externalist response to  76–7 Moore’s response to  75 relationship to perceptual relativity  69–70

in Stoicism  66, 68 smell  319-49; see also olfaction distinguished from olfaction  316 social perception  218, 229–33, 513 and mirroring  230–2 sortals  837, 841–2, 850 sounds  4, 14–17, 19–20, 82–6, 237–40, 274–93, 335–8, 405–7, 460–2, 477–89, 491–7, 499, 501–3, 505, 512, 576–8, 680 as events  281–4, 287–8 instrumental 499–500 non-linguistic  476–7, 479–80, 484, 489–90 non-speech  477–9, 490 non-veridical 290–1 relation to vibrations  279–81 as temporally extended  275 space  103–4, 142–4, 305, 307–10, 441, 443–7, 449–53, 466–7, 519, 521–3, 525, 546, 553, 687–8; see also perception of space empty  308, 401, 543, 553 extra-personal  517, 519 oriented  442–3, 447, 450 peripersonal  221, 308, 617 structure of  102–4, 260–1, 265, 844 spatial experiences  142, 169, 466, 643 spatial perception  88, 90, 99–101, 104–7, 361, 441–3, 446, 456, 673 cross-modal 444 as egocentric  453–5 inverted 444 phenomenology of  443, 449, 451 as tied to a frame of reference  441–3 of three-dimensional objects  88 Special Properties Criterion  575–8 species theory of perception Descartes’s arguments against  84 Hobbes’s arguments against  85 speech  6, 16–7, 286, 320, 479–83; see also phonemes; speech perception across different languages  477–8, 481–2 compared to other sounds  17, 485–7 motor theory of  230, 493 sinewave 477–9 speech perception  477–92, 820 contents of  478–84 as cross-modal  487

922   subject index speech perception (Cont.) disorders of  483 in infants  482 mimicking and  24 modularity of  490–1, 763–4 motor theory of  230 objects of  484–9 phenomenology of  477, 479 processing of  489–91 spiritual change  56 SSDs (sensory substitution devices)  659–60, 664–5, 668, 672–3; see also vOICE; Brain-Port capabilities of users of  661–2, 668–9, 672 state nonconceptualism  182–4 stability  624, 629–30 of behaviour dispositions  453 of visual experience  193, 227, 593 Stoics  10–13, 29, 34, 67–8, 72 streams, ventral  156, 221–5, 265, 267, 272, 382 subjective probabilities  696, 700, 709 subjective threshold approaches  728, 731 subjectivism  321–2, 338, 340–1, 411, 414–15, 549, 559 in the Presocratics  35–6 subpersonal level  455–6 substitution of the senses, see sensory substitution sweetness  18, 59, 328, 331–5, 338, 578, 679 synaesthesia  21, 147, 604–5, 617–18, 640–58, 771, 775, 777–8, 830–1, 854, 862 of audition and vision  641, 650 definitions of  640, 643 distinguished into lower and higher kinds 642–3 as non-veridical  643–5 of number and location  642 as partly mind-dependent  648–9 processing of  651–2 and its relationship to its component experiences 645–7 of taste, space and touch  642 tactile-visual sensory substitution, see TVSS taste  18, 316–18, 321–48, 578–80 as a crossmodal or multimodal experience  331–2, 578–9

as the product of inputs from the tongue, touch, and smell  317 as triggering tactile sensations  642 divided into basic tastes  18, 323, 330–1, 333–4 experienced in the tongue or oral cavity 323 influenced by prior taste experiences  333 location of  325, 332 people with extraordinarily sensitive  644 perceptual reports about  237–8, 240, 252–4 Plato on  40n37 psychological contributions to  334–6 role of  316 subjectivity of  321, 333 vs. gustation  316 taste receptors  4, 323, 330, 332, 336, 572 temperature  45, 295, 299, 316, 333, 335–7, 339, 341, 355, 357, 532, 578, 604, 864 temporoparietal junction, see TPJ thalamus  263–5, 326, 328, 432, 535–7 Theophrastus  30, 33, 35–7, 48–50 thermoreceptors  355, 357, 578, 863–5 timbre  16, 20, 274–5, 479, 482, 484, 486, 488, 496–9, 501–2, 512–13, 648 definition of  499 time  304–5, 459–74 audition and  274–5, 280 causal theory of  467–71 contrasted with space in perception  466 demonstrative thought, across  836 as directional  461, 465, 467–9 illusion of  460, 469–73 objective passage of  460, 465–8, 472–3 as ordered  465, 467–9 phenomenology of  461, 468, 472–3 tone  20, 105–6, 496, 495–6, 498–9, 502–7, 510–12, 661, 679–80, 686–87, 690–3 tongue  4, 18, 30, 36, 39–40, 316–17, 322–5, 328, 330, 332–4, 336, 572, 582, 661–2, 664–6, 675 and the basic tastes  330–1 as a detector of electrical current  665 total hallucinations  200–2, 209–14 as differing in content from partial hallucinations 202

Subject index   923 as having no content  203 touch  294–309, 578–82, 604–5 active  145, 148, 578, 580–1, 604 Aristotle on  44n54 bipolarity of  300–2 compared with the interoceptive senses  353, 356, 365 distinguished from other senses by its relation to the body  294, 296–7 localization of  223n5, 302–3, 306, 308–9 moisture as an object of  45 as multisensory  579, 853 objects of  294–5 objectivity of  300–1 and olfaction  18 passive  325, 578, 581–2 perceiving absences through  542–3 Plato on  38, 39, 41 pressure theory of  297–301 spatiality of  304, 308 taste or flavour and  317, 330, 335–7, 578 template theory of  295–7 compared with vision  89, 90, 111, 308, 487, 517 tracking  171–2, 357, 364, 398–400, 582, 632, 671, 755 TPJ (temporoparietal junction)  519–23 training, see perceptual learning transducers  45, 354, 356, 429–30, 571–3, 577–9, 581–3, 603, 622, 630–1, 756, 762 transparency  130, 132, 269, 358, 593, 595, 600, 651 TVSS (tactile-visual sensory substitution)  582, 669, 671 types, formal syntactic  708–9 unconscious perception  19, 158, 371–87, 569, 726, 728–9, 733, 782, 789, 792 arguments against the existence of  378–80 as unavailable to executive processing 384–6 contrasted with conscious perception  383 evidence for  373–7 extent of, in vision  381–3 perceptual learning and  826 signal detection theory, and  719 varying grades of  374–7

unique hues  685, 692 unity of consciousness  41n42, 616, 775 value  22, 871–2, 874–5, 880–2 aesthetic 881–2 artistic  872, 881–2 hedonic  316, 338 veil of ideas  57, 91–3 relation to skepticism  91 veridical perception  22–4, 60, 124, 169–70, 172, 175, 187–8, 190, 198–213, 208, 210, 285, 291, 423, 425, 462, 599–600, 704, 707, 800 definition 199 vestibular sense  353–4, 361–3, 365, 525 definition of  361 phenomenology of  363 representationality of  361–2 vibrations  15, 17, 275, 279–85, 291–2, 295, 299, 335, 427, 487, 604, 661, 864–5 of objects  280–1 relation to sounds  279–81 vision  257–74; see also colour vision and absences  543–4, 546–8 binocular  105, 263–4, 692 divided according to anatomical area and function 381 divided into dorsal and ventral streams  221–2, 265–6 divided into action-guiding and object recognition processes  367 as dominant over proprioception  517 as dominant over touch  517, 610 intromissionist theories of  260 as the model for the other senses  257 objects of  269, 88–9, 269, 398, 444–5, 582, 761, 849–50 physiology of  104–6, 112–13, 258–61; see also eyes processing of, in the retina  262–4 processing of, in the brain  22, 191–4, 218, 221–5, 261–8, 379–82, 394–6, 433, 437, 663–8, 767, 771, 773, 838 sensorimotor account of 674 stereoscopic  105, 263–4 visual representations  17, 379, 381, 521, 762, 764, 778

924   subject index visual representations (Cont.) unconscious  381–2, 659 visual cortex  7, 222, 318, 607, 662–3, 666–8, 674–5, 815, 829 visual system, see vision, physiology of; vision, processing of, in the retina; vision, processing of, in the brain vOICe  661, 669 vomeronasal sex detection, see VSD VSD (vomeronasal sex detection),  573–4 arguments against its membership in the senses 573–4

warmth  11, 45, 47, 405–7, 418–19, 572, 863, 865 as a perceptible quality  45 warping 687–90 Wittgenstein  19, 119, 122, 129, 134, 443–5, 448–9, 453, 544, 760, 778 on the perception of orientation  444–6, 453 Young-Helmholtz theory of colour perception 429–31