Cognitive Semiotics: Integrating Signs, Minds, Meaning and Cognition 3030429857, 9783030429850

Citation preview

Perspectives in Pragmatics, Philosophy & Psychology 24

Claudio Paolucci

Cognitive Semiotics

Integrating Signs, Minds, Meaning and Cognition

Perspectives in Pragmatics, Philosophy & Psychology Volume 24 Editor-in-Chief Alessandro Capone, University of Messina, Italy Consulting Editors Keith Allan, Monash University, Australia Louise Cummings, The Hong Kong Polytechnic University, Hong Kong Wayne Davis, Georgetown University, Washington, USA Igor Douven, University of Paris-Sorbonne, France Yan Huang, University of Auckland, New Zealand Istvan Kecskes, State University of New York at Albany, USA Franco Lo Piparo, University of Palermo, Italy Antonino Pennisi, University of Messina, Italy Francesca Santulli, Ca' Foscari University of Venice, Italy Editorial Board Members Noel Burton-Roberts, University of Newcastle, Newcastle, Australia Brian Butler, University of North Carolina, Asheville, USA Marco Carapezza, University of Palermo, Palermo, Italy Felice Cimatti, Università della Calabria, Cosenza, Italy Eros Corazza, Carleton University, Ottawa, Canada Marcelo Dascal, Tel Aviv University, Tel Aviv, Israel Michael Devitt, City University of New York, New York, USA Frans van Eemeren, University of Amsterdam, Amsterdam, The Netherlands Alessandra Falzone, University of Messina, Messina, Italy Neil Feit, State University of New York, Fredonia, USA Alessandra Giorgi, Ca’ Foscari University of Venice, Venice, Italy Larry Horn, Yale University, New Haven, USA Klaus von Heusinger, University of Stuttgart, Stuttgart, Germany Katarzyna Jaszczolt, University of Cambridge, Cambridge, UK Ferenc Kiefer, Hungarian Academy of Sciences, Budapest, Hungary Kepa Korta, ILCLI, Donostia, Spain Ernest Lepore, Rutgers University, New Brunswick, USA Stephen C. Levinson, Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands Fabrizio Macagno, New University of Lisbon, Lisbon, Portugal

Jacob L. Mey, University of Southern Denmark, Odense, Denmark Pietro Perconti, University of Messina, Messina, Italy Francesca Piazza, University of Palermo, Palermo, Italy Roland Posner, Berlin Institute of Technology, Berlin, Germany Mark Richard, Harvard University, Cambridge, USA Nathan Salmon, University of California, Santa Barbara, USA Stephen R. Schiffer, New York University, New York, USA Michel Seymour, University of Montreal, Montreal, Canada Mandy Simons, Carnegie Mellon University, Pittsburgh, USA Timothy Williamson, University of Oxford, Oxford, UK Anna Wierbizcka, Australian National University, Canberra, Australia Dorota Zieliñska, Jesuit University of Philosophy and Education Ignatianum, Kraków, Poland More information about this series at http://www.springer.com/series/11797

Claudio Paolucci

Cognitive Semiotics Integrating Signs, Minds, Meaning and Cognition

Claudio Paolucci Department of Philosophy and Communication Studies University of Bologna Bologna, Italy

ISSN 2214-3807     ISSN 2214-3815 (electronic) Perspectives in Pragmatics, Philosophy & Psychology ISBN 978-3-030-42985-0    ISBN 978-3-030-42986-7 (eBook) https://doi.org/10.1007/978-3-030-42986-7 © Springer Nature Switzerland AG 2021 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

Preface

In this book, I advance a proposition concerning cognitive semiotics, and I try to shed light on some key problems in contemporary debates in semiotics, philosophy and the cognitive sciences such as subjectivity, representations, beliefs, perception, imagination, social cognition, mind and language. In my view, cognitive semiotics should not be conceived as a special type of semiotics but as an attempt to answer the following question: “How can we come to know the world through signs and languages?”. I am inspired in this by Umberto Eco and by his ideas on semiotics. Indeed, when Umberto Eco reflected, in old age, upon what might have been the driving idea of his work and what theme had obsessed him through the entirety of his scientific activity, he identified with great clarity the cognitive problem of “how we come to know the world”. “I think the theme I have addressed in an almost obsessive way, throughout all my work, is that of how we come to know the world. Perhaps this explains the growing attention I have dedicated to narrative universes: these constitute an Ersatz, a substitute for the world. (Eco 2004: 199) I do not want to be the critic or the reviewer of myself, but think of my novels: the issue in The Name of the Rose is a truth to uncover; Foucault’s Pendulum discusses how to construct a nonexistent world; The Island of the Day Before examines a world that exists, but the contours of which are still unclear. My work in semiotics, in the end, concerns the problem of how our signs give account of that which is or construct that which is not. My interest in avant-garde phenomena springs from the interest for a language that tends to break down and recreate our worldview. The years I have spent analysing mass media likewise concern the problem of truth (Eco 2001: 616–7)”.

Eco seems to suggest that “cognitive” semiotics is not, after all, a kind of semiotics but the vocation of semiotics itself or, at least, the vocation of the kind that he practiced. If the word “cognitive” refers to the question of “how we come to know the world”,1 and if only through signs, meanings, texts and languages are we able to  “Cognitive” and “cognition” are not thought in opposition to “emotion” or “passion” and do not recall in any way intellectual or rational values as opposed to “non-intellectual” or “non-rational” ones. Fear, feeling, emotion and passion are all cognitive forms that enable us to know the world (cf. Caruana and Viola, 2018), as Peirce forcefully argued in his anti-Cartesian essays. 1

v

vi

Preface

“give account of that which is” or to “construct that which is not”, then the object of cognitive semiotics is the way in which semiotic systems represent the background of our perception of the world and define the conditions under which cognition and knowledge are possible. It is not by chance that this topic represents the “almost obsessive way” of all of Umberto Eco’s work. It is, after all, a classical option embraced by semiotics in its totality; when Saussure, Hjelmslev and Greimas said that meaning could not be determined based upon the referent’s structure or upon the extralinguistic universe, this was because the problem was not that of investigating the relationship between the world and languages or questioning their significance within a given culture. On the contrary, the problem lays in focusing on the ways in which languages can provide a scaffolding for human cognition, directing our points of view and extending and strengthening our cognitive possibilities and ability to know the world.2 Peirce made a similar argument when—in open polemic with the phrenology of his era which sought to locate cognitive functions within areas of the brain—he stated that “in my opinion, it is far more true that the thoughts of a living writer are in any printed copy of his book than that they are in his brain” (CP 7.365). Far from arguing that anatomy does not matter, Peirce was instead putting forth the cognitive semiotics option, which represents a true philosophical antecedent to the extended mind thesis (cf. Paolucci 2011; Iliopoulos 2019): to study texts and languages that shape thought and express it is to study cognition and the way we think in a profound and non-­trivial way. For this reason, in my opinion, cognitive semiotics deals with the way in which a semio-linguistic reflection upon signs, meaning and language can shed light on both the ways in which we arrive at an understanding of the world through cognition and action, and our ability to make sense of our experience. Therefore, cognitive semiotics is committed to partecipating in contemporary debates concerning cognition, since many cognitive scientists nowadays face frequent semiotic dilemmas and deal with semiotic problems under a different name or rubric. In my view, in order to do that, cognitive semiotics has to be presented as a strong theoretical proposition, able to take sides in the current debate on cognition and to represent a reference point for both semiotic researchers interested in the cognitive sciences and cognitive scientists in search of valid, trustworthy knowledge about signs, meaning, language and cognition. This book presents this strong theoretical proposition concerning cognitive semiotics in the first chapter. Cognitive semiotics must be conceived as a “theory of the lie” that keeps together Radical Enactivism, Pragmatism and Material Engagement Theory. In the second chapter, I deal with a classic problem of philosophy—the problem of subjectivity—and I try to show how this problem can be better posed and solved

2  This is the correct way of reading Saussure’s adage, often confused with the Sapir–Whorf hypothesis, that thought is an amorphous mass prior to the appearance of language.

Preface

vii

through a semiotic theory of impersonal enunciation that deals with subjectivity in language. In the third chapter, I outline the features of a “semiotic mind”, starting from an interpretation of Peirce’s semiotics. From this perspective, based on an extended cognition framework grounded on signs and languages, I deal with three key notions at the core of the contemporary debate: habits, beliefs and representations. The fourth chapter is dedicated to the problem of social cognition and outlines the crucial role of semiotic narratives in explaining our capacity for attributing meaning to the others’ actions and for understanding, through semiotic interaction analysis, social cognition impairments like Autism Spectrum Disorders in infants. The fifth chapter deals with a theory of perception conceived as “controlled hallucination” or “controlled figuration”. Starting with the semiotics of perception outlined by Umberto Eco in Kant and the Platypus, I will deal with Andy Clark’s predictive processing and connect it with Jan Koenderink’s “Goethean” account of perception, focusing on the key role of imagination in perception. In my perspective, cognitive semiotics interprets cognition as (1) an enactive form of sense-making, involving interaction with the external world; (2) a form of action mediated by meaning, where meanings are not representations of the world or truth conditions, but interpretive habits and sense-making activities; (3) a perspective in which texts, languages and semiotic systems represent not the expression of a pre-existing thought located in our heads, but forms that structure the way in which we think and know reality, or as cognitive scaffolding which represents the background of our perception of the world. Bologna, Italy December 2020

 Claudio Paolucci

Acknowledgements

This book would not have been possible without the contribution of three people. Firstly, Shaun Gallagher, who has honoured me with his friendship and constant support during all these years. The periods spent at the University of Memphis in 2016 and 2018, where I had the possibility to give lectures and seminars to its extraordinary students, were the most important moments for refining the ideas of this book. Secondly, Antonino Pennisi, to whom I owe the gratitude for having convinced me that the connection between semiotics and the cognitive sciences—inaugurated by Umberto Eco in Kant and the Platypus—was fundamental. I owe a great deal to the many opportunities for discussion over the past 10 years at the University of Messina, with him and with his talented students, who showed me that a relationship between cognitive sciences, semiotics and philosophy of language was not only possible, but unavoidable. Finally, Lambros Malafouris, with whom I have built a truly special relationship over the last few year. The fellowship with the University of Oxford and the many opportunities for confrontation related to the ERC Handmade project allowed me to develop some of the extraordinary ideas in his Material Engagement Theory in my own approach to semiotics. A special thanks goes to the great generosity of Vittorio Gallese, who has always valued my work throughout all of these years, discussing it with me and reading my publications and my projects while supporting them publicly. I would also like to thank Paolo Leonardi, Patrizia Violi, Giovanni Matteucci, Dan Hutto, Andy Clark, Jean Petitot, Fausto Caruana, Pietro Perconti, Marco Carapezza, Göran Sonesson, Ruggero Eugeni, Rossella Fabbrichesi, Francesca Piazza, Antonis Iliopolus, Stefano Gensini, Alessandro Sarti, Rocco Ronchi, Liliana Albertazzi, Andrea Pinotti, Francesco Parisi, Marco Viola, Alessandra Falzone and Jordan Zlatev for discussing some of these ideas with me and reading some parts of this work.

ix

x

Acknowledgements

Another thank you goes to the young semioticians of the University of Bologna, who supported me in the drafting and revision of this book: Martina Bacaro, Paolo Martinelli, Laura Pielli, Paola Columbano, Luigi Lobaccaro, Gabriele Giampieri, Marta Caravà, Flavio Valerio Alessi and John Sykes. This book owes a great deal to the PRIN 2015 project on Perception, Performativity and the Cognitive Sciences, funded by the Italian Ministry of University and Research. A big support came also from the European Commission that funded my NeMo project (NEw MOnitoring Guidelines to Develop Innovative ECEC Teachers Curricula, Grant No: 2019-1-IT02-KA201-063340) on embodied interactions. Finally, the Research Center for Enactivism and Cognitive Semiotics of the University of Memphis contributed decisively to this book. Special thanks go to Mia Burnett for her extraordinary work, to the University of Memphis and, once again, to Shaun Gallagher who made so many things possible.

List of Abbreviations

Hjelmslev, Louis T. CC 1935. La catégorie des cas. Étude de grammaire générale. Acta Jutlandica, VII, pp. I–XII e pp. 1–184, I. Aarhus: Universitetsforlaget. NE 1985. Nouveaux essais. Paris: PUF.

Peirce, Charles S. CP

Collected Papers of Charles Sanders Peirce, vol. I–VI edited by Hartshorne, C. and Weiss, P. 1931–1935. vol. VII–VIII edited by Burks A. W. 1958. Cambridge, MA: Harvard University Press. W Writings of Charles Sanders Peirce. A Chronological Edition. 7 vols. Eds. Moore, E. C., Kloesel, C. J. W. et al. 1982. Bloomington: Indiana University Press. LW Semiotic and Significs. The Correspondence Between Charles Sanders Peirce and Victoria Lady Welby. Edited by Hardwick, C.  S. 1977. Bloomington: Indiana University Press. NEM The New Elements of Mathematics by Charles S. Peirce. vol. I–IV. Edited by Eisele, C. 1976. Mouton: The Hague. RLT Reasoning and the Logic of Things, edited by Ketner, K.  L. 1992. Cambridge, MA: Harvard University Press. MS Manuscripts in the Houghton Library of Harvard University, as identified by Richard Robin. 1967. Annotated Catalogue of the Papers of Charles S. Peirce. Amherst: University of Massachusetts Press. And in 1971 “The Peirce Papers: a Supplementary Catalogue”. Transactions of the C.  S. Peirce Society 7: 37–57.

xi

Contents

Cognitive Semiotics. Radical Enactivism, Pragmatism and Material Engagement������������������������������������������������������������������������   1 1 The Threshold Between Nature and Culture������������������������������������   1 2 The Three Dimensions of Cognitive Semiotics��������������������������������   6 3 Enactivism, Sense-Making and Pragmatism������������������������������������  10 4 Cognitive Semiotics Sense-Making��������������������������������������������������  13 5 Languages and Cognitive Systems ��������������������������������������������������  18 References����������������������������������������������������������������������������������������������������  22 For a Cognitive Semiotics of Subjectivity��������������������������������������������������������  27 1 Subjectivity in Language������������������������������������������������������������������  27 2 For an Enactive Semiotics of Subjectivity����������������������������������������  32 3 The Pre-Logical Mind����������������������������������������������������������������������  37 4 Persona. The “I”, “You”, “He” Hierarchy����������������������������������������  43 5 The Primacy of ‘He’ ������������������������������������������������������������������������  48 6 Illeity ������������������������������������������������������������������������������������������������  50 References����������������������������������������������������������������������������������������������������  59 The Semiotic Mind. Beliefs, Habits and Extended Cognition������������������������  63 1 Anti-Cartesian Semiotics������������������������������������������������������������������  64 2 Cognitive Semiotics and Pragmatism: Beliefs and the Extended Mind ��������������������������������������������������������������������  68 3 The Semiotic Extended Mind (1): Parity Principle and Cognitive Intertwining ��������������������������������������������������������������  76 4 The Semiotic Extended Mind (2): Phenomenology��������������������������  81 5 Semiotic Representations������������������������������������������������������������������  89 References����������������������������������������������������������������������������������������������������  92 Social Cognition and Autism Spectrum Disorders: From  Mindreading to Narratives������������������������������������������������������������������������  97 1 Early Mindreading and Narrative Practices��������������������������������������  97 2 Interaction Theory and Narrative Practices��������������������������������������  99 xiii

xiv

Contents

3 Radical Enactivism, Interactive Specialization and Social Cognition����������������������������������������������������������������������  103 4 Two Discoveries������������������������������������������������������������������������������  105 5 A Two-Levels Model. Low-Level: Action/Perception/ Imagination Matching��������������������������������������������������������������������  108 6 A Two-Level Model. High-Level: Narrativity��������������������������������  110 7 Narrativity and Developmental Trajectories ����������������������������������  115 8 From Deception to Pretend Play and Language Acquisition����������  118 9 Autism Spectrum Disorders and Cognitive Semiotics��������������������  119 References��������������������������������������������������������������������������������������������������  122 Perception as Controlled Hallucination ��������������������������������������������������������  127 1 Semiotics of Perception������������������������������������������������������������������  127 2 Kant, the Platypus and the Predictive Processing ��������������������������  129 3 Diagrammatic Thinking, Inference and Perception������������������������  132 4 The Bootstrap Problem and the Metaphor of Vision����������������������  138 5 The Laws of Imagination����������������������������������������������������������������  142 6 Perception as Controlled Hallucination������������������������������������������  146 7 Perception, Imagination and Narratives������������������������������������������  149 References��������������������������������������������������������������������������������������������������  156

Cognitive Semiotics. Radical Enactivism, Pragmatism and Material Engagement

1  The Threshold Between Nature and Culture As suggested by Goran Sonesson (2012: 208) in a seminal article, “cognitive semiotics has been invented many times during the last decades”. Cognitive semiotics can indeed be many things, and there are many ways to “invent” it. Some of these things and ways are de facto. Other things and other ways are de iure. I will not proceed to create a new invention. Rather, in this book I will try to clarify the articulation between what cognitive semiotics is de facto and what, in my perspective, should be able to be de iure. De facto, there is an international association of cognitive semiotics that brings together scholars from different disciplinary fields, encouraging their exchange and interaction. De facto, Cognitive Semiotics is an international journal in semiotics with an important history, many important papers in different disciplinary fields and a great international prestige. De facto, cognitive semiotics is a research direction practiced worldwide by many scholars who aim to link cognitive science with semiotics. All of this is of fundamental importance, and it entertains many of the connections that we will discuss in this book, which, however, is not about what cognitive semiotics currently is (de facto), but about what cognitive semiotics would and should be in the future (de jure). In my opinion, cognitive semiotics has to deal with the way in which a semio-linguistic reflection on signs, meaning and languages is able to make more intelligible the way in which we come to know the world through action and sense-making, and has to strongly commit to discussing the contemporary debate on cognition. This research perspective has a counter-effect. Since we engage with many heterogeneous ways of thinking that may be extraneous to the semio-­linguistic tradition, our comparisons between semiotics and the contemporary debate on cognition will entail important effects on the structure of semiotic theory itself. Also, this research perspective is needed because, as it often happens

© Springer Nature Switzerland AG 2021 C. Paolucci, Cognitive Semiotics, Perspectives in Pragmatics, Philosophy & Psychology 24, https://doi.org/10.1007/978-3-030-42986-7_1

1

2

Cognitive Semiotics. Radical Enactivism, Pragmatism and Material Engagement

in the current landscape of 4e cognition,1 many scholars in the cognitive sciences address semiotic problems by doing semiotics without calling it with this name.2 To do this, cognitive semiotics cannot be only a political attempt to keep together existing things. Nor can it be the attempt to mediate between distant instances, with the goal of establishing a dialogue between them. To be clear, I believe that both these dimensions are fundamental and that they have a decisive importance in the current international landscape of the sciences of language and cognition. Nonetheless, in order to make a contribution to what cognitive semiotics should be de jure, we need to present cognitive semiotics as a strong theoretical proposal, capable of taking a stand in the current debate, and of representing a reference point for researchers in semiotics who are interested in the cognitive sciences. My mentor Umberto Eco (1997: Chap. 2), whom I continue to be inspired by in the research group I am directing at the University of Bologna, called Charles Sanders Peirce’s semiotic theory “cognitive semiotics”, which had influenced him in a deep way. For Peirce himself, what Eco is calling “cognitive semiotics” is not a species or a type of semiotics that can be distinguished from other types of semiotics (on this topic, see Zlatev 2011) nor, for Peirce, can we make a theory of cognition without semiotics. His idea, already discussed in 1868 together with a real philosophical antecedent of the theory of the extended mind (Paolucci 2011, see also infra, Chap. 3), was that “whenever we think, we have present to the consciousness some feeling, image, conception, or other representation, which serves as a sign” (CP: 5.283). Every thought was a sign, and, according to Peirce, since “we have no power of thinking without signs” (CP: 5.265), a theory of cognition was inseparable from a phenomenological semiotics that studied “everything that is present to the mind”. Describing Peirce’s cognitive semiotics and how this ambitious and original program can be achieved through the connection with pragmatism goes beyond the scope of this first chapter (Paolucci 2010, 2011, infra, Chap. 3). What we are interested in here, instead, is to draw upon how Umberto Eco developed this idea. For Eco, Peirce’s idea of sign was too broad, and it needed to be reassessed. The very notion of “sign” was too tied to commonsense, and it needed to be substituted with the idea of semiotic system. This is interesting for two reasons. First, it placed cognition under the aegis of the idea of “system”, in accordance with the enactivist tradition in the cognitive sciences, that is built from the very beginning on the idea of system (see Maturana and Varela 1980; Chemero 2009; Gallagher 2017; Hutto and Myin 2017; Di Paolo et al. 2018). Second, this idea is aligned with the revolution of linguistics, in which the idea of system was pivotal (Eco 1997; Paolucci 2017a, b). Therefore, Eco advanced a synthesis between the two main semiotic traditions, which is also discussed in Kant and the Platypus (Eco 1997; Paolucci 2010). Hence, what is this idea of “semiotic system”? What does Eco mean by this notion? 1  “4e cognition” refers to a turn inside cognitive sciences and theory of mind that thinks of cognition being as embodied, extended, embedded and enacted. For an overview, see Rowlands 2010. 2  A very important exception is Material Engagement Theory (Malafouris 2013), which explicitly refers to a semiotic perspective on cognition. More on that in this very same chapter.

1  The Threshold Between Nature and Culture

3

Eco (1975), in two passages which are close together, gives two very different definitions, which he certainly thought to be compatible, but which clearly are not. The first definition is this one: there is a semiotic system when there is the socially conventionalized possibility of generating sign-functions, whether the funtives of such functions are discrete units called signs or vast portions of discourse, provided that the correlation has been previously posited by a social convention. This is why semiotics is a logic of culture. (Eco 1975: 7)

This definition, which identifies semiotics with “a general theory of culture” and “the semiotic” with “the cultural” has had enormous success, and it has influenced a very large part of the semiotic tradition. Indeed, semiotics has often dealt with cultural phenomena, also by endorsing Barthes’ idea of a critique of ideology, which had the aim of showing the cultural dimension of what presents itself as “natural”. It is precisely because of this definition and this identification that Eco considered zoosemiotics the “lower limit of semiotics”, i.e., because “it concerns itself with the communicative behavior of non-human (and therefore non-cultural) communities”. As both Rodriguez Higuera and Kull (2017) and Gensini (2018: 97–8) have shown, this definition by Eco and its endorsement by a large part of the semiotic tradition has produced more problems than it has solved. My aim is to eschew the very idea of the threshold between nature and culture, and the corresponding identification of semiotics with “what is cultural”. This will also allow me to build semiotics as a critique of ideology—a fundamental aspect of semiotics that I want to keep3—on a purely semantic and rhetoric ground, as Eco himself did in the very same chapter on semiotics and ideology in A Theory of Semiotics. Perhaps, this is why, in the very same book, Eco gave another definition of semiotics and of its object: A sign is everything which can be taken as significantly substituting for something else. This something else does not necessarily have to exist or to actually be somewhere at the moment in which a sign stands in for it. Thus, semiotics is in principle the discipline studying everything which can be used in order to lie. […] I think that the definition of a 'theory of the lie' should be taken as a pretty comprehensive program for a general semiotics. (1975: 8)

Now, cognitive semiotics is based on explicit criticism and rejection of the first definition in favor of the second. The “theory of the lie” definition is the very program of cognitive semiotics. From my point of view, we need to transpose Eco’s definition from the idea of sign to the idea of semiotic system; if there is a system capable of lying, then it is a semiotic system. This obviously does not necessarily entail that this system is cultural or conventionally established. There are many semiotic systems that we consider “natural” and that are able to lie. For example, James Lloyd studied the case of firefly femmes fatales specialized in mimicking the mating signals of other species of fireflies, with the purpose of attracting responding males to become their prey. These aggressive acts of mimicry are a major factor in the survival and reproduction of both prey and predator. “It is a case of deception  See Paolucci 2008, Fabbri 2017.

3

4

Cognitive Semiotics. Radical Enactivism, Pragmatism and Material Engagement

through active falsification of information that leads to efficient predation by femmes fatales fireflies and triggered evolutionary processes in their preys’ communicative behaviors” (El-Hani et al. 2009; Sonesson 2018). Even if we disagree with El-Hani et al. (2009), as Sonesson (2019) does,4 if we move from sign production and sign recognition to Eco’s idea of “semiotic system”, we see that such a system enables deception. Indeed, by “theory of lie”, Eco does not mean a voluntary deception by an individual person,5 but only that there is a system that makes deceiving others possible. A system is semiotic if it allows for the construction of significant surfaces capable of lying, if it allows for the deceiving of the other, by hiding the object for effective action (see infra, Chap. 2). Hence the radically anti-­representationalist nature of semiotics; cognition does not serve to represent the world, but to effectively act in it. In order to act in the world in an effective way, it is necessary to bring forth worlds through meaning, and meanings are not representations of the world, but “tendencies actually to behave in a similar way under similar circumstances in the future” (CP: 5.487), that is, habits. We are accustomed to think of truth and falsehood as being two sides of the very same phenomenon, due to the traditions of logic and the philosophy of language inspired by Wittgenstein, Quine and Davidson. However, from a cognitive point of view, truth and falsehood are not two sides of the same phenomenon at all. Indeed, as far as executive functions, theory of mind and cognitive processing are concerned, telling the truth is a totally different cognitive operation from lying (Diana 2014; Walczyk et al. 2014). Not only does lying require more cognitive resources (Vrij et al. 2006, 2011), but it has been demonstrated that deceptive responses are typically associated with increased response times and higher error rates compared with truthful responses (Van Bockstaele et al. 2012). Significant differences between liars and truth tellers have been highlighted based on the combination of three factors: cognitive load manipulation, multimodal data collection and t-pattern analysis (for an overview, Diana 2014). Moreover, regions in the frontal and parietal cortex show higher activation when people lie compared with when they are telling the 4  Sonesson (2019) does not agree with El-Hani et al. (2009)’s interpretation, since, according to Sonnesson’s point of view, “animal mimicry, as well as phenomena of the human Lifeworld comparable to it, are in a sense the opposite of signs. It has often been observed, not only within speech act philosophy, but also by the semiotician Luis Prieto, that a sign can only function as such once it is recognized to be a sign. Animal mimicry, camouflage, and the like, in contrast, only work as such, to the extent that they are not perceived as signs”. I will come back to this important debate in a future work. 5  Indeed, in his response to some of my solicitations at a conference on A Theory of Semiotics. Eco talked about the “theory of misguided inference” and he said that, although he no longer corrected the definition of “theory of lies” because of its enormous success, he believed that the idea of semiotic features should be extended to that of “error”, namely to cases where someone is wrong and does not deliberately lie. The classic example is Ptolemy. When he said that earth stays still at the center of the universe he was not lying, but he was wrong. Or, and this is the example given by Eco (https://www.youtube.com/watch?v=gSEEYdM7k9A), when a well-known Italian philosopher translates an English passage saying that “bees take bananas with sticks” he is not lying, but he is simply mistakenly translating “apes” with “bees”, since the Italian word for “bee” is “ape”. On this topic, see Gramigna 2020.

1  The Threshold Between Nature and Culture

5

truth (regardless of whether they were asked about their past experiences or opinions, see Abe 2009, Ganis 2015 and Ofen et al. 2017). Lie-related activation in the right temporal pole, precuneus and the right amygdala differ not only if compared to telling the truth but differ also in relation to the content of the lie (Ito et al. 2012; Ofen et al. 2017). Preparing to lie activates parietal and frontal brain regions that were distinct from those activated both while telling the truth and while executing lies (Abe et al. 2008; Ofen et al. 2017). In addition, it is even more important to move from the act of deceiving and/or telling a lie to the system that allows for deception and lie telling. Semiotic systems are used to coordinate, describe, design, project, communicate, imagine, and lie. Lying is not simply telling a falsehood but telling a falsehood making it seem true. This requires a lot more work, which explains the greater resources needed in its cognitive operations. But this also requires a system with greater complexity if compared to the one needed to tell the truth. In order to lie, you have to know the truth and know what the truth might look like and do something in order to build the lie accordingly.6 As cognitive semiotics cognitive semiotics does not endorse the idea of a “lower threshold of semiotics” which Eco spoke about, since it embraces the “theory of lie” definition and since it denies that a distinction between what is semiotic and what is not semiotic depends on the threshold between nature and culture, cognitive semiotics aims to fill the gap between biosemiotics and semiotics of culture, which in the last forty years have taken completely different paths. Indeed, with some exceptions stemming from those who have strongly embraced cognitive semiotics (Stjernfelt 2007; Sonesson 2012; Fusaroli and Tylén 2014; Zlatev 2015), scholars who have worked in the field of biosemiotics have not dealt with semiotics of culture, and vice versa. This has led to another strong distinction, which this work aims to rethink. Since Sebeok’s pioneering work, biosemiotics has been strongly based on Peircean semiotics, while semiotics of culture has been strongly based on the structuralist tradition, mediated by one of its most original versions, that of Jurij Lotman. A clear exception in this landscape was Umberto Eco’s semiotics, which was intentionally founded on the attempt of bridging Peircean semiotics and structuralism. However, Eco dealt with biosemiotics mainly in order to spell out its limits and boundaries, attempting to redefine that line between culture and nature that has always been a strong temptation of his thought and of his way of defining what semiotics is and what semiotics is not. Peirce’s semiotics has much to say about cultural phenomena, just as Greimas’ semiotics has much to say about the natural ones (for example, see our discussion of Greimas’ idea of narrativity in Paolucci 2019). This is so because the distinction between nature and culture is misleading, and it must be abandoned when it comes to defining what semiotics is and what it does.

6  I owe this idea to Paolo Leonardi (personal communication), even if Paolo used it in order to argue that truth has to be considered more important than falsity, since telling a lie is parasitic upon telling the truth.

6

Cognitive Semiotics. Radical Enactivism, Pragmatism and Material Engagement

2  The Three Dimensions of Cognitive Semiotics Starting from Eco’s definition, we can clearly indicate three principles grounding a cognitive semiotics and our idea of it: –– Radical Enactivism (meaning is not thought as content representing the world as being a certain way). –– Pragmatism (meaning is identified with habits and with sense-making). –– Material Engagement Theory (Extended Mind/Enactive Sign/Material Agency) We will now briefly outline these three principles, leaving space for the discussion of more technical aspects and more detailed insights to the following chapters and to future works. Conceiving of semiotics as the discipline that studies everything that can be used in order to lie is not equivalent to identifying it with a theory of truth, which thinks of language and meaning systems as forms of representation of the world. On the contrary, such a perspective on semiotics consists in thinking that systems and processes of signification serve to construct significant surfaces that are useful within the interaction between an organism and its environment, regardless of their truth-­ bearing content. As we have said, from a cognitive point of view, truth and lie are very different phenomena. Hence the radically enactivist and pragmatistic dimension of cognitive semiotics. Cognition does not serve to construct a true representation of the world, but it is a means to act in the world in effective ways. And, in order to act in the world in an effective way, it is necessary to build versions of the world that are capable of bringing forth a world, not to represent it. Hence the “anti-­ representational” nature of cognitive semiotics. Cognition is a form of effective action and, for action to be effective, it is necessary to know how to construct significant surfaces that can be used in order to lie. To summarize, three central claims are at the core of a cognitive semiotics’ perspective: 1. cognitive semiotics understands cognition as an “enactive” skillful activity that involves the ongoing interaction with the external world (Hutto and Myin 2017: 36); 2. if cognition serves to act in the world in an effective way, in order to do so it is sometimes necessary to represent it. To act effectively we need to know the context in which we act, expect something, and pay attention to anything other than what we expected. These are all representative dimensions. However, these are also high-level skills that are not constitutive of cognition tout court and, therefore, they cannot be used in order to define it (see chapter “The Semiotic Mind. Beliefs, Habits and the Extended Cognition” on representations). 3. very often, more than representing the world, cognition has to bring forth the world through meaning, and meanings are not representations of the world, but habits and sense-making activities.

2  The Three Dimensions of Cognitive Semiotics

7

Hence the primacy of lies over truth, which Eco always emphasized. Hence Eco’s cognitive approach to semiotics: making a shift from conceiving the fundamental task of cognition as “accurately representing an environment to continuously engaging that environment with a body so as to stabilize appropriate coordinated patterns of behavior” (Beer 2000: 97). In this way, it is easy to understand the first two main features of a cognitive semiotics. Cognitive semiotics does not think of meaning in terms of truth conditions and contents that represent the world as being a certain way. According to cognitive semiotics, content has nothing to do with truth conditions nor with some representations of some state of the world, but with a series of morphologies (semioticians speak of “forms of content”) that builds systems of values through which languages attune to their environment and constitute the background of our perception of the world. Signs and languages are not built to represent the world, but to enact every possible meaningful relation that we want to express while attuning a particular environment. This is why the semiotic tradition has always been non-representationalist, endorsing a radically anti-representationalist account of languages (and not only verbal ones). Consider Peirce’s insight, that will be discussed in chapter “The Semiotic Mind. Beliefs, Habits and the Extended Cognition”, that representations are signs; this means that the only representations that exist are things that stand for something else under some respect. Signs hide the world and take its place in order to build effective strategies of action. This is totally different from the classical idea of representation, where content represents the world as being a certain way that might or might not be. That is why cognitive semiotics is radically enactivist; languages do not represent the world, instead they build categorizations (forms of content) that install habits. This is the connection between cognitive semiotics and pragmatism, since it is worth mentioning that the father of cognitive semiotics, Charles Sanders Peirce, was also the father of pragmatism. To develop the meaning of something, we have, therefore, simply to determine what habits it produces, for what a thing means is simply what habits it involves. Now, the identity of a habit depends on how it might lead us to act, not merely under such circumstances as are likely to arise, but under such as might possibly occur, no matter how improbable they may be. What the habit is depends on when and how it causes us to act. […] Thus, we come down to what is tangible and conceivably practical, as the root of every real distinction of thought, no matter how subtile it may be; and there is no distinction of meaning so fine as to consist in anything but a possible difference of practice. […] Thought is an action, and that it consists in a relation. (CP: 5.399-400)

So, if thought is an action that consists in a relation, every cognition is a “doing”, not in the form of behavior—pragmatism has nothing to do with behaviorism -, but in the form of habit, a disposition to act in a certain way under some circumstances in the future. Since habits are not reserved to human beings alone and mean only “regularity”, “continuity”, “patterns of action that involve some iteration over time”, every material system can embody habits. In this way, according to cognitive

8

Cognitive Semiotics. Radical Enactivism, Pragmatism and Material Engagement

s­ emiotics, there is a material agency at the very core of cognition. This paves the way for Material Engagement Theory (MET). I believe that MET is a fully-fledged semiotic theory of cognition. Not only in its most immediately evident component, the notion of the “enactive sign”, but also in the other two dimensions on which it is built: the extended mind and material agency. Material engagement theory as an explanatory path is based on three interrelated working hypotheses, which can be summarised as follows. (a) The hypothesis of extended mind, which explores the constitutive intertwining of cognition with material culture; (b) the hypothesis of enactive signification, which explores the nature of the material sign not as a representational mechanism but as a semiotic conflation and cohabitation through matter that enacts and brings forth the world; and, finally, (c) the hypothesis of material agency, which explores agency not as a human property but as the emergent product of situated activity. The distinctive feature of the material engagement approach is the commitment to a view of thinking as a process that is distributed, enacted and situated, as well as assembled, from a variety of non-localizable mental resources spanning the boundaries of the individual brain and body. (Malafouris 2019: 196)

I have tried to show elsewhere (Paolucci 2011) how the extraordinary theory of the extended mind by Andy Clark (2008) is in fact a semiotic theory, which has a real philosophical antecedent in Peirce’s thought, in his pragmatism and in his theory of existential graphs. MET emphasizes the aspects related to the material culture and to the distribution of cognition even more, by making the extended mind theory a real core feature of cognitive semiotics (see Malafouris 2013, see also infra, Chap. 3). As far as material agency is concerned, it is MET itself that is indebted to a semiotic theory, since, as Malafouris (2013: 145) himself says, “the term material agency is used instead of Latour’s term actantiality”. Indeed, the idea of “actantiality” comes from semiotics, since Latour borrowed it from Greimas’ theory of narrativity (Greimas 1983), with whom he studied in the 1980s. An “actant” performs (subject) or undergoes (object) an action (or a passion),7 and Greimas emphasized the radical difference between actants and actors. The actant was in fact a role, a place within a system of values, and it was defined only by its relations with other actants and by the network of relations between actions and passions. As such, an actant was independent from any ontological determination, where ontological determinations were exclusively pertinent to the domain of actors, since an actantial role (its performing or undergoing of an action) could be played by a person, an animal, a thing, an institution or an abstract concept (for instance, the work ethic that forces us to a sort of “having to do”). This dimension of “doing” is a core feature of any cognitive semiotics based on pragmatism and enactivism, and the notion of material agency, at least in Malafouris’ formulation, underlines the active role of the environment in building the semiotic coupling that makes cognition possible.

7  As it can be seen, “actant” is a neutral term in Greimas’ theory, which does not distinguish between subjects and anything else which causes something to happen (this distinction would be rather be in the “thematic roles”). In this way, Greimas does not even need to talk of “material agency”, since agency can be material from the very beginning.

2  The Three Dimensions of Cognitive Semiotics

9

The semiotic dimension of MET is explicitly assumed and emphasized by Malafouris himself: In chapter 4, I argued against the dominant representational-computational view of mind. In this chapter, I want to explore the implications of that critique from a semiotic perspective. Underpinning the suggested constitutive intertwining of cognition with material culture is the capacity of material things to operate as signs. And indeed, insofar as all material culture has a semiotic dimension, understanding signification is important [… and] what l am seeking to understand is, essentially, the semiotic basis of the relationship between cognition and material culture. (Malafouris 2013: 89–90)

MET is a theory that thinks of the role of technical objects and material artefacts as constitutive of cognition. Artefacts are a part of the environment to which some cognitive functions are delegated, and that would not be possible to be performed within the head or within the biological body. However, if this function of the environment had already been emphasized by extended mind theory and by the enactivists, MET is able to go far beyond this, taking a series of positions that represent and should increasingly represent an inescapable source of inspiration for cognitive semiotics. Indeed Malafouris, who is an archaeologist, starts from a critique of archaeology and the type of classical explanations based on cultural meaning and social context that have characterized its enterprise. Starting out from the insights of enactivism, claiming that humans are fundamentally coupled with their ecological niches, according to Malafouris we need to understand the interactions between material environment and cognitive capacities as occurring in a manner of circular, dynamical causation and co-constitution. Malafouris argues for a methodological shift in the archaeological studies of material artefacts. Rather than asking about the role of the artefact in the hypothesized ancient society, or how the artefact might have been used, or the meaning that it could have inside a given culture, according to Malafouris we should ask about the ways by which the artefact might have scaffolded human cognition, or expanded and enhanced the possibilities of cognitive tasks. The very same shifting of the point of view must be assumed for semiotics. Texts, languages and semiotics systems scaffold the way humans come to know the world and represent the background of our perception of the environment. Therefore, cognitive semiotics can bring together the three dimensions that Malafouris considers to be constitutive of his theory: (1) the extended mind, (2) the enactive sign, and (3) material agency. It is not surprising that one of these three constitutive dimensions of MET is an enactive conception of semiotics, like the one I am endorsing and developing here. There is no meaning inherent in past or present material signs; there is only the capacity for meaning. Meaning is the temporally emergent property of material engagement, the ongoing blending between the mental and the physical. In the case of material signs, we do not read meaningful symbols; we meaningfully engage meaningless symbols. Material signs have no meaning in themselves; they merely afford the possibility of meaning, as a door affords the possibility of being opened. In real life, to interpret a material sign […] it is to become habituated with the interactive possibilities and consequences of its performance in

10

Cognitive Semiotics. Radical Enactivism, Pragmatism and Material Engagement context. […] Material signs do not represent; they enact. They do not stand for reality; they bring forth reality. (Malafouris 2013: 117–8)

The idea of an “enactive sign” requires us to go back to some of the key principles of enactivism, in order to show its connections with some ideas within semiotics.

3  Enactivism, Sense-Making and Pragmatism8 Enactivism assumes that cognition is the result of the interactive relations that occur between an organism and the environment in which it is situated (Varela et al. 1991). From an enactive point of view, every cognition, perception, or thought is the result of a living body engaging in its own environment. However, from a cognitive semiotics’ point of view, this environment is not a “natural” one, but a semiotic environment crowded with objects, norms, habits, institutions, and artefacts that shape our minds. The enactivist model for cognition is the living body, not the computer machine, as it was for the classic computational view originated by Alan Turing’s work. However, in an extended mind scenario, like the one introduced here through MET, this opposition between the natural environment and the semiotic environment has to be weakened, since artefacts are part of the environment structurally coupled with the organism and cognitive semiotics, as MET does, considers technical objects and material artefacts as being constitutive of cognition. Indeed, both enactivism and MET start from a strong continuity between mind and matter, between organism and environment, that is very close to the synechism of Charles Sanders Peirce (see Paolucci 2011; Zalamea 2012; Fabbrichesi and Leoni 2012). One of the main theses of enactivism is that the living organism, in order to survive and build its internal autonomy, creates a distinction between itself and the environment in which it is embedded. To exist, this distinction needs a continuous and precarious engagement between the organism and the environment called structural coupling (Maturana and Varela 1980). The capacity that allows the living organism to maintain and adjust its own coupling is called adaptivity (Di Paolo 2005), through which the organism tries continuously to avoid surpassing the limits of its own viability and seeks in the environment the conditions of his sustainability. In the enactivist view, habits are connected to this idea of adaptivity. To understand the kind of dispositions that feature in explanations of the habits of living things, we must recognize that livings things have special adaptive tendencies, which make them disposed—unlike purely mechanical or physical systems—to sensitively adjust in characteristic ways to the particularities of their situated circumstances. (Hutto and Robertson 2019: 6)

Maintaining a dynamical identity while facing material changes, organisms are engaged in the creation of an Umwelt in which they are able to act and survive

 My thanks to Luigi Lobaccaro for his help in discussing the first part of this paragraph.

8

3  Enactivism, Sense-Making and Pragmatism

11

(Varela and Weber 2002). This operation is called the sense-making9 of the living: “an autopoietic system always has to make sense of the world so as to remain viable. Sense-making changes the physicochemical world into an environment of significance and valence, creating an Umwelt for the system.” (Thompson 2007: 146). Sense-making is a semiotic operation and it is connected to non-representational meanings and to habits. It is a key concept for enactivism, because sense-making is conceived as the very form of all cognitions, from the most basic to the more complex ones. This is why enactivism is a key dimension for a cognitive semiotics: cognition in its most fundamental expression has a semiotic structure connected to the sense-making of the living. The idea is that through distribution of values and meanings, the organism reduces the complexity of the material world in which he lives in, selecting the pertinent stimuli for its own viability, which is an operation in which sometimes it can fail. And the “distribution of values and meanings” is a radically semiotic operation. A key precaution is due. In the enactivist sense, meaning is not understood as the classical content of a linguistic term and it is not conceived in terms of a truth-­ conditional semantics. On the contrary, for the enactivist tradition, meaning is thought of in a semiotic way. Indeed, meaning is the effect of the building of the pertinences enacted in order to reduce the complexity of the environment through a distribution of values (see Di Paolo et al. 2018). Since it is the general form of cognition, this idea of values that emerge from the coupling between organism and environment is a semiotic idea able to structure and drive the very same interaction between the two. As it is well known, the semiotic tradition thinks of meaning as the result of a categorization that brings forth a morphology of values (Hjelmslev 1943; Greimas 1970; Eco 1975). Every semiotic system builds its own pertinences, both on the expression plane and on the content plane. These pertinences can be translated between different systems, but they build their own specific systems of values. In recent years, the use of notion of sense-making has been criticized by Hutto and Myin (2013: 34ff, 2017), since they consider the use of the term “sense”, but also “significance” and “meaning”, a potential source of misunderstanding for enactivists’ ideas. Indeed, if “meaning” is thought in terms of analytic philosophy—as content connected to truth-conditions—the misunderstanding is just around the corner, since it would lead us to think of the most fundamental cognitive operation (sense-making) as a particular kind of representation of the world. This is why Hutto and Myin try to explain cognition through the concept of “basic mind”, without employing any kind of vocabulary connected to semantics. However, the very same problem disappears if we assume a semiotic idea of “meaning”, connected to habits and tied to pragmatism, as the one we are introducing here through cognitive semiotics. This is why “sense-making” seems to us a perfect word in order to explain the basic forms of cognition related to basic minds and connect them to higher 9  “Sense Making: The active adaptive engagement of an autonomous system with its environment in terms of the differential virtual implications for its ongoing form of life. The basic, most general form of all cognitive and affective activity manifested experientially as a structure of caring.” (Di Paolo et al. 2018, Glossary).

12

Cognitive Semiotics. Radical Enactivism, Pragmatism and Material Engagement

forms of cognitive skills related to problem-solving, norms and language, in a “synechist” way (Di Paolo et al. 2018). Also because the enactivist account of cognition gives high emphasis to the interaction with the others, to intercorporality and to intersubjectivity. Indeed, as “sense-­ making” is supposed to be the first form of engagement with the world, enactivism has developed a concept which constitutes the first and general form of engagement with other living systems: partecipatory sense-making10 (De Jaegher and Di Paolo 2007). The idea is that, in interactive situations, the pattern of coordination between two agents affects individual sense-making, creating a system in which the coupling with the environment is co-organized inside the inter-action. This participatory sense-making is coordinated in non-predictable ways and it applies to how the participants to the interactions make sense of each other and how they make sense of the world together. Such a general concept is not restricted to specific forms of human interaction, say a conversation. Rather, since it is defined in systemic terms, it applies in general across species too and for the whole gamut of interactive possibilities. The concept does specify, however, something that goes beyond the simple conjoining of sense-making and a social situation. It is not merely sense-making about something social (e.g., the expressions of another person, or their hidden motives) but it is sense-making performed socially, enacted as a shared practice. This implies sense-making that is perturbed, enabled, modulated, regulated, or even constituted by what goes on in the interaction (the activity of other participants or the interactive patterns or both). (De Jaegher and Di Paolo 2007: 82)

The social understanding in humans is thus anchored to a more general form of interaction that happens between living organisms thought as dynamical systems (for an overview on an enactivist approach to social cognition, see Gallagher and Zahavi 2008: Chap. 9; Paolucci 2019, 2020a). This holds true also for high-level forms of inter-actions, like the human ones connected to social and cultural practices, which can be perfectly explained in enactivist terms without introducing concepts like “representations” or “mindreading”, as Shaun Gallagher’s works have shown us in the last ten years (see Gallagher 2017, 2020, see also infra, Chap. 5). Hence, according to cognitive semiotics, cognitive processes, instead of being identified with an internal representational and computational machinery, are always thought to be “constitutively and already world involving” (Hutto and Myin 2013: 137), where this involvement of the world is enabled by the organism’s possible and actual actions, like in the pragmatist tradition. Pragmatism is the key concept that bridges radical enactivism with cognitive semiotics. Since meaning is not connected to content representing the world as being a certain way (truth-conditional semantics), radical enactivists not only claim that many of the cognitive processes which are at stake in cognitive sciences’ explanations are non-representational, but also

 “Participatory Sense Making: “Sense-making in the context of a social interaction as it is affected by coordination patterns, breakdowns, and recoveries undergone during social encounters. Participatory sense-making is how people understand each other and how they understand and act on the world together.” (Di Paolo et al. 2018; Glossary).

10

4  Cognitive Semiotics Sense-Making

13

hold that these processes are “content-less” or “content-free” (Hutto and Myin 2013, 2017). Indeed, pragmatism and enactivism share the idea that cognitive processes are not built in order to give account of the world, representing it as being in a certain way in our heads, but in order to afford possible actions. The argumentation in favour of a content-less approach to cognition is based on an analytical approach to the “hard problem of content” (Hutto and Myin 2013: 58), which is strictly tied to a representational and informational view of cognition and mental activities. According to Hutto and Myin (2013, 2017), since, at present, there is not a naturalistic account of information-as-content that is actually able to meet the constraints of a naturalistic explanation, there is no need to think mental representations entertain a truth-bearing relation with the things they represent. A successful explanation of basic cognitive processes such as perception, action, imagination and episodic recalling should eschew the concepts of representation and of content altogether. However, as far as this problem is concerned, it is important emphasize again that the idea of “content” against which Hutto and Myin argue comes from analytic philosophy and is thought in terms of truth conditions. As we have said, according to semiotics, content is related to morphologies that arise out of the attunement between organisms and environment and out of processes of categorization (“forms of content”). Moreover, semiotics defines meaning in the forms of habits and sense-­ making and if we think of “sense-making” in the terms of cognitive semiotics, there is no need of fearing any kind of misunderstanding of the enactivists’ ideas.

4  Cognitive Semiotics Sense-Making We can come back now to the notion of sense-making, which is crucial for a cognitive semiotics. From the starting point of the structural coupling between organism and environment, which leads to the organism generating its own environment via autopoiesis, “a subset of the myriad forms of coupling with the environment that constitutes the organism’s world - that is, those engagements that are appreciated as meaningful by the organism, emerge. Nothing occurs in this world that is not significant and in relation to which the organism positions itself (even in the particular case of indifference, which is also a position)” (Di Paolo et al. 2018: 48). In his last works, Varela called this relationship “sense-making”, thus identifying its semiotic structure from its name onwards. However, there is much more semiotics in sense-­ making than the name alone. To use Hjelmslev’s terminology, sense-making is the process involving ‘cutting’ a pre-semiotic continuum which, from the starting point of n possible relationships, selects the most relevant based on a system of meaning understood as a system of values. It is via sense-making operations that the world becomes a meaningful world connected to the organism’s position in the system. Certainly for the enactivists, this sense-making operation, which carves out relevant relationships from the starting point of other equally possible relationships, is implemented at the living organism

14

Cognitive Semiotics. Radical Enactivism, Pragmatism and Material Engagement

and science of life level and not at the linguistic or social institution level, as Hjelmslev and the semiotic tradition were accustomed to. But this is of even greater interest here, as our objective is precisely abandoning the threshold of culture as the place for semiotics, together with the very distinction between nature and culture, which historically ended up separating biosemiotics and zoosemiotics on the one hand, from semiotics understood as a logic of culture, on the other. As far as this crucial point is concerned, enactivism shows us how semiotic sense-making relations are foundational also of the organism’s very activity even at the life science level. No living system is possible unless this living system is also and first and foremost a semiotic system. It is thus clear that semiosis nestles also in the life processes and systems (biosemiotics) and not simply as far as cultural processes and systems are concerned (cultural semiotics). The syntagma which has frequently defined semiotics itself as “a discipline which studies the systems and processes of signification” is neutral with respect to the opposition between nature and culture, just as it is neutral regarding that between life and sociality. A living system is a system and a process of signification (sense-making). Sense-making is the capacity of an autonomous system to adaptively regulate its operation and its relation to the environment depending on the virtual consequences for its own viability as a form of life. Being a sense-maker implies an ongoing (often imperfect and variable) tuning to the world and a readiness for action. Through the combination of material and precarious self-individuation and adaptive regulation of the relations to the environment, sense-making naturalizes the concept of vital norms and lies at the core of every form of action, perception, emotion, and cognition, since in no instance of these is the basic structure of concern or caring ever absent. This is constitutively what distinguishes mental life from other material and relational processes. (Di Paolo et al. 2018: 49)

An idea which is extremely interesting for a semiotics of the enactive process of sense-making is that sense-making is not an activity which resides within the organism or the subject. Quite the contrary, sense-making is an activity which emanates from the system. Enactivism does not supply us with the idea of a subject which binds up the real into categories or that of an organism structurally coupled with its environment which adds its own meaning (or series of meanings) to this world and this environment. Quite the contrary, sense-making is an adding subtraction activity which carves out its own subset of relevant interactions from within a dynamic coupling between organism and environment which have value for the system and the organism themselves. In brief, sense making is the activity of adding subtractions which carves out a world of values and meanings (an “environmental world”) from the starting point of many other possible environments and worlds. In Paolucci 2020b, I conceived precisely of this “adding subtractions activity” as the act of enunciation, which is the semio-linguistic equivalent of the sense-making process. From the starting point of the possibilities opened by schemas, norms and uses, enunciation effectively carves out significant micro-universes (texts and enunciates) via an act (see infra, Chap. 2). I believe this is a key idea for semiotics, since it enables certain ideas from the semio-linguistic tradition to be juxtaposed and amended. The starting point of the sense-making situation is not that of an absence of sense (“manque”), a deficit

4  Cognitive Semiotics Sense-Making

15

which pushes us in the direction of a series of narrative transformations (see Greimas 1983; Ricoeur 1984). Quite the contrary, there is always too much sense and the problem is filtering it and constructing a semiotic niche capable of allowing for a whole of productive interactions. I would argue that the “semiotic niche” concept, one capable of flanking and supplementing the “ecological niche” concept from the starting point of an ‘excess of sense’, is a key idea in a new cognitively oriented semiotics that is capable of leaving behind mechanisms modelled on human (and Western) cultural practices connected to the “manque” notion by Greimas (1983) and the “absence of sense” which characterises semiotic practices according to Fontanille (2008). By contrast, an enactivist notion of semiotic practices, namely of inter-actions between organisms and environmental worlds, is an idea in which sense-making activity is an addition of subtractions capable of constructing semiotic niches. Let’s clarify this point. “Ecological niche” refers to the environment occupied by a species or population within its habitat, that is understood not as a physical space, but as the ensemble of the roles and functions which individuals take on in an ecosystem, using, modifying and building the resources of the habitat itself. Languages, but also norms, institutions and uses of the “speakers” are especially complex semiotic niches which have pruned possibilities and ecologically built semiotic environments which regulate in a certain way (and not in others) the interactions with the subjects inhabiting them. Within diverse languages, institutions and norms, we find diverse forms of regulation of interactions, which are the effect of different “addition of subtractions”. Meaning and related concepts (value, significance, etc.) apply only to the whole situated organism in relation to its associated milieu, which itself coemerges with the individuation and activities of organisms. To think that meaning resides in one part of a coupled organism-­ environment system is a fallacy of misplaced concreteness (like thinking the speed of a car can be located inside its engine). Moreover, sense-making does not “add” anything to the rich coupling with the environment, it “subtracts” from it by monitoring and responding to only a small and relevant subset of all the influences that impinge on the organism, a subset out of which the organism constitutes itself as a sense-maker. (Di Paolo et al. 2018: 53)

It is precisely in this way that the enactivist idea of agency as an addition of subtractions from semiotic niches (systems, norms, uses, collective appreciations, etc.) is connected to the theory of enunciation and, especially, precisely to the idea of the act of enunciation. Consider the production of art as an example, an act of sense-making which is a long way away from the life science domain and doubly bound up with the logics of culture, as artistic enunciation has frequently been conceived of as centred on the subject, on its creative individuality of worlds and producer of reality. However, for an artist a canvas is never empty, for a writer a page is never blank. Both canvas and page are always too full of the already said, the already painted, the clichés, stereotypes, interpretative habits, that is, of the whole of schemes, norms, habits and usages which regulate a system’s discursive formations. As Hjelmslev (1942) showed, every subjective act of enunciation depends on a whole of virtual structures, perfectly real in their virtuality, which regulate utterance and enunciates

16

Cognitive Semiotics. Radical Enactivism, Pragmatism and Material Engagement

production. When artists such as Cézanne11 or Francis Bacon12 say that the problem for painterly enunciation is avoiding clichés, they are saying so because enunciation filling up a canvas or a page is always bound up with stereotypes, norms, habits and usages which define the very moment they find themselves in. It is precisely to the extent that it is empty that a canvas is always too full, overflowing with things already said, precisely in its emptiness. But if this is true, what are these stereotypes, these clichés, these things already said which a blank page is so replete with precisely in its being blank? In Semiotics and the Philosophy of Language, Umberto Eco (1984) used the term “Encyclopaedia” to refer to “the whole of the already said”, the “library of all libraries”, “the recorded set of all interpretations”. For Eco, the Encyclopaedia is a semiotic environmental world representing the background to our perception of the world and our sense-making activities. Indeed, each sense-making activity generates meanings from the starting point of this virtual background of the already said, which traces and records all the enunciates already enunciated. As such, Encyclopaedia thus pertain to the Hjelmslev dimension of use, namely the semiotic dimension in which each individual act pulsates with its relationship with other individual acts. At the same time, an encyclopaedia is simultaneously a set of (virtual) schemes and norms, which represent the condition of possibility of the production of new enunciates. Indeed, each culture retains a memory of its various linguistic and non-linguistic uses, regulating these in accordance with collective appreciations and, thus, bringing forth relational schemas which prune down the system’s possibilities. For Eco, the Encyclopaedia is the potential for new sense-making productions. Attempting to reactivate a concept which, in Eco, is almost always a theoretical notion not sufficiently put to concrete empirical test,13 I would argue that we can speak of the Encyclopaedias as an “a priori effect’”. The Encylcopaedia is an “effect” to the extent that it is the product of all the acts of enunciation it records. It is a priori—materially and historically—to the extent that it locally represents the condition of possibility for new acts of enunciation. An Encyclopaedia is simultaneously structured by past enunciation and structuring of future enunciation. Thus, it is not precisely the Encyclopaedia which fills up the writer’s pages prior to his enunciation and obliges the artist to take away, by addition of subtractions? Is not the Encyclopaedia this set of schemes, norms and uses which an artist’s empty canvas is so replete with in its emptiness? When Francis Bacon (1993) argues that a whole preparatory work prior to painting needs to be done in order to free it of clichés and that a painting must always pass through a series of involuntary acts such as “throwing colour on a canvas”, “making marks which mean nothing”, “brushing a part”, “randomly rubbing out”,

 See Lawrence 1991: 99–106.  See Bacon 1993. 13  But the true concrete activation of the Encyclopaedia idea in Eco is to be found in his novels according to the principle “whereof one cannot theorize, one must narrate” (see Paolucci 2017a). 11 12

4  Cognitive Semiotics Sense-Making

17

etc., he means that a painter is never filling up a blank surface but rather emptying, clearing, cleaning off the canvas14 all those encyclopaedic stereotypes it is so replete with. For this reason, the painting is already there, in the encyclopaedic virtuality of cultures and the artist must remove it, adding substractions, exactly as Michelangelo said he did with his blocks of marble. When we move from linguistics to semiotics, abandoning Saussure’s distinction between langue and parole, every act of enunciation which makes us pass from the encyclopaedia to the enunciate is an addition of subtractions which enacts an encyclopaedic virtual with its stereotypes, its clichés and its overflowing “already said”. And this also when the act forcefully attempts to deviate from this set of schemes, norms, frames, uses and already enunciated enunciates, potentializing and virtualising them. An enunciation is always a singularity if compared to the schemes, the norms, the uses and the enunciates which make it possible. However, enunciative events are not born in isolation. Quite the contrary, they emerge opposing to other equally possible events which could equally have emerged within that system. Every act of enunciation effectively negates and contradicts others which would have been equally possible. It is the very existence of the enunciate which negates and contradicts other enunciates. For this reason, in his Archaeology of Knowledge, which formulates an important theory of impersonal enunciation (see infra, Chap. 2), Michel Foucault dealt simultaneously with the “regularity of the enunciates”—their dependence on a regular system of schemes, norms and uses—and their “rarity”, “the fact that all is never said” (Foucault 1969: 138). However numerous they are, enunciates are always in deficit [...] Enunciates are studied at the limit which separates them from what is not said, within the instance which bring them out to the exclusion of all others. (Foucault 1969: 138)

For a cognitive semiotics, regularity and rarity represent the two foundational principles of a semiotic agency capable of constructing semiotic niches. Objectively regulated and regular, without being the product of obedience to a system of rules by a subject, enunciates are at the same time rare, because their presence excludes the contemporary presence of other enunciates which would have been equally possible on the basis of the regularities of the system. This addition of subtractions, which makes us pass from the Encyclopaedia to the enunciate via schemes, norms and uses, is the semiotic agency linked to the act of enunciation. It tears out sense-making and the act of enunciation from the ownership of a subjective instance and consigns them to a structural coupling between the organism and the semiotic environment of the Encyclopaedia, in which the former generates the latter which generates the former (see Maturana and Varela 1980; Di Paolo et al. 2018). Indeed, Eco (1984) conceived of the relationship between the system of the Encyclopaedia and the subject in an asymmetrical way, since the subject was the “slime and debris” of semiosis, the effect of the semiotic activity within the encyclopaedia. It is, moreover, an idea which in some way conforms to the 14

 See Deleuze 1981: 157.

18

Cognitive Semiotics. Radical Enactivism, Pragmatism and Material Engagement

o­ riginal enunciation theory put forward in Benveniste’s structural linguistics (1966); it is subjectivity which emerges from the act of enunciation, not vice versa. Our enactivist position about semiotic systems and languages (see Bottineau 2016) is clearer now; in an autopoietic system like the Encyclopaedia, enunciation instances generate the encyclopaedic slime and debris which generate them (see infra, Chap. 2).

5  Languages and Cognitive Systems In this respect, let us look at an example which is complex enough to be capable of illustrating all these foundational dimensions to the act of enunciation. Consider someone who is learning wine tasting and trying to understand which taste sensations are to be used and acknowledged to gain an understanding of whether a certain wine is full bodied or otherwise, in order to verbally describe the wine or recognize it in a test.15 A situation such as this one involves certain profound oppositions of a differential type (“full bodied” versus “non-full bodied”) which are standardised and institutionalised to the extent of categorising the world of wines, differentiating products and generating pairings with specific dishes relating to certain cultural practices. But there is at play also a subjective part connected to a novice tongue which must render itself social and intersubjective via use,16 learning to tentatively acquire a series of interpretative habits capable of linking individual bodily taste sensations with socially consolidated categories. For the purposes of passing from the former to the latter and adequately recognising a wine via a correct enunciate, our novice can relate his own use to other uses, resorting, for example, to the experience of his own sommelier course teacher and measuring his own subjective experience with another’s subjective experience, one capable before him of combining his own individual sensations with the socially expected categorization and passing from the former to the latter. At work here are differential values (potential), social norms (virtual), inter-­ subjective uses (actual) and enunciates (realised). Some of these dimensions are enacted while others remain only virtual, however perfectly real and functioning in their virtuality. However, in no case the enacted forms outrun the virtual ones, that, on the contrary, guide and channel them. However, it is obviously impossible to learn to be a good sommelier by studying the textual categorisation of socially standardised wines. Learning takes place when the subjectivity of my tongue becomes commensurable with the material of wine and indexes some of the formants to become the expression of possible content (such as a “full bodied wine”). In both cases, my body must ultimately succeed in inhabiting another bodyliness, locally 15  See Festi 2003: 192. On this theme, within a semiotics of perception, see also Moutat 2016, 2019 and Marrone 2016. 16  On the theme of the body which must render itself social, see two important volumes by Gianfranco Marrone (2001, 2005).

5  Languages and Cognitive Systems

19

adapting to its significant singularities.17 In both cases, my body must succeed in leaving behind its current habits for the purposes of combining its specific features with an “other” system, while consolidating new personal habits. As Sarti et al. (2019: 18) have noted, “this process no longer corresponds with a categorisation but rather with the identification of the principal orientations of a perceptive flow even in the absence of any fixed stabilisation”. My tongue must render itself social, learning to make itself commensurable with an expected social categorisation, such as that between “full bodied” and “non-full-bodied”. It is, in fact, precisely a possible commensurability which is at stake and which constitutes what is at play in this type of experience. It is the individual singularities of my tongue and those of the wine which must become commeasurable in an encounter with an “other” I do not know and which I am learning to manage (and it is not to be taken for granted that I will succeed). It is precisely the sensations of my tongue in contact with the otherness of the wine which must be semiotised for the purpose of indexing some bodily formants (“tasting sensations”) which put themselves forward as possible expressions for a specific content (“full bodied wine”). But this process encompasses right from the outset what Marrone (2001) called “social bodies”, that is, a semantic categorization coming from other enunciating instances (norms, uses, schemes, texts, etc.) which are at work in the construction of a commensurability between the singularities of my body and those of an ‘other’ body with which I am attempting to construct a mediation and a possible passage. Cultural norms and habits produce a silent “discourse” inhabiting my own individual and subjective “discourse”, which is always generated in relation to that of other enunciating instances. In this regard, in A Theory of Semiotics, Umberto Eco (1975) examined the way a subject produces a sign’s expression plane from the starting point of a series of semiotic grammars and socially established conventions. According to Eco, the act generating the sign changes not only with these cultural and grammatical dimensions, but also with the material and corporeal substance which shores up every specific language. To use a banal example, it is one thing to express one’s point of view speaking a natural language and another to do so via a painting or a movie. For this reason, Eco elaborated an important typology, which can be useful within the current debate in cognitive sciences on the relationship between languages, cognition and meaning. Eco argues that accessing the signified plane requires, first and foremost, disentangling the way in which the figures of the signifier take shape from the starting point of the material substance and from the gesture of engraving it. The semiotic work required in order to generate an enunciation in a natural language is different in kind from the one needed in order to generate enunciations within other non-linguistic semiotic systems. However, whatever the differences and heterogeneities, Eco was able to identify two general types of possible relations depending on whether the language has a grammaticalized type of expression or not. Eco speaks of type/token ratio and distinguishes the act of semiotic

17

 See Fontanille 2004: 214.

20

Cognitive Semiotics. Radical Enactivism, Pragmatism and Material Engagement

production in function of this difference. Indeed, the ratio is facilis, when an expression token accords with a pre-existing expression type. On the other hand, the ratio is difficilis when a preformed and grammaticalized type of expression does not exist and the expressive token is directly correlated with its contents. There is a case of ratio facilis when an expression-token is accorded to an expression-type, duly recorded by an expression-system and, as such, foreseen by a given code. There is a case of ratio difficilis when an expression-token is directly accorded to its content, whether because the corresponding expression-type does not exist as yet or because the expression type is identical with the content-type. In other words, there is a ratio difficilis, when the expression type coincides with the sememe conveyed by the expression-token. Using a formulation that will be partly criticized in the following sections, one could say that in cases of ratio difficilis the nature or the expression is motivated by the nature of the content. (Eco 1975: 183)18

It is clear that very few languages are capable of institutionalising a system of expression. Eco cites the natural languages, the road sign system and certain formal languages like logic. All other languages, all other semiotic systems function via ratio difficilis, because they have no preformed expression type and the individual act which produces the sign must do so directly according with the content plane. Consider an artist painting a picture or a novice attempting to index his own tasting sensations to a socially expected categorisation; their acts of enunciation generate an expression plane in direct function of the contents they want to express. Now, it is very clear that virtually all languages and semiotic systems function via ratio difficilis. Natural language, with its disembodied structure and its ratio facilis is a radical exception: an exceptional product of evolution whose ability to radically reconfigure almost all human social practices is by no means coincidental (Lo Piparo 2003; Pennisi and Falzone 2010, 2017; Pennisi 2014). It would appear obvious that a theory of cognition cannot be constructed from the starting point of exceptions, as has historically been the case in the cognitive sciences, that have conceived of cognition as a language-like representation in a linguistic format. Natural language, with its content structure and its relation between signifier and signified founded on ratio facilis cannot be the model of cognition, representing an ultra-cogent exception. For instance, natural language is fully disembodied, as it can run on different hardware, something which is not true of cognition, as it is made up of many other dimensions. It is trivial to say, for instance, that spoken Italian, written Italian, Italian telegraphed by Morse code, Italian transmitted by flags according to the international code of Navy is, in all these cases, one and only language and not four different languages. The units that build it up differ from case to case, but the network of relations between these units remains the

 In Eco the semiotic ratio (1975) is a relationship between type and token. I cannot follow Eco on this point, as the type/token distinction seems to work virtually exclusively for the expression plane of verbal language and similar semiotic systems, while it would not appear to be adequate for taking account of what happens, for example, for visual or syncretic languages, where the existence of a type of expression is much more doubtful. Moreover, a type/token ratio would not seem to adequately describe what happens on the content plane. I would argue that ratio takes shape primarily between singularities, in the establishment of a commensurability relation.

18

5  Languages and Cognitive Systems

21

same, and allows to identify the language. Accordingly, the main object and the first aim for linguistics must be this network, while its concrete representations or the related manifestations have nothing to do with the language. (Hjelmslev 1959: 16)

However, virtually all other languages and semiotic systems do not function in this way. And yet, it is also important to avoid the opposite error. That is, by means of an excessive desire for paradigm shift, it is essential to avoid using the “embodied a priori” model as sole model for cognition, since it does not fully explain certain fundamental cognitive revolutions expressed in extremely powerful semiotic systems such as natural language. There should not be “sole” and “a priori” models in the cognitive sciences. This is why cognitive semiotics, with its knowledge on signs and languages, can be useful for the cognitive sciences. Indeed, the first thing cognitive semiotics teaches us is the difference between semiotic systems of a linguistic type (the natural languages and a few others) and semiotic systems of a non-linguistic type. The first ones are founded on ratio facilis, the second ones on ratio difficilis. This means that some semiotic systems are disembodied, and other semiotic systems are embodied. And, thus, if semiotic systems shape the mind, a war between embodied cognitive science and disembodied cognitive science simply means not recognising that certain things work in the first way and certain others in the second one. Cognition always implies a come and go between different systems. For instance, verbal language—which is primarily disembodied as far as its schemes and norms are concerned—always has a radically embodied dimension, referring to linguistic practices, enunciation and participatory sense-making. Of course, we can undoubtedly think that the enunciation dimension is central for natural languages and for the forms of life characterising them, too, as Di Paolo et al. (2018) and myself have done (Paolucci 2020b). However, reducing the semiotic system of natural languages to the enunciation dimension alone—because it is the embodied one and thus closer to the principles of the embodied turn—is a mistake, as verbal language is not exclusively a matter of this crucial dimension and presents many other dimensions of a different kind. In the same way, it is undoubtedly important to avoid the opposite mistake made by the first phase of the cognitive sciences with their works on cognition, language and signs. As Malafouris (2013) has very well shown, tracing material signs to the very same model used for natural language is mistaken, because their ratio difficilis structure makes them profoundly heterogeneous systems with many different specific semiotic features, if compared to those of the natural languages. They are thus to be kept distinct and explained locally without reference to a priori models. Looking closer, these models for cognition often come from an implicit cognitive semiotics. Indeed, Peter Strawson has underlined that the majority of the cognitive sciences has taken a specific form to the extent that it has been massively influenced by the model of language and by a certain conception of it, inspired mainly by both Chomsky and analytic philosophy. It would not be difficult to argue that the cognitive sciences have been modelled on an underlying cognitive semiotics inspired by

22

Cognitive Semiotics. Radical Enactivism, Pragmatism and Material Engagement

languages constructed on ratio facilis. And that the embodied turn has, by contrast, taken as a model those built on ratio difficilis. It is precisely for this reason, as this book deals with subjectivity, enunciation (chapter “For a Cognitive Semiotics of Subjectivity”), cognition, habits, representations (chapter “The Semiotic Mind. Beliefs, Habits and the Extended Cognition”), social cognition, narratives (chapter “Social Cognition and Autism Spectrum Disorders: from Mindreading to Narratives”) and perception (chapter “Perception as Controlled Hallucination”), that it must take account of the semiotic heterogeneity of all these dimensions, taking a tactical approach. With the exception, obviously, of the three dimensions delineated in this chapter, which I believe must build up a cognitive semiotics.

References Abe, N. (2009). The neurobiology of deception: evidence from neuroimaging and loss-of-function studies. Current Opinion in Neurology, 22(6), 594–600. Abe, N., Okuda, J., Suzuki, M., Hiroshi, S., Tetsuya, M., Etsuro, M., Minoru, T., & Toshikatsu, F. (2008). Neural correlates of true memory, false memory, and deception. Cerebral Cortex, 18(12), 281–289. https://doi.org/10.1093/cercor/bhn037. Bacon, F. (1993). Conversazioni con Michel Archimbaud. Genova: Le Mani Microart’s. English Edition: Bacon, Francis. 1994. Conversation withMichel Archimbaud. New  York: Phaidon Press. Beer, R. D. (2000). Dynamical approaches to cognitive science. Trends in Cognitive Science, 4(3), 91–99. Benveniste, É. (1966). Problèmes de linguistique générale. Paris: Gallimard. English Edition: Benveniste, Émile. 1971. Problems in General Linguistic, Transl. Maria Elizabeth Meek, Coral Gables Fl.: Miami University Press. Bottineau, D. (2016). Linguistique incarnée et “énactivisme”: quelles collaborations possibles avec les neurosciences? In A. Rabatel, M. Temmar, & J.-M. Leblanc (Eds.), Sciences du langage et neurosciences (pp. 211–232). Limoges: Actes du colloque 2015 de l’Association des Sciences du Langage, Lambert-Lucas. Chemero, A. (2009). Radical Embodied Cognitive Science. Cambridge, MA: MIT Press. Clark, A. (2008). Supersizing the mind. Embodiment, action, and cognitive extension. Cambridge, MA: Oxford University Press. De Jaegher, H., & Di Paolo, E. (2007). Participatory sense-making: An enactive approach to social cognition. Phenomenology and the Cognitive Sciences, 6(4), 485–507. Deleuze, G. (1981). Francis Bacon. Logique de la sensation. Paris: Éditions de la Différence. English edition: Deleuze, Gilles. 2003. Francis Bacon: The Logic of Sensation. London: Continuum Press. Di Paolo, E. (2005). Autopoiesis, adaptivity, teleology, agency. Phenomenology and the Cognitive Sciences, 4(4), 429–452. Di Paolo, E., De Jaegher, H., & Cuffari, E. (2018). Linguistic bodies. The continuity between life and language. Cambridge, MA: MIT Press. Diana, B. (2014). A cognitive approach to deception detection: multimodal recognition of prepared lies. Department of Human Sciences for Education “Riccardo Massa”. Milan: University of Milan, Bicocca. Retrieved from https://boa.unimib.it/retrieve/handle/10281/53250/80210/ phd_unimib_043265.pdf.

References

23

Eco, U. (1975). Trattato di semiotica generale. Milan: Bompiani. English edition: 1976. A theory of semiotics. Bloomington: Indiana University Press. Eco, U. (1984). Semiotica e filosofia del linguaggio. Turin: Einaudi. English edition: 1984. Semiotics and the Philosophy of Language. Bloomington: Indiana University Press. London: Macmillan. Eco, U. (1997). Kant e l’ornitorinco. Milan: Bompiani. English edition: 1999. Kant and the Platypus. Essays on Language and Cognition. London: Secker & Warburg. El-Hani, C. N., Queiroz, J., & Stjernfelt, F. (2009). Firefly Femmes Fatales: A case study in the semiotics of deception. Biosemiotics, 3, 33–55. https://doi.org/10.1007/s12304-009-9048-2. Fabbri, P. (2017). L’efficacia semiotica. Domande e risposte. Rome: Mimesis. Fabbrichesi, R., & Leoni, F. (2012). Continuità e variazione. Rome: Mimesis. Festi, G. (2003). Le logiche del sensibile. Un confronto tra la semiotica tensiva e il processo di naturalizzazione del senso. Semiotiche, 1(03), 175–196. Fontanille, J. (2004). Soma et séma. Figures du corps. Paris: Maisonneuve et Larose. Fontanille, J. (2008). Pratiques sémiotiques. Paris: PUF. Foucault, M. (1969). L’archéologie du savoir. Paris: Gallimard. Fusaroli, R., & Tylén, K. (2014). Linguistic coordination: Models, dynamics and effects. New Ideas in Psychology, 32, 115–117. https://doi.org/10.1016/j.newideapsych.2013.03.002. Gallagher, S. (2017). Enactivist interventions. Rethinking the mind. Oxford: Oxford University Press. Gallagher, S. (2020). Action and interaction. Oxford: Oxford University Press. Gallagher, S., & Zahavi, D. (2008). The phenomenological mind. London: Routledge. Ganis, G. (2015). Deception detection using neuroimaging. In P.  A. Granhag, A.  Vrij, & B.  Verschuere (Eds.), Detecting deception: Current challenges and cognitive approaches (pp. 105–121). Chicago, IL: Wiley. Gensini, S. (2018). Appunti su semiotica ed etologia: un dialogo (parzialmente) interrotto. Reti, Saperi, Linguaggi, 7, 97–110. Gramigna, R. (2020). Imagining others. Deception, prediction, and disguised intentions in strategic interactions. Versus—Quaderni di Studi Semiotici, 130(2020), 121–138. Greimas, A. J. (1970). Du sens. Paris: Seuil. Greimas, A. J. (1983). Du Sens II—Essais sémiotiques. Paris: Edition du Seuil. Higuera, C. J. R., & Kull, K. (2017). The Biosemiotic glossary project: The semiotic threshold. Biosemiotics, 10(1), 109–126. Hjelmslev, L. T. (1942). Langue et parole. Cahiers F. de Saussure, 2, 29–44. Hjelmslev, L. T. (1943). Prolegomena to a theory of language. Madison: University of Wisconsin Press. Hjelmslev, L. T. (1959). Essais linguistiques, Travaux du Cercle Linguistique de Copenhague vol. XII, 1959 (tr. it. Saggi di linguistica generale, Pratiche, Parma, 1981). Hutto, D.  D., & Myin, E. (2013). Radicalizing Enactivism. Basic Minds Without Content. Cambridge, MA: MIT Press. Hutto, D. D., & Myin, E. (2017). Evolving enactivism. Basic minds meet content. Cambridge, MA: MIT Press. Hutto, D. D., & Robertson, I. (2019). Clarifying the character of habits: Understanding what and how they explain. In F. Caruna & I. Testa (Eds.), Habit: Pragmatist approaches from cognitive neurosciences to social sciences. Cambridge, MA: Cambridge University Press. Ito, A., Abe, N., Fujii, T., Hayashi, A., Ueno, A., Shunji, M., Takahashi, S., & Mori, E. (2012). The contribution of the dorsolateral prefrontal cortex to the preparation for deception and truth-­ telling. Brain Research, 1464, 43–52. Lawrence, D. H. (1991). Scritti sull’arte. Syracuse: Tema Celeste. Lo Piparo, F. (2003). Aristotele e il linguaggio. Roma-Bari: Laterza. Malafouris, L. (2013). How things shape the mind. A theory of material engagement. Cambridge, MA: MIT Press.

24

Cognitive Semiotics. Radical Enactivism, Pragmatism and Material Engagement

Malafouris, L. (2019). Understanding the effects of materiality on mental health. BJPsych Bulletin, 43(5), 195–200. Marrone, G. (2001). Corpi sociali. Processi comunicativi e semiotica del testo. Turin: Einaudi. Marrone, G. (2005). La cura Ludovico. Turin: Einaudi. Marrone, G. (2016). Semiotica del gusto. Linguaggi della cucina, del cibo, della tavola. Rome: Mimesis. Maturana, H., & Varela, F. (1980). Autopoiesis and cognition. The realization of the living. London: Routledge. Moutat, A. (2016). Le dispositif énonciatif à l’épreuve de la remédiation. In T.  Migliore (Ed.), Rimediazioni. Immagini interattive (Vol. 2, pp. 233–250). Moutat, A. (2019). Le commentaire œnologique ou comment déguster le vin par les mots. Lexicons, Sensations, Perceptions and Emotions, 13. Retrieved from http://journals.openedition.org/lexis/3026. Ofen, N., Whitfield-Gabrieli, S., Chai, X. J., Schwarzlose, R. F., & Gabrieli, J. D. E. (2017). Neural correlates of deception: lying about past events and personal beliefs. Social Cognitive and Affective Neuroscience, 12(1), 116–127. https://doi.org/10.1093/scan/nsw151. Paolucci, C. (2008). Modelli di analisi non testuale di una semiotica interpretativa delle culture. Il caso del Partito Democratico. Versus-Quaderni di Studi Semiotici, 107–108, 73–100. Paolucci, C. (2010). Strutturalismo e interpretazione. Ambizioni per una semiotica minore. Milan: Bompiani. Paolucci, C. (2011). The external mind: Semiotics, pragmatism, extended mind and distributed cognition. Versus—Quaderni di Studi Semiotici, 112–113, 69–96. Paolucci, C. (2017a). Whereof One cannot theorize, thereof one must narrate. In T. Thellefsen & B. Sørensen (Eds.), Umberto Eco in his own words (pp. 165–174). Berlin: De Gruyter. Paolucci, C. (2017b). Eco, Peirce, and the anxiety of influence: The most Kantian of Thinkers. In S. G. Beardsworth & R. E. Auxier (Eds.), The Philosophy of Umberto Eco (pp. 251–278). Chicago: Open Court. Paolucci, C. (2019). Social cognition, mindreading and narratives: A cognitive semiotics perspective on narrative practices from early mindreading to Autism Spectrum Disorders. Phenomenology and the Cognitive Sciences, 18, 375–400. Paolucci, C. (2020a). A radical enactivist approach to social cognition. In A. Pennisi & A. Falzone (Eds.), The extended theory of cognitive creativity. Vol. 23: Perspectives in pragmatics, philosophy & psychology (pp. 59–74). Cham: Springer. Paolucci, C. (2020b). Persona. Soggettività nel linguaggio e semiotica dell’enunciazione. Milano: Bompiani. French translation 2020. Liège: Puliège. Pennisi, A. (2014). L’errore di Platone. Bologna: Il Mulino. Pennisi, A., & Falzone, A. (2010). Il prezzo del linguaggio. Evoluzione ed estinzione nelle scienze cognitive. Bologna: Il Mulino. Pennisi, A., & Falzone, A. (2017). Darwinian biolinguistics. Dordrecht: Springer. Ricoeur, P. (1984). Temps et récit II. La configuration dans le récit de fiction. Paris: Editions de Seuil. Rowlands, M. (2010). The New Science of the mind. Harvard: MIT Press. Sarti, A., Citti, G., & Piotrowski, D. (2019). Differential heterogenesis and the emergence of the semiotic function. Semiotica, 1–34. In press. Sonesson, G. (2012). The foundation of cognitive semiotics in the phenomenology of signs and meanings. Intellectica, 58(2), 207–239. Sonesson, G. (2018). No more faith in fakes: A natural history of counterfeiting. Versus—Quaderni di Studi Semiotici, 127(2), 259–274. Sonesson, G. (2019). On mimicry, signs and other meaning-making acts. Further studies in iconicity. Biosemiotics, 12, 99–114. https://doi.org/10.1007/s12304-018-9340-0. Stjernfelt, F. (2007). Diagrammatology. An investigation on the borderlines of phenomenology, ontology, and semiotics. Dordrecht: Springer.

References

25

Thompson, E. (2007). Mind in Life. Biology, Phenomenology and the Sciences of Mind. Cambridge, MA: Harvard University Press. Van Bockstaele, B., Verschuere, B., Moens, T., Suchotzki, K., Debey, E., & Spruyt, A. (2012). Learning to lie: Effects of practice on the cognitive cost of lying. Frontiers in Psychology, 3, 526. https://doi.org/10.3389/fpsyg.2012.00526. Varela, F. J., & Weber, A. (2002). Life after Kant: Natural purposes and the autopoietic foundations of biological individuality. Phenomenology and the Cognitive Sciences, 1, 97–125. Varela, F. J., Thompson, E., & Rosch, E. (1991). The embodied mind. Cognitive science and human experience. Cambridge, MA: MIT Press. Vrij, A., Fisher, R., Mann, S., & Leal, S. (2006). Detecting deception by manipulating cognitive load. Trends in Cognitive Sciences, 10(4), 141–142. https://doi.org/10.1016/j.tics.2006.02.003. Vrij, A., Granhag, P. A., Mann, S., & Leal, S. (2011). Outsmarting the liars: toward a cognitive lie detection approach. Current Directions in Psychological Science, 20(1), 28–32. https://doi. org/10.1177/0963721410391245. Walczyk, J.  J., Harris, L.  L., Duck, T.  K., & Mulay, D. (2014). A social-cognitive framework for understanding serious lies: Activation-decision-construction-action theory. New Ideas in Psychology, 34, 22–36. https://doi.org/10.1016/j.newideapsych.2014.03.001. Zalamea, F. (2012). Peirce’s logic of continuinity. Boston, MA: Docent Press. Zlatev, J. (2011). What is cognitive semiotics?. Semiotix: a global information bulletin, XN—6. Retrieved from https://semioticon.com/semiotix/2011/10/what-is-cognitive-semiotics/. Zlatev, J. (2015). Cognitive semiotics. In P. Trifonas Peter (Ed.), International handbook of semiotics (pp. 1043–1067). Dordrecht: Springer.

For a Cognitive Semiotics of Subjectivity

1  Subjectivity in Language This chapter will attempt to illustrate how a semio-linguistics approach specific to cognitive semiotics can make a concrete contribution to certain themes at the heart of the contemporary debate on cognition. Our topic will be subjectivity, and this chapter will put forward an enactivist explanation founded on a semiotic theory of effective action grounded on pragmatism. The central idea is that our ability to be subjects, that is, our ability to make ourselves the object of our own thoughts, is intrinsically linked to the semiotic capacity to lie and to the benefits which this capacity brings for the purposes of building strategic thought linked to effective action. This idea will be founded on a semio-linguistic theory of the person. In pre-cognitive semiotics, the main semio-linguistic theory, as far as the subjectivity in language is concerned, is Benveniste’s theory of enunciation.1 This very popular theory in both philosophy and psychology is essentially split into two parts: (1) the subject derives from the linguistic ‘I’; and (2) subjectivity is linked to the

1  It is worth saying that the word “enunciation” does not even appear in the English translation of the Problems in General Linguistics by Benveniste. Instead, the word used by the translator is “utterance”. However, “enunciation” has become more and more popular in semiotics and linguistics, so we prefer to use the French sounding word. More, the English translators of Semiotics and Language by Greimas and Courtes (1979) translate the French “énoncé” as “utterance”. This is a real mess, because, following the original translations, “utterance” would mean both the act of enunciation and the product of it! Therefore, we will use “enunciation” in order to mean the act of utterance which produces the sentence or the statement (“énoncé”). On the contrary, we will use “enunciate” in the general sense of “that which is uttered”. In contrast to enunciation, the enunciate “is the state resulting from enunciation, independently of its syntagmatic dimensions (sentence or discourse)” (Greimas and Courtes 1979: “Utterance”). In addition, we will not follow the decision of translating “enoncé” with utterance because, in the English language, “utterance” is much more related to the act than it is to the product of the act. Where possible, we will use “sentence” in order to express the product of the act of enunciation (“enunciate”), but “sentence” has a pure linguistic meaning, while an “enunciate” can also be an image or a picture.

© Springer Nature Switzerland AG 2021 C. Paolucci, Cognitive Semiotics, Perspectives in Pragmatics, Philosophy & Psychology 24, https://doi.org/10.1007/978-3-030-42986-7_2

27

28

For a Cognitive Semiotics of Subjectivity

semio-linguistic activity of enunciation. As far as the first point is concerned, Benveniste was perfectly aware that the ‘I’ is a very special linguistic sign in the sense that, in oral speech, its meaning and reference varies in accordance with who is saying ‘I’. This is true of very few other language entities (‘you’, ‘here’, ‘now’) and Benveniste remained fascinated, as did a great many later linguists and philosophers of language, by the “heretical transcendence” of these “ultra-special categories” which change meaning according to who it is saying them, later made famous by the name embrayeurs.2 These embrayeurs indicate an asymmetry that Benveniste found intriguing. “I” and “you” are not on the same plane as “he”, as the third person does not change meaning and reference in accordance with who is saying it. For this reason, according to Benveniste, ‘I’ (to an even greater extent that ‘you’) takes linguistic primacy in expressing subjectivity. As we will see, Benveniste’s first conclusion is true. “I” and “you” are not on the same plane as “he”. His second conclusion, on the other hand, is radically false. “I” has no linguistic primacy in expressing subjectivity. Quite the contrary, as the too frequently forgotten linguist Gustave Guillaume demonstrated, “he” has primacy in expressing subjectivity. Since Benveniste used to call the third person the “non-­ person”, given that almost all languages express the impersonal form in the third person (“il pleut”, “it rains” etc.), it is an impersonal conception of subjectivity which will be constructed here, from the starting point of a cognitive semiotics. These first considerations lead on very naturally to the second point in Benveniste’s theory. For him, the place of the subject was the place of an instance of mediation between the langue, namely the social and shared aspect of language, and the parole, namely the individual act giving shape to it. Benveniste called this “third” instance between the langue and the parole “enunciation” and believed that subjectivity in language was precisely a matter of this instance of enunciation which, for him, was occupied by the “ego” (first person). For Benveniste, the subject’s place was thus not that of the individual and subjective act, but rather that of the mediation instance which made us pass from the social and shared aspect of language to its individual appropriation within the discourse. For this reason the enunciation instance, which defined subjectivity’s place in language, was defined as “the very act of producing an enunciate” (sentence, statement etc.) which, for Benveniste, occurred by means of placing in discourse a series of virtual categories (langue) which could be actualised by means of an individual act (parole). It was precisely via this act that, according to Benveniste, the enunciation instance “constituted itself subject”, finding its privileged place precisely in those ultra-special language categories which include certain pronouns (“I”), deixis (“here”) and time indicators (“now”). Emile Benveniste gave the first formulation of enunciation as a process by which natural language (Saussure's langue) is turned into discourse. In between language (conceived of as a paradigmatic system) and speech — already interpreted by Hjelmslev as a syntagmatic 2  The expression is not from Benveniste, but comes from Ruwet’s translation of Roman Jakobson’s ‘shifter’ which indicated those language categories which “change reference person in accordance with who is saying them” (Jakobson 1963, Kleiber 2016).

1  Subjectivity in Language

29

system and now specified in its status as discourse — it is indeed necessary to supply mediating structures and to imagine how it is that language as social system can be assumed by the individual realm without as a result being scattered into an infinite number of examples of speech (Saussure's parole), outside all scientific cognizance. (Greimas and Courtes 1979: “Enunciation”)

As we have seen, according to Benveniste, language structures such as “I”, “here” and “now” were the privileged place of expressions of subjectivity because, in contrast to other language categories, their meaning was not immediately understandable within the language forms themselves. Understanding the meaning of “I” or “here”, in fact, requires leaving language behind and considering the concrete context of the discourse at which an instance takes on board the forms of language via an act. “Enunciation” is the name of this act that produces “utterances”. This subjectivity linked to an “I”, expressed by certain privileged language categories which inherently refer outside the language itself, seemed in perfect conformity not solely with the transcendence of a phenomenological “I”, like Husserl’s,3 but first and foremost, with certain psychological discoveries which used to connect, at least in temporal terms, the ontogenetic appearance of self-consciousness in infants with the acquisition of language. This is the origin of Benveniste’s enormous success, since he could ground in the dominant discipline of those times—linguistics—a series of approaches borrowed from philosophy and psychology. Benveniste guaranteed a semio-linguistic scientific character for these approaches, arguing that the subject finds his reality in the reality of language. De facto, the subject is nothing else than the “ego saying ego”. The “subjectivity” we are discussing here is the capacity of the speaker to posit himself as “subject”. […] Now we hold that that “subjectivity”, whether it is placed in phenomenology or in psychology, as one may wish, is only the emergence in the being of a fundamental property of language. “Ego” is he who says “ego”. That is where we see the foundation of “subjectivity”, which is determined by the linguistic status of “person”. (Benveniste 1966/1971: 224)

Benveniste’s move fits perfectly with that which founded cognitive semiotics. Semio-linguistic structures do not express cognitive structures nor are they an extrinsic means for something pre-existing occurring within the body, the mind or the brain. Quite the contrary, semio-linguistic structures build an indispensable scaffolding which brings cognitive structures forth. However, in Benveniste’s own terms, his move about subjectivity in language is untenable and entirely incompatible with the new knowledge coming from the cognitive sciences. We will thus retain his framework (semio-linguistic structures scaffold and bring forth cognition), but set out their specific features in an alternative form. In any case, a big part of the success of Benveniste’s enunciation theory is precisely due to the fact that it posed the subjectivity problem in a new and original way, even though no enunciation theorist after him ever used the word “subject”, preferring the more neutral expression “instance” at all times. However, it was 3  The close bond between Benveniste’s linguistics and phenomenology has been repeatedly highlighted. See, for example, Violi (2007), Coquet (2007).

30

For a Cognitive Semiotics of Subjectivity

Benveniste himself who devoted a great many pages—which later became ultra-­ famous—to the “subjectivity in language” issue. Greimas and Courtés (1979: “Enunciation”) noted that “Benveniste’s innovative contribution gave rise to a number of exegeses of a metaphysical or psychoanalytic bent, all of them exalting the unexpected reappearance of the subject and permitting the suppression of an ‘anonymous’ conception of language — credited — and discredited — for being a collective system of constraints”. Let us return, then, to the thesis which made Benveniste’s enunciation theory so successful. In perfect accordance with the “linguistic turn” he was writing within, his thesis was this: It is in and through language that man constitutes himself as a subject, because language alone establishes the concept of “ego” in reality, in its reality which is that of the being. (Benveniste 1966/1971: 224)

Clearly, for Benveniste, the subject derives from the linguistic “I”, but his thesis can certainly not be clearly understood without an understanding of what he meant by “subject” and in what way language can be said to be capable of founding the concept of “ego” in its reality. Fortunately, both are extremely clear in Benveniste’s work. The “subjectivity” we are discussing here is defined not by the feeling which everyone experiences of being himself (this feeling, to the degree that it can be taken note of, is only a reflection) but as the psychic unity that transcends the totality of the actual experiences it assembles and that makes the permanence of the consciousness. (Benveniste 1966/1971: 224)

The very close bond between Benveniste’s approach to the problem of subjectivity and the fields of psychology and philosophy is very clear, to the extent that Benveniste’s theories were formulated precisely in opposition to those of Sartre in The Transcendence of the Ego and Politzer in Critique of the Foundations of Psychology.4 By “subject”, in fact, Benveniste did not mean the non-linguistic person nor an embodied social actor with a sensitive, emotional component, but rather simply the psychic unit which transcends and brings together the lived experience. Thus, not consciousness, but rather what philosophers call self-consciousness, that unit (“I”) which must be able to accompany all lived experience, that “synthetic unity of apperception”, as Kant defined it. In classical philosophy, self-­consciousness is the never complete act with which the subject reflects on himself and subjectivity consists precisely in the ability to make oneself object of one’s own reflections. Consciousness is never solely a matter of intentionality directed at objects but one which is duplicated and reflects on itself. As such it is not solely consciousness but self-­consciousness. “I think” and “I think that I think” coincide in such a way that one cannot exist without the other. (Jaspers 1932: 195)

This is the capacity which, according to Benveniste, derives from the linguistic “I” and, more specifically, from the linguistic idea of “person”, which presides, for

 See Bianco 2008.

4

1  Subjectivity in Language

31

example, over the distribution and the correlation of personal pronouns such as “I”, “you” and “he”. The originality of Benveniste’s theory of subjectivity is its connecting self-consciousness with linguistic person theory, while thinking that self-­ consciousness derives (1) from “the correlation of personality opposing the I-you persons to the non-person he” (Benveniste 1966/1971: 204), and (2) from the “correlation of subjectivity operating within the preceding and opposing I to you” (Benveniste 1966/1971: 204), while identifying an “I” which is more profound than any point (1) type “I”. In fact, whilst “self-consciousness is possible solely by contrast” and neither “I” nor “you” “can conceive of themselves without the other”, according to Benveniste “ego” always retains a transcendent position and a primacy with respect to a “you” which the “I” itself poses. “I” is always transcendent with respect to “you”. When I get out of “myself” in order to establish a living relationship with a being, of necessity I encounter or I posit a “you”, who is the only imaginable “person” outside of me. These qualities of internality and transcendence properly belong to “I” and are reversed in “you”. One could thus define “you” as the non-subjective person, in contrast to the subjective person that “I” represents; and these two “persons” are together opposed to the "non-person" form (= he). (Benveniste 1966/1971: 201)

If it is the subject which is born with the linguistic “I”, it is because, for Benveniste, no self-consciousness exists without language, or rather, no identity of the subject exists outside the enunciation instance, namely outside the discourse situation in which someone enunciates, saying “I”. It is in this way that language is capable of founding the concept of ‘ego’ in its reality, and the “reality of language” which Benveniste writes about is precisely enunciation. Consciousness of self is only possible if it is experienced by contrast. I use I only when I am speaking to someone who will be a you in my address. It is this condition of dialogue that is constitutive of person, for it implies that reciprocally I becomes you in the address of the one who in his turn designates himself as I. […] As a matter of fact, language is responsible for it in all its parts. I posits another person, the one who, being, as he is, completely exterior to “me”, becomes my echo to whom I say you and who says you to me. (Benveniste 1966/1971: 224–5)

For Benveniste, then, (1) the subject derives from the linguistic “I”, (2) the linguistic I founds subjectivity and, thus, self-consciousness, in the “reality of its being”, which is enunciation. Thus, (3) enunciation is the condition of possibility of subjectivity, namely self-consciousness. This is a fundamentally important point. Benveniste (1966/1971: 226) says that “if one really thinks about it, one will see that there is no other objective testimony to the identity of the subject except that […] language is so organized that it permits each speaker to appropriate to himself an entire language by designating himself as I”. Having us believe that anyone who does not accept his view has effectively “not really thought” about something is a rhetorical mechanism which Benveniste makes frequent use of in his writings, at least when he makes statements which he cannot fully motivate, as seems to be the case with this statement. But this aspect of his argumentative strategies aside, if what Benveniste is saying was perhaps tenable in the period he was writing in, accepting it as true today is certainly no straightforward matter to the extent that we

32

For a Cognitive Semiotics of Subjectivity

now possess a great deal of other “objective testimony to the subject’s identity” which in no way seems to consist in the linguistic “I” and the person category.5 Quite the contrary, it would seem that precisely a certain reading of “he” and its delocutive capacity—a term capable of uniting “person” and “non-person”6—is able to identify the subject’s identity in the capacity a subject has to making himself object of his own semio-linguistic and cognitive activity. Thus, subjectivity would not consist in the “ego saying ego”, appropriating the language through discourse, as Benveniste argued, but rather in the subject’s ability to make himself object, or rather, the delocutive status which makes this duplication of the subject possible and enables him to think of himself as a person among other people. The second part of this chapter will construct a rigorous semio-linguistic theory on this matter, but, first of all, we must return to Benveniste’s idea that subjectivity is constituted within, and by means of, language and that the subject derives from the linguistic I. Is this really the case?

2  For an Enactive Semiotics of Subjectivity As it is well known,7 The Iliad’s characters certainly do not show formed subjectivity, such as that which we attribute to ourselves and which we are used to thinking in terms of—with Benveniste—whose very foundation is in language. But Achilles and Agamemnon certainly have language all the same, just as Odysseus does. However, the status of their subjectivity is profoundly different from Odysseus’ subjectivity, as well as our own. Man in the Iliad does not have a subjectivity like ours. He has no consciousness of his consciousness of the world, he has no inner mental space in which to exert introspection. Volition, planning, initiative are organised without any consciousness and are thus ‘told’ to the individual in the language familiar to him, sometimes with the visual aura of a friend dear to him or a figure of authority or a ‘god’, sometimes simply in a voice. The individual obeys these hallucinatory voices because he cannot not “see” for himself what to do. (Jaynes 1976: 101–2)

5  Psychologists such as Julian Jaynes (1976), classical antiquity philologists such as Bruno Snell (1948) and semiotics scholars such as Ugo Volli (1994) – and this without considering the cognitive science literature which moves practically in a diametrically opposite direction from Benveniste’s  - have done much work on this “objective testimony of the subject’s identity”, as Benveniste called it, which seemed absent to him in the period he was writing in. 6  ‘Delocutive’ means that which is outside locution, i.e. what is neither locutor nor interlocutor. We will shortly see how ‘he’, which is both the third person and the non-person, is capable of acting as extensive term within the linguistic category of ‘person’ with its ability to represent all positions. This is a reading inspired by the linguistic theories of Hjelmslev and Guillaume, which overtly contradicts Benveniste’s idea, according to which ‘he’ is the exclusive form of the ‘non-person’. 7  See Detienne and Vernant 1974, Volli 1994.

2  For an Enactive Semiotics of Subjectivity

33

Perhaps we can put it better. This delocutive “he” (voice, god, friend, etc.) which “tells” the individual his own plans, initiatives and will is this same individual’s “I”, the form of his person and subjectivity. Subjectivity is this “he” of the “I” who speaks to the “I” and modulates his perspective to the extent that when this mechanism does not function and the subject believes this “he” to be truly a “he”, he is frequently suffering from schizophrenia or hallucinations. This “subject”, this third person “he” speaking to the first person “I”, thus has a semiotic origin and structure which many of The Iliad’s subjects do not have. For example, when Agamemnon notifies Achilles that he will deprive him of his war booty in the person of the slave Briseis, Achilles’s first reaction is to pull out his dagger and kill Agamemnon. But after a brief movement, he stops. Certainly for a modern reader, Achilles’ change of mind would be based on a consideration that “revenge is a dish best served cold” and that he will take sweeter revenge and achieve better satisfaction by not killing Agamemnon, but by withdrawing from battle and obliging Agamemnon himself to beg him to return, together with his Myrmidons. Achilles, in other words: 1. must have reflected on himself, positioning his own subject as the very object of his reflections; 2. must have been able to build a strategy which delineates a possible world which is alternative to the real world; 3. must have bet that this possible world generated by his strategy and his ability to make himself object and act as a delocutive subject would ultimately become “the same world” as the real one in the future; 4. must have known how to hold back his rage in view of a more profitable future for himself. But Achilles has none of these four abilities, he has none of these four characteristics defining subjectivity. He does not know that he knows, he does not feel that he feels, he does not perceive that he perceives, he cannot make himself object of his own reflections, he cannot construct that duplication, that turning back on oneself in which a subject is capable of objectifying himself and duplicating himself to reflect on himself. Achilles cannot subjectify himself and is subject only to the extent that he is subject to the will of the goddess Athena, who, “sent by the heavens” and “visible only to him”,8 blocks his arm.9 The characters in the Iliad have no subjectivity—only the gods have this—the characters have only destiny. They are subjects only in the sense that they are subject to the will of the gods. While we consider ourselves full subjects capable of taking responsibility for our actions, thinking about our actions and generating

 Iliad I, 193, 198.  Snell comments (1948/1953: 57): “The poet had no need here of any ‘apparatus’. Achilles simply controls himself. Athena’s action is a disturbance for us but for Homer she was necessary here. We would expect a ‘decision’, namely reflection and action by Achilles. In Homer, by contrast, man is still not exponent of his decision”. For an important reactivation of linguistic and philosophical ideas connected to Iliad, see Piazza 2019. 8 9

34

For a Cognitive Semiotics of Subjectivity

strategies in which we objectify ourselves as characters among other characters, Achilles completely lacks this capacity, because he is totally lacking subjectivity. The narrative conventions as regards the characters are such as to exclude all forms of organisation of the mental space in which we can recognise the mediation of subjectivity. […] As in Homer a mental space in which the world is represented and we take decisions, make plans, assess the various promptings of our instincts and passions to achieve a unitary synthesis is lacking, the same is true of a unitary concept of the person, the ‘I’. (Volli 1994: 168, 171)

The opposite is true with Odysseus. From the Iliad onwards, he is a character whose specific features and isolation from his fellow adventurers is underlined. What makes Odysseus different from the other heroes is in no way that he has a superior intelligence or that he has a better ability to read the situation, but rather “cunning”, a shrewdness (metis), his semiotic capacity to come up with tricks and stratagems, setting up signifier surfaces10 capable of lying about the world’s state for the purpose of achieving his goals. It is this semiotic capacity to lie which gives Odysseus his subjectivity, that “I” which the other Homeric heroes are entirely lacking. See, for example, Odysseus’s refusal to punish his maids flirting with their suitors immediately in Canto XX of the Odyssey and compare it with Achilles’s decision not to fight Agamemnon. Firstly, Odysseus speaks to himself and tries to calm himself down and even goes as far as to objectify himself, using the word ‘you’ in reference to his heart, establishing an intersubjective relationship between locutor and interlocutor entirely interior to the person category, which is presented in the form of a dialogue: Then he smote upon his breast and rebuked his own heart, saying: ‘Endure, my heart; yea, a baser thing thou once didst bear, on that day when the Cyclops, unrestrained in fury, devoured the mighty men of my company; but still thou didst endure till thy craft found a way for thee forth from out the cave, where thou thoughtest to die. Endure, my heart; yea, a baser thing thou once didst bear (Odyssey, XX, 18–28)

It is Odysseus who performs the functions which, in the Briseis episode, are attributed to the gods; “he himself meditates, decides, holds back. It is not by chance that we frequently find him being referred to with the epithet ‘divine’ (theoios) and very frequently ‘luminous’, in kinship with the Zeus root” (Volli 1994: 176). It is precisely the subjectivity which Achilles lacked which makes Odysseus similar to the gods, to the extent that Odysseus’s heart obeys its subject just as Achilles obeyed the gods. Here we find the important topography of the instances of enunciation highlighted by Coquet (2007); a judgement instance (subject) considerably detached from a sensitive instance (the heart) which must “bear”, “submit to” and “believe to die” (non-subject). The subject (I) speaks to the non-subject (you) like a locutor speaks to an interlocutor in a dialogue enunciation context, but a third instance—the most fundamental of all—emerges in the third person (he), a delocutive q­ uasi-­subject  Signifier surface refers to the fact that one can consider the signifier (or expression plane) without the signified (meaning or content plane). This separability is significant in that we can use the signifier to convey meaning that is not related to how the world is (and for this reason can be used in order to lie).

10

2  For an Enactive Semiotics of Subjectivity

35

who belongs to neither the subject (I) nor the non-subject (you), but rather constitutes the former while it frees the latter: shrewdness (metis). What is metis, then, the quasi-subject cited in the third person which supervenes to the ‘I’ (subject), freeing the ‘you’ (non-subject)? Metis is “calculating reason”, “practical rationality”, the structuring form of an act capable of establishing the subject and the non-subject. In Theogony, Metis is Zeus’s first bride whose son is prophesied to overthrow his father. This is why Zeus swallows Metis while she is pregnant and Metis—“who know more than all the gods and mortal men”11—thus lives inside Zeus and advises him, giving him her prophesying ability. Zeus is, in fact, defined metíeta, the metióeis.12 This ability to see possible alternative worlds to the real world is precisely Odysseus’s great capacity together with his strategic thinking, his calculating reason (metis) and this semiotic ability to construct deceits, to the extent that Odysseus is frequently called polymetis, full of prophesying ability and blessed with a reason which knows many calculations and tricks. These two things—and this is our point—are not unconnected. It is, in fact, his ability to predict the possible behaviour of the Trojans when faced with the horse that enables Odysseus to win the war by deceit and tricks, constructing a signifier surface which can be used for lying. It is his semiotic ability to set up signs hiding objects and taking their places that enables Odysseus to free himself from Polyphemus’s grotto, calculating the wisdom of not killing him right away and reflecting on possible alternative worlds to the real one, in which he himself is one character amongst many others. Rethinking Polyphemus and other equally famous episodes, such as the building of the horse or that of the singing of the mermaids, it is easy to believe that metis is, first and foremost, the ability to offer up an opportune signifier surface to the world, i.e. to take on an outward appearance which does not correspond to one’s interior truth but is opportune in a given situation, suited to a certain plan of action. Odysseus escapes Polyphemus’s grotto and hides, hanging under the belly of a sheep which the blinded giant lets out. The Trojan horse works in the same way, hiding the warriors who will destroy the city under its outward appearance. In the case of the maids […] Odysseus does not let the rage that would betray him get the better of him but pretends to be calm and asleep, maintaining his beggar’s garb with which he protects himself in the midst of his enemies. To be calculating reason, metis must also be simulation and dissimulation, semiotic machine with which to construct appearances. It is a matter of lying, erecting meaning effects, making believe and not letting the truth be known and, at the same time, of establishing an appearance which conceals a secret. (Volli 1994: 180–1)

Thus metis is semiotic capacity to construct signifier surfaces which substitute objects, conceal them and take their places, saying things about them without these things necessarily being true.13 In fact, Umberto Eco (1975: 17), as we have seen in the previous chapter, defined semiotics precisely as the discipline which studies “all that which can be used in order to lie”. The semiotic capacity to lie serves to make action effective and strategic. For this to be possible, what is required is not only  Theogony, verses 886–7.  See Detienne and Vernant 1974: Chaps. 3 and 4. 13  See Detienne and Vernant 1974: 14–5. 11 12

36

For a Cognitive Semiotics of Subjectivity

predicting the behaviour of others (once again metis qualities) but first and foremost knowing how to envisage oneself in a multiplicity of possible alternative forms and worlds, acting as an other and objectifying oneself as a specific subject of one’s own reflection. From this envisaging, two different ‘I’s emerge; one who acts and one who watches him act, taking on himself the freedom of which he deprives himself (“endure my heart”) and evaluating his behaviour in various possible contexts. This duplication of the subject himself for the purposes of effective action, whether we call it “self-consciousness” or the “I”, exists only to the extent that a semiotic capa­ city to make oneself object of one’s own reflections exists, representing the foundation of subjectivity. Subjectivity is not born of the linguistic “I” but of the semiotic ability to lie, to construct signifier surfaces which build worlds that are alternative to the real one and place fictional objects within them. For this reason, metis presents simultaneously three Latour’s modes of existence, these being metamorphosis (2012: 181–206), technique (2012: 207–232) and fiction (2012: 233–258). The “I” with its self-consciousness ability and its personal structure capable of saying “ego” and “you”, is born of this strategic capacity to construct actions based on the image of the other and his potential responses to the action of the ego. This capacity is not born with language. It is born with semiotics in terms of effective action, and verbal language inherits this semiotic ability, constructing a formal apparatus capable of expressing it (formal apparatus of enunciation). The fact that subjectivity develops in children at the end of the secondary intersubjectivity phase (Trevarthen and Hubley 1978; Gallagher and Hutto 2008), frequently prior to the learning of language, is coherent with these ideas. From 9–18  months of age, in fact, children develop, firstly, a semiotic ability to deceive others via the non-verbal construction of possible alternative worlds to the real one (pretending to cry, vocalising, pointing, etc.) and an equally reinforced capacity to imagine linked to pretence (Reddy 2008), which leads to the development of pretend play, namely that form of play in which we pretend to be someone else. In this way not only does ontogenetic child development, as it appears in modern cognitive science, detach the subjectivity from the linguistic “I” to connect it up with the imagination, action and the semiotic ability to deceive others, but it also shows that the “I” is the last thing to develop. And that its development depends fundamentally on the intersubjectivity of “you” and the impersonal character of “he”. This completely turns Benveniste’s thesis in which the “I” takes primacy over the other linguistic persons on its head. We will look at this in the next section. Cognitive semiotics is thus capable of telling a completely different story on the person notion, on the relationship between the linguistic persons and the subjectivity linked to them, as compared to the pre-cognitive semio-­ linguistic tradition. For many reasons, then, Benveniste’s thesis, which sees subjectivity as dependent on the linguistic “I”, is no longer tenable today. The fact that “language is so organized that it permits each speaker to appropriate to himself an entire language by designating himself as I” (Benveniste 1966/1971: 314) is by no means a sign that subjectivity derives from the linguistic “I”. Quite the contrary, it is a sign of the fact that language expresses that semiotic capacity which renders the subject an instance

3  The Pre-Logical Mind

37

capable of making itself the object of its own reflections. It is extremely significant that language constructs certain ultra-special language categories such as ‘I’, ‘here’ and ‘now’ which cannot be entirely interpreted within language, but require us to put linguistic forms aside and refer to the concrete situation in which they are used. And within this “formal apparatus of enunciation”, it is the delocutive structure of the third person (“he”) which grounds the “I” via the “you”. Benveniste’s hierarchy is thus turned on its head: it is no longer “I-you” versus “he”, as Benveniste argued, but rather “I” versus “I + not-I” (he). We will now see how Gustave Guillaume perfectly expressed this primacy of the delocutive category (he)—which Benveniste mistakenly saw as ‘non-person’—over the person categories (I-you). This leads on to our “impersonal enunciation” thesis (Paolucci 2020; Metz 1991), namely a semiotic theory of enunciation which must be founded on the delocutive category of “he” as extensive linguistic subjectivity term, the union of person and non-person. Because only the third person is capable of being I and non-I at the same time and, thus, of expressing that ability to make ourselves object of our reflections which we call “subject”. We thus now need to clarify this fundamental point from the starting point of a semio-linguistic thought.

3  The Pre-Logical Mind In the first chapter we saw that one of cognitive semiotics’ first steps was replacing the classic, confused notion of sign with that of semiotic system, capable of describing general language functioning. Some of the notions of 4E cognition, which were our inspiration, are themselves founded on systems theory (Varela 1978; Chemero 2008), which have historically been powerfully connected to semiotic theory, thanks to the work of René Thom and, above all, the extraordinary thoughts of Jean Petitot (1985, 1992, 2011). The dynamical systems theory underlying enactivism comes, like that of semiotics, from the mathematical theory of singularity, but is structured from the starting point of the life sciences and complexity theory, without being founded on a clear semio-linguistic base,14 such as that which we are attempting to consolidate here. Even if they were inspired by some reflections on language and by linguistics, the classic cognitive sciences, thinking at the level of mediation specific to linguistic form representations, frequently founded their conception of language on a logical model. This led to them importing the very idea of meaning from analytical philosophy, where linguistic meaning refers to truth conditions. If this seemed adequate for a theory of cognition founded on the idea of representation, it cannot suffice for cognitive semiotics, according to which meaning refers to sense-making and content forms construction operations. The first step in a cognitive semiotics capable of dialoguing with 4E cognition must thus be specifying very clearly what we mean by

14

 For a comparison, see Paolucci 2017.

38

For a Cognitive Semiotics of Subjectivity

“semio-linguistic system” and how this is structured and articulated. If a semiotic systems theory is constituent of cognition and constitutes an indispensable scaffol­ ding, we will need an extremely clear understanding of the way a semio-linguistic system rearticulates sense-making in its own way, and contributes to what I consider the most fundamental of the cognitive operations (sense-making), as enactivists have shown us. This will have important repercussions precisely for a theory of subjectivity. I have shown elsewhere (Paolucci 2020), as further proof of a shared basis between dynamic systems theory and semio-linguistic systems theory, that the concept of system as formulated by Ferdinand de Saussure,15 which underlies modern linguistics, was imported into linguistic theory from Leibniz’s differential calculus principles. Saussure’s idea is that a linguistic system self-organises into a system of values which reciprocally determine themselves and that each element’s identity depends simultaneously (1) on its relationship with other elements in the system, and (2) on the system’s relationship with other heterogeneous systems. We see then that in semiological systems like language, where elements hold each other in equilibrium in accordance with fixed rules, the notion of identity blends with that of value and vice versa. […] Let us observe from the outset that even outside language all values are apparently governed by the same paradoxical principle. They are always composed: (1) of a dissimilar thing that can be exchanged for the thing of which the value is to be determined; and (2) of similar things that can be compared with the thing of which the value is to be determined. Both factors are necessary for the existence of a value. (Saussure 1916: 110, 115)

Saussure’s example is currency; the value of a five-franc note is determined by the fact “that it can be exchanged with a specific quantity of something else, such as bread, for example” (Saussure 1916: 115). However, it is also determined by “a similar value from the very same system, for example, a one franc note or a coin from another system (a dollar)” (Saussure 1916: 115). Semio-linguistic systems are inherently pluri-systemic. The identity of their elements depends on their own system and the heterogeneous systems with which this system is translated and compared. In brief, a semio-linguistic system is, for Saussure, a system which is structurally coupled with other non-linguistic systems with which it is reciprocally determined. Its identity is a relational identity which refers back to translation with other systems. It is precisely as a result of this fact of being a system composed of translation from other systems that the semio-linguistic system delineates a sui generis version of the sense-making cognitive operation (Di Paolo et al. 2019, see Sarti et al. 2019 for a mathematical model of the semiotic version of dynamical systems). Languages and semiotic systems are forms of interface between organism and environment which represent the background to our perception of the world. To their adaptive repertoires of norms, usages and habits, we attribute a social value which gives meaning to our experience, shaping it. The more specifically individual experience which delineates us is a kind of modulation of this meaning.  Notwithstanding the later fame of the ‘structural linguistics’ formulation, Saussure never spoke of ‘structure’ but always and only of ‘system’.

15

3  The Pre-Logical Mind

39

For this reason, the way in which a semio-linguistic system enacts its sense-­ making procedures is crucial. The semio-linguistic sense-making procedure prescribes that, in the structural coupling between a linguistic system and its environment, linguistic singularities of the various languages will correspond to the values of the differential relationships between the elements of the language. For example, Hjelmslev (CC, NE) showed that the category of cases is present in all languages, but the system of cases varies from language to language in accordance with the reciprocal relationships between the elements in a system. The category of cases is to be found invariably in Sanskrit, Latin and German. But the system of cases differs from one language to the next because the number of cases is not the same and because the correlations between the cases are specific to each language [...]. What is true of cases is also true of all other grammar categories in the same way. The system is thus the specific form in which the category is realised in a specific language. This form is defined by means of the number of terms and via the correlations between these. (Hjelmslev, NE: 27–8)

But, however variable the relationships between the elements in a system and however numerous their correlations, in another important work, Hjelmslev (NE) was able to identify the “general structure of linguistic correlations”, that is, the general structure which defines the systemic relationships between the elements of language, characterising sense-making operations. The linguistic system is not constructed as a logical system of oppositions between positive terms and negative terms. The linguistic system is free in relationship to the logical system it corresponds to. It can be orientated differently on the axis of the logical system and the oppositions it contracts are subject to the law of participation: there is no opposition between A and non-A, there are oppositions between A on one hand and A+non-A on the other. There is nothing surprising about this discovery because Lévy-Bruhl’s research shows that language bears the mark of a pre-logical mind (CC: 188).

This pre-logical mind which, for Hjelmslev, is at work in language, is of enormous interest here. We are interested in understanding not only in how the pre-­ logical mind is made but also, and primarily, what semiotic studies about language can tell us things about this mind, which is seen as underlying language. With the term “pre-logical mind”, Hjelmslev is not simply saying that language cannot be reduced to principles of ‘non-contradiction’ and “excluded middle” specific to classical logic, but also that language cannot be reduced to the classic principles of a certain structural linguistics with which Hjelmslev, well before Chomsky and 1990s cognitive linguistics, was in open conflict. The target of Hjelmslev’s argument and its concept of the pre-logical mind was precisely the way in which Roman Jakobson thought of the reciprocal determination relationships between semio-linguistic elements (structure). Inspired by the structure of our phonatory apparatus, Jakobson (1963) was, in fact, thinking of the differential relationships which made up his famous list of relations between phonemes in an exclusive way, namely on the basis of the idea that, in an opposition relationship, the presence of a term (A) excludes the simultaneous presence of the opposite term (non-A). This is natural given that—with the exception of some Tibetan monks after lengthy training or singers such as Demetrio Stratos (for this

40

For a Cognitive Semiotics of Subjectivity

reason studied at length precisely by phonetics laboratories)—humans cannot produce more than one sound at the same time. This obviously excludes the contemporary presence of opposite traits to that articulated. For Jakobson, exclusive relations are essentially of two types: (1) qualitative oppositions (A versus B) in which, on a common axis basis (for example, labiality) two elements oppose one another and are differentiated via the presence of two opposing traits (for example “bilabial” versus “labiodental”), or (2) privative oppositions (A versus non-A) in which the opposition is between the presence of a trait (A), which defines a term as marked, and its absence (non-A), which defines the opposite term as non-marked. Jakobson’s famous phonemes list (1963) is entirely made up of oppositions traceable to these two types. It is thus clear that many other elements of language—from meaning to enunciation—are unlikely to function in the same way in which the human mouth and phonatory system works.16 This leads on to a first semiotic-cognitive principle: there is no isomorphism between the expression plane and the content plane of language. Signified and signifier present heterogeneous and different structures. However, hi­storically and for not entirely banal motives (see Paolucci 2010), Greimas and Benveniste imported Jakobson’s qualitative (A versus B) and privative (A versus non-A) oppositions into the “elementary structure of signifying” (semiotic square) and the “person correlations” which constituted the “formal apparatus of enunciation” (“person” versus “non-person”). In “The General Structure of Linguistic Correlations” and The Category of Cases, Hjelmslev showed that this did not seem to be possible and that both the content plane structure and the person correlation presented an entirely different form of relationship. It would seem that a system is frequently organised on the opposition between a precise term on one hand and a vague term on the other. (NE: 33) Roth discovered that the opposition which takes place within a grammar dimension is not an opposition between positive and negative ideas but an opposition between a simple idea and a complex one. […] Roth’s principle can be generalised. Linguistic system structure is not such as to be capable of maintaining a distinction between a positive term and a negative term (at best an extremely rare case). The real and universal opposition is between a definite term and an indefinite term. (CC: 185-6)

Hjelmslev (NE: 39–40) termed the precise term (A) intensive, which he defined as a term which is installed in a single semantic zone and “incapable of crossing the frontiers”. By contrast, he termed the vague term (A  +  not-A) extensive as it is capable of extending its value to the totality of the category and thus take on the value specific to the term it opposes too. An extensive term possesses the ability to extend its meaning to the whole area whilst an intensive term installs itself definitively in a single zone and cannot cross the border. (Hjelmslev, NE: 40)

 Federico Albano Leoni (2009) recently showed that not even our phonatory system functions as Jakobson believed.

16

3  The Pre-Logical Mind

41

For example, the man/woman grammatical relationship is not a qualitative opposition between the feminine (A) and the masculine (B) or, as in psychoanalysis, between masculine (“presence of the phallus”) and non-masculine (“absence of the phallus”), but rather a participatory opposition between the feminine (A) and the feminine+non-feminine (A + non-A). In the semiotic square of sexuality, we can see how this relationship was conceived exactly as an opposition between a positive term and a negative term (polar opposition, “A versus B”). Here the masculine (B) is contrary to the feminine (A) while the non-feminine (non-A) is contradictory to the feminine (A). The feminine (A) is likewise contrary to the masculine (B) while the non-masculine (non-B) is contradictory to the masculine (B). The complex term (A + B) would then refer to a copresence of masculinity and femininity as in the case of hermaphroditic animals. However, Hjelmslev entirely rejects this understanding of this relationship in terms of polar opposition, choosing instead to characterize the underlying opposition as participatory. In contrast to the semiotic square, Hjelmslev demonstrates that this relationship is rather an opposition between a precise term and a vague term in which “woman” is intensive (or precise) and “man” extensive (or vague). In this case, “man” (A + non-A, rather than B as above) is opposed to “woman” (A) when masculine needs to be differentiated from feminine, but encompasses “woman” (the contrary term), non-man (the contradictory term above) and hermaphrodite (the complex term above) too in phrases like “man is an intelligent animal”. For this reason, as Hjelmslev has shown, whereas “woman” is a precise term, concentrating its meaning in a single semantic area (feminine), “man” is a vague term which disseminates into all categories, as it can represent the corresponding contrary term (‘man’), the contradictory term (non-woman) and the complex term (‘hermaphrodite’), as well as “woman”, the precise term (A), herself. Precisely because the masculine linguistic gender is extensive and does not mean “masculine”, but, rather, “feminine+non-feminine”, certain recent proponents of gender equality prefer to use “her” rather than “his” in vague gender attribution contexts, according the feminine form the extensive role normally accorded to the masculine in language. Hjelmslev is thus capable of demonstrating that it is exactly a form of tensive relation specific to a “pre-logical mind” which underlies all linguistic systems, as it is to be found not only in all gender systems and cases but also in those of time, manner, number, person, etc. (see CC: 185). For example, as far as linguistic cases are concerned, in English and Scandinavian the genitive is intensive (A) and the nominative extensive (A + non A). In French, as far as the number is concerned, the plural is intensive and the singular extensive. In English, German and Scandinavian, preterite is intensive where the present is extensive, etc. (NE: 34). This leads on to the decisive idea that a semio-linguistic system has a constituently participatory structure. Its elements do not possess an identity such as to allow oppositions of an exclusive type between positive and negative terms. On the contrary, they possess an identity in which an element can always participate in the specific value of the opposite element. For this reason, Hjelmslev defines the oppositional relationship forms specific to a semio-linguistic system not in the exclusive form “A versus

42

For a Cognitive Semiotics of Subjectivity

n­ on-­A”,17 but rather in the participatory form “A versus A+non-A”, where “A” is intensive and “A + non-A” extensive. It seems to us that this way of looking at things is closer to the real state of affairs that Jakobson’s: in the place of marked and non-marked, it is intensive and extensive that we need to use. The term intensive is not characterised by the absence of something but by the fact of being able to occupy any zone of the category area. […] “Marking” is an artificial medium introduced by this theoretician for the purposes of explaining something which, in the final analysis, is explained much better without it. “Marking” is a superfluous invention and a pointless complication. (Hjelmslev, NE: 42)

This is what this “pre-logical mind” is and this is how it is at work in language. Given any semio-linguistic category (gender, number, time, case, aspect, etc.) there will always be at least one vague term (A + non-A) capable of locally assuming the value (A, B, C, etc.) specific to all the other precise terms (intensives) within the category. That is how language works: it is a dynamic system of a semio-linguistic type structured by a “pre-logical mind”. It is certainly not accidental, then, that what, starting with Benveniste, has been called “person correlation”, was, for Hjelmslev, a participatory relationship between intensive (‘I’ and ‘you’) and extensive (‘he’) terms. Where ‘I’ and ‘you’ are, in fact, precise terms (intensives), ‘he’ is a vague term (extensive). For example, in French the third person (‘il’) is both the third person form (“il se promene”—he walks) and the non-person form (“il pleut”—it rains), but it is also capable in certain contexts of adopting “by law of substitution”, the forms “I” and “you”. In all these cases, where “I” and “you” define precise terms (intensive), “he” defines a vague term (extensive) capable of extending its meaning to the whole category. In this way, inspired by Hjelmslev, we can say that the opposition between ‘I-you’ and ‘he’ is by no means a privative opposition between the person (I-you) and the non-person (he) as Benveniste argued, but rather a participatory opposition between two intensive terms (“I” and “you”) and an extensive term (he), which is capable of taking on the value of the other two.18 There is a pre-logical mind at work in the semio-linguistic subjectivity correlation. Hjelmslev’s relationship is not the form of relationship which defines the person correlation for Benveniste and on which he founded enunciation theory as we know it. For this reason, we must take seriously Parret’s idea that the ideas of Hjelmslev could “act as conceptual horizons and a source of inspiration for much pioneering research” (Parret 1995: 1, see also Migliore 2017).  Where “non-A” can be interpreted also as the polar opposite of A, “B”. “non-A” is simply that which is not A”, “the absence of A”. 18  For Hjelmslev, the ‘he’ (NE: 37) defines the neutral category which is then transformed into the extensive term even though the participatory correlation it indicates (‘you’ versus ‘I-he’), and the opposition between a precise term (you) and two vague terms (I-he) differs from what has been and will be proposed here (‘I-you’ versus ‘he’) which, on the other hand, involves two precise terms and one vague one. What interests us here is the person correlation on which to found enunciation theory and not the general structure of linguistic correlations at a grammar level, which is what Hjelmslev was studying. But obviously what is important to us at this level is that the person correlation is a participatory opposition and not a privative opposition, as it was for Benveniste. 17

4  Persona. The “I”, “You”, “He” Hierarchy

43

4  Persona. The “I”, “You”, “He” Hierarchy We can now return to the issue of the origin of subjectivity in language. As we have seen, for Benveniste the subject depended on the linguistic “I” and the linguistic “I” depended on the person correlation. This correlation defines a privative opposition between the presence of the person (I-you) and the absence of the person (he) as “he” was defined exactly as a “non-person”. For Benveniste (1966) “I” takes primacy over the other linguistic persons. Indeed, ‘he” is a “non-person”, but also “you” depends on “I”, “just as the interlocutor depends on the locutor” and is defined in function of the locutor as “the person to whom ‘I’ speaks”. Furthermore, given that in performative sentences like “I swear” Benveniste found a linguistic act which functioned solely in the first person, he could argue that an “I” form more profound than any “I” and any “you” existed. Through this linguistic form, “things are done with words” (see Austin 1975). I swear is a form of peculiar value in that it places the reality of the oath upon the one who says I. This utterance is a performance; "to swear" consists exactly of the enunciation I swear, by which Ego is bound. The enunciation I swear is the very act which pledges me, not the description of the act that I am performing. In saying I promise, I guarantee, I am actually making a promise or a guarantee. […] Enunciation is identified with the act itself. But this condition is not given in the meaning of the verb, it is the ‘subjectivity’ of the discourse which makes it possible. The difference will be seen if you substitute je jure, “I swear”, with il jure, “he swears”. While je jure is a commitment, il jure is simply a description, on the same plane as il court, il fume, “he runs, he smokes”. (Benveniste 1966/1971: 229)

As it is clear, in the case of the performatives, “you” functions exactly like “he” and the oath applies only to the “I”. This is the origin of the Benveniste linguistic person hierarchy: “I-you-he”. As we have seen, it would seem that cognitive semiotics completely turns this hierarchy on its head, in the direction of the “he-you-I” hierarchy. What is needed now is to demonstrate the semio-linguistic reasons behind this overturning. Benveniste drew his linguistic person hierarchy not from the system of language, but from the concrete allocutive situation, which he calls “discourse” and defines as the “language in action and necessarily between partners” (Benveniste 1966/1971: 220). It is, in fact, solely within this oral discourse situation formulated in real presence that “I-you” differs from “he” and the “person” (I-you) differs from the “non-­ person” (he). And it is truly strange that Benveniste does not hesitate to find this same correlation, based exclusively on the behaviour of the reference in concrete allocutive situations and untenable outside this framework, also within the linguistic system, with very evident distortions. A linguistic theory of verbal person can be constituted only on the basis of the oppositions that differentiate the persons; and it will be summed up in its entirety in the structure of these oppositions. In order to uncover this structure, we could start with the definitions used by the Arab grammarians. For them, the first person is al-mutakallimu 'the one who speaks'; the second, al-mubiitabu 'the one who is addressed'; but the third is al-yii'ibu 'the one who is absent.' A precise notion of the relationships among persons is implied by these

44

For a Cognitive Semiotics of Subjectivity d­ enominations; precise especially in that it reveals the disparity between the first and second persons and the third. Contrary to what our terminology would make us believe, they are not homogeneous. […] “I” designates the one who speaks and at the same time implies an utterance about “I”; in saying “I”, I cannot not be speaking of myself. In the second person, “you” is necessarily designated by “I” and cannot be thought of outside a situation set up by starting with “I”. […] But in the third person a predicate is really stated, only it is outside “I-you”; this form is thus an exception to the relationship by which “I” and “you” are specified. Consequently, the legitimacy of this form as a “person” is to be questioned. […] Indeed, it is always used when the person is not designated and especially in the expression called impersonal. […] The “third person” is not a “person”; it is really the verbal form whose function is to express the non-person. It is precisely the “absent person” the Arab grammarians spoke of. (Benveniste 1966/1971: 196–9)

It is very clear that Benveniste defines the “I-you” versus “he” relationship on the basis of a privative opposition of an exclusive type (“presence” versus “absence” of the person). At the same time, this privative opposition is grounded exclusively on an allocutive, “in presence”, model. I refers to the locutor (who can become ‘you’), you to the interlocutor (who can become ‘I’) and “he” to as “what is outside the locutive act”. From there, with the help of the fact that, in most languages, the impersonal form is expressed by the third person form, Benveniste homogenises the “third person” and the “impersonal” in the form of non-person (the absence of the person). But this is clearly unacceptable. As Joly highlighted: The whole of Benveniste’s theory is based on the idea that the third person is a ‘non-­ person’. I have had the opportunity, with others at the time, to show that this idea - and a fortiori, the conclusions drawn by Benveniste himself and many other semiotics scholars and literary critics - is founded, in the best hypothesis, on a misunderstanding and, in the worst, on nonsense relating to the nature of the person within language. (Joly 1994: 48)

Indeed, if Benveniste had turned his attention to the linguistic forms rather than the dialogue situation of discourse, he would have realised that the opposition between “I-you” and “he” is a participatory opposition of the type: “A” versus “A+non-A” (“presence of the person” versus “presence of the person+absence of the person”).

And not a privative opposition of the following type: “A” versus “non-A” (“presence of the person” versus “absence of the person”),

as it is the one that he proposes. In all languages, in fact, “he” defines both the form of the third person and the form of the impersonal, as well as performing various “substitution roles”, as Hjelmslev calls them (NE). For this reason, if we turn to the language system, as Benveniste does not do, the foundational opposition grounding enunciation is in no sense a privative opposition which opposes the “person” (I-you) to the “non-person” (he) but rather a participatory opposition which opposes the “person” (I-you) to the “person+non-person” (he/she/it). In every language, the “third person” functions alternatively both as person and non-person (the “impersonal”) and, for this reason,

4  Persona. The “I”, “You”, “He” Hierarchy

45

its opposition to “I-you” has a participatory form of the “A versus A+non-A” type, which is specific to a pre-logical mind. We find a highly significant dichotomy here which we also saw at work in enactive theory of language: the semio-linguistic system cannot be reduced to the interaction of a system of dialogue and to participatory sense-making. It can be grounded on that, but it cannot be reduced to that, as Di Paolo et al. (2019) demonstrated. On the contrary, this is exactly what Benveniste is primarily doing, “losing” language and its correlation system, focusing his attention only on dialogue and participatory sense-making. However, enunciation is not solely a question of face to face interaction. Indeed, at least one other enunciation model exists, the one which inspired this work, that is at the heart of Bruno Latour’s book Enquetes sur les modes d’existence. For Latour (2012), enunciating is first and foremost e-nunciating (ex-nuncius), sending a nuncius, a messenger who speaks for us. This messenger can be a diffe­rent subject (you) or an enunciate (he), a text or a work produced by the enunciation instance, but “enunciating” does not refer to dialogue, but primarily to the operations of sending, mediating and delegating. In Latin “ex” means “outside of” and the nuncius is “he who takes messages”, “he who lets us know”.19 In enunciation, the nuncius—the enunciate—is thus distanced from he who has sent it (ex), it takes the place of the enunciating instance and speaks for it, even when it is absent. Distancing, substitution and delegation: enunciation is a theory which has to do with sending and with what is present solely by delegation but, in actual fact, remains absent. And what remains absent can also be the true intentions of the person enunciating, such as in lying or strategic action, when we want to give an image or message of ours which differs from what we actually are. This is the basis for what I said regarding how subjectivity depends on our cognitive ability to think of ourselves as characters among other characters, within a story that we want to tell with our own words or with our own actions. Enunciating is the act of sending a nuncius, a messenger, a “word-bearer” who speaks for us. The theory of enunciation is thus first and foremost a theory of delegates, a theory of mediators, a theory which enquires into who speaks for us. It is a theory concerning those instances and semio-linguistic entities to whom we delegate our words, who mediate in our relationships with things and represent the background to our perceptions of the world. For this reason, in the place of enunciation there is never solely “the subject”—ego—but there are many other “enunciating instances”. We all know, for example, that society generates discourses, and some say that the body is itself a discourse generator. It is thus no longer a question of ‘subjects’ but rather instances of discourses, enunciating instances. The other enunciating instances can thus not be amputated from the enunciation, reduced traditionally and mistakenly to the ego. (Coquet 2016)

Inside every discourse there are semiotic entities which cannot be reduced to the “I”. For this reason, Benveniste’s theory is unsatisfactory: it is radically subject-­ centric and ignores the other enunciating instances reverberating within every 19

 See Pianigiani 1907.

46

For a Cognitive Semiotics of Subjectivity

l­ anguage act. This is a further ambiguous aspect of Benveniste’s theory which needs to be underlined and taken on board, rather than clarified and disproved. As Kleiber noted (2016: 34–5), “what characterises the third person and separates it from “I” and “you” is its ability to refer to both individual human beings and to non-human beings and objects”. This idea, already present in the ideas of the Port-Royal grammarians, was one of the principal reasons behind Benveniste’s concept of the third person as a ‘non-person’. There is an ontology, i.e. a specific distribution of humans and non-humans,20 behind Benveniste’s decision to juxtapose privately “I-you” (person) against “he” (non-person) and deny ‘he’ any person status whatsoever, despite the evident fact that, at the grammar level, “he” is obviously a lingui­ stic person. The ability to refer to ‘things’ was not the only argument put forward by Benveniste to justify the divorce between I/you and he but Benveniste considered it one of the most revealing: “Lastly it is important to be fully aware of the specific features of the ‘third person’ which is the only way in which a thing can be predicated verbally”.21 (Kleiber 2016: 36)

Indeed, Maillard (1974: 60) observed that, in Benveniste’s analysis, there was “a frequent ambiguity in the use of the term ‘person’, which sometimes refers to a purely formal grammatical category and sometimes to a substantial human presence in the enunciate”. It is obviously not a coincidence that a theory whose goal is to found the subjectivity of the self-consciousness on the linguistic capacity of “the ego saying ego” views ‘person’ as exclusively that which can be taken on by human actors (I-you) and ‘non-person’ as what can be taken on by either human actors or non-human actors (he). However, if we take the very same opposition “human” versus “non-human”, it is perfectly clear also in this regards that “he” is by no means a “non-person” (non-human), but rather a “person+non-person” (human+non-­ human). At this level, too, what is at stake is a participatory opposition involving an extensive term and not a privative opposition involving an absence of the person, as Benveniste argued. It is, indeed, precisely Benveniste’s use of the French “personne” to say “nobody”, as in the expression “Qui as-tu vu? – Personne”. In this regard, a revealing structure for our purposes can be found in the two different Greek and Latin etymologies of the word ‘person’, which both arrive to the “grammatical person” sense, but by means of totally opposite semantic genealogies. As Létoublon (1994) has shown, Greek prosôpon primarily means “what is before the eyes”, “face”, “outward appearance” (sense 1). The second meaning of the word is, on the other hand, “physical person” (sense 2), while only much later, via Etruscan influences, prosôpon began to mean “mask” (sense 3), “character” (sense 4) and “grammatical person” (sense 5). By contrast, in Latin persona primarily and originally means “theatre mask”, “character” (sense 1), and only later “human person, individual” (sense 2) and, finally, “grammatical person” (sense 3).

20 21

 See Descola 2005.  Benveniste 1966: 230.

4  Persona. The “I”, “You”, “He” Hierarchy

47

We have before us here the two souls of the theory of enunciation and subjectivity in language. On the one hand, we have (1) face to face communication with someone, equivalent to the “in person” locution (“what is present before our eyes”, “in presence”) and opposed to the representation via the mediation of a “third”. On the other hand, we have (2) the semiotic idea of constructing a signifier surface which can be used in order to lie (the mask), the sending of a messenger, a person to speak for us, a “third party mediator” whom we delegate to speak in our place. This is Benveniste’s idea of enunciation (face to face communication, “what is before the eyes”) as opposed to Latour’s notion of the same concept (delegation, “sending a messenger”). According to Latour, enunciation consists in the acts of “sending” and “mediation” which make us present via a nuncius when we are absent (see Latour 1999, 2012). In this work, which is inspired by Latour’s idea of enunciation, “person” is to be understood in the Latin sense of mask, character, and thus of “actant” rather than “actor”. And, as we saw with respect to material agency (Malafouris 2013), actants, in semiotics, are completely indifferent to “human” versus “non-human” opposition, which relates solely to actors, the concrete entities which occupy the places and positions specific to semio-linguistic entities. Semiotic actants, that, as is well known, profoundly influenced Latour’s idea of “networks of humans and non-humans” (Actor-Network Theory),22 are as entirely indifferent to the ontological opposition between humans and non-humans as they are to the oppositions between “person”, “thing”, “idea” and “concept”. Hence the profound bond between these enunciating instances and the idea of “person” which does indeed mean “character”, “mask”, but also “face” and “face to face”. Indeed, also in face to face communication we effectively wear masks and construct our characters. This leads to an important consequence. If language is something which can always be used for lying, it is possible that the function of the semio-linguistic system is to establish positions of subject capable of making a “staging” possible, distributing “masks” to the actors which occupy those positions. Thus, there is something extremely valuable and powerful in the very notion of “third person”. The third person is, in fact, a “third element”, a mediator outside the locutor-interlocutor relationship,23 just as it is also a “third” in the “human” versus “non-human” opposition. The power specific to the third person, which is an extensive term defining both the person (A) and the non-person (non-A), becomes clear if we consider Gustave Guillaume’s position (1991) in his Leçons de linguistique.

22 23

 See Latour 2005.  See Maillard 1974: 61.

48

For a Cognitive Semiotics of Subjectivity

5  The Primacy of ‘He’ Guillaume opens up an interpretation of the person correlation which is entirely alternative to that of Benveniste (see Joly 1994; Kleiber 2016). His account is another classic way of presenting the I/you/he triad, which is an alternative to thinking of it as locutor (I), interlocutor (you) and non-locutor (he). In this alternative vision, the third person is not seen as a ‘non-person’ in the sense of being “outside the locutive exchange” (Maillard 1974: 61). This other classic representation of the person correlation consists of conceiving the ‘I’ as he who speaks, or locutor, the ‘you’ as he who is spoken to, or interlocutor, and the ‘he’ as what is spoken of, or delocutive. This conception has been expressed very clearly since antiquity (see Colombat 1994), and the impersonal use of the third person is an exception to this delocutive conception. We will return to this exception because the third person is in no way the delocutive object referred to in phrases such as “it rains”. Despite this exception, when the “delocutive” idea comes on stage, everything seems to change. If the first tripartite distribution in locutor, interlocutor and non-­ locutor was, in fact, entirely internal to the level of enunciation as dialogue locution, in the “locutor-interlocutor-delocutive” triad, the three terms are no longer on the same level: The definitions of the first two persons (I and you) are effectively located at the enunciation level while the third person (he) is valid at the enunciate level. I and you are enunciation instances, and thus are attributed an enunciation role while the others are instances of the enunciate and thus are attributed a role within the enunciate. (Kleiber 2016, § 4)

The enunciate (the sentence, the product of language) is obviously a fundamental dimension in enunciation theory, since “enunciation” is precisely defined as “the very act of producing the enunciate” (in Benveniste) or “the ability of the enunciate to express the act that produced it” (in Greimas). However, when the level of the enunciate is introduced, many things change radically: If the level of the enunciate is taken on board it is clear that the opposition between he and I and you no longer works: if ‘he’ is certainly a delocutive and thus “what is spoken of”, I and you are also delocutives in the same way, namely subjects spoken of, as in saying, for example “I left on Sunday morning” or “You left on Sunday morning”, the locutor is obviously speaking of himself or his interlocutor. This is not a new observation. Apollonius Dyscolus and later, in the early Middle Ages, Priscian, highlighted that persons were not to be analysed solely in relation to the role they play in enunciation, but also in relation to the role they play within the enunciate, i.e. as objects of discourse. (Kleiber 2016, § 4)

It is worth noting the short circuit between subject and object: ‘I’, ‘you’ and ‘he’ are simultaneously subjects spoken of and objects of discourse. Both Benveniste and Guillaume were fully aware of this aspect,24 but interpreted it in diametrically different ways. For Benveniste, the fact that “he” represents what is spoken of and works at a different level (enunciate) as compared to that of “I” and “you” (enunciation) is simply a further motive for thinking of it as a non-person, privatively 24

 See, for example, Benveniste 1966: 228 and Guillaume 1991: 114.

5  The Primacy of ‘He’

49

opposed to “I” and “you”. By contrast, the fact that the delocutive is also present within “I” and “you” prompts Guillaume to postulate a primacy of “he” over the other two persons, as the third person is the only one of the three to present the de­locutive trait exclusively. Looking at things from closer to - something which has been avoided on this matter - the delocutive person is not absent from any of the three persons. Because we are always speaking of a person which, in the case of the locutive person, is the same person speaking and, in the case of the allocutive person, the same person spoken to. If I say to someone: “You behaved badly”, I’m speaking to him but in speaking to him I am speaking to him of him. What we are witnessing is the simultaneous appearance of the allocutive person and an implicitly conceived delocutive person. It is the same when I say: “I believe this”. It is me speaking but in my words it is me spoken of. Thus the delocutive person is implicitly associated with the locutive person. (Guillaume 1991, t. 10: 114).

There is a “third person” in every “I” and in every “you” to the extent that subjectivity in language is defined in its very essence precisely by this joint presence. The fact that the third person is in some way implicitly present also within “I” and “you” means that, within Guillaume’s theory, it takes primacy over the other persons and ends up becoming the most important element in the person correlation specific to the formal apparatus of enunciation. The third person, underpinning all persons, is the foundation of the enunciation system because, when we speak, we are always speaking of someone (or something). For Guillaume a third person underlies all the other linguistic persons (Joly 1994: 48).

We cannot but be on Guillaume’s side on this point. It is, here, a matter of forcefully asserting this primacy of “he” which takes us in the direction of a theory of impersonal enunciation, founded on the category which, for Benveniste, defined the non-person. Indeed, we will see that the impersonal defines the delocutive form as an event which opens up positions of subject in which something or someone is spoken of. It is extremely clear that the third person (delocutive) defines a person who expresses the object of the discourse, the “what or who is spoken of”. Or perhaps it would be better to say that the third person defines the subject of the discourse, as the subject of a discourse is precisely “what is spoken of”, “what underlies” (sub-iectum) the language lens, “the main subject of a discourse”? This short circuit between subject and object, between delocutive third person status and the subject of a discourse, is extremely stimulating in semio-linguistic theory. The third person can be simultaneously subject and object and this assigns it a decisive role for a semiotic theory of the subjectivity in language, since “subjectivity” is precisely the capacity of a subject to make itself object of its own reflection. It is thus the third person, with its “person+non-person” participatory structure, which holds subject and object together, neutralizing their opposition. The neutral position of the third person as regards the “subject-object” opposition is flanked by a corresponding neutral position as regards the “human-non-human” opposition, as “he” is both a “who” and a “what”, whereas “I” and “you” can only be a “who”. If we thus adopt Guillaume’s point of view, we see emerge at the level of all the linguistic persons that fundamental property which we identified as the subjectivity in language’s trademark. What emerges is the subject’s capacity to make itself object

50

For a Cognitive Semiotics of Subjectivity

of its own reflections, which Benveniste saw as the exclusive domain of “I”, of “ego saying ego”. The distinction between a locutive person speaking, an allocutive person spoken to and a delocutive person spoken of is, certainly, absolutely exact. It could not be otherwise given the straightforward nature of the facts observed. But this distinction, however exact, presents things in an incomplete way. The locutive person is not solely the person speaking. It is also the person who, in speaking, speaks of himself. In the same way, the allocutive person is not solely the person spoken to. It is also the person to whom he is spoken of. Only the third person is truly one, as it is solely the person spoken of. (Guillaume 1991, t10: 114)

This is what “subjectivity in language” really is: certainly not Benveniste’s “ego saying ego”, but, rather, the delocutive structure underpinning all the linguistic persons (extensive term). This “delocutive form” is to be found in the third person in what we might call its ‘pure’ form. This form defines that duplication inside language in which, even when we say “I” or speak to a “you”, this “I” and this “you” become the object (or subject) of the discourse. Guillaume shows us this with no room for misunderstanding; it is the delocutive structure of the “he” which, in language, expresses subjectivity, namely the capacity of each subject to make himself the object of his reflections and of his words.

6  Illeity For reasons which will soon be clarified, I refer to the (non) person expressing subjectivity in language as illeity (from “il” in “il pleut”). An extract from Maurice Blanchot will help clarify. So it is not enough for me to write “I am unhappy”. As long as I write nothing else, I am too close to myself, too close to my unhappiness, for this unhappiness to become really mine in the form of language: I am not yet truly unhappy. It is only from the moment I arrive at this strange substitution, “He is unhappy”, [il est malheureux] that language begins to be formed into a language that is unhappy for me, to sketch out and slowly project the world of unhappiness as it occurs in him. So, perhaps, I will feel myself implicated, and my unhappiness will be felt by this world from which it is absent. (Blanchot 1949/1995: 21–22)

The way in which, for Blanchot, pain will show itself “in a linguistic sense” must firstly be underlined. Pain will show itself in a world from which it is absent, it will make itself present to the extent that it is absent. However, for Blanchot, this personal expression of a pain that is truly mine is possible only through a mysterious substitution of “I” with “he”. Blanchot is clearly dealing here with the enunciative assumption by a subject (“so, perhaps, I will feel myself implicated”). A phenomenological manifestation of a somatic type perceived as regarding the subject (“my unhappiness will be felt”) depends on this assumption, with its “presence of an absence” form. The pheno­ menon (“unhappiness”) for Blanchot thus performs the function of an enunciative assumption and an enunciative assumption does not take place in a first-person enunciation (“je suis malheureux”), but rather in the third person (“il est

6 Illeity

51

­ alheureux”). There is a sort of collective of enunciating instances that brings forth m the subject as its own product. As Coquet argued, body is always a discourse generator (the pain of my unhappiness), the other is always a discourse generator (the unhappiness of the “he”), and language, with its social and intersubjective structure, is a discourse generator, a very special kind of phenomenological light capable of “sketching out and slowly project the world of unhappiness”. It is only at this point that the subject feels to be implicated, achieves self-consciousness, knowing that he knows and feeling that he feels in the first person. Indeed for Blanchot, as long as I am solely saying “I am unhappy” I am too close to myself for my unhappiness truly to be mine. The unhappiness is like a hand in front of my eyes, which is absolutely the nearest thing to me but nonetheless cannot be seen and, in fact, stops me from seeing. The same is true of an unhappiness founded on “I-here-now”, on “my unhappiness” enunciated in the first person. According to this idea, the “I” is not only not the origin of subjectivity—as Benveniste argued it was—but neither is it the origin of the semio-linguistic category of person, as Guillaume suspected. There is a semiotic principle which echoes the theory of semio-linguistic systems behind this idea of Blanchot’s. An element’s identity consists in the network of relationships it is a part of and it is for this reason that unhappiness becomes mine solely and exclusively at such time as it becomes part of a relationship which transcends it. Thus, the unhappiness is mine only when it comes up against an “otherness” which defines its identity, exactly like individuality is a real individuality when it becomes part of a relationship which defines it in its most profound and subjective forms. As Latour said (1999: 66), when we are working on enunciation, we have to start “from passage and relationship, not accepting as starting point any being which did not come out of this relationship”. The “I” is no exception, and for this reason, establishing its identity requires that we establish the network of relations that it forms a part of, the “mini scene”—to use Lucien Tesnière’s expression—which assigns its role. It is semiotics’ very processes. We have first and formost the “mini scene” of enunciation, rather than the object and the subject, and in this “mini scene” the places and the positions are more important than the elements that occupy them.25 Blanchot thus helps us to take on board this “scene” of enunciation when he tells us that what is personal (the pain which belongs to the self) depends on a non-­ person “third person” pain (“il”) which makes the I-you possible. As Deleuze noted (1980): In this passage there is, I believe, something which Blanchot can teach us, not solely in literary but also in linguistic terms as, at least as far as I know, he is the only scholar arguing for this type of approach in linguistics. That is, is it precisely in Blanchot that the elements of a critique of the theory of embreyeus exists, a critique of the linguistic theory of embreyeurs. (Deleuze 1980)

What does “il est malhereux” actually mean? What is the deeper meaning of this mysterious substitution by Blanchot? 25

 See Deleuze 1969.

52

For a Cognitive Semiotics of Subjectivity

As we have seen, in the third person, French, like Italian and many other languages, has both the third person form and the impersonal, but French, like English for example, expresses it by placing the personal pronoun ‘il’ before an impersonal phrase, in a “personality of impersonality” which is precisely pre-logical and participatory26: “il pleut”, “it rains”, “it happens”. So Blanchot is playing precisely on the vagueness and indecipherability which this specifically participatory “personality of impersonality” brings with it: “il est malheurex”, that is “he is unhappy” but also “it is unhappy”. This is what Lévinas (1974) called illeity: “il”, which is both simultaneously “il pleut” and “il se promene,” is “the third person pronoun in the impersonal form of the verb: not an author of the action who we do not know well, but the character of this same action which, in some way, has no author” (Lévinas 1974: 50). This is the profound meaning of Blanchot’s substitution which starts from the “I-you” structure of the person (je suis malheureux), to which he opposes a personal pronoun “he” (il) on which “I-you” depends (unhappiness is mine only and exclusively when it is detached from an “I” towards a “he”), but this “he” is not the form of a person that can be called “third”, but rather the more profound form of the impersonal. “il est malhereux”, “it is unhappy”. What is this third person ‘il’ which no longer belongs to any person? I would say that, very far from being an ‘il’ pertaining to what is anonymous, it is actually an ‘il’ pertaining to pure singularity. This ‘il’ truly pertains to pure singularity. That is, singularity detached from any person. […] When I say “it happens” or when I say “it is raining”, here, too, we are faced with two formulas which tend to be extreme. It rains (Il pleut). What is this “it” (il)? What does this sign designate? It no longer designates a person. What does it designate then? It designates an event. There is thus the “it” (il) of an event. This ‘il’ is there in the event of the “here it is” formula (“il y a”). “Il y a” or the “il” in “il pleut” (“it rains”) refers to an event. An event is not a person. […] Quite the contrary, an event is extremely singular and identifiable. So, we need to say that identifying an event is not of the same type as identifying a person. (Deleuze 1980)

There is an unhappiness event which happens according to the illeity process, according to the impersonal “il”, and distributes subject positions in which my place—my unhappiness—depends on the place of what does not relate to me, that unhappiness which does not affect me. This is what illeity is: the impersonal event which opens up subject positions. And it is this impersonality of an event capable of making the subject and the person possible (illeity), which is to be placed centre stage in the semiotic theory of enunciation and subjectivity in language. As Esposito noted (2007: 131), “in such cases - expressions such as úei, tonat, it rains  – the process is understood as something objective for which no agent is responsible, but rather an event with no subject which is constitutive of the subject”. That the event category can include the semio-linguistic category of subject is an accepted idea by Benveniste himself when he writes that “volat avis does not mean “the bird is flying” but rather “flying, the bird”. The form volat is itself sufficient and whilst it is not personal, it includes the grammatical notion of subject” (Benveniste 1966/1971: 175). This leads on to the network of relationships defined 26

 It is, for example, not like this in Italian.

6 Illeity

53

by Blanchot, which moves from the first and second persons (I-you) to the third person (il est malheureux), on which the first and second persons depend. And then it moves from the third person “il” to a more profound “il” which no longer designates any third person, but rather the impersonal form of a singular event, or a point at which something happens (“il pleut”, “il y a”), which is illeity. There are two great opposing traditions of thinking on propositions and the place of the subject within the phrase and the language. The first one, the Aristotelian tradition, is based on the form “S is P” where S is the subject and P the predicate. In this tradition, the subject is the very heart of the proposition and it is on this subject, expressed by name, that certain properties expressed by adjectives, for example, are predicated. From logic to linguistics, from analytical philosophy to philosophical hermeneutics, this tradition has certainly been a winning one and predominates in Western thought to the extent that Nietzsche advanced the suspicion that many of our philosophical problems depend precisely on the fact that our languages favour a concept of language based on the centrality of subject in the “subject-predicate” structure.27 However, since antiquity, an alternative tradition—profoundly different and ­revolutionary—has never ceased moving parallel to this more pervasive tradition, rejecting its very presuppositions. Indeed, the Stoics28 did not believe that the subject was the centre of the proposition. It is not accidental that the Stoics were neither Greek nor native Greek speakers and thus foreigners in the language they wrote in, “guests” in their own language.29 The Stoics believed that the centre of the proposition was the event expressed by the verb and that this event was “incorporeal” (asomaton). Indeed, the event was certainly embodied in actors (“I”), spaces (“here”) and times (“now”), but could in no way be reduced to these as the event represented the condition of possibility of these actors, spaces and times (in language). It is not the statement that “the tree is green” (S is P) but rather the “greening” of the tree as the spring event, which opens up positions for the subjects and the persons which come to occupy them. This Stoic tradition was taken up at different times by two giants of contemporary semiotics and linguistics who did not know each other and had probably not read each other’s work, but rather incredibly made the same points using the same metaphors and the same identical examples (see Paolucci 2018). Charles Sanders Peirce and Lucien Tesnière reformulated the ideas of the Stoics within a structural analysis of language which they called, respectively, Logic of Relatives and Structural Syntax. Both Peirce and Tesnière began with a radical critique of the Aristotelian

 See Gentili 2017.  On these arguments in philosophy, see Brehier 1928, Deleuze 1969, Manetti 1987 and Pohlenz 1948. 29  Paolo Leonardi (personal communication) reminded me that there is a tradition that half goes back to Russell, and, later, Quine, Sloat, Burge, Fara and others that considers names to be predicates. This tradition does not think like the Stoics do, but still does not assume the centrality of the subject. 27 28

54

For a Cognitive Semiotics of Subjectivity

Figs. 1 and 2 The proposition “John gives John to John” corresponds in its constitution, as Figs. 1 and 2 show, precisely to ammonia” (CP 3.469) F

predication model30 which placed at the centre of the proposition the “nominative subject”, which Peirce defined “as an unfortunate habit of our race or, maybe as a mysterious instinct, just like the wasps take care of their own eggs” (MS 200: 59). The opposition of subject and predicate makes impossible to distinguish the structural balance of the sentence, because it brings the analyst to isolate one of the actants as a subject, excluding the others and pushing them inordinately in the predicate with the verb and the circumstants. That means to assign to one of the elements of the sentence too much importance, while no linguistic element justifies that. [...] The opposition of the subject and the predicate introduces an asymmetry factor, because each actant is on a different level according to its being or not the subject. [...] On the contrary, in the sentence Alfred gives the book to Carl, Carl and also the book, while not directly acting, are anyway actants in the same way as Alfred. (Tesnière 1959: 76, 73)

While Peirce was studying ergative languages such as Basque, which have a very different way of expressing agency when compared to transitive languages (see Violi 2007), he referred to this Aristotelian structure which stops us from understanding the structural equilibrium of a phrase as “Arian syntax” (MS 595: 27). He also added that the Logic of Relatives “had no reason to consider, for example, the fact of giving as more pertinent to the donor than to the receiver or the gift”. In a triadic fact, say, for example A gives B to C, we make no distinction in the ordinary logic of relations between the subject nominative, the direct object, and the indirect object. [...] If we call A, B, C, D four subjects of the proposition and “_ sells _ to _ for the price of _” a predicate, we represent the logical relation, well enough, but we abandon the Aryan syntax (MS 595: 27).

Against this Aristotelian model grounded on the centrality of the subject, both Peirce and Tesnière proposed an evenemential model which placed the event expressed by the verb at the centre of the proposition and the language. Every event brought up a specific “verbal valence” and opened up a variable number of subject positions which could be variously filled. Just as nineteenth century chemists learned to classify elements according to valence or the number of open bonding sites on the atom, Peirce and Tesnière classified relatives and actants, the relational  It is of even greater interest in Peirce who, in his introductory essay, On a New List of Categories, and then again in the anti-Cartesian essays in which he founded semiotics, was inspired precisely by predication and the Aristotelian model incarnated in the S is P form (“the stove is black”). This model was then completely abandoned with the Logic of Relatives watershed, which had revolutionary consequences which have perhaps not yet been fully understood, including for Peirce’s semiotics theory. 30

6 Illeity

55

elements of a proposition, according to the number of open places for insertion of an indexical sign. A chemical atom is quite like a relative in having a definite number of loose ends or “unsaturated bonds”, corresponding to the blanks of the relative. In a chemical molecule, each loose end of one atom is joined to a loose end, which it is assumed must belong to some other atom, [...] The proposition “John gives John to John” corresponds in its constitution, as Figs. 1 and 2 show, precisely to ammonia. Thus, it becomes plain that every node of the bonds is equivalent to a relative; and the doctrine of valence is established for us in logic (CP 3.469-3.471).4

The relative terms are thus nothing more than pure positions defined by their verbal valence and Peirce frequently used a notation which identified them as blank spaces disseminated by the structure of the phrase, such as the gift example: “― gives ― to ―” (see CP 3.340, 3.471, MS 595).31 In this way, Logic of Relatives is no more than an original form of topology of proposition,32 in which the classification of the elements is functional to the number of places available for the addition of a new term capable of combining with the verb considered. In Tesnière these terms went by the name of actants,33 in Peirce relative terms (or correlates).34 In this passage from subject/nominative logic (Aristotelian) to logic of relatives (Stoic), a revolution took place. This revolution is somehow parallel to that of modern subjectivity and the way it is expressed in language. As is well known, for Aristotle there is a correspondence between subject and substance: on one hand, the substance is the foundation on which the accidental qualities lean (metaphysical subject); on the other hand, in language, its mirror image is precisely the grammatical subject of which the various predicates are said (logical subject). It is not accidental that Aristotle affirms that “in the first place the substance would seem to be the subject of everything” (Metaphysics, VII, 3). By contrast, for the Logic of Relatives the subject is an empty position opened by the event and completely dependent on it: the subject is subjected to the evenemential structure of the relationship, from which its position is determined. Not by chance the word “subject”, which derives from the Latin subiectus (past participle of subicere, made up of sub, “under” and iacere, “throw”, thus “to subdue, to subject”) literally means “he who is under”, “he who is below” and, that is, “subjected”,

 Frege has a similar idea and he also uses the chemical metaphor. However, Frege refers to concepts and not to language. 32  See Burch 1991. 33  “Protoactants” in Greimas 1983, “positional actants” in Fontanille 1998. 34  Peircean terminology in the Logic of Relatives is much differentiated and often not coherent (cp. Fabbrichesi Leo 1992: 136–137). For instance, Peirce distinguishes between relative, relation, relationship, and relate (CP 3.466). Our argumentation will be referring to the extraordinary exposition from 1897, in which a relative is a rhema based on a determined form of relation of the form “_is a lover of_”. In this case, the rhema (or relative) is dyadic (form of relation) and the positions that are constructed by its valence are the relative terms or correlates of the relative in its form of relation. In this case, the relative would be equivalent to the small Tesnierian drama, with its verbal valence, and the relative terms, or correlates, would be equivalent to the actants (we do not distinguish here between actants and circumstants in Tesniere, cf. Petitot 1985: Chap. 4). Cp. MS 544 and CP 3.456.552. 31

56

For a Cognitive Semiotics of Subjectivity

“dependent”. Clarifying the place of subjectivity in language thus requires recovering its subjection to the event and the relationship which is characteristic of the subject in both its linguistic and metaphysical form (“sub-ject” as “sub-stance”). Saying that a sentence like “Alfred speaks” involves only two elements means analyzing it in a superficial way, a purely morphological one, and neglecting the essential element, which is the syntactic bind. In chemistry, we have an analogous phenomenon: the combination of chlorine (Cl) and of sodium (Na) generates a compound, cooking salt or sodium chloride (NaCl), which is a completely different substance and has completely different features from both chlorine and sodium (Tesnière 1959: 30).

For “Stoic” approaches like Peirce’s and Tesnière’s, substance is always the effect of a combination of relationships and not something which itself possesses properties which can be predicated for it when it takes on the position of subject of the phrase within a proposition.35 Within this perspective, the subject has positions only, which are determined by the event expressed by the verb they depend on. The person is a mask, a character occupying a role and position in the proposition’s “scene”. At multiple points, Peirce and Tesnière make a whole series of examples of verbs with various valances expressing events which open up a variable number of subject positions. For instance, “it rains” is 0 valence (“it rains” is considered a process which takes place on its own, without the involvement of people or things”); “A is a man”36 has a valence of 1 (one subject position), “A kills B” or “A hits B” has a valence of 2 (two subject positions) and “A gives B to C” has a valence of 3 (three subject positions). Tesnière calls these “zero actants predicates”, “single actant predicates” (A falls), “two actants predicates” (“A hits B”) and “three actants predicates” (“A gives B to C”), in accordance with the verb’s valence, and speaks of the phrase as a “predicative scene”.37 First and foremost, language expresses events which open up positions of subject. Nouns and pronouns fill up the empty positions opened up by verbal valence and build the enunciate as a “saturated whole”, that the event expressed by the verb “knots up, so to speak, into a single bundle” (Tesnière 1959: 31) Each event brings up a specific verbal valence and opens up a variable number of positions of subject which can be filled in various ways. “I” is thus nothing more than one of the (pro) nouns which can occupy one of the subject positions opened up by the event expressed by the verb. It is because an impersonal unhappiness event occurs that this unhappiness can truly become “mine, in a linguistic way” (“Je suis malheureux”).  This was, on the other hand, exactly Peirce’s theory in On a New List of Categories (CP 1.545–59). 36  It is very clear here how predication is retraced to a specific case in Logic of Relatives, that ­relating to monadic predicates. Aristotelian theory of language and the proposition founded on predication are thus revealed to be a specific case of Stoic theory of the proposition founded on the event expressed by the verb. This latter takes account of the former as a specific case, but the contrary is not true. 37  See Tesnière 1959: 79–83. 35

6 Illeity

57

For this reason, several of Benveniste’s famous statements need to be turned on their heads. It is no longer the case that “language is possible only because each speaker sets himself up as a subject by referring to himself as I in his discourse” (Benveniste 1966: 225), but rather that ‘the subject (the “I”) is possible only to the extent that language opens up positions of subject which a personal instance can variously occupy’. It is no longer that “because of this, I posits another person, the one who, being, as he is, completely exterior to “me”, becomes my echo to whom I say you and who says you to me” (Benveniste 1966/1971: 225), since the third person as an extensive term for “person+non-person” defines an impersonal event within which it is possible to say “I” and “you” (illeity). What is the relationship between event and person? Is an injury an event? Yes, if I’m injured. It is the expression of something which is happening to me or happened to me. Good. How is an injury identified? Is it identified because it happens to a person? Well, would I really call person he who the injury happens to? […] An event only exists to the extent that it is effectuated. There are no not-effectuated events. On this we agree: no “Platonic idea of injury” exists. However, at the same time, we need to say that within an event a part which goes further, which goes beyond its very effectuation always exists. […] Joe Busquet used to say: “My injury existed before me. I was born to embody it”. That is, yes, it took place in me, but it contains something for which it is no longer ‘my’ injury. It is ‘its’ injury (C’est “il” blessure). (Deleuze 1980)

This is Blanchot’s “il est malheureux”, or illeity. According to Blanchot, the event is prior to the different subjects who come to effect it, occupying the actant positions opened up by the event expressed by the verb. According to Busquet’s fascinating formula, “my injury existed before me. I was born to embody it”. Illeity’s evenemential structure distributes the position of subject, the masks that are faces for the linguistic persons who will occupy them in the mini-scene. We find here, at a semio-linguistic level, the same internal hierarchy of the persons at work at the cognitive level. The illeity of the “he” (third person) is an impersonal event which opens up the intersubjectivity of the “I-you” (second person) which, in turn, makes possible the position of the subject (first person). Indeed, a newly born child does not have the capacity to make himself the object of his reflections (subject) and, in order to develop it, he must undertake a long and winding road of interactions with others. This interaction with others is punctuated by various enunciating instances as the society, the bodies, the norms and the habits of his caregivers define the form of the events within which the child can encounter the others in interaction. From the starting point of these illeities, the construction of a subjectivity involves a long and tortuous journey. Human children (and many other social animals) exhibit a special responsiveness to the intentional attitudes of others early on (3 months), discriminating human actors from mechanical agents and goal-related actions from mechanical movements. Indeed, during primary intersubjectivity, human infants and some other social animals (Myowa-Yamakoshi et al. 2004) are already able to see bodily movements as goal-directed, meaningful intentional movements and to perceive other persons as agents, discriminating a biological movement from other kinds of movements (Simion et al. 2011). Embodied interactions during primary intersubjectivity

58

For a Cognitive Semiotics of Subjectivity

(between 1 and 9 months of age) constitute our primary access for understanding the others and the skills we exhibit during this crucial period ground our more sophisticated social cognition abilities developed later on in ontogenesis. These skills include proto-mimesis, emotional interchange, imitation and the parsing of perceived intentions (Gallagher 2005, 2006, 2009). During primary intersubjectivity, children are already able to interpret the actions and the expressive movements of the others through meaning (Zlatev 2008). Around the age of 1 year, but also earlier (9 months), together with the development of a semiotic competence (Vygotskij 1986; Eco 1975), the infant goes beyond person-to-person immediacy and enters contexts of shared attention, where he begins to tie actions to pragmatic contexts (Reddy 2008). The capacity for joint attention is a major addition to the repertoire of social skills exhibited during primary intersubjectivity. Not only are children now able to re-enact to completion goal-directed behavior that someone else fails to complete, but, in those shared situations, an object or an event can become a focus between people. For instance, they can focus on an object of value that can be communicated about (Hobson 2002). “The child, on seeing an adult who tries to manipulate a toy and who appears frustrated about being unable to do so, quite readily picks up the toy and shows the adult how to do it” (Gallagher and Hutto 2008: 24). When a “third” enters the interaction, the child modulates his sense-making capacities, making objects the aim of his own desires. Projecting himself into possible worlds which are not identical with the one he lives it, imagining himself as a character alongside other characters, the child learns that he can lie and use signs to manipulate others within shared practices (think, for example, of pretend crying, which takes place long before the advent of language). In semiotics, débrayage (“disengagement”) is the capacity that a subjective instance has to break away from its own presence, in order to think of itself as a character among other characters (“not-I”), inhabiting another place and another time from those he really lives (“not-­ here”, “not-now”). When I write of myself, as I am now, I am projecting into the enunciate a simulacrum (not-I), which inhabits other spaces and other times from those I am currently living as I write these lines on the computer at my desk. However, subjectivity in language is not a matter of this dialectic between “I” and “not-I”, but rather a matter of the participatory structure which opposes an “I” to an “I + not-I”, a “now” to a “now+not-now”, a “here” to a “here+not-here”. For this reason, subjectivity is the capacity a subject (‘I’) has to make himself object (‘not-­ I’) of his reflections. Subjectivity consists in an ‘I + not-I’ expressing what Hjelmslev called a “pre-logical mind” which was the “the general structure of all linguistic correlations”. This capacity is not born with the linguistic “I” and neither does it require language acquisition. It consists only of the acquisition of that sense-making capacity which is specific to semiotic systems and connected to strategic thought, the ability to lie and effective action. This is the origin of our semiotic and enactive conception of subjectivity. Our ability to duplicate ourselves, our capacity to make ourselves object of our own reflections that we call “subject” has brought us enormous cognitive benefits. This ability derives from effective action and the need to deceive the others, sending a

References

59

messenger for ourselves capable of make believing what we are not, what we do not want.

References Albano Leoni, F. (2009). Dei suoni e dei sensi. Bologna: Il Mulino. Austin, J. L. (1975). How to do thing with words (2nd ed.). Harvard: Harvard University Press. Benveniste, É. (1966). Problèmes de linguistique générale. Paris: Gallimard. English Edition: Benveniste, Émile. 1971. Problems in General Linguistic, Transl. Maria Elizabeth Meek, Coral Gables Fl.: Miami University Press. Bianco, G. (2008). Il professor Benveniste contro gli analogisti. Sulla differenza tra gioco e sacro. Aut Aut, 337(January/March), 75–92. Blanchot, M. (1949). La part du feu. Paris: Gallimard. English Edition: Blanchot, Maurice. 1995. Work of Fire. Cambridge (MA): Cambridge University Press. Brehier, É. (1928). La théorie des incorporels dans l’ancien stoïcisme. Paris: Vrin. Burch, R. W. (1991). A Peircean reduction thesis: Foundations of topological logic. Lubbock, TX: Tech University Press. Chemero, A. (2008). Self-organization, Writ Large. Ecological Psychology, 20(3), 257–269. Colombat, B. (1994). Remarques sur le développement de la notion de personne dans l’histoire de la linguistique. La Personne, 3, 15–27. Coquet, J.-C. (2007). Physis et Logos. Une phénoménologie du langage. Paris: Presses Universitaires de Vincennes. Coquet, J.-C. (2016). L’énonciation, fondement de la phénoménologie du langage. In M. Colas-­ Blaise, L. Perrin, & G. M. Tore (Eds.), L’énonciation aujourd’hui. Un concept clé de sciences du langage (pp. 295–302). Limoges: Lambert-Lucas. Deleuze, G. (1969). Logique du sens. Paris: Les éditions de Minuit. Deleuze, G. (1980). Dernier cours à Vincennes: 03.06.1980. La voix de Gilles Deleuze en ligne. Retrieved from http://www2.univ-paris8.fr/deleuze/article.php3?id_article=214 and http:// www2.univ-paris8.fr/deleuze/article.php3?id_article=215. Descola, P. (2005). Par-delà nature et culture. Paris: Gallimard. Detienne, M., & Vernant, J.-P. (1974). Les ruses de l’intelligence. La metis des Grecs. Paris: Flammarion. Di Paolo, E., De Jaegher, H., & Cuffari, E. (2019). Linguistic bodies. The continuity between life and language. Cambridge, MA: MIT Press. Eco, U. (1975). Trattato di semiotica generale. Milan: Bompiani. English edition: 1976. A Theory of Semiotics. Bloomington: Indiana University Press. Esposito, E. (2007). Sul ri-uso. Pratiche del testo e teoria della letteratura. Milano: Franco Angeli. Fabbrichesi Leo, R. (1992). Il concetto di relazione in Peirce. Milan: Jaca Book. Fontanille, J. (1998). Sémiotique du discours. Limoges: PULIM.  English Edition: Fontanille, Jacques. 2006. The Semiotics of Discourse. Trans. Bostic, Heidi. New  York: Peter Lang Publishing. Gallagher, S. (2005). How body shapes the mind. Oxford: Oxford University Press. Gallagher, S. (2006). The narrative alternative to theory of mind. In R.  Menary (Ed.), Radical enactivism: Intentionality, phenomenology and narrative: Focus on the philosophy of Daniel D. Hutto (pp. 223–229). Amsterdam: John Benjamins. Gallagher, S. (2009). Philosophical antecedents to situated cognition. In P. Robbins & M. Aydede (Eds.), The Cambridge handbook of situated cognition (pp.  35–51). New  York: Cambridge University Press.

60

For a Cognitive Semiotics of Subjectivity

Gallagher, S., & Hutto, D.  D. (2008). Understanding others through primary interaction and narrative practice. In J.  Zlatev, T.  Racine, C.  Sinha, & E.  Itkonen (Eds.), The shared mind: Perspectives on intersubjectivity (pp. 17–38). Amsterdam: John Benjamins. Gentili, C. (2017). Introduzione a Nietzsche. Bologna: Il Mulino. Greimas, A. J. (1983). Du sens II. Essais sémiotiques. Paris: Seuil. Greimas, A. J., & Courtes, J. (1979). Sémiotique. Dictionnaire raisonné de la théorie du langage. Paris: Hachette. English Edition: Greimas Algirdas J. and Courtes, Jospeh. 1982. Semiotics and Language: An Analytical Dictionary. Bloomington: Indiana University Press. Guillaume, G. (1991). Leçons de linguistique 1943–1944. Québec-Lille: Presses Universitaires de Lille. Hobson, P. (2002). The cradle of thought. London: McMillan. Jakobson, R. (1963). Essais de linguistique générale. Paris: Les Éditions de Minuit. Jaspers, K. T. (1932). Philosophie. Chicago: University of Chicago Press. English edition: Jaspers, Karl. 1969–1971. Philosophy. Trans. E. B. Ashton 1969–1971. Jaynes, J. (1976). The origin of consciousness in the breakdown of the bicameral mind. Boston, New York: Houghton Mifflin/Mariner Books. Joly, A. (1994). Eléments pour une théorie générale de la personne. Faits de langue, 3, 45–54. Kleiber, G. (2016). Énonciation et personne ou Quelques moments de la vie d’un couple. In M. Colas-Blaise, L. Perrin, & G. M. Tore (Eds.), L’énonciation aujourd’hui. Un concept clé de sciences du langage (pp. 33–50). Limoges: Lambert-Lucas. Latour, B. (1999). Piccola filosofia dell’enunciazione. In Eloquio del senso. Dialoghi semiotici per Paolo Fabbri, eds. Basso, Pierluigi. e Corrain, Lucia, Costa & Nolan, Ancona-Milano: 71–94; now in 2001. In F. Paolo & M. Gianfranco (Eds.), Semiotica in nuce II (pp. 64–77). Rome: Meltemi. Latour, B. (2005). Reassembling the social. An introduction to Actor-Network-Theory. Oxford: Oxford University Press. Latour, B. (2012). Enquête sur les modes d’existence. Une anthropologie des Modernes. Paris: La Découverte. English edition: Latour, Bruno. 2013. An Inquiry into the Modes of Existence. An Anthropology of the Moderns. Trans. Porter, Catherine. Cambridge, MA: Cambridge University Press. Létoublon, F. (1994). La personne et ses masques: remarques sur le développement de la notion de personne et sur son étymologie dans l’histoire de la langue grecque. Faits de langue, 3, 7–14. Lévinas, E. (1974). Autrement qu'être ou au-delà de l'essence. Paris: Le livre de poche. English edition: Lévinas, Emmanuel. 1991. Otherwise than Being, or Beyond Essence. Trad. Lingis, Alphonso. Dordrecht: Springer. Maillard, M. (1974). Essai de typologie des substituts diasporiques. Langue Française, 21, 55–71. Malafouris, L. (2013). How things shape the mind. A theory of material engagement. Cambridge, MA: MIT Press. Manetti, G. (1987). Le teorie del segno nell’antichità classica. Milan: Bompiani. Metz, C. (1991). L’ènonciation impersonelle ou le site du film. Paris: Klincksieck. English edition: Metz, Christian. 2015. Impersonal enunciation, or the place of film. Trans. Deane, Cormac. New York: Columbia University Press. Migliore, T. (2017). L’enunciazione in Louis Hjelmslev. In A. Zinna & L. Cigana (Eds.), Louis Hjelmslev (1899–1965). Le forme del linguaggio e del pensiero (pp.  123–147). Toulouse: Éditions CAMS/O. Myowa-Yamakoshi, M., Tomonaga, M., Tanaka, M., & Matsuzawa, T. (2004). Imitation in neonatal chimpanzees (Pan troglodytes). Developmental Science, 7(4), 437–442. Paolucci, C. (2010). Strutturalismo e interpretazione. Ambizioni per una semiotica minore. Milan: Bompiani. Paolucci, C. (2017). System and structure. Semiotics as encyclopedic theory of complexity. In F. La Mantia, I. Licata, & P. Perconti (Eds.), Language in complexity. The emerging meaning (pp. 83–98). Dordrecht: Springer.

References

61

Paolucci, C. (2018). Logic of relatives and semiotics in Peirce. From the “subject-predicate” inferential structure to the Synechistic topology of interpretation. In M. Švantner & V. Gvoždiak (Eds.), How to make our signs clear (pp. 36–56). Boston: Brill. Paolucci, C. (2020). Persona. Soggettività nel linguaggio e semiotica dell’enunciazione. Milano: Bompiani. French translation 2020. Liège: Puliège. Parret, H. (1995). PrÉhistoire, structure et actualitÉ de la thÉorie hjelmslevienne des cas. Nouveaux Actes SÉmiotiques, 38, 1–27. Petitot, J. (1985). Morphogenèse du sens. Paris: PUF.  English edition: Petitot, Jean. 2004. Morphogenesis of meaning. Trans. Manjali, Franson. Bern: Peter Lang. Petitot, J. (1992). Physique du sens. De la théorie des singularités aux structures sémio-narratives. Paris: Éditions du CNRS. Petitot, J. (2011). Cognitive morphodynamics. Dynamical morphological models of constituency in perception and syntax. Bern: Peter Lang. Pianigiani, O. (1907). Vocabolario etimologico della lingua italiana. Milan: Sonzogno. Piazza, F. (2019). La parola e la spada. Violenza e linguaggio attraverso l’Iliade. Bologna: Il Mulino. Pohlenz, M. (1948). Die Stoa. Geschichte einer geistigen Bewegung (Vol. 1 and 2). Gottingen: Vandenhoeck & Ruprecht. Reddy, V. (2008). How infants know minds. Cambridge, MA: Harvard University Press. Sarti, A., Citti, G., & Piotrowski, D. (2019). Differential heterogenesis and the emergence of the semiotic function. Semiotica, 1–34. In press. Saussure, F. d. (1916). Cours de linguistique générale. Paris: Payot. Course in General Linguistics. Eds. Charles Bally & Albert Sechehaye. Trans. Roy Harris. La Salle, Illinois: Open Court. 1983. Simion, F., Di Giorgio, E., Leo, I., & Bardi, L. (2011). The processing of social stimuli in early infancy: From faces to biological motion perception. Progress in Brain Research, 189, 173– 193. https://doi.org/10.1016/B978-0-444-53884-0.00024-5. Snell, B. (1948). Die Entdeckung des Geistes. Studien zur Entstehung des europaeischen Denken bei den Grieschen (2nd ed.). Hamburg: Classen Verlag. English edition: Snell, Bruno. 1953. The discovery of the mind. The Greek origins of European Thought. Trans. Rosenmeyer, T. G. Cambridge, MA: Harvard University Press. Tesnière, L. V. (1959). Éléments de syntaxe structurale. Revised and corrected edition 1966. Paris: Éditions Klincksieck. English edition: Tesnière, Lucien V. 2015. Elements of structural syntax. Amsterdam: John Benjamins. Trevarthen, C., & Hubley, P. (1978). Secondary intersubjectivity: Confidence, confiding, and acts of meaning in the first year. In A. Lock (Ed.), Action, gesture and symbol: The emergence of language (pp. 183–122). New York, San Francisco: Academic. Varela, F. J. (1978). On Being Autonomous: The Lessons of Natural History for Systems Theory. In G.  J. Klir (Ed.), Applied General Systems Research. Vol. 5: NATO Conference Series (pp. 77–84). Boston, MA: Springer. Violi, P. (2007). Lo spazio del soggetto nell’enciclopedia. In C. Paolucci (Ed.), Studi di semiotica interpretativa (pp. 177–202). Milan: Bompiani. Volli, U. (1994). La cicatrice di Odisseo. Il Piccolo Hans, 79–80(Autunno-Inverno), 162–194. Vygotskij, L. S. (1986). Thought and language. Cambridge, MA: MIT Press. Zlatev, J. (2008). The co-evolution of intersubjectivity and bodily mimesis. In J. Zlatev, T. Racine, C. Sinha, & E. Itkonen (Eds.), The shared mind: Perspectives on Intersubjectivity (pp. 215– 244). Amsterdam: John Benjamins.

The Semiotic Mind. Beliefs, Habits and Extended Cognition

The semiotics tradition has always been radically anti-mentalist.1 Daddesio (1995) showed how hard semiotics tried to avoid a mentalistic language and any kind of psychologism. Peirce himself, who was the real founder of a cognitive semiotics, wrote that any reference to a mind in his semiotics was “a sop to Cerberus”, because he despaired “of making my own broader conception understood” (LW: 81). On this topic, the semiotic tradition followed a noble legacy in the history of philosophy, one whose founders are Kant and Husserl. According to this conception, thought is not related to a psychology of the mind, but rather to the logical operations which represent the conditions of possibility of the latter. The ordinary logicians talk of the acts of the mind, concepts, judgements, acts of concluding […]. All these ideas of the mind are, however, representations, or signs. We must begin by getting diagrammatic notions of signs from which we skip away, at first, all reference to the mind, and after we have made those ideas just as distinct as our notion of a prime number or an oval line, we may then consider, if need be, what are the peculiar characteristic of a mental sign. […] But there is nothing to compel the object of such a formal definition to have the peculiar feeling of consciousness. That peculiar feeling has nothing to do with the logicality of reasoning, however, and it is far better to leave it out of account. (Peirce, NEM IV: 54)

We will come back on this very important identification of representations with signs, since Peirce is saying that all representations are signs, but not that all signs are representations. The anti-mentalistic attitude is evident here and, in an interview with Umberto Eco from 2006,2 while discussing the resurgence of semiotics in the cognitive sciences that he himself explored in Kant and the Platypus, he explicitly said that “according to semiotics, mind was a bad word”. The semiotic tradition was in some sense correct. If the mind is an inner layer of representations “in the head”,

 It has also been anti-cognitive if we take into account the generative tradition.  Youtube. 2006 (16th July), Umberto Eco, Semiotica: origini, definizione, sguardo sul presente. Interview in Monte Cerignone.) 1 2

© Springer Nature Switzerland AG 2021 C. Paolucci, Cognitive Semiotics, Perspectives in Pragmatics, Philosophy & Psychology 24, https://doi.org/10.1007/978-3-030-42986-7_3

63

64

The Semiotic Mind. Beliefs, Habits and Extended Cognition

thought in language-like format inspired by the idea of “content” as “truth condition,” then semio-linguistic systems have nothing to do with that. However, if mind is extended and enacted, if representations are not always the object of cognitive science anymore, and if content is not thought of in terms of truth-conditions, there is no need for this “anti-mentalistic” attitude anymore. Nor is there need for any suspicion towards the cognitive sciences, since the cognitive sciences—and 4e cognition in particular—are fields in which many semiotic problems are posed, even if they are not usually labeled as semiotic problems. Furthermore, a semiotic mind is something that can usefully be thought of, as Charles Sanders Peirce did with his definition of the mind as an “external sign”. Actually, as it involves a definition of the mind which tries to bring it back to its semiotic underlayer and explain it by reference to this latter (and not viceversa), the purpose of the semiotic tradition is far more sophisticated than a mere anti-mentalist—or anti-­ cognitive—position. The idea, consistent with the modern arguments of Material Engagement Theory (MET), was to show how the mind was constructed by semiotic operations. Signs, texts and languages were not, in Peirce’s theory, the external expression of a preformed mind localized inside the brain and the body, but were rather thought of instead as something that shapes the mind. Therefore, the aim of semiotics was to show how signs shape the mind. Hence, Peirce started with logical operations of a non-psychological kind in order to show their semiotic structure, since, according to him, logic was a part of semiotics3 (Paolucci 2010; Bellucci 2017). For instance, abduction was the inference of a minor premise in a “Barbara” syllogism and, according to Peirce, a logical operation of substitution in which the conclusion stands for the premises was a semiotic operation. This is why, in the second of his famous “anti-Cartesian essays”, Peirce highlighted the way sensation, emotion, attention and perception were all semiotic operations. This is why a semiotic mind is crucial. It is no longer necessary to start from mental and cognitive operations in order to arrive at signs and languages with which to express them. On the contrary, we have to start from signs, texts and languages, in order to understand how semiotic systems shape the mind.

1  Anti-Cartesian Semiotics It is not by chance that the three essays formulated by Peirce between 1867 and 1868, which are at the foundation of semiotics, are known as the “anti-Cartesian essays”. Besides, it is not surprising that Umberto Eco considered them at the basis of a “cognitive semiotics”. We have to start from these anti-Cartesian essays. Since the first line of the first of his anti-Cartesian essays, Peirce puts cognition at the core of his interest, by opposing a perceptive model based on “mere 3  “Logic, in its general sense, is, as I believe I have shown, only another name for semiotic (σημειωτική), the quasi-necessary, or formal, doctrine of signs.” (CP 2.277) MS L 751902, MS 478, MS 693, MS 640.

1  Anti-Cartesian Semiotics

65

c­ ontemplation” of a cognition to an inferential model working in the absence of the object. Peirce shows a phenomenological way of thinking, in which “cognition” means something (feeling, image, conception, thought etc.) “we have present to the consciousness” (5.283) and mind is the “theater” in which these phenomena are manifested. Peirce deals with a “phenomenological mind” (Gallagher and Zahavi 2008) and it is not surprising that, later, he often spoke of phaneroscopy as connected to the notion of a semiotic mind. Therefore, the problem Peirce is dealing with is how this “cognition” can become “present to the mind”. While, in the perceptive model, cognition is determined directly by the external object, in the inferential model, the actual cognition we have present to the consciousness is determined by previous cognitions that are extracted from thoughts in signs (CP 5.213). Peirce asked if it was really possible to draw a cognition directly from the object, without any previous cognition in the phenomenological structuration of this cognition itself playing a part. Through numerous examples, Peirce denied this possibility and described it as one of his famous four incapacities (anti-intuitionism):witnesses are unable to distinguish between what they have seen and what they have inferred (CP 5.216).4 The third dimension of space and the continuity of the perceptual field are indeed not seen but learnt, inferred and reconstructed, otherwise we would perceive a blind spot in our visual field and we would not be subject to perceptual illusions, such as, for instance, those linked to magic tricks (CP 5219–223). In addition, perceptions and sensations are not intuitions, but complex forms of inference in which a series of complex predicates are replaced by a simple one (CP 5.266–309) etc. However, it is important to emphasize that Peirce is not, despite what has been argued, opposing an inferential model of a semiotic type against a perceptual model based on direct perception of affordance in the world, as in Gibson (1979) or Gallagher’s Interactive Theory (2005, 2020). Nothing of the kind. What Peirce is saying is that in every cognition it is impossible to distinguish by intuitive means (that is without reference to previous knowledge) which part of cognition comes from the world and which, on the other hand, comes from previous knowledge of the world. For this reason, a form of appraisal of the cognition which makes itself present to the mind, whose purpose is to identify its various provenances, can take place only by semiotic means, from the starting point of a ‘reasoning from signs’. The “semiotic mind” issue is thus a classic philosophical issue, that of the division of the composites. In every cognition there are always components which come from objects from the world and components which come from previous knowledge of the world. De facto, experience supplies us with nothing else other than mixtures: cognition makes no exceptions and behind the notion of “cognition” we arbitrarily group things which differ in kind. We have somehow lost the ground of composites. What is needed is thus to rediscover the articulations which make up these mixtures beyond what is manifested in our cognitive experience (de facto). If the composite r­ epresents

4  Obviously, it is not a problem of memory; it is the fact that perception relies on habits built up from previous cognitions (priors).

66

The Semiotic Mind. Beliefs, Habits and Extended Cognition

the fact, it must be divided into tendencies or into pure directions that only exist in principle (de jure). These conditions are constitutive of experience. This is the starting point for certain of Bergson’s most famous words on the art of dividing the mixtures which are generated in experience, identifying pure tendencies which only exist de jure (Bergson 1889). It is also the starting point of certain parts of the Phaedrus, in which Plato compares philosophers with skilled cooks who know how to cut meat without breaking its bones, following the structure accorded it by nature (Deleuze 1966: 21–29). Cognition makes no exceptions and Peirce takes his place here in a thousand of years long tradition in which the problem is carving, cutting the object—the cognition which is made present to the consciousness—in an adequate way. An example will help us to understand this better. For my mother’s generation not vaccinating your children was inconceivable. This was not because people believed in vaccination more than they do today, but because it never occurred to anyone to take up an opinion on such sensitive matters which had been handed over to a series of experts (i.e., doctors), who were considered to really know what they were talking about. In today’s world, where the ‘conspiracy’ culture so disliked by Eco (1988, 2000) accompanies a general distrust of others and an uncontrolled quantity of information freely circulating on the internet, people have come to believe that doctors—drug companies’ partners in crime—hide certain fundamental damaging information on vaccines, because they gain financially from vaccinating children. An extremely wide-ranging movement of parents who decide not to vaccinate their children was thus born, exposing those children (and other families’ children) to considerable risk. But what do these parents know about vaccines? Does the knowledge on the basis of which they decide whether or not to vaccinate their children come directly from objects? Have they ever been to a laboratory? Have they ever looked into a microscope? Have they ever read an essay on scientific medicine? Usually they have not. Their knowledge on vaccines derives from previous knowledge circulating in the non-scientific open space, with all the manipulation and ingenuity which is so characteristic of it. While the scientific expert community has shown that vaccines do not constitute a real danger to health, dozens of cases contradicting this idea circulate on Facebook and other social networks, and on sites not managed by doctors. Today, many people decide to believe these sources rather than those of the scientific community. This is an example of what Peirce was talking about: our knowledge of the world is formed from elements deriving from objects and from elements deriving from previous knowledge circulating in the community. Knowing is thus a form of appraisal, surveying knowledge circulating in the community and weighing it epistemically. Peirce called this previous knowledge which shapes our cognitions and represents the background to our perception of the world “interpretants”. Peirce thought of these interpretants not as located in the individual mind, but rather as

1  Anti-Cartesian Semiotics

67

distributed in the intersubjectivity of the interpreting community.5 The interpretants are illeities (cfr. Supra, Chap. 2) impersonal points of view on objects circulating in the community space (supra-personal level) and which determine personal points of view in the first person (personal level). This was the starting point of his appeal to the “scientific community of interpreters” to survey these interpretants. This is the activity of interpretation for Peirce: an act of appraisal, surveying our knowledge exactly as we would a house or something we want to buy, in order to determine its value and pay the fair price for it. Effectively interpres was originally precisely a mediator, he who guaranteed an object changing hands for the right value. If we are to believe language historians, the word interpres originally designated a middleman in a transaction, he whose good offices are needed for an object to change ownership [...]. Interpres thus ensures a passage and simultaneously takes care to accord the object concerned an exact value and assist in transmitting it in such a way as to certify that the object is changing hands in its entirety. (J. Starobinski 1974: 23)

The information circulating about vaccines is not all of the same quality and value. For this reason, it is crucial to determine which part of a cognition comes from objects and which derives from previous knowledge: this process cannot be accomplished intuitively, but only via reasoning from signs. Semiotics is the surveying of interpretants circulating in a community via a semiotic mind, for the purposes of determining value (appraisal). It is a form of participatory sense-making which deals with structural coupling between a “community of interpreters” and an “environmental world of interpretants”. This is the first and central starting point for a semiotic mind. If cognition is a phenomenon which makes itself present to the consciousness, this phenomenological level of a personal type refers to a further level of a supra-personal type, populated by interpretants. A cognition making itself present to the mind is thus a sign determined by other previous knowledge circulating in the community. But cognition is not determined solely by previous cognitions but also by the objects themselves, for example by that which Peirce calls the Dynamical Object, which is “the Reality which by some means contrives to determine the Sign to its Representation”. On this subject, Umberto Eco (1997) talked about a “sense before any sensate articulation effected by human cognition” that ensures “that some things that we say about it and for it cannot and must be not taken as holding good” (Eco 1997: 53, see Paolucci 2016, 2017a). We use signs to express a content, and this content is carved out and organized in different forms by different cultures (and languages). […] It is like beef or veal: in different cultures

5  This idea of previous knowledge circulating in the community, that will inspire Umberto Eco’s idea of Encyclopedia, can be usefully thought as a cognitive semiotics’ institution, a set of human and non-human interrelationships embedded in different semiotic and cognitive systems. Each of them may be defined not simply by semiotic and cognitive principles, but by “norms and practices, and by less formal and imperfect social interactions that may involve a variety of biases. Different task-players are dynamically related in a gestalt arrangement such that an intervention (above a certain threshold) on one node or element in the system will lead to modulations in other nodes or elements, or in the whole”. (Gallagher et al. 2019: 7).

68

The Semiotic Mind. Beliefs, Habits and Extended Cognition the cuts vary, and so the names of certain dishes are not always easy to translate from one language to another. And yet it would be very difficult to conceive of a cut that offered at the same moment the tip of the nose and the tail. There are things that cannot be done (or said). The fact that these things were said once upon a time does not matter. (Eco 1997: 52–3)

This point is very important for debates such as that concerning vaccines, as we can see. And once again it refers to cutting meat, as in Plato or Bergson. This is a cutting which respects natural structure, as Plato said, and is determined by it. The way in which the object determines the cognition which makes itself present to the consciousness refers to a further, third, level inherent to a semiotic mind, the sub-­ personal level, which refers to a semiotics of perception. We will examine this level in chapter “Perception as Controlled Hallucination”. However, even at the current state of our debate, we can grasp the deeper meaning for a cognitive semiotics of Peirce’s famous definition of sign: A sign, or representamen, is something which stands to somebody for something in some respect or capacity. (CP 2.228) Whenever we think, we have present to the consciousness some feeling, image, conception, or other representation, which serves as a sign. (CP 5.283)

From all this, the structure of a semiotic mind is easy to understand. A thought, a sensation, a perception or an image which makes itself present to the consciousness (cognition) is a sign in the place of an object (sub-personal level) and in the place of previous knowledge (interpretants) of the same object (supra-­personal level). Or rather, a phenomenon which makes itself present to the mind is a sign which stands for an object to a series of previous knowledge, circulating in the interpretant community. “Object” and “interpretant” are the “something” and the “somebody” of the Peircean definition of sign; interpretants are not interpreters. Rather, they are illeities, the result of the structural coupling between a semiotic mind and a world/ environment constituted by somethings and somebodies.

2  C  ognitive Semiotics and Pragmatism: Beliefs and the Extended Mind Cognitive semiotics claims that “thinking” refers to building “cognitions” and that cognitions are phenomena that are “present to the mind”. It also claims that these phenomena are determined by objects and interpretants and stand for them in some respect or capacity. In order to determine the particular way in which cognition stands to somebody for something we need a cognitive semiotics, that is, a “reasoning from signs”, since we cannot tell through intuition alone if a cognition is determined by its object or by interpretants circulating inside a community. This is why it is fundamental for cognitive semiotics to enquire into the way through which interpretants are built, fixated and discussed by the community. Pragmatism is

2  Cognitive Semiotics and Pragmatism: Beliefs and the Extended Mind

69

exactly the theory devoted to answering this question. According to Peirce, who was the inventor of the word and the founder of this key way of doing philosophy, it is a theory of the “Fixation of Belief” that tells us “How to Make Our Ideas Clear”, by enquiring into how the individual stream of thought tends to stabilize in a set of beliefs that find in the intersubjectivity of the community their only warranty. The centrality of beliefs is a key position for cognitive semiotics, which is tied to pragmatism, since beliefs play characteristic roles in shaping our thinking and acting in a very different manner than similar attitudes, such as expressing an opinion or hoping. This is why beliefs are the core example both for pragmatism and the extended mind theory, as promoted by Clark and Chalmers (1998). A comparison between the two will enlighten some key features of a cognitive semiotics grounded on pragmatism. In order to do that, I will now briefly reconstruct Peirce’s pragmatism in relation to his theory of cognition, in order to show the epistemological frame that connects it to the theories of the extension of the mind (see Clark 2008), to enactivism (Hutto and Myin 2013; Gallagher 2017) and to distributed cognition (Hutchins 2001; Alaĉ 2011). According to Peirce, pragmatism is a semantic theory which has to do with cognition, but gives special attention to the meaning of beliefs. Indeed, people who decide not to have their children vaccinated that vaccines are dangerous or useless; they do not simply opine that it is so. As we have said, according to cognitive semiotics, meaning is not content thought in truth-conditions nor is it representational, since it does not have “the function of saying or indicating that things stands thus and so” (Hutto 2013: 62). On the contrary, meaning is connected to “practical bearings” and to “habits”. In Peirce’s words, “to develop meaning, we have simply to determine what habits it produces, for what a thing means is simply what habits it involves” (CP 5.400). For pragmatists, meaning is subject to the community and to the extension of thought in the environment. As the “pragmatic maxim” makes clear, it is the main output of pragmatism itself in its very first formulation, where Peirce claims that we have to “come down to what is tangible and conceivably practical, as the root of every real distinction of thought, no matter how subtle it may be, and there is no distinction of meaning so fine as to consist in anything but a possible difference of practice” (CP 5.400). In order to throw light on all this, I would like to go back to the essays “The Fixation of Beliefs” and “How to Make Our Ideas Clear”, in which Peirce establishes his “social theory of logic” (CP 5.431), and connects it with his previous theory of cognition on which it relies. Pragmatism is, indeed, inseparable from the cognitive semiotics promoted by Peirce (although the expression itself was coined by Umberto Eco). It is no coincidence that in one of the foundational essays of pragmatism, Peirce starts from semiosis and from the continuous dynamic of thought-sign (cognition) and defines the following two structural properties as being fundamental for a pragmatist theory of knowledge: 1. the semiotic movement of interpretants tends toward its stabilization and toward rest. If the thought-sign is a “a thread of melody”, its development tends toward “the demi-cadence which closes a musical phrase” (belief). This stabilization of

70

The Semiotic Mind. Beliefs, Habits and Extended Cognition

semiosis sets up a regular tendency to act in a regular way in the future, that Peirce calls habit (CP 5.397–8). 2. Every “moving thought”, viz. every interpretation, always arises from a ground of regular habits, emerging as a figure that comes out from its background. In this sense, Peirce can define “habit” as the “guiding principle” of every inference leading us from one sign to its interpretant sign (CP 5.367). That which determines us, from given premises, to draw one inference rather than another, is some habit of mind, whether it be constitutional or acquired. The habit is good or otherwise, according as it produces true conclusions from true premises or not; and an inference is regarded as valid or not, without reference to the truth or falsity of its conclusion specially, but according as the habit which determines it is such as to produce true conclusions in general or not. The particular habit of mind which governs this or that inference may be formulated in a proposition whose truth depends on the validity of the inferences which the habit determines; and such a formula is called a guiding principle of inference. (CP 5.367)

Thus, it is evident, from a cognitive semiotics’ point of view, that habits take primacy over inference. This has very significant consequences for cognitive sciences, which can be identified, for example, at the level of a theory of perception, in particular in the so-called “bootstrap problem” connected to the priors of the predictive processing (see Clark 2016; Hutto and Myin 2017). We will deal with this issue in chapter “Perception as Controlled Hallucination”. As will be clarified later in this work, like enactivist theories, which are indeed inspired by pragmatism, Peirce works at the level of systems theory, claiming a continuity between mind and matter, which is the main position of his synechism (see Zalamea 2003; Fabbrichesi and Leoni 2005; Havenel 2008; Paolucci 2011; Malafouris 2013; Iliopoulos 2019). Thus, three founding principles of a cognitive semiotics can be identified: (1) a semiotic system always tends toward stabilization, whatever it might be, (2) the purpose of stabilizing the system is not to represent the world in the most accurate and truthful way, but rather to act in the environment, avoiding states of instability as far as possible, and (3) regularity has a value and a priority within semi-linguistic systems. In fact, interpretation arises from the regularity of previously established habits and goes back to them as a figure that stands out from its background, in order to modulate this very same background. This is why Peirce can claim that we are not interested in believing something to be true, but rather in believing what we already believe, in order to minimize the cognitive effort required to make the system work. It is only when the system does not work, namely, when its habits are no longer capable of making action effective, that we then try to change these habits. Pragmatism helps cognitive semiotics build a processual theory of thought-sign (cognition). Its task is to lead to a general theory of habit formation—or regularities—and of the transformation of these regularities. This is the popular theory of the “fixation of belief”, which leads, in Peirce’s thought, to a pragmatist semantic theory that identifies meaning with the regularity of habit (CP 5.400). Peirce says that (1) the true function of the semiotic flow of cognitions is to install habits of action, and that (2) the concatenation of our actions is always a function of the

2  Cognitive Semiotics and Pragmatism: Beliefs and the Extended Mind

71

h­ abits guiding the series of actions. With pragmatism, Peirce develops a semantic theory that is intended to account for the processual dynamic of thought and action (semiosis), seeking to define a semantics of action in its relationship with cognition (see Ricoeur 1977). This processual dynamic consists in the union of (1) a stable state, called “belief”, that tends not to be questioned and thus presides over the concatenation of actions and (2) an unstable state, which Peirce called “doubt” which also tends to become stabilized. Each of these states holds an affective value. Belief is euphoric, doubt is dysphoric. This is why the semiotic system tends to maximize the first one and to minimize the second one. The transition from instability to stability is the famous Peircean inquiry (CP 5.374). Doubt is an uneasy and dissatisfied state from which we struggle to free ourselves and pass into the state of belief; while the latter is a calm and satisfactory state which we do not wish to avoid, or to change to a belief in anything else. On the contrary, we cling tenaciously, not merely to believing, but to believing just what we do believe. (CP 5.372)

In its original cognitive semiotics formulation, pragmatism tries to determine the way in which we tend to attribute meaning and values. In Peirce’s view, there is in each community a tendency to “continue in its being” that defines the attitude that communities hold towards their own signs and that comes before any philosophical “pursuit of truth”: We may fancy that this is not enough for us, and that we seek, not merely an opinion, but a true opinion. But put this fancy to the test, and it proves groundless; for as soon as a firm belief is reached, we are entirely satisfied, whether the belief be true or false. […] The most that can be maintained is, that we seek for a belief that we shall think to be true. But we think each one of our beliefs to be true, and, indeed, it is mere tautology to say so. (CP 5.375)

Peirce’s pragmatism is not a theory of truth, but a theory of meaning. And it is precisely because we lazily tend to adhere to what we believe that we need a scientific method for fixing beliefs, which is capable of breaking this trend in favor of truth. In short, while logic studies how we should think (see Hookway 2009), the social theory of logic studies how we actually think. According to Peirce we never think, as we should, in a “logical” manner. The scientific method for the fixation of beliefs is indeed necessary because Peirce’s analysis highlights a tendency to attribute meaning according to that previous meaning which is sedimented in our habits and in our beliefs and to act according to it, regardless of whether it is actually true or not. How can we make our ideas clear? How can we fix beliefs inside our communities in a scientific way? Pragmatism’s answer to this question, which remains crucial also for our contemporary debate, lies in the pragmatic maxim. Consider what effects, that might conceivably have practical bearings, we conceive the object of our conception to have. Then, our conception of these effects is the whole of our conception of the object. (CP 5.440)

This idea, which leads to an identification of beliefs with habits, is central to cognitive semiotics since it allows us to find a criterion with which to evaluate the narratives and the interpretants circulating within a community.

72

The Semiotic Mind. Beliefs, Habits and Extended Cognition

Indeed, according to Peirce “reality, like every other quality, consists in the peculiar sensible effects which things partaking of it produce. The only effect which real things have is to cause belief, for all the sensations which they excite emerge into consciousness in the form of beliefs” (CP 5.406). But the pragmatic maxim must also be applied to beliefs, in order to identify their meaning. The essence of belief is the establishment of a habit; and different beliefs are distinguished by the different modes of action to which they give rise. If beliefs do not differ in this respect, if they appease the same doubt by producing the same rule of action, then no mere differences in the manner of consciousness of them can make them different beliefs, any more than playing a tune in different keys is playing different tunes. (CP 5.398)

The pragmatist concept of belief and its identification with habit perfectly explain the state of mind of people whose speech and action suggest both that they ought to be ascribed the belief that p (according to speech) and the conflicting attitude of believing ~p (according to action). Schwitzgebel (2010) provides a rich phenomenology of these occurrences: Juliet, the implicit racist; Kaipeng, the trembling Stoic; and Ben, the forgetful driver. In all of these cases, With genuine conviction and complete sincerity, you endorse some proposition P. Every time you think about P, you reaffirm it; to you, it seems unquestionably true. Yet if we look at the overall arc of your behavior – at your automatic and implicit reactions, at your decisions, at your spontaneous remarks on nearby topics – there’s a decidedly un-P-ish cast. What should we say you believe in such cases? (Schwitzgebel 2010: 531).

All of these cases are problematic from the standpoint of a theory that conceives beliefs in terms of propositional attitudes, like analytical philosophy does, but they are quite unsurprising and unchallenging for a pragmatist approach grounded on cognitive semiotics. Since the meaning of a belief depends on how it might lead us to act, its meaning is not thought as content conceived in truth conditions nor in terms of an attitude towards a propositional content. On the contrary, the meaning of a belief lies in the habit that it produces, and all the rest is only the way you try to represent yourself by bringing forth a world that can be used in order to lie during participatory sense-making. Of course, beliefs are not always non-representational; beliefs can also have content. And of course, there are beliefs that can be understood in terms of propositional attitudes. However, they are special cases of a broader cognitive phenomenon that is better understood in different terms through pragmatism. Daniel Hutto (2013; Hutto and Myin 2017) makes an important distinction between beliefs as intentional attitudes and beliefs as propositional attitudes. Beliefs as intentional attitudes are the sort of beliefs that both we and other animals adopt towards situations and that lead us to act (for instance, the dog barks because he believes a stranger is coming closer). Beliefs as propositional attitudes “are also directed to situations—and so qualify as intentional attitudes, but they are linguistically mediated intentional attitudes” (Hutto 2013: 55). This matters because all beliefs target a particular state of affairs, but their intentionality implies neither intensionality (with the “s”) nor extensionality, since the aim of a belief is to grant effective action and not to represent the world through content. This is a “twin argument”, similar to that ­relating to

2  Cognitive Semiotics and Pragmatism: Beliefs and the Extended Mind

73

the idea of meaning conceived as a representation linked to truth conditions. Similarly, aliefs—associative beliefs which are automatic, a-rational and which we share with other non-human animals—and beliefs as propositional attitudes are contentful states. The content of beliefs and aliefs is representational and thought in terms of truth-conditions. As Stalnaker (1998) showed us, “it is essential to propositional content that they have truth conditions”. However, as Peirce emphasized, beliefs are not meant to represent the world. Indeed, we tend not to modify beliefs even when we are in the presence of a truth, as long as they still contribute to effective action or forms of life which have value for us. As Hutto (2013: 66) pointed out, while opposing a semiotic conception of belief to a semantic conception based on truth conditions, “if we reject teleosemantics in favor of teleosemiotics, we can […] accept that organisms often act successfully by making appropriate responses to objects or states of affairs in ways that are only directly mediated by their sensitive responding to natural signs, where this responding does not involve contentfully representing the objects or states of affairs in question, i.e. in terms of contentful states of mind that refer or have truth or accuracy conditions”. This idea also offers a very elegant explanation of the reason why it is so difficult to alter pure intentional attitudes by direct rational means, such as arguments or linguistically mediated contents relying on “truths”. Moreover, it also offers a straightforward explanation of why “we will have reason to ascribe a beliefas-propositional attitude that p to certain individuals while at the same time ascribing them a belief-­as-­intentional attitude that is directed at the same state of affairs in ways that apparently performatively conflict with the person’s professed belief. […] What we are dealing with is a tension between a contentful propositional attitude and a contentless purely intentional attitude” (Hutto 2013: 71). During a conference on the idea of belief, a colleague and friend of mine with an analytic background and an analytic approach to the philosophy of the mind, asked me how I could explain, by supporting a pragmatist perspective and refuting an idea of representation based on truth conditions, the belief that there were 17 people present in the room (something which, obviously, I myself believed). My answer to that question was that I would have never ordered 16 pizzas if I had not known that at least one of us wanted to skip lunch. This was, indeed, a way to consider the “conceivable practical bearings” highlighted by Peirce and a way of translating our beliefs into possible actions and habits, while explaining their meaning on the basis of how these beliefs might have led us to act. Applying these ideas to the beliefs of Otto and Inga, the key example for Extended Mind Theory, will help us to understand even better. The famous case of Otto and Inga is one in which a belief is conceived in terms of propositional attitudes and it is the example given by Clark and Chalmers (1998) in their very first formulation of the extended mind. It is an extremely interesting example, which confirms that fame does not always do justice to an idea. Indeed, the example was formulated by David Chalmers, whose approach is analytical, whereas Clark (personal communication) was not certain about the appropriateness of using the idea of belief conceived as propositional attitude, in order to introduce

74

The Semiotic Mind. Beliefs, Habits and Extended Cognition

their theory. Notwithstanding this, as often happens, over the last two decades Clark has had to deal extensively with this example and this idea of belief, often quoted as the main instance of his Extended Mind theory. What he was truly interested in, as far as Otto and Inga’s example was concerned, was the principle of cognitive impartiality, according to which there is parity between Inga’s biological memory and Otto’s notebook. On the contrary, he was not that interested in the content of a belief thought in terms of propositional attitudes, which was a contribution proposed by David Chalmers. Indeed, in their seminal first paper, which founded the very idea of the extended mind, Clark and Chalmers (1998) seem to promote a conception of cognition in terms of mental states or contents, like beliefs, information, representations, etc. This position has been deeply weakened, if not even abandoned, in further works by Clark, such as Supersizing the Mind (2008) and, in particular, Surfing Uncertainty (2016). In this latter work, a view of cognition in terms of cognitive processes and activities, like problem solving, meaning attribution, perception and judging is often proposed. On the contrary, in Clark and Chalmers (1998) first paper, the mind is something constituted by beliefs, desires, representations, propositional attitudes and informational states, and the problem seems to be the localization of these states and contents. This is why it has been rightly noted (see Gallagher 2011, 2017) that the extended mind theory certainly aimed at extending the res cogitans into the res extensa, but did not aim at either tackling the privilege of the mind for cognition or actually abandoning “a concept of the mind that the extended mind hypothesis is really trying to challenge” (Gallagher 2011: 59). Starting from the Anti-Cartesian essays, in which Charles Sanders Peirce founded cognitive semiotics, in this paragraph I have tried to introduce an idea of both cognition and belief that has nothing to do neither with content nor with mental states. According to this pragmatist approach, the meaning of a cognition consists in its “conceivable practical distinctions” (CP 5.409) and if two cognitions or two beliefs lead to the same practical distinctions, whatever difference between them “has no more to do with our real meaning than the difference between the French idiom “Il fait froid“ and its English equivalent “It is cold” (CP. 5.404). Let’s apply all this to the often-quoted example of Otto, the man affected by the Alzheimer disease to whom Clark and Chalmers attribute an experimental “extended mind” (see Fabbrichesi 2016). Otto uses a notebook on which he writes the location of the museum that he wants to visit. Any time he wants to go there, he reads his notes. The notebook plays the role usually played by biological memory in people who do not suffer from memory disorders, as in the case of Inga, who relies on her own memory to remember the museum’s location. Otto’s notebook works exactly like the information constituting an ordinary non-occurrent belief; it simply happens that this kind of information lies beyond the skin. In Otto’s case, as in a normal person’s case, the information is available to consciousness and available to guide action, exactly in the same way in which a belief usually is. And when an external device plays the same role that a belief plays, it can be considered equally “mental.” A pragmatist like Peirce would have remarked that Otto’s notebook has to be considered part of cognition simply because it guides habits of action (the habit of walking quietly to the museum) and that his notebook is equivalent to Inga’s

2  Cognitive Semiotics and Pragmatism: Beliefs and the Extended Mind

75

­ emory because they have the same meaning, both installing the same habit of m action. In Peirce’s view, as far as cognition is concerned, we leave the domain of a mind with its states and contents and we enter the domain of meanings through signs and habits of action. This is cognitive semiotics. In order to explain cognition, meaning and habits are crucial and states and contents are signs which call for an appraisal. This is why Peirce can claim that “the whole function of thought is to produce habits of action” and that “the identity of a habit depends on how it might lead us to act” (CP 5.398). This is an opportunity to say something important on the nature of beliefs in Otto and Inga’s cases, as well as on the meaning of the different entities supporting them. If the essence of belief is the establishment of a habit and if different beliefs are distinguished by the different modes of action to which they give rise, then no mere differences in the manner of consciousness of them or in the different supports that serve to implement them can make them different beliefs, any more than playing a tune in different keys is playing different tunes. As far as beliefs are concerned, Otto’s notebook and Inga’s biological memory are the same tune played in different keys. And so are the beliefs that they implement. Otto’s notebook and Inga’s biological memory are part of the same cognitive machinery because they both have the same meaning and lead to the same habit of action, and not because they install the same cognitive circuitry, embodied in different supports. Like one of its constitutive dimensions, Material Engagement Theory, cognitive semiotics subscribes to a locationally neutral account of cognition, but not to a substrate-neutral account of cognition (see Fusaroli and Paolucci 2011): “what minds are made of matters as much as how minds are functionally organized” (Malafouris 2013: 76). And this is for technical reasons: as we have seen in our first chapter, natural language is the only disembodied semiotic system that can be implemented in different substrates, but all the other semiotic systems are not and their structures depend on how the form of their expression emerge from the physical substrates. Painting, architecture or music are perfect examples for dependence on physical substrates, because if you change their substrates, you change their identity. Of course, this does not hold true for language, since “it is trivial to say, for instance, that spoken Italian, written Italian, Italian telegraphed by Morse code, Italian transmitted by flags according to the international code of Navy is, in all these cases, one and only language and not four different languages” (Hjelmslev 1959: 16). However, the Principle of Cognitive Impartiality by Andy Clark is crucial for a cognitive semiotics, that remains locationally uncommitted, or—as the computer scientist William Clancey (2009: 28) puts it—remains committed to “antilocalization”. Cognition and mind are not things that can be localized. Rather, they are something that emerges from the constitutive intertwining of brains, bodies and objects shaped by semiotics systems in a specific cultural setting. Cognitions call for meanings and different meanings are distinguished by the different modes of action to which they give rise. If different cognitions localized in different supports give rise to the same mode of action, they are not different cognitions and do not have different meanings. This is cognitive semiotics’ way to keep together pragmatism and extended mind theory, tackling the privilege and the standpoint of the mind

76

The Semiotic Mind. Beliefs, Habits and Extended Cognition

in explaining cognition. Cognition has to do with meanings and habits more than with contents, states and their localizations. As Malafouris (2013: 15) claims, the cognitive scientist who is searching for the mind behind the meaning of signs and the habits of actions “is committing the same ‘category mistake’ as the foreign visitor to Cambridge or Oxford who, having seen the colleges, the libraries and the departments, asks to be shown the university”. This is why cognitive semiotics and 4e cognition owes Andy Clark a lot. According to different ideas and theories (and also according to personal preferences), one can prefer the “equal partners” argument by Hutto and Myin (2013, 2017) or some of the important tunings of the Principle of Cognitive Impartiality (PCH) authored by different scholars inside 4e cognition. One can also see the dangers of a subterranean return to functionalism, since Clark (2010) himself used a functionalist argument against Adams and Aizawa’s (2010) critiques. However, recent work by Andy Clark (2016) moves in the very opposite direction and both Sutton (2010), Kirchhoff (2012) and Menary (2018) spoke of a “second wave” and “third wave” of PCH, moving from functionalism to cognitive complementarity and integration. However, as it can be seen, the actual debate in cognitive sciences and philosophy of mind has been changed by Andy Clark’s PCH and it owes him a lot. And it owes him even more if we think that this very idea has been formulated more than 20 years ago, when, for the cartesian cognitive science, thinking that the mind could be extended also into the res extensa was unpalatable. Moreover, as far as embodied cognition is concerned, PCH represented a major step in a landscape which was too dominated by an idea of the mind as neurophysiologically embodied in the brain and in a culture of cognition in which “almost everything that has to do with the thought seems to have to do with a few images of the brain” (Clark 2008: xxvii). It is not by chance that Charles Sanders Peirce formulated a pilot prototype of the extended mind theory more than 150 years ago. And in this extended mind theory, the central role that mind plays in cognition was questioned and tackled.

3  T  he Semiotic Extended Mind (1): Parity Principle and Cognitive Intertwining The notion of “Mind” is a recurrent one in contemporary debates, but, like any other notion, its meaning has modified repeatedly over time.6 Cognitive semiotics can help throw light on this. For instance, it might be helpful to briefly recall that in The Iliad the word that is usually translated as “mind” is expressed by terms which refer to the different parts of the body: phren, the lungs; ker, the heart; thymos, the breathing spirit; noos, the sharp sight. All these terms refer to externally induced bodily modifications, that force the hero to respond through a series of actions. These are  I follow here Fabbrichesi 2016.

6

3  The Semiotic Extended Mind (1): Parity Principle and Cognitive Intertwining

77

not tied to physical organs as we understand them today, but to cosmic powers of divine origin, capable of dominating and guiding men’s life. In this view, men feel and reason, choose and move because their breath, beating heart, and sight dictate that they do so. The word “mind”, which has its etymological root in menos, the impetus that runs through the body and compels to act, develops through the Odyssey, Plato, and the Stoics, until it becomes the Latin mens, in which it is not possible to find any reference to body and physical emotions anymore. Through a long and complex process of semantic transformations, the “mind” ends up being localized in the head, “inside the skin and the skull,” where 4e cognition finds it and move it out, finding again the places where it was in the beginning. It is only the priority of alethic and rational knowledge made possible by the “Cogito”, connected to the idea that the body and the senses are unreliable sources of information, that has led to understand the “mind” as the privileged seat of thoughts, beliefs, sentiments, and self-consciousness. For millennia, the idea of mind as we thought of it before Andy Clark’s work, had been simply unimaginable (see Fabbrichesi 2016). Indeed, we have seen in the previous chapter how “metis” and self-consciousness were connected to efficient action, to cognitive semiotics and to the idea of an enacted mind. “Mind” is not even a concept that had an existence and a corresponding word. This is the reason why Ugo Volli (1994: 171) could say that in Homer there is no mental place where the world is represented and where decisions, plans, evaluations of the different forces of instinct and passion are made in order to get to a unitary synthesis. Nothing compared to a res cogitans inside the head came close to existing. And, as far as cognition was concerned, the very same distinction between “internal” and “external” was a clumsy one. Indeed, in order to identify a cognition, a physical principle based on perception was adopted (the “internal vs external” dichotomy). However, it should be noted that, according to Cartesian cognitive science, cognition is considered mental and, therefore, not physical. All in all, to use a terminology already introduced in the second chapter, in classical cognitive science, the opposition between mental and physical—like the one between internal and external—were exclusive and not participative. Conversely, in cognitive semiotics these oppositions are participative. Thus, in the last paragraphs of the second of his anti-Cartesian essays, “Some Consequences of Four Incapacities”, Peirce asked the most fundamental of all questions: “in what consists the reality of the mind?” (CP 5.313). His answer was extremely radical: mind is an “external sign developing according to the laws of inference” (CP 5.313–4). This was not simply the reaffirmation of the incapacity of introspection to which “all knowledge of the internal world is derived by hypothetical reasoning from our knowledge of external facts”, but a conception of the mind that was completely original for its time and that only very recently has been reconsidered by studies on cognition. Peirce was able to formulate a sort of “parity principle”, which consisted in his synechism, or the theory of the continuity between mind and matter that Peirce would formulate exactly as “The Law of Mind” and that would later influence William James’ thought. In order to proceed with this, let’s read the excerpt:

78

The Semiotic Mind. Beliefs, Habits and Extended Cognition To begin with the psychologists have not yet made it clear what Mind is. […] Feeling is nothing but the inward aspect of things, while mind on the contrary is essentially an external phenomenon. The error is very much like that which was so long prevalent that an electrical current moved through the metallic wire; while it is now known that that is just the only place from which it is cut off, being wholly external to the wire. Again, the psychologists undertake to locate various mental powers in the brain; and above all consider it as quite certain that the faculty of language resides in a certain lobe […]. In my opinion it is much more true that the thoughts of a living writer are in any printed copy of his book than that they are in his brain. (CP 7.364) A psychologist cuts out a lobe of my brain and then, when I find I cannot express myself, he says, "You see your faculty of language was localized in that lobe." No doubt it was; and so, if he had filched my inkstand, I should not have been able to continue my discussion until I had got another. Yea, the very thoughts would not come to me. So my faculty of discussion is equally localized in my inkstand. […] It is plain enough that the inkstand and the brain-lobe have the same general relation to the functions of the mind. (CP 7.365)

In Supersizing the Mind, Andy Clark’s argumentation about Richard Feynman’s work perfectly overlaps with Peirce’s argumentation. Clark (2008) begins his book reporting the debate between the historian Charles Wiener and the physicist Richard Feynman, concerning some Feynman’s freshly published manuscripts. According to Wiener, these manuscripts represented a record of Feynman’s daily work. Feynman’s answer was the following: “I actually did the work on the paper”. “Well,” Weiner said, “the work was done in your head, but the record of it is still here.” “No, it’s not a record, not really. It’s working. You have to work on paper and this is the paper. Okay?” (from Gleick 1992: 409)

A conflict between different perceptions can be identified in this passage. On one hand, from Weiner’s point of view, work is done in the head and paper is simply an aid. On the other hand, from Feyman’s point of view, the work is done on the paper and through it and, if the external part was removed, the very nature of the work would change. Below is Clark’s comment: Such considerations of parity, once we put our bioprejudices aside, reveal the outward loop as a functional part of an extended cognitive machine. […] Feynman’s suggestion is, at the very least, that the loop into the external medium was integral to his intellectual activity (the “working”) itself. But I would like to go further and suggest that Feynman was actually thinking on the paper. The loop through pen and paper is part of the physical machinery responsible for the shape of the flow of thoughts and ideas that we take, nonetheless, to be distinctively those of Richard Feynman. It reliably and robustly provides a functionality which, were it provided by goings-on in the head alone, we would have no hesitation in designating as part of the cognitive circuitry. (Clark 2008: xxv)

It is clear that extended mind theory, in both Peircean and Clarkean formulations, is somehow grounded on a very popular semiotic operation called the “commutation test” (Hjelmslev 1959) which generated the structural linguistics’ revolution within the language sciences. To sum up, discovering whether an element is a constitutive part of a system requires replacing it with another one (hence “commutation”) and seeing if the meaning of the system changes. If it does not change and the system in its entirety remains the same, then that element is a “variant” of the ­system

3  The Semiotic Extended Mind (1): Parity Principle and Cognitive Intertwining

79

and not a constitutive component of it, one which the semio-linguistic tradition called a “formant” of the system. Conversely, if the meaning changes and the system in general has been modified by the commutation, then that element is constitutive of the identity of the system and cannot be commuted. For instance, as far as linguistic systems are concerned, different regional pronunciations of the same word are variants, whereas the different phonemes are formants of the system (the couple “b/c” changes the meaning of the linguistic system, as far as the English “bar” and “car” are concerned). The very same thing is true of cognitive systems. The pen, the paper and the inkstand which Peirce refers to, are commutable with a tablet or a capacitive stylus. It is not random that this tablet and capacitive stylus are digital versions of ordinary writing tools. Therefore, they are variants in that system. However, the external artefact is not a variant at all, since its removal would dramatically alter the meaning and identity of the distinctive operations of the cognitive system itself. As Feynman puts it, “it is the work” and “you have to work on the paper”. An example may be useful here. As regards the archaeology of the mind, Malafouris (2013) analysed the way in which cognitive functions have changed due to the introduction of writing and its consequences for memory activities. This is an important case study, because it encompasses the Otto and Inga problem in the very same way in which those of Peirce and Feynman do. Malafouris (2013: 79) shows that the introduction of the Linear B tablet for a Mycenean society was “a situated technology instantiating a new way of remembering and a new way of forgetting. The Mycenean simply read what the Linear B tablet remembers. In fact, being able to read, that person no longer needs to remember”. The transition from “remembering” to “reading” is crucial and cannot be underestimated. Before tablets were invented, remembering was a cognitive activity that did not involve reading. However, the cognitive activity of remembering is now profoundly bound up with that of reading. And this holds true for Inga, too, and not only for Otto (and it will be more and more connected to certain innovative digital operations in the future). Thus, the invention of tablets has produced a radical reorganization of the cognitive system in its “internal” part, too, and not only as a system in its entirety. Indeed, as it is very well known, reading and literacy are cognitive operations involving very different regions of the brain from the regions for the working memory. This is why when the external part is added, it reconfigures the system in its entirety, changing the distribution of “formants” and “variants”. The semiotic idea of “commutation test” tackles Adams and Aizawa’s (2008) distinction between “causation” and “constitution”, between “mere interaction” and “participation”. According to the “coupling-constitution” fallacy (Aizawa 2010), pen, paper and tablet do not remember and are not part of the work. They simply help us to remember and to work. According to a semiotic terminology (Greimas 1983), they are helpers, not subjects. But this can be claimed only because Adams and Aizawa do not tack on the pre-eminence of the mind and of the individual in cognition and do not think in terms of “cognitive systems” with their “commutation tests”, thus ignoring the reorganization of the cognitive system in its internal and neural part. The external loop which transforms a remembering operation into a reading

80

The Semiotic Mind. Beliefs, Habits and Extended Cognition

o­ peration does not solely change the cognitive processes underlying the cognitive system, but also the brain areas involved, generating a new system with different formants and different variants. The distinction between causation and constitution can only be accepted if the mind is considered as a property or a possession of the individual and the individual has an ontological and agentive priority for the cognitive operations. But if we tack on the pre-eminence of the mind and think in terms of “cognitive systems” rather than “individuals” and “representations”, the commutation test tells us a totally different story. Peirce’s arguments about a semiotic extended mind can help us understand this cognitive system. Since the argument is very original, complex and surely provocative (even more so if situated within the debates of its time), we have to go deeper into this first foundation of a cognitive semiotics. First of all, Peirce claims that the mind has been erroneously confused with “the inward aspect of things” and presents “the feeling” as one of the objects of this confusion. Anyway, emotions, attention, representation, introspection, belief, disposition toward action and—more generally—all those phenomena that in his previous works Peirce had already reduced to semiotics or to the pragmatic maxim could be other possible examples. What Peirce wants to fight is, in fact, an idea of the mind as a set of states and contents like feelings, beliefs, information, representations, etc. According to Peirce, this idea of the mind is wrong because it mistakes the mind with its internal effects, while, on the contrary, these are simply inferred hypothetically from external facts. As a matter of fact, the first part of Peirce’s argument consists in reaffirming the incapacity of introspection—that Peirce will never abandon—and the stating that those who think the mind as constituted by beliefs, desires, representations, feelings, informational states and other propositional attitudes are actually mistaking it with the internal aspect of systemic phenomena. Just as we say that a body is in motion and not that motion is in a body, we ought to say that we are in thought and not that thoughts are in us. (CP 5.289)

Such a rejection of a conception of the mind as constituted by beliefs, desires, representations, feelings and propositional attitudes is a point of originality specific to cognitive semiotics. It puts into brackets the distinction between “causation” and “constitution” pointed out by Adams and Aizawa (2008) that is built on the idea of representation in the internal mind, and replaces it with the cognitive semiotics’ idea of “commutation test”, when singling out the constitutive parts of the cognitive system is required. Cognitive semiotics’ extended mind is an extended mind theory without the classic representations going inside the internal mind. The parity of outer and inner cognitive elements advocate here does not imply a functional isomorphism, but the idea of a constitutive intertwining of brain, bodies, interpretants, habits and things. If you cut away the pen and paper, if you cut away the interpretants and the habits they give rise to, the “motion” of the thoughts we are engaged in changes its flow. When writing enters the scene and the cognitive operation of remembering involves reading, it is another part of the brain that has to be cut out in order to lose that faculty.

4  The Semiotic Extended Mind (2): Phenomenology

81

The second part of Peirce’s argument is the most provocative one. Mind is a phenomenon, and it is an essentially external phenomenon. This provocative stance can be understood only by according the same importance to both the word “phenomenon” and the word “external”, and not only to “external”, as one would be tempted to do in the beginning. At the same time, this provocation cannot be understood if we do not pay attention to the “metallic wire” example. Put briefly, Peirce of course affirms that the mind is external, but “external” has a very specific and polysemic meaning that we need to disambiguate, if we want to understand the depth of Peirce’s position and its theoretical consequences.

4  The Semiotic Extended Mind (2): Phenomenology Let’s start from the idea that the mind is a phenomenon. Peirce can be said to be the founder of phenomenology—that he called phaneroscopy—even if he has never been fully credited for that, as is the case with many other of his “inventions”.7 In his phaneroscopy, Peirce defines the phenomenon as “anything that becomes present to the mind”. How can it be possible for the mind, that should be the “theater of manifestation of phenomena”, to be itself a phenomenon and, moreover, an “external” one? Back in 1861, Peirce talked about the “field of consciousness” as the theater for the manifestation of phenomena (W1: 60–1).8 He would maintain this formulation until the subsequent phaneroscopic turn (cf. CP 1.284–417, RLT: 197–268). In any case, conscious life with its contents is usually thought of as the domain of a “self” to which it is supposed to belong, independent of any nature—whether mental, personal or transcendental—that will be attributed to this same “self”. The first Peircean move consists in expelling the self from the field of consciousness, thus assuming that the theater for the manifestation of phenomena is not the mind, the subject or the individual—which Peirce considered as phenomena among the others—but something else that needs to be discovered. It is not accidental that Peirce decided to replace the terms “Subject” and “Ego” with the more classical “Soul”.9 Now, the Peircean Soul is in no way a knowing subject, since Peirce describes it as a known object or, at least, as an object that can be subject to knowledge. As an object among other objects, the subject (the Soul) is in itself an element of the field of consciousness inside which we can come to know it. For this reason, Peirce highlights in 7  Obviously, this is not true inside the “Peircean community”, where many important works have been published on this topic. In my opinion, the most important is still Parker 1992. The main writings by Peirce on phenomenology are MS 1101, 831, 478, 304–6, 336–7 and 908: all published in The Essential Peirce vol. 2. An important Italian work which deals with many of the problems we are discussing here is Calcaterra (ed.) 2006. 8  On Peirce’s ideas about subjectivity, see Colapietro 1989. 9  This abandon of the subject will be confirmed in W1: 75 & 78 and will represent a constant in Peirce’s thought, up to its complete refusal in the Cognitive Essays from 1868 (cf. CP 5.313).

82

The Semiotic Mind. Beliefs, Habits and Extended Cognition

many points of his writings10—by using the etymological root of upokeimenon as “principal argument of a discourse” or “grammatical subject”—that at a semio-­ linguistic level there is no difference between subject and object, so that, inside a predication, the object of cognition will always be the subject of the proposition.11 This idea will be reformulated many times and will find its final structuration in the essay “On a New List of Categories” (1867), where the sub-jectum—literally “what is situated below”—will be transformed into sub-stantia (see CP 1.547–8).12 In this way, the notion of self, of subject, of “I”, will no longer be attributed the field of consciousness as an opening of its own specificity and, rather than “having to be able to accompany every single representation”, it will be an effect, a correlate of the field of consciousness and of the phenomena inside of it: Accordingly, we have the testimony of consciousness that the subject is not thought but thought of, that is does not enter into the field of consciousness. (W1: 154)

After having expelled the self from the field of consciousness, the Peircean problem becomes that of discovering (1) what the constitutive elements of this field are, and (2) what nature should be attributed to the field itself. As far as point (1) is concerned, Peirce calls thought the element of the field of consciousness and underlines how “thought” for him is equal to “phenomenon”, viz. to what “is present to the consciousness” or “the idea as it appears to the consciousness” (W1: 61). One would be tempted to identify the “field of consciousness” with the “mind”, given that the phenomena that are manifested there are thoughts. Anyway, the Peircean notion of “thought” assumes a very particular meaning that makes it entirely incompatible with all forms of Cartesian theater as he writes that “thought is not personal at all. It is a pure Form” (MS 920). Already in “Some Consequences of Four Incapacities”, Peirce succeeded in defining what this “pure form” that defines “thought”, its “formant”, might be: Whenever we think, we have present to the consciousness some feeling, image, conception, or other representation, which serves as a sign. But it follows from our own existence […] that everything which is present to us is a phenomenal manifestation of ourselves. This does not prevent its being a phenomenon of something without us, just as a rainbow is at once a manifestation both of the sun and of the rain. When we think, then, we ourselves, as we are at that moment, appear as a sign. (CP 5.283)

According to Peirce, semiotics is the form of thought as well as the form of the phenomenon. The content of the mind—what is present to the consciousness—is an  W1: 75, 152, 154 & 1.559, cf. Apel (1967: 203–4).  As for example: “You will observe that under the term ‘Subject’ I include not only the subject nominative, but also what the grammarians call the direct and the indirect object, together, in some cases, with nouns governed by prepositions” (CP 4.543). 12  Peirce’s example is renowned: “the stove is black”, where “stove” is defined as sub-stantia and takes the place of the sub-jectum of the proposition of which you predicate some properties (its blackness). We can recall here Aristotle’s definition: “The subject is the one of which everything else is predicated, while it is itself not predicated of anything else. And so we must first determine the nature of this; for that which underlies a thing primarily is thought to be in the truest sense its substance.” (Metaphysics, VIII, 3, 1029. Cf. also Categories c, 5, 2). 10 11

4  The Semiotic Extended Mind (2): Phenomenology

83

effect, something that stands for something else. It is a sign and signs are interpretants distributed in the community and in the environment according to complex mediations and habit-based regularities. In this way, Peirce is able to establish a coupling between internal and external—a rainbow between the sun and the rain – that extends the mind and distributes it in network of signs ruled by the community. Indeed, with his idea of the mind as an “external phenomenon”, Peirce tackles the privilege of the mind for cognition. Mind is an effect, a “phenomenon”, “something which appears” and not the starting point for cognition. Mind emerges from something else and this “something else” is the very core of cognition. Indeed, when Peirce becomes aware that all thoughts are not only phenomena, but also signs, the “field of consciousness” becomes nothing else than the space for the manifestation of signs, a “space” which corresponds to the community of interpretants (CP 5.311). Regarding this point, Peirce was very explicit since the publishing of the anti-­ Cartesian essays: “we have no power of introspection, but all knowledge of the internal world is derived by hypothetical reasoning from our knowledge of external facts” (CP 5.265). This is the reason behind his statement that “we have no power of thinking without signs” (CP 5.265). All cognitions are signs and signs call for interpretants and interpretants are illeities distributed in the community. This is the reason why Peirce can write that “in [his] opinion it is much more true that the thoughts of a living writer are in any printed copy of his book than that they are in his brain” (CP 7.364). Here we can see that Peirce’s phenomenology fills a gap with Continental philosophy. Husserl only tackles the problems of intersubjectivity and of the “community of monads”13 very late and he comes at it starting from the transcendental subjectivity grounded on the correlation “noesis-noema”, that must be recognized also to alter-egos. On the contrary, Peirce grounded his own phenomenology—his theory of “thought that becomes present to the mind”—not on the transcendentalism of the subject, but on the transcendentalism of the community. And this holds true up to the point of defining the “real” as the object of a true belief that the community will go on to affirm in the long run (CP 5.407–8). For this very same reason, Apel (1967, 1991) has been able to describe Peirce’s thought as a transcendental philosophy without a subject, in which what makes it possible for the phenomena to become present to the mind is exactly the semiotic community: Unless we make ourselves hermits, we shall necessarily influence each other's opinions; so that the problem becomes how to fix belief, not in the individual merely, but in the community. (CP 5.378)

We face illeities and our problem is the way we connect our beliefs to these impersonal public interpretants that circulate in our communities. According to Peirce, the “subject” does not stand at the origin of the manifestation of phenomena, but it is simply a phenomenon among others. The same can be applied to the mind, which is a phenomenon just like “feelings”, “images”, “conceptions” or  The expression “community of monads” is itself symptomatic of Husserl’s approach to the problem.

13

84

The Semiotic Mind. Beliefs, Habits and Extended Cognition

“­ representations”; it is true that it manifests on the inside, but its structure is that of a sign which stands for something external. This accords perfectly with the idea of inter-­subjectivity (second person) as the condition of possibility of subjectivity (first-­person) and with the idea of an impersonal third person (the “IT” of the Encyclopaedia as a collection of norms, usages and habits) as the condition of possibility of inter-­subjectivity (see supra, Chap. 2). Moving from habit to habit, from semiotic system to semiotic system, we modify the meanings through which we attribute sense to our actions, and we act in the world, so that these steps reconfigure our minds, our thought and action skills. Individual human thought and reason are not activities that occur solely in the brain or even solely within the organismic skin-bag. This matters, because it drives home the degree to which environmental engineering is also self-engineering. In building our physical and social worlds, we build (or rather, we massively reconfigure) our minds and our capacities of thought and reason. (Clark 2008: xxviii)

Since cognition implies a constitutive intertwining between brains, bodies, things and material cultures, by reconfiguring our semiotic systems and the interpretants circulating within them, we also massively reconfigure ourselves. This is the deep meaning of the famous Peircean identification of “man” with “sign”, in which man and word “educate each other” according to a paideia relation in which to build and reconfigure new meanings also means to build and reconfigure ourselves (cf. CP 5.313). It does not seem possible to account for the cognitive accomplishments of our species by reference to what is inside our heads alone. One must also consider the cognitive roles of the social and material world. […] Cognitive processes may be distributed across the members of a social group […]. Humans create their cognitive powers in part by creating the environments in which they exercise those powers. (Hutchins 2001: 9)

For this reason, Peirce could oppose the “series of cognitions the community will always continue to re-affirm” to the “series of cognitions relative to private inward determinations” and define only the latter as the “real cognitions” (cf. CP 5.311–2). As a matter of fact, thought is distributed in the community of interpretants and mind is extended in interaction, in intersubjectivity, in material artifacts and, more generally, in all the signs that we use to make sense of our experience and that define the conditions of possibility of our cognition. In the semiotic phenomenology of Peirce, the subject’s point of view is by no means a light that makes visible the phenomena, as in Husserl’s phenomenological intentionality or as in Heidegger’s Time and Being,14 but it is itself a derived function of the field of manifestation of phenomena that has nothing to do with the mind, with the subject or with intentional consciousness. This will open the field of phaneroscopy, in which the mind is nothing else than a phenomenon among others and is subjected to the theater of manifestation of phenomena that is represented by the community. For Peirce, the community has the same function that the intentional  In Time and Being (1927), the Dasein is casting his light (being, Sein) on the object (Seinde). Cf. Paolucci 2010: Chap. 5.

14

4  The Semiotic Extended Mind (2): Phenomenology

85

consciousness or the transcendental subjectivity has for Husserl: it establishes the conditions for the emergence of phenomena, i.e. “thoughts”. According to Peirce, the “subject” does not stand at the origin of the manifestation of phenomena but is simply a phenomenon among others. The same applies to the mind, which is a phenomenon just like “feelings”, “images”, “conceptions” or “representations” are. It is true that the mind manifests on the inside, but its structure is that of a sign which stands for something external. In fact, Peirce affirms that the conception of the internal mind can be compared to a conception of electricity as contained in the metallic wire, which was current until it was understood that that was the only place in which it could not be found. The same thing goes for the mind, that was thought to be situated in interiore homine, while this is the only place where the mind cannot be found. What is then the “mind”? By “mind”, Peirce means the field of manifestation of phenomena that makes cognitions—including feelings, emotions, beliefs, representations and habits—possible. As for the metallic wire, Peirce says, this field was thought to be located inside the subject, but it is not. The mind is only the phenomenon, that is, the sign of something external from which it is inferred and of which the states in interiore homine are an effect. As the metallic wire, the internal mind with its internal states is just a conductor that stands for something else that determines the internal states like emotions, beliefs, representations, habits, etc. This is why the mind is indeed an “external phenomenon”, something that unfolds and unveils in interiore homine, but which identity stands for something else, a “something else” which is the community and the external interpretants that make identity possible. For this reason, Peirce defines the community as “The Interpreting Mind” (CP 8.177): it is in fact the community that is responsible for defining that field of the manifestation of phenomena that, following Cartesian usage, we usually call “mind”. The internal mind with its cognitions is an “external sign”, something that stands for something else and to which it refers. According to cognitive semiotics, the cognition present to the mind (personal level) has a semiotic structure, which refers it to a sub-personal and a supra-personal level that strongly determine cognition at the personal level. Hence the construction of a “Semiotic Mind”, according to which the “internal”, “in the head” mind is nothing but an epiphenomenon that stands for something else. “Mind” is somewhat like a computer screen: what is shown on a computer screen has its origins elsewhere and continues to work even if the screen is turned off. However, using the computer without the screen can be extremely problematic for the user. The same thing goes for what we call the mind. It has its origins elsewhere but using our cognitive functions without a mind can be extremely difficult for us. However, Cartesian cognitive science seems to merge the screen with the computer, and it is not by chance that the mind has been identified with the software, not with the screen. It should by now be clear enough that Peirce and cognitive semiotics do not deny that the mind is also something internal. Peirce only says that this internal mind is the phenomenon—the sign—of something else to which it refers, and which makes it possible. In short, by using the expression “external mind”, Peirce does not want to say that the mind is external, but that the mind is something that cannot be defined

86

The Semiotic Mind. Beliefs, Habits and Extended Cognition

using the opposition internal/external. He only uses the expression “external mind” with a polemic intent, because he is against a whole set of old-standing positions according to which the mind is internal, situated in interiore homine. Semiotically speaking, as we have seen in the previous chapter, the internal/external opposition is a participative one,15 in which the term “external” is the extensive term that functions both as an opposing term (the mind is external because Peirce wants to attack those who think at it as internal) and as the totality of the category (the mind is something that cannot be defined using the internal/external opposition). As a matter of fact, to Peirce the mind is neither internal nor external, since it is neutral in respect to the opposition of internal versus external. This is why Peirce says that if you cut out someone’s brain, a person would be unable to express himself, but that the same thing would happen with the inkstand, that, if filched, would prevent the continuation of his discussion until he had got another. Peirce argues against the idea of localizing the mind. The mind cannot be localized, because it is a distributed external phenomenon grounded on a Hypothesis of Cognitive Impartiality. Our problem-solving performances take shape according to some cost function or functions that, in the typical course of events, accord no special status or privilege to specific types of operations (motoric, perceptual, introspective) or modes of encoding (in the head or in the world). [...] It states that the biological control system does not care about differences of location or type of resource but simply uses whatever it can relative to some cost-benefit trade-off, to get the job done.” (Clark 2008: 21–22).

Indeed, in Peirce’s cognitive semiotics this Hypothesis of Cognitive Impartiality is not a purely theoretical formulation at all, since the semiotic laws of the distribution of thought on paper have been already formulated in his famous theory of existential graphs. This theory is, to our knowledge, the most advanced theory that is able to show the parity between “thinking on paper” and “thinking in the head”. The graphs express through distribution laws on paper and through diagrammatical transformation between the relations on the paper16 the whole range of logical algebra (Alpha graphs), the relative logic (Beta graphs) and the “modal calculi” (Gamma graphs). Peircean existential graphs give greater strength to the positions of those who—like Andy Clark—support a parity principle between

 Recall that an opposition is defined as “participative” when one of the terms of the opposition can also assume, in certain contexts, the semantic value which is usually attributed to its opposing element. For example, the linguistic opposition “man versus woman” is participative because the term “man” can locally also assume the semantic value of “woman” in sentences like “Man is a rational animal”. In such cases, “woman” is defined as an intensive term, because it concentrates meaning in one single zone of the semantic category, while “man” is an extensive term, because its meaning is distributed across the whole semantic category. In the above sentence, “man” doesn’t only mean “male”, but “male+female”. Similarly, in Peirce’s definition of the “external mind”, “external” has exactly this kind of “extensive” value. Peirce is saying that the mind is external+internal and he adds that the internal part of the mind, that has been considered as the only real one, is in fact a phenomenon and thus a sign of the external part. 16  See Paolucci 2017a, Bellucci and Pietarinen 2016, Dondero and Fontanille 2012. 15

4  The Semiotic Extended Mind (2): Phenomenology

87

thinking on paper and thinking in the head and, moreover, they support the idea that the distribution of thought on paper is part of the cognitive process itself. According to Peirce, “existential graphs arise as a sharp tool to analyze inference in the logic of relatives” (Zalamea 2003: 123). In order to understand their role, we need to take into consideration the fact that, for Peirce, inference is the way through which we move from sign to sign (from cognition to cognition) and the Logic of Relatives is the discipline who studies all the possible forms of relation. Thus, according to Peirce, the system of existential graphs is a tool to analyze the way we move from cognition to cognition (inference), and it represents the distribution of cognition on paper, its extension beyond every “mental” boundary and beyond any opposition between internal and external. In the theory of existential graphs of 1906, which follows and improves the already fundamental results obtained in 1885 and 1896,17 a graph is a relation between cognitions (a relation among signs) and “scribing” existential graphs on the sheet amounts to their conjunction (asserting them on the sheet), drawing an “oval” around them amounts to their negation (detaching them from the sheet) and connecting them with “identity lines” amounts to their existential instantiation. Nice diagrammatic rules — on allowed topological maneuverings of the graphs, crossings between "ovals" and "lines", and analytic continuations — provide then diverse combinatorics between conjunction, negation and existence, paving the way to the construction of amazing logical systems, in which topological permissions correspond to logical deductions. (Zalamea 2003: 123)

We do not have enough time here to present the technical system of Peircean graphs.18 A single example could be the way in which Peirce, with great topological and mathematical elegance, formalizes the classical conjunction, negation and inferential relations through the Alpha graphs system: Alpha's inference rules (or "illations") are the following permissions, natural and symmetric, which allow introducing, moving and forgetting information in series of nested cuts: Al: ("Erasure") Any evenly enclosed graph may be erased. A2: ("Insertion") Any graph may be inserted in any oddly enclosed region. A3: ("Iteration") Any graph may be iterated (i.e., repeated) in a strict region for that graph. A4: ("Deiteratiori") Any graph whose occurrence could result from iteration may be deiterated (i.e., erased). A5: ("Double cut") A double cut may be inserted or erased around any graph in any region. […] It is easy to check that Alpha's inference rules correspond to the following valid logic rules:      •  Erasure and Insertion: rule of conjunction p ˄ q → p, and contrapositive ¬ p → ¬ (p ˄ q).

 An introduction to the Peircean existential graphs system can be found in Roberts 1973 and a wonderful reading which connects it to the theory of continuity and to the Logic of Relatives can be found in Zalamea 2003. 18  Cf. Zeman 1964, Roberts 1973 and Zalamea 2003. 17

88

The Semiotic Mind. Beliefs, Habits and Extended Cognition      •  Iteration and Deiteration: generic intuitionistic rule of negation: p ˄ ¬ q ↔ p ˄ ¬ (p ˄ q).      •  Double Cut: classical rule of negation: ¬ ¬ p ↔ p. To show how inferences work in Alpha, […] we will write Fw&G to denote that graph F illatively deduces (rules A1-A5) graph G. […]

   1. Inference of [p → (q → p)] * - 1

As Zalamea states19: Distributed cognition has some wonderful technical expressions in Peircean graphs. For instance, some conditions of distribution of classical quantifiers correspond to extensions of identity lines in the Gamma existential graphs’ system. Extended Mind and Distributed Cognition can find in such graphs a beautiful technical model.

As a matter of fact, existential graphs allow for relations between cognitions to be mapped on a sheet of paper and to show how we move from thought to thought, from cognition to cognition, from sign to sign. And this is done in a form that extends the mind outside of the boundaries within which it is usually consid19

 Personal communication. See also Zalamea 2003.

5  Semiotic Representations

89

ered conceived. As well as any cognition can be erased, replaced by other cognitions, iterated, conjoined, denied and connected with other cognitions or simply cut away, any existential graph can represent these processes through its diagrammatic system. With his system of graphs Peirce is able to build a perfect identity between thought and space, building a perfect concrete instantiation of the cognitive impartiality that grounds his semiotic extended mind.

5  Semiotic Representations What happens to the concept of “representation” in a pragmatist framework in which thought is a form of action connected to signs, meaning and habits and not a set of mental states in the “extended mind”? The debate on representations is harsh and very complex nowadays. However, cognitive semiotics has to say something on this topic, since Peirce identified representations with signs, but not vice versa. This identification is a key point that calls for explanation. Of course, since cognitive semiotics is tied to pragmatism, there is nothing further from it than that view of human cognition so purified and so detached from the world that is often connected to a representationalist point of view, “a disembodied input-output device characterized by abstract, higher-level logical operations” (Malafouris 2013: 29). When cognitive science was born20 in the 50s, its revolutionary object was cognition, a level of “mental representation” that was analyzed at a wholly separate level from the biological or neurological on the one hand, and the sociological or cultural, on the other. Mental representation was that level of internal mediation between input and output, perception and action that was largely ignored by behaviorism and could be described as language-like components and language-like structural composition, composed and manipulated according to purely formal criteria.21 There are many different positions on this notion of representation nowadays. As Rowlands (2012: 133) pointed out very rightly, “the idea of a mental representation has come to mean too many different things to too many different people”. However, taken for granted their huge variety, “representations” have always been thought as constructed internal “images” of something external (see Ramsey 2007; Chemero 2009; Clark 2016; Clowes and Mendonça 2016; Hutto and Myin 2017). Mental or "internal" representations can be understood as referring to the representational content of a certain intention or belief about the world. Mental representations, in other words, provide an internal intracranial structure that makes it possible for the objects of

 The plural form “cognitive sciences” start having a currency only a few decades later, following Gardner’s (1987) still insightful reconstruction, that individuates the birth of cognitive science in two main conferences: “Cerebral Mechanisms in Behavior” held in September 1948 at CalTech, to which von Neumann, McCullough and Lashley participated; and “Symposium on Information Theory” held at MIT in September 1956, to which Newell, Simon, Chomsky, Miller, Bruner, Goodnow and Austin took part. 21  See Fusaroli (2011: 99), Fusaroli and Paolucci (2011). 20

90

The Semiotic Mind. Beliefs, Habits and Extended Cognition thought and perception to be present inside the cognitive system […]. More precisely, representation is conceived as the principal disembodied mechanism by which we feed our brains with information from the world, process that information, then externalize our mental contents into the world. On that construal, when we perceive, classify, remember, or simply think about an object X, we don't think about the “real” object; rather, we think about its internalized representational substitute or replicator, X’. (Malafouris 2013: 26–7)

As we have seen, the semiotic idea of “representation” has nothing to do with all that. Therefore, it may be extremely useful to take the semiotic idea according to which internal representations are signs erroneously confused “with the inward aspects of things” seriously. In two very important papers written in the context of a debate with Mark Rowlands (2006, 2012), Shaun Gallagher (2008, 2015) singled out six constitutive traits of the idea of representation, which “find agreement in both proponents and critics of representation” (Gallagher 2015: 3): 1. Representation is internal (image, symbol, neural configuration). 2. Representation has duration (a representation is a discrete identifiable thing). 3. Representation bears content that is external to itself (its content refers to or is about something other than itself—representations involve intentionality). 4. Representation requires interpretation—its meaning derives from a certain processing that takes place in the subject. 5. Representation is passive (it is produced, enacted, called forth by some particular situation). 6. Representation is decouplable from its current context (it can be part of an offline process; it can stand in for something that is not present). Since there is no agreement on the necessity of all these six criteria, we cannot take these as necessary and sufficient conditions.22 For instance, Ramsey (2007: 18) does not agree on point 5 and claims that representations are often considered causally active, “mental states that do various things”. However, my aim is simply to show that all representations are signs, as sustained by Peirce, but not all signs are representations, since they do not present most of these features. As far as point (1) is concerned, we have already shown why a semiotic representation is not internal at all. So, there is no need to come back on the ideas that (1) representation are signs, that (2) signs are phenomena we have present to the consciousness, and that (3) the mind is an external sign mistaken for the inward aspect of things. As far as point (2) is concerned, Peirce claimed that his cognitive semiotics was the first part of his theory of synechism, that consisted in affirming the continuity between “mind” and “matter”, between “thought” and “world”, arguing that the systems that are not usually conceived as cognitive obey the same semiotic rules that regulate the cognitive ones. Peirce calls his synechism “The Law of Mind” (CP 6.102–6.163). A clear example is provided by habits: “habit is by no means exclusively a mental fact. Empirically, we find that some plants take habits. The stream of water that wears a bed for itself is forming a habit” (CP 5.492). But Peirce also 22

 For a discussion, see Caravà 2019.

5  Semiotic Representations

91

affirms that there is thinking going on in the dance of bees and in the morphology of crystals. However, the sign can of course be a discrete entity, having duration and being an identifiable thing. The opposition between continuous and discrete has always been a tricky one (see Paolucci 2004; Serres 1980); signs can be discrete entities as well as they can be continuous ones. With regard to point (3), signs can bear content, but they do not necessarily have to do so, if by “content” we mean truth conditions that connect the signs with the world. Signs bear meaning, but, as we have already said, cognitive semiotics does not think meaning in terms of contents connected to truth conditions at all. Signs always bear interpretants, habits and dispositions to act. They can also bear content, because, in order to act in the world in a proper way, we sometimes need to represent it. But, for instance, as far as basic minds are concerned, semiotic representations do not bear content at all (see Hutto and Myin 2017; Paolucci 2020), and all external signs connected to material culture cannot be explained in terms of content. Respecting point (4), signs, like representations, always require interpretation. However, a sign’s meaning does not derive from a certain processing that takes place in the subject, but from external interpretants distributed inside the community. The word “interpretant” chosen by Peirce in order to name the knowledge that builds up cognition is not a random one. They are “interpretants” because they interpret the world before we interpret the world and they orientate our own interpretation. As to point (5), signs are not passive at all. Signs install habits of action, manipulate, do various things and make the others do other things. Signs have their own material agency and their own semiotic intentionality. As far as point (6) is concerned, all signs stand for something, but this does not always imply decoupability, that is, the standing in for something that is not present. This is a crucial point, often misunderstood when semiotics is concerned. The decoupability condition is not always present as far as signs are concerned, as we are going to see in the chapter on the semiotics of perception. Indeed, when I see something in front of myself, the object does not stand for itself: it is itself, it is the object. But this does not mean that interpretants, meaning and habits—the three key words for a cognitive semiotics approach to cognition—are not at work also in perception (see infra, Chap. 5). The “standing for” of the sign does not imply the decoupability conditions: indexes are signs, but they are not decoupable from the objects. The weathervane indicating the direction of the wind stands for the wind, but it is a sign of it exactly because it is not decoupable from it. Put it inside the house, decoupling it from its object, and it will stop to be a sign of the wind. Summing up, as far as signs are concerned, only two conditions of representations hold firmly: the “standing for” condition (6) and the “requiring interpretation” (4) one. But, if we remove decoupability and individual processing, “standing for” and “requiring interpretation” are the properties of every sign as a sign and not the properties of a representation. Internal representations are a somehow misleading attempt to give account of the intertwining of the human mind with the brain and the material world by way of a more familiar model, that of the external sign. Indeed,

92

The Semiotic Mind. Beliefs, Habits and Extended Cognition

external signs “stand for” and “need interpretation” too; smoke stands for fire according to some point of view, but it can also be produced by a cinema team while shooting, due to esthetic reasons. This is why signs ask for interpretations, since they can always be used in order to lie. Every sign is something that stands for something else according to some interpretants. This is why Peirce could say that representations are signs; the only properties of representations that hold—when cognition is thought through a cognitive semiotics perspective—are the properties of the sign. Since we live in a world full of external representations, where material culture introduces external signs to which many cognitive activities are attributed, the semiotic model of external signs has been borrowed in order to account for the internal one. However, as far as the internal sign—the “rainbow between the sun and the rain”—is transformed into a cognitive representation, a series of further conditions are introduced. This is why claiming that representations are signs does not mean also claiming that signs are representations. Since signs are neither passive nor internal, since they do not always fulfill the decoupability condition and since they do not always bear content, signs are not representations. Peirce calls them “cognizables” and, since they can be used in order to lie during strategic action, they bring forth worlds with the aim of acting effectively in our environments. To represent the world in the classical vocabulary of cognitive sciences means to narrow and to impoverish a semiotic world where cognition is tied to interpretants, habits and meaning. This semiotic world is not an internal and passive world decoupled from an environment that it represents, and that it misrepresents. The semiotic world is a sense-making world where values are established through habits and meaning, in order to act in an effective way inside semiotic niches. We have thus understood not only why signs are not representations, but also why representations are signs. In order to explain cognition effectively, the only properties we need are those distinctive of the sign, namely the “standing for” and “interpretability” through sub-personal external signs. As Material Engagement Theory claims, “the only representations that count and that may have any true significance in the study of the mind are those to be found, beyond the skin and the skull, in the outside world. For all these reasons, it would be fair to say that Material Engagement Theory requires us to rethink the idea of representation radically, but not necessarily to abandon it altogether” (Malafouris 2013: 31). I hope cognitive semiotics have helped us reconsider the idea of representation radically; representations are signs, but signs are not representations.

References Adams, F., & Aizawa, K. (2008). The bounds of cognition. Oxford: Blackwell. Adams, F., & Aizawa, K. (2010). Defending the bounds of cognition. In R. Menary (Ed.), The extended mind (pp. 67–80). Cambridge, MA: MIT Press.

References

93

Aizawa, K. (2010). The coupling-constitution fallacy revisited. Cognitive Systems Research, 11(4), 332–342. Alaĉ, M. (2011). Handling digital brains: A laboratory study of multimodal semiotic interaction in the age of computers. Cambridge, MA: MIT Press. Apel, K.-O. (1967). Der Denkweg von Charles S. Peirce: Eine Einführung in den amerikanischen Pragmatismus. Frankfurt: Suhrkamp. English Edition: Apel, Karl-Otto. 1995. Charles S. Peirce: From Pragmatism to Pragmaticism, Atlantic Highlands, New Jersey: Umanity Press International. Apel, K.-O. (1991). Le problème de l’evidence phènomènologique à la lumière d’une sèmiotique transcendentale. Critique, 46, 79–113. Trad. J. Poulain. Bellucci, F. (2017). Peirce’s speculative grammar. Logic as semiotics. London and New  York: Routledge. Bellucci, F., & Pietarinen, A.  V. (2016). Existential graphs as an instrument of logical analysis: Part I. Alpha. The Review of the Symbolic Logic, 9, 209–237. https://doi.org/10.1017/ S1755020315000362. Bergson, H. (1889). Essais sur les données immédiates de la conscience. Thèse pour le doctorat présentée à la Faculté des lettres de Paris. Paris: Félix Alcan, coll. «Bibliothèque de philosophie contemporaine». English Edition: Bergson Henri. 2001. Time and Free Will: An Essay on the Immediate Data of Consciousness. New York: Dover Publications. Calcaterra, R. (2006). Semiotica e fenomenologia del sé. Roma: Aragno. Caravà, M. (2019). The threshold of representations: Integrating semiotics and the cognitive sciences. VS˗Quaderno di Studi Semiotici, 128(1), 157–174. https://doi.org/10.14649/94104. Chemero, A. (2009). Radical embodied cognitive science. Cambridge, MA: The MIT Press. Clancey, W. J. (2009). Scientific antecedents of situated cognition. In P. Robbins & M. Aydede (Eds.), The Cambridge handbook of situated cognition (pp.  11–34). New  York: Cambridge University Press. Clark, A. (2008). Supersizing the mind. Oxford: Oxford University Press. Clark, A. (2010). Coupling, constitution, and the cognitive kind: A reply to Adams and Aizawa. In R. Menary (Ed.), The extended mind. Cambridge, MA: The MIT Press. Clark, A. (2016). Surfing uncertainty. Oxford: Oxford University Press. Clark, A., & Chalmers, D. (1998). The extended mind. Analysis, 58(1), 7–19. Clowes, R.  W., & Mendonça, D. (2016). Representation Redux: Is there still a useful role for representation to play in the context of embodied, dynamicist and situated theories of mind? New Ideas in Psychology, 40(A), 26–47. https://doi.org/10.1016/j.newideapsych.2015.03.002. Colapietro, V. M. (1989). Peirce’s approach to the self. A semiotic perspective on human subjectivity. New York: Albany State of New York University Press. Daddesio, T. D. (1995). On minds and symbols. The Relevance of cognitive science for semiotics. Berlin: De Gruyter. Deleuze, G. (1966). Le Bergsonisme. Paris: PUF.  English Edition: Deleuze, Gilles. 1988. Bergsonism. New York: Zone Books. Dondero, M. G., & Fontanille, J.. (2012). Des images à problèmes. Le sens du visuel à l’épreuve de l’image scientifique. Limoges: Pulim. Eco, U. (1988). Il pendolo di Foucault. Bompiani: Milano. English Edition: Eco, Umberto. 1989. Foucault’s Pendulum. Translated by William Weaver. London: Secker & Warburg. Eco, U. (1997). Kant e l’ornitorinco. Milano: Bompiani. English Edition: Eco, Umberto. 1999. Kant and the Platypus: Essays on Language and Cognition. Translated by Alastair McEwen. London: Secker & Warburg. Eco, U. (2000). Sulla letteratura. Milano: Bompiani. English Edition: Eco, Umberto. 2004. On Literature, Boston: Houghton Mifflin Harcourt. Fabbrichesi, R. (2016). Peirce, Mead, and the theory of extended mind. Commens. The Digital Companion to C.S.  Peirce. Retrieved November 2016, from http://www.commens.org/ encyclopedia/article/fabbrichesi-rossella-peirce-mead-and-theory-extended-mind.

94

The Semiotic Mind. Beliefs, Habits and Extended Cognition

Fabbrichesi, R., & Leoni, F. (2005). Continuità e variazione. Leibniz, Goethe, Peirce, Wittgenstein. Con un'incursione kantiana. Roma: Mimesis. Fusaroli, R. (2011). The social horizon of embodied language and material symbols. VS-Quaderni di Studi Semiotici, 112–113, 97–123. Fusaroli, R., & Paolucci, C. (2011). The ‘external mind’: An introduction. VS-Quaderni di Studi Semiotici, 112–113, 3–30. Gallagher, S. (2005). How the body shapes the mind. Oxford: Oxford University Press. Gallagher, S. (2008). Are minimal representations still representations? International Journal of Philosophical Studies, 16(3), 351–369. Gallagher, S. (2011). The overextended mind. VS-Quaderni di Studi Semiotici, 112–113, 57–68. Gallagher, S. (2015). Re-presenting representation. Philosophical Inquiries, 3(2), 1–14. Gallagher, S. (2017). Enactivist interventions. In Rethinking the mind. Oxford: Oxford University Press. Gallagher, S. (2020). Action and interaction. Oxford: Oxford University Press. Gallagher, S., & Zahavi, D. (2008). The phenomenological mind. New York: Routledge. Gallagher, S., Mastrogiorgio, A., & Giorgio, P. (2019). Economic reasoning and interaction in socially extended market institutions. Frontiers in Psychology, 10, 1856. https://doi. org/10.3389/fpsyg.2019.01856. Gardner, H. (1987). The mind’s new science. New York: Basic Books. Gleick, J. (1992). Genius: the life and science of Richard Feynman. New York: Pantheon Books. Gibson, J. (1979). The ecological approach to visual perception. Boston: Houghton Mifflin. Greimas, A. J. (1983). Du sens II. Essais sémiotiques. Paris: Seuil. Havenel, J. (2008). Peirce’s clarifications of continuity. Transactions of the Charles S.  Peirce Society, 44(1), 86–133. https://doi.org/10.1353/csp.0.0001. Heidegger, M. (1927). Sein und Zeit. Tübingen: Max Niemeyer Verlag. English edition: Heidegger, Martin. Being and Time. 1962. London: SCM Press. Hjelmslev, L. (1959). Essais linguistiques (Vol. 12, Italian ed.). Travaux du Cercle Linguistique de Copenhague. Hookway, C. (2009). Logic and the natural history of thought. International conference Ripensare il pragmatismo. Milan: University of Milan. Retrieved October 7–8 2009. Hutchins, E. (2001). Distributed cognition. In N. J. Smelser & P. B. Baltes (Eds.), International encyclopedia of the social and behavioral sciences (pp.  2068–2072). New  York: Elsevier Science. Hutto, D. D. (2013). Why believe in contentless beliefs? In N. Nottelmann (Ed.), New essays on belief: Constitution, content and structure (pp. 55–74). New York: Palgrave Macmillan. Hutto, D. D., & Myin, E. (2013). Radicalizing enactivism. Basic minds without content. Cambridge, MA: MIT Press. Hutto, D. D., & Myin, E. (2017). Evolving enactivism. Basic minds meets content. Cambridge, MA: MIT Press. Iliopoulos, A. (2019). Material engagement theory and its philosophical ties pragmatism. Phenomenology and the Cognitive Sciences, 18(1), 39–63. Kirchhoff, M. D. (2012). Extended cognition and fixed properties: Steps to a third-wave version of extended cognition. Phenomenology and the Cognitive Sciences, 11(2), 287–308. Malafouris, L. (2013). How things shape the mind. A theory of material engagement. Cambridge, MA: MIT Press. Menary, R. (2018). Cognitive integration: How culture transforms us and extends our cognitive capabilities. In A. Newen, L. De Bruin, & S. Gallagher (Eds.), The Oxford handbook of 4E cognition (pp. 187–215). Oxford: Oxford University Press. Paolucci, C. (2004). Piegature della continuità: semiotica interpretativa e semiotica generativa. In Versus (Vol. 97, pp. 111–149). Milano: Bompiani. Paolucci, C. (2010). Strutturalismo e interpretazione. Ambizioni per una semiotica minore. Milan: Bompiani.

References

95

Paolucci, C. (2011). The ‘external mind’: Semiotics, pragmatism, extended mind and distributed cognition. VS-Quaderni di Studi Semiotici, 112–113, 67–94. Paolucci, C. (2016). Umberto Eco. In Tra ordine e avventura. Milan: Feltrinelli. Paolucci, C. (2017a). Eco, Peirce, and the anxiety of influence: The most Kantian of thinkers. In S. G. Beardsworth & R. E. Auxier (Eds.), The philosophy of Umberto Eco (pp. 251–278). Chicago: Open Court. Paolucci, C. (2020). A radical Enactivist approach to social cognition. In: The Extended Theory of Cognitive Creativity, Cham, Springer, 2020, 59–75. Parker, K. A. (1992). The Principle of Continuity in C.S. Peirce’s Phenomenology and Semeiotic, PhD Dissertation submitted to the Faculty of Graduate School of Vanderbilt University, Nashville, Tennessee. Ramsey, W. M. (2007). Representation reconsidered. Cambridge: Cambridge University Press. Ricoeur, P. (1977). La sémantique à l'action. Paris: Editions du Centre national de la recherche scientifique. Roberts, D. D. (1973). The existential graphs of Charles S. Peirce. The Hague: Mouton. Rowlands, M. J. (2006). Body language: Representation in action. Cambridge, MA: MIT Press. Rowlands, M. J. (2012). Representing without representations. Avant: Trends in Interdisciplinary Studies, 3(1), 133–144. Schwitzgebel, E. (2010). Acting contrary to our professed beliefs, or the Gulf between occurrent judgment and dispositional belief. Pacific Philosophical Quarterly, 91, 531–553. https://doi. org/10.1111/j.1468-0114.2010.01381.x. Serres, M. (1980). Hermès V. Le Passage du Nord-Ouest. Paris: Les Éditions de Minuit. Stalnaker, R. (1998). On the representation of context. Journal of Logic, Language and Information, 7, 3–19. https://doi.org/10.1023/A:1008254815298. Starobinski, J. (1974). Trois Fureurs. Paris: Gallimard. Sutton, J. (2010). Exograms and interdisciplinarity: History, the extended mind, and the civilizing process. In R. Menary (Ed.), The extended mind (pp. 189–225). Cambridge, MA: MIT Press/ Bradford. Volli, U. (1994). La cicatrice di Odisseo. Il Piccolo Hans, 79–80(Autunno-Inverno), 162–194. Zalamea, F. (2003). Peirce’s logic of continuity: Existential graphs and non-Cantorian continuum. The Review of Modern Logic, 9(29), 115–162. Zeman, J. J. (1964). The Graphical Logic of C. S. Peirce, PhD Thesis, University of Chicago.

Social Cognition and Autism Spectrum Disorders: From Mindreading to Narratives

In this chapter I will work on the crucial role of semiotic narratives for social cognition. I will try to stress the key position that a semiotic theory of narrativity can occupy in explaining our capacity of attributing meaning to the others’ actions and understanding, through semiotic interactions analysis, social cognition’s impairments like Autism Spectrum Disorders in infants. I will be inspired by Shaun Gallagher and Daniel Hutto’s (2008) Narrative Practice Hypothesis (NPH) and I will try to develop the Narrative Practice Hypothesis using insights from the semiotic tradition. Indeed, according to Hutto (2008, 2009), narratives are “representational artefacts”. However, a distinction between narratives and narrativity can be introduced and while narratives can be thought of as representational artefacts (Gallagher and Hutto 2019), semiotic narrativity cannot, since it is neither an artefact, and much less a representational one.1 In this chapter, I will outline what semiotic narrativity is and what its role can be for social cognition and Autism Spectrum Disorders.

1  Early Mindreading and Narrative Practices Things have changed a lot in the last 10 years inside the debates about mindreading and social cognition. Mindreading skills were originally connected with the ability to pass a linguistic false belief test around the age of four. The idea of grounding a human child’s ability to understand and represent another’s false belief in her narrative competency, without the need to bring mindreading in, made a lot of sense for children who have been shown to successfully pass the standard false belief task at that age. As Shaun Gallagher and Daniel Hutto (2008: 25) have stressed, “it is

1  On the “performative function” of narratives, thought as “experience with” and irreducible both to representations and to the “experience of” something, see Matteucci 2019: 76–9.

© Springer Nature Switzerland AG 2021 C. Paolucci, Cognitive Semiotics, Perspectives in Pragmatics, Philosophy & Psychology 24, https://doi.org/10.1007/978-3-030-42986-7_4

97

98

Social Cognition and Autism Spectrum Disorders: From Mindreading to Narratives

n­ otable that many false-belief tests are presented in the form of a narrative and could be interpreted as tests for a certain level of narrative competency”. In the last 15 years, starting from the Onishi and Baillargeon (2005) experiments, early mindreading entered the scene, becoming little by little the core business of ongoing research in social cognition. Indeed, the distinction between implicit and explicit knowledge has played an increasingly important role not only in cognitive development, but in cognitive science at large. There are now several studies suggesting that implicit knowledge precedes explicit understanding. One important source of evidence is the finding that children learn to perform particular tasks before they can understand and talk about what they are doing.2 Moreover, false belief tasks seem to require many other cognitive abilities in addition to mindreading skills. For this reason, implicit mindreading could be at play for a child’s understanding long before they have the explicit capacity to pass a false belief linguistic task that is structured in a narrative form. Many experiments during these last 10 years are supposed to show that this is the case. According to Onishi and Baillargeon (2005), 15-month-old children have the same mindreading skills adults have, even if in an “implicit” and “more rudimentary” form. According to Onishi and Baillargeon, mindreading abilities simply refine and specialize themselves during the development, but they are already at work in infants before the acquisition of language and of those “explicit” skills that allow them to pass a false belief task. This position, shared also by Carruthers (2013) and Leslie (2005), has been challenged.3 For instance, Ruffman and Perner (2005a: 215) think that infants need longer looking time simply because they must form a new association between the box, the actor and the object if compared to the one “still sustained in the frontal cortex when babies are exposed to the test stimuli”. They also think that babies are simply using behavior rules and do not appeal to others’ mental states (goals, perceptions and beliefs) to make sense of their actions (Ruffman and Perner 2005b: 463). A comparable interpretation grounded on behavioral abstractions concerning theory of mind experiments in primates has been provided by Daniel Povinelli4 (Povinelli and Vonk 2004; Gallagher and Povinelli 2012). In any case, even if we think that early-mindreading deals with beliefs,5 there is certainly no need to buy Onishi and Baillargeon’s (2005: 257) idea that “children are born with an abstract computational system that guides interpretation of other’s behavior”. As I will claim here, mindreading can be a very specific skill that children develop from interaction with their caregivers and from other cognitive skills  For an overview, Clements and Perner 1994, Ellis 2009.  Low and Wang (2011) extended to infant data the original idea on mindreading in primates by Povinelli and Vonk (2004): no single behavioral experiment has been designed that can be uniquely explained by a mindreading account rather than behavioral rules. Shortly said, there are no experimental protocols intended to test mindreading capacities that could not also in principle be explained by the use of purely behavioral, non-mentalistic rules. 4  See also Povinelli and Vonk (2004) and, for the so called “Povinelli’s problem”, Lurz et al. (2014) and Andrews (2015: chap. 6). 5  See infra, § 8. 2 3

2  Interaction Theory and Narrative Practices

99

before their 15 months of life. It has been argued that while explicit theory of mind tasks involve a fully-fledged theory of mind that operates on belief representations, implicit tasks recruit a minimal theory of mind that relies on some simpler states that are not belief representations, but make it possible to encode relations between the agent and its environment (Butterfill and Apperly 2013, see also Kovacs 2016). Obviously, there is no general agreement on the interpretation of these data, but both the anticipatory looking and the violation of expectation methodologies,6 which are grounded on eye-tracking techniques, allow us to go further than the original false belief tests, which assess only explicit understanding, in the sense that children are asked a question about the protagonist’s belief or action that they have to answer. Both methodologies have been tested also on primates (Martin and Santos 2014, 2016; Krupeneye et al. 2016), showing that human infants and some nonhuman animals are actually able to solve some theory of mind tasks (cf. Apperly and Butterfill 2009; Butterfill and Apperly 2013; Buttelmann et al. 2017). Is this early mindreading a challenge for the Narrative Practice Hypothesis (NPH) (Hutto 2008, 2009; Gallagher and Hutto 2008)?

2  Interaction Theory and Narrative Practices The NPH is part of a larger enactive account of interaction sometimes referred to as “Interaction Theory (IT)” (Gallagher and Zahavi 2008; Gallagher and Hutto 2008). Inside this theory, NPH plays the role of an externalist way of accounting for the kind of third-person “high level” ways of thinking about others’ intentional states that explain ostensible behavior, like beliefs, intentions and desires (see infra, § 4). One claim of IT is that in early-mindreading experiments (both in infants and primates), we are not dealing with beliefs or other folk psychological states at all. IT claims that when the infants or the monkeys see that the agent is not in the position to see the shift of the toy from box A to box B, they are not inferring that the agent has a false belief about the location (Gallagher and Povinelli 2012: 147). On the contrary, “seeing what the other is doing is framed in terms of the possible actions I would take if I were to engage with that agent”, without requiring the infant to “infer the agent’s mental states (Gallagher and Povinelli 2012: 149). Since infants normally spend their entire first year interacting with others, they gain through interactions all those skills that are sufficient to handle meaning in situations like the ones at play in early-mindreading experiments. During primary intersubjectivity (1–9 months of age), human infants, as well as some other social animals (Myowa-­ Yamakoshi et al. 2004), are already able to see bodily movements as goal-directed, 6  Here is the general scenario of those experiments: (1) Agent puts toy in A, or sees toy put in A. (2) Infant sees that the agent can see where the toy is. (3) The toy is shifted from A to B, but the agent is not in a position to see this. (4) Infant sees that the agent couldn’t see the shift. (5) When the agent still looks in B, there is a violation of the expectation of the infant. Infants’ looking time and places are recorded through eye-tracking techniques.

100

Social Cognition and Autism Spectrum Disorders: From Mindreading to Narratives

meaningful intentional movements and to perceive other persons as agents, perfectly discriminating a biological movement from other kinds of movements (Simion et  al. 2011). During their embodied interactions, infants perform proto-­mimesis, emotional interchange, imitation and parse perceived intentions, interpreting the actions and the expressive movements of the others through meaning (Zlatev 2008). Later, during secondary intersubjectivity (after 9  months), children gain abilities like joint attention, joint actions and the reference to pragmatic contexts of action and this is all that is needed in order to explain early-mindreading experiments. Without appealing to mental states nor to behavioral rules, IT suggests that infants perceive the world and the others in terms of how the infant can act and interact with the agent and the world. According to Gallagher, this can be seen in some of the more interactive experimental designs used to study infants’ social cognition. In a study by Buttelmann et al. (2009) 18-month-olds try to help an agent retrieve a toy while taking into account the fact that the agent doesn’t know about a switched location (the false belief situation). In that situation, when the agent focuses on the wrong container (the original location, A), the infant is ready to lead him to the correct box (B), but not in the situation when the agent does know about the switch, i.e., the true belief situation, and still goes to A. In the latter case the infant goes to assist the agent at A. (Gallagher and Povinelli 2012: 149)

In my opinion, this is a key experiment not only for the IT approach to social cognition, but for the whole debate on mindreading. It is significant that, since the experiment cannot be interpreted in terms of behavioural rules, the infant sees exactly the same thing in the case of true belief (when the agent knows there has been a shift from A to B) as in the case of false belief (when the agent does not know about the shift).7 The most obvious way to give an account for this result would be to introduce different mental states (“true belief” and “false belief”) in order to explain behaviour, but Gallagher denies that and interprets the experiments as an example of knowledge-ignorance: The fact that the infant knows either that the agent has seen the switch or not, plus the agent’s situated behavior with respect to A (e.g., moving to A and attempting to open it), is enough to specify the difference in the agent’s intention. […] The infant does not have to make inferences to mental states since all of the information needed to understand the other and to interact is already available in what the infant has seen of the situation. (Gallagher and Povinelli 2012: 149–50)

Buttelmann et al. (2009, 2017) explicitly deny that all of the information needed to understand the other and to interact with her is actually already available in what the infant has seen of the situation. This is why they think that their experiments

7  In their 2017 version of the experiment (with apes), Buttelmann et  al. (2017) set up a second experiment in an “ignorance control condition” (the actor didn’t know which box contained the object), demonstrating that apes were not responding based on a rule like “whenever someone is ignorant about where his object is, he will be looking for it”. One of the main results of the Buttelmann et al. (2017)‘s study is that apes behaved exactly like infants in the Buttelmann et al. (2009)‘s experiments.

2  Interaction Theory and Narrative Practices

101

cannot be interpretable in terms of knowledge-ignorance. Buttelmann et al. emphasize this. In all studies of false belief understanding, a key interpretive challenge is to distinguish an understanding of false belief from an understanding of knowledge-ignorance. This challenge applies even to the standard verbal false belief task: the child in the false belief condition might reason that the returning protagonist is ignorant about the location of the toy (which was switched when she was out of the room) and so she will just go where she saw it last (which is the correct answer). […] We also do not believe the knowledge-ignorance interpretation is a plausible one for our study. [...] The key moment is when the agent is trying unsuccessfully to get into the box, and clearly needs help. The child wants to help. What should she do? (Buttelmann et al. 2009: 341-2)

In the true belief condition, 75% of two-and-a-half-year old children correctly opened the wrong box with no toy which the agent had just tried to open, “assuming that since he knew the toy was in the other box, he must want to open this one for some reason” (Buttelmann et al. 2009: 339). In the false belief condition, 83.3% of children opened the other box, the one containing the toy, apparently assuming that the agent, who had a false belief, was trying to open the first box in order to get the toy. The percentages were respectively 84%/72% with 18-month-old infants and at 56%/80% with 16-month-old children. In this way, Buttelmann et al. (2009: 340) claimed that “infants clearly make use of their understanding of others’ false beliefs to help them appropriately”. Is there a way to interpret Buttelmann’s experiments without introducing a mental “in the head” explanation, or without thinking that the information needed to understand the other and to interact with her is actually already available in what the infant has seen of the situation? Buttelmann et al. (2017) replicated the very same experiments with great apes (chimpanzees, bonobos and orangutans), finding out the very same results they had with 16-month-old infants, with striking similarities also regarding percentages in both true and false beliefs conditions. This is why they claimed that “apes may have a basic understanding of others’ false beliefs”. According to Buttelmann et al., the main question is on what basis did infants and apes act as they do in the false belief condition. Why did they not just help the agent open the box as in the other condition? The most plausible explanation for this behavior, in our opinion, is that children imagined (as it were) a thought bubble in agent’s head containing the desired toy in the box he put it in: he is trying to get into the box because he believes the toy is in there (he saw it go in there but did not see it go out), and so that must be what he wants. If children in this false belief condition were simply attributing to the agent a blank thought bubble of ignorance, then they would have no reason to go retrieve the toy – unless one thinks that somehow the toy is especially salient on its own. But then – flipping back to the true belief control – one must explain why they do not find the toy especially salient when everything is exactly the same but the agent was in the room when its location was switched. To override the tendency to simply help the agent open the box, which is their most frequent natural response (and a kind of ‘pull of the real’), children in the false belief condition had to attribute to the agent the false belief that the toy was in there. Thus, the main logic of the current study – and what makes it a study of false belief, in our opinion – is that without an understanding of the

102

Social Cognition and Autism Spectrum Disorders: From Mindreading to Narratives

agent’s false belief, children cannot help him appropriately because they cannot know that he wants the toy. (Buttelmann et al. 2009: 342)

This is the “in the head” interpretation given by Buttelmann et al. (2009), but, even if we do not buy it, in my opinion their experiments are still not fully explainable in terms of perceptual and interaction processes of primary and secondary intersubjectivity, as IT claims. In order to act as they do, infants have to imagine two different contents driving different behaviors. Ignorance is not enough, as Buttelmann et al. (2009) have said, but in the true belief condition infants can help opening the empty box only if they imagine that the agent wants to open the empty box for some other reason, different from the one of getting the toy. The world can afford a different possibility of action only because the child bets something about the kind of cognitive content driving the agent’s behavior. That is why in the false belief condition they open the other box, imagining that the agents wants to get the toy, but in the true belief condition they help the agent to open the empty box, since they are imagining that another kind of content is driving her behavior. The idea of cognitive content driving the behavior of someone who acts for reasons seems to be needed here, although it is lacking in IT.8 However, in order to account for folk psychology competence in terms of “acting for reasons” without introducing the capabilities of inferring other’s mental states, IT has to refer to narrative practices, which cannot be involved in those kind of early-mindreading embodied interactions, since the Narrative Practice Hypothesis, in Gallagher and Hutto’s formulation, is actually linked to stories, to language acquisition and to a full-fledged development of the self. So, we face an explicatory gap. We are dealing with some kind of “high-level” practice of making sense of intentional actions in terms of reasons, involving some way of thinking about o­ thers’ intentional states that explain ostensible behavior. However, in order to account for all that without introducing some internal, “in the head”, mechanism of cognition (as Buttelmann et al. do), IT has to rely on the Narrative Practice Hypothesis, but, how can those mindreading-like skills be grounded on narrative practices, if both children and primates do not even have language or a fully developed standard narrative competence? A semiotic conception of narrativity can solve this explanatory gap and show that narrative practices do play a central role even in early mindreading skills. This will help IT to account for social cognition skills relying exclusively on narrative embodied interactions. Of course, we need to overcome a language-based conception of the Narrative Practice Hypothesis (NPH), that holds that “we only acquire a practical understanding of belief by engaging in conversation” and “it is only those linguistically competent actors that […] are the proper subjects of folk psychological narratives” (Hutto 2006: 207, 238). In order to achieve this goal, I will introduce one main founding idea of the semiotics tradition, this being that narrativity is the deep structure of meaning, so that we always make sense out of our own actions and

8  Of course, as we have usually done in this book, we do not use the term “content” in terms of truth-conditions, but in a semiotic sense.

3  Radical Enactivism, Interactive Specialization and Social Cognition

103

out of the action of others by placing them in a narrative framework, in a technical sense that I will outline below. As we will see, according to semiotics, narrativity is derived neither from meaning nor from language. On the contrary, meaning and language are built on the structure of narrativity. In this way, reframing the NPH on a semiotic ground means claiming that social cognition skills related to narrative practices develop in babies before their acquisition of language, in agreement with the proposals of Onishi and Baillargeon (2005), Southgate et  al. (2007) and Tomasello (2014). We can move thus towards a new conception of social cognition that sees mindreading as a very specific skill developed from semiotic and prelinguistic narrative practices in which language extends beyond embodied interactions9 for quite a long time before we are able to pass the “false belief” test. This idea, that I am going to develop now, can integrate and give strength to narrative theories as an alternative to the theory of mind inside the current debate of social cognition. In brief, we are going to propose a Narrative Practice Semiotic Hypothesis (NPSH), bringing together a semiotic approach with the narrative theory originally developed by Shaun Gallagher and Daniel Hutto. This asks for a radical enactivist account of social cognition, according to which narrative competence is not always connected to narratives thought as representational artefacts.

3  R  adical Enactivism, Interactive Specialization and Social Cognition I will give a radical enactivist account of social cognition, showing how we move from neurons to mindreading skills without the need of introducing any idea of mental content or internal representation, according to our cognitive semiotics perspective. This will involve three steps: (1) an enactivist interpretation of the biological basis of social cognition; (2) a semiotic reformulation of the enactivist Narrative Practice Hypothesis by Gallagher and Hutto (2008), and (3) an overview of the ontogenetic process that leads to mindreading through joint attention, semiotic competence, deception skills, pretend play and language acquisition. The expression “mindreading” (or “theory of mind”) is generally used as a “shorthand for our ability to attribute mental states to self and others and to interpret, predict, and explain behaviour in terms of mental states such as intentions, beliefs and desires” (Gallagher and Zahavi 2008: 171). For classical cognitive scientists (Frith and Happe 1999; Currie and Sterelny 2000; Malle 2002), mindreading was thought to be the prerequisite for social cognition, such that in everyday life we have to infer the mental states of the others (beliefs, desires, knowledge) in order to shape their behaviour through meaning. The idea that beliefs, desires, sensations guiding our actions depend on a specific corpus of knowledge that explains how our mental states connect and interact has been called “Theory Theory” (TT). Its name  Cf. Fusaroli et al. (2013). For a new perspective on embodiment, see Pennisi and Falzone 2017.

9

104

Social Cognition and Autism Spectrum Disorders: From Mindreading to Narratives

indicates that the specific corpus of knowledge controlling action is a kind of “theory” on the basis of which we act. Furthermore, this theory is also the basis of our reading of others’ actions, beliefs, desires and intentions; we use a “theory” about how people behave (folk psychology) in order to infer (mind-read) beliefs, desires and intentions that give meaning to others’ actions. Of course, the use of those “theories” is not always explicit and conscious, but nevertheless, it controls our attribution of meaning to all those phenomena (cf. Crane 1995). In cognitive sciences, this theory reigned undisputed for years, until the arrival of the Simulation Theory (ST) (cf. Gordon 1986; Heal 1998; Goldman 1989, 2006). ST proposes a radically different model, arguing that, in order to understand others, we use our own mind as a model and through it we simulate beliefs, desires and other intentional states that we then project onto others’ minds, in order to explain or predict their behaviours. According to both Theory Theorists and Simulation Theorists, understanding if infants and apes were capable of mindreading meant asking if they could have social cognition skills like we do (Ruffman and Perner 2005a; Apperly and Butterfill 2009; Povinelli and Vonk 2004; Buttelmann et al. 2009, 2017). A radical enactivist account of social cognition challenges the classical view of cognitive sciences, both in its Theory Theory (Baron-Cohen 1995; Wellman et al. 2001) and in its Simulation Theory version (Goldman 2006), claiming that we do not have social cognition skills in the way we thought we do. By moving towards a conception of social cognition that considers semiotic, prelinguistic practices as the basis for mindreading, we will see how language extends them beyond embodied interactions long before the explicit ability to pass a false belief test. Instead of being the condition of possibility of social cognition, mindreading can be thought as an emerging propriety of social cognition, that develops starting with embodied interactions. Our thesis is inspired by the enactivist account of social cognition by Shaun Gallagher and Daniel Hutto and by the interactive specialization framework (Johnson 2000; Simion et al. 2011). We will start with the latter. According to this neuroconstructivist point of view, “cognitive specialization is an activity-dependent and an experience-dependent process, strictly linked to the exposure to certain experiences occurring over a particular period of time, called a critical or sensitive period” (Simion et al. 2011: 174). Even if the human system begins life broadly tuned to detect social stimuli, there is no need to hypothesize dedicated abstract modules for detecting social stimuli. Newborns do not manifest any visual preference for a human face if compared to a scrambled face with more elements in the upper part (Simion et al. 2006), showing that it is not necessary to hypothesize the existence at birth of an experience-independent module of face recognition (Fig. 1). However, this general preference for some structural and mereological properties that faces share with other visual stimuli changes over development and become tuned to human faces due to the extensive and repeated experience with this kind of elements provided by the environment within the very first months of life (Scott et  al. 2007; Simion et  al. 2011). The very same three-month-old infants who, as newborns, prefered an upright stimulus with three blobs randomly located in the

4  Two Discoveries

105

Fig. 1  Faces employed to test face recognition’s preferences in newborns (from Simion and Di Giorgio 2015)

upper part over a real face, now prefer to look at faces even when they are contrasted with scrambled face configurations with more elements in the upper part (Turati et al. 2002), “corroborating the idea that 3 months of visual experience are sufficient to change and tune the face representation” (Simion and Di Giorgio 2015: 4). This results in the development of expertise, since we are generally better at discriminating between stimuli we have experienced. We share this “perceptual narrowing” phenomenon with primates as well (Sugita 2008). Something similar is supposed to go on with mindreading skills. Indeed, while some basic capacities for social navigation and interaction are undoubtedly built-in, others may be acquired or soft-assembled in ontogeny (see Hutto 2009: 19). Starting from a sub-personal action/perception/imagination matching mechanism implemented in mirror neurons and from the reward network—encouraging us to prefer the stimuli and the experiences that lead to a positive sanction—it is interaction, with its narrative logic, that turns a very general embodied matching system into a mindreading module tuned to the meaning of the actions of the others during interaction.

4  Two Discoveries In the last 20 years, there have been two discoveries that have shaken the primacy of the theory of mind in social cognition. On the one hand, the discovery of mirror neurons has led to very different accounts of social cognition, if compared to the mindreading-based ones. On the other hand, the Narrative Practice Hypothesis (NPH), grounded on a radically enactivist theory of cognition (Hutto and Myin 2017) and on Interactive Theory (Gallagher 2017; Menary 2006), has introduced an externalist way of accounting for the kind of third-person, high level ways of

106

Social Cognition and Autism Spectrum Disorders: From Mindreading to Narratives

t­hinking about others’ intentional states that explain ostensible behavior (Hutto 2006, 2007, 2008, 2009). It is not by chance that, in his fundamental paper on social cognition of 2007, Vittorio Gallese quotes Daniel Hutto in his first and in his last page, in order to move against “the traditional view in the cognitive sciences that holds that humans are able to understand the behaviour of others in terms of their mental states”. It seems preposterous to claim that our capacity to reflect on the intentions, beliefs and desires determining the behaviour of others is all there is in social cognition. [ ...] A growing sense of discomfort towards a blind faith in folk psychology to characterize social cognition is indeed surfacing within the field of philosophy of mind. It has recently been stressed that the use of folk psychology in social cognition of the belief-desire propositional attitudes is overstated (see Hutto 2004). [...] Our sophisticated mind-reading abilities probably involve the activation of large regions of our brain, certainly larger than a putative and domain-­ specific theory of mind module. My point is that these brain sectors do encompass the premotor system and, in particular, the mirror neuron system [ ... which] might also be crucial in the course of the long learning process children require to become fully competent in how to use folk psychology. This learning process greatly benefits from the repetitive exposure to the narration of stories about the actions of various characters (for a putative role of narrative practices in the development of a competent use of folk psychology, see Hutto 2004). (Gallese 2007)

The Narrative Practice Hypothesis, originally formulated by Daniel Hutto (2006, 2007, 2009) and then strengthened through the work of Shaun Gallagher (2009, Gallagher and Hutto 2008; Gallagher and Hutto 2019) claims that the normal route by which children acquire their folk psychological competence is through exposure to narratives of a special sort, those that “make explicit mention of how mental states (most prominently, beliefs and desires) figure in the lives, history and larger projects of their owners, inter alia” (Hutto 2009: 11). The core claim of the Narrative Practice Hypothesis is that direct encounters with stories about reasons for acting, supplied in interactive contexts by responsive caregivers, is the normal route through which children become familiar with both (1) the core structure of folk psychology and (2) the norm-governed possibilities for wielding it in practice, knowing how and knowing when to use it. (Hutto 2006: 231–2)

By folk psychology, Hutto means our everyday practice of making sense of intentional actions (our own and those of others) in terms of reasons (beliefs, desires etc.)10 and “denies that its acquisition depends on the existence of any kind of dedicated mindreading mechanisms. Nor is it forged by theorizing activity” (Hutto 2008: 177). Hutto’s idea (2008: 177) is that our cognitive folk psychological competence is developed by and rooted in a socially shared set of narrative practices, and that this competence “requires that children are appropriately immersed in intersubjective narrative activities and supported by others”. Rather than by reading others’ minds, by simulation or by a theory, our cognitive folk psychological competence is shaped by “stories about those who act for reasons (folk psychological narratives)”, since stories of this special kind “provide the crucial training set needed  Hutto makes an important departure from classical theories of mind explicitly excluding prediction from it and concentrating only on explanation based on reasons.

10

4  Two Discoveries

107

for understanding reasons. They do this by serving as exemplars, having precisely the right features to foster an understanding of the forms and norms of folk psychology”. (Hutto 2007: 53) The central claim of the NPH is not compatible with TT, ST or TT-ST combos where these theories seek to explain the basis of our core FP [folk psychology]-competence. If the NPH is true, FP-competence does not equate to or derive from having a Theory of Mind (ToM). […] Our minds do not literally contain the basic FP principles. The NPH eschews any crude internalizing stories that claim that whenever we learn a competence we must store it as a set of propositional rules in our ‘heads’. (Hutto 2008: 178)

This kind of position radically undermines the internalism which had characte­ rised much of the cognitive science paradigm. It is no coincidence that, for these internalist theories, what controlled actions and their meaning was exactly a theory of mind. Furthermore, the theory of mind was the very condition of possibility for intersubjectivity and for the construction of a social world, which constitutively depended on it. This kind of position was widely shared inside the scientific community: Mind-reading appears to be a prerequisite for normal social interaction: in everyday life we make sense of each other’s behavior by appeal to a belief-desire psychology. (Frith and Happe 1999: 2) It is hard for us to make sense of behavior in any other way than via the mentalistic (or “intentional”) framework. […] Attribution of mental states is to humans as echolocation is to the bat. It is our natural way of understanding the social environment. (Baron-Cohen 1995: 3–4) Mind-reading and the capacity to negotiate the social world are not the same thing, but the former seems to be necessary for the latter. […] Our basic grip on the social world depends on our being able to see our fellows as motivated by beliefs and desires we sometimes share and sometimes do not. (Currie and Sterelny 2000: 145)

In the more radical proposals, like the one by Malle (2002), a cognitive theory of mind has been considered as the equivalent of the Kantian categories for cognition and for the human social behaviour based on cognition: Theory of mind arguably underlies all conscious and unconscious cognition of human behaviour, thus resembling a system of Kantian categories of social perception — i.e., the concepts by which people grasp social reality. (Malle 2002: 267)

Once the idea of narrative competence appears, the very idea of social cognition stops necessarily concerning the mind and the processes occurring under the individual’s skin. Instead, folk psychology and, more in general, cognition starts to depend on the construction of a social world and on intersubjectivity, which both define the conditions of possibility of our inner mental states. A complementary idea is that other kinds of narrative competences enable a less mediated interpretation of the other's actions and intentions, that is, without the need for a folk psychology of mental states. Normally, coming to understand another's reasons is not a matter of comprehending their discrete “mental states” but rather their attitudes and responses as whole situated persons. I encounter the other person, not abstracted from their circumstances, but in the middle of something that has a beginning and that is going somewhere. I see them in the framework of a story in which either I have a part to play or I don't. The narrative is not primarily about what is “going on inside their heads”; it's about what is

108

Social Cognition and Autism Spectrum Disorders: From Mindreading to Narratives

going on in our shared world and about how they understand and respond to it. In this sense, our commonsense understanding of others does not consist of folk psychological theory, but of the skilful practical reasoning that depends on a developed narrative competency. (Gallagher and Zahavi 2008: 193)

It seems promising to argue that we make sense out of the action of other individuals based on what they believe, wish and know—not because we read their minds, rather because we are able to read their intentions within embodied interactions and place them in narrative frameworks (Gallagher and Zahavi 2008; Gallagher and Povinelli 2012). Is it possible to keep together this view together with Gallese’s one and move towards a new conception of social cognition that sees mindreading as a very specific skill developed from semiotic and prelinguistic narrative practices, without introducing any kind of “in the head”, representationalist view referring to mental states and contents? In order to achieve that, we will introduce a two-levels model.

5  A  Two-Levels Model. Low-Level: Action/Perception/ Imagination Matching Even if it has been interpreted as an “embodied simulation” (Gallese and Goldman 1998; Goldman 2006; Rizzolatti and Vozza 2011), the discovery of mirror neurons has shaken both the Theory Theory (TT) and the Simulation Theory (ST). It has pushed TT to a minor role and has forced ST to integrate a sub-personal level, giving birth to a hybrid view, according to which there are two routes to social cognition. The full version of Goldman’s (2006) simulation-based approach to mindreading involves a distinction between low-level and high-level tasks of mindreading achieved by two distinct kinds of processes of mental simulation; whereas so-called processes of “mirroring” (exemplified by the activity of mirror neurons) underlie tasks of low-level mindreading, the so-called processes of “enactment-­ imagination” underlie tasks of high-level mindreading (Jacob 2011). The radical enactivist account of social cognition I am defending claims that the “low-level”, grounded on mirror neurons (“mirroring”), has to be reinterpreted in a different manner and that the “high-level” (“enactment-imagination”) has to be replaced by a semiotic and narrative logic of inter-action. Aside from the interpretation in terms of ST, the neuroscientific discovery of mirror neurons basically shows us a common neurobiological ground for action, perception and imagination (Rizzolatti and Craighero 2004). It tells us that specific social cognitive endowments of our species are the evolutionary outcome of the selection of mechanisms that are not intrinsically cognitive or, at the very least, certainly not mindreading-specific (Gallese et al. 2013; Gallese 2001). They come from action and, as we are about to see, from the action’s motor and rewarding biological structures. The main claim is that key aspects of human social cognition are

5  A Two-Levels Model. Low-Level: Action/Perception/Imagination Matching

109

“underpinned by neural exploitation”; that is, the adaptation of sensory-motor and reward integrating brain mechanisms “to serve new roles in thought and language, while retaining their original functions as well” (Gallese 2007: 666, see also Gallese and Lakoff 2005). Indeed, the discovery of mirror neurons tells us that the perception of others’ actions automatically activates the same brain areas in the prefrontal cortex and in the Broca’s area that become active when we ourselves perform a similar action. Moreover, overlapping of neural areas occurs in regions of the prefrontal and parietal cortices in the following conditions: (1) when I perform an intentional action, (2) when I observe someone else performing an intentional action, (3) when I imagine myself or someone else performing that action, and finally (4) when I prepare to imitate someone else’s action (Gallese 2005; Rizzolatti and Sinigaglia 2006). All this seems to go in the direction of an enactivist theory of perception (Noë 2006) rather than towards a Simulation Theory, since it poses a common ground for perception and action, which is the strong argument made by the enactivists (Menary 2006; Gallagher 2017; Hutto and Myin 2017). If perception is an enactive, sensory-­ motor process, and not simply a process of sensory reception, the resonance processes discovered by neuroscientists in Parma can be easily considered as being part of the perceptual process when perception focuses on others’ actions. Shaun Gallagher (2006, 2017) has proposed an enactivist alternative to the ST interpretation in which mirror neuron activation is not the beginning of a simulation, rather it is part of an intersubjective perception of what the other is doing. Neural systems do not activate themselves, but they are activated by the other’s action. This is not a simulation, it is a perceptual event that is a response to the other’s action (Gallagher and Zahavi 2008: 268–279). This seems to be just the opposite of what the ST argues, namely that our mind is activated as a model in order to “put ourselves in the other’s shoes”. The main difference that we need to underline is that for the “embodied simulation” interpretation of the mirror neurons discovery, we are dealing with a “first person-third person matching (i.e. link ‘I do and I feel’ with ‘he does and he feels’)” (Gallese et al. 2004: 396). On the contrary, for the enactivist point of view, we are dealing with a “first person-second person” matching, where the subject is not an observer of others’ actions, but a “fellow character” interacting with others inside a practice. An action-oriented perspective on cognition, like enactivism is, changes the role of the individual from a passive observer (for instance, the person observing the behavior of the others in the theory of mind experiments) to an actively engaged agent interacting in a closed loop with the world as well as with others. This is a key point, because for an enactivist account of cognition, interaction has to be the “model” for observation and not vice versa. It is through understanding interactions that we will cast light to the assignment of meanings during observations, because (1) we share an embodied biological system that matches action and perception, and (2) we spend our entire first 2 years of life interacting with others. Mirror neurons are crucial for that; matching action, perception and imagination, they ground meaning in our motor system and, through habits, are preparatory for action.

110

Social Cognition and Autism Spectrum Disorders: From Mindreading to Narratives By matching a perceived motor act with those contained in one’s own motor repertoire, [the Mirror system] allows an inner, experienced, non-cognitively mediated, recognition of others’ action goal. (Gallese et al. 2013: 17)

The fact that the “recognition of the others’ goal” is “experienced” and “non-­ cognitively mediated” is the main point for an enactivist theory of social cognition and is a common view in both “third person” and “second person” interpretations of the discovery of mirror neurons. However, no traces of mindreading can be seen here, since mindreading is a cognitively mediated recognition and representation of the others’ intentions, beliefs and desires. How can we call attention to the importance of inherited, embodied practices and social interactions, without relying on the processing or manipulating of informational contents? In order to do that, we have to move to the “high-level” of our model.

6  A Two-Level Model. High-Level: Narrativity In order to understand how narrativity leads us to minds and beliefs starting from basic perceptions, emotions and embodied enactive interactions without the need of mindreading, it is important to stress that humans are creators of stories. There is no human culture which is not founded on a set of stories and that, through narratives, does not elaborate explanations and teachings related to its world and its experiences, processed in the form of myths and legends. There is no single human child that does not grow up in an environment where stories represent a fundamental phase of his growth and development, a kind of main gateway to fuller intersubjectivity, which opens up to the realm of others’ actions and the motivations that push them to act the way they do (Hutto 2007, 2009). However, while narratives are cultural products, the ability to narrate stories (narrativity) is not. Narrativity is a cognitive skill by which we shape experience through meaning (Talmy 2000; Herman 2003; Paolucci 2012). This is an everyday experience in situations like anxiety or restlessness, when something important has happened and we still cannot really focus on the main point. What do we do? We call a friend, or a girlfriend, and we need to tell her our story. This is because storytelling has its own logic and its own structure, so telling what has happened to us means that we are shaping it through the structure of narrative and shaping it through this kind of structure means giving it meaning. That is why we usually feel better when we have told our story: we have given meaning to experience through the structure of narrative. That is why narratives do not simply “represent” experience. Narratives instead shape experience through meaning, and this has a looping effect, in the sense that it gets reincorporated back into our experience (see Gallagher and Hutto 2019). The idea that narrativity is a cognitive skill by which we shape experience was originally developed by Jerome Bruner (1986, 1991), who thought that there are two modes of cognitive functioning, “each providing distinctive ways of ordering

6  A Two-Level Model. High-Level: Narrativity

111

experience.” These were logical thought (paradigmatic) and narrative thought. The first deals with truth, the second “with the broader question of how we come to endow experience with meaning” (Bruner 1986: 13). Before and after Bruner, the semiotic tradition (Eco 1979; Greimas 1983; Paolucci 2012, 2019) has shown that narrative thought has a stable pattern, which is also at play in pre-linguistic and non-­ linguistic experiences, and is a key aspect in social cognition. For the aims of this work, we claim that we make sense out of the action of others through a narrative mind. We will show (1) how a narrative mind does not depend on the acquisition of language, but develops from more basic perceptions, emotions and embodied interactions with others, especially the ones between babies and caregivers; (2) how narrativity—the deep cognitive structure that shapes narratives— permeates the interactive competencies of pre-linguistic children and social nonhuman animals; and finally, (3) how mindreading skills in human and primates come from a narrative logic of inter-action. In order to understand this narrative logic of inter-action, I will first refer to the semiotic tradition. Indeed, while NPH entrusts narratives with such a great power at the level of cognition, it does not offer or incorporate a theory of narrative competence, failing to provide a definition of what narratives are. Cognitive semiotics can help establish a definition of narrative via integration with NPH. Indeed, as Pierre Jacob (2011) pointed out, in order to accomplish its tasks, NPH has to answer two questions: (1) what is distinctive about the cognitive ability of framing and understanding a narrative, and (2) how it could be the basis of one’s understanding of other minds beyond the limits of the interactive approach, as in the enactivist theory of action and perception developing through intersubjectivity (cf. Jacob 2011: 533–4). Let’s start from the definition of narratives. What are narratives? This is a tricky question and providing a good answer to it is beyond the scope of this paper. A very minimal definition will suffice for our purposes. Larmarque tells us that for something to be a narrative “at least two events must be depicted in a narrative and there must be some more or less loose, albeit non-logical relation between the events. Crucially, there is a temporal dimension in narrative”. (Gallagher and Hutto 2008: 31)

This definition is somewhat too minimal, because the main problem concerns exactly the nature of those “non-logical relation between the events” which constitutes narratives. Cognitive semiotics is able to unpack what this non-logical relation might be. With “narrativity”, the semiotic tradition means a prelinguistic skill able to shape experience through meaning (Paolucci 2012; Lorusso et al. 2012), which is prototypically expressed in narrative texts and practices, but is at work even in non-­ narrative ones (Eco 1979). For semiotics, narrativity is a morphology with a regular pattern that structures meaning and experience. This form is irreducible to either cause-effect logic or type-token logic. The idea comes from two of the most important semioticians, the French-­ Lituanian Algirdas J.  Greimas and the Italian Umberto Eco. In The Role of the Reader (1979), Umberto Eco lays out a well-known analysis that shows not only that the structure of narrativity is also present in non-narrative texts, such as Spinoza’s Ethics, but also that narrative is constitutive of a peculiar way of ­organising

112

Social Cognition and Autism Spectrum Disorders: From Mindreading to Narratives

actions and events in time. In the semiotic view, stories provide forms of organisation able to help human cognition organise the manifold domains of experience and knowledge, each one with its set of beliefs, practices and procedures.11 Narratively accounting for facts and happenings is a way of filling the cognitive gap between general encyclopaedic knowledge of the world (e.g., the knowledge that water freezes at 0 °C) and the way that knowledge is used in a particular situation (e.g., the fact that I slipped on the ice yesterday).12 If we make a distinction between a particular experience and the understanding of some elements as tokens of abstract schemes, we can place narrativity between those two extremes, in a position of mediation. According to semiotics, narrativity plays the role of schematism (in the Kantian sense) between type and token, and between our phenomenological experience and our capacity to interpret it as the token of a type. This is exactly the “non-logical” relationship between events which constitutes the particular form through which narratives shape experience. Narrativity mediates intersubjectivity and my own personal story and that is why, as Hutto perfectly states, our own folk psychological competence is acquired through exposure to stories that demonstrate how beliefs and desires figure in the lives and projects of their owners. This addresses Pierre Jacob’s first question concerning NPH, showing what is distinctive about the cognitive ability to frame and understand a narrative. Indeed, semiotics’ idea of narrativity is an improvement of Jerome Bruner’s argument, according to which narrative thought and paradigmatic thought “are two modes of cognitive functioning”, which “each providing distinctive ways of ordering experience, of constructing reality.” The two (though complementary) are irreducible to one another. […] Perhaps Richard Rorty is right in characterizing the mainstream of Anglo-American philosophy (which, on the whole, he rejects) as preoccupied with the epistemological question of how to know truth-which he contrasts with the broader question of how we come to endow experience with meaning, which is the question that preoccupies the poet and the storyteller. […] Scientists, perhaps because they rely on familiar stories to fill in the gaps of their know­ ledge, have a harder time in practice. But their salvation is to wash the stories away when causes can be substituted for them.” (Bruner 1986: 11–3).

But obviously the cognitive side of the story is not enough. According to semiotics, narratives show a regular pattern, a deep structure which stays constant. Semioticians call it “narrativity” and think that narrativity is the form of narratives, but it is also the form of our attribution of meaning to experience. This is why semioticians think meaning has a narrative form, which is at work even in non-­ narrative texts or in social practices, institutions, etc. Inspired by Vladimir Propp’s study of the Russian folktale (1928), by Claude Levi-Strauss’ analyses of the structures of myths (1958) and by a dialogue with the philosopher Paul Ricoeur, who authored Time and Narrative, Greimas (1970, 1983)  Cf. Lorusso et al. (2012). See also Herman (2003: 165).  Cf. Danto 1985: 238. See also Mink 1978 and Talmy 2000. This fits perfectly well with Laura’s example in Gallagher and Hutto (2008: 26–8).

11 12

6  A Two-Level Model. High-Level: Narrativity

113

argued that narrativity is at the core of the organisation of meaning, since it defines a set of structural relationships involved in any course of action. According to Greimas, and to semiotic tradition, narrativity is not a property of a particular text type (tale, story etc.), but instead constitutes a pattern of organisation which can be found in any text or practice concerned with action, be it a fairy tale, a cooking recipe or a scientific paper.13 Narrativity is a particular way of shaping a process of transformations and to give meaning to experience means to organize it in a narrative way. This is why narratives cannot simply be “a special kind of representational artefacts” or “complex representations”; narratives do not represent experience, they shape it through meaning and this has a looping effect, in the sense that it gets reincorporated back into our experience.14 In the semiotic tradition, narrativity involves a set of positions called actants that can be filled by various concrete entities called actors.15 We existing individuals are actors that fill the positions of the actants, taking the positions of the other that can also be filled by other actors and not by us, which is why we gain folk psychology from narratives. We sometimes occupy the same position that can be filled by others. I would say that folk psychology and mindreading are topological properties of narratives. If the variety of actors is infinite, or, at least, indefinite and vary from story to story, the structure of the actants is much more stable and it is exactly with this structure that semiotics identifies narrativity. Briefly, narrativity is a processual form of interrelated positions organized in a suitable and stable way beyond the superficial variations characterizing any single story. According to semiotics, narrativity is not the story, but the structure of positions that gives shape to the story and embodies in it. These positions are interrelated according to a prototypical structure developing in four steps. These four steps have been identified by studying narratives (tales, myths etc.), but it has been shown by the semiotic tradition that they can be found in any discourse or practice involving acting for reasons.16 The first step of this narrative logic of inter-action is often called contract and is supposed to establish the system of values which will frame further actions. This step is characterized by “manipulation” and we can have different kind of manipulations between actants here. These types of manipulation include authority, which Greimas describes as the transfer of a “having to do” (e.g. a king asking a hero to rescue a princess in order to save the village), persuasion (described as the transfer

 See Robichaud (2003: 39).  Gallagher and Hutto (2019) define narratives as “a special kind of representational artefacts” in order to i) distinguish narratives from action, ii) derive them from embodied experience and iii) avoid pan-narrativism. Summing up, according to enactivism, narrativity is derived from experience, while, according to cognitive semiotics, it shapes experience. 15  Cf. Greimas (1983). See also Lorusso et al. (2012). 16  The bibliography on this particular topic blends with the whole semiotic tradition and I believe that this is one of the most important discoveries of the whole discipline. However, a clear introduction can be found in Bertrand (2000) (chaps. 4 and 5). Of course, Greimas (1970, 1983) is the main developer of this idea. 13 14

114

Social Cognition and Autism Spectrum Disorders: From Mindreading to Narratives

of a “willing to do”), and permission (for example someone who asks permission for acting, as in academia assistant professors ask of a full professor). In all these variable cases, what really matters is the establishment of something worth acting for, or the establishment of values that need actions in order to be preserved or transformed. The second step after contract is often called competence. In order to act a subject must have the necessary abilities, the “knowing how to do”, as Greimas calls them. A number of important transformations enacted in narratives address the issue of abilities, transforming the agent into an agent particularly competent to perform the transformation at the core of the story. We will have further different relations between actants here. Actants will either be helpers or opponents, assisting or creating obstacles for the subject that is looking for the “knowing how to do.” For instance, in order to free the princess, a hero of a tale may need a magical sword or some important information and loads of secondary characters and objects can fill the role of his helpers and opponents. Or in order to pass an exam, a student needs books to build his own competence and these are helpers. The third step is performance, a transformation in which the subject transforms the undesired state by obtaining his object(s) of value (which, of course, can also be a nonphysical object, a state of mind or whatever else he aims for). This third step represents the core transformation in any narrative, the transformation for which the subject was previously prepared by being manipulated and made competent. The fourth step of the semiotic narrative schema is sanction. At this stage, the subject receives an acknowledgment of his success or failure by virtue of a given system of values (usually the one established in the first step). As we will see, this fourth step is extremely important and connected to impairments in social skills. According to Greimas (1970, 1983), this simple list of four patterns of transformations could account for all the narrative constructions he had ever encountered. This pattern of transformation is what makes a sequence with non-logical relations between the events a narrative. Behind this simple construction, semioticians would argue, there lies a fundamental organization of transformations that will be found in any description of action if it is to be meaningful. This is why, according to semiotics, narrativity is exactly the form of sense that shapes actions in their being meaningful for us. Obviously, this kind of pattern can be found in every detective story, in every fairy tale, in every blockbuster movie, but also in complex practices like an exam at the university, where an institution sets a system of values, the student must acquire a competence, perform and be judged by the teacher. Also, narrativity, in this ­semiotic sense, can be found in the majority of animal cultures. The immature animal has to gain competence about getting food, has to perform and get sanctioned based on its own behaviors (Marrone and Mangano 2018). The semiotic idea of narrativity is not an explicit “narrative storytelling” at all, but the deep pattern that structures narratives and develops the ability to see and to frame others in a detailed pragmatic and social context, where values can be shared and transformed.

7  Narrativity and Developmental Trajectories

115

7  Narrativity and Developmental Trajectories “Infant research” (Stern 1992, 1995) and semiotic studies (Violi 2012; Paolucci 2019) have shown that this kind of pattern is at play both in what Colwyn Trevarthen (1979) and Trevarthen and Hubley (1978) called “primary intersubjectivity” (1–9 month-old infants) and “secondary intersubjectivity” (9–18 month-old infants). This aspect should be carefully considered in order to understand the role played by narrativity in early mindreading and folk psychology competence. Human children and many other social animals exhibit a special responsiveness to the intentional attitudes of others early on (3  months), discriminating human actors from mechanical agents and goal-related actions from mechanical movements. Indeed, during primary intersubjectivity, human infants and some other social animals (Myowa-­Yamakoshi et al. 2004) are already able to see bodily movements as goal-directed, meaningful intentional movements and to perceive other persons as agents, discriminating a biological movement from other kinds of movements (Simion et al. 2011). Embodied interactions during primary intersubjectivity constitute our primary access for understanding the others and the skills we exhibit during this crucial period ground our more sophisticated social cognition abilities, developed later on in ontogenesis. As noted previously, these skills include protomimesis, emotional interchange, imitation and the parsing of perceived intentions (Gallagher 2005, 2006, 2009). During primary intersubjectivity, children are already able to interpret the actions and the expressive movements of the others through meaning (Zlatev 2008). In all these embodied interactions during primary intersubjectivity, a narrative competence (in the semiotic sense) is already at play. When a 6  months-­old infant starts perceiving various bodily movements as meaningful, understanding that he can answer them in interactive response, having back her mother’s smile as a reward, keeping on doing this because the reward is intrinsically motivating, for a semiotician there is an evident narrative competence already operating, able to transform the values involved and shape experience through meaning (Violi 2012). The presence of values which are worth acting (contract and sanction) is extremely important, since our radical form of enactivism do not only include the idea of sensorimotor contingencies, but also emphasize other fundamental aspects of embodiment such as affectivity, reward, interest, motivation and embodied social interaction (see Dominey et al. 2016). However, for mindreading skills to emerge, secondary intersubjectivity is crucial. Around the age of 1 year, but also earlier (at 9 months), together with the development of a semiotic competence (Vygotskij 1986; Eco 1975), the infant goes beyond person-to-person immediacy and enters contexts of shared attention, where he begins to tie actions to pragmatic contexts (Reddy 2008). The capacity for joint attention is a major addition to the repertoire of social skills exhibited during primary intersubjectivity. Not only children are now able to re-enact to completion the goal-directed behavior that someone else fails to complete, but, in those shared situations, an object or an event can become a focus between people: an object of value that can be communicated about (Hobson 2002). “The child, on seeing an adult who

116

Social Cognition and Autism Spectrum Disorders: From Mindreading to Narratives

tries to manipulate a toy and who appears frustrated about being unable to do so, quite readily picks up the toy and shows the adult how to do it” (Gallagher and Hutto 2008: 24). This emergence of a “third” beyond the caregiver/infant interaction—that can be a value for both—is crucial for developing a narrative framework in which the infant starts to learn what things mean and what they are for, together with the idea that these values can be transformed and manipulated. Infants not only cooperate with the others, but, more deeply, begin to build their own stories in order to cheat others, like when they false cry to get an object and not only their mother’s attention. This is crucial for developing mindreading and all the cognitive skills required in order to handle and understand false belief situations properly. At around 9 months of age, children start to build worlds which are alternatives to the ones they are living in. Through imagination and strategic behavior, they start to build situations decoupled from the actual ones. They start to imagine fictional worlds and—more importantly—they start working in order to make these fictional imagined situations become actual ones (see Paolucci 2019). For instance, consider that they want an object, a “third” that is of value to them. In order to get it, they begin to obtain the competence they need, and they actually perform something in order to own it, false crying in order to manipulate the mother and make her bring the object they want. They learn that they can manipulate, and be manipulated, that others can believe in things that are not actually true, that the caregiver can really believe they are actually crying and that all this can change their world in a way they want it to be. That is how human abilities to represent another’s false belief are connected to narratives and to all those semiotic skills which allow someone to construct signs that stand for other things and can be used to lie. After all, this was exactly the most beautiful definition by Umberto Eco (1975: 13): “semiotics is the discipline that studies everything that can be used in order to lie”. This answers the second question raised by Pierre Jacob, showing how semiotic narratives can be the basis of one’s understanding other minds beyond the limits of the interactive approach, like in the enactivist theory of action and perception we have dealt with in the previous paragraphs. But this can also tell us something important about theory of mind and mindreading skills. Infants have at least implicit mindreading skills. Animals also have social cognition skills; they read others’ actions, desires, intentions and, according to very recent experiments by Buttelmann et al. (2017) and Krupeneye et al. (2016), they also read and handle false beliefs. They have to do that in order to survive in the wild (see Andrews 2015). But animals do not have language, or a theory of mind connected to propositional attitudes like we are supposed to have. Using Daniel Hutto’s terminology, they only have intentional attitudes (Hutto 2013). However, they also interact with their caregivers during their development and, as said, during these embodied practices they also shape experience through semiotic narrative patterns. The same thing goes with social cooperation and joint activities inside the animals’ social network. During these interactions, the puppy attunes its skills and behaviours to its environment and to its social group. This is why we believe that mindreading is connected with narrative patterns inside embodied interactions and that the difference between great apes and human social cognition does not lie in their

7  Narrativity and Developmental Trajectories

117

basic capacity to read other minds. Indeed, if deception is the key element in order to move from embodied social interactions to mindreading skills, primates and many other social animals always show astonishing deceptive skills. The focus on deception is crucial for a radical enactivist account of mindreading and social cognition like the one we are defending. When deception is at play, animals or humans alike are set in an unfamiliar context that requires the drawing of some kind of inference in order to predict a new behaviour. This holds true both for the deceiver—who has to construct a strategy that takes into account the image of the possible reactions of the others—and the deceived, who has to react in an unfamiliar situation, one for which he has not any behavioural abstractions (see Gallagher and Povinelli 2012). It is not by chance that Lurz (2011; Lurz and Krachun 2011) connects the adaptive function of mindreading to camouflage. “Lurz suggests that the reason why we mindread is that our ancestors found it useful to predict the behaviour of conspecifics who were looking at ambiguous stimuli, such as camouflaged predators or preys” (Andrews 2015: 167, see also Lurz 2011; Krachun et al. 2016). If the adaptive function of mindreading is to predict others’ behaviour in opaque environmental contexts, another important scenario that needs to be introduced in our cognitive semiotics account of social cognition is opaque cooperative social behaviour. This underlines another strong evolutionary claim. Sociality does not only mean to be competitive and gain advantage through deception or strategic behaviour, but also to collaborate, share knowledge, build communities and engage in joint activities that may hugely benefit from mindreading skills. This is why in shared activities inside a social cooperative world, a strange behaviour is not immediately sanctioned negatively. Since primates and humans can handle meaning during interactions, they often bet that behind a strange behaviour in a collaborative task, there may be some sort of opaque reasons to engage in that particular behaviour by an individual inside a cooperative community. In situations of cooperation of that sort, that animal social behaviour does not always move to the sanction step, but goes back to the contract one, engaging in some sort of recognition that a particular behaviour can be valuable and worth being performed, even if it is not immediately clear. Our everyday practice of making sense of intentional actions in terms of reasons (beliefs, desires, etc.) is a skill that comes from social narrative practices in which we manipulate others, in which we try to make others do things inside a shared system of values, in which we gain the competences needed to do these kind of things, in which we act and we get judged on our actions. This kind of activity grounds the relationship between caregiver and baby before verbal language deve­ lops, and is a pattern of action that can be found not only in primates, but probably in a variety of other social animals. A radical enactivist theory of social cognition grounded on semiotics and narrative practices claims that social cognition skills come from this mechanism, without involving any kind of inferences about beliefs or desires understood as mental states or representations. Semiotic narrativity is the key bridge that leads us to mind and beliefs starting from basic perceptions, emotions and embodied enactive interactions, and the kind of social cognition that

118

Social Cognition and Autism Spectrum Disorders: From Mindreading to Narratives

o­ perates in implicit false belief task competency is developed out of this narrative logic of interaction.

8  From Deception to Pretend Play and Language Acquisition According to our Narrative Practice Semiotic Hypothesis (see Paolucci 2019), a radical enactivist account of social cognition claims that our mindreading skills are grounded first and foremost in the embodied interactions between infants and caregivers during primary intersubjectivity (1–9 months), but they fully develop during secondary intersubjectivity connected to the semiotic capacity of deception and lie (9–18 months). Only in its full-blown form developed during secondary intersubjectivity, narrativity can lead to mindreading skills. This encourages pretend play, which is also connected to semiotic skills and narrative thought (Hutto 2006, 2008, 2009), that typically begins in a child’s second year, at around 18 months. As Leslie (1987, 2005) already noted, pretense signals the beginnings of the so-called second-­ order mental processes that underwrite the development of theory of mind. The conjunction of deficits in both pretense and theory of mind in Autism Spectrum Disorders (ASD) is consistent with this idea. Pretend play is a key moment for developing a folk psychology competence and not only because it highlights and fosters children’s abilities to understand and coordinate multiple mental perspectives. If folk psychological competence refers to our everyday practice of making sense of intentional actions (our own and those of others) in terms of reasons (beliefs, desires etc.), it implies both a “first-person/second person” and a “first-person/third-person” matching. “I am Spiderman” and “I am mommy and you are daddy” exactly represent this kind of matching during pretend play activities. When children are playing pretend, they are playing ‘as if’ something or someone is real. This is why pretend play is actually connected not only with the beginning of the second-order mental processes that lead to mindreading, but also with a narrative logic that shapes their experience, since this “as if”—as we have shown before—is acquired through semiotic narratives during secondary intersubjectivity, without being immediately related to language acquisition or to a representational content. Children simply start putting themselves in the position of the others; they become actors occupying other characters’ positions. Pretend play is a merging between fiction and reality, between the actual world of reality and the possible world of imagination. For example, a child might be placing a doll in a bed and, to the child, the doll is alive and really sleeping and so the child will have to wait until the doll wakes up. Pretend play is a fully developed narrative practice, involving skills that do not depend upon language acquisition. Language extends these skills beyond embodied interactions, fine tuning an already developed narrative competence connected to mindreading and to folk psychology skills. The same thing goes with shared attention skills developed during secondary intersubjectivity. Basic forms of co-referential shared interactive activity like

9  Autism Spectrum Disorders and Cognitive Semiotics

119

p­ ointing or gaze following are already at play by 9 months, but full-blown prelinguistic joint attention skills are normally well established by 18 months, when pretend play also shows up. At this point, a full-blown mindreading competence, already at play in a more rudimentary way starting from 9 months, is usually established. This is consistent with data coming from early mindreading experiments (see Paolucci 2019; Baillargeon et al. 2010). Impairments in both pretend play and mindreading skills in children with ASD is also consistent with this thesis. During the second year of life, language shows up, extending in important ways all these capacities. The Narrative Practice Hypothesis by Daniel Hutto, linked to verbal language and propositional attitudes, explains perfectly well why three-year-­ old children consistently fail standard verbal-based false belief and related meta-­ representational tests, showing us what happens when language enters the scene of mindreading life of children from 2 to 5 years. However, this does not necessarily mean that early mindreading is not a real form of mindreading and does not involve folk psychological states at all. As we have already said, data from very recent experiments by Buttelmann et al. (2009, 2017) with infants and primates seem not to be fully interpretable in terms of an enactivist theory of embodied interactions without introducing a narrative competence, which has to be at play before language acquisition (Paolucci 2019). Semiotic narrativity provides narrative competence before language acquisition inside a radical enactivist account of social cognition. Of course, Hutto believes in non-propositional forms of believing (i.e., intentional attitudes) and non-discursive narratives, but those forms and those narratives do not enter in the Narrative Practice Hypothesis for folk psychology, which deals only with linguistically mediated propositional attitudes. But it looks like there is more continuity between these types of beliefs, and our Narrative Practice Semiotic Hypothesis tries to give account of this continuity. Indeed, as far as mindreading and folk psychological competence are concerned, starting from the sub-personal action/perception/imagination matching mechanism implemented in mirror neurons and from the reward network—encouraging us to prefer the stimuli and the experiences that lead to a positive sanction—it is action with its narrative logic that turns a very general embodied matching system into a mindreading module, tuned to the meaning of the actions of the others during interaction. And all this without the need of representational and linguistic content in the explanation of cognition in basic minds (see Hutto and Myin 2017).

9  Autism Spectrum Disorders and Cognitive Semiotics In order to test all this, let us consider cases in which social cognition skills do not work properly, as in Autism Spectrum Disorders (ASD). We all know that the value of something shows up when we lose it. Infants begin life with general mechanisms dedicated to processing intentions, goals as well as other social stimuli, that subsequently become “tuned” to mindreading as a direct consequence of the extensive experience with social interaction provided by the species-typical environment

120

Social Cognition and Autism Spectrum Disorders: From Mindreading to Narratives

within the first months of life. As we have shown, joint attention, semiotic competence, deception skills and pretend play all play a key role in this process. Since infants spend their entire first 2  years interacting with others, this represents of course a critical moment, because social skills impairments, like ASD, are the proof that this cognitive attunement can fail. We know that children with ASD consistently fail false belief tasks (Baron-Cohen 1995; Wellman et  al. 2001) and clear signs of disattunements of this kind are actually visible during embodied interactions in secondary intersubjectivity, particularly in the sanction and contract phases of the narrative development of the embodied interaction. This is crucial for our radical enactivist account of social cognition grounded on semiotics. In a paper of 2013, Gallese, Rochat and Berchio worked extensively on the potential role of the mirror mechanism in Autism Spectrum Disorder, focusing on the deficit in chaining motor acts into a global action and in understanding others’ intentions when children with ASD have to rely on motor cues during embodied interactions. But what about a more general incapacity of succeeding in social cognition tasks which is at work in children with ASD? Can it be related to the high-­ level order of our model for social cognition? I will refer here to the Social Motivation Theory of Autism Spectrum Disorders (Dawson et al. 2005; Chevalier et al. 2012). According to this theory, one of the possible causes underlying the difficulties of social interaction in subjects with ASD could be the presence of a motivational deficit, closely associated with anomalies in the reward network brain structures. Following these anomalies in the brain reward system, subjects with ASD would have a reduced sensitivity to social gratification and would fail to attribute a value of reward, which is usually intrinsically motivating, to socially relevant stimuli. According to the Social Motivation Theory, the lack of experience of the social world would prevent the specialization of neural circuits underlying these domains and we know that their development and their specialization is based on experience. The idea that social motivation deficits play a central role in Autism Spectrum Disorders (ASD) has recently gained increased interest. This constitutes a shift in autism research, which has traditionally focused more intensely on cognitive impairments, such as Theory of Mind deficits or executive dysfunction, while granting comparatively less attention to motivational factors. […] ASD can be construed as an extreme case of diminished social motivation and, as such, provides a powerful model to understand humans’ intrinsic drive to seek acceptance and avoid rejection. […] Social motivation is a powerful force guiding human behavior and that disruption of social motivational mechanisms may constitute a primary deficit in autism. In this framework, motivational deficits are thought to have downstream effects on the development of social cognition and deficits in social cognition are therefore construed as a consequence, rather than a cause, of disrupted social interest. (Chevalier et al. 2012: 231–2)

It is extremely difficult to tell if social motivation deficits are the cause of ASD impairments—like Social Motivation Theory claims—or the effect of them. Indeed, if you have difficulties in playing soccer, it will be extremely rare that you will be motivated to play it, since every attempt will emphasize your difficulties. The same thing could be at play with ASD, where difficulties with interacting result in motivation deficits for doing so. However, this does not matter as far as early diagnosis—

9  Autism Spectrum Disorders and Cognitive Semiotics

121

and not the explanation of the causes—is concerned. A semiotic analysis of video recordings of interactions between caregivers and infants later diagnosed with ASD clearly shows impairments in the first and fourth steps of the semiotic narrative account for action. Children with ASD problems usually do not care about being judged positively, they do not prefer—like other children do—tasks that give them back a social reward. Social reward is not an attractive and motivational property of a stimulus for them. Accordingly, they do not completely share our own system of values and this brings them to carry on their own business without paying attention to our own (resulting in impairments in contract phase). This is why we claim that the reward network can be involved in ASD impairments. If compared to the motor level, the brain structures that compose the reward system are very different. In particular, ASD deficits “appear to be rooted in biological disruptions of the orbitofrontal-striatal-amygdala circuitry as well as in dysregulation of certain neuropeptides and neurotransmitters” (Chevalier et al. 2012: 239). Since early diagnosis has proven to be essential for effective treatment for ASD, recognizing that a small baby that does not care about returning his mother’s smile when he acts is extremely important. Children with ASD usually do not prefer socially rewarding tasks. Rather, their attention is not captured by the mother’s voice compared to other stresses, and this is visible from the very first months of their life. Social motivation models of ASD posit that early-onset impairments in social attention set in motion developmental processes that ultimately deprive the child of adequate social learning experiences and that the resulting imbalance in attending to social and non-social stimuli further disrupts social skill and social cognition development. (Chevalier et  al. 2012: 234)

Moreover, unlike ordinary theory of mind accounts of ASD, cognitive semiotics grounded on social motivation models would also explain apparently non-social deficits in ASD, such as repetitive behaviours and restricted interests. This indicates a very promising future for the Narrative Practice Semiotic Hypothesis. We do not really know how ASD children behave in early mindreading tasks, because the ASD diagnosis usually arrives more than 1 year later. What we know is that there is a strict connection between experience and the specialisation of neurons and networks in the reward areas. We also know that all the parents of babies with an ASD diagnosis reported indifference to social stimuli and to reception of a positive sanction from the others since the very first months of life. Diminished social interest deprive the developing child of social interest and learning opportunities, which leads to diminished expertise in social cognition. An early diagnosis based on the monitoring of the sanction aspects during practices between children and caregivers could be an extremely important diagnostic aim that a work on narrative practices can help to achieve. Indeed, a diagnosis of ASD is usually provided around the age of 2 years and half and usually follows a developmental delay in children’s linguistic skills. However, clear signs of impairments in the social skills that can lead to ASD can be seen and read much earlier, during embodied and prelinguistic practices between infants and

122

Social Cognition and Autism Spectrum Disorders: From Mindreading to Narratives

caregivers, when the toddler is between nine and 18 months old (secondary intersubjectivity). This is why, in our Cognitive Semiotics Unit at the University of Bologna, we have developed a semiotic methodology meant to help teachers, families and caregivers detect clear signs of possible future disorders of social skills, which may later lead to a diagnosis of ASD.17 The only way this can be done is through the observation of prelinguistic interactions between caregivers and infants, using a cognitive semiotics methodology. Using video recordings, we have built up a deep comparison between the regular pattern of a typical interaction between infants and caregivers and a non-regular pattern of the same kind of interaction shown through an analysis of videos of interactions of infants later diagnosed with ASD. By resorting to a database that is already in construction at the Department of Philosophy and Communication of the University of Bologna, and thanks to the contribution of organizations, families, schools and all the partners of a project funded by the European Commission, our semiotic methodology is intended to allow for the recognition of the regular patterns of development readable during the interaction. Different levels of this kind of interaction (i.e. emotional, sensorimotor, gestural, imitative, behavioral) have been considered in developing this methodology. Starting from video recordings, this multilayered analysis of embodied interactions will allow the building of models aimed at detecting deviations from a typical pattern for each of the levels mentioned above. The results of our analysis will also provide knowledge for the realization of a model and of a range of educational tools (e.g. videos, webinars, etc.), aimed at teaching pre-primary teachers, families and caregivers to detect the signs of a possible future neurodevelopmental disorder. This will lead to the individuation of recurrent patterns that could be interpreted as worrying signals, which may be examined more closely through medical consultation. Thus, cognitive semiotics applied to interactions analysis can also help to reduce disparities in learning outcomes between children who have developed typically and children with ASD.  Indeed, earlier evaluation is very important, since early treatment by specialized professionals has been proven to be much more successful. In particular, early diagnosis significantly improves the chances for effective treatment, which may foster a form of high functioning autism.

References Andrews, K. (2015). The animal mind. London and New York: Routledge. Apperly, I. A., & Butterfill, S. A. (2009). Do humans have two systems to track beliefs and belief-­ like states? Psychological Review, 116, 953–970. Baillargeon, R., Scott, R. M., & He, Z. (2010). False-belief understanding in infants. Trends in Cognitive Sciences, 14(3), 110–118.

17

 See our NeMo European Project: https://site.unibo.it/nemoproject/en

References

123

Baron-Cohen, S. (1995). Mindblindness: an essay on Autism and Theory of Mind. Cambridge: MIT Press. Bertrand, D. (2000). Précis de sémiotique littéraire. Paris: Nathan. Bruner, J. (1986). Actual minds, possible worlds. Cambridge: Harvard University Press. Bruner, J. (1991). The narrative construction of reality. Critical Inquiry, 18, 1–21. Buttelmann, D., Carpenter, M., & Tomasello, M. (2009). Eighteen-month-old infants show false belief understanding in an active helping paradigm. Cognition, 112(2), 337–342. Buttelmann, D., Buttelmann, F., Carpenter, M., Call, J., & Tomasello, M. (2017). Great apes distinguish true from false beliefs in an interactive helping task. PLoS One, 12(4), 1–13. Butterfill, S. A., & Apperly, I. A. (2013). How to construct a minimal theory of mind. Mind & Language, 28(5), 606–637. Carruthers, P. (2013). Mindreading in infancy. Mind & Language, 28(2), 141–172. Chevalier, C., Kohls, G., Troiani, V., Brodkin, E. S., & Schultz, R. T. (2012). The social motivation theory of autism. Trends in Cognitive Sciences, 16(4), 231–239. Clements, W. A., & Perner, J. (1994). Implicit understanding of belief. Cognitive Development, 9(4), 377–395. Crane, T. (1995). The mechanical mind. London: Penguin Books. Currie, G., & Sterelny, K. (2000). How to think about the modularity of mind-reading. The Philosophical Quarterly, 50(199), 145–160. Danto, A. (1985). Narration and knowledge. New York: Columbia University Press. Dawson, G., Webb, S. J., & McPartland, J. (2005). Understanding the nature of face processing impairment in autism: insights from behavioral and electrophysiological studies. Developmental Neuropsychology, 27(3), 403–424. Dominey, P., Prescott, T., Bohg, J., Engel, A., Gallagher, S., Heed, T., Hoffmann, M., Knoblich, G., Prinz, W., & Schwartz, A. (2016). Implications of action-oriented paradigm shifts in cognitive science. In A. K. Engel, K. J. Friston, & K. Danica (Eds.), The pragmatic turn: Toward action-­ oriented views in cognitive science (pp. 333–356). Cambridge, MA: MIT Press. Eco, U. (1975). Trattato di semiotica generale. Milano: Bompiani. English edition: Eco, Umberto. 1976. A Theory of Semiotics. Bloomington: Indiana University Press. Eco, U. (1979). Lector in fabula. Milano: Bompiani. English edition: Eco, Umberto. 1981. The Role of the Reader, Bloomington, Indiana University Press. London: Hutchinson. Ellis, R. (2009). Implicit and explicit learning, knowledge and instruction. In E. Rod, L. Shawn, E. Catherine, R. Hayo, E. Rosemary, & P. Jenefer (Eds.), Implicit and explicit knowledge in second language learning (pp. 3–26). Buffalo/Toronto: Multilingual Matter. Frith, U., & Happe, F. (1999). Theory of mind and self-consciousness: What is it like to be Autistic. Mind & Language, 14(1), 1–22. Fusaroli, R., Gangopadhyay, N., & Tylén, K. (2013). The dialogically extended mind: Language as skilful intersubjective engagement. Cognitive Systems Research, 29–30(1), 31–39. https://doi. org/10.1016/j.cogsys.2013.06.002. Gallagher, S. (2005). How body shapes the mind. Oxford: Oxford University Press. Gallagher, S. (2006). The narrative alternative to theory of mind. In R.  Menary (Ed.), Radical enactivism: Intentionality, phenomenology and narrative: Focus on the philosophy of Daniel D. Hutto (pp. 223–229). Amsterdam: John Benjamins. Gallagher, S. (2009). Two problems of intersubjectivity. Journal of Consciousness Studies, 16(6– 7), 1–20. Gallagher, S. (2017). Enactivist interventions. Rethinking the mind. Oxford: Oxford University Press. Gallagher, S., & Hutto, D. (2008). Understanding others through primary interaction and narrative practice. In J. Zlatev, T. Racine, C. Sinha, & E. Itkonen (Eds.), The shared mind: Perspectives on intersubjectivity (pp. 17–38). Amsterdam: John Benjamins. Gallagher, S., & Hutto, D.  D. (2019). Narratives in embodied therapeutic practice: Getting the story straight. In P.  Helen, K.  Sabine, T.  Jennifer, & F.  Thomas (Eds.), The Routledge

124

Social Cognition and Autism Spectrum Disorders: From Mindreading to Narratives

International handbook of embodied perspectives in psychotherapy. London: Routledge. https://doi.org/10.4324/9781315159416-4. Gallagher, S., & Povinelli, D. (2012). Enactive and behavioral abstractions accounts of social understanding in chimpanzees, infants and adults. Review of Philosophy and Psychology, 3, 145–169. Gallagher, S., & Zahavi, D. (2008). The phenomenological mind. London: Routledge. Gallese, V. (2001). The ‘shared manifold’ hypothesis: From mirror neurons to empathy. Journal of Consciousness Study, 8, 33–50. Gallese, V. (2005) . Being like me: self-other identity, mirror neurons and empathy. In Perspectives on imitation: from cognitive neuroscience to social science, vol. 1 (eds S. Hurley & N. Chater), pp.101–118. Cambridge, MA: MIT Press Gallese, V. (2007). Before and Below ‘Theory of Mind’: Embodied simulation and the neural correlates of social cognition. Philosophical Transactions of the Royal Society, B: Biological Sciences, 362(1480), 659–669. Gallese, V., & Goldman, A. (1998). Mirror neurons and the simulation theory of mindreading. Trends in Cognitive Science, 12, 493–501. https://doi.org/10.1016/S1364-6613(98)01262-5. Gallese, V., & Lakoff, G. (2005). The bra in’s concepts: The role of the sensory-motor system in conceptual knowledge. Cognitive Neuropsychology, 22(3), 455–479. https://doi. org/10.1080/02643290442000310. Gallese, V., Keysers, C., & Rizzolatti, G. (2004). A unifying view of the basis of social cognition. Trends in Cognitive Science, 8(9), 396–403. Gallese, V., Rochat, M., & Berchio, C. (2013). The mirror mechanism and its potential role in autism spectrum disorder. Developmental Medicine and Child Neurology, 55(1), 15–22. Goldman, A. I. (1989). Interpretation psychologized. Mind & Language, 4, 161–185. Goldman, A. I. (2006). Simulating minds. The philosophy, psychology and neuroscience of mindreading. New York: Oxford University Press. Gordon, R. M. (1986). Folk psychology as simulation. Mind & Language, 1, 158–171. Greimas, A. J. (1970). Du sens. Paris: Seuil. Greimas, A. J. (1983). Du sens 2. Paris: Seuil. Heal, J. (1998). Understanding other minds from the inside. In A. O’Hear (Ed.), Current issues in philosophy of mind (pp. 83–99). New York: Cambridge University Press. Herman, D. (2003). Stories as a tool for thinking. In D. Herman (Ed.), Narrative theory and the cognitive science (pp. 163–194). Stanford: CSLI. Hobson, P. (2002). The cradle of thought. London: McMillan. Hutto, D. D. (2004). The limits of spectatorial folk psychology. Mind & Language, 19, 548–573. https://doi-org.ezproxy.unibo.it/10.1111/j.0268-1064.2004.00272.x. Hutto, D. D. (2006). Narrative practice and understanding reasons. In R. Menary (Ed.), Radical enactivism: Intentionality, phenomenology and narrative: Focus on the philosophy of Daniel D. Hutto (pp. 231–247). Amsterdam: John Benjamins. Hutto, D. D. (2007). The narrative practice hypothesis: Origins and applications of folk psychology. Philosophy. Royal Institute of Philosophy Supplement, 82, 60. Also in Narrative and Understanding Persons, ed. H. Daniel: 43–68. Cambridge: Cambridge University Press. Hutto, D.  D. (2008). The narrative practice hypothesis: clarifications and implications. Philosophical Explorations, 11(3), 175–192. Hutto, D.  D. (2009). Folk psychology as narrative practice. Journal of Consciousness Studies, 16(6–8), 9–39. Hutto, D. D. (2013). Why believe in contentless beliefs? In N. Nottelmann (Ed.), New essays on belief: Constitution, Cture (pp. 55–74). New York: Palgrave Macmillan. Hutto, D. D., & Myin, E. (2017). Evolving enactivism. Boston: MIT Press. Jacob, P. (2011). The direct-perception model of empathy: A critique. Review of Philosophy and Psychology, 2011(2), 519–540. Johnson, M. H. (2000). Functional brain development in infants: Elements of an interactive specialization framework. Child Development, 71(1), 75–81.

References

125

Kovacs, A.  M. (2016). Belief files in theory of mind reasoning. Review of Philosophy and Psychology, 7(2), 509–527. Krachun, C., Lurz, R., Russell, J.  L., & Hopkins, W.  D. (2016). Smoke and mirrors: Testing the scope of chimpanzees’ appearance-reality understanding. Cognition, 150(2016), 53–67. https://doi.org/10.1016/j.cognition.2016.01.012. Krupeneye, C., Kano, F., Hirata, S., Call, J., & Tomasello, M. (2016). Great apes anticipate agents’ actions based on their false beliefs. Science, 354(6308), 110–114. https://doi.org/10.1126/science.aaf8110. Leslie, A. M. (1987). Pretense and representation: The origins of “theory of mind.”. Psychological Review, 94(4), 412–426. https://doi.org/10.1037/0033-295X.94.4.412. Leslie, A.  M. (2005). Developmental parallels in understanding minds and bodies. Trends in Cognitive Sciences, 9(10), 459–462. Levi-Strauss, C. (1958). Anthropologie structural. Paris: Plon. Lorusso, A. M., Paolucci, C., & Violi, P. (Eds.). (2012). Narratività. Temi, Problemi, prospettive. Bologna: Bononia University Press. Low, J., & Wang, B. (2011). On the long road to mentalism in children’s spontaneous false-belief understanding: are we there yet? Review of Philosophy and Psychology, 2(3), 411–428. Lurz, R. (2011). Mindreading animals. Cambridge: MIT Press. Lurz, R., & Krachun, C. (2011). How could we know whether nonhuman primates understand others’ internal goals and intentions? Solving Povinelli’s problem. Review of Philosophy and Psychology, 2(3), 449–481. Lurz, R., Kanet, S., & Krachun, C. (2014). Animal mindreading: A defense of optimistic agnosticism. Mind & Language, 29(4), 428–454. Malle, B. F. (2002). The relation between language and theory of mind in development and evolution. In T. Givón & B. F. Malle (Eds.), The evolution of language out of pre-language (pp. 265– 284). Amsterdam: John Benjamins. Marrone, G., & Mangano, D. (Eds.). (2018). Semiotics of animals in culture. Dordrecht: Springer. Martin, A., & Santos, L. R. (2014). The origins of belief representation: Monkeys fail to automatically represent others’ beliefs. Cognition, 130(3), 300–308. Martin, A., & Santos, L. R. (2016). What cognitive mechanisms support primate theory of mind? Trends in Cognitive Sciences, 20(5), 375–382. Matteucci, G. (2019). Estetica e natura umana. In La mente estesa tra percezione, emozione ed espressione. Roma: Carocci. Menary, R. (Ed.). (2006). Radical enactivism. Focus on the philosophy of Daniel D.  Hutto. Amsterdam/Philadelphia: John Benjamins. Mink, L. O. (1978). Narrative form as a cognitive instrument. In R. Canary & H. Kozicki (Eds.), The writing of history: Literary form and historical understanding (pp. 129–149). Madison: University of Wisconsin Press. Myowa-Yamakoshi, M., Tomonaga, M., Tanaka, M., & Matsuzawa, T. (2004). Imitation in neonatal chimpanzees (Pan troglodytes). Developmental Science, 7(4), 437–442. Noë, A. (2006). Action in perception. Cambridge: MIT Press. Onishi, K.  H., & Baillargeon, R. (2005). Do 15-month-old infants understand false beliefs? Science, 308(5719), 255–258. Paolucci, C. (2012). Social cognition between theories, simulations and narratives: The narrative practice semiotic hypothesis. Intellectica, 58(2012), 173–196. Paolucci, C. (2019). Social cognition, mindreading and narratives. A cognitive semiotics perspective on narrative practices from early mindreading to Autism Spectrum disorders. Phenomenology and the Cognitive Sciences, 18(2), 375–400. Pennisi, A., & Falzone, A. (2017). Darwinian biolinguistics. Dordrecht: Springer. Povinelli, D. J., & Vonk, J. (2004). We don’t need a microscope to explore the chimpanzee’s mind. Mind & Language, 19(1), 1–28. Propp, V. (1928). Morfologija skazki. Leningrad: Academia. Reddy, V. (2008). How infants know minds. Cambridge: Harvard University Press.

126

Social Cognition and Autism Spectrum Disorders: From Mindreading to Narratives

Rizzolatti, G., & Craighero, L. (2004). The mirror-neuron system. Annual Review of Neuroscience, 2004(27), 169–192. https://doi.org/10.1146/annurev.neuro.27.070203.144230. Rizzolatti, G., & Sinigaglia, C. (2006). So quel che fai, Il cervello che agisce e i neuroni specchio. Milano: Raffaello Cortina Editore. Rizzolatti, G., & Vozza, L. (2011). Nella mente degli altri. Neuroni specchio e comportamento sociale. Bologna: Zanichelli. Robichaud, D. (2003). Narrative institutions we organize by: the case of municipal administration. In B. Czarniawska & P. Gagliardi (Eds.), Narrative we organize by (pp. 37–54). Amsterdam: John Benjamins. https://doi.org/10.1075/aios.11.04rob. Ruffman, T., & Perner, J. (2005a). Infants’ insight into the mind: How deep? Science, 308(5719), 214–216. Ruffman, T., & Perner, J. (2005b). Do infants really understand false belief? Trends in Cognitive Sciences, 9(10), 462–463. Scott, L. S., Pascalis, O., & Nelson, C. A. (2007). A domain general theory of perceptual development. Current Directions in Psychological Science, 16(4), 197–201. Simion, F., Turati, C., Valenza, E., & Leo, I. (2006). The emergence of cognitive specialization in infancy: The case of face preference. In M. Johnson & Y. Munakata (Eds.), Attention and performance XXI, processes of change in brain and cognitive development (pp. 189–208). Oxford: Oxford University Press. Simion, F., Di Giorgio, E., Leo, I., & Bardi, L. (2011). The processing of social stimuli in early infancy: From faces to biological motion perception. Progress in Brain Research, 189, 173– 193. https://doi.org/10.1016/B978-0-444-53,884-0.00024-5. Simion, F., & Di Giorgio, E. (2015). Face perception and processing in early infancy: inborn predispositions and developmental changes. Frontiers in Psychology, 6, 969, 1–11. https://doi. org/10.3389/fpsyg.2015.00969. Southgate, V., Senju, A., & Csibra, G. (2007). Action anticipation through attribution of false belief by 2-year-olds. Psychological Science, 18(7), 587–592. Stern, D. (1992). Diary of a baby. What your child sees, feels and experience. New York: Basic Books. Stern, D. (1995). The motherhood constellation. New York: Basic Books. Sugita, Y. (2008). Face perception in monkeys reared with no exposure to faces. Proceedings of the National Academy of Sciences of the United States of America, 105, 394–398. Talmy, L. (2000). A cognitive framework for narrative structure. In L.  Talmy (Ed.), Toward a Cognitive Semantics (Vol. 2, pp. 417–482). Cambridge, MA: MIT Press. Tomasello, M. (2014). A natural history of human thinking. Cambridge, MA: Harvard University Press. Trevarthen, C. (1979). Communication and cooperation in early infancy: A description of primary intersubjectivity. In M. Bullowa (Ed.), Before speech (pp. 321–348). Cambridge: Cambridge University Press. Trevarthen, C., & Hubley, P. (1978). Secondary intersubjectivity: Confidence, confiding and acts of meaning in the first year. In L. Andrew (Ed.), Action, gesture and symbol: The emergence of language (pp. 183–229). Academic Press: London. Turati, C., Simion, F., Milani, I., & Umiltà, C. (2002). Newborns’ preference for faces: What is crucial? Developmental Psychology, 38(6), 875–882. Violi, P. (2012). Nuove forme della narratività. In A. M. Lorusso, C. Paolucci, & P. Violi (Eds.), Narratività. Temi, Problemi, prospettive (pp. 105–132). Bologna: Bononia University Press. Vygotskij, L. S. (1986). Thought and language. Cambridge: MIT Press. Wellman, H., Cross, D., & Watson, J. (2001). Meta-analysis of theory of mind development: The truth about false belief. Child Development, 72(3), 655–684. https://doi. org/10.1111/1467-8624.00304. Zlatev, J. (2008). The co-evolution of intersubjectivity and bodily mimesis. In J. Zlatev, T. Racine, C. Sinha, & E. Itkonen (Eds.), The shared mind: Perspectives on intersubjectivity (pp. 215– 244). Amsterdam: John Benjamins.

Perception as Controlled Hallucination

1  Semiotics of Perception In this chapter, I will deal with the problem of perception from the point of view of cognitive semiotics.1 I will try to underline the crucial role of imagination, claiming that perception is a form of “controlled hallucination” (Koenderink 2010; Clark 2016), where, by “controlled hallucination”, I mean the product of the imagination controlled by the world. The way in which we match the “hallucination” of imagination with the “control” of the world is through diagrams and narratives. The main idea is that “hallucination” is the model of perception and not a deviant form of it. With “hallucination”, as defined in perception studies and in the neurogeometry of vision (Koenderink 2010; Sarti et al. 2008), I mean the morphological activity of the production of forms by the imagination, which remains crucial both when it is not controlled by the world—as in the case of hallucination, imagination or dream— and when it is controlled by the world, as in the case of online perception. Maybe, the word “hallucination” can be misleading, since perceptual phenomena under the aegis of hallucination may seem to lose the concreteness that I want to characterize them as having. It is possible that “figuration” would fit better with the ideas I will develop here, since no Sartrean “derealization” is involved (see Matteucci 2019). However, since our brains try to guess what is out there, and to the extent that that guess matches the evolving sensory data, we perceive the world, in Surfing Uncertainty, Clark (2016) recalls the slogan coined by the vision scientist Ramesh Jain that perception is controlled hallucination. This is the direction I am going to take in this chapter. But, exactly because “this view of perception puts us in genuine cognitive contact with the salient aspects of our environment”, Clark (2016: 326, see also paragraph 6.10) suggests considering hallucination as a form of “uncontrolled perception”. However, Clark’s view is the classical one that thinks of 1  On this topic, see Pozzato and Marmo 1989, Marrone 1995, Basso Fossali 2009; Paolucci 2010: Chap. 4.

© Springer Nature Switzerland AG 2021 C. Paolucci, Cognitive Semiotics, Perspectives in Pragmatics, Philosophy & Psychology 24, https://doi.org/10.1007/978-3-030-42986-7_5

127

128

Perception as Controlled Hallucination

p­ erception as grounding both hallucination and imagination, which are supposed to be “deviant” or “uncontrolled” forms of perception. Since we want to claim the opposite, in this chapter I will continue to use “hallucination”, and since there is a well-­established tradition regarding this concept in the field of perception studies, I will do so with the caveat that “hallucination” does not imply any kind of “derealization” of perceptual phenomena from a phenomenological point of view (Gallagher and Zahavi 2014). In order to do all this, I want to start with the crucial semiotic problem of Kant and the Platypus. How does one perceptually interpret the platypus? How does one construct the percept of an item of which he does not have the concept yet? How does one account for what taxonomists do when they see a platypus for the first time, or for what Marco Polo does when faced with the rhinoceros? How does one account for the bricolage of cultural units and previous knowledge that is at work when we try to adapt a schema to an unknown perceptual content and then, on the basis of the latter, readjust the concept so that it fits with the perceptual content? For example, let us consider Marco Polo, who saw what we now know were rhinoceroses on Java. Since he had the previous knowledge of the unicorn, coming from his culture, he designated those animals as unicorns. Later, as he was a honest chronicler, he hastened to tell us that these unicorns were rather strange, given that they were not white nor slender, but had “the hair of the buffalo” and “the feet of an elephant”. Therefore, he advised against sending a virgin girl to capture it as was suggested by the legends regarding unicorns (see Eco 1997: Chap. 2). A similar problem occured when the platypus was discovered in Australia. A pelt and a sketch of the animal were sent back to Great Britain by Captain John Hunter, but British scientists’ initial hunch was that the animal was a hoax, that it had been produced by some Asian taxidermist by putting together pieces of other animals. It was thought that somebody had sewn a duck’s beak onto the body of a beaver-like animal. Marco Polo and the scientists both attempted to interpret the rhinoceros and the platypus respectively in a top-down manner, trying to fit the percepts into their previous knowledge to predict and interpret what was in front of their eyes. This is the reason why the rhino was a “clumsy and dark unicorn” and only later, on the basis of new perceptions that could not fit previous knowledge, did Marco Polo readjust the concept of unicorn to include the features that did not fit with it. While it can be maintained that semiotic processes are involved in the recognition of the known, because it is precisely a matter of relating sense data to a (conceptual and semantic) model, the problem, which has been debated for a long time now, is to what extent a semiotic process plays a part in the understanding of an unknown phenomenon (Eco 1997/2000: 59).

In Kant and the Platypus, Eco formulated and anticipated with his own style a problem of perception which is now at the core of contemporary debates regarding the predictive processing of the so-called Bayesian brain.

2  Kant, the Platypus and the Predictive Processing

129

2  Kant, the Platypus and the Predictive Processing In order to explore the connections between Eco and predictive processing, let us consider Andy Clark’s Surfing Uncertainty as the best application of the principles of the Bayesian brain to perception. In Clark’s formulation of the predictive processing, perception (in its rich, world-revealing forms) is active and works from the top-down. The brain’s key job is to predict the present sensory signal, by construc­ ting it for itself using stored knowledge about the world. In perception, the incoming sensory signal is matched (courtesy of multiple local exchanges) by a complex flow of top-down prediction. As we have seen in chapter “The Semiotic Mind. Beliefs, Habits and the Extended Cognition”, cognitive semiotics and pragmatism share this view about the nature of how sensory signals and top-down predictions mesh. For instance, Peirce claims that we cannot have any “cognition” without “previous knowledge”. “Interpretants” in cognitive semiotics are analogous to “priors” in predictive processing. According to predictive processing, as long as the matching proceeds, some po­pulations of cells send predictions downwards, and others send residual errors upwards, signaling mismatches with what is currently predicted (“prediction errors”). When the top-down flow sufficiently matches the sensory evidence, “the system has unearthed the most likely set of worldly causes that would give rise to the sensory barrage”, and a stable world-revealing percept is formed (see Clark 2016: Chap. 6). This is where “a potent data compression strategy” known as “predictive coding” comes into the mix. Given that the brain is already active, busily predicting its own present sensory inputs and the resulting states at every level of processing, cognitive systems need only bother about whatever escapes the predictive net. What drives further processing of what escaped the predictive net (e.g. the selection of a new predictive model, or the amendment of an old model) is just the residual difference (the “prediction error”) between an actual current signal and a predicted signal. As Andy Clark used to say, in some sense, “perception carries only the news”. This is exactly what Umberto Eco is communicating with the story of the rhinoceros and Marco Polo. The process of integrating new perceptions with prior concepts using semiotics and pragmatism is not referred to as “predictive coding,” but there is not much difference between the accounts. Marco Polo forms a top-down bricolage using his previous knowledge coded in the encyclopedia (priors) in order to match the sensory data, and it is only when he sees that he simply cannot work out what the rhinoceros is in this way, that he thinks of constructing a new “coding” for the new sensory data input. We perceive a structured external world and not just patterns of light and sound, because we meet the transduced pixels with a top-down cascade of interacting represented distal causes. The so-called ‘transparency’ of perceptual experience (that we seem to simply see tables, chairs, and bananas, not any sensory intermediaries) seems to fall quite naturally out of these models. Perception and understanding are here inseparable. To perceive like that is to guess how the sensory evidence was generated by means of interacting distal causes, and how it is likely to evolve on multiple spatial and temporal scales. Perceiving and

130

Perception as Controlled Hallucination

u­ nderstanding are thus co-constructed and percepts are always at least weakly “conceptualized”. (Clark 2016)

The main idea is that we can use stored knowledge (interpretants) and regularities of the environment (habits) to generate a kind of analogue of the driving sensory signal as it unfolds across multiple layers and types of processing. This is the reason why percepts are always at least weakly conceptualized. Of course, this opens a lot of questions from the perspective of cognitive semiotics. What is the nature of this analogue? How does it match with the sensory signals? If priors and interpretants play a crucial role for perception, where do they come from (this is otherwise known as the “bootstrap problem”)? And, last but not least, what is the relationship between percepts and concepts, since they seem to be radically intertwined? The relationship between the percept and the concept is the same problem that Umberto Eco presents in Kant and the Platypus. The key for the solution is, according to his point of view, a reformulation of the Kantian notion of schematism. Here we start to get close to our problem, because schematism is the product of imagination. For Kant, we have concepts, that are intellectual, we have sense data, that comes from aesthetics as a theory of sensibility, but the art of making those two communicate in perception is the art of imagination. Eco’s argumentations on this topic are very famous. I will briefly sum-up two difficult points, which I think should be problematized, especially in the light of some contemporary studies inside cognitive sciences. 1. According to Eco, the construction of the percept is grounded on a process of “subsuming something under rules or concepts”, (it is precisely on this topic that Eco feels the need of developing his own interpretation of Kant’s schematism). According to Eco, the percept defines a token that, in order to be interpreted, must be unified under a general type. It must be “coded”, as predictive processing teaches us. As Eco (1997/2000: 97) says, in order to interpret perceptual phenomena, we have to construct a rule. “Rather than observe (and thence produce schemata), the reflective faculty of judgment produces schemata in order to be able to observe” (top-down predictive process). This process of “subsuming under rules” is at work in cases in which the empirical concept is given (i.e. the cases that Eco traces back to Kant’s determinant judgment), as well as in cases in which the concept have to be formulated starting from sensory data (i.e., the cases that Eco traces back to Kant’s reflective judgment). This second set of cases includes the rhinoceros described by Marco Polo and the platypus described by the British taxonomists after its discovery in Australia. 2. In both cases, in order to make what was said at point (1) possible, Eco claims that a third element is always required. This third element is a mediating one, that allows for “representing an image according to a certain concept” (Eco 1997/2000: 82). This third mediating element is needed for constructing the concept or the rule aimed at interpreting an image that is not known yet (e.g. in the case of the rhinoceros or in that of the platypus). According to Eco, for empirical concepts like “man”, “platypus” or “rhinoceros”, this third element that ­mediates

2  Kant, the Platypus and the Predictive Processing

131

between sensibility and intellect (namely Kant’s schema) has a morphological nature. While the schema of the circle is not an image but a rule for constructing the image if necessary, the empirical concept of the plate should nonetheless include the notion that its form may be constructed in some way—in a visual sense, to be exact. One must conclude that when Kant thinks of the schema of the dog, he is thinking of something very similar to that which, in present-day cognitive science, Marr and Nishishara (1978) call a "3-D Model," which they represent as in figure 2.2.

In perceptual judgment the 3-D model is applied to the manifold of experience, and we distinguish an x as a man and not as a dog. (Eco 1997/2000: 85–86).

I believe that both these points are problematic and should be elaborated further (and possibly abandoned). Indeed, a type/token schema grounded on a process of “unifying under concepts” seems inadequate to account for the phenomenological emergence of the percept. In addition, it does not seem possible to individuate a morphological machinery at the level of the schematism, namely between intuitions and concepts. Contrary to what Eco claims, morphologies seem to be an emergent phenomenon. If predictive processing is right, we are able to build a percept starting from priors, taking care only of the residual difference (the “prediction error”) between an actual current signal and a predicted one, without the need of a figurative resemblance between a schema and the percept, since the so-called ‘transparency’ of perceptual experience is the effect of the fact that perceiving and understanding are co-constructed and percepts are always at least weakly “conceptualized”. We perceive a structured external world and not just patterns of light and sound not because we apply a morphological 3-D model to experience, but because we meet the sensory signals with a top-down cascade of priors and interpretants (previous knowledge). Somehow, we perceive through the filter of these interpretant signs and it is through these interpretant signs that we can perceive aspects that we could not otherwise perceive when observing the object directly without the cognitive semiotic filter. And this mediation is worth exploring.

132

Perception as Controlled Hallucination

3  Diagrammatic Thinking, Inference and Perception A key concept of cognitive semiotics is diagrammatic thinking (see Stjernfelt 2007; Paolucci 2017; Zalamea 2003). The notion is crucial because it allows us to invert the commonsense idea coming from semiotics that perception is inferential. According to cognitive semiotics, the inverse is the case. Inference is perceptive, and if there is inference within perception is because perception grounds inference. But why are we introducing the concept of diagram here, in the middle of a discussion on schematism, on the powers of imagination and on the relationship between percepts and concepts? Simply because Peirce claimed that schemata are diagrams and that even the simplest syllogistic conclusion can only be drawn by observing the relations of the terms in the premises and conclusion in a diagram built by the imagination. Kant's whole philosophy turns upon his logic. He gives the name of logic to the greater part of his Critic of the Pure Reason, and it is a result of the great fault of his logical theory that he does not extend that name to the whole work. This greatest fault was at the same time the greatest merit of his doctrine: it lay in his sharp discrimination of the intuitive and the discursive processes of the mind. […] He drew too hard a line between the operations of observation and of ratiocination. He allows himself to fall into the habit of thinking that the latter only begins after the former is complete; and wholly fails to see that even the simplest syllogistic conclusion can only be drawn by observing the relations of the terms in the premises and conclusion. His doctrine of the schemata can only have been an afterthought, an addition to his system after it was substantially complete. For if the schemata had been considered early enough, they would have overgrown his whole work. (CP 1.35)

As it is well known, in Kantian gnoseology there is a problem of commensurability between the two sources that should grant our knowledge of objects. Intuitions, which are sensory (aesthetic), and concepts, which are intellectual (logical), do not share the very same nature. In order to enable concepts to determine intuitions, a mediating representation is needed. This mediating representation would ensure a possible commensurability between these two heterogeneous systems. The mediating representation that makes these heterogeneous worlds communicate is the schematism of imagination. It is clear what “the greatest fault” of Kantian doctrine is according to Peirce. Kant is famous for having said that intuitions without concepts are blind and concepts without intuitions are empty, yet, he still distinguishes intuitive processes from discursive processes and observational operations from ratiocinative operations. Only in the schema do these elements communicate. However, for Peirce, intuitions and concepts, observation and ratiocination, aesthetics and logic, percepts and concepts do not differ by nature and are always radically intertwined. This is why perception is always needed in order to explain inference. This is also why— more on this later—imagination is needed in order to explain both perception and inference. Eco notes this intertwinement and, in order to think of schemas of empirical concepts as being endowed with morphological features, he looks back at this extraordinary idea by Peirce. Eco (1997/2000: 50) notices that Peirce, with no hesitation, claims that Kantian schemas are diagrams. While Eco does not fully buy this

3  Diagrammatic Thinking, Inference and Perception

133

Peircean idea, in order to explain what a Kantian schema is, he still refers to the concept of diagram. For example, he talks about flowcharts of calculators: In order to gain a better understanding of the concept of schema, perhaps we need to consider what computer operators call a flowchart. The machine is capable of “thinking” in terms of IF... THEN GO TO […]. Given one operation, at a certain juncture in the process a possible alternative is produced, and, depending on the answer that appears, a choice needs to be made; depending on the new answer, it is necessary to return to a higher node of the chart, or proceed beyond, and so on. The chart has something that can be intuited in spatial terms, but at the same time it is substantially based on a temporal course (the flow), in the same way as Kant observes that the schemata are based fundamentally on time (Eco 1997/2000:82–83).

It is important to keep in mind that, according to Kant, a schema is a determination of time. In diagrams theory, space is much more studied if compared to time. However, as far as perception is concerned, diagrams of time are crucial, as we will see later, when we will deal with the default condition of perception, that is to “perceive the immediate future” (see Adams et al. 2013; Clark 2016). But what do we mean by “diagram” if, according to Peirce, a diagram is a Kantian schema? In his extraordinary attempt to construct “a system of diagrammatization by means of which any course of thought can be represented with exactitude” (CP. 4.530), Peirce gives us this delightful example: But why do that [build a System of diagrammatization], when the thought itself is present to us?” Such, substantially, has been the interrogative objection raised by more than one or two superior intelligences, among whom I single out an eminent and glorious General. “General, you make use of maps during a campaign, I believe. But why should you do so, when the country they represent is right there?” Thereupon, had he replied that he found details in the maps that were so far from being “right there,” that they were within the enemy’s lines, I ought to have pressed the question, “Am I right, then, in understanding that, if you were thoroughly and perfectly familiar with the country, no map of it would then be of the smallest use to you in laying out your detailed plans?” To that he could only have rejoined, “No, I do not say that, since I might probably desire the maps to stick pins into, so as to mark each anticipated day’s change in the situations of the two armies”. To that again, my sur-rejoinder should have been, “Well, General, that precisely corresponds to the advantages of a diagram of the course of a discussion. Indeed, just there, where you have so clearly pointed it out, lies the advantage of diagrams in general.” (CP 4.530)

As we see here, a diagram is a sign that displays features that remain only virtual in the object before the intervention of the sign and that only the sign is able to manifest. Through the direct observation of the diagrammatic sign “other truths concerning its object can be discovered than those which suffice to determine its construction” (CP 2.279). A diagram is a filter through which it is possible to perceive otherwise unnoticeable properties of the object. In the case of the diagram, this filter is essentially cognitive, since the diagram is a logical image or “schematic image,” as Peirce says (see NEM IV: 238). Moreover, in order to discover new proprieties of the object not directly observable without the sign, the sign has to be manipulated. According to cognitive semiotics, cognition must always pass through some sort of “doing”, like manipulating

134

Perception as Controlled Hallucination

a diagrammatic analogue in the imagination (priors, interpretants etc.). It is like in Blowup by Michelangelo Antonioni. When you are in the park and a murder happens exactly while you are at the park, you cannot perceive it. However, at home, through the pictures you have shot there, through the manipulation of their semiotic structure (zooming in, cutting etc.), you can perceive things that you were not able to perceive while observing the object directly. This is why Peirce states that there is something in the object that can be perceived and acknowledged only through the sign and that only in the sign is directly perceivable. What is manifested via the sign and only via the sign is crucial for perception. Diagrammatic thinking is a key element for a theory of knowledge grounded on the impossibility of separating “reasoning” and “observation”, “knowledge” and “action”, “perception” and “inference”. This is not because perception is inferential, but rather because that inference is perceptive. I suppose it would be the general opinion of logicians, as it certainly was mine, that the Syllogism is a Symbol, because of its Generality. But there is an inaccurate analysis and confusion of thought at the bottom of that view; for so understood it would fail to furnish Evidence. Now necessary reasoning makes its conclusion evident. What sort of a Sign can communicate this Evidence? It is a very extraordinary feature of Diagrams that they show […] that a consequence does follow, and more marvellous yet, that it would follow under all varieties of circumstances accompanying the premises. (NEM 4:318) Even simple syllogism, involves an element of observation; namely, deduction consists in constructing an icon or diagram of the relations of whose parts shall present a complete analogy with those of the parts of the object of reasoning, of experimenting upon this image in the imagination, and of observing the result so as to discover unnoticed and hidden relations among the parts. (CP 3.363)

Evidence, in an inference, is something that is observed and follows manipulation (see Stjernfelt 2007). This idea can be generalized: in the diagram of the hierarchical structure of a company, you perceive the generality of the relations in the token, you see it. And you can manipulate the diagram, in order to make emerge new forms of relations which were not noticed in constructing the diagram. A Diagram, in my sense, is in the first place a Token, or singular Object used as a Sign; for it is essential that it should be capable of being perceived and observed. It is, however, what is called a General sign; that is, it denotes a general Object. […] The Diagram represents a definite Form of Relation. This Relation is usually one which actually exists, as in a map, or is intended to exist, as in a Plan. […] The pure Diagram is designed to represent and to render intelligible, the Form of Relation merely. Consequently, Diagrams are restricted to the representation of a certain class of relations; namely, those that are intelligible. We may make a diagram of the Battle of Gettysburgh, because in a certain sense, it may thus be rendered comprehensible. (NEM IV: 315–6)

Thus, the expression “logical image” means that a diagram is a sensory, observable, and manipulable token that embodies purely logical (intelligible) relations. This is the reason why Peirce says that it is a “schematic image,” in the sense in which, according to Kant, the schematism of imagination does not consist in an image, but in aesthetic relations (time-space ones) that embody or realize purely

3  Diagrammatic Thinking, Inference and Perception

135

logical (conceptual) relations.2 “A Diagram is an icon or schematic image embodying the meaning of a general predicate; and from the observation of this icon we are supposed to construct a new general predicate” (NEM IV: 238). Since “a Diagram is mainly […] an Icon of intelligible relations” (CP 4.531), Peirce suggests, a diagram unites logical features (the relations that it displays are intelligible) and aesthesic features (the diagram is a sensory token that can he perceived and manipulated). Therefore, Peirce can assign to the diagram the role carried by the schematism in Kant (see CP 1.35). In Kant and the Platypus, comparing Peircean diagrams and Kantian schemata, Eco (1997) gives us a precious piece of advice on this topic. He notices that the schema in Kant was an “image” only in the sense of the Bild in Wittgenstein,3 according to which a proposition shares a common form of relation with the fact that it represents, but, at the same time, it differs from the fact in nature. It is exactly this idea that is at the root of the Peircean conception of the diagram. Eco seems to notice this since, at the end of the discussion on schematism, he inserts the Peircean diagram of the sentence “every mother loves some child of hers”.

 See Deleuze 1963, Eco 1997.  For more on the connections between Peircean semiotics and Wittgenstein, see Fabbrichesi 2014.

2 3

136

Perception as Controlled Hallucination

Here it is exactly a logical image, a Bild, a schema in which aesthetic elements embody or realise purely logical relations in observable and manipulable tokens that share the same form of relation. A logical image of this kind is then completely neutral to the opposition between arbitrary sign and motivated sign (see Eco 1997: Chaps. 2 and 5), or between symbol and icon/index, since the relations between the parts of the diagram are motivated by the form of relation expressed in the proposition, but it is only thanks to the conventional rules of the graph that they are expressed. As Peirce suggests, according to the rule of transformation that allows manipulating the diagram while experimenting on it, the transformed diagram is the image of the original diagram and both are images of the object that they “show,” thus making purely virtual properties visible. In this specific and non-trivial sense, there exists an image of something only when there is something that is able to make us pass from one “universe” to another, guaranteeing a possible commensurability between the two systems and operating as a translation, although just a partial one. Like when we speak of the “image of a function” (its correspondence in another set) or when we say that “the parliament is the image of the population”. In order to explain this point, Peirce introduces the second equation of optics:

let f1 and f2 be the two distances of the two foci of a lens from the lens. Then,



1 / f1  1 / f2  1 / f0 This equation is a diagram of the form of the relation between the two focal distances and the principal focal distance; and the conventions of algebra [...] in conjunction with the writing of the equation, establish a relation between the very letters f1, f2, f0 regardless of their significance, the form of which relation is the very same as the form of the relation between the three focal distances that these letters denote. This is a truth quite beyond dispute. Thus, this algebraic Diagram presents to our observation the very, identical object of mathematical research, that is, the Form of the harmonic mean, which the equation aids one to study. (CP 4.530)

3  Diagrammatic Thinking, Inference and Perception

137

Through this example, Peirce says that the physical object, the letters of the algebraic equation and the corresponding graph, all embody the same form of relation, the one that is found among the two focal distances. Every single element embodies the same ratio, the one that prescribes, for instance, that to have magnification it is necessary to preserve a certain distance between the eye and the lens and the lens and the observed object. Thus, the diagram displays sensitively (in a perceptual way in the graph, in a physical way in the lens) the same logical relations that are in its object, making them observable and experimentally manipulable. Just like a Kantian schema, a diagram establishes commensurability between heterogeneous systems, and it does so exclusively through relations of relations. Its way of functioning has nothing to do with figurative analogies. This can be clearly seen looking at the diagram of the proposition considered by Eco. The diagram embodies logical relations, presenting them as observable and perceptually manipulable tokens; which is why it functions as a schema, but it does not look like a proposition, of which it preserves only the same form of relation. So, if predictive coding is right, it is through a diagrammatic imagination that the forms of sensory data are built into the ‘transparency’ of our perceptual experience, where we seem to simply see tables, chairs, and bananas, not any sensory intermediaries. More, perceiving and understanding are thus co-constructed: percepts are always ‘conceptualized’ and logical relations are always seen. Every addition or improvement to our knowledge, of whatsoever kind, come from an exercise of our powers of perception. In necessary inference my observation is directed to a creation of my own imagination, a sort of diagram or image in which are portrayed the facts given in the premises; and the observation consists in recognizing relations between the parts of this diagram which were not noticed in constructing it (NEM IV: 105).

I want to take very seriously, I would say literally, this cognitive semiotics’ idea that every improvement of our knowledge comes from perception, that inference depends on perception, that perception depends on imagination and that imagination implies embodied and enacted creativity—a doing—a kind of performativity through which we manipulate forms of relations and recognize something which was not noticed before the construction of imagination through the doing. Perceiving is not sampling the properties of the external world in order to build a representation. It is a creative and embodied activity through which we generate percepts starting from priors and interpretants, matching them with the sensory signals through diagrammatic reasoning. However, in Kant and the Platypus, Eco, being influenced by Kant and by Marr’s classical cognitive science account of perception, does not take Peirce’s identification of schematism and diagrammatic thinking seriously. He does not fully buy the idea that schemas are nothing but diagrams and that Kantian schematism is just a specific kind of semiotic diagrammatic process of the imagination. On the one hand, Eco seems to be willing to endorse a flowchart-like diagrammatical process for the Kantian application of categories to the manifold of sensible intuition, on the other hand, he does not seem to be willing to apply it to empirical concepts, such as “rhinoceros”, “man”, or “unicorn”.

138

Perception as Controlled Hallucination

Because it is clear that our nice little analogy with the flowchart, which might help us understand how the schematic construction of the triangle proceeds, works far less well in the case of the dog. A computer can certainly construct the image of a dog, provided it is given suitable algorithms: but it is not by examining the flowchart for the construction of the dog that a person who has never seen a dog can have a mental image (whatever a mental image may be) of one. (Eco 1997/2000: 85).

Is this really the case, though? The whole argument that Eco develops concerns (1) the problem of vision; (2) and the bootstrap problem. As far as point (1) is concerned, the idea is that, given a diagram (e.g. a flowchart or Peirce’s algebraic equation), a person that has never seen “x” is not able to construct a mental image of “x”. This is to say that she or he cannot see something that looks like a dog, a man, a platypus looking directly at the diagram. Thus, for empirical concepts, the schema would rather be something like a 3D model, like David Marr’s one. If I see a 3D model, I can construct a mental image. As far as point (2) is concerned, the problem of Eco is the so-called “bootstrap problem” of perception (Clark 2016; Hutto and Myin 2017). If perception needs priors, what happens if I do not have any good prior to use, as was the case for the rhinoceros and the platypus? Put in the terms of the contemporary debate in cognitive science, where do we (or our brains) get the conceptual resources and background knowledge (priors and interpretants) needed to make inferences in perception? Here we enter the main door of our problems of perception, of vision and of the role of imagination.

4  The Bootstrap Problem and the Metaphor of Vision Let’s start from the bootstrap problem, our point (2). According to Clark (2016: 19), since the role of our brains in perception is “guessing the world”, they must try to optimize their inferences starting with only the extremely “slim pickings of the sensory evidence’”. In such prediction-driven learning, very well embodied in the bricolage of cultural units trying to fit the sensory data in front of Marco Polo, “the world can be relied upon to provide a training signal allowing us to compare current predictions with actual sensed patterns of energetic inputs. This allows well-­ understood learning algorithms to unearth rich information about the interacting external causes that are actually structuring the incoming signal” (Clark 2016: 19). This notion of reliability is also present in Eco’s explanation in which there is a “hard core of being” (Eco 1997: Chap. 1) able to falsify our predictions. However, it seems to me that Eco is underestimating the role that habits play in perception, especially in the bootstrap problem. Let’s take an example that I borrow from Umberto Eco’s book From the Tree to the Labyrinth, where I am supposed to have kicked him in the shins (Eco 2007: 476). Eco wonders what is the cognitive status of the sensation of pain that arises in the script of my breakfast in the morning, when I accidentally touch the coffee pot.

4  The Bootstrap Problem and the Metaphor of Vision

139

According to him, the burning sensation produced by the coffee pot is a Firstness, “the emerging of something new” (since we don’t have the habit of feeling pain during breakfast). However, Firstnesses “do not spring up isolated; for if they did, nothing could unite them. They spring up in reaction upon one another, and thus in a kind of existence” (CP 6.199). According to cognitive semiotics, the emerging of the Firstnesses through their being opposed to each other (Secondness) starting from the regularity of the habit (Thirdness) is an event (CP 6.200), i.e., a singularity, a point at which something occurs. In this way, the spontaneity of Firstness, whose irregular and singular nature Peirce underlines (CP 6.54), turns out to be nothing other than an infinitesimal deviation from the law and from the regularity on whose basis it is produced (CP 6.59). Peirce calls habit, or Thirdness, this very regularity starting from which it is possible to generate the singular spontaneity of the Firstnesses in their opposition to one another (Secondness). In other words, the very spontaneity of the emergence of something new (Firstness) is nothing but the habit of a regular series (Thirdness) which differentiates itself at certain given points: the singular emerges from the regular from which it detaches itself as a consequence of an instability effect. And this is a theorem of the mathematical theory of s­ ingularities by Renè Thom (1988). In this way, the feeling of pain that emerges in the example of my morning coffee (Firstness) is a quality that emerges from a background of experiential habits (getting up in the morning, picking up the coffee pot, putting it on the burner, not turning the gas up too high, placing the coffee pot in just the right place; a whole syntax of habits and regularities of everyday experience). Thus, the sensation of pain (Firstness) arises against a background of habits (Thirdness) that did not imply it (it is not regular to encounter pain in the breakfast scenario) and pain can only arise in opposition (Secondness) to this background of habits. So, on the basis of a series of regularities and habits that define the rules of my morning breakfast (Thirdness), a tendency to be distinguished from it may be created, out of which something new emerges, something for which the regularity of the local system does not make allowance. Here we have the cognitive semiotics version of the bootstrap problem; “how does the knowledge that powers the predictions that underlie perception and action arise in the first place?” Only habits are needed in order to answer this problem. Predictive processing does not have to be understood in terms of matching or mismatching contents that we have in our heads (the “mental image” Eco was talking about: more on that later) to the world. On the contrary, from a cognitive semiotics point of view, the anticipations that we usually do—casting our questions to the world trying to anticipate it during perception—are grounded in an organism’s story of interactions inside semiotic niches (think about Marco Polo with the unicorn). These priors are shaped through experience, development and evolution and are embodied into habits that lead to changes in what we expect to experience (see Hutto and Myin 2017: 67–70). But do habits define norms, or rules, that make us behave and cognize in a lazy way, bringing the present situation back to some regularities stored somewhere? Not at all. Habits are exactly what make possible to perceive the future in the present and

140

Perception as Controlled Hallucination

anticipate the world we are expecting. Habits are what is needed in order to create a virtual analogue of the flow of perception in the absence of the object, actively accommodating and attuning organisms to specific aspects of the environment. And this leads to our point (1), where Eco tells us that a person that has never seen rhinoceros or a platypus is not able to construct a mental image of these animals ­starting from diagrams like the one of the proposition “every mother loves some child of hers”. It seems to me that Eco’s argument is grounded on and clings to one of the oldest metaphors of Western thought: the metaphor of vision (Eco 1997/2000: 100), according to which, “knowing” is “seeing” (for instance, “do you see the problem?” in order to ask if you understand it). To say that we cannot see the dog if we just look at its diagram is like saying that we cannot see a proposition if we look at a Mayan codex. We simply do not know what to look for (Rorty 1979: 26) because we are not yet able to locate what we see into an interpreting semiotic system. If someone shows me a Mayan codex, I will see scribbles, since I do not know the Mayan language. Given my priors, my interpretants and my encyclopedic expertise, I will produce a certain perceptive interpretant. But if I learn the Mayan language, then I will see a Mayan codex, having acquired the ability to cut out the encyclopedia adequately. Why if I see something like Peirce’s diagram of the proposition, I do not see the proposition, a cat or the Mona Lisa? Why if I see a pixilation code by which a computer is able to reproduce the image of the Mona Lisa on the screen, I do not see the Mona Lisa? For the very same reason for, when I see this formula f(x) = x3—3x, I do not immediately see its corresponding diagram (Fig. 1): On the contrary, mathematicians easily do see the diagram, because they know how to move from one semiotic system to another, building a diagram that keeps the very same form of relation. Also, many musicians, when they “see” a score, can “hear” its corresponding music, if we want to follow these ugly sensory metaphors that seem to mislead Eco’s argument. We do not, but they do. This is to say that mathematicians and musicians know what to look for. As Peirce would say, they have acquired a habit, so they can move from the diagram to the mental image, whatever a mental image is. The problem with Peirce’s flowchart taken into account by Eco is that we do not know what to look for when we do not have the priors. This is the reason why we are not able to construct a mental image in the cases of the rhinoceros and the platypus. Following Peirce, we are not able to construct an interpreting sign which is Fig. 1  Diagram of the function f(x) = x3—3x

4  The Bootstrap Problem and the Metaphor of Vision

141

different in nature from the first sign. This interpreting sign would embody the same relational form of the first sign, but under a different aspect (e.g. equation/diagram). We are not able to carry on the semiosis, constructing a diagram that translates the first sign into another interpreting sign. We are not able to move from one point of these heterogeneous domains to another one. We are not able to establish a ratio between these heterogeneous domains, and they continue to be heterogeneous. Interpretation freezes, but it does so because we do not know what to look for. Therefore, the impossibility of constructing mental images starting from diagrams is by no means a counterargument against the possibility of extending a diagrammatical conception of schematism to empirical concepts too. The diagrammatic schematism of imagination is completely unrelated to any figurative analogy between the schema and its object, contrary to what Eco claimed, linking schemas to David Marr’s 3D model. This diagrammatic schematism works much more like a computer does when it builds up the image of Leonardo’s Mona Lisa from a non-figurative code. This perfectly fits with Andy Clark’s idea that we perceive a structured external world (that we seem to simply see tables, chairs, and bananas, not any sensory intermediaries) and not just patterns of light and sound, because “we meet the transduced pixels with a top-­down cascade of interacting represented distal causes”. According to cognitive semiotics, our cognition is diagrammatic. This means that it continuously moves through heterogeneous semiotic domains, without the need of any figurative analogy between them. “Every mother loves some child of hers” is a proposition. But its Peircean diagram is also the very same proposition and fits even better with the purpose of training a machine (Gangemi et al. 2018). Thus, what is the idea behind a diagrammatic cognition? It is the idea that, as far as perception is concerned, aesthesic elements embody forms of relations in observable and manipulable tokens called diagrams. The diagram displays sensitively the same logical relations that are in its object, making them directly observable and experimentally manipulable (in the world or through the imagination). Through diagrammatic thinking, we can move from knowledge to cognition, from priors to the present flow of online perception, simply through a form of doing, through the manipulation of a diagram, that can be done in the world (as we do with ordinary called diagrams) or in the imagination, as Peirce has shown us. So, if diagrammatic cognition is right, it is through a diagrammatic imagination that the forms of sensory data are built into the ‘transparency’ of our perceptual experience, where we seem to simply see tables, chairs, and bananas, not any sensory intermediaries, without the need of any figurative analogy between domains. I propose that animals like us can use stored knowledge and regularities of the environment to generate a kind of virtual diagrammatic analogue of the driving sensory signal as it unfolds across multiple layers and types of processing. This paves the way for processes able to form and evolve in the absence of the usual sensory flow, like dreaming and imagination, that both have the same form of relations of online perception. The morphological processes underlying the construction of those forms are the same. This matters because perceptions simply happen to the observer and percepts are presentations rather than representations, there being nothing to represent prior to presentations. In other words, as Koenderink

142

Perception as Controlled Hallucination

(2010) has shown us, to see or to perceive is an action of the Verstand (presentational awareness), whereas to look at things is an action of the Vernunft (discursive mind). To blend the two is in some sense a dangerous move.4 Of course, priors are important and play a crucial role in standard online perception. However, in many situations, we simply do not update our perception if our priors are changed, as in many visual illusions like the Müller-­Lyer one. In all these cases, we know that what we see is wrong, but we keep on seeing wrongly.5 In order to give account for that, we need to give account for the phenomenological construction of the percept and we cannot do it in the standard way of predictive processing, which deals with how the perceptive system recognizes what is out there in the world, given that its access is limited to sensory input (see Gallagher 2017: Chap. 6). A Gibsonian account of perception in terms of affordance is not enough in order to deal with the phenomenological construction of the percept. We will be inspired by what has been called a “Brentanian” account of perception. Perception in Brentanian terms is neither the re-presentation of an objective external reality (the inferential approach) nor is it simply a direct or indirect resonance of such a reality due to action (Gibsonian and enactive approaches). Rather, perception is the presentation of a unitary occurring event of which the perceiver’s subjective structure is a nonindependent part. […] The central idea in Brentano’s work, that of perception as presentation, has been entirely missing from cognitive science and has only recently been introduced into contemporary dialogue. (Albertazzi et al. 2010: 8)

If the brain’s key job is to predict the present sensory signal, by constructing it for itself using stored knowledge about the world, in order to understand presentations one needs to study the “Laws of Imagination” rather than the “Laws of Thought” or the “Laws of the Brain”.

5  The Laws of Imagination What are the laws of the imagination that shape perception? Gestalttheorie has shown us that perceiving is an activity in which the sensory signal is organized according to immanent laws, able to give account for the scene in front of us, which, of course, is not necessary only a percept, since it can be a scene that happens also in a dream or in our imagination. Indeed, these laws are not meant to represent the world according to the stimulus at all.

4  About the relationship between presentation and perception, also in connection with knowledge and predication and with the necessity of not to blend them, see Matteucci 2019, Chap. 4. See also Albertazzi et al. 2010. 5  I owe this idea to Daniel Hutto and to his PhD classes at the University of Bologna, during June 2019.

5  The Laws of Imagination

143

For instance, in Kanizsa’s triangle (Kanizsa 1980), the triangle is the only thing that is not there (the image above is a simple collection of three pac-man figures). However, we all see it, no matter our cultural differences, the way the outer world is or the priors we have. According to Gestalttheorie, the qualities of the “percepts” are totally independent from the quality of their constitutive parts. Gestalttheorie’s example is melody: we can recreate the very same melody playing it in different keys (thus changing the notes and the physical frequencies of the sound) or playing it with different instruments. In all these cases, the melody will not change, since it is a perceptive quality. According to Gestalttheorie, percepts are like melodies. This is why perception works in a diagrammatic way: it moves from one domain to another, keeping the form of the relation. Gestalttheorie calls this diagrammatic feature of perception “principle of transposability of a Gestalt”. Gestalt laws do not come from experience, but they shape it. They are our way of constructing saliences in order to shape perception through meaning. A brief overview of Gestalt laws will suffice.

The first law of the Gestalt is Figure/Ground, which express the tendency of the perceptive system to look for edges, in order to simplify a scene into the main object

144

Perception as Controlled Hallucination

that we are looking at (the figure) and everything else that forms the background (or ground).

The second law is the Law of Proximity. We group elements that are closer together, separating them from those farther apart, as shown in the above figure.

The third law is the Law of Similarity (or Law of Invariance): we seek differences and similarities in an image and link similar elements.

The fourth law is the Law of Closure. In perception, we prefer complete shapes, so we automatically fill in gaps between elements to perceive a complete image (see “a”). Gestalt laws are ordered in a hierarchy. As it can be seen, the Law of Proximity is stronger than the Law of closure (see “b”).

5  The Laws of Imagination

145

According to the Law of Continuation, we follow and “flow with” lines and we group elements accordingly.

Finally, according to the Law of Prägnanz (or “Good Form”), we perceive complex images as simple ones. We tend to differentiate elements that are similar in color, form, pattern, etc. from others—even when they overlap—and cluster them together. Despite the significance of these laws, imagination is not limited to the laws of Gestalt. Indeed, as we have seen in our previous chapter, if we look at how it works during the ontogenesis—for instance during secondary intersubjectivity (Trevarthen and Hubley 1978; Gallagher 2020)—we see that at around 9 months of age, babies start to build worlds which are alternatives to the ones they are living in. Through imagination and strategic behavior, they start to build situations that are decoupled from actual ones. They start to imagine fictional worlds and—more importantly— they start working in order to make these fictional, imagined, situations become the actual ones, as when they fake cry in order to manipulate the mother, for instance (see Paolucci 2019). Through imagination, they bet that the possible world they have built according to their desires and values can be—or can become through their action—the real world. We will see that this kind of working of the imagination is crucial for perception. Furthermore, we will also see that this process of matching the two worlds through a combination of imagination and action has a narrative form. For instance, it is the key narrative process of detective stories. Indeed, in order to build a semiotics of perception grounded on imagination, there are three things that we have to understand: (1) the first is what we mean by

146

Perception as Controlled Hallucination

“controlled hallucination”; (2) the second is the way controlled hallucination is connected to perception; (3) the third is the connection between imagination and narratives in perception.

6  Perception as Controlled Hallucination As far as point (1) is concerned, with “hallucination”, or “figuration”, I indicate a process of microgenesis (Brown 2002) that continuously produces the next thread of perceptual experience while the current one fades, and does so without voluntary control (Koenderink 2010). I claim that meaning guides this microgenesis, or hallucination, and that imagination is the engine of this microgenesis guided by mea­ ning. For instance, in Kanizsa’s triangle, the triangle is totally a product of our hallucination, since it is the only thing that is not present in the stimuli. It is not by chance that, as Reddy et al. (2010) show, not only online perception and ­imagination are closely related in the brain, but perception, as it occurs in creatures like us, is co-emergent with imagination (Ganis et al. 2004). What we perceive is literally (not metaphorically) the future, not the present, because perception is the anticipation of the next thread of sensory information through previous knowledge. Indeed, we build a diagrammatic analogue with the very same form of relation of the world that we expect. There is a beautiful experiment by Adams et al. (2013) that shows that, in some circumstances, we hear the presence of the absence, that is, we hallucinate something that is not there, but we expect to be there. When silence arrives we literally do not hear it as we are supposed to do (i.e. as nothing playing). In its place, we hear the presence of the absent sound that we were expecting. Adams’ et  al. (2013) experiment runs as follows. They used a simple computer simulation of birdsong. A multi-layer prediction machine processes sequences of simulated bird-chirps. The simulations were then repeated but omitting the last three chirps of the original signal. At the first missing chirp, the network responds with a strong burst of prediction error. But, most suggestively, at the precise moment where the first missing chirp should have occurred, the system generated a brief, transient illusory percept. This hallucinated percept was not strong, but the timing was correct with respect to the missing chirp. Thus, our perceptive system first dimly ‘perceived’ (hallucinated) the missing chirp, before responding with a strong error signal when the actual absence of the anticipated sensory evidence became apparent. Of course, Kanizsa’s triangle is a visual correspondent of Adams et al. (2013) experiment. This is what I call perception as a “controlled hallucination”, which is the general functioning of our rich, world-revealing perception at any level, since it concerns an organism structurally coupled with its environment trying to minimize disorder and surprise. I refer here to what Jan Koenderink (2010), a cognitive scientist and mathematician who works on the connection between theory of singularities and perception

6  Perception as Controlled Hallucination

147

(Sarti et  al. 2008), used to call the “Marrian” and the “Goethean” accounts of perception. • The Marrian account: perception is the result of standard computations on optical data. • The Goethean account: perception is controlled hallucination, or “controlled figuration”. The mainstream view in cognitive science and neuroscience, which is often also the commonsense view, is that perception is all about a kind of passive imprinting of the world upon the sense organs and the brain. As Egner et al. (2010) say, “on traditional accounts the visual system was seen as a passive analyzer of bottom-up sensory information”. This is a view of the perceiving brain as highly stimulus-­ driven, taking energetic inputs from the senses and turning them into a coherent percept by a kind of inwards flowing stream. The Predictive Processing account of perception takes a different direction and, as we have seen, includes a top-down predictive aspect in its account. However, Predictive Processing thinks of perception as a kind of new schematism (the “matching”) between aesthetics (the sensory data) and the concepts (the priors). Our brains are proactive. They are constantly buzzing as they try to predict the sensory signals arriving across all modalities. When such proactive brains ‘match’ the incoming sensory signal, we perceive the world, understand it and are immediately positioned to imagine it and to act in it too. As I have tried to show in connection to Eco’s semiotics of perception, a semiotic theory of perception must deal with the phenomenological account of the percept as an effect of a diagrammatic process of emerging. This is done through imagination, through “controlled hallucination”: It is perhaps not superfluous to stress that “hallucination” is different from the mainstream notion of “prior”. A prior—as used in Bayesian inference—is a generic, usually statistical, property (Purves et al. 2001). For example, “light comes from above” is such a prior (if put in suitable format). It applies, on the average, for terrestrial animals that live in open spaces. Such priors package “frozen” prior experience as it were. “Hallucinations” differ by not being frozen, applying to the actual situation. Hallucinations can be regarded as specific, necessarily tentative, instantions of the observer’s present “situational awareness” instead of its average past. (Koenderink 2010: 32)

This is important, because, as we have said, we do not always update our perceptions according to our past experience and according to the changes in our priors. Indeed, presentations are not necessarily an outcome of so called top-down influences but are always the effect of the morphological machinery responding to the Gestalt laws of imagination (grouping, shaping and sense-making). Perceptual qualities, in the Brentanian sense are those that are, in principle, independent of so-called top-down or cognitive modifications due to geographical, cultural, linguistic, or social environment; to emotional and aesthetic phenomena; or to subjective preferences. (Albertazzi et al. 2010: 9)

As far as vision is concerned, in the mainstream account of cognitive science, the optical input is described as “data”. This notion should be deeply troubling from a semiotic point of view, because the term “data” implies a sender and a receiver and

148

Perception as Controlled Hallucination

a communication over some channel, as though the world were sending something to someone else located in a different place, which is definitively not the case, since perception is the effect of a dynamical system that couples organism and environment. In perception, “data” are constructed through an attunement of the organism and the world, where the organism looks for elements that are worth for him to look for. For instance, as in Kanizsa’s triangle, we look for edges and we perceive edges even if edges are the only thing that are not present in the stimulus. This is because, given evolutionary pressure and experience driven plasticity, “we are wired to the environment in order to produce states that track edges when exposed to discontinuities. The system is physically attuned to such things, ‘set up to be set off’ by such visual discontinuities” (Gallagher 2017: 120). Exploring the world, the organism casts his questions to the environment through imagination and predicts its answers until he encounters resistance. This very action turns optical or sound structure into “data”, which are not sent from the world to the organism through senses but are the actual product of the autopoietic structure of the system of perception. Perception occurs when the top down activity of the imagination succeeds at generating the sensory data for itself, building a coherent story that paves the way for efficacious action. When it can generate a diagram of the future sensory data, the agent perceives the world. When it cannot, encountering a resistance, it tries a new attunement—a sort of diagrammatic manipulation—or changes the world through action. Therefore, data are built up because we produce them in looking for what we need for action, in order to minimize our work in the environment. The difference between the mainstream view, where data are sent by the environment and processed through perception, and the view where they are the product of what we look for in a coupled environment, can be operationalized as follows: • Perception, as far as vision is concerned, is optically guided potential behavior (see Koenderink 2010: 32). Potential is key here. Perception is grounded on imagination, as it concerns the potential behavior connected, as we shall see, to a coherent story we are building in order to act in the world and minimize disorder. Perception as optically guided potential behavior is meant to reveal a world of salient, meaningful, interacting causes selected in the light of human needs and possibilities, which is exactly what pragmatists had in mind when they were telling us that the meaning of something consists in its conceivable practical bearings. This is exactly why visual perception is optically guided potential behavior. This also marks the difference between pragmatism and behaviorism, since perception is not action nor behavior. This pragmatist idea is consistent with the Affordance Competition Hypothesis originally introduced by Cisek (2007), which is a key hypothesis also for the Predictive Processing by Andy Clark (2016: Chap. 6). The brain processes sensory information to specify, in parallel, several potential actions that are currently available. These potential actions compete against each other for further processing, while information is collected to bias this competition until a single response is selected. (Cisek 2007: 1585)

7  Perception, Imagination and Narratives

149

If we go beyond Cisek’s terminology of “processing information”, like I think we have to do in order to focus on the main problem of perception as controlled hallucination, we see that, at the moment, we are seeing a good deal of confirmation regarding this “pragmatist” approach from neurosciences (see Cuccio and Caruana 2015). The classic distinction among perception, cognition and action simply does not reflect not only the phenomenology of our experience, but also the global functional architecture of the brain, where, for instance, motor systems are active and play a huge part also in perception, decision making, social cognition and problem solving (Gallese 2009; Borghi and Caruana 2013; Gallese et al. 2004; Rizzolatti and Sinigaglia 2007). Increasing and highly suggestive evidence challenges the view of core cognitive capacities (such as planning and deciding) as neurophysiologically distinct from the circuitry of sensorimotor control. For example, decisions concerning eye movements and the execution of eye movements recruit highly overlapping circuits in lateral intraparietal area (LIP), frontal eye fields (FEF), and the superior colliculus […]. In the same vein, a perceptual decision task (one in which the decision is reported by an arm movement) revealed marked responses within premotor cortex corresponding to the process of deciding upon a response (Romo et al. 2004). Quite generally, wherever a decision is to be reported by (or otherwise invokes) some motor action, there looks to be an entwining of perceptuo-motor processing and decision-­making, leading Cisek and Kalaska to suggest that ‘decisions, at least those reported through actions, are made within the same sensorimotor circuits that are responsible for planning and executing the associated action’ (Cisek and Kalaska 2011: 274). In cortical associative regions such as posterior parietal cortex (PPC), Cisek and Kalaska go on to argue, activity does not seem in any way to respect the traditional divisions between perception, cognition, and action. Instead we find neuronal populations that trade in shifting and context-responsive combinations of perceiving, deciding, and acting, and in which even single cells may participate in many such functions (Andersen and Buneo 2003). (Clark 2016: 178)

If perception is supposed to work as a process that is continuous with action, the Marrian casual chain is inverted. Instead of “data” arriving at the eye, being processed, being further processed and finally resulting in a “representation” of the scene in front of us, the agent explores the world in any conceivable direction, casting his questions and producing data looking for what he needs for action, until it encounters resistance, as for Marco Polo’s rhinoceros.

7  Perception, Imagination and Narratives In order to see at work this mechanism of controlled hallucination, where the organism casts his questions to the environment through imagination and predicts its answers until he encounters resistance, let’s see what happens with this scene in front of us. Here, not only perception hallucinates something, but what we see depends on what we look for and on how we group elements according to what we look for.

150

Perception as Controlled Hallucination

I will follow here Koenderink (2010). Here the “scene in front of us” is a collection of black blobs on a white background. However, we are likely to “see” either one of two views of a cube, although infinitely many alternative interpretations are possible. The two views may alternate in time. We can also see our own location (either above or below the cube), an occluding screen with circular apertures through which we see a black space behind and gray “connecting lines” where there is no local optical structure. Thus, what we face here is one of many possible presentations. We could have grouped pieces in many other different ways, but we haven’t, since laws of Gestalt apply. Instead, we have looked for something, we have somehow explored the optical structure and we have built up a semiotic expression plan that conveys meaning (the cube), out of many other possible configurations that would have been totally meaningless, that is, that would not have stabilized any expression plan out of the optical structure. This is not amazing at all, since the optic nerve has a channel capacity of 108–109 bits per second, whereas estimates of the structural complexity of perception, are generally below 100 bits per second, so perception has to be very selective and meaning does exactly this job of selection. This fits perfectly with the “adding of subtractions” we were talking about in chapter “Cognitive Semiotics. Radical Enactivism, Pragmatism and Material Engagement”. Somehow, we have looked for salient cues, we have built them in our coupling with the environment. It is not by chance that Pinna and Albertazzi (2010) have shown through the method of experimental phenomenology that “there is no perception without three forms of organization: forms of grouping, forms of shape and forms of meaning” (Albertazzi et al. 2010: 20). What is there if we do not look for something guided by meaning? Just an arrangement of grey and black with no edges at all. But we see edges and we see a couple of different cubes alternating in time, changing also our own position in the scene. Cues we have selected are the junctions of three curves, collinearity and parallelity: all highly non generic features of 2D projections of 3D wireframe configurations. In this way, we have picked up a highly non-generic and non-ordinary

7  Perception, Imagination and Narratives

151

pattern in our searching for edges. And we have also used the fact that the eight local patterns we have picked up may form a coherent whole, an unlikely coincidence in the stimulus that comes from our Gestaltheorie laws of imagination. I want to emphasize that this is actually only an arrangement of gray and black that could have been grouped in many other possible ways. The “cube” is entirely our hallucination, even if all other agents are not unlikely to entertain similar visions, no matter their priors and their cultural preferences and habits. So, what have we done? We have created our cube by looking for something, exploring the world and turning optical structures into meaningful data. Basically, we created the cube through two main moves: (1) we have selected cues turning them into clues, (2) and we have built up a coherent story, a coherent whole, starting from the clues that we have created from the cues we have selected. That’s why, as I will show now, it is through narratives that we match the hallucination of imagination with the control of the world, giving birth to perception as controlled hallucination. The couple “cue” and “clue” is a key one for a cognitive semiotics perspective on perception. “Cue” often means “signal for action”, or “a hint or indication about how to behave in particular circumstances” or “a piece of information or circumstance that aids the memory in retrieving details not recalled spontaneously”. This latter meaning is related to that of “clue”, “a piece of evidence or information used in the detection of a crime or solving of a mystery”, or “a fact or idea that serves as a guide or aid in a task or problem”. This meaning is of interest not only because it connects us to narratives and to semiotics, but because it hints at the fact that cues are selected. This is also true of perception, since in some way the organism “selects” certain cues and “ignores” others. For instance, in the above example, we all were looking for edges, grouping elements accordingly, following Gestalttheorie laws. The observer selects optical structure and promotes it to cue status and then turns cues into clues, in order to build a coherent whole. This double action—a doing, a diagrammatic manipulation—turns optical structure into meaningful data and we hallucinate the cube. Such an idea is quite natural. Consider the well-known Sherlock Holmes stories or the Miss Marple/Hercule Poirot ones by Agatha Christie (see Eco and Sebeok 1983). Throughout the story, Sherlock Holmes collects clues, mostly apparently irrelevant trivialities. The important point here is that the world offers infinite numbers of equally trivial observations, and Sherlock Holmes selects ones that will become meaningful in order to construct a coherent whole starting from the pieces at the conclusion of the story. As Holmes (or Marple, or Poirot) explains the case, all these apparently irrelevant trivial observations become very meaningful, whereas the infinitely many observations that could just as well have been made are never even mentioned. It is only Holmes’ explanation (truly Holmes’ perception) of the plot that renders the clues meaningful. Thus, the clues are necessary to construct the plot, and the plot as a coherent whole is necessary to be able to select (or rather, create) the clues. We look for figures, so we create cues, but we create cues because we look for figures; it is a perfect autopoietic dynamical system. Brains cannot wait, they act before they know what to do and what they are doing is not to construct a

152

Perception as Controlled Hallucination

correct image of the stimulus. What they are doing is trying to act efficaciously (and that’s why we track edges). The problem of selection of cues is central to vision, and it can only be solved by way of figuration and hallucination. As Koenderink (2010) has shown, there is simply no way to transform mere optical structure into meaning. We as perceivers structurally coupled with an environment must supply meaning. A pure optical or sonic fact is entirely meaningless on its own; it will remain unnoticed or is bound to be forgotten the next moment, if it is not built up as a cue through a semiotic process of microgenesis, that turns optical data into an expression plan conveying meaning. It is but a mere structure, unless it fits into our phenomenological present “situational awareness”: then it is noticed and promoted to cue status. This phenomenological level driven by meaning, which matters for perception, seems not to be the core business for Predictive Processing theories, which tend to focus more on the laws of the brain (subpersonal level), while the phenomenological laws of the imagination ground perception as controlled hallucination. Also, this modus operandi may sound suspiciously nonscientific to mainstream cognitive scientists. In following the correct “scientific method”, we are not supposed to pick and choose our own data, or, and it would be even worse to create this data. But this is exactly what we usually do in perception, in which we shape stimuli through meaning. As Koenderink (2010) shows, this is indeed exactly why the Goethean account is so unpalatable to the mainstream cognitive scientists. A hallucination, or anything imaginary that you build up looking for something trying to match the world, is by definition non-veridical, being “an experience involving the apparent perception of something not present”: a sign that can be used in order to lie. According to cognitive scientists, only straight computations on optical data (the Marrian account) can ever hope to be veridical, “coinciding with reality”. This all too common notion is misguiding, however, because it assumes that the radiance at a point completely and unambiguously specifies the scene in front of the observer; which is far from being the case. Any radiance is compatible with an infinity of possible phenomenological scenes. Each such scene would equally well “explain” the optical structure available to the observer. Thus, the Marrian problem is hopelessly underconstrained. […] In order to arrive at the representation of the scene in front of the observer one needs additional tools that allow one to weed out the “wrong ones” from the “actual” one. The current mainstream way to answer this dilemma is the “Bayesian approach to perception”. (Koenderink 2010: 36)

This “Bayesian approach” is predictive processing’s approach, that thinks of “coincidence with reality” as a “matching between priors and sensory data” and “veridicality” as “efficacious optically guided behavior.” But then one is no longer forced to prefer the Marrian, inferential, helmholtzean account over the Goethean account. Controlled hallucination can very well evoke uncontradicted (efficacious) optically guided behavior as long as the hallucinations are checked against the actual optical structure. This approach mirrors Sherlock Holmes’ way of solving a crime. Holmes imagines a plot (a hallucination, not a perception) and checks it against the available evidence. This process effectively weeds out mere fancies and allows him to select the structure that will become evidence in a bootstrap fashion. (Koenderink 2010: 36)

7  Perception, Imagination and Narratives

153

Exploring the world, the organism casts his questions to the environment through imagination and predicts its answers until he encounters resistance. When the ima­ gination does not succeed at generating the sensory data for itself, it tries a new attunement that creates new data. Imagination tries to build a possible world that bets to be the very same world of the actual real world, as far as efficacious action is concerned. When the bet fails, it tries a new attunement and refines its bet on the structure of the world. Standing before an unknown, surprising event that requires explanation, the organism fallibly gambles that this event could be the part of a story it can create through cues and clues (see CP 2.624). If this is verified, thus discovering that the unknown event matches the coherent story we have built, that event thereby stops being surprising and stops being a resistance that requires new bets. It simply matches the story we have built through imagination. As we have seen in chapter “Social Cognition and Autism Spectrum Disorders: from Mindreading to Narratives”, according to cognitive semiotics, narrativity is a Gestalt property that shape experience through meaning and that we use in order to build coherent wholes starting from otherwise irrelated elements. Umberto Eco gives a masterful explanation of this aspect in his paper “Horns, Hooves, Insteps,” analyzing what allows Voltaire’s Zadig to construct a “normal story” starting from some surprising events and perceptions. Eco shows how Zadig could have interpreted the various “visual utterances” that he has to do with as a “disconnected series” or as a “coherent sequence, that’s to say a text or a story.” And he shows how it is only by following this second path that Zadig manages to make sense of his experience. He [Zadig] decides to interpret the data he had assembled as if they were harmoniously interrelated. He knew before that there was a horse and that there were four other unidentified agents. He knew that these five agents were individuals of the actual world of his own experience. Now he also believes that there was a horse with a long tail, fifteen hands high, with a golden bit and a silver hoof. But such a horse does not necessarily belong to the actual world of Zadig’s experience. It belongs to the textual possible world Zadig has built up […] imagination is a world-creating devices. […] So Zadig is in a position to make a “fair guess” according to which both the horse and the dog of his own textual world are the same as those known by the officers. This kind of move is the one usually made by a detective: “The possible individual I have outlined as an inhabitant of the world of my beliefs is the same as the individual of the actual world someone is looking for”. (Eco 1983: 214)

Eco says that behind this connection between imagination and narratives there is a Kantian idea, namely, that we must interpret the world as if it were a story. What is this Kantian idea? It consists in the fact that without our structuring it into “stories” and “texts”, we could never have any knowledge or experience of the world. We interpret the world as if it were a “text” or a “story”—that is to say—as if the possible structure we have delineated with our imagination is the same one that constitutes the real world. Even if this should not be true—this is the predictive error part—it is the only way in which we can have fallible experience of the world, in trying to predict what the world does not match. I would not put things in this Kantian way as Eco does. I would be more radical. In my account, narrativity is a

154

Perception as Controlled Hallucination

Gestalt propriety that shapes perception and experience and, in doing so, makes our attunement to the environment possible. This is why I was claiming that the way in which we match the hallucination of imagination to the control of the world is through semiotic narrativity, in this technical sense. However, according to Eco, there is one major difference regarding the “strength” of narrativity when it operates within the real world and when it operates within the fictional world, as it does in a novel. Unlike what happens in the real world, a detective like Sherlock Holmes never makes mistakes and never hesitates to gamble that the world which he has charted out is the same world as the “real” world, since he has the privilege of living in a world constructed by his inventor, Sir Arthur Conan Doyle, who constantly verifies all the bets, predictions and stories. In the real world, on the other hand, there is nothing which corresponds to the author of a novel. To be sure that the mind of the detective has reconstructed the sequence of the facts and of the rules as they had to be, one must believe a profound Spinozistic notion that “ordo et connexio rerum idem est ac ordo et connexio idearum”. The movements of our mind that investigates follow the same rules of the real. If we think "well," we are obligated to think according to the same rules that connect things among themselves. If a detective identifies with the mind of the killer, he cannot help but arrive at the same point at which the killer arrives. In this Spinozist universe, the detective will also know what the killer will do tomorrow. And he will go to wait for him at the scene of the next crime. But if the detective reasons like that, the killer can reason like that as well: he will be able to act in such a way that the detective will go and wait for him at the scene of the next crime, but the victim of the killer's next crime will be the detective himself. And this is what happens in "Death and the Compass," and in practice in all of Borges's stories, or at least in the more disturbing and enthralling ones. (Eco 1990: 160)

This is also the case in The Name of the Rose: it is no coincidence that, in writing it, Eco had Borges’ Death and the Compass in mind, which not by chance inspired the evil blind librarian who is the protagonist of the novel’s finale. Having understood that William of Baskerville had gambled that the murderer was following the “scheme of the Apocalypse” for his crimes, Jorge de Burgos begins following the scheme himself, in order to kill William of Baskerville, even though all the previous murders were actually the result of chance and not an elaborate or preconceived plan. While William of Baskerville was trying to attune his imagination with the world, Jorge was trying to match its actions with the imagination of William, buil­ ding a world through action that was exactly the same world William had built through his imagination, in order to match the surprising state of affairs. Since narratives are built in order to match our imagination with the environment, narratives can also be constructed in order to lie, establishing a match that looks real, but is not. This narrative construction relies on the very structure of perception itself, in which we select cues and turn them into clues, in order to build a coherent story and attune the morphologies produced by our imagination with the optical data that guide efficacious behavior. As we have seen, veridicality is not the core business of perception, which, on the contrary, is the tentative of minimizing disorder and free energy, attuning the world through efficacious behavior (Bruineberg et al. 2018: 2; see Friston 2011).

7  Perception, Imagination and Narratives

155

There is nothing in the structure of the world that guarantees our conjectures. On the contrary, in Eco’s opinion, man is structurally devoted to falsity and error, constantly running the risk of remaining a victim of his own semiotic creations, which he uses to try to decipher the disorder of the world, but which often do nothing besides show him his own signs and references. This is the “force of falsity” that Eco always puts on stage in his novels. Starting from false ideas and incorrect conjectures, man is certainly able to discover many truths, but these discoveries are often the result of chance and error, just like those of William of Baskerville, who manages to discover the true murderer of the abbey and his mysteries by following the false scheme of the Apocalypse. That’s why the continuity between the mind and world, “Synechism” as Peirce used to call it, according to Eco is not a constitutive principle, but rather a regulative one. We act “as if” our perceptions and conjectures correspond to the structure of the real world. We would be unable to gain experience without this gamble. However, there is nothing in the structure of the world that guarantees the Spinozean parallelism that serves to secure the products of our imagination, just as Sir Arthur Conan Doyle did with those of Holmes. Exactly because perception is controlled hallucination, in the real world, unlike that of fiction, the matching between the organism and the environment is something that we tentatively try to impose on the world’s chaos, in order to produce an order. However, without any guarantee that this order exists, and corresponds to that of the world. We produce the order we need to find in the world in order to minimize disorder and free energy. I have never doubted the truth of signs, Adso; they are the only things man has with which to orient himself in the world. What I did not understand was the relation among signs. I arrived at Jorge through an apocalyptic pattern that seemed to underlie all the crimes, and yet it was accidental. I arrived at Jorge seeking one criminal for all the crimes and we discovered that each crime was committed by a different person, or by no one. I arrived at Jorge pursuing the plan of a perverse and rational mind, and there was no plan, or, rather, Jorge himself was overcome by his own initial design and there began a sequence of causes, and concauses, and of causes contradicting one another, which proceeded on their own, creating relations that did not stem from any plan. Where is all my wisdom, then? I behaved, stubbornly, pursuing a semblance of order, when I should have known well that there is no order in the universe […] The order that our mind imagines is like a net, or like a ladder, built to attain something. But afterward you must throw the ladder away, because you discover that, even if it was useful, it was meaningless. (Eco 1980: 426)

I don’t know if Umberto Eco is right. Maybe, as an enactivist working on Material Engagement Theory, I like to think that Synechism, the continuity between mind and matter, organism and the environment, can also be a constitutive principle of perception and not only a regulative one. However, and this is really a nice idea, this is exactly why Eco finds it necessary to “laugh at the truth”. Truth, being a form of correspondence between intellect and things on the one side and between the mind and the world on the other side, is a form of order that we try to impose on the world in order to gain experience. However, many other orders are possible. This is why it is always a good idea not to take just one too seriously. This is why cognitive semiotics is the discipline that studies everything that can be used in order to lie.

156

Perception as Controlled Hallucination

References Adams, R. A., Shipp, S., & Friston, K. J. (2013). Predictions not commands: Active inference in the motor system. Brain Structure and Function, 218(3), 611–643. https://doi.org/10.1007/ s00429-012-0475-5. Albertazzi, L., Thonder, V., Gert, J., & Vishwanath, D. (Eds.). (2010). Perception beyond inference: The information content of visual processes. Cambridge, MA: MIT Press. Andersen, R.  A., & Buneo, C.  A. (2003). Sensorimotor integration in posterior parietal cortex. Advances in Neurology, 93, 159–177. Basso Fossali, P. (2009). La tenuta del senso. Per una semiotica della percezione. Roma: Aracne. Borghi, A.  M., & Caruana, F. (2013). Embodied Cognition, una nuova psicologia. Giornale Italiano di Psicologia, 35(1), 23–48. Brown, J. W. (2002). Self–embodying mind: Process, brain dynamics and the conscious present. Barytown, NY: Barytown Ltd. Bruineberg, J., Kiverstein, J., & Rietveld, E. (2018). The anticipating brain is not a scientist: The free-energy principle from an ecological-enactive perspective. Synthese, 195(6), 2417–2444. Cisek, P. (2007). Cortical mechanisms of action selection: The affordance competition hypothesis. Philosophical Transactions of the Royal Society B, 362, 1585–1599. Cisek, P., & Kalaska, J. F. (2011). Neural mechanisms for interacting with a world full of action choices. Annual Review of Neuroscience, 33, 269–298. Clark, A. (2016). Surfing uncertainty: Prediction, action, and the embodied mind. Oxford: Oxford University Press. Cuccio, V., & Caruana, F. (2015). Il corpo come icona. Abduzione, strumenti ed Embodied Simulation. VS-Quaderni di Studi Semiotici, 120, 93–103. Deleuze, G. (1963). La Philosophie Critique de Kant. Paris: PUF. Eco, U. (1980). Il Nome della Rosa. Milan: Bompiani. English edition: Eco, Umberto. 1983. The Name of the Rose (trans: Weaver, W.). Boston: Houghton Mifflin Harcourt. Eco, U. (1983). Horns, hooves, and insteps: Some hypothesis on three types of abduction. In U.  Eco & T.  Sebeok (Eds.), The sign of the three. Holmes, Dupin, Peirce (pp.  198–220). Bloomington: Indiana University Press. Eco, U. (1990). I Limiti dell’Interpretazione. Milan: Bompiani. English Edition: Eco, Umberto. 1990. The limits of interpretation. Bloomington: Indiana University Press. Eco, U. (1997). Kant e l’ornitorinco. Milan: Bompiani. English edition: Eco, Umberto. 2000. Kant and the Platypus: Essays on Language and Cognition (trans: McEwen, A.). London: Secker & Warburg. Eco, U. (2007). Dall’albero al labirinto. In Studi storici sul segno e l’interpretazione. Milan: Bompiani. English edition: Eco, Umberto. 2014. From the Tree to the Labyrinth: Historical Studies on the Sign and Interpretation (trans: Oldcorn, A.). Cambridge: Harvard University Press. Eco, U., & Sebeok, T. (Eds.). (1983). The sign of the three. Holmes, Dupin, Peirce. Bloomington: Indiana University Press. Egner, T., Monti, J. M., & Summerfield, C. (2010). Expectation and surprise determine neural population responses in the ventral visual stream. Journal of Neuroscience, 30(49), 16601–16608. Fabbrichesi, R. (2014). Peirce e Wittgenstein: un incontro. Immagine, prassi, credenza. Milan: Mimesis. Friston, K. J. (2011). Functional and effective connectivity: A review. Brain Connectivity, 1, 13–36. Gallagher, S. (2017). Enactivist interventions: Rethinking the mind. Oxford: Oxford University Press. Gallagher, S. (2020). Action and interaction. Oxford: Oxford University Press. Gallagher, S., & Zahavi, D. (2014). Primal impression and enactive perception. In V. Arstila & D. Lloyd (Eds.), Subjective time: The philosophy, psychology, and neuroscience of temporality (pp. 83–99). Cambridge: MIT Press.

References

157

Gallese, V. (2009). Mirror neurons and the neural exploitation hypothesis: From embodied simulation to social cognition. In J. A. Pineda (Ed.), Mirror neuron systems (pp. 163–190). New York: Humana Press. https://doi.org/10.1007/978-1-59745-479-7_8. Gallese, V., Keysers, C., & Rizzolatti, G. (2004). A unifying view of the basis of social cognition. Trends in Cognitive Sciences, 8(9), 396–403. Gangemi, A., Alam, M., & Presutti, V. (2018). Amnestic forgery: An ontology of conceptual metaphors. Frontiers in Artificial Intelligence and Applications, 306, 159–172. Ganis, G., Thompson, W. L., & Kosslyn, S. M. (2004). Brain areas underlying visual mental imagery and visual perception: An fMRI study. Cognitive Brain Research, 20(2), 226–241. Hutto, D. D., & Myin, E. (2017). Evolving enactivism. Boston: MIT Press. Kanizsa, G. (1980). Grammatica del vedere. Saggi su percezione e Gestalt. Bologna: Il Mulino. Koenderink, J. (2010). Vision and information. In L.  Albertazzi, V.  Thonder, J.  Gert, & D.  Vishwanath (Eds.), Perception beyond inference: The information content of visual processes (pp. 27–57). Cambridge, MA: MIT Press. Marrone, G. (1995). Il dicibile e l’indicibile. Palermo: L’epos. Matteucci, G. (2019). Estetica e natura umana. In La mente estesa tra percezione, emozione ed espressione. Roma: Carocci. Paolucci, C. (2010). Strutturalismo e interpretazione. Ambizioni per una semiotica minore. Milan: Bompiani. Paolucci, C. (2017). Semiotics, schemata, diagrams and graphs: A new form of diagrammatic Kantism by Peirce. In K. Hull & R. K. Atkins (Eds.), Peirce on perception and reasoning. From icons to logic (pp. 117–133). New York and London: Routledge. Paolucci, C. (2019). Social cognition, mindreading and narratives. A cognitive semiotics perspective on narrative practices from early mindreading to autism spectruem disorders. Phenomenology and the Cognitive Sciences, 18(2), 375–400. Pinna, B., & Albertazzi, L. (2010). From grouping to visual meanings: A new theory of perceptual organization. In L. Albertazzi, V. Thonder, J. Gert, & D. Vishwanath (Eds.), Perception beyond inference: The information content of visual processes (pp. 287–344). Cambridge, MA: MIT Press. Pozzato, M. P., & Marmo, C. (1989). Dai sensi al senso: Annotazioni a margine di un seminario. Carte Semiotiche, 6, 124–147. Purves, D., Lotto, R. B., Williams, S. M., Nundy, S., & Yang, Z. (2001). Why we see things the way we do: evidence for a wholly empirical strategy of vision. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 356, 285–297. Reddy, L., Tsuchiya, N., & Serre, T. (2010). Reading the mindʼs eye: Decoding category information during mental imagery. NeuroImage, 50(2), 818–825. Rizzolatti, G., & Sinigaglia, C. (2007). Mirror neurons and motor intentionality. Functional Neurology, 22(4), 205–221. Romo, R., Hernandez, A., & Zainos, A. (2004). Neuronal correlates of a perceptual decision in ventral premotor cortex. Neuron, 41(1), 165–173. Rorty, R. (1979). Philosophy and the mirror of nature. Princeton: Princeton University Press. Sarti, A., Citti, G., & Petitot, J. (2008). The symplectic structure of the primary visual cortex. Biological Cybernetics, 98, 33–48. Stjernfelt, F. (2007). Diagrammatology: An investigation on the borderlines of phenomenology, ontology and semiotics. Dordrecht: Springer. Thom, R. (1988). Esquisse d’une sémiophysique. Paris: InterEditions. English translation: Thom, R. 1990. Semio physics: A sketch. Aristotelian physics and catastrophe theory. Boston: Addison Wesley. Trevarthen, C., & Hubley, P. (1978). Secondary intersubjectivity: Confidence, confiding, and acts of meaning in the first year. In A. Lock (Ed.), Action, gesture and symbol: The emergence of language (pp. 183–122). New York, San Francisco: Academic. Zalamea, F. (2003). Peirce’s logic of continuity: Existential graphs and non-Cantorian continuum. The Review of Modern Logic, 29, 115–162.