Bloomsbury Semiotics Volume 2: Semiotics in the Natural and Technical Sciences 9781350139329, 9781350139350, 9781350139343

Citation preview

BLOOMSBURY SEMIOTICS VOLUME 2

Bloomsbury Semiotics General Editor: Jamin Pelkey Volume 1: History and Semiosis Edited by Jamin Pelkey Volume 2: Semiotics in the Natural and Technical Sciences Edited by Jamin Pelkey and Stéphanie Walsh Matthews Volume 3: Semiotics in the Arts and Social Sciences Edited by Jamin Pelkey, Susan Petrilli and Sophia Melanson Ricciardone Volume 4: Semiotic Movements Edited by Jamin Pelkey and Paul Cobley

BLOOMSBURY SEMIOTICS

SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES VOLUME 2

Edited by Jamin Pelkey and Stéphanie Walsh Matthews

BLOOMSBURY ACADEMIC Bloomsbury Publishing Plc 50 Bedford Square, London, WC1B 3DP, UK 1385 Broadway, New York, NY 10018, USA 29 Earlsfort Terrace, Dublin 2, Ireland BLOOMSBURY, BLOOMSBURY ACADEMIC and the Diana logo are trademarks of Bloomsbury Publishing Plc First published in Great Britain 2022 Copyright © Bloomsbury Publishing Plc, 2022 Jamin Pelkey and Stéphanie Walsh Matthews have asserted their right under the Copyright, Designs and Patents Act, 1988, to be identified as Editors of this work. For legal purposes the Acknowledgements on p. xv constitute an extension of this copyright page. Cover design: Tjaša Krivec Cover illustration by Rebecca Heselton All rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or any information storage or retrieval system, without prior permission in writing from the publishers. Bloomsbury Publishing Plc does not have any control over, or responsibility for, any third-party websites referred to or in this book. All internet addresses given in this book were correct at the time of going to press. The author and publisher regret any inconvenience caused if addresses have changed or sites have ceased to exist, but can accept no responsibility for any such changes. A catalogue record for this book is available from the British Library. A catalog record for this book is available from the Library of Congress. ISBN: HB: 978-1-3501-3932-9 ePDF: 978-1-3501-3934-3 eBook: 978-1-3501-3933-6 Set: 978-1-3501-3944-2 Series: Bloomsbury Semiotics Typeset by Integra Software Services Pvt. Ltd. To find out more about our authors and books visit www.bloomsbury.com and sign up for our newsletters.

CONTENTS

L ist

of

F igures 

vii

L ist

of

T ables 

viii

L ist

of

C ontributors 

ix

A cknowledgements 

xv

L ist

xvi

of

A bbreviations 

Introduction

1

Stéphanie Walsh Matthews 1 Semiotics in Mathematics and Logic Ahti-Veikko Pietarinen and Frederik Stjernfelt

13

2 Semiotics in General Biology  Kalevi Kull and Donald Favareau

35

3 Semiotics in Ecology and Environmental Studies  Timo Maran

57

4 Semiotics in Ethology and Zoology  Morten Tønnessen

75

5 Semiotics in Evolutionary Linguistics Jamin Pelkey and Prisca Augustyn

93

6 Semiotics in Health and Medicine  John Tredinnick-Rowe and Donald E. Stanley

119

7 Semiotics in Psychiatry and Psychology  Norbert Andersch

143

8 Semiotics in Neuroscience and Cognition  Kristian Tylén and Jijo Kandamkulathy

173

9 Semiotics in Computing and Information Systems  Martin Irvine

203

10 Semiotics in Economics and Finance Todd Oakley

239

vi

CONTENTS

11 Semiotics in Law and Jurisprudence  Clara Chapdelaine-Feliciati

259

12 Semiotics in Architecture and Spatial Design  Gabriele Aroni

277

13 Semiotics in Graphic Design  Steven Skaggs

297

14 Semiotics in Marketing and Branding  Kristian Bankov and Dimitar Trendafilov

319

I ndex 

342

LIST OF FIGURES

  6.1 Example of an EEG reading with associated hyperkinetic movements

129

  8.1 fMRI brain imaging

177

  8.2 Paul Broca’s original record of Lebourgne autopsy

179

  8.3 Contemporary version of the Wernicke-Gerschwind model of language processing

180

  8.4 Illustration of the cortical areas involved in early visual perception

184

  8.5 Areas of the ‘Social Brain’

187

13.1 Poster, Cranbrook Graduate Program in Design, 1989

301

13.2 Compositional styles carry semantic connotations

303

13.3 Page grammar

310

13.4 Functional matrix

311

13.5 Cluster analysis of gastronomical terms from inhabitants of the arctic

312

13.6 The Visual Gamut with fifteen nodes illustrating blends of iconicity, indexicality and symbolicity

313

13.7 Signature of designer Saul Bass, c. 1980

314

14.1 Commercial communication as a system consisting of four main agents (D.T)

323

14.2 Brand meaning model

326

14.3 A discourse scheme of the Apple brand on a synchronic level

327

14.4 A grouping of the codes according to their place in time and their role in culture 328 14.5 The axiology of consumption scheme

332

14.6 The typology of the travellers

333

14.7 The brand identity audit of Levi’s

335

14.8 Semiotic mapping of the values of the brand

336

14.9 Trend mapping in the alimentary sector

338

LIST OF TABLES

6.1 Stereo electroencephalography, HMS – hypermotor seizure

130

9.1 Computer system levels: Overview of the semiotic system stack

209

9.2 Important concepts in Peirce’s writings on Logic as Semeiotic (1902–12) 212 9.3 Major developments in computing and semiotics

213

9.4 Research and theory on semiotics and computing, 1980s to present

216

9.5 Computer system homologies

219

9.6 Programming and software: Source code to dialogic interaction

223

LIST OF CONTRIBUTORS

Norbert Andersch (MD, MRCPsych) is a neurologist and psychiatrist with a special interest in psychopathology, neuro-traumatology and the concept of schizophrenia. He has practised as a clinician and medical expert for more than four decades in Germany and England. After 2000 he worked as Consultant Psychiatrist in East London reconstructing local Mental Health Teams and leading Psychosis Crisis and Community Teams in Central and South London (for Maudsley NHS Foundation Trust/King’s Health Partners) from 2003 to his retirement in 2016. He researches the impact of symbol-formation on the make-up of consciousness and Ernst Cassirer’s symbol theory in connection to mental illness. His book Symbolische Form und psychische Erkrankung [Symbolic Form and Mental Illness] was published in 2014. He is a Member of the Royal College of Psychiatrists and its Philosophy Special Interest Group. He is currently lecturing in psychopathology at Sigmund Freud Privat University (Vienna, Berlin, Paris). Gabriele Aroni is Assistant Professor in the School of Cultural Technology at Xi’an Jiaotong-Liverpool University, Suzhou, Jiangsu, China. His research is situated at the intersection of semiotics, architecture, game studies and cultural heritage. He published articles in Well Played, the Journal of Media Research, Annali di Architettura and Southern Semiotics Review as well as a book on Renaissance architecture (Mimesis: Milan, 2016). He presented at conferences in China, Canada, Romania, Portugal and Italy. Dr Aroni is also a researcher at the Global Academic Alliance for Partnership on a Community with Shared Future at the Communication University of China in Beijing, as well as the Multilanguage Cultural Heritage Lexis Research Project at the University of Florence, Italy, and the International Scientific Committee on Interpretation and Preservation of Cultural Heritage Sites for ICOMOS. Prisca Augustyn is Full Professor of Linguistics and German Studies in the Department of Languages, Linguistics, and Comparative Literature at Florida Atlantic University, in Boca Raton, USA. She teaches graduate and undergraduate courses in linguistics, semiotics and German Studies, and is coordinator of the German language program. She holds a PhD in Germanic Linguistics from UC Berkeley (2000), where she studied with Irmengard Rauch. She is co-author with Nikolaus Euba (UC Berkeley) of Welten (2021), an introductory German language program, and Stationen 4th Ed. (2019), an intermediate German language program with Cengage. In the context of language learning and teaching, she is interested in the role of translation, vocabulary acquisition and extensive reading. Her recent publications deal with connections between Biosemiotics and Biolinguistics, the work of Jakob von Uexküll, Ecolinguistics, Cognitive Linguistics, and the relationship between language and belief.

x

LIST OF CONTRIBUTORS

Kristian Bankov is Full Professor of Semiotics at the New Bulgarian University, Sofia, Bulgaria and visiting professor at Sichuan University, Chengdu, China. He is Chair of the Southeast European Center for Semiotic Studies at NBU. Prof Bankov has been teaching courses in Marketing and Brand Semiotics since 1995. He is editor-in-chief of the journal Digital Age in Semiotics and Communication. He has written five books and more than hundred articles in Bulgarian, English and Italian. He has been visiting professor or taught courses in Helsinki University, Sichuan University, University of Turin, Hanoi University. In 2014 he was elected as the Secretary General of the International Association for Semiotic Studies (IASS). His interests in semiotics started during the early 1990s when he was studying in Bologna, following the courses of Ugo Volli and Umberto Eco. In 2000 he defended his PhD in Helsinki with Eero Tarasti. In 2006 he became Associate Professor in Semiotics, and he became Professor in 2011. In 2011–12 he served as vice rector for International Affairs and Public relations of New Bulgarian University. Clara Chapdelaine-Feliciati is a licensed lawyer (Barrister and Solicitor) with the Law Society of Ontario, Canada, and Assistant Professor at the School of Public and International Affairs at York University, Toronto. She holds a PhD in Law from Oxford University, a Juris Doctor and Bachelor of Civil Law (J.D. and B.C.L.), McGill University, and a Master of Laws (LL.M.), King’s College London. She is the author of Feminicides of Girl Children: An International Human Rights Law Approach (Brill 2018), and published articles in the Cambridge International Law Journal, International Journal of Children’s Rights, Semiotica, and International Journal for the Semiotics of Law. She co-founded the Girls’ Studies Research Network and is conducting a bilingual study on girl performers and the law in Canada, funded by the Social Sciences and Humanities Research Council. Her research interests include legal significs, semiotics and semioethics, international law, child law and feminist legal theory. Donald Favareau is an Associate Professor at the University Scholars Programme at the National University of Singapore, where he has been teaching the first university course on biosemiotics in Asia since his arrival there in 2004. An editor of the Springer Book Series in Biosemiotics, he is also the co-founder and Vice-President of the International Society for Biosemiotic Studies (2005–12 and 2015 to the present), and the author of numerous publications on biosemiotics, including the 2011 anthology with commentary Essential Readings in Biosemiotics, the edited volumes Co-operative Engagements in Intertwined Semiosis (2018) and, with Paul Cobley and Kalevi Kull, A More Developed Sign: Interpreting the Work of Jesper Hoffmeyer (2012). His primary research interests are biosemiotics, cognitive science and the history and philosophy of science. Martin Irvine is Associate Professor and Founding Director of the Graduate Program in Communication, Culture & Technology (CCT) at Georgetown University, Washington, DC, USA. Prof Irvine has maintained several parallel interests: languages and literature; philosophy; semiotics; intellectual history; and computing and programming. He set up the first Web server at Georgetown University in 1993 (the Labyrinth, the first humanities site), and founded the CCT Graduate Program at Georgetown University in 1995, where he now teaches courses on design theory, semiotics and computing, and the design principles of AI and computing systems. Recent publications have been on

LIST OF CONTRIBUTORS

xi

Peircean dialogism and semiotic principles of art and music genres. His current research focuses on the intellectual history of computing from Leibniz to the present in a Peircean semeiotic framework, including a comprehensive study of Peirce’s unpublished papers. His forthcoming book will be a new synthesis of the history of logic, mathematics, semiotics and computation, which reveals the essential semiotic foundations of modern computing, information and all things digital. Kandamkulathy Jijo is a PhD Researcher on the interface between semiotics and cognitive neuroscience at Saint Joseph University, Macau, China. Former Vice Principal of Saint Claret College, Ziro, he has presented a paper on Transmodal Arbitrariness in Berkeley in the Annual Biosemiotics Gathering. He currently researches on semiotic substitution of vidisigns and haptisigns. Kalevi Kull is Full Professor of Biosemiotics in the Department of Semiotics at the University of Tartu, Estonia. His research area includes biosemiotics, theoretical biology, semiotic ecology and general semiotics. He is co-series editor (with Paul Cobley) of Semiotics, Communication and Cognition (de Gruyter Mouton), Biosemiotics and Tartu Semiotics Library, he co-edits the journal Sign Systems Studies. He is the fifth Thomas A. Sebeok Fellow of the Semiotic Society of America and President of the International Society for Biosemiotic Studies. Timo Maran is Full Professor of Ecosemiotics and Environmental Humanities at the University of Tartu, Estonia. His research interests are semiotic processes in ecosystems and ecocultures, Estonian nature writing, zoosemiotics and species conservation, and semiotics of biological mimicry. He is the author of Mimicry and Meaning: Structure and Semiotics of Biological Mimicry (Springer, 2017) and Ecosemiotics: The Study of Signs in Changing Ecologies (Cambridge University Press, 2000). He has been an editor of the journal Biosemiotics (Springer, 2013−20) and the series Environmental Humanities (Cambridge University Press, since 2018). Timo Maran is also a writer and has published five poetry collections. Todd Oakley is Full Professor and Chair of the Department of Cognitive Science at Case Western Reserve University, Cleveland, Ohio, USA. His research and teaching interests include cognitive linguistics, semiotics and rhetorical theory, each of which is represented in his most recent book, Rhetorical Minds: Meditations on the Cognitive Science of Persuasion (2020). He is also author of From Attention to Meaning (2009) and co-editor of Mental Spaces in Discourse and Interaction (2008). He is the past president of the International Association for Cognitive Semiotics, and an Associate Editor of the journal Cognitive Semiotics. Jamin Pelkey is Associate Professor and Program Director in the Department of Languages, Literatures, and Cultures, Toronto Metropolitan University. He serves as Co-Editor of Semiotica, president of the International Association for Cognitive Semiotics, and as executive board member of the International Cognitive Linguistics Association, the International Society for Biosemiotic Studies, and the Semiotic Society of America. Jamin has edited or co-edited fifteen collections in linguistics and semiotics. He is the 2017 recipient of the Mouton d’Or Award for best article in Semiotica, and his research explores semiotic dimensions of language evolution and embodied cognition,

xii

LIST OF CONTRIBUTORS

with publications in venues such as Cognitive Semiotics, Diachronica, The Journal of Literary Semantics, Linguistics of the Tibeto-Burman Area, Reviews in Anthropology, Semiotica, Sign Systems Studies, Studies in Language, and Symmetry. His authored books include Dialectology as Dialectic (De Gruyter, 2011) and The Semiotics of X (Bloomsbury, 2017). Ahti-Veikko Pietarinen is Full Professor in the Nurkse Department of Innovation and Governance, Tallinn University of Technology, Estonia. He has published over 150 refereed papers in philosophy of science, philosophy of language, philosophy of mathematics, natural-language semantics and pragmatics, game theory, American pragmatism, history of analytic philosophy, phenomenology and cognitive neurosciences, and history of ideas. His books include Signs of Logic: Peircean Themes on the Philosophy of Language, Games, and Communication (Springer Synthese Library, 2006), Game Theory and Linguistic Meaning (CRiSPI, Oxford: Elsevier Science, 2007), and the multivolume Logic of the Future (De Gruyter) in his edited book series Peirceana. His journal publications include papers in Synthese, Journal of the History of Ideas, Semiotica, Perspectives on Science, History and Philosophy of Logic, Linguistics and the Human Sciences, Studia Logica, Cognitio, History of Philosophy Quarterly, Notre Dame Journal for Formal Logic and Pragmatics & Cognition. Steven Skaggs is Professor of Design at the University of Louisville. He has worked as both a creative artist and theorist, with particular interests in typography, calligraphy, and the variety of ways verbal content is transformed when it is made visual. His art is in the permanent collections of the Akademie der Künste (Berlin), Klingspor Museum (Offenbach) and the Sackler Collection of Visual Poetry (University of Iowa). He’s the author of Logos: The Development of Visual Symbols (Crisp, 1994) and FireSigns: A Semiotic Theory for Graphic Design (MIT, 2017), the latter being named an Outstanding Academic Title by the Association of Research Libraries. In basic semiotic matters, he considers himself a ‘neo-Peircean’, although he rejects the universal necessity of the dynamical object, an apostasy that puts him at odds with Peirceans of a more traditional cast. Donald E. Stanley (FCAP, FASCP) is a semi-retired pathologist, with a special interest in cytology, trained at the University of Vermont and Karolinska Radiumhemmet. His interest is in medical decision-making and how the use of language impedes or allows for the patient to understand complex treatment interventions. He reviews randomized clinical trials for McMaster Online Rating of Evidence and is an associate editor at Dynamed. His most recent publications are ‘Selecting Clinical Diagnoses: Logical Strategies Informed by Experience’ (with D. G. Campos in the Journal of Evaluation in Clinical Practice, 2015), ‘Medical Reasoning and Doctor-patient Communication’ (with S. R. Sehon in the Journal of Evaluation in Clinical Practice, 2019) and ‘Strategies in Abduction: Generating and Selecting Diagnostic Hypotheses’ (with R.  Nyrup in The Journal of Medicine and Philosophy, 2020). Frederik Stjernfelt is Full Professor of semiotics, intellectual history and the philosophy of science in the Department of Communication and Psychology at Aalborg University, Copenhagen, Denmark. He is a critic for Weekendavisen. His books in English include Diagrammatology (Springer, 2007), Democratic Contradictions of Multiculturalism

LIST OF CONTRIBUTORS

xiii

(with J. M. Eriksen, Telos, 2012) and Natural Propositions (Docent, 2014), and his papers are published in Semiotica, Synthese, Sign Systems Studies, Transactions of the Charles S. Peirce Society, among venues. He is PI (with David Budtz) of the Humanomics Center, conducting meta-studies in the state of the humanities, the philosophy of science and related disciplines. A selection of papers can be found at: http://frederikstjernfelt.dk. Morten Tønnessen is Professor of Philosophy and Head of department of the Department of Social Studies at the University of Stavanger (UIS), Norway. His PhD was defended at University of Tartu´s Department of Semiotics, Estonia (2011). Tønnessen has been an editor-in-chief of the journal Biosemiotics (2013–20, lead editor-in-chief 2018–20), and is currently President of the Nordic Association for Semiotic Studies (2017–). He is also a member of Norway´s Council for Animal Ethics (2017–), and chair of the steering group for Cognitive and behavioural neuroscience lab (Cognitive lab, UIS). Recent books include The Semiotics of Animal Representations (Rodopi 2014, co-edited), Animal Umwelten in a Changing World: Zoosemiotic Perspectives (Tartu University Press 2016, co-edited and co-written), Thinking about Animals in the Age of the Anthropocene (Lexington Books 2016, co-edited) and Semiotic Agency: Science beyond Mechanism (Springer 2021, co-written with Alexei Sharov). John Tredinnick-Rowe is a Research Fellow in complex interventions at the Peninsula Medical School, Plymouth University, UK. His research interests include health services, trials methodology, medical education, knowledge exchange and commercialization (KEC) and medical semiotics. He has written widely about the intersection of semiotics and healthcare. Extending from theory development to the design and implementation of clinical consultation models for family doctors (SHERPA). He also holds an honorary Senior Research Fellowship at Exeter Medical School, and board-positions with the Royal College of Radiologists and Medical Research Council’s working group on Health Informatics. Dimitar Trendafilov is MBA and PhD in Theory of Culture (semiotics) from New Bulgarian University. The focus of his research is applied semiotics in the realm of branding and commercial communication. He has been a full-time lecturer in marketing and brand management and a researcher at NBU since 2015, as well as a member of the South-East European Center for Semiotic Studies. In 2017 he published his dissertation entitled Semiotic Studies on Brand: Tools, Analyses, Results and he published the monograph Mobile Consumer in 2021. Most of his articles and collection chapters in the last decade were dedicated to various case studies and applications of semiotics in marketing practice. Dimitar has had successful projects completed for companies in various industries in Bulgaria, including banking, cosmetics, communications, tobacco and fashion. Kristian Tylén is Associate Professor at the Department of Linguistics, Cognitive Science and Semiotics, School of Communication and Culture, at Aarhus University, Denmark. His main research interests include the study of cognitive and social mechanisms underlying the evolution of human symbolic behaviour, the psycholinguistics of dialogue, collective problem solving and creativity, meaning-construction, and the role of objects in human cognition. He approaches these topics using a broad range of research methods from the cognitive sciences including behavioural experimentation, fMRI brain imaging,

xiv

LIST OF CONTRIBUTORS

physiological measurements, statistical modelling as well as conceptual/theoretical analysis. He was among the initiators of the International Association of Cognitive Semiotics and is still serving on the board. Stéphanie Walsh Matthews is Associate Professor in the Department of Languages, Literatures & Cultures, Toronto Metropolitan University, Canada and one of the Editors-in-Chief for the International Association of Semiotic Studies’ flagship journal, Semiotica. She is currently editing a section of Springer’s Handbook of Cognitive Mathematics, under Professor Marcel Danesi’s editing leadership. She is the co-director of the Meaning Lab at Ryerson University and leads a research group using humanoid robots to investigate the language practices of children with autism spectrum disorder. Along with her publications in semiotics, language and culture, her research also includes post-colonial and magical realism in literature. She supervises students in the joint graduate program of Communication and Culture.

ACKNOWLEDGEMENTS

Out of all the hundreds of individuals involved in this project, we owe a big thank you up front to Andrew Wardell, Senior Commissioning Editor of Linguistics at Bloomsbury from 2017–20. Without Andrew’s invitation and his ensuing vision, enthusiasm and patient support for this project, it would never have gott off the ground; and it certainly wouldn’t have grown from a single volume ‘companion’ into a four-volume ‘major reference work’. Andrew issued the invitation for this project while still serving as Editorial Assistant to Gurdeep Mattu. Many thanks are also due to Becky Holland, Editorial Assistant to Andrew from 2018–21; to Morwenna Scott, Senior Commissioning Editor of Linguistics since 2020; and to Laura Gallon, Editorial Assistant to Morwenna since the end of 2021 – each of whom played key roles in shepherding the project along to publication against the bottomless backdrop of tragedy, uncertainty and delay that marked the Covid-19 pandemic. Additional vital support from Bloomsbury during the project’s final stages came from Production Editor, Elizabeth Holmes, and from Dharanivel Baskar, Team Lead for project management at Integra Software Services, both of whom engaged untold support from their own dedicated teams. It has been a pleasure to work with all of you: thank you once again. During the final two years of intensive work, this project benefitted financially from a number of research grants, including a 2020–21 collaborative publishing grant from the Faculty of Arts and Creative Industries, Middlesex University, London (via Paul Cobley) and three grants from Toronto Metropolitan University – including a 2020–21 Faculty Research Grant from the Faculty of Arts (via Jamin Pelkey), a 2021 graduate research assistant grant from the TMU-York Graduate Program in Communication and Culture (via Stéphanie Walsh Matthews) and a 2021–22 Work Study Research Assistant Grant from the Office of the Vice President of Research and Innovation (via Jamin Pelkey). These funds paid for logistical and copy editing support involving seven student researchers: three PhD researchers (Sophia Melanson Ricciardone, Richard Rosenbaum and Jan Vykydal) and four undergraduate researchers (Sari Park, Irene Storozhuk, Leonard Pamulaklakin and Kai Maurin-Jones). Many thanks to all. Appreciation is also due to nine anonymous peer reviewers for their invaluable guidance and feedback, including a converging series of recommendations that ultimately led to the expansion of the project from one volume to four. The project also benefited from countless consultations, conversations and inspirations afforded by connections with semioticians around the world. Let us say thank you in this regard to colleagues in the Semiotic Society of America, the International Association for Cognitive Semiotics, the International Cognitive Linguistics Association, the International Society for Biosemiotic Studies, and the International Association for Semiotic Studies, many of whom have played a formative role, suggesting in the process the need for additional complementary volumes of this nature to offer expanded scope, depth and range, in terms of topics, angles of coverage, and qualified participants. Semiotics, after all, is still only just getting started; and if these volumes serve the enterprise en route to its next milestone, it is thanks to everyone involved.

LIST OF ABBREVIATIONS Guide to Critical Peirce Editions

Charles S. Peirce: Primary sources

CD

CN

CP

EP 1

EP 2

LoF

LI

MS

Peirce, C. S. (1889–91), entries in The Century Dictionary: An Encyclopedic Lexicon of the English Language, W. D.Whitney (ed), New York: Century Co. Cited as CD followed by page number. Peirce, C. S. ([1901–8] 1975–9), Charles Sanders Peirce: Contributions to ‘The Nation’, 3 vols., K. L. Ketner and J.E. Cook (eds), Lubbock: Texas Tech University Press, 1979. Cited as CN followed by volume number and page number. Peirce, C. S. ([1866–1913] 1931–58) The Collected Papers of Charles Sanders Peirce, vols. 1–6, C. Hartshorne and P. Weiss (eds), vols. 7–8, A. Burks (ed), Cambridge: Harvard University Press. Cited as CP, followed by volume number and section number. Peirce, C. S. ([1867–93] 1992), The Essential Peirce: Selected Philosophical Writings, vol. 1, in N. Houser and C. Kloesel (eds), Bloomington: Indiana University Press. Cited as EP 1, followed by page number. Peirce, C. S. ([1893–1913] 1998), The Essential Peirce: Selected Philosophical Writings, vol. 2, Peirce Edition Project (eds), Bloomington: Indiana University Press. Cited as EP 2, followed by page number. Peirce, C. S. ([1895–1910] 2019–21), Logic of the Future: Writings on Existential Graphs. A.-V.Pietarinen (ed), vol. 1: History and Applications; vol. 2/1: The Logical Tracts; vol. 2/2: The 1903 Lowell Lectures; vol. 3/1: Pragmaticism; vol. 3/2: Correspondence, Berlin: De Gruyter. Cited as LoF followed by volume number and page number. Peirce, C. S. ([1891–1910] 2009), The Logic of Interdisciplinarity: The Monist Series, E. Bisanz (ed), Berlin: Mouton de Gruyter. Cited as LI followed by page number. Peirce, C. S. ([1857–1914] 1787–1951), The Charles S. Peirce Papers Harvard University, Houghton Library, MS Am 1632. Individual papers are referenced by manuscript number in R. Robin (ed), Annotated Catalogue of the Papers of Charles S. Peirce, Amherst: University of Massachusetts Press, 1967, and in Robin, ‘The Peirce Papers: A Supplementary Catalogue’, Transactions of the Charles S. Peirce Society 7, 1971: 37–57. Cited as MS, followed by manuscript number and page number.

LIST OF ABBREVIATIONS

xvii

Note: Houghton Library catalog available online: https://hollisarchives.lib. harvard.edu/repositories/24/resources/6437. Many of Peirce’s papers and letters are available in the Microfilm Edition, Harvard University Library, 38 Reels (1966–70). Digital images of the microfilm, in the Robin catalog number sequence, are also available online: https://rs.cms.hu-berlin.de/peircearchive/pages/home.php. NEM

PoM

PPM

SS

SWS

W

Peirce, C. S. ([1866–1913] 1976), The New Elements of Mathematics, 4 vols., C. Eisele (ed), The Hague: Mouton Press. Cited as NEM, followed by volume number and page number. Peirce, C. S. ([1888–1908] 2010), Philosophy of Mathematics: Selected Writings, M. E. Moore (ed), Bloomington: Indiana University Press. Cited as PoM. Peirce, C. S. ([1903] 1997), Pragmatism as a Principle and Method of Right Thinking: The 1903 Harvard Lectures, P. Turrisi (ed), New York: SUNY Press. Cited as PPM, followed by page number. Peirce, C. S. and V. L. Welby ([1903–1911] 1977), Semiotic and Significs: The Correspondence between Charles S. Peirce and Victoria Lady Welby, C. Hardwick and J. Cook (eds), Bloomington: Indiana University Press. Cited as SS, followed by page number. Peirce, Charles S. ([1894–1912] 2020), Selected Writings on Semiotics, 1894–1912, F. Bellucci (ed), Berlin: Mouton De Gruyter. Cited as SWS followed by page number. Peirce, C. S. ([1857–92] 1982–2010), Writings of Charles S. Peirce, 7 vols. (1–6, 8), Peirce Edition Project (eds), Bloomington: Indiana University Press. Cited as W, followed by volume number and page number.

xviii

Introduction Semiotics in the Natural and Technical Sciences STÉPHANIE WALSH MATTHEWS

If the world is ‘perfused with signs’, as C. S. Peirce suggests, and even potentially ‘composed exclusively of signs’ (1906: EP 2.394), there could be no world without signs. In that case, all sciences that study our world, whether natural, technical or otherwise, would be sign sciences. Readers who are familiar with the many applications of semiotics to disciplines associated with the social sciences and humanities may be somewhat surprised to see chapters in the second volume of this reference set focused on applications to the natural and technical sciences. One may ask if semiotics might be emerging from its basis in philosophy and linguistics, established a little over a hundred years ago, and seeking to insert itself into uncharted territories – a paradigm shift that may confound. Such a shift may appear to be out of character to those who continue to see semiotics as the handmaiden of fields such as critical theory, or to those who continue to espouse poststructuralist views of signs as indicators of meaning slippage. Such traditional users of semiotics, and others besides, may well wonder what semiotics has to offer those disciplines that seek to understand aspects of the world that are evolutionary, biological, ecological, neurological, technological, financial, legal and architectural. Are such applications practicable or even possible? In fact, as Peirce knew, the world can only be understood by deciphering the signs that are immanent in it. Moreover, semiotic analysis itself stems from that world, which provides not only our objects of inquiry, but also their sign vehicles and interpretants in processes of mutual entanglement. Understanding the nature of these relationships is a pursuit germane to all disciplines, including the natural and technical sciences (see, e.g., Nöth 2001; Midtgarden 2020). The main thrust of the sciences is an attempt to provide models of the world, and then to use these models to build intellectual structures and institutions; the models themselves are sign structures that provide ‘views’ as to why and how things work (Anderson et al. 1984; Cunningham 1998; Sebeok and Danesi 2001). Hence, semiotics is at home in the natural and technical sciences and should be applied in the disciplines that study these domains, starting with how they inform the web of meaning, called the semiosphere, which reveals that the species-specific world, or Umwelt, in which we live, is interrelated with those of other species in a shared existence with all other realities – whether organic or inorganic, imagined or real, that make up the world (Kull 1998). It is of little surprise that, with the panoply of sign systems that

2

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

surround us, sign systems are remarkably uniform in their functioning. As we humans are afforded the unique ability to think of signs as signs, we are particularly well-suited to explore nature with sign structures, from the study of mental structures to the study of the very cells that provide a dynamic biological unity among animals and humans. Studying this unity involves mapping the study of signs, semiotics, onto the study of nature in all of its manifestations (Nöth 2001). We use signs; they do not use us (Pablé 2016). They are the DNA of cognition that have allowed humans to evolve in a unique way – through nature and history. This volume is dedicated to documenting the contributions semiotics has made, and stands to make, to disciplines that study the semiotic origins of this evolutionary paradigm and its consequences. There is thus no paradox to this volume. Although many of the chapters collected here deal with various complex perspectives of the so-called ‘hard sciences’, the theme running through them is that these perspectives are based on semiosis and thus that understanding the natural world starts with understanding how we represent it semiotically. This, in turn, includes embracing all perspectives under the rubric of semiotics, from biology to mathematics, which end up shaping not only the semiosphere but also the biosphere that we occupy. Many of the chapters in the first half of the volume function as what can be called natural semiotics. As is well known, semiotics emerges as a discipline in antiquity as a study of natural (bodily) signs, which were subsequently differentiated from conventional (human-made) signs. Semiotic practice during this stage served to differentiate the natural world from the conventional world. It was chiefly interested in the manifestations of bodily symptoms, starting with Hippocrates, who saw a connection between these symptoms and the environment in which a patient lived: the body and the world. Far from being a radical break from semiotic theory, these chapters constitute a retrieval of its origins, as a method capable of studying and explaining natural phenomena, including interactions between humans and the environment in which they exist. The volume not only retrieves natural semiotics by exploring its contributions to the study of medicine, animal communication, ecology, biological systems, etc. but also extends to what can be called artificial semiotics, which includes contributions to the technical disciplines – such as the study of mathematics, logic, neuroscience, artificial intelligence on the one side – and studies of our built worlds on the other – including economic, legal, architectural, design and marketing systems that are intrinsic to the semiosphere and which affect our worldviews according to place and time. In other words, these latter domains of inquiry and practice reify our experience of being in the world, through complex networks of signification, all of which require interpretation on the part of the scientist. Long before Peirce expounded the need to embed phenomenological semiotics in logic, nonverbal sign use and its basis in producing reasoning systems were studied largely through the analysis of mathematical systems. Even today, many nonverbal signs are built mathematically, such as those in artificial intelligence systems. In the first chapter of this volume, Stjernfelt and Pietarinen’s ‘Semiotics in Logic and Mathematics’ demonstrates that notation, history, innovation and proof owe a great deal not only to semiotics as a perspective on these disciplines, but to our ability to produce meaningful representations of the world in which we exist. In fact, everything that has been invented for ‘math’s sake’ is, in actuality, built by mathematical means. Mathematical notations and language are actually much more intertwined in natural information processing than may have been previously thought. Semiotics, therefore, will provide key insights into how the intertwining is based on parallel sign structures at work.

INTRODUCTION

3

Moreover, mathematical notation (often considered the ideal system of semiotics) may be more powerful cognitively than language, since it utilizes diagrammatic representations that are packed with more meaning than assemblages of words. Stjernfelt and Pietarenen systematically demonstrate, in fact, how semiotics is itself intrinsically mathematical (as is logic) and how the diagrams of mathematicians put on display the reasoning they use. In fact, math and logic have ensured scientific progress thanks to the use of notation as a semiotica utens. Diagrams provide implications that allow us to construct analogies that elucidate the relations between the parts of phenomena. Moving through abduction and induction, math allows us to apprehend the world (or at least perceive it through thought experiments) via its diagrammatic notation, allowing us to grasp it holistically, as Kant often pointed out, and thus better understand it (Kant [1786] 2004). From the intuitive way in which we sense how it is to be in the world, natural semiotics allows us to examine concretely what the world is. In their chapter on ‘Semiotics and General Biology’, Kull and Favareau review the semiotic processes behind life’s processes. Indistinguishable from life itself, semiosis, the process of meaning-making, explicates how biology needs semiotics. In general terms, all living things need to make meaning; without it, there is no life of which to speak, since meaning is what defines life in a sentient way. Aiming to provide a unified model of semiosis, Kull and Favareau’s chapter makes a claim for establishing a broader semiotic biology (or biosemiotics). Starting with an indepth survey of the contributions to this field by von Uexküll, Sebeok, Deely, Morris, Deacon, Kull, Maran, Favareau and Tembrock, the authors make annotations throughout regarding the implications of using semiotics in biology, given that it provides an insightful expliqué of the experience of being in the world. The authors argue that a semiotics in biology will avoid the pitfalls of nominalism, by focusing on the various evolutionary and teleological elements of life for which signs have been devised as models of those elements, even though they may be re-shaped to serve specific human intents. This thematic approach is extended in Chapter Three, ‘Semiotics in Ecology and Environmental Studies’, by Maran, who establishes important connections between general semiotics and ecological science, which, like semiotics, is interested in relations between the human and natural worlds and actually surfaces in the ideas of structural thinkers of the nineteenth century. Maran seeks to provide an overview of notions of population, community, ecosystems and ecology from the perspective of semiotics, highlighting the pre-existing relations and similarities between the two disciplines. Maran dispels the notion that the continuum between nature and culture is typologically hierarchical, arguing that it is holistic, whereby humans and other organisms interact in various modes of environmental relationships with each other. Presenting notions from the subfield of open semiotics, such as ecofield and semiotic niche, Maran maintains that the study of the diversity of meaning systems is derived from relations inherent in differing environment conditions. Maran’s model of the relationship between species and their environments meshes the previous Umwelt approach with Kull’s notion of ecological fitting, a co-existence modality that makes for a more rapid process of relational fitting. This is a marked departure from notions such as the slow erosion of time. Rapid processes result from relational fitting, as initially discussed by Hoffmeyer (2008: 107–8,  345), constituting specific interactions between species and environments that manifest themselves in patterns and structures that show how the different species react to the world in terms of specific ecological codes. Stressing that ‘semiotic regulation in the ecosystem is contextual and cumulative’, Maran’s chapter allows us to better understand that ecosystems provide the conditions for both biological and semiotic adaption. All

4

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

of these evolve, and co-evolve, fitting in and forging forward. Maran thus moves away from the typically hierarchical, binary, dualistic treatments of culture and nature. Rather, he presents multi-perspectives based on local cosmologies and Peircean categories. He discusses the related notions of ecosemiosphere, characterized as an ecological bubble of meaning, and semiocide, or the negative impact of culture on nature. The fourth chapter, ‘Semiotics in ethology and zoology’, by Tønnessen, reflects on semiotic contributions to the study of communicative and behavioural abilities among non-human animals, treated since 1963 under the rubric of zoosemiotics: ‘the scientific study of signaling behavior in and across animal species’ (Sebeok 1963: 465). Tønnessen reviews various overlapping terms in this subfield, focusing on semiotic affordances that have shaped biology from the outside. The semiotic study of non-human animals has allowed for a bridging of the natural sciences with the humanities and social sciences via semiotic method. Like some of the other chapters in this volume, Tønnessen starts with the ideas Jakob von Uexküll, the founder of the semiotic approach to ethology and zoology, which have been elaborated by semiotic thinkers such as Morris, Sebeok, Magnus, Martinelli, Maran and Tønnessen himself, all of whom made non-human animals both a subject and object of semiotics. From their perspectives and approaches, zoosemiotics has become a fertile interdisciplinary and intertheoretical field offering helpful perspectives on the complexity of biological systems. Tønnessen concludes with a plea on the need for future studies in animal-human relations, incorporating Actor-Network Theory and Maran’s approach to Latour’s work on objects and subjects. What sets humans apart from other animals, in both an objective and semiotic sense, is language. In ‘Semiotics in Evolutionary Linguistics’, Pelkey and Augustyn, in keeping with the views of Deacon, Sebeok, Fauconnier, Turner, Harari, Tattersall and others, argue that language is the system that binds virtual and real worlds in human consciousness. The chapter looks specifically at the many points of contact between evolutionary linguistics and semiotics (semiotics being concerned with the nature of the relationship of language to other sign systems). The study of linguistic evolution as a means to understand human phylogeny had existed long before Darwin’s theory of evolution; hence the inclusion of evolutionary linguistics in this initial suite of chapters developing links between semiotics and the natural sciences. An example of how linguists and biologists use the same type of metaphorical understanding can be seen, for instance, in the concept of ‘tree structure’ used in both biology and historical linguistics. The chapter provides a critical review of evolutionary patterns from a semiotic perspective, thus situating semiotics within the larger frame of historical and comparative linguistics. Through this, the importance of diagrammaticality is highlighted as key for understanding language as an adaptation of cognitive structures, which were themselves later exapted for verbal and signed communication, rather than being an evolutionary adaptation from primate non-verbal communication. The ability to think about signs underscores the singularity of human language, which serves as a superior modelling system, in sync with Uexküll’s Umwelt notion, that is both a part of, yet separate from, all other animal communication systems, since it is a reflexive tool for thought, for example. The chapter strongly emphasizes, following Chomsky, Peirce and others, that language is not a mechanical communicative device or the unique locus of symbolic capacity; rather, it is a complex modelling system for modelling possible worlds that scaffolds the entire species’ experience of the world. The relationship of linguistics to semiotics seems an obvious one, given Saussure’s semiology, which includes linguistics as one of its branches. A lesser known and oft overlooked area of connectivity is the historical relationship of semiotics to medicine and

INTRODUCTION

5

healthcare, which is, as mentioned above, a key aspect of the history of semiotics. This is a point that the sixth chapter on ‘Semiotics in Health and Medicine’ by Tredinnick-Rowe and Stanley also brings out. Semiology, in this original iteration, is implicitly taught to all medical students to this day as the way to identify disease and to categorize it (known as aetiology). Drawing from both the Saussurean glottocentric perspective, whilst extending it to the non-anthropocentric domains (where all sensory stimuli systems are reviewed), Tredinnick-Rowe and Stanley identify the production of signs without intention as signa naturalia, recalling St. Augustine’s terminology – e.g. symptoms of disease have no utterer per se. Discussing the lineage of medicine to semiotics, from antiquity through to the eighteenth and nineteenth centuries, the authors argue for a re-focusing on the historical ties between semiotics and medical practice. Referring to Stanley and Hyrup’s 2020 diagnostic vignette, the chapter examines in particular the narrative modelling of health as a means to understanding the experiences of patients. As the authors state, future emphasis on clinician-patient communication via semiotics would also usher in a working relationship between biosemiotics and semiotics in medicine that also would continue to bridge the divide in the nature and culture split – an artificial dichotomy already well understood both in biosemiotics and by French semioticians influenced by the key ideas of Greimas. Andersch’s chapter ‘Semiotics in Psychiatry and Psychology’ reminds the reader of the cline between nature and culture by positioning symbolism at the core of the psyche’s pathology and ways in which the clinician can use this understanding to engage effectively with patients. Additionally, the chapter provides a historical review of the roles of semiotics and symbolic research more broadly, advocating for a semiotic and symbolic approach to psychopathology as a fundamental therapeutic tool for the clinician. The post-war world saw a fracture between the study of symbolism and the typical empirical investigations into the brain, with psychiatry mainly concerning itself with the need to prove its worth as a medicine and not as a ‘magical’ enterprise, as was believed at the time. Already with Freud, there was a call for more human interaction and less ‘brain centric’ approaches, based on the importance of symbols to human consciousness. With Jung, symbols and individuation became inseparable, and consciousness was bonded to the symbolic realm via its repository of archetypal structures. Cassirer then argued that the human cultural world, endowed symbolically, constitutes the irrevocable break from the animal world (1979). From here, Andersch discusses the inner workings of the human psyche and ways in which serious mental illnesses, such as bipolar disorder, psychosis and schizophrenia, can be viewed as losses to the interactive (pattern based) symbolic safety-net. The chapter provides key insights into the practices of clinicians, as well as surveying the views of key theorists and their schools of thought from the Genetic Structuralists, to the Freudians, Empiricists, and Pragmatists, identifying the Russian/Baltic/Scandinavians as the forerunners of the psycho-semiotic world. Andersch also suggests that future research on mental pathologies will be better informed if theorists and practitioners revisit the interplay of ideas shared between Cassirer and Peirce, which elucidate the nexus between symbolic and mental aspects of consciousness. As the eighth chapter by Tylén and Kandamkulathy, ‘Semiotic in Neuroscience and Cognition’, suggests, symbols and their interpretation are as organic as they are inorganic and intrinsic to our meaning-making processes. The chapter explores semiotic contributions to the discipline of cognitive neuroscience via cognitive developments in ‘neurosemiotics’ – a term first used by Bouissac in 1985. Although neurosemiotics could

6

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

be construed as a branch of biosemiotics, the authors show that the functional neural structures for human meaning making are central and thus deserving of autonomous (although interrelated) study. What is of specific interest here is the intersection between semiotics, the cognitive sciences and the neurosciences. As discussed in other chapters, our concepts are not singularly and uniquely linked to the physical world. As such, neuro and cognitive semiotics approach conceptualizations as culturally situated phenomena (albeit based on the same neurobiology worldwide). However, these phenomena are in no way to be construed as abstract on their own, as in a Cartesian approach, but rather as products of the interaction between the body, the mind and the world. Neurosemiotics concretely examines the mediated processes that emerge between human semiotic experiences and the world they model. Contemporary research shows that the way we conceptualize is deeply integrative of these three dimensions – body, mind and culture. It is also not limited to the brain as an autonomous organ, but rather in terms of the parity principle which sees the world as part of our cognitive processes. This approach is highly supportive of von Uexküll and Peirce’s notion of ‘structures of meaning’. The chapter also offers a review of research on the distinction between structure and brain functioning, and, by way of historical and empirical examples, revisits the polemic between models of distribution and localization within the brain, as well as bottom-up and top-down processing. Interestingly, by reviewing predictive coding, the authors conclude that many of the previous divisive discussions are rendered moot by the naturalizing cognitive mechanisms that socialization requires. Chapter Eight, ‘Semiotics in Computing and Information Systems’, by Irvine explores semiotic contributions to computer science linked to the computational capacity of the human brain and utilizing the concept of ‘blackboxing’ in computer science, which Irvine claims must be revised semiotically, entailing the need to focus on the exigencies of human symbolic thought rather than a strict machinery analogy among natural and artificial systems. Like the opening chapter, this one too strongly emphasizes the linkage among logic, mathematics and symbolic processes and how it is utilized to express formalized operations. And, like Chapter 5, the analogy of speech as exaptation of language is also discussed. Logic is an adaptation of general cognitive functions, while mathematical operations are exaptations. The chapter reinforces the importance of understanding computational operations as initiating interpretations of logical processes, in terms of symbols – a translation process that is central to computers. The isomorphism between computer science and semiotics involves an analysis of system levels. From cybernetic theory to tokenization, the chapter offers multiple perspectives on the similarities between histories of computer design and theories of cognition, such as frame theory. Most importantly, the telic aims of computer science are strongly shaped by the individual views of designers of computer systems, highlighting the role of human agency in the development of computer systems. The chapter also connects the history of computer science with the development of information systems. The externalization process that is afforded by looking at external brains, so to speak, meets with stark limitations when applied to the disciplines under consideration in the next chapter, Oakley’s ‘Semiotics in Money and Finance’, which provides a robust history with semiotic interventions on the many systems at play in the study of money and finance. Money is as old as civilization itself; so it is fitting that the chapter starts with Mark Twain’s short story The £100,000,000 Bank-Note. From here, the fabulated turns to reality, with a focus on the role money plays in shaping our socio-cultural environments

INTRODUCTION

7

and their worldviews. Oakley takes the reader through the various facets of money as ‘metal and meddle’, shaping civilization to this day. Reviewing money as myth, money is considered as a set of interacting variables – as exchangeability signage, as part of logic, representation, interpretation, conceptualization, schematization and more, rather than as barter, language, tribute, quantification, gift and a motivation for violence (as it has been traditionally portrayed). Social structures are also explored through perspectives such as the heterarchy of money, the hegemony of money, the Marxist approach to money, and that of the Poststructuralists. Finally, through a cogent metaphor of money as language and writing, the idea of money systems as being like dialects, supports the subtext in many of the chapters that envision the importance of semiotics in understanding the foundational cognitive structures that repeat themselves through human institutions. The  binding property of money is irrelevant without the social and institutional glues that bind it to human activities (Maurer 2006). In the chapter ‘Semiotics in Law and Jurisprudence’, Chapdelaine-Feliciati, argues overall that semiotics and the study of the law are one and the same. She demonstrates from the early correspondences of Welby and Peirce, Welby’s own corpus and later Kevelson’s work, that law and semiotics had already found a common ground. Lady Welby’s significant work establishes an intrinsic relationship between order and meaning. In her correspondence with Peirce, we can see her insistence on the need to consider them in tandem, not separately. Furthermore, like any system of signs, the law is mutable and thus its meaning influences social order; so Welby cautions against the ability of professional litigators to play with the interpretative values of signs and warns against the linguistic war that hovers in legal systems. With de Hann, Welby’s significs approach is further entrenched into the discourse of law and the legal realm, raising interest among semioticians and scholars of law. However, Peirce, as Kevelson notes, had already woven into his semiotics the rule and role of the law. For instance, even a brief syllogism brings the concept of law as a rule system into effect – there is no law without a system of logic. Peirce also demonstrated, however, how reason can be skewed, highlighting the unequal relationship between ethics and the law. Most importantly, Peirce insisted that a sign as a proposition need not be asserted or denied, since it retains meaning regardless of what one ‘does’ with it. For Peirce scholars, the relationship of signage to the law is encapsulated in his notion of Legisign, which is in essence a law that is a sign and which signifies through its application (Replica) and through other sign modalities, such as Argument and Symbol. Kevelson further developed this notion as a provisional judgement. ChapdelaineFeliciati also provides an interesting historical review of the Supreme Court’s relationship to Peirce who, indirectly through the Metaphysical Club, with his logic of search and discovery, may have indirectly influenced the whole of American Common Law. Chapdelaine-Feliciati, moreover, looks at the ambiguities used to describe the main approaches in which legal semiotics investigates law systems. One of these is the ‘semioticsin-law’, which considers the law as a system of signs; and another is the ‘semiotics-of-law’, which looks at legal information exchanges via the study of sign systems. The semioticsin-law approach is characterized by three aspects: (1) the legal realm viewed as an infinite system of signs, (2) semiotic methods in examining the law, (3) semiotics as providing a common language for other sciences and approaches. Legal semiotics has yet to be integrated into any formal Law School pedagogy. However, as argued in this chapter, the role of language and argumentation strongly supports that Semiotics is deeply and fundamentally ensconced in law. Contemporary legal semiotics, as exemplified by Wagner, Petrilli and Danesi explores signs and meaning

8

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

within the context of law systems in practical and theoretical terms, exploring the ways in which legal systems and meaning structures unfold at the social level. The final three chapters in the collection shift the discussion into semiotic contributions to the sciences of technical design, starting with Aroni’s chapter on ‘Semiotics in Architecture and Spatial Design’, which provides an all-encompassing overview on the communicative aspects of landscape structures and the semiotics of architectural design. Using the ideas of Eco as a referential frame, Aroni suggests that architecture needs to distinguish between what it communicates semiotically and what its function is. Architecture has been traditionally plagued by contrasting dialectic approaches – including the mechanical versus the aesthetic and the representational versus the objective. Whatever the theoretical justification, there is little doubt that we glean meaning from the built environment via the action of semiosis. This inner tendency leads us to shape the space we inhabit. From antiquity onwards, the many theoretically varied questions regarding architecture have produced a fragmented discipline of architectural hermeneutics. Within the semiotic paradigm, however, a certain unity of thought emerges, given that it envisions buildings as semiotic objects that accrue socially significant connotative meanings. An analogy often made is that of architecture as a language. This has suggested to some that it could be studied in terms of a ‘grammar’ – a conceptualization that dates back to the Roman Vitruvius, who had adopted it already in the first century BCE. In such a framework, Saussurean notions can be easily applied, such as envisioning an architectural project as a signified and the design principle utilized to realize it as the signifier. Interestingly, Vitruvius’s view of the meaning and functions of architecture was infused by various perspectives, which today would be called interdisciplinary. Vitruvius’s views extended well into Renaissance perspectives of building design. In actuality, this view has persisted well into modernity, wherein architecture continues to be characterized as a language that communicates specific meanings via its own grammar – a grammar that shares basic semiotic properties with actual languages. Focus on these properties has guided work in various contemporary semiotic approaches to buildings, whereby even the parts of buildings have been analogized with parts of language – for example, mouldings with vowels and elements with consonants. When combined in the design, the result are architectural words guided by the syntax of architectural language. Aroni takes a different path, emphasizing the importance of symbolic theories, rather than analogies to language, since symbols and cultural evolution are intertwined, showing how the synchronic aspects of contemporary design are amalgams of symbolic practices of the past. Like several others before him, Aroni affirms the role of semiotics in decoding the meanings of architectural and spatial design, independently of, but not neglectful of, Saussurean approaches. He also looks at how semiotics itself has evolved in an increasingly technology-based world, and how it can counteract the mechanicalpractical interpretations of buildings that is currently spreading throughout the modern world. Aroni’s insistence that meaning (not practicality) is at the core of building design is a well-taken one, in line with previous work within semiology and phenomenology which has sought to understand the semiotic source of the functional elements of architecture. Also in contrast to the discourses of postmodernism in architecture – it is relevant to note that the term postmodernism is coined in the field of contemporary architecture in the 1970s – Aroni sees a core of meaning structures in buildings, rather than their denial. As such, architectural semiotics is better informed by Peircean triadic theories, which invariably involve the interpretant as a major factor in shaping design.

INTRODUCTION

9

Skaggs’s chapter, ‘Semiotics in Graphic Design’, shows how the forms of design are grounded in meaning structures, fitting into the broader theme of this volume on adaptive and exaptive processes that are guided by semiosis and its effects on cognitive evolution. Much like architecture, graphic design involves composite visual sign structures that interact with cognitive structures in the service of what Skaggs calls ‘usefulness’ in design – that is, the utilization of emotive visual signs that are projected onto a specific domain, which, in turn, become persuasive subconscious elements of interpretation. This telic usefulness factor separates graphic design from fine art proper. Graphic designs should move things along, so to say, being functional features in the everyday world, conveying useful (and persuasive) information through our eyesight. Starting with the origins of graphic design as a distinctive field in 1922, Skaggs shows how it quickly became part of the marketing world, given its psychological power to persuade people to see things in a certain way. By the 1960s, the field of graphic design became almost formulaic, abandoning the artistry of previous decades, and focusing on functionality. Cursive, aesthetically pleasing styles were replaced by monolithic forms adopted as standard. It was in the late twentieth century, that a reaction to this depersonalized approach surfaced, ushering in a highly ‘psychedelic’ approach that has become standard. This new trend was influenced by postmodern ideas in semiotics generally, which emphasized the subjectivity of the viewer and creator, not the standardized forms of design. Skaggs then provides an interesting side-story of semiotics via the history of graphic design by underlining the use of virtually identical theoretical and methodological approaches in the two fields. Central to both is the opposition of the denotative versus the connotative, or to employ Barthes’s terminology, of first-order and second-order levels of meaning. Also intrinsic to the history of both fields is the work of pragmatist philosophers and semioticians, influenced by Peirce, Morris and Dewey, which Skaggs utilizes to suggest that visuality, like Arnheim before him, is central to cognition, even superseding verbality in many domains of signification. Skaggs finally turn his attention to the emergence of multimodality in graphic design, which, however, is not divorced from the basic cognitive-semiotic approaches of the past – indeed, it constitutes an overlapping approach. The final chapter in this volume, ‘Semiotics in Marketing and Branding’, by Bankov and Trendafilov, serves as a bridge to Volume 3 (Semiotics in the Arts and Social Sciences) by providing an overview of one of the most common disciplinary applications of semiotics today – applications to the world of marketing and branding. As a field at first defined semiotically in the 1960s by Barthes and Levy, marketing has since become an ipso facto branch of semiotics, ensconced into it in the subsequent decades by Sebeok and others, starting with the paradigmatic conference of 1986 on marketing semiotics organized by Sebeok. Within this branch, various approaches have emerged, such as the narratological one based on the ideas of Greimas and the visual-based approach promoted by Floch initially. Interestingly, it was via its applications to marketing that semiotics catapulted to the forefront of public awareness at the end of the previous century. Of special importance in this field is branding, which is essentially a way of utilizing the semiotic resources of a culture (linguistic, visual, aesthetic, etc.) to create powerful advertising messages. Today, branding semiotics itself has become a distinctive area of analysis, within various cognate fields, such as anthropology, culture studies and media studies. Bankov and Trendafilov invoke Semprini’s work to elucidate the composition of the brand as a sign, providing models of marketing from around the world. From here, behavioural approaches and the use of predictability measures are introduced and

10

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

compared to the more semiotic ones. A clear and detailed account of code theories within branding and marketing domains is also provided, which reveals the broad cultural impact of branding on the contemporary state of consumer culture specifically. These fourteen chapters serve as a catalogue of evidence demonstrating the contributions, interventions and inroads that semiotic perspectives are making into disciplines associated with the natural and technical sciences. More than a survey of the state-of-the-art, however, these chapters also expose substantial areas for further research and ongoing involvement in each of the domains surveyed for those who are interested to join in and continue developing more meaning-centred approaches to these and related disciplines. The open questions and priority research areas for semiotic development in these fields are in many cases imposing, if not overwhelming, in their scale, scope and implications; and it is for this reason in particular that we hope the chapters in this volume will inspire more theorists and researchers to get involved in the development and application of semiotics in the disciplines affiliated with the natural and technical sciences.

REFERENCES Anderson, M., J. N. Deely, M. Krampen, J. Ransdell, T. A. Sebeok, and T. von Uexküll (1984), ‘A Semiotic Perspective on the Sciences: Steps towards a New Paradigm’, Semiotica, 52 (1–2): 7–47. Bonfantini, M. (1987), La semiosi e l’abduzione, Milan: Bompiani. Bouissac, P. (1985), ‘Neurosemiotics: A Definition’, Recherches sémiotiques / Semiotic Inquiry, 5 (3): 323–5. Cassirer, E. (1979), Symbol, Myth, and Culture: Essays and Lectures, New Haven, CO: Yale University Press. Cobley, P. (2018), ‘Observership, “Knowing” and Semiosis’, Cybernetics and Human Knowing, 25 (1): 23–47. Cunningham, D. J. (1998), ‘Cognition as Semiosis: The Role of Inference’, Theory & Psychology, 8 (6): 827–40. Deely, J. (1990), Basics of Semiotics, Bloomington: Indiana University Press. Deely, J. (2011), ‘Thomas A. Sebeok and Semiotics of the 21st Century’, in P. Cobley, J. Deely, K. Kull, and S. Petrilli (eds), ‘Semiotics Continues to Astonish’: Thomas A. Sebeok and the Doctrine of Signs, 123–60, Berlin: de Gruyter Mouton. Hoffmeyer, J. (2008), Biosemiotics: An Examination into the Signs of Life and the Life of Signs, trans. J. Hoffmeyer and D. Favareau; ed. D. Favareau, Scranton: University of Scranton Press. Kant, I. ([1786] 2004), Metaphysical Foundations of Natural Science, Cambridge Texts in the History of Philosophy, ed. and trans. with introduction by M. Friedman, Cambridge: Cambridge University Press. Kull, K. (1998), ‘On Semiosis, Umwelt, and Semiosphere’, Semiotica, 120 (3–4): 299–310. Kull, K. (2009), ‘Biosemiotics: To Know, What Life Knows’, Cybernetics and Human Knowing, 16: 81–8. Kull, K. (2011), ‘The Architect of Biosemiotics: Thomas A. Sebeok and Biology’, in P. Cobley, J. Deely, K. Kull, and S. Petrilli (eds), ‘Semiotics Continues to Astonish’: Thomas A. Sebeok and the Doctrine of Signs, 223–50, Berlin: de Gruyter Mouton.

INTRODUCTION

11

Midtgarden, T. (2020), ‘Peirce’s Classification of the Sciences’, Knowledge Organization, 47 (3): 267–78. Nöth, W. (2001), ‘Ecosemiotics and the Semiotics of Nature’, Sign Systems Studies, 29 (1): 71–81. Peirce, C. S. ([1893–1913] 1998), The Essential Peirce: Selected Philosophical Writings, vol. 2, Peirce Edition Project (eds), Bloomington: Indiana University Press. Cited as EP 2. Sebeok, T. A. (1963), ‘Review’, Language, 39: 448–66. Sebeok, T. A. and M. Danesi (2000), The Forms of Meaning: Modeling Systems Theory and Semiotic Analysis, Berlin: Mouton de Gruyter. Maurer, B. (2006), ‘The Anthropology of Money’, The Annual Review of Anthropology, 35: 15–36. Merrell, F. (1996), Signs Grow: Semiosis and Life Processes, Toronto: University of Toronto Press. Nöth, W. (1990), Handbook of Semiotics, Bloomington: Indiana University Press. Nöth, W. (2001), ‘Ecosemiotics and the Semiotics of Nature’, Sign Systems Studies, 29 (1): 71–81. Pablé, A. (2016), ‘Global Semiotics vs. Human Semiology’, Chinese Semiotic Studies, 10 (4). Sebeok, T. and M. Danesi (2001), Forms of Meaning: Modeling Systems Theory and Semiotic Analysis, Berlin, Boston: De Gruyter Mouton. Threadgold, T., ed. (1986), Language, Semiotics, Ideology, Sydney: Pathfinder Press. Uexküll, J. V. (1957), ‘A Stroll through the Worlds of Animals and Men’, in C. Schiller (ed.), Instinctive Behavior: The Development of a Modern Concept, 5–80, New York: International Universities Press. Uexküll J. V. (2001), ‘An Introduction to Umwelt’, Semiotica, 134 (1/4): 107–10.

12

CHAPTER ONE

Semiotics in Mathematics and Logic AHTI-VEIKKO PIETARINEN AND FREDERIK STJERNFELT

INTRODUCTION Mathematics and logic differ from most other sciences in the sense that they have always been innovative in exploring the forefronts of meaning and signification, that is, are semiotic disciplines, however tacitly or implicitly. The invention and use of special signs and notations and their precision-targeted signification practices are central, if not simply definitory, to those of ‘exact’ sciences. Among the exact sciences, we count computational, data and AI sciences as well as any field whose progress is predicated on the success of the exact sciences. Mathematics and logic could be their methodological drivers, or means of explanation, or even the foundations. Physics, chemistry and biology may well be among such, as may anything involving the characteristics of complex open systems, including human collective, economic and political action and behaviour. Where most other scientific disciplines and scholarly fields use specialized versions of ordinary language enriched by the development of novel, regional terminologies, mathematics and logic add to this the deliberate construction of new, non-linguistic signs and their manipulation in order to address their subject matter. Such semiotic formalisms, in turn, may migrate into many other disciplines making use of mathematics and logic. Danesi and Bockarova (2014: 7) have gone as far as to argue that even since ‘the advent of mathematics as a distinct discipline, mathematicians have been doing semiotics without knowing it’.1 The present chapter attunes to the history, theory and philosophy of notation in mathematics, and to its involvement with logical notations and their signification practices. The chapter recounts the point of semiotics (theory of signs) to serve as a theory and methodology for exact mathematical and logical practices. Such practices involve, primarily, decisions on which notational means one chooses to conduct mathematical explorations in and how to assess their advantageousness to discovery and proof. Well-conceived changes in notations facilitate conceptual change in science; thus the commitments implicit in privileging one sign system over the other should be systematically articulated. This chapter tells a much-condensed story of how the semiotic and philosophical field of notation studies the constraints and possibilities that different notational systems can manifest. In particular, diagrams are such signs par excellence, congenital to mathematical and logical reasoning. Beyond psychological effects, they serve important logical and epistemological functions. A structural constraint may receive

14

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

a description that needs no appeal to cognitive faculties or abilities. Logical notations employing iconic signs like diagrams are a prime example the characteristic feature of which consists in their departure from the language-like linear strings of conventional signs that are often, though rather illicitly, equated with the exactitude of logical and mathematical notations. All such choices are semiotically motivated, thus rendering mathematical and logical discoveries likewise amenable to evolving semiotic analyses and conceptualizations. Our chapter by no means aims at a complete, or even a semi-complete overview of those masses of relevant research one encounters in the intersections of semiotics, logic and mathematics. Our focus is on the origins of these encounters when they explicitly connect with sign systems, especially as conceived in the work of Charles Sanders Peirce (1839–1914). There are many promising and recent encounters that the future research may bring to the fore, such as a potential integration of Peirce’s relational and evolutionary theory of the structure of the world conceived as being built upon indecomposable triadic structures – or perhaps asymmetric graphs as interpreted by Dipert (1997) – and the computational hypergraph constitution of the universe (Wolfram 2020). But where an explicitly sign-theoretic nomenclature is wanting and we do not as yet precisely know what significance we should assign to mathematical findings such as how collections of abstract relations between abstract elements may build up the most complex of forms, we relegate the semiotic significance of such mathematical abstractions for future research to discover. Likewise, there is no shortage of work – both recent and historical – in logic and reasoning that has been or is prone to be semiotically pertinent and applicable. Peirce’s mature view was that the study of the systems of logic is tantamount to the study of the systems of signs, and that the theory and classification of signs plays an important and indispensable role in both how we conceive inferential relations and how we act according to the leading principles of reasoning. Thus anyone accustomed to think of logic in relevant semiotic terms is at once thinking of the notions such as meaning, significance, models, diagrams, semantics and pragmatics as a natural progression to how this wide semiotic perspective to theories and systems of logic has evolved in the past and how it may be evolving into new directions in the future. Mathematics is a way of thinking that has certain special characteristics. Peirce explains the ‘“mathematical” ways of thinking’ as ‘making a diagram on paper or in imagination, whether it contains lines or is merely an array of points, and experimenting on the possible variations of it, and generalizing the results’ (Peirce to Cousin Jo, 26 June 1909: RL 366). Given the importance of diagrams for the discussion, we provide some initial context for their appreciation. Peirce’s definitions of diagrams are invariably found within the context of mathematical reasoning. In a piece entitled ‘Logic’ (c. 1893–4: R S-64) which may be related to his planned chapter on ‘Graphs and Graphical Diagrams’ for a book on logic that was never finished, he writes: Definition 1. A mathematical hypothesis is an imaginary state of things proposed for study. Definition 2. Any imaginary state of things is mathematically possible for which the hypothesis leaves room. Definition 3. A diagram is a drawing whose parts are related like the elements of a hypothesis sufficiently to assist the study of it.

SEMIOTICS IN MATHEMATICS AND LOGIC

15

Examples of such diagrams follow these definitions. Most of the relevant writings that pertain to the topic of reasoning and diagrams derive from Peirce’s later years of 1904–8, which he produced in connection to his mature restatements of the philosophy of pragmaticism, while continuing to emphasize the imaginary character of diagrams that serve as the corollary theory of reasoning to his mature pragmaticistic philosophy. At all events, mathematics is the science of studying imaginary and hypothetical states of affairs and their deductive consequences, while logic – conceived as semiotics – is a normative science that analyses the processes of reasoning in exact sciences and elsewhere.2 Peirce proposed that there are two uses of deduction – its practice and its analysis – and that all deduction is mathematical in the sense that it is diagrammatic in nature (1902: NEM 4.47–8, 1903: CP 1.54). It was clear early on that the method of diagrammatization applies even to the most rudimental patterns of inference: For instance, take the syllogistic formula, All M is P All S is M ⸫ All S is P This is really a diagram of the relations of S, M, and P. The fact that the middle term occurs in the two premises is actually exhibited, and this must be done or the notation will be of no value. (1885: W 5.164) Whoever reasons deductively, and even does so without access to any external signs, must rely in reasoning upon some construction and observation of such construction in the imagination. Later in 1904, Peirce writes: All reasoning in mathematics is performed by the aid of diagrams, using this word in a broad sense. In pure mathematics, we do not trouble ourselves much about the meanings of the diagrams. We can upon occasion, assign to them any appropriate meanings we like. The principal thing is to learn how to work with the diagrams, and to go through the motions of reasoning. (1904: R 693) In deduction, a reasoner ‘examines the state of things asserted in the premises, forms a diagram of that state of things, perceives in the parts of the diagram relations not explicitly mentioned in the premises’, and finally ‘satisfies itself by mental experiments upon the diagram that these relations would always subsist, or at least would do so in a certain proportion of cases, and concludes their necessary, or probable, truth’ (c. 1896: CP 1.66). Examination of diagrams refers to imaginary activities and mental constructs as the term ‘diagram’ is intended to be ‘used in the peculiar sense of a concrete but possibly changing mental image of such a thing as it represents’ (1910: R 616, LoF 1). The rest of the chapter is organized as follows. The next section outlines a brief history of mathematical and logical notations. This is followed by a section on Peirce and his pioneering work on the theory of signs as the resource for a synthetic semiotic theory of mathematical and logical notations, including the classification of reasoning and even that of mathematical proofs that split into their abductive, deductive and inductive stages. The success of his synthetic semiotic theory supervenes on his central notion of creative mathematical and exact reasoning as a fundamentally diagrammatic facility. The

16

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

penultimate section, entitled ‘Theorematicity: A Case for Semiotics in Proof Discovery,’ continues on the contribution that the semiotic perspective makes to the theory of innovative elements involved in proofs.

A BRIEF HISTORY OF MATHEMATICAL AND LOGICAL NOTATIONS Danesi and Bockarova’s argument implies that ever since antiquity, mathematics in particular has been a semiotic science, purposefully furthering its development and conquering new territory by means of notational discussion and innovation. This is found as early as in special notation systems for numbers such as Chinese, Egyptian, Greek, Roman, Arabic (that is, Hindu) and other numerals, which stand out among other word classes in written languages by using different para- or non-alphabetic scripts. Already before Christ, the invention of the Zero sign and a place value notation of numbers in India was a decisive step in the development of arithmetic. Early Greek use of geometrical diagrams, that is, drawings accompanied by special definitions, reading conventions and restrictions such as the compass-and-ruler manipulation of diagrams was the formative semiotic instrument of Pythagorean and Euclidean geometry, and through the history of mathematics, new breakthroughs have, again and again, been facilitated by the semiotic invention of new notational tools. To name but a few: the invention of the tree notation for concepts and subconcepts by Porphyry in the third century, and the formalization of syllogisms in Boethius in the sixth century, taken further by Abélard and Buridan in the twelfth and fourteenth centuries. In mathematics, a whole current of semiotic innovation marked the late medieval and early modern period: the introduction of the plus sign by Nicole Oresme in 1360, the minus by Johannes Widmann in 1489, while in the late fifteenth century, Nicolas Churquet invented the superscript notation for powers or exponents and introduced negative numbers as powers. The radical (square and other roots) sign was made by Christoff Rudolf in the early sixteenth century, and most importantly, the invention of parentheses to indicate articulated content and stepwise procedure, crucial to algebra, by Niccoló Tartaglia in 1556. The sign of equality by Robert Recorde followed the year after, the introduction of decimals by Simon Stevin in 1585 and the decimal point by John Napier around 1600. The multiplication sign x was coined in 1631 by William Oughtred who also invented the slide ruler by putting two logarithmic scales besides each other. Major seventeenth-century breakthroughs were Descartes’ introduction, in 1631, of the signs x, y, z to refer to unknown variables, with a, b, c referring to known variables; he went on, in 1637, to invent the Cartesian plane with two orthogonal arithmetical axes facilitating the development of analytical geometry. Similarly important were Isaac Newton’s and Gottfried Leibniz’ independent inventions of calculus in the last decades of the century, involving the famous priority controversy between the two. Leibniz invented the ‘long S’ integral sign, while his notation invention of ‘dx’ to signify infinitesimally small quantities was opposed by Newton, ridiculed by Berkeley (‘ghosts of departed quantities’) and later reinterpreted by the doctrine of limits and neighbourhoods, but still it won out in the long run. Condorcet added the partial derivative sign in 1770, and from the eighteenth century, the semiotic innovation of mathematical notation virtually exploded. Leibniz was also one of the first explicitly to reflect about notational choices (Couturat 1901; Cassirer 1923; Geymonat 1947, 150–1; Serfati 2001). He saw Aristotle as a

SEMIOTICS IN MATHEMATICS AND LOGIC

17

pioneer who, by carefully tracking which syllogisms expressed in natural language are truth-preserving and which are not, ‘wrote mathematically outside mathematics’ ([1674] 1903: GP 7.519). Leibniz held that, when we think by means of symbolic signs, we neither see nor perceive the idea conveyed; rather, we proceed straight to the objects intended (Dascal 1978; Burkhardt 1980; Couturat 1901; Gensini 1991). He dreamed of a language where all logical consequences would be perfectly transparent. In the twentieth century, Ludwig Wittgenstein echoed this aspiration with his assertion that ‘what can be said at all can be said clearly’ (from the preface to his 1921). One may align the thought with the Einsteinian quip ‘everything should be made as simple as possible, but not simpler’.3 In between Leibniz and Wittgenstein, Peirce recognized that the primary purpose of one’s notational resources is not merely to find efficient roundabouts to deductive inference or to establish it as a powerful calculus, but to invent new methods for logical analysis: that is, to break down the inferential steps in their simplest occasions so as to see what really constitutes the purported argument from the premises to its possibly multiple conclusions (see the next section). An equally important service is performed by the preservation and cultivation of freedom in our scientific modelling and representational tasks (Hadamard 1949; Feynman, Leighton and Sands 1964). As an example whose repercussions we are only beginning to better appreciate is provided by von Neumann (1958), who recognized that the internal languages of the brain are structurally quite different from what we are acquainted with in our accustomed ordinary use of conventional symbols. Brown and Porter (2006: 7) echoed this recognition in their more recent conviction that it is ‘unreasonable to suppose that a purely linear mathematics can express reasonably the complex interactions that occur in the brain’. All such probings, old and new, tend to suggest that notations and languages closer to our actual information processing are well worth developing. Indeed, alternatives to linear notations of mathematics and its cognate fields such as theoretical physics are of no shortage these days. In modern science, such examples abound from higher-order algebras to category theory; from synthetic differential geometry to concepts involving objects such as pre-sheaves, topoi, homologies, cobordisms, ∞-groupoids, all of which are emerging as respectable and profitable methods to supplant earlier conceptions, involving new nonlinear and diagrammatic notations to supplant ordinary languagelike expressions and regimentations of thoughts that had been straightjacketed by such predated representational forms and notations. In logic, the tradition in ‘natural logic’ may be another though a different one, as it seeks to ground untranslated inferential tasks directly on the patterns and units of one particular medium, that of natural language, its lexical units and their vague and fluctuating sense boundaries (Cruse 2011; Endrullis and Moss 2015). Such examples are only a modest sample of the kind of work semiotics can do when involved in the study of the basal characters of intellectual cognition. According to Leibniz, an important corollary follows from the fact that all thinking is indeed in signs: ‘Since in order to think the human mind needs signs, then the better we will reason by means of them the more those signs express the relationships among things’ (Leibniz to Huygens, 1691). A sign which represents the relations among the parts of the object by corresponding relations among the parts of the sign is called by Leibniz a ‘character’. When ideas are too complex to be grasped at once by the understanding, characters furnish the material and observable support needed to assist reasoning. To Leibniz, the chief aim of any characteristics is to find a notation from which all the consequences

18

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

could be routinely and monotonically derived. The debate among J.H. Lambert (1764), Gottfried Ploucquet (1766) and Georg J. von Holland about the virtues of algebraic and graphical symbolism exemplified the debate about the possibility of achieving this aim (Hailperin 2004; Capozzi and Roncaglia 2009; Bellucci, Moktefi and Pietarinen 2013). Immanuel Kant, one might notice, opposed this Leibnizian view. He sought to differentiate the synthetic method of mathematics from the analytic procedures of philosophy and general logic (Hintikka 1973). General logic is, for Kant, a canon of reason, not its organon (Barone 1957; Capozzi 2002). Logic prescribes laws of correct use but contains no principle for the expansion of knowledge (Kant 1998, B viii–ix). Hence, Kant held that logic could not be concerned with individual signs instantiating general concepts, as geometry and algebra do. Logic is therefore no ars inveniendi, or universal algebra. To Kant, mathematics, by contrast, is in need of construction involving intuition, in the shape of pure ‘schemata’, artfully uniting space-time presentation in the intuition with general universal concepts of the understanding. It is commonly believed that the birth of new formal logic(s) at the end of the nineteenth century rendered redundant Kant’s belief in intuition as that what unifies the manifold of our experience, because logic was now grounded solely upon a formal language and an apparatus of axioms and rules (see Kneale and Kneale 1962: VII–VIII; Coffa 1991). Like Frege (1884), Russell and Couturat went on to maintain that the logicist program had shown that intuition has no place in mathematics at all, and that Kant’s philosophy of mathematics was thus conclusively refuted. ‘Leibniz’s dream’ (as Peano called it) seemed vindicated.

PEIRCE’S SYNTHETIC SEMIOTICS OF MATHEMATICS AND LOGIC However, the standard historical sketch narrated above neglects quite a bit. Indeed, an alternate line of development is clearly discernible through the work of Friedrich A. Lange and Peirce who was much inspired by Lange. To Lange (1877), all apodictic reasoning is the same in kind, for spatial intuition is the source of both mathematical and logical necessity (see Thiel 1993; Bellucci 2013; Stjernfelt 2014). Likewise, Peirce held that the notion of intuition is not redundant, since it is what allows necessary reasoning to yield informative truths. Thus, one is not drawn into logicism that would render mathematics an ‘applied logic’ (Pietarinen 2009, 2010). In fact, for Peirce all necessary reasoning rests upon the observation and manipulation of diagrams (see e.g. Bellucci and Pietarinen 2016a, 2016b; Hintikka 1980; Hookway 1985; Pietarinen 2006; Stjernfelt 2007), explicitly taking Kantian schematism with its double involvement of intuition and understanding, of imagination and generality, as his point of departure. This view sees intuition as embodied in a public commodity amenable to careful scrutiny. It is the diagram, in the Lange-Peirce paradigm, which functions as the core instrument of synthetic thought. On this view, linear, language-like formalisms are not opposed to diagrams but rather form a particular diagram subtype.

Philosophy of notation Recent work in the philosophy of notation has been much invigorated by this ‘diagrammatic turn’ (Gardner 1958; Baron 1969; Coumet 1977; Allwein and Barwise 1996, Shin 1995, Bernhard 2001; Greaves 2002; Moktefi and Shin 2012; the ongoing ‘Diagrams’

SEMIOTICS IN MATHEMATICS AND LOGIC

19

Conference Series, and so on). This semiotic subfield grew out of this development as a descriptive, critical and creative enterprise. It is descriptive, because it seeks to describe the various ways that notational systems work. It is critical, because it recognizes that some systems work better than others do. Finally, it is creative, because it invites us to seek out systematic means of expression that we yet cannot quite envision, and for indefinite use and application possibilities. As mentioned, advances in mathematics have often been facilitated by advances in notation (Giaquinto 2007) – and perhaps nowhere in more salient fashion than in the notation of diagrams applied to exact sciences. Feynman diagrams are central examples of diagrammatic notations that were destined to arise  – in the sense that if Feynman would not have proposed them, then someone else would soon have – so fundamental a tool have those diagrams been for theoretical physics ever since the late 1940s. And since advances in science have often been prompted by advances in mathematics, new and emerging notational systems are non-replaceable resources for discovery and proof feeding into numerous fellow disciplines. To rewind a little, the breakthrough of modern formal logic in the second half of the nineteenth century was executed by the introduction of new semiotic notation systems. The pioneering proposals by Augustus de Morgan and George Boole in mid-century came to full force with Gottlob Frege’s 1879 Begriffsschrift with a wholly new, graphic notation system allowing for a complete and consistent representation of propositional and predicate logic, and with Peirce’s two published Algebra of Logic papers of 1880–5  – and many more in his manuscripts – which independently achieved similar new logics by algebraic means carried out in linear formal languages. The latter became – through Schröder, Peano and Russell – the roots of the systems still used by modern formal logic. Simultaneously, Peirce’s 1885 paper explicitly introduced the idea of a ‘philosophy of notation’ initiating modern semiotic meta-reflection on sign use in mathematics and logic. In this strong sense of continuous notational contributions to scientific progress, mathematicians and logicians have always been practicing semioticians – rather than psychologists – and though they may not always have consciously reflected upon any explicit doctrine of semiotica docens, then at least, in their fertile developing of signs and sign systems, they have implicitly been practising their field as a semiotica utens, a semiotics in use. Peirce called the use of logic in mathematics and the study of logic, respectively, logica utens and logica docens, and his philosophy of logic and mathematics simply saw logic and semiotics as one and the same thing (Pietarinen 2005). His reflections of the issue would later develop in concert, from 1896, with his second class of systems formalizing logic and reasoning, the graphic systems known as ‘Existential Graphs’ (Peirce [1895–1913] 2019–22). His mature reflections on the semiotics of logic and mathematics after 1900 still inspire much work in the field; the reason why we shall devote the main part of this article to those reflections and their implications.4

Peirce on mathematics and logic ‘Science that draws’ versus ‘science of drawing’ necessary conclusions  To Peirce, the crucial difference between mathematics and logic was that the former, in the words of his father, the mathematician Benjamin Peirce, is ‘the science that draws necessary conclusions’ (1903: R 459, LoF 2/2). Logic, by contrast, is the science studying such drawing of conclusions. Thus, mathematicians use their spontaneous ‘logica utens’ and need not study logic, while logicians take it as their object to understand how mathematicians and other reasoners in science and everyday life ought to think.

20

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

To begin with the former, Peirce saw that mathematics always works with diagrams in the special, extended sense of the word which he developed. The core example of diagrams, to Peirce, was Euclidean geometry diagrams, facilitating the proving of theorems by ingenious manipulation with those diagrams, but he developed and generalized the concept far beyond that prototypical example. Thus, another central example of diagrams, to Peirce, are algebras. This comes to the fore in his early Philosophy of Notations, in the second Algebra of Logic article of 1885 indeed aptly subtitled ‘A Contribution to the Philosophy of Notation’, which is one of the first publicized expressions of his notion of diagram experiment: The truth, however, appears to be that all deductive reasoning, even simple syllogism, involves an element of observation; namely, deduction consists in constructing an icon or diagram the relations of whose parts shall present a complete analogy with those of the parts of the object of reasoning, of experimenting upon this image in the imagination, and of observing the result so as to discover unnoticed and hidden relations among the parts. (1885: CP 3.363) Geometrical diagrams and algebras are but examples of the general category of diagrams developed by Peirce particularly in his final fertile period after 1900. The idea is that the central sign type used in all branches of mathematics is that of diagrams. They are skeletal signs, stripping some idea or hypothesis, as far as possible, of irrelevant features in order to focus upon those relations scrutinized. Then, they are investigated by interpreting, manipulating and experimenting upon the diagram in order to chart which ‘unnoticed and hidden’ necessary implications can be drawn from it. The mathematical intuition relies on the diagrammatic construction of the model or the state of things upon which ‘the reasoner then observes this diagram and notices some relation between parts of it which had not been expressly introduced in the construction of the diagram’, Peirce explains in a letter draft to Francis C. Russell (1908: RL 386, CSP to FCR, September 18). All deductions, then, are performed by means of diagrams, explicitly or not, and all of mathematics are characterized by so deducing implications from diagrams. Deductions, with their nearly complete certainty, are possible only in the ideal realm of mathematics because in idealizations we may have full control over the few, cleansed relations investigated – in contradistinction to empirical sciences where empirical data are always, to some degree, messy and uncontrollable and hence in need of Peirce’s other inference types of abductions (qualified guesses) or inductions (empirical sample investigation tied to credences and likelihoods of the tentative status of one’s knowledge). Epistemology of diagrams  Peirce’s epistemologically central notion of diagrams as the semiotic gateway of mathematics need now be further qualified (Stjernfelt 2019). Diagrams are a sort of icons, characterized by a relation of similarity to their object – in this case, the relevant relations under scrutiny. But any diagram token, be it on the pages on a math book, on the blackboard, the computer screen or elsewhere, is a physical sign vehicle which necessarily has a number of excess properties, even in the skeletonized shape of a diagram. Thus, the representation of a geometrical object such as a triangle has to have a certain size and colour, and its lines must have some thickness, all of which are absent from the proper mathematical general triangle. So, an important feature in

SEMIOTICS IN MATHEMATICS AND LOGIC

21

the mathematical use of diagrams is the process of ‘prescission’ deliberately bracketing such superfluous features of the diagram token. By this important semiotic step, the pure, general diagram type may be observed, through the lenses of prescission, as it were. By this procedure, the mathematician becomes able directly to observe general features of the object. This is important to the semiotics of mathematics: generality is not a later, secondary result of higher, linguistically informed mathematical cognition, but is at stake already in the initial interaction with the diagram. The diagram, however, even if primarily iconical, also has important symbolic and indexical aspects. The diagram, as a general sign, is also a symbol, and typically it is informed by other symbols, that is, other diagrams, accompanying text, etc. They make clear what it is a diagram of, which kind of object it depicts, and thereby informs us about which manipulations of the diagram are allowed. If the triangle on the page is a geometrical triangle in the plane, a certain number of transformations are allowed: the addition, movement and manipulation of lines – if it is a triangle on a spherical surface, other rules hold. This is important, for the very purpose of a diagram is manipulating and experimenting with it to produce another diagram which is true in case the former diagram is. This shows that the diagram, moreover, is the predicate part of a proposition. If the triangle token is accompanied by the word token ‘triangle’, the two taken together constitute the proposition ‘This [diagram] is a triangle’, and from that  proposition others may be proved by ingenious manipulation, such as ‘A triangle’s sum of angles equals the sum of two right angles’. Oftentimes, such a diagram, furthermore, is equipped with indices in order to be able clearly to refer to specific parts or aspects of the diagram, such as the letters a, b, c to denote the three angles or the letters A, B, C to denote the three sides of the triangle, constituting further propositions such as ‘The angle sum is defined by a + b + c’. Thus, mathematics proceeds by beginning with an ideal hypothesis, such as ‘Let us assume this is a triangle’. It is hypothetical, because it claims nothing at all about the empirical world, and it is ideal because an ideal hypothesis is cleansed from all irrelevant properties first by design and then by prescission. This is why Peirce supplied his father’s definition of mathematics by the notion that mathematics is through-and-through hypothetical – all of its truths are ‘if-then’ conditionals, not categorical propositions.5 The identification of mathematics with diagram deduction has the important corollary that whenever deductions are undertaken elsewhere, in the empirical sciences or in everyday reasoning, an underlying mathematical diagram is effective, explicitly or implicitly. This takes place, of course, in the many disciplines or practices explicitly borrowing from the rigor and conceptuality of mathematical tools, patterns and structures – in physics, systems biology, social statistics, construction, etc. – but it may also take place implicitly as soon as a deduction is performed. So, a priori or apodictic reasoning in empirical fields necessarily possesses a diagrammatic-mathematical core and should be measured on its correctness. Given this elementary idea – that mathematics is the practice of deductive experimentation with diagrams – Peirce adds a number of further distinctions important for a semiotics of mathematics, that between first-order and second-order diagrams (hypostatic abstractions) the latter taking the former as their objects; that between logical analysis and demonstration; that between simple, ‘corollarial’ proofs and more complicated ‘theorematic’ proofs requiring the introduction of further auxiliary parts to the diagram; and that between diagram subtypes such as maps, graphs and algebra, roughly corresponding to the large mathematical subcontinents of geometry, analysis, algebra, categories, etc.

22

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Mathematical discovery and proof In Peirce’s semiotic philosophy of mathematics, mathematical objects have their objective origins in the mental realm in Peirce’s broad, non-psychologistic meaning of ‘mental’. This is supported by nomenclature some of which is technical, such as his classifications of signs (including, but by no means limited in their importance to, the famous one into icons, indices and symbols); the three modes of reasoning of abduction, deduction (including theorematic reasoning) and induction; the formation of entia rationis by abstraction; the processes of generalization and harmonization that contributes to mathematical discovery; and the experimentation upon diagrammatic constructions in discovering and articulating the deductive inferential steps; and finally, the details of the ensuing theory of what constitutes the actual practice of mathematics and its historical progress throughout the millennia. We will provide a snapshot of a selection of these topics next. Mathematics as an observational science  The doctrine that there must be a senseperceptual, perhaps predominantly visual, element in all thinking has been commonplace since Aristotle: ‘It is impossible ever to think without a mental picture (φαντάσματος). The same affection is involved in thinking as in drawing a diagram (διαγράφειν)’ (De sensu et de memoria, 450a 1–3; translated in Hintikka 1987: 206). To Leibniz ([1674] 1903: 98–9), the purpose of the ‘universal characteristic’ was to oblige reasoning to leave ‘visible traces’, in order to ‘spare the imagination’ – a claim Peirce (n.d.: R 328.42) later echoed in insisting on the diagrammatic character of both geometric and algebraic thinking. Peirce recalls claims such as Gauss’s that mathematics is the ‘science of the eye’ (1896: CP 1.34), that mathematics is an observational science (c. 1899: R 1292), and that mathematicians in general are masters of teasing out the observational elements involved in demonstrative and inventive procedures. To wit, Peirce wrote that mathematics ‘mainly consists in constructing diagrams (continuous in geometry, arrays of signs in algebra) according to general precepts and then observing in the parts of these diagrams relations not explicitly required in the precepts’ (c. 1899: R 1292). By ‘observations’ no simple-minded visual appearance of phenomena is intended. Admittedly, what ‘sensible to the eye’ tends to imply is our intuition of what a diagram is in its ordinary usage. Certainly, appeals to the ‘visual’ status of diagrams are typically met in research on ‘visual’ logics and languages (Hammer 1995; Shin 2002), including visual thinking in mathematics (Mancosu, Jørgensen, and Pedersen 2005). Peirce, however, though certainly including such dimensions in diagrammatic reasoning, did not approve of visuality per se to be the essential feature of diagrams. He repeatedly mentions the possibility of non-visual yet diagrammatic systems of logic: Such a diagram has got to be either auditory or visual, the parts being separated in the one case in time, in the other in space. (1892: CP 3.418) [E]very Deduction involves the observation of a Diagram (whether Optical, Tactical, or Acoustic). (1908: NEM 3.869–70) None of this is to deny visuality as the characteristic mode for diagrammatic representations as far as much of human reasoning is concerned. Certainly, he took it that ‘the best way to think clearly is to think in visual diagrams’. But as every topologist knows it is often

SEMIOTICS IN MATHEMATICS AND LOGIC

23

infinitely better to perform kinaesthetic thought-experiments along the lines of Peirce’s invitation to ‘imagine how things would feel, not how they would look. You must imagine that you move yourself. It is well to imagine oneself free from gravity and swimming about in air or in a liquid’ (R S-25), implicitly referring to the data from our kinaesthetic and tactile senses that contribute to the brain’s drawing of conclusions. The diagrammaticity of syntax evidently is found in spoken language and other auditory means of signification as well, and in species with refined olfactory competences, organizing smell impressions spatially and diagrammatically in order to reason from them is probably a standard evolutionary protocol.

Application of logic to mathematics Abduction, deduction, induction  A proper application of logic to mathematics is, in Peirce’s pioneering view, an application of the logic of science to the discovery of mathematical results. Although mathematics ‘interests itself merely in tracing out the consequences of hypotheses’ (1901: R 250), in order to study the ‘matter of reasoning’ mathematics appeals to logic in its broad sense of the logic of science, bringing out elements of abduction and induction that both precede and entail such ‘tracing out the consequences of the hypotheses’. For also mathematical hypotheses and proofs need to be invented: the emergence of new ideas involves abduction. Truths of mathematics need a corroboration in the wider context of mathematical cultures and practices of argumentation, and those, in turn, involve induction. Above all, mathematical discovery is a discovery of patterns and structures as mental creations: All reasoning in mathematics is performed by the aid of diagrams, using this word in a broad sense. In pure mathematics, we do not trouble ourselves much about the meanings of the diagrams. We can upon occasion, assign to them any appropriate meanings we like. The principal thing is to learn how to work with the diagrams, and to go through the motions of reasoning. (1904: R 693) So even if, ideally, a finished, mathematical proof may be presented in the shape of a stepwise, validated deduction series, the discovery of that proof typically necessitates the whole claviature of inference tools, involving also abduction and induction. Deduction  In the deductive process of reasoning, a reasoner ‘examines the state of things asserted in the premises, forms a diagram of that state of things, perceives in the parts of the diagram relations not explicitly mentioned in the premises’, and finally ‘satisfies itself by mental experiments upon the diagram that these relations would always subsist, or at least would do so in a certain proportion of cases, and concludes their necessary, or probable, truth’ (c. 1896: CP 1.66). Deduction is, however, not exhaustive of what is going on in the practice of mathematical reasoning, as it is preceded by abduction while traced by induction. Abduction contributes to the framing of mathematical hypothesis (conjecture), while induction contributes to the verification and historical and experiential validity of mathematical results. Sandwiched between the two, then, is deduction, the struggle to articulate the minute steps of reasoning into the form known as ‘the proof’. Even deduction itself splits in Peirce’s account into two kinds, namely logical analysis (definition), and demonstration (Pietarinen and Bellucci 2014). It is the latter that is typically associated

24

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

with what deductive mathematical proofs mean. But even so, there are elements in how these proof steps are actually carried out that contribute to the discovery and novelty of mathematical thought, and those presuppose proper definitions, capacity for logical analysis, and ultimately, the theorematicity of proofs. Before going into the latter, we review the involvement of the stages of abduction and induction. Abduction  Blais (1989) has described the general situation as one in which ‘very little human knowledge’ – including mathematical knowledge – ‘is possible within the severe constraints of “keeping to the facts, just the facts”’ (1989: 331). This general methodological lesson is borrowed from the method of science, where it is well recognized that at some point in the investigation the facts will run out, yet science has to keep on progressing. At some point, facts cannot be grounded upon other facts and some facts remain groundless. How to look beyond facts, then? What is the leap of faith to be exercised in guessing about what the future states of affairs may look like? With ungrounded facts at hand, what matters is the evaluation of the totality of what follows as consequences from those facts. Some facts are grounded on nothing else than those conceivable consequences no matter how improbable they may be. As to why such guesswork may nevertheless be trustworthy and far from any random process, Peirce pointed out that since our minds have evolved in affinity with nature, our mental realm is endowed with an ability to discover tentative working hypotheses on the way the world – just as the imaginary world of the mathematics – may be. The process theory of how new ideas are created by reasoning was termed by Peirce abduction. Its leading principle – namely that the mind has evolved in affinity with nature – justifies the schema of abductive inference in which one conjectures the antecedent of the subjunctive in major premise from (2) and (1): (1) A surprising (mathematical) object, X, is discovered. (2) If A were to exist, X would be the matter of course. (3) Thus there is reason to conjecture that A is the case. For example, the major premise could state the subjunctive conditional ‘If there were such things as Peano axioms, then natural numbers would be a credible mathematical phenomenon’. Here the surprise concerns a discovery of an integer. While natural numbers as such (or the relations characterizing them) may not be that surprising to an educated mind, there certainly are endless surprising properties of natural numbers that have been, and are still to be, discovered. Induction  The inductive side of mathematical investigation concerns the degrees of certainty that can be achieved with it: Mathematical certainty is not absolute certainty. For the greatest mathematicians sometimes blunder, and therefore it is possible, – barely possible, – that all have blundered every time they added two and two. Bearing in mind that fact, and bearing in mind the fact that mathematics deals with imaginary states of things upon which experiments can be enormously multiplied at very small cost, we see that it is not impossible that inductive processes should afford the basis of mathematical certainty; and any mathematician can find much in the history of his own thought, and in the public history of mathematics to show that, as a matter of fact, inductive reasoning

SEMIOTICS IN MATHEMATICS AND LOGIC

25

is considerably employed in making sure of the first mathematical premisses. Still, a doubt will arise as to whether this is anything more than a psychological need, whether the reasoning really rests upon induction at all. (1903: R 492, LoF 2/1) This passage helps to explain why Peirce believed (though he left some leeway for doubt on this) that induction contributes to the fallibility of not only scientific experience but also the mathematical one: induction is one of the ways in which all reasoning is justified alongside with abduction and deduction. Still, mathematics is a special case in the sense of its ‘practical infallibility’, that is, a systematic blundering of the temporal collective of the community of mathematicians is ‘barely possible’ – but it is not inconceivable, presupposing that there is an inductive element involved in what constitutes the validity of all reasoning, and therefore the validity of mathematical results. In sum, then, even if the final proof may be presented in its valid outfit, a cleansed product comprising a protracted series of deductive steps, the real discovery and construction of such series of inferential steps hides the messy truth about how the theorems have been arrived at by a variety of semiotic practices and cultures of mathematics. Those involve intellectual guesses, experience, countless dead ends and contradictions, inductive inspiration from the network of related results, the involvement of cumbersome historical pathways, and the constant conceptual flux at the basic level of meaning and signification. To Peirce, such processes were facilitated by the ability of human beings to exert iterated self-control of reasoning up to five or six levels, thus taking a critical, meta-semiotic stance towards their own reasoning, terminology and semiotic conventions (Stjernfelt 2021). Even more, as much as we tend to think of theorems as clear ‘results’ that follow from mathematical axioms through the establishment of their proofs as virtually tautological consequences of those axioms, in practice it is very often rather the axioms that become reinterpreted, adjusted, reworked and perhaps at some point adequately justified by the theorems that they establish. Oftentimes, a reverse methodology takes predecence in deciding which axioms are necessary to account for the proof of a certain theorem, just like the amount of possible consequences in remote parts of mathematics of a certain theorem, in a sort of inductive step, is included in the assessment of its truth (Friedman and Simpson 2000). No wonder that Peirce was in his own words a ‘total disbeliever in axioms’ (1903: CP 7.621) to provide deep insights into this give-and-take dialogical nature of how the process of mathematical knowledge is receiving its evidential affirmations. In some of his last writings on the topic, Peirce maintained that diagrams belong to those ‘quite other systems of signs into which [mathematicians and logicians] are accustomed to translate words and forms of words’ (1910: R 654, LoF 1), that is, to systems of signs other than natural language. Now logical diagrams form ‘the simplest possible system that is capable of expressing with exactitude every possible assertion’ (1910: R 654). In this system of diagrams, Peirce writes, ‘there are none of the ordinary parts of speech. […] its expressions are diagrams upon a surface, and indeed must be regarded as only a projection upon that surface of a sign extended in three dimensions’ (1910: R 654). Peirce then proceeds to present one of the central tenets of logical diagrams for the analysis of the meaning of assertions, namely that ‘three dimensions are necessary and sufficient for the expression of all assertions; so that, if man’s reason was originally limited to the line of speech (which I do not affirm), it has now outgrown the limitation’ (1910: R 654). ‘A drawing or model’, Peirce adds, ‘may be employed to aid the imagination; but the

26

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

essential thing to be performed is the act of imagining’ (1910: R 616, LoF 1). Thus, diagrams overcome the limitations of linearity customarily and traditionally assumed in the notations of logic just as in those of natural languages. This leads to the last topic highlighted in the next section.

Ethics of notation Another fledgling subfield of semiotics of mathematical and logical notations is the ethics of notations and the principles and norms of notational design. Peirce defined the regulating maxim of the ethics of notation to be the following: I set out from the following ethical principle which must appeal to the conscience of every right-minded person: The person who introduces a conception into science has both the right and the duty of prescribing a terminology and a notation for it; and his terminology and notation should be followed except so far as it may prove positively and seriously disadvantageous to the progress of science. If a slight modification is sufficient to remove the objection, a much greater one should be avoided. (1901: R 530, LoF 1) This principle was borrowed from biology where it originated with Linnaeus’ botanical nomenclature. Yet it is fair to state that a proper theory of the ethics of notation has not trespassed us of yet despite Peirce’s repeated accentuation of the importance of agreeing upon such norms over a century ago. Now the paragon of an ‘ethical’ design in novel diagrammatic notations in mathematics may well be category theory (Lambek and Scott 1986; Lawvere and Schanuel 2009). Categories comprise morphisms that preserve a structure of a collection of objects. Ascertaining commutation of a category is to uncover a structure-preserving mapping by uncovering a diagrammatic property. Icons (preservation of a structure) are paramount, leading through experimentation to a discovery of such new properties. Categories are thus a modern meta-level vindication of Peirce’s vision of diagrammatic reasoning as the mode of reasoning by which new structures are discovered by constructions within the mathematical mind, and it has achieved that by the manifest need to expand the notational inventory to the directions in which the rate of mathematical thoughts would be best upregulated. As noted above, philosophy of notation is an emerging science that critically attends to the moves and commitments implicit in privileging any proposed system of signs, as media of communication, design features aiding innovation, models of formal systems or otherwise (Pietarinen 2003).

THEOREMATICITY: A CASE FOR SEMIOTICS IN PROOF DISCOVERY Josiah Royce, Peirce’s devoted student and colleague, ought to be credited for putting his finger on these matters early on. With reference to Poincaré’s problem in his book Science and Hypothesis (1902) on how creative yet mathematical reasoning is possible, Royce emphasizes that what Peirce had called attention to is the issue of the ‘fecundity of the deductive process’ (Royce 1913). Such fertility is, according to Royce, ‘a logical fact’ and not a mere psychological phenomenon. The consequences of mathematical principles ‘are such that novel results in vast numbers are annually discovered […] not stowed away in the premises in any such way’ (1913: 137). Royce explains Peirce having

SEMIOTICS IN MATHEMATICS AND LOGIC

27

shown that any proposition whatever which has a definite meaning permits you to draw from it an infinite number of deductive inferences, all of which are possible without formulating any other basis for the deductions in question than the assertion of the original proposition and the synthetic power which one indeed has in his hands who is capable of understanding certain simple processes of the construction of relations. (1913: 137–8) Such deductions ‘have the character of novelty’, used as instruments of ‘enormous power in those branches of mathematics in which one builds one system of relations upon another system’ (1913: 138). With an allusion to diagrammatic reasoning, Royce then explains how ‘a given set of premises we construct in terms of symbols, diagrams, or figures, hereby expressing the meaning of these premises’. This is the ‘live process of deduction’ which ‘consists of the reading of the meaning [of premises] from a new point of view’. This relation is not the matter of the psychology of producing beliefs in the conclusions but to ‘give us insight into a connection of premises and conclusion’ (1913: 143). In the same period, Peirce proposes his fundamental distinction between corollarial and theorematic deductions – the distinction stemming from his comments upon Euclid, distinguishing small reasoning steps on the one hand (often marked with a small corolla, a wreath, in the margin by commentators), and major theorems to be proved, on the other (Stjernfelt 2011). Far from being a mechanical, automatable routine of calculating conclusions from the premises, deduction of a major theorem oftentimes involves a creative, ‘theoric’ step requiring the reasoner to reconceptualize the problem from a new point of view or to add further auxiliaries to the premises. While ‘corollarial’ deductions can be read directly off of the colligated premises, such is not the case with ‘theorematic’ reasoning. In proving the angle sum of a triangle, Euclid, for instance, has to add certain auxiliary lines to the original triangle diagram. Such a ‘theoric’ step requires semiotic experimentation, as it may not be evident which auxiliary objects to choose or which Gestalt shift of the problem to pursue (Shin 2010; Stjernfelt 2014). Peirce saw it as a major semiotic-epistemological goal to investigate, and so he listed some of the major types of such ‘theoric steps’, an aim whose systematic exposition and expansion lamentably remains unfulfilled. Peirce’s insistence on the difference between the investigation process of finding a proof, on the one hand, and the deductive structure of that proof itself, on the other, exemplifies Reichenbach’s later distinction between ‘context of discovery’ and ‘context of justification’. Where the analytical tradition, however, has taken that distinction as a pretext for philosophy to focus upon the latter predominantly, Peirce insists on involving the discovery procedure in his account. The semiotics of mathematical-logical discovery thus stands out as an important research project to pursue, closing the circle with a return to the opening question of what constitutes good notations: which notational devices, design practices and innovations are conducive to theorematic reasoning?

CONCLUSIONS A semiotic perspective on the exact sciences is both natural and historically justified, even if, in everyday conduct of science, it is something that might strike us a rather unfamiliar undertaking remote to current concerns. Why have the practising scientists as well as scholars attuned to fundamental issues often neglected many of the methods and concepts that semiotics has had in the offing over the years, decades or even centuries?

28

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

This question is a rather open-ended one, and possible answers may range from the long-term unavailability of original texts from Peirce to the ever-advancing ‘barbarism of specialization’ (to borrow Ortega ý Gasset’s phrase that he found appropriate already in the 1930s) – a mindset that not only sees it more appropriate to develop one’s own methodologies within the disciplinary boundaries rather than crossing over them, but is motivated to stay local by short- to mid-term expected gain instead of long-term impact to science and humanity. Other problems and future priorities for the study of mathematics and logic from a semiotic perspective may include the related questions of the strength of the integrative glue that semiotics might provide across mathematics and logic, and increasingly, in the computational renderings of the basic processes and structures in the universe, including origins of life and how complex forms may arise out of simple but abstract relationships. On top of all these, whatever the future directions may turn out be, they are bound to engage with questions arising from similar concerns that occupy semiotics and philosophy of notation too. Indeed, notational choices in logic and mathematics may alter not only inferential perspicuity but cultivate entire ‘innovation ecosystems’ in those sciences and other disciplines applying them. Such changes are the lifeblood of critically attending to the moves and commitments implicit in privileging a given sign system. The semiotic field of notation studies the constraints and possibilities that different notational systems manifest. Although diagrammatic signs clearly have beneficial and important psychological effects, they also – and even predominantly – serve important logical and epistemological functions. A structural constraint may receive a description that needs no appeal to cognitive faculties or abilities. Logical notations employing iconic signs like diagrams are a prime example the characteristic feature of which consists in their departure from the language-like linear strings of conventional signs that are often, though rather illicitly, equated with the exactitude of logical and mathematical notations. All such choices are semiotically motivated, thus rendering mathematical and logical discoveries likewise amenable to evolving semiotic analyses and conceptualizations. The present chapter has highlighted some such contributions largely as they have arisen in Charles S. Peirce’s engagement with these issues.

ACKNOWLEDGEMENTS Work supported (Pietarinen) by the TalTech grant SSGF21021, and The Chinese National Social Science Fund ‘The Historical Evolution of Logical Vocabulary and Research on Philosophical Issues’ (20& ZD046) and (Stjernfelt) by the KHK "Cultures of Research" at the RWTH Aachen 2021–22.

NOTES 1 For more on the contribution of mathematicians to notational innovations, see Cajori 1928–9, Serfati 2005, Dutilh Novaes 2012. According to Dutilh Novaes (2012), for example, a formal language, just as a system of diagrams, may be conceived as a kind of cognitive technology. Thus it matters how that technology is engineered, that is, what the best notations for it may be, and what the norms and reasoning practices are that govern such notational engineering feats.

SEMIOTICS IN MATHEMATICS AND LOGIC

29

2 This is not to state that the two, mathematics as the science of discovery by hypotheses, reasoning and diagrams, and the normative sciences, would be disconnected areas of study. There is much sign theory involved in the analysis of mathematics, which Peirce well acknowledged: ‘Mathematics wholly relates to hypothetical states of things, perfectly exact, but of a kind that only affect a few very general characters. A mathematical operation is a triadic relation. Two of the correlates are analogous to causes. One is the operator the other the operand. The third correlate is analogous to an effect. It is the result. All mathematical operations are determined by entirely general definitions, expressive of the essence of the ideas, and are quite unaffected by any existential conditions’ (1907: R 277, 15 October). 3 The quip itself is a simplified version of what Einstein apparently articulated in the words ‘the supreme goal of all theory is to make the irreducible basic elements as simple and as few as possible without having to surrender the adequate representation of a single datum of experience’ (‘On the Method of Theoretical Physics’, the Herbert Spencer Lecture, Oxford, 10 June 1933). 4 Some recent contributions to the semiotics and its role in mathematics, logic and their education include Mortensen & Roberts 1997; Rotman 2000, 2008; Presmeg et al. 2018. 5 Given the all-pervading hypothetical character of mathematical statements, we can for example further argue that Peirce was not endorsing the ‘classical’ logic for mathematical proofs (namely the invariant truth of the Law of the Excluded Middle) but that the hypothetical view of mathematics generalizes over the classical versus constructive mathematical proofs: certain mathematical theorems are those explicitly recognized to be implied by LEM (Richman 1990).

REFERENCES Allwein, G. and J. Barwise, ed. (1996), Logical Reasoning with Diagrams, Oxford: Oxford University Press. Baron, M. E. (1969), ‘A Note on the Historical Development of Logic Diagrams’, The Mathematical Gazette, 53 (383): 113–25. Barone, F. (1957), Logica formale e logica trascendentale, Torino: Edizioni di Filosofia. Barwise, J. and J. Etchemendy (1996), ‘Visual Information and Valid Reasoning’, in A. Gerard and J. Barwise (eds), Logical Reasoning with Diagrams, 179–200, Oxford: Oxford University Press. Bellucci, F. (2013), ‘Diagrammatic Reasoning. Some Notes on Charles S. Peirce and Friedrich A. Lange’, History and Philosophy of Logic, 34 (4): 293–305. Bellucci, F. and A.-V. Pietarinen (2016a), ‘Peirce’s Existential Graphs as an Instrument of Logical Analysis. Part I: Alpha’, The Review of Symbolic Logic, 9 (2): 209–37. Bellucci, F. and A.-V. Pietarinen (2016b), ‘From Mitchell to Carus: Fourteen Years of Logical Graphs in the Making’, Transactions of the Charles S. Peirce Society, 52 (4), 539–75. Bellucci, F. and A.-V. Pietarinen (2017), ‘Two Dogmas of Diagrammatic Reasoning: A View from Existential Graphs’, in K. Hull and R. K. Atkins (eds), Peirce on Perception and Reasoning: From Icons to Logic, 174–95, London: Routledge. Bellucci, F., A. Moktefi, and A.-V. Pietarinen (2013), ‘Diagrammatic Autarchy: Linear Diagrams in the 17th and 18th Centuries’, in James Burton and Lopamudra Choudhury (eds), Proceedings of the International Workshop on Diagram Logic and Cognition, 23–30, CEUR Workshop Proceedings. Bernhard, P. (2001), Euler-Diagramme, Paderborn: Mentis. Blais, M. J. (1989), ‘A Pragmatic Analysis of Mathematical Realism and Intuitionism’, Philosophia Mathematica, s2–s4 (1): 61–85. Bordron, J. F. (2011), L’iconicité et ses image: Études sémiotiques, Paris: PUF.

30

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Brown, R. and T. Porter (2006), ‘Category Theory: An Abstract Setting for Analogy and Comparison’, in Giandomenico Sica (ed.), What Is Category Theory? 257–74, Monza: Polimetrica. Burkhardt, H. (1980), Logik und Semiotik in der Philosophie von Leibniz, München: Philosophia Verlag. Cajori, F. (1928–9), A History of Mathematical Notations, 2 vols., Chicago: Open Court. Capozzi, M. (2002), Kant e la logica, Napoli: Bibliopolis. Capozzi, M. and G. Roncaglia (2009), ‘Logic and Philosophy of Logic from Humanism to Kant’, in Leila Haaparanta (ed.), The Development of Modern Logic, 78–158, Oxford: Oxford University Press. Cassirer, E. (1923), Philosophie der symbolischen Formen. I. Die Sprache, Berlin: Bruno Cassirer. Coffa, J. A. (1991), The Semantic Tradition from Kant to Carnap: To the Vienna Station, Cambridge, UK: Cambridge University Press. Cooke, E. F. (2003), ‘Peirce, Fallibilism, and the Science of Mathematics’, Philosophia Mathematica, 11: 158–75. Coumet, E. (1977), ‘Sur l’histoire des diagrammes logiques, figures géométriques’, Mathématiques et sciences humaines, 60: 31–62. Couturat, L. (1901), La logique de Leibniz: d’après des documents inédits, Paris: Alcan. Cruse, A. (2011), Meaning in Language: Introduction to Semantics and Pragmatics, Oxford: Oxford University Press. Danesi, M. and M. Bockarova (2014), Mathematics as a Modeling System: A Semiotic Approach, Tartu: Tartu University Press. Dascal, M. (1978), La sémiologie de Leibniz, Paris: Montaigne. Dipert, R. (1997), ‘The Mathematical Structure of the World: The World as Graph’, The Journal of Philosophy, XCIV (7): 329–58. Dondero, M. G. and J. Fontanille (2012), Des images à problèmes: Le sens du visuel à l’épreuve de l’image scientifique, Limoges: Presses universitaires de Limoges. Dutilh Novaes, C. (2012), Formal Languages in Logic: A Philosophical and Cognitive Analysis, Cambridge, UK: Cambridge University Press. Eco, U. (1976), A Theory of Semiotics, Bloomington: Indiana University Press. Edeline, F., J. M. Klinkenberg, and P. Minguet (1992), Traité du signe visuel: pour une rhétorique de l’image, Paris: Seuil. Endrullis, J. and L. S. Moss (2015), ‘Syllogistic Logic with “Most”’, in Logic, Language, Information, and Computation, 124–39, Berlin: Springer. Euler, L. (1835), Letters of Euler on Different Subjects in Natural Philosophy, vol. 1, New York: J. and J. Harper. Feynman, R. P., R. B. Leighton, and M. Sands (1964), The Feynman Lectures on Physics, vol. 2, New York: Addison-Wesley. Frege, G. (1884), Die Grundlagen der Arithmetik, Breslau: Verlag von Wilhelm Koebner. Friedman, H. and S. Simpson (2000), ‘Issues and Problems in Reverse Mathematics’, Contemporary Mathematics, 257: 127–44. Gardner, M. (1958), Logic Machines and Diagrams, New York: McGraw-Hill. Greaves, M. T. (2002), The Philosophical Status of Diagrams, Stanford: CSLI. Gensini, S. (1991), Il naturale e il simbolico: Saggio su Leibniz, Roma: Bulzoni. Geymonat, L. (1947), Storia e filosofia dell’analisi infinitesimal, Torino: Levrotto e Bella. Giaquinto, M. (2007), Visual Thinking in Mathematics, Oxford: Clarendon.

SEMIOTICS IN MATHEMATICS AND LOGIC

31

Glasgow, J., N. H. Narayanan, and B. Chandrasekaran, eds (1995), Diagrammatic Reasoning: Cognitive and Computational Perspectives, Cambridge, MA: MIT Press. Goldblatt, R. (1979), Topoi, the Categorial Analysis of Logic, Studies in the Foundations of Mathematics 98, Amsterdam: North Holland Elsevier. Goodman, N. (1968), Languages of Art, Indianapolis: Bobbs-Merrill. Hadamard, J. (1949), The Psychology of Invention in the Mathematical Field, Princeton: Princeton University Press. Hailperin, T. (2004), ‘Algebraical Logic 1685–1900’, in Dov M. Gabbay and John Woods (eds), The Rise of Modern Logic from Leibniz to Frege, 323–88, Amsterdam: Elsevier North Holland. Hammer, E. M. (1995), Logic and Visual Information, Stanford: CSLI. Hintikka, J. (1980), ‘C. S. Peirce’s “First Real Discovery” and Its Contemporary Relevance’, Monist, 63 (3): 304–15. Hintikka, J. (1973), Logic, Language-Games and Information: Kantian Themes in the Philosophy of Logic, Oxford: Clarendon Press. Hintikka, J. (1987), ‘Mental Models, Semantical Games and the Varieties of Intelligence’, in L. Vaina (ed.), Matters of Intelligence, 197–215, Dordrecht: D. Reidel. Hoffmann, M. H. G. (2011), ‘Cognitive Conditions of Diagrammatic Reasoning’, Semiotica, 186: 189–212. Hookway, C. (1985), Peirce, London: Routledge and Kegan Paul. Kahneman, D. (2011), Thinking, Fast and Slow, New York: Farrar, Straus and Giroux. Kant, I. ([1781] 1998), Critique of Pure Reason, ed. P. Guyer and A. W. Wood, Cambridge: Cambridge University Press. Kneale, W. C. and M. Kneale (1962), The Development of Logic, Oxford: Clarendon. Lambek, J. (1989), ‘On the Sheaf of Possible Worlds’, in J. Adamek and S. Mac Lane (eds), Categorial Topology, Singapore: World Scientific. Lambek, J. and P. J. Scott (1986), Introduction to Higher Order Categorical Logic, Cambridge: Cambridge University Press. Lambert, J. H. (1764), Neues Organon, Leipzig: Wendler. Lange, F. A. (1877), Logische Studien, Iserlohn: J. Baedeker. Lawvere, F. W. (1974), ‘Continuously Variable Sets: Algebraic Geometry = Algebraic Logic’, in H. E. Rose and J. C. Sheperdson (eds.), Proc. A.S.L. Logic Coll, 135–56, Bristol: NorthHolland Publishers Amsterdam. Lawvere, W. and S. Schanuel (2009). Conceptual Mathematics: A First Introduction to Categories, 2nd edn, Cambridge: Cambridge University Press. Legg, C. (2012), ‘What Is a Logical Diagram?’ in Sun-Joo Shin and Amirouche Moktefi (ed.), Visual Reasoning with Diagrams, 1–18, Dordrecht: Springer. Leibniz, G. W. ([1674] 1903), Opuscules et fragments inédits de Leibniz, ed. Louis Couturat, Paris: Alcan. MacLane, S. (1981), ‘Mathematical Models: A Sketch for the Philosophy of Mathematics’, Amer. Math. Monthly, 88: 462–72. Magnani, L. (2011), ‘External Diagrammatization and Iconic Brain Co-Evolution’, Semiotica, 186: 213–38. Mancosu, P., K. F. Jørgensen, and S. A. Pedersen, eds (2005), Visualization, Explanation and Reasoning Styles in Mathematics, Dordrecht: Springer. Moktefi, A. and S. J. Shin (2012), ‘A History of Logic Diagrams’, in Dov Gabbay, Francis J. Pelletier and John Woods (eds), Logic: A History of Its Central Concepts, Amsterdam: North-Holland, 611–82.

32

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Moore, M. E., ed. (2010), New Essays on Peirce’s Mathematical Philosophy, Chicago and La Salle: Open Court. Mortensen, C. and L. Roberts (1997), ‘Semiotics and the Foundations of Mathematics’, Semiotica, 115: 1–27. Nakatsu, R. T. (2010), Diagrammatic Reasoning in AI, Hoboken, NJ: John Wiley. Peirce, C. S. ([1857–1914] 1787–1951), The Charles S. Peirce Papers Harvard University, Houghton Library, MS Am 1632. Individual papers are referenced by manuscript number in R. Robin (ed.), Annotated Catalogue of the Papers of Charles S. Peirce, Amherst: University of Massachusetts Press, 1967, and in Robin, ‘The Peirce Papers: A Supplementary Catalogue’, Transactions of the Charles S. Peirce Society, 7, 1971: 37–57. Cited as R. Peirce, C. S. ([1866–1913] 1931–58), The Collected Papers of Charles Sanders Peirce, vols. 1–6, ed. C. Hartshorne and P. Weiss, vols. 7–8, ed. A. Burks, Cambridge: Harvard University Press. Cited as CP. Peirce, C. S. ([1866–1913] 1976), The New Elements of Mathematics, 4 vols., ed. C. Eisele, The Hague: Mouton Press. Cited as NEM. Peirce, C. S. ([1857–92] 1982–2010), Writings of Charles S. Peirce, 7 vols. (1–6, 8), ed. Peirce Edition Project, Bloomington: Indiana University Press. Cited as W. Peirce, C. S. ([1867–93] 1992), The Essential Peirce: Selected Philosophical Writings, vol. 1, ed. N. Houser and C. Kloesel, Bloomington: Indiana University Press. Cited as EP 1. Peirce, C. S. ([1893–1913] 1998), The Essential Peirce: Selected Philosophical Writings, vol. 2, ed. Peirce Edition Project, Bloomington: Indiana University Press. Cited as EP 2. Peirce, C. S. ([1888–1908] 2010), Philosophy of Mathematics: Selected Writings, ed. M. E. Moore, Bloomington: Indiana University Press. Cited as PoM. Peirce, C. S. ([1895–1913] 2019–22), Logic of the Future: Writings on Existential Graphs, ed. A.-V. Pietarinen, vol. 1: History and Applications; vol. 2/1: The Logical Tracts; vol. 2/2: The 1903 Lowell Lectures; vol. 3/1: Pragmaticism; vol. 3/2: Correspondence, Berlin: De Gruyter. Cited as LoF. Pietarinen, A.-V. (2003), ‘Peirce’s Theory of Communication and Its Contemporary Relevance’, in K. Nyìri (ed.), Mobile Learning: Essays on Philosophy, Psychology and Education, 81–98, Vienna: Passagen Verlag. Pietarinen, A.-V. (2005), ‘Cultivating Habits of Reason: Peirce and the Logica Utens vs. Logica Docens Distinction’, History of Philosophy Quarterly, 22 (4): 357–72. Pietarinen, A.-V. (2006), Signs of Logic: Peircean Themes on the Philosophy of Language, Games, and Communication, Dordrecht: Springer. Pietarinen, A.-V. (2009), ‘Pragmaticism as an Anti-Foundationalist Philosophy of Mathematics’, in B. Van Kerkhove, R. Desmet, and J. P. Van Bendegem (eds), Philosophical Perspectives on Mathematical Practices, 305–33, London: College Publications. Pietarinen, A.-V. (2010), ‘Which Philosophy of Mathematics Is Pragmaticism’, in M. Moore (ed.), New Essays on Peirce’s Mathematical Philosophy, 59–79, Chicago, IL: Open Court. Pietarinen, A.-V. (2011), ‘Existential Graphs: What a Diagrammatic Logic of Cognition Might Look Like’, History and Philosophy of Logic, 32 (3): 265–81. Pietarinen, A.-V. (2012), ‘Peirce and the Logic of Image’, Semiotica, 192: 251–61. Pietarinen, A.-V. and F. Bellucci (2014), ‘New Light on Peirce’s Conceptions of Retroduction, Deduction and Scientific Reasoning’, International Studies in the Philosophy of Science, 28 (4): 353–73. Ploucquet, G. (1766), Sammlung der Schriften, ed. August Friedrich Bök, Tübingen: Cotta. Poincaré, H. (1902), La Science et l’Hypothèse (Science and Hypothesis, 1905).

SEMIOTICS IN MATHEMATICS AND LOGIC

33

Presmeg, N., L. Radford, W.-M. Roth, G. Kadunz, eds (2018), Signs of Signification: Semiotics in Mathematics Education Research, Cham: Springer. Richman, F. (1990), ‘Intuitionism as Generalization’, Philosophia Mathematica, 5 (1–2): 124–8. Roberts, D. D. (1973), The Existential Graphs of Charles S. Peirce, The Hague: Mouton. Rotman, B. (2000), Mathematics as Sign: Writing, Imagining, Counting, Stanford: Stanford University Press. Rotman, B. (2008), Becoming beside Ourselves: The Alphabet, Ghosts, and Distributed Human Being, London: Duke University Press. Royce, J. ([1913] 2001), ‘Some Psychological Problems Emphasised by Pragmatism’, in F. M. Oppenheim (ed.), Royce J. Josiah Royce’s Late Writings: A Collection of Unpublished and Scattered Works, vol. 1, 129–46, Bristol: Thoemmes Press. Semetsky, I. (2015), ‘Interpreting Peirce’s Abduction through the Lens of Mathematics’, in M. Bocharova, M. Danesi, D. Martinovic, and R. Núñez (eds), Mind in Mathematics: Essays on Mathematical Cognition and Mathematical Method, 154–66, Munich: Lincom. Serfati, M. (2001), ‘Mathématiques et pensée symbolique chez Leibniz’, Revue d’histoire des sciences, 54 (2): 165–221. Serfati, M. (2005), La révolution symbolique: La constitution de l’écriture symbolique mathématique, Paris: Editions Petra. Shimojima, A. (1996), On the Efficacy of Representation, Diss., Indiana University, Bloomington. Shin, S. J. (1995), The Logical Status of Diagrams, Cambridge, UK: Cambridge University Press. Shin, S. J. (2002), The Iconic Logic of Peirce’s Graphs, Cambridge, MA: MIT Press. Shin, S. J. (2010), ‘Peirce’s Two Ways of Abstraction’, in M. E. Moore (ed.), New Essays on Peirce’s Mathematical Philosophy, 41–58, Chicago and La Salle: Open Court. Simone, R., ed. (1995), Iconicity in Language, Amsterdam: John Benjamins. Stjernfelt, F. (2007), Diagrammatology: An Investigation on the Borderlines of Phenomenology, Ontology, and Semiotics, Dordrecht: Springer. Stjernfelt, F. (2011), ‘Peirce’s Notion of Diagram Experiment: Corollarial and Theorematical Experiments with Diagrams’, in Richard Heinrich, Elisabeth Nemeth, Wolfram Pichler, David Wagner (eds), Image and Imaging in Philosophy, Science and the Arts, vol. 2, 305–40, Frankfurt: Ontos Verlag. Stjernfelt, F. (2014), Natural Propositions: The Actuality of Peirce’s Doctrine of Dicisigns, Boston: Docent. Stjernfelt, F. (2019), ‘Dimensions of Peircean Diagrammaticality’, Semiotica, 228: 301–31. Stjernfelt, F. (2021), ‘Conscious Self-Control as Criterion for Reasoning’, Cognitive Semiotics, 14 (1): 71–99. Thiel, C. (1993), ‘Friedrich Albert Langes bewundernswerte Logische Studien’, History and Philosophy of Logic, 15 (1): 105–26. Van Langendonck, W. (2007), ‘Iconicity’, in Dirk Geeraerts and Hubert Cuyckens (eds), The Oxford Handbook of Cognitive Linguistics, 394–418, Oxford: Oxford University Press. Venn, J. (1881), Symbolic Logic, London: Macmillan. von Neumann, J. (1958), The Computer and the Brain, New Haven: Yale University Press. Wittgenstein, L. (1921), Tractatus Logico-Philosophicus, trans. C. K. Ogden and F. P. Ramsey, London and New York: Routledge. Wolfram, S. (2020). ‘A Class of Models with the Potential to Represent Fundamental Physics’, Complex Systems, 29 (2): 107–536.

34

CHAPTER TWO

Semiotics in General Biology KALEVI KULL AND DONALD FAVAREAU

INTRODUCTION General biology is the discipline that describes and explains the organization and interactions of all living systems, much of which is based on organisms’ abilities for perceiving, knowing and communicating. Accordingly, the existence of meaning-making, interpretation and sign-use in many species besides the human is the primary reason that applying semiotics in biology is unavoidable. But there is also a second reason, and this concerns the phenomenon of life itself. Included in the very definition of ‘being alive’ are the phenomena of function, value and meaningfulness, which are themselves, by definition, semiosic qualities. The relationship between semiotics and biology has been formulated in a radical form in Sebeok’s Thesis: ‘The phenomenon that distinguishes life forms from inanimate objects is semiosis’ (Sebeok 2001: 1). An analogical consequence of this posit is that cognition, knowing, is a general feature of life, since knowing is obviously a semiosic phenomenon. While discussions continue as to whether semiosic phenomena are universal to life, or restricted to only some of its forms (e.g. only eukaryotes, or animals, or vertebrates), the existence of semiosis in the biological realm is generally accepted by almost all semiotic schools, and the study of life as a semiosic phenomenon has been regularly present in biology (see reviews in Favareau 2010; Ingensiep 2001). Accordingly, the study of interpretation and meaning-making in the realm of biology has at least two important aims: 1. To discover the primary conditions for the emergence of meaning in living beings, and together with this, to build the basis for general semiotics; 2. To re-conceptualize the theory of general biology so that its relationship to semiotics will be explicit. The scope of semiotics as appropriate for biology could be thus formulated as the study of patterns, processes, products and consequences of interpretive behaviour (Pap 1979: 3401); or, equivocally: the study of semiosis: its preconditions, forms and products, the latter including codes and artefacts. At a more mature and fully elaborated stage, then, general biology will include – alongside the subfields of biophysics, biochemistry, cytology, histology, physiology, ethology, morphology, ontogenetics, phylogenetics, ecology – also a subfield of biosemiotics, which describes the ways in which living beings acquire and use knowledge, as well as the types of interpreting agents and the mechanisms of interpretation. In this way, the study of the subjectivity of organisms will have an explicitly semiotic basis.

36

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

This chapter provides a brief review of how the processes of life have been seen from the semiotic point of view in general biology thus far, and indicates the ways in which the application of a more explicitly semiotic understanding of life may aid in the development of general biology in the future. It argues that since the processes of meaning-making, i.e. semiosis, cannot be eliminated in the study of living beings, general biology will benefit greatly by drawing from and building upon the work that has already been done in investigating meaning-making in the discipline of semiotics. In the process, certain important changes in, and of benefit to, the general model of semiosis will accrue, as an important part of this more unified perspective will include more foundational definitions and analyses of interpretation, choice, categorization, meaning, habit, representation, learning, evolution and translation. Finally, some of the outstanding problems, lacunae, and research priorities for the development of a more semiotic general biology will be addressed.

HISTORY As of this writing, reviews of the application of semiotics in the history of biology have been limited to only a few, although rather extensive, accounts (e.g. Kull 1999; Favareau 2010; Maran et al. 2011). Yet even these were not able to cover the whole scope of biosemiotic work in its relationship to the development of knowledge in, and importance for, general biology. And indeed, although the careful study of signs in living systems has taken place throughout all of history of human-animal communication and medicine (Deely 2001), the semiotic program in modern biology largely began with Jakob von Uexküll’s Theoretische Biologie (in its extensive form, Uexküll 1928). Conducting and reporting detailed case studies on the physiological mechanisms mediating perception and action in a number of different species (particularly invertebrates), he formulated the general semiotic model of the functional circle (Funktionskreis), which was seen by Uexküll as the dynamic mechanism of meaning-making, and the system of processes that creates the organism’s subjective world, its umwelt. Uexküll developed a theory of meaning based on biology (Uexküll 1940), and a theory of general biology in which the concept of meaning-making had remarkable pride of place. And while Uexküll did not use the terms ‘semiotics’ or ‘semiology’, he did use the concepts of sign and meaning. Importantly, he also introduced the diagnosis for what he saw as limiting the further development of biology: Bedeutungsblindheit, blindness to the reality and importance of meaning in the organization and interaction of living beings (1940: 1). The attempts to correct this problem are what has motivated the incorporation of semiotics into biology ever since. Interestingly enough, Uexküll’s own approach found only few such followers until the last decades of the twentieth century, and this was because the prevailing arguments used in biology since the 1930s (which began the period of dominance of neo-Darwinism) were based on evolutionary explanations, whereas Uexküll was critical of Darwinian theory and the role of natural selection; he emphasized and used instead mainly synchronic, rather than diachronic, explanations for the understanding of animal meaning-making. A rediscovery of Uexküll’s works began in the 1970s, however, particularly by Thomas A. Sebeok, who recognized these ideas as being revolutionarily semiotic. Sebeok at first characterized Uexküll’s ‘Theory of Meaning’ (Uexküll 1940) as a ‘pioneer monograph on zoosemantics’ (Sebeok 1972: 160) and soon after, Uexküll’s work was included in the list of semiotics

SEMIOTICS IN GENERAL BIOLOGY

37

classics (T. v. Uexküll 1981). The attention towards Uexküll’s work grew further, especially after the 1990s (Kull 2001; Brentari 2015; Michelini, Köchy 2020). A separate path towards semiotic biology was initiated by the works of Charles W.  Morris, starting as early as 1925 (Petrilli 1999). Morris argued that there is ‘[n]o reason why sign-processes, for all their immediate sense of familiarity, should not be as complex as any chemical structure or biological functioning’ (Morris 1955: 11). Morris was influenced by the semiotic realism of Charles Peirce, and made it clear that semiotics should also deal with animal communication, using Peirce’s pragmatic theory of meaning. These ideas were picked up by ethologist Peter Marler (1961; see also Anastasi 2017), who applied a semiotic analysis to the description of the types and function of bird songs, and by entomologist Rudolf Jander (1981). Morris’s insistence that the semantic, syntactic and pragmatic dimensions of semiosis should be included into every analysis of a sign, was applied by Günter Tembrock in his study of bio-communication (1971) and, if somewhat less explicitly, by oncologist Giorgio Prodi in his attempts to fashion a semiotics of the immune system (1977, 1988). Morris’s approach to semiotics was also used by Friedrich Rothschild, who was the first (according to our knowledge) to use the term ‘biosemiotics’ in its contemporary sense (Rothschild 1962; 1968); subsequently, the term ‘biosemiotics’ has been independently coined a number of times (by Stepanov, Florkin and perhaps some others).2 Other early works appearing under the label ‘biosemiotics’ were attempts at applying the structuralist semiotics of the Saussurean tradition to the findings of biology. Prominent among these was Marcel Florkin (1974), whose ‘molecular biosemiotics’ was an attempt to interpret the biochemical processes of the cell. Florkin modelled his concept of the molecular ‘bioseme’ upon Saussure’s signifié/signifiant duality (1974: 15). Similarly, after the discovery of genetic code, a number of works on the application of semiotics to genetics appeared, and the language metaphor for genes became rather widely used (and still persists in the layman’s discourse today). Roman Jakobson’s writings (especially Jakobson, Waugh 1987: 69), with their ‘double-articulation’ analogy of nucleotides as phonemes and codons as morphemes, reflect the tone of such discussions. Somewhat deeper applications of structuralist semiotics were made in developmental biology. When Conrad H. Waddington (1972: 289) concluded that ‘[i]t is language […] that I suggest may become a paradigm for the theory of General Biology’, it was semiotic structuralism that he was referring to. Approaches in this line included works by René Thom (1980), Brian Goodwin (1972), Howard Pattee (1972), Stuart Kauffman (1996) and particularly the Osaka Group for the Study of Dynamic Structure (Ho 1989), as well as the criticisms of neo-Darwinism by Gregory Bateson (1973), Giuseppe Sermonti (1999), Marcello Barbieri (1985) and others. In parallel, the work on biocybernetics contributed to the lines of semiotic thought in biology by developing a theory of controls.3 Important work on relational biology, close in its approach to semiotic biology, was done by Robert Rosen (1991; see also Gare 2019). In the fields of animal cognition, behaviour and communication – all areas in which semiotic models and analysis can be effectively applied – the explicit usage of semiotic concepts in ethology, surprisingly, has been relatively rare. Exceptions include, besides Marler, for instance, the works of Heini Hediger and Günter Tembrock (see reviews by Martinelli 2010; Maran et al. 2011), and, more recently, Karel Kleisner (2008; drawing on the work of Adolf Portmann), Morten Tønnessen et al. (2016), Günther Witzany (2014) and El-Hani et al. (2009). Almo Farina’s decades-long work on avian (and, more

38

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

recently, marine) soundscapes must be mentioned here (Farina 2012), as well as that of the zoo- and zoösemiotician Aleksei Turovski (2011). Systematic work on developing an explicitly semiotic theory of life (together with the formation of a group from Copenhagen and Tartu who later institutionalized biosemiotics) started in the 1990s (Sebeok, Umiker-Sebeok 1992; Hoffmeyer 1996; 2008; Deacon 1997; 2012; Barbieri 2001, 2007; Markoš 2002; Brier 2008; Bruni 2008; Kull et al. 2009; Favareau 2010; Emmeche, Kull 2011; Favareau et al. 2012; Rattasepp, Bennett 2012; Sharov 2013; Stjernfelt 2014; etc.),4 and for the most part, though not unanimously, this development also saw a turn towards a more Peircean approach, in parallel with such a turn in the whole of semiotics. Biological codes and organic artefacts are methodologically simpler to study than are processes of interpretation. The concept of organic code was elaborated by Marcello Barbieri (2001), who also pointed out this methodological difference, with a proposition towards its inclusion in the framework of biosemiotics. In response, it was argued that codes, being necessary for semiosis, however, are not sufficient for semiosis (Markoš 2010; Deacon 2015; Gare 2019; Kull 2020). In sum, the development of biosemiotics since the 1990s goes hand in hand with – and, indeed, is part of – widespread changes going on within general biology, characterized by a turn away from reductionist-mechanical approaches and towards more complex dynamic systems thinking, the epigenetic turn, cognitive biology (Tommasi et al. 2009; Auletta 2011; the latter includes a chapter on semiotics), and the ‘extended synthesis’ post-neo-Darwinian understanding of evolution, which includes ideas contributed to it by biosemiotics.

METHODOLOGIES In addition to providing new models to biology for understanding and describing life, semiotics is also important for further substantiating methodology in biology. Below are some significant problems in this regard that semiotics is, or can be, helpful for.

Scientific argumentation and proofs If there are regularities of living systems that are not deterministic nor stochastic, but are instead based on interpretation on the part of the organisms themselves, as biosemiotics argues, then the methods of measurement, and the evaluation of models, will have to take into account the deep differences in measurement uncertainty, and thus of methodology, between biology and the sciences of non-living matter. In other words, given the semiotic nature of the biological objects of study, the appropriate research methodology must also reflect these features. A remarkable aspect of non-physicalist methodology stems from the role and origin of exceptions. Since semiotic regularities (such as rules and habits), unlike physical laws, by their nature, may have exceptions, this implies that observed cases that appear to contradict the rule cannot be used as facts arguing for the inconsistency of the rule. A typical example is the existence of species’ boundaries: the existence of intermediate forms cannot disprove the existence of boundaries. This is related to the nature of many biological systems that are formed on the basis of family resemblance. Likewise, the claim that the regions of the brain are densely interconnected does not necessarily entail that some parts of the brain are not more (or less) interconnected than others, and may form functional or

SEMIOTICS IN GENERAL BIOLOGY

39

morphological wholes. The logic of the living does not map on neatly to the logic of the mathematician or the architect. Accordingly, biology has always used qualitative methods of observation rather extensively, in parallel with quantitative ones. Their role appears evident and necessary if one is to capture the semiotic nature of the objects under study.5

Reality of meaning and function Biological function and meaning are closely related concepts. However, they are not identical. Biological function has been described as ‘some process or part that serves some purpose, in the context of the maintenance and reproduction of the whole organism’ (Emmeche 2002: 15). Due to its polysemic use and history, ‘meaning’ is either not used in semiotic theories as a technical term, or it is taken as a primary semiotic concept and is left undefined. We might think of it as the entire possible world of interpretation. The function of an organ, as well as the meaning of a mediator (such as a molecule, word or sign), is defined by their relation to something other than themselves (an ‘absential’ relation in Terrence Deacon’s terms; a fundamental ‘being-towards-another’ in the words of John Deely). The effects of this relation can (sometimes) be well seen by a researcher, or an external observer, but if the relation itself is to be seen as a genuinely intrinsic component feature of an organism (and not just as a nominalistic posit), it must exist as such – i.e. in its being as this ‘mediation’ – for the organism itself. Thus, the relationally and experientially embodied ‘function of the eye’ (something to use in order to see), as well as the similarly relationally and experientially embodied ‘meaning of the object seen by the eye’ (e.g., something to catch), must be assumed to actually exist to the organism, if they are to have any genuine ‘reality’ about them at all. Whereas something like the ‘size of the eyeball’ is a relationally embodied, but not experientially embodied, reality that may be seen by an external observer, but not necessarily assumed to exist as such for the organism itself. Nominalism, long thought to be a first-person category error imposed on objectivity, would thus in this instance turn out to be a third-person category error imposed on subjectivity. Traditional biology has long found the concepts of function and meaning for the organism exceedingly hard to define and explain, as it may be that any device capable of detecting the ‘existence’ of meaning must itself have the ability to create that very meaning. The application of semiotics to biology may be exactly what is needed, however, in order to be able to explain the reality of meaning, as well as its necessary and sufficient conditions, in living beings.

Umwelt-research Umwelt is not only species-specific, but, being subjective, it also varies from individual to individual. The principles for the empirical study of umwelt were laid down by Jakob von Uexküll almost a century ago. According to his approach, the description of umwelt requires at least a double mode of study of both physiology and behaviour, both of which can be experimental. Physiological study describes the capacities of sense organs (receptors) to react to various physical factors, the capacities of motor organs (effectors) to change the state of the environment (and of the organism), and the connection networks between the receptor and the effector organs. Behavioural study describes the behaviour of the organism in natural conditions, and the qualities that contribute to, and do not

40

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

contribute to, behavioural decision-making. Accordingly, the integration of physiological and behavioural descriptions allows one to construct, in a rather rich way, the description of umwelt.

Gradualism versus discontinuity Whether the world is modelled as more continuous or more discrete relies on a metaphysical preference and cannot have a final empirical solution. This difference can be well seen in biological debates, for instance, between gradualism and punctualism in evolutionary studies, or between gradualism and emergentism both in origin of life and in levels of organization research. Semiotics has introduced into biology an additional debate about the thresholds between the types of sign use, and by extension, types of sign users, in the living world. Despite the strong influence of the researchers’ metaphysical stance, it is still possible to arrive at agreement on the relative disparity of changes or differences, since the extent of difference is obviously not the same everywhere. Two particular semiotic problems for general biology are the existence and placement of the lower semiotic threshold (according to the current viewpoint, close to, if not coextensive with, the life/non-life boundary), and similarly, the symbolic threshold (often claimed to fall within the language/pre-linguistic sign systems boundary). The question of phytosemiosis – the extent, nature and even existence of genuinely semiotic processes in plants – was introduced by Martin Krampen in 1981, and can arguably be seen as yet another ongoing debate about thresholdism in biology.

Multiple semiotic schools Semiotic theory in biology, as elsewhere, is not integrated. It includes, minimally, the extant incompatible conceptual systems of Peircean, Saussurean, Jakobsonian, Batesonian, Lotmanian, hermeneutic and other approaches – and the differences and debates between these conceptual systems and semiotic schools are inherited in the debates about the semiotic problems of biology. As noted previously, a generally Peircean model of semiosis has become most prevalent in biosemiotics for at least a couple of decades now (as championed by, among others, Sebeok, Hoffmeyer, Stjernfelt and Deacon). However, it is evident that the Peircean approach also has been seen to have some limitations, as actively discussed in biosemiotics (Vehkavaara 2003; Karatay et al. 2016; Rodríguez Higuera 2019; see also the section on Priorities below).

Facing the fundamentals of semiotics Since biosemiotics is working at the most fundamental level of semiotics, it has a potential to integrate many different semiotic approaches and models. Studying the order in which the features of semiosis appear in embryonic development, which is an important area of study for biosemiotics, may help to build the basal models of semiosis, communication and the typology of signs in the right way, thus contributing importantly to general semiotics.

THE STATE-OF-THE-ART Important aspects of semiotic studies in biology include (a) mechanisms of categorization (including speciation, differentiation and perceptual categorization); (b) the primary mechanisms and conditions of ‘choice’; (c) habituation and learning; (d) the formation and persistence of codes; (e) the semiotic theory of agency and of organism; (f) the typology

SEMIOTICS IN GENERAL BIOLOGY

41

of pre-linguistic meaning-making (or signs) and the emergence of new types of semiosis in ontogeny; (g) consequences of semiotic models for the theory of evolution; (h) the semiotic basis of general communicative ecology. We describe some of these semiotic aspects of biology below, in the context of the major issues of general biology: i.e. the nature of life, codes, speciation (self-categorization), communication, learning, evolution and ecology.

Studies of semiosis and the nature of life The main features of living systems have been derived from three basic processes, which underpin the major classes of biological explanations and theories – these are reproduction, homeostasis and semiosis. Obviously, however, all these processes must co-exist in the living, and therefore the proper description, as well as understanding, of life processes must include all three. The semiotic aspect, the need for the understanding of which was developed latest, largely completes the general model of life. Let us briefly examine how these three necessary aspects of life have resulted in three different explanatory models: 1. Reproduction. This is a simple feature, generally described as a cyclic autocatalytic process: O+R→2O, in which a structure, O, assimilates a resource, R, and as a result produces two structures of O, the original and its replica. Here O can be a simple molecule. However, in order to make diverse and still self-reproducing products (open evolution) available, O should have a variety of forms that all provide the template-based synthesis for the same kind of O. Polycondensate nucleic acids are ideal molecules of such kind. Provided that these can serve additionally as specific catalysts for other processes, each kind of O can have a unique conglomerate of substances and processes surrounding it, forming a cell, or organism. Random imprecisenesses in the reproduction process open the way to diversification and evolution. This is the core of basic Darwinian theory and the neo-Darwinian mechanism of evolution. Survivorship of particular kinds of O can be described by their fitness, or reproductive success, and all diversity of life can be seen as a consequence of precisely this kind of self-reproduction, along with its occasionally ‘lucky’ errors. 2. Homeostasis. Homeostasis (‘keeping in being’) is a process which includes autocatalytic sub-processes that are responsible for activity and work, and can be describable as a system of positive feedback, to which another process provides negative feedback, after taking the output of the former as its input. One homeostatic system can engage with, or become entwined with another homeostatic system, readjusting itself as a result, without losing its own homeostaticity, due to the selfregulated persistence of such feedback cycles. Such a model is describable as the core of an organism or an organismic system. Organicist, biocybernetic, homeorhetic and autopoietic theories are all derivable from this core model. According to the class of homeostatic models, reproduction is seen as a secondary feature, appearing under the control of organismic feedback, while plasticity turns out to be an important feature. Therefore, theories based on this model have been often seen as alternatives to the neo-Darwinian perspective. 3. Semiosis. This is an action undertaken in the situation of indeterminacy, more commonly described as interpretation. This requires an agent, obviously including an autocatalytic process (or more commonly, processes), responsible for its work. An indeterminate situation appears when the agent has more than one way to act in a particular situation and must actively choose only one. The multiplicity

42

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

of potential actions is based on more than one functional circle being applicable in the same situation. In other words: possibilities appear due to incompatible rules that organize the organism’s behavior (which can be thought of as a kind of ‘multivitality’). Active interpretation – semiosis – is the activity by which the organism resolves these moments of indeterminacy. Why is the concept of ‘rules’ central to this definition? Because a rule is that the following of which, can fail. Following a law, as in the laws of the non-living world, is something that cannot be failed. And why must ‘more than one’ rule be simultaneously in play? Because if there is only one rule for action or reaction, what one has is still determinism, not interpretation or semiosis. Such a rule can fail, but it cannot be chosen, as the availability of only one rule is not an indeterminate situation. Failure, the possibility to be wrong in one’s interpretation, is therefore not an option that can be chosen. But if there are two incompatible rules, then the situation is qualitatively completely different. For then the deviation from one, and the going along the other, is not a failure, it is, rather, a choice. This capacity is what distinguishes the living from the non-living, and semiosis from determinism. Seen as such, the conditions for semiosis can be fulfilled in rather simple organisms. Uexküll, for instance, describes such functional circles in a unicellular protist (Uexküll 1928: 126, 147). Yet the precise mechanism of semiosis is still under discussion. Since it provides freedom and is a point where subjectivity begins, however, scholars often refuse to call it a mechanism (Henning, Scarfe 2013).

The study of codes An understanding of biological semiosis requires, first, the proper concept of code, or rule. This notion was introduced into semiotic study from Saussure, developed rather independently in Information Theory, and was introduced widely into biology through the concept of genetic code. It was elaborated in biosemiotics most especially by the works of Marcello Barbieri (2001). The processes that make organisms, and that machines can be built to use, rely on regularized links between things and events that, by themselves, would not be and cannot be spontaneously and regularly linked. The basis of such linking is mediation – the usage of specifically developed independent mediators. Being artefacts that have been constructed, such code-making mediators are independently restorable or reproducible, and a decisive feature of a code is the arbitrarity of the link it executes. For instance, the correspondence between certain codons and certain amino acids in the genetic code is regular; it is, however arbitrary, in the sense that there is no physico-chemical reason why this, and not some other, correspondence would have had to have been necessarily instantiated. (And here lies the big difference between genetic translation and transcription – the latter is based on direct stereochemistry, not on the presence and activity of mediators.) The particular choice of correspondence that is fixed in a code can therefore not be calculated on the basis of stereochemical forces, to the extent that these leave several possibilities free in the case of mediators. Codes, therefore, are the products of historical contingency, not physical necessity. As such, they can also be the results of learning, as what is meant by learning always assumes a certain freedom to arrive at a different correspondence. This (often hidden) choice is the basis for arbitrarity, which Howard Pattee calls the ‘epistemic cut’ in discussing the distinction between the interpreter who takes the measurement and that which is being measured. Since ‘choice’ is commonly spoken of as a self-conscious act, and the more primitive forms of ‘learning’ may, on the surface, appear to be rather predictable, there has been

SEMIOTICS IN GENERAL BIOLOGY

43

much misunderstanding of, and resistance to, the interpretation of the organic codes (and, convergently, iconic and indexical relations) as being ‘arbitrary’. Arbitrariness, however, is not the opposite of Motivatedness, nor is it in any way, at least in the living world, the synonym of ‘randomness’ – quite the opposite! The capacity for arbitrariness is, rather, the precondition for any choice, whether motivated or not, although would be highly unlikely to find a completely unmotivated choice. Thus, codes in biology, while not random, are always arbitrary.

Categorization and speciation The concept of species – one of the most widely used in biology – was initially imported from scholastic logic, as Linnaeus applied it in the development of his taxonomies. A remarkable turn was made in the 1930s, when Ernst Mayr paid attention to the processes that are (or can be) responsible for the formation of ‘natural categories’ of organisms due to the possibility for reproductive interaction between organisms, and introduced what he called the ‘biological concept of species’. However, according to Mayr, the relevant phenomena were mostly isolation barriers that might restrict organisms from mating with each other, thus resulting in the separated groups called ‘species’. The concept was critically elaborated by Hugh Paterson, who demonstrated that the processes responsible for the categorization of organisms into species were mainly those of mutual recognition enabling mating, and resulting in the boundaries between species as secondary or derivative. This acknowledgement of the recognition process being central led to the biosemiotic concept of species. The biosemiotic concept of species describes the categorization of communicatively interacting organisms as being, to a large extent, analogous to the intercellular processes of perceptual categorization. This means speciation is only one more example of the more general phenomenon of semiotic categorization. A particularly important feature characterizing semiotic categorization is what Ludwig Wittgenstein called ‘family resemblance’ – i.e. due to the fact that categorization is based on pairwise recognition, and each organism (or cell, in the case of perceptual categorization) can be, and usually is, at least slightly individual, the resulting category may not have any one particular, or even general, distinctive feature that would separate the organisms of one species from the organisms of their neighbouring species. Nevertheless, the boundaries that do arise are rather stable, with intermediate specimens rather frequently existing. Here again, we see that the logic of biology is not the logic of the excluded middle. And thus the critical need for a semiotic perspective within biology.

The study of communication and bio-translation What distinguishes communication from simpler forms of interaction is that it by necessity includes interpretation. Or, according to another definition, communication is interaction based on signs. Under either formulation, communication is therefore one of the defining attributes of life. Communication per se, it must be noted, does not always entail the sending of any messages, i.e. it may be unintentional. For instance, if two butterflies follow each other, both using visual interpretation, this interaction is a case of communication, without any intentional messages being sent by either one. Auto-communication, i.e. interpretation of oneself, is likewise a form of individual communication. Communication research, not surprisingly, is therefore a major field of semiotics. Semiotic communication models have been widely applied in biology (Tembrock 1971), and Roman Jakobson’s communication model has been analysed from a biological point of view by Stefan Artmann, Timo Maran

44

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

and others, while Günther Witzany has examined a wide variety of species using the Theory of Communicative Action by Jürgen Habermas. Communication is also the process by which social systems, including those of nonhuman sociality – e.g. families, herds, flocks, schools and superorganisms – are created. If intercellular interactions can be shown to be communicative (i.e. actions based on interpretation, and not necessity nor stochasticity), then multicellular organisms and embryogenesis will have to be explained on the basis of semiotic models. Likewise, what happens in many instances of cellular communication has been described in terms of a ‘translation process’, where ‘translation’ can be defined as ‘evaluative interpretation’ (including on the level of the cell and between the various internal systems of the body). Here translation studies (see, e.g., Marais 2019) may have important insights to offer for the development of a general semiotic biology.

The study of learning The existence of at least a minimal capacity for learning has been demonstrated in almost every living system. Despite ‘knowledge’ (the product of learning) being not a classical biological concept, it is a fundamentally semiotic one – therefore the application of the concept of ‘knowledge’ in the biological studies of learning should be appropriate. Due to the popular concept of ‘machine learning’, however, it is necessary to distinguish between deterministic and indeterministic ‘learning’. In the computational approach, learning is commonly modelled as an effect in certain networks brought about through the action of logic gates (formal neural networks). The work of such logic gates is algorithmically deterministic, and therefore what emerges from it is deterministic learning (if the term ‘learning’ is even appropriate here; sometimes the less metaphoric terms ‘system entrainment’ or ‘recurrent optimization’ are used to describe the human act of crafting, via computers, statistically emergent output functions from multiple streams of under-specified input data). The semiotic concept of learning describes learning as a naturalistic process that starts with behavioural indeterminacy (situations of incompatibility, confusion, logical conflict, problem or ambiguity). These are situations in which there are multiple simultaneously available options, or possibilities, to choose from. Semiotic learning appears when the choices made by such interpretations become retained and re-used in similar situations. Four major types of learning can be distinguished, which correspond to four types of semiosis and signs: imprinting, conditioning, imitation, and instruction (Zlatev 2002; Kull 2018). 1. Imprinting. Learning by attending to a ‘point’ (datum of experience) that is one of several options present. It is the simplest form of choice, since it does not assume any additional reasoning during the making of the choice (it even does not require any goal). What is chosen turns out itself to become the object. This is an iconic relation. 2. Association (in a narrow sense). Learning by attending to two ‘points’ that are co-present. This includes attending to one datum of perceptual experience, and also a second that is different from the first (however close to the first, co-present, or co-located), and thus establishing a relation between these two. Such established co-relatedness, or regular co-presence, between the two becomes interpreted as an indexical sign. Strengthened by conditioning, it represents an existing correlation (see examples in Hawkins, Byrne 2015).

SEMIOTICS IN GENERAL BIOLOGY

45

3. Imitation. Learning by attending to a spatio-temporal ‘track’ between two or more ‘points’ that are co-present. In addition to (2), it includes learning to attend to the movements from one point to another. Tracking thus appears as the modelling of form. The corresponding sign is an emon (Kull 2018: 140–1). 4. Instruction or learning by attending to an intentionally created other for the form. Learning to attend to that which has been made upon on the basis of the form recognized, but is not co-extensive with the original form itself. Rather, learning by instruction consists precisely in attending to a form that has been made differently from the form that is given in perceptual experience, and coming to know the latter through the former. This corresponds to the symbolic relation, an example of which is naming. Because of this independent form-making upon dependent forms, symbolic operations assume the capacity for the construction of the opposite, the non-existent, the negation. This is a detached, un-grounded6 relationship that has to be learned from other symbol users how to be made and used, which is why ‘instruction’ plays such a major role in the human use of signs. It should be noted that these four types of choices, related to learning and the types of signs, are presented here in order of their growing complexity, where each higher type of learning necessarily includes all of the lower ones. Here, again, we see the need for semiotic analysis to understand the whole of learning we see being done by living systems.

The study of evolution According to the semiotic model, an organism can be seen as a communicative system, both externally and internally. Organismic being includes semiosis – a searching and problemsolving process that underlies the decisions organisms make, on both the individual level, as well as the level of the lineage. These include, among others, decisions resulting in genome expression (phenotype-building), in behaviour, and in reproduction (these features of autonomous activity have also often been defined as constituting ‘agency’). Seen thusly, semiosis is the epigenetic process par excellence, in that it induces profound changes in the organism independently of genetic mutation or DNA sequence change. Rather, the organizational power of semiosis means the capacity for learning, and thus adaptive plasticity. Accordingly, the recognition of the activity of the organism as being based on semiosis (together with certain across-generation effects of habitat choice, food choice, partner choice, niche construction, etc.) provides a model for organism evolution that is fundamentally different from the neo-Darwinian one. The incorporation of the semiotic perspective into general biology reveals that the evolution of living systems is based on the instantiation and inheritance of sign relations (many of which have become ‘habits’ in the Peircean sense). New sign relations result from learning and can be seen as exaptations (changes of function via changing one or more of the relata in a relation) in the work of pre-existing sign processes. The four broad processes of learning described in the section above all work as problem-solving mechanisms that can establish new sign relations (or habits, or rules). The description of evolutionary mechanisms that include epigenetic inheritance and the role of organisms’ activity is sometimes called ‘the extended theory of evolution’. According to this inherently (if not always explicitly realized) semiotic understanding, evolutionary innovation can and often does result from organisms’ choices. In the extended theory, in addition to natural selection, both biological drives and semiotic  fitting  play

46

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

an  evolutionary  role, suggesting that evolution can be based on needs and semiotic fittedness, and not only on reproductive success.7 In sum, the mechanisms of semiotic evolution include, and also reveal, a large number of inter-related processes and phenomena that require detailed (re)description and (re)explanation on a semiotic basis. We can thus foresee a rich period to come for semiotic evolutionary study.

Semiotic ecology An important insight that semiotics provides to ecology stems from the observation that most of both the interspecific and the intraspecific bonds that establish the ecological network are based on recognition processes. This means that organisms’ capacity for interpretation is the necessary condition for the formation and persistence of biological communities. A semiotic approach in ecology means a description or study that pays attention to: 1. The distinctions that organisms themselves make and the ways how the organisms themselves perceive the world, i.e. the study of umwelt or organic categorization; 2. The intentionality of organisms’ behavior,8 the role and types of organic drives (needs), and the changes resulting from individual organisms’ searching, learning, adaptation (fitting) and habituation; 3. Communication and its role at all levels of living systems; the formation of organic forms as communicative structures; 4. The production of ecosystems as the result of emergent organic design by the multiple organisms living in its communities; and 5. The types of sign processes as they differ and vary in the processes resulting in the production and reduction of diversity. Thus, according to the semiotic view, an ecosystem is a network of organisms' umwelten capable of designing and preserving functional bonds, together keeping the element cycles running with a tendency to balance the respective flows.

PRIORITIES Semiotics has not yet been properly integrated into general biology, and most of the work of doing so lies ahead. But doing so is not merely a matter of applying the pre-existing semiotic models to the pre-existing questions and problems of biology. Instead, the problems in contemporary semiotic biology that require proper analysis are not limited to just those of biology, but also concern the development of semiotic theory itself. Below we list the focuses of some contemporary debates, or rather problems waiting, where more fine-grained semiotic analysis is needed.

Upgrading current models of semiosis and interpretation The current models of semiosis existing in semiotics are not properly prepared to be effectively incorporated into biology. It seems obvious that an elaborated operational model of semiosis can only be worked out with considerable help from biosemiotics. One may notice that some of the definitions of semiosis used above differ slightly from those, for instance, of Charles Sanders Peirce. Indeed, contemporary biosemiotics

SEMIOTICS IN GENERAL BIOLOGY

47

uses many concepts, terms and ideas of Peirce, but deliberately does not hew precisely to his model. For while Peirce wrote about signs primarily in the context of logic, he did not write about signs primarily in the context of biological being per se. Yet when asking about life qua life, our question is not only about logic, but more radically, about existence. Logic does not care about time, space and matter, while existence cannot do without these. Temporal aspects of existence, as well as material particularity and spatial relationship, are particularly important for the organization and interaction needed to bring about even the simplest forms of life. Similarly, interpretation, the crux of semiosis, is, for Peirce, a logical operation. In life, however, it is more – it takes time, it should be embodied, it is situational and metabolic. If we do not notice this difference, then we could easily reach the conclusion – given that logic is computation – that there is computation and only computation all the way down. Because interpretation, from the point of view of logic, can be described as an algorithm, and everything algorithmic can be executed as computation. Seen thusly, the lower semiotic threshold, as well as the reality of organismic ‘choice’ and possibility, would disappear. When we adopt a biological approach, instead, both will become undeniably apparent again.

Qualifying the lower threshold of life Related to the question of the lower semiotic threshold, a question that is discussed over and over again in biology is that of establishing the lower threshold of life. For instance, the structure and functioning of viruses are rather well researched and understood. But whether viruses are living or non-living beings lacks a final answer, not because we do not know what viruses are, but – more importantly – because no general agreement has yet been reached within biology as to how most properly to define ‘life’. Various candidate definitions listing characteristics of ‘what it is to be alive’ are in use – but as yet there is no one final agreement among scientists regarding exactly in what the phenomenon of ‘life’ per se consists. Here, we believe that a semiotic perspective can provide some help. According to an intuition shared by many biologists, life includes certain meaning-making elements (i.e., triadic relations), and this intuition may well contribute to a definition of life. For if semiosis is at the root of any meaning-making process, then we could define life as any process requiring semiosis for its realization. In this case, Sebeok’s dictum that life and semiosis are co-extensive would be true by definition, and it would only require a very precise operational understanding of semiosis. This, as we can see, has not yet really been achieved. We have noted earlier that semiosis is the process of acquiring knowledge, in that semiosis manifests in two information-bearing activities: interpretation and habitformation. In other words, semiosis underlies choice, and it underlies the introduction of rules. This accepted, we can use an existing detailed description of, e.g., intracellular processes, and examine in detail whether or not precisely this process of semiosis can be found there. For example, there is the metabolic network, which requires energy and material, and which is therefore related to the processes of respiration and nutrition. There is also catalysis, feedback and self-organization in the organization and maintenance of living being. All of that is necessary for life, one might ask, but is it also per se semiosis? Here again, the semiotic perspective may aid general biology: For under this understanding, we see that interpretation assumes indeterminacy. For a process to be interpretative, it should have the freedom to occur in alternative ways. Interpretation

48

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

therefore includes at least a primitive possibility for choice and decision-making, a primitive form of problem-solving. But do such things take place in the cell? While still lacking an exhaustively rigorous definition of semiosis of some specificity, we tend to hypothesize that, yes, semiosis may happen in the cell. And if so, then, a further question becomes that of determining what exactly defines a minimal semiotic unit in a cell? Obviously, it will need to include much more than simply the strain of nucleic acid or the cell membrane per se. Nevertheless, an important and interesting problem of the relationship between interpretation and habit appears here. Interpretation is a unique decision. Habit is a decision that has little or no uniqueness, a process that is generally the same as time goes on. Interpretation, even in the cell, would use one of the several simultaneously available options of doing something (e.g. to regulate a process) together with leaving a temporary trace of this choice – i.e. the production of an interpretant. If this trace is influencing further behaviour when repeated in similar situations, it has already formed a habit. Indeed, the majority of processes that we see in cells today are habit-based. A habit that neither changes nor can be modified has become a deterministic process of efficient causality, from which new information will not appear. Under such circumstances, there is no semiosis, regardless of whether or not there may have been in engendering the habit, a long time in the past. Moreover, habits can appear purely by chance, without the involvement of semiosis, and perhaps indeed some of the more persistent chemical reactions we see in all life may have had their early origin in this way. We hypothesize that habits are more often the products of semiosis, however, and seemingly so also even in the cell. It is quite probable that indeterminate situations in which a new kind of behaviour can occur are rather rare in the cell. If so, we should accept that semiosis in the cell is not an ever-ongoing process – but, rather, that semiosis takes place only once a certain threshold of anomalous conditions has manifested, works out a new habit or modifies an earlier one, and then falls dormant, or causally disappears, until a next such anomalous occurrence brings it forth again. Here a temporary, shortliving sign relation results in a steady, long-term habit which has replaced the need for, and eliminates the potential instability of, continual semiosis. According to contemporary knowledge in biology, scenarios like the above are sometimes possible, although it is not yet clear how often they occur. But if this account is sound, we can make an interesting conclusion about the nature of semiosis in the cell, that modifies the understanding of the ‘lower semiotic threshold’ as formulated by Umberto Eco. For as described: The lower threshold itself may not be sharp; instead, there is but a threshold zone. Accordingly, semiosis may manifest for a moment, and then disappear, while cells (the minimal organisms) carry on the habits – which are themselves the conditions for semiosis to reappear.

Reconsidering historical explanations in biological meaning-making The place of historical explanations in biological meaning-making is not clear. This often manifests as the debate between the diachronic versus the synchronic pictures of meaning-making, or between evolutionary versus ontogenetic sources of meaning, and takes the form of such questions as to whether or not ‘natural selection’ can, in any way, be responsible for creating meaning? A teleosemantic explanation (Millikan 1989) would answer positively, while acknowledging the difficulty in the problem of novel contents: i.e. how could ‘natural selection’ account for the ability to represent properties that are evolutionarily novel? (Garson, Papineau 2019). One possible solution is to accept that

SEMIOTICS IN GENERAL BIOLOGY

49

meaning-making, i.e. semiosis, only ever takes place in a short moment, given that it by necessity involves choice and the conditions for choice – i.e. a state of presently existing indeterminacy and unpredictability. If so, then only the conditions for meaning-making can be a matter of evolution, but the creation of meaning itself cannot, since meaning is always created anew, in the moment, irrespective of how similarly it has been done so previously, or how ‘persistent’ in this sense it may appear to be.

Responding to anti-representationalism There is a debate between representationism and anti-representationism that is apparently not yet resolved. The issue here takes several forms, the two most prominent being the need (or not need) of program code tokens and robotic action repertoires to be ‘representational’ of real-world objects for the successful development of Artificial Intelligence or Artificial Life, and the related issue in Cognitive Science and Philosophy of Mind as to whether the human mind itself works on a collection of ‘representations’ that are ‘interpreted’ by the neural version of an internal Cartesian homunculus. Anti-representationalists in both domains appeal instead to the power of emergent dynamics operating across agents’ bodies, actions and worlds (which includes the actions of other agents) in enabling successful negotiation of complex, adaptive, real-time experience without the need for ‘contentbearing’ representations and their internal ‘interpreters’ (see reviews in Chemero 2011; Gallagher 2017). The rise of anti-representationalist thinking across many domains of inquiry, including that of distributed cognition, enactivism, language study and anthropology, to name just a few, poses a direct and often overtly stated challenge to those who would advance a semiotic view of knowing and of interaction wherein terms like ‘interpretation’ and ‘representation’ are freely used, and where ‘sign relations’ do indeed have a ‘contentful’ aspect – though in all three cases, not in the way that the anti-representationalist misunderstand our use of these terms here. Indeed, an overarching criticism of antirepresentationalism that gets repeatedly made, even among its proponents, is that they ‘lack a proper operationalization of the notion of representation’ to begin with (Haselager et al. 2003). And here too, can the definition of ‘interpretation’ that we have been presenting in this contribution help to clarify the points of agreement and disagreement in the semiotic versus the non-semiotic understandings of what a ‘representation’ is. For as was made clear in the above discussions, ‘interpretation’ is a bodily response in the presence of indeterminacy or mutual incompatibility, and the action itself – the ‘choice’ enacted, as it were – often sets the conditions upon which the next act of semiosis will have to occur. It is precisely this enactively produced ‘trace’ or ‘interpretant’ change in the system that constitutes the ‘content’ that is ‘re-presented’ in subsequent acts of interpretation that have become habitualized. At its base, then, it is a life process far removed from the caricature of ‘representation’ painted by the anti-representationalists as that of an ‘internal little man’ contemplating ‘pictures in the mind’. But here, too, is a place where much work needs to be done in clarifying these ideas for use within a semiotic biology.

A more testable semiotic biology Another important and as yet not well-resolved problem in developing a semiotic general biology concerns the testability and development of methodologies for testing semiotic hypotheses in biology. Steps towards experimental biosemiotics and quantitative

50

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

biosemiotics, however, are on the way (Galantucci, Garrod 2012; Compagno 2018). The development of a more testable semiotic biology goes hand in hand with the development of fresh innovative thinking about the appropriate methodological work needed, and understanding that the methods that may be needed and most fruitful here will have to be different and more complex than the methods that have proven most useful in the sciences of the non-living. The detailed semiotic study of the sign-processes and umwelten of many species, likewise, lies ahead.

Cross-disciplinary outreach and growth Finally, there exist many works which deal with biosemiotic problems without using the rich traditions of semiotics as part of their conceptual apparatus. These are works on semiotic problems in biology that themselves do not use semiotic theory, models or concepts, usually focusing on the emergence of consciousness and knowing, either in evolution (Godfrey-Smith 2016; Ginsburg, Jablonka 2019; Koch 2019), or as a contemporary problem in neuroscience (Edelman 1989; Damasio 1999; Ramachandran, Blakeslee 1999), and often both (e.g. Fuster 2003, who gets the closest by explicitly drawing on Uexküll’s model of the Funktionskreis in developing his notion of the ‘cognitive unit’ or ‘cognit’ joining perception and action). With Sebeok, we feel that many of these biologists and researchers are working along similar paths as ours, without yet fully grasping how much nascent semiotics that they are already doing – and, with Sebeok, we feel that their own work will only grow in richness when they eventually do. Thus, one of the outstanding priorities for the further development of semiotic biology is cross-disciplinary outreach and growth.

CONCLUSION The incorporation of semiotics into biology has taken place as part of a larger set of changes in biology, such as the move towards more dynamical systems and processual approaches, the epigenetic turn, 4E (embodied, embedded, enacted and extended) cognition, and the development of a post-Darwinian theory of evolution, all the while piece by piece rebuilding theoretical biology upon a more secure and explicitly semiotic footing. But the development of a semiotic general biology is seemingly just as, if not even more, important for semiotics, as it offers new, foundational (re)definitions and (re)analyses of interpretation, choice, categorization, meaning, representation, habit, learning, evolution and translation as they manifest in their original and most basal forms in the relational processes of living being, which we have sketched out in abbreviated form throughout this chapter, and which we summarize below as follows: The conditions for Choice are the contexts of indeterminacy or mutual incompatibility. An indeterminate situation appears when an agent is simultaneously presented, via signs, with more than one way to act in a particular situation, and is capable of doing either (this capability can be called ‘multivitality’). Semiosis, meaning-making, manifests at this moment, to the extent that the agent’s actions will be neither random nor deterministic. Such actions will be Interpretations in that, sensu Peirce, they are called forth by circumstances (prospects) appearing to the agent in the form of signs (distinctions), towards which they generate interpretants (responses) that themselves can be a source of some new indeterminacy necessary to call forth (i.e. manifest) a subsequent act of semiosis (more developed sign).

SEMIOTICS IN GENERAL BIOLOGY

51

We see, then, that Interpretation is sign-based action taken within the context of indeterminacy. Understanding this allows us to make an important and interesting clarification in the relationship between interpretation and habit. Interpretation is a unique decision. Habit is a decision that has little or no uniqueness, a process that is the generally same as time goes on. Interpretation, even in the cell, would use one of the several simultaneously available options of doing something (e.g. to regulate a process) together with leaving a temporary trace of this choice – i.e. the production of an interpretant. If this trace is influencing further behaviour when repeated in similar situations, it has already formed a Habit. As such, the particular representamen-object-interpretant triadic relationship of this semiotically produced ‘trace’ (change in the system) made by the interpretation constitutes the ‘content’ (meaning) that is ‘re-presented’ in subsequent acts of interpretation that have become habitualized. Representation here refers to not static noun-like entities within the system, but to those particular representamen-objectinterpretant triadic relationships that have been manifested before and may be appropriate for manifesting again when faced with a situation of similar indeterminacy. Learning is the establishment of new information, via interpretation, and corresponding to four types of semiosis and signs: imprinting (icon making), conditioning (index making), imitation (emon making) and instruction (symbol making) where each higher type of learning necessarily includes all of the lower ones. Both Translation (‘evaluative interpretation’ across sign systems) and Categorization (based on pairwise recognition in the sense of what Wittgenstein calls ‘family resemblance’) are the results of learning, and take place in all living systems, both within and between organisms. Finally, the recognition that the activity of the organism is based on semiosis provides a model for organismic Evolution that is complementary to the neo-Darwinian one. The incorporation of the semiotic perspective into general biology reveals that the evolution of living systems is based on the instantiation and inheritance of sign relations. These new sign relations result from learning and can be seen as exaptations (changes of function via changing one or more of the relata in a relation) in the work of pre-existing sign processes. The four broad processes of learning described above all work as problem-solving mechanisms that can establish new sign relations (or habits, or rules). In this way semiotic evolutionary innovation can and often does result from organisms’ choices, suggesting that evolution can be based on needs and semiotic fittedness, and not only on reproductive success. As shown above, the development of a more semiotic general biology stands to benefit the work of both semioticians and biologists, who, working collaboratively towards a more integrated perspective, only together can bring it about.

ACKNOWLEDGEMENT This research was supported by National University of Singapore grant number AcRF R-124-000-114-115 and by the Estonian Research Council grant PRG314.

NOTES 1 The emphasis on interpretive behaviour rather than on signs means that the concept of sign is often understood too narrowly. The emphasis on interpretation rather than signs also means that ‘biosemiotics’ and ‘biohermeneutics’ can be treated as almost synonymous. 2 Kull 2022 has since discovered an early usage of the term ‘biosemiotics’ (Biosemiotik) that dates back to the 1850s, in the writings of Austrian chemist Vincenz Kletzinsky (1826–1882). 3 See also Abel 2009.

52

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

4 Including the establishment of the International Society for Biosemiotic Studies (Favareau 2005; https://www.biosemiotics.org). 5 ‘Einstein’s reply when asked if everything, in theory, could eventually be reduced to a correct physical description, was: “Yes, that is conceivable, but it would be of no use. It would be a picture with inadequate means, just as if a Beethoven symphony were presented as a graph of air pressure”’ (Pattee 2005: 292). See also Kauffman 2012. 6 See also Cangelosi 2005. 7 See the special issue of Biosemiotics on evolution (Sharov et al. 2016; Hoffmeyer, Stjernfelt 2016). 8 Or, ententionality (Deacon 2012). See also Hoffmeyer 2012.

REFERENCES Abel, D. L. (2009), ‘The Biosemiosis of Prescriptive Information’, Semiotica, 174 (1/4): 1–19. Anastasi, A. (2017), ‘Biology, Learning, and Evolution of Vocality: Biosemiotics of Birdsong’, Cognitive Semiotics, 10 (1): 19–39. Auletta, G. (2011), Cognitive Biology: Dealing with Information from Bacteria to Minds, Oxford: Oxford University Press. Barbieri, M. (1985), The Semantic Theory of Evolution, Chur: Harwood Academic Publishers. Barbieri, M. (2001), The Organic Codes: The Birth of Semantic Biology, Ancona: Pequod. Barbieri, M., ed. (2007), Introduction to Biosemiotics: The New Biological Synthesis, Berlin: Springer. Bateson, G. (1973), Steps to an Ecology of Mind: Collected Essays in Anthropology, Psychiatry, Evolution and Epistemology, London: Paladine. Brentari, C. (2015), Jakob von Uexküll: The Discovery of the Umwelt between Biosemiotics and Theoretical Biology (Biosemiotics 9), Cham: Springer. Brier, S. (2008), ‘The Paradigm of Peircean Biosemiotics’, Signs, 2: 30–81. Bruni, L. E. (2008), ‘Hierarchical Categorical Perception in Sensing and Cognitive Processes’, Biosemiotics, 1 (1): 113–30. Cangelosi, A. (2005), ‘Approaches to Grounding Symbols in Perceptual and Sensorimotor Categories’, in H. Cohen and C. Lefebvre (eds), Handbook of Categorization in Cognitive Science, 719–37, Amsterdam: Elsevier. Chemero, A. (2011), Radical Embodied Cognitive Science, Cambridge: MIT Press. Compagno, D., ed. (2018), Quantitative Semiotic Analysis (Lecture Notes in Morphogenesis), Cham: Springer. Damasio, A. (1999), The Feeling of What Happens: Body and Emotion in the Making of Consciousness, Boston: Houghton Mifflin Harcourt. Deacon, T. (1997), The Symbolic Species: The Co-Evolution of Language and the Brain, New York: Norton. Deacon, T. (2012), Incomplete Nature: How Mind Emerged from Matter, New York: W. W. Norton. Deacon, T. (2015), ‘Steps to a Science of Biosemiotics’, Green Letters: Studies in Ecocriticism, 19 (3): 293–311. Deely, J. (2001), Four Ages of Understanding: The First Postmodern Survey of Philosophy from Ancient Times to the Turn of the Twenty-first Century, Toronto: University of Toronto Press. Edelman, G. (1989), The Remembered Present: A Biological Theory of Consciousness, New York: Basic Books.

SEMIOTICS IN GENERAL BIOLOGY

53

El-Hani, C. N., J. Queiroz, and C. Emmeche (2009), Genes, Information, and Semiosis (Tartu Semiotics Library 8), Tartu: Tartu University Press. Emmeche, C. (2002), ‘The Chicken and the Orphean Egg: On the Function of Meaning and the Meaning of Function’, Sign Systems Studies, 30 (1): 15–32. Emmeche, C. and K. Kull, eds (2011), Towards a Semiotic Biology: Life Is the Action of Signs, London: Imperial College Press. Farina, A. (2012), ‘A Biosemiotic Perspective of the Resource Criterion: Toward a General Theory of Resources’, Biosemiotics, 5 (1): 17–32. Favareau, D. (2005), ‘Founding a World Biosemiotics Institution: The International Society for Biosemiotic Studies’, Sign Systems Studies, 33 (2): 481–5. Favareau, D., ed. (2010), Essential Readings in Biosemiotics: Anthology and Commentary (Biosemiotics 3), Berlin: Springer. Favareau, D., P. Cobley, and K. Kull (2012), A More Developed Sign: Interpreting the Work of Jesper Hoffmeyer (Tartu Semiotics Library 10), Tartu: University of Tartu Press. Florkin, M. (1974), ‘Concepts of Molecular Biosemiotics and of Molecular Evolution’, Comprehensive Biochemistry, 29A: 1–124. Fuster, J. M. (2003), Cortex and Mind: Unifying Cognition, Oxford: Oxford University Press. Galantucci, B. and S. Garrod, eds (2012), Experimental Semiotics: Studies on the Emergence and Evolution of Human Communication, Amsterdam: John Benjamins. Gallagher, S. (2017), Enactivist Interventions: Rethinking the Mind, Oxford: Oxford University Press. Gare, A. (2019), ‘Biosemiosis and Causation: Defending Biosemiotics through Rosen’s Theoretical Biology; or, Integrating Biosemiotics and Anticipatory Systems Theory’, Cosmos and History, 15 (1): 31–90. Garson, J. and D. Papineau (2019), ‘Teleosemantics, Selection and Novel Contents’, Biology and Philosophy, 34 (36): 1–20. Ginsburg, S. and E. Jablonka (2019), The Evolution of the Sensitive Soul: Learning and the Origins of Consciousness, Cambridge: The MIT Press. Godfrey-Smith, P. (2016), Other Minds: The Octopus, the Sea, and the Deep Origins of Consciousness, New York: Farrar, Straus and Giroux. Goodwin, B. C. (1972), ‘Biology and Meaning’, in C. H. Waddington (ed.), Towards a Theoretical Biology. Volume 4: Essays, 259–75, Edinburgh: Edinburgh University Press. Haselager, P., A. D. Groot, and H. V. Rappard (2003), ‘Representationalism vs. Antirepresentationalism: A Debate for the Sake of Appearance’, Philosophical Psychology, 16 (1): 5–23. Hawkins, R. D. and J. H. Byrne (2015), ‘Associative Learning in Invertebrates’, Cold Spring Harbor Perspectives in Biology, 7 (5): a021709 (1–17). Henning, B. G. and A. C. Scarfe, eds (2013), Beyond Mechanism: Putting Life Back into Biology, Lanham: Lexington Books. Ho, M. W. (1989), ‘A Structuralist of Process: Towards a Post-Darwinian Rational Morphology’, in B. C. Goodwin, A. Sibatani, and G. C. Webster (eds), Dynamic Structures in Biology, 31–48, Edinburgh: Edinburgh University Press. Hoffmeyer, J. (1996), Signs of Meaning in the Universe, Bloomington: Indiana University Press. Hoffmeyer, J. (2008), Biosemiotics: An Examination into the Signs of Life and the Life of Signs, Scranton: Scranton University Press. Hoffmeyer, J. (2012), ‘The Natural History of Intentionality: A Biosemiotic Approach’, in T. Schilhab, F. Stjernfelt, and T. Deacon (eds), The Symbolic Species Evolved, 97–116, Dordrecht: Springer.

54

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Hoffmeyer, J. and F. Stjernfelt (2016), ‘The Great Chain of Semiosis: Investigating the Steps in the Evolution of Semiotic Competence’, Biosemiotics, 9 (1): 7–29. Ingensiep, H. W. (2001), Geschichte der Pflanzenseele: Philosophische und biologische Entwürfe von der Antike bis zur Gegenwart, Stuttgart: Alfred Kröner Verlag. Jakobson, R. and L. R. Waugh (1987), The Sound Shape of Language, 2nd edn, Berlin: De Gruyter. Jander, R. (1981), ‘General Semiotics and Biosemiotics’, in R. T. De George (ed.), Semiotic Themes, 225–50, Lawrence: Lawrence University of Kansas Publications. Karatay, V., Y. Denizhan, and M. Ozansoy (2016), ‘Semiosis as Individuation: Integration of Multiple Orders of Magnitude’, Biosemiotics, 9 (3): 417–33. Kauffman, S. (1996), At Home in the Universe: The Search for the Laws of Self-Organization and Complexity, Oxford: Oxford University Press. Kauffman, S. (2012), ‘From Physics to Semiotics’, in S. Rattasepp and T. Bennett (eds), Gatherings in Biosemiotics (Tartu Semiotics Library 11), 30–46, Tartu: Tartu University Press. Koch, C. (2019), The Feeling of Life Itself: Why Consciousness Is Widespread but Can’t Be Computed, Cambridge: The MIT Press. Kleisner, K. (2008), ‘The Semantic Morphology of Adolf Portmann: A Starting Point for the Biosemiotics of Organic Form?’, Biosemiotics, 1 (2): 207–19. Krampen, M. (1981), ‘Phytosemiotics’, Semiotica, 36 (3–4): 187–209. Kull, K. (1999), ‘Biosemiotics in the Twentieth Century: A View from Biology’, Semiotica, 127 (1–4): 385–414. Kull, K., ed. (2001), Jakob von Uexküll: A Paradigm for Biology and Semiotics. Semiotica, 134 (1–4), Berlin: Mouton de Gruyter. Kull, K. (2018). ‘On the Logic of Animal Umwelten: The Animal Subjective Present and Zoosemiotics of Choice and Learning’, in G. Marrone and D. Mangano (eds), Semiotics of Animals in Culture: Zoosemiotics 2.0 (Biosemiotics 17), 135–48, Cham: Springer. Kull, K. (2020), ‘Codes: Necessary, but Not Sufficient for Meaning-making’, Constructivist Foundations, 15 (2): 137–9. Kull, K., T. Deacon, C. Emmeche, J. Hoffmeyer, and F. Stjernfelt (2009), ‘Theses on Biosemiotics: Prolegomena to a Theoretical Biology’, Biological Theory: Integrating Development, Evolution, and Cognition, 4 (2): 167–73. Kull, K. (2022), “The Term ‘Biosemiotik’ in the 19th Century”, Sign Systems Studies, 50 (1): 173–178. Marais, K. (2019), A (Bio)Semiotic Theory of Translation: The Emergence of Social-Cultural Reality (Routledge Advances in Translation and Interpreting Studies), New York: Routledge. Maran, T., D. Martinelli, and A. Turovski, eds (2011), Readings in Zoosemiotics (Semiotics, Communication and Cognition 8), Berlin: De Gruyter Mouton. Markoš, A. (2002), Readers of the Book of Life, New York: Oxford University Press. Markoš, A. (2010), ‘Biosemiotics and the Collision of Modernism with Postmodernity’, Cognitio, 11 (1): 69–78. Marler, P. (1961), ‘The Logical Analysis of Animal Communication’, Journal of Theoretical Biology, 1: 295–317. [Reprinted in Maran et al. 2011: 253–77.] Martinelli, D. (2010), A Critical Companion to Zoosemiotics: People, Paths, Ideas (Biosemiotics 5), Berlin: Springer. Michelini, F. and K. Köchy, eds (2020), Jakob von Uexküll and Philosophy: Life, Environments, Anthropology (History and Philosophy of Biology), London: Routledge. Millikan, R. G. (1989), ‘Biosemantics’, Journal of Philosophy, 86: 281–97.

SEMIOTICS IN GENERAL BIOLOGY

55

Morris, C. W. ([1925] 1993), Symbolism and Reality: A Study in the Nature of Mind, PhD diss., Chicago: University of Chicago. Morris, C. W. ([1946] 1955), Signs, Language and Behavior, New York: George Braziller. Pap, L. (1979), ‘On the Scope of Semiotics: A Critique and Redefinition’, in S. Chatman, U. Eco, and J. M. Klinkenberg (eds), A Semiotic Landscape: Proceedings of the First Congress of the International Association for Semiotic Studies, Milan, June 1974, 336–40, The Hague: Mouton Publishers. Pattee, H. H. (1972), ‘Laws and Constraints, Symbols and Languages’, in C. H. Waddington (ed.), Towards a Theoretical Biology 4: Essays, 248–58, Edinburgh: Edinburgh University Press. Pattee, H. H. (2005), ‘The Physics and Metaphysics of Biosemiotics’, Journal of Biosemiotics, 1: 281–301. Petrilli, S. (1999), ‘Charles Morris’s Biosemiotics’, Semiotica, 127 (1–4): 67–102. Prodi, G. (1977), Le basi materiali della significazione (Nuovi saggi italiani 21), Milano: Bompiani. Prodi, G. ([1977] 2021), The Material Basis of Meaning (Tartu Semiotics Library 22), Tartu: University of Tartu Press. Prodi, G. (1988), ‘Signs and Codes in Immunology’, in E. Sercarz, F. Celada, N. A. Michison, and T. Tada, (eds), The Semiotics of Cellular Communication in the Immune System: Proceedings of the NATO Advanced Research Workshop on the Semiotics of Cellular Communication in the Immune System, 53–64, Berlin: Springer. Ramachandran, V. S. and S. Blakeslee (1999), Phantoms in the Brain: Human Nature and the Architecture of the Mind, New York: William Morrow. Rattasepp, S. and T. Bennett, eds (2012), Gatherings in Biosemiotics (Tartu Semiotics Library 11), Tartu: Tartu University Press. Rodríguez Higuera, C. J. (2019), ‘Everything Seems so Settled Here: The Conceivability of Post-Peircean Biosemiotics’, Sign Systems Studies, 47 (3–4): 420–35. Rosen, R. (1991), Life Itself: A Comprehensive Inquiry into the Nature, Origin, and Fabrication of Life, New York: Columbia University Press. Rothschild, F. S. (1962), ‘Laws of Symbolic Mediation in the Dynamics of Self and Personality’, Annals of New York Academy of Sciences, 96: 774–84. Rothschild, F. S. (1968), ‘Concepts and Methods of Biosemiotic’, Scripta Hierosolymitana, 20: 163–94. Sebeok, T. A. (1972), Perspectives in Zoosemiotics, The Hague: Mouton. Sebeok, T. A. (2001), Signs: An Introduction to Semiotics, 2nd edn, Toronto: University of Toronto Press. Sebeok, T. A. and J. Umiker-Sebeok, eds (1992), Biosemiotics: The Semiotic Web 1991, Berlin: Mouton de Gruyter. Sermonti, G. (1999), Dimenticare Darwin: Ombre Sull’evoluzione, Santarcangelo: Rusconi. Sharov, A. A. (2013), ‘Minimal Mind’, in L. Swan (ed.), Origins of Mind (Biosemiotics 8), New York: Springer, 343–60. Sharov, A., T. Maran, and M. Tønnessen (2016), ‘Comprehending the Semiosis of Evolution’, Biosemiotics, 9 (1): 1–6. Stjernfelt, F. (2014), Natural Propositions: The Actuality of Peirce’s Doctrine of Dicisigns, Boston: Docent Press. Tembrock, G. (1971), Biokommunikation: Informationsübertragung im biologischen Bereich, Teil 1, 2, Berlin: Akademie-Verlag. Thom, R. (1980), ‘L’espace et les signes’, Semiotica, 29 (3/4): 193–208.

56

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Tommasi, L., M. A. Peterson, and L. Nadel, eds (2009), Cognitive Biology: Evolutionary and Developmental Perspectives on Mind, Brain, and Behavior (Vienna Series in Theoretical Biology 11), Cambridge: The MIT Press. Tønnessen, M., R. Magnus, and C. Brentari (2016), ‘The Biosemiotic Glossary Project: Umwelt’, Biosemiotics, 9 (1): 129−49. Turovski, A. (2011), ‘The Need for Impression in the Semiotics of Animal Freedom: A Zoologist’s Attempt to Perceive the Semiotic Aim of H. Hediger’, in C. Emmeche and K. Kull (eds), Towards a Semiotic Biology: Life Is the Action of Signs, 133–41, London: Imperial College Press. Uexküll, J. v. (1928), Theoretische Biologie (2te gänzlich neu bearbeitete Auflage), Berlin: Verlag von Julius Springer. Uexküll, J. v. (1940), Bedeutungslehre (Bios: Abhandlungen zur theoretischen Biologie und ihrer Geschichte sowie zur Philosophie der organischen Naturwissenschaften 10), Leipzig: Verlag von J. A. Barth. Uexküll, T. v. (1981), ‘Die Zeichenlehre Jakob von Uexkülls’, in M. Krampen, K. Oehler, R. Posner, and T. V. Uexküll (eds), Die Welt als Zeichen: Klassiker der modemen Semiotik. Berlin: Severin und Siedler, 233–79. [English version: Uexküll, T. v. 1987, ‘The sign theory of Jakob von Uexküll’, in M. Krampen, K. Oehler, R. Posner and T. v. Uexküll (eds), Classics of Semiotics, New York: Plenum Press, 147–79.] Vehkavaara, T. (2003), ‘Natural Self-interest, Interactive Representation, and the Emergence of Objects and Umwelt: An Outline of Basic Semiotic Concepts for Biosemiotics’, Sign Systems Studies, 31 (2): 547–87. Waddington, C. H. (1972), ‘Epilogue’, in C. H. Waddington (ed.), Towards a Theoretical Biology 4: Essays, 283–9, Edinburgh: Edinburgh University Press. Witzany, G., ed. (2014), Biocommunication of Animals, Dordrecht: Springer. Zlatev, J. (2002), ‘Meaning = Life (+ Culture): An Outline of a Unified Biocultural Theory of Meaning’, Evolution of Communication, 4 (2): 253–96.

CHAPTER THREE

Semiotics in Ecology and Environmental Studies TIMO MARAN

INTRODUCTION There are fundamental similarities and connections between ecology and semiotics. Both disciplines derive from the same episteme of systemic or structural thinking established in the late nineteenth and early twentieth centuries. Both classic ecology and semiotics (semiology) have also viewed their research objects as possessed of structural organization. In 1935, Arthur G. Tansley proposed the concept of the ecosystem as the structural and functional system of organisms and the environment they inhabit. In a similar way, early semiotics saw language (but also mythologies and literature) as intrinsically organized and composed of elements and their relations (e.g. in the semiology of Ferdinand de Saussure and the structural anthropology of Claude Lévi-Strauss). Both fields have given a principal ontological position to relations and go to great lengths to study relations. In ecology, ecological relations such as predation, parasitism, competition, herbivory and so on, have been the principal entities of research. Meanwhile, in Peircean semiotics, a main object of study – the sign – is understood as a mediated relation. There are also many examples wherein semiotic theories and concepts have been included within ecology and environmental studies. The adoption of semiotics within ecology has aided in foregrounding information and communication processes in nature, and also in articulating the relations between human culture and ecosystems. This chapter provides an overview of semiotics in population, community and ecosystem ecology, and also overviews various usages of semiotics in environmental studies, as well as describes the ecosemiotic paradigm as an explicit synthesis of ecology and semiotics. In the second half of the twentieth century, the influence of ecology and other biosciences on the humanities became noticeable. This movement led to the development of various novel paradigms (e.g. media ecology, cultural ecology) that adopted ecological concepts (environment, ecosystem, symbiosis), and also led to the rise of interest towards environmental issues as research objects (e.g. in ecocriticism and environmental history). In semiotics, the introduction of the concept of the semiosphere by Juri Lotman (2005) as a sphere of sign processes in a loose metaphoric relation with the biosphere, and the adoption of the Umwelt concept originally coined by Jakob von Uexküll (1982) in biosemiotics to denote species-specific perceptual worlds, are some markers of this development. Furthermore, there have been sub-disciplines in semiotics that have come  close to the

58

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

subject matter of ecology because of their interest in spaces, spatial relations and artefacts (e.g. urban semiotics, Krampen 1979). The interrelations between the biosciences and the humanities culminated in the development of the environmental humanities at the beginning of the twenty-first century, which as of now forms a contemporary context for interpreting the relations holding between semiotics and ecology. Besides these general developments, in the second half of the twentieth century specific paradigms of zoo-, eco- and biosemiotics emerged with the aim of bridging different fields of biology and semiotics, and with an ambition towards interdisciplinary syntheses. With its longer history, biosemiotics has been especially active in making many semiotic concepts accessible to ecology. It is also worth noting that several people working in the field (e.g. Kalevi Kull, Almo Farina, Riin Magnus) have professional involvement in both semiotics and ecology. For the most part, the historical influences between ecology and semiotics appear to be unidirectional – ecological concepts have been adopted by different paradigms of the humanities, but very seldom do we find cases wherein semiotic concepts and methods have been used in ecology (although the number of such works has been growing in recent decades). So what, in principle, could the role of semiotics be in ecology and environmental studies, and why should we aspire to such a synthesis? Based on a review of the literature, the following main motivations for incorporating the semiotic approach into ecology can be distinguished: 1. Including animal agency in the understanding of ecological processes. The question of how animals perceive, select and modify their environments is relevant for a number of ecological topics, such as protecting habitats for endangered species or controlling pest damage in medicine, agriculture or forestry (e.g. Shaw et al. 2013). Including the animal perspective is mostly achieved by applying Uexküll’s Umwelt theory or its elaborations (such as the landscape of fear, Bleisher 2017, or environmental continua, Manning et al. 2004). 2. Including human communicative and cultural processes in the subject matter of ecology. This is especially relevant for topics where environmental processes depend on human culture or behaviour. For instance, studies of urban ecology or semi-natural ecosystems would remain incomplete without the inclusion of the human cultural component, due to the effect that humans have in shaping these environments (e.g. Hess-Lüttich 2016). 3. Bridging the sciences and the humanities for the purpose of enriching the theory, conceptions and methods of ecology, or for envisioning the synthesis of the ecologies and the humanities into one sphere of knowledge. This approach is often related to the process of proposing new concepts for ecology (e.g. propagating the informational, cybernetic or cognitive approach, Farina et al. 2005; Farina and Pieretti 2013), or for building new methodological frameworks (as in ecosemiotics). Upon examining how semiotics has been conceptually included in ecological studies, there are several methods of concept-building which can be distinguished: existing ecological concepts are reinterpreted by adding some semiotic content (e.g. semethic interaction, semiotic niche, applying semiotic concepts directly within ecology; signs in the form of environmental signs, a code as an ecological code); original concepts that have derived

SEMIOTICS IN ECOLOGY AND ENVIRONMENTAL STUDIES

59

from the theoretical synthesis of semiotics and ecology (eco-field, sign-field) are proposed; or concepts which have a broader interdisciplinary usage and have been naturalized in both semiotics and ecology (Umwelt, affordance) are used. As it stands, all of these conceptual tools have been proposed by different authors, often one concept at time and based on diverse disciplinary insights. Integrating ecology and semiotics seems promising as a number of dedicated research methods have been provided and applied as part of their synthesis. For instance, Italian semiotician and landscape ecologist Almo Farina has developed the method of ecofield analysis that combines Uexküll’s Umwelt theory with the spatial description of landscapes and the allocation of resources. The ecofield is a meeting point of an animal’s biological needs and the properties and resources of the landscape. ‘The term eco-field is the contraction of the words “ecological field”, and means the physical (ecological) space and the associated abiotic and biotic characters that are perceived by a species when a functional trait is active’ (Farina and Belgrano 2004: 108). In addition, Farina proposes describing organism-environment relations as a Need-Function-Ecofield (or interface)Resource sequence (Farina 2012: 23) wherein functions and resources are mediated by a semiotic component – the ecofield – that animals need to perceive and interpret correctly in order to make use of resources. Ecofield analysis has been practically tested by Roberto Pizzolotto (2009) in a study of the distribution of Carabid beetles in various natural and anthropogenic habitats in Italy. He concludes that the eco-field is a valuable approach for developing tools to reveal natural trends, and that the study of life strategies as descriptors of an organism’s perception of the natural environment may lead to a practical application of the eco-field hypothesis […] ecofield is not merely an eco-mathematical model; its ecological dimensions result from the life histories and interactions of living organisms. The eco-field has been strictly related to species traits, which are one of its determining characteristics when species interfere with perceived ecological factors. (Pizzolotto 2009: 146) Anther more established method is Kalevi Kull’s distinction between 0-, 1-, 2- and 3-nature as different levels of environmental mediation in the nature-culture continuum: Zero nature is nature itself (e.g., absolute wilderness). First nature is the nature as we see, identify, describe and interpret it. Second nature is the nature which we have materially interpreted, this is materially translated nature, i.e. a changed nature, a produced nature. Third nature is a virtual nature, as it exists in art and science. (Kull 1998: 355) Kull’s typology is an effective conceptual tool for analysing semi-natural communities, hybrid natures and the interrelations between culture and nature. The typology has been applied in organizing the intertwining of culture and nature in sacred landscapes (Heinapuu 2016) and herbal medicine (Sõukand 2005). Other proposals for semiotically motivated research methods are the Naturesyns model (Møller 2009), Ecological Repertoire Analysis (Maran 2020) and Anxious Semiotics (Whitehouse 2015). The presence of several original research methods also signals the strength of the synthesis between ecology and semiotics.

60

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

POPULATION ECOLOGY AND SPECIES CONSERVATION In ecology, a division is often made between population ecology and ecosystem ecology, while in-between these major ecological schools we may locate community ecology. In population ecology, the research interests are in the demographics of different species, a species’ usage of resources and the relations they have with other species living in the same habitats (ecological relations). Population ecology also has a strong link with conservation biology, predicated on the question of what requirements are needed for the given species to thrive. In the study of animal populations, semiotics can be employed in various ways. First, (bio)semiotics can be used to explain the ecological role of animal morphology – what meanings an animal can achieve in ecological relations. Kalevi Kull has emphasized the role of organic form as a constituent of ecosystems: ‘production of ecosystem as the result of multiple “organic design” by the organisms living in the ecosystem’ (Kull 2008: 3213). Animal form has had a central focus in the Germanlanguage biological tradition, e.g., in the work of Adolf Portmann (animal appearance), and has more recently been elaborated in Czech biosemiotics under the concepts of semiotic co-option and semantic organs (Kleisner 2015). An important principle of this approach is understanding that animal form is semiotically open and able to gain new meanings, and that these can become engaged in new ecological relations. Several authors at the crossroads of ecology and semiotics have contemplated the semiotic mediatedness of organism-environment relations. Aside from Almo Farina’s ecofield concept (described above), Jesper Hoffmeyer (2008) has proposed the semiotic niche concept to emphasize the many properties and resources of the ecological niche which are presumed to be involved for the interpreting subject. For instance, in making use of an environment for nesting, a bird like the chaffinch needs to recognize a suitable location for nesting, secure the territory from rivals and build a nest from twigs, moss and other vegetative material (on the semiotic aspects of niche construction, see Peterson et al. 2018). Here an animal obtains an active role in creating a correspondence between its own genetic and bodily information and environmental information. In ecological studies, an animal’s individual connection to the environment is often discussed under the Umwelt concept (e.g. in landscape ecology, Manning et al. 2004; in sensory ecology, Jordan and Ryan 2015). With regard to the ecological relations holding between species, the semiotic approach is mostly related to the role of communication in ecological relations. It is generally accepted in ecology that the availability of information and communication may have quite a significant effect on ecological relations (Schmidt et al. 2010). Interspecies communication may have an effect on habitat selection, migratory routes, accessibility, food preferences and so on: The evidence is strong that interspecific information transfer influences the distributions of animals relative to each other, as evidenced by the number of studies […] Information flow between species can influence the position in space and time of different species, whether it be temporary groups around a predator or resource or stable associations between species in mixed-species groups or between species with shared territorial locations. (Goodale et al. 2010: 359)

SEMIOTICS IN ECOLOGY AND ENVIRONMENTAL STUDIES

61

The semiotic approach may shed light on these aspects in more specific ways, including how the particular messages, codes and media that are used in animal communication influence ecological and interspecies relations. The semiotic approach in ecology also helps to decipher how the physical environment acts as a medium in the message exchange between species. Russian ecologists and semioticians Elina Vladimirova and John Mozgovoy (2003, also Vladimirova 2009) have proposed signal field theory to describe the type and abundance of animal tracks and traces in a given area, and thereafter use the latter as the basis on which to analyse the diversity of meaning-relations in the environment. Sign field analysis has also enabled the characterization of the semiotic intensity of the environment, inasmuch as a number of functional classes of environmental objects provoke reaction on behalf of the animal. Focusing on the semiotic mediatedness of the animal-environment relationship is important for animal conservation and species protection, as it is in animal Umwelten wherein the environment becomes usable for other animals. Distinctions and categorizations between ‘structural habitat units (e.g. land cover types) as perceived by humans may not represent functional habitat units for other organisms’ (Van Dyck 2012: 144). The Umwelt approach allows for better scrutinizing the accessibility of environments for animals in the face of various anthropogenic effects like anthropogenic niche construction or human interference in information processing (Van Dyck 2012). It may also be that human-altered environments inhibit animals not because of the lack of some resource, but because of the deficient competencies of animals in recognizing their resources. For instance, if the abundance of prey species diminishes quickly, will a predator be capable of finding and developing a novel image of prey, as has been noticed in black-footed ferrets with regard to the declining prey populations of prairie dogs (Candland 2005)? The Umwelt-centred view will lead to different responses and countermeasures as regards environmental change. Besides protecting physical environments and land areas, attention needs to be on working with animal Umwelten in order to provide animals with necessary cues for orienting in the environment, or for altering animal Umwelt by training them to survive in novel environments (Van Dyck 2012; Shier 2016). In several works, Morten Tønnessen has elaborated on Umwelt theory to make it more suitable for analysing the changing relations between an animal and its environment (including concepts of Umwelt transition and Umwelt trajectory: see Tønnessen 2009, 2014). Umwelt theory may also be a valuable tool in more general strategies of nature protection as it makes it possible to describe and value the environment from the perspective of different non-human species (e.g. in regard to wolves, Tønnessen 2010; Drenthen 2016).

COMMUNITY ECOLOGY Community ecology focuses the combination, distribution and dynamics of species in local biological communities or ecosystems. Here the semiotic approach may be included to untangle the role that semiotic processes such as interspecies communication, animal cultures and ecological heritage, play in shaping the biological communities. Semiotics can further target specific communicational conventions that are used in biological communities. An example of such an approach can be found in Sánchez-García et al. (2017), wherein various relations of bark beetles in the forest ecosystem have been analysed using eco-field networks (representamen networks). According to this study,

62

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

relevant information is shared among forest species via the combination of various scents that together comprise the communication medium of odourtope. Kalevi Kull has proposed the concept of the consortium, which emphasizes the basic semiotic structure of biological communities. A consortium is a ‘group of organisms connected via (sign) relations, or groups of interspecific semiosic links in biocoenosis’ (Kull 2010: 347). It may be that in the evolution and development of the community structure, semiotic processes like habitat choice, recognition and learning play a major role. This is the idea behind the concept of ecological fitting wherein organisms themselves select and adjust their location and relations with the resources, species mates and other species within the ecosystem (Janzen 1985; Kull 2020). According to the ecological fitting hypothesis, species’ co-existence in biological communities does not result from the longterm co-evolution of the species, but results from more rapid processes of relational fitting. An example of such a fitting process can be habitat preference, which is based on the risk assessments that animals make concerning the presence of predators (such descriptions of the environment have been called landscapes of fear, Bleicher 2017). By looking for, and finding, a good spot to inhabit, individuals belonging to various species, are, in fact, creating the composition of the biological communities. Jesper Hoffmeyer (2008) has proposed describing ecological relations and food webs by foregrounding semiotic relations and by applying the ideas of semiotic habituation and symbolization. He has proposed the concept of semethic interaction to describe the way existing patterns, structures and routines tend to become sources of interpretation between species: ‘Whenever a regular behaviour or habit of an individual or species is interpreted as a sign by some other individuals (conspecific or alter-specific) and is reacted upon through the release of yet other regular behaviours or habits, we have a case of semethic interaction’ (Hoffmeyer 2008: 189). To give an example, the blossoms of daisies, dandelions and other plants transmit messages about the presence of nectar to flies, butterflies and other insects; the habits of pollinators to visit colourful plants is in turn used by the crab spiders Thomisidae, who lurk in blossoms; and the crab spiders’ habit of sitting in the blossoms is made use of by the parasitic mud dauber wasps Sceliphron sp., many species of which are specialized in catching spiders. Semethic interactions form cascades of linkages based on habits and the recognition occurring between different species in biological communities (e.g. in predator-prey networks). There are several approaches in ecology that aim to study the spatial organization of biological communities (e.g. landscape ecology, acoustic ecology). Spatial ecological analysis often uses large-scale modelling and applies it to big data sets. For instance, in acoustic ecology, hundreds of microphones can be simultaneously used to map the changing patterns of biophonies, geophonies and technophonies. Here semiotics can be included as a method of organizing and categorizing data. Farina and colleagues (2016) have proposed an Ecoacoustic Event Detection and Identification (EEDI) method wherein properties and meanings attributed to earlier events are combined with computerized analyses of datasets. The methods allow for analysing soundscapes in large land areas and in long-term monitoring programs. In addition to organismal activity and semethic interactions, we may also assume broader communicative conventions and codes arising in local communities. The idea of interspecies communicative conventions has been proposed by several authors and designated by many terms: acoustic codes (Malavasi et al. 2013), ecoacoustic codes (Farina 2018a), ecological codes (Kull 2010; Maran 2017a). Malavasi and colleagues (2013) argue that birds’ songs establish cross-species conventions (acoustic codes), and that these

SEMIOTICS IN ECOLOGY AND ENVIRONMENTAL STUDIES

63

conventions allow birds themselves to regulate their density in biological communities. By listening, adapting and tuning in to the morning chorus, individual birds receive information about the crowdedness of the habitat and location of unoccupied areas and resources. Maran (2017a) has proposed a broader interpretation of ecological codes as partially shared and distributed interspecies conventions wherein the code incorporates both environmental and communicational aspects, and wherein every participant uses a partial variation of a code. The further distinction in ecological codes can be made between: (1) distribution codes, where animal activities and communication through the process of self-assembly organize the spatial and temporal organization of the animals; (2) significational codes, where an environmental affordance is perceived and interpreted similarly by various species that results in shared or non-random use of the corresponding resource; (3) identity codes, where the ecological code is centred on a species or group that has significance or is charismatic to a broad number of species in the given ecological community; (4) symbolic codes, where the code is centred on the specific patterns of colour (or another modality) that have a shared meaning for a number of species; and (5) archetypical codes, where the ecological code is centred on the meaning relation that is valid for a broad number of different species due to the general physiological, ecological or behavioural constitution of the organisms. (Maran 2017a: 130–1) Examples of ecological codes include eyespots, yellow-black coloration, hissing and other warning signs. The presence of ecological codes opens up possibilities for complex interspecies regulation in ecosystems.

SYSTEM ECOLOGY System ecology studies the large-scale structure of ecosystems by paying attention to the distribution and movement of matter and energy (described as pools, flows, trophic levels, bioproduction, etc.). According to this objective, system ecology has developed a complex body of mathematical methods to model ecosystems. On this scale of generalization, the semiotic activity of organisms as well as the dynamics of populations is usually considered as a variation below statistical relevance. At the same time, it has been proposed that besides webs of energy and matter, ecosystems also consist of information networks. Strong positive feedbacks in information processing can define or reinforce levels of organization – from a cell to an individual to symbioses all the way to an ecosystem and the biosphere […]. Information stored at higher order levels of organization, such as social groups, communities or ecosystems, can be used by lower level systems, such as individual organisms and cells. In this way, information processing occurs across scales of space and time, and can create and maintain physical or energetic structures. (O’Connor et al. 2019) System ecologists Bernard C. Patten and Eugene Odum (1981) have argued that this informational layer consisting of an enormous number of local feedback cycles is the main reason why ecosystems retain their relatively stable structure and do not disperse into myriads of chaotic events. They further describe informational processes that allow for

64

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

the connection of different entities and layers of the ecosystem as mapping – processes which correspond to iconic representation in semiotic jargon – and amplification, in which a small causal trigger can have a major output effect on account of semiotic mediation. The informational layer of the ecosystem mostly comprises the activities of organisms which through homeostasis and self-preservation aim for stability at the local level. This property of living systems to retain their organization has been called coherence by Robert Ulanowicz (2010), who connects the latter with communication and biosemiosis. Through the feedback organismal regulation may cumulate in a more general system-wide regulation (lasting population densities or community structures). In some cases, however, also the non-living environment may become an agent carrying system-wide information (Lévêque 2003: 96). An example of such process would be the seasonal changes in coastal ecosystems, wherein the change of chemical composition (pH, DH, organic compounds) in water signals the beginning of the reproduction cycle for a variety of species. The semiotic regulatory layer of the ecosystem has been termed an information network (Patten and Odum 1981) or communication network (Lévêque 2003: 95). Danish system scientist Søren N. Nielsen (2007) has rightly recognized the semiotic character of this layer and proposed that it should be called semiotype in parallel to genotype and phenotype. Nielsen has argued for the inclusion of the semiotic approach in ecological modelling that would take into account the role of meaning-making (qualities) at different levels of ecological systems. Following Bernard C. Patten (1990), organismal meaning-making can be included within ecological models as input and output environs (formalized but phenomenological equivalents of Uexküll’s Merkwelt and Wirkwelt). Nielsen suggests that system ecology would benefit from including the understanding of second-order (cybernetic) systems, which are underdetermined, partly autonomous, ontically open and reactive (Nielsen 2016). Nielsen has further proposed that the particular approach could be named ecosystem semiotics and that ‘in order to understand the action of humans toward our environment and fellow/companion species on our planet, it is very important to have a further look on and improved understanding of the semiotic processes in the ecosystems’ (Nielsen 2007: 100). Semiotic regulation in the ecosystem is contextual and cumulative; it includes and combines patterns and perceivable properties of the inanimate environment with the perception, interpretation and behavioural action of single organisms together with their memory, experience and evolutionary past, as well as the communication networks in and between species. This makes the semiotic layer of the ecosystem very difficult to rationalize by conventional scientific methods. The numerous tiny acts of meaning-making organize and regulate the ecosystem in its every joint and connection, forming a complex multilayered network (Nielsen 2016). Describing these indirect regulating hubs in semiotic terms is notably present in the works of eminent system ecologist Bernard C. Patten. He has considered the ecosystem as a ‘model-making complex adaptive system’, wherein the internal model-making of living agencies together with physical resources and the forces of natural selection lead to the active auto-evolutionary self-design (Patten 1998: 151). In a more global, biospheric sense, the semiotic regulatory layer of the ecosystem can be described as the (bio)semiosphere, following Jesper Hoffmeyer: ‘the semiosphere is a sphere just like the atmosphere, the hydrosphere, and the biosphere. It penetrates to every corner of these other spheres, incorporating all forms of communication: sounds, smells, movements, colours, shapes, electrical fields, thermal radiation, waves of all kinds, chemical signals, touching, and so on. In short, signs of life’ (Hoffmeyer 1996: vii).

SEMIOTICS IN ECOLOGY AND ENVIRONMENTAL STUDIES

65

ENVIRONMENTAL STUDIES Environmental studies is a denotation for a broad interdisciplinary field that combines the approaches of the humanities, social sciences, environmental sciences and biology for studying human interactions with the environment. There are various possibilities for using semiotic theories in environmental studies. In this overview, I will cover approaches that focus on human communicational or cultural relations with the environment and that afford the environment with some realist or agential properties. I exclude numerous works wherein semiotic theories have been applied for analysing environmental representations as objects within media discourses (e.g. Douglas and Veríssimo 2013; Dobrin 2018), as well as poststructuralist criticism (mostly departing from the philosophies of Deleuze, Foucault, Baudrillard, but see Beever’s 2013 critical synthesis) of Western societies that occasionally include environmental topics. An adjunct field that has some overlap with semiotics is ecolinguistics as developed by Arran Stibbe (2012, 2015) and colleagues to critically scrutinize the functioning of language in ecological crises. In the works of several authors we find the aspiration to integrate perspectives of ecological science and the humanities into a non-dualistic interdisciplinary framework. Often such endeavours derive their motivation from semiotics. Combining semiotics and Actor-Network Theory (ANT) has been proposed for the non-dualistic treatment of culture-natures (Ivakhiv 2002; Jepson et al. 2011; Maran 2015). Adrian Ivakhiv has called his ANT-inspired approach multicultural ecology and described this as a ‘perspective that acknowledges the cultural embeddedness of any and all ideas nature, accepts the coexistence of multiple cultural-ecological practices, and gests, at least in a preliminary way, a normative dimension by which such practices can be compared and evaluated’ (Ivakihiv 2002). It is worth noting that ANT as originally developed by Bruno Latour was influenced by Greimas’ actant analysis and has also been called ‘material semiotics’ (Law 2008). Another surface of syntheses lies between semiotics and multi-perspectivist anthropology (mostly deriving from the works of Eduardo Viveiros de Castro). In this tradition, Eduardo Kohn (2007, 2013) has studied the relations between humans, domestic animals and wild animals in South American nature-cultures by combining local cosmologies with the Peircean typology of signs. According to this approach, humans and other animals are grounded by iconic and indexical semiosis which creates the united ecology of selves. In a similar way, Nils Lindahl Elliot (2019) has applied Peircean categories in analysing tourist experience in tropical America, considered in terms of the degrees of mediatedness organizing the perception and observation of wildlife. On the opposite side of the Earth, Almo Farina has found inspiration from traditional Mediterranean agricultures in suggesting the rural sanctuary model for promoting the co-existence of small-scale economy, local cultures and biological diversity. The rural sanctuary ‘is defined as an area where farming activity creates habitats for a diverse assemblage of species that find a broad spectrum of resources along the season […] A Rural Sanctuary represents an ecosemiotic agency in which human eco-fields and animal eco-fields interact’ (Farina 2018b: 139). Farina emphasizes the positive impact of human activities on other species as traditional agriculture often makes landscapes patchier and more heterogenic. There are also approaches that undermine the boundaries of human material or literary culture by aiming to build a direct synthesis between culture and the environment. This view is expressed by medievalist Alfred K. Siewers in his vision of an ecosemiosphere

66

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

that ‘literally means an ecological bubble of meaning (borrowing the term “semiosphere” from semiotics)’ (Siewers 2014: 4), wherein the term ‘extends earlier definitions of specific symbolic cultures as semiospheres, or meaningful environments, into physical environments’ (Siewers 2011: 41). Deriving from the works of Juri Lotman, Kati Lindström (2010) has suggested that the landscape may enter into a dialogic relationship with culture by providing perceptual markers that act as a second code of communication. Such a relationship leads to enhanced cultural autocommunication and thus enhances cultural creativity. Kadri Tüür (2009, 2016) has argued for overcoming the representational view of nature and using the concept of biotranslation for describing relations between animal meaning-making and literary depictions. It is noteworthy that semiotics has been influential in many applied fields of environmental studies such as landscape studies (Abrahamsson 1999; Claval 2005; Lindström et al. 2011), ecological restoration (Rochford 2017), ecological design (Ávila 2020) and environmental education. In environmental education especially, communicating and mediating environmental knowledge to students is a practical concern, as in the case of learning about plants and other organisms that are very different from humans (Affifi 2013). In this context, environmental literacy with a semiotic emphasis on interpretation has been used both as a theoretical concept (Stables and Bishop 2001), and also for encouraging the practical skill of reading traces and tracks (Lekies and Whitworth 2011). There are a number of critical concepts which denote the collapse of nature-cultures or the negative effects of human activities on the meaning-making of other species. Ivar Puura’s (2013) concept of semiocide describes ‘a situation in which signs and stories that are significant for someone are destroyed because of someone else’s malevolence or carelessness, thereby stealing a part of the former’s identity’. Examples of semiocide are the replacement of natural meadows with golf courses or primeval forests with mono-cultural plantations. A similar critical concept to describe harmful semiosis is that of semiotic pollution, as in the effect of excessive light or sound signals produced by modern human civilization to other life forms (Posner 2000). Semiotic pollution may disturb the code, contact, message, participants and other aspects of the sign process. German semiotician Ronald Posner draws our attention to the parallel between chemical contamination and semiotic pollution as both increase physiological stress in biological organisms. In some cases, the environmental object may also demonstrate dissent or non-concordance with the human interpretation. The concept of dissent was used in this context by Australian semiotician David Low (2008), who emphasized the necessity of including the environment as a semiotic subject into the study of environmental communication. According to his view, environmental processes enter into environmental communication as dynamical objects of the sign in the sense of C. S. Peirce. For example, the pollutants in water act as dynamical objects, whereas their perceived characteristics and effects act as immediate objects of the sign. In such situations, people search for the correspondence between dynamical and immediate objects – that is, they adjust and adapt their signmediated knowledge towards the environmental processes themselves. In another type of dynamic interaction, cultural norms are projected onto nature and the material environment through human activities. Here, cultural oppositions like city and forest or native and non-native may through applied rules and actions influence the structure of biological communities (Maran 2015; Magnus and Remm 2018). Prisca Augustyn (2013) has further demonstrated the role of language structures (framing, metaphors, oppositions) in the human understanding, appreciation and manipulation of nature. Semiotic modelling provides a tool to explicate the grounds of human

SEMIOTICS IN ECOLOGY AND ENVIRONMENTAL STUDIES

67

understandings of the environment, and to playfully rearrange these by altering the grounds of modelling (Maran 2020, forthcoming). On a more general scale, the type of semiosis dominant in culture may also influence culture-nature relations and local ecologies. Alf Hornborg (1996, 2001) has demonstrated how the dominance of abstract sign systems may lead to ecosystem dissolution. He has distinguished between sensory, linguistic and economic signs, and has shown how each subsequent sign type becomes more detached from the practices of human living within the ecosystem. Using the example of native South American peoples, Hornborg has shown that adoption of abstract economical sign systems becomes a main reason for the dismantling of local nature-cultures. As seen from this overview, semiotics is mostly applied in environmental studies for establishing a common framework in the study of nature-cultures, or for critical treatment of human effects on the environment.

ECOSEMIOTICS Ecosemiotics (also semiotic ecology) is an explicit synthesis of ecology and semiotics that started to develop in the 1990s. The concept was originally proposed by German semiotician Winfried Nöth (1996), although there is also a prehistory with earlier variations of the name used in the early 1990s (ecological semiotics, environmental semiosis). In the development of ecosemiotics, it is especially professor Kalevi Kull of the Tartu School who has had a leading role (for an overview, see Maran 2018). Over the years, many authors have explicitly elaborated ecosemiotics, including Almo Farina, Timo Maran, Morten Tønnessen, Riin Magnus, Alf Hornborg, Ernst Hess-Lüttich, Alfred K. Siewers, Kadri Tüür and Matthew Clements. Depending on the author, the scope of ecosemiotics has been understood as having either a more humanitarian or scientific focus. For instance, ecosemiotics has been defined as ‘the study of sign processes which relate organisms to their natural environment’ (Nöth 2001: 71), ‘a branch of semiotics that studies sign processes as responsible for ecological phenomena’ (Maran and Kull 2014: 41), or as the semiotic discipline investigating ‘human relationships to nature which have a semiosic (sign-mediated) basis’ (Kull 1998: 351). In the twenty-first century, ecosemiotics has gained more disciplinary unity and developed a shared framework that covers both semiosis in the biological realm and cultural representations of nature. For instance, Kalevi Kull has described the aim of the field holistically: The role of semiotics for ecology is to constitute a certain theoretical frame that would allow to approach, without any dualism, the analysis of semiosphere as the natureculture whole. This includes description and explanation of natural emergence of meaningfulness in organic communication, and of the communicative basis of organic forms and relations. (Kull 2008: 3211) Deriving from the more recent concepts of socio-ecological systems (Bodin 2017) and biocultural diversity (Sobo 2016), we can claim that ecosemiotics studies semiotic processes present in and responsible for constituting local biocultural wholes (or naturecultures). Contemporary ecosemiotics can be characterized as treating sign processes taking place on many different levels of the biosemiosphere: from the potential of the

68

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

environment to evoke semiosis, to the meaning-making and communication of animals, to semiotic networks in ecosystems, up to cultural representations and the symbolization of nature in culture (cf. Maran 2020, forthcoming). Ecosemiotics has categorized environmental semiosis and signification (elaborating concepts of perceptual affordance, tacit signs, environmental meta-signs, Maran 2017b) and combined these with cultural representations of nature (under concepts like nature-text, environmental-cultural hybrid signs, Maran 2017b, 2020). As Morten Tønnessen (2020) has demonstrated, ecosemiotics is also effective across various scales of generality in its study of both local and globalized semiotic processes. This broad scope allows ecosemiotics to analyse very different objects (nature writing, eco-cinema, urban trees, alien species, medicine plants, etc.) by pinpointing interactions, cross-effects and hybridizations between different levels of semiotic phenomena. Maran and Kull (2014) have brought out eight main principles of ecosemiotic research: 1. The structure of ecological communities is based on semiosic bonds; 2. Changing signs can change the existing order of things. Living organisms change their environment on the basis of their own images of that environment; 3. Semiosis regulates ecosystems. Meaning-making both stabilizes and destabilizes them; 4. Human symbolic semiosis (with its capacity of de-contextualization) and environmental degradation are deeply related; 5. Energetically and biogeochemically, human culture is a part of the ecosystem. Semiotically, culture is both a part and a meta-level of the semiosic ecological network; 6. The environment as a spatial-temporal manifestation of an ecosystem functions as an interface for semiotic and communicative relations; 7. Narrative description is inadequate for the description of ecological semiosis; 8. The concept of culture is incomplete without an ecological dimension. A theory of culture is incomplete without the ecosemiotic aspect. These principles are comprehensive in the sense that they have a broad focus which covers the semiotics of ecological relations as well as the semiotics of human connections with ecosystems, and in a metalevel role covers the ecological dimension within the humanities. Aside from its usage in environmental humanities and education science, ecosemiotics has also found applications in practical ecological research. The ecosemiotic approach appears to be very suitable for analysing semi-natural environments and hybrid natures. For instance, Low and Peric (2012) have applied ecosemiotics in analysing human agency in the distribution of weeds and the related construction of meanings, Maran (2015) has started from ecosemiotics in his survey of the spread and cultural interpretations of novel species (golden jackal, Canis aureus), and Magnus and Remm (2018) have provided an ecosemiotic analysis of the distribution and cultural history of urban tree species. Morten Tønnessen (2020) has recently elaborated on Umwelt theory in his analysis of the change in Amazonian culture-natures by focusing on two species of monkeys, the red howler monkey (Alouatta seniculus) and the black-headed squirrel monkey (Saimiri vanzolinii). Marcos S. Karlin (2016) has further applied ecosemiotics for mapping the resilience of the local semiosphere by juxtaposing locally and globally available species knowledge in Salinas Grande, Argentina. Based on the examples above, ecosemiotics appears to be

SEMIOTICS IN ECOLOGY AND ENVIRONMENTAL STUDIES

69

practically usable in various case studies and applied to various research objects. The development of ecosemiotics in the last twenty-five years is an encouraging sign for the viable synthesis of ecology and semiotics.

PERSPECTIVES AND CHALLENGES Semiotics appears to possess strong potential for contributing to the ecological sciences, owing to a robust analytical framework that covers informational and communicational processes in both the biological realm and human culture. There are encouraging examples of such integration (e.g. research done by Almo Farina), but the broad-scale synthesis of ecological and semiotic research is a task still to be undertaken. In working towards this synthesis, the following challenges must be addressed: 1. Developing a non-structural semiotics that would include conceptual tools and research methods for working with heterogenic, open, dynamical and potentially unlimited systems. Compared to human linguistic and cultural systems that have been the model object for semiotics, ecosystems are special due to the presence of a vast number of species and energetic and material openness. Analysing such systems would require a critical revision of semiotic methodology. 2. Elaborating the frame of analysis to cover processes with different semiotic complexity. The success of applying semiotics in ecological research appears to depend on methods that can address the interrelations of different types of semiotic processes (e.g. environmental affordances, signification, animal communication, cultural representation, meaning-making and symbolization in human discourses). Some progress has been achieved in this integration, especially in ecosemiotics, but there is still progress to be made. 3. From an ecological perspective, a critical question appears to be how to include and integrate qualitative descriptions and animal phenomenal perspectives into existing ecological methods and conceptual systems. There are topics where this integration has been quite successful (ecoacoustics, theory of niche construction), but on a broader scale integration still needs to be achieved. There are ongoing detrimental processes to our planet – climate change, species loss, accumulation of waste – that urge science to find new and effective ways to address environmental problems. Part of this challenge is bridging the natural sciences with the humanities, and on the object-level bridging biodiversity conservation with human discourses and meanings. In this context, developing the synthesis between ecology and semiotics is an endeavour both timely and very necessary.

ACKNOWLEDGEMENTS This work was supported by the Estonian Research Council grant (PUT 1504).

REFERENCES Abrahamsson, K. V. (1999), ‘Landscapes Lost and Gained: On Changes in Semiotic Resources’, Human Ecology Review, 6 (2): 51–61. Affifi, R. (2013), ‘Learning Plants: Semiosis between the Parts and the Whole’, Biosemiotics 6: 547–59.

70

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Augustyn, P. (2013), ‘Man, Nature, and Semiotic Modelling or How to Create Forests and Backyards with Language’, Sign Systems Studies, 41 (4): 488–503. Ávila, M. (2020), ‘(De)sign Responses as Response Diversity’, Biosemiotics, 13: 41–62. Beever, J. (2013), ‘Baudrillard’s Simulated Ecology’, Sign Systems Studies, 41 (1): 82–92. Bleicher, S. S. (2017), ‘The Landscape of Fear. Conceptual framework: Definition and Review of Current Applications and Misuses’, PeerJ, 5: e3772. Bodin Ö. (2017), ‘Collaborative Environmental Governance: Achieving Collective Action in Social-Ecological Systems’, Science, 357 (6352): eaan1114. Candland, D. K. (2005), ‘The Animal Mind and Conservation of Species: Knowing What Animals Know’, Current Science, 89 (7): 1122–7. Claval, P. (2005), ‘Reading the Rural Landscapes’, Landscape and Urban Planning, 70 (1–2): 9–19. Dobrin, S. I. (2018), ‘Environmental Image Analysis’, in N. Castree, M. Hulme, and J. D. Proctor (eds), Companion to Environmental Studies, 423–9, London: Routledge. Douglas, L. R. and D. Veríssimo (2013), ‘Flagships or Battleships: Deconstructing the Relationship between Social Conflict and Conservation Flagship Species’, Environment and Society: Advances in Research, 4 (1): 98–116. Drenthen, M. (2016), ‘Understanding the Meaning of Wolf Resurgence. Ecosemiotics and Landscape Hermeneutics’, in M. Tønnessen, K. Oma, and S. Rattasepp (eds), Thinking about Animals in the Age of the Anthropocene, 109–26, Lanham: Lexington Books. Farina, A. (2012), ‘A Biosemiotic Perspective of the Resource Criterion: Toward a General Theory of Resources’, Biosemiotics, 5 (1): 17–32. Farina, A. (2018a), ‘Ecoacoustic Codes and Ecological Complexity’, Biosystems, 164: 147–54. Farina, A. (2018b), ‘Rural Sanctuary: An Ecosemiotic Agency to Preserve Human Cultural Heritage and Biodiversity’, Biosemiotics, 11: 139–58. Farina, A. and A. Belgrano (2004), ‘The Eco-Field: A New Paradigm for Landscape Ecology’, Ecological Research, 19 (1): 107–10. Farina, A. and N. Pieretti (2013), ‘From Umwelt to Soundtope: An Epistemological Essay on Cognitive Ecology’, Biosemiotics, 7: 1–10. Farina, A., J. Bogaert, and I. Schipani (2005), ‘Cognitive Landscape and Information: New Perspectives to Investigate the Ecological Complexity’, Biosystems, 79 (1–3): 235–40. Farina, A., N. Pieretti, P. Salutari, E. Tognari, and A. Lombardi (2016), ‘The Application of the Acoustic Complexity Indices (ACI) to Ecoacoustic Event Detection and Identification (EEDI) Modeling’, Biosemiotics, 9: 227–46. Goodale, E., G. Beauchamp, R. D. Magrath, J. C. Nieh, and G. D. Ruxton (2010), ‘Interspecific Information Transfer Influences Animal Community Structure’, Trends in Ecology & Evolution, 25 (6): 354–61. Hansen-Møller, J. (2009), ‘Natursyns Model: A Conceptual Framework and Method for Analysing and Comparing Views of Nature’, Landscape and Urban Planning, 89 (3/4): 65–74. Heinapuu, O. (2016), ‘Agrarian Rituals Giving Way to Romantic Motifs: Sacred Natural Sites in Estonia’, Sign Systems Studies, 44 (1–2): 164–85. Hess-Lüttich, E. W. B. (2016), ‘Urban Discourse – City Space, City Language, City Planning: Eco-Semiotic Approaches to the Discourse Analysis of Urban Renewal’, Sign Systems Studies, 44 (1–2): 12–33. Hoffmeyer, J. (1996), Signs of Meaning in the Universe, Bloomington: Indiana University Press. Hoffmeyer, J. (2008), Biosemiotics: An Examination into the Signs of Life and the Life of Signs, Scranton: University of Scranton Press.

SEMIOTICS IN ECOLOGY AND ENVIRONMENTAL STUDIES

71

Hornborg, A. (1996), ‘Ecology as Semiotics: Outlines of a Contextualist Paradigm for Human Ecology’, in P. Descola and G. Pálsson (eds), Nature and Society: Anthropological Perspectives, 45–62, London: Routledge. Hornborg, A. (2001), ‘Vital Signs. An Ecosemiotic Perspective on the Human Ecology of Amazonia’, Sign Systems Studies, 29: 121–52. Ivakhiv, A. (2002), ‘Toward a Multicultural Ecology’, Organization & Environment, 15 (4): 389–409. Janzen, D. H. (1985), ‘On Ecological Fitting’, Oikos, 45 (3): 308–10. Jepson, P., M. Barua, and K. Buckingham (2011), ‘What Is a Conservation Actor?’ Conservation and Society, 9 (3): 229–35. Jordan, L. A. and M. J. Ryan (2015), ‘The Sensory Ecology of Adaptive Landscapes’, Biology Letters, 11: 20141054. Karlin, M. S. (2016), ‘Ethnoecology, Ecosemiosis and Integral Ecology in Salinas Grandes (Argentina)’, Etnobiología, 14 (1): 23–38. Kleisner, K. (2015), ‘Semantic Organs: The Concept and Its Theoretical Ramifications’, Biosemioticsi, 8: 367–79. Kohn, E. (2007), ‘How Dogs Dream: Amazonian Natures and the Politics of Transspecies Engagement’, American Ethnologist, 34 (1): 3–24. Kohn, E. (2013), How Forests Think: Toward an Anthropology beyond the Human, Berkeley: University of California Press. Krampen, M. (1979), Meaning in the Urban Environment, London: Pion. Kull, K. (1998), ‘Semiotic Ecology: Different Natures in the Semiosphere’, Sign Systems Studies, 26: 344–71. Kull, K. (2008), ‘Semiotic Ecology’, in S. E. Jørgensen and B. D. Fath (eds), Systems Ecology, Vol 4 of Encyclopedia of Ecology, 3210–14, Oxford: Elsevier. Kull, K. (2010), ‘Ecosystems Are Made of Semiosic Bonds: Consortia, Umwelten, Biophony and Ecological Codes’, Biosemiotics, 3 (3): 347–57. Kull, K. (2020), ‘Semiotic Fitting and the Nativeness of Community’, Biosemiotics, 13: 9–19. Law, J. (2008), ‘Actor Network Theory and Material Semiotics’, in B. S. Turner (ed.), The New Blackwell Companion to Social Theory, 3rd edn, 141–58, Oxford: Blackwell. Lekies, K. S. and B. Whitworth (2011), ‘Constructing the Nature Experience: A Semiotic Examination of Signs on the Trail’, The American Sociologist, 42: 249–60. Lévêque, C. (2003), Ecology: From Ecosystem to Biosphere, Boca Raton: CRC Press. Lindahl Elliot, N. (2019), Observing Wildlife in Tropical Forests, vol. 1: A Geosemeiotic Approach, Bristol: Delome Publications. Lindström, K. (2010), ‘Autocommunication and Perceptual Markers in Landscape: Japanese Examples’, Biosemiotics, 3 (3): 359–73. Lindström, K., K. Kull, and H. Palang (2011), ‘Semiotic Study of Landscapes: An Overview from Semiology to Ecosemiotics’, Sign Systems Studies, 39: 12–36. Lotman, J. (2005), ‘On the Semiosphere’, Sign Systems Studies, 33 (1): 215–39. Low, D. (2008), ‘Dissent and Environmental Communication: A Semiotic Approach’, Semiotica, 172: 47–64. Low, D. and Z. Peric (2012), ‘An Ecosemiotic Approach to Weed Biosecurity’, Pakistan Journal of Weed Science Research, 18: 801–9. Magnus, R. and T. Remm (2018), ‘Urban Ecosemiotics of Trees: Why the Ecological Alien Species Paradigm Has Not Gained Ground in Cities?’ Sign Systems Studies, 46 (2–3): 319–42.

72

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Malavasi, R., K. Kull, and A. Farina (2013), ‘The Acoustic Codes: How Animal Sign Processes Create Sound-Topes and Consortia via Conflict Avoidance’, Biosemiotics, 7: 89–95. Manning, A. D., D. B. Lindenmayer, and H. A. Nix (2004), ‘Continua and Umwelt: Novel Perspectives on Viewing Landscapes’, Oikos, 104: 621–8. Maran, T. and K. Kull (2014), ‘Ecosemiotics: Main Principles and Current Developments’, Geografiska Annaler: Series B, Human Geography, 96 (1): 41–50. Maran, T. (2015), ‘Emergence of the “Howling Foxes”: A Semiotic Analysis of Initial Interpretations of the Golden Jackal (Canis aureus) in Estonia’, Biosemiotics, 8 (3): 463–82. Maran, T. (2017a), Mimicry and Meaning: Structure and Semiotics of Biological Mimicry, Cham: Springer. Maran, T. (2017b), ‘On the Diversity of Environmental Signs: A Typological Approach’, Biosemiotics, 10 (3): 355–68. Maran, T. (2018), ‘Two Decades of Ecosemiotics in Tartu’, Sign Systems Studies, 46 (4): 630–9. Maran, T. (2020), ‘Ecological Repertoire Analysis: A Method of Interaction-Based Semiotic Study for Multispecies Environments’, Biosemiotics, 13 (1): 63–75. Maran, T. (forthcoming), Ecosemiotics: The Study of Signs in Changing Ecologies, Cambridge: Cambridge University Press. Nielsen, S. N. (2007), ‘Towards an Ecosystem Semiotics: Some Basic Aspects for a New Research Programme’, Ecological Complexity, 4 (3): 93–101. Nielsen, S. N. (2016), ‘Second Order Cybernetics and Semiotics in Ecological Systems. Where Complexity Really Begins’, Ecological Modelling, 319: 119–29. Nöth, W. (1996), ‘Ökosemiotik’, Zeitschrift für Semiotik, 18 (1): 7–18. Nöth, W. (2001), ‘Ecosemiotics and the Semiotics of Nature’, Sign Systems Studies, 29 (1): 71–81. O’Connor, M. I., M. W. Pennell, F. Altermatt, B. Matthews, C. J. Melián and A. Gonzalez (2019), ‘Principles of Ecology Revisited: Integrating Information and Ecological Theories for a More Unified Science’, Frontiers in Ecology and Evolution, 7: 219. Patten, B. C. (1990), ‘Environ Theory and Indirect Effects: A Reply to Loehle’, Ecology, 71 (6): 2386–93. Patten, B. C. (1998), ‘Network Orientors: Steps toward a Cosmography of Ecosystems: Orientors for Directional Development, Self-Organization, and Autoevolution’, in F. Müller and M. Leupelt (eds), 137–60, Berlin: Springer-Verlag. Patten, B. C. and E. P. Odum (1981), ‘The Cybernetic Nature of Ecosystems’, The American Naturalist, 118 (6): 886–95. Peterson, J. V., A. M. Thornburg, M. Kissel, C. Ball, and A. Fuentes (2018), ‘Semiotic Mechanisms Underlying Niche Construction’, Biosemiotics 11: 181–98. Pizzolotto, R. (2009), ‘Characterization of Different Habitats on the Basis of the Species Traits and Eco-Field Approach’, Acta Oecologica, 35 (1): 142–8. Posner, R. (2000), ‘Semiotic Pollution’, Sign Systems Studies, 28: 290–307. Puura, I. (2013), ‘Nature in Our Memory’, Sign Systems Studies, 41 (1): 150–3. Rochford, F. (2017), ‘Designing the Environment – The Paradox of Eco-Restoration’, Griffith Law Review, 26 (2): 202–20. Sánchez-García, F. J., V. Machado, J. Galián, and D. Gallego (2017), ‘Application of the EcoField and General Theory of Resources to Bark Beetles: Beyond the Niche Construction Theory’, Biosemiotics, 10: 57–73. Schmidt, K. A., S. R. Dall, and J. A. Gils (2010), ‘The Ecology of Information: An Overview on the Ecological Significance of Making Informed Decisions’, Oikos 119: 304–16.

SEMIOTICS IN ECOLOGY AND ENVIRONMENTAL STUDIES

73

Shaw, I. G., J. P. Jones, and M. K. Butterworth (2013), ‘The Mosquito’s Umwelt, or One Monster’s Standpoint Ontology’, Geoforum 48: 260–7. Shier, D. M. (2016), ‘Manipulating Animal Behavior to Ensure Reintroduction Success’, in O. Berger-Tal and D. Saltz (eds), Conservation Behavior: Applying Behavioral Ecology to Wildlife Conservation and Management, 275–304, Cambridge: Cambridge University Press. Siewers, A. K. (2011), ‘Pre-modern Ecosemiotics: The Green World as Literary Ecology’, in T. Peil (ed.), The Space of Culture – The Place of Nature in Estonia and Beyond, 39–68, Tartu: University of Tartu Press. Siewers, A. K. (2014), ‘Introduction: Song, Tree, and Spring: Environmental Meaning and Environmental Humanities’, in A. Siewers (ed.), Re-imagining Nature: Environmental Humanities and Ecosemiotics, 1–41, Bucknell: Bucknell University Press. Sobo, E. J. (2016), Dynamics of Human Biocultural Diversity: A Unified Approach, London: Routledge. Sõukand, R. (2005), ‘Loodus eesti rahvameditsiinis’, in T. Maran and K. Tüür (eds), Eesti Looduskultuur, 55–79, Eesti Kirjandusmuuseum. Stables, A. and K. Bishop (2001), ‘Weak and Strong Conceptions of Environmental Literacy: Implications for Environmental Education’, Environmental Education Research, 7 (1): 89–97. Stibbe, A. (2012), Animals Erased: Discourse, Ecology and Reconnection with the Natural World, Middletown: Wesleyan University Press. Stibbe, A. (2015), Ecolinguistics: Language, Ecology and the Stories We Live By, London: Routledge. Tønnessen, M. (2009), ‘Umwelt Transitions: Uexküll and Environmental Change’, Biosemiotics, 2 (1): 47–64. Tønnessen, M. (2010), ‘Wolf Land’, Biosemiotics, 3: 289–97. Tønnessen, M. (2014), ‘Umwelt Trajectories’, Semiotica, 198: 159–80. Tønnessen, M. (2020), ‘Current Human Ecology in the Amazon and Beyond: A Multi-Scale Ecosemiotic Approach’, Biosemiotics, 13: 89–113. Tüür, K. (2009), ‘Bird Sounds in Nature Writing: Human Perspective on Animal Communication’, Sign Systems Studies, 37 (3–4): 226–55. Tüür, K. (2016), ‘Semiotics of Textual Animal Representations’, in T. Maran, M. Tønnessen, and S. Rattasepp (eds), Animal Umwelten in a Changing World: Zoosemiotic Perspectives, 222–38, Tartu: University of Tartu Press. Uexküll, J. V. (1982), ‘The Theory of Meaning’, Semiotica, 42: 25–82. Ulanowicz, R. E. (2010), ‘Process Ecology: Stepping Stones to Biosemiosis’, Zygon: Journal of Religion & Science, 45 (2): 391–407. Van Dyck, H. (2012), ‘Changing Organisms in Rapidly Changing Anthropogenic Landscapes: The Significance of the “Umwelt”-Concept and Functional Habitat for Animal Conservation’, Evolutionary Applications, 5 (2): 144–53. Vladimirova, E. (2009), ‘Sign Activity of Mammals as Means of Ecological Adaptation’, Sign Systems Studies, 37 (3–4): 614–38. Vladimirova, E. and J. Mozgovoy (2003), ‘Sign Field Theory and Tracking Techniques Used in Studies of Small Carnivorous Mammals’, Evolution and Cognition, 9 (1): 1–17. Whitehouse, A. (2015), ‘Listening to Birds in the Anthropocene: The Anxious Semiotics of Sound in a Human-Dominated World’, Environmental Humanities, 6: 53–71.

74

CHAPTER FOUR

Semiotics in Ethology and Zoology MORTEN TØNNESSEN

INTRODUCTION The chapter presents semiotic studies in ethology and zoology, starting with foundational work and contemporary developments and proceeding to relevant methodologies and prospective future studies. Current research needs include further developments of methodology, which is why a section on prospective future studies ends the chapter. Throughout this chapter, the semiotic study of animals will frequently be referred to in terms of zoosemiotics, since this notion is so central in the semiotic discourse about animals. The conception of zoosemiotics as a field of study has evolved considerably since Thomas Sebeok coined the term in 1963 (Sebeok 1963), and currently signifies a far more comprehensive field than what Sebeok’s original definition of zoosemiotics as ‘the scientific study of signalling behaviour in and across animal species’ indicates (1963: 465). In contrast, Maran et al. (2011a: 8) distinguish between ethological zoosemiotics and anthropological zoosemiotics, with the former overlapping with Sebeok’s initial notion of zoosemiotics and the latter programmatically expanding the conception of zoosemiotics by addressing ‘the semiotic interaction between human beings and other animals’.1 This includes communicative human-animal interaction, which is typically ‘reciprocal and – [with some reservations] – intentional’ (Maran et al. 2011a: 8), and may be said to be of equal relevance to ethology and studies of human behaviour and experience. Notably, it also includes significational and representational instances of human semiosis involving animals. The contemporary notion of zoosemiotics thus entails that the semiotic study of animals cannot simply be understood as a synthesis between semiotics and ethology, although such a synthesis remains at its core. It must furthermore be understood as engaging and intersecting with ecology, cultural studies and other fields of study where animals appear in one form or another. Though those who label themselves as ‘zoosemioticians’ are still quite few in number, the semiotic approach to studying animals has gained widespread attention, especially among scholars in the humanities and social sciences who aim to draw on or relate to natural science. For example, Donna Haraway, one of the leading thinkers within the posthumanities, makes extensive use of semiotics in her now classic work When Species Meet (Haraway 2008), which deals with human–animal encounters and relations. In an interview (Haraway 2016: 209–10) she furthermore acknowledges that her equally

76

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

influential ‘A Cyborg Manifesto’ (Haraway 1985) was inspired, among other things, by the impact the Umwelt theory of Jakob von Uexküll was having on some contemporary thinkers.2 Von Uexküll’s work is solidly positioned at the foundations of semiotic studies of animals. With the Umwelt theory, developed over a period of some thirty years (from von Uexküll 1909 to von Uexküll 1940), the Baltic-German biologist offers theoretical perspectives and tools for a semiotic study of animal behaviour and experience. In his development of the theory, von Uexküll’s own understanding was that he was applying the epistemological and ontological outlook of Immanuel Kant ([1778] 1996) to the realm of biology (see particularly von Uexküll 1928). Such philosophical references, and more importantly, philosophical aspects of von Uexküll’s original theory development, may be part of the reason why the Umwelt theory has been received with interest and curiosity among philosophers from von Uexküll’s own time and all the way up to contemporary times. Among philosophers, von Uexküll’s work on the nature of animal lifeworlds has been discussed by prominent figures such as Martin Heidegger (1995, drawn from lectures Heidegger held in a course 1929–30), Maurice Merleau-Ponty (2003, drawn from lectures Merleau-Ponty held in three consecutive courses on ‘The Concept of Nature’ in 1956–7, 1957–8, and 1959–60), and Gilles Deleuze and Feliz Guattari (1987) (see also Buchanan 2008), among many others. Von Uexküll’s semiotically framed work in biology has influenced ethology, the scientific study of animal behaviour, directly in form of the impact he had on contemporary German biology in his time, and also by way of Konrad Lorenz (1935, [1949] 1961) who is generally regarded as one of the founders of ethology and made some use of von Uexküll’s work. Von Uexküll himself is also frequently referred to as a pioneer within ethology. However, ethology as it is conceived of and practised today has quite a different appearance than the one von Uexküll programmatically laboured to give it. To simplify, mainstream contemporary ethology is predominantly an objective field of study primarily making use of quantitative methods, whereas von Uexküll’s research agenda was that of a subjective biology (see particularly von Uexküll 1928) in which qualitative methods would play an important role in the study of animal behaviour. With this in mind, we can safely observe that von Uexküll’s Umwelt theory has, until now at least, been more influential in shaping how scholars think about biology outside biology than it has been within biology. As already alluded to, a semiotic approach to studying animals has been particularly influential among scholars in the humanities and social sciences with an interest in learning from natural science. These scholars have in many cases opposed drawing any sharp distinction between ‘two cultures’ of scholarly study (Snow 2001), and favoured some kind of integration between natural science, the humanities and social science. This inclination is also evidently a feature of contemporary biosemiotics and zoosemiotics. The journal Biosemiotics, published by Springer Nature, is probably the one journal in the world that currently publishes the most articles that apply semiotic perspectives to studies of animals.3 Tellingly, in the ‘Web of Science’ indexing system now operated by Clarivate Analytics, the journal is categorized both under ‘Evolutionary and developmental biology’ (i.e. as a biology journal) and under ‘History and philosophy of science’ (i.e. as a history or philosophy journal). This illustrates the ambivalent position of contemporary semiotic studies of animals, and raises the question of whether there is any chance that biology can be fundamentally transformed ‘from the outside’.

SEMIOTICS IN ETHOLOGY AND ZOOLOGY

77

FOUNDATIONAL WORK Zoosemiotics, the semiotic study of topics of ethology, zoology and other animal-related fields, historically emerged as a precursor to biosemiotics and ecosemiotics, with common roots in the Umwelt theory of Jakob von Uexküll (1921, 1934), among other sources of inspiration (see, e.g., Sebeok 2001; Kleisner 2008). Thomas Sebeok (1963) coined the term, established von Uexküll as a key figure in zoosemiotics and made initial connections between semiotic theory and animal studies. While relating to Peirce’s semiotics as a framework, Sebeok also pinpointed the relevance of the semiotic work of Charles Morris (1901–79), e.g. Morris 1946. Building further on Sebeok’s perspective, Maran et al. (2011b: 248) note ‘the inherent significance of Charles Morris for zoosemiotic studies: he has developed an explicitly zoosemiotic interpretation of Peircean semiotics and has often used animal behaviour and communication to illustrate his theoretical models’.4 As Maran, Tønnessen, Magnus et al. (2016) emphasize, zoosemiotics approaches animals as both subjects and objects of semiotics. The Umwelt theory of von Uexküll has been particularly central in the study of animals as subjects of semiotics, which also has an impact on semiotics at large given its implicit expansion of the range of semiotic (or semiosic) subjects beyond human subjects. In Winfried Nöth’s Handbook of semiotics (Nöth 1990), zoosemiotics is thus presented as ‘the study of the semiotic behaviour of animals’ (a definition that implicitly recognizes animals as subjects of semiotics) and ‘a transdisciplinary field of research’ that is situated ‘between biology and anthropology’ (Nöth 1990: 147). The conception of semiotics that is presented in this chapter has some resonance within the semiotic community, but it should be noted that it diverges from certain other predominant conceptions, at least in its semiotically informed philosophical anthropology (i.e. its semiotic understanding of the human species). While most semioticians recognize that sign use occurs in animals, opinions differ on what distinguishes human from nonhuman animal sign use. For instance, John Deely (2010) regards the human being as the only ‘semiotic animal’, ‘alone among the lifeforms of planet earth’ in that it is ‘capable of knowing that there are signs, and (quite a further step again) what signs are’ (2010: 114). In Deely’s view, by using signs but not understanding signs as signs (as distinguished from things and objects), animals are merely semiosic, whereas human beings stand out as the only truly semiotic animal. Moreover, central thinkers within cognitive semiotics, such as Göran Sonesson (2006) and Jordan Zlatev (2009), stress that only human beings are capable of the most advanced forms of sign use. In contrast with biosemioticians, they prefer to reserve the notion of ‘sign’ for contexts that involve complex cognitive processes, thus limiting the scope of zoosemiosis to the most cognitively complex animals. While Sonesson and Zlatev recognize that any animal relates to meaning, they argue that this fact is not in itself proof that they are using signs. It should be noted that even within biosemiotics, which is often understood to incorporate zoosemiotics within it, there may be diverging conceptions of the semiotic agency of animals. Both Jesper Hoffmeyer (1996, 2009) and Marcello Barbieri (2003, 2015) take for granted that animals do have semiotic agency (or subjecthood), with both scholars drawing heavily on von Uexküll’s Umwelt theory in this regard. The two generally agree that animals interpret signs. What they disagree on is first and foremost the threshold for interpretative semiosis, with Barbieri restricting this to animals with a nervous system, while Hoffmeyer had a wider understanding of interpretative semiosis (and thus operated with a lower threshold). In the context of zoosemiotics, this implies

78

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

that whereas Barbieri by and large limits interpretative semiosis to so-called higher animals, Hoffmeyer applied the notion also to ‘lower animals’ and unicellular organisms that have at times been classified as animals. However, neither in the work of Hoffmeyer nor in that of Barbieri does the semiotic behaviour and capacities of individual animals feature as particularly central. In contrast, another central biosemiotician, Kalevi Kull, has repeatedly contributed to developing our understanding of animals in the semiotic context, even though he is originally trained as a botanist (see for instance his distinction between vegetative, animal and cultural semiosis in Kull 2009, and his portrayal of zoosemiotics as ‘the study of animal forms of knowing’ in Kull 2014). Kull has also been central in the dissemination and contemporary contextualization of the pioneering work of Jakob von Uexküll (Kull 2001, cf. also Rüting 2004).

Jakob von Uexküll (1864–1944) ‘Umwelt’, a term for the subjective lifeworld of certain organisms, is undoubtedly the central notion in the work of Jakob von Uexküll and the reason why his work is of foundational importance for any semiotic study of animals. In von Uexküll’s work, the Umwelt notion is applied on animals, humans and microorganisms, though not on plants nor fungi.5 The Umwelt notion in its classical, Uexküllian version is thus universally applicable within the semiotic study of animals, but also goes beyond this. Although von Uexküll’s Umwelt theory, with its depiction of subjective animal lifeworlds, was semiotic from the outset, its semiotic aspects are most developed and most precisely articulated in his very last main work, Bedeutungslehre (von Uexküll 1940, cf. also von Uexküll and Kriszat 1956, translated to English as von Uexküll 2010). In the words of Tønnessen, Brentari and Magnus (2016: 144): In Bedeutungslehre (von Uexküll 1940) […] the connections between different Umwelten are brought into focus, and ‘meaning’/’significance’ [Bedeutung] as the major organizing principle of the composition of different Umwelten is emphasized. Hence the Umwelten are here, in this last major work of Uexküll, described in an explicitly ecological and semiotic framework. The later biosemiotic interpretations of ‘Umwelt’ appear to combine Uexküll’s earlier focus on the Umwelten of individual (species) and the meaning-related explanations of Umwelt formation which is more explicit in Uexküll’s later works.

Thomas Sebeok (1920–2001) The coinage ‘zoosemiotics’ first appeared in print in a review by Thomas Sebeok of several books (Sebeok 1963), in the following sentence: ‘The term zoosemiotics – constructed in an exchange between Rulon Wells and me – is proposed for the discipline, within which the science of signs intersects with ethology, devoted to the scientific study of signalling behaviour in and across animal species’ (1963: 465). This first definition of zoosemiotics frames the semiotic study of animals as intersecting with ethology, the study of animal behaviour, and implies that the only animal behaviour that is of a semiotic character is that of signalling behaviour.6 In ‘“Talking” with animals: Zoosemiotics explained’ (Sebeok 1990c), Sebeok elaborates by stating that ‘[s]emiotics is, quite simply, the exchange of messages’, and that zoosemiotics refers to ‘that segment of the field which focuses on

SEMIOTICS IN ETHOLOGY AND ZOOLOGY

79

messages given off and received by animals, including important components of human nonverbal communication, but excluding man’s language and his secondary, languagederived semiotic systems’ (as reprinted in Maran et al. (eds) 2011b: 87). As we see in this latter citation, in Sebeok’s view there are elements of zoosemiosis in the human species, to the extent that humans exhibit generic ‘animal’ traits that are also found in animals. However, he thought that the study of more specifically human traits should be left to other branches of semiotics. In Sebeok’s own retelling (Sebeok [1972b] 2011: 96–7), the initial reactions to the introduction of zoosemiotics were ‘by and large, negative’, although he ‘was unprepared for its incipient espousal in several branches of zoology’ ([1972b] 2011: 96). In 1965, in an article in Science (Sebeok 1965), he explained that he chose the term ‘zoosemiotics’ ‘to emphasize its necessary dependency on a science which deals, broadly, with coding of information in cybernetic control processes and the consequences that are imposed by this categorization where a living animal is the transcoder in a biological version of the traditional information-theory circuit’. In Thomas Sebeok’s conception of semiotics, zoosemiotics had a central position: ‘According to Sebeok’s own subdivision of semiotics, zoosemiotics is one of the three major branches of semiotics, along with anthroposemiotics and endosemiotics’ (1972a: 163, qtd. in Nöth 1990: 147–8). A majority of Sebeok’s most central writings on zoosemiotics are included in two essay collections (Sebeok 1972a, Sebeok 1990a). This includes a proposed division of zoosemiotics into three parts, namely pure, descriptive and applied zoosemiotics (Sebeok 1972a: 132). According to his views, pure zoosemiotics should be concerned with the development of language and models designed to deal scientifically with signs in animals; descriptive zoosemiotics should study the pragmatic, semantic, and syntactic aspect of animal communication; and applied zoosemiotics should study the possibilities to use and manipulate the semiotic activity of animals for practical goals. (Maran et al. 2011a: 9)

Heini Hediger (1908–92) In ‘Biosemiotics: Its Roots, Proliferation and Prospects’, Thomas Sebeok (2001) portrays the work of the Swiss zoo biologist Heini Hediger as representing one of three ‘20th century iterations’ of biosemiotics, with von Uexküll’s Umwelt theory representing the first, and the work of Giorgio Prodi the last (Favareau 2010: 223). By presenting Hediger’s ‘lifelong attempt to understand animals’ as ‘a milestone in the elucidation of’ biosemiotics, he stresses the central role of studies of animals, and the importance of empirically oriented work, within biosemiotics (2010: 225). Hediger met with von Uexküll and, later on, knew Sebeok, and in this sense he stands as a transition figure, as it were, between the two in the history of semiotic studies of animals. In his biological works, Hediger (1964, 1980) referred to von Uexküll and further developed an approach to zoo biology that was informed by fundamental insights drawn from Umwelt theory. In Chrulew’s depiction (2020), von Uexküll himself emphasized that zoo ‘enclosures should include objects and apparatuses that take on significance within the unique sign system of their occupants, thereby encouraging meaningful and healthy activity on the part of the animals that will, in turn, enliven the exhibit for viewers’. His influence on Hediger ‘is particularly prominent within

80

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Hediger’s theorisation of the key task of zoo biology: to understand and remediate the transformations produced by captivity’ (2020). Sebeok (2001: 225) particularly highlights how the Umwelt theory had a ‘decisive influence’ on Hediger’s ‘highly original analyses of the psychology and biology of animal flight response (or negative territoriality)’. Underlining the significance of promoting the psychological and physical comfort of zoo animals, Hediger sought to facilitate and enhance the natural speciesspecific behaviour of zoo animals by identifying and eliminating abnormal behaviours caused by the captive setting (2020). ‘All sorts of problems encountered in the zoo’, in Hediger’s view, ‘could be most effectively combated or solved by a consideration of what is significant for the animals’ (Chrulew 2020). In Sebeok’s understanding (2001: 226), ‘Hediger totally accepted the principles of zoosemiotics – which of course constitutes a substantial segment of biosemiotics’.

CONTEMPORARY DEVELOPMENTS In contemporary times, Dario Martinelli (2010) and others have developed a history of zoosemiotics as well as contributed to expanding and further developing the range of semiotic studies of animals (see also Maran et al. 2011b). Importantly, leading scholars have supplemented the classical ethological zoosemiotics, ‘dealing with animal sign action’ (Maran et al. 2011a: 8), with anthropological zoosemiotics (Martinelli 2007; Maran et al. 2011b), with the latter being concerned with ‘the study of semiotic relations between humans and other animals’ (Maran et al. 2011a: 1). Instead of limiting zoosemiotics to signalling behaviour in animals, as Sebeok (1963) initially did, this implies defining zoosemiotics as ‘the study of semiosis within and across animal species’ (Martinelli 2007: 28, emphasis in the original). As Dario Martinelli (2007: 28) makes explicit, zoosemiotics does not only deal with animal communication, it ‘is interested also in another important semiotic phenomenon, that of signification, occurring when the receiver is the only subject taking part in the semiosis, and a true sender is missing’. Furthermore, contemporary zoosemiotics is also concerned with representations. In Martinelli’s work, this is presented as a ‘sub-category within anthropological zoosemiotics’ (Maran et al. 2011a: 8–9); ‘here, the non-human animal is a pure source of meaning, an object, rather than a subject, of representation’. Maran et al. (2011: 1) incorporate these different categories of semiosis into the definition of zoosemiotics by defining the field as ‘the study of signification, communication and representation within and across animal species’ (emphasis in the original). Some people may wonder how zoosemiotics can be defined as the study of semiosis ‘within and across animal species’ (Martinelli 2007: 28) at the same time as a part of zoosemiotics is said to be ‘anthropological’. The explanation is that in this outlook, the human species is regarded as an animal species (Martinelli 2007: 28, cf. Maran et al. 2011a: 2). From a biological point of view this stand is commonsensical and unproblematic. It is also to some extent consistent with Sebeok’s view on this point, in that Sebeok too held that ‘important components of human nonverbal communication’ were of a zoosemiotic nature (Sebeok 1990c as reprinted in Maran et al. 2011b: 87). However, the rhetoric is different, and Sebeok might not have agreed that the study of human representations of animals in the form of, e.g., ‘texts, names, depictions, beliefs, myths or factual knowledge about animals’7 should be framed as partaking in the enterprise of zoosemiotics. After all, even though such representations have animal content (i.e. they are about animals), the representational activity involved is distinctly human.

SEMIOTICS IN ETHOLOGY AND ZOOLOGY

81

At this crossroad, whether or not the human being is regarded as an animal turns out to be of practical importance: If it is, this conception of zoosemiotics makes sense, if it isn’t, it does not. Contemporary zoosemiotics is markedly post-Sebeokian in that it deviates from Sebeok’s outlook in the framing of the human being’s position within nature at large. In similar terms, it also deviates from Sebeok’s worldview in the framing of culture’s position within nature. In the first case, contemporary zoosemiotics is arguably more consistently pluralistic, and has less of an anthropocentric bias, than Sebeokian zoosemiotics. In the second case, in its view on the relation between culture and nature, it is arguably more consistently holistic. These developments, representing deviations from classical Sebeokian zoosemiotics, appear to be regarded as advances by most scholars and students in the newest generation of zoosemioticians – and to be more in line with the contemporary Zeitgeist among concerned scientists and environmental and animal protection activists.8 Key developments in semiotic approaches to animal studies in recent years have their origin in Italy, France, the Czech Republic and Estonia, among other countries. Some examples of contemporary research represent continuations of classical work within zoosemiotics, with updated knowledge. For example, the topic of ape language experiments, towards which Sebeok was critical, has been revisited by Mirko Cerrone (2018). Hediger’s work on zoological gardens has been continued in Nelly Mäekivi’s work on communication and welfare aspects of current zoo biology and practices (see also Kull 2016, on the semiotic work of Estonian zoologist Aleksei Turovski, Tallinn Zoo). Likewise, Riin Magnus (2016) has developed a contemporary zoosemiotic perspective on guide dogs for the blind, a topic which preoccupied Jakob von Uexküll and his colleagues in his days. All of these examples involve empirical case studies. Further contemporary empirical case studies, without similarly clear historical precedents, include topics such as, e.g., zoomusicology (Martinelli 2009), cats and their relation to humans (Jaroš 2016), and communicative interaction between captive elephants and humans (Ribó 2019). Attempts at synthesis with related fields or approaches have been made, e.g., with regard to ecology and ecological economics (Farina 2012), Actor-Network Theory (Maran 2015), posthumanities, anthropology (e.g. Brentari 2015) and Human–animal studies/ anthrozoology. Other developments are related to work contextualizing Umwelt theory, e.g. within philosophy (including phenomenology, see Buchanan 2008, and philosophical anthropology, see Brentari 2015), or criticizing the classical version of Umwelt theory while aiming to update it (see, e.g., Tønnessen 2009, cf. also Burghardt 2008, Allen 2014). Furthermore, zoosemiotics has been contextualized within ethology at large (see, e.g., Maran 2010, cf. also Allen 2014). Critical discussion and integration with related approaches is now resulting in novel models and methods, many of which originating in a growing number of case studies (see, e.g., Maran, Tønnessen and Oma et al. 2016). Focus is shifting from objective descriptions of differences in sensory apparatus, communication channels, etc. to more dynamical representations with more emphasis on interaction (e.g. Lestel 2011) and hybridity, thus challenging mainstream dichotomies. Particularly, there is rising interest in human–animal interaction, and many authors frame descriptive accounts about such interaction as involving an ethical outlook or being pertinent for ethical considerations. In doing so, they underline the moral relevance of human–animal sign exchange and the ways in which human semiosis ultimately affect animals. Further developments in the application of semiotic perspectives in ethology and zoology are related to the study of zoosemiosis in humans (e.g. the animality of

82

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

humans), and an enforced focus on representations of animals in human culture (see, e.g., Tüür and Tønnessen 2014; Marrone and Mangano 2018) and semiosis between humans and animals. The latter development points towards integration of zoosemiotic and ecosemiotic perspectives, particularly in the context of the environmental crisis. For many scholars, this involves what we could characterize as an ethical turn in zoosemiotics in particular and to some extent semiotics generally. This ethical turn is discernible in the increasing focus, within the semiotic community, on environmental problems and human responsibility for the effects of our human semiosis. It is furthermore detectable in the growing interest in animal welfare concerns and related outlooks.

METHODOLOGIES FOR SEMIOTIC STUDIES IN ETHOLOGY AND ZOOLOGY Maran, Tønnessen, Tüür et al. (2016) mention Umwelt analysis, communication analysis, environmental analysis, and analysis of animal representations as basic methodological approaches in zoosemiotics. They further emphasize the central role of modelling theory, and contemporary elaborations of Umwelt theory (cf. also Maran 2014), for zoosemiotic methodology. Furthermore, they observe that zoosemiotic studies often involve hybrid objects that require a multifaceted and flexible approach, often requiring the development of ad-hoc methods. A scholar that has contributed originally to a biosemiotically informed approach to the study of animals is the French philosopher Dominique Lestel. He has called for a more interactive ethology (Lestel 2011, cf. also Lestel 2002), where the ethological researcher does not approach the animal as a passive object, but rather as an active subject capable of interacting with other sentient beings such as the said researcher. A basic hypothesis in this work is that animal behaviour is best displayed during interaction (cf. also Lestel, Brunois and Gaunet 2006; Chrulew 2014). Recently, Pauline Delahaye (2019) has contributed with a monograph on semiotic methodology for animal studies. She argues for ‘intertheoricity’ and building bigger models that are more suitable for complex subjects, counteracting the overall tendency in science towards increased specialization. Delahaye further emphasizes that zoosemiotics should aim to have an impact on methodology in ethology, and the importance of addressing ethical issues. The topic of anthropocentrism and anthropomorphism in the study of animals has recently been addressed by Jaroš and Maran (2019) in an article on different ‘Narratives of Anthropological Difference’, namely Gradualism, Transformativism, Unitarism and Pluralism. They see these various ‘[n]arrative structures […] as being rooted in ontological stances’ that are pertinent for ‘the broader philosophical, disciplinary, and methodological framework of a given theory that influences not just the way that hypotheses are formulated, but also how empirical events are interpreted’ (2019: 383). In their interpretation, Sebeok applied Pluralism in regard to both cognition and culture, but applied Transformativism in his approach to communication.9

FUTURE STUDIES In the near future, work is needed that connects semiotic studies in ethology and zoology with issues in global human ecology, develops flexible zoosemiotic tools and methodology

SEMIOTICS IN ETHOLOGY AND ZOOLOGY

83

for application by practitioners such as field ethologists, veterinarians and zookeepers, and makes further connections between semiotic studies of animals and phenomenology, ethnography, and anthropology by developing tools and methodology tailor-made for studies related to human agents and their dealings with animals. Some indications for these various prospective research avenues are given in the following.

Zoosemiotic tools and methodology for the study of animal–human relations More research is needed on how animals relate to humans. In modern science, the agency and subjectivity of animals has tended to be neglected. This has largely limited studies of animals to quantitative methods, whether in the study of animals in themselves, or in the study of how animals relate to human beings. The result is that only a limited repertoire of methods apt for quantification has been widely applied in biology at large. What zoosemiotics can offer is a flexible tool-box of qualitative methods aimed at studying semiotic phenomena in animals and humans, which allow for novel approaches to animal agency and subjectivity. With its foundation in Umwelt theory, which recognizes far more genuine subjects in nature than many other biological theories do, methods of zoosemiotics are applicable across the human/animal distinction, in the study of animal–human (and, as we shall see, human–animal) relations. Individual subjectivity and personality traits in animals – which any zookeeper is tangibly aware of – are of particular interest. As Chrulew (2020) observes, while Hediger posed ‘the important question of keeper effects – that is, how captive animals are changed by their relationships with their keepers’, such phenomena have yet ‘to be properly broached in zoo-biological research’. Chrulew rightly also remarks, on a more general note, that ‘the anthropogenic transformation of animal behaviour and environments only grows more significant today, in the face of unravelling webs of ecological meaning and of hybrid environments in which intensifying and diversifying human-animal relationships are rife with catalytic effects’. On a final note, in the last few years there has been increasing interest in a ‘One Health’ research agenda, and a related ‘One Welfare’ research agenda, both of which align well with the interdisciplinary inclinations of semiotic studies of animals in general and of human–animal relations in particular. As Day (2011) details, the idea of ‘One Health’ is not new, as comparative medicine and comparative anatomy has been practised as far back as in Ancient Greece, if not even earlier. Human and veterinary (i.e. animal) medicine saw quite some interaction in the eighteenth and nineteenth centuries, when modern veterinary medicine was in its early stages of development. In the twentieth century the two disciplines specialized and diverged (Zinsstag et al. 2005; Day 2011). In contemporary times, the ‘One Health’ agenda is seen as an extension of the US epidemiologist Calvin Schwabe’s agenda for ‘one medicine’ (Schwabe 1984), launched ‘to focus attention on the similarity between human and veterinary health interests’ (Zinsstag et al. 2005: 2142). According to Zinsstag et al. (2005: 2142), ‘the “one medicine”’, which they associate with One Health, ‘is the general science of all human and animal health and disease. It builds on a common pool of knowledge from anatomy, physiology, pathology, epidemiology, and aetiology in all species’. Approaching human and animal health in context has potential health benefits for both humans and animals. ‘Over the last 15 years’, Wondwossen et al. (2014) write, ‘our planet has faced more than 15 deadly zoonotic or vector-borne global outbreaks, both viral […] and bacterial’. A ‘“one health” system could reduce the time to detect emerging zoonoses and accelerate control and prevention’ (Zinsstag et al. 2005: 2143). Semiotic studies could contribute to this cross-species endeavour by developing a form of comprehensive medical semiotics

84

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

incorporating zoosemiotic and anthroposemiotic subject matters of health and disease (see Tønnessen, forthcoming). Day (2011) takes a broader view of One Health, stating that the One Health concept implies ‘that there should be a seamless interaction between veterinary and human medicine with clinicians, researchers, agencies and governments working together for the benefit of domestic and wild animal and human health and the global environment’ (emphasis added). The latter is sometimes referred to in terms of ‘planetary health’, which expands the health notion to also apply to ecological circumstances.10 Echoing these concerns, Wondwossen et al. (2014) point to the ‘need for One Health collaborative efforts that include the coordinated involvement of human and animal medicine as well as agriculture, wildlife, and environmental experts’. Tackling the issues addressed by such a research agenda ‘requires working across discipline […] barriers’ (2014) and aiming to redesign how science is organized, since the ‘current fragmentation of science is often incongruent with actual needs’ (Zinsstag et al. 2005: 2143). This broader conception of the One Health agenda would, on the semiotic side, require integration not only of zoosemiotic and anthroposemiotic dealings with health and disease, but furthermore integration of these with ecosemiotic work (cf. Chapter Three, this volume, ‘Semiotics in ecology and environmental studies’). As a supplement to the One Health agenda, Pinillos et al. (2016) are among those who have suggested a One Welfare research agenda which would incorporate human and animal welfare issues and concerns. They argue (2016: 412) that ‘there is a strong link between animal welfare and human wellbeing’. Citing Colonius and Earley (2013), Pinillos et al. claim that separation between the studies of human and animal welfare ‘is an artificial compartmentalization’ given that these ‘disciplines rely on the same set of scientific measures and heavily depend on each other in an ecological context’. The claim that the two disciplines ‘rely on the same set of scientific measures’ is not altogether accurate. In short, while the topic matter (i.e. welfare) might be fundamentally ‘the same’ for all sentient beings, the available methods are not, at least not as far as subjective welfare is concerned.11 Specifically, unlike qualitative methods for assessing subjective human welfare, qualitative methods for assessing subjective animal welfare cannot rely on data from the study objects themselves gathered through interviews or questionnaires. Instead, they must primarily rely on observations of the animals at issue (these may in some cases feasibly be conducted in a participatory, interactive manner that elicit responses from the animals). Here a methodological question of great importance is to what extent the subjective biology of von Uexküll can be made operational in assessing the subjectively experienced welfare of humans and animals. Umwelt-centred assessments may also incorporate data drawn from interviews with or surveys aimed at animal keepers.

Zoosemiotic tools and methodology for the study of human–animal relations More research is needed on how humans relate to animals. In anthropology and, more widely, any field of study that makes use of methods of ethnography, the objects of study are typically restricted to those that are directly related to human perceptions, attitudes and practices. By and large this also applies to descriptive phenomenology, with its roots in phenomenology and psychology, and to sociology. In cultures and societies across the world, many human perceptions, attitudes and practices involve animals, but, again, in modern science the agency and subjectivity of animals has tended to be neglected. In

SEMIOTICS IN ETHOLOGY AND ZOOLOGY

85

consequence, whereas studies of animals and animal phenomena have been dominated by quantitative approaches, both qualitative and quantitative approaches have been applied in studies of humans and human phenomena. This has resulted in one-sided descriptions of animals in human–animal relations, drawn from a hegemonic perspective of human interests. Descriptive phenomenology is widely used within the social sciences, where it represents one of the most central qualitative methods. In its current manifestation it tends to rely on specifically human forms of data such as interviews and texts, and to be limited to the study of human subjects. However, drawing on Umwelt theory, descriptive phenomenology may be practised so as to allow for qualitative studies of the lifeworlds of animals as well (see Tønnessen, forthcoming, 2022). In anthropology, ethnography, descriptive phenomenology and sociology, a semiotic approach to studies of animals drawing on the pluralistic perspective of von Uexküll can contribute to work that is non-anthropocentric, or at the very least less anthropocentric and more pluralistic than many other approaches. In this sense zoosemiotics and related approaches have some of the answers to contemporary calls for a more-than-human perspective on phenomenology (Abram 1997), an anthropology that goes beyond the human (Kohn 2013; Ingold 2013; Schroer 2019), multispecies studies (Haraway 1985, 2008, 2016) and so on. What these interdisciplinary approaches and research programs have in common is a critique of the anthropocentric bias ingrained in so many fields of study that involve humans or animals or both. This bias can be explicit (i.e. articulated) or merely implicit, and often results from drawing an artificially sharp boundary between humans and animals as objects of study, where the lives and minds of humans are believed to be accessible in ways that the lives and minds of animals are not. In the social sciences and the humanities, human beings are generally acknowledged as subjects on par with researchers [sic], but the same grace is seldom shown to animals, who are traditionally regarded as mere objects bereft of any similar kind of subjectivity or agency. Another way of expressing this is stating that in the social sciences and the humanities, researchers have tended to operate with various ingroups and outgroups, but the various ingroups have tended to consist of human beings only, and animals have tended to belong, explicitly or implicitly, to outgroups.12 In the words of anthropologist Tim Ingold (2013: 12), ‘[it] has long been customary, in the world of us and them, to refer to the former as “subjects” and to the latter as “objects”’. Bruno Latour, who has inspired Actor-Network Theory (ANT), rejects this subject/object distinction – for ‘all it does is to impose, a priori, a wholly spurious asymmetry between, on the one hand, the world of human intentional action and on the other, a material world of causal relations’ (Ingold 2013: 12–13). The traditional subject/ object distinction in science is paralleled by an equally sharp distinction society/nature. In place of such dichotomies, ‘Latour calls for a symmetrical approach which would bring both humans and non-humans to the table as transacting parties’ (2013: 13). ‘Objects and subjects’, as Latour points out, ‘can never associate with one another; humans and non-humans can’ (Latour 2004: 76, original emphasis, cited in Ingold 2013: 13). This line of thinking has had some influence on contemporary zoosemiotics. With reference to Latour’s work, Maran (2015) makes use of actor-network theory in combination with methodology from biosemiotics and cultural semiotics in a semiotic analysis of the emergence of the Golden Jackal in Estonia.

86

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

One big challenge for the artificially sharp boundary between humans and animals as objects of study, and the anthropocentric bias it results in, is the fact that the standard notion of the adult, rational human being does not apply to all human beings, nor to all life stages. For human beings that are capable of talking and writing intelligibly, interviews and various forms of text analysis represent methods that in effect give researchers access to thoughts, feelings and experiences that would otherwise be harder to study. And yet there are human beings that lack the ability to express themselves in these ways, either permanently or temporarily (in any human being’s normal development). Even so, few would say that qualitative studies of their lives are altogether untenable, since various forms of observation are possible. Why then rule out, or neglect, qualitative studies of animals? After all, some of the same methods can be applied in the study of both humans and animals. In allowing for more diverse approaches in studies of humans and animals and their interaction, the zoosemiotic research agenda could not only contribute to less anthropocentric studies of how human beings relate to animals. It could potentially also contribute to studies of humans and human phenomena that acknowledges human diversity to a fuller extent than what is the case today. Greater attentiveness to all the exceptions and nuances to the standard notion of the adult, rational human being in the semiotic study of how humans relate to animals would align well with contemporary developments in the posthumanities, gender studies, etc.13

SUMMARY AND CONCLUSION Today, zoosemiotics is arguably the theoretically and empirically soundest approach to ‘taking the animal’s perspective’.

ACKNOWLEDGEMENTS The author acknowledges his involvement in the University of Stavanger program areas for research ‘The Greenhouse: An environmental humanities initiative at University of Stavanger’ (IN-11621) and ‘Philosophy and subjectivity’ (IN-11746).

NOTES 1 ‘This branch was’, as Maran et al. (2011a: 8) observe, ‘projected, although not systematically defined, by Sebeok and by the zoologist Heini Hediger’. Cf. also Sebeok 1990b. 2 Haraway writes: “In many ways, ‘A Cyborg Manifesto’ was trying out some of the knowledges that hadn’t been mine that I was getting from my colleagues and the graduate students in the program [the History of Consciousness program at University of California Santa Cruz], and that came to be part of poststructuralism and deconstruction in various ways – some of the theories of Jakob von Uexküll and Roland Barthes and many others”. 3 For the sake of full disclosure: The author has been main Editor-in-Chief of Biosemiotics. 4 ‘Nowadays’, Maran et al. (2011b: 248) remark, ‘his works are largely discarded from mainstream semiotics because of their behaviouristic flavour, but their reinterpretation from the contemporary zoosemiotic perspective is a task still waiting to be carried out’. 5 The latter are in stead said to have Wohnhüllen (cf. von Uexküll 1940). Importantly, both Umwelten and Wohnhüllen are of a semiotic nature and involve interpretation of either ‘Merkzeichen’ (perceptual signs) and ‘Wirkzeichen’ (action signs), in Umwelten, or of

SEMIOTICS IN ETHOLOGY AND ZOOLOGY

6

7 8

9

10

11

12

13

87

‘Bedeutungsfaktoren’ (meaning factors), in the Wohnhüllen of plants and fungi. Unlike in Umwelten, however, there are no functional cycles at work in Wohnhüllen, according to von Uexküll. Note that Sebeok does not in this initial definition refer to zoology, the study of animals more broadly, even though the attribute zoo- is used in the coinage ‘zoosemiotics’. The implication of this is that Sebeok not only initially limited zoosemiotics to signalling behaviour as opposed to other kinds of animal behaviour, but furthermore limited zoosemiotics to behaviour as opposed to other aspects of animal biology. Mentioned in Maran et al. (2011a: 1) as examples of study objects of anthropological zoosemiotics. These disagreements also have implications for the conception of semiotics at large: In this contemporary zoosemiotic view, cultural semiotics should not be conceived of as being entirely distinct from the semiotics of nature (including zoosemiotics). Rather, these subfields are intertwined and cannot meaningfully be approached entirely in separation from each other. In the words of Maran et al. (2011a: 2), ‘zoosemiotics investigates a field of knowledge that includes both natural and cultural elements.’ This contrasts with Darwin’s Gradualism and Tomasello’s Transformativism in all three contexts, and de Waal’s Unitarism (with regard to cognition and culture) and Gradualism (with regard to communication). For instance, The Rockefeller Foundation-Lancet Commission on Planetary Health, in alliance with the United Nations Framework Convention on Climate Change (UNFCCC), indicates that ‘planetary health’ refers to ‘the health of our planet’ (see https://unfccc.int/ climate-action/momentum-for-change/planetary-health). Some would say that this ecological conception of ‘health’ is metaphorical, while reference to human and animal health stands on more solid ground. It should also be noted that human and animal welfare diverge in some respects, in that some needs and abilities that contribute to high welfare (such as use of written language, spirituality, political discourse) are genuinely and uniquely human. In a similar manner, some animals have needs and abilities that humans do not have. Of course, the outgroups have also tended to include some humans, with an overall emphasis on the division of human beings into ingroups and outgroups (which makes sense given the anthropocentric bias). It would also align well with political philosopher Will Kymlicka’s (2018) argument for a non-anthropocentric foundation for human rights built on what unites us with other sentient beings rather than on what separates us from them.

REFERENCES Abram, D. (1997), The Spell of the Sensuous. Perception and Language in a More-Than-Human World, New York: Vintage Books. Allen, C. (2014), ‘Umwelt or Umwelten? How Should Shared Representation Be Understood Given such Diversity?’ Semiotica, 198: 137–58. Barbieri, M. (2003), The Organic Codes: An Introduction to Semantic Biology, Cambridge: Cambridge University Press. Barbieri, M. (2015), Code Biology: A New Science of Life, New York: Springer. Brentari, C. (2015), Jakob von Uexküll: The Discovery of the Umwelt between Biosemiotics and Theoretical Biology (Biosemiotics 9), Cham: Springer. Buchanan, B. (2008), Onto-Ethologies: The Animal Environments of Uexküll, Heidegger, Merleau-Ponty, and Deleuze, New York: SUNY Press.

88

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Burghardt, G. M. (2008), ‘Updating von Uexküll: New Directions in Communication Research’, Journal of Comparative Psychology, 122 (3): 332–3. Cerrone, M. (2018), ‘Umwelt and Ape Language Experiments: On the Role of Iconicity in the Human-Ape Pidgin Language’, Biosemiotics, 11 (1): 41–63. Chrulew, M. (2014), ‘The Philosophical Ethology of Dominique Lestel’, Angelaki: Journal of the Theoretical Humanities, 19 (3): 17–44. Chrulew, M. (2020), ‘Reconstructing the Worlds of Wildlife: Uexküll, Hediger, and Beyond’, Biosemiotics, 13 (1): 137–49. Colonius, T. J. and R. W. Earley (2013), ‘One Welfare: A Call to Develop a Broader Framework of Thought and Action’, Journal of the American Veterinary Medical Association, 242: 309–10. Day, M. J. (2011), ‘One Health: The Importance of Companion Animal Vector-borne Diseases’, Parasites & Vectors, 4: 49. Deely, J. (2010), Semiotic Animal: A Postmodern Definition of ‘Human Being’ Transcending Patriarchy and Feminism, South Bend: St. Augustine’s Press. Delahaye, P. (2019), A Semiotic Methodology for Animal Studies (Biosemiotics 19), Cham: Springer. Deleuze, G. and F. Guattari (1987), A Thousand Plateaus: Capitalism and Schizophrenia, trans. Brian Massumi Minneapolis: University of Minnesota Press. Farina, A. (2012), ‘A Biosemiotic Perspective of the Resource Criterion: Toward a General Theory of Resources’, Biosemiotics, 5 (1): 17–32. Favareau, D., ed. (2010), Essential Readings in Biosemiotics: Anthology and Commentary, Dordrecht: Springer. Haraway, D. J. (1985), ‘Manifesto for Cyborgs: Science, Technology, and Socialist Feminism in the 1980s’, Socialist Review, 80: 65–108. Haraway, D. J. (2008), When Species Meet, Minneapolis: University of Minnesota Press. Haraway, D. J. (2016), Manifestly Haraway: The Cyborg Manifesto. The Companion Species Manifesto. Companions in Conversations (with Cary Wolfe), Minneapolis: University of Minnesota Press. Hediger, H. (1964), Wild Animals in Captivity: An Outline of the Biology of Zoological Gardens, trans. G. Sircom, New York: Dover. Hediger, H. (1980), Tiere Verstehen: Erkenntnisse eines Tierpsychologen, Munich: Kindler Verlag. Heidegger, M. (1995), The Fundamental Concepts of Metaphysics: World, Finitude, Solitude, Bloomington: Indiana University Press. Hoffmeyer, J. (1996), Signs of Meaning in the Universe, Bloomington: Indiana University Press. Hoffmeyer, J. (2009), Biosemiotics: An Examination into the Signs of Life and the Life of Signs, Scranton: University of Scranton Press. Ingold, T. (2013), ‘Anthropology beyond Humanity’, Suomen Antropologi: Journal of the Finnish Anthropological Society, 38 (3): 5–23. Jaroš, F. (2016), ‘Cats and Human Societies: A World of Interspecific Interaction and Interpretation’, Biosemiotics, 9 (2): 287–306. Jaroš, F. and T. Maran (2019), ‘Humans on Top, Humans among the Other Animals: Narratives of Anthropological Difference’, Biosemiotics, 12 (3): 381–403. Kant, I. ([1787] 1996), Critique of Pure Reason, trans. Walter Pluhar, Indianapolis: Hackett Publishing. Kleisner, K. (2008), ‘The Semantic Morphology of Adolf Portmann: A Starting Point for the Biosemiotics of Organic Form?’ Biosemiotics, 1 (2): 207–19.

SEMIOTICS IN ETHOLOGY AND ZOOLOGY

89

Kohn, E. (2013), How Forests Think: Toward an Anthropology beyond the Human, Berkeley: University of California Press. Kull, K. (2001), ‘Jakob von Uexküll: An Introduction’, Semiotica, 134 (1/4): 1–59. Kull, K. (2009), ‘Vegetative, Animal, and Cultural Semiosis: The Semiotic Threshold Zones’, Cognitive Semiotics, 4 (Spring): 8–27. Kull, K. (2014), ‘Zoosemiotics Is the Study of Animal Forms of Knowing’, Semiotica, 198: 47–60. Kull, K. (2016), ‘Need for Impressions: Zoosemiotics and Zoosemiotics, by Aleksei Turovski’, Sign Systems Studies, 44 (3): 456–62. Kymlicka, W. (2018), ‘Human Rights without Human Supremacism’, Canadian Journal of Philosophy, 48 (5): 763–92 Latour, B. (2004), Politics of Nature: How to Bring the Sciences into Democracy, Cambridge, MA: Harvard University Press. Lestel, D. (2002), ‘The Biosemiotics and Phylogenesis of Culture’, Social Science Information, 41 (1): 35–68. N. Scott (trans). Reprinted in Maran, Martinelli and Turovski (eds) 2011, 377–409. Lestel, D. (2011), ‘What Capabilities for the Animal?’ Biosemiotics, 4 (1): 83–102. Lestel, D., F. Brunois, and F. Gaunet (2006), ‘Etho-ethnology and Ethno-ethology’, Social Sciences Information, 45: 155–77. Lorenz, K. (1935), ‘Der Kumpan in der Umwelt des Vogels’, Journal für Ornithologie, 83: 137–213. Lorenz, K. ([1949] 1961), King Solomon’s Ring, trans. Marjorie Kerr Wilson, London: Methuen. Mäekivi, N. (2016), ‘Communication in the Study of Zoological Gardens’, in T. Maran, M. Tønnessen, K. Armstrong Oma, L. Kiiroja, R. Magnus, N. Mäekivi, S. Rattasepp, P. Thibault, and K. Tüür, Animal Umwelten in a Changing World – Zoosemiotic Perspectives (Tartu Semiotics Library 18), 204–21, Tartu: Tartu University Press. Mäekivi, N. (2018), ‘Freedom in Captivity: Managing Zoo Animals According to the “Five Freedoms”’, Biosemiotics, 11 (1): 7–25. Magnus, R. (2016), ‘The Semiotic Challenges of Guide Dog Teams: The Experiences of German, Estonian and Swedish Guide Dog Users’, Biosemiotics, 9 (2): 267–85. Maran, T. (2010), ‘Why Was Thomas A. Sebeok Not a Cognitive Ethologist? From “Animal Mind” to “Semiotic Self”’, Biosemiotics, 3 (3): 315–29. Maran, T. (2014), ‘Dimensions of Zoosemiotics: Introduction’, Semiotica, 198: 1–10. Maran, T. (2015), ‘Emergence of the “Howling Foxes”: A Semiotic Analysis of Initial Interpretations of the Golden Jackal (Canis aureus) in Estonia’, Biosemiotics, 8 (3): 463–82. Maran, T., D. Martinelli, and A. Turovski (2011a), ‘Introduction’, in Maran, Martinelli and Turovski (eds), Readings in Zoosemiotics, 1–20, Berlin: DeGruyter Mouton. Maran, T., D. Martinelli, and A. Turovski, eds (2011b), Readings in Zoosemiotics (Semiotics, Communication and Cognition 8), Berlin: DeGruyter Mouton. Maran, T., M. Tønnessen, K. Armstrong Oma, L. Kiiroja, R. Magnus, N. Mäekivi, S. Rattasepp, P. Thibault, and K. Tüür (2016), Animal Umwelten in a Changing World – Zoosemiotic Perspectives (Tartu Semiotics Library 18), Tartu: Tartu University Press. Maran, T., M. Tønnessen, R. Magnus, N. Mäekivi, S. Rattasepp, and K. Tüür (2016), ‘Introducing Zoosemiotics: Philosophy and Historical Background’, in T. Maran, M. Tønnessen, K. Armstrong Oma, L. Kiiroja, R. Magnus, N. Mäekivi, S. Rattasepp, P. Thibault, and K. Tüür, Animal Umwelten in a Changing World – Zoosemiotic Perspectives (Tartu Semiotics Library 18), 1–20, Tartu: Tartu University Press.

90

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Marrone, G. and D. Mangano, eds (2018), Semiotics of Animals in Culture: Zoosemiotics 2.0 (Biosemiotics 17), Cham: Springer. Martinelli, D. (2007), Zoosemiotics: Proposals for a Handbook (Acta Semiotica Fennica 26), Imatra and Helsinki: Finnish University Network of Semiotics; International Semiotics Institute; Semiotic Society of Finland. Martinelli, D. (2009), Of Birds, Whales and Other Musicians: An Introduction to Zoomusicology, Scranton: University of Scranton Press. Martinelli, D. (2010), A Critical Companion to Zoosemiotics: People, Paths, Ideas (Biosemiotics 5), Berlin: Springer. Merleau-Monty, M. (2003), Nature, compiled by Dominique Séglard; trans. Robert Vallier, Evanston: Northwestern University Press. Morris, C. W. (1946), Signs, Language and Behavior, New York: Prentice Hall. Nöth, W. (1990), Handbook of Semiotics, Bloomington and Indianapolis: Indiana University Press. Pinillos, R. G., M. Appleby, X. Manteca, F. Scott-Park, C. Smith, and A. Velarde (2016), ‘One Welfare – a Platform for Improving Human and Animal Welfare’, Veterinary Record, 179: 412–13. Ribó, I. (2019), ‘Dialogical Communicative Interaction between Humans and Elephants: An Experiment in Semiotic Alignment’, Biosemiotics, 12 (2): 305–27. Rüting, T. (2004), ‘History and Significance of Jakob von Uexküll and of His Institute in Hamburg’, Sign Systems Studies, 32 (1–2): 35–72. Schwabe, C. (1984), Veterinary Medicine and Human Health, Baltimore: Williams and Wilkins. Sebeok, T. A. (1963), ‘Review’, Language, 39: 448–66. Sebeok, T. A. (1965), ‘Animal Communication’, Science, 147 (3661): 1006–14. Sebeok, T. A. (1972a), Perspectives in Zoosemiotics (Janua Linguarum, Series Minor 122), The Hague: Mouton de Gruyter. Sebeok, T. A. ([1972b] 2011), ‘The Word “Zoosemiotics”‘, in Sebeok 1972a: 178–81, reprinted in T. Maran, D. Martinelli and A. Turovski (eds), Readings in Zoosemiotics (Semiotics, Communication and Cognition 8), 95–8, Berlin: DeGruyter Mouton. Sebeok, T. A. (1990a), Essays in Zoosemiotics (Monograph Series of the TSC 5), Toronto: Toronto Semiotic Circle; Victoria College at the University of Toronto. Sebeok, T. A. ([1990b] 2011), ‘Zoosemiotics: At the Intersection of Nature and Culture’, in Sebeok (1990a), 37–47, reprinted in T. Maran, D. Martinelli and A. Turovski (eds), Readings in Zoosemiotics (Semiotics, Communication and Cognition 8), 77–86, Berlin: DeGruyter Mouton. Sebeok, T. A. ([1990c] 2011), ‘Talking’ with Animals: Zoosemiotics Explained, in Sebeok, 105–13, reprinted in T. Maran, D. Martinelli and A. Turovski (eds), Readings in Zoosemiotics (Semiotics, Communication and Cognition 8), 87–94, Berlin: DeGruyter Mouton. Sebeok, T. A. (2001), ‘Biosemiotics: Its Roots, Proliferation, and Prospects’, Semiotica, 134 (1/4): 61–78, reprinted with commentary in Donald Favareau (ed.) (2010): 221–36. Schroer, S. A. (2021), ‘Jakob von Uexküll: The Concept of Umwelt and Its Potentials for an Anthropology beyond the Human’, Ethnos, 86 (1): 132–52. Snow, C. P. (2001 [1959]), The Two Cultures, London: Cambridge University Press. Sonesson, G. (2006), ‘The Meaning of Meaning in Biology and Cognitive Science: A Semiotic Reconstruction’, Sign Systems Studies, 34 (1): 135–213. Tønnessen, M. (2009), ‘Umwelt Transitions: Uexküll and Environmental Change’, Biosemiotics, 2: 47–64.

SEMIOTICS IN ETHOLOGY AND ZOOLOGY

91

Tønnessen, M., C. Brentari, and R. Magnus (2016), ‘The Biosemiotic Glossary Project: Umwelt’, Biosemiotics, 9 (1): 129–49, with Appendix (supplementary material available online): https://link.springer.com/article/10.1007/s12304-016-9255-6. Tønnessen, Morten, forthcoming. Nosology and Semiotics. In Carlo Guido Musso & Adrian Covic (eds), Organ Crosstalk in Acute Kidney Injury: Basic Concepts and Clinical Practices. Springer Nature. Tønnessen, Morten, forthcoming, 2022. Umwelt Theory for Practitioners: Semiotic Guidelines for Application in a More-Than-Human Descriptive Phenomenology. In: Amir Biglari (ed), Open Semiotics. Paris: L’Harmattan. Tüür, K. and M. Tønnessen, eds (2014), The Semiotics of Animal Representations (Nature, Culture and Literature 10), Amsterdam: Rodopi. Uexküll, J. v. (1909), Umwelt und Innenwelt der Tiere, 1st edn, Berlin: Verlag von Julius Springer. Uexküll, J. v. (1921), Umwelt und Innenwelt der Tiere, 2nd edn, Berlin: Verlag von Julius Springer. Uexküll, J. v. (1928), Theoretische Biologie, 2nd edn, Berlin: Verlag von Julius Springer. Uexküll, J. v. (1934), Streifzüge durch die Umwelten von Tieren und Menschen: Ein Bilderbuch unsichtbarer Welten, Hamburg: Rowohlt. Reprinted in J. v. Uexküll and G. Kriszat (illustrations) ([1934/40] 1956/2010), A Foray into the Worlds of Animals and Humans – With a Theory of Meaning (Posthumanities 12), trans. J. D. O’Neil, Minneapolis: University of Minnesota Press. Uexküll, J. v. (1940), Bedeutungslehre (= Bios, Abhandlungen zur theoretischen Biologie und ihrer Geschichte sowie zur Philosophie der organischen Naturwissenschaften Bd. 10), Leipzig: Verlag von J. A. Barth. Uexküll, J. v. and G. Kriszat (illustrations) ([1934/1940]1956), Streifzüge durch die Umwelten von Tieren und Menschen: Ein Bilderbuch unsichtbarer Welten, Bedeutungslehre, Hamburg: Rowohlt. Uexküll, J. v. ([1934/40] 2010), A Foray into the Worlds of Animals and Humans – With a Theory of Meaning (Posthumanities 12), trans. J. D. O’Neil, Minneapolis: University of Minnesota Press. Wondwossen, A., J. Dupouy-Camet, M. J. Newport, C. J. B. Oliveira, L. S. Schlesinger, Y. M. Saif, S. Kariuki, L. J. Saif, W. Saville, T. Wittum, A. Hoet, S. Quessy, R. Kazwala, B. Tekola, T. Shryock, M. Bisesi, P. Patchanee, S. Boonmar, and L. J. King (2014), ‘The Global One Health Paradigm: Challenges and Opportunities for Tackling Infectious Diseases at the Human, Animal, and Environment Interface in Low-Resource Settings’, PLOS Neglected Tropical Diseases, 8 (11): e3257. Zinsstag, J., E. Schelling, K. Wyss, and M. B. Mahamat (2005), ‘Potential of Cooperation Between Human and Animal Health to Strengthen Health Systems’, Lancet, 366: 2142–5. Zlatev, J. (2009), ‘The Semiotic Hierarchy: Life, Consciousness, Signs and Language’, Cognitive Semiotics, 4: 169–200.

92

CHAPTER FIVE

Semiotics in Evolutionary Linguistics JAMIN PELKEY AND PRISCA AUGUSTYN

INTRODUCTION We share a biological world – and a deep evolutionary history – with hundreds of thousands of other species whose features and behaviours are in many ways similar to our own.1 And yet in spite of our affinities with other animals, as Terrence Deacon observes, ‘we also live in a world that no other species has access to’ (1997: 21): ‘We inhabit a world full of abstractions, impossibilities, and paradoxes’, a world in which we can ponder our existence, recognize our ignorance, adopt new beliefs, think of things in terms of other things, reflect on how things might have been and could yet be. It is a realm of stories in which we reconstruct past experiences and invent new experiences. ‘We even make use of these stories to organize our lives. In a real sense, we live our lives in this shared virtual world’ (Deacon 1997: 21–2). The nature and origins of this virtual world and our shared ability to create it and inhabit it are still poorly understood. The closest we have come to anything approaching consensus on the matter is the insight that this ability has something to do with what we call ‘language’ (see, e.g., Sebeok 1987, Fauconnier and Turner 2002, Harari 2014, Tattersall 2018).2 The origins of language and the dynamics of language change are the concerns of the field of evolutionary linguistics. The nature of language and its relationships with other sign systems are the concerns of semiotics. This chapter explores the interface of semiotics and evolutionary linguistics in terms of the current status of their dialogue, along with future priorities and possibilities for ongoing collaboration between the two. This is accomplished in eight sections. Following this introductory section, we offer a brief history of evolutionary approaches to the study of language. This is followed by a discussion of semiotic contributions to the field of historical and comparative linguistics. In the fourth section, we discuss the role of semiotics in understanding the origins of language. This is followed by a discussion of specific insights from biosemiotics applied to linguistic semiosis, with a focus on modelling systems or ‘Umwelt’ cognition (Uexküll 1920; Kull 2010). We then provide a comparative discussion of linguistic semiosis and animal communication before turning to a discussion of the relationships between semiotics and biolinguistics. The chapter then concludes with a summary of progress, problems and priorities for the study of semiosis in language evolution. If the field has any shape of its own, this is initially due to Ferdinand de Saussure’s influential suggestion that ‘evolutionary linguistics’ should function in opposition to ‘static

94

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

linguistics’ as two distinct sciences of language (1916: 79–100). According to Saussure, the evolutionary versus static opposition maps onto diachronic linguistics (involving evolutionary, time-based analyses) and synchronic linguistics (static, timeless analyses), respectively (1916: 81), with linguistic synchrony receiving preferential treatment. This pivotal set of suggestions led to a disciplinary revolution that effectively swept aside several centuries of focus on process-oriented phenomena in the language sciences, in favour of a new focus on the timeless semantic codes and syntactic relations of linguistic ‘synchrony’ – a paradigm shift that held sway throughout the twentieth century. Whether or not the distinction itself was actually necessary or valid to begin with is a question that has received little attention (cf. Anttila 1989; Pelkey 2011, 2019). Regardless, around the turn of the twenty-first century, disciplinary attention in the language sciences slowly began to shift once again to time-based analyses, due especially to cross-disciplinary questions surrounding the nature of human culture, cognition and pre-history (see, e.g., Croft 2000; Christiansen and Kirby 2003; Evans and Levinson 2009, Nölle, Hartmann, and Tinits 2020). Notably, one of the most influential works responsible for instigating this shift was Terrence Deacon’s Symbolic Species (1997), a study written from the point of view of semiotics – or at least heavily influenced by the semiotic thinking of Charles S. Peirce. We return to Deacon’s semiotic influence on the field in a later section. First it is important to better orient our discussion to the field itself. Broadly conceived, the contemporary study of language evolution encompasses at least six potential foci, all of which are interrelated: 1. Language ontology 2. Language origins 3. Language acquisition 4. Language variation 5. Language history 6. Language process This chapter is primarily concerned with the first two and last two items in this list (1, 2, 5, 6), though items 3 and 4 remain highly relevant for more in-depth reflection. Research on language ontology (i.e. the meaning or definition of ‘language’, especially concerned with its situated status and distinctive nature) is perhaps the most neglected in linguistic approaches to language evolution, and it is here that semiotics stands to make some of its most important contributions to the field. There is a widespread tendency to assume that questions about the nature or definition of language are already settled. As a result, the meaning of ‘language’ is usually either conflated with verbal (and signed) language use in socio-cultural communication contexts, or, alternatively, it is conflated with the cognitive architecture that informs this specific ability. The former is the usual assumption of researchers in the newly emergent sub-field of ‘language evolution’ studies, while the latter is the usual assumption of researchers in the field of biolinguistics. According to Steels (2011), biolinguists tend to assume that ‘there is a strong biological determination of the structure of language’ while language evolution researchers (who Steels frames as ‘evolutionary linguists’) tend to assume that language is primarily shaped by cultural forces. Naturally, researchers in both camps, as well as those who straddle the two, are often willing to admit the necessity of biology and culture working in tandem through processes of co-evolution in the emergence of language. But this brings up

SEMIOTICS IN EVOLUTIONARY LINGUISTICS

95

numerous issues, conceptual/presuppositional problems in particular; so before going further down this path, we turn to a more detailed historical summary of the field, preand post-Saussure, noting relationships with semiotics in the process.

EVOLUTIONARY LINGUISTICS: HISTORY AND SEMIOSIS Until the late nineteenth century, the scientific study of language was known as ‘philology’. Philologists wrote grammars, compiled dictionaries, carried out etymological research and critical textual research, and undertook historical-comparative research focused on linguistic relationships for purposes of language classification, proto-language reconstruction and other modes of language comparison (such as deciphering hieroglyphs and runes). Although philologists usually relied on written documents as their primary sources of evidence, their rigorous methodologies (most notably the comparative method) and dramatic findings (most notably Proto-Indo-European) put the study of language at the forefront of scientific inquiry in general from the 1780s to the 1860s (Greenberg 1957; McMahon 1994; Alter 1999; Wyhe 2005; Atkinson and Gray 2005). Joseph Greenberg (1957) even argues that William Jones’s ([1788] 1824) discovery and reconstruction of Indo-European makes him the original discoverer of evolution.3 The implications of this claim, their history and veracity, are the focus of this section.

The discovery of evolution Greenberg’s proposal is noteworthy for at least two reasons: first of all, because the history of dialogue between linguistics and biology through most of the nineteenth century actually bears this out – in spite of contemporary assumptions to the contrary – and, second, because this is a decidedly semiotic hypothesis: i.e. a domain-general proposal on the nature of evolution, potentially constituting evidence for a relationship of identity between ‘evolution’ and ‘semiosis’ (see Peirce 1890–1892; Deely 2009: 98, 165; Pelkey 2015). As to the first point, in fact, leading linguists of Darwin’s day such as August Schleicher and Max Müller argued forcefully that philologists were the first to discover evolution (see Schleicher [1869] 1983: 32–5; Müller 1887: xi); and even Darwin himself admits that language diversification and biological speciation are ‘curiously parallel’ (1882: 90). As Atkinson and Gray point out, ‘phylogenetic understanding and methodology in linguistics had already developed rapidly before Darwin, and this continued throughout the nineteenth century’ (2005: 517). Fitch also notes that Darwin drew on the theory and methodology of historical philology (including everything from the comparative method to tree diagrams) as a major source of inspiration (Fitch 2008: 373). Indeed, it is ironic that we have come to think of the life sciences as the only true domain for evolution (language ‘evolution’ and cultural ‘evolution’ somehow being merely a dim analogy); when, in fact, at a conceptual level historically, the situation worked quite the other way around. Naturally, at the (pre)historic level, biological evolution has priority; but this does not mean the evolution of language is somehow artificial; nor does it preclude the possibility that the same (more general) processes might somehow be at work in both. This, in turn, brings us back around to the second reason Greenberg’s proposal is noteworthy: i.e. the potentially domain-general nature of evolution. C. S. Peirce (1890–1892) argues that evolution takes place in three intertwining modes: Chance, Law and Habit Taking (also Tychasm, Anancasm and Agapasm), the third of which tends to

96

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

be neglected in mainstream theories of evolution to this day. Building on the semiotic philosophy of C. S. Peirce, John Deely (2009, 2016) has argued that this third mode corresponds with semiosis itself: i.e. a ‘vis a prospecto’ [‘future-oriented’] mode of causation, working in tandem with the other two modes of evolution that typify standard biological accounts: i.e. chance genetic mutations and the mechanical causality involved in copying genetic codes. As Pelkey argues (2015), this third mode of evolution is not actually foreign to biological theory. Rather it shows up under rubrics often considered to be marginal to evolution proper, in processes known as ‘ontogeny’,4 and in ‘evo-devo’ theories of development. From a semiotic perspective, however, instead of being marginal, this third mode of evolution should be understood as central (Pelkey 2015, 2019).

Peircean evolutionary linguistics Arguably, linguistic evolution also proceeds by the ongoing interaction of the same three modes of evolution identified by Peirce, with processes of variation-based analogy corresponding to Peircean ‘chance’, processes of automated chunking and the maintenance of inherited linguistic constructions corresponding to Peircean ‘law’ and processes of future-oriented pattern solving corresponding to Peircean ‘habit taking’; or, to put it more succinctly, ‘analogy’, ‘automation’ and ‘diagrammatization’ (Pelkey 2013a, 2015, 2019). These proposals are made on the basis of empirical evidence analysing comparative data from dozens of languages through a cross-theoretical lens drawing on advances in historical linguistics, cognitive linguistics and Peircean semiotics. While further work is necessary to bear these relationships out and to refine them theoretically, critically and empirically, it is important for the present discussion to note that the third mode of evolution under consideration in these accounts is drawn from the work of Michael Shapiro (1991, 2002; building on Anttila 1989), a linguist who argues that the notion of Peircean ‘diagrams’ and ‘diagrammatization’ are crucial for the future of linguistic theory. Diagrams are networks of patterned resemblance (or ‘iconicity’) between two domains of inquiry that serve as an aid to thought by representing ‘the relations […] of the parts of one thing by analogous relations in their own parts’ (1902: CP 2.277). Diagrammatization involves the incremental modification or improvement of diagrams towards better fit between parts and wholes internally or between two or more domains of inquiry. In language evolution, diagrammatization is a ‘process by which unconformities in languages are reduced or eliminated over time’ (Shapiro 2002: 118) as a community of speakers works unwittingly to better organize and modify linguistic phenomena ranging from semantic relations to grammatical paradigms via ‘the collective legitimation of fluctuations’ (Shapiro 1991: 5). Notably, linguistic diagrams and diagrammatization are ‘panchronic signs’, subtending ‘both linguistic synchrony and linguistic diachrony’; as such, they are ‘states in synchrony and real tendencies in diachrony’ (Shapiro 2002: 118), thus delegitimising any strict dichotomy between the two. Winfried Nöth (2008) concurs that diagrammatic relations should be given pride of place in the future of linguistic theory. From a Peircean perspective, language is not merely a collection of symbols, and it is certainly not a system composed primarily of arbitrary relations. Instead, Nöth argues that Peirce’s position on language corresponds with a complex system that functions in the aggregate as a ‘rhematic, iconic Legisign’: i.e. a vague diagram type that can only be realized via ‘iconic Sinsigns’ (i.e. replicas, or diagram tokens) (Peirce 1903: CP 2.246; Nöth 2008: 94).

SEMIOTICS IN EVOLUTIONARY LINGUISTICS

97

Contemporary language evolution research Notably in this connection, researchers in the contemporary ‘language evolution’ movement introduced above share an appreciation for the pervasively iconic nature of linguistic systems (following up on Jakobson’s indictment of Saussure’s ‘dogma of arbitrariness’)5 but have so far neglected to rally around the more overarching importance that diagrammatic iconicity and diagrammatization play in the organization and evolution of language,6 especially when considered in a broader theory of domain-general evolution by which the chance guesswork of analogy (a la Anttila 1994, 2003) and the automated chunking of repeated constructions (a la Bybee 2010) are mediated by future-oriented pattern solving according to goodness of fit between parts and wholes in language change (as introduced above). On the other hand, the robust applications of mixed-methodologies, cross-disciplinary research and multimodal attention that mark the contemporary language evolution movement are implicitly semiotic in themselves, often making potentially important contributions to semiotic theory and methodology in their own right in addition to working overtly to develop the ‘experimental semiotics’ approach of Galantucci and Garrod (2011) in the process (see also Roberts and Sneller 2020). Semioticians in search of better models and relevant empirical findings for theorizing the nature of human meaning construal at the intersection of language, culture and cognition would do well to learn from, and interact with, the movement. Among other means, this can be accomplished by learning from regular contributions to three dedicated journals and two biennial conference series. See Brill’s Language Dynamics and Change (since 2011), Oxford’s Journal of Language Evolution (since 2016), Benjamin’s Evolutionary Linguistic Theory (since 2019). Also of note are EvoLang and Protolang: two international conference series devoted to the study of language origins and language evolution.7

SEMIOTICS IN HISTORICAL AND COMPARATIVE LINGUISTICS Following the Darwinian revolution, which led to the ascendancy of biology over linguistics in the sciences, and the Saussurean revolution, which led to the ascendancy of synchrony over diachrony in linguistics, historical-comparative approaches to linguistic analysis faded into the background. But in spite of the ensuing mainstream neglect of historical-comparative approaches to linguistic analysis for most of the twentieth century, time-based analyses continued to be maintained; and semiotic interventions into the theory and methods of historical linguistics made important headway. Irmengard Rauch (1981) is perhaps the first to catalogue these emerging contributions in response to a claim by Akmajian, Demers and Harnish that ‘we currently have little idea what causes language change’ (1979: 216). Rauch responds by drawing attention to studies on the role of diagrammatic iconicity in language organization and change in the work of Shapiro (1980) and Haiman (1980). She then draws attention to abductive analogy (i.e. analogical guesswork) as a factor in language change in the work of Anttila (1972) and Andersen (1973). All four of these studies develop ideas from the pragmatist semiotics of C. S. Peirce; and in the coming two decades, detailed applications of Peircean semiotic for the explanation and analysis of language change would go on to be most attentively developed in the successive works of Raimo Anttila (e.g. 1989, 1992, 1994, 2003) and Michael Shapiro (e.g. 1985, 1987, 1991, 2002).8

98

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

In addition to Shapiro’s development of the theory of diagrammatization in language change, introduced in the previous section, he applies Peirce’s theory of the ‘interpretant’ (the meaningful local potential or constraints that enable a system or an individual to act on or interpret a sign relation in some particular way) as an explanatory upgrade for markedness theory in linguistics, with insightful implications for the theory and analysis of language change. Shapiro’s ideas on markedness and the interpretant work in tandem with his ideas on diagrammatization to suggest a plausible, secular theory of end-directed causation in language change. As for Anttila’s semiotic contributions to the field, his innovative textbook Historical and Comparative Linguistics (published in two editions, in 1972 and 1989, with a successive reprinting in 2009) is suffused with and guided by Peircean semiotic. He works from the premise that a robust understanding of the linguistic sign is necessary for adequate theoretical explanation and practical understanding of both language change and linguistic reconstruction. Anttila precedes Shapiro in developing Jakobson’s claim that the relational nature of language is iconic and diagrammatic at every level. These diagrammatic relations provide patterns for the analogical guesswork or ‘abduction’ of individual speakers and groups of speakers that help drive language change towards specific ends (although the actual triadic process is necessarily more complicated: see Pelkey 2019). Anttila’s semiotic approach to historical linguistics concludes that final causation should be readmitted into theories of language change and linguistic reconstruction. Also of note, for Anttila, the theory, methodology and object of historical linguistic research are congruent: all being guided by the search for collateral evidence in the mode of pragmatist semiotic inquiry (1994; see also Pelkey 2014): i.e. involving analogical guesswork according to a pattern, working to find better fit between parts and wholes with reference to a more general type. For this reason, Anttila keeps analogy at the forefront of his approach to historical linguistic theory and methodology, even arguing that it is ‘the backbone of universal grammar’ (2003: 439). And, while proponents of universal grammar do not necessarily subscribe to this view, the general (and provocative) point Anttila is making has a surprising degree of purchase across a range of theoretical perspectives, a point that is better situated within a discussion of questions on the nature of language itself or language ‘ontology’ to which we turn in the next section.

SEMIOTICS AND THE ORIGINS OF LANGUAGE To what degree do the study of language evolution and the study of language change share the same object? Put differently, ‘did language evolve through language change?’ Mendívil-Giró, who frames the question this way, answers ‘no’ (2019); and while others beg to differ (see, e.g., Bybee 2010, Hartmann 2020); a necessarily prior question is easy to miss (or dismiss) in the debate: i.e. what do we mean by ‘language’? As Jackendoff notes, ‘your theory of language evolution depends on your theory of language’ (2010). To understand the origins and evolution of language or the nature of language change, it is first necessary to clarify the concept of ‘language’ itself. Although the problem is fraught (and impossible to settle here) it is on this point that semiotics stands to make yet another lasting contribution to the field. With that in mind, we turn to the question of language ontology in order to better approach questions of language origins, pointing to ways that semiotic interventions can help clarify ideas and move the field forward.

SEMIOTICS IN EVOLUTIONARY LINGUISTICS

99

Clarifying concepts: Sebeok on language ontology According to Peirce, pragmatist semiotics is primarily concerned with the clarification of concepts (1903: CP 5.212). Working in this vein, Thomas A. Sebeok argues that it is crucial to distinguish between ‘language’, ‘verbal communication’ (a.k.a. ‘speech’) and ‘non-verbal communication’ (1986a, 2001d), noting that human beings are capable of all three, while other species are capable only of non-verbal communication. Nonverbal communication includes a vast array of potential options ranging from vocal communication, acoustic communication, kinesics and gesture, to olfactics, haptics, chronemics and the like. Verbal communication, by contrast, is the human-specific conventional mode of linguistic exchange (whether by means of speech, signed language or script) that is most often discussed as ‘language’ in the literature. But since all three of Sebeok’s distinctions are often discussed as ‘language’ in English – in addition to other senses, including ‘language’ varieties (French, Indonesian, Basque, etc.) and ‘language’ styles (formal, informal, crude, etc.) – it is important to clarify the idea of ‘language’ critically and conceptually when attempting to carry out analysis or theory building on the topic with any degree of precision or shared understanding – especially when researching the origins of ‘language’. Whatever language is, for example, did it originally emerge for purposes of verbal communication? This is a forgone conclusion in many circles. For researchers in the ‘language evolution’ movement introduced above, language is the capability for socioculturally embedded verbal communication. For researchers in biolinguistics (discussed further below), language is the biologically situated capacity for verbal communication known as ‘language competence’. Both groups tend to defer the question of language ontology since the status of the capability or capacity in question is focused in the end on verbal performance or the cognitive mechanism and underlying structures that enable verbal performance. Sebeok argues that verbal communication was rather an exaptation of language  – radically separating verbal (spoken/signed) communication from any other mode of communication in the biological realm. This would mean that language itself was not originally dedicated to communication at all (see also Cobley 2016: 29–44); it would also mean that verbal (spoken/signed) communication did not develop out of non-verbal modes of communication. As Bateson (1968) has pointed out, the evolution of speech/sign from other modes of animal communication would imply the relative attrition of nonverbal modes; but, on the contrary, human non-verbal communication has since evolved to be far more complex than anything observed in the rest of the animal kingdom. What then is ‘language’ – i.e. the capacity itself apart from its communicative potential?

Language, reflexivity and virtual worlds Sebeok answers that it is a unique species-specific ‘modeling system’9 that not only echoes aspects ‘of “what is out there” but can, additionally, [enable us to] dream up a potentially infinite number of possible worlds’ (1987: 347). This brings us back to the reflection with which this chapter began – Deacon’s virtual realm ‘full of abstractions, impossibilities, and paradoxes’ (1997: 21). Such feats of modelling are not primarily concerned with enhanced social communication or efficient information transfer. They are rather concerned with the fruits of reflexive consciousness: i.e. the creation of virtual worlds that offer new and unlimited meaning potential (including the possibility for new beliefs

100

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

and the ability to doubt these same beliefs). The augmentation of reason, imagination, self-control and creativity that resulted from this shift in consciousness had profound consequences not only for Homo sapiens but also for the cognitive abilities of all earlier species in the genus Homo, according to Sebeok (1986a, 1986b, 1987). Although Sebeok does not register consonance with C. S. Peirce on these points, there is much agreement between the two regarding the root sense of language. Peirce argues that the human ‘faculty of language’ is ‘a phenomenon of self-control’ that enables us to ‘criticize our thought logically’ due to the uniquely human ability to think of signs as signs (c. 1905: CP 5.534; see also Rauch 1999: 64–5, Deely 2010). For Peirce, then, human language is a special mode of versatile, reflexive thought. Peirce reiterates, in the same passage, that ‘all thinking is by signs’, including symbolic signs; but this does not mean that language in its root sense requires words. Such an assumption would be a logical fallacy or conceptual conflation based on a separate English sense of ‘language’ (i.e. language as a referential code or system, whether functional or formal, for social communication: Sebeok’s ‘speech’ or ‘verbal communication’). Nor does this mean that the appearance of language in its root sense marks the genesis of symbolic thought and behaviour, as many have come to assume.

Symbolic species or semiotic animal? The widely influential success of Deacon’s (1997) Symbolic Species served to cement earlier proposals10 suggesting that the nature of language is somehow tied to the emergence of ‘symbolic’ reference. According to Deacon, language is ‘a mode of communication based on symbolic reference’ that involves a higher-order grammar that ‘maintains and regulates’ the system (1997: 41, 100). Given Deacon’s usage of ‘symbol’ in relation to other key terms from Peircean semiotic (1997: 69–101), one might expect a thorough integration of his ‘symbol’ concept within Peircean semiotic. But as successive reviews have demonstrated, this does not appear to be the case. In addition to conflicts between Peirce and Deacon’s respective accounts of language that should be apparent in light of the discussion above, two of the most cogent critiques have come from Pietarinen (2012) and Stjernfelt (2014: 141–77, see also 2007, 2012). In Pietarinen’s estimate, Peirce would disagree with Deacon’s symbolic threshold due to the necessity of shared continuity between all sign systems (whether human or otherwise). Furthermore, from a Peircean perspective, any discussion of symbols and symbolic systems would focus on salient processual relationships involving habit and convention (Pietarinen 2012). Stjernfelt’s critique goes further into the details of Peircean semiotic to argue that symbolic dicisigns (also known as ‘propositions’) naturally employ basic concepts and conceptual systems that function along a cline from abstract to perceptual throughout animal cognition (2014: 40–1; drawing on Hurford 2007). Stjernfelt argues further that it is misleading to tie the emergence of human cognition to any particular sign classification or presupposed opposition since for Peirce even arguments (presumably the most advanced sign class) are understood to be implicitly present in the most basic modes of semiosis (2014: 1–4). For a nuanced review of Stjernfelt’s critique of Deacon, see Bennett (2021: 175–89). Other problems intrinsic to Deacon’s symbolic threshold hypothesis can be noted in the literature as well (see, e.g., Queiroz and Ribeiro 2002; Pattee 2007; Stjernfelt 2012, 2007: 241–55; Queiroz, Stjernfelt, and El-Hani 2014; Maran 2017: 59–61). Sebeok himself held a different position as can be noted in claims like the following: ‘instances

SEMIOTICS IN EVOLUTIONARY LINGUISTICS

101

of the sign category, which philosophers (especially since C. S. Peirce) routinely call “symbols”, occur naturally not only throughout the animal kingdom, but […] in the metabolic code that governs endosemiotic processes inside animal bodies’ (1987: 354). On the other hand, Deacon’s ‘virtual worlds’ discussion introduced at the beginning of this chapter is clearly consonant with a Peircean (and Sebeokean) semiotic account of language emergence. So, if this unique ability is not tied to a symbolic threshold, what is it tied to? Perhaps human consciousness is rather a new kind of self-controlled, reflexive awareness, as Peirce suggests above – not so much a new ability to use symbols in general but, rather, an awakening to a distinctive mode or sub-type of symbolic activity and symbolic potential – a new reflexive mode of symbol use. If so, this new ability would have more to do with what Peirce discusses as ‘hypostatic abstraction’ and ‘prescission’ (see, e.g., Peirce c. 1902: CP 4.235, c. 1905: CP 5.534; Stjernfelt 2007: 241–55, 2014: 162–77, Champagne 2018).11 Notably, both abilities rely on a prior capacity for ‘metasemiosis’ (Deely 2010): the ability to distinguish between sign vehicles and their objects, or the ability to think of signs as signs. With these distinctions in mind, along with the virtual worlds they enable, it may indeed be more accurate to count human beings as one among myriad symbolic species that also happens to be uniquely aware of symbolic reference in a reflexive capacity, due to a more general ability for semiotic reflection. On related grounds, Stjernfelt suggests ‘abstract animal’ as a more apt replacement for Deacon’s ‘symbolic species’ (Stjernfelt 2007: 255), and Deely (2010) suggests ‘semiotic animal’, pointing to a necessarily earlier threshold separating semiosic activity in general (animal being, in which humans participate) from semiotic reflection itself (human being beyond animal awareness).

Language as adaptation; speech as exaptation: How, why and when? The discussion above lays the groundwork for a plausible account of the pre-sapiens origins of language in its root sense. Sebeok (1986b, 1987) himself argues that language as a modelling system began with Homo habilis some two million years ago and was then slowly exapted for verbal (spoken/signed) communication – with stages apparently increasing in complexity along the way, as implied by expansions in brain size found in successive species, along with developments in tool use, increasingly sophisticated ritual and material culture, among other sources of evidence – all leading up to Homo sapiens and the eventual emergence of civilization. Barham and Everett’s (2020) gradualist, meaning-centric account of the origins of linguistic communication12 is a helpful development of (the exaptation of) pre-sapiens language origins. Building on the work of Deacon (1997), Hurford (2007), Everett (2017), and many others, Barham and Everett consolidate evidence for an initial stage in the exaptation of language for socio-cultural communication purposes as early as Homo erectus. But, assuming that linguistic communication is an exaptation of language and that language began as an adaptation of general primate cognition that ushered in semiotic consciousness, as described above, what could account for the original cognitive adaptation itself? What exactly was adapted and how? And similarly for the exaptation of language for verbal communication, how did the exaptation happen? Several have pointed to ‘analogy’ as a shorthand for the language faculty and its features (e.g. Deely 2002, Anttila 2003, Bybee 2010: 75, Brand et al. 2021), but analogy is not

102

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

restricted to human thought. Fauconnier and Turner (2002) describe a gradient cline of analogical thinking or ‘conceptual blending’ abilities shared between human beings and alloanimals that leads to a new kind of complex blending found only in humans. This they dub ‘double-scope conceptual blending’, a blending activity that undergirds all feats of imagination in which aspects of two or more things are thought of in terms of each other, leading to the creation and contemplation of new possibilities. This is a plausible, meaning-centric account; but much work remains to develop it (and translate it into semiotic), thematically, empirically and logically. It is tempting to default to discussions of grammar and words when considering language and speech, but thinking does not require words any more than verbal communication requires complex syntax. Most of our thought processes are composed not of words, but of geometric shapes, images, lines and feelings based on body memories of moving and interacting in space (Arnheim 1969: 134; McGilchrist 2019: 106–10; Sheets-Johnstone 2011; Johnson 1987, 2007). In addition to traditional domains of research such as cranial volume, skull morphology, tool use and ritual remains, other more meaning-oriented factors influencing transitional stages in the evolution of language and speech/sign may include music and song (e.g. Dunbar 2004; Mithen 2006), movement and dance (e.g. Sheets-Johnstone 2011; Fink et al. 2021), gesture and pantomime (e.g. Bouissac 2006; Zlatev et al. 2020), material artefacts (e.g. Botscharow 1990; Tylén et al. 2020) and more. In short, the story is complex and much work remains; but for theorizing the origins and development of human cognition nothing is more promising than the evolution of upright posture (Sheets-Johnstone 1990a, 1990b; Van Lier 2003, 2010; Pelkey 2017a, b, 2018). After all, upright bipedalism is an attribute just as striking as language13 when it comes to the unique affordances of the human Umwelt.

BIOSEMIOTICS AND LINGUISTIC SEMIOSIS Jakob von Uexküll was convinced that ‘[as] scientists we know that whatever we say about the natural world is inadequate, because human language is no currency for the truths of nature’ (Uexküll 1964: 144).14 Uexküll’s Umwelt theory rests on the following main ideas: (A) [True] reality (Natur) that lies beyond or behind the nature that physicists, chemists, or microbiologists conceive of in their scientific systems reveals itself through signs. These signs are therefore the only true reality, and the rules and laws to which the signs and sign-processes are subject are the only real laws of nature. […] (B) The methodology of Umwelt research, which aims to reconstruct this ‘creating’ of [reality] […] means, therefore, reconstructing the Umwelt of another living being. […] (C) The aim of Umwelt research is to create a theory of the composition of nature […] [by exploring] the sign-processes that govern the behaviour of living subjects. (T. von Uexküll 1982: 4–8) An awareness of the limitations of this ‘creating of reality’ with language is consistent with the view that language is a tool of thought or modelling system (e.g. Sebeok and Danesi 2000) rather than a mere communication system. According to this view, language serves

SEMIOTICS IN EVOLUTIONARY LINGUISTICS

103

the cognitive function of modeling, and, as the philosopher Popper as well as the linguist Chomsky have likewise insisted […], not at all for the message swapping function of communication. The latter was routinely carried on by nonverbal means, as in all animals, and as it continues to be in the context of most human interactions today. (Sebeok 1991a: 334) Upon accepting the view that linguistic semiosis is more about the fixation of belief (e.g. Peirce 1877) than it is about ‘swapping messages’, the semiotic perspective on language aims at a better understanding of this specifically human ‘creating of reality’. About all we can say at a general level is that the words of our language provide complex perspectives that offer us highly special ways to think about things – to ask for them, tell people about them, etc. Real natural language semantics will seek to discover these perspectives and the principles that underlie them. People use words to refer to things in complex ways, reflecting interests and circumstances, but the words do not refer. (Chomsky 2007: 58) This puts many linguistic abstractions in perspective and defines a clear path of discovery. The goal to better understand ‘how people use words to refer to things in complex ways’ may target the acoustic level to examine the production and perception of speech sounds or areas like verb cognition to address the particular characteristics of certain verbs and the difference between he sprayed the wall with paint and he sprayed paint on the wall. Uexküll’s Umweltlehre inspired Sebeok’s definition of semiosis as ‘the processual engine which propels organisms to capture “external reality” and thereby come to terms with the cosmos in the shape of species-specific internal modeling systems’ (Sebeok 2001b: 15). The processual nature of all semiosis, including linguistic semiosis, can be illustrated with another important biosemiotic principle that is Uexküll’s Funktionskreis. The constant feedback between action and perception is a model of organism-environment interaction Jakob von Uexküll first established through muscular physiology. Uexküll formulated a fundamental law of neuromuscular regulation (the principle of negative feedback) that explains how any outside impulse to the body of an organism is received by the muscles and nerves that are already engaged, forming a feedback loop of perception and action that is fundamental to all organism/environment interaction. Like walking on a rocky path requires a constant feedback loop between feeling the surface and placing the next step, speech depends on a constant feedback between hearing and producing sound. Uexküll’s Funktionskreis is a multi-level model of semiosis that can be applied to all aspects of language from the physiological level of speech perception and production to the most complex aspect of this ‘creating of reality’ that depends on the constant reprocessing, recalibrating, reconsidering that is best exemplified in Peirce’s concept of the interpretant. Never fixed, the ‘creating of reality’ depends on the ongoing process of semiosis. The idea that our beliefs are formed by the metaphors and narratives we are exposed to, represented in Peirce’s famous dictum my language is the sum total of myself (1893: CP 5.314), has been developed in cognitive linguistics through the work of Goffman

104

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

(1974), Fillmore (1976) and Lakoff & Johnson (1980). Its most recent iteration includes the possibility that our beliefs form physical structures in the brain (Lakoff 2009), another idea best represented by the incessant feedback-loop of the Funktionskreis. The more often you hear, read or see something, the stronger the neuronal connection, and the more firmly established the associated beliefs. When Uexküll wrote that ‘human language is no currency for the truths of nature’, he understood that the stories we live by are always in flux, and that the best explanations for what he called Natur (i.e. reality) is to understand our ability to create our Umwelt. What we refer to as science are, in fact, stories we live by that require constant revision. Frans Verhagen (2008) identifies the following metaphors or stories about nature: Nature as scala naturae Nature as machine Nature as factory/workshop Nature as web Nature as storehouse Nature as mother Nature as measure Kalevi Kull has offered a similar analysis of basic metaphors in the field of biology throughout the ages in the metaphors ladder, tree and web (Kull 2003); representative of hierarchy, geneaology and ecology respectively. The story of nature as a storehouse, for instance, comes into play wherever humans are harvesting resources or raising and eating animals. The story of nature as web is connected to the concept of ecology. The story of nature as machine resonates with our stories about atoms and molecules, genes and neurons. We have cognitive or practical habits associated with all of them. Our attitudes towards other animals also draw on different narratives depending on the context. The stories we live by determine how we think and act, whether we live with animals or without them, tolerate or eradicate them, use or abuse them, eat or abstain from them, admire or ignore them, protect them or leave them to extinction, see them as something or someone. The metaphor of the hierarchy is paramount and depends on our stories about the sentience and intelligence of non-human organisms; the key characteristic of exclusivity of the faculty of language. Language, the old story goes, makes us different from the rest.

ANIMAL COMMUNICATION AND LINGUISTIC SEMIOSIS The American linguist Charles Hockett (1960) made a list of design features of human language to compare language with ‘animal communication’. The list included, for instance, features like displacement (the ability to refer to things not present at the time and place of signifying), productivity (the ability to create new combinations of signs), cultural transmission (the teaching of particular signs from generation to generation) and learnability (the ability to learn new sign systems). Č adková (2015) explained that Hockett’s design features are inadequate not only because some of them are not exclusive to human natural languages, but because it is inherently unscientific to even expect the sign systems of other animals to have comparable characteristics in the first place. The concept of Umwelt prevented Jakob von Uexküll from such misguided comparisons, because he knew that the capacity to signify and interpret signs varies with an organism’s species-specific semiotic profile.

SEMIOTICS IN EVOLUTIONARY LINGUISTICS

105

Hockett’s design features emerged in the era of behaviourism and B. F. Skinner’s Verbal Behavior (1957). While the main ideas of behaviourism had been derived from studies with animals, the cognitive/semiotic capacities of the animals involved were ignored. It was irrelevant whether the subject of the experiment was a rat or a pigeon. On the contrary, von Uexküll considered both an anthropocentric perspective on other animals and a purely mechanical approach to their behaviour to be inadequate and unscientific. Just as Uexküll rejected a universal learning theory based on the behaviourist paradigm, Chomsky rejected a behaviourist approach to human language (Chomsky 1959). To validate behaviourist models of learning and to challenge the innateness hypothesis, psychologists and primatologists in the mid-twentieth century conducted long-term primate studies starting in the 1960s. These studies were based on the fact that primates share most of our DNA and general physiology and the assumption that only physiological limitations prevent primates from speech. The assumption that animals that look most like us are closest to us in intelligence resonates with the hierarchical story of the scala naturae. In these studies, chimpanzees or bonobos were raised by humans with ASL (e.g. Gardner and Gardner 1984) or abstract symbol systems (e.g. Savage-Rumbaugh 1986) to find out if primates have something like grammar even though they are apparently not using it. The neglect of species-specific semiotic abilities in all these experiments was obvious to Chomsky and Sebeok, who voiced their opposition (cf. Sebeok & Umiker-Sebeok 1980). While Sebeok considered Terrace’s experiments (cf. Terrace 1979) to adhere to high scientific standards, he was certain that ‘the alleged language experiments with apes divide into three groups: one, outright fraud; two, self-deception; three, those conducted by Terrace. The largest class by far is the middle one’ (Sebeok quoted by Wade 1980). Chomsky and Sebeok knew that there was nothing to be learned about human language or the cognitive capacities of primates (Sebeok & Umiker-Sebeok 1980). In an interview about his opposition to these experiments, Chomsky explained: Would it be of any interest to train grad students to more or less mimic apes? We would learn nothing about apes from the fact that grad students can be trained to more or less mimic them – try to get an NSF contract to study that – just as we learn nothing about humans from the fact that apes can be trained to mimic humans in some respects. Language is a notorious failure, exactly as any biologist and paleo-anthropologist would have expected. But if, say, Nim had succeeded, we would still have learned nothing about language acquisition, gaining neither more nor less wonderment, though we would have a biological problem. Namely, if apes have this fantastic capacity, surely a major component of humans’ extraordinary biological success (in the technical sense), then how come they haven’t used it? It’s as if humans can really fly but won’t know it until some trainer comes along to teach them. (Cicchiaro 2007/2008) It took several decades until primate researchers reoriented their focus towards the species-specific sign-systems, vocal and non-vocal, and primates were studied in the wild or in more species-appropriate semi-wild habitats (e.g. Halloran 2012). It is unfortunate that Uexküll’s basic premise that each animal species has a specific way of interacting with its environment that is articulated in the notion of Umwelt took so long to permeate into the mainstream sciences.

106

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Animals processing human language Unlike wild animals, domesticated animals that have evolved with us have long tuned in to human language. While most dog owners tend to overestimate the role language plays in their interaction with dogs (e.g. Horowitz 2009), the ability of domestic dogs to process human language is deserving of scientific attention. Alexandra Horowitz attributes her approach to studying dog behaviour to the work of Jakob von Uexküll: The scientific study of animals was changed by a German biologist of the 20th century named Jakob von Uexküll. What he proposed was revolutionary: anyone who wants to understand the life of an animal must begin by what he called their Umwelt, their subjective or ‘self world.’ Umwelt captures what life is like as the animal. (Horowitz 2009: 20) It is from this starting point that Horowitz explains the perception and action profiles of dogs for a better understanding of how they signify and interpret human signs. Dogs learn by reading our body first, and then associate vocal commands with what we communicate with our bodies. One of the most important lines of research that has come from Jakob von Uexküll’s institute in Hamburg at the beginning of the twentieth century was methods for the training of guide dogs for the visually impaired (cf. Magnus 2015). The ability in dogs to interpret the vocalizations of another species is indicative of high intelligence (cf. Herzing 2015), but exactly how they process language is only beginning to be understood (e.g. Andics et al. 2016). Horowitz points out that ‘[researchers] tend to speak of the species as though all members of the species were identical’ (2009: 8) and cautions that ‘[the] results of many wellperformed experiments may eventually allow us to reasonably generalize to all dogs, period. But even then, the variations among individual dogs will be great’ (2009: 9). Like with human experiments in neuroscience, it may be too soon for generalizations to be made about the minds/brains of intelligent organisms regarding such fundamental questions as specification. If we look at all animals under the assumption that they construct their own subjective self-world according to their species-specific semiotic profiles, but also based on their unique experiences, the generality and explanatory power of individual brain scans appears in a different light. Fortunately, it appears that the twenty-first century has finally acknowledged the complexity and systematicity of the signifying abilities of other intelligent animals. At least some scientists have found ways to investigate animal communication with a new attitude that is consistent with the Uexküllian premise that their ways to make sense of the world are different from ours. For the language sciences, this new attitude requires zooming out from human language as completely separate from the sign-systems of nonhuman organisms and at the same time not anthropomorphizing by comparing all sign systems to human language. In a recent paper on how to analyse non-human types of intelligence, Herzing explains that [intelligence] has historically been studied by comparing nonhuman cognitive and language abilities with human abilities. Primate-like species, which show human-like anatomy and share evolutionary lineage, have been the most studied. However, when comparing animals of non-primate origins our abilities to profile the potential for intelligence remains inadequate. (Herzing 2014: 676)

SEMIOTICS IN EVOLUTIONARY LINGUISTICS

107

SEMIOTICS IN BIOLINGUISTICS Rooted in ethology Biolinguistics emerged from dominant paradigms in the language sciences in the middle of the twentieth century. This new approach originated from ethology and comparative psychology initiated by Jakob von Uexküll, Konrad Lorenz and Nikolaas Tinbergen (cf. Lenneberg 1967). Uexkuell, the forerunner of modern ethology, [proposed] that every species has its own world-view. The phenomenological implications of this formulation may sound old-fashioned today, but students of animal behavior cannot ignore the fact that the differences in cognitive processes (1) are empirically demonstrable and (2) are the correlates of species-specific behavior. (Lenneberg 1967: 372) Uexküll’s Umwelt – the subjective species-specific world created by an organism – is central to the ethological approach to human language shared by biolinguists (see also Augustyn 2009a, b, 2013; Tønnessen 2015). According to Chomsky’s description, biolinguists work to adapt the frameworks of comparative psychology and ethology ‘to the study of human cognitive organs and their genetically determined nature, which constructs experience – the organism’s Umwelt, in ethological terminology – and guides the general path of development, just as in all other aspects of growth of organisms’ (Chomsky 2006: x). Eric Lenneberg’s Biological Foundations of Language (1967) especially ‘anticipated many themes of the coming decades’ (Jenkins 2000: 3) and Chomsky declared that ‘[linguistics] is really a theoretical biology’ (Sklar 1968: 218) just as Jakob von Uexküll had believed in ‘making [linguistics] a biological science’ (see Kull 2001: 3).

Language as a tool of thought Biolinguists see human language foremost as a cognitive tool, since the species was capable of communication before it emerged. Sebeok agrees with the biolinguistic perspective, arguing that language serves primarily ‘the cognitive function of modeling’ (1991a: 334): [A]s the philosopher Popper as well as the linguist Chomsky have likewise insisted […], not at all for the message swapping function of communication. The latter was routinely carried on by nonverbal means, as in all animals, and as it continues to be in the context of most human interactions today. (1991a: 334) Sebeok regarded language as a secondary modelling system that allows the species to create models of reality by augmenting the primary modelling system that is the speciesspecific perceptual system (cf. Anderson & Merrell 1991, Sebeok & Danesi 2000). ‘Organisms create their umwelten’ (Kull et al. 2009: 167–73) and humans do so with their senses as well as with language; and a better understanding of its underlying principles promises to be ‘a most illuminating probe with which to explore the organization of mental processes’ (Chomsky 2006: 83).

108

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Three factors For biolinguists, three factors constitute the human faculty of language: 1. the genetic endowment 2. organism–environment interaction (species-specific Umwelt) 3. abstract principles not specific to the faculty of language. (cf. Chomsky 2005, 2007) The genetic endowment, or innateness of human language, is widely known and accepted  as Universal Grammar. It explains why there is no community of people, no matter how small or isolated, without language. Organism–environment-interaction (i.e. species-specific Umwelt) is the crucial component of the growth of language in the individual. The principles that are not specific to language make language an exaptation, a repurposing of cognitive structures specific to the human cognitive profile.

The generative principle, abduction and habit-taking In search of this overlap of abstract principles that serve language as well as other human cognitive abilities, the Peircean concepts of abduction and habit-taking have also played a role in biolinguistics. The analysis of the deep structure of abstract operations of formal grammar is like a ‘Peircean logic of abduction’ (Chomsky 2006: 86). Peirce argued that the general limits of human intelligence are much more narrow than might be suggested by romantic assumptions about the limitless perfectibility of man […]. He held that innate limitations on admissible hypotheses are a precondition for successful theory construction, and that the ‘guessing instinct’ that provides hypotheses makes use of inductive procedures only for ‘corrective action.’ […] To understand how knowledge is acquired, in the rationalist view that Peirce outlined, we must penetrate the mysteries of what he called ‘abduction’. (Chomsky 2006:79–80) The fundamental questions for biolinguistics, according to Jenkins (2000: 1), are: 1. What constitutes knowledge of language? (Plato’s problem) 2. How is this knowledge acquired? (Humboldt’s problem) 3. How is this knowledge put to use? (Descartes’ problem) To these three fundamental questions, the following have been added cautiously: 4. What are the related brain mechanisms? 5. How did language evolve in the species? Questions of brain mechanisms and evolution have been treated with caution by biolinguists, and Chomsky has warned linguists since the 1960s ‘not to relate the postulated mental structures and processes to any physiological mechanisms or to interpret mental function in terms of “physical causes”’ (2006: 12) and encouraged researchers, instead, to explore the creative/generative principles of language use. In particular, biolinguistics has been critical of the many confident pronouncements coming from neuroscience, because ‘neuroscientists still do not understand how brains generate minds’ and ‘principles underlying brain development and evolution remain only dimly understood’

SEMIOTICS IN EVOLUTIONARY LINGUISTICS

109

(Fitch 2009: 285). Likewise, neuroscientists do not know how brains generate language, and there is still very little collaboration between neurolinguists and theoretical linguists (cf. Andrews 2011).

Language as a natural object This scepticism towards physiological and behavioural evidence is consistent with the antipsychologism inherent in semiotics. In the context of science in general (or linguistics in particular), psychologism is a reductionist approach where behavioural and physiological evidence of cognitive processes (or rather the correlates thereof) replaces the semiotic or logical analysis of the underlying concepts (cf. Stjernfelt 2013, 2014). The main issue is that when languages are studied like natural objects, some units of description (e.g. syntactic units like noun and verb, subject or object) are natural kinds, but the spatiotemporal events related to individuals processing language are not. In other words, while the syntactic analysis of sentences (and many other types of linguistic analysis that deal with natural kinds) is anti-psychologistic, measuring processing speed in individuals (and other types of analysis based on behavioural or physiological evidence) are prone to psychologism. To be clear, it is not experimentalism or empiricism but psychologism that may well be behind the fact that only 36 per cent of studies in the top three psychology journals can be replicated successfully (Carey 2015). While biolinguistics focuses on the observable facts related to the natural kinds of language, there is humility in admitting that ‘many mysteries still lie beyond the reach of the form of human inquiry we call “science”, a conclusion that we should not find surprising if we consider humans to be part of the organic world’ (Chomsky 2007:18). While biolinguistics focuses strongly on the natural kinds (noun, verb, determiner, etc.) related to language, there are ‘unresolved semiotic challenges [that] pose problems for any aspect of cognition’ (Fitch 2009): About all we can say at a general level is that the words of our language provide complex perspectives that offer us highly special ways to think about things – to ask for them, tell people about them, etc. Real natural language semantics will seek to discover these perspectives and the principles that underlie them. People use words to refer to things in complex ways, reflecting interests and circumstances, but the words do not refer. (Chomsky 2007: 58) Biolinguistics does not separate nature and culture. The notion of optimal design in the Minimalist approach, exemplified by the analogy between natural languages and snowflakes can therefore be understood as the central unifying principle that sees language as a natural object (Boeckx & Piatelli-Palmarini 2005: 461).

SEMIOSIS IN LANGUAGE EVOLUTION: PROBLEMS AND PROGRESS Contemporary evolutionary linguistics shares much common ground with semiotics; and this is due, in part, to the ongoing influence of semiotic theory among practitioners – especially Peircean and Uexküllean theories. As in other disciplines, though, ideas from semiotics are too often applied in piecemeal fashion, neglecting the broader context and full

110

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

potential of the theory or system of origin. Adopting a more thoroughgoing Peircean semiotic perspective would be especially helpful for the field since the point of Peirce’s pragmatist semiotic is to clarify concepts through an architectonic logic, approached from a fallibilist stance, seeking to establish continuity between systems – and induce doubt, when necessary (over unwarranted assumptions). Among other concepts that stand in need of ongoing clarification in contemporary evolutionary linguistics, are vague concepts of high salience like ‘language’ and ‘symbol’. Another research priority in this connection is the need for ongoing inquiry into the domain-general nature of ‘evolution’ and the interaction of multi-layered modelling systems in the evolution of human cognition. The relevance of diagrammatic iconicity for the field is also an important reason for reorientation to semiotic interventions. Diagrammatization (as a process) and diagrammatic relations (as a state) not only expose the false dichotomy of synchrony and diachrony, but should also serve as an upgrade for research on iconicity among language evolutionists – just as they should serve as an upgrade for historical linguistic research on language change and historical reconstruction. Since diagrammatization is end-directed, this implies the need for reincorporating final causation into evolutionary linguistic theory. Elsewhere, links between language, analogy, blending, imagination and their potential for developing a semiotic theory of the origins of language and speech are also in need of further work. This includes the need for more inclusive, gradient models that investigate basic conceptual blending abilities that we share with other animals. And the grandest open project of all in need of semiotic collaboration is the never-ending reconstruction of master narratives for language evolution and speech exaptation – and their co-evolution with embodied cognition and polysemiotic resources across deep time. And it is here that semiotics may offer the most useful perspective of all: common grounds for dialogue between language evolutionists and biolinguists.

NOTES 1 This collaboration draws on our complementary backgrounds in historical linguistics and language evolution (for J. P.) and biolinguistics and biosemiotics (for P. A.) and our shared appreciation for the thought of C. S. Peirce and Thomas A. Sebeok. Pelkey is responsible for the Introduction, the Conclusion, and Sections 2–4, while Augustyn is responsible for Sections 5–7. 2 Unfortunately, English is ill-suited for such discussions since our ‘language’ about ‘language’ in the English ‘language’ is fraught with pernicious polysemy – a point that is germane for later sections of this chapter. 3 Though, like Darwin who came after him, Jones was building on the work of others. 4 The growth of an organism from seed to maturity, or the growth of a population through space and time. 5 Jakobson 1965: 426, 1977; see also Rauch 1981; Nöth 1999, 2008. For further examples and discussion among ‘language evolution’ researchers, see Perniss, Thompson, and Vigliocco 2010; Dingemanse et al. 2015; Pleyer et al. 2017; Perlman 2017; Nölle, Hartmann, and Tinits 2020. 6 Though discussions of the evolution of ‘systematicity’ in language change may be vaguely related (see, e.g., Dingemanse et al. 2015). 7 See https://www.evolang.org/ and http://www.protolang.org/ for more information. Closely related is the annual conference of the Cultural Evolution Society: https:// culturalevolutionsociety.org/.

SEMIOTICS IN EVOLUTIONARY LINGUISTICS

111

8 See Pelkey (2013a, b, 2015, 2019) for further discussion and application. 9 Rendered here using Sebeok’s default American spelling; rendered elsewhere using project commonwealth spelling, except in quoted material that uses the former; For Sebeok, modelling systems and Umwelt cognition share a kind of identity. See Umwelt discussion in the next section: ‘Biosemiotics and linguistic semiosis’. 10 See e.g., Cassierer 1923–9, 1944; Sapir 1921, 1929; Donald 1991. 11 To illustrate, hypostatic abstraction is implicated in the ability to think of states and predicates like ‘soft’ or ‘see’ as subjects and objects like ‘softness’ or ‘sight’, respectively. Prescission is the ability to imagine qualities apart from the specific contexts or phenomena they ultimately rely on or inhere in, as when we tease apart ‘sweetness’ from ‘honey’ or ‘light’ from ‘morning’. 12 This work, like so many others, confuses ‘language’ with verbal communication; hence this phrasing. 13 See Tattersall 2018: 289 14 My translation – P. A.

REFERENCES Aames Cucchiaro, M. (2007/2008), On the Myth of Ape Language. Interview with Noam Chomsky. https://chomsky.info/2007____/ Akmajian, A., R. A. Demers, and R. M. Harnish (1979), Linguistics: An Introduction to Language and Communication, Cambridge: Harvard. Alter, S. G. (1999), Darwinism and the Linguistic Image, Baltimore: Johns Hopkins University Press. Andersen, H. (1973). ‘Abductive and Deductive Change’, Language, 49 (4): 765–93. Anderson, M. and F. Merrell, eds (1991), On Semiotic Modeling, Berlin: Mouton de Gruyter. Anderson, M., J. N. Deely, M. Krampen, J. Ransdell, T. A. Sebeok, and T. v. Uexküll (1984), ‘A Semiotic Perspective on the Sciences: Steps towards a New Paradigm’, Semiotica, 52 (1–2): 7–47. Andics, A., A. Gábor, M. Gácsi, T. Faragó, D. Szabó, and A. Miklósi (2016), ‘Neural Mechanisms for Lexical Processing in Dogs’, Science, 353 (6303): 1030–2. Andrews, E. 2011, ‘Language and Brain: Recasting Meaning in the Definition of Human Language’, Semiotica, 184 (1): 11–32. Anttila, R. ([1972] 1989), Historical and Comparative Linguistics, 2nd edn, Amsterdam: John Benjamins. Anttila, R. (1977), Analogy, The Hague: Mouton. Anttila, R. (1992), ‘Historical Explanation and Historical Linguistics’, in G. W. Davis and G. K. Iverson (eds), Explanation in Historical Linguistics, 17–39, Amsterdam: John Benjamins. Anttila, R. (1994), ‘Collaterality and Genetic Linguistics’, in The Peirce Seminar Papers: Essays in Semiotic Analysis, vol. 2, M. Shapiro (ed.), 29–46, Oxford: Berghahn. Anttila, R. (2003), ‘Analogy: The Warp and Woof of Cognition’, in B. D. Joseph and R. D. Janda (eds), The Handbook of Historical Linguistics, 425–40, Oxford: Blackwell. Arnheim, R. (1969), Visual Thinking, London: Faber & Faber. Atkinson, Q. D., and R. D. Gray (2005), ‘Curious Parallels and Curious Connections: Phylogenetic Thinking in Biology and Historical Linguistics’, Systematic Biology, 54 (4): 513–26. Augustyn, P. (2009a), ‘Uexküll, Peirce, and Other Affinities between Bio-linguistics and Biosemiotics’, Biosemiotics, 3 (1): 1–17.

112

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Augustyn, P. (2009b), ‘Translating Jakob von Uexküll – Reframing Umweltlehre as Biosemiotics’, Sign Systems Studies, 37 (1/2): 281–98. Augustyn, P. (2011), ‘On the Concept of Code in Linguistics and Biosemiotics’, Biosemiotics, 4 (3): 281–9. Augustyn, P. (2013), ‘What Connects Biolinguistics and Biosemiotics?’ Biolinguistics, 7 (4): 96–111. Augustyn, P. (2018), ‘Animal Studies in the Language Sciences’, Biosemiotics, 11 (1): 121–38. Barham, L. and D. Everett (2020), ‘Semiotics and the Origin of Language in the Lower Palaeolithic’, Journal of Archaeological Method and Theory, 28: 535–79. Bateson, G. (1968), ‘Redundancy and Coding’, in Th. A. Sebeok (ed.), Animal Communication, 614–26, Bloomington: Indiana University Press. Bennett, T. J. (2021), Detotalization and Retroactivity: Black Pyramid Semiotics, Tartu University, PhD Dissertation. https://dspace.ut.ee/handle/10062/71156 Boeckx, C. and M. Piattelli-Palmarini (2005), ‘Language as a Natural Object: Linguistics as a Natural Science’, The Linguistic Review, 22: 362–79. Botscharow, L. (1990), ‘Paleolithic Semiotics: Behavioral Analogs to Speech in Acheulean Sites’, in M. L. Foster and L. Botscharow (eds), The Life Of Symbols, 81–94, London: Routledge. Bouissac, P. (2006), ‘Gesture in Evolutionary Perspective’, Gesture, 6 (2): 189–204. Brand, C. O., A. Mesoudi, and P. E. Smaldino (2021), ‘Analogy as a Catalyst for Cumulative Cultural Evolution’, Trends in Cognitive Sciences, 25 (6): 450–61. Bybee, J. (2010), Language, Usage and Cognition, Cambridge: Cambridge University Press. Č adková, L. (2015), ‘Do They Speak Language?’ Biosemiotics, 8 (1): 9–27. Carey, B. (2015), ‘Psychologists Welcome Analysis Casting Doubt on Their Work’, New York Times, 28 August. Cassirer, E. ([1944] 1972), An Essay on Man: An Introduction to a Philosophy of Human Culture, New Haven: Yale University Press. Cassirer, E. ([1923–9] 2001–2), Philosophie der symbolischen Formen, 3 vols., Ernst Cassirer Werke Bd, 11–13, Hamburg: Meiner. Champagne, M. (2018), Consciousness and the Philosophy of Signs: How Peircean Semiotics Combines Phenomenal Qualia and Practical Effects, Cham: Springer. Chomsky, N. (1959), ‘Review of Verbal Behavior by B.F. Skinner’, Language, 35 (1): 26–58. Chomsky, N. (2005), ‘Three Factors in Language Design’, Linguistic Inquiry, 36 (1): 1–22. Chomsky, N. (2006), Language and Mind, 3rd edn, Cambridge: Cambridge University Press. Chomsky, N. (2007), ‘Biolinguistic Explorations: Design, Development, and Evolution’, International Journal of Philosophical Studies, 15 (1): 1–21. Chomsky, N. ([1966] 2009), Cartesian Linguistics: A Chapter in the History of Rationalist Thought, 3rd edn, Cambridge: Cambridge University Press. Cobley, P. (2016), Cultural Implications of Biosemiotics, Cham: Springer. Christiansen, M. H. and S. Kirby, eds (2003), Language Evolution, Oxford: Oxford University Press. Croft, W. (2000), Explaining Language Change: An Evolutionary Approach, London: Longman. Darwin, C. (1882), The Descent of Man and Selection in Relation to Sex, London: John Murray. Deacon, T. W. (1997), The Symbolic Species: The Co-evolution of Language and the Brain, New York: W.W. Norton. Deely, J. (2002), ‘The Absence of Analogy’, The Review of Metaphysics, 55 (3): 521–50. Deely, J. (2009), Purely Objective Reality, Berlin: Mouton de Gruyter.

SEMIOTICS IN EVOLUTIONARY LINGUISTICS

113

Deely, J. (2010), Semiotic Animal: A Postmodern Definition of Human Being Transcending Patriarchy and Feminism: To Supersede the Ancient and Medieval ‘Animal Rationale’ along with the Modern ‘res cogitans’, South Bend, ST: Augustine’s Press. Deely, J. (2016), ‘Thirdness in Nature’, SCIO: Revista de Filosofía, 12: 75–80. DeSilva, J. (2021), First Steps: How Upright Walking Made Us Human, New York: Harper. Dingemanse, M., D. E. Blasi, G. Lupyan, M. H. Christiansen, and P. Monaghan (2015), ‘Arbitrariness, Iconicity, and Systematicity in Language’, Trends in Cognitive Sciences, 19 (10): 603–15. Di Sciullo, A. M. and C. Boeckx, eds (2011), The Biolinguistic Enterprise: New Perspectives on the Evolution and Nature of the Human Language Faculty, Oxford: Oxford University Press. Donald, M. (1991), Origins of the Modern Mind: Three Stages in the Evolution of Culture and Cognition, Cambridge, MA: Harvard University Press. Dunbar, R. (2004), The Human Story: A New History of Mankind’s Evolution, London: Faber & Faber. Evans, N. and S. C. Levinson (2009), ‘The Myth of Language Universals: Language Diversity and Its Importance for Cognitive Science’, Behavioral and Brain Sciences, 32 (5): 429–92. Everett, D. L. (2017), How Language Began: The Story of Humanity’s Greatest Invention, New York: Liveright. Fauconnier, G. and M. Turner (2002), The Way We Think: Conceptual Blending and the Mind’s Hidden Complexities, New York: Basic Books. Fillmore, C. F. (1976), ‘Frame Semantics and the Nature of Language’, Annals of the New York Academy of Sciences: Conference on the Origin and Development of Language and Speech, 280: 20–32. Fink, B., B. Bläsing, A. Ravignani, and T. K. Shackelford (2021), ‘Evolution and Functions of Human Dance’, Evolution and Human Behavior, 42 (4): 351–60. Fitch, W. T. (2008), ‘Glossogeny and Phylogeny: Cultural Evolution Meets Genetic Evolution’, Trends in Genetics, 24 (8): 373–4. Fitch, W. T. (2009), ‘Prolegomena to a Future Science of Biolinguistics’, Biolinguistics, 3 (4): 283–320. Galantucci, B. and S. Garrod (2011), ‘Experimental Semiotics: A Review’, Frontiers in Human Neuroscience, 5: 1–11. Gardner, R. A. and Gardner, B. T. (1984), ‘A Vocabulary Test for Chimpanzees (Pan troglodytes)’, Journal of Comparative Psychology, 98 (4): 381. Goffman, E. (1974), Frame Analysis: An Essay on the Organization of Experience, London: Harper and Row. Greenberg, J. H. (1957), ‘Language and Evolutionary Theory’, in J. H. Greenberg (ed.), Essays in Linguistics, 56–65, Chicago: University of Chicago Press. Haiman, J. (1980), ‘The Iconicity of Grammar: Isomorphism and Motivation’, Language, 56 (3): 515–40. Halloran, A. (2012), The Song of the Ape: Understanding the Languages of Chimpanzees, New York: St Martin’s Press. Harari, Y. N. (2014), Sapiens, London: Harvill Secker. Hartmann, S. (2020), ‘Language Change and Language Evolution: Cousins, Siblings, Twins?’ Glottotheory, 11 (1): 15–39. Hauser, M. D. and T. G. Bever (2008), ‘A Biolinguistic Agenda’, Science, 322: 1057–9. Hauser, M. D. et al. (2002), ‘The Faculty of Language: What Is It, Who Has It, and How Did It Evolve?’ Science, 298: 1569–79.

114

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Haworth, K. A., and T. J. Prewitt (2020), The Evolution of Human Consciousness and Linguistic Behavior: A Synthetic Approach to the Anthropology and Archaeology of Language Origins. Lanham: Rowman & Littlefield. Heine, B. (1997), Cognitive Foundations of Grammar, Oxford: Oxford University Press. Herzing, D. (2014), ‘Profiling Non-human Intelligence: An Exercise in Developing Unbiased Tools for Describing Other “Types” of Intelligence on Earth’, Acta Astronautica, 94 (2): 676–80. Herzing, D. (2015), Dolphin Communication and Cognition, Cambridge: The MIT Press. Hockett, C. ([1960] 1982), ‘The Origin of Speech’, in Human Communication: Language and its Psychobiological Bases, New York: Scientific American. Horowitz, A. (2009), Inside of a Dog: What Dogs See, Smell, and Know, New York: Simon & Shuster. Hurford, J. R. (2007), The Origins of Meaning: Language in the Light of Evolution, Oxford: Oxford University Press. Jackendoff, R. (2010), ‘Your Theory of Language Evolution Depends on Your Theory of Language’, in H. Yamakido, R. K. Larson, and V. Déprez (eds), The Evolution of Human Language: Biolinguistic Perspectives, 63–72, Cambridge: Cambridge University Press. Jakobson, R. (1977), ‘A Few Remarks on Peirce, Pathfinder in the Science of Language’, MLN, 92 (5): 1026–32. Jenkins, L. (2000), Biolinguistics. Exploring the Biology of Language, Cambridge: Cambridge University Press. Jenkins, Lyle, ed. (2004), Variation and Universals in Biolinguistics, London: Else-vier. Johnson, M. (1987), The Body in the Mind: The Bodily Basis of Meaning, Imagination, and Reason, Chicago: University of Chicago Press. Johnson, M. (2007), The Meaning of the Body: Aesthetics of Human Understanding, Chicago: University of Chicago Press. Jones, W. ([1788] 1824), Discourses Delivered before the Asiatic Society: And Miscellaneous Papers, on the Religion, Poetry, Literature, etc., of the Nations of India, London: C. S. Arnold. Kirby, S., M. Tamariz, H. Cornish, and K. Smith (2015), ‘Compression and Communication in the Cultural Evolution of Linguistic Structure’, Cognition, 141: 87–102. Kull, K. (1999), ‘Biosemiotics in the Twentieth Century: A View from Biology’, Semiotica, 127: 385–414. Kull, K. (2001), ‘Jakob von Uexküll. An Introduction’, Semiotica, 134 (1): 1–50. Kull, K. (2003), ‘Ladder, Tree, Web: Ages of Biological Understanding’, Sign Systems Studies, 31 (2): 589–602. Kull, K. (2010), ‘Umwelt and Modelling’, in P. Cobley (ed.), The Routledge companion to semiotics, 43–56, London: Routledge. Kull, K., T. Deacon, C. Emmeche, J. Hoffmeyer, and F. Stjernfelt (2009), ‘Theses on Biosemiotics: Prolegomena to a Theoretical Biology’, Biological Theory, 4 (2): 167–73. Lakoff, G. (2009), The Political Mind: A Cognitive Scientist’s Guide to Your Brain and Its Politics, New York: Penguin Books. Lakoff, G. and M. Johnson (1980), Metaphors We Live By, Chicago: University of Chicago Press. Lenneberg, E. (1967), Biological Foundations of Language, New York: John Wiley. Levinson, S. C. and N. Evans (2010), ‘Time for a Sea-Change in Linguistics: Response to Comments on “The Myth of Language Universals”’, Lingua, 120: 2733–58.

SEMIOTICS IN EVOLUTIONARY LINGUISTICS

115

Linden, E. (1987), Silent Partners: The Legacy of the Ape Language Experiments, New York: Balantine Books. Lorenz, K. (1941), ‘Kants Lehre vom apriorischen im Lichte gegenwärtiger Biologie’, Blätter für Deutsche Philosophie, 15: 94–125. Lorenz, K. (1971), Studies in Animal and Human Behaviour, vol. 2, Cambridge, MA: Harvard University Press. Magnus, R. (2015), The Semiotic Grounds of Animal Assistance: Sign Use of Guide Dogs and Their Visually Impaired Handlers, Dissertation. Tartu: Tartu University. Maran, T. (2017), Mimicry and Meaning: Structure and Semiotics of Biological Mimicry, Berlin: Springer. McConnell, P. (2003), The Other End of the Leash. Why We Do What We Do around Dogs, New York: Ballantine Books. McGilchrist, I. (2019), The Master and His Emissary: The Divided Brain and the Making of the Western World, 2nd edn, New Haven: Yale University Press. McMahon, A. M. S. (1994), Understanding Language Change, Cambridge: Cambridge University Press. Mendívil-Giró, J. (2019), ‘Did Language Evolve through Language Change? On Language Change, Language Evolution and Grammaticalization Theory’, Glossa, 4 (1.124): 1–30. Mithen, S. J. (2006), The Singing Neanderthals: The Origins of Music, Language, Mind, and Body, Cambridge, MA: Harvard University Press. Müller, M. (1887), The Science of Thought, London: Longmans, Green and Co. Nölle, J., S. Hartmann, and P. Tinits (2020), ‘Language Evolution Research in the Year 2020: A Survey of New Directions’, Language Dynamics and Change, 10 (1): 3–26. Nöth, W. (1999), ‘Peircean Semiotics in the Study of Iconicity in Language’, Transactions of the Charles S. Peirce Society, 35 (3): 613–19. Nöth, W. (2002), ‘Charles Sanders Peirce, Pathfinder in Linguistics’, Interdisciplinary Journal for Germanic Linguistics and Semiotic Analysis, 7 (1): 1–14. Nöth, W. (2008), ‘Semiotic Foundations of Natural Linguistics and Diagrammatic Iconicity’, in K. Willems and L. De Cuypere (eds), Naturalness and Iconicity in Language (Iconicity in Language and Literature 7), 73–100, Amsterdam: John Benjamins. Nöth, W. (2015), ‘Biolinguistics and Biosemiotics’, in E. Velmezova, K. Kull, and S. J. Cowley (eds), Biosemiotic Perspectives on Language and Linguistics, 151–68, Cham: Springer. Pattee, H. H. (2007), ‘The Necessity of Biosemiotics: Matter–symbol Complementarity’, in M. Barbieri (ed.), Introduction to Biosemiotics, 115–32, Dordrecht: Springer. Peirce, C. S. ([1857–1914] 1787–1951), ‘The Charles S. Peirce Papers Harvard University, Houghton Library, MS Am 1632’, Individual papers are referenced by manuscript number in R. Robin (ed.), Annotated Catalogue of the Papers of Charles S. Peirce, Amherst: University of Massachusetts Press, 1967, and in Robin, ‘The Peirce Papers: A Supplementary Catalogue’, Transactions of the Charles S. Peirce Society 7, 1971: 37–57. Cited as MS. Peirce, C. S. ([1866–1913] 1931–58), The Collected Papers of Charles Sanders Peirce, vols. 1–6, C. Hartshorne and P. Weiss (eds), vols. 7–8, A. Burks (ed.), Cambridge: Harvard University Press. Cited as CP. Peirce, C.S. (1877), ‘The Fixation of Belief’, Popular Science Monthly, 12 (November): 1–15. Peirce, C. S. ([1890–2] 2010), ‘The Monist Metaphysical Project’, in Writings of Charles S. Peirce: A Chronological Edition, vol. 8 (1890–1892), Peirce Edition Project (eds), 83–205, Bloomington: Indiana University Press. Cited as W8. Peirce, C. S. ([1893–1913] 1998), The Essential Peirce: Selected Philosophical Writings, vol. 2, ed. Peirce Edition Project, Bloomington: Indiana University Press. Cited as EP 2.

116

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Pelkey, J. (2011), Dialectology as Dialectic: Interpreting Phula Variation, Berlin: de Gruyter. Pelkey, J. (2013a), ‘Analogy, Automation and Diagrammatic Causation: The Evolution of Tibeto-Burman *lak’, Studies in Language, 37 (1): 144–95. Pelkey, J. (2013b), ‘Chiastic Antisymmetry in Language Evolution’, The American Journal of Semiotics, 29 (1): 39–68. Pelkey, J. (2014), ‘Diagnostic Dialectology: Interpreting Ngwi Variation in China’s Red River Valley’, in A. Barysevich, A. d’Arcy, and D. Heap (eds), Proceedings of Methods XIV: Papers from the Fourteenth International Conference on Methods in Dialectology, 2011, 236–48, Frankfurt: Peter Lang. Pelkey, J. (2015), ‘Deep Congruence between Linguistic and Biotic Growth: Evidence for Semiotic Foundations’, in Biosemiotic Perspectives on Language and Linguistics, E. Velmezova, S. J. Cowley, and K. Kull (eds), 97–119, Berlin: Springer. Pelkey, J. (2017a), ‘Greimas Embodied: How Kinesthetic Opposition Grounds the Semiotic Square’, Semiotica, 214 (1): 277–305. Pelkey, J. (2017b), The Semiotics of X: Chiasmus, Cognition, and Extreme Body Memory, London: Bloomsbury Academic. Pelkey, J. (2018), ‘Upright Posture and the Meaning of Meronymy: A Synthesis of Metaphoric and Analytic Accounts’, Cognitive Semiotics, 11 (1): 1–18. Pelkey, J. (2019), ‘Peircean Semiotic for Language and Linguistics’, in Tony Jappy (ed.), The Bloomsbury Companion to Contemporary Peircean Semiotics, 391–418, London: Bloomsbury Academic. Perlman, M. (2017), ‘Debunking Two Myths against Vocal Origins of Language: Language Is Iconic and Multimodal to the Core’, Interaction Studies, 18 (3): 376–401. Perniss, P., L. R. Thompson, and G. Vigliocco (2010), ‘Iconicity as a General Property of Language: Evidence from Spoken and Signed Languages’, Frontiers in Psychology, 227: 1–15. Pietarinen, A.-V. J. (2012), ‘Peirce and Deacon on the Meaning and Evolution of Language’, in T. Schilhab, F. Stjernfelt, and T. Deacon (eds), The Symbolic Species Evolved, 65–80, Berlin: Springer. Pleyer, M., S. Hartmann, J. Winters, and J. Zlatev (2017), ‘Interaction and Iconicity in the Evolution of Language: Introduction to the Special Issue’, Interaction Studies, 18 (3): 303–13. Queiroz, J. and S. Ribeiro (2002), ‘The Biological Substrate of Icons, Indexes and Symbols in Animal Communication: A Neurosemiotic Analysis of Vervet Monkey Alarm Calls’, in M. Shapiro (ed.), The Peirce Seminar Papers, vol. 5, 69–78, Oxford: Berghahn Books. Queiroz, J., F. Stjernfelt, and C. N. El-Hani (2014), ‘Dicent Symbols and Proto-propositions in Biological Mimicry’, in V. Romanini and E. Fernández (eds), Peirce and Biosemiotics (Biosemiotics 11), 199–213, Dordrecht: Springer. Rauch, I. (1981), ‘The Semiotic Paradigm and Language Change’, in J. Deely and M. D. Lenhart (eds), Semiotics 1981, 193–200, New York: Plenum Press. Rauch, I. (1999), ‘Peirce and Linguistics’, in I. Rauch (ed.), Semiotic Insights: The Data Do the Talking, 61–72, Toronto: University of Toronto Press. Roberts, G. and B. Sneller (2020), ‘Empirical Foundations for an Integrated Study of Language Evolution’, Language Dynamics and Change, 10 (2): 188–229. Sapir, E. (1921), Language: An Introduction to the Study of Speech, New York: Harcourt, Brace and Company. Sapir, E. (1929), ‘The Status of Linguistics as a Science’, Language, 5 (4): 207–14.

SEMIOTICS IN EVOLUTIONARY LINGUISTICS

117

Saussure, F. de (1916), Cours de linguistique générale, ed. C. Bally and A. Sechehaye, Paris: Payot. Savage-Rumbaugh, E. S. (1986), Ape Language: From Conditioned Response to Symbol, New York: Columbia University Press. Schleicher, A. ([1869] 1983]). ‘Darwinism Tested by the Science of Language’, in E. F. K. Koerner (ed.), Linguistics and Evolutionary Theory: Three Essays (Amsterdam Studies in the Theory and History of Linguistic Science 6), 3–72, Amsterdam: John Benjamins. Sebeok, T. A. (1972), Perspectives in Zoosemiotics, The Hague: Mouton. Sebeok, T. A. (1986a), ‘Communication, Language, and Speech: Evolutionary Considerations’, in T. A. Sebeok (ed.), I Think I Am a Verb: More Contributions to the Doctrine of Signs, 10–16, New York: Plenum Press. Sebeok, T. A. (1986b), ‘The Problem of the Origin of Language in an Evolutionary Frame’, Language Sciences, 8 (2): 169–76. Sebeok, T. A. (1987), ‘Toward a Natural History of Language’, Semiotica, 65 (3/4): 343–58. Sebeok, T. A. (1991a), ‘In What Sense Is Language a Primary Modeling System?’ in M. Anderson and F. Merrell (eds), On Semiotic Modeling, 327–40, Berlin: Mouton de Gruyter. Sebeok, T. A. (1991b), ‘Linguistics and Semiotics’, in T. A. Sebeok (ed.), A Sign Is Just a Sign, 59–67, Bloomington: Indiana University Press. Sebeok, T. A. (1998), ‘The Estonian Connection’, Sign Systems Studies, 26: 20–41. Sebeok, T. A. (1999), ‘Editor’s Note: Towards a Prehistory of Biosemiotics’, Semiotica, 127 (1/4): 1–3. Sebeok, T. A. (2001a), ‘Biosemiotics: Its Roots and Proliferations’, Semiotica, 134 (1): 18–31. Sebeok, T. A. (2001b), Global Semiotics, Bloomington, IN: Indiana University Press. Sebeok, T. A. (2001c), Signs: An Introduction to Semiotics (Toronto Studies in Semiotics and Communication), 2nd edn, Toronto: University of Toronto Press. Sebeok, T. A. (2001d), ‘Nonverbal Communication’, in P. Cobley (ed.), The Routledge Companion to Semiotics and Linguistics, 14–27, London: Routledge. Sebeok, T. A. and J. Umiker-Sebeok (1980), Speaking of Apes: A Critical Anthology of Two-way Communication with Man, New York: Plenum. Sebeok, T. A. and M. Danesi (2000), The Forms of Meaning: Modeling Systems Theory and Semiotic Analysis, Berlin: Mouton de Gruyter. Shapiro, M. (1980), ‘Russian Conjugation: Theory and Hermeneutic’, Language, 56 (1): 67–93. Shapiro, M. (1983), The Sense of Grammar: Language as Semeiotic, Bloomington: Indiana University Press. Shapiro, M. (1985), ‘Teleology, Semeiosis, and Linguistic Change’, Diachronica, 2: 1–34. Shapiro, M. (1987), ‘Sapir’s Concept of Drift in Semiotic Perspective’, Semiotica, 67: 159–71. Shapiro, M. (1991), The Sense of Change: Language as History, Bloomington: Indiana University Press. Shapiro, M. (2002), ‘Aspects of a Neo-Peircean Linguistics: Language History as Linguistic Theory’, in M. Shapiro (ed.), The Peirce Seminar Papers: Essays in Semiotic Analysis, vol. 5, 108–25, Oxford: Berghahn. Sheets-Johnstone, M. (1990a), The Roots of Thinking, Philadelphia: Temple University Press. Sheets-Johnstone, M. (1990b), ‘The Origin of Counting: A Rethinking of Upright Posture’, in M. L. Foster and L. Botscharow (eds), The Life Of Symbols, 45–62, London: Routledge. Sheets-Johnstone, M. (2011), The Primacy of Movement (2nd edn), Amsterdam: John Benjamins. Skinner, B. F. (1957), Verbal Behavior, La Jolla: Copley Publishing Group.

118

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Sklar, R. (1968), ‘Chomsky’s Revolution in Linguistics’, The Nation, 213–17. Steels, L. (2011), ‘Modeling the Cultural Evolution of Language’, Physics of Life Reviews, 8 (4): 339–56. Stjernfelt, F. (2007), Diagrammatology: An Investigation on the Borderlines of Phenomenology, Ontology, and Semiotics, Berlin: Springer. Stjernfelt, F. (2012), ‘The Evolution of Semiotic Self-Control’, in T. Schilhab, F. Stjernfelt, and T. Deacon (eds), The Symbolic Species Evolved, 39–63, Dordrecht: Springer. Stjernfelt, F. (2013), ‘The Generality of Signs’, Semiotica, 194 (1): 1–33. Stjernfelt, F. (2014), Natural Propositions: The Actuality of Peirce’s Doctrine of Dicisigns, Boston: Docent Press. Tattersall, I. (2018), ‘Language Origins: An Evolutionary Framework’, Topoi, 37 (2): 289–96. Tønnessen, M. (2015), ‘Umwelt and Language’, in E. Velmezova, K. Kull, and S. J. Cowley (eds), Biosemiotic Perspectives on Language and Linguistics, 77–96, Cham: Springer. https:// doi.org/10.1007/978-3-319-20663-9_5. Terrace, H. S. (1979), Nim, New York: Knopf. Tylén, K., R. Fusaroli, S. Rojo, K. Heimann, N. Fay, N. N. Johannsen, F. Riede, and M. Lombard (2020), ‘The Evolution of Early Symbolic Behavior in Homo Sapiens’, Proceedings of the National Academy of Sciences, 117 (9): 4578–84. Uexküll, G. v. (1964), Jakob von Uexküll. Seine Welt und seine Umwelt, Heidelberg: Carl Winter. Uexküll, J. v. (1902), ‘Psychologie und Biologie in ihrer Stellung zur Tier-seele’, Ergebnisse der Physiologie, 1: 212–33. Uexküll, J. v. (1920), Theoretische Biologie, Heidelberg: Suhrkamp. Uexküll, J. v. (1982 [1940]), ‘Theory of Meaning’, trans. B. Stone and H. Weiner, Semiotica, 42 (1): 25–47. Uexküll, T. v. (1982), ‘Introduction: Meaning and Science in Jakob von Uex-küll’s Concept of Biology’, Semiotica, 42 (1): 1–24. Uexküll, T. v., W. Geigges, and J. M. Herrmann (1993), ‘Endosemiosis’, Semiotica, 96 (1/2): 5–51. Van Lier, H. (2003), ‘Around Homo in 80 Theses: A Fundamental Anthropogeny’, M. Lo (ed.), P. Lottefier (trans.), Online: Anthropogenie.com. Van Lier, H. (2010), Anthropogénie, Paris: Impressions nouvelles. Velmezova, E., K. Kull, and Stephen J. Cowley, eds (2015), Biosemiotic Perspectives on Language and Linguistics, Berlin: Springer. Verhagen, F. C. (2008), ‘Worldviews and Metaphors in the Human-Nature Relationship: An Ecolinguistic Exploration through the Ages’, Language & Ecology, 2 (3): 1–19. Wade, N. (1980), ‘Does Man Alone Have Language? Apes Reply in Riddles, and a Horse Says Neigh’, Science, 208: 1349–51. Wirth, L. (1928), ‘Theoretical Biology by J. von Uexküll’, D. L. Mackinnon trans. (book review), The American Journal of Sociology, 33 (6): 995–8. Wyhe, J. v. (2005), ‘The Descent of Words: Evolutionary Thinking 1780–1880’, Endeavour, 29 (3): 94–100. Zlatev, J., P. Żywiczyński, and S. Wacewicz (2020), ‘Pantomime as the Original Human-specific Communicative System’, Journal of Language Evolution, 5 (2): 156–74.

CHAPTER SIX

Semiotics in Health and Medicine JOHN TREDINNICK-ROWE AND DONALD E. STANLEY

INTRODUCTION This chapter provides an overview of the role semiotics has played in medicine and healthcare. We outline its roots in classical antiquity; specifically covering the emergence of Greek and Roman use of signs. Further to which we provide inferences about the use of medical semiotics in the Latin Age. The discussion then moves into the role medical semiotics played in eighteenth- and nineteenth-century diagnostics techniques, prior to the rapid technological developments in medicine that have shaped modern healthcare. The most current use of an explicitly semiotic method is found in clinical skillcourses, where medical students are taught semiology as an introduction to the basics of disease identification, and categorization (aetiology). This method is most heavily used in psychiatry and neurology in Latin American countries. We also cover the use of Peircean abductive reasoning to explain diagnostic decision-making. A clear explanation of how Peircean semiotics can be applied in the form of vignettes of diagnostic scenarios is provided. The chapter concludes by discussing contemporary issues in medical semiotics, some prominent topics and authors, as well as proposing areas for future work. There are many areas of clinical practice in which semiotics could be applied. However, some of the issues medical semiotics faces are consistent with concerns in semiotics more widely. Namely, the stranglehold of en vogue postmodern theorists dominates doctor-patient discourse analysis, which could be a fecund area for semiotics. Nevertheless, we see a future for semiotics to function as a qualitative counterpoise to existing bio-statistical approaches in medicine and healthcare.

Overview The use of semiotics in medicine and healthcare, in particular, remains a marginal topic in comparison to other established topics, such as socio-semiotics, or biosemiotics for which there are designated journals and book series. However, the reader will find a rich vein of semiotic literature spanning several millennia in this chapter. We provide an overview of the role semiotics has played in medicine and healthcare, organized in concise summaries. The chapter starts with the historical development of semiotics in medicine, healthcare and clinical subjects more widely. As an area inquiry, medical semiotics has its roots in classical antiquity, and we specifically cover  the

120

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

emergence of Greek1 and Roman use of signs in medicine, and to a lesser extent the use of signs in Mesopotamian divination. The text moves through the enlightenment and into the use of medical semiotics in the eighteenth and nineteenth centuries. The chapter then transitions into the current methodological approaches used in the subject. Here, there is a particular focus on the use of semiotics in diagnosis, clinical decision-making and Peircean abduction, explained through vignettes of diagnostic scenarios. We move into a discussion about contemporary works that use semiotics from a large variety of medical and healthcare specialities, disease categories and other medical topics. We conclude with proposed areas for future research drawing on the corpus of literature covered forthwith.

Background In the most extensive monograph of the twentieth century on medical semiotics, Bær (1988: 1) states that the ‘art of healing, in Greek antiquity, was called techne semeiotike, a craft of having to do with signs’. This term he describes as meaning a semiotic craft or ‘the skill to interpret semeia, signs’ (1988: 41). Baer suggested that the value of applying semiotics to medicine was that it could provide medicine with: A grammar of signs, refining the syntax of symptoms, disclosing their pluridimensional semantic richness, and proposing a dialogistic pragmatics of how to interact with the patient. (Bær 1988: 2) This statement is a concise summary of the literary corpus around semiotics use in medicine. In this chapter, we hope to broaden the view of medical semiotics both chronologically, linguistically (as many texts are Anglophone) and culturally, to incorporate ideas originating in medical semiotics outside of the Greek/Western world. The reader will also find the semiotics of medicine can have a substantial overlap with other subjectdisciplines such as biology (biosemiotics), zoology (zoösemiotics), ethics (semio-ethics), education (edusemiotics) and cognitive semiotics – see Jensen Thomas et al. (2019). A fleeting glance to the etymological roots of semiotics will help to foreground some of the often entwined and coextensive historical developments in sign use and medicine. Whilst it is well-established that semiotics as a term derives from the Greek sēmeion (σημεῖον), ‘a sign, a mark’, less frequently considered is the root of the noun, which is not a general name for the doctrine of signs, but rather ‘a synonym for that specific branch of medicine concerned with one class of Greek σημεία or Latin signa naturalia, namely, symptoms, the signs of diseases’ (Deely 2006: 76). In the seventeenth century, sign use still retained natural connotations as the signa naturalia in the works of Locke 1690 or Poinsot 1632 (Ars Logica). A period of time that John Deely (2009) considers protosemiotic, in that Augustine was the first to describe what the sign does, and Poinsot the first to show how signs mediate culture and nature. However, moving out of the enlightenment, if we inspect lexicographical definitions of semiotics (as a medical topic) it is clearly coterminous with semiology, as a branch of medical science concerned with symptoms. This interrelationship is noted in Goodrich (1850) and Ogilvie (1863) as well as being a synonym for symptomatology (Deely 2006). By the late nineteenth century the terms had shed their specifically medical connotations.

SEMIOTICS IN HEALTH AND MEDICINE

121

Deely (2006) further illustrates the conflation of the two terms in the 1883 dictionary of Annandale:

–

Semeiological, a. Relating to semeiology or the doctrine of signs; specifically, pertaining to the symptoms of diseases.

–

Semeiotic, a. Relating to semeiotics; pertaining to signs; specifically, relating to the symptoms of diseases; symptomatic.

–

Semeiotics, n. [Gr. σημείον, a mark, a sign] 1. The doctrine or science of signs; the language of signs. – 2. In pathology. that branch which teaches how to judge of all the symptoms in the human body, whether healthy or diseased; symtomatology; semeiology. (Annandale 1883)







In contemporary scholarship the fundamental distinction between semiology (de Saussure) and semiotics (Peirce) is that ‘semiology concerned itself only with intentional communication acts, such as speaking and writing, or other related forms such as gesture and Morse code, Peircean semiotics included all sensory stimuli that could create another idea in the receiver’s mind’ (Daylight 2012: 37). Furthermore, in the modern era, Ferdinand de Saussure’s semiology has become regarded as a limited subset of semiotics due to its glottocentric perspective (Daylight 2012: 37). To couch this in relevance to medicine, semiotics can account for the unintentional use of signs as well as signa naturalia, whereas semiology is applicable only to intentional communicational acts. This point is extended by Thomas Sebeok’s medical commentary on indexical signs: The essential point here is that the indexical character of the sign would not be voided if there were no interpretant, but only if its object were removed. An index is that kind of a sign that becomes by virtue of being really (i.e., factually) connected with its object. ‘Such is a symptom of disease’ (Peirce 8.119). All ‘symptoms of disease’, furthermore, ‘have no utterer’, as is also the case with ‘signs of the weather’ (8.185). We have an index, Peirce prescribed in 1885, when there is ‘a direct dual relation of the sign to its object independent of the mind using the sign […] of this nature are all natural signs and physical symptoms’ (3.361). (Sebeok 2001b: 70–1) One point to take away from this lexicographical discussion is that the study of medical signs is antecedent primum mobile for all modern incarnations of semiotics, be it Saussurean,2 Peircean, Barthesian, etc., which function as an extension of the original symptomatic and physical doctrine of signs; projected into social and non-anthropocentric domains (i.e. biosemiotics, zoosemiotics, ecosemiotics, etc.), many of which can be found elsewhere in this volume.

HISTORY In this section, the reader is presented with a historiographic account of clinical methods involving semiotics and semiology. From this etymological basis, we can establish a firm Greek pedigree for the use of semiotics in medicine¸ described by Cobley (2009: 264) as ‘the process that professional physicians followed in evaluating signs of body disorder understanding their cause, offering therapy where beneficial, and prognosticating

122

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

the patient’s future’. This is indeed the case for how the doctrine of signs progressed into modern Western medicine. However, these processes did not occur solely in the Hellenic world, and as a starting point we must consider Mesopotamia and other ancient cultures.

Cuneiform divination Carlo Ginzburg (1979) characterized the divination processes of the Mesopotamian cultures as employing abductive reasoning (of protracted a priori inferences) with no small measure of luck. Typically, this involved a series of complex propositions using what we would call operators today ‘IF, OR, AND etc.’ Leading inevitably to a future tense prognosis. This clinical decision-making takes the form ‘If P then Q’, similar to the propositional logic for diagnosis. More specifically, the diagnostic process involved moving from protasis (a clause in a conditional statement) to apodosis (the main clause in a conditional statement). One difference between the Mesopotamian and Greek diagnostic procedures was that typically context was linked through a material form for the Mesopotamians and substance for the Greeks (Manetti 1993). Continuing our diachronic analysis, we progress to the Greek father of Western medicine. It would be remiss to omit Hippocrates, who was the first to move medicine as a subject from the province of divine soothsayers to secular applications; in so doing, taking the first step to explicate the subject from superstition and put it on a naturalistic and rational footing (Baer 1991). Near-Eastern and Hellenic foundation for medical semiotics has been established. But it must be clearly stated that concurrent developments in medical sign system occurred in other classical cultures such as China (Xia, Staiano-Ross and Day 2014), and can be found in Sanskrit Āyurvedic texts (Selby 2005), or one could turn to the semiotic analyses of health communication in Quranic for example (Basit 2017). It is almost certain that use of signs as the basis for divinatory or ritual healing processes fed into the medical practice of other classical civilizations outside of Euphrates-Tigris river basin, and the Greek poleis. However, what is often lacking is the written traditions of such cultures (or translations into major European languages), from which we are able to reconstruct medical practices and customs.

Medicine semiotics from the enlightenment The Enlightenment in Europe represents a period in which a burgeoning medical literature emerged that we can interrogate for its use of signs/diagnostic approaches. Looking at medical practitioners in this period Thomas Sydenham’s Observationes Medicae (1676) inspired key figures such as Etienne de Condillac (1714–80) and Pierre-Jean-Georges Cabanis (1757–1808), who created nosographic categories based on signs. De Condillac and Cabanis raised objections to how medical sense-data would typically be considered in isolation from medical theories of signs and symptoms. That is, along the lines of Hume’s criticism of naïve empiricism (Rudnytsky and Charon 2008). Similar attention should be paid to Johann Georg Zimmermann’s Erfahrung in der Arzneykunst (1763) and Philippe Pinel’s Nosographie Philosophique (1789), whose works transformed traditional prognosis-orientated symptomatology into a more semiotic endeavour. Several other relevant medical texts from this period that draw upon sign-systems worthy of note include Herman Boerhaave’s posthumous surgical text (Boerhaave 1742), Hieronymus David Gaub’s work on disease theory (Gaub 1797) and his Institutiones

SEMIOTICS IN HEALTH AND MEDICINE

123

pathologiae medicinalis (1759) in which he goes into great depth about the symptomata of a wide variety of diseases (Gaub 1759). Synchronously, Ernst Anton Nicolai developed a theorem of the hallmarks of the internal state of the human body. Nicolai (1756) presents a semiotic interpretation of the physiologist and pathologist Dr Samuel Schaarschmidts’s work, in particular, he is critical of older medical semiotics text’s idée fixe with bodily functions, rather than writing about a causal analysis of them. The German-language pathology and psychopathology literature in the nineteenth century retains a more general concept of signs as a diagnostic unit, such as Feodorovich and Hippius’s 1892 work Semiotics and Diagnosis of Childhood Diseases. Or Hufeland (1823) in the Journal of Obstetric Practice who wrote on the value and importance of semiotics, similarly Becker (1832) addressed the role of semiotics concerning cardiology in his Zur Physiologie und Semiotik der Herzthätigkeit. This corpus represents part of a wider shift in nineteenth-century medicine away from the nosological classification of diseases towards pathogenesis (Maehle 1995). Typically, the clinical method at this time would draw from autopsies, chemistry and microscopy3 to explain symptoms. In the twentieth and twenty-first centuries, the link between semiology as a form of pathology continued in standard medical texts in Europe and South America. However, with the advent of technological solutions for analysing neurophysiological disorders like seizures and fits, semiology moved into a distinctive new phase, as more in-depth analysis of bodily functions became possible. The distinct semaphores and non-verbal communication of patients (particular physical movement) is the semiology. This is used to differentiate neurological disease categories, whereby the doctors’ clinical reasoning corresponds to the correct identification of a patient’s semiology with the electroencephalograms (EEG) or MRI reading to triangulating data sources (semiotics and extra-semiotic) therefore locating the part of the brain with neurological damage. Further illumination of current medical semiotic research is given in the Methodologies section of this chapter (see below). This interrogation of the historical roots of semiotics and medicine has helped to isolate the use of medical signs from semiotics more generally. Medical semiotics not only birthed Western medicine but continues to be taught today in clinical training. In some instances, in the Dominican Republic for example, semiotics has been taken as the basis of entire medical school curricula (San-Martín, Delgado-Bolton, and Vivanco 2017). The longevity of the subject speaks to its fundamental importance in medicine, even if its prestige has diminished since the end of the twentieth century (MacBryde and Blacklow 1971). In short, semiotics with its ability to describe objectivity, subjectivity, inter-subjectivity and suprasubjective relations (Deely 2014) is a higher fidelity representation of reality than the mere accumulation of isolated material facts loosely linked by differing physical laws. That is to state when Sebeok (1978: 181) glossed that medical semiotics was a tripod of medicine, linguistics and philosophy it was to demonstrate the depth and breadth of the subject. Similarly, Peirce’s statement that all this universe is perfused with signs, if not composed exclusively of them (1906: CP 5.448, fn 6) reflects this same fundamental point that human beings are essentially Semiotic Animals – animal semeioticum (Deely 2005). This is perhaps one of the fundamental statements that undergird the semiotics of health and medicine. In Sebeok’s poignant phase, life is co-existential with semiosis (Sebeok 2001a).

124

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

METHODOLOGIES Methodologically speaking, there is a clear delineation between medicine and medical semiotics as disciplines, even if one is the progeny of the other. However, if we consider the overall structure of both subjects, a basic comparison can be made. According to a conversation between Kull (2017: 200) and Umberto Eco, Umberto remarked: It is not a science [semiotics] in the way physics is a science because it is not accumulative like science is. […] So semiotics, I always say, is like medicine. […] I think that semiotics is something like this – a confederation of disciplines, sometimes using different methods. One can see that structurally as disciplines, both medicine and semiotics contain a variety of heterogeneous, but linked specialities, which may use different methods. Similarly, in this entry, it is clear that a variety of different methodological approaches from different semiotic schools and epochs can be applied spanning pathology, person-centred medicine and population health/epidemiology. In this peculiar way, the two areas share a structural affinity through their diverse approaches, as well as a common origin.

Medical semiotics and clinical education In real-world clinical scenarios, the most current use of explicitly semiotic methods is in undergraduate clinical skills courses, where medical students are taught semiology as an introduction to basic disease identification, and categorization (aetiology). This semiological approach is typically introduced as a general method, applicable to many areas of medicine and will more often be found in South and Central American schools. For example, see Pérez Bada and Quintana López (2018). However, there exits medical education literature for specific specialities that draw from a semiotic approach, e.g. pulmonary radiology (Carette et al. 2008), paediatrics (Petit 2011), neurophysiology (Müller and Wolff 2003). Once more, the reader might be interested to note the interconnections between medical semiotics and the semiotics of education (Stables and Semetsky 2014; Olteanu 2015) in the way medicine is taught (Tredinnick-Rowe 2018). In varying degrees, the clinical skills courses in modern universities do not deviate from the sign ‘(P) – disease (Q) relationship (If P ⊃ Q)’ used by Mesopotamian and Greek auguries. Either through a direct, combined or weighted causation process, aetiological discussions follow similar lines of varying complexity as the student matures. That is, if I smell sweetness (the representamen) on a patient’s breath, this indicates a chemical known as ketones (object) occurring in a process known as ketoacidosis (build-up of ketones). Ketoacidosis is a serious condition in a person with type-1 diabetes, the symptoms of which include thirst, fatigue and problems with vision. This syllogistic movement could be (and probably was) performed without any knowledge of ketoacidosis, matching the olfactory sign to the physical symptom of diabetes (thirst, fatigue, poor vision). As such, what undergraduate medical students learn is the same syllogistic diagnostic logic as their classical forebears, this ‘pre-diagnostic semeiology’4 (Hess 1993) has remained largely unaltered since it was written in cuneiform up until the early eighteenth

SEMIOTICS IN HEALTH AND MEDICINE

125

century (Steinert 2013). After which the gradual union of specific biochemical and biostatistical knowledge around a condition became ever more foundational for medical and healthcare students’ education. That is to state, whilst the original aetiological approach with a syllogistic base in signa naturalia as a diagnostic method remain similar, it has been overshadowed by the vast developments in philosophia naturalis5 since the early eighteenth century. Consequently, the tail-end of millennia old medical practice has been worn down to the first step a junior doctor takes on the journey to becoming a clinical professional.

Biosemiotics and medicine Despite its long-standing history, the literature concerning semiotics in medicine and health has achieved minimal connection with its disciplinary counterpart, Biosemiotics. The two areas remain largely separate in academic terms. A lowly number of publications inhabit this intersection, the most extensive being the edited collection Biosemiotic Meaning by Goli (2016). Also, we find the following titles that touch on healthcare or medical topics: 1. Addiction (Alter 2014) 2. Cancer (Ehlers and Krupar 2012) 3. Disability (Rogers and Swadener 2001) 4. Homoeopathy (Brands 2006) 5. Obesity (McLaren 2015) 6. Symptoms (Staiano-Ross 2012; Malterud et al. 2015) 7. Psychoanalysis (Litowitz and Epstein 1991; Aragno 2012) 8. Psychopathology (Bielecka and Marcinów 2017) The hesitancy of the biosemiotic scholars to engage with healthcare-related issues may be due to medical science’s continued insistence on separating sema (meaning and signification) from the soma (physical body) (Hoffmeyer 2012). One potential avenue to reconnect biosemiotics and medicine would be the corpus of work around Jakob von Uexküll’s Umwelt theory, as noted by Maran (2014: 299). Biosemiotics is firmly a product of the Tartu-Moscow School, with the CopenhagenTartu school emerging in more recent years. In Copenhagen, leading biosemiotic figures such as Jesper Hoffmeyer have ruminated about medical and healthcare topics on occasion, see Hoffmeyer (1997: 2000). Of particular note is Hoffmeyer’s work a biosemiotic approach to health (Hoffmeyer 2010). Outside of the Tartu-Copenhagen schools of semiotics Umberto Eco (2018)6 outlined the importance of the Italian oncologist Giorgio Prodi (1928–87). His work on the material bases of signification (Prodi 1988b), and immunology (Prodi 1988a) form another possible bridge between a medical discipline and biosemiotics. One of the most prolific medical semiotics scholars, Kathryn Staiano-Ross, who rooted her work in anthropological studies of ethnic groups in Southern Belize (Staiano 1986), formed an approach known as biocultural semiotics – grounding the theory in medical anthropology and Foucauldian biopolitics. Specifically, ‘proposent une approche qu’ils definissent comme une semiotique bioculturelle, et qui considere le corps en tant qu’objet culturellement marque et la culture comme etant corporalisee’ (Staiano-Ross and Khanna

126

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

1999: 2). The cultural and sociological overtones of Staiano-Ross’ biocultural semiotics approach can be seen in her analysis of Gulf War Syndrome as a socio-political event, shaped through discourse (Staiano-Ross 2007). However, elsewhere she uses extensively Peircean sign categories (Staiano 1982, 1979; Staiano-Ross 2012). Situating her body of work sui generis at the intersection of medical and biological semiotics. This approach is currently exemplified by Puumeister and Ventsel’s Biopolitics Meets Biosemiotics: The Semiotic Thresholds of Anti-Aging Interventions (Puumeister and Ventsel 2018). The analysis of semiotic thresholds is through Peirce’s categories of icon, index and symbol, whilst also drawing on Canguilhem’s and Foucault’s development of biopower. This remains largely virgin territory, but a potentially fecund area for future research in biosemiotics, medicine and health.

Semiotic and clinician-patient interaction The bulk of medical sociology literature is concerned with the speech acts or discourse between clinical staff and patients. This area of work is easily adaptable to socio-semiotic analyses, and the application of semiotics to social-linguistic methods (Skopek 1979). For example, semiotics as a theoretical tool is easily integrable into discourse (Jamani 2011) and conversation analysis (Menz et al. 2010). An attempt to blend semiotic theory and conversation analysis can be found in Bonnin (2019) where he produces a detailed multilingual dissection of psychoanalytical discourse between practitioners and patients. This is done through an interrogation of discursive methodologies supplemented by additional theoretical positions, including linguistic/semiotic landscapes, Bakhtinian theory and deixis. In short ‘discourse analysis […] is analysing semiotic data (predominantly linguistic in my research) as contextualized and embedded in a specific context’ (2019: 17). Bonnin (2019) expands the existing theoretical conceptions of linguistic landscapes (Landry and Bourhis 1997) and semiotic landscapes (Jaworski and Thurlow 2011), by proposing the term discursive landscape, where ‘the linguistic component only exists in some sort of semiotic materiality’ (2011: 69). Similar approaches to hospital ethnography with a semiotic approach can be found in Kahn (1981). The semiotoization of public health via adapting the semiotic theories of landscapes is a potentially fruitful future area for medical semiotics (Lindström, Palang, and Kull 2012).

Abduction: Vignettes of diagnostic scenarios The historically different approaches to diagnosis through signs have been detailed in the previous sections. Here we will illustrate a diagnostic logic rooted in the semiotics of Charles Sanders Peirce and his concept of abduction. Namely, how one moves from generating new diagnostic hypotheses, selecting hypotheses for further pursuit and evaluating their likeliness in light of the available evidence (Stanley and Nyrup 2020). We do this through the use of a vignette of a diagnostic scenario.

Clinical vignette: Malignant lymphoma Patient [Pt]: I have been referred to you from my local doctor who has treated me for 6-months because of sinusitis on my right side. Doctor [DR]: Please tell me how this began? For example, did you have pain, nasal discharge? A cold or the flu?

SEMIOTICS IN HEALTH AND MEDICINE

127

[Pt]: It began about eight-months ago with a feeling of fullness over my right cheek and a bit of stuffiness on both sides of my face. I did not have any nasal discharge, though I sneezed from time to time. [DR]: Did your local Dr advise any testing? [Pt]: After a course of antibiotics, the fullness did not seem to disappear, so I was sent to the ENT specialist who found a 6 mm polyp attached to the septal mucosa and removed it. That relieved my symptoms for a few days. But my doctor ordered an x-ray, the report was that the right maxillary sinus was somewhat opaque but no other remarks. So, I was given another course of different antibiotics over 6 weeks. [DR]: Did you notice any difference after the polypectomy and these two courses of antibiotics? [Pt]: I still feel this stuffiness. [DR]: You said that you did not have any discharge from your nose earlier, is that still the case? [Pt]: A few weeks ago, when I blew my nose, the mucus was pinkish or slightly redpink. I thought it resulted from the polyp biopsy. [DR]: When you went to the radiology department for your x-ray, do you recall what part of your face was examined? [Pt]: Only the right side restricted to the maxilla. [DR]: I would like to examine your neck? [Pt]: Yes, of course, [DR]: Palpates face angle of jaw, neck and throat area. [DR]: You are now retired? [Pt]: Yes, I worked for a company building passenger buses and specifically in the export division examining service contracts and follow-up on repairs. [DR]: Did you travel abroad? [Pt]: Yes, I lived in Argentina, Tucuman, in Perth, Australia and in Bangkok, Thailand. [DR]: How long did you live in Thailand? [Pt]: Three years. [DR]: Do you recall ever having ‘kissing fever’ or mononucleosis? [Pt]: I think that as a young man, while in university, I did have a long recovery from mono, as it was called and spent almost a week in the infirmary because of fatigue. [DR]: ‘Would you give me permission to have a surgical biopsy of the sinus cavity’. [Pt]: Is that the next step? I think, maybe, I think my stuffiness is a little improved? [DR]: I will review the x-rays and see you next week. [DR]: Nasal stuffiness, unilateral, history of Epstein Barr mononucleosis, residence in far East, questionable nasal discharge, short-term response to polypectomy, little or no response to two 6-week courses of different antibiotics, questionable blood-tinged nasal mucous. This patient requires a sinus cavity biopsy to exclude malignant lymphoma of NK-T cell type. Biopsy shows T cell lymphoma expressing CD2, CD56 and cytoplasmic CD3, no surface CD3.

Abductive method explained Peircean abduction is built on several strands of reasoning using different sets of data (Peirce 1903: CP 5.145). The doctor adopts a hypothesis to integrate and to explain the historical, clinical and investigational findings. Abduction follows the examination

128

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

routine to the extent the procedure is processional: i.e. a complete physical examination with a focus on physiologic responses that accounts for an initial evaluation. This is the background knowledge required to begin the process. Our vignette isolates the following information: 1. NK-T cell lymphoma is endemic in the Far East, and the patient resided there. [Epidemiological (population-level) fact]. 2. Epstein-Barr virus is the probable causative virus, as the patient had exposure as a young man [individual fact – medical history]. 3. Sinusitis is common, polyps are also common, but removal usually alleviates symptoms, and sinusitis usually resolves without antibiotic treatment [Epidemiological (population-level) fact]. The patient’s complaint is long-standing, at least 4 months [individual fact]. [False positives – information to discard, narrowing in on associated rule between general disease category and individual presentation of symptoms]. 4. Most cases of NK-T cell lymphoma are stage 1, without nodal involvement [Epidemiological (population-level) fact], the patient has no palpable lymph node [individual fact] and radiology did not show ulceration or lymph node involvement [Confirmation through a diagnostic test – hypoicon based in a rule (thirdness) derived from qualisigns (firstness) to link population-level knowledge to the individual patient]. The diagnosis is, therefore, malignant NK lymphoma of the right maxillary sinus and will be treated after complete staging with radiotherapy. This diagnosis is built on weaving together facts into a strong inferential chain, the more strands in the chain, the stronger the abductive inference. Some strands may not fit (polypectomy), and these red herrings need to be explained, interpolated or ‘fished out’ from strands of the woven chain. This can include heuristic observations such as overt bodily or facial expression, and confirmed by physical examinations. The physician must link together signs, history and symptoms that are indicative of a particular pathology. Background knowledge is the crucial requirement to choose the most fertile hypothesis to test. In the vignette, the logic vacillates in stages between population-level medical facts and the individual patient’s symptoms to establish a causal link, confirmed by the use of matching iconic representations of the disease in the form of radiology exams.

Medical semiotics and semiology Previously we established that Saussurean semiology does not account for natural signs, referential objects or other extra-linguistic functions but rather psychical entities. Here, a very different approach to semiology will be introduced that does not contain a concept of L’arbitraire du signe, but rather anchored in medicine. The medical research on seizures and epilepsy retains the concept of semiology in its literature, mainly concerning the identification of common bodily movements in different types of seizures. This involves combinations of graphical electroencephalography (EEG) data and known patterns of seizure semiology. Despite many years of research, using the visual semiology data of observable patterns to correctly categorising seizure types can be just as effective as using neurological scans. An example EEG is given by Stefan and Gollwitzer (2019) in Figure 6.1.

SEMIOTICS IN HEALTH AND MEDICINE

129

FIGURE 6.1  Example of an EEG reading with associated hyperkinetic movements.

To illustrate this point, consider Wang et al. (2018) who cross-tabulates the semiological features of a seizure (verbal and movement artefacts), corresponding and MRI localization. As such, the relationship between the bodily site of trauma (object), observable semiological features (indexical legisigns) and corresponding interpretation in the MRI output (hypoicon sinsign) is presented. This is shown in Table 6.1. The paralinguistic signs of gesture are used as one piece of evidence to triangulate the location and effect of the seizure. In short, the specific bodily movement (signs) gives intimations to the specific location of the seizure in the brain’s different lobes. Consequently, we see the potential for seizure semiology (as semiotics) to play an important part in the pre-surgical evaluation of epilepsy, and the diagnostic process more widely (Ahmedt-Aristizabal et al. 2018).

STATE-OF-THE-ART There are several different strands of healthcare and medical semiotic literature. A  division in approach and tenor of work is often determined by the author’s clinical status (Tredinnick-Rowe 2018), with medical practitioners adopting more data-driven, case study-based methods, while semioticians without clinical training tend to focus on theoretical and anthropological/socio-semiotic topics. There is an entire literature concerning general issues in medicine and health from a semiotic perspective (Crookshank 1923; Staiano 1979; Von Uexküll 1982; Von Uexküll 1986; Bær 1988; Nessa 1996; Hess 1998; Brands, Franck, and Van Leeuwen 2000; Sebeok 2001c, 1985; Staiano-Ross 2011; Zukowski and Danesi 2019). Closely linked to this is symptomology and semiotics as a diagnostic method (Kahn 1978; Bær 1982; Sebeok 1986; Burnum 1993; Staiano-Ross 2012; Stanley and Campos 2013; Stanley and Sehon 2019; Stanley and Nyrup 2020).

TABLE 6.1  Adapted from Wang et al. (2018) SEEG – Stereo electroencephalography, HMS – hypermotor seizure, dots on the schematic mean the location of the EEG, and the arrows refers to the spread direction. Histopathology

Imaging characteristics

Aura

Effective components

Motor components SEEG or Surgery resection

MRI negative/ bilateral OFC hypometabolism

Uncomfortable feeling in the heart

Fearful expression [Physical Sign]

Sitting up and slight coughing, rocking back and forth, vocalization, frenetic and rapid pedalling movements [Physical Sign]

FCD IIa

MRI negative/R. Temporal pole. R. mesial OFC hypometabolism

Déjà vu

Angry Expression [Physical Sign]

Huffing heavily and sitting up, slapping bed vigorously with bilateral hands [Physical Sign]

FCD Ia

Bottom of R. Superior frontal sulcus abnormality/ R. superior frontal sulcus hypometabolism

Fear/electric feeling in chest

Laughing [Verbal Sign]

Vocalization, rotation to left and right, rapid paddling of bilateral legs, sitting up [Physical Sign]

FCD IIb

SEMIOTICS IN HEALTH AND MEDICINE

131

The literature alternatively can be divided by way of profession in accordance with clinical speciality. Medicine: 1. Gerontology (Stafford 1988) 2. Immunology (Sercarz and Celada 1988; Prodi 1988a; Prodi 1988b) 3. Psychiatry (Shands 1970; Ablamowicz 1994; Davtian and Chernigovskaya 2003; Kuperman and Zislin 2005; du Plessis 2012; Rejón, Altable, and Dening 2013; Huertas 2014; Andersch 2017) 4. Psychosomatic medicine (Uexküll 1979, 1985; Langewitz 2009) 5. Public Health (Brookes and Harvey 2014; Lazard et al. 2017; Holden et al. 2021) Healthcare: 1. Homoeopathy (Walach 1991; Schemm et al. 2002) 2. Neuro-linguistics (Laughlin, McManus, and Stephens 1981; Chernigovskaya Tatiana 1999; Andrews 2011) 3. Nursing (Donnelly 1987; Melo et al. 2017) 4. Occupational Health (Barley 1983) 5. Pharmacy (Nuessel 2002) 6. Psychoanalysis (Aragno 2012) 7. Therapy and Psychotherapy (Peyrot 1987; Lee and Beattie 2000; Barclay and Kee 2001; Keinänen 2003; Kozin 2003; Muntigl 2004; Jensen Thomas et al. 2019) Alternatively, a focus on particular organs or bodily systems can be used to segment the literature, e.g. neuro-semiotics (Grzybek 1993; Roepstorff 2001; Favareau 2002), brain trauma (Goldberg 2017), dermatological-semiotics (Hasan and Haddao 2019), back pain (Hadler 2004). The literature is amenable to division by nosological classification, i.e. specific medical conditions such as aphasia (Price-Williams and Sabsay 1979; Volpe 1991; Landzelius 2009; Novaes Pinto 2013), Alzheimer’s (Gubrium Jaber 1988), anorexia (Prewitt 1992), autism (Smith and Bell 2001; Oakley and Vidanović 2014), chronic pain (Priel, Rabinowitz, and Pels 1991; Honkasalo 2001), depression (Donnelly and Irvin 1990; Catt 2012), dementia (Fleche 2009), fibromyalgia (Quintner et al. 2003), HIV (Tulloch 1992; Namaste 1993; Scalvini 2010; Ferguson 2013), obesity (Jutel 2005; Murray 2007) and schizophrenia (Frow 2001). Beyond this, there are general cross-cutting thematic topics, such as: 1. eHealth (Caiata Zufferey and Schulz 2010; Camerini, Diviani, and Tardini 2010; Neuhauser and Kreps 2010; Schulz and Rubinelli 2010) 2. Nutrition (Knuf and Caughlin 1993) 3. Suicide (Utriainen and Honkasalo 1996) 4. Mindfulness (Chinen 1988) 5. Placebo effect (Schonauer 1993; Miller and Colloca 2010; Goli, Rafieian, and Atarodi 2016)

132

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Umberto Eco’s ideas that medicine and semiotics have an isomorphic structure as a collection of disparate methods were stated at the introduction of this section. To close we wish to expand the idea by drawing from specific points made by Kull (1998: 362) that ‘semiotics is more a standpoint than a set of methods or ideology’. We should therefore not approach medical semiotics from inside the existing frameworks of medicine, but rather be looking to extend the viewpoint and horizons of medical researchers and practitioners who are rooted in pure empiricism.

PRIORITIES Perhaps one of the most obvious but missing uses of semiotics in medicine is its application to discourse studies of clinician-patient communication (Elwood 2012). This could take several different forms, for example, semiotics could easily have a role in the analysis of patient-doctor dialogue, or the dialogue between inter-professional clinical groups, i.e. modern healthcare work is regularly delivered by teams. An example from Bonnin (2019) with a focus on patient voice illustrates one possible way that semiotics can offer an alternative to or complement in vogue poststructuralist French literature that currently dominates this area. A priority for the subject is a more comprehensive dialogue between biosemiotics and the use of semiotics in medicine. Staiano (1992) has led the way in this regard by introducing  medical ethnography from a biosemiotic perspective (biocultural semiotics),  further advanced by Hoffmeyer (2010, 2008) and Puumeister and Ventsel (2018). Beyond biosemiotics, there is an emerging literature that connects semiotics to public health marketing and branding as a behaviour change mechanism, covering topics such as smoking cessation (Moran Stritch et al. 2020). This area is very much at the beginning stage due to the embryonic state of brand-semiotics as an area of inquiry. There is a rich vein of work for semiotician in the literature on seizure semiology, whose approach already involves the pairing of semaphore/non-verbal communication and gesture (see Tantam 1986; Sebeok and Umiker-Sebeok 1981; Nespoulous, Perron, and Lecours 1986) with medical datasets as part of their semio-logical clinical decisionmaking. There is potential for Peircean abductive logic or the semiotics of gesture to pragmatically extend the quantitative, data triangulation and deductive reasoning techniques commonly used in medicine. More widely semiotics can function as a qualitative counterpoise for these techniques. Beyond this discipline, there is considerable overlap between medical semiotics and theoretical areas of psychiatry and psychopathology (Andersch and Cutting 2014; Andersch 2017, 2018). An important current development in this area is Norbert Andersch’s letter to the Editors of World Psychiatry, calling for a World Psychiatry Association Section on ‘Semiotics and Symbol Research.7 This is important because an established semiotics section within a global organization like the WPA would enable it to collect, analyse and disseminate information. One of the fundamental struggles of medial semiotics since the end of the nineteenth century has been its disinvestment in medical schools and teaching hospitals. Having a de facto section  within a global medical institution would be a significant step to rectifying this issue.

SEMIOTICS IN HEALTH AND MEDICINE

133

CONCLUSION From its documented origins, semiotics in medicine has negotiated a labyrinthine path of Babylonian auguries to Hellenic medicine, continuing as a preeminent diagnostics technique up until the nineteenth century in Western civilization. In the modern era, its  use persists in seizure semiology, diagnostic logic and socio-semiotics explanations of health. The divisions between these epochs can be summarised depending on the prevailing attitude to sign-use in medicine and healthcare. The nosological categorization of disease based in signs and the prognostication of indexical bodily sinsigns characterizes much of classical literature up to the eighteenth century (Hasar 2015). The eighteenth and nineteenth centuries shifted from the nosological classification of disease towards an understanding of its wider determinants, i.e. pathogenesis (Maehle 1995). In conjunction with an exponentially increasing understanding in medical conditions, there was a corresponding expansion in the research of social contexts of disease, and hence the role legisigns play. The final medical paradigm to emerge in the late twentieth century was the notion of Salutogenesis, i.e. rather than looking at the causes of illness we are concerned with a paradigm which focuses on the determinants of health and well-being. In conclusion, the role of semiotics in medicine and healthcare has emerged from nosological beginnings that looked to categorize disease, to a formative ‘protosemiotic’ stage with a focus on pathogenesis (causes of disease). To its current approach which takes a salutogenic-basis, uncovering the wider determinants of health. One can see medical semiotics functions over a vital continuum from cells to cultures, based in categorization, causation and the uncovering of determinants. Human beings are, as John Deely so neatly described in his tout court phrase, ‘semiotic animals’ – animal semeioticum (Deely 2005), from this all semiotic scholarship in health and medicine follows.

NOTES 1 An excellent introduction to the semiotics of Galenic medicine (humoralism) can be found in Sebeok (2001c). 2 Deely (2006: 3) would dispute this, stating that ‘Saussure proposed a “science of signs” that determinately excluded the whole order of σημεία (Simeia – sign/mark) in favour of an exclusive concentration upon the realm of σύμβολα (Symbola) … Even though his proposal contains the expression “natural signs”, Saussure does not at all mean by this signs “natural” in the sense of symptoms (σημεία). He means “natural” in the sense of iconic, and specifically such as can be represented in social behavior and interaction on the basis of convention, as he expressly says (Saussure 1916: 68)’. 3 See Schultz (2008) for an in-depth discussion about Rudolf Virchow’s influence on the concurrent development of microscopy and symptomatology. 4 Translation by Risse (1996). 5 Not to be confused with Von Schelling, Hegel’s or later Kant’s idealist Naturphilosophie, where the conception of objects in a priori intuition excludes subjects such as biology and chemistry as science. Rather I refer to the term relating to Bentham (Chrestomathia) and Peirce’s idioscopic conception of science as specialized scientific inquiry rooted in experimentation, but not exclusively rooted in natural science (moderate methodological naturalism); see Peirce’s 1903 ‘A Syllabus of Certain Topics of Logic’ and Gava (2019).

134

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

6 This is the English text of Umberto Eco’s presentation from 1988, made in honour of the Giorgio Prodi ‘Giorgio Prodie la soglia inferiore della semiotica’ 7 In press.

REFERENCES Ablamowicz, H. (1994), ‘Shame as Abject Communication a Semiotic View’, American Journal of Semiotics, 11 (3/4): 155–70. AbdulHasan, A. T. and K. M. Haddao (2019), ‘Non-Adherence among Iraqi ACNE Patients: Dermatosemiotics Study’, Indian Journal of Public Health Research & Development, 10 (11): 2747–50. Ahmedt-Aristizabal, D., C. Fookes, K. Nguyen, S. Denman, S. Sridharan, and S. Dionisio (2018), ‘Deep Facial Analysis: A New Phase I Epilepsy Evaluation Using Computer Vision’, Epilepsy & Behavior, 82: 17–24. Alter, J. S. (2014), ‘Spirits and Exorcism: On the Semiotics of Healing and Recovery’, Ethos, 42 (4): 399–414. Andersch, N. (2017), ‘Towards a Synthesis of Psychiatry and Semiotics. Para uma Síntese Entre Psiquiatria e Semiótica’, Psicopatologia Fenomenológica Contemporânea, 6 (2): 93–111. Andersch, N. (2018), ‘The Unused Potential of Semiotic/Symbolic Research in Psychopathology: A New Key towards the Understanding of Mental Illness’, Proceedings of the WPA Congress: Mental Health an Education. Mental health of a person of the XXI century. Moscow, Russia, 19–23. Andersch, N. and J. Cutting (2014), ‘Ernst Cassirer’s Philosophy of Symbolic Forms and Its Impact on the Theory of Psychopathology’, History of Psychiatry, 25 (2): 203–23. Andrews, E. (2011), ‘Language and Brain: Recasting Meaning in the Definition of Human Language’, Semiotica, 2011 (184): 11–32. Annandale, C. (1883), The Imperial Dictionary of the English Language, London: Blackie & Son. Aragno, A. (2012), ‘The Marriage of Psychoanalytic Methodology with the Biosemiotic Agenda’, Biosemiotics, 5 (2): 247–67. Bær, E. (1982), ‘The Medical Symptom Phylogeny and Ontogeny’, American Journal of Semiotics, 1 (3): 17–34. Bær, E. (1988), Medical Semiotics, Lanham: University Press of America. Bær, E. (1991), ‘Semiotics Presuppositions of Healing’, in B. E. Litowitz and Phillip S. Epstein (eds), Semiotic Perspectives on Clinical Theory and Practice, 1–24, Berlin: Mouton de Gruyter. Barclay, M. W. and M. Kee (2001), ‘Towards a Semiotic Psychotherapy: Semiotic Objects and Semiotic Selves’, Theory & Psychology, 11 (5): 671–86. Barley, S. R. (1983), ‘Semiotics and the Study of Occupational and Organizational Cultures’, Administrative Science Quarterly, 28 (3): 393–413. Basit, A. (2017), ‘Health Communication in The Quran: Charles Saunders Pierce’s Semiotic Analysis’, Malaysian Journal of Communication, 33 (4): 76–88. Becker, F. W. (1832), ‘Zur Physiologie und Semiotik der Herzthätigkeit’, Med. Ztg. v. Ver. f. Helik, 4 (1/15): 63–8. Bielecka, K. and M. Marcinów (2017), ‘Mental Misrepresentation in Non-human Psychopathology’, Biosemiotics, 10 (2): 195–210. Boerhaave, H. (1742), Opera Omnia Medica, Venice, Italy: Laurentium Basilium.

SEMIOTICS IN HEALTH AND MEDICINE

135

Bonnin, J. E. (2019), Discourse and Mental Health: Voice, Inequality and Resistance in Medical Settings, Oxford: Routledge. Brands, M. (2006), ‘Biosemiotics: A Framework for Complementary Medicine?’, Focus on Alternative and Complementary Therapies, 11 (s1): (6). Brands, M., D. Franck, and E. V. Leeuwen (2000), ‘Epistemology and Semiotics of Medical Systems: A Comparative Analysis’, Semiotica, 132 (1–2): 1–24. Brookes, G. and K. Harvey (2014), ‘Peddling a Semiotics of Fear: A Critical Examination of Scare Tactics and Commercial Strategies in Public Health Promotion’, Social Semiotics, 25 (1): 57–80. Burnum, J. F. (1993), ‘Medical Diagnosis through Semiotics: Giving Meaning to the Sign’, Annals of Internal Medicine, 119 (9): 939–43. Caiata Zufferey, M. and P. J. Schulz (2010), ‘Potentialities and Limits of Internet Health Communication in Chronic Care: Results from a Qualitative Study’, Social Semiotics, 20 (1): 61–75. Camerini, L., N. Diviani, and S. Tardini (2010), ‘Health Virtual Communities: Is the Self Lost in the Net?’, Social Semiotics, 20 (1): 87–102. Carette, M. F., A. Parrot, M. Fartoukh, M. Tassart, and A. Khalil (2008), ‘Semiologie de la Vascularisation Systemique Normale et Pathologique du Poumon en Scanner’, Journal de Radiologie, 89 (10): 1403–4. Catt, I. E. (2012), ‘Communicology and the Worldview of Antidepressant Medicine’, American Journal of Semiotics, 28 (1/2): 81–103. Chernigovskaya T. V. (1999), ‘Evolutionary Perspective for Cognitive Function: Cerebral Basis of Heterogeneous Consciousness’, Semiotica, 127 (1–4): 227–38. Chinen, A. B. (1988), ‘Modes of Understanding and Mindfulness in Clinical Medicine’, Theoretical Medicine, 9 (1): 45–71. Cobley, P. (2009), Routledge Companion to Semiotics, Abingdon, Oxford: Routledge. Crookshank, F. G. (1923), ‘The Importance of a Theory of Signs and a Critique of Language in the Study of Medicine’, in C. K. Ogden and I. A. Richards (eds), The Meaning of Meaning: Study of the Influence of Language upon Thought and of the Science of Symbolism, 337–55, New York: Harcourt, Brace. Davtian, S. and T. Chernigovskaya (2003), ‘Psychiatry in Free Fall in Pursuit of a Semiotic Foothold’, Sign Systems Studies, 31 (2): 533–44. Daylight, R. (2012), ‘The Difference between Semiotics and Semiology’, Gramma: Journal of Theory and Criticism, 22: 37–50. Deely, J. (2005), ‘Defining the Semiotic Animal: A Postmodern Definition of Human Being Superseding the Modern Definition “Res Cogitans”’, American Catholic Philosophical Quarterly, 79 (3): 461–81. Deely, J. (2006), ‘On “semiotics” as Naming the Doctrine of Signs’, Semiotica, 158: 1–33. Deely, J. (2009), Augustine and Poinsot: The Protosemiotic Development, 2nd edn, Postmodernity in Philosophy Poinsot Trilogy, Chicago: University of Chicago Press. Deely, J. (2014), ‘Subjectivity, Suprasubjectivity, and Semiosis’, Chinese Semiotic Studies, 10 (3): 375–407. Donnelly, E. (1987), ‘Semiotics and Nursing’, in T. Sebeok and J. Umiker-Sebeok (eds), Semiotic Web, 1986: A Yearbook of Semiotics, 379–87, Berlin Mouton de Gruyter. Donnelly, E. and D. J. Irvin (1991), ‘Depression Diagnosis and Experience’, in J. N. Deely (ed.), Semiotics: 373–82, Lanham: University Press of America. du Plessis, R. (2012), ‘Constructing Patient–Psychiatrist Relations in Psychiatric Hospitals: The Role of Space and Personal Action’, Social Semiotics, 23 (3): 424–43.

136

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Eco, U. (2018), ‘Giorgio Prodi and the Lower Threshold of Semiotics’, Σημειωτκή – Sign Systems Studies, 46 (2–3): 343–51. Ehlers, N. and S. Krupar (2012), ‘The Body in Breast Cancer’, Social Semiotics, 22 (1): 1–11. Elwood, T. W. (2012), ‘Viewing Health Care through a Semiotic Veil of Signs’, Journal of Allied Health, 41 (1): 3–13. Favareau, D. (2002), ‘Beyond Self and Other: On the Neurosemiotic Emergence of Intersubjectivity’, Sign Systems Studies, 30 (1): 57–99. Feodorovich, F. N. and A. Hippius (1892), Semiotik und Diagnostik der Kinderkrankheiten, vol. 8, Stuttgart: Verlag von Ferdinand Enke. Ferguson, A. A. (2013), ‘The Rhetoric of AIDS and the Semiosis of Rage’, in J. Pelkey (ed.), Semiotics 2013: Why Semiotics?, 273–80. Ottawa: Legas. Fleche, A. (2009), ‘Beyond Linguistics: Deixis, Dementia, and the Theatricality of Speech in Alzheimer’s Memoir’, Semiotica, 2009 (174): 165–80. Frow, J. (2001), ‘Metaphor and Metacommunication in Schizophrenic Language’, Social Semiotics, 11 (3): 275–87. Gaub, H. D. (1759), Institutiones Pathologiae Medicinalis, Vienna: Bibliopolae. Gaub, H. D. (1797), Anfangsgründe der Medicinischen Krankheitslehre, trans C. G. Gruner, Berlin, Germany: Boßischen Buchhandlung. Gava, G. (2019), ‘Peirce and Methodological Naturalism’, in Paul Giladi (ed.), Responses to Naturalism: Critical Perspectives from Idealism and Pragmatism, 208–29, Abingdon, Oxford: Routledge. Goldberg, G. (2017), ‘Toward a Postmodern Pragmatic Discourse Semioethics for Brain Injury Care’, Physical Medicine and Rehabilitation Clinics of North America, 28 (2): 393–411. Goli, F., S. Rafieian, and S. Atarodi (2016), ‘An Introduction to the Semiotic Approach to the Placebo Responses’, in F. Goli (ed.), Biosemiotic Medicine. Studies in Neuroscience, Consciousness and Spirituality, Cham: Springer. Goli, F. (2016), Biosemiotic Medicine: Healing in the World of Meaning (Studies in Neuroscience, Consciousness and Spirituality 5), Mill Valley: Springer. Goodrich, C. A. (1850), An American Dictionary of the English Language, Springfield, Mass: George and Charles Merriam. Grzybek, P. (1993), Psychosemiotik, Neurosemiotik = Psychosemiotics, Neurosemiotics, Bochumer Beiträge zur Semiotik, Bochum, Germany: Brockmeyer. Gubrium Jaber, F. (1988), ‘Incommunicables and Poetic Documentation in the Alzheimer’s Disease Experience’, Semiotica, 72 (3–4): 235–54. Hadler, N. M. (2004), ‘Editorial: The Semiotics of Backache’, Spine, 29 (12): 1289. Hasan, A. T. A., and K. M. Haddao (2019), ‘Non-Adherence among Iraqi ACNE Patients: Dermatosemiotics Study’, Indian Journal of Public Health Research & Development, 10 (11): 2747–50. Hasar, R. V. (2015), ‘Symptom without Transcendental Syntax’, Sign Systems Studies, 43 (1): 29–47. Hess, V. (1993), Von der Semiotischen zur Diagnostischen Medizin: Die Entstehung der Klinischen Methode Zwischen 1750 und 1850, vol. 66, Abhandlungen zur Geschichte der Medizin und der Naturwissenschaften, Husum: Matthiesen Verlag. Hess, V. (1998), ‘Medical Semiotics in the 18th Century: A Theory of Practice?’, Theoretical Medicine and Bioethics, 19 (3): 203–13. Hoffmeyer, J. (1997), ‘Somethings or Someones?’, Advances: The Journal of Mind-Body Health, 14 (4): 22–4.

SEMIOTICS IN HEALTH AND MEDICINE

137

Hoffmeyer, J. (2000), ‘The Biology of Signification’, Perspectives in Biology and Medicine, 43 (2): 252–68. Hoffmeyer, J. (2008), ‘The Semiotic Body’, Biosemiotics, 1 (2): 169–90. Hoffmeyer, J. (2010), ‘A Biosemiotic Approach to Health’, in S. J. Cowley, J. C. Major, S. V. Steffensen, and A. Dinis (eds), Signifying Bodies: Biosemiotics, Interaction, and Health, 21–42, Braga: The Faculty of Philosophy of Braga Portuguese Catholic University. Hoffmeyer, J. (2012), ‘The Natural History of Intentionality. A Biosemiotic Approach’, in T. Schilhab, F. Stjernfelt, and T. Deacon (eds), The Symbolic Species Evolved, 97–116, Dordrecht: Springer Netherlands. Holden, A. C. L., P. Neville, B. Gibson, and H. Spallek (2021), ‘Taking Responsibility for the Tooth: A Semiotic and Thematic Analysis of Oral Health and Disease in the TV Show “Embarrassing Bodies”’, Health, 25 (6): 739–56. Honkasalo, M. L. (2001), ‘Pain, Self and the Body’, American Journal of Semiotics, 17 (4): 9–41. Huertas, R. (2014), ‘Subjectivity in Clinical Practice: On the Origins of Psychiatric Semiology in Early French Alienism’, History of Psychiatry, 25 (4): 459–67. Hufeland, C. W. (1823), ‘Ueber den Werth und die Bedeutung der Semiotik’, Journal der Practischen Heilkunde, 8 (128): 3–9. Jamani, K. J. (2011), ‘A Semiotics Discourse Analysis Framework: Understanding Meaning Making in Science Education Contexts’, in S. C. Hamel (ed.), Semiotics Theory and Applications, 191–208, New York: Nova Science Publishers, Inc. Jaworski, A. and C. Thurlow (2011), Semiotic Landscapes: Language, Image, Space, Advances in Sociolinguistics, London: Continnuum-3P. Jensen, T. W., B. T. Sarah, S. B. Marie, and V. S. Sune (2019), ‘“You Know It, How I Feel, I Mean You Just Did It:” The Emergence of We-ness Through Re-enactment in Psychotherapy’, Cognitive Semiotics, 12 (2): 1–15. Jutel, A. (2005), ‘Weighing Health: The Moral Burden of Obesity’, Social Semiotics, 15 (2): 113–25. Kahn, J. Y. (1978), ‘A Diagnostic Semiotic’, Semiotica, 22 (1–2): 75–106. Kahn, J. Y. (1981), ‘The Semiotic Crisis in Contemporary Hospitals’, in J. N. Deely (ed.), Semiotics 1981, 337–43, Lanham: University Press of America. Keinänen, M. (2003), ‘Charles Peirces Sign Terminology, Psychosemiosis, and Psychotherapy: A Clinical Approach’, Semiotica, 2003 (146): 213. Knuf, J. and J. Caughlin (1993), ‘Weighty Issues: Semiotic Notes on Dieting as a Secular Ritual’, Health Communication, 5 (3): 161–79. Kozin, A. (2003), ‘The Sign of Love Gregory Bateson and the Family Therapy Paradigm’, American Journal of Semiotics, 19 (1/4): 221–41. Kull, K. (1998), ‘Semiotic Ecology: Different Natures in the Semiosphere’, Sign Systems Studies, 26 (1): 344–71. Kull, K. (2017), ‘Umberto Eco and John Deely: What They Shared’, Sign Systems Studies, 45 (1/2): 194–204. Kuperman, V. and J. Zislin (2005), ‘Semiotic Perspective of Psychiatric Diagnosis’, Semiotica, 2005 (155): 1–13. Landry, R. and R. Y. Bourhis (1997), ‘Linguistic Landscape and Ethnolinguistic Vitality: An Empirical Study’, Journal of Language and Social Psychology, 16 (1): 23–49. Landzelius, M. (2009), ‘Spatial Reification, or, Collectively Embodied Amnesia, Aphasia, and Apraxia’, Semiotica, 2009 (175): 39–75.

138

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Langewitz, W. (2009), ‘A Theory of Psychosomatic Medicine: An Attempt at an Explanatory Summary’, Semiotica, 2009 (173): 431–52. Laughlin, C. D. J., J. McManus, and C. D. Stephens (1981), ‘A Model of Brain and Symbol’, Semiotica, 33 (3–4): 211–36. Lazard, A. J., A. Schmidt, H. Vu, M. J. Byron, E. Peters, M. H. Boynton, and N. T. Brewer (2017), ‘Icons for Health Effects of Cigarette Smoke: A Test of Semiotic Type’, Journal of Behavioral Medicine, 40 (4): 641–50. Lee, V. and G. Beattie (2000), ‘Why Talking about Negative Emotional Experiences Is Good for Your Health: A Microanalytic Perspective’, Semiotica, 130 (1–2): 1–82. Lindström, K., H. Palang, and K. Kull (2012), ‘Semiotics of Landscape’, in I. Howard and H. Thompson (eds), The Routledge Companion Landscape Studies, 97–107, London: Routledge. Litowitz, B. E. and P. S. Epstein (1991), Semiotic Perspectives on Clinical Theory and Practice: Medicine, Neuropsychiatry and Psychoanalysis, Approaches to Semiotics [AS], Berlin: De Gruyter Mouton. Locke, J. (1690), An Essay Concerning Humane Understanding, London: Printed by Elizabeth Holt for Thomas Basset. MacBryde, C. and R. S. Blacklow (1971), Signs and Symptoms: Applied Pathologic Physiology and Clinical Interpretation, 5th edn, Philadelphia: J.P. Lippincott. Maehle, A. H. (1995), ‘Volker Hess, Von der Semiotischen zur Diagnostischen Medizin: Die Entstehung der Klinischen Methode Zwischen 1750 und 1850, Abhandlungen zur Geschichte der Medizin und der Naturwissenschaften’, Medical History, 39 (4): 513–14. Malterud, K., A. D. Guassora, A. H. Graungaard, and S. Reventlow (2015), ‘Understanding Medical Symptoms: A Conceptual Review and Analysis’, Theoretical Medicine and Bioethics, 36 (6): 411–24. Manetti, G. (1993), Theories of the Sign in Classical Antiquity, Bloomington: Indiana University Press. Maran, T. (2014), ‘Biosemiotic Criticism: Modelling the Environment in Literature’, Green Letters, 18 (3): 297–311. McLaren, G. (2015), ‘The Obesity Crisis and Semiotic Corruption: Towards a Unifying Biosemiotic Understanding of Obesity’, Cosmos and History: The Journal of Natural and Social Philosophy, 11: 181–220. Melo, G. D. S. M., M. P. Tibúrcio, C. C. S. de Freitas, Q. L. D. D. A. Q. D. Vasconcelos, I. K. F. Costa, and G. D. V. Torres (2017), ‘Semiotics and Semiology of Nursing: Evaluation of Undergraduate Students’ Knowledge on Procedures’, Revista Brasileira de Enfermagem, 70: 249–56. Menz, F., J. Lalouschek, M. Sator, and K. Wetschanow (2010), ‘Medizinsemiotik und Gesprächsanalyse’, in F. Menz, J. Lalouschek, M. Sator, and K. Wetschanow (eds), Sprechen über Schmerzen – Linguistische, Semiotische und Kulturelle Analysen, 128–30, Duisberg: Universitätsverlag Rhein-Ruhr. Miller, F. G. and L. Colloca (2010), ‘Semiotics and the Placebo Effect’, Perspectives in Biology and Medicine, 53 (4): 509–16. Moran Stritch, M. J., F. Houghton, D. O’Doherty, D. McInerney, and B. Duncan (2020), ‘Tobacco Control and “Plain Packaging”: Performativity, Cigarettes and the Semiotics of Brand Names’, Irish Journal of Medical Science, 189 (1): 405–7. Müller, A. and J. R. Wolff (2003), ‘Semiotische Aspekte der Neurophysiologie: Neurosemiotik’, in R. Posner, K. T. Robering, and T. A. Sebeok (eds), Semiotik: Ein Handbuch zu den zeichentheoretischen Grundlagen von Natur und Kultur, 2667–97, Berlin: De Gruyter Mouton.

SEMIOTICS IN HEALTH AND MEDICINE

139

Muntigl, P. (2004), ‘Ontogenesis in Narrative Therapy: A Linguistic-Semiotic Examination of Client Change’, Family Process, 43 (1): 109–31. Murray, S. (2007), ‘Corporeal Knowledges and Deviant Bodies: Perceiving the Fat Body’, Social Semiotics, 17 (3): 361–73. Namaste, K. (1993), ‘Semiotics and/as Social Theory: AIDS/HIV Treatment Information and Semantic Intervention’, American Journal of Semiotics, 10 (1/2): 177–200. Nespoulous, J. L., P. Perron, and A. R. Lecours (1986), The Biological Foundations of Gestures: Motor and Semiotic Aspects, Neuropsychology and Neurolinguistics) (Neuropsychology and Neurolinguistics Series, New York: Psychology Press. Nessa, J. (1996), ‘About Signs and Symptoms: Can Semiotics Expand the View of Clinical Medicine?’, Theoretical Medicine, 17 (4): 363–77. Neuhauser, L. and G. L. Kreps (2010), ‘eHealth Communication and Behavior Change: Promise and Performance’, Social Semiotics, 20 (1): 9–27. Nicolai, E. A. (1756), Dr. Samuel Schaarschmidts Semiotic oder Lehre von den Kennzeichen des innerlichen Zustandes des menschlichen Körpers mit Zusätzen vermehret, Berlin, Germany: Gottlieb August Lange. Novaes Pinto, R. (2013), ‘The Aphasic Utterance: A Significal Perspective’, Semiotica, 2013 (196): 457–72. Nuessel, F. (2002), ‘Errors in Medical Prescriptions A Semiotic Perspective’, in J. N. Deely (ed.), Semiotics 2002, 60–75, Ottawa: Legas. Oakley, T. and Đorđe V. (2014), ‘Deixis and Personhood: Autism, Intersubjectivity, and the Fluidity-to-Inelasticity Hypothesis’, Cognitive Semiotics, 7 (2): 191. Ogilvie, J. (1863), The Imperial Dictionary, English, Technological, and Scientific; Adapted to the Present State of Literature, Science, and Art, Glasgow: Blackie and Son. Olteanu, A. (2015), Philosophy of Education in the Semiotics of Charles Peirce: A Cosmology of Learning and Loving, Oxford: Peter Lang. Peirce, C. S. ([1866–1913] 1931–58) The Collected Papers of Charles Sanders Peirce, vols. 1–6, eds. C. Hartshorne and P. Weiss, vols. 7–8, ed. A. Burks, Cambridge: Harvard University Press. Cited as CP. Peirce, C. S. (1903), A Syllabus of Certain Topics of Logic, pamphlet. Boston: Alfred Mudge & Son. Pérez B. E. and L. A. A. Quintana (2018), ‘Evaluation of the Development of Clinical Skills in 3rd Year Medicine Students, Villa Clara’, EduMeCentro, 10 (1): 92–108. Petit, A. (2011), ‘Enseignement Simule de Pediatrie en Deuxieme Cycle des Etudes Medicales. Simulated Teaching of Pediatrics in the Second Cycle of Medical Studies’, Masters in Medical Education, Hôpitaux Universitaires Paris Est, Université Pierre et Marie Curie. Peyrot, M. (1987), ‘Circumspection in Psychotherapy: Structures and Strategies of CounselorClient Interaction’, Semiotica, 65 (3–4): 249–68. Pinel, P. (1789), Nosographie Philosophique, ou, La Méthode de l’analyse Appliquée a la Médecine, Paris: Chez Maradan. Prewitt, T. (1992), ‘Unholy Anorexia Blood Myth and the Signs of the Flesh’, in J. N. Deely (ed.), Semiotics 1992, 300–19, Lanham: University Press of America. Price-Williams, D. and S. Sabsay (1979), ‘Communicative Competence among Severely Retarded Persons’, Semiotica, 26 (1–2): 35–64. Priel, B., B. Rabinowitz and R. J. Pels (1991), ‘A Semiotic Perspective on Chronic Pain: Implications for the Interaction between Patient and Physician’, British Journal of Medical Psychology, 64 (1): 65–71. Prodi, G. (1988a), ‘Signs and Codes in Immunology’, in E. E. Sercarz and F. Celada (eds), The Semiotics of Cellular Communication in the Immune System, Berlin: Springer-Verlag.

140

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Prodi, G. (1988b), ‘Material Bases of Signification’, Semiotica, 69 (3–4): 191–242. Puumeister, O. and A. Ventsel (2018), ‘Biopolitics Meets Biosemiotics: The Semiotic Thresholds of Anti-Aging Interventions’, Theory, Culture & Society, 35 (1): 117–39. Quintner, J., D. Buchanan, M. Cohen, and A. Taylor (2003), ‘Signification and Pain: A Semiotic Reading of Fibromyalgia’, Theoretical Medicine and Bioethics, 24 (4): 345–54. Rejón Altable, C. and T. Dening (2013), ‘Psychopathology beyond Semiology. An Essay on the Inner Workings of Psychopathology’, History of Psychiatry, 24 (1): 46–61. Risse, G. B. (1996), ‘Review of the book Von der Semiotischen zur Diagnostischen Medizin: Die Entstehung der Klinischen Methode Zwischen 1750 und 1850’, Bulletin of the History of Medicine, 70 (4): 715–16. Roepstorff, A. (2001), ‘Brains in Scanners: An Umwelt of Cognitive Neuroscience’, Semiotica, 2001 (134): 747–65. Rogers, L. J. and B. B. Swadener (2001), Semiotics and Dis/Ability: Interrogating Categories of Difference, New York: State University of New York Press. Rudnytsky, P. L. and R. Charon (2008), Psychoanalysis and Narrative Medicine, S U N Y Series in Psychoanalysis and Culture, New York: State University of New York Press. San-Martín, M., R. Delgado-Bolton, and L. Vivanco (2017), ‘Role of a Semiotics-Based Curriculum in Empathy Enhancement: A Longitudinal Study in Three Dominican Medical Schools’, Frontiers in Psychology, 8 (2018): 1–8. Saussure, F. de (1916), Cours de linguistique générale, ed. C. Bally and A. Sechehaye, Paris: Payot. Scalvini, M. (2010), ‘Glamorizing Sick Bodies: How Commercial Advertising Has Changed the Representation of HIV/AIDS’, Social Semiotics, 20 (3): 219–31. Schemm, W., M. Konitzer, N. Freudenberg, and G. C. Fischer (2002), ‘Therapeutic Interaction through Metaphor: A Textual Approach to Homeopathy’, Semiotica, 2002 (141): 1–27. Schonauer, K. (1993), Semiotic Foundations of Drug Therapy: The Placebo Problem in a New Perspective, Berlin: Mouton de Gruyter. Schultz, M. (2008), ‘Rudolf Virchow’, Emerging Infectious Diseases, 14 (9): 1480–1. Schulz, P. J. and S. Rubinelli (2010), ‘Internet-enhanced Health Communication’, Social Semiotics, 20 (1): 3–7. Sebeok, T. (1978), The Sign and Its Masters, Austin, TX: University of Texas Press. Sebeok, T. (1985), ‘Vital Signs’, American Journal of Semiotics, 3 (3): 1–27. Sebeok, T. (1986), I Think I Am a Verb: More Contributions to the Doctrine of Signs, New York: Plenum Publishing. Sebeok, T. (2001a), ‘Biosemiotics: Its Roots, Proliferation, and Prospects’, Semiotica, 134 (1/4): 61–78. Sebeok, T. (2001b), Signs: An Introduction to Semiotics, 2nd edition, Toronto Studies in Semiotics and Communication, Toronto: University of Toronto Press. Sebeok, T. (2001c), ‘Galen in Medical Semiotics’, in Global Semiotics, 44–58, Indianapolis: Indiana University Press. Sebeok, T. and J. Umiker-Sebeok (1981), ‘Nonverbal Communication, Interaction, and Gesture’, in A. Kendon (ed.), Approaches to Semiotics, The Hague: De Gruyter Mouton. Selby, M. A. (2005), ‘Narratives of Conception, Gestation, and Labour in Sanskrit Āyurvedic Texts’, Asian Medicine, 1 (2): 254–75. Sercarz, E. E. and F. Celada (1988), The Semiotics of Cellular Communication in the Immune System (NATO ASI Series 23: Cell Biology), Berlin: Springer-Verlag. Shands, H. C. (1970), Semiotic Approaches to Psychiatry, The Hague: Mouton.

SEMIOTICS IN HEALTH AND MEDICINE

141

Skopek, L. (1979), ‘Doctor-Patient Conversation: A Way of Analyzing Its Linguistic Problems’, Semiotica, 28 (3–4): 301–12. Smith, H. A. and K. S. Bell (2001), ‘Video as Semiotic Vehicle in the Development of a Theory of Mind for Children with Autism: The Case of Peter’, International Journal of Applied Semiotics, 2: 73–87. Stables, A. and I. Semetsky (2014), Edusemiotics: Semiotic Philosophy as Educational Foundation, New Directions in the Philosophy of Education, Abingdon, Oxford: Routledge. Stafford, P. B. (1988), ‘Towards a Semiotics of Old Age’, in T. Sebeok and J. Umiker-Sebeok (eds), The Semiotic Web, 271–300, Berlin: Mouton de Gruyter. Staiano, K. (1982), ‘Medical Semiotics: Redefining an Ancient Craft’, Semiotica, 38 (3–4): 319–80. Staiano, K. (1986), Interpreting Signs of Illness: A Case Study in Medical Semiotics (Approaches to Semiotics 72), Berlin: De Gruyter Mouton. Staiano, K. (1992), ‘Biosemiotics, Ethnographically Speaking’, in T. A. Sebeok, J. UmikerSebeok, and E. P. Young (eds), The Semiotic Web 1991: Biosemiotics, 407–26, Berlin: De Gruyter Mouton. Staiano-Ross, K. (2007), ‘Wounded Warriors: Further Explorations into a Biocultural Semiotics’, Semiotica, 2007 (166): 1–44. Staiano-Ross, K. (2011), ‘Quarantine’, Semiotica, 2011 (187): 83–104. Staiano-Ross, K. (2012), ‘The Symptom’, Biosemiotics, 5 (1): 33–45. Staiano-Ross, K. and S. Khanna (1999), ‘A Body of Signs: An Introduction to Biocultural Semiotics: Sémiotique et médecine’, Recherches sémiotiques, 19 (1): 3–24. Staiano, K. (1979), ‘A Semiotic Definition of Illness’, Semiotica, 28 (1–2): 107–26. Stanley, D. E. and D. G. Campos (2013), ‘The Logic of Medical Diagnosis’, Perspectives in Biology and Medicine, 56 (2): 300–15. Stanley, D. E. and S. R. Sehon (2019), ‘Medical Reasoning and Doctor-patient Communication’, Journal of Evaluation in Clinical Practice, 25 (6): 962–9. Stanley, D. E. and R. Nyrup (2020), ‘Strategies in Abduction: Generating and Selecting Diagnostic Hypotheses’, The Journal of Medicine and Philosophy, 45 (2): 159–78. Stefan, H. and S. Gollwitzer (2019), ‘Ictal Semiology, Functional Anatomy and Multimodal Diagnostic in Patients with Insular Epilepsies’, Acta Epileptologica, 1 (1): 8. Steinert, U. (2013), ‘Fluids, Rivers, and Vessels: Metaphors and Body Concepts in Mesopotamian Gynaecological Texts’, Le journal des medecines cuneiformes, 22: 1–23. Sydenham, T. (1676), Thomas Sydenham’s ‘Observationes Medicae’ (London, 1676) and his ‘Medical Observations’ (Manuscript 572 of the Royal College of Physicians of London), with new transcripts of related Locke MSS. in the Bodleian Library, G. G. Meynell (ed.), Folkestone, England: Winterdown Books. (1991). Tantam, D. (1986), ‘A Semiotic Model of Nonverbal Communication’, Semiotica, 58 (1–2): 41–58. Tredinnick-Rowe, J. (2018), ‘Can Semiotics Be Used to Drive Paradigm Changes in Medical Education?’, Sign Systems Studies, 46 (4): 491–516. Tulloch, J. (1992), ‘Discoursing AIDS and Sexuality: Popular Media, Magazines and Young Audiences’, Social Semiotics, 2 (2): 113–51. Uexküll, T. V. (1979), Lehrbuch der Psychosomatischen Medizin, Munich: Urban & Schwarzenberg. Uexküll, T. V. (1982), ‘Semiotics and Medicine’, Semiotica, 38 (3–4): 205–16. Uexküll, T. V. (1985), Grundfragen der Psychosomatischen Medizin, Hamburg: Rowohlt TB-V. Uexküll, T. V. (1986), ‘Medicine and Semiotics’, Semiotica, 61 (3/4): 201–17.

142

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Utriainen, T. and M. L. Honkasalo (1996), ‘Women Writing Their Death and Dying: Semiotic Perspectives on Women’s Suicide Notes’, Semiotica, 109 (3–4): 197. Volpe, A. D. (1991), ‘Cohesiveness in Aphasia’, in J. N. Deely (ed.), Semiotics 1991, 264–71, Lanham: University Press of America. Walach, H. (1991), ‘Homeopathy as Semiotic’, Semiotica, 83 (1–2): 81–96. Wang, X., W. Hu, K. Zhang, X. Shao, Y. Ma, L. Sang, Z. Zheng, C. Zhang, J. Li, and J. Zhang (2018), ‘The Anatomo-Electrical Network Underlying Hypermotor Seizures’, Frontiers in Neurology, 9 (243): 1–12. Xia, Y., K. Staiano-Ross, and H. Day (2014), ‘An Early Semiotic’, Semiotica, 2014 (200): 49–83. Zimmermann, J. G. (1763), Von Der Erfahrung in Der Arzneykunst, Zürich: Heidegger und Compagnie. Zukowski, N. and M. Danesi (2019), Medical Semiotics: Medicine and Cultural Meaning, München: Lincom GmbH.

CHAPTER SEVEN

Semiotics in Psychiatry and Psychology NORBERT ANDERSCH

INTRODUCTION AND BRIEF ORIENTATION TO THE FIELD The international congress on ‘The Symbolic Construction of Reality’ in Tokyo (2016) came up with the following statement: The concept of ‘symbol’ is without doubt in the core of the theoretical framework of the human sciences. Its relevance is beyond question and a great variety of differing notions of the symbol were developed by social behaviorists, pragmatists, cultural anthropologists, psychoanalysts, literary theorists, philosophers of life, semioticians and many more. In order to highlight the significance of symbols for the constitution of human life, (the philosopher) Ernst Cassirer defined the human being as ‘animal symbolicum’ (Cassirer 1944), because he considered the concept of rationality as inadequate to describe human cultural forms and realities. Yet, it remains unclear how these theoretical positions are connected to each other and to what extent they can be combined with each other. Furthermore, a concise and systematic theoretical examination of the concept of symbol is rather underrepresented. These conclusions fully apply to the field of psychiatry and general psychopathology, somewhat less to psychology and psychoanalysis. In recent decades, the importance of semiotic and symbolic research on the make-up of consciousness and mental crisis has been widely ignored. Despite a promising start in the first half of the twentieth century, symbol research was nearly wiped out by Nazi rule and Second World War. In the postwar area, language researchers and linguists have also lost their focus on issues of mental crisis and illness, on the plight of patients and their clinical cases, and even more so on the breakdown of semiotic and symbolic features in the make-up of our interactive mental matrix during psychosis and schizophrenia. Contemporary psychiatry – in the public eye and in its clinical content – has become increasingly identified with the simplistic descriptive catalogue systems known as the Diagnostic and Statistical Manual of Mental Disorders (DSM) or (parts of) the International Classification of Diseases (ICD-10, see Sartorius 2002). Theoretical psychopathology has lost the breadth and depth of ideas and approaches compared to the interdisciplinary discussions throughout the scientific community a hundred years ago. The philosophical paradigm shift from a view on substance to a focus on function was ignored, as was the

144

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

rapidly growing importance of semiosis and symbolic formation in the make-up of human consciousness which were central aspects for the contemporary work of philosophers such as Charles S. Peirce, Alfred North Whitehead, Susanne Langer, Ferdinand de Saussure and Ernst Cassirer. German psychiatrist Karl Jaspers’ fall for existentialism (after having published his General Psychopathology in 1913), and Ludwig Binswanger’s turn away from a semiotic perspective on psychopathology towards a post-fascist Heideggerian approach, marginalized the impact of both on the psychiatric discourse. After the Second World War, gestalt- and symbol- research in psychiatry could not regain the impact it had had in the 1920s, since it had been, seriously damaged by the total disruption of its scientific networks and the forced exile and early deaths of many of its main protagonists. Thus, a merely brain-based and a predominantly biological approach forced its way back to clinical ‘superiority’, promoting simple and standardized forms of diagnosis and treatment, be it ECT or psychopharmacology. After 1960 doctors and therapists worldwide were pressured (if not coerced) into using ‘classification systems’ (ICD and DSM) which, first used for administrative purposes and as research tools, have since been changed to facilitate diagnosis-based ‘pathwaytreatments’. With their purely descriptive approach to mental conditions, they are ‘blind’ to the complexity of human interaction, to meaning, intentionality and resonance; to the constant change of basic pattern of behaviour, gestalt-building or the very specific ‘as-if’ mentality of human consciousness. Even the massive surge of scientific efforts in biosemiotics, neurosemiotics and psychosemiotics in the new millennium has passed by most of the psychiatric establishment and its research units; but, facing growing criticism, the WHO and DSM Working Groups have realized ‘that there is a need to resolve or at least to face some fundamental questions [ … on the] treatment or on a particular hypothesis about mental functioning’ (Sartorius 2002: 74/75). In June 2018 Mario May, editor of World Psychiatry, concluded: The usefulness of diagnostic categories in psychiatry has been overemphasized. These categories have been initially charged with implications in terms of pointing to a specific treatment and prospectively a specific etiology and/or pathogenesis […] The fact is, however, that these implications are less significant than originally believed. […]. In addition […] to a given diagnosis […] we should start to promote the construction and validation of tools guiding the clinician systematically in the characterization of the individual case. (May 2018) Having worked as a neurologist and clinical psychiatrist for several decades, my view is that a semiotic/symbolic approach to psychopathology could well turn out to be one of these tools.

SYMBOL RESEARCH AND SYMBOLIC FORMATION IN PSYCHIATRY AND PSYCHOLOGY: A PROMISING START Over centuries, healers, market barbers and street gossip alike had thrived on real or invented symbol-shaped relations in order to bind human madness to magical, mystical and religious speculation. This is why, during the first century of ‘modern psychiatry’,

SEMIOTICS IN PSYCHIATRY AND PSYCHOLOGY

145

symbolic approaches to mental disturbance were dismissed as unscientific explanations. Previous symbolic connotations had to be overcome once and for all by the biological allocation of – and rational thinking about – brain mechanisms. But fairly soon, the importance of symbols was stressed again, albeit in the name of the newly adopted scientific paradigm. German medic Ferdinand Finkelnburg (1870) while trying to work out the multitude of symptoms in aphasia came to the conclusion that the use of symbols amounts to a kind of artificial creation of conventional signs, exclusively practiced by human beings, and that their proper usage, including a detached and abstract view on reality, gets lost in the psychopathological process. In the following decades, neurological researchers like Spamer (1876), Kussmaul (1874) and Pick (1908) also presented clinical cases distinguishing the mere loss of clinical capacity from its representational importance as a tool fostering meaning and generalization. Henry Head’s publication ‘Disorders of Symbolic Thinking and Expression’ (1926) gave a boost to symbol research. Even more so Ernst Cassirer’s main oeuvre Philosophy of Symbolic Forms (three volumes: 1923, 1925, 1929) which produced a remarkable impact on the psychopathological discourse throughout Europe in the 1920s and 1930s. Psychologists (and more so psychoanalysts) had started a symbol debate in the 1890s with S. Freud’s ‘Die Abwehr-Neuropsychosen’ (1894) and ‘Die Traumdeutung’ (1900) and remained engaged in fierce discussions within their own ranks, which dragged on throughout the twentieth century. S. Freud’s psychoanalytic method was a first attempt to take the psychopathological focus away from the brain and to replace it with the relational setting connecting subject and milieu, thus unconsciously shifting the clinician’s view from a focus on the brain (substance) towards human interaction (function), a paradigm-changing view only later fully explored by Ernst Cassirer’s philosophical investigation in 1910. Freud, in his early publications, regarded symbol appearance as a typical sign of either the unconscious, the ‘primary process’ and basic dream experience, or psychopathological decline and regression (Freud 1894/1900). Only dream symbols could (on rare occasions) rise to a position of transpersonal structural elements. Freud’s position in connecting symbol building to pathological events, and, more so, to sexual symptomatology was backed up by S. Ferenczi (1913: 102) and extended by Ernest Jones (1916), prompting Freud’s verdict on symbols as the centrepiece and dogma of psychoanalysis. In his book, Theory of Symbols, Jones was consequently led to connect symbolic features to ‘primitive thinking’. Anna Freud marginalised the meaning of symbols even more, as ‘by-products of dream interpretation’ and as the most basic access to ‘Id’-impulses (A. Freud 1936), thus excluding the symbolic discourse from the centre stage of psychoanalytical discussion and consciousness research. Carl Gustav Jung – having split from Freudian psychoanalysis – elaborated his theories during the 1910s and 1920s without any closer affiliations to the lively contemporary symbol discourse. In contradiction to Freud’s view, for Jung the content of symbols has a major impact on the individuation process and the development of a mature inner self. Thus, the symbolic process is an indispensable requirement and constant companion of the make-up of consciousness. Symbols are transformers of energy as they assimilate mental complexes to the conscious part of our personality. Jung highlights the capabilities of the symbol, its pattern-based ‘Gestalt’, its inner structure, its ego-building format and its unique power of anticipation. Highlighting the collective dimensions of symbols and their importance to generate preformed ‘archetypical’ patterns of mental energy, he brings in a completely new aspect to the psychological/psychiatric debate. Looking at therapy,

146

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

his view is that reinstating symbols between inner self and social environment has an indispensable healing power. There were many more opinions in the vibrant symbol debate of the early twentieth century: Rank and Sachs portrayed the role of symbols as primitive tools of mental adaptation, while Mead and Pavlov put more emphasis on their importance as balancing forces and a means of saving mental energy. Silberer (1912) and Luria (1992) regarded symbols as natural forces of form- and pattern-building; Lewin (1926) was the first researcher to consider symbols as living interactive suspension-bridges, thus facilitating the energy-loaden fundament of consciousness and intentionality, a brilliant idea but only picked up by Leuner (1962) in his inventive studies on artificial psychoses. Psychoanalytic researchers such as Stekel, Szondi, Klein and Sechhaye thought of symbols as facilitators in accessing suppressed mental complexes, while psychiatrists such as Hanfmann, Arieti (1956), Kasanin (1944) and Bash (1955) considered symbols in their natural role of fostering human intelligence. A more semiotic approach (but still structurally similar to Cassirer’s) was at the centre of Jean Piaget’s research (1973) in Switzerland. From about 1930 on, he established an understanding of conscious development in children – based on the semiotics of Saussure (1916/1985) and the mathematical models of the Bourbaki group (1948) – which points to the incorporation of mental tools by which parallel ontologies and representational models of ‘reality’ come about. In the United States, Charles S. Peirce died in 1914. His limited take on psycho(patho) logy had missed out on the ingenious findings on developmental stages in child behaviour and language in the first half of the twentieth century, which also excluded early Peircean semioticians from discussions about consciousness and mental illness. In hindsight, the three-volume oeuvre of Ernst Cassirer’s ‘Philosophy of Symbolic Forms’ (1923–9) is seen by many as the ‘semiotic turn’ in the history of philosophy and anthropology (Bermes 1997). Cassirer centred his whole philosophical approach around the emergence of ‘symbolic form’ as the missing link between the individual biological being and civilization. In his opinion there is no human reality (Wirklichkeit) without or beyond ‘symbolic formation’. The emerging human cultural world represents an irrevocable break with its organic animal tradition. This is reflected in the change of interaction between human intentionality and ‘civilization’, progressing from instinct and preformed mental patterns to different levels of ‘world-making’ facilitated by symbolic forms. The latter appear on the human stage as magic, myth, language, religion, bodily experience, politics, science, the arts and others, taking the form of a universal metamorphosis of cultural creations, woven into a matrix of mental formation called consciousness. Strengthened by the philosopher’s unusually close working relationships with neurologists (Goldstein 1926/1934 and Gelb 1969), psychiatrists (Binswanger 1924) and psychologists (Lewin 1926; Gelb 1969) its results were widely discussed beyond Germanspeaking scholars, and influenced researchers in philosophy, neurology, psychiatry, psychology and anthropology in Scandinavia, the Soviet Union, France, United States, UK, Italy and other countries. This transnational interdisciplinary discourse was brutally halted by fascism and Second World War when Jewish or Socialist researchers as Goldstein, Lewin, Fuchs, Buehler (1934), Cassirer himself and many others, were driven into exile, and had their scientific work interrupted and destroyed. Whilst Cassirer fled to England, Sweden and later the United States, C. G. Jung – after ousting Freud as Head of the ‘Psychoanalytische Vereinigung’ – became the doyen of its Germanic branch under Nazi rule. Jung and Cassirer never had any personal or other connections. Nonetheless,

SEMIOTICS IN PSYCHIATRY AND PSYCHOLOGY

147

there are a surprising number of structural and parallel aspects to be found in both their symbol theories (Brumlik 1993: 143; Pietikainen 1999). Jung’s symbol concept is built on general laws of symmetry, geometry and physics – as is Cassirer’s. Both show evidence of a similar complexity in building mental ‘Gestalt’, and both assume a multi-layered matrix of ‘Sinnstiftung’ in the make-up of consciousness – which both consider not to be merely brain activity but a living process of creating interactive events.

SEMIOTICS AND SYMBOL-RESEARCH POST-SECOND WORLD WAR: PUSHED OUT OF THE MAINSTREAM In the United States and Canada, immigrants such as Cassirer, Goldstein and Buehler with a semiotic/symbolic approach had serious difficulties making themselves heard within a post-war mainstream psychiatry that meandered between conservative psychoanalysis, ECT-treatment, medicalization and behaviourism. Heinz Werner (1964), Susanne Langer (1957), F. J. Hacker (1965) and David Shakov (1969) made important contributions to the link between symbol formation and psychopathology, but were sidelined from the mainstream discourse, as were the results of a major US conference on ‘Psychology and the Symbol’ in 1963 with Bertalanffy, Rappaport and Royce as speakers. Its findings (Royce 1965) were overshadowed by the industry-promoted public hype about the first patient trials using anti-psychotic medication. For decades most of the exiled pre-war symbol researchers (with psychoanalysts as the exception) vanished from mainstream psychology and psychiatry. Only in hindsight recent research confirms (Andersch 2014a) that Cassirer’s pre-war ideas on the symbolic make-up of mental formation (in connection with Goldstein’s concept of psychopathology) had a hidden, but lasting influence on the scientific discourse in the last decades of the twentieth century. In Russia, Lurija (1992), Leontiev (1977), Vygotsky (1934/1974) and Saporoshez (1958) modelled parts of their theories on a symbolic approach, as did Bourdieu (1992), van Ey (1952), Canguilhem (1943/1974), Merleau-Ponty (1966), Lacan (2006) and Foucault (1968) in France; Langer (1974/79), Kasanin (1944), Royce (1965), Werner (1963), Kaplan (1963), Goodman (1978), Hacker (1965), Rappaport, Stack-Sullivan, Segal (1957/59) and von Bertalanffy (1965) in North America; Bash (1955) and Ciompi (1982/88) in Switzerland; Foulkes (1964) and Bion (1962) in England; and Leuner (1962), Peters (1973/4), Lorenzer (1970) and Mentzos (2009) in Germany. Group-analysis, founded and fostered by S. Foulkes in England during the Second World War, who had spent his pre-exile years (then named Sigmund Fuchs) as research-assistant of Goldstein at Frankfurt University) owes much of its matrix-concept to Cassirer’s symbol concept (Nitzgen 2010). Kurt Goldstein’s research on brain-injured patients – focusing on their regression from abstract to concrete behaviour – was successfully repeated with psychotic patients in the Soviet Union in the 1940s, but results obtained by Vigotsky and Luria – demonstrating a loss of symbolic capacity – were sidelined in Western countries during the Cold War. Russian psychologist Leontjew, who described Gestalt-building by symbolic formation as a ‘mechanism of building mechanisms’ as early as the 1950s was ignored (in Western countries up to the 1970s), as were research findings of Saporoshez (1958).1 Internationally renowned German psychiatrist Hanscarl Leuner used LSD-25 on healthy probands in his research trials in the 1960s. He attributed the chaos following mental break-down in psychosis to a mixture of damaged symbolic levels, the re-emergence

148

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

of prefabricated mental patterns and attempts of symbol-reformation, all of which can be seen as potential building blocks of a new reality. Based on extensive research trials of his own, and theoretically on findings by Lewin and Cassirer from the 1920s, he successfully introduced ‘Symbol Therapy’ (‘katathymes Bilderleben’, see Leuner 1970) into clinical practice as a treatment for mental illness. Psychiatrist, psychoanalyst and sociologist Alfred Lorenzer (1970) conducted extensive research in therapeutic interaction highlighting the major importance of symbols. While focusing on Ernst Cassirer’s and Susanne Langer’s philosophy of conscious symbol formation he critically reviewed (the narrow) Freudian approach to symbols. His sophisticated review and theoretical re-enactment of the psychoanalysist’s detailed interaction with patients provides a brilliant insight into the human mental make-up and symbolic capacity. He published a number of books, well reviewed but hardly understood within the (clinically unexperienced) academic and sociological discourse of the 1968 student revolt. Unfortunately, up to today his groundbreaking work remains ignored by mainstream psychiatry and psychology (Andersch 2018a, b) and has until now not been translated into other languages. In the 1970s, theories on social psychiatry by Swiss psychiatrist Ciompi (1982/88) were based on the integration of Saussure’s and Cassirer’s structural and semiotic/symbolic models but only occasionally used in clinical practice. The closest structural link to the importance of symbolic formation on mental health was the concept of ‘pensée operatoire’, presented in France by Marty and M’Uzan (1978), which focused on plausible origins of psychosomatic illness. The authors describe a breakdown of emotional rapport, which becomes one-dimensional, without the skills of symbolization. This concept of de-symbolized activity may well be used to better understand the (up to now unexplained) ‘negative symptoms’ following severe schizophrenic or psychotic crisis. Also a number of psychosomatic concepts – based on the theories of Thure von Uexkuell and Bertalanffy – were modelled on the semiotic capacity in the allocation and practical use of ‘meaning’, or its loss in regressive ‘re-somatisation’ (Sifneos 1975; Mentzos 2009). Recently much attention has been given to Fonagy’s Mentalisation Project (2003), started in the late 1990s. His working group (with findings on different levels of mental representation) obviously avoids any connection to its historical roots and discourses of symbol-research. His project draws heavily on speculation about mirror neurons and relies on the analysis of, particularly, childhood experience, yet ignores the emerging and changing complexity of symbolization as an ever-present process at all age levels.

THE ‘SYMBOLIC ANIMAL’: DEVELOPMENTAL STAGES, SYMBOLIC MATURATION, MAKE-UP, LOCATION AND SYMBOL-PATHOLOGY When I take a closer look at patients with severe forms of mental illness such as bipolar disorder, psychosis and schizophrenia, trying to characterize specific features in their clinical presentation, what stands out are not the ICD/DSM listed ‘first rank symptoms’ (as described by Kurt Schneider in 1939) but their remarkable inability to endure unsolved emotional tension. What really causes their heavy states of anxiety, panic, disorder and hallucinations is a loss of an interactive (pattern-based) symbolic safety-net: a going back from spontaneity to

SEMIOTICS IN PSYCHIATRY AND PSYCHOLOGY

149

rigidity, a loss of planning capacity, a sudden inability in anticipating the future; previously visible possibilities are lost, flexible thoughts narrowed down to a fixation on an isolated matter or a strange object, the patient’s mind being obsessed with past anxieties. It is their abstract thinking being replaced by concrete behaviour, the patient’s perspective and framing capacity being lost and drawn into compulsive action. Patients’ feelings are like ‘standing naked in the cold’ or being ‘stripped of your protective skin’. Spontaneous reactions to such overwhelming experiences are hasty attempts to re-activate earlier levels of functioning, helpless efforts to handle the ruins and lost fragments of broken concepts, a surge for archaic and instinctive preformed defensive patterns, and frantic attempts to be shielded against the incursion of unknown and unconscious forces (while protective layers of self-defence get thinner by the minute). What looks like chaos from the outside and what is experienced as total turmoil by the individual is, nonetheless, an understandable, yet highly complex, breakdown of a pattern-based symbolic matrix. Pattern building is the most natural companion of all biological development, and an indispensable cornerstone of semiosis and symbolic functioning, aiming at and making use of structural binding potentials that reach beyond its physical boundaries. Nonetheless, over eons the global biosphere was structured in a way that functioned well without semiotic systems of higher complexity. But even then, natural phenomena like tropism, taxis or reflexive action functioned on a quasi-semiotic level. Later on, mainly in animals with a developed central nervous system, conditional and operational reflexes work in a semiotic modus – an impulse functions as a sign, which is followed by a standardized motor activity. Observing later developmental steps of their psyche, mental activity finally turns towards a semantic/semiotic mood (Portnov 1993: 228). With growing intentionality and a more sophisticated reflection of outside reality, these forms of ‘signs’ emerge as an important source of information. Communication among animals is a continuous vice-versa positioning of partners, but a clear gap still remains in comparison to the human species, where an overall ‘shared intentionality’ (Tomasello 2019) prevails as the basis of all social interaction. One must keep in mind, though, that early sign systems can only gain their crucial developmental impact insofar as their usage becomes socially implemented as a main component of human language, interpretation and group interaction.2 All living matter, and its adequate adjustment to the environment, is determined by time frames of evolving semiotic complexity at three levels: 1. Basic biological circuits of functions are predominantly guided by genetic codes, where instincts regulate reaction and (motor) performance (as the outcome of past experiences). 2. Among animals, metastability evolves as an additional interactive element within herds or groups growing out of the complementary complex set of activities and drives of interactive correspondents. The background of socially improved protection via habitat or herd creates space for a more ‘individual’ existence. This means that ‘automatisms’ from the genetic setting become slowly reframed by new behavioural laws more strongly focused on actual/present interactive experience. 3. It is the final step of the human being into the symbolic empire which provides for a synthesis of semiotic patterns (in anticipation of relational dynamics) to create meaningful symbols (variable ‘Gestalten’). The variety of these parallel levels of ‘world-making’ is future-oriented stages of action; a dancing ground for human creativity, potential and possibilities. This final third step radically changes the

150

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

paradigm; the symbolic function is a new framework to an open field of activity: the indisputable, underlying fundament of consciousness. The major importance for mental health professionals (in knowing about developmental steps and time frames) lies in the fact that symbolic maturity and features change constantly throughout childhood and adolescence.3 Every breakdown of the (adult) mental matrix leads to a symptomatology driven by the re-emergence of earlier mental paradigms (or parts or fragments thereof), by which the severity of a crisis can be identified and appropriate therapies be considered. Consciousness, therefore, is not an inherited capacity but must be gained in an unusually lengthy period of mental maturation. It is a temporary activity thriving on contradictions and complementarities, preserved and re-enacted in Gestalt figures and symbols. Symbolic formation’s unique capacity is not a biological given but must be drawn up in continuous interaction (in four major steps) like a mental membrane – connective, protective, selective and separating at the same time: 1. By reacting towards, and applying meaning to, fascinating parts of the human environment, thereby linking physical complexities to one’s own mental properties (identification); 2. by intensifying this process in weaving a safety-net between complementary complex patterns of subject and habitat/group (condensation); 3. by exploring the underlying make-up of its architecture, by separating it from the concrete background into which it was first built (separation and fixation); and 4. by incorporating the emerging ‘symbolic form’ into the tool set of personal activities, and by using it freely to create a sphere of possibilities and future potential (reference/pattern). Thus, the multitude of human activities culminates in a limited number of ‘symbolic forms’ such as magic, myth, religion, law, science, the arts and a few others (all of them indicating a different relationship between subject and group complexity), while their underlying patterns can be used again and again in endlessly changing settings.4 Symbolic formation aims towards ‘Gestalt’-building, a unification, something that (at least) looks like a functioning totality. But in people with mental illness, its construction is defective in the first place, or else lost or severely altered by trauma or conflict. Seen from today and judged from a perspective of clinical relevance, the whole complex of symbolic/semiotic research (with regard to psychopathology) can be divided into four main groups. There is first a complex which may be called ‘Genetic Structuralism’ with Cassirer, Goldstein, Goodman, Head, Piaget (1973), Buehler (1934), Langer (1967/1972/1982), Whitehead (1927/85) and others as prominent philosophers and clinicians at the helm where symbolic form and function is seen as cultural creation and tool of human anticipation and creativity. As a second group, there is the Freudian branch of traditional psychoanalysis with Freud, Jones, Ferenczi and others who regarded symbols as symptoms of the primary process, pathological thinking and dream life. These ‘regressive’ views have been altered by new generations of psychoanalysts like Foulkes, Bion, Kubie, Lacan, Mentzos, Lorenzer and others.5 There is a further, mainly American group of Empiricists and Pragmatists with Peirce, Mead and Morris in the lead, where symbols are seen as reference tools and origins of mental complexity and social construction. This approach draws critical views from developmental research because

SEMIOTICS IN PSYCHIATRY AND PSYCHOLOGY

151

of its ‘Adulto-Centrism’ (Fetz 1981), as it mostly deals with mature and adult forms of semiotic processes. Finally, there is the Russian (ex-Soviet), Baltic and Scandinavian semiotic school with a long tradition in semiotic research going back to the 1960s with Sebeok, Portnov, Deglin (all in Grzybek 1993a), Hoffmeyer, Zlatev and Kull (1999/2009) as prominent researchers. They are today’s frontrunners in Neuro- and Psychosemiotic discussion and publications. Signs and symbols are seen by the Baltic School as essential membranes of life, on all different levels of development. Background to the Genetic and Baltic School are von Uexküll’s biological and (from today’s view) biosemiotic writings. The pragmatic American discourse circles have a closer connection to linguistics and Saussure, but there is much overlap in content and references.

HUMAN SYMBOLIC CAPACITY: GRASPING MORE BACKGROUND DETAILS ABOUT ITS TRAJECTORY AND BUILD-UP There is a remarkable joint approach in the theories of Peirce, Cassirer, Langer, Whitehead, Goodman, Saussure, Head and Piaget in their focus on human interactive experience, on thought, language and learning as some kind of ‘symbol-transformation’ which, at times, is guided by conscious rules (as mental frames or interpretation) but most of the time takes its course behind our backs, shielded away from our conscious awareness. In working with patients in conflict, in psychological crisis and in (severe) mental illness all stages of symbol transformation must reveal their mechanisms, their very specific principles of construction: how its arcs of suspension are created, how the semiotic net is woven (long before the final symbolic capacity as a tool of reference is reached) or how its stabilizing frames break down in mental illness or psychological crisis. It is the quest for the early rules and invariants in which the making and structure of symbols is bound together; a process obviously guided by the living ‘architecture of human experience’. What symbolism and semiotics try to locate in the events of conscious interaction, logic tries to find in physics and geometry. So it is no surprise that most of the (above mentioned) scientists were heavily influenced by mathematical and geometrical concepts.6 Yet, in their symbol-theories there is no search for simplistic elements as mediators between subject and environment, but rather for ‘real abstractions’ and ‘invariants of experience’ as correspondents of vital and creative ‘arcs of suspension’. ‘Ideal abstractions and invariants’ emerge as lawful sophisticated relations between complexities (and further abstractions of those complexities) among which abstractions have been created (Whitehead in: Hampe 1998: 31). While in the theory of relativity, space and time are relations between particles of matter, Whitehead – as mathematician and philosopher – now considers them to be relations between events. From early on, humans are trained to experience everything in a timespace-matter format. But this – for Whitehead – is only one setting among other possible perspectives. He considers even a single point (obviously the most basic operational element in geometry) as a complex matter. Watching out for ‘Recognita’ – a system of eternal rules of interconnection where its figures are constitutive (and the underlying pattern) of countless other events, is the prevailing ‘holistic’ view crucial to Whitehead’s symbol-concept (Hampe 1998: 76/77). According to Wiehl (in Whitehead 1971: 38/9), in Whitehead’s theory of symbolic reference ‘we can find an analogy to the dialectic theorem of the speculative sentence where the subject is not just characterized (externally

152

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

determined) by the predicate, but where there is a vice-versa mirroring influence in the mutual make-up of both correspondents’. This also applies to the negative emergence of facts (and not only to positive experiences); it applies to ‘comparative feelings’ (as in Kurt Lewin’s maintenance of parallel arcs of suspension); and last but not least to ‘suspended judgements’, seen by Whitehead not as a wavering attitude or indecisiveness but as an intuitive consideration, as a fruitful form of hesitation, leading to much better conclusions compared to the outcome of hasty decisions, thus preventing (too) ‘early closure’ (Van de Vijver 2000) as we often find in patients’ distortion of reality in psychosis or paranoia. But Whitehead’s slowing down of complex decisions needs facilitation: by an ‘imaginative element’ – a consciously constructed contrast between perceived reality and its reaction to it (i.e. a verbal expression in the experience of sentence building). This is an intentional mental act of interconnection – a symbolism – leading to either true conviction or the indisputable experience of failure. Whitehead’s ‘suspended judgement’ means upholding unsolved tension, strengthening attention towards new experiences, opening up to unknown solutions – a sophistication of skills of observation, a source of stored and provided energy and a general progress of knowledge. All in all, an approach coming very close to a method named ‘abduction’ by Charles S. Peirce, Peirce’s operation of a ‘non-formalized procedure of closure’ (Wirth 1995), is in some way comparable to Whitehead’s ‘imaginative elements’ – to Whitehead’s favourable view on speculation in science – as ‘abduction’ means the proposal of plausible hypotheses and suggesting premises: the only logical way of introducing a really new idea (Peirce 1903: CP 5.172), a unique synthetical mode of closure (Peirce 1901: CP 2.777), not only capable of finding the explanation for mysterious and surprising circumstances, but also in inventing new theories. Abduction includes retrograde and reverse references, means to unlock intentions, to identify and recognize hidden traces – and is a creative way of injecting a new vocabulary by courageously renaming seemingly well-known phenomena (Wirth 1995). This very functional view had already been taken up by Cassirer in his 1910 treatise on ‘Substance and Function’. In the 1920s he writes: ‘Our world cannot be defined as an entity of solids “in” space, nor as a specific event “in” time, but has to be considered as a “system of events”: and in the specification of these events, in its rule of law, space and time are integrated as crucial and necessary aspects’ (1995: 98). Cassirer and Whitehead (and in a similar sense also Peirce) agree on a quest for invariants (of experience) or ‘ideal elements’ in understanding the symbolic process: these lawful interrelations can never be found as grounding eidetic moments (which according to Husserl’s phenomenology have always been there – but have never been found7); on the contrary: symbols are always ‘the result of a permanent process of abstraction and construction, not unconditional, but conditional and constructed’ (Hampe 1998). Their joint understanding of ‘late closure’ is fully reflected in the suspension-loaded symbol construction which in 1926 psychologist Kurt Lewin suggested as the grounding concept of human consciousness. In direct reference to Cassirer’s symbol research, his model of adult awake intentionality is based ‘on the active maintenance of parallel arcs of mental suspension’, each of these providing for a distinct way of world making (Andersch 2014). Mental tension here is considered to be an indispensable source of energy (much in contrast to Freud’s overemphasized and generalized view about the human drive to escape mental tension); a symbolic net made up of complementary complexities between self and world, changing levels between the growing complexity of the subject and his/ her rooms of resonance.

SEMIOTICS IN PSYCHIATRY AND PSYCHOLOGY

153

Ihmig (2003) takes a closer look at the structural laws within Cassirer’s theory of invariants. They can be identified by specific moments in all (different) geometrical ‘Bewegungsformen’, which are part of all human interaction without ever being fully repeated in the single concrete sensual experience. Cassirer picked up the idea of the formation of invariants, as applied by Felix Klein in the ‘Erlanger Programm’ for the purpose of classifying geometries, and developed it further as a general epistemological method for ascertaining what defines the object of scientific knowledge as a whole.8 Coming surprisingly close to the above-mentioned concepts of Peirce and Whitehead, ‘Cassirer’s approach to symbolic formation is not based on the elementary application of a sphere of subjective fictional signs or terms to the perception of given objects in the surrounding environment. Instead, his concept requires the application of rules and structures that can be transferred from each of both spheres into the other’ (Ihmig 2003). This means that the structure of each correlative element is not perceived as a given but only created in the process of ‘Gestaltung’ itself, thereby emerging as a complementary complexity of underlying patterns of both sides. Cassirer’s concept, and this is the major difference with the classic Gestalt theory – and a prerequisite to understanding his approach – can be extended to theoretical or virtual spheres which are no longer bound to an empirical construct of perception (Ihmig 1997: 361/362). The method of classifying invariants relative to certain operations offers the eminent advantage that the procedure can be iterated at will. Therefore, a transition to increasingly higher levels of abstraction, or ‘levels of objectivity’, becomes possible. Each set of operations can be fixed by means of a symbol. This symbol, in turn, can be viewed as an element of a superior set of operations. Looking at semiosis as a whole, we still have to remember Cassirer’s distinction: All the phenomena which are commonly described as conditioned reflexes are not merely very far from but even opposed to the essential character of human symbolic thought. Symbols – in the proper sense of this term – cannot be reduced to mere signals. Signals and symbols belong to two different universes of discourse: a signal is a part of the physical world of being; a symbol is a part of the human world of meaning. Signals are ‘operators’; symbols are ‘designators’. Signals, even when understood and used as such, have nevertheless a sort of physical or substantial being; symbols have only a functional value. (1944: 51) Cassirer later refers to the example of transformation-groups in mathematics to clarify the way in which levels of world-making – based on underlying invariant structures – can be connected to one other and transformed into one another ([1937] 1999). Thus, certain projections emerge as finally compatible with each other, while previously these qualities were considered as logically contradictory. Looking further at Cassirer’s theory of numbers which – beyond its quantitative aspect – always represent the result of an abstraction, a categorial inclusion or a reconstruction, the question is whether our known Arabic numerals (from 0 to 9) also should be seen as (symbolic) carriers of complexity (as mutual intertwinements of complementary geometries), as Andersch (2014) suggests in his model of a ‘Matrix of Mental Formation’. In summary: the reference character of symbols is a very late, adult result of symbolconstruction which progresses over numerous steps at all age levels. Much more attention has to be given to the developmental make-up of symbol-formation and to the

154

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

sophisticated crossover structure of abstract and concrete patterns on both sides of the interactive correspondents, where changing settings of complementary complexities are fostering a setting of late closure, and creating a potential of binding and storing mental energy, thus turning otherwise destructive forces into a generating power for conscious awareness, anticipation and creativity of the human personality.

PEIRCEAN SEMIOTICS AND PSYCHOPATHOLOGY: QUESTIONS, GAPS, PROBLEMS AND FUTURE PRIORITIES In his treatise ‘A semiotic model of mind’ (2000), psychiatrist and psychoanalyst David D. Olds comes up with his ‘project of exploring the unifying, syncretic properties of semiotic theory’ based on the works of ‘Peirce […], Sebeok (1986), Eco, and Deely (1986), among others, (who) have brought semiotics to the fore’. However – according to Olds, these authors are not well known to analysts, and difficulties can arise from the concepts, which seem at once simple and baffling. The jargon is at times inconsistent and confusing. I myself have found it tough going to grasp these ideas, and even harder to communicate them to those not already immersed in the theory. (2000: 499) Olds’ initiative (see also 1990, 1992, 1994) sums up that, up to the 1990s, a specific semiotic approach (as might have been drawn from Peirce’s basic research) had not yet fully entered the medical or psychological stage and discourse. Though highly important to all mental sciences, at the turn of the twentieth century the early stages of child development were just not on Peirce’s mind. ‘A system of Logic, considered as semiotics’ was at first meant to be the headline for Peirce’s collected presentations and writings. Different from symbol researchers as Head, Cassirer, Goldstein, Lewin, Foulkes, Piaget, Lacan, semioticians had hardly any connections to the field of psychopathology, nor close contact to neurologists or psychiatrists. They were trained neither in clinical practice nor in research settings with patients. This means there is (and remains) a focus on the more adult, abstract, mature ways of human thought and interconnections – in clear contrast to proponents of the symbol-function concept whose efforts were especially aimed at pre-logical, developmental (magic, mythical, religious, etc.) levels of world-making (including S. Freud and C. G. Jung). The exception to this ‘adulto-centrism’ could have been Charles W. Morris, who studied psychology at the University of Chicago in the 1920s and originally planned a career in psychiatry (Posner 1987). Finding out about the overarching importance of sign processes, he changed his professional plans. He finished his doctoral dissertation ‘Symbolism and Reality – A Study in the Nature of Mind’ under George H. Mead in 1925, trying to prove that thought and mind are not entities, nor even processes involving a psychical substance distinguishable from the rest of the reality, but are explicable as the functioning of parts of the experience as symbols to an organism of other parts of experience. Being then the symbolic portion of experience, the psychical or mental can neither be sharply opposed to the rest of experience nor identical with the whole of experience. (Morris [1925] 1993: 3–4)

SEMIOTICS IN PSYCHIATRY AND PSYCHOLOGY

155

Morris, later teaching as professor for social research at Rice University, Chicago University and Harvard, never came back to take a closer look connecting his main theme of ‘signs and values’ to the subject of mental illness or psychopathology. Another protagonist taking an early semiotic approach to brain mechanisms was German Jewish psychiatrist Friedrich S. Rothschild, who worked alongside Kurt Goldstein in Frankfurt in the late 1920s and published a book on the symbolism of brain structure already in 1935. He is the first one to have coined the term ‘biosemiotics’ in 1962. Rothschild’s considerations are wide-ranging, including philosophical, anthropological and ethnic/religious considerations, but do not enter into detailed psychopathological discussion. In a review article from 2003, Anderson sees Rothschild in line with some other personalities who in hindsight might be called ‘proto-semioticians’ (or, as Kull in 1999 suggests, ‘endemic-semioticians’). She concludes that Rothschild assumes that ‘understanding humans biosemiotically would help address scourges ranging from population explosion, industrial waste, weapon manufacturing to war and selfdestruction’. Two decades earlier, Saussure himself had seen semiotics as part of sociology9; his disciples, A. Sechehaye and C. Bally, who edited Saussure’s works (1916), posthumously argued in favour of integrating semiotics into psychology, a consideration later backed up by the extensive research on child development by Jean Piaget. As Saussure himself never considered semiotics as a discipline in its own right, Roman Jakobson and many others ‘redefined Saussure’s terms and concepts and disregarded the psychological implications of Saussure’s definition’. In doing so (Grzybek 1993b: 7), ‘Roman Jakobson modified all the original sources without making such modifications explicit.’10 It took half a century until Sebeok’s introduction of the notion and term of ‘Zoosemiotics’ in 1963, followed by Krampen’s proposal of ‘Phytosemiotics’ in 1981, and C. Pearson’s ‘The Cognitive Sciences: A Semiotic Paradigm’ (Simon and Scholes 1982: 225–40) established a first and genuine approach to the field of biology, consciousness and subsequently psychopathology, and laid the foundations for a fuller development of today’s biosemiotics and psychosemiotics, spearheaded by the work of Jesper Hoffmeyer (2008/2015) and Kalevi Kull (1999/2009) among many others. In the mid-1990s, in his essay ‘Psychosemiotics – Neurosemiotics: What Could/Should It Be?’, Peter Grzybek (1993b: 2) appealed to his own colleagues to leave behind the selfcontained attitude of semioticians, to abandon logocentrism, the self-attributed role as a meta-science and its de-psychologized status. He complained that semiotics has not been particularly empirically orientated; rather, it has been (or become) a more or less self-contained discipline (as has linguistics) […] within the ‘semiotic field’ there are only few concrete attempts to relate empirical data (i.e., psychological or neuropsychological) back to theoretical questions focusing on the concept of sign and the process of its generation. The few exceptions from this statement are semioticians from the Soviet Baltic School as Ivanov (1978), Portnov and Deglin (both in Grzybek 1993a). After the turn of the millennium, a number of remarkable papers on neurodevelopment in children (Zlatev 2009, 2015), a detailed outlook on representational meaning-making ‘signways’ (Muller 1996; Smith 2005)11 and valuable contributions by Davtian (2003) on ‘variants of sign malfunctions’ as a new semiotical approach to psychopathology (see also

156

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Chernigovskaya 2007) have been published but not yet found access to the psychiatric mainstream discourse. Despite Olds’ promising initiative and the growing presence of semiotic institutes at a number of (mainly Baltic and Scandinavian) universities three decades on, the lack of semiotic approaches to clinical questions has not changed fundamentally. In Germany, Leferink (2008) draws the radical conclusion that ‘semiotics has not dealt with psychology’ and may even have a hostile attitude towards it. Psychosemiotics remains focused on administrational problems in medicine (Nöth 1990), on general views on mental health, on language attitudes and on problematic aspects in labelling illnesses, but hardly on the real problems at the centre of psychopathological research, on the structure and make-up of symbol pathology or how a loss of semiotic capacity contributes to the start and to the chronicity of mental illness. At some stage, the Peircean side of the semiotic discourse would be well advised to acknowledge and open up further towards the contributions made by the proponents of the symbolic function concept (as based i.e. on Cassirer’s work): the latter having been involved in the psychopathological discourse for about a century. Its researchers (e.g. Buehler, Freud, Jung, Piaget, Goldstein, Foulkes, Lacan) had thorough experience as doctors or therapists themselves or have been closely involved in clinical trials, as Cassirer (1929) with a special treatise on symbol pathology.12 They have tried to disentangle the complex developmental make-up of the ‘symbol’ before it reaches its mature (commonly quoted) referential capacity. Detecting steps and levels of this genetic structural trajectory is the only way to access the complex and puzzling symptomatology of severe mental illness and to implement appropriate steps for treatment programs (as H. C. Leuner did in his symbol therapy starting in the 1960s). Seen from today, a wider review of previously ‘contradictory’ outcomes in semiotics and symbol research in psychopathology leads to a more unifying result by re-framing old, contradictory dualisms via functional pattern-building, laws of complementarity, gestalt and symbol-processes. There are also a number of promising attempts in closing the philosophical and theoretical gap between Cassirer and Peirce, looking out for a more inclusive definition of symbols as a force of creative power, including the complex detailed make-up of human semiotic capacity (Grzybek 1993a; Otte 1997; v. Heusden 2003; Krois 2004; Innis 2009; Stjernfelt 2012; Hoffmeyer 2015; etc.) and its inventive (and destructive) role in thinking, language and ‘ways of world-making’.13

STATE OF THE ART: THE CURRENT STATUS OF SEMIOTIC INSIGHT INTO THE FIELD International classification systems such as ICD and DSM and their diagnostic criteria are applied to about 100 million patients in more than ninety countries every year. Critics claim that this approach disregards the specific symbolic make-up of human culture, that it is not based on an underlying natural, relational or genetic order, and that it remains reduced to a mere superficial description of behavioural phenomena, measured against a ‘norm’ of individuals in the (post-)industrialized milieu of the Western cultures (Berrios 1999; Gorostiza/Manes 2011). The ongoing diversification of descriptive diagnoses of individualized ‘disorders’ (from about 350 to 400 now) may look like a sign of scientific progress; instead, it is only evidence of the fact that the present approach is unable to grasp the underlying architectural and relational symbolic framework which constitutes human consciousness and its breakdown in mental crisis.

SEMIOTICS IN PSYCHIATRY AND PSYCHOLOGY

157

Semiotic and symbolic approaches to psychopathology have been marginalized and misunderstood. This is even more astonishing as all human thinking and all psychiatric symptoms are – without exception – linked to symbol processes and its dysfunction or breakdown. Semioticians would benefit from making a first step towards the clinical sciences (as would psychopathology) with suggestions for conducting joint empirical research and thereby focusing on early developmental stages in the make-up of semiotic structures of consciousness. Translated into clinical terms, a new joint approach would lead to a much wider understanding of the multi-layered architecture of mental health. It allows for a sustainable point of reference in defining ‘mental illness’, and it might help us understand yet-unexplained symptom changes during the course of treatment. A semiotic/symbolic approach reaches beyond the indisputable limits of a mere phenomenological analysis which – at present – is propagated as the state-of-the-art approach to mental illness. Summarizing these developments, we can condense the symbolic/semiotic take on the mental matrix of consciousness as follows: 1. The human psyche is not a closed system which strives for a reduction of tension. To the contrary: several layers of symbolically preserved arcs of mental suspension are fundamental to, and are the basic condition of, an adult interactive matrix of consciousness. 2. Consciousness starts and proceeds as the subject’s ability to activate those parts of their group/milieu/environment which are complementary to one’s own (changing) levels of mental complexity – and to stabilize and preserve the resulting net of experience and resonance. 3. Symbolic forms in their bipolar make-up are activating, generating and maintaining variable trained/learned patterns of behaviour even without having understood the complex technical or hidden details of the mental processes involved. 4. Human consciousness moves very fast away from storing early picture sequences, and towards a field of abstract thoughts. These are not made up of (sequences of) inner images but facilitated by ordinary pattern building (towards complementary complexity on both sides of the subject versus social divide). 5. The subject’s mental complexity grows with age, nurtured by a symbol transfer from group/environment to the individual (and later, back from the mature subject to the group), going along with a subsequent loss of complexity in the complementary correspondents. What is even more crucial, the short-lived entities subject and environment deal not with empirical sense data but symbols throughout. 6. Thus, the variable use of symbols (as tools of reference) is only the final result of several developmental steps on a trajectory passing through a number of mental paradigms (from body-related pre-symbolic to tool-shaped free-symbolic) which can be found in every individual human’s progress as in the different layers of cultural history. 7. Symbolic forms are of transcultural, universal value; they can be described and classified for diagnostic purpose. They are a binding force for complementary and contradictory patterns; thus generating, storing and activating mental energy. Their built-in structural resilience becomes weakened and disintegrates depending on the severity of mental illness. The symptoms of breakdown can be used to identify and ‘repair’ gaps in different developmental stages of conscious interaction between subject and milieu.

158

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

To comprehend the multitude of symptoms in the breakdown of semiosis and symbol formation with its changing levels of complexity, mixed-up timelines, restructuring activities and provisional helplines, an interactive moving model of consciousness close to clinical practice and experience is required. Well-advanced starting points to such concepts can be found in A. N. Whitehead’s ‘process-paradigm’ ([1927] 1985), in E. Cassirer’s model of different layers of ‘symbolic formation’ or in J. Piaget’s ‘genetic structuralism’.14 All of them focus on a process of mental differentiation which starts from a basic togetherness of those compartments which, in its later development, will emerge as divided acting correspondents: subject and object. Their early connection to matter, reification and thing-ness will be changed to an ever more abstract functional process. This is how pattern-based interrelations (musterfähige Bewegungsformen) can emerge, where intentional Gestaltung15 can open up variable polarities: metastable stages of performance for potential and creativity (Fetz 1999). Dependent on the stability of this ‘Matrix of Mental Formation’ (Andersch 2007), humans can thrive in anticipation, creativity and building up potential and future. At the same time, this safety net covers for our anxieties, for the fragmentation of our daily procedures, for our attempts of ‘trial and error’ as well as for disappointments and efforts which fail. Even in those unfortunate moments, its stretched ropes hold tight, providing us with a feeling of meaning in life.

A MATRIX OF MENTAL FORMATION While the structure of the physical functions of logic and its laws are seemingly free of contradictions, our healthy and stable human mind is obviously made up of something entirely different: an arc of preserved tension, a kind of living suspension bridge where its stabilizing ropes are created by an ongoing back-and-forth between subject intentionality and subsequent resonance from group or environment. Consciousness therefore is a temporary activity thriving on contradictions and complementarities, preserved and reenacted in Gestalt figures and Symbols. People with serious mental illness quite obviously have a problem of setting up and maintaining this ‘Matrix of Mental Formation’ (Andersch 2014). Within this concept, mental stability is not ‘a function of the brain’ but a functioning social construct – as is a good marriage, a decent education or respectable science. All very much real and no ‘myth’ – but not as a substance or an observable object in our brains; rather as a relational order (wherein our brain plays a crucial role). In recent decades, the growing importance of semiotic and symbolic research in relation to the make-up of consciousness and mental crisis has been recognized in the psychological discourse, but ignored and left behind in the mainstream debate of psychiatry. Despite recent efforts in uniting a split symbolic/semiotic approach to the theory of psychopathology (including psychiatry, psychology and psychoanalysis), the absence of a common conceptual framework is prevailing. More money and more focused research are needed. Only a cooperative review of previously ‘contradictory’ research results can lead to a much more unifying model of a Matrix of Mental Formation, linking humans and their world by re-framing old dualisms via functional pattern-building, laws of complementarity, gestalt and symbol-processes and semiotic scaffolding. Semiotic and symbolic research and action should play a vital and decisive role in a move towards a new and reformed ‘science of mental health’ as laid out in the theses below: 1. Our shared capacity of ‘world making’ and our variability in changing mental paradigms are the fundamental cultural achievements of the human species. They

SEMIOTICS IN PSYCHIATRY AND PSYCHOLOGY

159

emerge in symbolic forms and functions as magic, myth, religion, law, bodily experience, science, the arts and others, carried forward by sign systems centred around human language. Its mental spaces secure the plausibility of everyday experience, interaction, anticipation and creativity. 2. The living suspension bridges between different levels of intentionality, and their complementary rooms of resonance are a source of mental energy which (in its entirety) is known as consciousness. Serious gaps in the functioning of this variable interactive matrix are experienced as mental crisis or illness. 3. Mental illness is always a dysfunction of being together (with other humans). It is neither a matter of damage to the brain nor to its organic function but a breakdown of symbolic formation and/or the inability to perform the adequate changes between different forms/or frames of ‘world-making’. 4. Mental dysfunction is more likely to be detected/observed in individuals compared to group settings. Nonetheless, both sides of the social fabric are always actively involved in the breakdown of the symbolic matrix. 5. Typical symptoms in mental crisis include anxiety and the inability to endure unsolved tensions, a reduction of mental complexity and a regressive move in (re)activating ancient (auto-regulative) pattern of thought and behaviour (hyperreflexivity). 6. What looks like mental chaos in crisis is always a mixture of broken paradigmatic frames, substitutes for the loss of sense-making and helpless efforts in repairing damaged levels of abstraction. Psychiatrists and psychologists should focus their involvement on restructuring the symbolic matrix or reintroducing temporary suspension bridges to foster creative impulses of the patient. 7. The temporary reduction of mental abstract flexibility towards a more concrete level of awareness – and a subsequent self-centred attitude – should not be simplified as a symptom of mental disorder. It can be a natural form of selfprotection and  – when treated with acceptance and respect – can turn out to become a source of personal maturation within the healing process. This is why psychopathology should move away from looking at magical, mythical, religious and spiritual early levels of ‘world-making’ as pathological. 8. If a mental disorder is caused by an identifiable organic cause (such as concussion, intoxication, infection, epilepsy or dementia) the first line of treatment should be determined by a neurologist/internal medic. 9. The aim of psychiatric and psychological intervention lies not in regaining lost ‘standards of normality’ but in finding a new internal balance between personal intentionality and the subject’s social room of resonance. 10. Therapy is meant to encourage group-involvement, reactivate rituals and group support and foster changed levels of world-making by internalizing these safety nets by attaching them to symbolic tools. 11. All new classification systems (of mental crisis as previously ICD and DSM) should replace the hierarchical model of human and personal development by adopting a matrix of variable paradigms of symbolic function (which allow for parallel and integrative settings). They should include local frame-settings and methods in identifying group pathologies.

160

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

12. Research efforts should focus on detecting the hidden pathways and detailed matrices of the semiotic make-up and fabric of consciousness – especially in which ways biological circuits are interconnected with social sign systems. A new approach should promote a stronger acceptance of different and deviating human behaviour and strongly reduce the use of psychotropic medication. 13. There must be a much stronger awareness that constantly progressing signsystems and the power of symbolic form and function in itself can lead to mass manipulation, abuse and exploitation. If applied to clinical tasks, a symbolic/semiotic model enables therapists (nurses, doctors, psychologists) to reach a new form of joint understanding with the patient. This complementary stance towards the patient’s vulnerable position merges into a short moment of unification, which fosters healing and a process of becoming, while re-experiencing early mental structures with a potential to change their damaged pattern of interaction. Due to the application of a new symbolic methodology, certain settings and clinical symptoms which were seemingly contradictory beforehand now emerge as being compatible within a newly created, more abstract geometry of interrelations. Approached from this ‘symbolic’ angle, mental health could be defined as the human ability to stabilize early patterns of personal experience, to successfully create, change and integrate symbolic forms of social interaction, while establishing an equilibrium between the demands and intentions of self-regulation and environment, adding its newly found results to human tradition. Mental illness subsequently would no longer be misidentified as a mere dysfunction of the brain but regarded as the inability to (stabilize and/or) integrate one’s own patterns of behaviour into a social framework, leading to a breakdown of (different and multiple) layers of ‘symbolic formation’, while the balance between cultural interaction and the emergence of inner preformed patterns is continuously (or constantly) changed towards the latter. In psychiatry, psychology and psychopathology, organized efforts in collecting and coordinating findings on semiotic and symbolic research should be fostered, supported and funded, thus contributing to a ‘science of meaning’ (or salience) beyond the mere biological function of our animal brains and in integrating this important human source of knowledge into the regular discourse of our discipline. Symbol research thus can contribute to psychological as well as to biological considerations. Organized efforts should be made by the medical profession to seriously cooperate with already-existing academic projects in bio-, neuro- and psychosemiotics. Its unlocked potential can provide a ‘new key’ to some of the unsolved riddles of mental illness and theoretical psychopathology.

NOTES 1. Saporoshez (1958) pointed out that only human behaviour relies on a proper use of tools, and that typical copying activities in small children (echokinesis, echomimia, echolalia) come to an end early in the second year before being replaced by symbolic patterns (Leontjev 1977). Its emergence is less fostered by a rewarding stimulus than by the coming-together of the child’s own activity with its imagined purpose. 2. In hindsight – and adding up clinical research from recent decades (Kupferberg 2010, 2010a) – the importance of linguistic considerations for mental illness appears to be exaggerated.

SEMIOTICS IN PSYCHIATRY AND PSYCHOLOGY

3.

4.

5.

6.

7.

161

Karl Buehler’s Organon model (Andersch 2020b) and Roman Jakobson’s extensions to this approach could not be used in solving the complex pathways of language disorder in schizophrenia (Chernigovskaya 2007; Natterer 2010). Soviet/Russian researchers have repeatedly stressed the point that human communication is always facilitated on different levels of semiosis. Portnov (1993: 275) finally concludes: ‘the only way forward is not only to hint at the different levels of complexity among the semiotic layers, but to compare the structure of mental activity and consciousness to the different types and layers of the semiotic process’. Cassirer never developed a full system of ‘symbolic formation’, yet he had explicitly envisaged this possibility in the first volume of his Philosophy of Symbolic Forms (1923: 18): ‘If there was a way to gain a systematic perspective over the different directions of that kind [i.e., the entirety of symbolic forms, N. A.] – and: if there was the possibility of deconstructing its typical and invariant pattern, as well as its specific internal order and hidden differences, the ideal form of “general characteristics” for the entirety of mental productivity might emerge – just as Leibniz had called for with regard to human knowledge in general’ (translated by the author, N. A.). The early Freudian view on symbols was challenged throughout the twentieth century but fundamental criticism came only up in the United States in the 1950s (Segal 1957/59; Kubie 1953a, b). In his definition of the symbolic process. L.S. Kubie (1953b: 5) comes close to Cassirer’s own version: ‘symbolic representations of every conceptual process are rooted in both the body and the outside world […] symbolic representation must inevitably develop through an interplay of meanings which begin internally and extend subsequently to include external points of reference’. Winnicott, Segal, Bion, Foulkes and many other psychoanalysts did their best to redress Freud’s unilateral view in the 1950/60s. So did Lorenzer; Mentzos; Benedetti/Rauchfleisch 1988; and Bernhardt/Link-Wieczorek thereafter. Cassirer made an extensive and sophisticated comment on Einstein’s ‘Theory of Relativity’. In his treatise ‘Determinismus und Indeterminismus in der modernen Physik’ he also discussed the implications and consequences of Quantum Theory. In a letter to Cassirer, the eminent quantum physicist Max Born agreed with Cassirer’s views and also supported his view on the ‘Theory of Invariants’ (Andersch 2014: 66). In a preface to the English edition (1956) of ‘Determinism and Indeterminism in modern Physics’ Henry Margenau, one of the leading quantum physicists of the time, praised Cassirer’s findings as: ‘ahead of its day; its thesis was revolutionary and radical […] Cassirer saw more deeply and perceived a basic change in the meaning of reality […] he showed that the causal controversy […] was an outgrowth of a more fundamental issue’. The Russian semiotician Aleksandr Portnov (in Grzybek 1993a: 246/7) also points to the fact that various types of semiosis (phylogenetically from different periods) – according to the theories of atomic physician Niels Bohr – can be considered in analogy to different experimental systems of observation, whereby only the (newly) complementary complex intertwinement of these ‘Collectives of Signsystems’ allows for a coherent modelling of (previously incompatible) objective and subjective realities. Seen from a quantum-theoretical perspective this means that mental conditions and brain conditions do not challenge each other (fighting for causal predominance) but can now cooperate in the sense of the correlation by which they are connected. This may lead to the emergence of synchronic states: related to each other by laws of correlation, but not by causality. Scholars close to Edmund Husserl such as Aaron Gurwitsch (1949) and Alfred Schutz – both well aware of the semiotic/symbolic discourse in the 1940s and 1950s – were courageous enough to admit that ‘Husserl’s attempt to ground the constitution of transcendental intersubjectivity on the conscious efforts of the transcendental ego has failed’ (Schutz 1957).

162

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Schutz assumed that human self-evidence, accepted by humans as an unproven fact and based on trained cultural rules, has to be seen as fundamental to intersubjectivity. This view was shared by the phenomenological Heidelberg School of Psychiatry (Prof Dr Ch. Mundt) in the late 1990s which stated that ‘regarding all practical and therapeutic consequences the mental condition of a human being can never be based on a mere phenomenological analysis’ (Kick/ Diehl 1998). The much-quoted phenomenological method of Edmund Husserl (Staiti 2012) is of limited value to psychopathology and psychiatry (Andersch 2013). 8. Autochthon symbolic spaces – seen from Goodman’s ‘Ways of World making’ – (and: using the Heisenberg cut as a conceptual instrument, separating an ontic system from its background/ environment) can be seen as a decision for a very specific perspective on the system in observation/under research which (at least temporarily) excludes all other perspectives (whereby only all of them can produce totality). In a similar way, Atmanspacher (2008) brings up the (same) problem of ontic versus epistemic conditions, i.e. of those which can be accessed operationally versus those which cannot be observed empirically, nor be measured. 9. It is worth noting that many sociologists like Bourdieu (1974/1992); Honneth (1986); Dreher (2007), Gurwitsch (1949) and Magerski (2005) invested more research efforts and emphasis on the intertwinement between social systems and symbolic forms (compared to contemporary semioticians) and were thus coming much closer to analysing the group impact on the becoming and the healing aspects in mental crisis. 10. Also, Charlotte Buehler complained about R. Jakobson’s attitude of having plagiarized and massively altered Karl Buehler’s ‘Organon-Model of Language’ without ever referring to the original and its creator. 11. The social-personal signway is characterized by an early intersubjectivity that underlines the essential socio-cultural nature of the human species. This intersubjectivity develops into an increasingly sophisticated form that leads directly into personal elements and the affiliated evolving sense of self and identity. From the abundant research on the intersubjectivity of infants Muller (1996: 21) advances the following semiotic claims: 1. The mother-infant interaction is governed by an exchange of cues structured by a code. 2. This code has the essential features of a semiotic code insofar as it specifies cues as signs, indicates their legitimate substitution and combination, and organizes the pragmatics of turn-taking for the positions of sender and receiver of these cues. 3. The infant learns to use and respond to such cues. 4. The mother recognizes the infant as actively cueing. 5. The infant’s role as semiotic partner impacts the mother’s semiotic behaviour. 6. The mother’s violation of the semiotic code is disruptive to the infant, and this indicates that the infant has learned the basic rudiments of the code. 7. The semiotic rules for the interaction are culturally distinct. 8. The code that structures the interaction stands as a third term to the dyad, as the holding environment for both mother and infant. 9. The mother’s distinctive responsibility, what distinguishes her from other objects in the infant’s environment, is not as a desired object but rather as a desired subject. 10. The process of mutual semiotic recognition leads to the emergence of subjectivity in the infant, eventually effected in the use of ‘I’ and ‘you’. (Muller 1996 in Smith 2005, Psychosemiotics and its Peircean Foundation) 12. Cassirer visited Goldstein’s clinic in Frankfurt where their joint observations provided the background for his chapter ‘on the pathology of the symbolic function’ in the third volume of The Philosophy of Symbolic Forms. Cassirer conceived of this chapter as a kind of negative proof of his theory of symbolism: ‘the process of the world’s ‘symbolization’ discloses its

SEMIOTICS IN PSYCHIATRY AND PSYCHOLOGY

163

value and meaning where it no longer operates free and unhindered, but must struggle and make its way against obstacles’ (Cassirer 1957: 277). 13. what made Peirce and Cassirer and their virtual interconnections so interesting and fertile is that they incarnate, each on their continent, the final development of philosophy before the split between analytical and continental traditions during the first half of the twentieth century. […] Peirce in his pragmatism and semiotics, Cassirer in his doctrine of symbolic forms. Both, furthermore, aimed at founding these ambitious doctrines on systematical ontological assumptions, Peirce in his phenomenological list of categories, Cassirer in his little known theory of Basisphänomene, basic phenomena, which became a special focus field for John Krois […] and the basic connection which he established between Peirce and Cassirer on the following five main themes: categorizations, pragmatism, images, semiotic evolution and embodiment. (Stjernfelt 2012) 14. There is such a striking similarity between Cassirer-based concepts of psychopathology and those of Piaget that, despite the historical non-existence of dialogue between the two of them, researchers now speak of a joint ‘genetical semiology’ (Fetz 1981). 15. Philosopher Moritz Schlick has hinted at the fact that numerous human situations – acted out on the underlying stage of symbol- and gestalt-like stability – are not ‘Gestalten’ in themselves, but mere fragments or random summaries of facts, terms and relations. The question of whether we have to deal with ‘sum or gestalt’ cannot be decided easily. Nonetheless, Schlick emphases that ‘gestalt’ or symbolic formation does remain crucial, wherever the ‘matrix of invariants’ (‘Invariantenmatrix’) is at stake (i.e. the stage of action itself, on which the construct of our undertakings has to be carried out): ‘a gestaltist description will nowhere be the only/single choice, with the exception of those cases, where “invariants” are concerned, those laws/rules and relations which remain unaltered amongst the constant change of the scenery’ (Schlick 1936: 264, translated by the author, N.A.).

REFERENCES Akhutina, T. V. (2003), ‘Roman Jakobson and the Development of Russian Neurolinguistics’, Journal of Russian and East European Psychology, 41 (3–4): 129–58. Akhutina, T. V. (2006), ‘Vygotsky-Luria-Leontiev’s School of Psycholinguistics: The Mechanism of Language Production’, in R. Alanen and S. Poeyhoenen (eds), Language in Action: Vygotsky and Leontievian Legacy Today, 32–56, Newcastle: Cambridge Scholars Publishing. Alanen, R. and S. Poeyhoenen, eds (2007), Language in Action: Vygotsky and Leontievian Legacy Today, Newcastle: Cambridge Scholars. Andersch, N. (2007), ‘Symbolische Form und Gestalt – Ein kreatives Spannungsverhältnis’, Gestalt Theory, 29 (4): 279–93. Andersch, N. (2014), Symbolische Form und psychische Erkrankung. Argumente für eine ‘Neue Psychopathologie’. Klinische und philosophische Ueberlegungen, Würzburg: Königshausen & Neumann. Andersch, N. and J. Cutting (2014a), ‘Ernst Cassirer’s Philosophy of Symbolic Forms and Its Impact on the Theory of Psychopathology’, History of Psychiatry, 25 (2): 203–23. Andersch, N. (2018), Der Psychiater und Psychoanalytiker Alfred Lorenzer (1922–2002). Vordenker theoretischer Symbolpathologie. Schriftenreihe der DGGN Band 24, Würzburg: Koenigshausen & Neumann.

164

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Andersch, N. (2018), ‘The Unused Potential of Semiotic/Symbolic Research in Psychopathology: A New Key towards the Understanding of Mental Illness’, in Presentation at the Second WPA Congress on Mental Health: Meeting the Needs of the XXIst Century, 5–7, Moscow, Russia. Andersch, N. (2020a), ‘Letter to the Editors of “World Psychiatry” Calling for a WPA Section on “Semiotics and Symbolic Research in Psychiatry”’, academia.edu: bit.ly/WPEdsLtr. Andersch, N. (2020b), ‘Whatever Happened to Karl Bühler …’ Aufstieg und Ende der wissenschaftlichen Karriere des Arztes, Psychologen und Sprachforschers’, in Schriftenreihe der DGGN Band 26, 455–73, Würzburg: Koenigshausen & Neumann.Band 26. Anderson, M. (2003), ‘Rothschild’s Ouroboros’, Sign Systems Studies, 31 (1): 301–14. Atmanspacher, H. and S. Rotter (2008), ‘Interpreting Neurodynamics: Concepts and Facts’, Cognitive Neurodynamics, 2: 297–318. Arieti, S. (1956), ‘Some Basic Problems Common to Anthropology and Modern Psychiatry’, American Anthropology, 58 (1): 26–39. Bash, K. W. (1955), Lehrbuch der allgemeinen Psychopathologie, Stuttgart: Thieme. Benedetti, G. and U. Rauchfleisch (1988), Welt der Symbol, Göttingen: Vandenhoeck & Ruprecht. Bermes, C. (1997), Bedeutung als Bestimmung und Bestimmbarkeit. Trierer Studien zur Kulturphilosophie. Bd1, Würzburg: Koenigshausen & Neumann. Berrios, G. E. (1999), ‘Classifications in Psychiatry: A Conceptual History’, Australian and New Zealand Journal of Psychiatry, 33: 145–60. Binswanger, L. (1924), ‘Welche Aufgaben ergeben sich für die Psychiatrie aus den Fortschritten der neueren Psychologie?’, Zeitschrift f. d.ges. Neurologie u. Psychiatrie (Sonderdruck aus Bd. XCI/ Heft 3/5). Bion, W. R. (1962), ‘Eine Theorie des Denkens’, in E. Bott-Spillius (ed.), Melanie Klein heute. Entwicklungen in Theorie und Praxis, Bd. 1, 225–35, Stuttgart: Stuttgart Press. Blankenburg, W. (1973), Der Verlust der natürlichen Selbstverständlichkeit, Berlin: Springer Bott-Spillius, E., ed. (1990), Melanie Klein heute. Entwicklungen in Theorie und Praxis, Bd. 1, Stuttgart. Bourbaki, N. (1948), ‘L’architecture des mathématiques’, in Le Lionnais (ed.), Les grands courants de la pensée mathématique, 35–47, Paris: Édition des «Cahiers du Sud». Bourdieu, P. (1992), An Invitation to Reflexive Sociology, Chicago: University of Chicago Press. Bourdieu, P. (1992), ‘Sozialer Raum und symbolische Macht’, in P. Bourdieu (ed.), Rede und Antwort, 135–54, Frankfurt: Suhrkamp. Bredekamp, H., M. Lauschke, and A. Arteaga, eds (2012), Bodies in Action and Symbolic Forms: Zwei Seiten der Verkörperungstheorie, Berlin: Akademie Verlag. Brumlik, M. (1993), C. G. Jung zur Einführung, Hamburg: Junius. Brock, A. (1994), Whatever Happened to Karl Bühler?, Canadian Psychology, 35 (3): 319–29. Bühler, K. (1982 /1934), Sprachtheorie: Die Darstellungsfunktion der Sprache, Stuttgart, New York: Fischer. Canguilhem, G. (1974 [1943]), Das Normale und das Pathologische, München: Hanser. Cassirer, E. (1910), Substanzbegriff und Funktionsbegriff. Untersuchungen uber die Grundlagen der Erkenntniskritik, Berlin: Verlag von Bruno Cassirer. Cassirer, E. (1929), ‘Etudes sur la pathologie de la conscience symbolique’, Journal de la Psychologie, 26 (1929): 289–336 and 523–66. Cassirer, E. (1956 [1937]), Determinism and Indeterminism in Modern Physics: Historical and Systematic Studies of the Problem of Causality, New Haven: Yale University Press.

SEMIOTICS IN PSYCHIATRY AND PSYCHOLOGY

165

Cassirer, E. (1972 [1944]), An Essay on Man: An Introduction to a Philosophy of human Culture, New Haven: Yale University Press. Cassirer, E. (1995), Nachgelassene Manuskripte und Texte. Bd. 1: Zur Metaphysik der symbolischen Formen, eds. M. Krois und O. Schwemmer, Hamburg: Meiner. Cassirer, E. (2001–2 [1923–29]), Philosophie der symbolischen Formen, Ernst Cassirer Werke Bd 11/12/13, Hamburg: Meiner. Chernigovskaya, T. V. (2007), ‘The Mirror Brain, Concepts and Language: The Price of Anthropogenesis’, Neuroscience and Behavioural Physiology, 37 (3): 293–302. Churchland, P. S. (1986), Neurophilosophy: Toward a Unified Theory of the Mind/Brain, Cambridge: MIT Press. Ciompi, L. (1982), Affektlogik. Über die Struktur der Psyche und ihre Entwicklung: Ein Beitrag zur Schizophrenieforschung, Stuttgart: Klett-Cotta. Cobley, P., ed. (2009), The Routledge Companion to Semiotics, London: Routledge. Covington, M. A., H. Congzhou, C. Brown, L. Naci, J.T. Mc Clain, B. S. Fjordbak, J. Semple, and J. Brown (2005), ‘Schizophrenia and the Structure of Language: The Linguist’s View’, Schizophrenia Research, 77: 85–98. Cristalli, C. (2017), ‘Experimental Psychology and the Practice of Logic: Charles S. Peirce and the Charge of Psychologism 1869–1885’, European Journal of Pragmatism and American Philosophy, 9 (1). https://doi.org/10.4000/ejpap.1006. Crow, T. J. (1997), Is Schizophrenia the Price That Homo Sapiens Pays for Language?, Schizophrenia Research, 28 (2–3): 127–41. Davtian, S. and T. V. Chernigovskaya (2003), ‘Psychiatry in Free Fall: In Pursuit of a Semiotic Foothold’, Sign Systems Studies, 31 (2): 533–46. Deacon, T. (1997), The Symbolic Species, New York: Norton. Deely, J. (1986), ‘The Coalescence of Semiotic Consciousness’, in Frontiers in Semiotics, J. Deely, B. Williams, and F. E. Kruse (eds), 5–34, Bloomington: Indiana University Press. Deely, J. (2015), ‘Semiotics “Today”: The Twentieth-Century Founding and Twenty-FirstCentury Prospects’, in P. P. Trifonas (ed.), International Handbook of Semiotics, 29–113, Berlin: Springer. Deglin, V. L. (1993), ‘Die paradoxale Mentalität oder: Warum Fiktionen die Realität ersetzen’, in Grzybek (ed.), Psychosemiotik – Neurosemiotik. Bochumer Beiträge zur Semiotik Bd 41, 55–96, Bochum: Brockmeyer., op. cit. Dreher, J. (2007), ‘Symbolische Formen des Wissens’, in R. Schutzeicher (ed.), Handbuch Wissenssoziologie und Wissensforschung, 463–71, Konstanz: UVK-Verlags-Gesellschaft. Eschbach, A. (1984), Bühler Studien I/II, Frankfurt: Suhrkamp. Ferenczi, S. (1980 [1912]), Symbolism: First Contributions to Psychanalysis, London: Maresfield Reprints. Ferenczi, S. (1913), ‘Zur Ontogenese der Symbole’, Internationale Zeitschrift für (ärztliche) Psychoanalyse, 1: 436–38. Fetz, R. L. (1981), ‘Genetische Semiologie. Symboltheorie im Ausgang von Ernst Cassirer und Jean Piaget’, Freiburger Zeitschrift für Philosophie und Theologie Freiburg, 28: 434–70. Fetz, R. L. (1999), ‘Whitehead, Cassirer, Piaget: drei Denker-ein gemeinsames Paradigma’, Salzburger Theologische Zeitschrift, 3 (2): 154–68. Finkelnburg, F. C. (1870), ‘Niederrhein. Gesellschaft in Bonn: Medicin. Sektion, Sitzung am 21/3/1870’, Berliner Klinische Wochenschrift, 37 (8): 449/50 and 460/62. Fonagy, P. and M. Target (2003), Frühe Bindung und psychische Entwicklung, Gießen: Psychosozial-Verlag.

166

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Forster, O. (1984), ‘Zu einem Organon Modell pathologisch verzerrter Sprache’, in Eschbach A (1984), 343–59. Foucault, M. (1968), Psychologie und Geisteskrankheit, Frankfurt: edition suhrkamp. Foulkes, S. H. (1964), Therapeutic Group Analysis, London: Karnac Books. Freyberg, S. (2014), Peirce in Germany, European Journal of Pragmatism and American Philosophy. http://journals.openedition.org/ejpap/488 (accessed 21 September 2019). Freud, S. (1894), ‘Die Abwehr-Neuropsychosen’, in Sigmund Freud Gesammelte Werke, Bd.I, 59–74, Frankfurt: Fischer Verlag. Freud, S. (1900), Die Traumdeutung, GW Bd II-III, Frankfurt: Fischer. Friedman, J. T. and S. Luft, eds (2015), The Philosophy of Ernst Cassirer: A Novel Assessment, Berlin and Boston: de Gruyter. Friedrich, J., ed. (2018), Karl Bühlers Krise der Psychologie. Positionen, Bezüge und Kontroversen im Wien der 1920er/30er Jahre, Cham, Swiss: Springer. Friedrich, J. (2018a), ‘Bühlers neues Programm der Lebenspsychologie’, in J. Friedrich (ed.), Karl Bühlers Krise der Psychologie. Positionen, Bezüge und Kontroversen im Wien der 1920er/30er Jahre, 137–64, Cham, Swiss: Springer. Gelb, A. (1969), Zur medizinischen Psychologie und philosophischen Anthropologie, Darmstadt: Wissens. Buchgesellschaft. Goldstein, K. (1934), Der Aufbau des Organismus, Den Haag: Martin Nijhoff. Goldstein, K. (1939), ‘Methodological Approach to the Study of Schizophrenic Thought Disorder’, in Kasanin, Language and Thought in Schizophrenia, 17–40, New York: Norton. Goodman, N. (1978), Ways of Worldmaking, Indianapolis, Cambridge: Hackett Publishing Company. Grzybek, P., ed. (1993a), Psychosemiotik – Neurosemiotik. Bochumer Beiträge zur Semiotik Bd 41, Bochum: Brockmeyer. Grzybek, P. (1993b), ‘Psychosemiotics – Neurosemiotics: What Could/Should It Be?’, in P. Grzybek (ed.), Psychosemiotik – Neurosemiotik (Bochumer Beiträge zur Semiotik Bd 41), 1–14, Bochum: Brockmeyer. Gurwitsch, A. (1949), ‘Gelb-Goldstein’s Concept of “Concrete” and “Categorial” Attitude and the Phenomenology of Ideation’, Philosophy and Phenomenological Research, 10 (2): 172–96. Hacker, F. (1965), ‘Psychology and Psychopathology of Symbolism’, in J. R. Royce (ed.), Psychology and the Symbol: An Interdisciplinary Symposium, 73–87, New York: Random House. Hampe, M. (1998), Alfred North Whitehead, München: Beck. Head, H. (1926), ‘Disorders of Symbolic Thinking and Expression’, Brit J Psychol Vol XI, Part 2: 179–93. Hoffmeyer, J. (2015), ‘Introduction: Semiotic Scaffolding’, Biosemiotics, 8: 153–8. Hoffmeyer, J. (2008), Biosemiotics: An Examination into the Signs of Life and the Life of Signs, Scranton: Scranton University Press. Honneth, A. (1986), ‘The Fragmented World of Symbolic Forms: Reflections on Pierre Bourdieu’s Sociology of Culture’, Theory, Culture & Society, 3 (3): 55–66. Ihmig, K. N. (1997), Cassirers Invariantentheorie der Erfahrung und seine Rezeption des ‘Erlanger Programms’, Hamburg: Meiner. Ihmig, K. N. (2003), ‘Cassirers Philosophie der Mathematik’, in H. J. Sandkühler and D. Paetzold (eds), Kultur und Symbol. Ein Handbuch zur Philosophie Ernst Cassirers, 232–47, Stuttgart: Weimar. Innis, R. E. (1980), ‘Notes on the Semiotic Model of Perception’, Philosophical Inquiry, 2 (2–3): 496–507.

SEMIOTICS IN PSYCHIATRY AND PSYCHOLOGY

167

Innis, R. E. (2009), Susanne Langer in Focus: The Symbolic Mind, Bloomington: Indiana University Press. Ivanov, V. V. (1978), Č et i neč et: Asimmetrija mosga i znakovych system, Moskva: Sovjetskoe radio. Jakobson, R. O. (1942), ‘La théorie saussurienne en rétrospection’, Linguistics, 22 (2): 161–96. Jakobson, R. O. (1965), ‘Quest for the essence of language’, Diogenes, 13 (51): 21–37. Jaspers, K. (1946), Allgemeine Psychopathologie (4.Aufl.), Berlin-Heidelberg-New York: Springer. Jones, E. (1987/1916), Die Theorie der Symbolik, Frankfurt: Athenaeum. Jung, C. G. (1948), Allgemeine Gesichtspunkte zur Psychologie des Traumes. Psychologische Abhandlungen VI, Zürich: Rascher Verlag. Jung, C. G. (1950), Gestaltungen des Unbewussten. Psychologische Abhandlungen VII, Zürich: Rascher Verlag. Jung, C. G. (1991), Grundwerk in neun Bänden, Olten: Walter-Verlag. Kasanin, J. S., ed. 1944), Language and Thought in Schizophrenia, New York: Norton. Keller, P. (2015), ‘Cassirer’s Retrieval of Kant’s Copernican Revolution in Semiotics’, in J. T. Friedmanand S. Luft (eds), The Philosophy of Ernst Cassirer: A Novel Assessment, 259–88, Berlin: de Gruyter. Kick, H. and H. U. Diehl (1998), ‘Klinische Phänomenologie und therapeutische Situation’, Fundamenta Psychiatrica, 12: 53–7. Knobloch, C. and S. Schallenberger (1993), ‘Sprechhandlung und Sprachbedeutung in der Sprachpsychologie um 1930’, Historie Epistomologie Langage, 15 (1): 81–109. Koenig, D. (2019), ‘Ernst Cassirer und der mathematische Raum – vom Erkenntnisproblem zum Symbolproblem’, in R. A. O. Kroemer (ed), Mathematik in der Tradition des Neukantianismus, 35–55, Siegen: Universitaetsverlag. Krampen, M. (1981), ‘Phytosemiotics’, Semiotica, 36 (3/4): 187–209. Krampen, M., K. Oehler, R. Posner, Th. von Uexkuell, eds (1987), Classics of Semiotics, New York: Springer. Krois, J. M. (2004), ‘Ernst Cassirer’s Philosophy of Biology’, Sign Systems Studies, 32 (1/2): 277–95. Krömer, R., G. Nickel, D. Koenig, and S. Shokrani (eds) (2019), Mathematik in der Tradition des Neukantianismus. Siegener Beiträge zur Geschichte und Philosophie der Mathematik. Bd 11, Siegen: Universitaetsverlag. Kubie, L. S. (1953a), ‘The Distortion of the Symbolic Process in Neurosis and Psychosis’, Journal of the American Psychoanalytic Association, 1: 59–86. Kubie, L. S. (1953b), ‘The Central Representation of the Symbolic Process in Psychosomatic Disorders’, Psychosomatic Medicine, XV (1): 1–6. Kull, K. (1999), ‘On the History of Joining Bio with Semio’, Sign Systems Studies, 27: 128–38. Kull, K. (1999a), ‘Biosemiotics in the Twentieth Century: A View from Biology’, Semiotica, 127 (1–4): 385–414. Kull, K. (1999b), ‘Towards Biosemiotics with Juri Lotman’, Semiotica, 127 (1–4): 115–31. Kull, K., T. Deacon, C. Emmeche, J. Hoffmeyer, and F. Stjernfelt (2009), ‘Theses on Biosemiotics: Prolegomena to a Theoretical Biology’, Biological Theory, 4 (2): 167–73. Kull, K. (2014), ‘Towards a Theory of Evolution of Semiotic Systems’, Chinese Semiotic Studies, 10 (3): 485–95. Kull, K. (2016), ‘Alexandr Levich (1945–2016) and the Tartu-Moscow Biosemiotic Nexus’, Sign Systems Studies, 44 (1/2): 255–66.

168

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Kupferberg, J. R. (2010), ‘Language in Schizophrenia. Part 1 Lang’, Linguist Compass, 4 (8): 567–89. Kupferberg, J. R. (2010a), ‘Language in Schizophrenia. Part 2: What Can Linguists Bring to the Study of Schizophrenia … and Vice Versa?’, Lang. Linguist Compass, 4 (8): 590–604. Kussmal, A. (1874/78/81), ‘Die Störungen der Sprache’, in Ziemßen (1874) Handbuch der Krankheiten des Nervensystems Bd XII Verlag F.C.W. Vogel, Leipzig Lacan, J. et al. (2006), Écrits: The First Complete Edition in English, New York: W.W. Norton. Lang, A. (1992), ‘Zeichen nach innen, Zeichen nach aussen – eine semiotisch-ökologische Psychologie als Kulturwissenschaft’, in P. Rusterholz and M. Svilar (eds), Welt der Zeichen – Welt der Wirklichkeit. Berner Universitätsschriften, 55–84, Bern: Paul Haupt. Langer, S. K. (1957), Philosophy in a New Key: A Study of the Symbolism of Reason, Rite, and Art, 3rd edn, Cambridge, MA: Harvard University Press. Langer, S. K. (1967/1972/1982), Mind: An Essay on Human Feeling. I–III, Baltimore: J. Hopkins University Press. Leferink, K. (2008), ‘Psychosemiotik –Ein Ansatz zur Kritik der Identitaet’, Journal Philosophie der Psychologie. http://www.jp.philo.at/texte/LeferinkK1.pdf. Leontjew, A. N. (1977), Probleme der Entwicklung des Psychischen, Kronberg, TS: Athenaeum. Leuner, H. C. (1970), Katathym-imaginative Psychotherapie. Stuttgart: Thieme. Leuner, H. C. (1997/1962), Die experimentelle Psychose, Berlin-Göttingen-Heidelberg: Springer. Levin, F. M. (1991), Mapping the Mind, Hillsdale, NJ: Analytic Press. Lewin, K. (1926), Vorsatz, Wille, Beduerftnis, Berlin: Julius Springer. Linask, L. (2019), ‘Vigotskij’s Natural History of Signs’, Sign Systems Studies, 47 (1/2): 257–304. Litowitz, B. (1990), ‘Elements of Semiotic Theory Relevant to Psychoanalysis’, in B. L. Litowitz and P. S. Epstein (eds), Semiotic Perspectives on Clinical Theory and Practice: Medicine, Neuropsychiatry and Psychoanalysis, Berlin: Mouton de Gruyter. Lofts, S. G. (1994), ‘L’ordre symbolique de Jaques Lacan a la lumiere du symbolique d’Ernst Cassirer’, in S. G. Lofts and P. Moyaert (eds), La pensee de Jaques Lacan, Paris: Edition Peeters. Lorenzer, A. (1970), Kritik des psychoanalytischen Symbolbegriffs, Frankfurt: Suhrkamp. Lurija, A. R. (1992), Das Gehirn in Aktion. Einführung in die Neuropsychologie, Reinbek bei Hamburg: Rowohlt. Magerski, C. (2005), ‘The Consecrating Power of Symbolic Forms: From Cassirer’s Philosophy of Symbolic Forms to Bourdieu’s Sociology of Symbolic Forms’, Z. f Soziologie, Jg,.34 (2): 112–27. Marty, M. and M. de M’uzan (1978), ‘Das operative Denken (pensée opératoire)’, Psyche, 32 (10): 974–984. May, M. (2018), ‘Editorial: Why the Clinical Utility of Diagnostic Categories in Psychiatry Is Intrinsically Limited and How We Can Use New Approaches to Complement Them’, World Psychiatry, 17 (2): 121–2. Mentzos, S. (1997), Psychose und Konflikt, Göttingen: Vandenhoeck & Ruprecht. Mentzos, S. (2009), Lehrbuch der Psychodynamik, Göttingen: Vandenhoeck & Ruprecht. Mies, T. and D. Nitzgen (2013), ‘Symbolbegriff und Symboltheorie in Psychoanalyse und Gruppenanalyse’, Psychosozial, 36 I Nr.131: 5–7. Mininni, G. (1993), ‘Psychosemiotics: Theoretical and Applied’, in P. Grzybek (ed.), Psychosemiotik – Neurosemiotik (Bochumer Beiträge zur Semiotik Bd 41), 157–98, Bochum: Brockmeyer.

SEMIOTICS IN PSYCHIATRY AND PSYCHOLOGY

169

Mormann, T. (2015), ‘Mathematics to Quantum Mechanics. On the Conceptual Unity of Cassirer’s Philosophy of Science (1907–1937)’, in J. T. Friedman and S. Luft (eds), The Philosophy of Ernst Cassirer: A Novel Assessment, 31–63, Berlin: de Gruyter. Morris, C. W. (1938), Foundations of the Theory of Signs, Chicago: University of Chicago Press. Mueller, U., B. Sokol, and W. F. Overton (1998), ‘Reframing a Constructivist Model of the Development of Mental Representation: The Role of Higher Order Operations’, Developmental Review, 18: 155–201. Mueller, U., B. Sokol, and W. F. Overton (1998a), ‘Constructivism and Development: Reply to Smith’s Commentary’, Developmental Review, 18: 228–36. Muller, J. P. (1996), Beyond the Psychoanalytic Dyad: Developmental Semiotics in Freud, Peirce and Lacan, New York: Routledge. Natterer, P. (2010), ‘Eckdaten der Psycholinguistik und Kognitiven Linguistik zum Zusammenhang von Realität, Denken und Sprache, Skript 2010 [1999]’. https://www.paulnatterer.de/file/99fbc4f6c7a8e729ffff80f6fffffff0.pdf (accessed 1 July 2020). Neumann, E. (1995), Ursprungsgeschichte des Bewußtseins, Frankfurt: Fischer TBV. Nitzgen, D. (2010), ‘The Matrix Reloaded: Ernst Cassirer – Kurt Goldstein – S.H. Foulkes’, Psychosozial, 119: 25–39. Nöth, W. (1990), Handbook of Semiotics, Bloomington: Indiana University Press. Olds, D. D. (1990), ‘Brain-centered Psychology: A Semiotic Approach’, Psychoanalysis and Contemporary Thought, 3: 331–63. Olds, D. D. (1992), ‘The Physicality of the Sign’, Semiotics, 9: 166–73. Olds, D. D. (1994), ‘Connectionism and Psychoanalysis’, Journal of the American Psychoanalytic Association, 42: 581–611. Olds, D. D. (2000), ‘A Semiotic Model of Mind’, Journal of the American Psychoanalytic Association, 48 (2): 497–529. Otte, M., Th. Mies, and M. Hoffmann (1997), Die Symmetrie von Subjektbezug und Objektivität wissenschaftlicher Verallgemeinerung. Untersuchungen zur Begründung wissenschaftlicher Rationalität im Anschluß an die mathematische Philosophie von Charles S. Peirce (Occasional Paper 162). Bericht für die Deutsche Forschungsgemeinschaft, Bielefeld. Piaget, J. (1945), La formation du symbole chez l’enfant. Imitation, jeu et reve, image et representation, Neuchatel et Paris: Delachaux et Niestle. Piaget, J. (1973), Einführung in die genetische Erkenntnistheorie, Frankfurt: Suhrkamp. Peirce, C. S. ([1866–1913] 1931–58), The Collected Papers of Charles Sanders Peirce, vols. 1–6, eds. C. Hartshorne and P. Weiss, vols. 7–8, ed. A. Burks, Cambridge: Harvard University Press. Cited as CP. Pelkey, J., S. Melanson, and R. Rosenbaum (2019), ‘Introduction: Cognitive Semiotics’, The American Journal of Semiotics, 35 (1–2): 1–9. Peters, U. H. (1973), ‘Wortfeld-Störung und Satzfeld-Störung: Interpretation eines schizophrenen Sprachphänomens mit strukturalistischen Mitteln’, Arch Psychiat Nervenkr, 217: 1–10. Peters, U. H. and J. A. Prelle (1974), ‘Schizophasie: Verschleierung einer Sprachstörung durch Sprache?’, Arch Psychiatr Nervenkr, 219: 277–84. Pick, A. (1908), ‘Asymbolie, Apraxie, Aphasie’, in Congres internationale de psychiatrie, Amsterdam 1907: 341ff. Pietikainen, P. (1999), C. G. Jung and the Psychology of Symbolic Forms, Helsinki: The Finish Academy of Science and Letters.

170

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Portnov, A. N. (1993), ‘Evolutionsgenetische Probleme der Psychosemiotik’, in Grzybek (ed.), Psychosemiotik – Neurosemiotik (Bochumer Beiträge zur Semiotik Bd 41), 225–53, Bochum: Brockmeyer. Posner, R. (1987), ‘Charles Morris and the Behavioural Foundations of Semiotics’, in Krampen et al., eds, Classics of Semiotics, 23–58, New York: Springer. Rothschild, F. S. (1935/1989), Symbolik des Hirnbaus: Erscheinungswissenschaftliche Untersuchungg über den Bau und die Funktion des Zentralnervensystems der Wirbeltiere und des Menschen, Bonn: Bouvier. Rothschild, F. S. (1962), ‘Laws of Symbolic Mediation in the Dynamics of Self and Personality’, Annals of the New York Academy of Sciences, 96: 774–84. Rothschild, F. S. (1994/1986), Creation and Evolution: A Biosemiotic Approach, Mevasseret Zion, Israel: L.Ph.Hes, C. Sorek. Royce, J. R., ed. (1965), Psychology and the Symbol: An Interdisciplinary Symposium, New York: Random House. Rusterholz, P. and M. Svilar, eds. (1993), Welt der Zeichen – Welt der Wirklichkeit. Berner Universitätsschriften, Bern: Paul Haupt. Ryckman, T. (2015), ‘A Retrospective View on “Determinism and Indeterminism” in modern Physics’, in J. T. Friedman and S. Luft (eds), The Philosophy of Ernst Cassirer: A Novel Assessment, 65–102, Berlin: de Gruyter. Sandkühler, H. J. and D. Paetzold, eds (2003), Kultur und Symbol, Stuttgart Weimar: JB Metzler. Saporoshez, A.W. (1958), Die Bildung willkürlicher Bewegungen, Habil (in Russian language), Turku: Annates Universitatis Turkuensis. Sartorius, N. (2002), Understanding the ICD-10 Classification of Mental Disorders, London: Science Press. Sass, L. (1992), Madness and Modernism: Insanity in the Light of Modern Art, Literature, and Thought, New York: Basic Books. Saussure, F. ([1916] 1962), Cours de linguistique Générale, Paris: Edition préparée par Tullio de Mauro. Schlick, M. (1936), ‘Über den Begriff der Ganzheit’, Actes du Huitième Congrès International de la Philosophie à Prague, 2–7 September 1934, 85–99. Schneider, K. (1939), Psychischer Befund und psychiatrische Diagnose, Leipzig: Thieme. Scholes, R. (1982), Semiotics and Interpretation, vol. 465, New Haven, CT: Yale University Press. Schur, M. (1955), ‘Comments on the Metapsychology of Somatizatization’, Psychoanalytic Study of the Child, 10: 119–64. Schutz, A. (1957), ‘Das Problem der transzendentalen Intersubjektivität bei Husserl’, Philosophische Rundschau, 5 (2): 81–107. Schutz, A. (1975), Collected Papers III, The Hague: Martinus Nijhoff. Sebeok, T. A. (1972), Perspectives in Zoosemiotics, The Hague: Mouton. Sebeok, T. A. (1975), ‘Zoosemiotics: At the Intersection of Nature and Culture’, in T. A. Sebeok (ed.), The Tell-tale Sign, 85–95, Lisse: Peter de Ridder. Sebeok, T. A. (1986), ‘Preface’, in T. A. Sebeok (ed.), Encyclopedic Dictionary of Semiotics, vol. 1, v–viii, Berlin: Mouton de Gruyter. Segal, H. (1957), ‘Notes on Symbol-Formation’, Int J Psycho-Anal, 38: 391–97. Segal, H. (1959), ‘On Symbolism’, Int J Psycho-Anal, 59: 315–19. Shakov, D. (1968), ‘Contributions from Schizophrenia to the Understanding of Normal Psychological Function’, in M. L. Simmel (ed.), The Reach of Mind: Essays in the Memory of Kurt Goldstein, 173–202, New York: Springer.

SEMIOTICS IN PSYCHIATRY AND PSYCHOLOGY

171

Sifneos, P. E. (1975), ‘Problems of Psychotherapy in Patients with Alexithymie Characteristics and Physical Diseases’, Psychotherapy and Psychodynamics, 26: 65–70. Silberer, H. (1912), ‘Über die Symbolbildung. Jahrb f psychoanal. und psychopathol. Forschung’, 621–60, Wien: Deuticke. Simmel, M. L., ed. (1968), The Reach of Mind: Essays in the Memory of Kurt Goldstein, New York: Springer. Simon, T. W. and R. J. Scholes, eds (1982), Language, Mind, and Brain, New York: Hillsdale. Smith, H. A. (2005), Psychosemiotics and Its Peircean Foundation, Kingston, Ontario: Queen’s University. Smith, L. (1998), ‘On the Development of Mental Representation’, Developmental Review, 18: 202–27. Smyth, W. E. and J. Jorna (1998), ‘The Signs We Live By: The Relationship between Semiotics and Psychology’, Theory & Psychology, 8 (6): 723–30. Spamer, C. (1876), ‘Über Aphasie und Asymbolie nebst dem Versuch einer Theoriebildung’, Arch f Psychiatr, 6: 496ff. Staiti, A. (2012), ‘Human Culture and the One Structure: On Luft’s Reading of the Late Husserl’, Comparative and Continental Philosophy, 4 (2): 317–30. Stjernfelt, F. (2012), ‘Peirce and Cassirer – The Kroisian Connection: Vistas and Open Issues’, in H. Bredekamp et al. (eds), John Krois’ Philosophical Semiotics, Bodies in Action and Symbolic Forms: Zwei Seiten der Verkörperungstheorie, 37–46, Berlin: Akademie Verlag. Tomasello, M. (2019), Becoming Human: A Theory of Ontogeny, Cambridge: Harvard University Press. Trifonas, P. P., ed. (2015), International Handbook of Semiotics, Berlin: Springer. van de Vijver, G. (2000), ‘Identification and Psychic Closure: A Dynamic Structuralist Approach of the Psyche’, Annals of the New York Academy of Sciences, 901 (1): 1–12. van Ey, H. (1952), ‘Grundlagen einer organo-dynamischen Auffassung der Psychiatrie’, Fortschr. Neurol. Psychiatr., 20 (5): 195–209. van Heusden, B. and R. Jorna (2003), ‘Why Representation Will Not Go Away: Crisis of Concept or Crisis of Theory?’, Semiotica, 143–1 (4): 113–34. Velmezova, E., K. Kull, and S. J. Cowley, eds (2015), Biosemiotic Perspectives on Language and Linguistics, Heidelberg: Springer. Vigotskij, L. (1934), ‘Thought in Schizophrenia’, Arch Neurol Psychiatr, 31: 1063ff von Berthalanffy, L. (1965), ‘On the Definition of the Symbol’, in J. R. Royce (ed.), Psychology and the Symbol: An Interdisciplinary Symposium, 26–72, New York: Random House. Von Uexküll, J. (1909), Umwelt und Innenwelt der Tiere, Berlin: Springer. Werner, H. and B. Kaplan (1964), Symbol Formation, New York and London and Sidney: John Wiley & Sons. Whitehead, A. N. (1927/1985), Symbolism: Its Meaning and Effect, New York: Fordham University Press. Whitehead, A. N. (1929/1971), The Function of Reason, Princeton, NJ: Princeton University Press. Wirth, U. (1995), ‘Abduktion und ihre Anwendungen’, Zeitschrift f Semiotik Bd, 17 (3–4): 405–24. Zahavi, D. (2010), ‘Empathy, Embodiment and Interpersonal Understanding: From Lipps to Schutz’, Inquiry, 53 (3): 285–306.

172

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Zlatev, J. (2009), ‘Levels of Meaning, Embodiment and Communication’, Cybernetics and Human Knowing, 14 (3–4): 149–74. Zlatev, J. (2015), ‘Cognitive Semiotics’, in P. P. Trifonas (ed.), International Handbook of Semiotics, 1043–67, Berlin: Springer.op. cit. Zlatev, J. and M. Andrén (2009), ‘Stages and Transitions in Children’s Semiotic Development’, in J. Zlatev, et al. (eds), Studies in Language and Cognition, 380–401, Newcastle: Cambridge Scholars. Zlatev, J., M. Andrén, Falck M. Johansson, and C. Lundmark, eds (2009), Studies in Language and Cognition, Newcastle: Cambridge Scholars, 380–401.

CHAPTER EIGHT

Semiotics in Neuroscience and Cognition KRISTIAN TYLÉN AND JIJO KANDAMKULATHY

Together with a general ‘cognitive turn’ in the humanities during the 1980s and 1990s, the brain and its cognitive systems have become an indispensable part of the discussion of (human) meaning making. While in the structuralist semiotic tradition, meaning was often portrayed as a more or less intrinsic quality of objects and texts, most semiotic scholars today endorse an approach to meaning-making as unfolding in a dynamic relation between a ‘cognizing’ agent and its environment (e.g. objects and texts), involving processes of non-mechanistic and non-deterministic, context-dependent interpretation. Only cognitive systems (in the broadest sense of the word) are capable of such interpretations, which places cognition in the centre of semiosis. This naturally also leads to scientific curiosity concerning the brain as one of the main (but possibly not the only) loci of cognitive processes. While it is important to stress that there is to date no such thing as a unified research field of ‘neurosemiotics’, the term is increasingly used regarding research into the neurophysiological basis of semiotic behaviour. The term is used in two rather different ways. Neurosemiotics is sometimes considered a branch of biosemiotics; and, in this context, the term is mostly used about electrophysiological cellular processes redescribed in terms of signs and interpretations. The term, however, is also considered a branch of cognitive semiotics, and is used, in this context, about the study of functional neural structures subserving cognitive processes of (human) meaningmaking. In the following, we will take this second perspective, presenting central tenets in the intersections between semiotics, cognitive science and neuroscience with particular emphasis on empirical studies applying brain imaging techniques such as fMRI, PET and EEG to inform investigations of semiotic processes. Many of these studies are not originally motivated or framed in semiotic terminology, but are recognized by the semiotician to have deep implications for the way in which our brains organize and process meaning.

MEANING AND MIND The field of semiotics is diverse, and concepts are often used in quite different ways in different traditions. Before diving into the particularities of brain structures and processes involved in human meaning-making, it would thus be useful to briefly outline a conceptual framework in an attempt to establish terminological clarity. It is, however, important to stress that the main purpose of this chapter is not to contribute to the development of fine-grained conceptual distinctions as we would easily get lost in the conceptual entanglements also so characteristic of the field of semiotics.

174

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

The concept of meaning is absolutely central to any theory of semiotics, yet it is still strangely intangible and challenging to define, and consequently a subject of controversy in philosophy of language and semiotics (Frege 1948; Wittgenstein 1953; Fodor 1975; Levinas 1987; Landau 2000; Croft 2010; Tylén et al. 2013; Stjernfelt 2015). Most theories, however, endorse some version of the primary Saussurean distinction between ‘referent’ (e.g. an object in the physical world) and its ‘meaning’ (how it presents itself to the organism), with the implication that the same physical object can mean different things and different physical referents can mean the same, depending on the organism or the context of the encounter (Saussure [1916] 1959). For instance, the same stone can be perceived as a geological phenomenon in one context, and a piece of art or a useful tool in another context (Tylén, Wallentin, and Roepstorff 2009; Tylén, Bjørndahl, and Weed 2011). Along similar veins, different languages are observed to carve up the same physical world in quite different ways, suggesting that our concepts are not a one-toone reflection of the physical world, but an expression of an active process of culturally situated interpretation (Majid et al. 2004, 2007; Malt et al. 2008; Malt 2019). While other classical approaches assume meaning to have an innate core (e.g. Fodor 1975), belong to an ideal, platonic realm independent of the human mind (e.g. Frege 1948), or consciousness (e.g. Witzany 2016), a grounding assumption in cognitive semiotics is that meaning is intimately connected to processes of embodied perception and cognition (Zlatev 2009, 2012; Brandt 2011) as well as the social, cultural and ecological processes guiding and constraining these processes (Vogeley and Roepstorff 2009; Donald 2001; Hutchins 2011; Tylén et al. 2013). This makes the human mind and its contextualized interactions with the surroundings world a natural object of scientific scrutiny. In other words, cognitive semiotics is (mainly) preoccupied with cognitive processes of construal mediating between the world and human phenomenal experience and how they are shaped through our social and communicative practices. Adopting concepts from cognitive science and cognitive linguistics, these are often discussed under the headline of representation, categorization, conceptualization, schemas, frames and scripts (Johnson 1989; Lakoff and Johnson 1999; Talmy 2000). While a few of these cognitive constructs might arise from hardwired primitives (e.g. the so-called principle of persistence, Baillargeon 2008; Bremner, Slater; and Johnson 2015), they are mainly thought to have an experiential origin, that is, formed through our bodily interactions with the material, social and cultural environment – hence the emphasis on embodiment (Lakoff and Johnson 1999; Sonesson 2007; Zlatev 2009; Tylén et al. 2013). In this respect, neurosemiotics can be regarded as the scientific attempt at naturalizing the representation processes that give shape to our experiences as they unfold at various timescales and in various areas of the human brain.

COGNITION AND BRAIN The relation between meaning, consciousness, cognition and the electro-chemical processes of the brain is also non-trivial and subject of continuous theoretical controversy. Importantly, neural activity in the brain goes beyond that of supporting cognition, as it is involved in a large number of basic low-level processes, such as a regulation of respiration, heartrate, hormonal processes, etc. (Gazzaniga 2009). Most of these autonomic processes are beyond conscious awareness and control, and carried out by

SEMIOTICS IN NEUROSCIENCE AND COGNITION

175

the deeper and evolutionarily primary brain structures (sometimes called the reptilian brain) largely shared with other species. In contrast, most cognitive functions are thought to reside in the (neo)cortex, which is relatively larger and more developed in humans (Gazzaniga 2009). However, it is not straight-forward to establish clear principles for the relation between cognition and brain processes, also due to the varying definitions of cognition. In classical cognitivist approaches – inspired by early work in artificial intelligence – ‘cognition’ mainly referred to explicit processes of thinking, understanding, deciding and recalling that were assumed to unfold as algorithmic transformations of symbolic representations following the principles of formal logic (Thagard 1996). More low-level or implicit processes for instance related to affect and emotions were largely ignored, just as perception and action were regarded input and output systems and thus not part of cognition-proper (Abrahamsen and Bechtel 2012). An influential version of this idea is formulated in Jerry Fodor’s Language of Thought (LOT, 1975). The theory assumes a process of translation between modality specific perceptual information into an amodal symbolic code sometimes referred to as ‘mentalese’ – or the language of thought. The brain thus carries out its computations as manipulations of symbolic strings that are eventually translated back, for instance as bodily action or expressions in a natural language. More contemporary approaches converge in treating perception, action and limbic processes as ‘cognitive’ and often even intimately connected to, for instance, processes of attention, memory and language (Pecher and Zwaan 2005). Moreover, a number of recent approaches question the extent to which cognition is in fact best defined as amodal information processing, or whether perceptual modalities, gesture and action are integral to even more high-order cognitive processes (Alibali and Goldin-Meadow 1993; Goldin-Meadow Alibali, and Church 1993; Barsalou 1999; Goldin-Meadow, Kim, and Singer 1999; Barsalou et al. 2003; Hauk, Johnsrude, and Pulvermuller 2004; Arbib 2008; Barsalou 2008; Spivey 2008), and abstract reasoning – for instance, through analogy and metaphoric extension (Lakoff and Johnson 1980, 1999; Hofstadter 2001; Holyoak, Gentner, and Kokinov 2001; Fauconnier and Turner 2008; Gibbs Jr 2011). Furthermore, recent studies have found cognitive processes related to joint problem solving, social affiliation and literature reading to modulate even basic physiological processes such as heart rate pointing to the deeply integrative nature of cognition (Konvalinka et al. 2011; Wallentin et al. 2011a; Fusaroli et al. 2016). Another, yet related, discussion concerns the boundaries of human cognition. Most handbook definitions portray cognition as ‘mental’ processes (cf. e.g. Wikipedia, Merriam-Webster, The Cambridge English Dictionary, etc.). This has the implication that cognition is confined to internal workings of the brain. The strong emphasis on ‘mental representations’ is also characteristic of second generation cognitive science, cognitive linguistics and early advances in cognitive semiotics (Croft 1998; Sinha 2007). However, inspired mainly by work in phenomenology, psychology and activity theory (e.g. Heidegger, Merleau-Ponty, Vygotsky, and Leont’ev), during the late 1990s, a new ‘active externalism’ arose under headlines of the extended mind (Clark 1997; Clark and Chalmers 1998) and distributed cognition (Hutchins 1995a), challenging the assumed mental ontology of cognition. While the two approaches (born in philosophy and anthropology, respectively) differ in important respects (see, e.g., Hutchins 2011), they converge in suggesting that cognitive processes extend beyond the human brain. Not unlike classical cognitivism, cognition is thus defined in functional terms as ‘information processing’. However, the processes are thought to be flexible in terms of  their

176

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

implementation and will often straddle the internal-external divide as cognitive processes of perception, memory and decision-making involve interactions with material artefacts and structures (Hutchins 1995b; Latour 1996; Clark 2006; Roepstorff 2008; Bjørndahl et al. 2014; Tylén et al. 2014; Tylén and McGraw 2014), or other agents and cultural institutions (Hutchins 2011, 1991; Fusaroli, Gangopadhyay, and Tylén 2014). The basic tenets of the external mind theory are summarized in Clark and Chalmers’ so-called parity principle: If, as we confront some task, a part of the world functions as a process which, were it done in the head, we would have no hesitation in recognizing as part of the cognitive process, then that part of the world is […] part of the cognitive process. (Clark and Chalmers 1998: 29) These ideas have resonated well in the cognitive semiotics community as they are found to have precursors in the work of C. S. Peirce (Paolucci 2011; Atã and Queiroz 2014) and J. von Uexküll (Sonesson 2015; Cárdenas-García and Ireland 2017), and point to the active role played by our engagement of sign-vehicles in structuring, exploring and sharing meaning, for instance, when manipulating diagrammatical representations (Goodwin 2000; Stjernfelt 2007, 2014a; Tylén, Bjørndahl, and Weed 2011; Bjørndahl et al. 2014; Tylén et al. 2014). However, the implications for cognitive neuroscience are non-trivial. Rather than being the locus and hardware for cognition, the brain and its anatomical structures become components of larger ‘coupled systems’ flexibly assembled to comprise also external artefacts, cultural practices and other agents. The cognitive process of recalling a piece of information – for instance an address – can thus either involve our biological memory system and thus brain structures such as the hippocampus and the temporal lobes, or external artefacts such as a notebook or a smartphone (Donald 1991; Clark and Chalmers 1998; Clark 2006). Practically, given their distributed nature, such coupled systems are hard to experimentally record or ‘measure’ with the standard methodological repertoire (brain imaging technologies, reaction times, eye- and motion tracking, etc.), which means that most existing research in extended mind and distributed cognition has been conceptual and anthropological, while cognitive neuroscientists have remained more conservative in their conception of cognition (noteworthy exceptions include Dotov, Nie, and Chemero 2010; Mirza et al. 2018; Kirsh and Maglio 1994; Jack and Roepstorff 2002; Tylén et al. 2012; Hasson et al. 2012; Tylén et al. 2016). Other related discussions concerning the nature of human cognition have unfolded under the headline of, for instance, enactive, situated and embedded cognition (Newen, De Bruin, and Gallagher 2018). While a more in-depth discussion of these approaches is beyond the scope of this chapter, it is important to emphasize that they each have potential farreaching implications for the relation between cognition and brain. Today, however, there is still a widespread conceptual disconnect between such models of cognition and empirical studies of brain function. Since the methods and measurements of cognitive neuroscience are designed and largely constrained to seek for answers in the brain processes of the individual, the evidence produced from brain imaging studies tends to (implicitly or explicitly) support a conception of cognition as a property of the individual brain. There are, however, a few promising attempts at looking at connectivity across brains (Montague et al. 2002; Astolfi et al. 2010; Dumas et al. 2011; Cui, Bryant, and Reiss 2012; Liu et al. 2016; Miller et al. 2019).

SEMIOTICS IN NEUROSCIENCE AND COGNITION

177

THE BRAIN AND ITS PROCESSES When approaching the role of the brain in human meaning making, a useful first distinction is between the structural and the functional brain. The structural brain is mainly studied with brain imaging techniques such as positron emission tomography (PET), computerized tomography (CT), magnetic resonance (MR) and diffusion tensor imaging (DTI) as well as autopsy of in vitro brains, with the purpose of detailed mapping of the brain’s anatomical structures and their connections (Carter and Shieh 2015). In contrast, studies of the functional brain aim to record in vivo unfolding neuronal activity mostly in response to particular perceptual or cognitive tasks. Neuronal activity can be directly measured using single cell recording, where electrodes are inserted directly into a neuron of interest (Gazzaniga 2009). However, this method is very invasive thus not considered ethical to perform on human subjects (with the exception of some presurgery contexts). Most studies of brain function thus rely on more indirect measures of neuronal activity. One of the most prevalent methods for the study of functional brain activity is functional magnetic resonance imaging (fMRI; see Figure 8.1). fMRI measures the so-called BOLD signal (blood-oxygen-level-dependent). Neurons use oxygen, and increased neuronal activity in an area of the brain will thus cause and upregulation of the blood flow to that area. The BOLD signal is an expression of the relation between oxygenated and deoxygenated blood at various times and in various areas of the brain, and can thereby be used as an indirect proxy for patterns of neuronal activity in response to stimuli and tasks (Friston, Jezzard, and Turner 1994; Friston et al. 1995). For instance, using fMRI, Zeki and Marini contrasted participants’ brain activation to pictorial stimuli depicting red and blue strawberries (1998). By subtracting the neural responses of the normal from abnormal colour stimulus events, they found enhanced blood flow to regions of the dorsolateral prefrontal cortex apparently specific for the perception of incongruent colours, with potential implications for aesthetic experiences in the perception of art (Zeki 2000).

FIGURE 8.1  fMRI brain imaging. A: The MR scanner setup used in fMRI experiments. B: Examples of resulting brain images. The coloured patterns produced reflect a statistical analysis investigating the average increased blood flow (BOLD) to certain regions across a number of experimental participants in response to particular stimuli or tasks.

178

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

While fMRI is particularly well-suited to study the location of neuronal activity in the brain (it has a high spatial resolution), the regulation of blood flow is a relatively slow process (~2–4 s) making the technique less sensitive to the fast temporal patterns of brain activity (low temporal resolution). Similarly, PET can be used for functional brain imaging, but since it relies on the injection of a radioactive tracer with potential health risk for the participant, is very expensive, and since it has an even lower temporal resolution than fMRI, today, it is mostly used for studies with clinical purposes and populations (Bailey et al. 2005; Carlson 2012). Another prevalent method for studying functional brain activity is electroencephalography (EEG). EEG records the subtle electrophysiological voltage fluctuations generated from neuronal activity using electrodes non-invasively attached to the scalp of the participant (Niedermeyer and da Silva 2005; Gazzaniga 2009). While the method can also generate maps pointing to the proximate location of neuronal activity, the spatial resolution is relatively poor, and constrained to cortical activity (while it cannot reliably capture activity in deeper structures of the brain). However, EEG has an excellent temporal resolution making it well fit to investigate questions pertaining to the unfolding of neuronal processes on the millisecond level. In particular, a derivative component of EEG called event-related potentials (ERP) has been used to study time-locked responses to linguistic or visual stimuli (Chwilla, Brown, and Hagoort 1995; Posner, Rugg, and Coles 1995; Luck 2005). An example of a well-studied ERP component is the N400 (i.e. a negative voltage fluctuation around 400 milliseconds after stimulus onset), which is interpreted as an indication that the brain is surprised by some contextual incongruency. In a classical study, Kutas and Hillyard (1980) contrasted regular sentences such as ‘she put on her high-heeled shoes’ with target sentences like ‘he spread the warm bread with socks’. Approximately 400 milliseconds after the word ‘socks’ they observed a negative fluctuation of the ERP signal (an N400), suggesting that this word was unexpected and difficult to integrate in the context. More recent methodological advances in the cognitive neurosciences include magnetoencephalography (MEG) and functional near-infrared spectroscopy (fNIRS). Contrary to PET, fMRI and EEG, the hardware for these is particularly designed for the study of neuro-cognitive processes, and is thus mostly found in dedicated neurocognition research labs (they are not part of the standard hospital equipment, and therefore currently less widespread). Similar to EEG, MEG uses sensitive magnetometers to record the electrophysiological signal from neuronal activity. However, MEG is recorded in using a large stationary system and comes with high spatial resolution as well as temporal resolution (Hamalainen et al. 1993; De Pasquale et al. 2010). Like fMRI, fNIRS records the relative oxygenation of blood flow, however, using infrared light electrodes on the scalp, which has the advantage that it is potentially portable and allow for more natural engagements with the surroundings (Bunce et al. 2006; Ferrari and Quaresima 2012).

ON LOCALIZATION VERSUS DISTRIBUTION OF BRAIN FUNCTION While studies of the structural (anatomical) approaches could seem less informative of inquiries into human semiotic behaviour and meaning-making than functional, historically lesion studies have been the golden standard in connecting cognition and brain function.

SEMIOTICS IN NEUROSCIENCE AND COGNITION

179

FIGURE 8.2  Paul Broca’s original record of Lebourgne autopsy. A large lesion is observed in left inferior frontal cortex, later called Broca’s area. Credit: CC BY/Wellcome Trust Images (The Wellcome Trust 2014).

Long before the advent of contemporary neuroimaging techniques, the cognitive neuroscientist would make observations of a patient’s abnormal behaviour in order to later associate it with lesions or other anatomical anomalies identified, for instance, by post-mortem autopsy. Famous examples include studies of the aphasic patient Lebourgne, that led Paul Broca’s to conclude that language is left-lateralized and processed in an area of the inferior frontal gyrus, now known as Broca’s area (Broca 1865; Gazzaniga 2009), or Scoville and Milner’s studies of the patient H. M., which led to the distinction between short- and long-term memory and early ideas about the corresponding brain structures (Scoville and Milner 1957; Eichenbaum 2013). While these kinds of case studies have generated many robust and reproducible results, the lesion method has also guided particular long-lived assumptions regarding the functional architecture of the brain. While not uncontested (see, e.g., Cole and Engeström 1993), the predominant conceptualization guiding neurocognitive research through the twentieth century has been that of the one-to-one mappings between cognitive functions and brain structures, an idea originally born in the doubtful discipline of Phrenology (Franz 1912; Simpson 2005). Broca’s studies were thus thought to successfully locate ‘the language area’ of the brain, and following the same basic scheme, twentieth-century cognitive neuroscience has largely been preoccupied with locating the ‘X area’ of the brain (Viola 2017). While not fully coinciding, this approach has resonated with the theory of ‘the modular mind’: that human cognition is compartmentalized in separate, innate modules each with their designated function (analogues to organs in the body, Fodor 1983; Coltheart 1999). While there are indeed many cognitive functions that appear consistently related to particular brain structures (e.g. face recognition in the fusiform gyrus or memory encoding in the hippocampus, Gazzaniga 2009), the dispute continues as new conceptual and methodological approaches challenge the idea of localizability as the main organizational principle of the brain. An increasing amount of studies thus approach the brain as distributed and hierarchically structured networks (Friston 2002, 2010;

180

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Friston et al. 2011a), or complex dynamic systems (Kelso 1995, 2012; Spivey 2008). The emerging picture seems to be one where ‘low-level’ cognitive functions, for instance pertaining to early processes of perceptions and action, seem more localized, while higher-order functions related to abstract reasoning, decision making and language are more distributed (Sporns 2002). The study of how language is processed in the human brain serves as an illustrative example. Following Paul Broca’s original findings, Carl Wernicke located another area in the left hemisphere superior temporal sulcus related to language (Wernicke 1874; Bogen and Bogen 1976). The two areas later formed the main constituents of the WernickeGerschwind model of language processing: Broca’s area was thought to be related to language production, while Wernicke’s area was thought to be the locus of language comprehension or the ‘mental lexicon’ (Gazzaniga 2009). The model is still referenced in contemporary research articles or handbooks of neurocognition, both because it was found to have some explanatory value in regard to aphasia (e.g. Faroqi-Shah and Thompson 2003; Ruiz et al. 2018), but possibly also because it fitted well with theoretical ideas about innateness, modularity and universality of language (Chomsky 1975, 1986; Pinker 1994; Hauser, Chomsky, and Fitch 2002). However, in more recent times, a host of studies have pointed to the limitations – if not misconception – of the original model (Binder et al. 2009; Hagoort 2013; Binder 2017). In the Wernicke-Gerschwind model, the comprehension of spoken or written language implied looking up word meanings in a ‘mental lexicon’ thought to reside in Wernicke’s

FIGURE 8.3  Contemporary version of the Wernicke-Gerschwind model of language processing. Broca’s area was thought to be involved in language production, while Wernicke’s area was thought to be involved in language comprehension. In a word repetition task, the signal would thus proceed from perceptual analysis in the primary auditory cortex to semantic analysis in Wernicke’s area, after which it would be passed on to Broca’s area for production planning before it is send to the motor cortex.

SEMIOTICS IN NEUROSCIENCE AND COGNITION

181

area (left posterior temporal cortex). While the idea of an anatomically encapsulated lexicon fared well with the approach to meaning as an amodal symbolic encoding of features, the theory of embodiment – together with new methods for studying functional brain activation – has come with a whole new set of predictions. In this approach, the meaning of a word is suggested to be related to the activation of larger networks of brain areas pertaining to, for instance, the perceptual or motoric qualities of the semantics. In an fMRI brain imaging study, participants were thus presented with action related words such as kick, pick and lick. Along with activation of classical language areas, the researchers found activation patterns in leg, arm and face-related areas of the motor cortex for each of the words respectively, pointing to the possible involvement of bodily simulation in the comprehension of linguistic stimuli (Hauk, Johnsrude, and Pulvermuller 2004; Shtyrov, Hauk, and Pulvermuller 2004). Likewise, the presentation of linguistic stimuli pointing to motion and space seems consistently to evoke activation of temporal and parietal areas also involved in (non-linguistic) motion perception and spatial navigation (Wallentin, Lund, et al. 2005; Wallentin, Østergaard, et al. 2005; Wallentin et al. 2006, 2008, 2011b; Rocca et al. 2020), while emotional content seem to activate limbic structures such as the amygdala (Wallentin et al. 2011a). A recent set of studies using novel modelling methods have mapped even more radically distributed networks of activation in response to natural language comprehension, organized along several semantic dimensions and covering, in fact, most of cortex bilaterally (Huth et al. 2012; Huth et al. 2016; de Heer et al. 2017). Another set of evidence that call for reinterpretation of the classical phrenological approach to brain function is the observation of structural and functional plasticity. The brain is not static in its architecture, but subject to constant development, reorganization and change as a function of experience, learning or trauma (Kolb and Whishaw 1998). On the structural side, a study found that London taxi drivers had a relative enlarged hippocampus, possibly developed over time in response to professional demands for spatial memory (Maguire et al. 2000), while other studies suggest that even as little as three month of juggling training affects the structural organization of the brain (Draganski et al. 2004). Plasticity is even more pronounced when it comes to functional processes. Experts in a particular field will thus display different patterns of brain activity than novices, suggesting that the brain rewires as it builds competences with a particular task or field of knowledge. For instance, it is shown that while untrained participants respond to rhythmical deviations in right hemisphere brain regions, the analogous responses are left-lateralized in expert jazz-musicians, indicating that they have evolved a different processing strategy for these kinds of signals (Vuust et al. 2005). Other observations of brain plasticity follow brain trauma such as stroke. Studies suggest that functions that were once dependent on brain areas affected by the stroke can often be re-established relying on very different areas (Murphy and Corbett 2009). That is, in these cases the same  behavioural component is subserved by different brain areas before and after the stroke. Compelling evidence of plasticity is also observed in the context of sensory substitution. In a seminal study by Bach-Y-Rita and colleagues (Bach-y-Rita et al. 1969), technologies were invented that would converge a video stream into tactile stimulations to the back of the participant, allowing blind individuals to ‘see’ objects in their environment. Later advances have allowed researchers to present stimulations to the tongue of the participant which has a higher resolution of sensitivity (Sampaio, Maris, and Bach-y-Rita 2001), and a series of studies have looked into the neural representation of the stimulations (Kupers

182

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

and Ptito 2004; Ptito et al. 2005). Interestingly, after relatively short episodes of training, these tongue stimulations were found to elicit patterns of neural activation in occipital areas of the blind subjects, suggesting that the stimulations were treated ‘as vision’ by the brain. Other examples of sensory substitution include studies of auditory-to-visual (e.g. Arno et al. 1999) and tactile-vestibular substitution (Tyler, Danilov, and Bach-yRita 2003). The studies of sensory substitution have potential far-reaching implications for discussions of the representation of meaning in the brain and have sparked deep philosophical discussions (Bach-y-Rita and Kercel 2003): are blind individuals actually seeing when using the sensory substitution technologies (Morgan 1977)? Or would they be experiencing a kind of controlled synaesthesia (Hurley and Noë 2003)? An alternative perspective is presented by Elisabeth Pacherie who suggests a separation of sensation and perception, that is, that we can have visual perception disconnected from visual sensation (Pacherie 1997).

MEANING IN PERCEPTION In parallel to the discussion on localization and distribution of neuro-cognitive processes, run discussions about the particular temporal unfolding of these processes. In particular, approaches differ in the extent to which they believe our experiences of the world to mainly rely on bottom-up or top-down processes (Noesselt, Shah, and Jancke 2003; Connor, Egeth, and Yantis 2004; Kintsch 2005; Dijkstra et al. 2017). In this context, bottom-up processes refer to the way percepts are incrementally constructed from the external stimulation of the senses. A bottom-up model of perception is thus one that assumes our experience of the surrounding world to be predominantly shaped by the incoming stimulus, a view held by many classic models of perception (e.g. Marr 1980). In contrast, a top-down model of perception is one that suggests our experiences to be construed also (or even predominantly) by expectations, beliefs and prior experiences of the world, which again might be shaped by situational, social and/or cultural factors (Frith and Dolan 1997). In other words, our conscious experience of the physical surroundings does not always perfectly coincide with the incoming stimulus. Rather, we sometimes experience what we expect to be there (cf. e.g. the phenomenon of ‘change blindness’, Clark 2014; Spratling 2016). While most neuro-cognitive approaches to perception endorse some kind of contribution of both bottom-up and top-down components of perception, they differ in the relative weight or prominence given to these processes. In the following, the visual system will serve to illustrate the basic principles of bottomup perceptual processing, however, similar organization is found for other senses (Gazzaniga 2009). From a bottom-up perspective, visual perception concerns the processes involved from the moment light reflected by the surrounding environment hits the retina, to the conscious experience of a complex scene. Perceptual analysis starts already in the eye itself, where photoreceptor cells (so-called rods and cones) differentially activate for certain light contrasts and wavelengths relevant for, for instance, edge detection and colour vision. From here the stimulus is projected via the optic nerve and the basal ganglia to the occipital cortex where it undergoes incremental structural analysis as it progresses through visual areas V1–V5, starting with detection and orientation of edges and contours, to depth perception and grouping (following gestalt principles), and further on to colour and motion detection (Zeki et al. 1991; Moutoussis and Zeki 1997). A number of studies have explored the role of early visual processing in relation to meaning-making in visual art (Ramachandran and Hirstein 1999; Zeki 2000; Bundgaard

SEMIOTICS IN NEUROSCIENCE AND COGNITION

183

2015; Boccia et al. 2016). Studies find that artworks will often exploit foundational organizing principles of visual perception to elicit aesthetic effects. For instance, low-level gestalt principles of ‘best continuation’ or ‘proximity’ and ‘grouping’ of elements can be used to support a higher-order conceptual connection between elements (Bundgaard 2009). Along similar veins, studies have explored neurocognitive roots of the evolution of early symbolic behaviour (e.g. practices of pattern engraving and decoration) with reference to similar processes of visual perception (Hodgson 2014, 2006, 2019; Tylén et al. 2020). Along the visual pathway the signal is split into the ventral stream (or, in popular terms, the ‘what’-stream), and the dorsal stream (or ‘where’/‘how’-stream) (Ungerleider and Mishkin 1982; Goodale and Milner 1992; Milner and Goodale 2008). The ventral stream goes to the fusiform gyrus and temporal structures associated with memory, and is usually associated with object recognition. It incorporates, for instance, the fusiform face area involved in face recognition, the visual word-form area thought to be involved in the recognition of letters, and the parahippocampal place area thought to play a role in the recognition of places, buildings and scenes (Grill-Spector 2003; Grill-Spector and Sayres 2008). In contrast, the dorsal stream goes to the parietal cortex and towards motor areas, and is associated with the location of objects, in particular in relation to action planning and tool use (Culham et al. 2003; James et al. 2003). While the dorsal stream is fast and largely preconscious, the ventral stream is slower and conscious, which has the implication that we can sometimes experience reflexive reactions to approaching objects even before we consciously recognize what they are. Or, as put by Frederik Stjernfelt: The overall picture is that the dorsal stream directs attention to locations and subsequently action affordances and objects in the visual field, which are, in turn, analysed and categorized by the ventral stream. The dorsal stream seems to provide a precise online analysis of egocentric visual action space, less subject to visual illusions than the more semantically heavy ventral stream. (2014b: 63) The establishment of the ventral-dorsal split has informed foundational discussions about the very nature of human categorization and meaning-making. Classical constructivist approaches to perception assume functional and value related aspects of objects (e.g. whether an entity is approachable or dangerous) to be features of the category/concept (Gregory 1970). Consequently, object recognition is thought to topdown guide the perception of an object’s value and function. However, numerous experimental and lesion studies suggest that we perceive functional features ‘directly’ and potentially independently of object recognition, supporting ideas originally formulated by James J. Gibson and ecological psychology (Greeno 1994; Gibson 1977). In other words, the dorsal pathway might be associated with the perception of affordances, that is, the way an object invites or allow for (inter)action (e.g. whether an object is graspable, Arbib 2010; Sakreida et al. 2016; Rocca et al. 2020). The ventral-dorsal split has also been discussed in relation to propositional semantic models of meaning. In an influential paper on animal cognition, James Hurford suggests that the basic logical structure of propositions is naturalized with the pre-attentional dorsal-stream deictically pointing to the argument of a logical proposition, while the ventral recognition systems connect it to its category or predicate (Hurford 2003). When a vervet monkey thus makes an alarm call, it communicates a deictic component,

184

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

FIGURE 8.4  Illustration of the cortical areas involved in early visual perception. From the eye, the signal is projected to the occipital lobe V1 and V2 (visual area 1 and 2) for initial analysis of contours and edges, V3 for integration in gestalts, V4 for colour analysis and V5 for motion analysis. The two blue arrows depict the main visual pathways. From the primary visual areas, the Ventral Pathway goes to the inferior temporal areas for object recognition/categorization. This pathway thus involves, among others, the Fusiform Face Area (face recognition), the Visual Word Form Area (recognition of letters) and the Para-hippocampal Place Area (recognition of places). The Dorsal Pathway proceeds to parietal areas involved in egocentric location, e.g. involved in action planning and tool use.

that some potential danger is detected in the periphery of the monkey, and a predicate component expressing if it is a snake, leopard or eagle. The ideas have subsequently been are criticized and elaborated with reference to Peirce’s doctrine of the Dicisign (Stjernfelt 2014a, 2014b). The previous section has mostly approached visual perception from a bottom-up perspective, appreciating how a percept is incrementally build up and analysed in terms of its constituent features to eventually enter conscious experience as meaningful categories, events or scenes. However, another important component to perception is the top-down processes that guide and contribute to the construal of experience. In the context of neuro-cognition these are often termed forward models of cognition (Friston 2012; Ridderinkhof 2014). There is little agreement on the extent to which our experiences are sole products of bottom-up stimulation of the senses, or what we experience is a predictive model of the world, with sensations only serving to update this model when it fails. However, there is neurocognitive evidence of predictive processes all the way down to very early, low-level and preconscious processes of perception (e.g. Guo et al. 2007). Relying on the method of EEG, an ERP component called mismatch negativity (henceforth MMN) is a subtle voltage fluctuation appearing approximately 150–250 ms after a stimulus event when the cognitive system finds it to be a deviant from the expected pattern (it is related to the later N400 component mentioned in an earlier section of the chapter, Näätänen et al. 2007; Näätänen 1995). It is thus observed in contexts where a sequence of, for instance, auditory notes are played back to the experimental participant

SEMIOTICS IN NEUROSCIENCE AND COGNITION

185

and infrequently a note is slightly deviating in pitch, intensity or meter. An MMN signal is indicating that the brain picked up the deviation. Interestingly, the MMN is a preconscious and pre-attentive signal and can be recorded from any of our sensory cortexes even if the experimental participant is preoccupied with something else (e.g. watching a video) and not able to consciously report the presence of the deviant. One of the prevalent interpretations of the MMN is that the cognitive system is sensitive to any kind of pattern in sensation, which are used to continuously build up ‘expectations’ in the form of predictive models of the world (Garrido et al. 2009). Our conscious experience is an expression of such models (more than the actual incoming stimulation which will often be varied and noisy). The MMN is an indication that the cognitive system’s current model failed to predict part of the incoming signal (= prediction error), and might have to be updated (Wacongne, Changeux, and Dehaene 2012). Recently, a particular forward-modelling conceptual framework has gained attention, referred to as predictive coding, which forms part of an attempt to sketch a unified theory of brain function, the Free Energy Principle (Friston 2005; Friston and Kiebel 2009; Friston 2010). Based on the principles of Bayesian statistical inference, the predicting coding model assumes the brain to be a hierarchical dynamic system of predictive processes cascading from low-level perception (e.g. MMNs and the slightly later N400 signal) to higher level inference (Clark 2013). The brain is continuously presenting hypotheses about what the world will be like and incoming sensations are compared to the hypothetical model. In cases of incongruencies, prediction error signals are propagated upward in the system depending on the cognitive systems ability to ‘explain away’ the error (Friston and Kiebel 2009). This will often engage the motor action system through attentional processes, since prediction errors are likely to make us direct attention to the potential causes of the error (e.g. saccading to something that is ‘surprising’ given our model expectations) or even to actively organize our world to minimize surprise. In this sense, the model is suggesting tight links between perception, action and learning (cf. the concept of ‘active inference’, Friston et al. 2011b; Friston et al. 2015). Several studies of semiotic practices have recently endorsed this explanatory framework for meaning-making, brain and cognition. In particular, it is suggested that aesthetic effects in music rest on tensions between anticipation and surprise. Music often presents predictable regularities in rhythmic, harmonic and melodic structure, but will also feature variations and ‘events’ that deviate from these regularities in more or less systematic ways. In other words, a lot of music is composed to set up and break expectations at both the low and higher levels of auditory perception (Vuust and Frith 2008; Vuust and Kringelbach 2008, Koelsch, Vuust, and Friston 2019). For instance, in a group of studies, researchers presented participants with varying levels of training in music with beats featuring different degrees of incongruities or syncopations – some which were ‘meaningful’ in the context and others which would be experienced as ‘mistakes’ – and showed how ERP signals differed systematically according to their manipulations (Vuust and Roepstorff 2008; Vuust et al. 2009; Vuust and Witek 2014; Vuust et al. 2018). Similar findings are reported for harmonic patterns (Leino et al. 2007).

MEANING IN COMMUNICATION From a semiotic point of view, it is relevant to make a distinction between (i) the way we cognitively construe the phenomenal world from direct perception of the referents, and (ii) from communicative signs pointing to these referents. In the latter case, what

186

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

we perceive is thus not the referent itself, but auditory or written utterances, gestures, pictures or other expressive behaviours and artefacts that are engaged to intentionally share attention to some co-present, displaced or imaginary referent scene (Brandt 2011). We have in previous sections already touched upon aspects of language processing in the brain, and, in particular, the observation that the meanings of words and sentences seem to activate brain areas also involved in non-linguistic processing of the corresponding phenomena, following the predictions of embodiment and simulation theory (Barsalou 1999, 2008; Barsalou et al. 2003; Simmons et al. 2007). However, importantly, meaning in communicative interactions comes with an element of additional social constitution. When we recognize a communicative sign (e.g. a spoken word or a gesture) to mean ‘dog’, we are not only engaging a representation of this category in our mind, we are also comprehending in which respect the concept is brought to our attention by the speaker: the speaker might wish to be informative, to warn us, to share attention to the dog’s cuteness, might intent it as an insult (‘you are a dog’), etc. In other words, the meaning of a communicative sign is inseparable from its pragmatic intention (Sperber and Wilson 1986; Clark 1996; Tomasello et al. 2005; Tomasello 2008). And understanding these intentions is suggested to engage particular brain areas putatively summarized under the heading of the social brain (Frith 2007). The social brain can be portrayed as an attempt at naturalizing cognitive mechanisms involved in our understanding of other people including their actions/behaviour, beliefs, intentions, emotions, perspectives, etc. It thus imports concepts from philosophy of mind such as theory of mind (ToM), simulation and mirroring and associates them with neurocognitive activation patterns in response to social stimuli and tasks. Through a large number of brain imaging experiments, researchers have thus mapped a network of brain structures that seem involved whenever the experimental participant is presented with tasks that require them to make inferences about the mental states of others, understand their actions, emphasize with their emotions or interact with them (Rilling et al. 2002; Iacoboni et al. 2005; Frith and Frith 2006a, 2006b). One such pattern consists of areas in the medial prefrontal cortex (mPFC) and the temporo-parietal junction (TPJ) that seem consistently associated with tasks in which participants are instructed to make inferences about the mental states of others (i.e. mentalize/ToM) based on social stimuli. For instance, in an fMRI study by Helen Gallagher and colleagues (2000), participants were shown cartoons that either presented a scene involving physical causality or an element of mentalizing. By contrasting brain activations two the different cartoon conditions, the researchers were able to show that mentalizing cartoons selectively engaged areas of the medial prefrontal cortex (Gallagher et al. 2000). Another set of findings concerns a network of areas activated when participants observe the actions of others and are thus considered to mediate action understanding (i.e. simulation, imitation and/or mirroring). These are typically found in the premotor areas of the inferior frontal gyrus and the inferior parietal lobes (Gallese and Goldman 1998; Iacoboni 2005; Iacoboni et al. 2005; Rizzolatti 2005; Newman-Norlund et al. 2007; Newman-Norlund et al. 2008), and are often presented as the human analogue of the mirror neuron system originally found in macaque monkeys (Rizzolatti and Craighero 2004; Rizzolatti 2005). A third network seem to be engaged when participants emphasize with the emotions of someone else, for instance, in contexts of pain, disgust or fear, and include areas such as the anterior cingulate, the insula and the amygdala (Wicker et al. 2003). For instance,

SEMIOTICS IN NEUROSCIENCE AND COGNITION

187

participants in an fMRI scanner had activation patterns in brain areas associated with the feeling of pain when observing or imaging their loved one getting exposed to pain (Singer et al. 2004; Cheng et al. 2010). Last, the three former networks can all be argued to be primarily engaged in social observation, that is, making sense of other individuals from an observational point of view (Tylén et al. 2012). A set of studies have thus made attempts at investigating the brain areas involved when individuals interact with each other in real time. These studies are typically carried out by having one experimental participant lying in a brain scanner while performing a collaborative task with an individual in the scanner control room (e.g. Redcay et al. 2010; Newman-Norlund et al. 2008; Noordzij et al. 2009). While results are generally more mixed, several studies have found the posterior-superior temporal sulcus (pSTS) – also associated with the perception of biological motion – tend to be involved, while interestingly areas related to ToM and mirroring are less often reported (Tylén et al. 2012). Interesting attempts have also been made at studying intercorrelations of brains in contexts of social interaction using so-called hyperscanning techniques, that is, situations where two individuals are simultaneously scanned while they interact (Montague et al. 2002; Astolfi et al. 2010; Dumas et al. 2011; Cui, Bryant, and Reiss 2012; Liu et al. 2016; Miller et al. 2019). For example, in a study by Guillaume Dumas and colleagues, pairs of participants performed a hand-movement imitation task while both pair members were scanned with EEG (Dumas et al. 2010). Synchronization of movement was found to be correlated with synchronization of brain processes in the so-called alpha mu-band over central parietal areas. As noted by Barraza and colleagues (2019), hyperscanning is still in

FIGURE 8.5  Areas of the ‘Social Brain’. The mPFC (medial prefrontal cortex) and TPJ (temporo-parietal junction) are consistently found in studies of Theory of Mind/Mentalizing. The premotor area and the IPL (inferior parietal lobe) often show up in studies of mirroring and are suggested to form part of the human Mirror Neuron System. The pSTS (posterior superior temporal sulcus) is found in studies of biological motion and is often reported in studies of social interaction.

188

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

its infancy, and there are currently no common conventions for the experimental setup and analyses protocol making it hard to access if results replicate across studies. Inter-individual brain synchronization was also found in an fMRI brain imaging experiment studying speakers and listeners (Stephens, Silbert and Hasson 2010). Listeners were found to synchronize (with a short delay) to the brain activity of a speaker telling a story across a number of areas including medial prefrontal cortex, TPJ and precuneus. Interestingly, higher levels of synchrony were correlated with better understanding of the story. Relying on similar methods, two studies investigated the brain structure involved in the integration of complex narrative meaning using silent movies (Hasson et al. 2008) and auditory narratives (Lerner et al. 2011). Researchers found activation in a number of spatially contingent areas in the posterior temporal cortices hierarchically organized from low level perception – responsive to patterns on shorter time scales (e.g. the level of individual speech sounds) – to high order areas responsible for semantic integration on longer time scales (e.g. the level of discourse). These observations have resonated with conceptual ideas about human memory processes (Donald 2007). For instance, Merlin Donald suggests that humans are evolutionarily special due to their engagement in socially distributed processes of collaborative reasoning. This has created adaptive pressures on the human brain to evolve a new memory process – what Donald calls The Slow Process – that support the integration of events and information into larger complexes (e.g. narratives), even when these occur temporally/spatially dispersed or are interrupted. Some further empirical support is provided for this idea in an fMRI experiment presenting participants with coherent and scrambled stories (that refuse integration in a coherent plot structure), suggesting that cumulative plot formation is supported by a macro-scale network sometimes referred to as the brain’s Default Mode Network (Tylén et al. 2015). While in previous sections, we have mostly focused on brain areas and processes associated with linguistic meaning-making, there are also studies looking into other modalities of semiotic behaviour. For instance, a number of studies have looked at brain processes involved in our understanding of hand and bodily gesture (Rizzolatti and Arbib 1998; Arbib 2005; Ozyurek et al. 2007; Willems and Hagoort 2007; Dick et al. 2009; Schippers et al. 2009; Xu et al. 2009). Interestingly, a number of these studies report activation in Broca and/or Werniche’s areas, suggesting that these are not specialized for verbal language (as has sometimes been suggested, cf. e.g. Embick et al. 2000). In an fMRI study, researchers thus presented participants with spoken words and gestural pantomimes or emblems of corresponding meanings. While they found modality-specific patterns for verbal and gestural signs in inferior and superior temporal regions, there were overlapping activation patterns in response to both modalities in Broca and Wernicke’s areas (Xu et al. 2009). In addition, another study suggests that the perception of symbolic artefacts such as a bucket of flowers, national flags or road signs, or even everyday objects contextualized to intentionally communicate something (e.g. a chair put out into the street to reserve a parking lot) seem to engage Broca’s area, among others (Tylén, Wallentin, and Roepstorff 2009). Last, a study by Tylén and colleagues (2016) investigated how symbolic artefacts acquire meaning through their functional engagement in contexts of joint action. Groups of four-six experimental participants were asked to collaboratively build models in LEGO bricks reflecting their shared understanding of abstract concepts such as ‘responsibility’ or ‘justice’. On the following day, the individual group members were scanned with fMRI and presented with photographic representations of their models and models build by other groups. When asked to rate the physical attributes

SEMIOTICS IN NEUROSCIENCE AND COGNITION

189

of the models (how fragile/robust the model was) – that is, consider it as a bare physical object – activity was recorded in the precentral gyrus associated with haptic perception and motor control, and the fusiform gyrus associated with object recognition. In contrast, when participants were asked to relate to the intended meaning of the LEGO models (i.e. rate how well they represented the abstract target concepts), activation was found in Broca’s area suggesting that the LEGO artefacts were perceived as sign vehicles, as well as mPFC and TPJ associated with social cognition. Together, these observations suggest that while intentional semiotic behaviour might engage a large and distributed network of brain areas dependent on the particular modality of expression and aspects of semantics, areas traditionally considered ‘language-areas’ are still playing a role and thus potentially involved in the expression and interpretation of cross-modal aspects of meaning (Tylén and Allen 2009; Xu et al. 2009).

LIMITATIONS IN THE STUDY OF BRAIN AND MEANING The study of brain processes provides interesting new ways of investigating semiotic processes. Generally, there has been a lot of optimism in the field of humanities and social sciences in terms of how the study of brain function could enlighten grand questions and theories about human behaviour, consciousness and meaning-making. Many of these expectations have, however, not been fulfilled. While there is no doubt that today we know much more about the brain and its processes than we did thirty, let alone fifty, years ago, it is not clear if studies of the brain have provided critical new insights in human meaning-making. There are several reasons for this. One is methodological: standard brain-imaging techniques are roughly constrained to answer the questions where in the brain does it happen? Or – with EPRs – when does it happen? Whenever we want to inform grand, foundational discussions of human meaning, we are thus faced with the challenge to operationalize this as a question about ‘where does it take place in the brain’. Clearly this is hard. Often it has led researchers to engage in fallacies of reverse inference (Poldrack 2011), that is, attempts at making inferences about mental states from brain activation patterns. It is thus fine to experimentally contrast canonical and scrambled sentences in order to conclude that certain brain areas are selectively activated by canonical sentences. It is, however, more problematic to show movies to participants, observe activation in the insula and from these observations infer that the participant had an emotional experience. Brain areas are often found to be involved in multiple mutually unrelated functions; and, although there are interesting advances using machine learning decoding techniques, we are not in a situation where we can reliably predict experiences from brain data (Naselaris et al. 2011). For the same reasons, neuroscience has somewhat limited potential to inform grand theories of semiotics. There are also other methodological limitations. Functional brain imaging is a young science and although a lot of sound research has been conducted in the field, it is also clear that there are deep methodological issues that warrant caution. For instance, the colourful maps of brain activation patterns found in fMRI studies are products of quite complex statistical models trying to balance signal and noise in the recorded data. Due to the costs and technical complexity of these kind of studies, many of the first generation of fMRI studies were systematically underpowered (i.e. they included too few participants and trials), and relied on statistical models that did not control for potentially confounding factors to sufficient extent. This is richly illustrated in a study that found brain activation patterns in a dead salmon in response to pictures of emotional faces (Bennett, Miller, and

190

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Wolford 2009). However, even when studies are well powered and the approaches to analysis are seemingly robust, recent meta-analyses suggest that there are still problems replicating patterns across studies and individuals, which is a major challenge for the future of cognitive neuroscience (Elliott et al. 2020).

REFERENCES Abrahamsen, A. and W. Bechtel (2012), ‘History and Core Themes’, in The Cambridge Handbook of Cognitive Science, 9–28, Cambridge: Cambridge University Press. Alibali, M. W. and S. Goldin-Meadow (1993), ‘Gesture-speech Mismatch and Mechanisms of Learning: What the Hands Reveal about a Child’s State of Mind’, Cognitive Psychology, 25 (4): 468–523. Arbib, M. A. (2005), ‘From Monkey-like Action Recognition to Human Language: An Evolutionary Framework for Neurolinguistics’, Behavioral and Brain Sciences, 28 (2): 105–24; discussion 125–67. Arbib, M. A. (2008), ‘From Grasp to Language: Embodied Concepts and the Challenge of Abstraction’, Journal de Physiologie Paris, 102 (1–3): 4–20. Arbib, M. A. (2010), ‘Mirror System Activity for Action and Language Is Embedded in the Integration of Dorsal and Ventral Pathways’, Brain and language, 112 (1): 12–24. Arno, P., C. Capelle, M. C. Wanet-Defalque, M. Catalan-Ahumada, and C. Veraart (1999), ‘Auditory Coding of Visual Patterns for the Blind’, Perception, 28 (8): 1013–29. Astolfi, L., J. Toppi, F. De Vico Fallani, G. Vecchiato, S. Salinari, D. Mattia, F. Cincotti, and F. Babiloni (2010), ‘Neuroelectrical Hyperscanning Measures Simultaneous Brain Activity in Humans’, Brain topography, 23 (3): 243–56. Atã, P. and J. Queiroz (2014), ‘Icon and Abduction: Situatedness in Peircean Cognitive Semiotics’, in L. Magnani (ed.), Model-based Reasoning in Science and Technology, 301–13, Berlin: Springer. Bach-y-Rita, P. and S. W. Kercel (2003), ‘Sensory Substitution and the Human–Machine Interface’, Trends in Cognitive Sciences, 7 (12): 541–6. Bach-y-Rita, P., C. C. Collins, F. A. Saunders, B. White, and L. Scadden (1969), ‘Vision Substitution by Tactile Image Projection’, Nature, 221 (5184): 963–4. Bailey, D. L., M. N. Maisey, D. W. Townsend, and P. E. Valk, eds (2005), Positron Emission Tomography, vol. 2, London: Springer. Baillargeon, R. (2008), ‘Innate Ideas Revisited: For a Principle of Persistence in Infants’ Physical Reasoning’, Perspectives on Psychological Science, 3 (1): 2–13. Barraza, P., G. Dumas, H. Liu, G. Blanco-Gomez, M. I. van den Heuvel, M. Baart, and A. Pérez (2019), ‘Implementing EEG Hyperscanning Setups’, MethodsX, 6: 428–36. Barsalou, L. W. (1999), ‘Perceptual Symbol Systems’, Behavioral and Brain Sciences, 22 (4): 577–609, discussion 610–60. Barsalou, L. W. (2008), ‘Grounded Cognition’, Annual Review of Psychology, 59: 617–45. Barsalou, L. W., W. K. Simmons, A. K. Barbey, and C. D. Wilson (2003), ‘Grounding Conceptual Knowledge in Modality-Specific Systems’, Trends in Cognitive Sciences, 7 (2): 84–91. Bennett, C. M., M. B. Miller, and G. L. Wolford (2009), ‘Neural Correlates of Interspecies Perspective Taking in the Post-mortem Atlantic Salmon: An Argument for Multiple Comparisons Correction’, Neuroimage, 47 (Suppl 1): S125. Binder, J., R. Desai, W. Graves, and L. Conant (2009), ‘Where is the Semantic System? A Critical Review and Meta-analysis of 120 Functional Neuroimaging Studies’, Cerebral Cortex, 9 (12): 2767–96.

SEMIOTICS IN NEUROSCIENCE AND COGNITION

191

Binder, J. R. (2017), ‘Current Controversies on Wernicke’s Area and Its Role in Language’, Current Neurology and Neuroscience Reports, 17 (8): 58. Bjørndahl, J. S., R. Fusaroli, S. Østergaard, and K. Tylén (2014), ‘Thinking Together with Material Representations: Joint Epistemic Actions in Creative Problem Solving’, Cognitive Semiotics, 7 (1): 103–23. Boccia, M., S. Barbetti, L. Piccardi, C. Guariglia, F. Ferlazzo, A. M. Giannini, and D. W. Zaidel (2016), ‘Where Does Brain Neural Activation in Aesthetic Responses to Visual Art Occur? Meta-Analytic Evidence from Neuroimaging Studies’, Neuroscience & Biobehavioral Reviews, 60: 65–71. Bogen, J. E. and G. M. Bogen (1976) ‘Wernicke’s Region – Where Is It’, Annals of the New York Academy of Sciences, 280 (1): 834–43. Brandt, P. A. (2011), ‘What Is Cognitive Semiotics? A New Paradigm in the Study of Meaning’, Signata. Annales des sémiotiques/Annals of Semiotics, 2: 49–60. Bremner, J. G., A. M. Slater, and S. P. Johnson (2015), ‘Perception of Object Persistence: The Origins of Object Permanence in Infancy’, Child Development Perspectives, 9 (1): 7–13. Broca, P. (1865), ‘Sur le siège de la faculté du langage articulé (15 juin)’, Bulletins de la Société Anthropologque de Paris, 6: 377–93. Bunce, S. C, M. Izzetoglu, K. Izzetoglu, B. Onaral, and K. Pourrezaei (2006), ‘Functional NearInfrared Spectroscopy’, IEEE Engineering in Medicine and Biology Magazine, 25 (4): 54–62. Bundgaard, P. F. (2009), ‘Toward a Cognitive Semiotics of the Visual Artwork – Elements of a grammar of intuition’, Cognitive Semiotics, 5 (3): 42–65. Bundgaard, P. F. (2015), ‘Feeling, Meaning, and Intentionality – a Critique of the Neuroaesthetics of Beauty’, Phenomenology and the Cognitive Sciences, 14 (4): 781–801. Cárdenas-García, J. F. and Ireland, T. (2017), ‘Human Distributed Cognition from an Organism-in-its-environment Perspective’, Biosemiotics, 10 (2): 265–78. Carlson, N. R. (2012), Physiology of Behaviour, Boston: Pearson Education. Carter, M. and J. C. Shieh (2015), Guide to Research Techniques in Neuroscience, London: Academic Press. Cheng, Y., C. Chenyi, C. P. Lin, K. H. Chou, and J. Decety (2010), ‘Love Hurts: An fMRI Study’, Neuroimage, 51 (2): 923–9. Chomsky, N. (1975), Reflections on Language, 1st edn, New York: Pantheon Books. Chomsky, N. (1986), Knowledge of Language: Its Nature, Origin, and Use, Convergence, New York: Praeger. Chwilla, D. J., C. M. Brown, and P. Hagoort (1995), ‘The N400 as a Function of the Level of Processing’, Psychophysiology, 32 (3): 274–85. Clark, A. (2006), ‘Material Symbols’, Philosophical Psychology, 19 (3): 1–17. Clark, A. (1997), Being There: Putting Brain, Body, and World Together Again, Cambridge: MIT Press. Clark, A. (2013), ‘Whatever Next? Predictive Brains, Situated Agents, and the Future of Cognitive Science’, Behavioral and Brain Sciences, 36 (3): 181–204. Clark, A. (2014), ‘Perceiving as Predicting’, in D. Stokes, M. Matthew, and S. Biggs (eds), Perception and its Modalities, 23–43, New York: Oxford University Press. Clark, A. and D. Chalmers (1998), ‘The Extended Mind’, Analysis, 58 (1): 7–19. Clark, H. H. (1996), Using Language, Cambridge: Cambridge University Press. Cole, M. and Y. Engeström (1993), ‘A Cultural-Historical Approach to Distributed Cognition’, in G. Salomon (ed.), Distributed Cognitions: Psychological and Educational Considerations, 1–46, Cambridge: Cambridge University Press. Coltheart, M. (1999), ‘Modularity and Cognition’, Trends in Cognitive Sciences, 3 (3): 115–20.

192

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Connor, C. E., H. E. Egeth, and S. Yantis (2004), ‘Visual Attention: Bottom-up versus TopDown’, Current Biology, 14 (19): R850–2. Croft, W. (1998), ‘Linguistic Evidence and Mental Representations’, Cognitive Linguistics, 9 (2): 151–74. Croft, W. (2010), ‘The Origins of Grammaticalization in the Verbalization of Experience’, Linguistics, 48 (1): 1–48. Cui, X., D. M. Bryant, and A. L. Reiss (2012), ‘NIRS-based Hyperscanning Reveals Increased Interpersonal Coherence in Superior Frontal Cortex during Cooperation’, Neuroimage, 59 (3): 2430–7. Culham, J. C., S. L. Danckert, J. F. X. De Souza, J. S. Gati, R. S. Menon, and M. A. Goodale (2003), ‘Visually Guided Grasping Produces fMRI Activation in Dorsal but Not Ventral Stream Brain Areas’, Experimental Brain Research, 153 (2): 180–9. de Heer, W. A., A. G. Huth, T. L. Griffiths, J. L. Gallant, and F. E. Theunissen (2017), ‘The Hierarchical Cortical Organization of Human Speech Processing’, Journal of Neuroscience, 37 (27): 6539–57. De Pasquale, F., S. Della Penna, A. Z. Snyder, C. Lewis, D. Mantini, L. Marzetti, P. Belardinelli, L. Ciancetta, V. Pizzella, and G. L. Romani (2010), ‘Temporal Dynamics of Spontaneous MEG Activity in Brain Networks’, Proceedings of the National Academy of Sciences, 107 (13): 6040–5. Dick, A. S., S. Goldin-Meadow, U. Hasson, J. I. Skipper, and S. L. Small (2009), ‘Co-Speech Gestures Influence Neural Activity in Brain Regions Associated with Processing Semantic Information’, Human Brain Mapping, 30 (11): 3509–26. Dijkstra, N., P. Zeidman, S. Ondobaka, M. A. J. van Gerven, and K. Friston (2017), ‘Distinct Top-Down and Bottom-Up Brain Connectivity during Visual Perception and Imagery’, Scientific Reports, 7 (1): 1–9. Donald, M. (1991), Origins of the Modern Mind: Three Stages in the Evolution of Culture and Cognition, Cambridge: Harvard University Press. Donald, M. (2001), A Mind So Rare: The Evolution of Human Consciousness, 1st edn, New York: Norton. Donald, M. (2007), ‘The Slow Process: A Hypothetical Cognitive Adaptation for Distributed Cognitive Networks’, Journal of Physiology-Paris, 101 (4–6): 214–22. Dotov, D. G., L. Nie, and A. Chemero (2010), ‘A Demonstration of the Transition from Readyto-hand to Unready-to-hand’, PLoS One, 5 (3): e9433. Draganski, B., C. Gaser, V. Busch, G. Schuierer, U. Bogdahn, and A. May (2004), ‘Changes in Grey Matter Induced by Training’, Nature, 427 (6972): 311–12. Dumas, G., F. Lachat, J. Martinerie, J. Nadel, and N. George (2011), ‘From Social Behaviour to Brain Synchronization: Review and Perspectives in Hyperscanning’, IRBM, 32 (1): 48–53. Dumas, G., J. Nadel, R. Soussignan, J. Martinerie, and L. Garnero (2010), ‘Inter-brain Synchronization during Social Interaction’, PloS one, 5 (8): e12166. Eichenbaum, H. (2013), ‘What HM Taught Us’, Journal of Cognitive Neuroscience, 25 (1): 14–21. Elliott, M. L., A. R. Knodt, D. Ireland, M. L. Morris, R. Poulton, S. Ramrakha, M. L. Sison, T. E. Moffitt, A. Caspi, and A. R. Hariri (2020), ‘What Is the Test-Retest Reliability of Common Task-Functional MRI Measures? New Empirical Evidence and a Meta-Analysis’, Psychological Science, 31 (7): 792–806. Embick, D., A. Marantz, Y. Miyashita, W. O’Neil, and K. L. Sakai (2000), ‘A Syntactic Specialization for Broca’s Area’, Proceedings of the National Academy of Sciences, 97 (11): 6150–4.

SEMIOTICS IN NEUROSCIENCE AND COGNITION

193

Faroqi-Shah, Y. and C. K. Thompson (2003), ‘Effect of Lexical Cues on the Production of Active and Passive Sentences in Broca’s and Wernicke’s aphasia’, Brain and language, 85 (3): 409–26. Fauconnier, G. and M. Turner (2008), The Way We Think: Conceptual Blending and the Mind’s Hidden Complexities, New York: Basic Books. Ferrari, M. and V. Quaresima (2012), ‘A Brief Review on the History of Human Functional Near-Infrared Spectroscopy (fNIRS) Development and Fields of Application’, Neuroimage, 63 (2): 921–35. Fodor, J. (1983), The Modularity of Mind, Cambridge: MIT Press. Fodor, J. A. (1975), The Language of Thought, vol. 5, Cambridge: Harvard University Press. Franz, S. I. (1912), ‘New Phrenology’, Science, 35 (896): 321–8. Frege, G. (1948), ‘Sense and Reference’, The Philosophical Review, 57 (3): 209–30. Friston, K. (2002), ‘Beyond Phrenology: What Can Neuroimaging Tell Us about Distributed Circuitry?’, Annual Review of Neuroscience, 25: 221–50. Friston, K. (2005), ‘A Theory of Cortical Responses’, Philosophical Transactions of the Royal Society B, 360 (1456): 815–36. Friston, K. (2010), ‘The Free-Energy Principle: A Unified Brain Theory?’, Nature Reviews Neuroscience, 11 (2): 127–38. Friston, K. (2012), ‘Prediction, Perception and Agency’, International Journal of Psychophysiology, 83 (2): 248–52. Friston, K. and S. Kiebel (2009), ‘Predictive Coding under the Free-Energy Principle’, Philosophical Transactions of the Royal Society B: Biological Sciences, 364 (1521): 1211–21. Friston, K., P. Jezzard, and R. Turner (1994), ‘Analysis of functional MRI time-series’, Human Brain Mapping, 1: 153–71. Friston, K. J., A. P. Holmes, K. J. Worsley, J. B. Poline, C. D. Frith, and R. S. Frackowiak (1995), ‘Statistical Parametric Maps in Functional Imaging: A General Linear Approach’, Human Brain Mapping, 2: 189–210. Friston, K. J., B. Li, J. Daunizeau, and K. E. Stephan (2011a), ‘Network Discovery with DCM’, Neuroimage, 56 (3): 1202–21. Friston, K., J. Mattout, and J. Kilner (2011b), ‘Action Understanding and Active Inference’, Biological Cybernetics, 104 (1–2): 137–60. Friston, K., F. Rigoli, D. Ognibene, C. Mathys, T. Fitzgerald, and G. Pezzulo (2015), ‘Active Inference and Epistemic Value’, Cognitive Neuroscience, 6 (4): 187–214. Frith, C. and R. J. Dolan (1997), ‘Brain Mechanisms Associated with Top-Down Processes in Perception’, Philosophical Transactions of the Royal Society of London Series B: Biological Sciences, 352 (1358): 1221–30. Frith, C. D. (2007), ‘The Social Brain?’, Philosophical Transactions of the Royal Society B: Biological Sciences, 362 (1480): 671–8. Frith, C. D. and U. Frith (2006a), ‘How We Predict What Other People Are Going to Do’, Brain Research, 1079 (1): 36–46. Frith, C. D. and U. Frith (2006b), ‘The Neural Basis of Mentalizing’, Neuron, 50 (4): 531–4. Fusaroli, R., N. Gangopadhyay, and K. Tylén (2014), ‘The Dialogically Extended Mind: Language as Skillful Intersubjective Engagement’, Cognitive Systems Research, 29: 31–9. Fusaroli, R., J. S. Bjørndahl, A. Roepstorff, and K. Tylén (2016), ‘A Heart for Interaction: Shared Physiological Dynamics and Behavioural Coordination in a Collective Creative Construction Task’, Journal of Experimental Psychology: Human Perception and Performance, 42 (9): 1297–310.

194

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Gallagher, H. L., F. Happé, N. Brunswick, P. C. Fletcher, U. Frith, and C. D. Frith (2000), ‘Reading the Mind in Cartoons and Stories: An fMRI Study of ‘Theory of Mind’ in Verbal and Nonverbal Tasks’, Neuropsychologia, 38 (1): 11–21. Gallese, V. and A. Goldman (1998), ‘Mirror Neurons and the Simulation Theory of MindReading’, Trends in Cognitive Sciences, 2 (12): 493–501. Garrido, M. I., J. M. Kilner, K. E. Stephan, and K. J. Friston (2009), ‘The Mismatch Negativity: A Review of Underlying Mechanisms’, Clinical Neurophysiology, 120 (3): 453–63. Gazzaniga, M. S. (2009), The Cognitive Neurosciences, Cambridge: MIT press. Gibbs Jr, R. W. (2011), ‘Evaluating Conceptual Metaphor Theory’, Discourse Processes, 48 (8): 529–62. Gibson, J. (1977), ‘The Theory of Affordances’, in R. Shaw and J. Bransford (eds), Perceiving, Acting, and Knowing, vol. 1, 67–83, Hillsdale: Lawrence Erlbaum Associates. Goldin-Meadow, S., M. W. Alibali, and R. B. Church (1993), ‘Transitions in Concept Acquisition: Using the Hand to Read the Mind’, Psychological Review, 100 (2): 279–97. Goldin-Meadow, S., S., Kim, and M., Singer (1999), ‘What the Teacher’s Hands Tell the Student’s Mind about Math’, Journal of Educational Psychology, 91 (4): 720. Goodale, M. A. and A. D. Milner (1992), ‘Separate Visual Pathways for Perception and Action’, Trends in Neurosciences, 15 (1): 20–5. Goodwin, C. (2000), ‘Action and Embodiment within Human Situated Interaction’, Journal of Pragmatics, 32: 1489–522. Greeno, J. G. (1994), ‘Gibson’s Affordances’, Psychological Review, 101 (2): 336–42. Gregory, R. L. (1970), The Intelligent Eye, London: Weidenfeld & Nicolson. Grill-Spector, K. (2003), ‘The Neural Basis of Object Perception’, Current Opinion in Neurobiology, 13 (2): 159–66. Grill-Spector, K. and R. Sayres (2008), ‘Object Recognition: Insights from Advances in fMRI Methods’, Current Directions in Psychological Science, 17 (2): 73–9. Guo, K., R. G. Robertson, M. Pulgarin, A. Nevado, S. Panzeri, A. Thiele, and M. P. Young (2007), ‘Spatio-temporal Prediction and Inference by V1 Neurons’, European Journal of Neuroscience, 26 (4): 1045–54. Hagoort, P. (2013), ‘MUC (Memory, Unification, Control) and Beyond’, Frontiers in Psychology, 4: 416. Hamalainen, M., R. Hari, R. J. Ilmoniemi, J. Knuutila, and O. V. Lounasmaa (1993), ‘Magnetoencephalography – Theory, Instrumentation, and Applications to Noninvasive Studies of the Working Human Brain’, Reviews of Modern Physics, 65 (2): 413–97. Hasson, U., E. Yang, I. Vallines, D. J. Heeger, and N. Rubin (2008), ‘A Hierarchy of Temporal Receptive Windows in Human Cortex’, Journal of Neuroscience, 28 (10): 2539–50. Hasson, U., A. A. Ghazanfar, B. Galantucci, S. Garrod, and C. Keysers (2012), ‘Brain-to-Brain Coupling: A Mechanism for Creating and Sharing a Social World’, Trends in Cognitive Sciences, 16 (2): 114–21. Hauk, O., I. Johnsrude, and F., Pulvermuller (2004), ‘Somatotopic Representation of Action Words in Human Motor and Premotor Cortex’, Neuron, 41 (2): 301–7. Hauser, M. D., N. Chomsky, and W. Tecumseh Fitch (2002), ‘The Faculty of Language: What Is It, Who Has It, and How Did It Evolve?’, Science, 298 (5598): 1569–79. Hodgson, D. (2006), ‘Understanding the Origins of Paleoart: The Neurovisual Resonance Theory and Brain Functioning’, Paleoanthropology, 2006: 54–67. Hodgson, D. (2014), ‘Decoding the Blombos Engravings, Shell Beads and Diepklo of Ostrich Eggshell Patterns’, Cambridge Archaeological Journal, 24 (1): 57–69.

SEMIOTICS IN NEUROSCIENCE AND COGNITION

195

Hodgson, D. (2019), ‘The Origin, Significance, and Development of the Earliest Geometric Patterns in the Archaeological Record’, Journal of Archaeological Science: Reports, 24: 588–92. Hofstadter, D. R. (2001), ‘Analogy as the Core of Cognition’, in D. Gentner, K. J. Holyoak, and B. N. Kokinov (eds), The Analogical Mind: Perspectives from Cognitive Science, 499–538, Cambridge: MIT Press. Holyoak, K. J., D. Gentner, and B. N. Kokinov (2001), ‘Introduction: The Place of Analogy in Cognition’, in D. Gentner, K. J. Holyoak, and B. N. Kokinov (eds), The analogical mind: Perspectives from cognitive science, 1–19, Cambridge: MIT Press. Hurford, J. R. (2003), ‘The Neural Basis of Predicate-Argument Structure’, Behavioral and Brain Sciences, 26 (3): 261. Hurley, S. and A. Noë (2003), ‘Neural Plasticity and Consciousness’, Biology and Philosophy, 18 (1): 131–68. Hutchins, E. (1991), ‘The Social Organization of Distributed Cognition’, in L. Resnick, S. D. Teasley, and J. M. Levine (eds), Perspectives on Socially Shared Cognition, 283–307, Washington, DC: American Psychological Association. Hutchins, E. (1995a), Cognition in the Wild, Cambridge: MIT Press. Hutchins, E. (1995b), ‘How a Cockpit Remembers Its Speeds’, Cognitive Science, 19 (3): 265–88. Hutchins, E. (2011), ‘Enculturating the Supersized Mind’, Philosophical Studies, 152 (3): 437–46. Huth, A. G., W., de Heer, T. L. Griffiths, F. E. Theunissen, and J. L., Gallant (2016), ‘Natural Speech Reveals the Semantic Maps that Tile Human Cerebral Cortex’, Nature, 532 (7600): 453–8. Huth, A. G., S. Nishimoto, A. T. Vu, and J. L. Gallant (2012), ‘A Continuous Semantic Space Describes the Representation of Thousands of Object and Action Categories across the Human Brain’, Neuron, 76 (6): 1210–24. Iacoboni, M. (2005), ‘Neural Mechanisms of Imitation’, Current Opinion in Neurobiology, 15 (6): 632–7. Iacoboni, M., I. Molnar-Szakacs, V. Gallese, G. Buccino, J. C. Mazziotta, and G. Rizzolatti (2005), ‘Grasping the Intentions of Others with One’s Own Mirror Neuron System’, PLoS. Biol, 3 (3): e79. Jack, A. and A. Roepstorff (2002), ‘Introspection and Cognitive Brain Mapping: From Stimulus-Response to Script-Report’, Trends in Cognitive Sciences, 6 (8): 333–9. James, T. W., J. Culham, G. K. Humphrey, A. D. Milner, and M. A. Goodale (2003), ‘Ventral Occipital Lesions Impair Object Recognition but Not Object-Directed Grasping: An fMRI Study’, Brain, 126 (11): 2463–75. Johnson, M. (1989), ‘Image-Schematic Basis of Meaning’, Recherches Semiotique, Semiotic Inquiry, 9: 109–18. Kelso, J. A. S. (1995), Dynamic Patterns: The Self-organization of Brain and Behaviour, Cambridge: MIT Press. Kelso, J. A. S. (2012), ‘Multistability and Metastability: Understanding Dynamic Coordination in the Brain’, Philosophical Transactions of the Royal Society B: Biological Sciences, 367 (1591): 906–18. Kintsch, W. (2005), ‘An Overview of Top-Down and Bottom-Up Effects in Comprehension: The CI Perspective’, Discourse Processes, 39 (2–3): 125–8. Kirsh, D. and P. Maglio (1994), ‘On Distinguishing Epistemic from Pragmatic Action’, Cognitive Science, 18 (4): 513–49.

196

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Koelsch, S., P. Vuust, and K. Friston (2019), ‘Predictive Processes and the Peculiar Case of Music’, Trends in Cognitive Sciences, 23 (1): 63–77. Kolb, B. and I. Q. Whishaw (1998), ‘Brain Plasticity and Behaviour’, Annual Review of Psychology, 49 (1): 43–64. Konvalinka, I., D. Xygalatas, J. Bulbulia, U. Schjødt, E. M. Jegindø, S. Wallot, G. Van Orden, and A. Roepstorff (2011), ‘Synchronized Arousal between Performers and Related Spectators in a Fire-Walking Ritual’, Proceedings of the National Academy of Sciences, 108 (20): 8514–19. Kupers, R. and M. Ptito (2004), ‘“Seeing” through the Tongue: Cross-Modal Plasticity in the Congenitally Blind’, International Congress Series, 1270: 79–84. Kutas, M. and S. A. Hillyard (1980), ‘Reading Senseless Sentences: Brain Potentials Reflect Semantic Incongruity’, Science, 207 (4427): 203–5. Lakoff, G. and M. Johnson (1980), Metaphors We Live By, Chicago: Chicago University Press. Lakoff, G. and M. Johnson (1999), Philosophy in the Flesh, New York: Basic Books. Landau, B. (2000), ‘Concepts, the Lexicon and Acquisition: Fodor’s New Challenge’, Mind & Language, 15 (2–3): 319–26. Latour, B. (1996), ‘On Interobjectivity’, Mind, Culture, and Activity, 3 (4): 228–45. Leino, S., E. Brattico, M. Tervaniemi, and P. Vuust (2007), ‘Representation of Harmony Rules in the Human Brain: Further Evidence from Event-related Potentials’, Brain Research, 1142: 169–77. Lerner, Y., C. J. Honey, L. J. Silbert, and U. Hasson (2011), ‘Topographic Mapping of a Hierarchy of Temporal Receptive Windows Using a Narrated Story’, Journal of Neuroscience, 31 (8): 2906–15. Levinas, E. (1987), ‘Meaning and Sense’, in A. Lingis (trans.), Collected Philosophical Papers, 75–107, Netherlands: Springer. Liu, N., C. Mok, E. E. Witt, A. H. Pradhan, J. E. Chen, and A. L. Reiss (2016), ‘NIRS-based Hyperscanning Reveals Inter-Brain Neural Synchronization during Cooperative Jenga Game with Face-To-Face Communication’, Frontiers in Human Neuroscience, 10: 82. Luck, S. J. (2005), ‘Ten Simple Rules for Designing ERP Experiments’, in T. C. Handy (ed.), Event-related Potentials: A Methods Handbook, 17–32, Cambridge: MIT Press. Maguire, E. A., D. G. Gadian, I. S. Johnsrude, C. D. Good, J. Ashburner, R. S. J. Frackowiak, and C. D. Frith (2000), ‘Navigation-related Structural Change in the Hippocampi of Taxi Drivers’, Proceedings of the National Academy of Sciences, 97 (8): 4398–403. Majid, A., M. Bowerman, S. Kita, D. B. M. Haun, and S. C. Levinson (2004), ‘Can Language Restructure Cognition? The Case for Space’, Trends in Cognitive Sciences, 8 (3): 108–14. Majid, A., M. Bowerman, M. Van Staden, and J. S. Boster (2007), ‘The Semantic Categories of Cutting and Breaking Events: A Crosslinguistic Perspective’, Cognitive Linguistics, 18 (2): 133–52. Malt, B. C. (2019), ‘Words, Thoughts, and Brains’, Cognitive Neuropsychology, 37 (5–6): 241–53. Malt, B. C., S. Gennari, M. Imai, E. Ameel, N. Tsuda, and A. Majid (2008), ‘Talking about Walking: Biomechanics and the Language of Locomotion’, Psychological Science, 19 (3): 232–40. Marr, D. (1980), ‘Visual Information Processing: The Structure and Creation of Visual Representations’, Philosophical Transactions of the Royal Society of London B: Biological Sciences, 290 (1038): 199–218. Miller, J. G., P. Vrtička, X. Cui, S. Shrestha, S. M. Hadi Hosseini, J. M. Baker, and A. L. Reiss (2019), ‘Inter-Brain Synchrony in Mother-Child Dyads during Cooperation: An fNIRS Hyperscanning Study’, Neuropsychologia, 124: 117–24.

SEMIOTICS IN NEUROSCIENCE AND COGNITION

197

Milner, A. D. and M. A. Goodale (2008), ‘Two Visual Systems Re-Viewed’, Neuropsychologia, 46 (3): 774–85. Mirza, M. B., R. A. Adams, C. Mathys, and K. J. Friston (2018), ‘Human Visual Exploration Reduces Uncertainty about the Sensed World’, PloS one, 13 (1): e0190429. Montague, P. R., G. S. Berns, J. D. Cohen, S. M. McClure, G. Pagnoni, M. Dhamala, M. C. Wiest, I. Karpov, R. D. King, and N. Apple (2002), ‘Hyperscanning: Simultaneous fMRI during Linked Social Interactions’, Neuroimage, 16 (4): 1159–64. Morgan, M. J. (1977), Molyneux’s Question: Vision, Touch and the Philosophy of Perception, Cambridge: Cambridge University Press. Moutoussis, K. and S. Zeki (1997), ‘Functional Segregation and Temporal Hierarchy of the Visual Perceptive Systems’, Proceedings of the Royal Society of London, Series B: Biological Sciences, 264 (1387): 1407–14. Murphy, T. H. and D. Corbett (2009), ‘Plasticity during Stroke Recovery: From Synapse to Behaviour’, Nature Reviews Neuroscience, 10 (12): 861–72. Naatanen, R. (1995), ‘The Mismatch Negativity: A Powerful Tool for Cognitive Neuroscience’, Ear and Hearing, 16 (1): 6–18. Näätänen, R., P. Paavilainen, T. Rinne, and K. Alho (2007), ‘The Mismatch Negativity (MMN) in Basic Research of Central Auditory Processing: A Review’, Clinical Neurophysiology, 118 (12): 2544–90. Naselaris, T., K. N. Kay, S., Nishimoto, and J. L. Gallant (2011), ‘Encoding and Decoding in fMRI’, Neuroimage, 56 (2): 400–10. Newen, A., L. De Bruin, and S. Gallagher (2018), The Oxford Handbook of 4E cognition, Oxford: Oxford University Press. Newman-Norlund, R. D., J. Bosga, R. G. Meulenbroek. and H. Bekkering (2008), ‘Anatomical Substrates of Cooperative Joint-Action in a Continuous Motor Task: Virtual Lifting and Balancing’, Neuroimage, 41 (1): 169–77. Newman-Norlund, R. D., M. L. Noordzij, R. G. Meulenbroek, and H. Bekkering (2007), ‘Exploring the Brain Basis of Joint Action: Co-Ordination of Actions, Goals and Intentions’, Social Neuroscience, 2 (1): 48–65. Niedermeyer, E. and F. H. Lopes da Silva, eds (2005), Electroencephalography: Basic Principles, Clinical Applications, and Related Fields, Philadelphia: Lippincott Williams & Wilkins. Noesselt, T., N. J. Shah, and L. Jancke (2003), ‘Top-Down and Bottom-Up Modulation of Language Related Areas – an fMRI Study’, BMC Neuroscience, 4: 13. Noordzij, M., S. Newman-Norlund, J. P. De Ruiter, P. Hagoort, S. C. Levinson, and I. Toni (2009), ‘Brain Mechanisms Underlying Human Communication’, Frontiers in Human Neuroscience, 3: 14. Ozyurek, A., R. M. Willems, S. Kita, and P. Hagoort (2007), ‘On-line Integration of Semantic Information from Speech and Gesture: Insights from Event-Related Brain Potentials’, Journal of Cognitive Neuroscience, 19 (4): 605–16. Pacherie, E. (1997), ‘Du problème de Molyneux au problème de Bach-y-Rita’, in J. Proust (ed.), Perception et intermodalité. Approches actuelles de la question de Molyneux, 255–293, Paris: Presses universitaires de France. Paolucci, C. (2011), ‘“The ‘External Mind”: Semiotics, Pragmatism, Extended Mind and Distributed Cognition’, VS Quaderni di studi semiotici, 112–113: 69–96. Pecher, D. and R. A. Zwaan (2005), Grounding Cognition: The Role of Perception and Action in Memory, Language, and Thinking, Cambridge: Cambridge University Press. Pinker, S. (1994), The Language Instinct, 1st edn, New York: W. Morrow and Co. Poldrack, R. A. (2011), ‘Inferring Mental States from Neuroimaging Data: From Reverse Inference to Large-Scale Decoding’, Neuron, 72 (5): 692–7.

198

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Posner, M. I., M. D. Rugg, and M. G. H. Coles, eds (1995), ‘Event-related Brain Potentials: An Introduction’, in Event-Related Brain Potentials and Cognition, 1–26, Oxford: Oxford University Press. Ptito, M., S. M. Moesgaard, A. Gjedde, and R. Kupers (2005), ‘Cross-modal Plasticity Revealed by Electrotactile Stimulation of the Tongue in the Congenitally Blind’, Brain, 128 (3): 606–14. Ramachandran, V. S. and W. Hirstein (1999), ‘The Science of Art: A Neurological Theory of Aesthetic Experience’, Journal of Consciousness Studies, 6 (6–7): 15–51. Redcay, E., D. Dodell-Feder, M. J. Pearrow, P. L. Mavros, M. Kleiner, J. D. E. Gabrieli, and R. Saxe (2010), ‘Live face-to-face Interaction during fMRI: A New Tool for Social Cognitive Neuroscience’, Neuroimage, 50 (4): 1639–47. Ridderinkhof, K. R. (2014), ‘Neurocognitive Mechanisms of Perception–Action Coordination: A Review and Theoretical Integration’, Neuroscience & Biobehavioral Reviews, 46: 3–29. Rilling, J. K., D. A. Gutman, T. R. Zeh, G. Pagnoni, G. S. Berns, and C. D. Kilts (2002), ‘A Neural Basis for Social Cooperation’, Neuron, 35 (2): 395–405. Rizzolatti, G. (2005), ‘The Mirror Neuron System and Its Function in Humans’, Anatomy and Embryology, 210 (5–6): 419–21. Rizzolatti, G. and M. A. Arbib (1998), ‘Language within Our Grasp’, Trends in Neuroscience, 21 (5): 188–94. Rizzolatti, G. and L. Craighero (2004), ‘The Mirror-Neuron System’, Annual Review of Neuroscience, 27: 169–92. Rocca, R., K. R. Coventry, T. Tylén, M. Staib, T. E. Lund, and M. Wallentin (2020), ‘Language beyond the Language System: Dorsal Visuospatial Pathways Support Processing of Demonstratives and Spatial Language during Naturalistic Fast fMRI’, NeuroImage, 216: 116128. Roepstorff, A. (2008), ‘Things to Think With: Words and Objects as Material Symbols’, Philosophical Transaction of the Royal Society of London B: Biological Sciences, 363 (1499): 2049–54. Ruiz, M. L., M. L. Rodríguez Sarasa, L. Sanjuán Rodríguez, J. Benito-León, E. García-Albea Ristol, and S. Arce Arce (2018), ‘Current Evidence on Transcranial Magnetic Stimulation and Its Potential Usefulness in Post-Stroke Neurorehabilitation: Opening New Doors to the Treatment of Cerebrovascular Disease’, Neurología (English Edition), 33 (7): 459–72. Sakreida, K., I. Effnert, S. Thill, M. M. Menz, D. Jirak, C. R. Eickhoff, T. Ziemke, S. B. Eickhoff, A. M., Borghi, and F. Binkofski (2016), ‘Affordance Processing in Segregated Parieto-Frontal Dorsal Stream Sub-Pathways’, Neuroscience & Biobehavioral Reviews, 69: 89–112. Sampaio, E., S. Maris, and P. Bach-y-Rita (2001), ‘Brain Plasticity: “Visual” Acuity of Blind Persons via the Tongue’, Brain Research, 908 (2): 204–7. Saussure, F. de. ([1916] 1959), Course in General Linguistics, New York: Philosophical Library. Schippers, M. B., V. Gazzola, R. Goebel, and C. Keysers (2009), ‘Playing Charades in the fMRI: Are Mirror and/or Mentalizing Areas Involved in Gestural Communication?’, PLoS ONE, 4 (8): e6801. Scoville, W. B. and B. Milner (1957), ‘Loss of Recent Memory after Bilateral Hippocampal Lesions’, Journal of Neurology, Neurosurgery, and Psychiatry, 20 (1): 11. Shtyrov, Y., O. Hauk, and F. Pulvermuller (2004), ‘Distributed Neuronal Networks for Encoding Category-Specific Semantic Information: The Mismatch Negativity to Action Words’, The European Journal of Neuroscience, 19 (4): 1083–92.

SEMIOTICS IN NEUROSCIENCE AND COGNITION

199

Simmons, W. K., V. Ramjee, M. S. Beauchamp, K. McRae, A. Martin, and L. W. Barsalou (2007), ‘A Common Neural Substrate for Perceiving and Knowing about Colour’, Neuropsychologia, 45 (12): 2802–10. Simpson, D. (2005), ‘Phrenology and the Neurosciences: Contributions of FJ Gall and JG Spurzheim’, ANZ Journal of Surgery, 75 (6): 475–82. Singer, T., B. Seymour, J. O’Doherty, H. Kaube, R. Dolan, and C. Frith (2004), ‘Empathy for Pain Involves the Affective but Not Sensory Components of Pain’, Science, 303 (5661): 1157–62. Sinha, C. (2007), ‘Cognitive Linguistics, Psychology and Cognitive Science’, in D. Geeraerts and H. Cuyckens (eds), The Oxford handbook of Cognitive Linguistics, 1266–94, Oxford: Oxford University Press. Sonesson, G. (2007), ‘From the Meaning of Embodiment to the Embodiment of Meaning: A Study in Phenomenological Semiotics’, in T. Ziemke, J. Zlatev, and R. M. Frank (eds), Body, language and mind, vol. 1 Embodiment, 85–128, Berlin & Boston: De Gruyter Mouton. Sonesson, G. (2015), ‘From Remembering to Memory by Way of Culture. A Study in Cognitive Semiotics’, Southern Journal of Semiotics, 5 (1): 25–52. Sperber, D. and D., Wilson (1986), Relevance: Communication and Cognition, vol. 142, Cambridge: Harvard University Press. Spivey, M. (2008), The Continuity of Mind, Oxford: Oxford University Press. Sporns, O. (2002), ‘Network Analysis, Complexity, and Brain Function’, Complexity, 8 (1): 56–60. Spratling, M. W. (2016), ‘Predictive Coding as a Model of Cognition’, Cognitive Processing, 17 (3): 279–305. Stephens, G. J., L. J. Silbert, and U. Hasson (2010), ‘Speaker–listener Neural Coupling Underlies Successful Communication’, Proceedings of the National Academy of Sciences, 107 (32): 14425–30. Stjernfelt, F. (2007), Diagrammatology, Netherlands: Springer Verlag. Stjernfelt, F. (2014a), Natural Propositions: The Actuality of Peirce’s Doctrine of Dicisigns, Boston, Massachusetts: Docent Press. Stjernfelt, F. (2014b), ‘Dicisigns and Cognition: The Logical Interpretation of the VentralDorsal Split in Animal Perception’, Cognitive Semiotics, 7 (1): 61–82. Stjernfelt, F. (2015), ‘Dicisigns’, Synthese, 192 (4): 1019–54. Talmy, L. (2000), Toward a Cognitive Semantics, 2 vols, Cambridge: MIT Press. Thagard, P. (1996), ‘Cognitive Science’, in E. N. Zalta (ed.), The Stanford Encyclopedia of Philosophy, Metaphysics Research Lab: Stanford University. Tomasello, M., M., Carpenter, J., Call, T., Behne, and H., Moll (2005), ‘Understanding and Sharing Intentions: The Origins of Cultural Cognition’, Behavioral and Brain Sciences, 28 (5): 675–91, discussion 691–735. Tomasello, M. (2008), Origins of Human Communication, Cambridge: MIT Press. The Wellcome Trust (2014), ‘Thousands of Years of Visual Culture Made Free through Wellcome Images’, Wellcome Trust. https://wellcome.org/press-release/thousands-yearsvisual-culture-made-free-through-wellcome-images. Tylén, K., M. Allen, B. K. Hunter, and A. Roepstorff (2012), ‘Interaction vs. Observation: Distinctive Modes of Social Cognition in Human Brain and Behavior? A Combined fMRI and Eye-tracking Study’, Frontiers in Human Neuroscience, 6: 331. Tylén, K., J. S. Bjørndahl, and E. Weed (2011), ‘Actualizing Semiotic Affordances in a Material World’, in S. J. Cowley (ed.), Distributed Language, 81–100, Amsterdam: John Benjamins Publishing Company.

200

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Tylén, K., P. Christensen, A. Roepstorff, T. Lund, S. Østergaard, and M. Donald (2015), ‘Brains Striving for Coherence: Long–term Cumulative Plot Formation in the Default Mode Network’, Neuroimage, 121: 106–14. Tylén, K., R. Fusaroli, J. S. Bjørndahl, J. Rączaszek-Leonardi, S. Østergaard, and F. Stjernfelt (2014), ‘Diagrammatic Reasoning: Abstraction, Interaction, and Insight’, Pragmatics and Cognition, 22 (2): 264–83. Tylén, K., R. Fusaroli, P. F. Bundgaard, and S. Østergaard (2013), ‘Making Sense Together: A Dynamical Account of Linguistic Meaning-Making’, Semiotica, 2013 (194): 39–62. Tylén, K. and J. McGraw (2014), ‘Materializing Mind: The Role of Objects in Cognition and Culture’, in M. Galloti and J. Michael (eds), Perspectives on Social Ontology and Social Cognition, 135–48, Dordrecht: Springer. Tylén, K., M. Wallentin, and A. Roepstorff (2009), ‘Say It with Flowers! An fMRI Study of Object Mediated Communication’, Brain and Language, 108 (3): 159–66. Tylén, K. and M. Allen (2008), ‘Interactive Sense-Making in the Brain’, in A. Cassara, F. Morganti, and G. Riva (eds), Enacting Intersubjectivity: Paving the Way for a Dialogue between Cognitive Science, Social Cognition and Neuroscience, Proceedings from the International Workshop, 224–41, Lugano, Switzerland. Tylén, K., R. Fusaroli, S. Rojo, K. Heimann, N. Fay, N. N. Johannsen, F. Riede, and M. Lombard (2020), ‘The Evolution of Early Symbolic Behavior in Homo Sapiens’, Proceedings of the National Academy of Sciences, 117 (9): 4578–84. Tylén, K., J. S. Philipsen, A. Roepstorff, and R. Fusaroli (2016), ‘Trails of Meaning Construction: Symbolic Artifacts Engage the Social Brain’, Neuroimage, 134: 105–12. Tyler, M., Y. Danilov, and P. Bach-y-Rita (2003), ‘Closing an Open-Loop Control System: Vestibular Substitution through the Tongue’, Journal of Integrative Neuroscience, 2 (2): 159–64. Ungerleider, L. G. and M. Mishkin (1982), ‘Two Cortical Visual Systems’, in D. J. Ingle, M. A. Goodale, and R. J. W. Mansfied (eds), Analysis of Visual Behavior, Cambridge: MIT Press. Viola, M. (2017), ‘Carving Mind at Brain’s Joints: The Debate on Cognitive Ontology’, Phenomenology and Mind, (12): 162–72. Vogeley, K. and A. Roepstorff (2009), ‘Contextualising Culture and Social Cognition’, Trends in Cognitive Sciences, 13 (12): 511–16. Vuust, P. and C. Frith (2008), ‘Anticipation Is the Key to Understanding Music and the Effects of Music on Emotion’, Behavioural Brain Research, 31: 599–600. Vuust, P. and M. L. Kringelbach (2008), ‘The Pleasure of Music’, in M. L. Kringelbach (ed.), Pleasures of the Brain, 255–69, Oxford: Oxford University Press. Vuust, P., K. Pallesen, C. Bailey, T. van Zuijen, A. Gjedde, A. Roepstorff, and L. Østergaard (2005), ‘To Musicians, the Message Is in the Meter Pre-Attentive Neuronal Responses to Incongruent Rhythm Are Left-Lateralized in Musicians’, NeuroImage, 24 (2): 560–4. Vuust, P. and A. Roepstorff (2008), ‘Listen Up! Polyrhythms in Brain and Music’, Cognitive Semiotics, 3: 131–59. Vuust, P., M. J. Dietz, M. Witek, and M. L. Kringelbach (2018), ‘Now You Hear It: A Predictive Coding Model for Understanding Rhythmic Incongruity’, Annals of the New York Academy of Sciences, 1423 (1): 19–29. Vuust, P., L. Østergaard, K. J. Pallesen, C. Bailey, and A. Roepstorff (2009), ‘Predictive Coding of Music–Brain Responses to Rhythmic Incongruity’, Cortex, 45 (1): 80–92. Vuust, P. and M. A. G. Witek (2014), ‘Rhythmic Complexity and Predictive Coding: A Novel Approach to Modeling Rhythm and Meter Perception in Music’, Frontiers in Psychology, 5: 1111.

SEMIOTICS IN NEUROSCIENCE AND COGNITION

201

Wacongne, C., J. P. Changeux, and S. Dehaene (2012), ‘A Neuronal Model of Predictive Coding Accounting for the Mismatch Negativity’, Journal of Neuroscience, 32 (11): 3665–78. Wallentin, M., T. Lund, S. Østergaard, L. Østergaard, and A. Roepstorff (2005), ‘Motion Verb Sentences Activate Left Posterior Middle Temporal Cortex despite Static Context’, NeuroReport, 16 (6): 649. Wallentin, M., S. Østergaard, T. E. Lund, L. Østergaard, and A. Roepstorff (2005), ‘Concrete Spatial Language: See what I Mean?’, Brain and Language, 92 (3): 221–33. Wallentin, M., A. Roepstorff, R. Glover, and N. Burgess (2006), ‘Parallel Memory Systems for Talking about Location and Age in Precuneus, Caudate and Broca’s Region’, Neuroimage, 32 (4): 1850–64. Wallentin, M., E. Weed, L. Østergaard, K. Mouridsen, and A. Roepstorff (2008), ‘Accessing the Mental space-Spatial Working Memory Processes for Language and Vision Overlap in Precuneus’, Human Brain Mapping, 29 (5): 524–32. Wallentin, M., A. H. Nielsen, P. Vuust, A. Dohn, A. Roepstorff, and T. E. Lund (2011a), ‘Amygdala and Heart Rate Variability Responses from Listening to Emotionally Intense Parts of a Story’, Neuroimage, 58 (3): 963–73. Wallentin, M., A. H. Nielsen, P. Vuust, A. Dohn, A. Roepstorff, and T. E. Lund (2011b), ‘BOLD Response to Motion Verbs in Left Posterior Middle Temporal Gyrus during Story Comprehension’, Brain and Language, 119 (3): 221–5. Wernicke, C. (1874), Der aphasiche Symptomenkomplex, Breslau: Cohen and Weigert. Wicker, B., C. Keysers, J. Plailly, J. P. Royet, V. Gallese, and G. Rizzolatti (2003), ‘Both of Us Disgusted in My insula: The Common Neural Basis of Seeing and Feeling Disgust’, Neuron, 40 (3): 655–64. Willems, R. M. and P. Hagoort (2007), ‘Neural Evidence for the Interplay Between Language, Gesture, and Action: A Review’, Brain and Language, 101 (3): 278–89. Wittgenstein, L. (1953), Philosophical Investigations, New York: Macmillan. Witzany, G. (2016), ‘The Biosemiotics of Plant Communication’, The American Journal of Semiotics, 24 (1–3): 39–56. Xu, J., P. J. Gannon, K. Emmorey, J. F. Smith, and A. R. Braun (2009), ‘Symbolic Gestures and Spoken Language Are Processed by a Common Neural System’, Proceedings of the National Academy of Sciences of the United States of America, 106 (49): 20664–9. Zeki, S., J. D. Watson, C. J. Lueck, K. J. Friston, C. Kennard, and R. S. Frackowiak (1991), ‘A Direct Demonstration of Functional Specialization in Human Visual Cortex’, The Journal of Neuroscience: The Official Journal of the Society for Neuroscience, 11 (3): 641–9. Zeki, S. (2000), Inner vision: An Exploration of Art and the Brain, Oxford: Oxford University Press. Zeki, S. and L. Marini (1998), ‘Three Cortical Stages of Colour Processing in the Human Brain’, Brain, 121 (9): 1669–85. Zlatev, J. (2009), ‘The Semiotic Hierarchy: Life, Consciousness, Signs and Language’, Cognitive Semiotics, 4: 169–200. Zlatev, J. (2012), ‘Cognitive Semiotics: An Emerging Field for the Transdisciplinary Study of Meaning’, Public Journal of Semiotics, 4 (1): 2–24.

202

CHAPTER NINE

Semiotics in Computing and Information Systems MARTIN IRVINE

INTRODUCTION: DEBLACKBOXING COMPUTER SYSTEMS AS SEMIOTIC SYSTEMS Computing and semiotics have been inextricably connected since the late seventeenth century. The intellectual history of computation is not a story about machines, but about discoveries in the structures of symbolic thought, specifically how the patterns of necessary reasoning in logic and mathematics can be formally symbolized at different levels of abstraction, and then physically ‘operationalized’ by assigning symbolic structures to physical structures. As leading historians of computing explain, The modern computer was not the inevitable outcome of technological advance. The crucial prerequisite for the useful application of technology to computing was the development of notation, or language systems, sufficiently comprehensive to satisfy both the need for representation, and the need to express and implement mechanisms for the transformation of expressions in the language. […] The real intellectual origin of the modern computer has much deeper roots in the themes of representation and of automatic methods of symbolic transformation. (Campbell-Kelly and Russ 1994: 701, 703) These special ‘language systems’ for ‘automatic methods of symbolic transformation’ (what we know as computer code + data) extend back to what Leibniz called a ‘mechanical thread’ (thinking with symbols that represent necessary patterns in logic and mathematics). We can trace this ‘thread’ from the era of Leibniz’s philosophy of symbols, his model for an arithmetical calculator, and a method for calculating with the binary (base 2) number system, including his design for the first binary calculator (c. 1700),1 through the era of Charles Babbage, George Boole and C. S. Peirce (1830s–1910s) (origins of formal logic and mechanical calculating ‘engines’) (Gabbay and Woods 2004), and on to the era of modern mathematical logic, the foundations of the modern electronic computing era, and digital information (1930s–50s) (Gabbay et al. 2014). Leibniz’s ‘thread’ appears in all physical devices designed to implement symbolic processes by assigning and delegating their representations and operations (mapped out in special symbols) to intentionally designed, corresponding components (Hilton 1963; Davis 2012; von Plato 2017).

204

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

The words compute and calculate were synonymous until recently. Both are derived from Latin words that refer to methods for counting with number symbols and doing arithmetic, and the term computer originally meant a person who did calculations with numbers and other formal symbols (Grier 2005). In fact, all designs for physical ‘computer systems’ (both earlier and modern) are extrapolations from how human computers in the nineteenth and early twentieth centuries worked out calculations with numbers, notation systems, formulas, calculating devices and reference books with ‘look-up’ tables of logarithms, trigonometry formulas and other pre-calculated values, to perform ‘computations’. Human computers are the models for the first CPUs (Central Processing Units) in digital computers: a coordinating agency for interpreting data representations as ‘inputs’, applying step-by-step, rule-governed operations (logic ‘outsourced’ to logic circuits), then ‘outputting’ (writing out) results in further sets of symbols, and repeating the process as needed. C. S. Peirce was an expert in these methods, and both his scientific work as a ‘computer’ and his theoretical work in mathematics and logic became the foundation for his semeiotic. The modern digital electronic computing era has all this history ‘built in’, and our ‘computers’ became technically possible with the unanticipated convergence of research and development in mathematics, logic, telecommunications and electrical engineering in the 1930s–40s (Ceruzzi 1983, 2003; Campbell-Kelly and Aspray 2014). This convergence was directly motivated by a core semiotic problem, one that weaves Leibniz’s ‘thread’ through many layers of complex design solutions: granted that we want to use electrical signals and components for speed and scalability in computation, how can we structure electricity and physical components to represent tokens of symbols to be computed, and then assign operations (necessary interpretations) that ‘go with’ the symbol tokens, and direct the system to transform the tokens as first represented (‘inputs’) into new tokens that represent the values or meanings as ‘computed results’ (‘outputs’), in a controlled, automatic process? The design solution for this semiotic problem is the story of modern computing, right down to all the devices, networks and media we use today. As Licklider explained: Digital computers deal essentially with discrete patterns that may represent names or pictures quite as readily as numbers, and […] numerical calculation is merely one of many things that processors of discrete patterns can do. For our purposes, it is beside the point that the main early applications of digital computers were numerical. It is more significant that textbooks now call them ‘general symbol processors’. (Licklider 1968: 274) How we get from the earlier room-size ‘number crunching’ electronic computers of the 1950s–60s to our contemporary computer systems for ‘general symbol processing’ is a story of applied semiotics: [T]he domain of computation actually comprises symbols – by which I mean things that represent other things (for example, a string of alphabetic characters) […] The act of computation is, then, symbol processing: the manipulation and transformation of symbols. Numbers are just one kind of symbol; calculating is just one kind of symbol processing. And so, the focus of automatic computation, Babbage’s original dream, is whether or how this human mental activity of symbol processing can be performed by

SEMIOTICS IN COMPUTING AND INFORMATION SYSTEMS

205

(outsourced to) machines with minimal human intervention. Computer science as the science of automatic computation is also the science of automatic symbol processing. (Dasgupta 2014: 12) We will unpack the assumptions about ‘symbol processing’ here, and complete this brief description to explain how computing now includes all digitally representable human symbol systems by filling in the details with C. S. Peirce’s semeiotic. Since the 1960s, and more extensively since the 1990s, modern semiotics and computing theory intersect across a wide interdisciplinary field that includes research programs in computer science, systems theory and engineering, design theory, philosophy, cognitive science, linguistics, logic and mathematics, as well as interdisciplinary work in the field of semiotic studies.2 In this context, an important viewpoint is emerging: everything in computing and information systems is based on underlying design principles for semiotic systems, which include structures for interactions and communications between and among semiotic agents (human and delegated agents in software). This chapter provides an orientation to this exciting and expanding field of study. The best way to make ‘computing and semiotics’ accessible for students and nonspecialists is through a unifying framework that reveals how semiotic functions are correlated with the design principles for computing systems, digital media, programming and software, and user interfaces. But because all the relevant disciplines are constituted by multiple schools of thought with varying terminology, we need a framework with a generally consistent vocabulary for making useful syntheses of concepts and semiotic principles regardless of the specialized terminology of any subdiscipline. A unifying semiotic framework can be provided by combining three interrelated views of computing that enable us to focus on universal design principles for computer systems as semiotic systems: 1. A Peircean semiotic systems view: extending and applying the key concepts in Peirce’s program of ‘Logic as Semeiotic’ (c. 1902–12) for the computing systems that his work anticipated. 2. The systems and design view: combining Peirce’s program for semeiotic with modern systems and design theory, as understood in all computing and information fields. 3. The cognitive-semiotic artefact view: defining the implemented designs in actual computer systems not as non-human ‘machines’ but as designed cognitive-semiotic artefacts, a view developed in cognitive science, anthropology and HCI (HumanComputer Interface Design). This combined framework provides a ‘deblackboxing’ method for discovering why and how computing systems are intentionally designed semiotic systems, even though semiotic principles are hidden from view (‘blackboxed’) in the implemented designs of computer systems as products. The conceptual metaphor ‘black box’ was originally an engineering term for any component designed to take in certain kinds of inputs (energy, signals, information, etc.) and convert them into specified outputs (e.g. a radio, a voltage transformer, a codec for converting digital into analog audio/video): the details inside the components can just stay ‘hidden’ (‘black-boxed’, ‘don’t need to know’, ‘built-in’), because only the outputs matter for the design purpose. The concept is now universally

206

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

used in systems and software design: ‘blackboxing’ is used to hide the internal complexities of a module (at one functional level) that other modules in a system ‘don’t need to know about’ for using the outputs communicated to the system (see below on systems and design theory). But in our contemporary political economy for intellectual-property-protected products, this design principle is also used intentionally to close off access to computing systems in ‘black-boxed’ manufactured devices, which are intended to maintain ‘users’ as passive consumers blocked from understanding the universal semiotic principles on which the devices depend. A semiotic systems de-blackboxing method, then, is required for exposing the implemented design principles that are everywhere presupposed and actively instantiated, but, by historical accident, have been artificially closed off from users’ understanding.3 The semiotic deblackboxing method introduced here exposes that all our interactions with computer systems are possible only by means of intentional logical ‘mappings’ between the levels of symbolic structures (with their interpretation processes) and corresponding levels in the design of computer systems (see below on homology). Briefly, the principle of mapping (correspondence relations between systems, domains or contexts) is used at many conceptual levels in mathematics and logic (e.g. functions, sets, diagrams, category theory), computer system design, software design and data design (databases, metadata schemes). Further, intra-system mappings are implemented in the physical structures of our devices (e.g. pixel coordinates mapped to graphics memory locations). For Peirce, mapping is a form of diagrammatic thinking in which relations among different levels of abstraction can be iconically represented, and also materially instantiated in designed artefacts (e.g. in actual maps and instruments).4 The semiotic foundations of computing systems can be described with both technical and conceptual accuracy, regardless of how computers and everything digital may be described in merely instrumental and operational language. I will therefore always use the term ‘computer system’, rather than ‘computer’, to remind us that we are always talking about designed semiotic systems, and not reified objects or products. The semiotic systems view also allows us to make implicit semiotic assumptions explicit, like mapping out the unconsciously operational grammar of any language. The intellectual history of mathematics, logic and computer system design is interwoven with implicit and tacitly presupposed theories of signs, symbols and symbolic processes, and the framework outlined here allows us to recover these implicit semiotic principles and make them explicit for our understanding today.5 As Peirce emphasized in many papers on disclosing the structures of logical inference in algebras and graphs, ‘it is the chief task of logic gradually to develop that which is implicit in thought and step by step to make it explicit, or, at least, to show how to do so’ (1897: MS 738.1). Peirce’s whole program of semeiotic is an application of this logical method, and we will follow a Peircean ‘step by step’ method that allows to make the implicit explicit by turning things inside-out, exposing how and why computer systems are designed semiotic systems, whether or not they are expressly recognized as such. By using this method, readers will notice that many authors who describe computer systems, user interface design, digital media and interactive software assume an underlying ‘symbolic systems’ view without presenting an explicit semiotic theory.6 Important work based on semiotic principles continues at the intersections of cognitive science, philosophy, theoretical computer science and AI,7 and all of this research and theory can be embraced and clarified in a unified Peircean semiotic systems view.

SEMIOTICS IN COMPUTING AND INFORMATION SYSTEMS

207

FRAMEWORKS FOR A SEMIOTIC SYSTEMS METHODOLOGY Peirce’s Logic as Semeiotic and semiotic systems Other chapters in this reference set provide an overview of Peirce’s semiotic theory (see esp. Vol. 1, Chps. 4 and 13), but for our context, I will focus on key concepts in Peirce’s works for describing computer and information systems as semiotic systems. Our reference model for semiotic theory and computation will be Peirce’s last version of his research program, termed ‘Logic [Considered] as Semeiotic’, which he did not live to complete (1890s–1913, and intensively during 1904–12).8 Peirce’s semeiotic, which must not be confused with post-1960s semiotics, was developed in the context of his work in mathematics, logic and scientific research (documented in Eisele 1979, 1985, and papers in NEM).9 Peirce can be considered the first ‘computer scientist’ in that most of his papers during the Logic as Semeiotic period include extensive drafts of his formal symbolic systems for representing necessary reasoning, and analyses of the possibilities for automated reasoning and ‘logical (or reasoning) machines’. He envisioned semeiotic as a logical unification program for understanding the structure of all sign systems and symbolic reasoning, which was extensible to the logical-symbolic design principles of algebraic notation, graphs and diagrams, technical instruments and artefacts, logic machines, and all devices used for logical analysis and computation. Peirce also had first-hand knowledge of the technologies and calculating machines of his era, and he designed his own scientific instruments that provided data for his computations. His contributions to Boolean logic and methods for formal symbolic notation became part of the symbolic logic tradition in the 1910s–30s,10 which in turn became the foundation for the formal symbolic ‘code’ used in the first programming languages, which is now ‘baked in’ to the code libraries used in all contemporary programming languages. Peirce’s grounding in mathematics, logic, instrument design and the logic machines of his era make his semeiotic the best extensible model for understanding the semiotic principles in the design of the computing and media systems that we use every day.

The systems and design view The key concepts in Peirce’s semeiotic can be readily combined with modern systems and design theory for developing consistent descriptions of computer and information systems as designed semiotic systems. This view requires a basic understanding of why and how digital computer systems are designed the way they are, rather than some other way.11 System design theory includes the method of levels of abstraction for composing and decomposing system functions in multiple, hierarchical, interconnected subsystems, each designed to implement functions at different levels or layers, all of which subserve the purposes of an overall architecture (the master design) of a larger complex system.12 The method of functional abstraction is essential because it allows us to design a complex multifunctional system not as a totalized whole, but by distributing functional levels to corresponding modular subsystems (like processor and memory functions, and hardware/software modules for graphics and audio/video). Each subsystem is designed to perform a function and communicate with other subsystems through interfaces (transfer gateways) in the architecture. Behind what we perceive at the user-facing levels, computer system design is a way of ‘orchestrating’ multiple unobservable levels of representation

208

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

within the system, all supporting and returning to what we do observe, interpret and interact with. An important pragmatist semiotic principle defined by Peirce underlies the unifying architecture of computer systems: the multiple levels of subsystems (also termed modules), from the most basic logic and memory components to software for representing interfaces for digital media, are designed to be ‘orchestrated’ as a telic (goal-directed, intentional, purposive) system. A computer system, by definition, must be an implementation of the purposes of semiotic agents (designers and users) directed into the whole system architecture. A computer system is made to exist only in service of the symbolic systems it is designed to instantiate, and the system is given telic direction in the way that metasymbolic programming code is designed to be interpreted as goal-directed operations in transition processes throughout the levels of the system (Gorn 1968, 1983; Horst 1996). Recognizing how and why functional levels of abstraction are universally used in computer system architecture, digital information and software enables us to establish semiotic levels of description that directly correspond with design levels of computer systems. The designed systems view thus provides a key to making all the implicit and embodied semiotic principles explicit and systematically interpretable.

The cognitive-semiotic artefact view Our framework reveals that ‘computers’ are not usefully defined as ‘machines’ at all. Combining concepts from cognitive science, philosophy of computation and anthropology, we find that computer systems are best defined as designs for cognitive-semiotic artefacts. An artefact, by definition, includes and presupposes its designers and makers. By making the principles for digital systems architecture accessible, we can reveal that a computer system is designed and implemented by and for semiotic agents.13 The cognitive artefact view also allows us to reveal how computer systems, information and networks exemplify what is now termed, in various fields, extended, distributed, delegated or off-loaded human cognition and agency.14 Because design principles for everything computational are telic (purposive), computer system designs exist only for implementing delegated processes of symbolic cognition, which must always include shared physical-perceptible representations and a provision for ongoing dialogic interpretation (which we realize in physical input/output devices, interactive interfaces and communication across networks). This view also allows us to recognize the deeper history of contemporary information systems as part of the longer continuum of technologies designed for representing, storing and transmitting symbolic systems in physical media. This framework allows us to reveal how computer systems, in designed levels of subsystems, are structurally and constitutively semiotic, and thus different in kind from anything else we call machines designed to perform other kinds of functions. Table 9.1 provides a system map for understanding computer systems levels and their corresponding implementations of semiotic levels.

Key concepts in Peirce’s semeiotic: Sign systems, symbols, technical implementations Important theoretical developments in Peirce’s papers during his Logic as Semeiotic period apply directly to the design principles of modern computing systems, and it only a historical accident that the trajectory of thought developed in Peirce’s writings was interrupted and then rediscovered, in part, in the 1930s–50s.15 Peirce’s important work

SEMIOTICS IN COMPUTING AND INFORMATION SYSTEMS

209

TABLE 9.1  Computer system levels: Overview of the semiotic system stack. Computer systems architecture: Subsystems, component modules

System functional levels: Levels of abstraction

Semiotic levels of description

The top ‘user-facing’ level of multimedia interfaces mapped to input/output devices (screens, audio outputs) in continuous refresh cycles for projecting computational results and states of interaction.

Decoding and transducing digital data into analogue (human perceptible) substrates (screens and audio outputs), as projections from software processes and symbol system tokenization mapping design (e.g. pixel patterns).

Semiotic agents interpret physical tokens of symbolic types in interface media, and direct further interpretations and representations; dialogic interactive interpretation and ongoing symbolic representations are physically instantiated with computer systems as ‘co-agents’.

GUI software/hardware mappings for interaction, conducting inputs/outputs to/ from system levels, and for dynamic updating of display and audio outputs.

Directing inputs (symbol token representations + intentions) and outputs (interpreted representations) in ongoing recursive process.

Semiotic agents direct the input/output structures in the physical subsystems for enacting dialogic interpretations, which project up to next level.

Active software and data levels. System modules orchestrate a ‘running’ software program by combining the levels of bytes in program code and bytes for data encoding (as indexed in different segments of active memory), for active operations on typed tokens.

Encoding symbol structures for data and operations (‘symbols that mean’ + ‘symbols that do’) in binary ‘machine interpretable’ byte representations in which operations transform tokens into new tokens in ongoing directed processes.

‘Source code’ program files (representing intentions in the metasymbolic code of a programming language) are translated by interpretant programs (compiler/ interpreter) into binary code for active processes to be directed by semiotic agents.

Binary information encoded and interpreted as both operations mappable to processors and as data types (data encoding for all forms of digital media) accessible in memory.

Binary representations differentiated and defined for system functions. Data types as interpretations of the contents of memory locations.

Digital (binary) information as semiotic subsystem for tokenization and typed representations. Tokenization and retokenization of digital data from/to storage and active system memory.

Physical system architecture modules: processors, memory units, storage devices, internal I/O (‘input/ output’), interfaces to upper level modules. The level of minimal binary information structures.

CPUs and GPUs for logic and programming instructions, and digital memory devices for holding long-term representations (storage), and shortterm (active RAM) data representations in physical locations indexed in the system.

Physical tokenization of symbolic structures for both data and operations. Every component at the first level implements a mapping of symbol structures and operations, the outputs of which are communicated ‘up’ the system stack.

210

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

on these topics has not yet been recovered for current research and theory. Some of the most important concepts are summarized here.16 Peirce frequently stated that his unified theory would embrace all forms of thought, reasoning and logic based on systems of ‘external signs’, including technologies designed to perform symbolic actions and relations (e.g. 1909: MS 637). He emphasized the constitutive physical-perceptible and cognitive-logical structures of all sign systems, and the necessity of a semiotic agency which activates triadic relations in ongoing generative interpretants. ‘[E]very reasoning is of the nature of a sign […] Sign will here be the general name for everything [used in reasoning], whether it be an instrument of music, a mental resolve, a voyage of discovery, or anything else that plays an essential part in the spread of intelligence’ (1907: MS 602.7–8; cf. 1904: MS 774, EP 2.326). Peirce continually refers to technologies that are based on semiotic and logical principles; for example: Reason […] only acts through signs, spoken or written or ‘scribed’ or imagined. That which has made all our wonderful engines, wireless telegraphs, telephones, phonographs, and a thousand other wonders possible, has been the differential calculus, by which scientific men are instructed how to make the experiments that will be important. What is this ‘differential calculus’? It is a system of signs invented by the great philosopher Leibniz. (c. 1911: MS 514.46–7) Peirce had thorough knowledge of the electrical signals technology of his era: he designed measuring instruments that used electromagnetic switches (as in telegraph systems), he drew the first diagram for using electrical switches to perform Boolean logic operations (1886: W5.421–3, and see Gardner 1958; Ketner and Stewart 1984), and he developed a binary system for encoding and encrypting Morse code (c. 1902: MS 1361). Peirce clearly understood how binary logic maps onto switched electrical circuits, and how a binary mathematical code could be used for electrical signals, but these applied semiotic ideas had to wait for their application in the 1930s, when rediscovered by Claude Shannon for telecommunication networks and binary data (Shannon 1938; 1948). Peirce also described how telecommunications signals create a semiotic subsystem that combines physical sign tokens, physical interpretants and human interpreters: Every thought, or cognitive representation, is of the nature of a sign. ‘Representation’ and ‘sign’ are synonyms. The whole purpose of a sign is that it shall be interpreted in another sign; and its whole purport lies in the special character which it imparts to that interpretation. When a sign determines an interpretation of itself in another sign, it produces an effect external to itself, a physical effect, though the sign producing the effect may itself be not an existent object but merely a type. […] Some signs are interpreted or reproduced by a physical force or something analogous to such a force, simply by causing an event; as sounds spoken into a telephone effect variations or the rate of alternation of an electric current along the wire, as a first interpretation, and these variations again produce new sound-vibrations by reinterpretation [ … T]he rate of alternation of an alternating current along the wire [is] a series of variations making up a sign that interprets, i.e. translates, the acoustic sign, and in its turn setting up new acoustic vibrations in the receiver, as a reinterpretation. (1904: MS 1476.4–5)

SEMIOTICS IN COMPUTING AND INFORMATION SYSTEMS

211

This statement is not only a summary of Peirce’s semiotic theory, but a clear example of how his semeiotic includes any technical design ‘that plays an essential part in the spread of intelligence’. We continue to use electrical signals for tokenizing symbolic structures in digital electronics, for which digital-analogue decoders and transducers ‘translate’ the binary symbolic sequences to and from human perceptible forms (all audio and visual media). In fact, Peirce is describing the elements of the electrical signals system that became the foundation of modern telecommunications and information theory, and the electronic representations required for digital computing that soon followed. Solving the problem for physically retokening (reproducing) electrical signals in predictable structures is what gave us Claude Shannon’s information theory (measuring and quantizing information in digital bits), and, ultimately, gave us the internet packet design principles that solved the problem of retokening digital data units across unlimited network connections. In his many discussions of the potential for automating necessary reasoning in formal symbol systems, Peirce continually emphasizes that such a system may be possible if it can be ‘self-controlled’ like human cognitive control over a logical process. Peirce explains that a symbol, as an intelligible representation given a physical existence in time, may, in its capacity as such, produce effects in the material universe [ … It] can have a history, may be affected by associations with other signs, and gradually may undergo a great change of meaning, while preserving a certain self-identity. Indeed […] connected with suitable machinery or other physical organism, being able to produce external effects by virtue of its signification, [a symbol] may by one branch of its signification act upon another branch of its signification; and there we have the first step toward self-control. (1905: MS 290.60) This description is close to our concepts for data representations and the metasymbolic levels of code interpreted in a computer program. Further, since we can consider ‘that a man is a machine with automatic controls’ it could be possible to delegate rational selfcontrol in a design for a mechanical symbolic process: ‘This operation of self-control is a process in which logical sequence is converted into mechanical sequences […]. There is a class of signs in which the logical sequence is at the same time a mechanical sequence’ (1905: MS L 390.39). These references are only a small sample from Peirce’s extensive writings on Logic as Semeiotic that reveal how the ideas for modern computing systems are based on a rediscovery of Peircean principles. In fact, as if answering Peirce’s requirement that an automated logical system must be a ‘self-controlled’, i.e. regulated as a telic (purposively directed) system, the design principles for modern computer architecture in the 1940s–50s solved the problem of internal logical control (CPUs), and interactive programming design since the 1970s solved the problem of semiotic agents controlling and directing running software as an active, ongoing, dialogic process. Peirce’s inclusive model of Logic as Semeiotic provides the semiotic structural details for describing modern digital computer systems as designed semiotic systems. Table 9.2 presents a brief summary of important terms and concepts in Peirce’s writings on semeiotic in 1902–12 that apply directly to concepts used in modern computing systems.17

212

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

TABLE 9.2  Important concepts in Peirce’s writings on Logic as Semeiotic (1902–12). Triadic structure of signs/symbols in systems of relations

Peirce extends his concepts for triadic symbol structures for theory on the properties of physical instantiation and sign actions, kinds of interpretants, the interpretant function as a generative principle, and semiotic agency.

Type/Token Relation

From 1906 on, Peirce used the terms token (physical instance) and type (general abstract pattern) for the correlation of perceptible representations to intelligible symbolic forms (replacing his earlier terms). Anything symbolic must be tokenizable and retokenizable as shared cognitive anchors.

Physical properties of signs, and how signs can be used to cause physical actions

Peirce describes how all signs/symbols require instantiation as physical-temporal-perceptible and intersubjectively available representations. Peirce’s concepts (representamen, tokens, physical indices) can be generalized as the principle for physical structured substrates, which hold perceptible patterns. Symbols thus instantiated can be used to cause physical actions in a designed system.

The dialogic principle

Peirce extends the model of meaning-development in interpretants in human dialog to the steps performed in logic as interactive interpretation with diagrams, notation systems, and technical devices. Externalized structures (graphs, diagrams, notation systems for computation) that support dynamic interpretation for reasoning are termed Quasi-minds.

Symbols and metasymbols: Formalizing necessary reasoning and potential for automation

From the 1880s on, Peirce developed concepts for formal symbols and symbols used at different levels of abstraction, which (in modern terms) are metasymbols for logical operations, abstractions, syntax and inference rules, in both algebraic and diagrammatic (graph) notations. From 1902 on, he describes how assigning the rulegoverned operations of formal symbols to physical structures could enable automation in a self-controlled system.

Boolean logic and binary (base 2) number system

Peirce drew the first diagram for performing Boolean AND OR operations with electromagnetic switches (1886), and in the 1880s–90s he became the leading American authority on Boolean symbolic logic. From 1904–12, he wrote hundreds of pages of mostly unpublished work on Boolean operations, the base 2 number system, and methods for binary computations.

HISTORIES OF DISCIPLINES Major contributions to semiotics and computing, Leibniz to multimedia There is a long intellectual history of semiotic thought before ‘semiotics’ as a post-1960s academic field (Eschbach and Trabant 1983; vols. 1–2 of Posner et al. 1997–2003), and this history is interwoven with a parallel history of computation (theory, design and implementation) before and after contemporary digital computers. Tracing the semiotic archaeology of computing with the framework presented here is important for our deblackboxing approach.

SEMIOTICS IN COMPUTING AND INFORMATION SYSTEMS

213

Table 9.3 presents a conceptual overview of major developments in this combined history of theory and technical implementations so that the underlying semiotic systems design principles can become accessible for further study.18

TABLE 9.3  Major developments in computing and semiotics: Theory and implementations, Leibniz to the Internet. 1679–1710

G. W. Leibniz, philosophy of symbols and symbolic operations; designed calculating machine, and model for a binary (base 2) calculator.19 Leibniz was a major influence on C. S. Peirce.

1820–50

Charles Babbage develops mechanical Difference Engine and designs for the Analytical Engine (1820s–40s). Develops symbolic notation system for symbolic-mechanical homologies. 1832–37: Samuel Morse designs Telegraphic Code as proto-binary ‘system of signs’ mapped to open/closed circuits in electromagnetic switches. The foundations for all future electronic communications. 1847–54: George Boole, Laws of Thought: the binary (two-value) algebra of logic.

1860s–1910s

C. S. Peirce: Develops and expands Boolean logic in algebraic and graphical systems for formal logic; develops a diagrammatic system of logic in ‘Existential Graphs’ (1880s–1912). Designs Boolean logic switches for a ‘logical machine’, and publishes an article on the design of ‘Logic Machines’ (1886–7: W 5.421–6, W 6.65–74). Develops methods for binary (base 2) computation (1904–12). Writes many drafts of papers for his unfinished program of ‘Logic as Semeiotic’ (1890s–1912), which includes semiotic concepts for technical systems.

1920s–30s

Beginnings of ‘Information Theory’ in telecommunications and electrical engineering: techniques for ‘shaping signals’, controlling electrical current and radio waves as a semiotic subsystem for transmitting representations.

1930s–40s

Alan Turing develops formal method for converting the steps in paper and pencil ‘computations’ into a discrete sequenced, automatable metasymbolic rule-governed process (1937). Claude Shannon rediscovers how to apply Boolean logic to electrical switches (1938); a design previously developed by Peirce. This design is developed further in the 1950s for clusters of ‘logic gates’ in all computing processors (CPUs). Charles Morris reinterprets Peirce’s semiotics and pragmatism; his writings become known in engineering and science communities (Morris 1938, 1946, 1964)

214

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

1940s

Electrical engineering information theory and computer design theory converge. Design principles for electronic computing system architecture first developed. 1945–48: John Von Neumann’s architecture for processor + memory units established (Von Neumann 1987). Claude Shannon establishes the mathematical theory for discretizing information in electronics based on the binary bit as a substrate for information representations (1948). Vannevar Bush, Memex (‘memory expander’): a design concept for bringing multiple symbol systems from different media into a unified, user-configurable desk ‘display’ (1945); a conceptual model for Engelbart and subsequent digital interface designs.

1950s–60s

John von Neumann, Arthur Burks, and colleagues standardize computing architecture for automating logic, and applying the binary system for data and code. Transition in computer science and engineering for reconceiving computer systems as general symbol processors for encoding any symbolic system, not merely ‘number crunchers’ (Hamming, Licklider, Engelbart, Newell and Simon). Donald MacKay, papers on Information Theory and Symbolic Systems (1952–68) (MacKay 1969). Colin Cherry, On Human Communication (1957), includes semiotic principles. Allen Newell & Herbert Simon develop ‘physical symbol systems’ theory: (1961, 1972, 1976, 2003); and Newell (1980, 1986); Simon (1993, 1996).

1960s–70s

J. C. R. Licklider, Doug Engelbart and Alan Kay redefine computing for multi-symbolic systems by developing engineering solutions for symbolic and interactive system concepts (see: Rheingold 2000; Moggridge 2007). Licklider redefines computers as a metamedium for symbolic systems, interactions, communication, and knowledge; funds Doug Engelbart’s lab (1960–77).20 Ivan Sutherland, Sketchpad (1963): proof of concept for graphical interface systems with a screen input device; display screen reconceived as two-way interface. Doug Engelbart, ‘Augmenting Human Intellect’ research program at SRI: computer systems redesigned as multi-symbol, cognitive, networked, interactive systems with integrative representational interfaces (Engelbart 1963). Invents ‘mouse’ controller, windowing system and hyperlinking documents over networked system (1960–8). Saul Gorn redescribes computer systems with pragmatist semiotics (Gorn 1967, 1968, 1983). Semiotics Societies established; Semiotics as an academic field of study formally begins.

SEMIOTICS IN COMPUTING AND INFORMATION SYSTEMS

1970s

215

1972–7: Alan Kay, Dynabook concept for computers as a metamedium. Develops Object-Oriented Programming and symbolic interface systems at Xerox/PARC (Kay 1972, 1984; Kay and Goldberg 1977). 1977–9: Kay co-develops Xerox PARC, Star/Alto PCs: first graphical interface, multimedia ‘personal’ computer systems with window layers, hypertext linking, icons and mouse pointer device. Semiotic terms – type/ token, icon, index – are adopted in GUI design at Xerox/PARC. ARPAnet – Internet (1970s–80s): Data packet protocols, implementing metadata layers and extending information theory for digital tokenization of data across networks of networks. Internet architecture and TCP/IP embodies all the requirements of a semiotic subsystem for end-to-end encoding and decoding (transmission/ reception) of all data types in the client/server architecture of the Internet. 1970s–90s: Emerging interest in computing systems for semiotic theory (early studies by Nake, Nadin and Andersen).

1980s

1984: Apple Macintosh: consumer system version of Xerox/PARC interactive graphical system concepts (without networking). ‘Personal computer’ GUI and interactive software design principles are established across operating systems, and soon become standardized for the design of all consumer and business PCs. Human Computer Interface design (HCI) becomes an interdisciplinary field (Shneiderman 1983, 1997; Norman and Draper 1986), combining computer science, design, programming, cognitive psychology, interaction theories and semiotic theory.

1990s

1991: Tim Berners-Lee develops the HTTP Web server system and HTML for hypertext linking among multiple networked documents as a protocol layer that uses the client/server architecture of the Internet. 1991: Unicode founded to standardize digital code for representations of written characters of all languages. Digital media standards widely adopted for digitization of all symbolic types (text, graphics, image, photo, video, audio). 1993–5: Graphical hypermedia interface software for the Web (Mosaic, Netscape). 1994: Internet and Web opened to private development and consumer use. Hypertext and hypermedia systems extend semiotic principles for indexical linking with the Internet client/server architecture as a subsystem. HCI now includes Internet/Web multimedia design.

2000s–

Internet and Web architectures scale and extend for all digital media and information services. Internet-connected massively distributed computing systems (Cloud) become standard architecture. AI and Machine Learning (ML) methods become viable with advances in computation and massive data sets.

216

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Research and theory on computation and information in semiotic studies: 1980s–present There are multiple schools of thought and disciplinary contexts for semiotic theory, and, thus, for ‘semiotics and computing’. The ongoing topics of research and theory in the semiotic studies community and related fields since the 1980s are summarized in Table 9.4, which presents representative work for points of entry in the field. Several scholars have led the way in applying Peircean principles to the study of computer and information systems: the studies by Frieder Nake (1997, 2002, 2008a, 2008b); Winfried Nöth (1997, 2002); Mihai Nadin (1988b, 1998, 2007, 2011); Peter Skagestad (1993, 1996, 1999); John Sowa (1984, 1991, 2000a, 2000b) and Joseph

TABLE 9.4  Research and theory on semiotics and computing, 1980s to present. Theories of computation: Semiosis and symbolic systems; Logic and computation; Computational theories of mind

Ketner and Stewart (1984) | Ketner (1988) Andersen, Holmqvist, Jensen (1993) | Skagestad (1996) Fetzer (1997, 2001) | Andersen (1997) | Nöth (1997, 2002) Andersen, Hasle, Brandt (1997) | Gudwin (1999) Nadin (1998, 2007, 2011) | Ransdell (2003) Rapaport (1999, 2012, 2018) | Sowa (2000a, 2006) Gomes, Gudwin, El-Hani, Queiroz (2007) Queiroz and Merrell (2009) | Tanaka-Ishii (2010)

Semiotic Foundations of Information Theory

MacKay (1969) | Gorn (1968, 1983) | Nadin (2011)

Interface Design and Interaction Programming

Engelbart (1988) | Shneiderman (1982)

Kockelman (2017a, b) Nadin (1988b, 1988a, 2017) | Kay (2001) Goguen (Goguen 1999; Malcolm and Goguen 1999) Goguen and Harrell (2005) | De Souza (2005) Murray (2012)

Digital Media as a Semiotic System; Software and Digital Art

Nake (1999, 2002, 2008a, 2009; Nake and Grabowski 2006) | Murray (2012) | Manovich (2013)

Knowledge Representation, Logic, and Conceptual Structures

Sowa (1984, 1991, 2000a, 2000b)

Semiotic Engineering Concepts

Liu (2000) | De Souza (2005) | Nadin (2017)

AI, Cybernetic Systems, and Semiotic Models of Data

Fetzer (Fetzer 1988, 1997, 2001)

Meunier (1989, 1998) | Holmqvist, Klein, Posner (1996)

Sowa (2000a, 2011) | Barbosa and Breitman (2017) Skagestad (1993, 1996) | Agre (1995, 1997) Jorna, Van Heusden, Posner (1993) | Ketner (2003)

SEMIOTICS IN COMPUTING AND INFORMATION SYSTEMS

217

Goguen (Goguen 1999, 2003; Malcolm and Goguen 1999; Goguen and Harrell 2005) are foundational. Other important studies appeared in the 1990s–2000s (Fetzer 1988, 1997, 2001, 2004; Andersen et al. 1997; Queiroz and Merrell 2009). The interdisciplinary scope of semiotics and computing now also includes work in anthropology and cognitive science (see Kockelman 2005, 2006, 2010, 2017a, 2017b).

METHODOLOGIES Describing semiotic levels in the design of digital computing systems With the framework outlined above and the knowledge base provided by the related disciplines, we can apply a semiotic deblackboxing method for describing how and why computer systems, software and digital media are semiotic systems, designed as systems of subsystems that serve human symbolic representation, interpretation and communication. With this method, we will also be able to discover our roles as semiotic agents, when everything computational and digital is restored to intelligibility as a semiotic system with purposiveness, agency and interpretability built in by design. Among the fundamental principles of computing and the necessary design features of digital systems, the following topics provide useful entry points for making semiotic principles explicit in a Peircean semiotic systems description.

Computation, automation and computer systems Our current PCs and computing devices incorporate nearly a hundred years of design solutions for technical implementations of semiotic systems.21 We can summarize the semiotic systems view of the design solutions to the core semiotic problem. To create an electronic computer system for automating operations (interpretive processes) on types of symbols and produce new token instances representing interpretations, we need to design a physical system that will allow us to (1) introduce physical tokens of human intentional sign systems and register them internally in structures in the system (i.e. take in inputs through an interface), (2) implement logical operations (interpretations) assigned to those symbols as defined at other levels in the system, (3) direct the system processes to generate (retokenize) tokens for internal transitions in the operations and further tokens for what the ‘input’ tokens must be transformed into as a result (‘outputs’) of the interpretations, and then (4) project the ‘output’ tokens into perceptible and interpretable physical media (i.e. interfaces). Further, having chosen the binary electronic architecture as the most efficient, we can use the same binary tokenization system to create byte representations of symbolic types (data), map the representations of human symbol systems (written characters, graphics, images, sounds) to indexed memory locations and assign to each symbolic type the pattern of interpretations (operations, relations, concept maps) that ‘go with’ the tokens instantiated in the system substrates. Models of computation for digital systems have gone through several stages of development since the 1940s, each in parallel with developments in supporting subsystems (e.g. memory units, processors, displays): (1) the first systems were algorithmic calculating machines designed ‘run’ one terminating program at a time in a system that encoded ‘symbols that mean’ and ‘symbols that do’ in corresponding binary substrates (the finite state machine ‘input-output’ model, 1940s–50s), (2) computing expanded as ‘general symbol processing’ designed to map any encodable symbol token structure and correlated patterns of interpretive ‘processing’ to binary memory units and processor

218

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

structures, and output corresponding representations to displays (screens) (1960s–70s) and (3) our current paradigm of interactive programming, dialogic graphical interfaces and networked, distributed computing in many layers of software, a macro design which subsumes earlier design stages. In our interactive paradigm, system levels are combined for dialogic processes, directed by agents who can continually modify, reinterpret and create new representations (1970s–present). Many leaders in computer science today focus the definition of computing and computer science on the implicit semiotic principles of representations, interpretive transformations and information processing, and not on computers as machines.22 ‘Representation-transformation can be a reference model of computing. An information process is a sequence of representations’ (Denning 2012: 808–89). Further, the core of computation is not simply representation, but operationalizing symbolic structures so that they cause controlled actions as operations, that is, as rule-directed interpretations of representations. Many descriptions of computation today follow the same view of signactions and symbolic processes that Peirce first developed: Computing emphasizes the transformation of information, not simply its discovery, classification, storage, and communication. Algorithms not only read information structures, they modify them […]. [T]he structures of computing are not just descriptive, they are generative. An algorithm is not just a description of a method for solving a problem, it causes a machine to solve the problem. The computing sciences are the only sciences with such a strong emphasis on information causing action. (Denning and Martell 2105: 16–17) These are the assumptions that enable computer systems to become semiotic systems. Assumed human cognitive-semiotic agency is ‘built in’ to all system levels (by assignment or delegation), and is structurally anticipated in the design principles for interactive and networked systems.

The principle of Homology: The key to the physical symbolic system There is a logical-semiotic ‘key’ for understanding why and how digital electronic computing systems can be designed to both instantiate physical symbolic representations (tokens of symbol types) and implement logical operations (perform assignable interpretations and instantiate further tokens) in a unified binary architecture. The key is in how the mathematical principle of homology (structural correspondence mapping between domains) can be used as a design principle; that is, by imposing a logical map of one-to-one correspondences between structures in symbolic systems and structures in an intentionally designed physical system. Peirce defined the homological mapping principle in his writings on mathematics and cartography (map-making): a homology (from Greek: homo: like, same + logos: structure, form, ratio, meaning) describes correspondences representing equivalences in structure or form. ‘Homological, having a structural affinity: distinguished from analogical’ (1889–91: CD 2868). The principle of one-to-one correspondence distinguishes homology from simple analogy: ‘homologous is corresponding in a system of one-toone correspondence’ (1894–5: MS 165, NEM 2.217). ‘A correspondence is a system of relationship between two sets of objects which connects all the objects of the first set each with the same number of objects of the second set’, a definition also equivalent to an injective function in mathematics. Homology is closely parallel with the concept of

SEMIOTICS IN COMPUTING AND INFORMATION SYSTEMS

219

projection in geometry and map-making: ‘[a projection] is a system by which the points on the surface of the earth […] are made to correspond one to one to the points of a map’ (1894–5: MS 94, NEM 2.286). Homologies and mapping projections are applications of the larger concept of mathematical functions: ‘The theory of projections [ … may] be said to be simply the theory of functions viewed under the strong perspective of a practical standpoint’ (1889–91: CD 4763). Digital computer systems as automated symbol processing systems would be impossible without this deeply assumed ‘practical standpoint’ for logical-symbolic mappings. A homology, used as a design principle, is a system of relations for mapping (or projecting) the structures of symbols and logic (represented in formal notation, graphs and diagrams) to intentionally corresponding physical structures, mapping one system onto or into another system (see Goguen 1999; Ambrosio 2014).23 Computer system architecture design provides the master plan for the homological mappings for each hardware and software subsystem so that the physical structures communicate back and forth, up and down, from and to, our input/output interfaces for interpretable token representations and communicating further semiotic agency into the system. This combination of internal and external physical substrates solves the core semiotic problem: how to ‘realize’ or ‘instantiate’ computations and symbolic representations in the physical affordances of the component structures (memory, processor units, user interfaces), electrical energy and time (Nisan and Schocken 2005; Denning and Martell 2015; Comer 2017; Rescorla 2017; see Table 9.5).

TABLE 9.5  Computer system homologies. Symbolic structures

Mapped to

Physical structures

Data: ‘Symbols that mean’ Structures of our main symbol systems (e.g. text, image patterns) tokenized as physical patterns of bit/byte units with data type assignments.

⇒

Tokenization of symbols in substrates in long-term storage devices, and in substrates for active short-term representation arrays of binary cells in RAM memory and in processor units.

Program code: ‘Symbols that do’ Metasymbolic symbols in programs are coded for operations and interpretive processes on/for data tokens. ‘Code’ is also tokenized in digital bit/byte units in a program file.

⇒

‘Running code’ is projected from locations in active memory to processor units with arrays of binary logic ‘gates’ that perform operations on data tokens by first ‘reading’ input data tokens, and ‘writing’ (tokenizing) results in memory.

Formal necessity Programming code is a sign system for translating necessary relations in logic and math (represented in formal symbols and metasymbols) into binary encoded algorithms and logic in software that anticipate interpretation in the architecture of a computer system for performing computations as actions.

⇒

Physical causality CPUs (and clusters of processors) translate the binary encoded representations in programs through arrays of physical logic gates into causal actions (interpretive processes) over physical (tokenized) data representations. CPU’s must also control and timesequence operation cycles for performing interpretive processes over physical time and spatial memory locations.

220

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

But viewed at the software, interface and media representation levels, we only attend to the higher levels of abstraction (the ‘user-facing’ levels) in the telic design: the observable levels of outputs and inputs are directly mapped to the lower unobservable physical levels, which are designed to ‘communicate up’ through the system. This ‘stacking’ of levels enables designers, programmers and users to take ‘the logical equivalence of hardware and software’ for granted, as Saul Gorn lucidly explained (Gorn 1968). At our observable ‘user’ levels, physical homologies are designed to disappear into pure functionality. Mapping principles are used in many contexts in computer system design, but viewed at a macro, unifying level, the homologous mapping principle is what enables us to create automated computation by translating formal (symbolic) necessity (represented in code) into physical causality (in computing components). The recognition of the formal-tophysical mapping principle, a Leibnizian ‘mechanical thread’, extends back to the formal symbolic logic systems developed by Peirce and his contemporaries in Boolean algebras and diagrammatic systems. The symbolic systems for formalizing necessary relations, developed from Peirce’s era to Turing’s in the 1930s, demonstrated that logical necessity could, in principle, be automated, provided that we can map the formal structures in a system of one-to-one correspondences for translating formal necessity into controlled physical causality (Robinson 1979; Robinson and Voronkov 2001; Rocchi 2013) (see Table 9.5). The mapping of formal metasymbolic structures (programming code) to physical architecture structures that perform actions is the sine qua non of digital electronic computation as a system of active, dynamic, interpretation processes.

Information and binary systems: Designing semiotic subsystems ‘Information theory’, as developed in electrical engineering, is an engineering solution to a semiotic problem: how can we impose a design on electrical current (and radio waves) for a system of predictable patterns that are invariant over places, times and material media, so that we can use the energy patterns to represent intentionally meaningful patterns in a communicable human sign system? Short answer: we can only efficiently impose this kind of controllable, predictable pattern on switched states of an electrical circuit: closed/open, on/off, voltage present/voltage absent. This is a binary, one-of-two-possible-states system, which maps exactly to the binary (base 2) number system and to the logical values in Boolean logic (T/F, yes/no). One unit of a switched state is a bit (binary unit); string them together in ordered patterns and we get bytes, which we can use to encode the structures of any digitized symbolic system. We implement the binary system map in matrices of miniature transistors (memory units) and chains of combined logic switches (logic ‘gates’ in processors). Digital information, then, is a design for a semiotic subsystem, a technique for tokenizing representations of symbolic structures in homologous physical substrates (mapping symbolic patterns to physical patterns).24 For an automated electronic computational system, then, only binary electronics allows us to create an exact system of one-to-one correspondences. This correspondence system allows us to (1) physically tokenize representations in formal-to-physical mappings in memory cells (bits and byte units: data) and (2) perform operations on representations by means of combinations of binary logic switches ‘hard-wired’ in millions of ‘logic gates’ in microprocessors. The combination of (1) and (2) is the definition of digital computation (see Table 9.5). Computer system design also includes a method for managing levels and types of binary bit representations. At digital bit-level representations, both ‘data’ and ‘program

SEMIOTICS IN COMPUTING AND INFORMATION SYSTEMS

221

code’ are stored as binary data in memory units, but computer systems are designed to index and type memory locations so that the tokenized bits and bytes can be referenced and internally retokenized to function at their assigned symbolic levels. Bits/bytes only ‘mean’ in a system of interpretation represented at another level, thus demonstrating the essential semiotic structure of all things digital. Unlike other material substrates in the history of sign systems (spoken language, traditional writing materials, image supports, analogue media), digital information is structurally semiotic in that the subsystem requires applying abstract symbolic thought itself to impose a logical structure on materials and energy that are meaningless in themselves (Blanchette 2011; Abbott 2019; Patt and Patel 2020). The mapping principle followed in engineering processes for implementation in electronic components allows us to instantiate bits and bytes as structure-preserving structures (replicable and transmittable patterns), creating the predictable, controllable structures required for all data tokenizing systems, from what we input through our keyboards and mouse clicks to internet packets sent to initiate a remote Cloud computing data process.

Computers, symbol processing and semiotic architecture Peirce’s semeiotic, which includes the principles for physical symbolic homologies and the logic of operations, allows us to complete and reframe the definitions of computers as ‘symbol systems’. Always aware of the necessity of physical instantiations of signs as shared cognitive anchors, Peirce also saw how sign systems, in ongoing patterns of representations and interpretations, can form semiotic ‘strata’ or levels of signs: In consequence of every sign determining an Interpretant, which is itself a sign, we have sign overlying sign. The consequence of this, in its turn, is that a sign may, in its immediate exterior, be of one of the three classes [icon, index, symbol], but may at once determine a sign of another class. But this in its turn determines a sign whose character has to be considered. This subject has to be carefully considered, and order brought into the relations of the strata of signs. (Minute Logic, MS 425 [1902]:134–5) This description is an excellent starting point for understanding the sign-system levels in the physical tokenization of structures of symbolic types and the delegated interpretants in software, which combine to make a computer system a designed semiotic system. We need only add Peirce’s extensive treatment of symbolic operations and interpretations to fill in the model for digital computer systems and the ‘strata’ of signs for digital media representations managed in the homologous maps for information, processing and interactive interface representations. A model for ‘symbol systems’ emerged the 1950s–70s, which became part of the discourse in computer science and AI. Allen Newell and Herbert Simon developed the ‘physical symbol system’ model, which combined the computational theory of mind in cognitive science with the concepts of symbols, logic and rule-governed operations in computer science (Newell and Simon 1976; Haugeland 1981b; Simon 1996). The ‘physical symbol system’ descriptions get us part way to a semiotic model, but the theory is based on an impoverished conception of signs and symbols, mostly modelled on the formal symbols of symbolic logic notation, with rules for logical operations and relations, that can be assigned in the computer architecture. The role of semiotic agency and

222

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

the function of interpretant relations in a full triadic symbol system model, in Peirce’s sense, are unaccounted for. However, the assumptions and terminology of the ‘physical symbol system’ hypothesis continue to inform arguments about symbols in theories of computation, cognition and AI (Marcus 2001; Nilsson 2007; Steels 2007; Conery 2012; Rapaport 2012). We use the binary subsystems for byte patterns of all digitizable symbolic types and methods for representation; and, at the digital token level, a computer system is designed as a dynamic system of unlimited retokenization: ‘tokens in’ and new ‘tokens out’ in the managed ‘strata of signs’. As Haugeland explains, using Peirce’s terms, in his classic study of AI: A computer is an interpreted automatic formal system. […] A digital system is a set of positive and reliable techniques (methods, devices) for producing and reidentifying tokens, or configurations of tokens, from some prespecified set of types. […] Digital techniques are write/read techniques. ‘Writing’ a token means producing one of a given specified type (possibly complex); ‘reading’ a token means determining what type it is. A ‘write/ read cycle’ is writing a token and then (at some later time) reading it; a write/read cycle is successful if the type determined by the read technique is the same as the type specified to the write technique. (Haugeland 1985: 48, 53–4) This is a useful general description of how all the unobservable symbolic homologies are designed to make what we do observe in our interface representations possible as components of a semiotic system.

Programming languages, code, running software and interfaces The design history of programming languages and all that we call code is a fascinating story of applied semiotics.25 At the beginnings of electronic computer system design and code for operations, John von Neumann (designer of the main homologous system architecture that we still use today) understood what Leibniz called ‘mechanical thread’, and he described the challenge of designing a code system for automated reasoning that mapped onto the state of components in the 1940s–50s: Our problem [for the coding of operations] is, then, to find simple, step-by-step methods […]. Since coding is not a static process of translation, but rather the technique of providing a dynamic background to control the automatic evolution of a meaning, it has to be viewed as a logical problem and one that represents a new branch of formal logics. (Goldstine and Von Neumann 1963: 83) Peirce would have fully agreed. Of course, we now have a full suite of ‘high level programming languages’ (e.g. the C family, Java, Python, JavaScript), for which teams of programmers work at high levels of abstraction above the physical systems. But even though the system homologies can be forgotten because they are built in and standardized (‘the logical equivalence of hardware and software’), coding a program ‘to control the automatic evolution of a meaning’ directed by semiotic agents continues to be the prime directive of coding software for computational telic systems.

SEMIOTICS IN COMPUTING AND INFORMATION SYSTEMS

223

Programming today begins by composing a ‘human readable’ file termed the ‘source code’, consisting of a metalanguage of formally specified terms, phrases and symbols for operations, relations and interpretations defined in the programming language. For the intentions encoded in the ‘source code’ file to ‘run’ in an actual computer system, the file must be translated into binary representations that can be directly installed (mapped into) computer memory and the structures of processors.26 The diagram in Table 9.6 provides a map of the interpretive processes that enables the ‘source code’, written in a high-level programming language, to be mapped to the digital computer system and become part of an active system with ‘users’ (semiotic agents). (The table assumes the interactive software paradigm.) Each step from source code to running code requires a delegated interpretant system, ‘meta-software’ designed to translate one encoded state into another. Of course, what we code in the symbols, logic and algorithms in a ‘source code’ program file is as equally motivated and directed by cumulative human agency as the active software that we ‘run’ and interact with for all our symbolic systems encoded as types of digital media.

Interfaces as dialogic semiotic substrates, and computers as a metamedium Material interfaces for enabling the symbolic pattern recognition (token → type relations) and dialogic interpretation processes with symbol representations have a deep cultural history, and digital interface design began by simulating our common two-dimensional representational substrates (surfaces for written symbols and images). Because all human sign systems must have physical-perceptible structures, symbolic structures come with built-in interfaces that enable inferences to systems of meanings outside physical instances. Contemporary pixel-based screens are controlled by graphics processors designed for rendering physical token structures (representations) of all our 2D symbol systems, and, by using projective geometry, for rendering simulations of 3D structures. The designers of our interactive graphical interfaces were both applied semioticians and systems engineers. The interface design concept that began in Doug Engelbart’s lab

TABLE 9.6  Programming and software: Source code to dialogic interaction. Source Code

Interpretant system

Binary ‘machine code’ file (or interpreted code at ‘run time’)

Interpretant system

⇒

⇒

⇒

⇒

Program text file (in Unicode bytecode representations) written in a high-level programming language (C++, Python, Java, etc.).

Complier program or interpreter translates source code text into binary ‘machine level’ code.

A binary code program file, as copied to a storage device, is assignable to a physical system as executable (‘runnable’) code.

Operating system ‘writes’ a tokenized ‘copy’ of the program into RAM, and CPUs initiate instructions for processes for specified data types.

‘Running code’ activated and directed by semiotic agents ‘Users’ are semiotic agents, dialogically interacting with the software for the symbolic systems interpreted and represented in the software, in a semiosic cycle.

224

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

and continued through all versions of windowing interfaces in PCs, distinguished three levels of ‘interfaces’: the physical, the cognitive and the conceptual (Card and Moran 1988; Moggridge 2007). To embody the interface concepts in the software behind what we see rendered in screens, graphical interfaces are designed with a ‘meta’ layer that we now take for granted in computer devices as two-way dialogic systems. The ‘interface’, as a semiotic substrate, is not simply a passive display for static representations, but incorporates an input system layer for communicating semiotic agency (intentions, choices, directions) back into the system for ongoing dialogic interaction with dynamic configurations of representations projected into the physical substrates of the screen. Our current interface designs support the ‘interactive computing paradigm’, which was developed by using implicit semiotic principles for designing non-terminating programs for multi-symbolic systems and recursive dialogic interpretive processes.27 Further, as Licklider and Kay envisioned, a digital multimedia interactive computer system is not correctly conceived as a medium, but as a metamedium, a medium for representing, interpreting, communicating and creating new instances of all symbolic media. Our contemporary computing paradigm is thus an implementation of Peirce’s dialogic model for dynamic symbolic systems, which includes different kinds of semiotic agency in the new combined system of human cognizers and distributed agency in many layers of software and networked systems.

CONCLUSIONS There can’t be a ‘semiotic approach’ to the study of computer systems, software, digital media, interfaces or the internet because these technologies are constitutively and structurally semiotic. That is, digital computing and information technologies are (1) complex-system artefacts designed by means of the cognitive-symbolic capacities of human sign-using communities, and (2) the whole architecture of subsystems and supporting technologies follows telic design principles for serving human symbolic systems and their corresponding patterns and actions of interpretation. Computer system design principles provide homological maps for symbolic to physical correspondences that enact assigned representations and operations. Any computer system, large, small or unobservable, represents an implementable design of applied semiotic structures in a unified architecture based on, and in the service of, human symbolic thought. From a pragmatist semiotic perspective, the computer system is not just the complex physical system of hardware, software and data (the hidden artefactual structures in machines, networks and stored information), even when correctly described as semiotic artefacts. The ‘computer system’ is actually the whole dialogic supersystem comprised of semiotic agents (aka ‘users’), who are not independent individuals but members of meaning-making communities, and computer systems embodying semiotic system design for dialogic interaction. As Engelbart originally envisioned, human cognizers + dynamic computational semiotic systems form a whole new third system not reducible to a sum of the constituents. We are members and agents of the designed systems, presupposed and included in the designs, not detached, empirical observers of a ‘machine’ (Winograd and Flores 1987; Winograd 1997). We activate the built-in agency position in all the interactive-dialogic relations with the physical architectures, in the ‘code’ of any running software, in the affordances of interfaces, and in all accessed networked information, near or far.

SEMIOTICS IN COMPUTING AND INFORMATION SYSTEMS

225

There are many other levels and contexts of semiotic functions in the design and use of software, databases, digital media, interfaces and AI, and these applications will open up for semiotic description by extending the semiotic systems deblackboxing method outlined here. Further, by extending Peirce’s semeiotic for our contemporary context, we have an open opportunity for bridge-building across disciplines, for embracing all knowledge domains relevant for semiotic research, and for reclaiming the foundational history of ideas woven with Leibniz’s ‘mechanical thread’.

NOTES 1 Leibniz’s philosophy of symbols and the metaphor of the ‘mechanical thread’ are in Leibniz 1975 (Loemker, ed.) and Dascal 1987. For the texts on Leibniz’s mechanical calculator, the binary (‘dyadic’) number system and his binary calculator, see Leibniz [1679] 2010 and [1710] 2009. 2 For background on the intellectual history and theory from different disciplinary viewpoints, see: Gorn 1968; Skagestad 1996; Andersen et al. 1997; Frank 2003; Gudwin and Queiroz 2005; Nadin 2007; Tedre 2014; Meunier 2018. 3 On ‘deblackboxing’, see Latour 1999: 183–93, and 2002; on closed, locked-in computing devices, see Zittrain 2009. 4 On mapping principles in diagrammatic reasoning and applications in computer systems, see Sowa 1984: 367–402; Glasgow, et al., eds. 1995; Goguen 1999; Sowa 2000a; Goguen and Harrell 2005; Stjernfelt 2007; Denning and Martell 2015: 123–35. 5 For background on the implicit and explicit semiotic foundations in the design history of computing, which includes earlier logic machines, diagrams, ‘paper machines’, and predigital methods for automating reasoning, see Gardner 1958; Webb 1980; Krämer 1988; Aspray 1990; Marciszewski and Murawski 1995; Priestley 2011; von Plato 2017, and for the intellectual history of computers as symbolic systems, see Mahoney 2011. 6 For example: Engelbart 1963; Winograd and Flores 1987; Card and Moran 1988, and other papers in Goldberg (ed) 1988; Rheingold 2000; Murray 2012; Manovich 2013; Rocchi 2013; Dasgupta 2014; Tedre 2014. 7 The following studies from various schools of thought on cognition, computation, and the computational theory of mind, include both implicit and explicit semiotic theory: Haugeland 1981a; Pylyshyn 1984; Schank and Childers 1984; Winograd and Flores 1987; Horst 1996; Agre 1997; Cummins and Cummins 2000; Scheutz 2002; Nilsson 2007; Clark 2008; Dror and Harnad 2008; Nilsson 2009; Rapaport 2012; Rescorla 2020. 8 Peirce’s explicit definitions for his later program of Logic [considered] as Semeiotic begin in 1896 (MSS 900, 900(s), Logic of Mathematics) and 1897 (MSS 738 and 798, On logic and semeiotic), and he develops the theory continuously from 1901–2 (in his drafts of the Minute Logic project, especially MS 425, which treats ‘reasoning by machinery’) to his final papers in 1913, a year before his death. On the first page of notebook pages from 1903, Peirce wrote the title, Mathematics as It Is to Be Treated in My Logic Treated as Semeiotics (MS 66). For general background, see Fisch 1986: 338–42, Colapietro 2003, Pietarinen 2006, and Bellucci 2014. I treat Peirce’s Logic as Semeiotic in relation to computing, information, AI and symbolic thought in a forthcoming book. 9 In my comprehensive survey of Peirce’s writings from 1890 to 1914 (in thousands of pages of his unpublished papers, and in his published articles and recent editions), I found that Peirce uses the term Logic as Semeiotic (and equivalent phrases) over fifty times. During this period,

226

10 11

12

13

14

15

16

17 18

19 20 21

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Peirce’s preferred spelling is semeiotic, sometimes semiotic, and very rarely in the plural form, semeiotics/semiotics (among over ninety uses of the terms). Peirce intended semeiotic to preserve the meanings in the traditions of logic (Greek: semeiotike), represented by the term used in works by John Locke and German logicians; but Peirce generalized semeiotic for formalizing the structures of all sign systems, especially the necessary structures of reasoning in mathematics and logic. See: Hintikka 1996; Anellis 2015; Øhrstrom 2017; and essays in Houser et al. 1997. The standard descriptions of computer system architecture and design principles are treated in all textbooks on the subject; the following provide accessible orientations: Heuring and Jordan 2003; Saltzer and Kaashoek 2009; and especially Tedre 2014 and Denning and Martell 2015. For thorough technical descriptions, see Blaauw and Brooks 1997; Comer 2017; Hennessy and Patterson 2017; Patt and Patel 2020. Valuable for systems theory concepts are Winograd and Flores 1987; Simon 1996; and Arthur 2011. For an orientation to the principle of levels of abstraction, subsystems, and system design see: Simon 1996; Baldwin and Clark 2000; Floridi 2008; Gobbo and Benini 2014; Denning and Martell 2015: 198–212; Rescorla 2017. Important studies that discuss or assume the cognitive artefact concept for computing, in different disciplinary contexts, are: Gorn 1968; Norman 1991; Hutchins 1999; Mahoney 2005; Houkes and Vermaas 2010; Nadin 2011; Borgo et al. 2014; Kockelman 2017b; Turner 2018; Anderson 2019; Sørensen et al. 2020. Important sources are Clark and Chalmers 1998; Latour 1999: 176–98; Hollan et al. 2000; Dascal and Dror 2005; Zhang and Patel 2006; Dror 2007; Clark 2008; Dror and Harnad 2008; Enfield and Kockelman 2017; Kockelman 2017a. The story of Peirce’s unrecognized contributions to, and anticipations of, the foundations of modern computing has yet to be told; for intellectual historical facts, connections, and insights, see Ketner and Stewart 1984; Ketner 1988; Gandy 1995; Skagestad 1996; Nöth 1997, 2003; Nadin 2011, 2017. Researchers will find the selections of papers in EP2, NEM (ed. Eisele 1985), and SWS (ed. Bellucci 2020) to be good starting points, but the most important writings from 1906 to 1912 that apply to computation, symbolic operations, and logical machines have not been published. I provide a catalogue of these papers, and edited selections of the most important sources, on my website: https://irvine.georgetown.domains/Peirce/. The sources for these concepts in Peirce’s papers are documented on my website: https:// irvine.georgetown.domains/Peirce/. The important developments in computing referenced here are documented in Ceruzzi 1983, 2003; Rheingold 2000; Ifrah 2001; Mahoney 2011; Davis 2012; and Campbell-Kelly and Aspray 2014. For the key concepts in interface and interaction design for our GUI systems from the 1960s on, see Goldberg (ed) 1988; and Moggridge 2007. Leibniz 1679, 1710; 1975 (Loemker, ed); Dascal 1987. Licklider 1960, 1965, 1977; Licklider and Clark 1962; Licklider and Taylor 1968; and see: Waldrop 2001. Accessible and ‘semiotics aware’ introductions to computing and computer systems are Tedre 2014; Denning and Martell 2015. Additional useful guides for the key concepts in computation are Hilton 1963; Smith 1998, 2002; Mahoney 2011; Davis 2012. More advanced accounts of the history of logic and automated reasoning, which reveal implicit semiotic principles, are Marciszewski and Murawski 1995; Rojas and Hashagen 2000; Priestley 2011; von Plato 2017.

SEMIOTICS IN COMPUTING AND INFORMATION SYSTEMS

227

22 See the papers from the ACM Ubiquity Symposium on ‘What Is Computation?’ (2011), available online: https://ubiquity.acm.org/symposia2011.cfm. The papers were also published in The Computer Journal 55 (7) (2012). This approach is also followed by Tedre (2014) and Denning and Martell (2015), and assumed throughout in Rheingold (2000). 23 To avoid confusion in terminology, we need to differentiate homology from analogy (any kind of likeness or comparison) and from the related terms isomorphism and hom(e)omorphism, used for strictly defined abstract, mathematical equivalences (as in category theory and topology) (Krömer 2007; Marquis 2009). The term homology is also used in other sciences, and you may find the terms from abstract mathematics used in the computing literature for mappings in digital architecture. But homology, in the general sense defined by Peirce, is the most appropriate term for the one-to-one, formal-to-physical imposed correspondences in digital computer system design. 24 ‘Information Theory’, as defined in electrical engineering and digital design, is continually misunderstood and mystified. For sources and useful explanations, see Shannon and Weaver 1949; Pierce 1980; Frank 2003; Gleick 2011; Nadin 2011; Denning and Martell 2015: 35–58; the implicit and explicit semiotic foundations of digital information are also discussed in MacKay 1969 and essays in Machlup and Mansfield (eds) 1983. 25 Accessible introductions are Turbak and Gifford 2008; Martin 2010; Denning and Martell 2015: 83–121. 26 The binary code translator programs are called ‘compilers’ and ‘interpreters’. 27 Important sources for supporting a semiotic view of interactive systems are Engelbart 1963; Hutchins et al. 1985; Agre and Rosenschein 1996; Wegner 1997; De Souza 2005; Goldin et al. 2006; Moggridge 2007; Murray 2012.

REFERENCES Abbott, R. (2019), ‘The Bit (and Three Other Abstractions) Define the Borderline between Hardware and Software’, Minds and Machines, 29/2: 239–85. Agre, P. E. (1995), ‘The Soul Gained and Lost: Artificial Intelligence as a Philosophical Project’, Stanford Humanities Review, 4/2: 1–19. Agre, P. E. (1997), Computation and Human Experience, Cambridge; New York: Cambridge University Press. Agre, P. E. and S. J. Rosenschein, eds (1996), Computational Theories of Interaction and Agency, Cambridge, MA: MIT Press. Ambrosio, C. (2014), ‘Iconic Representations and Representative Practices’, International Studies in the Philosophy of Science, 28/3: 255–75. Anderson, N. G. (2019), ‘Information Processing Artifacts’, Minds and Machines, 29/2: 193–225. Andersen, P. B., B. Holmqvist, and J. F. Jensen, eds (1993), The Computer as Medium, Cambridge: Cambridge University Press. Andersen, P. B., P. Hasle, and P. A. Brandt (1997), ‘Machine Semiosis’, in Roland Posner, Klaus Robering, and Thomas A., Sebeok (eds), Semiotik/Semiotics: A Handbook on the Sign-Theoretic Foundations of Nature and Culture, vol. 1, 548–71, Berlin and New York: Mouton De Gruyter. Anellis, I. H. (2015), ‘Peirce’s Role in the History of Logic: Lingua Universalis and Calculus Ratiocinator’, in Arnold Koslow and Arthur Buchsbaum (eds), The Road to Universal Logic, vols 1–2, vol. 2, 135–69, Cham: Birkhäuser and Springer.

228

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Arthur, W. B. (2011), The Nature of Technology: What It Is and How It Evolves, New York: Free Press. Aspray, W., ed. (1990), Computing before Computers, Ames, IA: Iowa State Press. Baldwin, C. Y. and K. B. Clark (2000), Design Rules, Vol. 1: The Power of Modularity, Cambridge, MA: MIT Press. Barbosa, Simone Diniz Junqueira, and Karin Breitman, eds (2017), Conversations Around Semiotic Engineering, Cham, Switzerland: Springer. Bellucci, F. (2014), ‘“Logic, Considered as Semeiotic”: On Peirce’s Philosophy of Logic’, Transactions of the Charles S. Peirce Society, 50/4: 523–47. Blaauw, G. A. and F. P. Brooks (1997), Computer Architecture: Concepts and Evolution, Reading, MA: Addison-Wesley. Blanchette, J. F. (2011), ‘A Material History of Bits’, Journal of the American Society for Information Science and Technology, 62 (6): 1042–57. Borgo, S., M. Franssen, P. Garbacz, Y. Kitamura, R. Mizoguchi, and P. E. Vermaas (2014), ‘Technical Artifacts: An Integrated Perspective’, Applied Ontology, 9 (3–4): 217–35. Bush, V. (1945), ‘As We May Think’, The Atlantic, July: 101–8. Campbell-Kelly, M. and S. B. Russ (1994), ‘Computing and Computers’, in Ivor GrattanGuinness (ed.), Companion Encyclopedia of the History and Philosophy of the Mathematical Sciences, vols. 1 and 2, 701–7, London; New York: Routledge. Campbell-Kelly, M. and W. Aspray (2014), Computer: A History of the Information Machine, 3rd edn, Boulder, CO: Westview Press. Card, S. K. and T. P. Moran (1988), ‘User Technology–from Pointing to Pondering’, in Adele Goldberg (ed.), A History of Personal Workstations, 489–521, New York: ACM Press and Addison-Wesley. Ceruzzi, P. E. (1983), Reckoners: The Prehistory of the Digital Computer, from Relays to the Stored Program Concept, 1935–45, Westport, CT: Greenwood. Ceruzzi, P. E. (2003), A History of Modern Computing, 2nd edn, Cambridge, MA: MIT Press. Cherry, C. (1957), On Human Communication: A Review, a Survey, and a Criticism, Cambridge, MA: MIT Press. Clark, A. (2008), Supersizing the Mind: Embodiment, Action, and Cognitive Extension, New York: Oxford University Press. Clark, A. and D. Chalmers (1998), ‘The Extended Mind’, Analysis, 58 (1): 7–19. Colapietro, V. (2003), ‘The Space of Signs: C. S. Peirce’s Critique of Psychologism’, in Dale Jacquette (ed.), Philosophy, Psychology, and Psychologism: Critical and Historical Readings on the Psychological Turn in Philosophy, 157–80, Dordrecht; Boston: Kluwer. Comer, D. E. (2017), Essentials of Computer Architecture, 2nd edn, Boca Raton: CRC Press. Conery, J. S. (2012), ‘Computation Is Symbol Manipulation’, The Computer Journal, 55 (7): 814–16. Cummins, R. and D. D. Cummins, eds (2000), Minds, Brains, and Computers: An Historical Introduction to the Foundations of Cognitive Science, Malden, MA: Wiley-Blackwell. Dascal, M. (1987), Leibniz – Language, Signs and Thought: A Collection of Essays, Amsterdam; Philadelphia: John Benjamins. Dascal, M. and I. E. Dror (2005), ‘The Impact of Cognitive Technologies: Towards a Pragmatic Approach’, Pragmatics & Cognition, 13 (3): 451–7. Dasgupta, S. (2014), It Began with Babbage: The Genesis of Computer Science, Oxford, UK: Oxford University Press. Davis, M. (2012), The Universal Computer: The Road from Leibniz to Turing, Boca Raton, FL: CRC Press.

SEMIOTICS IN COMPUTING AND INFORMATION SYSTEMS

229

De Souza, C. S. (2005), The Semiotic Engineering of Human-Computer Interaction, Cambridge, MA: MIT Press. Denning, P. J. (2012), ‘Opening Statement: What Is Computation?’, The Computer Journal, 55 (7): 805–10. Denning, P. J. and C. H. Martell (2015), Great Principles of Computing, Cambridge, MA: MIT Press. Dror, I. E. (2007), Cognitive Technologies and the Pragmatics of Cognition, Amsterdam; Philadelphia: John Benjamins. Dror, I. E. and S. R. Harnad, eds (2008), Cognition Distributed: How Cognitive Technology Extends Our Minds, Amsterdam; Philadelphia: John Benjamins. Eisele, C. (1979), Studies in the Scientific and Mathematical Philosophy of Charles S. Peirce, ed. R. M. Martin, The Hague: Mouton De Gruyter. Eisele, C., ed. (1985), Historical Perspectives on Peirce’s Logic of Science: Parts I and II, vols. 1–2, Berlin; New York: De Gruyter Mouton. Enfield, N. J. and P. Kockelman, eds (2017), Distributed Agency, Oxford; New York: Oxford University Press. Engelbart, D. C. (1963), ‘A Conceptual Framework for the Augmentation of Man’s Intellect’, in Paul W. Howerton and David C. Weeks (eds), Vistas in Information Handling, Volume I: The Augmentation of Man’s Intellect by Machine, 1–29, Washington, DC: Spartan Books. Engelbart, D. C. (1988), ‘The Augmented Knowledge Workshop’, in Adele Goldberg (ed.), A History of Personal Workstations, 185–248, New York: ACM. Eschbach, A. and J. Trabant, eds (1983), History of Semiotics, Amsterdam; Philadelphia: John Benjamins. Fetzer, J. H. (1988), ‘Signs and Minds: An Introduction to the Theory of Semiotic Systems’, in James H. Fetzer (ed.), Aspects of Artificial Intelligence, 133–61, Dordrecht and Boston: Kluwer. Fetzer, J. H. (1997), ‘Thinking and Computing: Computers as Special Kinds of Signs’, Minds and Machines, 7 (3): 345–64. Fetzer, J. H. (2001), Computers and Cognition: Why Minds Are Not Machines, Dordrecht: Springer. Fetzer, J. H. (2004), ‘Peirce and the Philosophy of Artificial Intelligence’, in M. Bergman and J. Queiroz (eds), The Commens Encyclopedia: The Digital Encyclopedia of Peirce Studies, New edn. http://www.commens.org/encyclopedia/article/fetzer-james-peirce-andphilosophy-artificial-intelligence. Fisch, M. H. (1986), Peirce, Semeiotic and Pragmatism, Bloomington: Indiana University Press. Floridi, L. (2008), ‘The Method of Levels of Abstraction’, Minds and Machines, 18 (3): 303–29. Frank, H. (2003), ‘Semiotik und Informationstheorie’, in Roland Posner, Klaus Robering, and Thomas A. Sebeok (eds), Semiotik/Semiotics: A Handbook on the Sign-Theoretic Foundations of Nature and Culture, vol. 3, 2418–37, Berlin and New York: Mouton De Gruyter. Gabbay, D. M. and J. Woods, eds. (2004), Handbook of the History of Logic, Vol. 3: The Rise of Modern Logic from Leibniz to Frege, Amsterdam; Boston: North Holland. Gabbay, D. M., J. H. Siekmann, and J. Woods, eds (2014), Handbook of the History of Logic, Vol. 9: Computational Logic, Amsterdam: North Holland. Gandy, R. (1995), ‘The Confluence of Ideas in 1936’, in Rolf Herken (ed.), The Universal Turing Machine: A Half-Century Survey, 50–102, Wien; New York: Springer. Gardner, M. (1958), Logic Machines and Diagrams, New York: McGraw–Hill. Glasgow, J., N. Hari, and B. Chandrasekaran, eds (1995), Diagrammatic Reasoning: Cognitive and Computational Perspectives, Cambridge, MA: AAAI Press.

230

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Gleick, J. (2011), The Information: A History, a Theory, a Flood, New York: Pantheon. Gobbo, F. and M. Benini (2014), ‘The Minimal Levels of Abstraction in the History of Modern Computing’, Philosophy & Technology, 27 (3): 327–43. Goguen, J. A. (1999), ‘An Introduction to Algebraic Semiotics, with Application to User Interface Design’, in Chrystopher L. Nehaniv (ed.), Computation for Metaphors, Analogy, and Agents, 242–91, Berlin and Heidelberg: Springer. Goguen, J. A. (2003), ‘Semiotic Morphisms, Representations and Blending for Interface Design’, in F. Spoto, G. Scollo, and A. Nijholt (eds), Algebraic Methods in Language Processing, vol. 21, 1–15, Twente: University of Twente. Goguen, J. A. and D. F. Harrell (2005), ‘Information Visualization and Semiotic Morphisms’, in Grant Malcolm (ed.), Multidisciplinary Approaches to Visual Representations and Interpretations, vol. 2, 83–98, Amsterdam; London: Elsevier. Goldberg, A., ed. (1988), A History of Personal Workstations, New York: ACM Press; AddisonWesley. Goldin, D., S. A. Smolka, and P. Wegner, eds (2006), Interactive Computation: The New Paradigm, Berlin; New York: Springer. Goldstine, H. H. and J. Von Neumann (1963), ‘Planning and Coding Problems for an Electronic Computing Instrument, Part II, vol. 1 (1946)’, in A. H. Taub (ed.), Collected Works, Vol. 5: Design of Computers, Theory of Automata and Numerical Analysis, vol. 5, 80–151, Oxford; London; New York: Pergamon Press. Gomes, A., R. Gudwin, C. N. El-Hani, and J. Queiroz (2007), ‘Towards the Emergence of Meaning Processes in Computers from Peircean Semiotics’, Mind & Society, 6 (2): 173–87. Gorn, S. (1967), ‘The Computer and Information Sciences and the Community of Disciplines’, Behavioral Science, 12 (6): 433–52. Gorn, S. (1968), ‘The Identification of the Computer and Information Sciences: Their Fundamental Semiotic Concepts and Relationships’, Foundations of Language, 4 (4): 339–72. Gorn, S. (1983), ‘Informatics (Computer and Information Science): Its Ideology, Methodology, and Sociology’, in Fritz Machlup and Una Mansfield (eds), The Study of Information: Interdisciplinary Messages, 121–40, New York: Wiley-Interscience. Grier, D. A. (2005), When Computers Were Human, Princeton: Princeton University Press. Gudwin, R. and J. Queiroz (2005), ‘Towards an Introduction to Computational Semiotics’, in International Conference on Integration of Knowledge Intensive Multi-Agent Systems (KIMAS 2005), 393–8, Waltham, MA: IEEE. Gudwin, R. R. (1999), ‘Umwelts and Artificial Devices: A Reflection on the Text of Claus Emeche: Does a Robot have an Umwelt ?’ Seminário Avançado de Comunicação e Semiótica 2: 51–6. Haugeland, J., ed. (1981a), Mind Design: Philosophy Psychology Artificial Intelligence, Cambridge, MA: MIT Press. Haugeland, J., ed. (1981b), ‘Semantic Engines: An Introduction to Mind Design’, in Mind Design: Philosophy Psychology Artificial Intelligence, 2–33, Cambridge, MA: MIT Press. Haugeland, J. (1985), Artificial Intelligence: The Very Idea. Cambridge, MA: MIT Press. Hennessy, J. L. and D. A. Patterson (2017), Computer Architecture: A Quantitative Approach, 6th edn, Cambridge, MA: Morgan Kaufmann. Heuring, V. P. and H. F. Jordan (2003), Computer Systems Design and Architecture, 2nd edn, Upper Saddle River, N.J: Pearson. Hilton, A. M. (1963), Logic, Computing Machines, and Automation, Cleveland, OH: Meridian Books.

SEMIOTICS IN COMPUTING AND INFORMATION SYSTEMS

231

Hintikka, J. (1996), ‘The Place of C. S. Peirce in the History of Logical Theory’, in Lingua Universalis vs. Calculus Ratiocinator: An Ultimate Presupposition of Twentieth-Century Philosophy, 140–61, Dordrecht: Kluwer. Hollan, J., E. Hutchins, and D. Kirsh (2000), ‘Distributed Cognition: Toward a New Foundation for Human-Computer Interaction Research’, ACM Transactions, ComputerHuman Interaction, 7 (2): 174–96. Horst, S. W. (1996), Symbols, Computation, and Intentionality: A Critique of the Computational Theory of Mind, Berkeley: University of California Press. Houkes, W. and P. E. Vermaas (2010), Technical Functions: On the Use and Design of Artefacts, Dordrecht; New York: Springer. Houser, N., D. D. Roberts, and J. Van Evra, eds (1997), Studies in the Logic of Charles Sanders Peirce, Bloomington: Indiana University Press. Hutchins, E. L. (1999), ‘Cognitive Artifacts’, in Robert A. Wilson and Frank Keil (eds), The MIT Encyclopedia of the Cognitive Sciences, 126–8, Cambridge, MA: MIT Press. Hutchins, E. L., J. D. Hollan, and D. A. Norman (1985), ‘Direct Manipulation Interfaces’, Human-Computer Interaction, 1 (4): 311–38. Ifrah, G. (2001), The Universal History of Computing: From the Abacus to the Quantum Computer, New York: Wiley. Jorna, R. J., B. Van Heusden, and R. Posner, eds (1993), Signs, Search and Communication: Semiotic Aspects of Artificial Intelligence, Berlin; New York: De Gruyter. Kay, A. (1972), A Personal Computer for Children of All Ages, Palo Alto, CA: Xerox PARC. Kay, A. (1984), ‘Computer Software’, Scientific American, 251 (3): 52–9. Kay, A. (2001), ‘User Interface: A Personal View’, in Randall Packer and Ken Jordan (eds), Multimedia: From Wagner to Virtual Reality, 121–31, New York: W.W. Norton. Kay, A. and A. Goldberg (1977), ‘Personal Dynamic Media’, Computer, 10 (3): 31–41. Ketner, K. L. (1988), ‘Peirce and Turing: Comparisons and Conjectures’, Semiotica, 68 (1/2): 33–61. Ketner, K. L. and A. F. Stewart (1984), ‘The Early History of Computer Design: Charles Sanders Peirce and Marquand’s Logical Machines’, The Princeton University Library Chronicle, 45 (3): 187–225. Kockelman, P. (2005), ‘The Semiotic Stance’, Semiotica, 2005 (157): 233–304. Kockelman, P. (2006), ‘Residence in the World: Affordances, Instruments, Actions, Roles, and Identities’, Semiotica, 2006 (162): 19–71. Kockelman, P. (2010), ‘Semiotics: Interpretants, Inference, and Intersubjectivity’, in Ruth Wodak, Barbara Johnstone, and Paul E. Kerswill (eds), The SAGE Handbook of Sociolinguistics, 165–78, Thousand Oaks, CA: SAGE Publications. Kockelman, P. (2017a), ‘Semiotic Agency’, in N. J. Enfield and Paul Kockelman (eds), Distributed Agency, 25–38, Oxford; New York: Oxford University Press. Kockelman, P. (2017b), The Art of Interpretation in the Age of Computation, New York: Oxford University Press. Krämer, S. (1988), Symbolische Maschinen: Die Idee Der Formalisierung in Geschichtlichem Abriss, Darmstadt: Wissenschaftliche Buchgesellschaft. Krömer, R. (2007), Tool and Object: A History and Philosophy of Category Theory, Basel; Boston: Birkhäuser. Latour, B. (1999), Pandora’s Hope: Essays on the Reality of Science Studies, Cambridge, MA: Harvard University Press. Latour, B. (2002), ‘Morality and Technology’, Theory, Culture & Society, 19 (5–6): 247–60.

232

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Leibniz, G. W. ([1679] 2010), ‘De Progressione Dyadica (Hannover MS), with Facsimile and Translation (Fr.)’ (Y. Serra, ed.). https://archive.org/details/69LeibnizDiadica. Leibniz, G. W. ([1710] 2009), ‘La machine arithmétique de Leibniz (1710) (with Illustrations)’ (Y. Serra and C.C. Adam, eds). https://archive.org/details/41Leibniz. Leibniz, G. W. (1975), Philosophical Papers and Letters: A Selection, 2nd edn, ed. and trans. L. E. Loemker, Dordrecht; Boston: Kluwer. Licklider, J. C. R. (1960), ‘Man-Computer Symbiosis’, IRE Transactions on Human Factors in Electronics, HFE-1/1: 4–11. Licklider, J. C. R. (1965), Libraries of the Future, Cambridge, MA: MIT Press. Licklider, J. C. R. (1968), ‘Computer Graphics as a Medium of Artistic Expression’, in Metropolitan Museum of Art (ed.), Computers and Their Potential Applications in Museums, 273–301, New York: Arno Press. Licklider, J. C. R. (1977), ‘User-Oriented Interactive Computer Graphics’, in Proceedings of the ACM/SIGGRAPH Workshop on User-oriented Design of Interactive Graphics Systems, 89–96, New York: ACM. Licklider, J. C. R. and W. E. Clark (1962), ‘On-Line Man-Computer Communication’, in Proceedings of the May 1–3,1962, Spring Joint Computer Conference, 113–28, New York: ACM. Licklider, J. C. R. and R. W. Taylor (1968), ‘The Computer as a Communication Device’, Science and Technology, 76: 21–38. Liu, K. (2000), Semiotics in Information Systems Engineering, Cambridge; New York: Cambridge University Press. Machlup, F. and U. Mansfield, eds (1983), The Study of Information: Interdisciplinary Messages, New York: Wiley-Interscience. MacKay, D. M. (1969), Information, Mechanism and Meaning, Cambridge, MA: MIT Press. Mahoney, M. S. (2005), ‘The Histories of Computing(s)’, Interdisciplinary Science Reviews, 30 (2): 119–35. Mahoney, M. S. (2011), Histories of Computing, Cambridge, MA: Harvard University Press. Malcolm, G. and J. A. Goguen (1999), ‘Signs and Representations: Semiotics for User Interface Design’, in Ray Paton and Irene Neilsen (eds), Visual Representations and Interpretations, 162–72, London; New York: Springer. Manovich, L. (2013), Software Takes Command: Extending the Language of New Media, London: Bloomsbury. Marciszewski, W. and R. Murawski (1995), Mechanization of Reasoning in a Historical Perspective, Amsterdam; Atlanta: Rodopi. Marcus, G. F. (2001), The Algebraic Mind: Integrating Connectionism and Cognitive Science, Cambridge, MA: MIT Press. Marquis, J. P. (2009), From a Geometrical Point of View: A Study of the History and Philosophy of Category Theory, Dordrecht: Springer. Martin, J. (2010), Introduction to Languages and the Theory of Computation, 4th edn, New York: McGraw-Hill. Meunier, J.-G. (1989), ‘Artificial Intelligence and Sign Theory’, Semiotica, 77 (1–3): 43–64. Meunier, J.-G. (1998), ‘Categorial Structure of Iconic Languages’, Theory and Psychology, 8 (6): 805–27. Meunier, J.-G. (2018), ‘Vers une Sémiotique Computationnelle?’, Applied Semiotics / Sémiotique appliquée, 26: 76–107. Moggridge, B. (2007), Designing Interactions, Cambridge, MA: MIT Press.

SEMIOTICS IN COMPUTING AND INFORMATION SYSTEMS

233

Morris, C. W. (1938), Foundations of the Theory of Signs, Chicago, IL: University of Chicago Press. Morris, C. W. (1946), Signs, Language and Behavior, New York: Prentice Hall. Morris, C. W. (1964), Signification and Significance: A Study of the Relations of Signs and Values, Cambridge, MA: MIT Press. Murray, J. H. (2012), Inventing the Medium: Principles of Interaction Design as a Cultural Practice, Cambridge, MA: MIT Press. Nadin, M. (1988a), ‘Interface Design: A Semiotic Paradigm’, Semiotica, 69 (3/4): 269–302. Nadin, M. (1988b), ‘Interface Design and Evaluation: Semiotic Implications’, in H. Rex Hartson and Deborah Hix (eds), Advances in Human-Computer Interaction, vol. 2, 45–100, Norwood, N.J: Ablex. Nadin, M. (1998), ‘Computer’, in Paul Bouissac (ed.), Encyclopedia of Semiotics (Online), Oxford: Oxford University Press. Nadin, M. (2007), ‘Semiotic Machine’, The Public Journal of Semiotics, 1 (1): 57–75. Nadin, M. (2011), ‘Information and Semiotic Processes: The Semiotics of Computation’, Cybernetics & Human Knowing, 18 (1–2): 153–75. Nadin, M. (2017), ‘Semiotic Engineering – An Opportunity or an Opportunity Missed?’, in Simone Diniz Junqueira Barbosa and Karin Breitman (eds), Conversations Around Semiotic Engineering, 41–63, Cham, Switzerland: Springer. Nake, F. (1997), ‘Der semiotische Charakter der informatischen Gegenstände’, Semiosis, 85–90: 24–35. Nake, F. (1999), ‘Bildgeschichten aus Zahlen und Zufall: Betrachtungen zur Computerkunst’, in Andreas Dress and Gottfried Jäger (eds), Visualisierung in Mathematik, Technik und Kunst: Grundlagen und Anwendungen, 117–36, Wiesbaden: Vieweg+Teubner. Nake, F. (2002), ‘Werk, Kunstwerk, Information, Zeichen’, in Udo Bayer and Karl Gfesser (eds), Kontinuum der Zeichen: Elisabeth Walther-Bense und die Semiotik, 9–13, Stuttgart: J.B. Metzler. Nake, F. (2008a), ‘Surface, Interface, Subface: Three Cases of Interaction and One Concept’, in Uwe Seifert et al. (eds), Paradoxes of Interactivity: Perspectives for Media Theory, Human-Computer Interaction, and Artistic Investigations, 92–109, Berlin: Boston: Transcript Verlag. Nake, F. (2008b), ‘Work, Design, Computers, Artifacts’, in Thomas Binder, Jonas Löwgren, and Lone Malmborg (eds) (Re)Searching the Digital Bauhaus, 309–31, London: Springer. Nake, F. (2009), ‘The Semiotic Engine: Notes on the History of Algorithmic Images in Europe’, Art Journal, 68 (1): 76–89. Nake, F. and S. Grabowski (2006), ‘The Interface as Sign and as Aesthetic Event’, in Paul A. Fishwick (ed.), Aesthetic Computing, 53–70, Cambridge, MA: MIT Press. Newell, A. (1980), ‘Physical Symbol Systems’, Cognitive Science, 4 (2): 135–83. Newell, A. (1986), ‘The Symbol Level and the Knowledge Level’, in Zenon W. Pylyshyn and William Demopoulos (eds), Meaning and Cognitive Structure: Issues in the Computational Theory of Mind, Norwood, NJ: Praeger. Newell, A. and H. A. Simon (1961), ‘Computer Simulation of Human Thinking’, Science, 134 (3495): 2011–17. Newell, A. and H. A. Simon (1972), Human Problem Solving, Englewood Cliffs, NJ: Prentice Hall. Newell, A. and H. A. Simon (1976), ‘Computer Science as Empirical Inquiry: Symbols and Search’, Communications of the ACM, 19 (3): 113–26.

234

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Newell, A. and H. A. Simon (2003), ‘Symbol Manipulation’, in Encyclopedia of Computer Science, 1731–5, Chichester, UK: John Wiley and Sons. Nilsson, N. J. (2007), ‘The Physical Symbol System Hypothesis: Status and Prospects’, in Max Lungarella, Fumiya Iida, Josh Bongard, and Rolf Pfeifer (eds), 50 Years of Artificial Intelligence, 9–17, Berlin and Heidelberg: Springer. Nilsson, N. J. (2009), The Quest for Artificial Intelligence: A History of Ideas and Achievements, Cambridge, UK: Cambridge University Press. Nisan, N. and S. Schocken (2005), The Elements of Computing Systems: Building a Modern Computer from First Principles, Cambridge, MA: MIT Press. Norman, D. A. (1991), ‘Cognitive Artifacts’, in John M. Carroll (ed.), Designing Interaction, 17–38, New York: Cambridge University Press. Norman, D. A. and S. W. Draper, eds (1986), User Centered System Design: New Perspectives on Human-computer Interaction, Hillsdale, NJ: Lawrence Erlbaum Associates. Nöth, W. (1997), ‘Representation in Semiotics and in Computer Science’, Semiotica, 115 (3/4): 203–13. Nöth, W. (2002), ‘Semiotic Machines’, Cybernetics & Human Knowing, 9 (1): 5–21. Nöth, W. (2003), ‘Semiotic Machines’, SEED Journal, 3 (3): 81–99. Øhrstrom, P. (2017), ‘C. S. Peirce’, in Alex Malpass and Marianna Antonutti Marfori (eds), The History of Philosophical and Formal Logic: From Aristotle to Tarski, 165–81, London; New York: Bloomsbury. Patt, Y. and S. Patel (2020), Introduction to Computing Systems: From Bits & Gates to C/C++ & Beyond, 3rd edn, New York: McGraw-Hill. Peirce, C. S. ([1857–92] 1982–2010), Writings of Charles S. Peirce, 7 vols. (1–6, 8), Peirce Edition Project (eds), Bloomington: Indiana University Press. Cited as W. Peirce, C. S. ([1857–1914] 1787–1951), The Charles S. Peirce Papers Harvard University, Houghton Library, MS Am 1632. Individual papers are referenced by manuscript number in R. Robin (ed.), Annotated Catalogue of the Papers of Charles S. Peirce, Amherst: University of Massachusetts Press, 1967, and in Robin, ‘The Peirce Papers: A Supplementary Catalogue’, Transactions of the Charles S. Peirce Society 7, 1971: 37–57. Cited as MS. Peirce, C. S. ([1866–1913] 1976), The New Elements of Mathematics, 4 vols., C. Eisele (ed.), The Hague: Mouton Press. Cited as NEM. Peirce, C. S. (1889–91), entries in The Century Dictionary: An Encyclopedic Lexicon of the English Language, W. D. Whitney (ed.), New York: Century Co. Cited as CD. Peirce, C. S. ([1893–1913] 1998), The Essential Peirce: Selected Philosophical Writings, vol. 2, eds. Peirce Edition Project, Bloomington: Indiana University Press. Cited as EP 2. Peirce, C. S. ([1894–1912] 2020), Selected Writings on Semiotics, 1894–1912, ed. F. Bellucci, Berlin: Mouton De Gruyter. Cited as SWS. Pierce, J. R. (1980), An Introduction to Information Theory: Symbols, Signals and Noise, 2nd edn, first publ. 1961. New York: Dover. Pietarinen, A. V. (2006), Signs of Logic: Peircean Themes on the Philosophy of Language, Games, and Communication, Dordrecht; London: Springer. Posner, R., H. Klein, P. B. Andersen, and B. Holmqvist (1996), Signs of Work: Semiosis and Information Processing in Organisations. Berlin: De Gruyter. Posner, R., K. Robering, and T. A. Sebeok, eds. (1997–2003), Semiotik/Semiotics: A Handbook on the Sign-Theoretic Foundations of Nature and Culture, 1–3 vols., Berlin; New York: Mouton De Gruyter. Priestley, M. (2011), A Science of Operations: Machines, Logic and the Invention of Programming, New York; London: Springer.

SEMIOTICS IN COMPUTING AND INFORMATION SYSTEMS

235

Pylyshyn, Z. W. (1984), Computation and Cognition: Toward a Foundation for Cognitive Science, Cambridge, MA: MIT Press. Queiroz, J. and F. Merrell (2009), ‘On Peirce’s Pragmatic Notion of Semiosis: A Contribution for the Design of Meaning Machines’, Minds & Machines, 19 (1): 129–43. Ransdell, Joseph (2003), ‘The Relevance of Peircean Semiotic to Computational Intelligence Augmentation’, SEED Journal, 3 (3): 5–36. Rapaport, W. J. (1999), ‘Implementation is Semantic Interpretation’, The Monist, 82 (1): 109–30. Rapaport, W. J. (2012), ‘Semiotic Systems, Computers, and the Mind: How Cognition Could Be Computing’, International Journal of Signs and Semiotic Systems, 2 (1): 32–71. Rapaport, W. J. (2018), ‘What Is a Computer? A Survey’, Minds and Machines, 28 (3): 385–426. Rescorla, M. (2017), ‘Levels of Computational Explanation’, in Thomas M. Powers (ed.), Philosophy and Computing: Essays in Epistemology, Philosophy of Mind, Logic, and Ethics, 5–28, Cham, CH: Springer. Rescorla, M. (2020), ‘The Computational Theory of Mind’, in Edward N. Zalta (ed.), The Stanford Encyclopedia of Philosophy, Metaphysics Research Lab, Stanford University. https://plato.stanford.edu/entries/computational-mind/. Rheingold, H. (2000), Tools for Thought: The History and Future of Mind-Expanding Technology, revised edn, Cambridge, MA: MIT Press. Robinson, J. A. (1979), Logic: Form and Function: The Mechanization of Deductive Reasoning, Edinburgh: Edinburgh University Press. Robinson, J. A. and A. Voronkov, eds (2001), Handbook of Automated Reasoning, 1–2 vols. Amsterdam and Cambridge, MA: North Holland and MIT Press. Rocchi, P. (2013), Logic of Analog and Digital Machines, Hauppauge, NY: Nova Science. Rojas, R. and U. Hashagen, eds (2000), The First Computers: History and Architectures, Cambridge, MA: MIT Press. Saltzer, J. H. and M. F. Kaashoek (2009), Principles of Computer System Design: An Introduction, Burlington, MA: Morgan Kaufmann. Schank, R. C. and P. G. Childers (1984), The Cognitive Computer: On Language, Learning, and Artificial Intelligence, Reading, MA: Addison-Wesley. Scheutz, M., ed. (2002), Computationalism: New Directions, Cambridge, MA: MIT Press. Shannon, C. E. (1938), ‘A Symbolic Analysis of Relay and Switching Circuits’, Transactions of the American Institute of Electrical Engineers, 57 (12): 713–23. Shannon, C. E. (1948), ‘A Mathematical Theory of Communication’, The Bell System Technical Journal, 27: 379–423, 623–56. Shannon, C. E. and W. Weaver (1949), The Mathematical Theory of Communication, Champaign, IL: University of Illinois. Shneiderman, B. (1982), ‘The Future of Interactive Systems and the Emergence of Direct Manipulation’, Behaviour & Information Technology, 1 (3): 237–56. Shneiderman, B. (1983), ‘Direct Manipulation: A Step Beyond Programming Languages’, IEEE Computer, 16 (8): 57–69. Shneiderman, B. (1997), Designing the User Interface, 3rd edn, Reading, MA: Addison Wesley. Simon, H. A. (1993), ‘The Human Mind: The Symbolic Level’, Proceedings of the American Philosophical Society, 137 (4): 638–47. Simon, H. A. (1996), The Sciences of the Artificial, 3rd edn, first published, 1969. Cambridge, MA: MIT Press.

236

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Skagestad, P. (1993), ‘Thinking with Machines: Intelligence Augmentation, Evolutionary Epistemology, and Semiotic’, Journal of Social and Evolutionary Systems, 16 (2): 157–80. Skagestad, P. (1996), ‘The Mind’s Machines: The Turing Machine, the Memex, and the Personal Computer’, Semiotica, 111 (3/4): 217–43. Skagestad, P. (1999), ‘Peirce’s Inkstand as an External Embodiment of Mind’, Transactions of the Charles S. Peirce Society, 35 (3): 551–61. Smith, B. C. (1998), On the Origin of Objects, Cambridge, MA: MIT Press. Smith, B. C. (2002), ‘The Foundations of Computing’, in Matthias Scheutz (ed.), Computationalism: New Directions, 23–58, Cambridge, MA: MIT Press. Sørensen, B., T. Thellefsen, and M. Thellefsen (2020), ‘A Peircean Semiotics of Technological Artefacts’, in Tony Jappy (ed.), The Bloomsbury Companion to Contemporary Peircean Semiotics, 253–75, London: Bloomsbury. Sowa, J. F. (1984), Conceptual Structures: Information Processing in Mind and Machine, Reading, MA: Addison-Wesley. Sowa, J. F., ed. (1991), Principles of Semantic Networks: Explorations in the Representation of Knowledge, San Mateo, CA: Morgan Kaufmann. Sowa, J. F. (2000a), Knowledge Representation: Logical, Philosophical, and Computational Foundations, Pacific Grove: Brooks/Cole; Thomson. Sowa, J. F. (2000b), ‘Ontology, Metadata, and Semiotics’, in Bernhard Ganter and Guy W. Mineau (eds), Conceptual Structures: Logical, Linguistic, and Computational Issues: 8th International Conference on Conceptual Structures, ICCS 2000, 55–81, Berlin; New York: Springer. Sowa, J. F. (2006), ‘Conceptual Graphs’, in Peter Bernus, Kai Mertins, and Günter J. Schmidt (eds), Handbook on Architectures of Information Systems, 295–320, Berlin and New York: Springer. Sowa, J. F. (2011), ‘Future Directions for Semantic Systems’, in Andreas Tolk and Lakhmi C. Jain (eds), Intelligence-Based Systems Engineering, 23–48, Berlin; Heidelberg: Springer. Steels, L. (2007), ‘Fifty Years of AI: From Symbols to Embodiment – and Back’, in Max Lungarella, Fumiya Iida, Josh Bongard, and Rolf Pfeifer (eds), 50 Years of Artificial Intelligence, 18–28, Berlin and Heidelberg: Springer. Stjernfelt, F. (2007), Diagrammatology: An Investigation on the Borderlines of Phenomenology, Ontology, and Semiotics. Berlin: Springer. Sutherland, I. E. (1963), ‘Sketchpad: A Man-Machine Graphical Communication System’, in Proceedings of the 21–23 May 1963, Spring Joint Computer Conference, 329–46, New York: ACM. Tanaka-Ishii, K. (2010), Semiotics of Programming, New York: Cambridge University Press. Tedre, M. (2014), The Science of Computing: Shaping a Discipline, Boca Raton: CRC Press. Turbak, F. and D. Gifford (2008), Design Concepts in Programming Languages, Cambridge, MA: MIT Press. Turing, A. M. (1937), ‘On Computable Numbers, with an Application to the Entscheidungsproblem’, Proceedings of the London Mathematical Society, Series 2, 42 (1): 230–65. Turner, R. (2018), Computational Artifacts: Towards a Philosophy of Computer Science, Berlin: Springer. Von Neumann, J. (1987), ‘First Draft of a Report on the EDVAC (1945)’, in William Aspray and Arthur W. Burks (eds), Papers of John von Neumann on Computers and Computing Theory, 17–82, Cambridge, MA: MIT Press.

SEMIOTICS IN COMPUTING AND INFORMATION SYSTEMS

237

von Plato, J. (2017), The Great Formal Machinery Works: Theories of Deduction and Computation at the Origins of the Digital Age, Princeton: Princeton University Press. Waldrop, M. M. (2001), The Dream Machine: J.C.R. Licklider and the Revolution That Made Computing Personal, New York: Viking Penguin. Webb, J. C. (1980), Mechanism, Mentalism and Metamathematics: An Essay on Finitism, Dordrecht: Reidel. Wegner, P. (1997), ‘Why Interaction Is More Powerful Than Algorithms’, Communications of the ACM, 40 (5): 80–91. Winograd, T. (1997), ‘The Design of Interaction’, in Peter J. Denning and Robert M. Metcalfe (eds), Beyond Calculation: The Next Fifty Years of Computing, 149–62, New York: Springer. Winograd, T. and F. Flores (1987), Understanding Computers and Cognition: A New Foundation for Design, Reading, MA: Addison-Wesley. Zhang, J. and V. L. Patel (2006), ‘Distributed Cognition, Representation, and Affordance’, Pragmatics & Cognition, 14 (2): 333–41. Zittrain, J. (2009), The Future of the Internet – And How to Stop It, New Haven, CT: Yale University Press.

238

CHAPTER TEN

Semiotics in Economics and Finance TODD OAKLEY

INTRODUCTION: MONEY AND OBJECTIFICATION ‘When the month was up, I had a million dollars to my credit in the London and County Bank’, recalls Henry Adams, the narrator and protagonist of Mark Twain’s short story, ‘The £100,000,000 Bank-Note’, a whimsical story published in 1893 amid Twain’s bankruptcy. A mining brokers clerk in San Francisco who liked to spend his free days sailing the bay, Henry found himself adrift in the Pacific Ocean one Saturday afternoon, only to be picked up by a British brig and subsequently deposited ragged and penniless in the streets of London. By hook and by crook, Henry became the subject of two wealthy brothers’ (Abel and Basil) bet, as he had what the brothers called an ‘honest face’: What will happen if we give him one of two £1-million Bank of England notes and leave him to his own devices for the month? Will he end up in jail, or will he return the note to us and, thereby, claim the reward of a ‘situation’ of his choosing? Now in possession of this conspicuous banknote that cannot be directly spent on sundry items of daily living – for no merchant can provide proper change – Henry nevertheless becomes something of a celebrity (‘the vest pocket million pounder’) able to secure room, board and the luxuries of high-class London. Every merchant, restaurateur and hotelier provision his every need for the month; all the while, his new friends secure his entry into the soirees of British nobility, such as the Duke and Duchess of Shoreditch or the Viscount Cheapside; all by this piece of paper. If that were not enough, every potential investor in his friend Lloyd Hasting’s mining interest bought shares solely based on Henry’s vouchsafing. To synopsize, Henry Adams entered London broke and hungry and left wealthy and betrothed to one of London’s finest young women, all because of single IOU. Readers of Twain’s story grasp the plausibility of his conceit, as each one is born and reared in a monetized economy, and even more, they have likely shuttled between different monetized economies. Thus, Henry’s ‘good fortune’ is embodied in a single piece of paper. At the same time, it is easy for readers to reflect on the strange, almost absurd fact that this piece of paper has nothing but the word of the Bank of England backing it, and that each one of Henry Adams’s emoluments hinges on an obligatory speech act from the signatory of said note. After all, people often fail to keep their promises; they are ‘just words’.

240

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Money is one of the most perplexing social structures ever to have existed. Classical (Adam Smith, David Ricardo, Jean-Baptiste Say, Thomas Malthus and J.S. Mill) and neoclassical economists (Alfred Marshall, Knut Wicksell, Irving Fisher and Léon Walras) generally follow the axiom that money is an exogenous variable to any market system, of no intrinsic importance to the internal dynamics thereof (see Skidelsky 2018), as such they regard money, a nominal source of value, as fundamentally different from the economy, the real value of goods and services produced within and traded between nation-states or empires. Much of neoclassical and neoliberal economic policy takes the view that the only role of the monetary authorities is to ‘get out of the way’ by assuring that central banks optimize the quantity of money to ensure that markets maintain their ‘natural’ equilibria. The banknote in Henry’s possession is simply a veil for barter, providing a work-around solution to the problem of ‘the coincidence of wants’. Henry does not need to possess a commodity of interest to the restaurateur, tailor, or hotelier for goods and services to be exchanged. Supply creates its own demand, according to Say’s law. If all goes according to plan, money is the dross of economic activity. Contemporary semiotic inquiry on the subject operates from the contradictory axiom that money is indeed a central concern of modern economic life. In concert with the above economists, most current semiotic analyses focus on money as a medium of exchange (Waller 2012; Bingham 2014). Henry’s possession of the note is a semiotic artefact encoding a speech act that also happens to be denominated in a unit of account (British Pound-Sterling) that functions to signify credit. According to this view, Money is language or speech. Bearers of British pound notes possess an articulation of value. Relatedly, the note is a form of potential and kinetic energy; it has power and does work – for good or ill. Semioticians are right to regard money as the lifeblood of market economies. Nevertheless, in focusing on money as a medium of exchange, both approaches tend to leave other vital functions in the background. The Great Financial Crisis of 2008 has made it apparent to vast swaths of the citizenry in the developed world that traditional economists were not only wrong but perhaps criminally negligent in building formal predictive models incapable of tracking real economies. Less apparent to semioticians and related scholars (none of whom have a ready audience of average citizens as the economists) is that their treatment of money as a medium of exchange disregards the other essential functions of money and money systems that are of even more significant consequence to economic life. The goal of semiotics is to understand meaning writ large, but such a profound gap in our understanding of one of the most potent social structures to have dominated human life for the last 4000 years almost will stunt the growth of inquiry. This chapter redresses this situation. I begin by explicating the functions of money as a semiotic phenomenon. The semiotic analysis of money follows a broadly Peircean scheme of triadic relations to highlight four interpretants underlying monetary relations. It is during these analyses that I engage with contemporary semiotic theories of money. With a clear grasp of the four interpretants of money, I proceed to discuss the different origin myths of money that undergird divergent economic paradigms, taking up the sociological work of Dodd (2014), the effect of which is to limn out the different semantic frames of money that comprise much of Western monetary, economic and political discourse. These origin myths set the background conditions for an exploration of money as social control. The exploration will examine money from distinct macroeconomic perspectives: the Austrian school; the Institutionalists and Chartalists; the Marxians; the Postructuralists; and the

SEMIOTICS IN ECONOMICS AND FINANCE

241

Metallists. The chapter concludes with two discussions. First, I argue that money is not so much the equivalent of speech but of writing and provide arguments from history and neurophysiology to support this contention, but I do concede that the money as speech equivalency is still useful in specific contexts. Generally, however, writing provides a more precise and nuanced equivalency for the semiotics of money. Second, I suggest that the semiotics of money should build more deliberately on Per Aage Brandt’s (2017) recent exploration of the ecological prerequisites of money.

INTERPRETANTS OF MONEY Semiotic theories of money typically begin with an economic dyad. For instance, Waller (2012) offers a broadly Peircean semiotic analysis of money in that Trader A, goods (referent) and money (signifier) form a signifying process mirrored in Trader B, who ‘translates’ money into goods. This analysis papers over many operations. It works in already stable money systems but says nothing about how those money systems come into being. To appreciate the nature of the problem, consider a simple Peircean signifying system of a backyard bee’s nest of Sign, Object and Interpretant. In this scenario, the sign is a perceived bulbous structure in the backyard, the Object is that of a bee colony and the Interpretant is the use of certain features of the way the sign signifies its object to generate and shape our understanding of the sign’s meaning. The object places constraints on the sign that it must meet if it is to signify the object. The size and colour of the nest may not be dispositive, since size varies and terrain colour varies, too. Shape, however, is likely to be dispositive of a species of bee, as nests are distinct in shape from, say, a hornet’s nest. Bee nest is an interpretant of the object and is thus a reliable sign. The generality of Peirce’s tripartite semiotic structure is intended to apply to all entities and organisms of the natural world – bees and humans alike. In that spirit, let’s assume Waller’s scenario of money as a means of exchange within this Peircean framework.1 First, we have a symbolic token – it does not matter what it is, just so it is durable, transportable, fungible and numerable. Trader A has X tokens aggregating to Y amount, which can be used to acquire any goods that are saleable in those tokens. In this case, ‘closing the semiotic triangle’, depends on each trader accepting the unit of account used to denominate the token. In this respect, Waller’s analysis is correct in that the goods held by Trader A can be translated into tokens held by Trader B, and that, by implication, the goods held by Trader A were once tokens held by Trader A. This chiastic circuit builds on prior circuits of exchange of the same type (but not necessarily involving the same tokens). Thus, it is possible to think of these exchanges as happening with greater or lesser frequency: hence, the ‘velocity’ of money is a measure of economic activity. Under this analysis, money exhibits what Bingham calls ‘positive movement’. One can learn much about the microeconomic dynamics of this semiotic scheme based on the intuition that money is a form of communication. However, once again, this analysis assumes that participants already operate in a stable semiotic system. In the final analysis, what does that tell us? As Hyman Minsky famously expresses it, ‘creating money is easy, getting it accepted is hard!’ Money as communication depends on trust, but how does trust develop? One of the problems with the money as communication thesis is what I call the unit of account problem. On what basis is the standard established? Now let us consider an ever so slightly different function of money: debt, which, as noted in the next section, has an historical provenance dating back to the development of

242

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

writing systems in Mesopotamia. Suppose instead that Trader A is Henry Adams. Adams secures his room and board through his ‘credit’ (Latin credere, ‘to trust’ or ‘to believe’). In this scenario, Adams possesses what the hotelier regards as credit (the possession of a denominated money token) that will not be used directly in the exchange of goods and services but indirectly as the basis for assessing the ‘credibility’ of Adams to eventually ‘redeem’ the debt. The interpretant here is the ‘loan worthiness’ of the person receiving the credit. Notice, however, that the hotelier could take Henry’s note and promise to give him proper change later, but that would amount to turning the hotelier into a debtor to one of his customers. It is better to have Adams owe him the smaller amount with the expectation of future recompense than try to come up with enough tokens to settle the account. Here money is debt; Henry’s banknote is what we call a ‘zero-perpetual bond’. That is, it is the equivalent of an interest-free bearer bond, for whoever possesses it acquires exactly the denominated amount of credit and no more. It just so happens that Henry possesses excessive credit. In this familiar scenario, the object of money tokens is ‘credit’, and the interpretant is the imputed ability of the debtor to settle the outstanding account. One of the primary social functions of money, according to Tony Lawson (2015), is to position community members as debtors and creditors, two primordial social categories (Graeber 2011). The dance of debtors and creditors exerts a ‘force of presence’ (Bingham 2014) on each other: the kinetic energy of debt is inversely proportional to the potential energy of credit. Henry bears a note that, in effect, represents an excessive store of potential energy such that he can translate that potential energy into any subaltern denomination of potential energy. Unlike physics, however, the energy itself need not be manifest in situ but instead can be symbolically ‘displaced’. Adams and the hotelier are taking value from the future. Money as a means of settlement contrasts with money as exchange in that the act of settling a debt effectively destroys the money (the only thing left over is the interest if any). Money’s destruction is why banks prefer that you not settle the debt prematurely. With exchange, money continually circulates among traders; it is not extinguished but instead migrates from node to node at different rates in a continuous circuit. A corollary of money as settlement is money as security or ‘protection’, or more commonly referred to by economists as a ‘store of value’. Money tokens signify protection from the vagaries of life. Keynes (1936) builds his macroeconomic theory on uncertainty as the overriding human condition. Saving money is a hedge against unexpected events. If the money token has ‘universal value’, then it confers a measure of protection. Part of its protective force issues from invocations of the divine, as perceptively emphasized by Brandt (2017). Precious metal coinage made its first known appearance in the Western world in the eighth century BCE in Lydia. According to Kurke (1999), the burgeoning democracy adopted gold and silver coinage precisely because the aristocracy used these precious metals as bodily adornments for signifying ‘bodily perfection’ (habrosunê) and was used in gift exchange to demonstrate an elite’s status or ‘meddle’. Under this dispensation, gold gave the elites direct access to the divine. In the eighth century, the political centre of power shifted from the spiritual elites to the bodily polis. Kurke (1999: 334) suggests that it was the contrast between phusis and nomos (‘essence’ and ‘function’), hataira and pornê (‘courtesan’ and ‘prostitute’), and dóro and nomisma (‘gift’ and ‘coin’) that formed the animating meanings of money. Gold was a way for the polis to co-opt the divine protection of the elites as a prophylactic against fortune and fate. Possession of a gold coin protected the bearer; it was a hedge against uncertainty because the bearer of the gold coin now has status and credit.

SEMIOTICS IN ECONOMICS AND FINANCE

243

The signification of status brings us to the money as a unit of account. It eventually turns out that the city controls the value (nomos) of the gold coins. The state has it both ways: it can at once engage in all the messy, earthy and profane lifeways through the mediation of the divine and noble essence of gold. We now arrive at sovereign money. What permits bearers of the money token to put their trust in those tokens as a medium of exchange, a means of settlement, and a source of security? In a word, the institution that adopts and enforces the standard unit of measure. The money token signifies a unit of account that becomes the only means of settling a debt with the sovereign. In a word, the sovereign places an obligation on its citizens, which can only be discharged through taxation. Taxation underwrites the basis of the Chartalist theory of money as the only acceptable unit of account. A subject does not come to use a specific form of money because she thinks the person on the other end of the trade is slightly more gullible than she; it is more likely that both parties accept the money. After all, it is the only thing each person can use to discharge each of their debts to the sovereign, the entity from whom one does not try to stiff. In this semiotic scenario, sovereigns and their institutions spend money into existence and then tax it out of existence. Taxation is a way for the government to have some measure of control over the money supply. A final derivative function of money is as a means of scorekeeping. Thorstein Veblen (1899) argues for ‘invidious comparison’, how we socially position ourselves relative to others to satisfy competitive appetites, as a pervasive feature of society. For example, a professor at University X exhibits anger at his institution for paying him 15 per cent less than the average salary of a professor at university Y, or a starting quarterback for an American Football team holds out in contract negotiations because he wants to be the highest-paid quarterback in the NFL. Now that we have a general account of the functions of money broadly within a social theoretical context, it is worth pausing for a moment to remind ourselves of the repeated interactions of economists, linguists and semioticians since the Enlightenment. Roman Jakobson (1990: 462) enumerates several semio-economic topics, from ‘dynamic synchrony’, ‘system contradictions’, ‘output-input’, ‘producer-consumer’, repeatedly given semiotic treatment by thinkers as diverse as Adam Smith and Ferdinand de Saussure. Jakobson notes as well that it was the eighteenth-century Russian economist Ivan Posoškov who popularized the slogan, ‘a ruble is not silver, a ruble is the ruler’s word’; and, earlier in England, John Law claiming that money is a ‘sign based on the prince’s signature’ (1990: 462). There is, therefore, a long tradition of economists and semioticians framing the problems of money as problems of language and communication, a theme to be noted in the sections below.

ORIGIN MYTHS Where did money begin? Did it start as the most efficient solution to the problems of barter, as most Classical, Neo-Classical and Austrian economists assume? Or did it begin with the semiotic advent of speech and exchange? Are we to look at the culturally universal but highly varied practices of gift-giving for its origins, or should we focus on the human capacity for complex quantification as the starting point? Or money did begin as a form of violence? The evidentiary state of money’s origins is so full of gaps that none of these discrete accounts can claim complete factual primacy, and all of them capture critical properties of money that are true enough to generate many of the background assumptions underlying

244

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

theories of money, economics, and finance. In this respect, they are myths, in the sense of providing heuristic frameworks for theories and models. After a description of each myth, I proceed to explore its implications for economic causality, its semantic profile and its broad relevance to political-economic theory and policy. I owe a great debt Nigel Dodd (2014) for his careful explication of these myths and their contribution to the social understanding of money, much of the material relies heavily on this source. However, while Dodd maintains scrupulous neutrality throughout his explication, I think it only fair and honest to confess that the money as tribute myth seems the most plausible as the impetus for the creation of sovereign money systems. Moreover, it is to sovereign money systems that shape the macroeconomic theories and policies under which most of us live.

Money as Barter The myth. Money emerged as a spontaneous solution to the problem of the double coincidence of wants in a barter exchange system. Does John have any goods that David wants and vice versa? What if one of them does not? Money solves the difficulty of finding two parties where each wants what the other is offering in exchange by settling on the third-party commodity. Some commodities are more stable than others, where stability means the facility with which a commodity can be sold. Gold and other precious metals make useful candidate commodities because they are always in demand, are relatively (but not excessively) scarce, do not oxidize or change colour drastically, and can be easily moulded, divided or otherwise denominated. Other examples include cowrie shells and other non-perishable goods. Water, though precious, is not a serviceable candidate, as it is cumbersome in large quantities, but more importantly, it evaporates. Once a commodity is thought of as ‘money’, its stability rises until everyone has an interest in acquiring it. The differentiation among monetary commodities and other commodities becomes marked and is apparent to all. Causality. Money arises from the needs of commerce. Markets pre-exist money and the role of the polity is as a mere enforcer of standards. Money’s origins are found solely in the market, not the polity. Ideally, money is apolitical, and its value was initially determined by the intrinsic value of the underlying commodity manifesting it. Profiling. Money as a commodity in markets is most salient; monetary authorities, standards of weights and measures, and law are inconspicuous or only salient as potential sources of friction or inefficiency of the markets. Participant roles of buyer and seller are profiled as equals. Barter is always construed as inefficient. The users of money are salient, while the issuers remain largely in the background. Political economy. The barter myth is still quite salient and commonplace, even though the evidence for the existence of barter communities in history or anthropology is virtually non-existent (Graeber 2011). This view is especially popular with Libertarians and economists of the Austrian school (e.g. Ludwig von Mises, Carl Menger) for they believe markets rather than states best organize that money. Enthusiasts of Bitcoin (and its developers) seem to work within this frame.

Money as language The myth. Money has no intrinsic value. It does not index some underlying ‘thing’ that, in turn, indexes some underlying ‘value’. Money is information, or, more specifically, code. What matters is the frequency with which this code migrates from station to station.

SEMIOTICS IN ECONOMICS AND FINANCE

245

Money arose as a semiotic code as signs whose power rests upon its coordination with other signs in a system of differences. Thus, money and language are close analogues. Just as a society needs to settle on a common koine of speech, which is arbitrary other than its being obligatory for members thereof, so does it need to settle on a conventional means of tokenized exchange. As societies develop, this code becomes increasingly ‘dematerialized’ and self-referential: money as gold or precious metal gives way to paper, which gives way to credit. This progression suggests that money starts as speech (spot exchange a la barter) but progresses to writing or code. Money is money only when it is in circulation, that is, when it moves from station to station. Much like a shark, if money stops moving, it dies! Causality. Money as symbolization correlates with the dominant mode of production (be it ancient, primitive, feudal, capitalist), socialist that is to say, that money and the mode of production are consubstantial; they constitute the economic system. Profiling. Money as code is salient, so is the preoccupation with security and encryption. Also, money is an arbitrary sign system, as a system of differences with no intrinsic value gets highlighted, as money is entirely relational. Money highlights the placement of an individual in a symbolic economy that begins with the ‘real’ or a state of nature before language, then proceeds to the idealist ‘ego’, before settling on the ‘symbolic’, or the internalization of the social order. Thus, to be monetized is to be symbolized as a member of such-and-such community. Political economy. This approach to money is embraced by several postmodernists and poststructuralists that take their cues from both de Saussure and Marx. While de Saussure made use of money as a productive heuristic for the structure of language and its synchronic study, he stopped short of embracing money as an equivalent of language. Jean-Paul Goux (Dodd 2014: 39–43), on the other hand, explicitly makes them equivalent. Marx, however, made it clear than money and language were distinctly different because money requires alienation – if anything, money is a form of translation for Marx. Rousseau also liked to compare money and language when thinking about the ‘social contract’. When money becomes ‘self-referential’, and everything becomes ‘monetized’, the social contract starts to break down.

Money as tribute The myth. Money emerged as payments associated with religious and political forms of tribute, not trade. Ancient rulers collected tribute long before the use of money as a means of exchange within markets developed. In this account, large public institutions are a necessary condition for the development of money, ‘not spontaneous interactions among utility-maximizing individuals’ (Dodd 2014: 23). Money originated as a means of payment to some sacred authority, and thus, money’s earliest manifestations were inextricably linked to violence, religious sacrifice and protection (either in the temporal or spiritual realms). There is considerably more evidence for this view than for barter (e.g. legal tablets of Babylonia and the Code of Ur-Nammu (2050 BCE), the Laws of Eshunna (1930 BCE) and the Babylonian Code of Hammurabi (1772 BCE). Wergild (lit. ‘Manmoney’) is an example of a system operating until the sixth century among the Germanic tribe, in which the governing authorities issued price ratios for bodies and body parts that were laid out in advance. Money becomes intimately associated with the ‘measure of the man’. Such systems of tribute, fines and later taxes, were prerequisites for establishing, maintaining, and strengthening a monetary system. Participants owe a ‘primordial debt’ to ‘society’, and each must acquire the only thing that will discharge that debt.

246

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Causality. The tribute frame arises from the needs of paying tribute to some authority, a supreme deity, and its appointed ministers (e.g, priests) and sovereigns. Critically, the payments are not between equals of buyer and seller. Profiling. Money as credit/debt is salient, as are the social roles of debtor and creditor. Large institutions – banks, governments and other institutions for clearing payments – are salient. De-emphasis on the spontaneous interactions between buyers and sellers and ‘price discovery’ takes a backseat to the issuers of money. Political economy. One normative facet of this frame is the proposition that money must be controlled by ‘the state’; money coevolved with our most ancient religious obligations. Many mainstream and heterodox Keynesians and Post-Keynesians operate according to this frame. To some degree, Marxians do too.2

Money as quantification The myth. What are the conditions of possibility for money to emerge? We begin with the dyad of two owners of objects, each of whom needs or wants what the other possesses. This mutual interest forms the basis of any exchange, which is the ‘act of reciprocal surrender’ or ‘mutual sacrifice’. Even a solitary act of, say, going jogging involves sacrifice insofar as something else, such as riding a bicycle, is foregone by running. The basis of money is an act of valuation: to gain something, we must simultaneously lose something else. We make sacrifices to ‘overcome the distance’ between ourselves and the object we need or want. Because exchanges involve sacrifice, the value that demands the sacrifice acquires a supra-subjective status; it must be ‘bigger than yourself’. Value, then, is intrinsically social, meaning that it is not a property that belongs to an object; it is a third category that comes to stand between us and the object. Nevertheless, through this intersubjective process, value comes to appear as though it is a real property of things themselves. When two or more objects of desire are compared, the immediacy of desire transcends the dualism of subject and object into something quantifiable. Quantification enables subjects to stand before us as independent values, and these values denote ‘resistance to our desire’ – the more valuable, the more quantifiable is the resistance to their being possessed by subjects. The development of universal value provides a way of managing the interplay between subjects, subjects and objects, and objects themselves. The management of subject and object makes money possible through quantification: only through a comparison of demands can definite and precise economic values be assigned to objects. Value is the distance between the subject and the object she or he desires. According to this myth, value and price are nearly identical. ‘Value is the epigone of price’ (Dodd 2014: 29). Money emerged to bring order to a world in which objects and values are in perpetual flux. Once in place, money distances subjects from objects by permitting the comparison between entirely different types of objects. We can compare apples and oranges; thus, while quantification allows for the valuation of everything according to an objectified standard, its application distances or alienates us from things. Hence, the moral concerns of monetizing everything. Causality. The causal reasoning runs from dyad to value to quantification, such that quantification of value becomes the ‘soluble’ for any exchange. Much like the barter myth, an intersubjective exchange is ground zero for the emergence of money. The causal direction is really where the similarities to barter stop. The quantification mythos wishes to account for the relationship between subjects and objects, whereas the barter

SEMIOTICS IN ECONOMICS AND FINANCE

247

mythos merely uses subjects and objects as abstract roles for establishing markets. The quantification myth remains agnostic about the status of markets as apolitical entities. Profiling. The most salient elements are objects as bearers of value, and value is to be construed in terms of sacrifice, of giving up something for something else. Value gets quantified, which then allows subjects to gauge the distance between themselves and the object. This distance is also construed force dynamically as the object’s resistance to the subject. The object has an agonistic force tendency that is either strong or weak, wherein strength is quantified according to some objectified unit of measure. The establishment of the unit of measure is incidental to the myth, however. What is of interest here is the motivation behind the need for a standard unit of measure. Ultimately, the salient view is of money as a ‘colourless’ and ‘anonymous’ function, even though its crucible is the hothouse of intersubjective exchange. Abstract value profiles exchange between strangers that are the hallmark of modernity. Political economy. This view of money correlates with the rise of the city and of modernity since population densities give rise to monetized relations. Quantification thrives in the company of strangers. The quantification myth has no strong political and economic identity, per se. One can see that any school of economics that focuses on the creation of value and price may operate within the frame. Its application might be more prevalent within a microeconomic analysis, and their focus on the mechanisms of price discovery.

Money as a gift The myth. Gifts are emotionally charged, morally loaded and reciprocal. If I give you a gift, you are thankful and may feel morally obligated to reciprocate. Under this dispensation, money originated as a gift exchange. Far from being a mere tool for reducing every social relationship to an anonymous function, money emerged from the constant give-and-take within communities marked by the continuous flow of goods and services. Consider Marcel Mauss’s interpretation of exchange among the Kula of Papua New Guinea. The system of gift exchange depends on what they call hau (also called mana) the spiritual power manifest in the gift that likewise underpins an obligation to reciprocate. Gifts must be passed on, for the receiver to retain what is given would be both morally and spiritually dangerous. Hau (‘circle’) creates a tie between givers and receivers and pertains not just between people but between souls, because ‘the thing itself possesses a soul’ (Mauss 1990: 16). Gift exchange is a social fact that permeates economic, tribal and moral life; its social mandate is pervasive. Now, certain items, such as emblazoned copper objects endemic to Northwest American natives or Samoan mats, fulfil some essential functions of money. They have the purchasing power that discharges debts. Thus, a specific number of copper objects can be used to purchase a specific number of blankets at rates that are fixed and publicly regarded. Gifts expressed social power; money expresses social power for managing the affective states of collective life. Money evolves as a means of expressing power over its members. Rather than a means of interacting with strangers, money as a gift manages the relations among familiars. Whatever effects it has on strangers is a by-product of gift-giving. Causality. Gift exchange causes the need to manage the circulation of goods. It is the talismanic power of the gift that drives the monetary circuit. Thus, money begins with a moral obligation. Profiling. The roles of the giver and receiver are salient, which can then be translated into creditors and debtors. Persons with money have social power, as they position themselves as creditors among debtors who must reciprocate in some fashion.

248

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Political economy. There is no mainstream school of politics and economics that fits comfortably within this framework, although the phenomenon of gift-giving has had some influence among heterodox economic theorists, such as Tony Lawson, who regards money as ‘social positioning’.

Money as violence The myth. Orthodox economics regards violence as extrinsic to money, as something contrary to its ‘natural’ mode of operation. Two French economists, Michel Aglietta and André Orléan (Dodd 2014: 43), have proposed that the violence is intrinsic to money’s origins. Taking their cue from René Girard’s equation of mimesis and violence, which argues that groups achieve delicate and precarious stability by balancing private desire with communal sacrifice. Our desire to attain what others have and to measure ourselves by ‘invidious comparison’ means that violence is always already immanent. Money, particularly gold as money, becomes a symbol of authority (of the sacred) that channels action in a way that can pacify. Money originates as a sacred object. Over time, the substance itself can cease to be linked to money (as when the Bretton Woods agreement broke down after Nixon took the US dollar off the gold standard) and is, thereby ‘desacralized’, but which requires an institution, a central authority or sovereign with a monopoly on violence, to ‘re-sacralise’ it and to render broad social ‘confidence’. Central  banks and their perceived independence have become the institution that re-sacralizes money. Causality. Money has two conflicting dynamics: it is at once a private object of desire and, on the other, it acts as a social bond or homogenizer. The causal dynamics of fragmentation and consolidation are what gives money its power; however, these dynamics are inherently unstable and subject to crises, which can pit citizen against citizen, citizen against the state and state against state. Profiling. Because money pacifies, staves off, or channels violence, its link to the sacred is salient in this mythic framework. Confidence in the form of a sacralising being or beings is also profiled, hence appeals to ersatz deities (e.g. ‘In God We Trust’) are commonplace sacralising speech printed on money tokens. The contrary of confidence is counterfeit (see the section on Poststructuralist money). It is violence against money, which is violence against its issuers and, by implication, its users. Political economy. The violence mythos has much in common with the tribute myth, insofar as the focus is on sovereignty. It can even be considered part of the same myth with the emphasis on protection from violence; money protects you from the vagaries of existence; it gets you out of trouble, or it can keep you from getting into trouble. It can also inflict violence and harm to others. Many who focus on the violence mythos are highly critical of the Euro, a monetary system that is powerless to avert violence in times of crisis.

MONEY AND SOCIAL STRUCTURE In this section, I follow the work of Carl Wennerlind (2001) on the way specific economic schools ‘hear’ what money is saying. The Austrian school hears a similar message as Classical economists, such as Hume, Ricardo and Smith; the Institutionalists and Chartalists hear an entirely different message, as do the Marxians, the postmodernists and the ‘Gold Bugs’, old and new.

SEMIOTICS IN ECONOMICS AND FINANCE

249

Heterarchical money: The Austrians Money communicates price and preference, signalling to others the opportunity for profit. When Henry shows the restaurateur his note and asks for change, he reveals to him not only that he can cover the costs of the meal at the reasonable profit mark-up but that he is likely to be a returning customer, a source of future revenue. The restaurateur and Henry form a spontaneous bond of trust, despite their erstwhile anonymity. The restaurateur does not even need to know his name; he is a participant within an exchange mechanism that resembles a language as universally agreed-upon mediator. Since Henry has an intrinsic interest in being initiated into the new social order in which he now finds himself, and the restaurateur likewise has an intrinsic interest in adding a new member to the fold, they both internalize the norms of the social order. For David Hume, Georg Simmel, Friedrich Hayek and most recently Steven Horwitz (1992), money is first and foremost a spontaneous distribution of trust among otherwise isolated individuals; for Hayek and Horwitz, the critical point is that markets emerge from the inherent disposition of individuals to truck, barter, and trade with money as the natural medium of preference, price and profit discovery. Markets are constituents of the natural order, which is not, in itself, amenable to some grand social plan. In this respect, the fact that Henry’s note bears the imprimatur of the Bank of England, and by extension, the Sovereign, is derivative. Money springs from the commercial instincts of individuals, just as language does (see Money as Language). For Austrian economists, money is social but not political. This view rests on the equivalence of money to language. Is language the correct analogy, however? A more accurate equivalent may be writing (more of which in the discussion section).

Hegemonic money: Institutionalists and neo-Chartalists Pecuniary cultures divide up the known world into commodities, producers and consumers. Each of these roles is constituents in the semiotic ‘code’ of capitalism. How does this code emerge? If you were to ask an Austrian economist, it emerges in the exchange process itself, with ‘trust’ as the inevitable by-product. This view essentially views all cultures through the lens of divisions of labour and market exchange: the ‘real’ economy. Money is the by-product of exchange. Full stop. Institutionalist economists since Thorstein Veblen have staked a quite different macroeconomic ground. Money performs multiple sociological functions within a society, with exchange being merely one of many. They argue that capitalists will always get capitalism wrong if they think that financial crises are somehow not intrinsic to pecuniary society. In essence, the business cycle depends on money as a store of value, which explains why those with money tend to hoard it (i.e. take it out of circulation) whenever they experience threats. If enough people with money to decide to save instead, society experiences a significant stagnation in aggregate demand, leading to debt deflation and depression. Crucially, money’s disappearance to the savings account, the mattress or the hole in the ground, deprives pecuniary society at large with a source of energy.3 Money is an institution – a set of rules and procedures that simultaneously enable and constrain social activities – that determine the conditions of social life beyond the narrow economic exchange. While Austrians consider trust as the output of monetary exchange, institutionalists regard monetary exchange as a manifestation of money: money is not a veil on barter, but a semiotic code for multiple lifeways within a monetized culture. Monetized culture takes many forms, not just capitalist.

250

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

To institutionalists like Dillard, Dyer and Veblen, pecuniary cultures engender Producers and Consumers. In capitalism, producers seek consumers through mechanisms that, optimally, turn consumers into debtors (e.g. auto dealerships provide or have an interest in the financing of a consumer’s new car). For institutionalists, the code of capitalism is the code of debt. The economy at large manages relationships of credit and debit under the guise of producer and consumer. Carsten Herrmann-Pillath (2013: 5, passim) further develops the institutionalist position on money from the perspective of evolutionary economics and conceptual blending; money constitutes an externalized, distributed system for blending the emotional dispositions of individuals with the social positioning of debtors and creditors within ‘institutionally guided behavioural patterns’, or IGBPs.4 If institutionalists are right, then money does not emerge meiotically from trader A and B, but parthenogenically from an organization that controls both sides of the ledger: it controls assets and liabilities at will. Enter the Chartalists. Chartalism and neo-Chartalism takes a similarly institutionalist and hierarchical view of money as a creature of the state. For over four thousand years, money has been a creature of the state (see Knapp (1924), Innes (1914), Wray (1998)), meaning that a sovereign determines the unit of account for which debts are denominated and then uses the power of taxation, fines, fees and tithes as the primary means of driving the currency. The reason you use a dollar, pound, euro, ruble, yuan or yen is not that you ‘think’ the person on the other side of your trade is slightly more gullible than you, but rather because you presume that the other person needs to use the credit so denominated to discharge her or his debts. Money becomes a universal solvent of exchange because it, and only it, can be used to pay taxes. (See money as tribute.) The interpretant of money is debt to the sovereign.

Monetary domination: The Marxians The institutionalists have much in common with Marxians, insofar as money is hegemonic. For Marxians, money signifies domination. The meaning of money in a capitalist economy is different from its meaning in primitive, archaic or feudal systems. In capitalism, money governs relations of labour, land and private property, and thus he who possesses money has the power to command labour and other resources to create commodities. What is more, land and labour themselves become commodified, such that the dispossessed are forced to sell their labour to the highest bidder. As with the institutionalists, money is debt, the holder of sufficient quantities (tied to the private accumulation of land and resources) commands social control. Money leverages power. Henry Adams, we should note, becomes an honoured member of London’s investment class: the possession of the note demonstrates his creditworthiness; his creditworthiness likewise gives him a pass into London’s capitalist cliques, which, in turn, allows him to pair speculative investors with his friend, Lloyd Hasting’s mining interest. Hasting himself can thereby buy labourers without having to spend a single day working in the mine. He is consuming their labour. For Marxians, capitalism allows consumers to consume the labour power of producers, turning producers into consumers under the thumb of capitalists. In short, money in a capitalist society is an expression of domination in which the real producers (labour) are colonized by parasites (capitalists) and, if left to fester, will be consumed thereby.

SEMIOTICS IN ECONOMICS AND FINANCE

251

Poststructuralist money: All money is counterfeit As with the Austrians, poststructuralist critical theorists tend to think of money as a language. Nigel Dodd (2014: 179–89) summarizes Jacques Derrida’s later writings on money, particularly Given Time. I, Counterfeit Money (1992a). As with languages, money is a dynamic system predicated on the infinite play of significations, such that pure meaning never emerges; meaning exists only in a state of perpetual insecurity. The meaning of a sign always gets its meaning to other signs (following de Saussure), but these sign relations take place in time, such that meaning is always and everywhere a measure of difference and postponement (hence difference, i.e. to at once ‘differ’ and ‘defer’), so it is never settled. It is quite beyond the scope of this chapter to evaluate Derrida’s theory of language, but only to point out that his view of money combines this poststructuralist theory of language with gift-giving (see the subsection, Money as a Gift). In Given Time (71–108), Derrida uses the story of two friends recounted in Baudelaire’s poem, ‘La Fausse Monnaie’ (‘Counterfeit Money’) as the subject of his discussion. In this story, the narrator and his friend are walking from a tobacconist’s shop and come upon a beggar. The narrator gives the beggar a small denomination coin, while his friend gives him a much larger denominated coin. The narrator quips to his friend that the beggar will undoubtedly be surprised by his generosity, to which the friend quips back, C’était la pièce fausse (‘It was the counterfeit coin’) and then mentioned in agreement, il n’est pas de plaisir plus doux que de surprendre un homme en lui donnant plus qu’il n’espère (‘there is no pleasure sweeter than surprising a man by giving him more than he had hoped for’). For the narrator, his friend wanted to, in effect, gagner quarante sols et le cœur de Dieu (‘gain forty sous and the heart of God’). Derrida then reflects on this dualism of a true versus false gift as precisely the logical contradiction subtending all money. If money is a form of gift-giving, then it cannot be a true gift, and thus is indistinguishable from counterfeit money. The logic goes something like this: for a gift to be a true gift, it must be given without any expectation or motive of reciprocity. According to Derrida, if the recipient owes something in return, it is no longer a gift (1992b: 165). The paradox of gift-giving is that a true gift can never show itself as a gift. The same paradox lies at the heart of money writ large. Both genuine and counterfeit money operate according to the same semiotic surfaces, such that the counterfeit can be counted as genuine, and the real note can be regarded as counterfeit. Remember, this was precisely Henry Adams’s initial worry in receiving the note: they will take one look at his raggedy appearance and assume it was fake and call the authorities, a question similarly posed by the brothers, Abel and Basil. As Dodd summarizes the Derridean position, money is essentially ‘the interplay of value, appearance, and effect’ (2014: 184), none of which is stable.

Money’s yoke: The metallic standard Where to find certainty? Not, of course, in the centralized authorities of government if you are Locke, Hume, Mill, Ricardo and most certainly not von Mises or Hayek. Money ‘answers all things’, says Locke, become of the ‘intrinsick [sic] value of Silver consider’d as money’ (1824 [1695]: 113). During the high-Victorian age of ‘sound money’, the £100,000,000 Bank-Note issued was not just a promise made by the Governor of the Bank of England in the name of Her Royal Highness, it was ultimately a promise to convert the nominal value thereof into an equivalent amount of gold. In the words of

252

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

John Stuart Mill, Money wields ‘a distinct and independent influence of its own when it gets out of order’ (quoted in Skidelsky 2018: 21), where ‘out of order’ typically means ‘too much’. To keep governments honest, especially in a republic of elected officials who are tempted to print more money in order to stay in power, money needs to be tied to an independent standard. Gold or silver makes excellent candidates, as they are durable but soft and easily denominated, resistant to oxidation, and scarce. Largely a nineteenth-century invention (Skidelsky 2018: 40–52), the gold standard was seen as the only real way to reign in government spending; it was the only way to secure proper exchange and trade among nations; and it was the backbone of the post-war Bretton Woods system of fixed exchange rates, where all other currencies were pegged to the US Dollar, and the US Dollar was pegged to gold at 35 dollars an ounce.5 Paeans to the gold standard rarely stop think about ontological quandary: what gives gold its value. For them, gold is value. As Ferry (2016): 76) has noted, gold has historically been linked to divinity and luxury, since as far back as Archaic Greece but most prominently through medieval Byzantine iconography. Byzantine iconography turns on the distinction between appearance and essence, and which has led to gold’s contested nature as ‘essence’ or ‘false idol’ (e.g. ‘The Golden Calf’). Thus, a commitment to precious metal is an implicit insistence in nature has value, and that once gold runs out, so does all value. According to Ferry, gold makes a paradoxical ‘semiotic claim’: it creates value ‘beyond semiosis’ (2016: 77). Gold has ultimate value becomes it exists beyond signification. The paradox of gold’s ultimate sign beyond signification is the implicit underwriter of Bitcoin (XBT), a distributed digital money system based on virtual gold. In this, the ultimate value is the block-chain code – trust the algorithmic replication of the code!6 In contrast to fiat currencies based on numerical infinity, the designers of Bitcoin have imposed a 21-million XBT, predicted to be reached in 2040, if not sooner. This inability to create XBT beyond a particular limit is supposed to assure us that its value will always be ‘as good as gold’, an attractive idea for anyone suspicious of institutionalized authorities, and the motives of would-be authoritarians. The monetary authorities cannot signify beyond the limiting powers of gold.

MONEY AS METAPHOR Money as language; money as writing A persistent theme in these discussions has been the tendency to equate money and language. Money is language underwrites many articulations of money as a means of social control, especially the Austrian and Classical views, where money more or less spontaneously arises whenever two or more people interact. When you do not know someone, and you begin to talk with him or her, you then start to build trust, at least for the duration of the exchange. In this view, money is the semiotics of exchange, just as language is the semiotics of exchange. Money supplements the exchange. That Money is a form of semiotics is clear, but I would like to push back a bit on the money/language equation. Once again, ‘anyone can adopt money’, says Hyman Minsky, ‘the difficulty is getting it accepted’. Minsky’s quip points to a fundamental problem of large-scale sovereign money systems. How do they develop? As we speak, many developing countries have sovereign currencies that do not and cannot provide for the needs and wants of its citizens: Zimbabwe dollars cannot buy all or even most of the food

SEMIOTICS IN ECONOMICS AND FINANCE

253

and energy of its citizenry. They must import these goods and pay for them in US dollars, which further weakens the currency, such that its exchange rate continues to drop relative to preferred reserve currencies; by importing food and energy, they import inflation. Moreover, they must repay their debts in a different currency. What does this suggest? The idea that money systems are fundamentally spontaneous dialects is, at best, partly right. Where there is human language, a dialect of language will emerge. This much is true, but it is not nearly sufficient. We are on firmer semiotic ground, I contend, by regarding money as something closer to writing than language as such. For money to emerge as a stable entity, it needs to be political (this in contrast to the advocates of Bitcoin who fervently wish money to be apolitical), and thus institutional. Money is more like writing in that it emerges from the institutional needs to distribute communicative intentions across time and space, and to ensure that the means of doing so is sufficiently standardized. A distributed standard takes more than spontaneous trust; even as spontaneous trust relations are an essential part of the process. Two reasons suggest that writing is a more precise semiotic analogue than speech. The first reason is historical. From the best of our knowledge, money systems developed almost precisely around the same time and place as writing, approximately four to five thousand years ago in Mesopotamia. What is more, current evidence strongly suggests that writing systems began principally as accounting systems for tracking numbers of goods bound for trade (see Schmandt-Besserat 1996), and that these symbols were developed as weights and measures under sovereign control, suggesting that money and writing are two sides of the same coin, so to speak. While speech may be a necessary condition for writing, it does not appear to be a sufficient condition. On historical-cultural grounds, the semiotics of money should not be predicated on language as a necessary and sufficient condition for its widespread development. The sufficient conditions of writing and money are both dependent on specific types of institutional and organizational imperatives, imperatives that are political through and through. The second reason is neurocognitive. A widely shared view within cognitive neuroscience is that the human neocortex is partly specialized for cultural activities, such as reading and arithmetic, that developed too late to have influenced genetic evolution. According to Stanislaw Dehaene and Laurent Cohen (2011, 2007), the capacity for representing letter strings and numbers occupies extensive cortical real estate of the left occipitotemporal and bilateral intraparietal cortex, in which these later cultural conventions invade or reuse evolutionarily older circuits for new purposes. More specifically, Dehaene and his colleagues have identified the left occipitotemporal sulcus as the visual word form area (VWFA) as a critical region in a neural circuit subtending reading-specific processes (including number identification) in an area associated iconic image processing. In contrast, the bilateral horizontal segment of the intraparietal sulcus (HIPS) appears to be the core number area for working with numbers. A reasonable conjecture is that these cortical regions work in coordination with core language-related areas when reasoning about the quantitative aspects of money. The take-away point is not that there is a ‘money module’ distinct from other domains of activity but to underscore the phylogenetic and ontogenetic consanguinity of with present evidence for the mutual emergence of writing and money systems in human history. Writing and money is, therefore, a more precise equivalent than money and language or speech, as it allows for more considerable discussion of activities that are distributed among communities in both time and space in a macroeconomy, while the emphasis on money as speech tends to focus almost exclusively on the microeconomic propensity

254

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

to truck, barter, and exchange, which can then lead to the fallacy of composition: the macroeconomy is the sum of all microeconomic activity. There is one potential area where money-speech equivalency has probative value for economic theory: monetary ‘leaks’. As Edward Sapir (1921: 39) notes, ‘all grammars leak’, meaning that dialectical innovations are always happening and, thus, the broader speech community will always produce ‘unsanctioned’ forms of speech. Money is leaky, too. That is, the government’s issuing of base money can be controlled through various monetary and fiscal operations (namely spending and taxation), but only to a point, as significant sums of ‘broad’ money in the form of loans, securities, derivatives and other financial innovations escape the control of the money authorities. For example, the quantitative easing programs in the United States, the United Kingdom and Europe from 2008 to 2016 were intended to (1) produce inflation that would (2) induce businesses and individuals to invest and banks to increase loans; in effect, to increase economic output through the banking pipeline, with relatively even distributional effects across the populations. Instead, QE never allowed the monetary authorities to reach their inflation targets because the money injected into the system ‘leaked’ out of the real economy into the reserve accounts of banks and, by extension, the asset portfolios of wealthy investors, who preferred to save their money rather than invest it. Money can leak out into a range of niches so that it is never under the kind of control supposed by the Quantity Theory of Money (as supply-side theory). In this respect, it is useful to think of money as a kind of spontaneous patois useful for specific purposes, and that it cannot be entirely and tyrannically consistent. Nevertheless, money as a sovereign system underlying political economies has much in common with authoritative writing systems that enforce and maintain standardization across communities and generations, even as there can be no perfectly controllable money system.

CONCLUSION: A BRANDTIAN SEMIOTICS OF MONEY I began this chapter just as many semiotic accounts begin by referencing a fanciful fictional story of money in use and the paradoxes it reveals. This is because our most easily graspable experiences with money are in second-person exchanges. As with much of human cognitive and social life, the second-person perspective is a fruitful way to begin. As I have argued most recently (Oakley 2020), these second-person engagements can lead us astray in grasping the dynamics of large social systems. The drama of Henry Adams is most saliently a drama of monetized exchanges between persons and classes, and readers must work pretty hard to ferret out features of the money system as such. Brandt (2017) has done semiotic theory a real service by focusing less on the exchanges themselves and more on the ecological prerequisites of money as a signifying system. For Brandt, we need to start thinking about money as part and parcel of the process for staving off entropy. Negentropy leads Brandt to propose an ecological scheme of three concentric ‘loops’: the organic, the technological and the symbolic. Organically, we are all beings living in a niche and drawing sustenance from it, beginning with sunlight. Technologically, we are beings who can construct additional facets of the niche to process the goods of the earth into objects and systems of higher power and use (e.g. farming, mining, metallurgy) that gives us ‘surplus value’. Symbolically, we endow these transient organic and technological fruits with ‘spiritual’ powers. (2017: 3). For Brandt, the meaning of money resides in the dynamic interrelations of these three extraction

SEMIOTICS IN ECONOMICS AND FINANCE

255

loops. In this sense, all we need do is posit social agents who collaborate to create near- to medium-term sustainable environments of laminated organic, technological and symbolic extraction. If Brandt is correct, and I think he is, then the semiotics of money and finance should focus precisely on the intersections of these three loops when thinking about money as large-population-wide social systems. Money can never be wholly symbolic, as there are organic and technological limits, as even a sovereign with numerical infinity on its side will come up against real resource limits. Money cannot be wholly organic, as gold and other precious metals have never counted as money in the absence of numerical and technological systems. In essence, the gold (organic) comprising the Lydian Stater is not money without an indexed nominal value (symbol) and authenticating stamp (technology). Money is political precisely because its meaning emerges from these vital social extraction loops that make up human life. The advantage of studying money in the manner broadly outlined in this chapter is that it offers an opportunity for semiotic inquiry to interject itself in the important political and economic problems of our time. At a time when policy debates and disagreements stem from stakeholders holding radically different conceptions of money, often without realizing that many of their disagreements stem from these different conceptualizations of this most enigmatic but consequential social structure, the study of money a few steps removed from this partisan wrangling might aid expert and laity alike in becoming wiser citizens of monetized societies.

NOTES 1 An anthropology of exchange, most notably conducted by Marcel Mauss (1990 [1925]) on gift giving, has long noted and granted much consideration of money as a social semiotic phenomenon. 2 Keynesians and Post-Keynesians refer academic economists who follow the English economist, John Maynard Keynes (1883–1946). Post-Keynesians (e.g. Robert Kahn, Michal Kalecki, Joan Robinson) comprise academic economists associated most prominently with Cambridge University who have ‘rescued’ the ‘real Keynesian’ approach from the adulterated version integrated with the neo-classical, ‘New Keynesians’ (e.g. John Hicks, Paul Samuelson, Paul Krugman). Marxian refers to economists (e.g. Duncan Foley, David Harvey, Rosa Luxemburg) who distinguish Marx’s economic theory from his political ideology. 3 This is precisely in line with Keynes’s argument in the General Theory of Employment, Interest, and Money (1936: 222), on the ‘paradox of thrift’ – in which excess savings lead to under-investment and falling aggregate demand. 4 The precise details of Herrmann-Pillath’s evolutionary approach to institutional economics are well beyond the scope of this chapter, but I do consider it an essential and promising integration of economics, cognitive science and semiotics. 5 Keynes was influential in the design of Bretton Woods, as he regarded a system of fixed exchange rates to be optimal for the post-war period of economic reconstruction. However, the system was not thoroughly Keynesian, as he thought the gold standard was foolish, and that the system should adopt what he called ‘the Bancor’, a supranational currency to act as a multilateral clearing system that would even out the balance of trade accounts between debtor and creditor nations. The idea of the Bancor proposal has been revived in recent years. 6 Bitcoin is but one type of digital currency, also known as crypto currency. Crypto, because the possession of any denomination of XBT occurs through a unique, anonymous identifier

256

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

that, if lost to the holder, means that it cannot be retrieved. The technical architecture of these digital currencies is known as ‘block-chain’, an algorithm allowing distributed ‘Bitcoin Miners’ to verify and certify transactions, allowing the system to bypass private and central banks. As of this writing most central banks are now beginning to use their own form of distributed block-chain technologies, and it turns out that ‘Bitcoin mining’ is energy intensive, resulting in excessive burning of fossil fuels in China, where most of the miners reside.

REFERENCES Bingham, C. (2014), ‘Money Talks: A Semiotic Exploration of Money as Communication’, in Kristian Bankov (ed.), Proceedings of the World Congress of the IASS/AIS, 165–72, Sophia: New Bulgaria University Press. Brandt, P. A. (2017), ‘The Meaning and Madness of Money: A Semio-Ecological Approach’, Cognitive Semiotics, 10 (2): 141–68. Dehaene, S. and L. Cohen (2007), ‘Cultural Recycling of Cortical Maps’, Neuron, 56 (2): 384–94. Dehaene, S. and L. Cohen (2011), ‘The Unique Role of the Visual Word Form Area in Reading’, Trends in Cognitive Science, 15 (6): 254–60. Derrida, J. (1992a), Given Time. I, Counterfeit Money, trans. P. Kamuf, Chicago: University of Chicago Press. Derrida, J. (1992b), ‘Given Time: The Time of the Kind’, trans. P. Kamuf, Critical Inquiry, 18 (2): 161–87. Dodd, N. (2014), The Social Life of Money, Princeton: Princeton University Press. Ferry, E. (2016), ‘On Not Being a Sign: Gold’s Semiotic Claims’, Signs and Society, 4 (1): 57–79. Graeber, D. (2011), Debt: The First 5000 Years, Brooklyn: Melville House. Herrmann-Pillath, C. (2013), Foundations of Economic Evolution, Cheltenham: Edward Elgar. Horwitz, S. (1992), ‘Monetary Exchange as an Extra-Linguistic Social Communication Process’, Review of Social Economy, 50 (2): 193–214. Innes, A. M. (1914), ‘The Credit Theory of Money’, The Banking Law Journal, 31: 151–68. Jakobson, R. (1990), On Language, eds. L. Waugh and M. Monville-Burston, Cambridge, MA: Harvard University Press. Keynes, J. M. (1936), General Theory of Employment, Interest, and Money, New York: Harcourt, Brace. Knapp, G. F. (1924), The State Theory of Money, 4th edn. London: Macmillan. Kurke, L. (1999), Coins, Bodies, Games, and Gold: The Politics of Meaning in Archaic Greece, Princeton: Princeton University Press. Lawson, T. (2015). ‘Social Positioning and the Nature of Money’, Cambridge Journal of Economics, 40 (4): 961–96. Locke, J. (1824 [1695]), ‘Further Considerations Concerning Raising the Value of Money’, in The Works of John Locke in Nine Volumes, Volume 4, London: C. Baldwin, Printer. Mauss, M. (1990 [1925]), The Gift, trans. W. D. Halls, New York: W.W. Norton. Oakley, T. (2020), Rhetorical Minds: Meditations on the Cognitive Science of Persuasion, Oxford: Berghahn Books. Sapir, E. (1921), Language, an Introduction to the Study of Speech, New York: Harcourt Brace. Skidelsky, R. (2018), Money and Government: The Past and Future of Economics, London: Penguin.

SEMIOTICS IN ECONOMICS AND FINANCE

257

Schmandt-Besserat, D. (1996), How Writing Came About, Austin: University of Texas Press. Twain, M. (1996 [1893]), ‘The £100,000,000 Bank-Note’, in The £100,000,000 Bank-Note and Other New Stories, Oxford: Oxford University Press. Veblen, T. (1899), The Theory of the Leisure Class, an Economic Study in the Evolution of Institutions, New York: Macmillan. Waller, C. (2012), ‘The Semiotic Theory of Money’, Economania. https://www.economania. co.uk/chris-waller/semiotic-theory-money1.htm. Wray, L. R. (1998), Understanding Modern Money: The Key to Full Employment and Price Stability, Cheltenham: Edward Elgar. Wennerlind, C. (2001), ‘Money Talks, but What Is It Saying? Semiotics of Money and Social Control’, Journal of Economic Issues, 35 (3): 557–74.

258

CHAPTER ELEVEN

Semiotics in Law and Jurisprudence CLARA CHAPDELAINE-FELICIATI

INTRODUCTION What is legal semiotics? How does it relate to semiotics more broadly? The concept of semiotics in law appears foreign to many lawyers and semioticians. Some may view legal semiotics as a special branch dedicated to the study of law within the science of semiotics, others as a sub-discipline within legal theory, and finally others as an interdisciplinary method to explore the legal realm. More specifically, does semiotics of law constitute a discipline, a meta-discipline, a doctrine or a school of thought, and how does it involve legal philosophy, legal positivism, legal realism, legal hermeneutics and critical legal theory? This chapter begins by providing a history of semiotics in law and jurisprudence, from influential modern semioticians who examined semiotics while also referring to the concepts of law and justice, notably Victoria Welby and Charles Sanders Peirce, whose theories were applied in the legal field, to Roberta Kevelson’s significant undertaking to disseminate and promote the raison d’être of legal semiotics. It offers a definition of legal semiotics and explores the conceptual notions of law, jurisprudence and the construal and assertion of rights. The chapter then turns to contemporary challenges in the field, and discusses that while numerous legal scholars explore the meaning and interpretation of legal texts, very few apply semiotics as a science and method. It notes a shift towards the recognition of semiotics of law as a new field of study. The chapter concludes with a discussion on future priorities.

MODERN SEMIOTICS AND THE LAW: WELBY AND PEIRCE Welby and Significs Victoria Lady Welby, the foremother of modern semiotics, explored various conceptual notions within this wide-ranging field, notably law, order and meaning-making. She noted in her correspondence with the lawyer and philosopher Norman Pearson (1886–8) ‘To me law and order are essential conditions of freedom; lawlessness or anarchy connotes the worst of slaveries to your own or others’ arbitrary wilfulness; it is liberty versus licence’ (Petrilli 2009: 176). Welby also discussed these concepts in a letter to Peirce (20 November 1904) in which she considered that ‘What is Order’ was the twin interrogation of ‘What is Meaning’ (Petrilli 2009: 390).

260

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

In her 1907 paper Primal Sense and Significs, Welby goes further by underscoring the distinction between natural law and legal positivism: ‘We talk of natural “law”; reducing its sphere to that of the “law-court” with its imposed decisions, forgetting that a law is a rule deliberately decreed and enforced or “passed” by consent and liable to abrogation’ (Petrilli 2009: 575). In this context, she underscores that law as a system of signs is not fixed and can indeed be modified. She nevertheless cautions against amending any law without careful consideration. In her letter to Ogden (16 May 1911), she declares: ‘Again you have it in law: the enormous growth of mechanical aids may make some laws unbearable burdens or almost practical jokes: but they must be repealed or modified with the utmost precaution and by experts, or you will engender “heresy” and lawabiding men will become lawless insurgents bringing social order into content’ (Petrilli 2009: 778–9). It is worthy of note that Welby, in her quest for precise language, refers to the international legal framework and the concepts of war and peace. In Grains of Sense (1897), she shows great concern for the possible effects of misinterpretation and mistranslation: ‘The mutual deafness, dumbness and blindness which is the mental condition of our “Modern Babel”, but which leaves us only too free for mutual collision, quarrel and destruction, must indeed make terribly for war, and even, through mutual hatreds thus engendered, for reversion to barbarism’ (1897: 138–9). She calls for the establishment of an International Court of Voluntary Appeal on all questions of expression, or an International Court of Linguistic Arbitration, to solve the ‘linguistic war’ to which we all fall victims (1897: 82–3). Welby argues that the language of expression, if perfected, constitutes a signifier or diplomatic weapon that is likely to address conflicts in the international order: [I]f the forces of international law and order are to triumph over those of anarchy and mutual extermination as they have already done within the borders of each national organisation, it must be done by arming them with weapons of expression of far greater power and perfection than any yet attained. (1897: 140) She contends that the ‘peace-maker of the future’ shall need this universal language to reach the hearts and minds of interpretants globally (1897: 140–1). Welby thus appears to consider language as a sine qua non condition for the establishment of law and peace, and arguably, the rule of law. It is worthy of note that her appeal occurred shortly before a century plagued with warfare, genocides, crimes against humanity and war crimes, as if she foresaw the importance of promoting goodwill and reconciliation. Her plea would be taken on by semioticians and significists in the field of international law a century later. Welby also showed concern for the way lawyers can shape meaning to their own advantage. In The Demand for Evidence (1906), she notes that the signs and symbols embodied in the notion of evidence can be untruthful: ‘And the first thing, the justifying thing, the true credential which the herald must have; that without which all other gifts would be in this case barren, is a keen sense of the value of evidence. And this not merely in the lawyer’s sense (as too often an asset to be manipulated) but in the scientific sense’ (Petrilli 2009: 812). She goes further in her Essay Truthfulness in Science and Religion (1888) by asserting that while ‘the obligation of the lawyer to give a sound opinion’ is a professional instinct, lawyers have often been ‘scoundrels’ and can be ‘unveracious’

SEMIOTICS IN LAW AND JURISPRUDENCE

261

(Petrilli 2009: 201). In this context, she questions the very integrity of lawyers as professionals willing to use any means to their end, including deception.

Jacob de Hann Welby’s theory on significs had a strong bearing on the development of legal semiotics in The Netherlands, most notably in the works of renowned lawyer, poet, semiotician and human rights advocate Jacob Israël de Haan (1881–1924). In his letter dated 11 February 1912, Frederik van Eeden wrote the following to Welby: Mr. De Haan is the most marvellous conquest you made. I gave him your book when he visited me (Significs and Language) hoping he would like it. And he was so enraptured that he could speak of nothing else the whole evening. He drank it in, like a thirsty man, water. Now he is going to startle the Professors of Law in Amsterdam by quoting you and writing articles about you in the Law reviews. (Petrilli 2009: 796) Van Eeden had met Welby at a Conference in 1892 and they had been corresponding ever since, sharing a common interest on the questions of meaning, language and communication. In this regard, Mannoury argues that van Eeden was the first scholar to transpose Welby’s theory on semiotics in The Netherlands, and to his field of expertise, psychiatry and psychology (Petrilli 2009: 754–5). Jacob De Haan followed the same path, focusing instead on the implementation of significs in the legal realm. De Haan applied Welby’s Meaning Triad (sense, meaning, significance) as a framework to analyse legal language in his essay ‘New Philosophy of legal language’ (1912), which he translated into Dutch as Zin (sense), Bedoeling (intention) and Waarde (value), as well as in his doctoral dissertation in 1916 (Petrilli 2009: 757). It is worthy of note that a hundred years later, Welby’s Law of the Three Stages was applied in another dissertation in law, this time at the University of Oxford (DPhil in Law, Chapdelaine-Feliciati 2016), due to Petrilli’s pivotal work on Welby’s opus and legacy (Petrilli 2009; Petrilli, Nuessel, and Colapietro 2013; Petrilli 2015). De Haan, on the occasion of the death of Welby, encapsulated her theory in the following terms (in Dutch): ‘The authoress characterizes significs as: the philosophy of Significance, that is: the philosophy of the human capacity of expression’ (Petrilli 2009: 256). He was interested in the volitional aspect of semiotics and notably the ethical responsibility of Master signifiers (Cyran 2011: 120) in the legal field. De Haan was very prolific in his study of legal significs, publishing two books and forty articles on the subject. In addition, he occupied the first Chair in Legal Significs and taught this discipline as a lecturer at the University of Amsterdam (Petrilli 2009: 758; Broekman and Fleerackers 2018: 65). Most notably, he coined the expression ‘legal significs’ to describe the study of semiotics in legal theory and legal practice (Broekman and Fleerackers 2018: 23). His work was regrettably interrupted when he moved to Jerusalem and was assassinated, on political grounds, shortly thereafter (Petrilli 2009: 758).

Peircean semiotics Peirce’s opus does not appear, at first glance, to explore legal semiotics. However, his writings do at times bear on the law, and his theories have been pursued by legal semioticians. As noted by Kevelson: ‘abundant material on law exists throughout [ … Peirce’s] voluminous writings. But one must search and be prepared to recognize allusions

262

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

to law even when mention of law is not explicit’ (1987: 14). In this regard, scholars such as Tiefenbrun consider that the conceptual notion of law permeates each part of Peirce’s philosophy: ‘in phenomenology, law is the continuity of experience; in logic, law is an operative symbol; in metaphysics, law is efficient reasonableness’ (2010a: 36). Peirce firstly notes that the notion of ‘right’ belongs to the legal realm, and underscores its association with the concepts of thought and meaning: ‘Now right is a matter of law, and law is a matter of thought and meaning’ (1903: CP 1.345). He also examines the concept of force without law or reason, or brute force (1896: CP 1.427). He notes that ‘It may be said that there is no such phenomenon in the universe as brute force, or freedom of will, and nothing accidental’, but continues ‘it still remains true that considering a single action by itself, apart from all others and, therefore, apart from the governing uniformity, it is in itself brute, whether it show brute force or not […] it is possible for a phenomenon in some sense to present force to our notice without emphasizing any element of law’. Peirce however argues that there is always an element of law, whether we perceive it or not: ‘That when more is taken into account, the observer finds himself in the realm of law in every case, I fully admit’ (1896: CP 1.428). In his exploration of the powers of reasoning, Peirce notes that the two notions – power and reason – are not inextricably linked. He illustrates this absence of correlation by noting that although lawyers are perceived as notables in society, they are not necessarily the strongest or more talented reasoners: For though the men who are most extraordinarily successful evidently do reason deeply about the details of their business, yet no ordinary degrees of good success are influenced – otherwise than perhaps favorably – by any lack of great reasoning power. We all know highly successful men, lawyers, editors, scientific men – not to speak of artists – whose great deficiency in this regard is only revealed by some unforeseen accident. (1898: CP 1.657) In this regard, he seems to share Welby’s lack of admiration for lawyers as a general category of professionals. He also refers to the legal process to illustrate how our perceptions can be skewed, and that signs and symbols may signify more than the human mind can perceive: ‘Every lawyer knows how difficult it is for witnesses to distinguish between what they have seen and what they have inferred’ (1868: CP 5.216). As was the case with De Haan, Peirce underscores the relationship between ethics and law, noting however that the former is not systematically considered in the latter: ‘Ethics is courteously invited to make a suggestion now and then in law, jurisprudence, and sociology’ (1902: CP 1.251). Peirce also employs legal signs and symbols to illustrate his theory on human conscience and decision-making: Conscience is like our Supreme Court, which intends to frame its decisions according to the principles of law. But when it has decided a point, its decision becomes law, whether the wisest counsels would have maintained it or not. For the actual law consists in that which the court’s officers will sustain. But according to the English logicians it is otherwise with rationality. Every reasoning holds out some expectation […] For the sole purpose of reasoning is, not to gratify a sense of rationality analogous

SEMIOTICS IN LAW AND JURISPRUDENCE

263

to taste or conscience, but to ascertain the Truth, in the sense of that which is SO, no matter what be thought about it. (c. 1902: CP 2.153) He appears, in this context, to equate the modus operandi of the supreme institutions of legal meaning-making with our own faulty mental mechanisms of rationalization. Supreme Court Justices employ legal principles that provide a justification for their judgments, present and future, which in turn create precedents that ensure the perpetuation of this self-fulfilling process of validation. Peirce emphasizes that such judgments are adhered to ‘whether the wisest counsels’ would have maintained them or not. Instead, the logician employs the notion of reason for the laudable pursuit of truth, regardless of whether the result shall comfort one’s sense of right and wrong. In his description of the third trichotomy of signs, and more specifically of the Argument, Peirce once again compares our own judgment to that of the legal system: What is the essence of a Judgment? A judgment is the mental act by which the judger seeks to impress upon himself the truth of a proposition. It is much the same as an act of asserting the proposition, or going before a notary and assuming formal responsibility for its truth, except that those acts are intended to affect others, while the judgment is only intended to affect oneself. (1903: CP 2.252) In this regard, the logician is not concerned with the psychological aspect of the act of judging, but rather with the nature of the sign that is a proposition, and which retains meaning, whether it is asserted or denied. Lastly, and most importantly, Peirce introduces the concept of the Legisign, which he defines as follows: ‘A Legisign is a law that is a Sign. This law is usually established by men. Every conventional sign is a legisign [but not conversely]. It is not a single object, but a general type which, it has been agreed, shall be significant. Every legisign signifies through an instance of its application, which may be termed a Replica of it’ (1903: CP 2.246). Within this first trichotomy of signs, he compares the Legisign to the Sinsign which is a single sign that is an actual existent thing or event (1903: CP 2.245) and the Qualisign as a quality that is a sign and cannot actually act as a sign until it is embodied (1903: CP 2.244). The Legisign is also a symbol, which belongs to the second trichotomy of signs (Icon, Index and Symbol) in as much as the latter is a general type of law. Moreover, within his third trichotomy of signs (Rheme, Dicent Sign, Argument), he observes that an Argument is a sign which, for its interpretant, is a Sign of Law (1903: CP 2.252), thereby emphasizing the role of the interpretant in meaning-making. When discussing Welby’s Threefold Laws of Meaning, Peirce refers to his other triadic classification of the immediate, dynamical and final interpretants, the latter defined as the ‘interpretation if consideration of the matter were carried so far that an ultimate opinion were reached’ and which he associates with Welby’s third level of analysis: ‘significance’ (n.d.: CP 8.184). Peirce also perceives the sign in a triadic relationship between the object (the dynamical object as it is in reality), the sign or representamen (the object as it is represented by the sign) and the interpretant, another sign that gives meaning to the representamen. He notes that the sign is a representamen with a mental interpretant (1903: CP 2.274). Peirce’s Legisign and his triadic conceptions of the sign were thereafter

264

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

applied by legal semioticians, such as Kevelson who devised her own conception of legisigns as ‘provisional judgments, held and acted on as if they were truths, although they are in fact the product of an ad hoc community that comes together out of common purpose so long as it is certifiable, verifiable, useful’ (Kevelson 1996: 51).

Oliver Wendell Holmes Jr. and the Metaphysical Club According to Kevelson, Peirce’s theories had a strong influence on Oliver Wendell Holmes Jr. (1841–1935), who went on to become a US Supreme Court Justice. Indeed, it is assumed that they exchanged ideas through their association at the Metaphysical Club, a discussion group that held several meetings in Cambridge, Massachusetts in the early 1870s. Also, Holmes’ correspondence with Laski discusses Peirce’s writings (Kevelson 1988b: 14). Kevelson contends that when Holmes was appointed to the US Supreme Court in 1902, he brought with him Peirce’s views in that his decisions are embedded in the ‘logic of search and discovery’, which is a logic of inquiry, an expanded logic that ‘goes far beyond syllogistic reasoning in its attempt to account for the actual process whereby all thinking persons may interpret their ideas in order to act upon them and thus to verify them with reference to the actual world’ (1988b: 13). For instance, Holmes’ declaration that the Constitution of the United States is an ‘experiment, as all life is an experiment’, appears to follow Peircian philosophy (Kevelson 1988b: 14). Moreover, Holmes emphasizes the importance of legal terminology in several judicial decisions and scholarly publications: If a man goes into law it pays to be a master of it, and to be a master of it means to look straight through all the dramatic incidents and to discern the true basis for prophecy. Therefore, it is well to have an accurate notion of what you mean by law, by a right, by a duty, by malice, intent, and negligence, by ownership, by possession, and so forth. I have in my mind cases in which the highest courts seem to me to have floundered because they had no clear ideas on some of these themes. (1897: 475) Holmes, who is known for his precise and meaningful opinions, is one of the most widely cited Supreme Court justices (Shapiro 2000: 423–4), and he acted as a Professor at his alma mater Harvard Law School, where several influential lawyers pursued their studies. Accordingly, his appreciation for Peirce’s principles and concepts could in this sense have greatly influenced American Common Law. Kevelson nevertheless acknowledges that these significant associations between Peirce and Holmes ‘are not sufficient in themselves to claim that Peirce’s ideas directly inform Holmes’ contributions to constitutional law’ and therefore suggests to explore their resemblance as concerns cause, discovery, interpretation (1988b: 14). Kevelson also considers that Peirce’s opus had an impact on the German-American jurist Francis Lieber (1798–1872), Professor at Columbia University and author of the Lieber Code which formed the basis of the first laws of war (Kevelson 1982: 64; Broekman and Fleerackers 2018: 64). She concludes that ‘Peirce’s semiotic philosophy was decisively instrumental in the development of much of both continental and Anglo-American law’ (1988b: 8). However, it is not until the late 1970s that the field of legal semiotics generated a renewed interest, notably among American scholars.

SEMIOTICS IN LAW AND JURISPRUDENCE

265

ROUND TABLES, RESEARCH CENTRES AND INTERNATIONAL JOURNALS The 1980s and 1990s were rich with creativity and meaning-making in legal semiotics. The First Symposium on Law and Semiotics, which discussed Peirce, Morris, Saussure and Eco, but not Welby, was held at the annual meeting of the Semiotics Society of America in 1983 (Kevelson 1987: 11). The following year, Kevelson founded The Penn State Center for Semiotic Research in Law, Government and Economics to supplement and explore Peirce studies, establish its mediative function for practical law, as well as provide a locus for conferences and publications (Kevelson 1987: 2). Additionally, Kevelson co-founded and sponsored the International Association for the Semiotics of Law, which provides newsletters on events as well as publication reviews, and the interdisciplinary International Journal for the Semiotics of Law (IJSL: 1988), which both still exist (Broekman and Fleerackers 2018: 25). She considered that ‘Law and semiotics is now in that stage of the process which is, in brief, agreeing to come to agreement. It is this initial agreement to which all subsequent dialogue will have reference, and which will possibl[y] establish the process of a legal semiotics as triadic’ (1987: 24). She would go on to organize eleven Round Tables on Law and Semiotics (1987–98) that brought together researchers with different approaches to the study of law, including philosophy of law, traditional jurisprudence, law as rhetoric, critical legal theory and social sciences perspectives on law. As noted by Kevelson: ‘scholars who came together to share and to explore would otherwise not be participants under the same roof. Semiotics provided the bridge; law was the common language’ (1988a: 4). Legal semiotics also became more prominent as a theory in legal scholarship and education. Duncan Kennedy, an important figure in Critical Legal Theory at Harvard Law School, participated in the Round Tables, where he discussed the semiotics of legal argument, and also incorporated – to some extent – legal semiotics in his research, notably an examination of Saussure’s ‘rationalist semiotics’, and Derrida’s ‘irrationalist semiotics’ of deconstruction (1989, 2001: 1175–82). As a former CIA agent, he was most probably aware of the power of decoding the master discourse. Balkin also attended the Round Tables, where he established a parallel between Saussure’s work in linguistics and the Hohfeldian approach to analytical jurisprudence (1989). Indeed, Hohfeld, in his examination of fundamental legal conceptions not only quotes Justice Holmes’ judicial decisions, but also underscores the power of language in law: Anyone who has seriously observed and reflected on the interrelation of ideas and language must realize how words tend to react upon ideas and to hinder or control them. More specifically, it is overwhelmingly clear that the danger of confusion is especially great when the same term or phrase is constantly used to express two or more distinct ideas. (Hohfeld 1917: 715) Balkin and Paul also examined, respectively, the Promise and the Politics of legal semiotics and the importance of incorporating semiotics in legal education, notably in core law courses (Balkin 1991; Paul 1991). At Penn State, Broekman pursued Kevelson’s tradition and designed and taught the Kevelson Seminars on Law and Semiotics, a three credit course offered at the Dickinson School of Law that explored legal semiotics in

266

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

different legal systems, notably Civil Law and Common Law, in an egalitarian perspective whereby students actively participate in exchanges on readings and write their own perspectives on the contribution of legal semiotics to teaching in law. This led to two Special Issues in the International Journal for the Semiotics of Law (Broekman and Pencak 2009, 2010).

LEGAL SEMIOTICS Semioticians and legal scholars often refer to the expressions ‘legal semiotics’ and ‘semiotics of law’ interchangeably, and views as to their respective meanings differ. Moreover, there are conflicting theories as to the origins of the expression ‘legal semiotics’. Roberta Kevelson was fascinated by the discipline she named ‘semiotics of law’ which she perceived as a special branch within the humanities that focuses on its object of study – ‘legal semiotics’ – a term she arguably introduced, possibly inspired by De Haan’s writings on legal significs, although she does not refer to him in this context (Broekman and Fleerackers 2018: 22–3). Balkin considers that Hohfeld was the first ‘legal semiotician’, while acknowledging that the latter was not consciously applying semiotics as a method (1989: 31–2). Additionally, Paul contends that Balkin and Kennedy coined the term ‘legal semiotics’ (Paul 1991: 1798). It is worthy of note that Welby is not mentioned in this context as a central semiotician and significist whose theories prompted the discipline of modern legal semiotics (Broekman and Fleerackers 2018: 65, 23). In terms of definitions, Tiefenbrun describes semiotics of law as ‘a specialized study of sign systems underlying legal informational exchanges’, based on Sebeok’s basic definition of semiotics as ‘the exchange of any messages whatever and of the systems of signs which underlie them’ (2010a: 24). Broekman, Mootz and Pencak refer to the subject of legal semiotics as ‘the scientific approach that regards law as a system of signs and meanings’ (2011: v). Following Welby’s and Peirce’s triadic approach to signs, it could be argued that legal semiotics comprises three central characteristics: (1) it studies the legal realm as an infinite sign system, (2) offers a method or methods to perform this ambitious undertaking and (3) provides a common language for scholars and practitioners. As concerns the first element – that of substance – of legal semiotics, it is pertinent to examine the concepts of law and jurisprudence. The term law is defined as ‘[t]he body of rules, whether proceeding from formal enactment or from custom, which a particular state or community recognizes as binding on its members and subjects’ (Oxford English Dictionary 2011). Hence there are two aspects to this definition: the term law refers to (1) a body of rules (2) that have been enacted by social institutions or master signifiers, be it a legislative, executive or judicial authority. In this regard, the notion of law, as presently described, is linked to positivism rather than to natural law: in order to be valid, laws or customs need not be moral, fair, or even divine. Their value rests in that they are ordained by the relevant authority (Davies 2008). The expression jurisprudence is inextricably linked to that of ‘the law’. Jurisprudence is defined firstly as a ‘knowledge of or skill in law’, secondly as a ‘science which treats of human laws (written or unwritten) in general’ and as ‘the philosophy of law’, and thirdly as a ‘system or body of law; a legal system’ (Oxford English Dictionary 2011). Moreover, it can refer to ‘case law’, thus to a body of court decisions, more specifically as ‘a collective term denoting the course of judicial decision, i.e. case law, as opposed to legislation’ (Gifis 1998). Hence the expression jurisprudence is offered both as an alternative to the term ‘law’, and to

SEMIOTICS IN LAW AND JURISPRUDENCE

267

the science that studies or understands the law. The latter meaning is often employed in legal theory as a synonym to ‘the philosophy of law’. Hence in the context of this chapter, jurisprudence is understood as the science that studies the law, or the ‘philosophy of law’. Central to the notions of both law and jurisprudence is that of individual and collective ‘rights’. While the latter expression has many meanings which could be the object of another study, it is pertinent to note that in the legal context ‘right’ usually refers to legal entitlements and justifiable claims that individuals have in a given society. Those encompass the right to or immunities, and include basic rights which are ‘necessary for living or maintaining a minimally acceptable human life’ as well as fundamental rights (Campbell 2008) usually referred to as human rights, such as the rights to life and protection against torture (United Nations 1966: 999 UNTS 171:arts 6, 7). Hohfeld criticizes the indiscriminate use of the term ‘right’ to refer to privilege, immunity or power rather than solely to a ‘right’ in the strictest sense. The Hohfeldian approach consists in categorizing legal relations as jural correlatives: (1) rights as claims – and the corresponding duty of others, (2) liberties (or privileges) – and the corresponding ‘no right’ of others, (3) powers – and the corresponding liability of others and (4) immunities – and the corresponding disability of others; as well as jural opposites: (1) rights – no rights; (2) privilege – duty; (3) power – disability and (4) immunity – liability (1913: 30). This leads to a ‘grammar of legal relationships’ (Hohfeld 1917; Jackson 2010: 25). Hohfeld furthers distinguishes a right as one’s affirmative claim against another, a privilege as one’s freedom from the right or claim of another, power as one’s affirmative control over a given legal relation against another, and immunity as one’s freedom from the legal power or control of another as regards some legal relation (1913: 55). In legal positivism, rights are both construed, interpreted and upheld by master signifiers: judges, lawyers, policy makers and legal scholars. Individuals can assert their rights, although they firstly need to be aware of their entitlements, and secondly, they must be able to decode relevant legal sign systems. Yet legal terminology is often complex and therefore inaccessible to the general public. As noted by Gifis, ‘[d]espite the increasing pervasiveness of law into every facet of modern life, the special language of the law remains a barrier to nonlawyers’ (1998: vi). Legal semioticians are thus tasked with the exploration of the significance of legal sign systems. Kevelson considers that legal semiotics comprises seven subdivisions: (1) relation between law and legal systems, (2) relation between legal discourse and legal practice, (3) exchange of signs between official legal actors and non-official members of the general public, (4) sign structures of inquests and trials, (5) intersystemic communication between codes of legality and codes of legitimacy, (6) relations between laws of logic and the logic of laws and (7) comparatist semiotic perspective on legal cultures (1982: 65). Just as Peirce underscores that all propositions are hypothetical, Kevelson notes that ‘[t]he legal argument is not a formal argument. Its premises are neither true nor false, but hypothetical’ (1988b: 5). She applies Peirce’s trichotomy as she conceives law as an index, as well as an icon and symbol, and the legal system as a network of sign relations and competing and conflicting legal subsystems. She argues that there is not one type of legal discourse, but rather conflicting modes of reasoning (1988b: 10, 17). Hence for the purpose of legal semiotics, the legal realm is not viewed in a restrictive sense as solely comprising legal concepts, such as the well-known actus reus and mens rea components of a crime, legal documents such as the factum, or legal institutions such as courtrooms. Rather, the legal sphere encompasses an infinite combination of interactions between the

268

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

signifier, the signified, their many interpretants and interpreters: lawyers, judges, scholars and the general public. The second important element of legal semiotics is that it offers methods and frameworks that allow for a critical examination of signs and symbols in the legal realm. Tiefenbrun underscores the role that semiotics plays as a method to interpret legal language: The semiotics of law would attempt to identify, classify, and describe in a systematic fashion – and in standardized language – modes of signification present in legal discourse that give rise to interpretation. (2010a: 24) Methodology constitutes the central test to identify whether one is participating in legal semiotics. In this regard, Kevelson notes that: ‘Properly speaking, one may speak of legal semiotics as such when investigators intentionally acknowledge semiotics as a theory and method and when they deliberately conduct their research according to the assumptions of semiotics’ (1988b: 19). Jackson argues that semiotics of law cannot be considered a discipline per se, since it does not follow a common set of methods (2010: 28–9), although several legal semioticians presently apply semiotics methods in their examination of the law (see section on new and future developments below). In fact, Kevelson remarks that legal semiotics can be viewed as a generic term that covers several methods of approach, with different, and possibly at times conflicting, objectives (1988a: 3). The third component is that legal semiotics provides a common language to decode icons and symbols that can be used by both semioticians and lawyers. It constitutes a bridge between law and norm, scholars and practitioners, the black letter law and various disciplines in the social sciences and humanities, such as philosophy, sociology, linguistics, anthropology and communication. Some employ legal semiotics to resolve theoretical problems, others in the hopes of influencing the legal sphere and modifying it to address its shortcomings, and finally, some for the purely intellectual and interdisciplinary exercise of exploring legal icons (Kevelson 1988a: 6). Moreover, legal semioticians can adopt a critical legal theory perspective by deconstructing the ‘discourse of power’ (Wagner and Broekman 2010: vi), and legal hermeneutics to explore the nature of legal meaning. They can serve as translators of legal terms and concepts, thereby acting as intermediaries between master signifiers and the general population. However, some lawyers use their knowledge of legal semiotics to decode legal signs for the sole purpose of succeeding in their own careers, as anticipated by Welby and Peirce (Watral 2010: 26–30). As concerns categorization, it is worth noting that while Kevelson considers that semiotics of law constitutes a discipline, and Landowski, a sub-discipline of semiotics, Jackson notes that for many it functions as a meta-discipline that provides a language to study the legal science, or an auxiliary discipline rooted in philosophy or sociology (Jackson 2010: 29). Tiefenbrun contends that semiotics has influenced several schools of thought, including legal hermeneutics and critical legal theory, and further traces a parallel between the development of legal positivism and legal realism with Saussure and Peirce’s theories, respectively (2010a: 25). One could also consider that legal semiotics constitutes a ‘doctrine’ of legal signs, inspired by Sebeok’s view of general semiotics as a ‘body of principles and opinions that vaguely go to form a field of knowledge’ (Petrilli 2010: 6). Hence legal semiotics has a dual nature which varies according to its reader

SEMIOTICS IN LAW AND JURISPRUDENCE

269

or interpreter: whereas it is perceived by some as inclusive and all encompassing, others view it as a specialized subdivision within larger categories of disciplines and theories. In this regard, Jackson notes that ‘[i]t is, perhaps, the absence of a common approach to the status of legal semiotics that makes the field at one and the same time puzzling yet vibrant’ (2010: 29).

SEMIOTICS IN LAW AND JURISPRUDENCE: STILL A NOVEL AREA Legal semiotics nevertheless remains an under-researched area among both legal scholars and semioticians. Although lawyers are constantly applying and adapting legal sign systems to their advantage, be it in the presentation of their client in court, the examination and cross-examination of witnesses or during jury selection, they do not knowingly apply semiotics as a methodology. Moreover, while legal scholars adopt an increasingly interdisciplinary approach to treaty and statute interpretation, notably in international law, there is currently a divide between legal interpretation and semiotics (Smolka and Pirker 2016). Hence, although the eleven round tables provided crucial discussions between their members, their impact on legal practitioners was rather limited (Broekman and Fleerackers 2018: 27). As noted by Broekman, Mootz III and Pencak ‘while many lawyers and legal scholars unknowingly are thinking and acting semiotically, the conferences and publications of the legal semiotic community exist at the margins of law schools and the publishing industry’ (2011: x). Tiefenbrun notes in this regard that only ‘a courageous few’ have entered the realm of legal semiotics (2010a: 24–5). It could be argued that a significant obstacle to the recognition of semiotics as a method is that it is virtually absent from law school curricula. Presently, most law faculties focus on teaching the black letter law and in training ‘superb legal technicians with deficiencies in humanity and humanities’ (Kevelson 1987: 13). Jurisprudence or legal philosophy and its various schools of thought are taught as optional upper-year courses, if at all, whereas contracts, torts and business are usually mandatory first and second-year six credit courses. Very few law schools delve into the political and philosophical implications of statutes and court decisions. And those that do, often fail to teach semiotics as a methodology. For instance, while McGill University’s Law Faculty, the present writer’s alma mater, offers a perfect setting for legal semiotics as it teaches Civil law and Common Law transsystemically and in a bilingual (English-French) setting, with a focus on the theory behind the law, it does not incorporate legal semiotics as a science or methodology. Moreover, an additional concern is that even at the Dickinson School of Law, the Roberta Kevelson Seminars of Law and Semiotics ended in 2013, and appeal for these seminars ‘faded away’, along with a ‘growing disinterest for legal theoretical issues’ (Broekman and Fleerackers 2018: vii–viii). Another significant challenge in legal semiotics is that given its novel nature, it may erroneously be attributed concepts that would arguably be placed in other fields. In this regard, Kevelson contends that: ‘At present many writings that discuss signs and sign function are classified as semiotics. But such a general use of the term obscures and trivializes the really distinctive contribution that semiotics makes to scholarship as a whole, across disciplines and with possible application to the practical arts and social institutions’ (1988b: 139). Furthermore, it could be argued that another impediment to the development of legal semiotics, and which stems from the previous observation, is that the term ‘semiotics’

270

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

may be instrumentalized by scholars for reasons other than the pursuit of knowledge and truth: ‘Opportunists took over, and used words like “sign”, “signification” or even “semiotics” itself as a buzzword to get their own narrative in the center of public interest’ (Broekman and Fleerackers 2018: 56). Broekman and Fleerackers note that this problem is still present today: ‘Those opportunists are still at work in the front lines of law, social sciences, ethics and other disciplines. They use “semiotics” as a concept to further their own vision on life and in doing so disregard what Peirce had stated so many times: each sign is a sign itself only in context’ (2018: 56).

NEW AND FUTURE DEVELOPMENTS There are nevertheless significant developments in semiotics in law and jurisprudence. The International Journal for the Semiotics of Law, which publishes articles in both English and French, has grown into an internationally renowned journal under the leadership of its Editor Anne Wagner. Its editorial board comprises semioticians from all continents; its submission proposals doubled in 2019; and in 2021, it increased its publication from four to five issues per year, which demonstrates both an interest in terms of subscriptions and submissions from legal scholars and semioticians. Routledge has a new Book Series on Law, Language and Communication, co-edited by Anne Wagner and Vijay Bhatia (2010); and Springer recently launched a Book series on Law and Visual Jurisprudence, co-edited by Anne Wagner and Sarah Marusek (2019). The International Roundtables, also Presided by Anne Wagner, hold a yearly bilingual conference (English-French). As in the original spirit of Kevelson, these roundtables bring together experienced semioticians, lawyers and young scholars to discuss new approaches to legal semiotics. The most recent Roundtable bore on ‘The Rearguard of Subjectivity’ and was held in honour of renowned legal semiotician Jan M. Broekman (KU Leuven, June 2021). Legal semiotics was also recently featured in Special Issues of semiotics journals. Semiotica, for example, recently published two separate thematic issues related to the field, including Hidden Meanings in Legal Discourse (Cheng and Petrilli 2016), Signs, Symbols and Meaning in Law (Wagner and Ye 2017); and the journal Social Semiotics recently published a thematic issue entitled Exploring Legal Discourse (Cheng and Danesi 2019). Also, law journals are dedicating special issues to legal semiotics, for instance the International Journal of Legal Discourse, with Anne Wagner and Aleksandra Matulewska as Guest Editors, recently focused on the instrumentalization of law as a socially constituted sign-system (2020). Moreover, scholars are exploring and creating several new subjects and subfields in this meta-discipline, which often intersect. These new main areas could be categorized as follows.

Legal translation, interpretation and legal-diplomatic discourse Professor Anne Wagner, a leader in contemporary semiotics at the Université de Lille (France) and member of the Centre de Recherche Droits et Perspectives du Droit, examines legal language, the complexities of interpretation and translation, including in different legal systems, as well as how they intersect with visual semiotics, jurisprudence, culture and communication (2016, 2017). Le Cheng, a lawyer with a PhD in Language and law, and Professor and Director of the Center for Legal Discourse and Translation at Zhejiang University, China, conducts research in the fields of law, legal translation, corpus linguistics and discourse analysis (2010, 2011). Deborah Cao, Professor at Griffith University,

SEMIOTICS IN LAW AND JURISPRUDENCE

271

Australia, combines her degrees in foreign language and interpretation from the Shanghai Foreign Language Institute and the Beijing Foreign Studies University with her law studies at the University of Queensland, Australia, to explore Chinese legal language and culture and its translation. She applies a semiotic framework to analyse legal concepts, such as the rule of law, noting that they are triadic in nature and their constituents are (1) relative, (2) relational and (3) contextual in the semiotic interpretative process (2001, 2003). Janny HC Leung, Professor in the School of English at the University of Hong Kong, also combines her studies in linguistics and translation with her legal education in England and the United States to adopt an interdisciplinary approach to law, linguistics and psychology, with a focus on legal bilingualism (2016, 2019). Sarah Marusek, Professor of Political Science at the University of Hawai‘i Hilo, conducts research in legal semiotics, legal geography and law and society (2014, 2017). Evandro Menezes de Carvalho studied law in Brazil as well as the Université de Paris-Sorbonne, is Professor of International Law at the Fundação Getúlio Vargas (FGV) Law School and the School of Law of the Fluminense Federal University in Rio de Janeiro, and examines legal-diplomatic discourse, complexities in translation across legal traditions and in multilingual treaties such as those of the World Trade Organization, and Brazil-China relationships in the BRICS context (2007, 2011). Finally, Susan Petrilli, renowned semiotician and Professor of Philosophy and Theory of Languages at the University of Bari, Italy, examines the intersection of semiotics and ethics, and the hidden meanings in legal discourse, notably the assumptions of the capacity to answer for self and to tell the truth (2016).

Semiotics in legal education Professor Jan M Broekman has brought an interdisciplinary and European training in Civil Law and Social Sciences from Leiden, The Netherlands, and in philosophy, psychiatry and education from Göttingen, Germany, to his teaching of semiotics in law at KU Leuven University as well as in the Common law North American context, at Penn State University. His research bears on legal semiotics, comparative law (Civil Law and Common Law), legal education and the language of law, notably the gap between the ‘artificial’ legal language employed by lawyers, judges and legal specialists, and everyday discourse, and he is presently supervising the preparation of the Kevelson Archive (Broekman, Mootz III, and Pencak 2011; Broekman and Catà Backer 2013, 2015). Also at Penn State University, Larry Cáta Backer, Professor of Law and International Affairs, and a lawyer, examines globalization, business and human rights. He adopts a political and  cultural approach to law and politics in the USA and Cuba, based on facts and theory, and his research is in part grounded in legal semiotics (Broekman and Catà Backer 2013, 2015). The late William Pencak, Professor of History at Penn State, investigated legal semiotics from a historical perspective, and notably argued that Kevelson was the pioneer of legal semiotics (2005: 210; Broekman and Pencak 2009, 2010). Another collaborator of Broekman is Francis J Mootz III, Professor of Law at the McGeorge School of Law of the University of the Pacific, and a lawyer, who explores law and contemporary European philosophy, including the hermeneutical and rhetorical traditions, and legal semiotics (2009; Broekman and Mootz III 2011). Lastly, Frank Fleerackers, Professor of Law at KU Leuven University, also a lawyer, studied law and philosophy both in Belgium and in the Common law system, notably at King’s College London, MIT and Harvard Law School, conducts research in legal theory and jurisprudence, and publishes on legal semiotics with Broekman (Broekman and Fleerackers 2017, 2018).

272

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Semiotics as a framework to decode international and domestic law Susan Tiefenbrun, Professor Emeritus and Director of the Centre for Global Legal Studies at the Thomas Jefferson School of Law, combines her practice as a lawyer to her studies in languages, including a PhD in French, in her examination of semiotics in law, notably in her influential monograph Decoding International Law: Semiotics and the Humanities (2010a). She publishes in the fields of semiotics in law and literature, international human rights and corporate law, and employs semiotics to explore the definition of legal terms, such as ‘terrorism’, their hidden meanings, connotations and denotations (1986, 2003, 2007, 2010a, b). Clara Chapdelaine-Feliciati, Professor at York University, Toronto, and a lawyer trained in the Civil law and Common law systems, applies legal semiotics and legal significs, notably Welby’s Meaning Triad, as a methodology to explore the meaning(s) of international legal instruments, and adopts a ‘legal semioethics’ approach in this context, inspired by Petrilli and Ponzio’s semioethics theory (2010, 2013, 2018a, b, 2020, 2021). Finally, Fuad Zarbiyev, Professor at the Graduate Institute of International and Development Studies in Geneva, examines the rules of treaty interpretation, and whether they bridge the gap between the signifier and the signified, or instead have no meaning of their own outside the interpretative experience, referring in this context to Derrida (2019).

Semiotics and legal philosophy, legal positivism, and critical legal theory Professor Bernard Jackson, a founding member of the International Association for the Semiotics of Law, has examined the relationship between semiotics and modern jurisprudence, including legal positivism in Hart, Dworkin, MacCormick, Kelsen, as well as Greimasian structural theories, notably in his seminal monograph on Semiotics and Legal Theory, with a special interest in the nature of law, including Jewish Law, that of semiotics, and their relationship (1985, 2000, 2006, 2010). At the University of Coimbra, Professor José Manuel Aroso Linhares explores legal philosophy, legal positivism, hermeneutics and deconstruction (2012a, b), and Professor Ana Margarida Simões Gaûdencio, jurisprudence, legal semiotics and critical legal studies (2016). Lastly, Professor Mario Ricca, of the Faculty of Law of the University of Parma in Italy, examines the relationship between religion, law, culture, anthropology, legal semiotics (semiotica giuridica) and human rights (2016, 2018). As can be observed, several scholars in this field were exposed to many cultures which could account for an openness to exploring different meanings across legal regimes. Many are also multilingual, and thus aware of the complexities of translating and transposing legal language in different contexts. These legal semioticians also edited several books in legal semiotics (Wagner, Summerfield, and Benavides Vanegas 2005; Wagner and Bhatia 2009; Wagner, Cheng, and Sin 2014). The influence of legal semiotics can also be witnessed in the judiciary. As recently as 2017, when Supreme Court Justice Neil Gorsuch appeared before the Judiciary Committee, and was asked about his views on the US Constitution, he provided a profoundly legal semiotic response. While one would have expected him to discuss the interpretation of a text of law, he noted instead that ‘text in law is in the first place a privileged and specified area of encounter’ echoing a legal semiotics perspective whereby texts constitute open spaces occupied by meaning, anchor places in space that are ‘always elsewhere, always on the move, always in the mode of encounter’ (Broekman and Fleerackers 2018: 34–5).

SEMIOTICS IN LAW AND JURISPRUDENCE

273

CONCLUSION As a novel concept in the field of law, semiotics still needs to be further defined and explored. Kevelson’s observation in this regard that ‘[p]aramount among tasks in the near future is the need to distinguish a semiotic concern with law from the various legal concerns that are not semiotically oriented’ (1988b: 53), is still pertinent today. It is hoped that legal semiotics will be incorporated in both legal theory and legal training, and thus in law schools as well as in other related disciplines, such as political science. Students should have the opportunity to question the master discourse, evaluate the strengths and weaknesses of icons, symbols and arguments, and apply semiotics as a framework to evaluate legal sign systems. These students will in turn become new generations of scholars and practitioners aware of the power of legal icons and symbols. One can only imagine the strength of this legal teaching, as stated by Holmes to his law students: ‘Your business as lawyers is to see the relation between your particular fact and the whole frame of the universe’ (Kevelson 1988b: 17). The twenty-first century may very well be a perfect space in time for a renaissance in the semiotics of law.

REFERENCES Balkin, J. M. (1989), ‘The Hohfeldian Approach to Legal Semiotics’, in Roberta Kevelson (ed.), Law and Semiotics, vol. 3, 31–50, New York: Plenum Press. Balkin, J. M. (1991), ‘The Promise of Legal Semiotics’, Texas Law Review, 69: 1831–52. Broekman, J. M. (2016), Meaning, Narrativity, and the Real: The Semiotics of Law in Legal Education IV, Dordrecht: Springer. Broekman, J. M. and L. C. Backer (2013), Lawyers Making Meaning: The Semiotics of Law in Legal Education II, Dordrecht: Springer. Broekman, J. M. and L. C. Backer, eds (2015), Signs in Law – A Source Book: The Semiotics of Law in Legal Education III, Dordrecht: Springer. Broekman, J. M. and F. Fleerackers (2017), Legal Conversation as Signifier, Elgar Studies in Legal Theory, Cheltenham: Elgar. Broekman, J. M. and F. Fleerackers (2018), Legal Signs Fascinate: Kevelson’s Research on Semiotics, Cham: Springer. Broekman, J. M. and F. J. Mootz III, eds (2011), The Semiotics of Law in Legal Education, Dordrecht: Springer. Broekman, J. M. and W. A. Pencak, eds (2009), ‘Lawyers Making Meaning: The Roberta Kevelson Seminar on Law and Semiotics at Penn State University’s Dickinson School of Law’, Special Issue of the International Journal for the Semiotics of Law, 22 (1). London: Springer. Broekman, J. M. and W. A. Pencak, eds (2010), ‘Signs of Law. The Roberta Kevelson Seminar on Law and Semiotics at Penn State University’s Dickinson School of Law’, Special Issue of the International Journal for the Semiotics of Law, 23 (1). London: Springer. Broekman, J. M., F. J. Mootz III, and W. A. Pencak (2011), ‘Preface – Semiotics in the Seminar’, in J. M. Broekman and F. J. Mootz III (eds), The Semiotics of Law in Legal Education, v–xiv, Dordrecht: Springer. Campbell, T. (2008), ‘Rights and Duties’, in P. Cane and J. Conaghan (eds), The New Oxford Companion to Law, Oxford: Oxford University Press. Cao, D. (2001), ‘Fazhi vs/and/or Rule of Law?: A Semiotic Venture into Chinese Law’, International Journal for the Semiotics of Law, 14 (3): 223–47.

274

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Cao, D. (2003), ‘Chinese Lawmaking as a Communicative Act?’ International Journal for the Semiotics of Law, 16 (3): 211–32. Carvalho, E. M. (2007), ‘The Decisional Juridical Discourse of the Appellate Body of the WTO: Among Treaties and Dictionaries as Referents’, International Journal for the Semiotics of Law, 20 (4): 327–52. Carvalho, E. M. (2011), Semiotics of International Law: Trade and Translation, trans. Luciana Carvalho Fonseca, New York: Springer. Chapdelaine-Feliciati, C. (2010), ‘Les droits de l’homme de la femme: polysémie ou androcentrisme’, International Journal for the Semiotics of Law, 23: 451–74. Chapdelaine-Feliciati, C. (2013), ‘The Sense, Meaning and Significance of the Twin International Covenants on Political and Economic Rights’, Semiotica, 196: 325–52. Chapdelaine-Feliciati, C. (2016), The Status of the Girl Child under International Law: A Semioethic Analysis, PhD dissertation in Law (DPhil), University of Oxford, Faculty of Law, Exeter College, UK. Chapdelaine-Feliciati, C. (2018a), ‘Deconstructing the Convention on the Rights of the Child: Semiotics, Significs and Semioethics of Gendercide’, Cambridge International Law Journal, 7 (1): 29–50. Chapdelaine-Feliciati, C. (2018b), Feminicides of Girl Children in the Family Context: An International Human Rights Law Approach, Family Law in a Global Society, Leiden: Brill. Chapdelaine-Feliciati, C. (2020), ‘The Semiotic Puzzle: Authentic Languages and International Law’, International Journal of Legal Discourse, 5 (2): 317–41. Chapdelaine-Feliciati, C. (2021), ‘Les réserves en droit international ont-elles des limites? Étude sémioéthique du droit à l’éducation de la Convention relative aux droits de l’enfant’, International Journal for the Semiotics of Law, 34 (4): 1–30. Cheng, L. (2010), ‘A Semiotic Interpretation of Genre: Court Judgments as an Example’, Semiotica, 182: 89–113. Cheng, L. (2011), ‘Administration of Justice and Multimodality in Media: Semiotic Translation, Conflict and Compatibility’, International Journal for the Semiotics of Law, 24 (4): 491–502. Cheng, L. and M. Danesi, eds (2019), ‘Exploring Legal Discourse: A Sociosemiotic (Re) construction’, Special Issue of Social Semiotics, 29 (3). London: Taylor and Francis. Cheng, L. and S. Petrilli, eds (2016), Hidden Meanings in Legal Discourse, Special Issue of Semiotica, 209, Berlin: De Gruyter. Cyran, E. J. (2011), ‘Common Law Lawyers Should Mind Their Trial Practices: Understanding, Identifying, and Correcting a Semiotic Imbalance’, in J. M. Broekman and F. J. Mootz III (eds), The Semiotics of Law in Legal Education, 115–28, Dordrecht: Springer. Davies, M. (2008), ‘Theory in Legal Research’, in P. Cane and J. Conaghan (eds), The New Oxford Companion to Law, Oxford: Oxford University Press. Gifis, S. H. (1998), Dictionary of Legal Terms: A Simplified Guide to the Language of Law, 3rd edn, New York: Barron’s Educational Series. Hohfeld, W. N. (1913), ‘Some Fundamental Legal Conceptions as Applied in Judicial Reasoning’, Yale Law Journal, 23: 16–59. Hohfeld, W. N. (1917), ‘Fundamental Legal Conceptions as Applied in Judicial Reasoning’, Yale Law Journal, 26: 710–70. Holmes, O. W. (1897), ‘The Path of the Law’, Harvard Law Review, 10 (8): 457–78. Jackson, B. S. (1985), Semiotics and Legal Theory, London: Routledge.

SEMIOTICS IN LAW AND JURISPRUDENCE

275

Jackson, B. S. (2000), Studies in the Semiotics of Biblical Law, Sheffield: Sheffield Academic Press. Jackson, B. S. (2006), Wisdom-Laws: A Study of the Mishpatim of Exodus 21: 1–22:16, Oxford: Oxford University Press. Jackson, B. S. (2010), ‘Legal Semiotics and Semiotic Aspects of Jurisprudence’, in A. Wagner and J. M. Broekman (eds), Prospects of Legal Semiotics, 3–36, Dordrecht: Springer. Kennedy, D. (1989), ‘A Semiotics of Legal Argument’, in R. Kevelson (ed.), Law and Semiotics Volume 3, 167–92, New York: Plenum Press. Kennedy, D. (2001), ‘A Semiotics of Critique’, Cardozo Law Review, 22: 1147–89. Kevelson, R. (1982), ‘Comparative Legal Cultures and Semiotics: An Introduction’, American Journal of Semiotics, 1 (4): 63–84. Kevelson, R. (1987), ‘Introduction to the First Round Table on Law and Semiotics’, in R. Kevelson (ed.), Law and Semiotics: Volume 1, New York: Plenum Press. Kevelson, R. (1988a), ‘Introduction to the Second Round Table on Law and Semiotics: Pragmatic Method and Some Consequences’, in R. Kevelson (ed.), Law and Semiotics: Volume 2, New York: Plenum Press. Kevelson, R. (1988b), The Law as a System of Signs, New York: Plenum Press. Kevelson, R. (1996), Peirce, Science, Signs, New York: Peter Lang. Leung, J. H. C. (2016), ‘Negotiating Language Status in Multilingual Jurisdictions: Rhetoric and Reality’, Semiotica, 209: 371–96. Leung, J. H. C. (2019), Shallow Equality and Symbolic Jurisprudence in Multilingual Legal Orders, Oxford: Oxford University Press. Linhares, A. and J. Manuel (2012a), ‘Evidence (or Proof?) As Law’s Gaping Wound: A Persistent False Aporia?’ Boletim Da Faculdade de Direito, 88 (1): 65–90. Linhares, A. and J. Manuel (2012b), ‘Law’s Cultural Project and the Claim to Universality or the Equivocalities of a Familiar Debate’, International Journal for the Semiotics of Law, 25 (4): 489–503. Marusek, S. (2014), ‘Visual Jurisprudence of the American Yellow Traffic Light’, International Journal for the Semiotics of Law, 27 (1): 183–91. Marusek, S. (2017), ‘Expiration Dates: Performative Illusions of Law and Regulation’, Semiotica, 216: 249–63. Mootz, F. J. III (2009), ‘Vico and Imagination: An Ingenious Approach to Educating Lawyers with Semiotic Sensibility’, International Journal for the Semiotics of Law, 22 (1): 11–22. ‘Oxford English Dictionary’, 2011, 3rd edn. Paul, J. (1991), ‘The Politics of Legal Semiotics’, Texas Law Review, 69 (7): 1779–829. Peirce, C. S. ([1866–1913] 1931–58), The Collected Papers of Charles Sanders Peirce, vols. 1–6, ed. C. Hartshorne and P. Weiss, vols. 7–8, ed. A. Burks, Cambridge: Harvard University Press. Cited as CP. Pencak, W. A. (2005), ‘Canada as a Semiotic Society: Harold Innis, Roberta Kevelson, and the Bias of Legal Communications’, International Journal for the Semiotics of Law, 18 (2): 207–15. Petrilli, S. (2009), Signifying and Understanding: Reading the Works of Victoria Welby and the Signific Movement, Berlin: De Gruyter Mouton. Petrilli, S. (2010), Sign Crossroads in Global Perspective : Semioethics and Responsibility, New Brunswick: Transaction Publishers. Petrilli, S. (2015), Victoria Welby and the Science of Signs: Significs, Semiotics, Philosophy of Language, New Brunswick: Transaction Publishers. Petrilli, S. (2016), ‘Two Assumptions in Legal Discourse: To Answer for Self and to Tell the Truth’, Semiotica, 209: 15–30.

276

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Petrilli, S., F. Nuessel, and V. Colapietro (2013), ‘On and Beyond Significs: Centennial Issue for Victoria Lady Welby (1837–1912)’, Semiotica, 196: 570. Ricca, M. (2016), ‘Klee’s Cognitive Legacy and Human Rights as Intercultural Transducers: Modern Art, Legal Translation, and Micro-Spaces of Coexistence’, Calumet – Intercultural Law and Humanities Review, 1–40. Ricca, M. (2018), ‘Ignorantia Facti Excusat: Legal Liability and the Intercultural Significance of Greimas’ “Contrat de Véridition”’, International Journal for the Semiotics of Law, 31 (1): 101–26. Shapiro, F. R. (2000), ‘The Most-Cited Legal Scholars’, Journal of Legal Studies, 29: 409–26. Simões Gaudêncio, A. M. (2016), ‘Fraternity and Tolerance as Juridical Boundaries’, Boletim Da Faculdade de Direito Da Universidade de Coimbra, 92 (2): 849–66. Smolka, J. and B. Pirker (2016), ‘International Law and Pragmatics: An Account of Interpretation in International Law’, International Journal of Language and Law, 5: 1–40. Tiefenbrun, S. (1986), ‘Legal Semiotics’, Cardozo Arts and Entertainment Law Journal, 5 (1): 89–156. Tiefenbrun, S. (2003), ‘A Semiotic Approach to a Legal Definition of Terrorism’, ILSA Journal of International and Comparative Law, 9: 357–402. Tiefenbrun, S. (2007), ‘The Semiotics of Women’s Human Rights in Iran’, Connecticut Journal of International Law, 23 (1): 1–82. Tiefenbrun, S. (2010a), Decoding International Law, Oxford: Oxford University Press. Tiefenbrun, S. (2010b), ‘Semiotic Definition of Lawfare’, Case Western Reserve Journal of International Law, 43 (1): 29–60. United Nations (1966), International Covenant on Civil and Political Rights, vol. 999 UNTS 171. Wagner, A. (2016), ‘A Space in-between – Legal Translation as a Third Space’, Journal of Civil Law Studies, 9 (1): 167–90. Wagner, A. (2017), ‘La sémiotique juridique verbale et nonverbale comme stratégie de communication du droit: Signs, symbols, and meanings in law’, Semiotica, 216: 1–18. Wagner, A. and V. K. Bhatia, eds (2009), Diversity and Tolerance in Socio-Legal Contexts: Exploration in the Semiotics of Law, Farnham: Ashgate. Wagner, A. and J. M. Broekman (2010), ‘Promises and Prospects of Legal Semiotics – An Introduction’, in A. Wagner and J. M. Broekman (eds), Prospects of Legal Semiotics, v–xviii, Dordrecht: Springer. Wagner, A. and N. Ye, eds (2017), Signs, Symbols, and Meanings in Law, Special Issue of Semiotica, 216, Berlin: De Gruyter. Wagner, A. and A. Matulewska, eds (2020), Instrumentalization of Law as a Socially Constituted Sign-system, Special Issue of the International Journal of Legal Discourse, 5 (2), Berlin: De Gruyter. Wagner, A., T. Summerfield, and F. S. B. Vanegas, eds (2005), Contemporary Issues of the Semiotics of Law, The Oñati International Series in Law and Society, Oxford: Hart. Wagner, A., L. Cheng, and K. Sin, eds (2014), The Ashgate Handbook on Legal Translation, Farnham: Ashgate. Watral, C. M. (2010), ‘Student Positions and Opinions’, International Journal for the Semiotics of Law, 23 (3): 3–39. Welby, V. (1897), Grains of Sense, London: J. M. Dent & Co. Zarbiyev, F. (2019), ‘The “Cash Value” of the Rules of Treaty Interpretation’, Leiden Journal of International Law, 32: 33–45.

CHAPTER TWELVE

Semiotics in Architecture and Spatial Design GABRIELE ARONI

The semiotics of architecture is intertwined with many other disciplines, such as aesthetics and sociology. This makes it difficult to find a precise stream of semiotics of architecture proper. Still, it is undeniable that semiotics has a very tangible and notable presence in everyone’s environment, since it is difficult to overlook the communicative components of landscape design, urban planning, architectural design and interior design, moreover ‘at the very heart of semiotics, a spatial logic (presence-absence, and an embodiment of this difference) is part of the definition of what it means to convey meaning’ (Sandin 2012: 175). Why, then, has relatively little attention been given to the semiotics of architectural design recently? The first reason is that, arguably, the architecture with which the vast majority of us deal on a daily basis apparently does not ‘communicate […] but functions’ (Eco 1997: 174), which raises the question as to whether representation in architecture is indeed possible, and especially how. Architecture, in fact, unlike figurative painting or sculpture, does not reproduce something that already exists, but necessarily creates something new. As such, it could be regarded as an asignifying and non-representative art, especially considering the mechanical role that it must serve. Despite this, there has always been a semiotic aspect one way or another: from the expression of natural or mathematical rules, to the imitation of historical styles, and expression of abstract rules. Much like Monsieur Jourdain in Molière’s Le Bourgeois gentilhomme, who did not know he was speaking in prose, ‘architects, unlike practitioners in many other fields, actually think in semiotic […] concepts directly in the process of their creative work’ (Broadbent 1994: 86). The meaning of the built environment is indeed a very present issue, even when not framed in semiotic terms. In a speech delivered at the Literary Symposium in Beijing in 2014, Chinese President Xi Jinping spoke against the ‘grandiose and weird western architecture’ that is present in China’s major cities (Morrison and MacLeod 2016), such as the CCTV building in Beijing, designed by internationally renowned Dutch architect Rem Koolhaas. The call for the avoidance of such architecture is clearly not a matter of economy, logistics or functionality, but rather of messages and symbols. A call hardly heeded, if the latest design for Shenzhen by the Dutch studio MVRDV, the Shenzhen Terraces, is any indication. Or, for that matter, the new glass façades that replace the concrete of old Brutalist buildings in many major cities such as Toronto, where the new ‘language’ of glass is superimposed to the old language of béton brut. This is indeed a semiotic manoeuvre: the insulating quality of new, high performance glass is arguably superior to the old concrete, but this is clearly not the reason for this completely new style: the glass façades are the symbol of the new city.

278

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

It is not by accident that many important semioticians taught in faculties of design. Indeed, Charles Morris was one of the founders of the New Bauhaus – now the IIT Institute of Design – in Chicago in the 1930s, Umberto Eco taught at the School of Architecture of the University of Florence in the 1960s (Eco [1968] 2016: sec. 8.6), and in the same years Tomás Maldonado introduced a course on semiotics at the Hochschule für Gestaltung, another sort of New Bauhaus founded in Ulm in 1953 (Mallgrave 2005: 332, 349). It should be noted that in this chapter, under the heading of the term ‘architecture’, I will intend any kind of design aimed at shaping the space around us. It thus includes landscape design, as well as urban planning and interior design. As architect Ernesto Nathan Rogers is known to have said referring to the scope of the work of architects: ‘from the spoon to the city’. In ancient treatises, for example, the distinction between architecture, urbanism and interior design is practically non-existent, and while these aspects are being treated specifically, they are never completely disjointed from each other, or considered separate disciplines altogether. That architecture is composed of several disciplines is formulated in the famous opening sentence of Roman architect Vitruvius’ De Architectura, the only treatise on architecture of the ancient Western world that came to us, which dates back to the first century BCE: ‘The science of the architect depends upon many disciplines and various apprenticeships which are carried out in other arts’ ([c. 20 BCE] 1931: 1.1.1). Indeed, ‘in the classical tradition the city is always the fulfilment of architecture’ (Westfall 2016: 51). As for the term ‘spatial design’, it is rather new, so much so that it is not present in dictionaries yet. The Bloomsbury Encyclopedia of Design defines it as ‘the design of the spatial environment’ that ‘currently cross[es] disciplines together and these may include architecture, interior design, landscape architecture, urban design, performance art, scenography, and exhibition design’ (Poldma 2016: 254). This definition brings back to Rogers’ idea, and the concept tries to make up for the fragmentation that the discipline of architecture – or design – suffered in the past decades, and suffers in today’s projects, where often the interiors are designed by a different team of professionals – interior designers – than the exteriors – architects – who are designing a building whose location and specifications are decided by a different team of specialists – the urban planners. The concept of space, however, is even broader, for instance Christian Norberg-Schulz states that: In architectural theory there is no reason to let the word ‘space’ designate anything but the tri-dimensionality of any building. But it is not said that this property is always of architectural importance. Expressions like ‘spatial experience’ or ‘spatial effect’ should therefore only be employed when the stereometric volume is of decisive importance. It is not practical to distinguish between ‘physical’ and ‘architectural’ space, but only between physical space and architecture. (1963: 20) For Norberg-Schulz, any built environment ‘is something more than a space with a varying degree of openness’, from single buildings to entire settlements (1979: 63). Pierre Pellegrino and Emmanuelle Jeanneret affirm that space is already ‘semiotic as much as pragmatic’ (2009: 271) before the creation of any architecture in it. It is however the built object that ‘is a semiotic object of connotation’ (2009: 284) and as such the focus of semiotic analysis.

SEMIOTICS IN ARCHITECTURE AND SPATIAL DESIGN

279

ARCHITECTURE AS A LANGUAGE Architect Peter Eisenman remarked about the 2014 Venice Biennale exhibition Elements of Architecture organized by Rem Koolhaas that ‘[a]ny language is grammar, so if architecture is to be considered a language, “elements” don’t matter. So for me what’s missing [from the show], purposely missing, is the grammatic’ (Ciuffi 2014). The fact that Koolhaas ‘doesn’t believe in grammar’ (2014) is stated as a negative fact. The orderly aspect of architecture, especially compared to the other arts, ‘makes it such an inviting field to the semiologist in the first place’ (Munro 1987: 121). In the very first chapter of the De architectura, Vitruvius makes a statement that has fascinated semioticians ever since: Both in general and especially in architecture are these two things found; that which signifies and that which is signified. That which is signified is the thing proposed about which we speak; that which signifies is the demonstration unfolded in systems of precepts. ([c. 20 BCE]: 1.1.3) We can already read the Saussurean pair of ‘signifier/quod significatur’ and ‘signified/quod significat’, albeit many commentators argue that Vitruvius did not properly understand the meaning of the original Greek terms semainon and semainòmenon, which are closer to our contemporary semiotic understanding (De Fusco 2019: 72). The signified is thus the actual project, whereas the signifier is the theory and the principles behind it. In semiotic terms, Alexandros Lagopoulos considers the first comparable to ‘performance’ whereas the second to ‘competence’ (2009: 195–6). Regardless, the fact that these terms are present in one of the oldest architectural treatises is indeed telling, especially since we can find how the dual valence of architecture, and especially its ante litteram semiotic one, permeates all of Vitruvius’ book. Many of the concepts that Vitruvius explains in his treatise are borrowed from other disciplines, in order to fulfil his aim of giving architecture the same status that more ‘regarded’ arts enjoyed, such as music and rhetoric. The latter is in fact one of the fields that inspired Vitruvius the most, as it was one of the most studied subjects in Rome, thanks to the writings of his fellow countrymen Marco Tullius Cicero and Quintilian, of whom Vitruvius was well aware, and of course the ones of Greek philosopher Aristotle. In the first book of the De Architectura, Vitruvius explains how temples erected in honour of different gods should be adorned by different orders so as to be ‘appropriate’ to the deity they represent: a sturdy Doric order for the temples of Minerva, Mars and Hercules; whereas an elegant Corinthian would be appropriate for temples dedicated to Venus or Flora. In the middle, the Ionic order, appropriate for deities such as Juno or Diana (c. 20 BCE: 1.2.5). To indicate this ‘appropriateness’, the concept that architectural design should be visually fitting for its role, Vitruvius uses the term decor, ‘the faultless ensemble of a work composed, in accordance with precedent, of approved details. It obeys convention, which in Greek is called thematismos, or custom or nature’ (c. 20 BCE: 1.2.5). In John Onians’ opinion, this application of the architectural orders has roots in rhetoric (1988: 38), since Cicero, in his De oratore writes of three types of oratory, explained more in detail later on by Quintilian: the coherent and pure Attic style from Athens, the pompous and grandiose Asiatic, and finally the intermediate Rhodian. As with

280

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

the architectural orders, each style was appropriate for a certain audience and situation, the aim was to communicate a message in an effective and suitable manner, be it the content of the oration for the orator, or the aspect of the building for the architect. In fact, for Vitruvius, a building fulfils its function ‘by making visible a character that expresses its purpose within the civil order’ (Westfall 2016: 21), thus by communicating to the people what it is and what it represents. Classical architecture, by its very nature, lends itself easily to a semiotic analysis and a comparison with language, especially taking into consideration the architectural orders (Munro 1987: 122; Scruton 2013: 148). Ferdinand de Saussure himself compared the linguistic unit to a classical column: From the associative and syntagmatic viewpoint a linguistic unit is like a fixed part of a building, e.g. a column. On the one hand, the column has a certain relation to the architrave that it supports; the arrangement of the two units in space suggests the syntagmatic relation. On the other hand, if the column is Doric, it suggests a mental comparison of this style with others (Ionic, Corinthian, etc.) although none of these elements is present in space: the relation is associative. ([1916] 2011: sec. 13.120) The theory of the orders, modelled after classical Greco-Roman architecture – in turn developed from more ancient predecessors, including Egyptian and Mesopotamian architecture – offers a ‘grammatical’ (Scruton 2013: 148) approach to architecture, which can already be read in Vitruvius’ treatise, but that will be fully codified only during the Renaissance. The architectural order is a plastic tridimensional composition, organized morphologically and proportionally controlled as well as structurally efficient (since it is based on the ancient constructive principle of post and lintel: a structural cell made of two vertical supports and one horizontal supported structure), which is moreover symbolically ‘eloquent’ regarding mythical-religious creeds, theoretical-scientific theories and poetic and literary metaphors; it is an aggregate of elementary components elaborated during several generations of classic Greek-Roman monuments, Renaissance classical buildings and beyond, always rigorously codified “in parallel” in the architectural treatises. [my translation] (Morolli 2013: 14) And it was the system that governed architecture until the twentieth century. Most importantly, ‘[b]efore they were commended as Classical and before they were defined with legal precision as orders, the columns, capitals, and mouldings which we know as Tuscan, Doric, Ionic, Corinthian and Composite were a material means of expression for communities, groups and individuals’ (Onians 1988: 3). The parallel to language is easy to make. Gabriele Morolli affirms that the origin of the alphabet in Phoenicia and the Peloponnese and the birth of a ‘code’ in the art of building are linked. At the same time as language was organized in a series of ‘conventional symbols’ shared by a community, the components of the architectural orders became the codification of the structural and symbolic mechanisms that ensured the structural solidity of buildings and their capability of fulfilling their functions. The orders are characterized by symbols in the design of their components: the capitals are a cup for libation in the Tuscan and

SEMIOTICS IN ARCHITECTURE AND SPATIAL DESIGN

281

Doric orders, sacred cushions in the Ionic, a funerary basket in the Corinthian, and a large vase surrounded by leaves and covered by a cup in the Composite. The orders were anthropomorphized from the simple and rustic Tuscan, associated with manual labour, the sturdy Doric linked to warriors, the balanced and elegant ionic representing a matron or an intellectual, the slender Corinthian a young maiden, and finally the richly decorated Composite, appropriate for a sovereign (Morolli 2013: 26). The parallel of the orders and language is also due to the very structure of their components. The mouldings, the most basic signs that compose the order, are associated to the vowels, and the elements, additional details, are the consonants. The combination of these two gives birth to the portions, which determine where this combination of mouldings and elements belong and their function and can thus be considered as the words of the architectural language of the orders. The arrangement of portions creates members: capitals, shafts and bases that in turn compose the main parts, that is, columns, entablatures and pedestal, the phrases of the syntactically intelligible sentence of the order. These components, like language, are arranged in more complex forms that result in buildings, blocks and whole cities (2013: 22–3). On this subject, Antoine de Quincy wrote that ‘[t]he elements [of architecture] are to architecture what words are to speech, what notes are to music’ ([1819] 1981: 1.29). The communicative aspect of architecture was indeed an important factor that contributed to the initial codification of architectural theory, as explained by Onians: [W]hat theorists proposed to patrons was that the power of buildings to affect those who saw and used them could be brought under their control and used directly for their benefit. They did this by first identifying the values with which the patrons wanted to associate themselves publicly and then demonstrating that architecture could be made to embody or express them. (1988: 6) In classical Greece, architectural orders also represented a geographical or ethnical identity, as the names themselves indicate: Ionia was a region of Greece, the Dorians an ethnic group and Corinth a city. This became particularly relevant during and after the Persian and Peloponnesian wars, such as at the stoa in Delphi, erected in the Ionic instead of the usual Doric style by the Athenians, to thank the Ionian Greeks for their help in the battle of Salamis against the Persians in 480 BCE (1988: 15). Vitruvius himself wrote the De architectura with the intent of fostering Roman architectural identity to counter the enormous Greek influence – what Virgil was doing for poetry and Dionysius of Halicarnassus for historiography – as part of a larger ‘romanization’ plan carried out by Augustus in the nascent empire. Vitruvius describes the tuscanicae dispositiones, the ancient Etruscan, and thus Italic, way of designing temples, to demonstrate that also the Romans had an indigenous architecture style, and did not borrow everything from the Greeks (Morolli 2013: 39–40). A similar situation can be found during the Renaissance, in 1420s Florence, when architect Filippo Brunelleschi, who was in charge of the redesign of the old church of San Lorenzo, refused to resort to the current foreign Gothic architecture, choosing instead the classical Roman style, infused with late-medieval elements typical of the Florentine traditions. For instance, in the Corinthian capitals he employs in the church, the traditional acanthus leaves used as a decoration are replaced by oak leaves, both as a reference to late-medieval Florentine capitals, and as a symbol of strength, since the

282

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

word vis in Latin can mean both oak and strength. This symbol was directed both to the martyr Saint Lawrence, to whom the church is dedicated, as well as the Medici family, who financed it, and whose power was raising in Florence. The same symbology is to be found in another part of the capitals, the volutes, which resemble ram horns – another symbol of strength – rather than the usual caulicoles (Aroni 2014). Brunelleschi’s contemporary and fellow architect Leon Battista Alberti wrote the first comprehensive architectural treatise of the Western world after Vitruvius, the De re aedificatoria, first published in Latin in 1450. Much like Vitruvius who wanted to assert a cultural independence from Greece in favour of ascending Rome, Alberti wanted to make sure that his treatise would be entirely Italic, and purged any trace of Greek terminology, largely used by Vitruvius, starting with architectura itself, which becomes the titular re aedificatoria. His nationalistic linguistic approach is of course echoed by his history of architecture, which in his opinion was born in Asia, flourished in Greece, but only reached its perfection in Italy (1485: 3.3). Centuries later, the famous engraver Giovanni Battista Piranesi will even affirm that Greek architecture actually derived from the Etruscan one (1761: chps. 16, 215), and for similar reasons: to re-establish the importance of Rome against the recent surge of interest in Greece that was taking place in the eighteenth-century academic and artistic circles. The meaning of architecture and spatial design as a symbol of a culture or a nation has many similar episodes. French king Louis XIV refused Gian Lorenzo Bernini’s design for the royal palace of the Louvre as too ‘Italic’, and imperialist powers often sought to establish their own architectural taste, which, in the opinion of Nikos Salingaros, still holds true. Today, contemporary architecture, especially in the case of large projects that receive ample media coverage – and praise – is ‘a symbolic invasion of traditional culture’, where there is ‘a Western expression of dominance encoded in contemporary architectural forms’ (Salingaros 2013: 217). Morolli observed the same phenomenon, for example as regards nineteenth-century architectural history. He noted that Viollet le Duc, in his Histoire de l’Habitation humaine, expresses disdain towards wooden Chinese architecture, and wooden architecture in general, such as Etruscan, was considered inferior. Or the ‘Calvinist presumption’ of the modernist historiography that considers that there is only one possible form that can appropriately respond to a certain demand (Morolli 1985: 37).

SEMIOTICS OF ARCHITECTURE AND SPATIAL DESIGN In more recent times, the study of the semiotics of architecture saw a particular resurgence in the late 1950s to the 1970s, especially as a response to the Modernist movement. Most of architectural theory throughout the twentieth century has been based on the conceptual – and formal – divide that separates ‘Modern’ architecture from everything that came before it. One of the bases of the modernist doctrine was that it was the endpoint of Western architectural development, and that its ‘machinelike’, abstract architecture was the definitive and only response to any architectural need (Mallgrave 2005: 195). The one-sided concerns with functionality and pure forms were seen as an impoverishment of design and triggered a reaction starting in the 1950s, when architectural theorists began putting an emphasis on the ‘meaning’ of architecture. This, as we will see, remains the main leitmotiv that guides the studies and debates of the semiotics of architecture and spatial design until this very day.

SEMIOTICS IN ARCHITECTURE AND SPATIAL DESIGN

283

Joseph Rykwert published his influential essay Meaning in Architecture (1960) in the Italian magazine Zodiac, where he affirmed the importance of the house as something more than a mere machine satisfying basic needs. Rykwert spent much of the 1950s living in Italy (Mallgrave 2005: 372), where the debate on semiotics and architecture was particularly active. Sergio Bettini published Semantic Criticism; and the Historical Continuity of European Architecture in 1958, and was one of the first scholars to identify architectural elements in terms of ‘signs’ (1958). The following year, Tomás Maldonado published Communications and Semiotics (1959) while he was teaching at the Technical Hochschule of Ulm, where he included semiotics in the curriculum. Then the 1960s were particularly prolific, with Giovanni Klaus Koenig’s Analisi del linguaggio architettonico (1964), Renato de Fusco’s Architettura come Mass Medium (1967), Maria Luisa Scalvini’s Simbolo e significato nello spazio architettonico (1968) and finally Umberto Eco’s La Struttura Assente ([1968] 2016). Part of Eco’s work was then published in English as Function and Sign: The Semiotics of Architecture, first in 1973 and later collected in the influential book Sign, Symbols and Architecture (Broadbent, Bunt, and Jencks 1980). Umberto Eco steered away from approaching the semiotics of architecture with a strict linguistic framework, as he affirmed that not all communicative phenomena can be explained with linguistic categories, and that instead a visual semiotic approach to architecture can demonstrate the independence of semiotics from linguistics ([1968] 2016: sec. 1.1.1). Eco gives architecture a double semiotic function: a primary denotative one, and a secondary connotative one. The primary denotative function is in virtue of the architectural object’s form and how humans interpret the environment around them (Eco 1997: 177): a flight of stairs will be understood as a device that enables us to move between two vertically different locations by virtue of its own shape, size and position. Munro (1987: 120) as well as Roger Scruton (2013: 153) brought up the issue with this line of reasoning that this is not much of a semiotic issue, since our understanding of basic geometric and architectural forms is not based on conventions, and that the aim of an architectural object is its function, not its cause or effect. The secondary function connotes instead ‘a certain ideology of the function’ (Eco 1997: 178). Remaining within the example of the stairs: an adorned spiral staircase surrounded by columns such as Borromini’s monumental staircase in Palazzo Barberini, while fulfilling the exact same mechanical and functional role as any bare concrete escape staircase in our average high-rise building, obviously ‘connotes’ a very different message that goes beyond its mere function. It is important to note that the primary and secondary functions are not in order of importance, and the connotative aspect is indeed a function in and of itself as much as the denotative one. Rather, the connotative function ‘rests on the denotation of the primary function’ (1997: 179). In France, the publication of Roland Barthes’ Mythologies ([1957] 2013) and Claude Lévi-Strauss’ Cultural Anthropology (1963), mostly based on the works of Swiss semiotician Ferdinand de Saussure, spurred great interest in the field of semiotics with repercussions in the arts, including architecture, especially with Barthes’ Elements of Semiology ([1964] 1977). Barthes applied semiotics to the study of the city, with the concept of urban semiology, already introduced by Lévi-Strauss. In his article Semiology and the Urban, Barthes highlighted the issues in modern urban planning: ‘in certain cases a conflict exists between the functionalism of a part of a city […] and what I call its semantic contents’ ([1967] 1997: 160). He affirms that urban semiology could help in

284

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

creating a scientific model for urban planning that is still lacking. However, he remarks that this analysis cannot be carried out on the basis of the univocal relationship between signifier and signified, since in an urban environment the signifieds are like mythical creatures, extremely imprecise, and at a certain point they always become the signifiers of something else; the signifieds are transient, the signifiers remain. The hunt for the signified can thus constitute only a provisional approach. The role of the signified when we succeed in discerning it is only to be a kind of witness to a specific state of the distribution of signification. ([1967] 1997: 162) Contemporary semiology, he continues, ‘never supposes the existence of a definitive signified’, and this concept should be applied to the study of urban planning ([1967] 1997: 163). The best approach to determine a semiology of the urban environment is thus to ‘multiply the readings of the city’ ([1967] 1997: 164), not in the form of functional surveys, but rather personal interpretations of the city from different categories of readers, from strangers to the city to its inhabitants. The urban language can then be scientifically defined from these interpretations in terms of syntax, units, etc., while keeping in mind that this language can only be valid for a certain historical moment or location, as the signifieds in the urban environment are always vague and changing. As concerns the English-speaking world, already in the mid-1960s the interest in semiotics was spreading in the field of architectural and design theory, thanks to the works of Rykwert and Christian Norberg-Schulz. The latter, inspired by the writings of Maldonado (Mallgrave 2005: 372), penned the book Intentions in Architecture (1963) where he introduced Morris’s triad of pragmatics, syntactics and semantics in architecture, that were to be used by Mario Gandelsonas and David Morton in their analysis of Peter Eisenmann’s and Michael Greaves’ architecture (1972). NorbergSchulz’s following works will abandon semiotics in favour of a phenomenological approach based on the theories of Husserl and Heidegger (Norberg-Schulz 1979, 2000). However, he maintained the importance of meaning in the built environment, affirming that ‘[t]he man-made environment where he lives is not a mere practical tool or the result of arbitrary happening, it has structure and embodies meanings. These meanings and structures are reflections of man’s understanding of the natural environment and his existential situation in general’ (1979: 50).

POSTMODERNISM The Postmodernist movement has particular relevance in the study of the semiotics of architecture and spatial design, as one of the core ideas of the movement regarded the meaning of the built environment, a topic that in one way or another is still one of the main cruxes of the architectural discourse today. Some of the first oppositions to the Modern movement came in the field of urban planning, with the writings of Jane Jacobs, who criticized the direction city developments had taken in North America and elsewhere in her seminal book The Death and Life of Great American Cities ([1961] 2016). One of the early texts that criticized orthodox modernism was from architect Robert Venturi, with Complexity and Contradiction in Architecture (1966), which was particularly popular in the English-speaking world. Venturi laments how before the

SEMIOTICS IN ARCHITECTURE AND SPATIAL DESIGN

285

modernist movement architecture was capable of ‘embodying multiple levels of meaning simultaneously’ (Watkin 2015: 660) and indeed shares many of the same ideas as Jacobs. Together with Denise Scott Brown, and Steven Izenour, Venturi wrote the controversial book Learning from Las Vegas ([1972] 1988), in which they used semiotics as a ‘supportive methodological backdrop’ (Baird 2014) for their attack on modernism. Venturi coins the ‘Duck versus Decorated Shed’ concept whereby buildings should not directly represent their functions, they should not be iconic signs of their functions. For Venturi, in modernist buildings, the structure, construction and volume are the decoration. To keep in line with the spirit of the age, an electronic society based on signage, the ‘decorated shed’ should be a simple building with attached signs, not unlike the ones on the strip in Las Vegas. The duck is the special building that is a symbol; the decorated shed is the conventional shelter that applies symbols. We maintain that both kinds of architecture are valid – Chartres is a duck (although it is a decorated shed as well), and the Palazzo Farnese is a decorated shed – but we think that the duck is seldom relevant today, although it pervades modern architecture. (Venturi, Scott Brown, and Izenour [1972] 1988: 87) Venturi draws a parallel between the use of styles and symbols in eclectic and contemporary architecture: ‘The hamburger-shaped hamburger stand is a current, more literal, attempt to express function via association but for commercial persuasion rather than theological refinement’ ([1972] 1988: 114). For Venturi, symbolism is essential in architecture, and the source material for this symbolism ought to be a model from a previous time or from the existing city ([1972] 1988: 131). Venturi’s approach is not strictly semiotic by his own admission ([1972] 1988: 131); despite sharing the same vocabulary, it is not an a priori theorization, but rather the theory is derived by practical examples. He shares however the same ideas as Alan Colquhoun about the necessity for representation in art, and thus architecture, and how architectural design should be the result of previous associations or aesthetic ideologies, rather than the application of physical and mathematical laws. Bruno Zevi as well mentions how ‘semiology is certainly essential, but by itself cannot solve architectural problems’, adding that the issue is not if architecture is a language, because regardless architects do communicate and ‘speak architecture’, so the important thing to do for modern architects is to ‘set down precisely what it implies to speak architecture in an anticlassical key’ (1978: 4). In Zevi’s opinion, one of the problems with semiotic inquiry in architecture is that in the search for a structure of an architectural language, it takes into consideration only typical, average buildings. In his opinion, the opposite should be done, and a code for architecture should be devised from masterpieces (1978: 73). Venturi suggests the use of direct symbols on buildings, the use thus of signs – iconic or otherwise – that refer to something else, placed on top of ‘shelter’ normal buildings. In his opinion this is more reflective of current society and thus more appropriate: ‘A sign on a building carries a denotative meaning in the explicit message of its letters and words. It contrasts with the connotative expression of the other, more architectural elements in the building’ ([1972] 1988: 100). For Venturi ‘[d]enotation indicates specific meaning; connotation suggests general meaning’. Elements that are denotative rely on their symbology, what he calls ‘heraldic characteristics’, thus elements that are visibly referring to something other than architectural, such as written texts, or ornamentation.

286

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Connotative elements instead rely on their physical, ‘physiognomic’ qualities for their meaning, their size, position and forms. Venturi gives the example of his Guild House building where he designed a large lettering on its façade: ‘the sign saying GUILD HOUSE denotes meaning through its words; as such, it is the heraldic element par excellence. The character of the graphics, however, connotes institutional dignity, while, contradictorily, the size of graphics connotes commercialism. The position of the sign perhaps also connotes entering.’ He continues about the semiotics of the other elements of the building: The white-glazed brick denotes decoration as a unique and rich appliqué on the normal red brick. The location of the white areas and stripes on the façade, we have tried connotatively to suggest floor levels associated with palaces and thereby palacelike scale and monumentality. The double-hung windows denote their function, but their grouping connotes domesticity and ordinary meanings. ([1972] 1988: 101) In his opinion, modern architecture has shied away from denotative signs, i.e. ornamentation and symbols, to rely exclusively on connotative elements that do not refer to anything other than the building itself. By doing this, by rejecting the use of denotative signs, Venturi argues that modern architects have replaced the decoration with these very elements, not unlike the Classical Orders of the Renaissance ([1972] 1988: 101, 135). Pellegrino and Jeanneret, drawing on Eco’s theory of denotation and connotation, argue the same, that modernist architecture is a model that reduces connotations to a mere metaphorical reproduction of a bi-univocal functioning thought in which the connotation of the first level of conception is to make believe that what determines needs […] is the only possible one having regard to a good relationship to the object coded for serial production. Modern architects thus reduced secondariness to a zero degree, they functionalized the connotation. Against this reduction, to give again meaning to their project, the postmodern architects implicitly reverse the diagram. (2009: 282) What the Modern movement did was to replace the symbolism of historical eclecticism with the industrial vernacular, using typological models based on their interpretation of the building technology brought forth by the industrial revolution because this type of design represented modernity (Venturi, Scott Brown, and Izenour [1972] 1988: 135). Venturi also points out how the classic structures of the Industrial Revolution, but also later on, such as the art deco factories of Albert Khan, were indeed what he would call a ‘decorated shed’ and were ornamented with a language that we now perceive as classic ([1972] 1988: 134–5). Venturi’s criticism of the modern movement is thus not on functional or technological grounds, but rather symbolic: ‘What we criticize is the symbolic content of the current Modern architecture and the architect’s refusal to acknowledge symbolism’ ([1972] 1988: 137). He shares a view very similar to Zevi, when he affirms that ‘the I-section on Mies van Der Rohe’s fire-resistant columns, for instance, is as complexly ornamental as the applied pilaster on the Renaissance pier or the incised shaft in the Gothic pier’ ([1972] 1988: 114). The problem with this symbology, in Venturi’s opinion, is that van Der Rohe’s

SEMIOTICS IN ARCHITECTURE AND SPATIAL DESIGN

287

I-beams are supposed to represent current technology, whereas it was a technology developed and already employed on a large scale since the Industrial Revolution. For Venturi, the ‘electronic technology’ was the real ‘modern’ technology, and it is from this semiotic association that he derives his idea of the ‘decorated shed’ ([1972] 1988: 115), a simple building with added symbolic signage. Venturi calls ‘heroic and original’ the kind of architecture that ‘gives off abstract meaning – or rather, expressions – recognizable in the physiognomic character of the architectural elements’. This is the design of brutalist or modernist buildings, where their ‘meaning’, or lack thereof, comes exclusively from their textures, scales and articulation of forms, all directly related to the architecture itself. Conversely, an ‘ugly and ordinary’ building ‘includes denotative meanings as well’ ([1972] 1988: 115). The symbolism of architecture also has social implications for Venturi. He advises to ‘learn from Levittown’ as much as to learn from Las Vegas. Levittown was a series of largescale suburban developments of single-family homes built between the late 1940s and 1950s for returning Second World War veterans, and it became the prototype of massproduced North American suburbia. Venturi argues that Modern architecture symbols are too detached from the understanding and taste of the common people, and they are built only ‘to suit their own particular upper-middle-class values, which they assign to everyone’ ([1972] 1988: 154). In Venturi’s opinion, buildings with social values should take inspiration not from the ‘industrial past’, but rather ‘from the everyday city around us, of modest buildings and modest spaces with symbolic appendages’ ([1972] 1988: 155). Zevi has a social outlook on the language of architecture as well. He considers that the language of classical architecture – in which he includes modernist architecture, in a reasoning similar to what we have read in Venturi et al. – is a result of political absolutism, and in fact facilities such as barracks, prisons and military installations display a rigidly geometric pattern (1978: 20). Five years after Venturi’s text, in 1977, the late Charles Jencks published what is now one of the most influential texts on the semiotics of architecture, The Language of PostModern Architecture ([1977] 2002), whose semiotic ideas gave much of the intellectual ‘verification’ to the Postmodern movement (Baird 2014). Jencks puts a date and place for the death of Modernism, with the demolition of the social housing complex Pruitt-Igoe in St. Louis in the United States in 1972, a series of buildings that embodied all the ideals of the CIAM (the Congress of International Modern Architecture) and even won an award from the American Institute of Architects in 1951 ([1977] 2002: 9). Similar episodes happened elsewhere in the world, such as in Naples, Italy, where the notorious housing project Vele di Scampia – suitably the set of crime film Gomorrah (Garrone 2008) – was demolished starting in 1997, about twenty years after its completion. The reasons were the same as the Pruitt-Igoe complex: rising maintenance costs, problems of criminality and disenfranchisement of its inhabitants. Jencks articulates a theory of architecture as a semiotic practice, in which ‘formal signifiers’ such as ‘forms, spaces, surfaces, volumes’ come to stand for signifieds as diverse as ‘space concepts and ideologies’ and ‘social customs and anthropological data’. Jencks argues that clear signage in the shape of the objects represented – or rather advertised – is easily understood by viewers and is thus an iconic sign with only a few denotative meanings rather than a symbol ([1977] 2002: 30). We can clearly see the influence of Venturi in these words, especially since Venturi also argued how the ‘relevant revolution today is the current electronic one’ ([1972] 1988: 151), to contrast it with the Industrial Revolution which was the basis of the Modern movement.

288

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Jencks follows the semiotic theory of Peirce distinguishing three types of signification: icon, index and symbol. He finds that architecture is similar to language, as we can find it organized in words, through syntax and semantics. The words are the architectural components: doors, windows, walls, etc.: ‘the architectural language, like the spoken one, must use known units of meaning’ ([1977] 2002: 34). However, ‘architectural words are more elastic and polymorphous than those of spoken or written language, and are more based on their physical context and the code of the viewer for their specific sense’ (Jencks [1977] 2002: 35). It doesn’t matter where we read a piece of poetry, whereas it changes a lot where we place a building, geographically speaking. Not unlike what was mentioned earlier in this chapter about the similarities between language and the classical architectural orders, components-words need to be assembled following a syntax: A building has to stand up and be put together according to certain rules, and methods of joinery. The laws of gravity and geometry dictate such things as an up and down, a roof and floor and various storeys in between, just as the laws of sound and speech formation dictate certain vowels, consonants and grammatical formations. These compelling forces create what could be called: syntax of architecture – that is, the rules for combining the various words of door, window, wall, ceiling and so forth. ([1977] 2002: 41) Jencks gives the example of Peter Eisenmann’s House III (1971), where the final, actual design signifies the creative process that led to that very design. Eisenmann calls it Cardboard Architecture, as the cardboard model is the ‘ultimate signifier’ (Jencks [1977] 2002: 42) of the design process. Mies van Der Rohe’ architecture exemplifies for Jencks the problem with modernist architecture from a semiotic perspective: Another masterpiece of the Modern Movement, the Chicago Civic Center designed by a follower of Mies, shows similar confusion in communicating the diversity of its content. The long horizontal spans and dark corten steel express ‘office building,’ ‘power,’ ‘purity,’ while the variations in surface express ‘mechanical equipment.’ All this is as intended, as far as it goes, but the primitive (and occasionally mistaken) meanings do not express anything deep or complex about working in the city. On a literal level the building does not communicate its important civic functions, nor the social and psychological meanings of this significant building task (a meeting place for the citizens of Chicago). ([1977] 2002: 13) Jencks affirms that Modernist architects avoided the use of symbolic signs ‘because they felt these historical elements signified lack of creativity’ ([1977] 2002: 35). In fact, they hoped to create a universal language based on functional types: ‘These signs would be indexical (either directly indicating their use, like arrows, linear corridors), or else iconic, in which case the form would be a diagram of its function (a structurally shaped bridge, and Venturi’s duck). Modern architectural words would be limited to these types of signs’ ([1977] 2002: 35). In Jencks’ opinion, this type of architectural language cannot work, as every living language is based on learned conventions, and thus on symbolic signs. Even more so in architecture, he continues, symbolic signs are more prevalent than

SEMIOTICS IN ARCHITECTURE AND SPATIAL DESIGN

289

iconic or indexical signs, for instance, the pitched roof. Reviled by the modernist, it was removed from every project, including housing. In Le Corbusier’s Cité Fruges, designed in the typical 1920s Modernist style, with flat roofs, terraces and strip windows, the inhabitants of the houses made extensive modifications to the design, adding mouldings and mullions to the windows, reducing their extensions, and some even added pitched roofs, in a search for the symbolic signs usually associated with architecture designed for human habitation. The authorities deemed the modifications ill-suited and reverted the buildings to their original look (Jencks [1977] 2002: 36). More focused with the social aspect of space is the work of Henri Lefebvre. In his La production de l’espace ([1974] 1991) he deals with how space can be read and decoded, and thus its semiotics. Lefebvre argues that semiotics can ‘be applied only to spaces already produced, and hence could not help us understand the actual production of space’, while at the same time conceding that of ‘the fact that space can signify there can be no doubt’ (Lefebvre [1974] 1991: 160). In his opinion, a produced space can be read and decoded, and while there is no ‘general code of space … there might have existed specific codes, established at specific historical periods and varying in their effect’ (Lefebvre [1974] 1991: 17) that could be understood by the members of that particular society. He applies Roland Barthes’ five codes ([1970] 1974) to the ‘reading’ of St. Mark’s Square in Venice by a visitor. First, the cultural code, with which visitors recall information they might know about Venice, the square and its monuments; then the proairetic code, similar to a ‘functional analysis’, where visitors know, or guess, the raison d’être behind the various buildings; these buildings also have symbols, hence the symbolic code, such as the lion, which, even if not understood clearly, are evidently standing for a meaning other than themselves. Other elements, such as personal feelings linked to the place, compose the semantic code, an additional extra-literal layer of meaning, and finally, ‘the simple empirical evidence of the paving-stones, the marble, the cafe tables’ pushed the visitor to ‘ask himself quite unexpected questions – questions about truth versus illusion, about beauty versus the message, or about the meaning of a spectacle which cannot be “pure” precisely because it arouses emotions’ ([1974] 1991: 161), i.e. the hermeneutic code. In Lefebvre’s opinion, even five codes, a number he deems arbitrary, might not suffice, and in fact he proposes an additional two: the ones of body and power. A person experiences space with all five senses, including smell and taste, and especially hearing, so it is not exclusively a visual experience, a factor that would likely put our experience of space in the field of biosemiotics. On the other end, Lefebvre enunciates power as an additional code, or rather an entity, identified in the state, that ‘has control of all existing codes’ ([1974] 1991: 162). More recently, Gunnar Sandin developed a temporal methodological approach to space called SMAMS (Sequential Merging of Actantial Models of Space) in which he combines Bruno Latour’s ‘emphasis on letting the actors themselves decide the grouping of will and matter’ with Foucault’s otherness approach and Manar Hammad’s typological variations. Sandin refers to a domain of space semiotics that he labels ‘actantial’ based on actors and the interrelation between the environment, culture and mind (2012: 174). In Sandin’s opinion there are two main approaches to spatial semiotics, a Greimasian structuralist one, which he calls ‘typological’ (2012: 183) where there is a finite set of principal actantial types – owners, visitors, authorizers and material partitions – exemplified in the writings of Hammad ([1990] 2002), and an ‘open’ (Sandin 2012: 183) model represented by the theories of Latour (2005). Sandin affirms that ‘both approaches render the production of space in societies or communities from an agency perspective, and these

290

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

models allow an analysis of architectural and urban space, that regards the environmental and material circumstances as vital’ (2012: 170). Hammad ‘operates the notion of actant on a systematic semiotic level, in a constrained typology of agency or actor-types’, while Latour ‘uses the notion of actant first of all in order to make room in sociology also for the non-human agency of spatial production’ (Sandin 2012: 178). In regard to Foucault’s heterotopia, Sandin states that they ‘bring vagueness in respect to what is more precisely pointed out’ since in his opinion ‘actors are found as they are studied, rather than thought up beforehand’ (2012: 170). At the same time he concedes that a ‘heterotopic point of view may nevertheless help to deconstruct authoritative categorization by way of making clear that specific places depend on the societal web and its overall historical authorization mechanism’ (2012: 170), and that ‘Foucault’s heterotopology has the ability to show that what we ordinarily think of as strictly separated spaces are – if not ideals, or illusions – mutually intertwined in defining each other’ (2012: 177). Sandin recalls an experiment done in 1984 by Hammad in Le Corbusier’s La Tourette, during a conference held in the former monastery. The experiment consisted in engaging in a series of rule-breaking behaviour related to the sociological rules of space, such as asking guests to change their room for reason of ranking, or occupy the wrong place at the dinner table. This experiment outlines how spatial access is a matter of time, based on temporal division of space (Sandin 2012: 181). Latour, on the other hand, criticizes the tendency to use predetermined categories that are invariable regardless of the analysis, and in his opinion, actors in a space are ‘able to propose their own theories of action to explain how agencies’ effects are carried over’ (2005: 57). It is through a combination of these two actantial analyses – Hammad’s ‘typological’ and Latour’s ‘open’ analysis – that we can ‘locate possible conjunctions and disjunctions of actants, and see how actants rise and expire, grow and diminish, transform or stay intact as the sociological process goes on’, and investigate the agencies of built space production (Sandin 2012: 183). Sandin suggests starting analyses of space with general spatial models, such as Foucault’s heterotopias or Lotman’s semiosphere, followed by Hammad’s typology as well as Latour’s actor-network-oriented theories.

CRISIS OF REPRESENTATION In 1982 the Graduate School of Design of Harvard University held a debate between the – then and now – avant-garde architects Peter Eisenman and Christopher Alexander on the topic of harmony and order in architecture. The debate confronted the opposite theories of the two architects, and while not touching on semiotics specifically, the topic of what the built environment should communicate was indeed a central issue, and one that is still relevant today. Eisenmann argued that the role of art and architecture ‘might be just to remind people that everything wasn’t all right’ since disharmony is part of the ‘cosmology that we exist in’, whereas for Alexander, architects and designers’ works should bring harmony and comfort to the users (Eisenman and Alexander 1983). Zevi, like Eisenman, advises for a ‘neurotic anxiety for certainty’. He argues that nothing can be done about it and it is part of everyday life, such as realizing that the Earth spins while it looks still to us, and that trying to avoid it is only ‘[f]ear of freedom and horror of irrational impulses’ (Zevi 1978: 11). Alexander also brings up the point that on the theoretical level, he finds himself agreeing with several architects, who then go and build something that looks radically

SEMIOTICS IN ARCHITECTURE AND SPATIAL DESIGN

291

different from what they seem to preach. This is an issue on which architectural historian James Curl concurs, who, while agreeing with much of Jencks reasoning, finds that the changes that Jencks argues Postmodernism brought are hard to see, and on the contrary the sense of alienation might have gotten worse, with still no recognizable meaning in architecture (Curl 2018: 305). Deconstructivism, too, is in his opinion hardly a ‘coherent style’; the difference is that it expresses its ‘sense of dislocation … by means of deformity, distortion, fragmentation and the awkward juxtapositioning of jarring, disparate grids, in opposition to established built fabric’ (2018: 307). Curl shares the same ideas as Nikos Salingaros when he writes that ‘[a]rchitecture only succeeds as architecture as an expression of gravitational control and stability: if it fails in these respects, it induces anxiety’ (2018: xxxiv). Onians affirms the same: ‘the post, pillars, and columns which have assured people in many cultures of their buildings’ structural stability have been just as critical in resolving other uncertainties and anxieties’ (Onians 1988: 3). Again, the meaning of architecture, in this case as expressing gravitation and structural laws, is central to the discourse.

THE SCIENTIFIC OUTLOOK Nikos Salingaros, drawing from the theories of Alexander (2002), with whom he collaborates, developed a theory of architecture based on mathematics, in particular the fields of fractals, complexity and – most interestingly from a semiotic point of view – information theory. In Salingaros’ opinion, the search for meaning in our built environment is a natural one, since ‘[w]e seek intelligibility and meaning from our environment and are repelled by environments that convey no meaning, either because they lack visual information, or because the information present is unstructured’ (Salingaros 2013: 100). In Salingaros’ opinion, the issue with modernist design is mainly one of information, of how humans process the visual message that they communicate, in short – despite his framing it in different terms – of semiotics. Research demonstrates that our brain has evolved to process visual information, and uniform fields and surfaces do not elicit a response. For this reason, ‘[m]inimalist surfaces and edges negate the way human beings have evolved to process information’ (Salingaros 2013: 107) and this lack of stimuli leads to depression; this opinion is shared by Nigel Coates, who states that ‘most architecture today deliberately avoids emotional engagement with its user’ (2012: 11). The reaction to unpleasant stimuli leads instead to the same reaction as to a disease or an injury, raising the levels of stress. In Salingaros’ model, ‘meaning is not assigned to external forms’ (2013: 188), but rather to how much the architectural forms are coformal to our brain process patterns. Architectural forms thus do not necessarily produce a ‘meaning’ in the proper sense of transmission of information, at least not in their simplest forms, where the ‘meaning’ can be limited to a positive or negative emotion (Salingaros 2013: 188). Conversely, if the architectural forms are incoherent, even in the presence of information, that would result difficult to synthesize and in turn ‘generates a negative emotion’ (2013: 188). This approach to the reading of architecture might generate an issue, from a semiotic standpoint, that was already addressed by Eco ([1968] 2016) and Maldonado (1977) about what should be considered a ‘sign’ or a ‘signal’. Maldonado pointed out that in architectural semiotics there is a confusion regarding what should be considered a sign or a signal, due to the fact that many definitions are directly borrowed from the study of

292

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

language and semioticians such as Peirce, Welby or Saussure. A bright light that makes one’s eyes close is just a natural impulse, and the reaction is triggered without an exchange of meaning. Eco however adds that if a light is pink and from the east, the receiver can read the signal as a sign, interpreting it as the sun that is about to rise. He thus adds nonintentional signs to the domain of semiotics, as long as there is an exchange of significance for which the receiver possesses the necessary code ([1968] 2016: sec. B.1.I.3). Back to Salingaros, visual information should thus be arranged in ordered fractal patterns, not dissimilar from arranging text along lines as in this book, so that we can easily read it. Salingaros mentions a pathology of the brain that leads to the inability to recognize visual patterns called ‘visual agnosia’, where ‘a person perceives detail but cannot integrate this information to recognize an overall form’ (2013: 111). In his opinion, if the built environment communicates to us through broken patterns and forms, human beings react in a manner similar to visual agnosia, as we are incapable of relating to it. The visual information presented by organized fractal patterns, for example in ornamentation, is different however from a written message. Unlike verbal or written texts, ornamentation and architectural forms communicate mostly through the subconscious, rather than with a readily understandable meaning. This idea could actually answer many of the reservations expressed about the validity of architecture as a language, such as the ones expressed by Maldonado, who argued that the lack of an ‘articulated semiosis’ precluded architecture from being a language proper (1977: 10). In fact, already in the seventeenth century, architect and scientist Claude Perrault, author of the famous colonnade of the Louvre, affirmed that architecture cannot be assimilated to language, since it lacks a common basic system, such as musical notes, that is universally shared. Thus, architecture should be based on habits and customs, rather than natural or scientific principles ([1683] 1993: 52). In a rebuttal that will find confirmation in Alexander and Salingaros’ studies, Étienne-Louis Boullée contested Perrault a century later, affirming that the fact that they did not find a scientific proof of a basic principle of architectural beauty does not mean that it does not exist, especially since many architects, in which he includes Perrault, seem to have been able to put it to work. Boullée believed that there is a natural proportional system that governs architecture, and that the work of the architect is to understand it and put it into work ([1790] 2005: 21–2). This system is different however from musical harmony, and it is independent from it, but follows another set of rules, of which Boullée admits he was unaware, but that modern mathematical and neurobiological research, such as that of Alexander (2002) and Salingaros (2013), is uncovering. These ideas, that architecture and our visual perception are rooted and obey a certain set of rules, albeit not necessarily the same as an existing code, such as language or music, have important implications for the semiotics of architecture. C. F. Munro stated that We should not expect architecture, if it is such a system, to resemble language, a different system, in all respects. We should perhaps try to see architecture as making its own ‘statement’, in its own terms, a statement that we may only with difficulty translate into another semiological system, such as cookery, or clothing, or even language. (1987: 122) The origins of architectural language were of great interest to architect John Soane as well, who in the nineteenth century claimed that architecture ‘speaks a language of its own … and above all, a building, like an historical picture, must tell its own tale’

SEMIOTICS IN ARCHITECTURE AND SPATIAL DESIGN

293

(Watkin 1998: 81). The independence of architectural, geometric language from spoken or written ones, entails that it could be considered a sort of ‘universal visual language’ that can somehow be understood independently of culture (Salingaros 2013: 174). For Salingaros there are two languages that guide architecture and urbanism: a ‘pattern language’ and a ‘form language’. The pattern language, a term first proposed by Alexander (1979), ‘is a set of inherited tried-and-true solutions that optimize how the built environment promotes human life and sense of well-being’ (Salingaros 2013: 273), not too dissimilar from the decor, the appropriateness that we can read in the treatises of Vitruvius and Alberti. The form language is instead ‘strictly geometrical’, and it is made of the architectural components that define the form of a building. Salingaros brings forth an important point regarding semiosis in architecture, reminiscent of the Threefold Laws of Meaning devised by nineteenth-century semiotician Victoria Welby. In fact, Welby’s ‘Significs’ studies encompassed more than the study of language, to include mathematic and the arts (Petrilli 2009: 255). Welby’s three levels of meaning are ‘Sense’, ‘Meaning’ and ‘Significance’. The first level of the triad is the most immediate, ‘from reference to the world of the senses understood in biological terms, the world of sensual perception, perceptual experience, to the properly human world of significance and its connection with values, ideology, and social programs’ (Petrilli 2009: 264). ‘Meaning’ is the intention behind the sign, what the creator of it desired to convey. Finally, the third level of signification is what she called ‘Significance’: how the sign is actually read by the interpretant in relation to the context, the ‘overall effect, import and value of signifying process’ (2009: 264). This aspect is relevant to Salingaros’ theory, as he affirms that ‘[t]he built environment communicates complex messages on many different levels. An architect may naively expect that his or her design communicates a specific message in the most obvious aspects of its form, materials, and surfaces. The built structure may communicate something else altogether: a mixed, much more complex message, or the message could be strongly negative’ (2013: 283). Eco as well mentioned how architects have very little control over how users ultimately interpret their creations, due to the complex codes of architectural design whose final meaning is also visible in more general, external and anthropological codes ([1968] 2016). It is thus important to evaluate the final message that buildings and planning are going to convey, i.e. their ‘significance’, rather that the personal intention of the designer, i.e. their ‘meaning’ – an opinion shared by Jencks: ‘The disparity between elitist and popular codes can be found everywhere in the Modern Movement and often the better the architect the more he attends to formal analogies and the less to obvious meanings’ ([1977] 2002: 16). Form languages and pattern languages that share a ‘comparable and compatible internal structure’ (Salingaros 2013: 294) are then joint, as occurs in music, where the sung text is combined with music.

REFERENCES Alberti, L. B. (1485), De re aedificatoria, Florence: Nicolai Laurentii Alamani. Alexander, C. (1979), The Timeless Way of Building, New York: Oxford University Press. Alexander, C. (2002), The Nature of Order: An Essay on the Art of Building and The Nature of the Universe, 4 vols., Berkely: The Center for Environmental Structure. Aroni, G. (2014), ‘Semiotic Analysis of the Corinthian Order in the Basilica of San Lorenzo in Florence 1420–1490’, Southern Semiotic Review, 2 (4).

294

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Baird, G. (2014), ‘Semiotics and Architecture’, in M. Kelly (ed.), Encyclopedia of Aesthetics, Oxford: Oxford University Press. Barthes, R. ([1970] 1974), S/Z, trans. R. Miller, New York: Hill and Wang. Barthes, R. ([1964] 1977), Elements of Semiology, trans. A. Lavers and C. Smith, New York: Hill and Wang. Barthes, R. ([1967] 1997), ‘Semiology and the Urban’, in N. Leach (ed.), Rethinking Architecture: A Reader in Cultural Theory, 158–72, London: Routledge. Barthes, R. ([1957] 2013), Mythologies, ed. R. Howard and A. Lavers, New York: Hill and Wang. Bettini, S. (1958), ‘Semantic Criticism; and the Historical Continuity of European Architecture’, Zodiac, 2: 191–203. Boullée, É. ([1790] 2005), Architettura. Saggio sull’arte, trans. C. Casonato, Piccola biblioteca Einaudi 287, Turin: Einaudi. Broadbent, G. (1994), ‘Recent Developments in Architectural Semiotics’, Semiotica, 101 (1): 73–101. Broadbent, G., R. Bunt, and C. Jencks, eds (1980), Signs, Symbols, and Architecture, Chichester: Wiley. Ciuffi, V. (2014), ‘Rem Koolhaas Is Stating “the End” of His Career, Says Peter Eisenman’, Dezeen, 9 June. Coates, N. (2012), Narrative Architecture, AD Primers, Chichester: Wiley. Curl, J. S. (2018), Making Dystopia: The Strange Rise and Survival of Architectural Barbarism, Oxford: Oxford University Press. De Fusco, R. (1967), Architettura come mass medium. Note per una semiologia architettonica, Bari: Dedalo. De Fusco, R. (2019), Linguistica, semiotica e architettura, Florence: Altralinea Edizioni. Eco, U. (1997), ‘Function and Sign: The Semiotics of Architecture’, in N. Leach (ed.), Rethinking Architecture: A Reader in Cultural Theory, 173–93, London: Routledge. Eco, U. ([1968] 2016), La struttura assente. La ricerca semiotica e il metodo strutturale, i Delfini 20, Milan: La nave di Teseo. Eisenman, P. and C. Alexander (1983), ‘Contrasting Concepts of Harmony in Architecture’, Lotus International, 40: 60–8. Gandelsonas, M. and D. Morton (1972), ‘On Reading Architecture’, Progressive Architecture, March 1972. Garrone, M. (2008), Gomorrah, 01 Distribution. Hammad, M. ([1990] 2002), ‘The Privatisation of Space’, in G. Sonesson (trans.), Studies in Theoretical and Applied Aesthetics 3, Lund: Lund University School of Architecture. Jacobs, J. ([1961] 2016), The Death and Life of Great American Cities, New York: Knopf Doubleday. Jencks, C. ([1977] 2002), The New Paradigm in Architecture: The Language of Post-Modernism, New Haven: Yale University Press. Koenig, G. K. (1964), Analisi del linguaggio architettonico, Florence: Libreria Editrice Fiorentina. Lagopoulos, A. P. (2009), ‘The Semiotics of the Vitruvian City’, Semiotica, 2009 (175). Latour, B. (2005), Reassembling the Social: An Introduction to Actor-Network-Theory, Oxford: Oxford University Press. Lefebvre, H. ([1974] 1991), The Production of Space, trans. D. Nicholson-Smith, Oxford: Blackwell.

SEMIOTICS IN ARCHITECTURE AND SPATIAL DESIGN

295

Lévi-Strauss, C. (1963), Structural Anthropology, trans. C. Jacobson and B. Grundfest Schoepf, New York: Basic Books. Maldonado, T. (1959), ‘Communications and Semiotics’, Ulm, July 1959. Maldonado, T. (1977), ‘Architettura e linguaggio’, Casabella, 1977. Mallgrave, H. F. (2005), Modern Architectural Theory: A Historical Survey, 1673–1968, Cambridge: Cambridge University Press. Morolli, G. (1985), Vetus Etruria: il mito degli etruschi nella letteratura architettonica nell’arte e nella cultura da Vitruvio a Winckelmann, Saggi e documenti 50, Florence: Alinea Editrice. Morolli, G. (2013), La lingua delle colonne, Florence: Edifir. Morrison, J. and C. MacLeod (2016), ‘China Bans “Grandiose and Weird” Western Architecture’, The Times, 25 February 2016. Munro, C. F. (1987), ‘Semiotics, Aesthetics and Architecture’, The British Journal of Aesthetics, 27 (2): 115–28. Norberg-Schulz, C. (1963), Intentions in Architecture, Cambridge: MIT Press. Norberg-Schulz, C. (1979), Genius Loci: Towards a Phenomenology of Architecture, New York: Rizzoli. Norberg-Schulz, C. (2000), Architecture: Presence, Language, Place, London: Thames & Hudson. Onians, J. (1988), Bearers of Meaning: The Classical Orders in Antiquity, the Middle Ages, and the Renaissance, Cambridge: Cambridge University Press. Pellegrino, P. and E. P. Jeanneret (2009), ‘Meaning of Space and Architecture of Place’, Semiotica, 2009 (175): 269–96. Perrault, C. ([1683] 1993), Ordonnance of the Five Kinds of Columns after the Method of the Ancients, trans. I. K. McEwen, Santa Monica: Getty Center for the History of Art and the Humanities. Petrilli, S. (2009), Signifying and Understanding. Reading the Works of Victoria Welby and the Signific Movement, Semiotics, Communication and Cognition 2, Berlin: De Gruyter Mouton. Piranesi, G. B. (1761), Della magnificenza ed architettura de’ romani, Rome. Poldma, T. (2016), ‘Spatial Design’, in C. Edwards (ed.), The Bloomsbury Encyclopedia of Design, 1st edn, 3, 254–5, London: Bloomsbury Academic. Quincy, A. ([1819] 1981), Précis de leçons d’architecture, vol. 1, Nordlingen: Uhl. Rykwert, J. (1960), ‘Meaning and Building’, Zodiac, 6: 193–6. Salingaros, N. A. (2013), A Theory of Architecture, Amherst: Levellers Press. Sandin, G. (2012), ‘Temporal Merging of Actantial Models of Space’, in Proceedings of the 11th World Congress of the International Association for Semiotic Studies, 170–86, Nanjing: IASS Publications & Hohai University Press. Saussure, F. ([1916] 2011), Course in General Linguistics, eds. P. Meisel and H. Saussy, trans. W. Baskin, New York: Columbia University Press. Scalvini, M. L. (1968), ‘Simbolo e significato nello spazio architettonico’, Casabella, 1968. Scruton, R. (2013), The Aesthetics of Architecture, Princeton: Princeton University Press. Venturi, R. (1966), Complexity and Contradiction in Architecture, The Museum of Modern Art Papers on Architecture, New York: The Museum of Modern Art. Venturi, R., D. Scott Brown, and S. Izenour ([1972] 1988), Learning from Las Vegas: The Forgotten Symbolism of Architectural Form, revised edn, Cambridge: The MIT Press. Vitruvius, M. P. ([c.20 BCE] 1931), On Architecture, vol. 1 (Loeb Classical Library 251), trans. F. Granger, Cambridge: Harvard University Press. Citations of specific sections follow the standard book-chapter-paragraph order, separated by periods.

296

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Watkin, D. (1998), ‘John Soane: Architecture and Enlightenment’, Casabella, October 1998. Watkin, D. (2015), A History of Western Architecture, 6th edn, London: Laurence King Publishing. Westfall, C. W. (2016), Architecture, Liberty and Civic Order: Architectural Theories from Vitruvius to Jefferson and Beyond, London: Routledge. Zevi, B. (1978), The Modern Language of Architecture, Canberra: Australian National University Press.

CHAPTER THIRTEEN

Semiotics in Graphic Design STEVEN SKAGGS

INTRODUCTION Why graphic design warrants its own chapter in this book A chapter on the semiotics of graphic design, placed within a reference set that includes chapters on pictorial semiotics and multimodal semiotics, has as its first task to explain why it exists as a stand-alone. That graphic design has something to do with making visual signs and often with pictures seems apparent, so what precisely is it that distinguishes graphic design from other visual semiotic modalities? Indeed, both pictorial semiotics and the semiotics of graphic design can be seen as sub-divisions of the broader class, visual semiotics. Nevertheless, various factors make graphic design worth considering as a special class. The first factor has to do with that somewhat problematic word, ‘pictorial’. If a picture (and hence, pictoriality) is a representation of something else – a likeness of a subject – then certainly the scope is too narrow for graphic design.1 While graphic design practice frequently makes use of both photographic and drawn pictures, it is a field that ranges well beyond the pictorial, encompassing typography and abstract elements that are not clear pictorial representations of any particular subject. A second factor, in some ways an amplification of the first, is that a display of graphic design combines many modes of visual communication in a complex ensemble (a book, a line of packaging, a web site, a branding identity system, etc.) rather than being comprised a single image. Thirdly, while many pictorial efforts are the result of creativity, graphic design foregrounds problem-solving creativity in a particularly salient way. Picture-making may or may not be a creative process. For instance, a remote field camera that snaps a picture whenever an animal walks by is not deciding to make an image, nor does it make decisions about framing and composition of the image. But arranging that picture on a web site (for example) is an act of graphic design that requires making a series of decisions contributing to the fulfilment of some intended purpose. In that respect, design’s generative, creative impulse must be served in any semiotic model that tries to encompass it, while pictorial semiotics may function without an assumption of the thinking that went into the picturing. Finally, extending this notion once again, all design is planning. This places, for graphic design, a special emphasis not only upon the creative act, but also upon the analysis of purpose and intended goals, in service of which the planning happens. Purpose-driven (teleological) inputs, in pursuit of anticipated outcomes, along with study of the contexts

298

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

that are likely to shape the interpretation for the receiver, are all important in design. These problem-solving factors, even if present, are neither central to the enterprise of a semiotics of the pictorial image nor the yet more general arena of visual semiotics. But they are essential to a semiotics of graphic design.

What graphic design is Graphic designers make, compose and assemble visual entities, for an intended audience, with a particular communication-driven purpose in mind. So perhaps we can take that description and hazard a definition: Graphic design is the planning and production of entities to act as signs when conveyed through the sense of sight.2 Graphic design is a transitive, utilitarian enterprise; whether the purpose is to persuade or to inform it is always to be useful. It points to something in the world other than itself. It disappears into the environment it helps construct. In that way graphic design separates itself from fine art in which the work is, by whatever definition of art you choose, intransitively attended to for what it is (in itself), not primarily used for what it helps you to do in the world.3 Graphic design’s essential role is to intentionally function as a link outward, towards some other thought or action, without pulling the attention back to it own materiality. So, although some design may be later praised and even exhibited for its aesthetic value, or perhaps someday valued as an important achievement of a culture (thereby bringing it closer to the realm of fine art), its initial use is never purposed in that self-referential way. One may appreciate the shape and feel of a finely wrought woodworking tool, but that tool’s success as an artefact of design is to be judged by how well it functions in working wood. Graphic design is such a tool, an implement towards getting along in the world. Graphic design serves a vast variety of functions, including such wide-ranging areas as logos and branding systems, typography, font design, advertisements, informational brochures, diagrams, maps, architectural signage, wayfinding, web sites, product interfaces, international traffic signs, exhibitions, data visualization, pictographic symbol systems and packaging. With the advance of digital technologies, user experience and user interface (UX/UI) have become important parts of graphic design and a portion of this chapter is devoted to that emerging area. Virtually anything that is developed for the purpose of conveying information through our eyesight can be considered a product of graphic design. Given that graphic design is such a large area of practice, and always concerned with the streaming of meaning, semiotics is a vital area of study for graphic designers; reciprocally, analysis of the process and products of design have become an important area of research for semioticians.

HISTORY OF GRAPHIC DESIGN Taken in one respect, graphic design is as old as history, as the written word itself depends upon a set of graphic characters which were designed by someone. Yet, the term ‘graphic design’ is of recent vintage. The industrial revolution in the late nineteenth century produced an increase in both literacy and manufactured products. For the first time, the supply of products vastly outstripped local demand. Magazines, becoming an increasingly popular medium due to the increase in literacy, began to use advertising as a means of generating revenue. Manufacturers used magazine advertising as a way to inform the public of new products and to stimulate demand. Railroads allowed

SEMIOTICS IN GRAPHIC DESIGN

299

the rapid distribution of goods. Meanwhile, new innovations such as linotype and chromolithography meant that printing could be made faster, cheaper, more colourful and freshly entertaining in content. This stew of influences resulted in an intense demand for artists to work with industry to fashion a growing assortment of visual communications. At first, the duties of these artists were quite separate: illustrators made paintings, type designers made letters, and printers were largely responsible for composing them for the press (Meggs 1983). But by the 1920s the execution of these various tasks were beginning to be placed into the hands of artist-managers (in advertising they called them ‘art directors’) who produced and coordinated visual content and delivered them to the printing house (Shaw, Jury). In 1922, W. A. Dwiggins, in an essay for a Boston newspaper,4 recounts the various duties the ‘advertising man’ is expected to perform, including ‘advertising artist’, ‘printing designer’, ‘artist’, ‘designer’, and apparently for the first time a new locution: ‘graphic design’. The term grew in prominence only after the Second World War, as universities began including graphic design curricula.5 By the 1960s, the functional-rationalist ideals of modernism, adopted by multinational corporations and prolifically spread through their expanding influence, had become so prevalent that, at least in the most technologically developed countries, the vernacular sign painter and traditional printer-composer were rapidly disappearing.6 A new selfawareness developed within the design community that realized graphic design as a powerful and increasingly eloquent communicative practice. Simultaneously, within boardrooms and marketing departments, the power of systematic visual communication planning was recognized as essential. Perhaps best represented by the International Style promoted by the Schule für Gestaltung Basel (Basel School of Design), modernist design of the late 1950s and 1960s employed sans serif typography, grid systems as organizing devices and favoured a minimalist approach to composition. Rational and coolly efficient, modernism expressed the authority of objectivity and the values of science. It aimed for a seamless and transparent process of interpretation on the part of the receiver, one in which the goal was to deliver ‘a content’ as directly and clearly as possible. Ideally, the viewer would be unaware that the display was even a designed artefact; rather the information was expected to be ‘handed over’ – immediately, intact – as if it were brute fact, a force of nature speaking with unquestioned assurance. Modernism’s goal of efficiency at once assumed, but hid, the sender’s authority. Whereas  an advertisement in the 1930s would bear traces of a particular artist’s hand  – embrace quirks of taste in illustration and in the peculiarity of layout, employ hand lettering or calligraphy – high modernist advertising in the 1960s was impersonal; neutral typefaces such as Helvetica had supplanted hand lettering, layouts were less idiosyncratic, and hand illustration was eschewed in favour of photography. The artist behind the message was de-emphasized in modernist design and this had the effect of ‘naturalizing’ the claims made in the messaging, resulting in an implicit dominance of sender over receiver. Ultimately, this elicited a late-twentieth-century reaction. The first impulse of this reaction was seen with the rise, in the late 1960s, of counter-cultural psychedelia, typified by the music venue posters of San Francisco’s Haight-Ashbury district, which luxuriated in the experience of florid materiality while content was obfuscated. Later, in a more consciously theoretical way, experiments of deconstructionist typography in the 1980s and 1990s heralded the rise of postmodern graphic design, in which the sender’s implicit

300

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

authority was challenged in a number of ways.7 While the first wave of reaction in the 1960s was embedded in the experiential lifestyles of the ‘tune-in, drop-out’ culture, this second wave was consciously semiotic, drawing fully from Barthes, Derrida and other continentalists.8 Since the 1990s it is impossible to put one’s finger on a dominant style of graphic design. There is, however, a generalization that might be suggested: graphic designers today work with a consciousness of this history as well as the social milieu within which the planned communication will be seen. Products of strong academic programs in Europe, Australia, USA and Asia, designers have taken design history and marketing courses that reinforce the interdependency of the artists and the communities they serve. Today, one is likely to find semiotics, if only at a rudimentary level, introduced within a design studio curriculum. Graphic design is evolving to be less about the technologies used in production and more about the essential and deeply semiotic transaction that occurs in a visual communication event. Given the upheavals in digital technology and the increasing rate of disruption in design’s production tools, a focus on the fundamentals of how people make meaning offers a comparatively welcome stability.

SEMIOTIC METHODOLOGIES IN GRAPHIC DESIGN Semiotics can be used at the front end of the design process, during the ideational, creative stage, or at the back end after the designed pieces are distributed into the world. Whether applied during the creative phase or in post-hoc analysis, three currents of semiotic thought tend to find favour: Saussurean/poststructuralist, pragmatist and transitional. The fundamental principles of these approaches are addressed elsewhere in this reference collection (see, e.g., Vol.1, Chps. 3–4).

Saussurean and poststructuralist methods The semiology that evolved from Ferdinand de Saussure has had a profound effect in all areas of communication studies. Based on a linguistic foundation, using a dyadic notion of the sign, Saussurean concepts developed to have a broader, extra-linguistic reach with Roland Barthes and the ‘Paris School’ advances of A. J. Greimas and Louis Hjelmslev. These later adaptations developed a semiology that was able to regard any mode of cultural transmission as a networked structure of codes along interacting planes of expression and content. Poststructuralist semiology reached its apparent end-state with the ‘deconstructionist’ ideology of Jacques Derrida who challenged all tidy architectonics of structuralism and stressed the unfixed nature of polysemy. Special note must be taken of this move by Derrida because it had a particular catalytic effect on radicalized design. Influenced by the Derrida’s writings, Katherine McCoy and her grad students at the Cranbrook Academy9 began to explore the visual ramifications. McCoy encouraged her students to force attention onto the materiality, rather than content, of words and images. In contrast to work in the modernist mainstream of the time – clean, efficient and invisible communications – the work issuing from Cranbrook in the late 1970s through the 1980s required considerable collaboration on the part of the viewer (Figure 13.1). Here is playfulness rather than authority, improvisation rather than rationality, opacity rather than transparency; a reader must slow down, take part in a game of transcription, always conscious of the process of encoding and decoding. Looking and reading are set

SEMIOTICS IN GRAPHIC DESIGN

301

FIGURE 13.1  Poster, Cranbrook Graduate Program in Design, 1989; Designed by Katherine McCoy (American, b. 1945); offset lithograph on heavy perforated card stock; 71.1 × 55.7 cm (28 × 21 15/16 in.); Gift of Ken Friedman; 1997-19-287.

side-by-side, the receiver alternating between being a reader and a viewer. Word is not privileged over image. Image is rarely used denotatively. Background and foreground, the most fundamental of gestalt dichotomies, often flip positions. We are made aware that we are looking at; indeed, it is only with effort do we manage to look through to the denotative content. Ironically, this direction – the style came to be called ‘decon’ – is, in part, a misreading of Derrida. Derrida’s notions of indeterminacy, play, erasure, différance, were targeted at conventional messaging that pretended to be clear and distinct. Derrida’s whole point was that language (especially the visual writing and printing of texts) implicitly hides or covers up its own instability; by intentionally and explicitly destabilizing the signifier, decon in a sense inverts Derrida’s point. However, by making the destabilization so emphatically visible, McCoy’s experiments forced the audience to be aware of the points Derrida was

302

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

making. The receiver questions the process of making and of interpretation, feels the tension between reading and looking and becomes mindful of the manipulation that has been enacted by the sender of a message.

Denotation and connotation Another contribution from the poststructuralist tradition that merits more detail because of its influence on design methodology is the distinction between denotation and connotation. While C. S. Peirce in also making this distinction, bases these terms along narrow logical lines, the version of the connotation-denotation dichotomy that has had more impact among designers is that which ensues along structuralist lines, especially those suggested and popularized by Roland Barthes (1957, 1977). In this conception, denotation is a signification that is direct and more explicitly code-dependent, while connotation is indirect, more nuanced, a second-level code of common association. The structuralist understanding of the denotation-connotation distinction is similar to the conventional dictionary description of these terms, with denotation being highly precise and prescribed by the code, and connotation being those associated implications which the sign arouses. Jean-Marie Floch (1995 [2000]) provides an example of a semiologist applying poststructuralist procedures, especially foregrounding in his work the role of connotation. His Visual Identities provides six case studies of detailed connotative analyses of brands, logos, packaging and advertising. His essays typify the ways in which the ‘continental’ tradition of semiology makes room for (often idiosyncratic) branching connections of connotations, memories, associations and transferences, each of which he accommodates within the notion of ‘bricolage’. Connotations, taken in the manner in which Floch uses the term, are perhaps the most covert, and most important, mode of communication in graphic design. Unlike denotation, in which the referent is highly constrained, connotation is often highly idiosyncratic. A person who has been bitten by a dog may have a heightened aversion to seeing an unleashed dog. Both the dog’s owner and the person with the fear of the dog recognize the animal, but the effect of that recognition softens to love on the part of the owner and hardens to fear for the once-bitten neighbour. Their personal histories contribute to the connotative meaning of the object. Shared connotations develop a sense of shared culture, and indeed, can be the basis for the connotations evolving into a symbol. We see this in the rise of popularity of certain humanitarians or heroes who, once their good works become widely recognized among a people, come to attain hero status and stand for an admirable life. Connotation is especially critical in graphic design because it is a visual discipline in which, unlike verbal language, the constraint of explicit coding is often absent. Display a photograph of a politician to a sympathetic or an unsympathetic audience, and although they may agree on the identity of the figure, their reactions are diametrically opposed and depend wholly on the associations elicited by the subject. The associative references are often more important to the message than the mere denotative identification of the politician. Other than indexical functions such as navigational efficiency, the choice of compositional style in graphic design is determined by intended connotation. For instance, the use of a grid as a compositional method connotes European modernism, while the use of symmetry in combination with ribbons and panels as graphic framings

SEMIOTICS IN GRAPHIC DESIGN

303

FIGURE 13.2  Compositional styles carry semantic connotations.

of text often connotes American vernacular (Figure 13.2). These connotative structures, supplemented by stylistic choices of typeface and imagery, contribute to establishing and working within (or defying) genre.

Pragmatist methods Pragmatist semiotics stems from the late-nineteenth- and early-twentieth-century work of Charles Sanders Peirce, John Dewey and Charles Morris. A fundamental premise in pragmatist semiotics, especially distinguishing it from Saussurean/Structuralist semiology, is its conception of a sign structure that is triadic rather than dyadic. Instead of a sign consisting of a signifier/signified, pragmatist semiotics conceives of any interpretable instance, or ‘semiotic moment’ (Skaggs 2017a: 42–9) as constituting a relation between a sign, its referent and an interpretant.10 For graphic design, pragmatist approaches offer two immediate allures. First, this three-part division resonates with a designer’s everyday work life. To understand this deep affinity, consider a fundamental situation in which a design studio is asked to design a piece and to later test whether their efforts have been effective. After an initial meeting with the client, a designer knows she has certain content to be put across (the referent); then she fashions a visual entity (the sign) that is intended to do the work of standing for the referent; which visual entity, upon being seen, engenders an effect (the interpretant) on members of a public. This process is then carried forward in a follow-up analysis: the test result is a sign; it refers to the effect the visual entity had on the public (the former interpretant); these results are now evaluated (the conclusion is now a new interpretant). This triadic chain, or semiosis, comprises stages with which every designer is familiar. This example is just one possible framing, but even here, in this basic exchange which is a fundamental and universal part of design life, there is an affinity between pragmatist ideas and a designer’s world. But the pragmatist notions take place in many ways, many

304

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

dimensions, many scales, that offer more analytical powers than the simple description of a largely commercial interaction given above. We will discuss some of these subtler potentialities and tools below when we come to the current state of semiotics in graphic design theory. The second allure, an especially sympathetic trait of pragmatism for the designer, is that unlike structuralism, pragmatism has absolutely no discernible nod, tilt towards, nor birthplace in, linguistics. As a result, pragmatist semiotics is more easily adaptable to the visual problems the designer is asked to solve; in graphic design, even when words appear, they do so through the visual manifestations of typography.11 The visual component of typography is precisely all that is not linguistically coded. Apart from these affinities, the pragmatist concept that has had the largest impact in graphic design is found in Peirce’s second trichotomy: the idea that a sign can relate to its object in three possible manners – iconic, indexical or symbolic.12 Indeed, this is a notion that is universally taught in design programs in colleges and art schools. However, this contribution to the design lexicon has also had the unfortunate effect of being nearly as universally misunderstood. Too often, one hears a designer speak of designing ‘an icon’ as if the visual entity can only be an icon, or a symbol, or an index; in fact, it can be all three depending on the context (or ground) upon which one is reviewing the situation. Peirce’s second trichotomy describes the three kinds of relation that may obtain between a sign and referent; there is no strict absolutism or categorical exclusivity implied. A pictorial emblem such as the Starbuck’s logo can be iconic of a mermaid while simultaneously being symbolic of the coffee chain it stands for.13 Among design theorists whose work has been influenced by pragmatist thought are Martin Krampen, Per Mollerup, Ellen Lupton, Johanna Drucker, Thomas Ockerse, David Crow and Richard Buchannan. Many of them have found the icon/index/symbol division to be fertile ground for exploration. Crow, Krampen and Mollerup have each employed Peircean schemes in various ways to classify logos and branding programs. With respect to logos, most theorists make a division between word-based and image-based logos. Image-based logos are usually called ‘pictographs’ and logos that are based on initials or words are usually called ‘wordmarks’. Some writers have found sub-categories. Mollerup, for example, lists ten beneficial semiotic attributes of logo use, from ‘uniqueness’ to ‘repetition’, before employing the icon/index/symbol schema to develop nine distinct classes: images, diagrams, metaphors, designations, reagents and symbols.14 Martin Krampen was one of the first semioticians to work specifically with graphic design (as opposed to photography, advertising or film). His conception, in the mid1960s, of a division of graphic signs into logograms, phonograms, pictographs and diagrams pointed the way towards classification schemes and astute semiotic analysis of graphic displays and sub-elements.15 Apart from its abstruse lexicon16 the central challenge for pragmatic semiotics is its general lack of ‘prescriptivity’. For the creative designer, tasked with generating ideas, can pragmatist semiotics offer conceptual tools to aid in the creative act? Moreover, does pragmatism offer the kinds of ready hypotheses, such as exist on the structuralist and poststructuralist side, to aid in a critique of visual culture? These stumbling blocks are beginning to be removed with some ‘neo-Peircean’ initiatives that we will get to below. Pragmatism has allowed both designers and analysts to observe fine divisions in the way logos and other kinds of graphic design function. The challenge for the pragmatist school is to more fully develop methods of creation and criticism.

SEMIOTICS IN GRAPHIC DESIGN

305

Transitional methods: Multimodal, social, cognitive and more I use the term ‘transitional’ here to refer to areas of semiotic study that begin to intersect with, or merge into, traditional well-established academic disciplines such as psychology, anthropology and sociology. Transitional methods may use methods within the traditional discipline but the investigator emphasizes processes of signification from either a structuralist or pragmatist basis and sometimes using a combination of the two foundational paradigms. These transitional methods may be thought of as various ways of pursuing what Charles Morris (1971: 21) called ‘pragmatics’ – the relation of the sign to the interpreters of the sign.17 Multimodal semiotics stresses that people simultaneously interpret many kinds of media including verbal, visual, haptic and auditory. A study of meaning-making, therefore, needs to account for holistic effect of them all and not be weighted towards only the linguistic. Multimodal approaches employ a mix of pragmatist and structuralist ideas in an attempt to analyse and integrate a more comprehensive set of inputs. In most multimodal work, the social context within which the communication event takes place plays a part. For instance, whereas Saussurean approaches consider the linguistic sign as well as many other code systems to be arbitrary and unmotivated, multimodal semioticians hold any sign systems that stems from a social group’s ancestral or unconscious beliefs and practices to be motivated. Because the multimodal method considers all varieties of sense-effects as interdependent, it does not recognize ‘visual semiotics’ per se. As in social semiotics, in multi-modal approaches the interpretant tends to reside in a social group or a culture rather than in an individual mind. Multimodal approaches often use empirical data gathered in procedures borrowed from linguistics and the social sciences. A close cousin of multimodal semiotics, social semiotics, focuses on the behaviour of the communicative group, whether it be a small neighbourhood, a socio-economic class or a larger cultural unit. In its most developed form, perhaps best seen in the work of Gunther Kress, Theo van Leeuwen and Robert Hodge in the 1980s and 1990s, and Jay Lemke in the 2000s, social semiotics broadens the subject of semiotic study towards the macro level of social interactions. These empirical assessments result in conclusions about common social habits of, say, visual composition, which can then offer instruction to ‘performers’ (i.e. makers or designers).18 For example, Kress and Van Leeuwen (1996: 227–9) noted the compositional styles of daily newspapers in Britain and Germany and summarized differences in the ways their readers parsed the visual elements on the page. They concluded that the German audiences expected many short articles all visible at once; the British public, on the other hand, seemed to prefer just a few articles presented at a time, but with large photographs and big typography. For the graphic designer (performer and maker), ethnographic analysis of this sort always begs certain questions: If the British public became thoroughly habituated to the German manner, wouldn’t we expect their interpretative context and graphic style begin to change? Does style influence culture as much as culture influence style? Also, such data collection methods sometimes have difficulty accounting for innovation. They describe what is, but they have a more difficult time suggesting what might be or should be. Multimodal and social semiotics overlap with each other to a significant degree and with other practices that collect demographic tendencies. The difference is that demographic preference data (for instance in market shelf-testing prospective package

306

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

designs) tends to be mute on the reasons consumers prefer one look over another, while multimodal and social semiotics are able to draw inferences about the motivations behind those preferences. A third transitional method, cognitive semiotics, establishes connections between semiotic analysis and the cognitive sciences. Cognitive semiotics looks at the fundamental mechanisms of perception and cognition at the micro level but translates the cellular and physiological components into a semiotic framework. Cognitive semiotics treats both mind and brain as semiosis. Marcel Danesi is one prominent semiotician who has recently been working in cognitive semiotics. Cognitive semiotics also has the potential to trade concepts with integrated information theory (which will be discussed below) and other recent developments in understanding how thought (including artificial intelligence) happens. Transitional semiotic methods function as doorways into the more traditional disciplines such as neuroscience, sociology, demographics and media studies. Each of the transitional areas is able to contribute the statistical, empirical and observational practices of the conventional disciplines to which they connect, while also retaining a decidedly semiotic point of view based on pragmatist or poststructuralist foundations. The influence of these transitional semiotic methods on the planning and analysis of design process and artefacts is expected to grow in coming years.

UX/UI AND HUMAN/COMPUTER INTERACTION User experience and user interaction imply two aspects of human-computer interactions. The study of both user experience (UX) and user interaction (UI) usually entail some variety of transitional methods, especially multimodal and cognitive semiotics. User experience takes an approach that is rooted in cognitive science and the psychology of perception; user interaction a kind of ‘descriptive phenomenology’ as revealed through observations of gesture and body movements as people use digital devices. Whereas the former regards human beings as processors of information, the latter approach tends to concentrate on the actions of eyes and limbs.19 These complementary approaches are somewhat difficult to reconcile because each endorses a dualistic Cartesian premise, implying a separation of mind and body. But as semiotics sees both mind and body as secondary to the process of semiosis, it provides a useful uniting paradigm. The cognitive approach tries to understand what is going on in our heads, while the embodied phenomenological approach is better at ‘describing the way in which we inhabit media-saturated environments’ (O’Neill, 43). Semiotics concentrates on ‘the role of the stuff in the world itself in terms of how it can signify what we mean when we manipulate it’ (O’Neill, 44). Semiotics therefore promises not only a broader and more general perspective, but also an infinitely scalable one that can be employed any place along a sequence of human/computer moves. As a recently developed sub-field of graphic design, only taking hold with the advent of the personal computer in the early 1980s, interface design was not as concerned with exploring layers of meaning as providing rudimentary functionality. Before screens were ubiquitous, it was necessary to signal in very definite ways what was an interactive button, or a field waiting for input, or a ‘hot link’ to send the viewer to another page. We still see, a half-century later, the relics of this primitive time fossilized into standard

SEMIOTICS IN GRAPHIC DESIGN

307

formats: underlined text to stand for links, rectangular colour panels as buttons, the word ‘home’ to return a visitor to the entry portal. These are all methods to indexically locate fields that allow – and signal – interactivities of one sort or another, and the habit of underlining, or boxing in, or calling a location a home, are all early methods to signify the kind of interaction to be undertaken. We can expect these traditional tropes to weaken; on a screen, the simple change from a cursor to a pointing finger20 or a change in colour is enough to index the interactive elements without the clichéd redundancies of the underscore and box. Peter Andersen’s early (1991) attempt to construct a taxonomy of computer interactive semiotics isolated five specific kinds of interface signs: interactive, actor, controller, object and layout signs. Interactive signs index the location for our manipulative interactions, actor signs each perform a specific function, controller signs change the properties of other signs, object signs are the targets of the interactive signs and layout signs present a backdrop behind actions (Andersen 1991: 199–213). Within a decade Mihai Nadin and others were sensing the need for a more detailed and articulated semiotic analysis of the digital graphic environment, even calling into question the novelty of a notion such as ‘interaction design’, saying one cannot not interact with anything in life (Nadin 1997). But the degree of complexity generated by a thorough semiotic accounting of visual digital media was becoming overwhelming. So when Shaleph O’Neill’s work was published in 2008 there was a significant tangle of concepts that had been put forward and which needed cleaning up. After recounting the questions and problems alluded to above – difficulties shared to some extent by researchers of other kinds of complex visual media such as film and television – O’Neill notes that the study of the design of screens has lagged far behind its potentialities. As an example he mentions the use of the pictorial icon: although screen-based interactive media ‘are extremely semiotic in character’, participants often engage them through simulations of archaic, physical real-world forms (O’Neill, 105). The result gives us a pictograph of a metal trashcan for deletions, a manila file folder for a directory, a representation of a piece of paper for document. Certainly these icons are anachronistic and in many cases inefficient.21 In order to develop a semiotics of UX/UI, O’Neill draws from several semiotic and philosophical currents. He builds, contests and to some extent reconciles everyone from Eco and Hjelmslev, to Peirce, Barthes, Heidegger and Sebeok. Ultimately, he adopts a theory of ‘embodied cognition’ which permits him to attempt to unify the action of interaction with the semiotics of comprehension. The state of semiotics with respect to interaction is in its infancy. Perhaps the key in adapting semiotics to interactive media is to realize that whereas older media, such as posters, books and television, were meant to be read or watched as solely communicative, contemporary smart phones, laptops and other digital platforms are intended to be actively utilized to perform work. In that sense, books and television are like supervisors telling you to dig a ditch; new media is the shovel. The digital media, in addition to informing, entertaining and persuading through words and images, also permits you to work with it as an implement. But unlike conventional tools such as the shovel, typewriter or printing press – machines with the capacity to do a single kind of work in the old world – this new digital work is not linear in its affordances. The digital user is able to branch out in all directions, so that the work is highly diversified and improvisatory.

308

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

The creative process with digital media demands one final analogy. Old work with graphic machines was like playing music from sheet music. It had to be defined, written out, through-composed. The new work is like jazz. One starts with a root musical theme in a given key, but where precisely the riff travels is anyone’s guess. So how does one plan for the myriad of possible sequences and leaps that a user may play? Mapping and describing these movements represents the biggest challenge for a semiotics of interaction design.

CURRENT STATE OF SEMIOTICS AND GRAPHIC DESIGN Graphic design as a discipline has been more responsive to technological change than to self-reflective theories. Ironically, this very sensitivity to tech is now acting as a catalyst for semiotic contributions. The arrival of digital tools in the 1990s ushered in a period of especially rapid change in the methods of graphic design production. One of the results of digitalization is a new level of graphic design ubiquity. Graphic design had been present before in books, newspapers and magazines, but now the general public was not only acutely aware of design but were given access to the designers’ toolbox. Starting with the development of ‘desktop publishing’ programs in the 1990s, and continuing with programs for DIY web and app design, production techniques that had been the province of specialized practitioners within design studios and ad agencies became accessible on everyone’s device. Not only are the tools at hand, but libraries of images, either curated or open, await on continuous social media feeds. The general public is now provided with internet resources for examining visual culture from Kyoto to Santiago. Each of us is exposed to hundreds of graphic communications, from all over the world, every hour. A single newly uploaded design proliferates, influencing other designs within the week, unfettered by lethargic print schedules, magazine circulation or national borders. The ease with which one can ‘cop a look’ found on the internet puts pressure on the concepts of copyright, plagiarism and intellectual property. It’s not that these legal concepts cannot be defined (although that is difficult enough) – it’s that they cannot be policed. An inexhaustive list of some of the positive and negative implications of this widespread availability of tools and exposure to graphic design includes: Positive aspects of graphic design ubiquity 1. Accessibility to a variety visual displays expands 2. More iterations can be produced in the process of designing 3. A single individual can author, design and mass publish 4. Publishing and distribution can happen almost immediately 5. Mixing and appropriation of styles, genres and vernaculars 6. Democratization of design (no curators) Negative aspects of graphic design ubiquity 1. Concept of intellectual property under threat 2. Memetic imitation leads to ‘follow the herd’ mentality 3. Authority and expertise have decreased importance 4. Democratization of design (no curators)

SEMIOTICS IN GRAPHIC DESIGN

309

The democratization of design is listed in both categories: it can be considered a positive or a negative influence depending on one’s perspective. Global graphic design practices encourage the development of a merged visual mono-culture, mixing elements of all cultures. This is seen in choices of colour, in the return of ornament (long spurned in the West), the style of graphic elements and universal adoption of place-specific genres (e.g. tattoo art, anime and manga). Whether this globalized mono-culture effect is considered a positive development or a negative one is an open question.

Growing awareness that semiotics is the centre of graphic design This new ubiquity of graphic design practice leads one to ask: What, in the end, comprises the unique, proprietary knowledge that graphic designers possess that defines their discipline? If everyone has access to the technology of design, and if design is based primarily upon the use of technology, then graphic design would seem to have no ‘special’ place or practice. This is causing a re-examination of what it means to be a graphic designer, and leading to a growing awareness that, at heart, what we do is not defined through our technology at all. Rather, the defining expertise of graphic design lies in creative decision-making that moves towards some purposeful end, and all the various technologies simply afford a means to reach that end. Without exception, this decision-making concerns the forming of functional systems of visual signifiers. The core of graphic design is to be found in understanding the meaningladen connections between the visual signs we create and the people to whom they are communicated; semiotics, then, becomes not only essential but indeed, the very heart of the profession.

New conceptual tools Perhaps reflecting this growing realization of the semiotic essence of the practice, new kinds of conceptual tools are being developed to guide creative decision-making in the design process. A few of these are mentioned here but many more may be expected to emerge.

Page grammar In Reading Images, Kress and van Leeuwen propose that when one sees a display surface, that surface is subtly broken into five semantically entangled sections. For Western cultures, top and bottom, centre and margin take on specific ‘information values’. This entails that every fixed page can be divided into quadrants, with an added zone in the centre of the page, and these five areas suggest latent interpretations (Kress and van Leeuwen (1996: 208) (Figure 13.3). Although one must always be cautious of being overly rigid when assigning definite interpretants to general syntactical categories,22 Kress and van Leeuwen more than anyone since Arnheim (1954, 1969), reawakened interest in the semiotic dynamics of the visual compositional surface itself. Beyond introducing the information value of page position, Reading Images also opened investigations into two other semiotic qualities of graphic displays: ‘salience’ (importance, or presence), and ‘framing’ (connecting/disconnecting, belonging-to/notbelonging-to).

310

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

FIGURE 13.3  Page grammar. Diagram after Kress and van Leeuwen (1996).

Zones of interaction O’Neill introduces several important questions that present opportunities for future study in UX/UI semiotics. He suggests that interactivity may begin to blend with ‘product semantics’ (Krippendorf 2006, Vihma 1995). He extends Kress’s page grammar into ‘zones of interaction’ within UX/UI. Since interactivity implies an embodiment of visual communication in a way not present in traditional media, he introduces a semiotics of embodied cognition.

Semantic profiles and the functional matrix Several ‘conceptual tools’ for graphic design practice based on pragmatist semiotics have been introduced by Skaggs. Two of these are semantic profiles and the functional matrix (Skaggs 2017a).

SEMIOTICS IN GRAPHIC DESIGN

311

FIGURE 13.4  Functional matrix.

Based on the premise that there are four modes in which a visual display influences the viewer, divided between affective and cognitive registers, establishing a design’s semantic profile allows the ‘personality’ of the designed display to be overtly determined as a goal against which iterations may be judged. The affective register concerns presence and expression. The conceptual register comprises denotation and connotation. A degree of relative emphasis, or valence (‘salience’ in multimodal terminology), is determined for each of these four influences. The result places the project into one of a number of classes or categories of semantic action.23 Semantic profiles can be used in post-design analysis, or in the research phase of a project to target the appropriate interaction of the visual display. The functional matrix (Figure 13.4) sets up oppositions between syntactic, semantic, denotative and connotative functions. The resulting matrix is used as a guide for the designer to ensure all aspects of a project are taken into consideration. The functional matrix, somewhat like Greimas’s semiotic square, exposes certain inherent tensions in the attributes of a design – tensions which cannot be expurgated and therefor with which a designer must work.

Connotation cluster analysis Carina Ren and Anders Munk (2019) conducted a study of connotations having to do with the arctic when considered in the context of gastronomy. Their results, making use of Graph API (Application Programming Interface), data-mined over one hundred million Facebook posts and sorted them into a graphical cluster analysis (Figure 13.5). The graph, based on which concepts were most closely related, revealed four distinct gastronomical ‘basins’: drinks, meats, fish and ice desserts. This kind of massive data mining, combined with cluster analysis, is likely to prove increasingly useful in providing input for graphic design, where these techniques can reveal visual cultural ‘hot spots’. Especially when combined with semiotic devices, such information, perhaps combined with A.I. and machine learning techniques, will aid the designer in image selection, hierarchical structures of text and compositional styles.

312

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

FIGURE 13.5  Ren and Munk (2019): Cluster analysis of gastronomical terms from inhabitants of the arctic.

PRIORITIES AND QUESTIONS Integration of paradigms These new techniques and conceptual tools indicate the potential for semiotics to contribute to design thinking. But before design can effectively incorporate semiotics, it would be helpful if there were more clarity or resolution of differences in the semiotic families. Poststructuralist, pragmatist and transitional semiotics already make substantial contributions, but their differing terminologies and conceptual platforms cause confusion. It would be helpful if a single clear consolidated semiotic method for design could emerge, a prospect that is delayed by the incongruity of semiotics’ famously interdisciplinary structure. How might such a partial reconciliation (at least) come about for graphic design? One possibility is to apply each branch as a set of specialized tools operating in particular, limited domains. For instance, given its linguistic dyadic character, perhaps Saussurean semiology’s insights into the performance of explicit networks of cultural codes could mesh with transitional semiotic methods to focus on broad social, behavioural, and demographic studies where explicit coding is at play, while Peircean semiotics becomes maps a larger and more fundamental theory on semiosis (sign process) within the envelope of which the others perform.24 Two related areas will need to be addressed, one lexical, the other empirical. Terminology needs to be agreed upon, in some cases modernized, in other cases simplified. This will be difficult, fraught with disagreements among scholars, but it is important to do this work. Within pragmatist semiotics alone, there is no consensus over such basic terms as sign, sign vehicle and representamen, or ground, context and frame of reference. Have we Peirceans yet settled on a word for the triadic set of relations itself?25 How can semiotics hope to play an important role going forward if there remain important stumbling blocks around fundamental concepts?

SEMIOTICS IN GRAPHIC DESIGN

313

The empirical initiatives can be expected to come from advances in such areas as cognitive semiotics and integrated information theory. Indeed, these studies, which shift the focus from abstract paradigms to the materiality of neurons, will undoubtedly introduce their own vocabulary, stimulating the evolving lexicon. Ultimately, it would be advantageous to have a single agreed-upon paradigm that accommodates the three semiotic branches while also providing a bridge to the physicalist aspects of cognition. That’s the holy grail.

Category variance Much of semiotics has to do with classifying the behaviour of the signs before us. Category variance has to do with recognizing that something may not be uncontroversially a member of a single class. While adherence to a category may be common in linguistics or logic, classes within visual semiotics are more difficult to pin down. It is widely recognized that in Peirce’s second trichotomy (which specifies three ways a sign may refer to its referent) a sign may be at once both an icon of x while being also an index of y. But the more extreme variance I am thinking of occurs in many situations within graphic design; a sign may be both an icon of x and to some degree also an index of x. If we accommodate such ‘degree-variance’, then all three elements of the icon-index-symbol trichotomy may be present in various mixtures so that a sign’s relation to its referent is as commonly mixed as the three primary colours of paint on an artist’s palette (Skaggs 2019). So too, especially with regard to connotation, a sign generally refers simultaneously to a variety of referents, and may manage this polysemy through proportionate combination of all three modes. In this view, the semantic sign-referent relation is best regarded not as a dipole or even a tripole, but as a triangular surface, or gamut, in which iconicity, indexicality, and symbolicity constitute apexes, maximal conditions, the plane between them mapping semantic space (Figure 13.6).26

FIGURE 13.6  The Visual Gamut with fifteen nodes illustrating blends of iconicity, indexicality and symbolicity.

314

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

FIGURE 13.7  Signature of designer Saul Bass, c. 1980 (Wikimedia Creative Commons Attribution-Share Alike 4.0 International licence).

For any semiotic moment (i.e. frame of analysis), the entire panoply of reference must be taken into account, and it is expected that these multiple sign/referent relations will engage different regions of the gamut surface between symbolic, indexic and iconic nodes. So, for example, every person’s signature is both symbolic and indexic but occasionally even iconic (Figure 13.7). The increasing awareness that graphic design involves not only the binaries of word and image (symbol and icon) but also the indexicality of touch and gesture (Ingold 2018, Skaggs 2017a,) introduces new arenas of investigation for the semiotic analyst.

Generative creative concepts Should a consolidated semiotic paradigm be adopted, it will then be important to determine the best practices for employing that paradigm not only descriptively in post-design analysis of visual culture, but also prescriptively within the creative act of designing. These are questions that have not been adequately answered, and until they are, graphic designers will probably continue to only dip a toe into semiotics rather than take the plunge that might be expected of a discourse so central to what they do.

Integrated Information Theory Integrated Information Theory (IIT) is an attempt to unite cognitive/neurological brain science with process engineering. IIT develops from the premise that consciousness consists in the largest integrated whole output of clustered neuronal interactions. Many IIT concepts are translatable into Peircean schemas in which the interpretant is the effect of a sign/object to a receiving ‘quasi-mind’ (1906: CP 4.536). In that case, all the input into the neural cluster constitutes the sign, the effect on the neural nets is the interpretant, perceived content is the immediate object, the output of the neural cluster is the ‘more developed sign’ (c. 1897: CP 2.228) leading to further semiosis (Skaggs 2017b: 322–3). It remains to be seen if and how semiotic theory plays a part in the development of IIT, or how IIT might influence visual and graphic design procedure. One suggestive

SEMIOTICS IN GRAPHIC DESIGN

315

implication is that eventually the digital/neural bridge could be built from the nexus of these theories, resulting in direct neural input, bypassing usual sense reception entirely. (Although it is not at all clear what graphic design would become once vision is taken out of the mix!)

CONCLUSION Graphic design and semiotics are like cousins who grew up in distant towns, learning of each other late in life. Although they are roughly the same age, and closely related as one is essentially the visual manifestation of the other’s object of study, until recently they hardly knew the other existed. They have matured in such different conditions and in such dissimilar environments, that their dialects make it hard to have a dialogue. But now they meet and understand they have a lot to talk about. Indeed, graphic design is the laboratory par excellence for studying visual semiotics, and semiotics is the tool of choice for foregrounding the semantic aspects of graphic design. They belong together – and more so all the time.

NOTES 1 The nature of pictoriality is an open question, however. One might imagine a definition that is less restrictive than the one assumed here (an iconic portrayal of a subject), but in so doing, one expands the term well beyond what most people mean by the word ‘pictorial’. 2 Some might want to further constrain this definition to two-dimensional signals or to static signals, but I prefer a broader envelope. By the definition given here, a television commercial would be considered an instance of graphic design. 3 Eco, following the line of thought of Hjelmslev, describes this intransitivity of fine art as returning attention to the materiality of the sign vehicle (1976: 261–78). Jakobson calls this the poetic function of communication: a message for its own sake. Design is for the sake of utility to a purpose other than itself. Skaggs and Hausman (2012) stress intransitivity as fundamental to art-making. 4 Dwiggins, W. A. ‘New Kind of Printing Calls for New Design’ Boston Evening Transcript 29 August 1922 [The Origns of Graphic Design in America 1870–1920 by Ellen Mazur Thomson (New Haven and London: Yale University Press, 1997), pp. 184–9.] 5 Paul Shaw (2014). 6 David Jury (2012), Philip Meggs (1983). 7 Poyner (2003) identifies five strategies to accomplish this: opposition, deconstruction, appropriation, techno and authorship. All five lead the receiver to question what they are seeing, to destabilize the apparent authority of the originator of the message, and to cause the audience to become viewers as much as readers of the message. 8 Katherine McCoy (1991). 9 In highlighting the Cranbrook work it is important to note that others, in both the United States and Europe, contributed to deconstructivist design and the weakening of the modernist paradigm. Especially noteworthy as other early contributors are Wolfgang Weingart in Switzerland, and April Greiman and Rick Valicenti in the United States. The work of these  designers is easily found online and in design journals such as Graphis and Communication Arts. 10 Different pragmatists and neo-pragmatists have used varying terms for these three elements. The most common of the alternative terms are representamen/sign/sign vehicle and object/

316

11

12

13 14 15 16

17

18 19

20

21

22

23 24

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

referent. I will use sign, referent and interpretant, and reserve the term ‘semiotic moment’ when the intent is to describe the triadic relations as an analysable whole. Johanna Drucker (1994) addresses this point when she speaks of the near-invisibility, to linguists, of the important semiotic effects of a change of typeface or position of words on a page. While ‘linguists could not recognize the visual material of the linguistic signifier sufficiently to theorize its active role’ (1994: 46) examining the visual materiality of the experimental typography of the early-twentieth-century avant-garde helps us to gain a broader perspective. Drucker uses both deconstructive concepts and Peircean perspectives in her analysis of Dadaist and Futurist typography. Indeed, it was likely the influence of Peirce on early programmers at PARC/Xerox, in the development of graphic interfaces for computers, that led them to label the pictographs of trash cans, file folders, loudspeakers and so on, ‘icons’. To avoid this ambiguity, it is best to emphasize the action of relating by using the adverbial forms of these terms: an iconic relation, an indexic relation, a symbolic relation. Mollerup. Marks of Distinction. London, Phaidon Press, 1997. Krampen (1965: 12–14): Design Quarterly 62, Walker Art Center Minneapolis. Another example: ‘rhematic indexic legisign’. While Peirce’s justifications for his terminology make sense from a theoretical perspective and are understood by the specialist, they have undoubtedly retarded the spread of his ideas. Morris postulated three branches of semiotic investigation: syntactics, the study of signs with respect to their material, physical or perceptual constitution; semantics, the study of the relations between signs and their referents; and pragmatics, the study of the relation of the sign to users, especially the effects signs have on a public. C. M. Johannessen (2010: 120): Forensic Analysis of Graphic Trademarks: A Multimodal Social Semiotic Approach. University of Southern Denmark PhD Dissertation. The following discussion will lean heavily on Shaleph O’Neill’s Semiotics of Embodied Interaction (2008), which provides an excellent basis for those wishing to investigate the particular challenges of the interactive environment. The pointing finger, which derives somewhat from Mickey Mouse’s gloved hand, is itself an icon of a more ancient marginal index: the ‘printer’s fist’ of the fourteenth to eighteenth centuries. This nostalgic tendency is changing. Telephony, for example, once a function performed by a particular object, is now a function that is diffused through many different objects and platforms, both physical and virtual. Indeed, the fluidity of new media may be expected to ‘splash-back’ upon traditional media as interactive media alter the cultural codes. Metaphors of screen interactivity spread ‘backward’ throughout graphic design, extending into the static, non-interactive spaces. An example of this rigidity may be found in the practice of graphology which purports to assign personality or character traits based on very specific mannerisms in handwriting. While it may be possible to identify a person through comparing samples of their handwriting, as each person’s expression through their handwriting (as well as other ways of moving) is probably unique to them, there is no independent empirical evidence to support the more extreme claims made by the graphologists. Their practice is based on a much too rigid code. The number of semantic classes is a function of how many degrees of valence the analyst chooses to use for the four semantic operations, as an exponent so that 42 (16) or 43 (64). Doing this smoothing-over-differences work cannot really hope to reconcile the distinction between dyadic and triadic approaches to ‘the sign’ at the most fundamental level, yet there may be a way to move the emphasis of analysis back and forth between triadic and dyadic, depending upon the purposes at hand.

SEMIOTICS IN GRAPHIC DESIGN

317

25 Some use ‘representamen’ as an element of the triad, calling the triad the ‘sign’ but others (including Peirce himself) also use ‘sign’ as the name for one of the triadic elements, leaving the triad itself unnamed. Personally, for many years now I prefer ‘sign’ for the unit of the triad and have been calling the entire triadic set the ‘semiotic moment’. 26 Skaggs (2017a, b).

REFERENCES Andersen, P. B. (1991), A Theory of Computer Semiotics, Cambridge: Cambridge University Press. Arnheim, R. (1954), Art and Visual Perception: A Psychology of the Creative Eye, Berkeley: University of California Press. Arnheim, R. (1969), Visual Thinking, Berkeley: University of California Press. Barthes, R. (1957), Mythologies, Paris: Editions du Seuil. Barthes, R. (1977), Image, Music, Text, trans. Stephen Heath, London: Fontana. Buchanan, R. (1985), ‘Declaration by Design: Rhetoric, Argument, and Demonstration in Design Practice’, Design Issues, 2 (1): 4–22. Buchanan, R., D. Doordan, and V. Margolin (2010), The Designed World: Images, Objects, Environments, Oxford and England: Berg Publishers. Crow, D. (2003), Visible Signs: An Introduction to Semiotics in the Visual Arts, London: Bloomsbury. Danesi, M. (2008), Why It Sells: Decoding the Meanings of Brand Names, Logos, Ads, and Other Marketing and Advertising Ploys, Lanham: Rowman & Littlefield. Derrida, J. (1976), Of Grammatology, trans. G. C. Spivak, Baltimore & London: Johns Hopkins University Press. Dewey, J. (1989 [1934]), Art as Experience (reprinted in 1989, John Dewey: The Later Works, 1925–1953. vol. 10), ed. J. Boydston, Carbondale: Southern Illinois University Press. Drucker, J. (1994), The Visible Word: Experimental Typography and Modern Art 1909–1923, 175–89, Chicago: University of Chicago Press. Dwiggins, W. A. (1922), ‘New Kind of Printing Calls for New Design’, Boston Evening Transcript, 29 August [The Origins of Graphic Design in America 1870–1920 by Ellen Mazur Thomson (New Haven and London: Yale University Press, 1997): 184–9.] Eco, U. (1976), A Theory of Semiotics, Bloomington: Indiana University Press, 261–73. Floch, J. (2000), Visual Identities, London: Continuum. Hjelmslev, L. (1976), Resume of a Theory of Language, Madison: University of Wisconsin Press. Ingold, T. (2018), ‘Touchlines: Manual Inscription and Haptic Perception’, in Christian Mosbaek Johannessen and Theo van Leeuwen (eds), The Materiality of Writing: A Tracemaking Perspective, 30–45 London: Routledge. Johannessen, C. M. (2010), Forensic Analysis of Graphic Trademarks: A Multimodal Social Semiotic Approach, University of Southern Denmark PhD Dissertation. Jury, D. (2012), Graphic Design before Graphic designers: The Printer as Designer and Craftsman 1700–1914, London: Thames and Hudson. Krampen, M. (1965), ‘Signs and Symbols in Graphic Communication’, Design Quarterly, 62 (1965): 1–31. Kress, G. and T. Leeuwen (1996), Reading Images: The Grammar of Visual Design, London: Routledge. Krippendorff, K. (2006), The Semantic Turn: A New Foundation for Design, London: Taylor & Francis.

318

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

Klaus, K. (2006), The Semantic Turn: A New Foundation for Design, London: Taylor and Francis. Lupton, E. (1996), Mixing Messages: Graphic Design in Contemporary Culture, Princeton: Princeton Architectural Press. Lupton, E. (2014), Type on Screen, Princeton: Princeton Architectural Press. McCoy, K. and M. McCoy (1991), Cranbrook Design: The New Discourse, New York: Rizzoli International. Meggs, P. (1983), A History of Graphic Design, New York: Wiley. Mollerup, P. (1997), Marks of Excellence: The History and Taxonomy of Trademarks, London: Phaidon. Morris, C. (1971), Writings on the General Theory of Signs, The Hague: Mouton. Nadin, M., ed. (1985), ‘The Meaning of the Visual: On Defining the Field’, Semiotica, 52 (3–4), Amsterdam: Mouton. Nadin, M. and R. Zakia (1994), Creating Effective Advertising, New York: Consultant Press. Ockerse, T. (1984), ‘De-Sign/Super-Sign’, Semiotica, 52 (3–4): 247–72. Ockerse, T. (1997), ‘The Semiosis of Design’, Zed, 4: 148–67. O’Neill, S. (2008), Semiotics of Embodied Interaction, London: Springer. Peirce, C. S. ([1857–92] 1982–2010), Writings of Charles S. Peirce, 7 vols. (1–6, 8), Peirce Edition Project (eds), Bloomington: Indiana University Press. Cited as W. Peirce, C. S. ([1866–1913] 1931–58), The Collected Papers of Charles Sanders Peirce, vols. 1–6, ed. C. Hartshorne and P. Weiss, vols. 7–8, ed. A. Burks, Cambridge: Harvard University Press. Cited as CP. Poyner, R. (2003), No More Rules: Graphic Design and Postmodernism, London: Lawrence King. Ren, C. and A. Munk (2019), ‘Digital Arcticism? Exploring Arctic Connotations on Facebook’, The Arctic Institute Center for Circumpolar Security Studies. Saussure, F. (1959 [1916]), Course in General Linguistics, ed. C. Bally and A. Sechehaye, trans. W. Baskin, New York: Columbia University Press. Shaw, P. (2014), ‘Graphic Design: A Brief Terminological History’, Blue Pencil. https://www. paulshawletterdesign.com/2014/06/graphic-design-a-brief-terminological-history/ Skaggs, S. (2017a), FireSigns: A Semiotic Theory for Graphic Design, Cambridge, MA: MIT Press. Skaggs, S. (2017b), ‘Integrating Peirce and IIT: How Integrated Information Theory and Peircean Semiotics Mesh with Respect to Conscious Systems’, Cognitio, 18: 322–3. Skaggs, S. (2019), ‘The Semiotics of Visual Identity: Logos’, The American Journal of Semiotics, 35 (3–4): 277–307. Skaggs, S. and C. R. Hausman (2012), ‘Toward a New Elitism’, Journal of Aesthetic Education, 46 (3): 83–106. Vihma, S. (1995), Products as Representations. A Semiotic and Aesthetic Study of Design Products, Publication Series of the University of Art and Design Helsinki UIAH A 14, 1995. Vihma, S. (2014), ‘On Design Semiotics’, Objects and Communication (Mediation Et Information), 30–31: 197–208.

CHAPTER FOURTEEN

Semiotics in Marketing and Branding KRISTIAN BANKOV AND DIMITAR TRENDAFILOV

One of the most successful applications of semiotic methods of analysis for academic and non-academic purposes is in the field of marketing, advertising and brand research. The way business has evolved in the last four decades has made consumption an ever more meaningful and communicative process, rather than the mere satisfaction of objective needs. For companies, developing strong brands with rich and exciting imagery becomes the only way to penetrate the minds of consumers in highly competitive markets and transform creativity into economic value.

CHRONOLOGICAL OVERVIEW Two seminal papers opened up the semiotic adventure in the world of marketing and brands more than sixty years ago: Sydney Levy’s ‘Symbols for Sale’ (1959) and Roland Barthes’ analysis of a print ad for Panzani brand pasta (1964). The two papers are very different in their intentions, method and language, and this divide ended up being prophetic for the future development of marketing semiotics. Indeed, the divide between the Anglo-American and the Franco-Italian schools of marketing semiotics remains until today, with barely any mutual influence. Levy’s paper comes from the side of marketing and brand research, Barthes’ from the side of semiotics. This is a transversal divide in both schools where the major contributions have been made either by academics or practitioners, and in some cases by authors balanced between the two sides, mostly after the 1990s. Yet what Barthes did in his analysis is not at all marketing semiotics. This was the time of Vance Packard’s Hidden Persuaders, when commercial messages were considered synonymous with manipulation. Barthes opened the critical perspective on advertising, which was the dominant semiotic mood at that time, demonstrating how advantageous a critical approach might be. Few years later, Umberto Eco followed the example with his famous analysis of print ad for Camay brand soap (Eco 1968: 169–77). In both cases the critical discourse on advertising is supported by a more technical model of that type of communication, which laid the ground for more pragmatic approaches. According to Oswald (2012), in the following decades the very popular critical paradigm contributed to marketing semiotics within two major academic fields: literary theory and cultural studies (Oswald 2012: 4–7), where feminist studies, gender, race, deconstruction and simulacra were the leading themes.

320

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

During these initial years, scholars in marginal branches of semiotic studies began applying constructive approaches to the semiotics of marketing as well. According to Pasquier’s (1999) very detailed account of, already in 1966 the first semiotic contributions appeared in the French Journal of Marketing (cf. Peninou 1996a, b, c). The first seminar on the semiotics of marketing and advertising was held in Paris in 1976, followed by bigger and bigger gatherings in the francophone world in 1983, 1986, 1987, 1992, 1996 (Pasquier 1999: 107). In the anglophone world, the maturation of the semiotic contribution to marketing and advertising took longer. The path, opened by Levy in 1959 and followed by other publications in the same framework of symbolic, aesthetic and lifestyle consumption (1963, 1978), attracted various academic scholars, mostly from the field of consumer research such as Belk, Holbrook, Mick, Kehret-Ward, Solomon and others. A great deal of those scholars’ work was semiotic research, although it was not labelled as such. The turning point came in 1986 when Tom Sebeok and Jean Umiker-Sebeok, with the support of Sidney Levy, organized the First International Conference on Marketing and Semiotics. Tom Sebeok, one of the leading figures in world semiotics, often said that many scholars in various fields are doing semiotics, but only few are aware of it. So, this conference was aimed at raising semiotic awareness in the direction of applied research; and it resulted in one of the most important books for our review  – Marketing  and Semiotics: New Directions in the Study of Signs for Sale (Umiker-Sebeok 1987). Following this, the Newsletter on Marketing Signs at the Center for Language and Semiotic Studies was established at Indiana University, and two issues of the International Journal of Research in Marketing (1988), entirely dedicated to marketing semiotics, were published, one of which had only French authors translated into English. Quantitative changes bring qualitative changes. According Pasquier (1999: 108) the second half of the 1980s was when semiotic research on marketing finally broke through from academia and appeared in market research agencies. In the anglophone world the pioneer in this business was Virginia Valentine, who, in 1988, founded the Semiotic Solutions agency and who opened the research frame of advertising codes (and about which we will have a closer look shortly). Many of the people from the initial team of Semiotic Solutions went on to find their own agencies, which now dominate the brand research market in the UK. In the United States, Marketing Semiotics is associated mainly with the work of Laura Oswald and her homonymous agency since 2000. On the Franco-Italian side, by the mid-eighties there was a significant number of semiotic publications, events and interest in marketing semiotics, but it was thanks to the outstanding theoretical insights and market research achievements of Jean-Marie Floch that it began to be used in business. Floch, who was among the closest collaborators of the founder of the Paris schools of semiotics Algirdas Greimas, formed a unit of semiotic researchers as part of the multinational market research company IPSOS. Thanks to the growing number of academic courses and programs in semiotics in these countries, it soon became a common practice for semioticians to be employed by large market research agencies, rather than founding their own independent agencies. Many academic researchers worked directly for the companies as freelancers. In the next section, we shall present the most important models of marketing and brand semiotics, which were shaped from the 1990s on by the main practitioners and academics in this domain in both traditions.

SEMIOTICS IN MARKETING AND BRANDING

321

THE GENERAL FRAMEWORK To put it briefly, semiotics has contributed significantly to theorization in advertising: of the choice and organization of signs within particular ads (i.e. inner sign system), deconstruction of the meanings of the discrete elements and structures within ads (especially of those which are not so overt), conceptualization of the signs’ rationalization process on behalf of the consumer, and deducing the philosophical, historical and sociocultural sources and reasons for realization of the signs in any advertising text (Mick et al. 2004: 21). The difference between academic and commercial semiotics is that the former searches for understanding of various aspects of culture and communication for the purposes of the critical or descriptive investigations, whilst the latter pays attention to the potential options in communication systems in order to use them for brand purposes in competitive and culture contexts (Evans 1999). Communication activity is more than just a primary tool in brand development – it represents its essence; a brand as a signifying construct is both (and simultaneously) a key differentiating idea in the marketplace and its dissemination and realization process. Brand analysis, according to the three branches of semiotics (introduced by Charles Morris, in Nöth 1990: 49), operates as follows: 1. Semantics – answers the question of what the brand will ‘mean’, what will be its fundamental positional appeal (McDonald’s stands for ‘fast food’, Starbucks – ‘the third place’, Nike – ‘successful sport performance’), 2. Syntax – addresses what goods and services are branded and represent the brand in front of the audience (the iconic Big Mac, famous French fries on every menu, McCafé; intensive/‘convenient’ distribution of the products (availability) such as big city-centres’ outlets of Starbucks, its own coffee-mixes and baristas’ positive attitude, as well as comfortable furniture; Nike’s wide range of sports shoes and equipment that covers almost all types of sports), 3. Pragmatics – concerns how the brand communicates, what language should be used – usually by a set of symbols (golden arches, Ronald McDonald, various menus; a siren from mythology, significant coffee-cup, green-coloured logo, ‘world music’ playing; the ‘Swoosh’ symbol, AirMax shoe line series, famous athletes’ presence). In other words, strategic brand management deals with brand meanings that should be transferred to and comprehended by the consumers. A company’s product policy controls the syntax, while communication policy covers the pragmatic branch. The division of the three is undoubtedly significant, but they act as elements of equal value within a whole. Especially in brand management practice, mistakes could appear in each of the branches. For instance, it is possible for communications on the surface of the system – the publicly visible part (interior of a shop, supermarket or showroom, as well as the look of the employees, branded cars, TV commercials, sponsored event) – to suffer from a lack of relation with the brand’s main message; or the products may have no capacity to provide the benefits that the advertising has promised; or the positioning message makes no difference in a given market or coincides with or copies some competitor’s message. In the current market situation, the wide variety of product choice and intensive communication activities of the regular consumer put branding practices on very different footing, since it gives to the consumer, if not exactly the leading role, at least a decisive one in the brand building process. Brands are not only a key element of the promotional

322

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

pillar of the marketing mix, providing a face and familiarity to consumers of companies and their products; they are also a long-term and intensive dialogue with the target audience. While in a certain previous stage of the market evolution managers aimed to load their brands with as much information as possible regarding the products they represent in order to find their space on the shelf, in a more mature market environment with stronger competition, brand managers’ task is to make their brands meaningful for somebody in order to distinguish them among the crowd and to enforce the support of the product from the consumers themselves. Putting the stress on the latter task, in one of his classical articles, Mick (1986: 196) specified that today’s market is ‘a web of meanings among consumers and marketers woven from signs and symbols ensconced in their cultural space and time’. In another significant paper, Malcolm Evans (1999: 12), one of the most experienced consultants and trainers currently in Europe, added: ‘This is the medium semiotics lives in – a fluid world of representations, a galaxy crisscrossed by codes, discourses and signs.’ In other words, the weight of the added value has been shifted from a ‘production-functional’ mode to an ‘emotional-symbolic’ one, a path to which is the creation of two-way communication with the consumer. The questions of why and how this happens are precisely those of interest to semiotics as a scientific discipline. Defining the scope of semiotics is not an easy task even for the scholars who have long been involved in the field. The difficulty arises from the broad set of issues with which semiotics is concerned, as well as from its innate interaction with other disciplines in the realm of humanities, such as linguistics, anthropology, culture and media studies, psychology and sociology. Semiotics has earned its place in the academic world as a science studying the nature and social life of the signs with which we engage, but this is actually quite a narrow view. More precisely, semiotics’ main object of study is the constellation of circumstances in which signs form the meanings they carry, the modifications they undergo and, last but not least, their relations with the objects and phenomena they refer to. Furthermore, semiotics is a valuable methodology that offers us explanations and options in the fields of communication, logic and our cognitive capacities, as well as regards processes available in culture. On this basis, semiotics also began searching for its own place in the realms of market research, advertising, branding and consumer behaviour. One of the key reasons for the existence of a brand is to build value through distinctiveness, and semiotics is the perfect tool for the investigation of that value, searching for sources of meaning in a given brand. Approximately a century ago, Ferdinand de Saussure stated that a semiotic process is available where there is a difference between the sound in a word pronounced by a human being and another sound left unpronounced, as well as between the meaning of the word in question and the meanings of all other potentially usable words. Along the same line of thinking, Italian semiotician Andrea Semprini (2006: 77) points out that the strength of brands is ‘always under double control of the perceiving agent and the other brands around it’, meaning that the survival of a brand’s message in a competitive environment depends equally decisively on whether it will be understood by the audience to which it is addressed and the comparison that the addressee of the message will make between the brand’s offer and those of other market players. If we decide to expand our reflections on Semprini’s inference, we can easily illustrate the configuration of agents of communication activity (see Figure 14.1). The square demonstrates that the composition of signs, which, in fact, is a general message by a given brand, is an information flow addressed to both the competitors and the target consumer group (i.e. it is in the triangle between A-B-C). In this way we search for both perception

SEMIOTICS IN MARKETING AND BRANDING

323

FIGURE 14.1  Commercial communication as a system consisting of four main agents (D.T).

of, and positive reaction to, the stimuli; the message is intended to point the consumer towards the brand while displacing competitors’ offers. The dotted line (В-С) shows us that communication between consumer and competition occurs in a ‘grey zone’ far from the hypothetical brand-sender (A). It is not very clear or predictable to brand management exactly what or how much the consumer knows about the alternatives available in the market, what experience s/he has with the products, how positively or negatively s/he is disposed toward other brands’ messages, etc. But in order for all this to occur, it is required to see the ‘big picture’ in which communication and exchange takes place; this exists within the triangle in right-hand side of the figure (between В-D-C): environmental conditions are not passive ‘observers’ of the processes in question but quite active elements, enforcing or deadening the messages of the brands in given category. Also, they form consumer perceptions and determine the nature and intensity of the communication noise. While the relations in the triangles between А-B-D and A-D-C are not identical at all, since they should persuade the consumer while at the same time weakening the competitors, the configurations under study (A-B-C and B-D-C) are, in a sense, mirror-images; any given brand is, to a very high degree, a product of the specific socio-cultural context not only of the market it was created in but even of the markets outside the original one. The case of McDonald’s is a good example of this: the brand carries the idea of ‘fast food’ as a type of service with a lot of obvious appeal to the American consumer, but its ‘content’, i.e. its menu, has been modified according to the specific tastes of consumers outside the United States. This is the reason why the success of the message of any brand depends on constant and deep studies of markets and cultural contexts, product positioning and trends in consumer attitude, which shed light on the language and specificities of the target audience’s perceptions of the brand.

MODELS OF MARKETING AND BRAND SEMIOTICS IN UK AND NORTH AMERICA Prominent agencies such as Added Value and Space Doctors, which exploit serious toolsets rooted in semiotics and ethnography, demonstrate that the interdisciplinary approach towards the market has already cemented its position and proven its benefits in business outcomes. Moreover, the processes of globalization are ongoing and the presence of some

324

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

brands simultaneously in various product categories requires timely and focused work in terms of brand management in different cultural contexts and taking into account different consumer perceptions, habits and values. There have already been many international corporations employing their own units of well-trained anthropologists and semioticians, or else which actively and permanently contract out the services of specialized agencies that help them in their brands’ development, including new product niche finding, redesign (or update) of packaging, communication fine tuning and important insights about consumer behaviour. The nature of brand as a specific carrier of information presupposes its being a central object of semiotic analysis. In her efforts to explain such an interesting market and social phenomenon, Cimatti (2006) is explicit: ‘Brand is […the] vehicle by which product – and all elements it embraces – occupies a very dynamic and emotional place in the everyday life of the consumer in the context of his/her socio-culture values.’ In her opinion, the meaning that a given brand installs in a consumer’s mind is an image which is actually a mental representation of a multitude of impressions, a complex idea including certain myths (half-true-half-false stories, stimulating communication between the brand and its audience), fictions (creating an idea using fabricated facts) and reveries (personal desires and dreams). However, the intention of the brand management team is not exactly representative of the environment we inhabit in terms of the objective world, but rather its recreation in the audience’s mind. The author herself chose to direct her work towards the multilevel and complex language which builds advertisements (words, gestures, colours, dances, etc.), by which companies communicate with the consumer. According to Cimatti (2006), brand plays the role of indicator, in the first place, since it indicates which product category contains the products it offers and which do not; but second, it is also a symbol, since it carries specific meaning attached to it by the producer. In the former case, we are talking about the principle of differentiation and basic consumer’s perception, while in the latter, shared experience is available, which should provoke emotional effect, calls for action or the interpersonal sharing of abstract ideas. Chevalier and Mazzalovo discuss the support of semiotics in the practice of marketing and branding in particular in their book Pro Logo: Brands as a Factor of Progress (2004). They assert that semioticians hold key positions between brand director (strategic control) and brand managers (brand identity director) in marketing companies: ‘Semiotics is at its most effective when it is coupled with a good, in-depth trend survey of society or the chosen consumer segments, and with good basic sales sense […] We can say here that semiotics aids in the management of coherence, but is only marginally helpful in managing relevance’ (2004: 124). Italian-born professor in semiotics at the University of Toronto, Marcel Danesi, in addition to his multiple publications on media and pop-culture, has contributed to the topic in his book Brands (2006) and articles such as What’s in a Brand Name? A Note on the Onomastics of Brand Naming (2011) and Semiotizing a Product into a Brand (2013). For many years he has also been editor-in-chief of Semiotica, one of the most prominent journals of semiotics, and he is former president of the Semiotic Society of America. Danesi suggested a model of explanation for how a product became a brand by means of semiotic instruments which he calls ‘brand wheels’. Referring to Barthes, Eco and linguistics, Danesi puts it this way: Semiotizing a product into a brand involves assigning a specific set of connotata to it by: 1) giving the product a name; 2) designing a logo for it that mirrors the

SEMIOTICS IN MARKETING AND BRANDING

325

connotata or complements them in some way; 3) devising a slogan and/or jingle for it that reinforces the connotata; 4) textualizing it by creating ads and commercials that communicate the connotata textually; and 5) designing the product or its package (or container), if relevant, to reflect the same set of connotata in terms of physical structure or appearance. (2013: 465) Among the many contributions of Danesi’s approach to branding is the way he takes all of the brand elements that managers can handle in order to achieve the highest level of brand equity possible and explained them via semiotic theory. This provides a clearer view for marketers of what exact effect would be produced by using one approach or another in naming (heritage, iconic, numeric, hyperbolic and so on), designing logos (letter, portrait, symbolic and so on) and slogan-jingle (a brief aphoristic-poetic statement or tune with metaphorical structure, metonymy, alliteration and so on), textualization (encoding messages in commercial communication) and design (unconscious reflection of the name and other features of the brand). A couple of years later, in Brand Meaning (2008), Mark Batey (a language graduate of Oxford University, and current consultant, trainer and speaker in brand culture, psychology and semiotics) produced a large and deep synopsis on the meaning generation process. He underlines the difference between primal and implicit meanings. The former are connected with the features of the categories in which brands operate, and are implicitly received by consumers as taken for granted; however, exactly for this reason consumers perceive brands as practically identical and cannot discern differences or added value (Batey 2008: 130). Thus, the primal meanings point out the direct benefits of products, while the second ones – symbolic meanings – are not subject to product attributes and market peculiarities, rather they originate in culture and resonate on the emotional and existential level. Their genesis must be located both in the particular dimensions of the category as it is and in the wider local context (fashion, technological, social, art movements and trends). The latter creates more stable and stronger relations between consumers and brands, and to protect to a great extent against consumers switching to other competitive brands, generating more and more steady sales for the companies (128). Batey’s model (Figure 14.2) aims to compare clearly and to put the limit between the scope and potential roads for accumulation of the both types of meanings in consumers’ minds. Primary meaning is based mainly on information received through the senses, but the result is adducing rational arguments supporting brand choice. This we can define as lowlevel value. As the diagram demonstrates, hidden (implicit) meaning is high-level value with roots in mythology, socio-cultural texts and collective unconsciousness. Precisely for that reason, this level is not easy achievable. Brands that offer such meanings to their consumers have managed to cultivate them and systematically communicate them over time, which imparts them uniqueness and authenticity – in practice this is the best defence against competitors’ messages (Batey 2008: 131). Here it should be noted that the ‘product functionality’ strategy in brand management, which counts on improvements in quality and the application of new benefits and applications, is not necessarily wrong. In a sense, it is even inevitable and even if it cannot be defined as ‘inspiring’ from the consumer’s perspective it still remains as a ‘door’ through which consumers gain access to the brand’s history and values. Even a legendary brand such as Harley-Davidson demonstrates that ‘meaning trajectory’ begins from primary

326

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

FIGURE 14.2  Brand meaning model (Batey 2008: 131).

meanings, since without the inimitable rumbling motorcycle, that brand’s narrative would seem like a cheap marketing trick (Batey 2008: 132). However, Figure 14.2 confirms the proposition that the ‘path’ from product to successful perception of the myth (i.e. its cultural meaning) is complicated, and it depends on various factors which exist outside of the economic explanation of mere exchange relations. As mentioned above, Laura Oswald is founder and director of Marketing Semiotics, a US-based agency that has been applying cross-cultural consumer research and ethnography, including semiotics, since 1991. The agency works internationally with projects in Asia and Europe, and with clients like American Express, Ford Motor Company, Coca-Cola, Pfizer, Burger King and many more. With a PhD degree in semiotics from New York University, she spent the four years between 2006 and 2010 as Associate Professor at the University of Illinois Urbana-Champaign. Along with a long list of academic articles, Oswald has published three major books of applied semiotics – Marketing Semiotics: Sign, Strategies, and Brand Value (2012); Creating Value: The Theory and Practice of Marketing Semiotics Research (2015); and Doing Semiotics (2020). She is also a submission reviewer for the Journal of Consumer Research. Unlike most of her academic colleagues, Oswald’s approach to research is interviewbased, searching for codes and meaning generators that support brand development. She sees discourse analysis as the appropriate method for discovering not only how advertising makes meaning for brands, ‘but also to the way brands sustain a positioning over time, remain distinctive in the competitive arena, relate to the culture of consumers, and engage the consumer-spectators in the brand world’ (Oswald 2012: 109). This predominantly targets the macro-level of the brand narrative, which takes into account the role of its detached signs such as logos, characters (that represents the brand) or a particular text (as opposed to the text in printed advertisements, for example, which would be microlevel). Oswald’s method aims to study the integral behaviour of the brand in terms of the

SEMIOTICS IN MARKETING AND BRANDING

327

FIGURE 14.3  A discourse scheme of the Apple brand on a synchronic level (adapted from Oswald 2012: 111).

wider discourse within culture at a particular moment of time, as well as the power and the roots of its positioning in the consumers’ mind. Discourse analysis operates in several dimensions at once in order to cover a broad spectrum of information. First, synchronic analysis of each touchpoint between brand and consumer – from logo to packaging (see Figure 14.3). A logo is a symbol or icon, rather than an arbitrary sign, and because of that it is a micro-narrative that fits into the culture of the target audience. Second, diachronic analysis, according to Oswald, has to deal with the big picture of the communication strategy going backwards in time, across media and different markets. This type of analysis embeds more detached signs. Its role is to detail the general discourse of the associations between the brand’s overall message and its individual signs; this is intended to discern the single shared perception of the brand by all of its consumers (rather than the perceptions of only some of them) (Oswald 2012: 110). Oswald concludes that in one wide, multicultural world, heterogeneous in traditions and languages, it is actually impossible for global brands simply to translate their advertisements from one market to another (Oswald 2012). Moreover, it is necessary to look deeper into market environments, and semiotics is a crucial tool in understanding the meanings that a given brand will portray, or is expected to portray, in different cultural environments with different consumer approaches and different means of understanding the value of goods and services. By presenting the benefits of semiotics working shoulder-to-shoulder with brand management, especially in achieving the major goal – building value beyond the physical product – the author subdivides the field of study in four zones. These zones are organized not hierarchically, but rather interweave from four directions. The first pair of zones consist of the mind, including ‘ego’, society and consumer behaviour; and of the sign, embracing trends, conditions and the impacts caused by the environment, novelties, etc. In other words, this first pair distinguishes personality from environment behaviour. The second pair of zones are that of semiotics and brand. As a scientific discipline, semiotics is oriented to symbolic communication by analysing codes, meaning and context, while brand management refers to decisions in market segmentation, positioning and strategies in advertising, packaging and point of sale. The brand message is intended to coincide with particular consumer needs, such

328

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

as status, belonging and recognition, and adds value to the level of imagination. This happens by means of elucidation and adjustment of the performative elements in brand communication (both verbal and visual) and of their accuracy towards cultural meaning. The next step of the process is the difficult work of semiotic analysis. As a part of the flow of texts in culture, brands must be studied through secondary sources of information about popular culture, special issues, classical pieces of art, advertising examples and so on, in order to determine how ‘repetition’ of cultural tropes forms the meaning of the product category and the position of the given brand within it. For instance, ‘ManMachine’ might serve as a cultural-contextual image of a brand for selling, say, personal computers; ‘Efforts-Achievements’ might serve the same purpose as a framework of the discourse in the sporting goods market (Oswald 2012: 110). The process may be compared to collecting puzzle pieces. Researchers acquire great piles of information, collate it, decode it, categorize and subcategorize it, and in the end double check everything carefully to produce a holistic model that should be in service of brand management both in terms of long-term strategic brand developing planning, and tactical correction in current brand message and positioning. The British school of semiotics, most popularly known as ‘cultural studies’, is focused on studying particular topics and cultural practices from everyday life, searching for culture in the ‘ordinary’ (Williams 1977). The idea of using cultural codes as an applied semiotic approach towards marketing was introduced in the 1980s by the research agency Semiotic Solutions, whose founder, Virginia Valentine, had left the academic world to initiate the second period of the gradually growing impact of semiotic contribution (after Floch) to the decision-making process in branding. The tool in question is fundamental for almost all the research and consultancy agencies that have since emerged in the UK, and is currently successful in many markets in Europe and Asia. Some researchers work for companies such as P&G, Coca-Cola and Nestlé not only ad hoc, sporadically, but also organizing training sessions and workshops toward the goal of constant adaptation and improvement of packaging, brand elements and messages (Maggio-Muller and Evans 2008). The Welsh novelist and Marxist theorist Reymond Williams originally formulated the theoretical categorization of codes (Valentine 2007: 19), pointing out that the complexity of culture could be noticed predominantly in the different processes and institutions that build it. There are two main directions of semiotic research into cultural codes: on the one hand, knowing the wider socio-culture framework and comparative analysis between changing codes in it, and on the other, codes deployed for exploitation by a given brand, on the other (Alexander 1999). Over the course of time, cultural codes end up grouped into three categories: Residual (or oddments), Dominant and Emerging (see Figure 14.4). In the first group (‘residual’) belong those codes that represent cultural values and uses of which are outdated or obsolete. Sometimes in synchronic analysis these may seem ‘out of place’, often their identification is difficult and they seem likely to disappear completely or to be replaced by new codes. The second group (‘dominant’) includes codes which are everywhere around us and which we actively use in the communication process

FIGURE 14.4  A grouping of the codes according to their place in time and their role in culture (Williams 1977: 121).

SEMIOTICS IN MARKETING AND BRANDING

329

(or at least which we understand with ease). Paradoxically, every now and then these become difficult to identify precisely because they are ubiquitous and have become diluted in the general cultural flow. Most often they serve to help draw a ‘current picture’ for researchers and brand managers of the communications common in particular market and, more specifically, in a product category. The ‘emerging codes’ group consists of the so called road signs to the future since, from the perspective of the current moment, these codes are nebulous, unclear, with uncertain potential and even a bit eccentric. Their identification and the preparation of possible scenarios in their development is comparable to a certain degree to the theoretical calculations in chaos theory, in that their full realization or even their total disappearance at an early stage depends on long and possibly unknowable number of factors. Yet from the business perspective, these are perhaps the most important codes of all; supposedly, even if not all of today’s emerging codes become ‘dominant’ codes tomorrow, some surely will, and knowledge about what they are and how to use them will provide a competitive advantage to brands. The ‘three code’ system is a type of discourse mapping that seeks to clarify models of communications in a given culture and subculture, as well as to certain ‘admissions’ and ‘meanings’ that circulate in it. We all are products of the culture we live in, and so by familiarizing ourselves with a culture, we are able to make particular conclusions about its members. Once a discourse is identified as ongoing, then it is obvious that a large group of people is its ‘motive force’ and its main ‘reader’. Semiotic approaches do not often have direct contact with consumers in the form of interviews and focusgroups; rather, they study secondary data in the form of cultural artefacts existing around us – material, media, language, etc. Usually, marketing research methods are based on the axiom that the consumer is extremely rational and controls her system of beliefs, assessments and attitudes towards objects and phenomena that impact her choices at the point of purchase. Semiotics diverges from this by accepting that, regardless of whether we want or realize it, we are part of some culture (a larger ‘consciousness’) which to a great extent shapes our perceptions about the world. For that reason, as a research tool, when semiotic approaches do engage with personal interviews with consumers, even these responses are taken not as direct answers but as texts among many other texts forming the discourse within which the details of a given culture ‘hides’. Since initially cultural codes may be obscured, semiotics tries to create a wider and more informed perspective on the context from which the consumers’ responses originate (Alexander 1999; Evans 1999; Valentine 2007). In practice, most often a researcher must prepare a list of codes, extracted from the general communication environment in the category under study; she then creates groupings according to the meaning framework (or cluster) that unites a number of codes together. For instance, in advertising (TV and print) usually the dominant bearers of meaning are love, friendship, nation, future and similar codes, which are abstract notions that cannot be presented directly to the audience. To communicate these meanings in an ad, then, a group of codes is deployed (visual, audio, verbal) that build product ‘validating’ concepts (Harvey and Evans 2001). That create a category ‘language’. That language clearly distinguishes one product category from others, for consumers and competitors. Advertising specialists do their best to avoid clichés in advertising by diverging from established patterns every now and then, but are typically careful not to wander too far either from the main message of the brand they work for or from the codes the audience is accustomed to seeing for products in this category. Over time, changes and modifications in codes accumulate, leading to cultural evolutions within a category.

330

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

These changes exist not so much in concepts of ‘valuation’ as such, but rather among the codes by which those concepts reach consumers via various media – TV spots, packages, websites, leaflets, in-store sign and so on. Semiotic decoding gives opportunities for: 1. cultural context and competition analysis; 2. new product development – empty niches and new opportunities for portfolio extensions; 3. assessment of changes in direction (in given category and in cultural codes); 4. identification of the most used codes in advertising for the purpose of new communicating offerings; 5. communication strategy – copywriting, packaging, media planning, etc.; 6. in-depth analysis of local cultural contexts (complete preliminary study of the history and development of the culture); 7. finding positive characteristics as well as what should be avoided; 8. cultural study aiming at the deconstruction and then amplification of particular concepts and the ideas and images connected with the concepts; 9. detailed study of the mass/local culture by searching for new ideas and metaphors (new bearers of meaning); 10. stimuli development for other marketing research methods (predominantly interview scenarios and materials for projective techniques). Evans (1999) summarizes this, stating that semiotics serves equally well both for fine-tuning of the current brand communication, for pre-coding of all of the forms of communication with the target audience and for preliminary analysis for future investments in new product categories and geographical markets. Semiotic approaches allow the researcher to study the features of the images in advertising as a second level of articulation, the smallest units of signification, and the issues coming from agrammatical uses in advertising representation.

MODELS OF MARKETING AND BRAND SEMIOTICS IN FRANCE AND ITALY We will begin this section with a short review of the contributions of Jean-Marie Floch, which have been decisive for the success of marketing semiotics in both academic and business contexts. This success was already inscribed in the project of his seminal book Semiotics, Marketing and Communication: Beneath the Signs, the Strategies (Floch [1990] 2001). On one hand, this book is an intentional continuation of a tradition of scientific rigor, inaugurated by Greimas and his school of semiotics. On the other hand, the models that result from the complex and intriguing theoretic work are clear and easy to apply. John F. Sherry, Jr. in the preface of the book, invites anyone to apply the models to any set of micronarratives of our everyday experience to see ‘how practical a good theory can be’ (Floch 2001: x). The extent of this review allows us to present only the three most important insights of Floch, although it is not easy to separate them from his overall argument. They are (1) the model of the axiology of consumption; (2) the typology of service users; (3) the visual design analysis of logos, other brand elements and spaces.

SEMIOTICS IN MARKETING AND BRANDING

331

The methodological tool which allows Floch to establish the standard of scientific rigor, which his most important followers will try always to preserve, is the text. ‘Beyond the text, no salvation’ is one of the slogans of Greimas, which Floch explains in his first chapter, and which allows him to set up his semiotic approach to marketing. Texts are not only the ‘intentional’ products of the companies as commercials, prints, logos, product designs, packaging, etc., but the experience itself and the context of their use and circulation as well. Textuality applies to all aspects of the whole cycle of production, distribution and consumption in all its variety of objects and interactions. Semiotics does not supplant existing methods of market and brand research, but it sharpens them and brings the operational awareness to a different level. In Floch’s words: ‘There can be value-added when there is an effort to acquire more intelligibility, more pertinence or more differentiation’ (Floch 2001: 6). The model of the axiology of consumption comes after a deep semiotic analysis of a commercial of Citroen BX from September 1982. The choice of this video is not arbitrary; its broadcast signalled a turning point in the symbolic capital of the major automotive brand, which during the 1970s lost its appeal for the young and playful audience. The generative trajectory of meaning is employed to demonstrate how the impressive creativity of images and words in the car commercials derives from a limited number of core values, articulated thanks to the semiotic square. The corporate narrative with the intermediation of the brand invests products and services with value for the consumers, transforming them into objects of value through narrative programs. Most of the commercials actualize two opposing categories of values, which constitute any meaningful narrative – utilitarian (instrumental) and existential values. Thanks to the two complementary positions on the semiotic square, Floch completes the typology of the core values (the axiology of consumption, see Figure 14.5) in the following way: Practical valorization, corresponding to instrumental values conceived as the opposite of base values (we can also call them ‘utilitarian’ values, which include such things as manoeuvrability, comfort, durability and so on). Utopian valorization, corresponding to base values conceived as the opposite, once again, of instrumental values (we could also call them existential values: identity, life, adventure and so on). Ludic valorization, corresponding to the negation of ‘utilitarian’ values (ludic valorization and practical valorization are therefore contradictory: ludic values include luxury, gratuitousness, refinement, an impulsive act or ‘small folly’). Critical valorization, corresponding to the negation of the ‘existential’ values with which one may invest the automobile (critical and existential valorizations are contradictory, the relationships of quality/price or innovation/cost are critical values) (Floch 2001: 120). Although conceived for a concrete case study in the automotive business the model results applicable to any sphere of consumption, corporate communication and strategic brand positioning. Floch himself gives examples with its application in the product design (Floch 2001: 135–7) and the design of the interior of a supermarket (2001: 138–64), but today exists an endless literature of its use, as well as many patented tools for real market research, based on it. Floch’s second great insight comes from his research for the Paris metro. This research was part of a large initiative by the state-owned French public transport company Régie autonme des transports parisiens (RATP) to improve the service of the metro and RER (Réseau Express Régional, Paris’s rapid transit system). The most complicated task was assigned to the team of semioticians, led by Floch. The task was to understand and

332

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

FIGURE 14.5  The axiology of consumption scheme (Floch 2001: 120).

conceptualize the experience of the commuters with the service in order to direct the management of RATP beyond the obvious technical and organizational improvements. The semiotic team designed a multistage research project, combining ethnography, semiotic desk analysis, qualitative and quantitative surveys. The main challenge was to create a reliable and comprehensive behavioural typology of the travellers, pertinent to the main purpose of the initiative (2001: 13–39). After the first stage of ethnographic observations the researchers gathered and put together the database of the registered multiplicity of invariant patterns of observed behaviour among the travellers, a kind of behavioural morphemes. The methodological step was brave: the experience of the travellers from the entrance to the exit of the metro was conceived as a text. A binary notion of opposing semantic poles was found to give the formal divide of the registered behavioural morphemes: discontinuity versus continuity. Thus, under the apparent multiplicity of individual behaviours, strategies arose valorizing the experience (a syntagmatic chain of behavioural morphemes) of the travellers as either continuous or discontinuous passage. The application of the semiotic square also in this case brought the examined object to its logical conclusion. In this way emerged four types of travellers, each type giving meaning to his/her experience with the service of the metro following a separate logic (see Figure 14.6): Once the formal typology was established, the determination of the qualitative description of the four types developed after several dozen interviews. Thus, for the surveyors, daily journeys are thought to be so many variations and transformational games that end up producing meaning. […] [they are] the most responsive to efforts to decorate or renovate. […] For the daydreamers – these quotidian jaunts are a neutral occurrence onto which they are able to graft other signifying practices, such as reading or

SEMIOTICS IN MARKETING AND BRANDING

333

FIGURE 14.6  The typology of the travellers (Floch 2001: 25).

knitting. […] They have perhaps the most physical relationship to the space in railway stations. They classify it according to the quality, density and consistency it presents to the flow of bodies in motion. […] Pros display the greatest interest in the accessibility of stations and the equipment. […] The trajectory the pro effectuates must be seen as an extensive attempt to desemanticize, abstract and formalize. The signifying practice of the stroller is, as one might imagine, exactly the opposite. […] Inasmuch as being amazed or surprised is valued above all, the stroller knows how to remain receptive to all the different, and sometimes strange, spectacles the metro and the RER can provide. (26–9) Just like the model of the axiology of consumption, the semiotic typology or the travellers’ behaviour turned out to be applicable in variety of ways, both in the future management of the metro as a quantitative survey and in other service spheres. It’s not difficult to see how easy the logical structure of the four types of users can be adapted to the customers of a supermarket, clients of a bank or students at a university, etc. The example of the integration of the semiotic square articulation of types with the qualitative and quantitative tools of research provided valuable capital for all future semiotic market researchers. Floch’s third great insight is concerned with the contribution of semiotics to the visual design of brands and marketing. Initially it was developed in his book from 1990, with the same rigor and practical orientation of the previous two models; afterwards Floch collected further analyses in a separate volume, Visual Identities (2000 [1995]) with more

334

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

of an aesthetic and anthropological orientation. With this book, Floch opened an entirely new perspective on the understanding of the deep levels of meaning making of logos. In one of the included case studies, Floch collaborated with the Creative Business agency on the task of creating a new logo for a big French bank. The contribution of semiotics was of strategic order, concerning the choice of a leading concept for the whole rebranding and communication strategy. The first task was to choose one among five possible such concepts which were: conviviality, reliability, directness, openness and clarity (Floch 2001: 42). The semiotic test indicated ‘clarity’ as the best option for the strategic goals of the bank; after this, Floch explored which semiotic tools would bear the greatest semantic potential for the concept, which were then systematized in order to be used for future campaigns. The most relevant contribution of semiotics to this project was the choice of a symbol for the logo which best expressed the semantic potential of clarity. To do this, Floch elaborated a complex theoretic frame for the role of the logo, not seeing it anymore as a static graphic mark for distinction, but rather as a narrative schema, establishing a long-lasting relation with the customers and the external world. The chosen symbol was the star, in a classical visual style, which has since become one of the most memorable bank logos in France, and is still being used today. Another exemplar exploration of the deep layers of meaning of logos is made in the IBM versus APPLE paper (Floch 2000: 33–62, 2001: 165–94), which demonstrates how a huge historical and ideological narrative can be encoded on a semi-symbolic level in the visual identity of the companies. The method is further developed and shaped as a standard for future analyses in the case study on the design of the packaging and leaflets of psychotropic drugs (2001: 73–107). Since the 1990s the theoretic, applied and essayistic production in French, Italian and Spanish within Floch’s paradigm is immense. We shall outline the contribution of three authors, who developed and upgraded the theory for the benefit of both practitioners and academics. Only two years after the publication of Floch’s seminal book, market researcher and professor Andrea Semprini, developed his semiotic model of the brand audit (Semprini 1992). The model was conceived after years of professional research for big and small companies in France and Italy. His methodology relies on the same textualist generative conception of meaning-making, but he enriches the tool for analysis with Umberto Eco’s notions of textual cooperation, encyclopaedia of production/encyclopaedia of reception, and the possible world of the brand. Semprini makes a considerable effort to represent the complex semiotic theory in a vocabulary close to that of the marketing and advertising professionals (74). The first step of his work on any brand is to establish the semiotic construct of its identity. The identity is the only condition to make the brand known, loved and chosen. The effect is long-lasting and the process, semiotically speaking, is like a stipulation of a contract of trust. Every brand at first is a void premise for a such contract and only with adequate marketing activities will it be filled with a mix of rational and emotional values. The purpose of the first phase of the brand audit is to establish the three levels of the content of the brand identity, which is achieved with desk analysis and qualitative interviews to obtain large amounts of data, concerning the communication activity of the company, the sociocultural context, competitors, customers’ perceptions, etc. These three levels include (1) axiological; (2) narrative; and (3) discursive (surface). The brand audit examines the semiotic coherence of those levels and creates a basis for concrete actions in case of deficits.

SEMIOTICS IN MARKETING AND BRANDING

335

The concise version of the Levi’s audit is detailed in Figure 14.7 below: The second phase of the audit is called ‘semiotic mapping’, and its purpose is to achieve further concreteness and operative value for the brand after the audit (see Figure  14.8). It operates entirely with Floch’s axiology of consumption, but enriches it with a full set of intermediate concepts, closer to those used by communication professionals and easier to apply: ‘The crucial question here is to show that starting from a purely semiotic issue – that of the articulation between basic values and use values – it is possible to design an analysis instrument that allows a great universality of applications’ (Semprini 1992: 74). The model is illustrated well and in detail with a case study of Benetton (Semprini 1992: 164–83), where the historical changes of the brand’s values are explained, together with a proposal for a map of the synchronic positioning of the then-present situation. Giulia Ceriani, founder of the semiotic research agency ‘BABA’, and adjunct professor at the University of Bergamo, has worked to further develop these types of semiotic

FIGURE 14.7  The brand identity audit of Levi’s (Semprini 1992: 59).

336

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

FIGURE 14.8  Semiotic mapping of the values of the brand (Semprini 1992: 132).

market research tools. In her book Marketing Moving: The Semiotic Approach (2001), she gives a systematic and extended description of the services which semiotics can provide to the marketing process, from the conception of the product to its ultimate realization. In her view, semiotics is a strategic tool, although one best used for fine tuning of what the company is doing (Ceriani 2001: 18). She strongly emphasizes that only the generative semiotics of the Paris school may give satisfactory results in marketing, and she makes explicit many of the implicit assumptions which were already present in the work of her teacher, Jean-Marie Floch. The methodological rigor of generative semiotics is projected on the marketing cycle of the company where not only the communication is seen as a discursive production, but so is every single element which is perceptible by the customers and other subjects of the socio-economic environment. Such a methodological move allows one to import structural integrity into strategic planning. Ceriani claims that this approach allows one ‘to manage with semiotics’ the marketing mix of the company (2001: 17, author’s

SEMIOTICS IN MARKETING AND BRANDING

337

italics). The generative method presupposes, as we have seen, a trajectory of meaning which starts from abstract fundamental values and ends on the discursive manifestation. In the language of marketing people, Ceriani calls the starting point of the marketing project ‘concept’. In the concept is encoded, in a very synthetic and virtual form, the core positioning idea of a brand, a service or a product, its meaningful DNA which afterwards will be actualized through its market realization. By way of example, Ceriani cites the stages that she has followed in a survey, aimed at identifying a new concept of perfume: 1. a scenario study, to understand the state of things, how the universe of reference was semantized (the whole of the beauty offer); 2. a first session with consumers, with a creative association task, to understand what the spontaneous associations were to the main trends indicated during phase 1; 3. some alternative transcriptions of the concept, to understand how its basic building blocks and the values they bring, could, taking different forms, determine effects of different meaning (desk analysis); 4. an expansion (also with visuals) of alternative concepts, endeavouring to pursue the investment and configuration hypotheses that these could take (desk analysis); 5. A second session with consumers to evaluate their options in relation to the selected concepts. (2001: 34) Once a strong and original concept is created, the guarantee of the semiotic management is to provide consistency (pertinence) in inventing product design, name, logo, slogan, packaging, points of sale, website, advertising directions, etc. (2001: 28–37). The next big step in Ceriani’s marketing semiotics is the development of a research tool upgrading the previous version with a new capacity to define market trends (Ceriani 2007). This might be considered as the structuralist equivalent of the less methodologically rigorous emerging codes monitor of the Anglo-American tradition. This tool was already conceived in the mid-1990s, but the growing dynamics of the markets, and the strategic urge to keep up with the rapid pace of market changes in the wake of mass penetration of the internet, inspired development of the patented tool Trend Monitor. Here again the theoretic background is the same, with very articulated use of the semiotic square. Trends are not seen as static entities to be predicted by a ‘fortune teller’, but as scenarios in which those who succeed to grasp and conceptualize them can take market advantages just by co-participating in the trends’ future actualization. In this way the creative challenge for the marketers is to invent brand, product or service concepts, the enactment of whose virtual form may benefit from the socio-economic tension of the identified trends. For instance, in the alimentary sector for 2003, the trend mapping showed ten particular concepts, situated on a grid of communicational and anthropological axes (see Figure 14.9). Easy action: simplify the processes Spotting: individuation and microsegmentation Pink hearts: desiring the desire Visual action: living in the imaginary Extrapower: consuming conquests Plenary: investing in the total living Ritualization: believing in the symbolic efficacy

338

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

FIGURE 14.9  Trend mapping in the alimentary sector (Ceriani 2007: 53).

Moralization: consuming intentions and principles Wireless: mobility and disconnection Emotional play: choosing the longest walk (Cerini 2007: 53–4) The last big contribution in this tradition of brand and marketing semiotics, and we would say its magnum opus, is Gianfranco Marrone’s book The Brand Discourse: Semiotic Models for Branding (2007). After two decades of successful expansion of semiotic research in academia and in the market, Marrone assumes the responsibility to re-establish the methodological canon inaugurated by Floch. Marrone’s task is considerable, since his theoretical project needed to encompass an impressive amount of very heterogeneous literature and organize it in terms of a generative hierarchy. The effect of this is twofold; on one side the reader sees how every aspect of the socio-economic occurrence of the brand is of a profoundly semiotic nature, and on the other side the book also works as an efficient promotion of the structural semiotics of the Paris school, illustrated in every theoretic detail with exciting examples of the global brandscape in which everyone is immersed.

SEMIOTICS OF BRANDS AND MARKETING TODAY In these hard times for humanities and semiotics, it is common for semiotics to survive in various universities thanks only to its applications to marketing and branding, as most of the job opportunities for semiotic researchers outside academia are to be found in market

SEMIOTICS IN MARKETING AND BRANDING

339

research companies. Journals specializing in marketing semiotics have proliferated, and the literature in this area is already vast and still constantly growing. Most of the market research agencies specializing in semiotic research are in Italy and the UK, the first of these being Semiotic Solutions founded by Ginny Valentine in 1988, but which also includes Space Doctors; The Flamingo Group; Sign Salad; Giulia Ceriani’s Baba Consulting; Andrea Zannin’s Beyond Research; and Truth. Even the more mainstream market research companies in the UK now offer semiotic research as part of their service packs; in Italy and France probably all of them are doing so, and IPSOS has a particular role in the affirmation of this practice as a standard. There are also agencies as Squadratti, whose brand identity comes from a ‘profanation’ of the semiotic square of Greimas, and Semio whose USP is ‘adaptive semiotics’, with particular know-how for the digital scenarios of marketing. In France the leading figure of marketing semiotics is Erik Bertin, who holds a top position in MRM/McCann Worldwide. In the United States, the industry tends to gravitate around Laura Oswald and her brand strategy and research boutique Marketing Semiotics; but other semiotically inclined marketing agencies, such as Added Value (Los Angeles), also exist. Thanks to the efforts of Chris Arning, since 2012 there has been an annual event dedicated exclusively to applied semiotics in the field of marketing and branding, itself branded as Semiofest. Semiofest is remarkable since on one hand it is probably the best place to start for newbies in marketing and branding semiotics; and, on the other, it attracts researchers and practitioners who offer genuine contributions to the field. Thus, for instance after the 2017 Semiofest conference in Toronto, The American Journal of Semiotics published a special issue of select, expanded papers from the conference, dedicated to ‘applied brand semiotics’ (Pelkey and Pereira 2018).

CONCLUSIONS It took a long time for semiotics to come down from the strict academic domain to ‘the streets’ as an applied discipline oriented to business practice. However, it appears that such applications are now here to stay; and the results this could provide for companies and brands will become more visible and valuable in their market performance. Based on theoretical models, marketing semiotics indisputably has demonstrated its merits in the domains of advertising and packaging design and redesign, and brand meaning and positioning engineering, but also in terms of launching product into new markets, adapting brands to new consumption trends and decoding competitors’ strategies and consumer behaviour. The challenges of the digital business environment will shape the next big thing in marketing and brand semiotics. In the twenty-first century, semiotic applications to marketing and branding continue to gain momentum by entering into the content of marketing research textbooks, which makes semiotics more familiar to a new generation of marketers, and by creating collaborations with applied psychology, cognitive science and anthropology as well. Moreover, global communications have made contacts between semiotic experts regular and more fruitful; shared knowledge and collaborations are growing year after year. Further evidence for the increased interest in this socio-cultural domain over in last few decades is its relevant publication base, which has also continued to expand. As a result, useful practices and case studies by applied semioticians from all over the world are now available in journals such as the Journal of Consumer Culture, the Journal of Consumer Culture Theory and the Journal of Cultural Marketing Strategy, to mention only a few.

340

BLOOMSBURY SEMIOTICS: SEMIOTICS IN THE NATURAL AND TECHNICAL SCIENCES

ACKNOWLEDGEMENTS We are particularly grateful to Malcolm Evans from the UK for updating us on marketing semiotics in the Anglo-American world, Giulia Ceriani from Italy for doing the same for the Franco-Italian world and Nina Denisova from IPSOS Bulgaria for sharing some information on the semiotic research in that multinational agency.

REFERENCES Alexander, M. (1999), Codes and Context: Practical Semiotics for the Qualitative Researcher. A Paper presented at Marketing Rese Society. Batey, M. (2008), Brand Meaning, London: Routledge. Ceriani, G. (2001), Marketing Moving: l’approccio semiotico. Analizzare il mix di comunicazione, gestirne gli effetti di senso, Milano: Franco Angeli. Ceriani, G. (2007), Hot Spots e sfere di cristallo: Semiotica della tendenza e ricerca strategica, Milano: FrancoAngeli. Chevalier, M. and G. Mazzalovo (2004 [2003]), Pro Logo: Brands as a Factor of Progress Hampshire; New York: Palgrave Macmillan. Cimatti, M. C. B. (2006), ‘Semiótica da Marca: Análise da Marca Contemporânea como Fenômeno de Linguagem’, Revista Caligrama, 2 (1): 1–14. Danesi, M. (2006), Brands, London: Routledge. Danesi, M. (2011), ‘What’s in a Brand Name? A Note on the Onomastics of Brand Naming’, Names, 59 (3): 175–85. Danesi, M. (2013), ‘Semiotizing a Product into a Brand’, Social Semiotics, 23 (4): 464–76. Eco, U. (1968), La struttura assente. Introduzione alla ricerca semiologica, Milano: Bompiani. Evans, M. (1999), Semiotics, Culture and Communication – Common Sense of 21st Century. Proceedings of the Australian Market Research Society. Floch, J. (2000 [1995]), Visual Identities, London and New York: Continuum. Floch, J. (2001 [1990]), Semiotics, Marketing and Communication, New York: Palgrave. Harvey, M. and M. Evans (2001), ‘Decoding Competitive Propositions: Semiotic Alternative to Traditional Advertising Research’, International Journal of Marketing Research, 43: 171–87. Levy, S. J. (1959), ‘Symbols for Sale’, Harvard Business Review, 37 (4): 117–24. Levy, S. J. (1963), Symbolism and Lifestyle, Proceedings. American Marketing Association Conference, décembre, 140–50. Levy, S. J. (1978), Marketplace Behavior: Its Meaning for Management, New York: AMACOM. Levy, S. J. (1981), ‘Interpreting Consumer Mythology: A Structural Approach to Consumer Behavior’, Journal of Marketing, 3: 49–61. Maggio-Muller, K. and M. Evans (2008), ‘Culture, Communication and Business: The Power of Advanced Semiotics’, International Journal of Market Research, 50 (2): 169–80. Marrone, G. (2007), Il discorso di marca. Modelli semiotici per il branding, Bari: Laterza. Mick, D. G. (1986), ‘Consumer Research and Semiotics: Exploring the Morphology of Sign, Symbols, and Significance’, The Journal of Consumer Research, 13 (2): 196–213. Mick, D. G. and L. Oswald (2006), ‘The Semiotic Paradigm on Meaning in the Marketplace.I’, in R. W. Belk (ed.), Handbook of Qualitative Research Methods in Marketing, 31–45, Cheltenham: Edward Elgar Publishing. Mick, D. G., J. E. Burroughs, P. Hetzel, and M. Y. Brannen (2004), ‘Pursuing the Meaning of Meaning in the Commercial World: An International Review of Marketing and Consumer Research Founded on Semiotics’, Semiotica, 152 (1/4): 1–74.

SEMIOTICS IN MARKETING AND BRANDING

341

Nöth, W. (1990), Handbook of Semiotics, Bloomington: Indiana University Press. Oswald, L. R. (1999), ‘Cultural Swapping: Consumption and the Ethnogenesis of Middle-Class Haitian Immigrants’, Journal of Consumer Research, 25 (4): 303–18. Oswald, L. R. (2012), Marketing Semiotics. Sign, Strategies, and Brand Value, Oxford and New York: Oxford University Press. Oswald, L. R. (2015), Creating Value. The Theory and Practice of Marketing Semiotics Research, Oxford and New York: Oxford University Press. Oswald, L. R. (2020), Doing Semiotics. A Research Guide for Marketers at the Edge of Culture, Oxford and New York: Oxford University Press. Pasquier, M. (1999), Marketing et sémiotique, Fribourg: Editions Universitaires. Pelkey, J. and M. Pereira, eds (2018), Applied Brand Semiotics, special issue of The American Journal of Semiotics, 34 (3–4): ii, 207 pp., Charlottesville: Philosophy Documentation. Peninou, G. (1966a), ‘Réflexions sémiologiques et creation publicitaire I: Genèse et objet de la recherche sémiologique en publicité’, Revue Française du Marketing, 19. Peninou, G. (1966b), ‘Réflexions sémiologiques et création publicitaire II. Eléments de méthode’, Revue Française du Marketing, 21. Peninou, G. (1966c), ‘Réflexions sémiologiques et création publicitaire Ill. Eléments de doctrine: structure du manifeste’, Revue Française du Marketing, 28. Peninou, G. (1972), Intelligence de la publicité, étude sémiotique, Robert Laffont, coll. Paris: Medias et Messages. Peninou, G. (1975), ‘Langage et image en publicité.I’, in La Publicité de A à Z, sous la direction de Claude Vielfaure, Paris: Editions Retz. Semprini, A. (1992), Le marketing de la marque, Paris: LIAISONS. Semprini, A. (2006), La Marca Postmoderna. Potere e Fragilitá della Marca nelle Societá Contemporanee, Milano: Franco Angeli s.r.l. Umiker-Sebeok, J., ed. (1987), Marketing and Semiotics: New Directions in the Study of Signs for Sale, Berlin: Mouton de Gruyter. Valentine, V. (2007), Semiotics, What Now, My Love? Paper presented at the Market Research Society Golden Jubilee Conference, Brighton, UK, 21–23 March. Williams, R. (1977), Marxism and Literature, Oxford: Oxford University Press.

INDEX

abduction 23–5, 98, 108, 120, 126–8, 152 Actor-Network Theory (ANT) 65, 85 Aglietta, M. 248 Alberti, L. B. 282, 293 Alexander, C. 290–3 algorithm 47, 175, 217–18, 252 alpha mu-band 187 Andersen, H. 97 Andersen, P. 307 animals 77–8 communication 37, 61, 79–80, 93, 99, 104–6, 149 environment and 61 higher 78 human and 81, 83–6, 87 n.11, 106 lower 78 non-human 4, 77, 80 populations 60–1 symbolic 148–51 zoo 80 Annandale, C. 121 anti-representationism 49 Anttila, R. 97–8 applied semiotics 204–5, 210, 222, 224, 283, 326, 328, 339 architecture 277–8 as language 279–82 post-modernism 284–90 representation 290–1 scientific proof 291–3 and spatial design 282–4 Aristotle 16–17, 22, 279 Arnheim, R. 309 artificial intelligence (AI) 2, 13, 175, 221–2, 225, 306 association learning 44 Atkinson, Q. D. 95 Atmanspacher, H. 162 n.8 Augustyn, P. 66 Bach-y-Rita, P. 181 Bær, E. 120 Balkin, J. M. 265–6

Bally, C. 155 Barbieri, M. 42, 77–8 Barham, L. 101 Barraza, P. 187 barter 240, 243–4, 246, 249 Barthes, R. 283–4, 289, 302, 319, 324 Bateson, G. 99 Batey, M. 325 Becker, F. W. 123 Bettini, S. 283 binary system 217, 220–1 Bingham, C. 241 Binswanger, L. 144 biolinguistics 94, 99, 107–9 biology 35–6 argumentation and proofs 38–9 Biological Foundations of Language (Lenneberg) 107 categorization and speciation 43 codes in 42–3 communication 43–4 and ecology 46 evolution, study of 45–6 fundamental level 40 gradualism vs. discontinuity 40 history of 36–8 learning, study of 44–5 meaning and function 39 meaning-making 48–9 mechanisms 41–2 problems 46–50 semiotic schools 40 testability 49–50 translation process 44 umwelt 39–40 biosemiotics 35, 37–8, 58, 60, 155 Biosemiotic Meaning (Goli) 125 ‘Biosemiotics: Its Roots, Proliferation and Prospects’ (Sebeok) 79 and linguistic semiosis 102–4 and medicine 125–6 Bitcoin (XBT) 252, 255–6 n.6 black box 205–6

INDEX

Blais, M. J. 24 blood-oxygen-level-dependent (BOLD) signal 177 Bockarova, M. 13, 16 Boerhaave, H. 122 Bonnin, J. E. 126, 132 Boole, G. 19 Boullée, É. 292 Bourbaki, N. 146 brain cognition and 174–6 communication 185–9 limitations 189–90 localization vs. distribution 178–82 meaning and mind 173–4 perception 182–5 and process 177–8 brand 321–2, 324. See also marketing and branding analysis 321 Apple 327 audit 334–5 Brand Meaning (Batey) 325 Brands (Danesi) 324 and consumer 327 Harley-Davidson 325–6 management 321–5, 327–9 meaning model 325–6 semiotic mapping 335–6 Brandt, P. A. 241–2, 254–5 Brentari, C. 78 Broca, P. 179–80, 188–9 Broekman, J. M. 269–71 Brown, R. 17 Brunelleschi, F. 281–2 Cabanis, P. -J. -G. 122 Čadková, L. 104 Cao, D. 270–1 Cassirer, E. 143–8, 150–4, 156, 158, 161–3 nn.4, 6, 12, 13 category variance 313–14 Ceriani, G. 335–7, 339 Cerrone, M. 81 Chalmers, D. 176 Chapdelaine-Feliciati, C. 272 Chevalier, M. 324 Chomsky, N. 103, 105, 107–8 Chrulew, M. 79, 83 Cimatti, M. C. B. 324 Ciompi, L. 148 Clark, A. 176

343

cluster analysis 311–12 Cobley, P. 121 codes 38, 41–3, 222 acoustic 62–3 archetypical 63 cultural 328 distribution 63 dominant 329 ecological 63 emerging 329 genetic 149 identity 63 significational 63 symbolic 63 cognitive neuroscience 176, 178–9, 190, 253 processes 6, 77, 107, 109, 173–6, 178, 182 semiotics 6, 9, 77, 105, 120, 173–6, 205, 208, 306, 313 Cohen, L. 253 Colonius, T. J. 84 communication 43–4, 60, 330 activity 321–2, 334 animal 37, 61, 79–80, 93, 99, 104–6, 149 Communications and Semiotics (Maldonado) 283 consumer and competition 323 human 58, 65, 161 intentional 121 linguistic 101 meaning in 185–9 non-verbal 99 signals 210 verbal 99 visual 297, 299–300, 310 comparative linguistics 97–8 Complexity and Contradiction in Architecture (Venturi) 284–5 computer systems 204, 224 deblackboxing 205–6, 212, 217, 225 design 205, 207–8, 220–2 digital 204, 207, 211–12, 219, 221, 223 homologies 219 levels 209 computing 203 automated 220 developments 213–15 and information 216–17 models 217–18 semiotic levels 217 de Condillac, E. 122

344

confidence 248 connotation 120, 145, 278, 285–6, 302–3, 311, 313 consciousness 5, 50, 101, 144–7, 150, 152, 155–61, 189, 300, 314, 325, 329 mental matrix of 157 consortium 62 consumption 319–20, 331–3, 335, 339 corollarial deduction 27 Cranbrook design 300–1, 315 n.9 critical valorization 331 Crow, D. 304 cultural codes 328–9 ‘Cyborg Manifesto, A’ (Haraway) 76, 86 n.2 cyclic autocatalytic process 41 Danesi, M. 13, 16, 306, 324–5 Darwin, C. 95, 97 Davtian, S. 155 Day, M. J. 83–4 Deacon, T. 39, 93–4, 99–101 De Architectura (Vitruvius) 278–9, 281 Death and Life of Great American Cities, The (Jacobs) 284 deblackboxing 205–6, 212, 217, 225 decoding 8, 189, 265, 330, 339 Decoding International Law (Tiefenbrun) 272 deduction 15, 20–5, 27 Deely, J. 77, 96, 120–1, 133, 133 n.2 De Haan, J. 261 Dehaene, S. 253 Delahaye, P. 82 Demand for Evidence, The (Welby) 260 denotation 65, 302–3 De re aedificatoria (Alberti) 282 Derrida, J. 251, 265, 272, 300–2 descriptive phenomenology 84–5, 306 Diagnostic and Statistical Manual of Mental Disorders (DSM) 143–4, 148, 156, 159 n.11 diagrams 3, 13–15, 18–20, 96, 325 diagrammatic iconicity 97, 110 and diagrammatization 15, 96–8, 110 epistemology of 20–1 systems of logic 22–3 digital computers 204, 207, 211–12, 219, 221, 223. See also computer systems Dipert, R. 14 Dodd, N. 240, 244, 251 dominant codes 328–9 Drucker, J. 316 n.11

INDEX

Dumas, G. 187 Dutilh Novaes, C. 28 n.1 Dwiggins, W. A. 299 Earley, R. W. 84 Ecoacoustic Event Detection and Identification (EEDI) method 62 eco-field 59, 61, 65 ecology community 61–3 population 60–1 semiotic approach in 46, 58, 61 system 63–4 ecosemiotics 67–9 Eco, U. 8, 48, 124–5, 278, 283, 291–3, 315 n.3, 319 Eisenman, P. 279, 284, 288, 290 electroencephalography (EEG) 128–9, 178 electronic computational system 220 Elliot, N. L. 65 emerging codes 329, 337 encoding 179, 181, 210, 240, 300, 325 Engelbart, D. 223–4 environmental studies 57–8, 65–7 Epstein-Barr virus 128 ethical turn 82 Evans, M. 322, 330 event-related potentials (ERP) 178 Everett, D. 101 evolution. See also evolutionary linguistics language 93–4, 96–9, 109–10 study of 45–6 evolutionary linguistics 93–5, 109–10 contemporary language 97 discovery 95–6 Peircean 96 Farina, A. 37–8, 59, 62, 65 Fauconnier, G. 102 Ferenczi, S. 145 Ferry, E. 252 Finkelnburg, F. C. 145 Fleerackers, F. 270–1 Floch, J. -M. 9, 302, 320, 330–1, 333–6, 338 Florkin, M. 37 Fodor, J. 175 Fonagy, P. 148 form language 293 Foucault, M. 289–90 Free Energy Principle 185 Frege, G. 19 Freud, A. 145

INDEX

Freud, S. 145–6, 148, 150, 152, 154, 161 n.5 functional brain 177–8, 181 functional magnetic resonance imaging (fMRI) 177–8, 181, 186–9 functional near-infrared spectroscopy (fNIRS) 178 matrix 310–11 Galantucci, B. 97 Gallagher, H. 186 gamut 313–14 Gandelsonas, M. 284 Garrod, S. 97 Gaub, H. D. 122 Genetic Structuralism 150 Gibson, J. J. 183 gifts 247–8 Ginzburg, C. 122 Given Time: I. Counterfeit Money (Derrida) 251 globalization 323–4 Goldstein, K. 147, 155 Goli, F. 125 Gollwitzer, S. 128 Goodrich, C. A. 120 Gorn, S. 220 Goux, J. -P. 245 Grains of Sense (Welby) 260 graphic design 297–8 awareness 309 category variance 313–14 cluster analysis 311–12 denotation and connotation 302–3 functional matrix 310–11 history of 298–300 IIT 314–15 page grammar 309–10 paradigms 312–13 positive and negative implications 308–9 pragmatist semiotics 303–4 Saussurean and poststructuralist methods 300–2 transitional methods 305–6 UX/UI 306–8 Gray, R. D. 95 Greenberg, J. H. 95 Greimas, A. J. 300, 311, 320, 330–1, 339 Grzybek, P. 155 Gurwitsch, A. 161 n.7

345

habits 45, 48, 62, 104 habit-taking 108 Haiman, J. 97 Hammad, M. 289–90 Haraway, D. J. 75, 86 n.2 Hediger, H. 79–81, 83 Herrmann-Pillath, C. 250, 255 n.4 Herzing, D. 106 Hillyard, S. A. 178 historical linguistics 97–8, 110 Historical and Comparative Linguistics (Anttila) 98 Hjelmslev, L. 300, 315 n.3 Hockett, C. 104–5 Hodge, R. 305 Hoffmeyer, J. 60, 62, 64, 77–8, 125 Hohfeld, W. N. 265–7 Holmes, O. W. Jr. 264, 273 homeostasis 41, 64 homology 218–20, 227 n.23 Homo sapiens 100–1 Hornborg, A. 67 Horowitz, A. 106 Horwitz, S. 249 House III (Eisenmann) 288 Hufeland, C. W. 123 Hurford, J. R. 101, 183 hypermotor seizure (HMS) 130 hyperscanning techniques 187 hypostatic abstraction 21, 101, 111 n.11 iconicity 96–7, 110, 313. See also diagrams Ihmig, K. N. 153 imitation learning 45 implicit meaning 325 imprinting learning 44 induction 3, 23–6 information system 220–1 Information Theory 42, 220, 227 n.24 Ingold, T. 85 Institutiones Pathologiae Medicinalis (Gaub) 122–3 instruction, learning 45 Integrated Information Theory (IIT) 314–15 Intentions in Architecture (Norberg-Schulz) 284 interfaces 223–4 International Classification of Diseases (ICD) 143–4, 148, 156, 159 n.11 International Journal for the Semiotics of Law (IJSL) 265–6, 270

346

interpretation 5–9, 35–9, 41–4, 46–51, 51 n.1, 62–3, 181, 185, 210, 217–18, 223–4, 270–2, 284, 299 Ivakhiv, A. 65 Jackendoff, R. 98 Jackson, B. S. 268–9, 272 Jacobs, J. 284 Jakobson, R. 37, 243, 315 n.3 Jander, R. 37 Jaroš, F. 82 Jaspers, K. 144 Jeanneret, E. 278, 286 Jencks, C. 287–9, 291, 293 Jenkins, L. 108 Jones, E. 145 Jung, C. G. 145–7 Kahn, J. Y. 126 Kant, I. 18, 76, 133 n.5 Karlin, M. S. 68 Kay, A. 224 Kennedy, D. 265–6 ketoacidosis 124 Kevelson, R. 261, 264–71, 273 Keynes, J. M. 242, 255 n.2, 255 n.5 Kohn, E. 65 Koolhaas, R. 277, 279 Krampen, M. 40, 304 Kress, G. 305, 309–10 Kull, K. 59–60, 62, 67–8, 78, 104, 124, 132 Kurke, L. 242 Kutas, M. 178 Kymlicka, W. 87 n.13 Lagopoulos, A. P. 279 Lange, F. A. 18 language 98, 329 adaptation 101 architecture as 279–82 competence 99 contemporary 97 English 99–100, 110 n.2 exaptation of 101 genetic endowment 108 Language of Thought (LOT) 175 money and 244–5, 252–3 natural kinds of 109 organism-environment-interaction 108 principles 108 problems and progress 109–10

INDEX

reflexivity and virtual worlds 99–100 Sebeok on 99 symbolic species 100–1 Latour, B. 85 law and jurisprudence De Haan, J. 261 education 271 Holmes, O. W. Jr. 264 international and domestic 272 interpretation and translation 270–1 legal-diplomatic discourse 271 legal semiotics 266–70, 272 Peircean semiotics 261–4 round tables 265 Welby and significs 259–61 learning Learning from Las Vegas (Venturi) 285 study of 44–5 Leeuwen, T. 309 Lefebvre, H. 289 Leferink, K. 156 legal semiotics 7, 259, 261, 264–73 legisigns 7, 96, 129, 133, 263–4, 316 n.16 LEGO model 188–9 Leibniz, G. W. 16–18, 22, 161 n.4, 203–4, 210, 213–15, 220, 222, 225, 225 n.1 Lenneberg, E. 107 Lestel, D. 82 Leuner, H. C. 146–8 Levi’s audit 335 Lévi-Strauss, C. 283 Levy, S. J. 319–20 Lewin, K. 146, 148, 152 Licklider, J. C. R. 204, 214, 224 Lieber, F. 264 Lindström, K. 66 linguistic semiosis animal communication and 104–6 biosemiotics and 102–4 Linhares, A. 272 logo 327 Lorenzer, A. 148 Lorenz, K. 76 Lotman, J. 57, 66 Low, D. 66, 68 ludic valorization 331 McCoy, K. 300–1 machine learning 44 magnetoencephalography (MEG) 178 Magnus, R. 68, 77–8, 81 Malavasi, R. 62

INDEX

Maldonado, T. 278, 283–4, 291–2 Manuel, J. 272 mapping 206, 220 Maran, T. 63, 68, 75, 77, 80, 82, 85, 86 n.4, 87 n.7 Marini, L. 177 marketing and branding 338–9 France and Italy 330–8 Marketing Moving: The Semiotic Approach (Ceriani) 336 Marketing Semiotics (Oswald) 326 UK and North America 323–30 Marler, P. 37 Marrone, G. 338 Martinelli, D. 80 Marty, M. 148 Marusek, S. 271 Marx, K. 245, 250 mathematics and logic 13–16 abduction 24 deduction 23–4, 27 epistemology of diagrams 20–1 ethics 26 history 16–18 induction 24–6 observational science 22–3 Peirce on 19–20 philosophy of notation 18–19 Matrix of Mental Formation (Andersch) 158–60 Matulewska, A. 270 Mauss, M. 247, 255 n.1 May, M. 144 Mazzalovo, G. 324 Mead, G. H. 154 meaning-making 35–6, 47, 64, 66, 68–9, 265, 305, 334 biological 48–9 brain, role of 177–8, 189 in communication 185–9 Meaning in Architecture (Rykwert) 283 Meaning Triad (Welby) 261, 272 and mind 173–4 in perception 182–5 mechanical thread 203–4, 220, 222, 225, 225 n.1 medial prefrontal cortex (mPFC) 186 medicine and healthcare 119–21 abduction 127–8 biosemiotics 125–6 clinical education 124–5 clinician-patient interaction 126

347

cuneiform 122 enlightenment 122–3 literature 129, 131–2 malignant lymphoma 126–7 priority 132 semiology 128–30 Mendívil-Giró, J. 98 mental dysfunction 159 illness 5, 148, 150–1, 155–60 Mesopotamian divination 120, 122, 124, 280 metasemiosis 101 Mick, D. G. 322 mirror neuron system 186 mismatch negativity (MMN) 184–5 Mollerup, P. 304 money 239–41, 254–5 barter myth 244 counterfeit 251 gift myth 247–8 gold standard 251–2 hegemony of 249–50 heterarchy of 249 interpretant of 241–3 and language 252–3 language myth 244–5 Marxians 250 quantification myth 246–7 tribute myth 245–6 violence myth 248 and writing 253–4 Mootz III, F. J. 269 Morgan, A. de 19 Morolli, G. 280, 282 Morris, C. W. 37, 77, 155, 305, 316 n.17 Morton, D. 284 Mozgovoy, J. 61 Muller, J. P. 162 n.11 Müller, M. 95 multimodal semiotics 305–6 multivitality 42, 50 Munk, A. 311 Munro, C. F. 283, 292 M’uzan, M. de 148 Mythologies (Barthes) 283 Nadin, M. 307 natural logic 17 neuronal activity 177–8 neurosemiotics 5–6, 173–4 Newell, A. 221 Newton, I. 16

348

Nicolai, E. A. 123 Nielsen, S. N. 64 NK-T cell lymphoma 128 Norberg-Schulz, C. 278, 284 Nöth, W. 67, 77, 96 Observationes Medicae (Sydenham) 122 Odum, E. 63 Ogilvie, J. 120 Olds, D. D. 154, 156 One Health 83–4 O’Neill, S. 307, 310, 316 n.19 one-to-one correspondences 220 Onians, J. 279, 281, 291 Orléan, A. 248 Oswald, L. R. 319–20, 326, 339 page grammar 309–10 parity principle 6, 176 Pasquier, M. 320 Paterson, H. 43 Pattee, H. 42 Patten, B. C. 63–4 pattern-building 149, 156, 158 pattern language 293 Pearson, N. 259 Peirce, B. 19 Peirce, C. S. 1–2, 14, 146, 163 n.13, 225–6 n.9, 262–3, 302 cognitive-semiotic artefacts 208 diagrams 14, 20–1 ethics of notation 26 evolutionary linguistics 96 Logic as Semeiotic 207–8, 210–12, 225–6 nn.8–10 mathematics and logic 19–21 and psychopathology 154–6 second trichotomy 304, 313 semiotic systems 205, 207–12, 261–4 systems and design view 207–8 trichotomy 263, 267, 304, 313 Pelkey, J. 96 Pellegrino, P. 278, 286 Pencak, W. A. 269, 271 Peric, Z. 68 Perrault, C. 292 ‘Philosophy of Symbolic Forms’ (Cassirer) 146, 161 n.4, 162 n.12 physical causality 186, 220 causes 108 symbol system model 218–22

INDEX

Piaget, J. 146, 155, 158 pictoriality 297, 315 n.1 Pietarinen, A.-V. J. 100 Pinillos, R. G. 84 Piranesi, G. B. 282 Pizzolotto, R. 59 plasticity 181 Popper, K. 103, 107 Porter, T. 17 Portnov, A. N. 161 n.3, 161 n.6 Positron Emission Tomography (PET) 178 Posner, R. 66 posterior-superior temporal sulcus (pSTS) 187 postmodernism 8, 284–90 Poyner, R. 315 n.7 practical valorization 331 pragmatism 303–5, 321 predictive coding 185 prescission 21, 101, 111 n.11 primal meaning 325 Primal Sense and Significs (Welby) 260 Prodi, G. 37, 125 programming languages 207, 222–3 Pro Logo: Brands as a Factor of Progress (Chevalier and Mazzalovo) 324 propositions 7, 21, 27, 100, 122, 183, 263 protosemiotic stage 120, 133 psychiatry 143–8, 154, 158, 160 psychologism 109 psychopathology 123, 143–4, 147, 154–60 ‘Psychosemiotics – Neurosemiotics: What Could/Should It Be?’ (Grzybek) 155 Puura, I. 66 quantification 246–7 Quincy, A. 281 Rauch, I. 97 Reading Images (Kress and Leeuwen) 309 Remm, T. 68 Ren, C. 311 reproduction 39, 41, 45, 64, 286 residual codes 328–9 retokenization 222 Ricca, M. 272 Rogers, E. N. 278 Rothschild, F. S. 37, 155 Royce, J. 26–7, 147 Russell, F. C. 20 Rykwert, J. 283–4

INDEX

Salingaros, N. A. 282, 291–3 Sánchez-García, F. J. 61 Sandin, G. 289–90 Sapir, E. 254 Saporoshez, A. W. 160 n.1 Saussure, F. de 4–5, 8, 37, 57, 93–5, 97, 121, 128, 133 n.2, 146, 148, 151, 155, 174, 245, 265, 268, 279–80, 283, 300–2, 305, 312, 322 Schleicher, A. 95 Schlick, M. 163 n.15 Schutz, A. 161–2 n.7 scorekeeping 243 Scruton, R. 283 Sebeok, T. A. 35–6, 47, 50, 75, 77–80, 82, 87 n.6, 99–101, 103, 107, 111 n.9, 121, 123 Sechehaye, A. 155 Second World War 143–4, 146–7, 287, 299 semantics 103, 181, 189, 321 Semiology and the Urban (Barthes) 283 semiosis 2–3, 8–9, 35–8, 40–2, 45–51, 67–8, 80, 93, 95–6, 103 semiotica utens 3, 19 semiotics applied 204–5, 210, 222, 224, 283, 326, 328, 339 claims 162 n.11 cognitive 6, 9, 77, 105, 120, 173–6, 205, 208, 306, 313 legal 7, 259, 261, 264–73 mapping 335–6 multimodal 305–6 natural 2–3 semeiotics 121, 204–12, 221, 225 ‘Semiotic Model of Mind, A’ (Olds) 154 Semiotics of Embodied Interaction (O’Neill) 316 n.19 Semiotic Solutions (Valentine) 320, 328, 339 social 305–6 subsystem 210, 220–1 Semprini, A. 322, 334 Sequential Merging of Actantial Models of Space (SMAMS) 289 Shapiro, M. 96–8 Siewers, A. K. 65–6 signa naturalia 5, 120–1, 125 Simões Gaudêncio, A. M. 272 Simon, H. 221 sinusitis 128 Skaggs, S. 310

349

Skinner, B. F. 105 social brain 186–7. See also brain software 205–8, 211, 217–25 Sonesson, G. 77 source code 223 species 3, 43–4, 59–66, 80, 100–1, 106–8 Staiano, K. 132 Steels, L. 94 Stefan, H. 128 Stereo electroencephalography (SEEG) 130 Stibbe, A. 65 Stjernfelt, F. 100–1, 183 strata of signs 221 structural brain 177 Sydenham, T. 122 symbols formation 146–8, 150, 153, 158–60, 161 n.4 Symbolic Species (Deacon) 94, 100–1 ‘Symbols for Sale’ (Levy) 319 systems 221–2 syntax 321 Tansley, A. G. 57 taxation 243 techne semeiotike 120 Tembrock, G. 37 temporo-parietal junction (TPJ) 186 Terrace, H. S. 105 theorematic deduction 27 theory of mind (ToM) 186–7 Theory of Symbols (Jones) 145 Threefold Laws of Meaning (Welby) 263, 293 threshold, semiotic 47–8 Tiefenbrun, S. 262, 266, 268–9, 272 Tønnessen, M. 61, 68, 77–8, 82 transitional methods 305–6 translation process 44 travellers 332–3 trend mapping 337–8 Trend Monitor 337 tribute 245–6 Truthfulness in Science and Religion (Welby) 260 Turner, M. 102 Tüür, K. 66, 82 Tylén, K. 188 Uexküll J. v. 36–7, 39, 42, 50, 76–8. See also umwelt Funktionskreis 36, 50, 103–4 Ulanowicz, R. 64

350

umwelt 4, 36, 39–40, 46, 57–61, 68, 76–85, 102–8 United States 146–7, 161 n.5, 254, 264, 287, 315 n.9, 320, 323, 339 user experience (UX) and user interaction (UI) 306–8 utopian valorization 331 Valentine, V. 320, 328 van Eeden, F. 261 van Leeuwen, T. 305 Veblen, T. 243, 249–50 Venturi, R. 284–8 Verbal Behavior (Skinner) 105 Verhagen, F. C. 104 violence 248 Visual Identities (Floch) 302, 333–4 visual word form area (VWFA) 253 Vitruvius, M. P. 8, 278–82, 293 Vladimirova, E. 61 von Neumann, J. 17, 214, 222

INDEX

Waddington, C. H. 37 Wagner, A. 270 Waller, C. 241 Wang, X. 129–30 Welby, V. 7, 259–63, 265–6, 268, 272, 293 Wennerlind, C. 248 Wernicke, C. 180–1, 188 Wernicke-Gerschwind model 180–1 When Species Meet (Haraway) 75 Whitehead, A. N. 151–3, 158 Williams, R. 328 Wittgenstein, L. 17, 43, 51 Wondwossen, A. 83–4 Zarbiyev, F. 272 Zeki, S. 177 Zevi, B. 285–7, 290 Zinsstag, J. 83 Zlatev, J. 77 zoosemiotics 4, 75–86, 87 n.6, 87 n.8, 155 zoosemioticians 75, 81