The Modern Legacy of Gibson's Affordances for the Sciences of Organisms [1 ed.] 9781032500195, 9781032500188

Citation preview

“The word ‘afordance’ is widely used, but many are not aware of its deep theoretical and philosophical roots. The unique editorial approach of Mangalam, Hajnal, and Kelty-Stephen aligns this volume’s diverse chapters along these roots. This strategy interconnects the entire volume in a manner both accessible to newcomers and stimulating to experts.” Rick Dale, Professor, University of California Los Angeles, USA “Embark on a mind-expanding expedition into the science of afordances with this illuminating book. Seamlessly merging philosophy, theory, and real-world applications, it navigates the fascinating terrain of how we perceive and interact with our surroundings. A thought-provoking read that reshapes our understanding of human-environment dynamics.” Benoît G. Bardy, Professor, Montpellier University, France “This wonderful book brings together many of the prominent scholars within ecological psychology to thoroughly discuss and refect on what affordances aford organisms as well as researchers nowadays. A must-read for anyone who is interested in learning about why people, animals, brains, robots and/or other systems do what they do in the environment that they live in.” Lisette de Jonge-Hoekstra, Assistant Professor, University of Groningen, The Netherlands “This edited book provides a broad ranging set of chapters (philosophy, perceptual psychology, neuroscience, applications) that takes stock of the contemporary theoretical and experimental work inspired by Gibson’s theory of afordances. Integration is harnessed by each paper addressing the same set of questions on afordances producing a unique resource on Gibson’s still evolving infuence.” Karl M. Newell, Professor Emeritus, The Pennsylvania State University, USA “A diverse group of scholars describes an extraordinary palette of research and innovation spawned by an antireductionistic theory once considered radically heterodox. These extant perspectives refect the exponential growth and transdisciplinary breadth of the theory’s scientifc and technical infuence. Such interpretations and applications are sure to inspire, challenge, and proliferate.” Gary Riccio, Ph.D., Nascent Science & Technology LLC, USA

THE MODERN LEGACY OF GIBSON’S AFFORDANCES FOR THE SCIENCES OF ORGANISMS This edited collection provides a comprehensive and empirically informed discussion on afordances and their role in studying goal-directed behavior, covering philosophical, experimental psychological, neuroscientifc, and applied perspectives. Showcasing the work of expert contributors from diferent backgrounds, the book inspires new directions for future research in afordances. Chapters address questions relating to the defnition and perception of afordances, their advantages over stimuli, the relationship between afordances and behavior, and how systems engage with afordances in diferent tasks and intentions. This question-based format provides a distinctive perspective that allows for a thorough exploration of the expansive feld of afordance research. This book serves as a crucial resource for seasoned scientists, researchers, and undergraduate and graduate students in the felds of ecological psychology, sensation and perception, cognition, and the philosophy of cognitive science, as well as non-academic individuals interested in mind sciences broadly construed. It provides valuable insights and knowledge in these felds, making it an essential reference for those seeking to deepen their understanding in the areas of perception and cognition. Madhur Mangalam is Assistant Professor in the Department of Biomechanics at the University of Nebraska at Omaha, USA. His research interests include nonlinear dynamical principles governing perception-action and embodied/ embedded cognition and the development of nonlinear analytical methods that can aid in discovering these principles. Alen Hajnal is Professor of Experimental Psychology at the University of Southern Mississippi, USA. Dr. Hajnal studies the interaction between body movements and perception in relation to the ecological theory of afordances. Damian G. Kelty-Stephen is Faculty Member at the Psychology Department at the State University of New York at New Paltz, USA. His research interests include perception-action relationships, embodied/embedded cognition, and the nonlinear dynamics that link them together.

Resources for Ecological Psychology A Series of Volumes Edited By Jefrey B. Wagman & Julia J. C. Blau [Robert E. Shaw, William M. Mace, and Michael Turvey, Series Editors Emeriti]

Afective Gibsonian Psychology Rob Withagen Introduction to Ecological Psychology A Lawful Approach to Perceiving, Acting, and Cognizing Julia J. C. Blau and Jefrey B. Wagman Intellectual Journeys in Ecological Psychology Interviews and Refections from Pioneers in the Field Edited by Agnes Szokolszky, Catherine Read and Zsolt Palatinus Places, Sociality, and Ecological Psychology Essays in Honor of Harry Heft Edited by Miguel Segundo-Ortin, Manuel Heras-Escribano and Vicente Raja The Ecological Brain Unifying the Sciences of Brain, Body, and Environment Luis H. Favela The Modern Legacy of Gibson’s Afordances for the Sciences of Organisms Edited by Madhur Mangalam, Alen Hajnal, and Damian G. Kelty-Stephen For more information about this series, please visit: www.routledge.com/ Resources-for-Ecological-Psychology-Series/book-series/REPS

THE MODERN LEGACY OF GIBSON’S AFFORDANCES FOR THE SCIENCES OF ORGANISMS

Edited by Madhur Mangalam, Alen Hajnal, and Damian G. Kelty-Stephen

Designed cover image: © Getty Images First published 2024 by Routledge 605 Third Avenue, New York, NY 10158 and by Routledge 4 Park Square, Milton Park, Abingdon, Oxon, OX14 4RN Routledge is an imprint of the Taylor & Francis Group, an informa business © 2024 selection and editorial matter, Madhur Mangalam, Alen Hajnal and Damian G. Kelty-Stephen; individual chapters, the contributors The right of Madhur Mangalam, Alen Hajnal and Damian G. Kelty-Stephen to be identified as the authors of the editorial material, and of the authors for their individual chapters, has been asserted in accordance with sections 77 and 78 of the Copyright, Designs and Patents Act 1988. All rights reserved. No part of this book may be reprinted or reproduced or utilised in any form or by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying and recording, or in any information storage or retrieval system, without permission in writing from the publishers. Trademark notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe. ISBN: 978-1-032-50019-5 (hbk) ISBN: 978-1-032-50018-8 (pbk) ISBN: 978-1-003-39653-6 (ebk) DOI: 10.4324/9781003396536 Typeset in Sabon by Apex CoVantage, LLC

CONTENTS

List of Illustrations List of Contributors Series Editor’s Foreword: Resources for Ecological Psychology Jeffrey B. Wagman and Julia J. C. Blau Foreword On Gibson’s Legacy William M. Mace Preface Madhur Mangalam, Alen Hajnal, and Damian G. Kelty-Stephen 50 Years On: What Does Affordance Afford Us? Madhur Mangalam, Alen Hajnal, and Damian G. Kelty-Stephen PART I

Ontology and Epistemology of Affordances 1 Why It Matters That Affordances Are Relations Taraneh Wilkinson and Anthony Chemero

xi xiv xviii xx xxii

1

9 11

viii

Contents

2 The Sociomaterial Theory of Afordances Julien D. Kiverstein 3 When It Comes to Afordances, What Do Animals Know and How Do They Know It? Jefrey B. Wagman, Tyler Dufrin, and Thomas A. Stofregen 4 Mental Action and the Scope of Afordance Perception Thomas McClelland and Max Jones 5 An Afordance-Based Approach to the Origins of Concepts Manuel Heras-Escribano, David Travieso, and Lorena Lobo 6 Toward an Ecological Theory of Time Brandon J. Thomas

26

42

71

83

101

PART II

The Role of Exploratory Activity and Scale in Perception of Afordances

119

7 The Dynamics of Afordance Emergence and Perception Matheus M. Pacheco and Ricardo Drews

121

8 Description of the World That Agential Systems Fit Into Tetsushi Nonaka

143

9 The Role of Exploratory Activity in Afordance Perception Alen Hajnal 10 Scaling Up: Lawfulness of Afordances Requires Independence From Any Single “Scale of Behavior” Damian G. Kelty-Stephen

161

176

Contents

PART III

ix

Afordances Through the Lens of Neuroscience

197

11 Afordance Switching in Self-Organizing BrainBody-Environment Systems Vicente Raja and Matthieu M. de Wit

199

12 What Is NExT for Afordances? Taking Brains Seriously in Organism-Environment Systems Luis H. Favela

214

13 Afordance and Tool Use: A Neurocognitive Approach François Osiurak and Giovanni Federico

232

14 From Turing to Gibson: Implications of Afordances for the Sciences of Organisms Madhur Mangalam, Louise Barrett, and Dorothy M. Fragaszy PART IV

Applications of the Ecological Theory of Afordances 15 Understanding Skilled Adaptive Behavior: The Role of Action, Perception, and Cognition in an Ecological Dynamics Perspective Ludovic Seifert, Duarte Araújo, and Keith Davids 16 Disability Through the Lens of Afordances: A Promising Pathway for Transforming Physical Therapy Practice Paula L. Silva and Sarah M. Schwab 17 “I Got All I Need to Know About Afordances From Norman”: What Engineers, Designers, and Architects Need to Know About Afordances Balagopal Raveendranath, Elenah Rosopa, and Christopher C. Pagano

249

269

271

291

327

x

Contents

18 Perception of Afordances in Interaction With Autonomous Systems Tri Nguyen, Corey Magaldino, Jayci Landfair, Matt Langley, and Polemnia G. Amazeen 19 On Afordances and Their Entailment for Autonomous Robotic Systems Mihai Andries, Lorenzo Jamone, Justus H. Piater, and Erol Sahin Index

348

363

376

ILLUSTRATIONS

Figures 1.1 Two defnitions of afordance. 2.1 The feld of inviting afordances and the landscape of afordances stand in a reciprocal or co-determining relation. 3.1 The world as a collection of properties. 3.2 The separation of animal from environment, and physical from mental, necessitates a description of knowing about the world and performing a given behavior in the world as one-way processes of translation and then transformation, followed by another one-way process of transformation and translation. 3.3 Research on afordances has tended to focus on afordances for behaviors such as reaching, grasping, stepping over, and ftting through (left). It has not tended to focus on afordances for behaviors such as completing a crossword puzzle, attending a museum exhibit, going on a date with a romantic partner, or obtaining a college degree (right). 3.4 For an object to aford picking up by a given animal, that object must be sufciently nearby, narrow, and detached in relation to the animal’s ability to extend, enclose, and pull with a given efector (left). The elaborated defnition of afordances is that they are opportunities for behavior that emerge from the multifaceted relationship between the animal and the environment under a particular set of circumstances (right).

12 31 43

44

47

49

xii

Illustrations

3.5

3.6

4.1

5.1 6.1

6.2 6.3 7.1 8.1 8.2

12.1 12.2 13.1 15.1

15.2

16.1

Describing the world, and what there is to be known about it, at the ecological scale allows for a description of knowing as the detection of lawfully generated information. In the consensus defnition of afordances, afordances for behavior can be known by detecting lawfully structured information, but afordances for higher-order cognition can only be known (in part) by doing so in the context of non-lawful, conventional constraints (left); in the amended defnition of afordances, all (both lower- and higher-order) afordances can be known by detecting lawfully structured information (right). Decision tree showing diferent ways of understanding mental afordances in terms of mental action, and the implications for the compatibility of the Mental Afordance Hypothesis with Embodied Cognition and Extended Cognition accounts. Example of how experience and habits interrelate. Minkowski space for an observer at a point in spacetime (collapsed from 4 to 3 dimensions: 2 spatial and 1 temporal for ease of demonstration). Ecological Minkowski space. Visual depiction of the co-additivity of events. Demonstration of inherent variability of the motor system. Flow feld around P. dumerilii larva, beating with its locomotor cilia. (a) Ventral surface of a fake detached by conchoidal fracture, as hit by a stone hammer in (b). Flake terminology is provided in (b). The NeuroEcological Nexus Theory (NExT) applied to the afordance of pass-through-able. Neural manifold hypothesis methodology. The three-action system (3AS) model. Illustration of the handholds characteristics for the control route of the learning protocol and for the three routes of the transfer test. Afordances exist in a landscape surrounding individuals and soliciting their actions, as long as they have relevant efectivities that are functional for interacting with a specifc performance environment. Illustrative example of perceptual-motor skills emerging from individual-environmental relations.

51

60

81 94

105 108 112 124 152

156 219 221 233

273

285 296

Illustrations

17.1

17.2 18.1 18.2

If a change in elevation is short relative to the size of an “actor,” then when the actor encounters the “step” the actor will be able to go up onto the step and continue in their current direction of travel (left column). This is the case whether the actor is animate (top row) or inanimate (bottom row). If the change in elevation is tall relative to the size of an actor, then a diferent mode of behavior will result when it is encountered (right column). The OctArm robotic limb attached to a tracked teleoperated robot. The driving simulator. The relationship between the agent and the environment is complicated by the addition of automation.

xiii

333 339 349 357

Tables 3.1 Fundamental characteristics of “stimulus” and “afordance.” 55 4.1 Examples of mental action that do not neatly line up with the covert/overt divide, as well as examples of covert and overt versions of each kind of mental action. 75 16.1 Summary of prominent models of disability. 305 16.2 Summary of ecological models of disability. 309

CONTRIBUTORS

Polemnia G. Amazeen, Department of Psychology, Arizona State University, Tempe 85287, USA. Mihai Andries, Department of Computer Science, IMT Atlantique Brest,

29280 Plouzané, France. Duarte Araújo, School of Human Kinetics, University of Lisbon, 1649004

Lisbon, Portugal. Louise Barrett, Department of Psychology, University of Lethbridge, Leth-

bridge, AB T1K 6T5, Canada. Anthony Chemero, Department of Philosophy, University of Cincinnati,

Cincinnati, OH 45221, USA. Keith Davids, Sport and Physical Activity Research Centre, Shefeld Hallam University, Shefeld S1 1WB, UK. Matthieu M. de Wit, Department of Neuroscience, Muhlenberg College, Allentown, PA 18104, USA. Ricardo Drews, Faculty of Physical Education and Physiotherapy, Federal University of Uberlândia, Uberlândia—MG, 38400-678, Brazil.

Contributors

xv

Tyler Dufrin, Department of Psychology, Illinois State University, Normal, IL 61790, USA. Luis H. Favela, Department of Philosophy and Cognitive Sciences Program, University of Central Florida, Orlando, FL 32816, USA. Giovanni Federico, IRCCS Synlab SDN, 80143 Naples, Italy. Dorothy M. Fragaszy, Department of Psychology, University of Georgia

Athens, Athens, GA 30602, USA. Alen Hajnal, School of Psychology, University of Southern Mississippi,

Hattiesburg, MS 39406, USA. Manuel Heras-Escribano, Departamento de Filosofía I, Universidad de Granada, 18071 Granada, Spain. Lorenzo Jamone, School of Engineering and Materials Science, Queen Mary University of London, London E1 4NS, UK

Max Jones, Department of Philosophy, University of Bristol, Bristol BS8 1TH, UK. Damian G. Kelty-Stephen, Department of Psychology, State University of New York at New Paltz, New Paltz, NY 12561, USA. Julien D. Kiverstein, Department of Psychiatry, Amsterdam University Medical Research, 1081 HJ Amsterdam, the Netherlands. Jayci Landfair, Department of Psychology, Arizona State University, Tempe,

AZ 85287, USA. Matt Langley, Department of Psychology, Arizona State University, Tempe,

AZ 85287, USA. Lorena Lobo, Departamento de Psicología y Salud, Universidad a Distancia

de Madrid, 28049 Madrid, Spain. Corey Magaldino, Department of Psychology, Arizona State University, Tempe, AZ 85287, USA.

xvi

Contributors

Madhur Mangalam, Division of Biomechanics and Research Development,

Department of Biomechanics, and Center for Research in Human Movement Variability, University of Nebraska at Omaha, Omaha, NE 68182, USA. Thomas McClelland, Department of History and Philosophy of Science, University of Cambridge, Cambridge CB2 12N, UK. Tri Nguyen, Department of Psychology, Arizona State University, Tempe, AZ 85287, USA. Tetsushi Nonaka, Graduate School of Human Development and Environment, Kobe University, Kobee 655-0872, Japan. François Osiurak, Laboratoire d’Étude des Mécanismes Cognitifs, Université de Lyon, 69001 Lyon, France; Institut Universitaire de France, 75005 Paris, France. Matheus M. Pacheco, Faculty of Sport, University of Porto, 4200-450

Porto, Portugal. Christopher C. Pagano, Department of Psychology, Clemson University, Clemson, SC 29634, USA. Justus H. Piater, Department of Computer Science, University of Innsbruck,

Innsbruck 6020, Austria. Vicente Raja, Department of Philosophy, University of Murcia, 30071 Murcia, Spain; Rotman Institute of Philosophy, Western University, London, ON N6A 3K7, Canada. Balagopal Raveendranath, Department of Psychology, Clemson University,

Clemson, SC 29634, USA. Elenah Rosopa, Department of Psychology, Clemson University, Clemson, SC 29634, USA. Erol Sahin, Department of Psychology, Middle East Technical University,

Ankara 06800, Turkey. Sarah M. Schwab, Department of Rehabilitation, Exercise, & Nutrition Sci-

ences, Cincinnati, OH 45221, USA.

Contributors

xvii

Ludovic Seifert, Faculty of Sport Sciences, University of Rouen Normandy

Mont-Saint-Aignan, 76000 Rouen, France; Institut Universitaire de France (IUF), 75005 Paris, France. Paula L. Silva, Department of Psychology, University of Cincinnati, Cincin-

nati, OH 45221, USA. Thomas A. Stofregen, School of Kinesiology, University of Minnesota,

Minneapolis, MN 55455, USA. Brandon J. Thomas, Department of Psychology, University of Utah, Salt Lake City, UT 84112, USA. David Travieso, Departamento de Psicología Básica, Universidad Autónoma de Madrid, 28049 Madrid, Spain. Jefrey B. Wagman, Department of Psychology, Illinois State University, Normal, IL 61790, USA. Taraneh Wilkinson, Department of Philosophy, University of Cincinnati,

Cincinnati, OH 45221, USA.

SERIES EDITOR’S FOREWORD

Resources for Ecological Psychology

This series of volumes is dedicated to furthering the development of psychology as a branch of ecological science. In its broadest sense, ecology is a multidisciplinary approach to the study of living systems, their environments, and the reciprocity that has evolved between the two. Traditionally, ecological science has emphasized the study of the biological bases of energy transactions between animals and their physical environments across cellular, organismic, and population scales. Ecological psychology complements this traditional focus by emphasizing the study of information transactions between living systems and their environments, especially as they pertain to perceiving situations of signifcance to planning and execution of purposes activated in an environment. The late James J. Gibson used the term “ecological psychology” to emphasize this animal-environment mutuality for the study of problems of perception. He believed that analyzing the environment to be perceived was just as much a part of the psychologist’s task as analyzing animals themselves, and hence that the “physical” concepts applied to the environment and the “biological” and “psychological” concepts applied to organisms would have to be tailored to one another in a larger system of mutual constraint. His early interest in the applied problems of landing airplanes and driving automobiles led him to pioneer the study of perceptual guidance of action. The work of Nikolai Bernstein in biomechanics and physiology represents a complementary approach to problems of the coordination and

Series Editor’s Foreword

xix

control of movement. His work suggests that action, too, cannot be studied without reference to the environment, and that physical and biological concepts must be developed together. The coupling of Gibson’s ideas with those of Bernstein forms a natural basis for looking at the traditional psychological topics of perceiving, acting, and knowing as activities of ecosystems rather than isolated animals. The aim of this series is to form a useful collection, a resource, for people who wish to learn about ecological psychology and for those who wish to contribute to its development. The series will include original research, collected papers, reports of conferences and symposia, theoretical monographs, technical handbooks, and works from the many disciplines relevant to ecological psychology. Jefrey B. Wagman and Julia J. C. Blau

FOREWORD ON GIBSON’S LEGACY William M. Mace

The editors of this volume, and the contributing writers they assembled, have taken on a daunting task—taking stock of the many ways that the concept of affordance is now being deployed in a wide range of psychology-related research. I say “psychology related” because the contributors come from many different academic departments and many countries. There are 19 chapters grouped under the four headings of Ontology and Epistemology, Exploratory Activity and Scale, The Lens of Neuroscience, and Applications. The editors asked each of the authors to address five questions (although questions 2, 3, and 4 add qualifying questions). This was a clever move to allow much more comparison than would otherwise be possible over such a variety of papers. The word “legacy” in the book’s title is a judicious word choice. As noted throughout the book, the understood meaning of “affordance” has grown in many directions and will be hard to tame. Thus, no easy convergence should be expected. The word “legacy” is sufficient to provide a goal and direction without promising a final synthesis. The editors of this collection have, quite reasonably, chosen to anchor the origin of the affordance concept in Gibson’s 1977 chapter, which was a draft of a chapter for his 1979 book. Some of the chapters recognize that Gibson did not invent the term or its substance so late as 1977, but I do want to make sure that readers know it was mentioned in his 1966 book, then gradually developed from there. For example, see Gibson (1971, unpublished). Many people also see the essential features of affordances in Gibson and Crooks’ (1938) paper on driving a car.

Foreword On Gibson’s Legacy

xxi

A historical question pertinent to this book involves the 1935 paper by Tolman and Brunswik (1935). Why did the “manipulanda properties” described in that paper, especially what we find on page 53, not strike the chords that “affordance” has? “The manipulanda of an object is, so-tospeak, its essential, behavioral core. They are the properties which make possible and support such and such actual behavioral manipulations. They are the object’s grasp-ableness, pick-up-ableness, chewableness, sit-onableness, run-through-ableness, and the like.” For readers who might not know, Edward Chace Tolman and Egon Brunswik were very prominent psychologists. Tolman studied under E. B. Holt at Harvard, just as James Gibson later studied under Holt at Princeton. Gibson spent time with both Tolman and Brunswik at Berkeley in 1955. Why wouldn’t Gibson and Tolman show some commonality? Nevertheless, talk of affordance is now common, and few have heard of manipulanda. As this volume shows, the concept of “affordance” has spread widely since 1977, if not metastasized. The editors want this collection to be a good resource to guide people to material that would have taken far too long to discover without it. I am very pleased that this volume now exists and express my gratitude to the editors and hardworking authors. William M. Mace Professor of Psychology Emeritus, Trinity College, Hartford, CT Reference list Gibson, J. J. (1971). A Preliminary Description and Classification of Affordances. Unpublished manuscript. https://commons.trincoll.edu/purpleperils/ 1970-1971/a-preliminary-description-and-classification-of-affordances/ Gibson, J. J., & Crooks, L. E. (1938). A theoretical field-analysis of automobiledriving. American Journal of Psychology, 51(3), 453–471. https://doi.org/ 10.2307/1416145 Tolman, E. C., & Brunswik, E. (1935). The organism and the causal texture of the environment. Psychological Review, 42(1), 43–77. https://doi.org/10.1037/ h0062156

PREFACE

The concept of afordance has been a cornerstone in ecological approaches to psychology, revolutionizing our understanding of how individuals construct their experiences and engage with their environment. Coined by James J. Gibson in 1977, afordance refers to the opportunities for action that the environment presents to an organism. It provided a more comprehensive framework than the traditional notion of stimuli, which had become increasingly limited and elusive in its ability to explain goaldirected behavior. Over the past fve decades, research on afordances has fourished, leading to exciting and diverse interpretations and applications of the concept. However, this abundance of work has also given rise to challenges and complexities, making it difcult for newcomers to navigate the feld according to their interests. It is in this context that the present work emerges, seeking to take stock of the current state of afordances in studying goal-directed behavior. This book brings together the contributions of 19 groups of scholars who refect on the modern legacy of Gibson’s afordance concept. They explore how afordances have evolved beyond Gibson’s original formulation, examining the promises fulflled, the challenges encountered, and the new questions that have arisen. The contributions are organized around four major themes, encompassing both philosophical and experimentalpsychological perspectives. The frst theme delves into the ontology and epistemology of afordances, exploring the abstract outlooks and logic required to fully utilize the afordance concept in theories of perceiving and acting. The second

Preface

xxiii

theme focuses on the role of exploratory activity and scale in perceiving afordances, shedding light on how afordances can inform empirical attempts at theory-building in perception-action. Moving beyond the traditional Gibsonian scope, the third theme explores the intersection of afordances with neuroscientifc approaches to the sciences of organisms. It addresses the crucial question of the nervous system’s role in an explanatory framework that incorporates afordances, emphasizing the compatibility and potential synergy between the two. The fourth and fnal theme examines the practical applications of afordance concepts, ranging from training and rehabilitation of organisms to the design of inclusive living spaces and tools, as well as their implications for artifcial autonomous and robotic systems. Each group of contributors answers the same set of questions, providing a unique and comprehensive exploration of the multifaceted landscape of afordance-themed work. This format ofers a unifed perspective while showcasing the diverse approaches and perspectives within the feld, enabling both newcomers and seasoned scholars to identify convergences and divergences in their research. Crucially, this book seeks to address an overarching concern: What can afordances still aford us after all this time? By bringing together these insights, we aim to provide a valuable resource for researchers interested in the sciences of organisms and the profound implications of the afordance concept. As we embark on this journey through the legacy and future of afordances, we invite you to join us in exploring the profound impact this concept has had and continues to have on our understanding of perception, action, and the intricate relationship between organisms and their environments. We extend our sincere gratitude to the authors of the book chapters, whose perceptive and stimulating viewpoints have enhanced this work. We also owe a debt of gratitude to our advisors and colleagues, who have served as a constant source of motivation for us throughout our careers in ecological psychology. We also thank the organizations and research groups that have helped and encouraged our ecological psychology work. We appreciate your support and encouragement. This book is a testimonial to the teamwork of an active and enthusiastic group of scholars who are attempting to better understand the function of afordances in the sciences of organisms. May our continued collaboration and exploration lead to further insights and advancements in ecological psychology. Madhur Mangalam, Alen Hajnal, and Damian G. Kelty-Stephen

50 YEARS ON What Does Afordance Aford Us? Madhur Mangalam, Alen Hajnal, and Damian G. Kelty-Stephen

Why Afordances?

Afordance has been a centerpiece of ecological approaches to psychology. Gibson developed this concept through the 1960s and 1970s, defning an “afordance” as an “opportunity for action” (Gibson, 1977). This concept ofered novel means to understand the role of individual actors in constructing their experience. For explaining action and response to ongoing events, afordance provided a broader substrate beyond the strictures and ambiguities of “stimulus” (Gibson, 1960). Stimuli had served experimental psychology and physiology well for decades. Nevertheless, by the middle of the 20th century, the concept of stimuli had grown murky, with stimuli becoming too brief, too small, and too impoverished to support goal-directed behavior—and difcult to pin down with any agreement. The concept of afordance promised to root our understanding of goal-directed behavior into an ontology better suited to active organisms. The past 50 years have been fruitful for scholarship and research on afordances. In many respects, this abundance of afordance-themed research is a ringing success. Then again, it has also produced its diversity of interpretations and uses of the afordance concept, unlike what Gibson (1960) found troubling about “stimulus.” The abundance of work inspired by the concept of afordance is exciting, daunting, and at times confusing—especially for a newcomer scholar who has not developed a knack for navigating according to their interests. It is thus worth asking, respectfully but critically, what does the concept of afordance aford us researchers interested in the sciences of organisms now, almost 50 years from frst coining? DOI: 10.4324/9781003396536-1

2

Madhur Mangalam, Alen Hajnal, and Damian G. Kelty-Stephen

The present work attempts to take stock of the current state of afordances in studying goal-directed behavior. We have assembled the contributions of 19 groups of scholars to provide an overview of the modern legacy of Gibson’s afordances for the sciences of organisms. We have invited these brave participants to refect on how they see the afordance concept growing beyond Gibson’s immediate reach, for example, the promises fulflled, the challenges encountered, and the new questions that this concept has uncovered. We have organized these contributions around four major themes. First, in a philosophical vein, we consider the ontology and epistemology of afordances. This theme addresses the abstract outlooks and logic needed to make the fullest use of the afordance concept in theories of perceiving and acting. Second, in a more experimental-psychological vein, we consider the role of exploratory activity and scale in perception of afordances. This theme addresses how afordances can motivate and support empirical attempts at theory-building in perception-action. These frst two themes encompass the traditional Gibsonian scope of discourse about afordances. This overview highlights how the afordance concept may have spread far beyond the original language of Gibson (1977) in our third and fourth themes. Some of the most impressive and instructive turns in the trajectory of the concept of afordance have been its appearance in neuroscientifc and applied felds. So, in our third theme, we consider what the concept of afordance brings to neuroscientifc approaches to the sciences of organisms. A major stumbling block in the discourse around afordances is the question of the nervous system’s status in an explanatory framework that enlists afordances. Ecological approaches to psychology have generally prompted critical reappraisals of the proper role of the brain and nervous system. For instance, Gibson (1966) highlighted that senses were whole perceptual systems, not merely the sensing tissues but also the tissues and constraints supporting those sensing tissues. One reading of his critique of stimulus amounts to a refusal to equate perception with a series of action potentials (Gibson, 1966). However, it is important to understand that embracing afordances does not amount to a denial of neural contributions. On the contrary, we ofer this third theme to clarify how the afordance concept and neuroscience may support one another. In our fourth and fnal theme, we consider what the afordance concept brings to the state-of-the-art applications of ecological approaches supporting organisms in everyday life. This theme addresses the training and rehabilitation of perceiving-acting organisms, the design of living spaces and tools providing inclusive support for everyday functioning, and the entailment of afordance concept for artifcial autonomous and robotic systems. In many ways, the afordance concept

50 Years On

3

refected Gibson’s (1979) hope that scientists might take organisms more seriously outside of the strictures of a traditional psychology lab. So, he might have been proud to see afordances spread to a wide range of domains extending beyond that lab, into the day-to-day concerns of organisms, and into our technological attempts to refashion our environments and develop new agents. All contributors have answered the same questions: 1. What do we understand by the term afordance? 2. What role does afordance play in perception? Is it the entity organisms perceive or the means through which organisms perceive? Are afordances the only perceptual dependent variables? 3. Why is afordance any better than stimulus? What does a theory of afordance suggest that stimuli cannot; how has it moved the needle past Gibson 1960’s recognition of how little we can defne stimuli? 4. What is the connection between afordances, behavioral scale, and intention? How distant or reluctant can a perceiving-acting system be before afordances come into play? Do afordances exert their role only when the system is close or willing enough? 5. How do systems engage with afordances as they move among tasks and intentions? Crucially, we initiated this project hoping that our colleagues might help us address an overarching concern: what afordances might aford us even after all this time? This format of answering the same questions ofers a unique vantage point for digging into a sprawling landscape of afordancethemed work. The uniformity ofers a neat complement to the rich diversity of approaches, making it easier for the newcomer and veteran alike to see how these multifaceted approaches might converge—or diverge—along the overarching priorities. Ontology and Epistemology of Afordances

Wilkinson and Chemero adopt a relational approach to afordances, acknowledging that environmental features do not solely dictate them. Instead, they recognize the infuence of complex interpersonal, social, and cultural dimensions that shape the human environmental niche. Through insights from phenomenology and enactivist discussions, these perspectives broaden the scope of afordance to encompass a wider range of human experiences, prompting inquiries into its applicability. Kiverstein explores relational theories of afordances, which transcend dualities and boundaries between diferent aspects of perception and action.

4

Madhur Mangalam, Alen Hajnal, and Damian G. Kelty-Stephen

He highlights Anthony Chemero’s perspective on afordances as relations between embodied abilities and environmental situations. Kiverstein’s work with Erik Rietveld defnes afordances in relation to the abilities present in a form of life, referring to the stable patterns of activity within a group of animals. The overarching question addressed in the chapter revolves around the role of social life in understanding afordances, with the sociomaterial theory emphasizing the inseparable intertwining of social and material aspects within afordances. Wagman, Dufrin, and Stofregen delve into a critical argument, proposing that the conventional defnition of afordances can be categorized into two distinct types, each with its mode of perception. Furthermore, they advocate for an amendment that considers afordances emerging from relationships between afordances, thereby unifying them under a single category and perception framework. This amendment aligns harmoniously with the all-encompassing ecological approach, emphasizing the existence of afordances and the governing laws within infnitely intricate contexts. McClelland and Jones delve into how broad our perception of afordances can be, specifcally focusing on the possibility of perceiving afordances for mental action. They discuss the ongoing debate surrounding extending the traditional list of afordances to include mental actions such as attending, imagining, and deliberating. The chapter explores three perspectives on mental action, highlighting the conceptual complexities involved, particularly in embodied and extended cognition. The authors aim to map out these views and their implications for the hypothesis that we perceive afordances for mental action without advocating for any specifc standpoint. Heras-Escribano, Travieso, and Lobo discuss how afordances can explain cognitive processes beyond perception and action, including language, imagination, and social practices. They argue that afordances play a crucial role in understanding the origins of concepts and propose the idea of implicit or embodied concepts as a bridge between basic and discursive cognition. These embodied concepts, formed through experiencing and acting upon afordances, are the missing link between experiential knowledge and explicit conceptual content. Thomas discusses the importance of afordances and afordance perception in academic research but highlights the lack of a principled explanation of time and space at the ecological scale within the theory of afordances. To address this gap, a theory of spacetime is proposed that focuses on organisms in their environment, considering actions and afordances as key factors. This theory introduces new concepts and tools,

50 Years On

5

reorganizes existing ones, and presents novel predictions for cognitive science, aiming to unify the psychology of experience with the laws of natural science. The Role of Exploratory Activity and Scale in Perception of Afordances

Pacheco and Drews discuss the signifcance of afordance perception in ecological psychology and highlight the need to understand its dynamics. They raise questions about how afordance perception occurs based on fndings in motor development and learning, noting the variable nature of motor control and the diference in dynamics between afordance existence and afordance perception. Drawing from the literature on social-afective psychology, the authors speculate on a potential process through which afordances emerge and are perceived. Nonaka presents the idea that biological agents actively seek to control their interactions with the environment, aiming for benefcial encounters and avoiding harmful ones. Understanding afordances involves perceiving the potential encounters that things in the environment can aford. To comprehend the perception and behavior of agents, it is crucial to have a comprehensive description of the world that aligns with their evolution, development, and behavior. The author explores how afordance provides a framework for bridging the gap between the physical properties of objects and the meanings they aford, highlighting the need for an appropriate level of description for evolving agential systems. Hajnal argues that while previous research has focused on identifying invariant properties that correspond to perceptual responses, less attention has been given to the details of information detection during perception. He highlights recent studies showing that exploratory activity is vital in facilitating information detection, with patterns of exploratory movements characterized by complex measures such as multifractality. Based on these fndings, he suggests that the complexity of exploratory activity across different scales infuences information detection and raises questions about traditional psychophysical methods and the concept of perceptual thresholds in studying perception and action systems. Kelty-Stephen refects on the relationship of afordance to scale. He argues that afordances ofer ecological psychology a lawful explanation of perception-action. But he points out that lawfulness may require afordances to exist across various scales, perhaps scale invariantly. The tradition of tying afordances to a “scale of behavior” requires indirect perception for interactions beyond immediate anatomical reach. He recognizes without

6

Madhur Mangalam, Alen Hajnal, and Damian G. Kelty-Stephen

question the importance of scale-dependent constraints within ecological psychology. However, he cautions against confating lawful afordances with scale-dependent constraints, which step could win short-term rhetorical gains but surrender afordances to computational explanations that prioritize inference over embodied relationships. According to Kelty-Stephen, the organism’s perceptual use of afordances for dexterous action occurs across multiple scales, supported by dimensionless afordances and recent models highlighting perception and action as a cascading, scale-invariant process. Afordances Through the Lens of Neuroscience

Raja and de Wit assert that organisms detect information about afordances in their surrounding energy arrays to guide their actions in the ecological approach to perception and action. At the behavioral scale, organisms responding to afordances are seen as self-organizing and reorganizing softly assembled synergies. Although ecological neuroscience has gained interest in studying this process at the neural scale, it remains an underexplored area, and the authors discuss existing empirical research and layout plans to investigate the neural dynamics associated with transitions between afordances within brain-body-environment systems. Favela proposes the NeuroEcological Nexus Theory (NExT) to address the criticism that Gibsonian Ecological Psychology lacks an understanding of the brain’s role in afordances. NExT suggests that afordances emerge through systematic relationships between ecological information and lowdimensional neural manifolds, drawing from recent neuroscience research on neural population dynamics. By incorporating manifold theory, the theory of afordances in Ecological Psychology can be enhanced, providing a more comprehensive understanding that avoids the caricatured portrayal of Gibsonian creatures as devoid of brain contributions. Osiurak and Federico introduce the three-action-system model (3AS), a framework that is based on fndings from behavioral and neuroscience studies. The model proposes that in humans, three distinct neurocognitive systems are responsible for processing diferent types of physical relationships: motor control/dorso-dorsal system for afordances, technical reasoning/ventro-dorsal system for mechanical actions, and semantic knowledge/ ventral system for contextual relationships. The authors draw from recent literature to explore the similarities and diferences between this framework and ecological psychology, particularly in relation to human tool use. Mangalam, Barrett, and Fragaszy discuss the relationship between the concept of “afordance” in ecological psychology and the concept of the “Turing machine” in computation. They argue that Turing’s theory of

50 Years On

7

computation supports Gibson’s ecological theory of afordances, suggesting that afordances can be perceived directly without the need for representations, similar to the confgurations of Turing machines. They also connect afordances with the discovery of ethological action maps in  the motor cortex, which align with the contemporary understanding of the mammalian brain and computation without representation, making the concept of afordance valuable for studying the connection between neural functioning and behavior. Applications of the Ecological Theory of Afordances

Seifert, Araújo, and Davids introduce an ecological dynamics approach to understanding afordances in the context of sports and the acquisition of skilled behaviors. Anchored in complex systems science, ecological psychology, and nonlinear dynamics, this multidisciplinary framework explores how individuals adapt their behaviors in response to surrounding constraints through processes of perception and action. The authors emphasize the role of information in guiding behavior regulation and highlight that afordances are relative to each individual’s abilities, with sports environments providing a rich landscape of multiple afordances that support the emergence of skilled behaviors. By integrating tools and concepts from nonlinear dynamics, the authors illustrate how information is embedded in the dynamics of a performance environment, ofering athletes multiple modes of action during sports performance. Silva and Schwab explore the impact of the concept of an afordance on disability studies and its potential to drive a paradigm shift in physical therapy practice, focusing on the experiences of people with disabilities. They highlight the challenges faced by individuals with disabilities in a world that often fails to accommodate their bodily capabilities, leading to the emergence of activist afordances through intentional and determined actions. The concept of activist afordances emphasizes an understanding of disability that is both embedded and embodied. The chapter concludes by proposing a new vision for physical therapy practice based on this understanding and outlining the need for further afordance research to support this transformative shift. Raveendranath, Rosopa, and Pagano explore the inconsistent application of afordances in engineering, architecture, and design, which deviates from the ecological theory of direct perception and creates two distinct interpretations. Emphasizing the pivotal role of theory, their chapter underscores the need for designers to transcend the current understanding of afordances in these felds and embrace the principles of ecological theory. The authors highlight the benefts of integrating this perspective

8

Madhur Mangalam, Alen Hajnal, and Damian G. Kelty-Stephen

into the design process by providing a concise overview of the key elements of the ecological theory of afordances. Nguyen et  al. examine the afordances of interactions with autonomous systems. They argue that while autonomous systems introduce more complex structured information, they do not fundamentally alter existing frameworks of afordance. Viewing autonomous systems from a systemic perspective, where the agent, the autonomous tool, and the environment dynamically interact, they consider these interactions a natural component of the information array that necessitates attunement, akin to learning to drive a car or ride a horse. Moreover, they highlight that the current application of autonomous systems, allowing for engagement or disengagement as needed, aligns with our established understanding of afordances in tool use, indicating that perception of multiple afordances in this context stems from a shift in the agent’s embodied capabilities rather than changes in intentions or environmental properties. Andries et al. discuss how afordances serve as a theoretical framework in robotics, guiding robots in their interactions with objects within their environment. They highlight the formalization of afordances to align with machine learning approaches, enabling robots to possess autonomous capabilities. The chapter explores the role of afordances in cognitive robots’ perception, particularly in the semantic mapping of the environment, emphasizing the interaction between afordances and intentions. Afordances are seen as a promising avenue for robotic action planning and are being extensively explored within cognitive robotics to enhance robots’ high-level cognitive abilities and achieve greater autonomy. Reference List Gibson, J. J. (1960). The concept of the stimulus in psychology. American Psychologist, 15(11), 694–703. https://doi.org/10.1037/h0047037 Gibson, J. J. (1966). The Senses Considered as Perceptual Systems. Houghton Mifin. Gibson, J. J. (1977). The theory of afordances. In R. E. Shaw & J.  Bransford (Eds.), Perceiving, Acting, and Knowing: Toward an Ecological Psychology (pp. 67–82). Lawrence Erlbaum. Gibson, J. J. (1979). The Ecological Approach to Visual Perception. Houghton Mifin.

PART I

Ontology and Epistemology of Afordances

1 WHY IT MATTERS THAT AFFORDANCES ARE RELATIONS Taraneh Wilkinson and Anthony Chemero

What Do We Understand by the Term “Afordance”?

James Gibson (1979) started the conversation with a defnition of afordances that diverged decisively from the traditional physicalist and reductionist ways of looking at the world. He claims that afordances are neither subjective nor objective, neither physical nor mental (Gibson, 1979, p.  129). This way of defning afordances is perplexing and smuggles in much baggage. Among that baggage is a non-dualism or neutral monism that Gibson inherited from his advisor Edwin Holt, who inherited it from William James. This baggage poses a problem for scientists and philosophers alike in that it is built on ontological assumptions that beg further clarifcation (Chemero, 2003, p. 182). This clarifcation is necessary, especially since afordances are meant to be “real” and empirically observable, not merely subjective constructs. The temptation has sometimes been to cast the notion of afordance as an object in the environment—emphasizing its role as a property of objects (Heras-Escribano, 2019; Reed, 1996; Turvey et al., 1981)—at others to defne afordance as a relation (Chemero, 2003, 2009), or more recently to emphasize the afordances in human social interactions (Baggs, 2021; Brancazio, 2020; Rietveld & Kiverstein, 2014). Since the close of the past century, many authors have converged on the view that afordances are “animal-relative properties of the environment” (Chemero, 2003, p.  192; Heft, 2001, 1989; Michaels, 2000; Stofregen, 2000; Turvey, 1992). There is general agreement that afordances cannot be features of the environment alone; they are instead features of animalenvironment systems (Stofregen, 2003). There is still some disagreement DOI: 10.4324/9781003396536-3

12

Taraneh Wilkinson and Anthony Chemero

over whether afordances are properties of the world of relations between animals and the world. Turvey et  al. (1981) defne afordances as objective, physical properties of the environment, which does not respect Gibson’s claims that they are both subjective and objective, both psychical and physical. However, the property-based defnition of afordances does several valuable things. First and foremost, this formulation of Gibson’s afordances has served as the basis of a successful scientifc research program in psychology. Second, in what Reed (1996) calls the “fundamental hypothesis of ecological psychology” (p. 18), having afordances be properties of the environment allows them to be the driver of evolution by natural selection. Afordances exist in the world; animal species evolve to be able to take advantage of them. In contrast, one of us has argued that afordances are relations between an animal’s abilities to act and a situation in the world (Chemero, 2003, 2009). Afordances, in this view, are neither properties of the environment nor properties of the actor; instead, they are the way actors ft into environments. In this, we are relying on a sentence from Gibson’s original defnition, “An afordance points both ways, to the environment and the observer” (Gibson, 1979, p. 129). This defnition has neither of the benefts mentioned earlier that the view of Turvey and colleagues does. It is somewhat mysterious and takes afordances out of the evolutionary processes. It has other advantages, though; the most important is that it is more faithful to Gibson’s intentions. Gibson considered himself a Jamesian radical empiricist (Heft, 2001). During this late-career era of theorizing, William James argued for two key points: (i) that relations are critical parts of our experience, and (ii) there is no ontological distinction between the physical and psychical. Moreover, as we will see later, understanding afordances as relations opens up new ways of understanding the afordances in social and cultural situations. (See Figure 1.1 for a graphical depiction of the diferences between the two views of afordances.)

FIGURE 1.1

Two defnitions of afordance.

Why It Matters That Afordances Are Relations

13

It is worth stressing that the diferences between afordances as relations and afordances as properties of the environment are not empirically consequential: you would do the same experiments and expect the same results, whichever you believe. Nonetheless, we prefer to understand afordances to be relations between what an animal can do and what its environment is like. The remainder of this document will focus on that understanding of afordances. The growing emphasis on afordances as relations between abilities and the environment has opened up a much broader discussion of the applicability of afordances to the skill-saturated human environmental niche. In recent years, discussions of afordances have targeted more of the abstract and nuanced dimensions of human experience, drawing on sociology, second-person neuroscience, and, increasingly, perspectives from critical phenomenology. This discussion has also had critics skeptical of designating distinctly social afordances when the term already implies social dimensions (Baggs, 2021). Regardless, even such skepticism confrms that afordances should account for social dimensions of organism-environment interactions for the human environmental niche. Further, in the push to expand the meaning of afordance, there is also a need to limit the expansion to keep the term meaningful (Withagen & Costall, 2021). In short, there is not yet a consensus on how many kinds of afordances there might be and to what extent human experience can be captured in the language of afordances. There has been talking of social afordances, interpersonal afordances, epistemic afordances, place afordances, and sociocultural afordances. In an earlier instance, Clark and Uzzell (2002) investigated the afordances of “home,” “neighbor,” and “school” in the case of adolescents. Some worry that extending the notion of afordances to include such social concepts will render the term meaningless. With recent trends to expand the use of the term “afordance” to capture more aspects of human experience, the question remains as to the scope of the term’s use. Recently, debates on potential expansions of the term have taken up the social and cultural dimensions of phenomenological experience, increasingly taking on questions of ethics and normativity. For instance, Rietveld and Kiverstein (2014) propose a new framework and defnition for afordances that take into account human’s various sociocultural niches. They adapt the Gibsonian “way of life,” introducing the Wittgensteinian concept of “form of life” to capture how diferent sociocultural niches enskill diferent groups of humans with diferent sets of abilities and difering ranges of available afordances according to language, culture, and social situation (pp.  327–328). In this case, Rietveld and Kiverstein emphasize that humans have a socioculturally infected environmental niche, “shaped and sculpted by the rich variety of social practices humans engage in”

14

Taraneh Wilkinson and Anthony Chemero

(p. 326). Nevertheless, rather than speaking of “higher” and “lower” levels of consciousness, Rietveld and Kiverstein speak of “level of ability” or “expertise” (p. 346). According to them, “lower” versus “higher” cognition represents a problematic dichotomy with cognitivist roots; instead, they propose an alternative to this dichotomy of perception as “openness to afordances” (p. 347). For humans, afordances are embedded in sociocultural practices. These sociocultural afordances are mediated through the organism’s ability, and we, as humans, are open to all kinds of afordances that emerge out of our specifc sociocultural contexts. Accordingly, Rietveld and Kiverstein ofer a new defnition of afordances: Afordances are relations between aspects [rather than features] of a material environment and abilities available in a form of life. —Rietveld and Kiverstein (2014, p. 335) By calling afordances “relations between aspects,” they take up and expand upon Chemero’s earlier redefnition of the concept: Afordances . . . are relations between the abilities of organisms and features of the environment. —Chemero (Chemero, 2003, p. 189) The new defnition ofered by Rietveld and Kiverstein afrms the relational nature of afordances while substituting “features” with “aspects.” And unlike Chemero’s defnition, this new defnition helps account for how abilities are embedded in sociocultural practice (Rietveld and Kiverstein, 2014, p. 334). At the same time, Rietveld and Kiverstein strongly afrm Chemero’s claim that “afordances and abilities are not just defned in terms of one another . . . but causally interact in real time and are causally dependent on one another” (Chemero, 2009, pp. 150–151).1 Yet, this defnition does not yet account for “population-wide realism” in the case of sociocultural niches, only realism at the individual level (Rietveld & Kiverstein, 2014, p. 336). To account for population-wide realism, it is possible to think of afordances as having “an existence that is relative to the skills available in practice [in the population] . . . to the abilities available in the form of life as a whole” (p. 337). This accounts for the realism of afordances among a given population. This exercise boils down to a distinction between an afordance in the form of life versus afordance for a particular individual. In other words, individuals with diferent abilities sharing the same form of life (sociocultural context) might perceive diferent afordances in the same situated context as a function of their difering skills and abilities. Moreover, since ability implies a correct or successful way of doing something, such ability entails normativity. Rietveld and Kiverstein (2014)

Why It Matters That Afordances Are Relations

15

stress that “there is a normative dimension to the abilities for picking up afordances” (p. 326). The normative dimension of afordances was already recognized in part by Gibson himself (cf. Jayawickreme & Chemero, 2008, p.  122). Importantly, direct perception of sociocultural afordances is acquired, not innate. One learns what counts as right and wrong perceiving from one’s community (Rietveld & Kiverstein, 2014, p. 332). Skill acquisition for humans involves “sociomaterial scafolding.” Skill acquisition cultivates attention. Thus, it is (for infants) about learning to care about the “right” thing (p. 331). The authors stress that this kind of normativity difers from justifcation and is a more basic kind. They call this more basic kind of normativity “situated normativity,” drawing on Rietveld (2008). The notion of normativity that we take to be applicable to a skilled individual’s engagement with afordances comes from that individual’s ability to distinguish correct from incorrect, better from worse, optimal from suboptimal, or adequate from inadequate activities in a specifc, concrete material setting (Rietveld & Kiverstein, 2014). Even if perception of sociocultural afordances is acquired, it is still direct, and the associated afordances are real in at least two ways: (i) their existence does not depend on a single individual’s use and (ii) sociocultural practices and their accompanying afordances are dependent on the material ofered by the environment. Since sociocultural afordances are meant to be real in these two ways, “the correctness of a judgment then will depend on both the material environment and the sociocultural practice” (Rietveld & Kiverstein, 2014, p. 334). On a similar note, Jayawickreme and Chemero (2008) have previously stressed the normative character of afordances. Again, normative value is tied to ability and meaning: “abilities are defned by how things are supposed to go”; per Gibson, “afordances are meanings and meanings are also normative” (p.  122). In stressing the normative aspects of afordances, Jayawickreme and Chemero (2008) defne afordances primarily as relations, “relations between abilities & behaviorally relevant aspects of situations.” The normative character is parsed in terms of moral afordances, “relations between morally relevant abilities and morally relevant situations” (p.  122). Moral afordances are not virtues, however. Instead, virtue maps onto the ability of the organism to respond to moral afordances. Jayawickreme and Chemero clarify that virtues are “abilities to behave appropriately in morally relevant situations,” and this means that virtues, rather than functioning similar to “global personality traits,” are rather “abilities to behave in very specifc contexts” (p. 122). They think that moral afordances understood in this sense fulfll requirements to be “partly normative,” “fully objective,” and empirically accessible (p. 122).

16

Taraneh Wilkinson and Anthony Chemero

In sum, as the use of the term “afordance” is applied to social and cultural dimensions of the human environmental niche, many of the original Gibsonian takes on the term remain central—for instance, their partly normative character, their instances as real objects, and our ability to empirically study them. Moreover, as we move into accounts of social and cultural afordances, the original neutral monism or non-dualism in Gibson’s initial understanding of the term comes to the fore, and it becomes all the more important to acknowledge afordances as both objective resources in the world and functions of a relationship between an organism and its environmental niche. At the same time, this raises new questions about the relationship between sociocultural afordances and material afordances and whether the distinction is ultimately meaningful in a relational view of afordances. The commitment to objectivity may inspire some to keep material afordances distinct and more primary than sociocultural ones. At the same time, those who take a robustly relational view of afordances might be tempted to follow Baggs (2021), who stresses that all material afordances are perceived in the context of a specifc environmental niche and, as such, will always be sociocultural in the case of a social species. What Role Does Afordance Play in Perception? Is It the Entity Organisms Perceive or the Means Through Which Organisms Perceive? Are Afordances the Only Perceptual Dependent Variables?

As far as the neutral monist is concerned, afordances are both the entity organisms perceive and the means through which organisms perceive; afordances are both resources and relations. They are both the objects of perception and the process by which organisms perceive anything at all. The difculty in taking a Gibsonian neither-singly-one-nor-solely-theother-and-also-both approach is that the tendency may be to use the term “afordance” indiscriminately to exclude other relevant perceptual dependent variables, should they exist. Afordances are “meanings available in the environment” (Jayawickreme & Chemero, 2008, p. 122; Reed, 1996). Of course, many meanings are available in an environment that is not afordance. Gibson included a sidebar in The Ecological Approach to Visual Perception (1979) entitled “To Perceive an Afordance Is Not to Classify an Object,” in which he distinguishes seeing that an object is a rock from seeing that it afords throwing. Seeing that an object is a rock perceives something other than an afordance. Both perceptions involve meaning but only one is an afordance. In 21st-century ecological psychology, direct learning theory

Why It Matters That Afordances Are Relations

17

(Jacobs & Michaels, 2007) requires that organisms perceive the adaptivity of their action to attune to the correct information in the environment and appropriately calibrate future actions. In addition to the information about afordances, there is information for learning. More mundanely, we perceive patterns, textures, works of art, and many other things outside the context of what activities they aford. Why Is Afordance Any Better Than Stimulus? What Does a Theory of Afordance Suggest That Stimuli Cannot; How Has It Moved the Needle Past Gibson 1960’s Recognition of How Little We Can Defne Stimuli?

Long before Gibson, Dewey (1896) pointed out that stimuli cannot be identifed without a response: a stimulus is something an organism responds to. The relationship between afordance and stimulus may depend on whether one takes a resource view or a relational view of afordances. Caiani (2014), who takes a resource view of afordance, parses a tight and nearly interchangeable relationship between afordance and stimulus; afordance is “univocally specifed by the structure of the stimulus, so that detecting the latter is the same as perceiving the former” (p. 280). On a more relational view, an afordance difers from a stimulus in terms of its ontology, and as a result, in the way it highlights the relationship between itself and the perceiving agent. A stimulus that leads to a response is similar to a relevant afordance or “solicitation” since not all afordances invite an agent equally. Not all individuals of the same species that share an environmental niche will perceive the same afordances, particularly in the case of humans. However, even a relevant afordance, or solicitation, difers from a stimulus in stressing its reality as a relation between the perceiver and the perceived—a neutral monism versus the implied substance ontology of stimuli. Withagen et  al. (2017), in their article “Inviting Afordances and Agency,” discuss the relation between the invitation of afordance and individual agency. Drawing on Reed’s selective retention theory, they note that the primary mode of an organism in relation to afordances is not self-making and refective but in unrefective responses to invitations. As they remark, the environment addresses us as organisms, and we respond. Or, put diferently, the environment is directed at the organism (Käufer & Chemero, 2021). Diferent available afordances solicit us with diferent strengths, and we can respond to or resist invitations. However, “invitations are not exclusively determined by the agent’s intentions” (Withagen et al., 2017, p. 13). As Heft (2001) has noted, drawing on Roger Barker’s concept of “behavior settings” (Barker, 1965), a social organism like an

18

Taraneh Wilkinson and Anthony Chemero

individual human move within and through various behavior settings. These behavior settings are abstract structures in the environment that arise from collective social actions. If an organism encounters a meaning-laden environment, the question arises as to how said organism orients itself to some meanings and not others. As Rietveld and Kiverstein (2014) put it: “How can we scientifcally account for an agent’s ability to select the action that is appropriate to the particularities of the situation given that there is always an open-ended number of possible actions available!” (p. 346). An answer to this question requires an account of agency. As Withagen et al. (2017) note, “[E]cological psychologists made room for the idea that animals can be the source of their own activity” (p. 16). While ecological psychology’s take on afordances has been criticized for lacking a robust account of agency, a relational afordance view that stresses its relational quality has resources for giving an account of an organism’s agency. On a relational view, an afordance difers from a stimulus because it already includes an account of the perceiver’s agency insofar as it is a relation. For instance, Withagen et  al. (2017) ofer a novel ecological conception of agency, defned as “the animal’s capacity to modulate the coupling strength with those afordances,” where “the agent [or animal] can infuence to what extent each invitation infuences him or her” (p. 14). On the one hand, “the agent is generally coupled to multiple afordances . . . that invite to diferent degrees” (p. 15); on the other, the agent can modulate the infuence of said invitations, and this modulation is an exercise of agency that can be modeled by the change in coupling strength between animal and afordance. Their account ofers a more robust take on agency, and the phenomenal experience of afordances can be modeled in a multi-dimensional space. They draw on enactivist work on conditions for agency, naming three conditions: individuality, normativity, and interactional asymmetry (Barandiaran et al., 2009). By focusing on coupling strength between agent and various afordances, Withagen et  al. (2017) establish interactional asymmetry, an important criterion for agency. What does this asymmetry look like? It can move in either direction, favoring the environment or animal, depending. If, on the one hand, afordances are largely invitations, then agency is reduced. Withagen et al. (2017) describe this as “giving in to the environment’s demands” (p. 16). On the other hand, when the agent can change the strength at which various afordances impose themselves on its perceptual feld, individual agency manifests. And as with above, how and whether afordances invite or exert their role depends on the ability of an organism.

Why It Matters That Afordances Are Relations

19

So, what does a theory of afordance suggest that a theory of stimuli cannot? Depending on the interpretation of afordances, it could provide not only an account of an organism’s feld of afordances, which some might equate to the structure of stimulus, but it could also give an account of the organism as an agent and how a given species’ environmental niche is shaped by both the organism and its surroundings. What Is the Connection Between Afordances, Behavioral Scale, and Intention? How Distant or Reluctant Can a PerceivingActing System Be Before Afordances Come Into Play? Do Afordances Exert Their Role Only When the System Is Close or Willing Enough?

Suppose we accept that afordances arise out of sociocultural niches, the possibility arises for distal invitations, that is, for afordances to be perceived directly yet still at an abstract distance from the material present. A perceiving-acting system that is enmeshed and enskilled in a specifc sociocultural niche will have a feld of afordances that refect both immediate and more distant invitations to action. For instance, middle-class American parents in the 21st century may have been habituated to considering a college education as a necessity or, at the very least, an important resource for their children’s future career success. In a job ofer that covers college tuition for dependents, they can directly perceive the ability to provide their children with that resource, much like they would directly perceive food on a table suitable for eating or ofering to their children. However, suppose their children are still quite young, the moment they leave for college is a long way away, yet this does not make the perception of the future ability to provide for their children’s college education as part of the job ofer any less immediate or direct. The ofer letter illustrates that individuals may directly perceive afordances that are quite distant in time and space, given a specifc position in culture and history of learning (Bruineberg et al., 2019). As intimated in the previous example, a relational approach to afordances opens up avenues for considering the role of temporal scale in afordances. Gastelum’s (2020) work on a temporal scale does just this. Drawing on a Husserlian account of “lived time,” she stresses that afordances are “shaped by a history of interactions” (p.  7). Afordances reference both “past abilities” and “future possibilities,” and, as a result, the full temporality of the process is inescapable (p. 8). According to Gastelum, “temporal scales matter because afordances serve a diferent explanatory role

20

Taraneh Wilkinson and Anthony Chemero

depending on which time scale you consider them at” (p. 9). An account of temporality is crucial for any theory of how abilities to perceive are learned and how organisms, as agents, can shift from answering the call of one invitation to attending to another afordance. How Do Systems Engage With Afordances as They Move Among Tasks and Intentions?

At any given moment, only a few afordances are also invitations. Which afordances quality as invitations at aby given moment is a function of what organisms are concerned with and what they are doing. Recent discussions of relevance and the inviting nature of afordances tend not to delve too deeply into the category of concern, except for some work on the neuroscience of afordances and how diferent psychiatric treatments might reshape a given individual’s feld of afordances. However, if we take the role of the sociocultural environment for humans seriously, afordances will also need to consider how some immediate concerns of individuals are shaped by systemic factors of their sociocultural context, sometimes even against the individual’s wishes. Let us start with an example from neuroscience. de Haan et al. (2013) take a phenomenological approach and map felds of afordance for patients receiving deep brain stimulation (DBS). According to de Haan et al., DBS treatment can produce major shifts in the experience of patients with obsessive compulsive disorder (OCD), especially with regard to how they perceive afordances in the environment. They explore and model the phenomenology of these patient experiences, speaking of the patients’ world in terms of “a feld of afordances” with three dimensions: (i) width or broadness of scope, (ii) depth or temporal horizon, and (iii) relevance of afordances or height. Expanding on the relational notion of afordances, de Haan et  al. bring in enactivism: “Enactivism ofers a dynamical systems perspective on the fundamental relatedness (or “coupling” in enactive terms) between person and world” (de Haan et al., 2013, p. 6). This fundamental relatedness is important. In this case, the relational quality is parsed in terms of dynamics and loops between animals and the environment, leading to dysfunction and impaired ability. An example of such a loop is the “hyper-refexivity trap” in which OCD patients are prone to be trapped (de Haan et al., 2013, p. 6). The hyper-refexivity trap starts from a place of discomfort and spirals into increasing ill ease through overfocus on the action. The hyper-refexivity trap calls for an account of how the direct perception of afordances fails us, that is, when the relationship between animals and the environment becomes pathological. De Haan et al. argue that, for instance, the feld of afordances of an individual

Why It Matters That Afordances Are Relations

21

with OCD is obstructed by afordances on the immediate temporal horizon with exaggerated relevance. The feld of afordances of an individual difers from the larger “landscape of afordances” that capture all possibilities for action open to a “form of life” or environmental niche (p. 7). The authors claim that after DBS treatment, many OCD patients experienced a phenomenal shift in their way of being in the world, writing that “the changed world that [their] OCD patients described can be feshed out in terms of changes in their feld of relevant afordances” (p. 7). The case of OCD is helpful in that patients with OCD may have difculties toggling concerns for diferent afordances due to the exaggerated relevance of afordances on the immediate temporal horizon. DBS treatment seemed to have some efect on the “existential stance” or “second order-relation” to “aspects of the person-world interaction” (p. 10). This possibility suggests that the ways in which agents move between tasks and intentions is a second orderrelation to the invitation of afordances, one we can only assume is mediated by specifc sociocultural contexts. Humans perceive and navigate our respective felds of afordances shaped by social, sometimes without the agent’s intention or will for it to be so. For social species like humans, afordances are perceived in social and cultural environments (Baggs, 2021; Gallagher & Ransom, 2016; Rietveld & Kiverstein, 2014). Brancazio (2020) has taken this a step further and recently argued that interpersonal afordances are a distinct type of afordance characterized by the interaction of two agents. She defnes interpersonal afordances as “actual possibilities for interaction with an agent” (p.  10), drawing on feminism and critical race theory insights as to how bodies can be treated as diferent social selves in diferent interactions (Anzaldúa, 1987; Barvosa, 2008; Harris, 2018; Ortega, 2015, 2019; Wing, 1990). Brancazio stresses the phenomenological immediacy of interpersonal afordances. According to Brancazio (2020), which afordances become invitations depends upon how one is seen, that is, how one’s identity is received and recognized by other agents. A sexist comment, for example, can suddenly and drastically change which of the available afordances are experienced as inviting action. Similarly, the concrete furniture of our social world, like medical equipment (Carbonell & Liao, 2021; Liao & Carbonell, 2022; Liao & Huebner, 2021), can embody biases that restrict which afordances become invitations, restricting the action possibilities that seem available to members of oppressed classes. Interpersonal afordances carry moral valence. In a parallel and more established trend, phenomenologists have applied critical insights from gender, race, and coloniality studies to think more carefully about how larger-scale social realities shape our frst-person and intersubjective experiences of the world (McMahon, 2017, 2018; Weiss

22 Taraneh Wilkinson and Anthony Chemero

et al., 2019; Young, 1980). Iris Marion Young has argued since the 1980s that oppressed bodies and their accompanying subjective experiences are partly constituted and negatively constrained by the external subjectivity of their oppressors. The oppressed body and consciousness experiences herself in part as object rather than subject or agent, and her very movements belie this external constraint. While Young writes from a feminist perspective, this insight has also been developed beyond the context of gender. Critical phenomenology has been applied to matters such as “compulsory able-bodiedness” (McRuer, 2020), coloniality (Mendieta, 2020), race (Karera, 2020), trans experiences (Bettcher, 2012; Salamon, 2010), and sexuality (Burke, 2020). Critical phenomenology can be traced as far back as the origins of many contemporary phenomenological debates, to the mid-20th century contributions of anti-racist decolonial philosopher Frantz Fanon and his engagement with the work of phenomenologists Merleau-Ponty (Käufer & Chemero, 2021). Relational accounts of afordances have already drawn extensively on phenomenology (Gastelum, 2020; Withagen et al., 2017). Critical phenomenology has many insights for a theory of afordances as a function of ability and temporal history within a sociocultural niche. The future of the question as to how systems engage with afordances as they move among tasks and intentions in a sociocultural context looks to be one of reconnecting with the phenomenological roots of afordances from new critical vantages. Note 1. The causal point that Chemero is trying to make is an interesting and difcult one to parse. However, there are potential analogs in other contexts, such as the Buddhist notion of co-dependent co-origination of things (pratītyasamutpāda), or more closely related to the history of phenomenology, F.D.E. Schleiermacher’s notion of Wechselwirkung that more or less captures the dynamical nature of mutual dependence between two phenomena (cf. Schmidt, 2005).

Reference List Anzaldúa, G. (1987). Borderlands: The New Mestiza. Aunt Lute Book Co. Baggs, E. (2021). All afordances are social: Foundations of a Gibsonian social ontology. Ecological Psychology, 33(3–4), 257–278. https://doi.org/10.1080/1 0407413.2021.1965477 Barandiaran, X. E., Di Paolo, E., & Rohde, M. (2009). Defning agency: Individuality, normativity, asymmetry, and spatio-temporality in action. Adaptive Behavior, 17(5), 367–386. https://doi.org/10.1177/1059712309343819 Barker, R. G. (1965). Explorations in ecological psychology. American Psychologist, 20(1), 1–14. https://doi.org/10.1037/h0021697

Why It Matters That Afordances Are Relations

23

Barvosa, E. (2008). Wealth of Selves: Multiple Identities, Mestiza Consciousness, and the Subject of Politics (Vol. 14). Texas A&M University Press. Bettcher, T. M. (2012). Full-frontal morality: The naked truth about gender. Hypatia, 27(2), 319–337. https://doi.org/10.1111/j.1527-2001.2011.01184.x Brancazio, N. (2020). Being perceived and being “seen”: Interpersonal afordances, agency, and selfhood. Frontiers in Psychology, 11, 1750. https://doi. org/10.3389/fpsyg.2020.01750 Bruineberg, J., Chemero, A., & Rietveld, E. (2019). General ecological information supports engagement with afordances for “higher” cognition. Synthese, 196(12), 5231–5251. https://doi.org/10.1007/s11229-018-1716-9 Burke, M. (2020). Heteronormativity. In G. Weiss, G. Salamon, & A. V. Murphy (Eds.), 50 Concepts for a Critical Phenomenology (pp. 161–168). Northwestern University Press. Caiani, Z. S. (2014). Extending the notion of afordance. Phenomenology and the Cognitive Sciences, 13(2), 275–293. https://doi.org/10.1007/s11097-0139295-1 Carbonell, V., & Liao, S. (2021). Materializing systemic racism, materializing health disparities. American Journal of Bioethics, 21(9), 16–18. https://doi.org/ 10.1080/15265161.2021.1952339 Chemero, A. (2003). An outline of a theory of afordances. Ecological Psychology, 15(2), 181–195. https://doi.org/10.1207/S15326969ECO1502_5 Chemero, A. (2009). Radical Embodied Cognitive Science. MIT Press. Clark, C., & Uzzell, D. L. (2002). The afordances of the home, neighbourhood, school and town centre for adolescents. Journal of Environmental Psychology, 22(1), 95–108. https://doi.org/10.1006/jevp.2001.0242 de Haan, S., Rietveld, E., Stokhof, M., & Denys, D. (2013). The phenomenology of deep brain stimulation-induced changes in OCD: An enactive afordance-based model. Frontiers in Human Neuroscience, 7, 653. https://doi.org/10.3389/ fnhum.2013.00653 Dewey, J. (1896). The refex arc concept in psychology. Psychological Review, 3(4), 357–370. https://doi.org/10.1037/h0070405 Gallagher, S., & Ransom, T. G. (2016). Artifacting minds: Material engagement theory and joint action. In G. Etzelmüller & C. Tewes (Eds.), Embodiment in Evolution and Culture (pp. 337–351). Mohr Siebeck GmbH & Co. KG. Gastelum, M. (2020). Scale matters: Temporality in the perception of afordances. Frontiers in Psychology, 11, 1188. https://doi.org/10.3389/fpsyg.2020.01188 Gibson, J. J. (1979). The Ecological Approach to Visual Perception. Houghton Mifin. Harris, A. P. (2018). Race and essentialism in feminist legal theory [1990]. In K. T. Bartlett & R. Kennedy (Eds.), Feminist Legal Theory: Readings in Law and Gender (pp. 235–262). Routledge. Heft, H. (1989). Afordances and the body: An intentional analysis of Gibson’s ecological approach to visual perception. Journal for the Theory of Social Behaviour, 19(1), 1–30. https://doi.org/10.1111/j.1468-5914.1989.tb00133.x Heft, H. (2001). Ecological Psychology in Context: James Gibson, Roger Barker, and the Legacy of William James’s Radical Empiricism. Psychology Press. Heras-Escribano, M. (2019). The Philosophy of Afordances. Palgrave Macmillan.

24

Taraneh Wilkinson and Anthony Chemero

Jacobs, D. M., & Michaels, C. F. (2007). Direct learning. Ecological Psychology, 19(4), 321–349. https://doi.org/10.1080/10407410701432337 Jayawickreme, E., & Chemero, A. (2008). Ecological moral realism: An alternative theoretical framework for studying moral psychology. Review of General Psychology, 12(2), 118–126. https://doi.org/10.1037/1089-2680.12.2.118 Karera, A. (2020). The racial epidermal schema. In G. Weiss, G. Salamon, & A. V Murphy (Eds.), 50 Concepts for a Critical Phenomenology (pp.  289–294). Northwestern University Press. Käufer, S., & Chemero, A. (2021). Phenomenology: An Introduction (2nd ed.). Polity Press. Liao, S., & Carbonell, V. (2022). Materialized oppression in medical tools and technologies. The American Journal of Bioethics, 1–15. https://doi.org/10.1080/ 15265161.2022.2044543 Liao, S., & Huebner, B. (2021). Oppressive things. Philosophy and Phenomenological Research, 103(1), 92–113. https://doi.org/10.1111/phpr.12701 McMahon, L. (2017). Phenomenology as frst-order perception: Speech, vision, and refection in Merleau-Ponty. In K. Jacobson & J. Russon (Eds.), Perception and Its Development in Merleau-Ponty’s Phenomenology (pp. 308–337). University of Toronto Press. McMahon, L. (2018). (Un)healthy systems: Merleau-Ponty, Dewey, and the dynamic equilibrium between self and environment. Journal of Speculative Philosophy, 32(4), 607–627. https://doi.org/10.5325/jspecphil.32.4.0607 McRuer, R. (2020). Compulsory able-bodiedness. In G. Weiss, G. Salamon, & A. V. Murphy (Eds.), 50 Concepts for a Critical Phenomenology (pp. 61–68). Northwestern University Press. Mendieta, E. (2020). Toward a decolonial feminist imaginary: Decolonizing futurity. Critical Philosophy of Race, 8(1–2), 237–264. https://doi.org/10.5325/ critphilrace.8.1-2.0237 Michaels, C. F. (2000). Information, perception, and action: What should ecological psychologists learn from Milner and Goodale (1995)? Ecological Psychology, 12(3), 241–258. https://doi.org/10.1207/S15326969ECO1203_4 Ortega, M. (2015). Latina feminism, experience and the self. Philosophy Compass, 10(4), 244–254. https://doi.org/10.1111/phc3.12211 Ortega, M. (2019). Spectral perception and ghostly subjectivity at the colonial gender/race/sex nexus: Ortega Spectral perception and ghostly subjectivity. Journal of Aesthetics and Art Criticism, 77(4), 401–409. https://doi.org/10.1111/ jaac.12673 Reed, E. S. (1996). Encountering the World: Toward an Ecological Psychology. Oxford University Press. Rietveld, E. (2008). Situated normativity: The normative aspect of embodied cognition in unrefective action. Mind, 117(468), 973–1001. https://doi.org/10.1093/ mind/fzn050 Rietveld, E., & Kiverstein, J. (2014). A rich landscape of afordances. Ecological Psychology, 26(4), 325–352. https://doi.org/10.1080/10407413.2014.958035 Salamon, G. (2010). Assuming a Body: Transgender and Rhetorics of Materiality. Columbia University Press. Schmidt, S. (2005). Die Konstruktion des Endlichen. De Gruyter.

Why It Matters That Afordances Are Relations

25

Stofregen, T. A. (2000). Afordances and events. Ecological Psychology, 12(1), 1–28. https://doi.org/10.1207/S15326969ECO1201_1 Stofregen, T. A. (2003). Afordances as properties of the animal-environment system. Ecological Psychology, 15(2), 115–134. https://doi.org/10.1207/ S15326969ECO1502_2 Turvey, M. T. (1992). Afordances and prospective control: An outline of the ontology. Ecological Psychology, 4(3), 173–187. https://doi.org/10.1207/ s15326969eco0403_3 Turvey, M. T., Shaw, R. E., Reed, E. S., & Mace, W. M. (1981). Ecological laws of perceiving and acting: In reply to Fodor and Pylyshyn (1981). Cognition, 9(3), 237–304. Weiss, G., Salamon, G., & Murphy, A. V. (2019). 50 Concepts for a Critical Phenomenology. Northwestern University Press. Wing, A. K. (1990). Brief refections toward a multiplicative theory and praxis of being. Berkeley Women’s LJ, 6, 181–201. Withagen, R., Araújo, D., & de Poel, H. J. (2017). Inviting afordances and agency. New Ideas in Psychology, 45, 11–18. https://doi.org/10.1016/j. newideapsych.2016.12.002 Withagen, R., & Costall, A. (2021). What does the concept of afordances aford? Adaptive Behavior, 1059712320982683. https://doi. org/10.1177/1059712320982683 Young, I. M. (1980). Throwing like a girl: A phenomenology of feminine body comportment motility and spatiality. Human Studies, 3(1), 137–156. https:// doi.org/10.1007/BF02331805

2 THE SOCIOMATERIAL THEORY OF AFFORDANCES Julien D. Kiverstein

The concept of afordance is a foundational concept for a post-cognitivist psychology that seeks to explain behavior in relation to the environment animals perceive and act in, not in terms of internal structures considered in isolation from this environment. Paraphrasing Mace, a post-cognitivist psychology invites us to explain behavior not in terms of what is inside of the animal’s head but by reference to what the animal’s head is inside of (Mace, 1977).1 I will explore the questions raised by the editors of this volume by asking about the place of social life in a theory of afordances. By a “theory of afordances,” I mean a theory in metaphysics that is concerned with accounting for what we mean when we say that afordances are real. By “social life” I mean what Wittgenstein (1953) called “forms of life,” the regular, relatively stable, and normatively regulated activities that take place between humans (and many non-human animals) within the groups and communities they form. What is distinctive about social forms of life as I will understand them is that the normativity that regulates the activities within these groups and communities is socially enforced. The activities of social animals are regulated, and not only regular. This means that social animals in regulating their activities are responsive to whether their actions conform with (or deviate from) the standards of correctness, appropriateness, or goodness defned within their group or community. My focus in this chapter will be on the afordances of an environment populated by humans. My argument however generalizes to other social forms of life. My aim is to make a plea for understanding the material afordances of the human environment as growing out of normatively regulated patterns of social interactions. The afordances available to social DOI: 10.4324/9781003396536-4

The Sociomaterial Theory of Afordances

27

forms of life have a reality that is, I will argue, both social and material. Humans inhabit environments that have been shaped by long histories of activity, and not always for the better if we consider the ecological devastation humans have wreaked. Human infants and children grow up experiencing afordances as members of communities within which they are introduced to afordances that have specifc (i.e., normatively specifed) meanings. Think for instance of what Costall (1995, 2012) has called “canonical afordances” such as the “sitability” of chairs. Our social peers play a prominent role in constructing, defning, and explaining to us as we develop the afordances of things. Infants are immersed in and surrounded by social life from the moment they are born. They never experience an environment of non-social afordances. What Do We Understand by the Term “Afordance”?

Gibson introduced the term “afordance” to refer to possibilities for action the environment furnishes to the creatures that live in it. In a much-cited passage, he adds that his concept of afordance “cuts across the dichotomy of subjective-objective and helps us to understand its inadequacy. It is equally a fact of the environment and a fact of behavior. It is both physical and psychical, yet neither” (Gibson, 1979, p.  129). What Gibson seems to be trying to articulate here is the metaphysical status of afordances as relations. Afordances are “equally facts of the environment” and “of behavior” because they have the metaphysical status of relations, where a relation is such that “no one constituent of an inquiry can be adequately specifed as fact apart from the specifcation of other constituents of the full subject-matter” (Dewey & Bentley, 1960, p.  122). On this reading, afordances cannot be specifed apart from a larger organism-environment system, which is the subject-matter of an ecological psychology. The afordances of the human environment belong to what Costall (1995) has described as a “humanized nature,” a natural world that has become social and cultural through human activity. The natural world humans grow up in does not permit the making of a distinction between what is natural, on the one hand, and what is artifcial, cultural, or humanmade, on the other. In earlier work with Erik Rietveld, I have argued for an understanding of afordances as relations between sociomaterial aspects of the environment and abilities available in forms of life (Rietveld & Kiverstein, 2014). We use the concept of “sociomateriality,” introduced to us through the work of our colleague Annemarie Mol, to foreground how material reality takes shape and is given form in everyday situations of social interaction. In the social setting of a scientifc laboratory, for instance, material reality can come to take the shape of “vaccinations,

28

Julien D. Kiverstein

microchips, valves, combustion engines, telephones, genetically manipulated mice and other objects .  .  . that carry new realities” (Mol, 1999, p. 75). We have proposed that what is true of materials in a scientifc laboratory is true more generally of afordances in the social setting of human forms of life (see also Heft, 2007). Afordances do not have a ready-made, pre-given reality prior to human activity but come into being within social life. Indeed, such a pragmatic, activity-dependent understanding of afordances generalizes to other non-social forms of life, though I will not make such an argument here. What Role Does Afordance Play in Perception? Is It the Entity Organisms Perceive or the Means Through Which Organisms Perceive? Are Afordances the Only Perceptual Dependent Variables?

The theory I will develop in this chapter, which I will call the “sociomaterial theory,” claims that afordances have a sociomaterial reality. The sociomaterial theory claims that when we experience an afordance, what we perceive is a relation, not an entity.2 The more standard view among ecological psychologists claims that afordances are dispositional properties of physical objects (Heras-Escribano, 2019; Turvey, 1992, 2018; Turvey et al., 1981) complemented by efectivities, understood as objectively measurable properties of animals (see, e.g., Turvey et al., 1992, 1981). Dispositional theories would hold that afordances, as dispositional properties of physical objects, are properties of the entities organisms perceive. The sociomaterial theory comes closer to claiming that afordances are the means through which organisms perceive. Dispositional theories take afordances to have a pre-given, determinate existence in physical objects that awaits our discovery before we act on them. Afordances are contained in physical objects in the way that the solubility of salt in water is contained in salt. Interestingly, this view of afordances as having a fxed and determinate reality prior to action is one that is also, at least implicitly, defended by some proponents of a relational theory of afordances. The constituents of the relation that Chemero (2009, 2003) appeals to in his analysis of what afordances are have a determinate existence at each point in time. On the environment side of the relation are what Chemero calls “features” of whole situations tied to particular times and places, which he is careful to distinguish from properties of objects (see Chemero, 2009). On the animal side of the relation are abilities of individuals, such as the ability to climb stairs. Chemero compares the notion of relation needed for understanding the reality of afordances with the logical relation “taller-than” that might hold between two

The Sociomaterial Theory of Afordances

29

individuals—John and Mary. To say Mary is taller than John is to ascribe a relation whose existence depends on both of them. As van Dijk (2021a) has recently noted, this logical understanding of the relation assumes that each relata has a determinate existence at any given point of time. Given this logical understanding of relations, afordances have an existence that pre-exists the activities of individuals. Afordances are found, not made. The sociomaterial theory, which I will develop in this chapter, holds that afordances are made and not found. The incompatibility of relational and dispositional theories of afordances has recently been challenged by Baggs and Chemero, who have proposed a possible reconciliation of relational and dispositional theories of afordances (Baggs & Chemero, 2021, 2019; Chemero, 2022; Chemero & Turvey, 2007; Wilkinson & Chemero, this volume).3 When an ecological psychologist designs an experiment that aims to probe the behavioral responses of the average or typical member of a species, they will be targeting afordances as belonging to the habitat. Baggs and Chemero (2019, 2021) claim that the afordances of a habitat are best conceived of along the lines laid out in mainstream ecological psychology, as dispositional properties of physical objects. In addition, they argue that ecological psychologists should also target the umwelt of an individual in their experiments. Thus, Chemero (2022) has recently observed, “[I]n real-time engagement with an umwelt, what is aforded is constantly changing; so too are our action capabilities” (p. 46; cf. Chemero, 2009, on afordances 2.0). To account for how individuals are able to successfully adapt their actions to ft the demands of a changing environment, Baggs and Chemero (2021, 2019) suggest a relational understanding of afordances may be what is needed. Baggs and Chemero (2021, 2019) argue that dispositional and relational theories of afordances can be seen to be consistent once one is careful to make a conceptual distinction between the habitat and umwelt. It is relative to our explanatory interests in conducting experiments in ecological psychology whether we talk about afordances as dispositions (as belonging to the habitat) or afordances as relations (as belonging to the umwelt). Both can perform important explanatory work depending on our purposes as scientists.4 I agree with Baggs and Chemero that it is important to distinguish afordances as experienced by an individual animal from the set of afordances that are available to any animal that can perceive and act on them. In our previous work, we have captured this diference by making a distinction between the landscape of afordances of a form of life and the feld of inviting afordances for an individual (Rietveld & Kiverstein, 2014; Bruineberg & Rietveld, 2014). I disagree however with Baggs and Chemero’s characterization of the habitat as a part of a world of physical objects that

30 Julien D. Kiverstein

becomes meaningful when considered from the point of a view of an ideal member of a species. Such a view of the habitat raises the following question: how could the point of view of a typical member of species confer meaning on what would otherwise be a meaningless physical world? First, who or what counts as a typical member of a species? Baggs and Chemero seem to have in mind here the typical participant in a laboratory experiment who can be taken to be representative of the average member of a species. However, the average or typical member of a species is a double abstraction: it involves frst a statistical averaging across members of a species to identify a typical member of a group. In addition, the notion of species is a second layer of abstraction, involving the taxonomizing of organisms based on shared phenotypical characteristics. These abstractions arguably fail to capture the diferences in forms of life that are the basis for the meaning of the afordances agents deal with in everyday life situations. There are diferences in social groups that are missed so long as one operates at the level of analysis of species and their average members. The dispositional theory of afordances Baggs and Chemero (2021, 2019) want to leave room for within ecological psychology runs the risk of “subjectivizing” social life (Costall, 1995), excising or excluding it from what the ecological psychologist investigates when they make use of the concept of afordance. If the experiments of ecological psychologist are concerned only with the dispositions of physical objects, they will abstract away from anything to do with social life. However, once social life has been excluded from our understanding of the habitat, as a dispositional understanding of the habitat would seem to require, the question arises of how to bring the social back into ecological psychology. Why Is Afordance Any Better Than Stimulus? What Does a Theory of Afordance Suggest That Stimuli Cannot; How Has It Moved the Needle Past Gibson 1960’s Recognition of How Little We Can Defne Stimuli?

Afordances, unlike physical stimuli, are meaningful for the animals that can perceive them. Those perceptual psychologists who understand perception as the outcome of the processing of meaningless physical stimuli run into the difcult question of how an animal ever comes to perceive a meaningful environment. A similar question arises for metaphysical theories that understand the reality of afordances in dispositional terms as properties of physical objects. The notion of meaning that is needed to characterize afordances is, I would contend, a normative notion. To explain how physical objects or stimuli become meaningful requires making reference to the diferent forms of life from which the norms and values in question

The Sociomaterial Theory of Afordances

31

have their origins. If one begins from development in an environment that is simultaneously natural, material, social, cultural, and historical, as the sociomaterial theory proposes to do, then the question of how afordances become meaningful admits of an answer that is otherwise unavailable.5 The sociomaterial theory of afordances holds that the separation of the social from the natural world doesn’t make sense from a developmental perspective. Infants do not frst encounter a natural world that only later becomes social. They are from the moment they are born, and perhaps already when they are growing in the womb, immersed in a meaningful world because of the social life going on around them. Briefy returning to Bagg and Chemero’s (2019, 2021) proposal, reviewed in the previous section, I suggest that we think of the habitat for humans not as a part of the physical world but as a landscape of afordances that has formed historically out of social and cultural interactions (Figure 2.1).

FIGURE 2.1

The feld of inviting afordances and the landscape of afordances stand in a reciprocal or co-determining relation.

32 Julien D. Kiverstein

The sociomaterial theory of afordances takes its lead from the last part of the famous quote from above in which Gibson tells us an afordance is “both physical, and psychical, yet neither.” Here Gibson seems to be alluding to William James’ notion of pure experience in radical empiricism, which James introduced to describe the fundamentally nondual way subjects experience the surrounding world in the fux of life.6 It is our concepts that enter into experience with linguistic practices, that divide and classify those experiences into discrete aspects of subjects and objects. Such dualities refect a stance removed from the dynamic fow and the felt relations that characterize an otherwise undiferentiated stream of experience. The relation of reciprocity between the feld and landscape is illustrated by the arrows forming a lemniscate (Kiverstein et  al., 2021; van Dijk, 2021b). The lemniscate can be viewed backward starting from the landscape of afordances. What one observes when one zooms out on the landscape are afordances as material structures that have formed through history of past activity within social forms of life. The fgure can also be traced by looking forward, starting from a feld of inviting afordances that contribute to continuing, maintaining, and further developing the afordances available in the landscape. James’ notion of pure experience can proftably be understood by contrast to an intellectualist view of experience as a mode of intellectually knowing a value-neutral, physical world. The intellectualist, in common with classical empiricism, conceives of all knowing as having its origin in sensory perception, which is caused by a physical stimulus empiricists called “sense impressions.” I am proposing to follow William James in thinking of perception as bound to an ongoing fow of world-directed activity in which the “body is the storm-centre, the origin of coordinates, the constant place of stress in all that experience train” (James, 1912, p. 86, quoted by Heft, 2001, p. 55). In this fow of activity, an animal smoothly adapts its activities to coordinate with a world understood in the practical terms of its habits and skills. What Is the Connection Between Afordances, Behavioral Scale, and Intention? How Distant or Reluctant Can a Perceiving-Acting System Be Before Afordances Come Into Play? Do Afordances Exert Their Role Only When the System Is Close or Willing Enough?

In my previous work with Erik Rietveld, we proposed to conceptualize afordances as forming a landscape within the niche of particular forms of life (Rietveld & Kiverstein, 2014).7 The notion of “forms of life” is an anthropological concept that can be concretely investigated from diferent

The Sociomaterial Theory of Afordances

33

perspectives. As van Dijk and Rietveld (van Dijk & Rietveld, 2017) note, we can for instance zoom out on the landscape and observe and describe the regular and stable patterns of practice and the persisting afordances that have been set up in the past by the participants in those practices. We can zoom in further within a specifc form of life or practice and take up the perspective of an observer of an agent as they engage with the landscape of afordances in a particular local behavioral setting (Barker, 1968; Heft, 2001). The example van Dijk and Rietveld (2017) discuss is of a person drinking cofee with friends in a café. What can be observed from this zoomed-in perspective is that the relatively persisting afordances in the landscape are actually dynamic and in fux. On the one hand, we can observe the afordances the café ofers such as the friendly waiting staf that create an inviting, warm atmosphere; the tables and chairs that aford sitting and chatting with friends; and the cofees that have been ordered that aford drinking. These afordances both enable certain actions but also constrain how those actions unfold. As the interaction between the friends unfolds so the landscape is changed. The friends may fnish drinking their cofees, for instance, and the waiters clear their mugs away from their table. Their socializing may begin to draw to a close as the friends recall an impending appointment on the other side of town and begin to coordinate with the distal afordance of catching a train. The local setting is undergoing constant change and how the person acts within this setting will require them to adapt to this change. The actions that can be observed crucially have an indeterminacy to them (van Dijk & Rietveld, 2017)—they can unfold in many diferent directions. As the people act, some of these possibilities are taken up and realized; others are neglected or closed-of such as the possibility to catch a tram that departs just as one arrives at its stop. Afordances belong to what Shotter (1983) described as “a world-inthe-making rather than a world-already-made” whose structure is waiting to be found. Afordances are possibilities in the sense that they have what van Dijk (2021a) has described as an “indeterminate” existence. As possibilities, they only take on a determinate, fully formed character once an action has been accomplished. As I sit in the café talking with my friends, the possibility to catch the train to arrive punctually to my appointment at 4 PM is an indeterminate and uncertain possibility (van Dijk & Rietveld, 2017). As I act to realize this possibility, the series of actions I take is not pre-determined in advance. What can be observed are a number of possibilities that become increasingly determinate in action. The metaphysical status of afordances as possibilities (and this is a general claim that applies to the afordances for non-social forms of life also) means that they do not fully exist prior to our activities but form, develop, and grow in practical activities that unfold over time.

34 Julien D. Kiverstein

We can also zoom in further and take up the frst-person perspective of an individual as they selectively engage with a feld of multiple inviting afordances (Bruineberg & Rietveld, 2014; Rietveld & Kiverstein, 2014).8 I suggest understanding the term “intention,” appealed to in the questions for this section, simply as a synonym for the selection of inviting afordances. Such an understanding of intention as the selection of afordances can be found in the writings of the ecological psychologist Edward Reed (1993, 1996). Reed rejected a conception of intentions as mental states that are the causes of actions. This is arguably the standard way of thinking about intention in cognitive science (see, e.g., Pacherie, 2008). Reed by contrast conceived of intentions as abilities “to select specifc afordances for the observer to become aware of and to use” (Reed, 1993, p. 65). Intentions are abilities for organizing cycles of perception and action in such a way that the subject becomes aware of, and/or makes use of only relevant afordances. Reed explains these abilities as developing in human children in social and cultural contexts in interaction with caregivers that scafold the child’s attention. Reed described this process in terms of a “feld of promoted action” that constrains which afordances the child attends to and utilizes by encouraging certain actions and discouraging others. The child in selecting among available afordances becomes sensitive to better or worse, correct or incorrect, appropriate or inappropriate patterns of activity within the social groups to which they belong. There is much that is worthy of further discussion in Reed’s proposal. I will focus here on Reed’s observation that the ability to select afordances is cultivated on the basis of sensitivity to what activities are considered normal and expected within a social group, such as one’s family or the cultural groups to which one belongs. When a person acts skillfully, they are typically sensitive to, which is to say they care about, whether their actions are adequate or inadequate, good or bad, efective or inefective. The standards of adequacy come in part from the social groups to which one belongs. It is this socially tuned concern with adequacy, appropriateness, efectiveness, and so on that we will take to provide an important clue to why certain afordances stand out as relevant. I borrow the term “concern” from the emotion psychologist Nico Frijda. He took an organism’s concerns to refer to “what gives events their emotional meaning” (Frijda, 2007, p. 123). Different emotions such as anger, sadness, fear, or surprise have in common that they safeguard a person’s concerns. The afordances that stand out as relevant are those that satisfy one’s concern as a skilled agent to perform adequately, if not excellently, in a given setting. Inviting afordances have afective allure—they are inviting, or soliciting, to experts (Rietveld, 2008; Toner et al., 2015; Withagen, 2022; Withagen et al., 2012) because of the concern of experts to perform at their best. What we learn from Reed is

The Sociomaterial Theory of Afordances

35

that the concern to perform well is cultivated, developed, and calibrated in interaction with other expert practitioners. Correct, adequate, and excellent are evaluations experts bring to bear on their performance based on what would satisfy other experts acting in similar situations. How Do Systems Engage With Afordances as They Move Among Tasks and Intentions?

In the ecological dynamics literature, inviting afordances have been identifed with afordances that inform the agent how to achieve their task goals. Araújo et al. (2019) write for instance: “For achieving task goals, some afordances are clear invitations, informing the perceiver how to proceed” (p. 562). Seifert et al. (2013) suggest in a similar fashion thus: “The increasing attunement of a performer to afordances can be understood by examining how an individual’s intentions converge with that of the task” (p. 171). Araújo et al. (2019) give as an example attackers in basketball who intend to dribble the ball around defenders in such a way as to get a chance to shoot a basket. It is the agent’s intentions they claim that specify which afordances are relevant to achieving these tasks’ goals. The concept of “intention” is here introduced to explain how agents determine which afordances to attend and which to ignore. Afordances are ignored (i.e., do not invite behavior) when they do not contribute anything to achieving task goals. I fnd such an appeal to intention to account for the selection of relevant afordances unhelpful. The landscape of afordances is exceptionally rich in terms of the possible actions it ofers. Yet individuals are typically ready to respond selectively to afordances in ways that are appropriate to the context, and that refect their own states of readiness and bodily preparedness that arise in relation to their situation. How is it possible that individuals are only responsive to relevant afordances in each particular situation? The very same afordances can stand out as relevant to an individual in one situation, while in a diferent situation the same afordance does not move them at all. What accounts for this diference? I do not think appealing to tasks, goals, and intentions helps to answer this question (van Dijk & Rietveld, 2017). It just invites the question: why does the agent have the goals and intentions that they do, or how does the individual perceive that they are in one task domain and not another? To appeal to goals, intentions, and tasks is just to postpone the task of explaining the selection of relevant afordances in a particular context. When an individual acts on an inviting afordance, they do so based upon a sensitivity to the norms of the forms of life to which they belong. They thus contribute, albeit in a small way usually, to sustaining and

36 Julien D. Kiverstein

maintaining the availability of the afordances that are set up within those forms of life. The sociomaterial theory claims that afordances are processes that form in unfolding activities. As we saw in the previous section, what one is invited to do in a particular setting is never fully determined in advance, since activities can always be continued in multiple ways. What activity is performed, for example, the conversation I have with my friends while drinking cofee in the cafe, is only fully determined once the activity is completed. The individual actions that a situation invites are what Shotter (1983) described as “formative causes.” Shotter ofers the useful analogy of the development of an oak tree from an acorn to explain what he means by formative causation. The oak tree is not an expression of something—for example, the set of instructions encoded in a genome—internal to the acorn, already fully specifed in advance. Similarly, afordances are not expressions of fully specifed possibilities contained in advance as dispositions in physical objects, or as pre-existing relata. The developmental transition that takes place as the acorn slowly transforms into a tree is a temporal, unfolding process of growth or production. It is an ordered transition from an indeterminate to a determinate form through an interaction of the acorn with the earth and its surroundings. Formative causation refers to this unfolding, inner activity of growth through which the tree is formed. The process of growth is unpredictable, responsive to the local contingencies of the earth in which the acorn is planted. Looking backward upon an action that has unfolded in the past, one can observe an afordance as a relatively persistent and stable structure that has been specifed and determined. What we can observe is a material structure that has taken on a particular determinate meaning as ofering particular actions because of the regular norm-governed patterns of social interaction in which this afordance has formed over time. The afordance, as a specifc possibility, has grown and further developed out of this past pattern of social interaction much like the oak growing from the acorn in response to the local contingencies of the soil in which it is planted. The form of life has set up possibilities in the form of material structures that can be further continued by other participants as they act in agreement with the norms of the social groups to which they belong. The history of past practice is continued by individuals as they respond to inviting relevant afordances. To borrow a helpful expression from van Dijk and Rietveld (2021), past patterns of practice “pave the way” for the possible responses of individuals going forward. What individuals have done in the past however doesn’t fully determine how an individual will respond to their present circumstances. Before the person acts, the action that is

The Sociomaterial Theory of Afordances

37

aforded to them by their present situation remains in part uncertain and open. How they respond will depend on local contingencies that allow for growth of the afordance in new and sometimes surprising directions much as is the case with the oak tree. Afordances are dynamic stabilities that form in fows of activity as people act in ways regulated by practices. The landscape of afordances is made up of material structures that have been given form through past patterns of social interaction. The afordances that make up the landscape set up conditions for the continuation, growth, and further development of practices. Looking forward on the feld of relevant afordances for an individual, one sees invitations to continue a productive or formative process. What future actions are invited is indeterminate and uncertain, in the process of being determined. Insofar as social practices are the formative causes of afordances, these future actions contribute to continuing, maintaining, and further developing the afordances available in the landscape. For this reason afordances have a precarious existence (van Dijk, 2021a, 2021b). Think for instance of the soil we depend upon for growing food. If our agricultural practices continue to lead to soil erosion, this soil may cease to aford the growing of food. Concluding Remarks

In this chapter, I have developed an argument for sociomaterial theory of afordances. The sociomaterial theory claims that afordances have an existence that is diachronically constituted in norm-governed, social forms of life. The sociomaterial theory is aligned with relational theories that have their historical roots in James’ concept of pure experience, which described a mode of experience prior to any dividing up and ontological separation of reality into subjects and objects. I have argued that the afordances available to animals that belong to social forms of life do not have a pre-given and determinate existence prior to what social animals do when they act according to the norms of the forms of life they take part in. I argued for this theoretical perspective on afordances indirectly by criticizing Baggs and Chemero’s (2021, 2019) proposal to understand afordances in an ecumenical way that makes room for both relational and dispositional accounts of what afordances are. I have argued that views that take afordances to be physical properties will struggle to account for the role of social life in the formation of afordances. The sociomaterial theory by contrast thinks of afordances as being formed, maintained, and developed in social life. Afordances are made not found; they stand in a relation of formative causation to social life.

38 Julien D. Kiverstein

Acknowledgments

I am grateful to Ludger van Dijk for helpful comments on an earlier draft of this chapter. I would also like to thank the audience at a wonderful UNAM workshop on social cognition where this chapter began to take shape. My thanks to Melina Gastelum, Laura Mojica, Sergio MartínezMunoz, Bruno Lara, Ale Ciria, Lenny Moss, Anthony Chemero, Vincente Raja, and Leana Budico for helpful discussion. This research was funded by a Vici grant from the Dutch Scientifc Organisation (awarded to Erik Rietveld). Notes 1. This proposition for how to move beyond cognitivist psychology might seem to some like a step backward to a pre-cognitivist, behaviorist psychology. However, the concept of afordance is arguably alien to much of what passed for behaviorist psychology, which tended to conceive of observable behavior as paired with a meaningless stimulus. Afordances, by contrast with stimuli, are understood by animals in pragmatic terms of the practical diference they make to their lives. 2. They claim that relations are perceived can be traced back historically to the radical empiricist philosophy of William James, the key claim of which was that in addition to particulars people also experience all manner of relations. 3. van Dijk (2021b) notes that Reed (1996) also made a distinction between afordances as changing relations between organisms and environments, and as persisting resources or dispositions of physical objects that pre-exist their perception by any individual animal (see Reed, 1996, p. 26). I focus on the more recent treatment of this distinction in Baggs and Chemero as it explicitly aims at reconciling the dispositional and relational theories of afordances. My aim is to show that these two theories are in competition because they are ontologically incompatible. 4. Their proposal has a second attractive feature that should be briefy mentioned: it allows for a unifcation of post-cognitivist approaches into a single metatheory that combines ecological psychology and the enactive approach to cognitive science (Di Paolo et al., 2017; Rosch et al., 1991; Thompson, 2007). They write, “If you are an ecological psychologist, you start from the objective, thirdperson point of view with the physical world. . . . The enactivist begins with the subjective, frst-person point of view with the umwelt enacted by an individual person” (Baggs & Chemero, 2021, p. 14). Earlier they explain how Turvey and colleagues have used the word “environment” to mean the habitat of a generic member of a species and most ecological psychologists have followed them in doing so. Enactive cognitive science, by contrast, approaches the animal-environment relation from the side of an animal and its internal organization. They seek to explain how, based on its internal organization, the animal makes sense of, and brings forth its umwelt. 5. Interestingly when Baggs and Chemero (2019) discuss the meaning of afordances, they seem to give a priority to an individual’s umwelt (pp. 13–14). It is the umwelt they write that is “given in experience” based on the individual’s “biological make-up, its development, its history of learning.” The individual’s development and its history of learning, if we are dealing with human

The Sociomaterial Theory of Afordances

39

infants and children, is typically social and cultural. Thus, Baggs, Chemero, and myself would seem to be in agreement that the meaning of afordances is social and cultural in origin. The point of view of the average member of a species is not something that can be made sense of apart from the social forms of life of this individual. 6. James’ notion of pure experience is often interpreted as a metaphysical thesis concerned with the intrinsic nature of material reality. James is sometimes read as claiming that the material of which everything is composed is neither fundamentally physical nor mental but is made from one kind of stuf that is neutral between these two metaphysical categories (see, e.g., Cooper, 2002; Lamberth, 1999). Pure experience has also been given a metaphysically neutral, phenomenological reading by Seigfried (1990). 7. Our concept of a landscape of afordances does some of the explanatory work the concept of habitat does for Baggs and Chemero (2019, 2021). It can account for instance for the relatively stable and persisting existence of afordances, and the sense in which the environment continues to furnish afordances even when individuals are not perceiving or acting on them. The term “form of life” however escapes the problems that Baggs and Chemero encounter in characterizing the habitat in relation to an abstract construct—the typical member of a species. 8. Our concept of feld closely overlaps with what Baggs and Chemero call the “Umwelt.” The feld is defned in relation to individuals and their lived experiences, as is the umwelt. A feld, like an umwelt as described by Baggs and Chemero (2019, 2021), is dynamic—it changes over developmental time-scales based on learning, and on shorter time-scales, based on the interests, needs, and purposes of the individual, and the experience they have of deviating from a good grip on the situation (Bruineberg & Rietveld, 2014). It is also experiencedependent, defned in relation to the lived experience of individuals.

Reference List Araújo, D., Dicks, M., & Davids, K. (2019). Selecting among afordances: A basis for channeling expertise in sport. In M. Cappuccio (Ed.), Handbook of Embodied Cognition and Sport Psychology (pp. 537–556). MIT Press. Baggs, E., & Chemero, A. (2019). The third sense of environment. In Perception as Information Detection: Refections on Gibson’s Ecological Approach to Visual Perception (pp. 5–20). Routledge. Baggs, E., & Chemero, A. (2021). Radical embodiment in two directions. Synthese, 198(9), 2175–2190. https://doi.org/10.1007/s11229-018-02020-9 Barker, R. G. (1968). Ecological Psychology. Stanford University Press. Bruineberg, J., & Rietveld, E. (2014). Self-organization, free energy minimization, and optimal grip on a feld of afordances. Frontiers in Human Neuroscience, 8, 599. https://doi.org/10.3389/fnhum.2014.00599 Chemero, A. (2003). An outline of a theory of afordances. Ecological Psychology, 15(2), 181–195. https://doi.org/10.1207/S15326969ECO1502_5 Chemero, A. (2009). Radical Embodied Cognitive Science. MIT Press. Chemero, A. (2022). A path to ecological psychology. In Z. Djebbara (Ed.), Afordances in Everyday Life: A Multidisciplinary Collection of Essays (pp. 33–40). Springer. Chemero, A., & Turvey, M. T. (2007). Gibsonian afordances for roboticists. Adaptive Behavior, 15(4), 473–480. https://doi.org/10.1177/1059712307085098

40 Julien D. Kiverstein

Cooper, W. (2002). The Unity of William James’s Thought. Vanderbilt University Press. Costall, A. (1995). Socializing afordances. Theory & Psychology, 5(4), 467–481. https://doi.org/10.1177/0959354395054001 Costall, A. (2012). Canonical afordances in context. AVANT: Pismo Awangardy Filozofczno-Naukowej, 2, 85–93. Dewey, J., & Bentley, A. F. (1960). Knowing and the Known (Vol. 111). Beacon Press. Di Paolo, E., Buhrmann, T., & Barandiaran, X. (2017). Sensorimotor Life: An Enactive Proposal. Oxford University Press. Frijda, N. H. (2007). The Laws of Emotion. Lawrence Erlbaum. Gibson, J. J. (1979). The Ecological Approach to Visual Perception. Houghton Mifin. Heft, H. (2001). Ecological Psychology in Context: James Gibson, Roger Barker, and the Legacy of William James’s Radical Empiricism. Psychology Press. Heft, H. (2007). The social constitution of perceiver-environment reciprocity. Ecological Psychology, 19(2), 85–105. https://doi.org/10.1080/10407410701331934 Heras-Escribano, M. (2019). The Philosophy of Afordances. Palgrave Macmillan. James, W. (1912). Essays in Radical Empiricism. Henry Holt. Kiverstein, J., van Dijk, L., & Rietveld, E. (2021). The feld and landscape of afordances: Kofka’s two environments revisited. Synthese, 198(9), 2279–2296. https://doi.org/10.1007/s11229-019-02123-x Lamberth, D. C. (1999). William James and the Metaphysics of Experience. Cambridge University Press. Mace, W. M. (1977). James J. Gibson’s strategy for perceiving: Ask not what’s inside your head, but what’s your head inside of. In R. Shaw & J. Bransford (Eds.), Perceiving, Acting, and Knowing: Towards an Ecological Psychology. Lawrence Erlbaum Associates, Inc. Mol, A. (1999). Ontological politics: A word and some questions. The Sociological Review, 47(1_suppl), 74–89. Pacherie, E. (2008). The phenomenology of action: A conceptual framework. Cognition, 107(1), 179–217. https://doi.org/10.1016/j.cognition.2007.09.003 Reed, E. S. (1993). The intention to use a specifc afordance: A conceptual framework for psychology. In R. H. Wozniak & K. W. Fischer (Eds.), Developing in Context: Acting and Thinking in Specifc Environments (pp. 45–76). Erlbaum. Reed, E. S. (1996). Encountering the World: Toward an Ecological Psychology. Oxford University Press. Rietveld, E. (2008). Situated normativity: The normative aspect of embodied cognition in unrefective action. Mind, 117(468), 973–1001. https://doi.org/10.1093/ mind/fzn050 Rietveld, E., & Kiverstein, J. (2014). A rich landscape of afordances. Ecological Psychology, 26(4), 325–352. https://doi.org/10.1080/10407413.2014.958035 Rosch, E., Thompson, E., & Varela, F. J. (1991). The Embodied Mind: Cognitive Science and Human Experience. MIT Press. Seifert, L., Button, C., & Davids, K. (2013). Key properties of expert movement systems in sport. Sports Medicine, 43(3), 167–178. https://doi.org/10.1007/ s40279-012-0011-z

The Sociomaterial Theory of Afordances

41

Seigfried, C. H. (1990). William James’s Radical Reconstruction of Philosophy. State University of New York Press. Shotter, J. (1983). “Duality of structure” and “intentionality” in an ecological psychology. Journal for the Theory of Social Behaviour, 13(1), 19–44. Thompson, E. (2007). Mind in Life: Biology, Phenomenology, and the Sciences of Mind. Harvard University Press. Toner, J., Montero, B. G., & Moran, A. (2015). Considering the role of cognitive control in expert performance. Phenomenology and the Cognitive Sciences, 14(4), 1127–1144. https://doi.org/10.1007/s11097-014-9407-6 Turvey, M. T. (1992). Afordances and prospective control: An outline of the ontology. Ecological Psychology, 4(3), 173–187. https://doi.org/10.1207/ s15326969eco0403_3 Turvey, M. T. (2018). Lectures on Perception: An Ecological Perspective. Routledge. Turvey, M. T., Burton, G., Pagano, C. C., Solomon, H. Y., & Runeson, S. (1992). Role of the inertia tensor in perceiving object orientation by dynamic touch. Journal of Experimental Psychology: Human Perception and Performance, 18(3), 714–727. https://doi.org/10.1037/0096-1523.18.3.714 Turvey, M. T., Shaw, R. E., Reed, E. S., & Mace, W. M. (1981). Ecological laws of perceiving and acting: In reply to Fodor and Pylyshyn (1981). Cognition, 9(3), 237–304. van Dijk, L. (2021a). Psychology in an indeterminate world. Perspectives on Psychological Science, 16(3), 577–589. https://doi.org/10.1177/1745691620958005 van Dijk, L. (2021b). Afordances in a multispecies entanglement. Ecological Psychology, 33(2), 73–89. https://doi.org/10.1080/10407413.2021.1885978 van Dijk, L., & Rietveld, E. (2017). Foregrounding sociomaterial practice in our understanding of afordances: The skilled intentionality framework. Frontiers in Psychology, 7, 1969. https://doi.org/10.3389/fpsyg.2016.01969 van Dijk, L., & Rietveld, E. (2021). Situated anticipation. Synthese, 198(1), 349– 371. https://doi.org/10.1007/s11229-018-02013-8 Withagen, R. (2022). Afective Gibsonian Psychology. Routledge. Withagen, R., de Poel, H. J., Araújo, D., & Pepping, G.-J. (2012). Afordances can invite behavior: Reconsidering the relationship between afordances and agency. New Ideas in Psychology, 30(2), 250–258. https://doi.org/10.1016/j. newideapsych.2011.12.003 Wittgenstein, L. (1953). Philosophical Investigations. Blackwell Publishing.

3 WHEN IT COMES TO AFFORDANCES, WHAT DO ANIMALS KNOW AND HOW DO THEY KNOW IT? Jefrey B. Wagman, Tyler Dufrin, and Thomas A. Stofregen

What Do We Understand by the Term Afordance?

Over the second half of his career, James Gibson developed the concept of afordance as a—or perhaps the—critical component of his Ecological approach to perception (Gibson, 1966, 1979; Reed, 1988; Wagman, 2020). Over the frst half of his career, Gibson came to suspect that (often implicit) centuries-old assumptions—about the world, what animals could know about the world, and how animals know such things—prevented a scientifc explanation of the plainly observable fact that all animals successfully perform everyday behaviors (see Blau & Wagman, 2022). Among these assumptions was that the most valid description of the world—and therefore of both what animals could know about the world and how they could know those things—was provided by the frameworks of Euclidean geometry and Newtonian mechanics. The World as a Collection of Objects and Properties

A fundamental goal of the Euclidean and Newtonian frameworks is to describe the world in objective (i.e., independent of any given animal) and abstract (i.e., independent of any given set of circumstances) terms. In this view, the world consists of (discrete and distinct) objects, each composed of abstract properties (P1, 2, 3 . . . n: e.g., length, height, diameter, mass, and location). Objects causally infuence each other by sequences of discrete and local mechanical processes (described by Newton’s laws of motion; Figure 3.1, top). DOI: 10.4324/9781003396536-5

What and How Animals Know About Afordances

FIGURE 3.1

43

The world as a collection of properties (Pn) (top). The world as a collection of niches relations between animals (An) and properties (Pn) (bottom).

Under these assumptions, knowledge about objects—say, a fallen twig on the forest foor—consists of the very same set of objective and abstract properties, and animals obtain knowledge about such properties by means of the very same sequences of discrete, local mechanical processes. Specifcally, energy of some kind—say, light—is transmitted from the object to the animal. A specifc part of the animal registers that energy—say, the retina—translates it into a particular pattern of neural activity and then transmits it to a diferent part of the animal—say, the visual cortex. This

44

Jeffrey B. Wagman, Tyler Duffrin, and Thomas A. Stoffregen

FIGURE 3.2

The separation of animal from environment, and physical from mental, necessitates a description of knowing about the world and performing a given behavior in the world as one-way processes of translation and then transformation, followed by another one-way process of transformation and translation.

neural activity is then transformed into a mental experience of the properties of that object (Figure 3.2, top). Gibson accepted that the frameworks of Euclidean geometry and Newtonian mechanics are invaluable for an abstract understanding a myriad of fundamental phenomena in mathematical and mechanical systems. However, he argued that those frameworks are inappropriate for understanding (or even describing) the successful performance of everyday behaviors, such as locomotion through a cluttered environment, manipulating objects, and interactions with other animals. By design, the frameworks of Euclidean geometry and Newtonian mechanics are independent of any given animal and any given set of circumstances. Therefore, under these frameworks, the properties of the objects of the world are also animal-independent and context-independent. Yet, Gibson argued that everyday behaviors are both relational and context-dependent. Any given behavior—say, picking up a fallen twig—is performed by a particular animal under a particular set of circumstances.

What and How Animals Know About Afordances

45

For Gibson, a description of what a given animal could potentially know about the world in terms of geometric and physical properties is wholly incommensurate with what that animal would need to know to successfully perform everyday behaviors. Moreover, a description of what an animal knows about the world in terms of a mental experience of Euclidean and Newtonian properties is wholly incommensurate with what that animal would need to do to successfully perform everyday behaviors. Put simply, successfully performing everyday behaviors requires that animals perform such behaviors with respect to the world itself and not with respect to the mental experience of the world (Reed, 1996). Consequently, Gibson concluded that the Euclidean and Newtonian frameworks create insurmountable barriers to a scientifc explanation of how any animal can know about the world such that it can successfully perform everyday behaviors (Gibson, 1979; Turvey et al., 1981). Moreover, to the extent that the evolution of species depends on the ability of individual animals to do exactly this, these centuries-old assumptions may also create insurmountable barriers to a scientifc explanation of evolution by means of natural selection (Reed, 1996; Wagman et al., 2023). To remove these barriers and replace them with a law-based explanation of the successful performance of everyday behavior, Gibson developed a new description of the world, what animals can know about the world, and how animals obtain this knowledge. His redescriptions formed the Ecological Approach to Perception and Action (Blau & Wagman, 2022; Gibson, 1979; Turvey, 2019). In what follows, we provide a brief overview of these redescriptions, followed by an expanded discussion of some of their implications for perception and action in relation to afordances. The World as a Collection of Niches

Rather than using Newtonian physics and Euclidean geometry to describe the world as existing independent of any given animal, Gibson developed Ecological physics and Ecological geometry to describe the environment as it exists for animals—that is, the environment as a setting for behavior. He described the world not as a collection of physical and geometric properties but as a collection of ecological niches (see Nonaka, 2020; Turvey, 2004, 1992). A niche is not where an animal lives but how it does so, given its relationship with surrounding substances, surfaces, and media (such as air or water). A niche is comprised exclusively of “afordances”—opportunities for behavior that emerge from the relationship between the animal and the environment (Gibson, 1979; Stofregen, 2003; Figure 3.1, bottom). Thus, by defnition, afordances—like behavior—are both relational and

46 Jefrey B. Wagman, Tyler Dufrin, and Thomas A. Stofregen

context-dependent. They are opportunities that exist for a particular animal under a particular set of circumstances. In Gibson’s Ecological approach, the world (and what can be known about it) consists of indefnitely many afordances. Moreover, an animal knows about afordances by means of a continuously unfolding process of detecting (relational and context-dependent) patterns in ambient energy arrays. Such patterns are lawfully related to the relationships between the animal and the surrounding substances and surfaces that defne afordances. In short, animals know about afordances by detecting information about afordances. The afordance concept has been highly infuential across many felds of study. Researchers have investigated perception of afordances for a variety of everyday behaviors by a variety of animal species occupying a variety of niches (see Wagman, 2020; Wagman et al., 2019). Yet, these eforts have tended to focus on a relatively limited subset of the indefnitely many afordances that exist for any given animal in any given niche. This subset consists of afordances (i) that emerge from a relationship between the animal’s physical capabilities and the environment and (ii) that are welldefned with respect to scale, scope, and duration. This subset includes afordances for reaching, grasping, stepping over, and ftting through (see Wagman, 2020; Figure 3.3, left). By contrast, researchers typically have not focused on afordances (i) that emerge from relationships between the animal’s cognitive capabilities and the environment and (ii) that are not easily defned in terms of physical scale or scope or that are characterized in terms of long duration. Examples include afordances for interacting with individuals of difering social status, completing a crossword puzzle, attending a museum exhibit, going on a date with a romantic partner, or obtaining a college degree (Figure 3.3, right). One possible reason for this focus in the literature is the possibility that (i) these two subsets of afordances represent fundamentally distinct categories of things that an animal can know about its niche and therefore (ii) there are distinct means by which an animal can know about the afordances in each category. For example, some scholars (see Bruineberg et al., 2019) have argued that afordances relating to physical capabilities—socalled “afordances for behavior” are fundamentally distinct from afordances relating to cognitive capabilities and cultural practices—so-called afordances for “higher order cognition.” Consequently, whereas an animal can know about afordances for behaviors by detecting lawful information about such afordances, an animal can know about afordances for “higher order cognition” only by detecting such information in the context of non-lawful, conventional constraints.

Research on afordances has tended to focus on afordances for behaviors such as reaching, grasping, stepping over, and ftting through (left). It has not tended to focus on afordances for behaviors such as completing a crossword puzzle, attending a museum exhibit, going on a date with a romantic partner, or obtaining a college degree (right).

What and How Animals Know About Afordances

FIGURE 3.3

47

48 Jefrey B. Wagman, Tyler Dufrin, and Thomas A. Stofregen

Modest Elaborations on the Defnition of Afordances

Our position is that afordances are a singular category of things that an animal knows (or could know) about its niche and that animals know about afordances only by detecting lawfully generated structure in ambient energy arrays—that is, by detecting information about afordances. We argue that this more general conception of afordances follows from three modest elaborations on describing afordances as opportunities for behavior that emerge from the relationship between the animal and the environment. First, afordances are opportunities—they are behaviors that an animal might or could perform but is not compelled or obligated to perform. At any given moment, for any given animal, there are indefnitely many afordances. At a particular moment, a particular object—say, a fallen twig— might aford picking up, stepping on, looking at, and gnawing on, among many others. Therefore, at any given moment, a given animal must choose which behaviors to perform as well as when and how to do so. Moreover, they must do so based on both the goals for which those behaviors are being performed and the circumstances under which those behaviors are being performed. Second, afordances are emergent. They exist neither in the animal independently of the environment (e.g., as representations) nor in the environment independently of the animal (e.g., as properties). Rather, afordances for a particular behavior emerge from a particular relationship between the animal and the environment under a particular set of circumstances. This is no diferent than how friction (of a particular magnitude) emerges from a particular relationship between two surfaces under a particular set of circumstances or how H2O (in a given state of matter) emerges from a particular relationship between H and O under a particular set of circumstances. A given afordance cannot be separated from the relationships from which or circumstances under which it emerges. Third, afordances almost always emerge from a multifaceted relationship between the animal and the environment. For a fallen twig to aford picking up by a given animal—for example—the twig must be sufciently nearby, small, and detached in relation to the animal’s ability to extend, enclose, and lift with a given efector (Figure 3.4, left). These three modest elaborations establish that afordances emerge from the multifaceted relationship between the animal and the environment under a particular set of circumstances. Consequently, a given afordance is qualitatively diferent than—is of higher order than—the animal or environment components from which it emerges, yet has the same ontological status as those components. Such higher-order relations are common in the

What and How Animals Know About Afordances

FIGURE 3.4

49

For an object to aford picking up by a given animal, that object must be sufciently nearby, narrow, and detached in relation to the animal’s ability to extend, enclose, and pull with a given efector (left). The elaborated defnition of afordances is that they are opportunities for behavior that emerge from the multifaceted relationship between the animal and the environment under a particular set of circumstances (right). [photo courtesy of Mathias Appel, CC0, via Wikimedia Commons].

natural world. For example, friction is of higher order than the surfaces from which it emerges, and H2O is of higher order than the hydrogen or oxygen molecules from which it emerges. We now make an analogous—but yet novel—claim that afordances also emerge from the multifaceted relationship between afordances under a particular set of circumstances. These afordances are qualitatively different than—are of higher order than—the afordances from which they emerge, which are of a lower order by comparison. Yet, lower- and higherorder afordance have the same ontological status. We explore the implications of this claim in the answers to subsequent questions. What Role Does Afordance Play in Perception? Is It the Entity Organisms Perceive or the Means Through Which Organisms Perceive? Are Afordances the Only Perceptual Dependent Variables?

Gibson was equally suspicious of other centuries-old assumptions that provided the ontological foundation for the Newtonian and Euclidean approaches described in answer to the previous question.

50 Jefrey B. Wagman, Tyler Dufrin, and Thomas A. Stofregen

The frst of these assumptions is that animals are logically and fundamentally separable from the world in which they exist. The second assumption follows from the frst—the Cartesian view that the world inside animals (so-called mental experience) is fundamentally and logically separable from the world outside animals (so-called physical reality). Together, these assumptions not only justify a description of the world as a collection of animal-independent properties but also necessitate a description of how it is that animals know the world as a one-way process of translation (of energy into neural activity) followed by transformation (of neural activity into mental experience). Analogously, these assumptions also necessitate a description of the execution of a given behavior as a one-way process of transformation (of mental experience into neural activity) followed by a translation (of neural activity into a series of muscular contractions; Figure 3.2). Gibson was convinced that rather than providing a useful backdrop for the investigation of scientifcally tractable problems, these and other centuries-old assumptions only served to generate hopelessly unsolvable mysteries such as: What are the local mechanisms by which abstract properties of the world are encoded into energy as transmissible data, and by which such energy is transformed (encoded again) into neural activity? And what are the intelligent, local mechanisms by which such neural activity (representations) is translated (i.e., decoded) into a subjective, mental experience of those properties? Replacing such mysteries with scientifcally tractable problems requires new descriptions of the world, what animals can know about the world, and how animals know such things. These new descriptions must be formulated at the ecological scale—the scale that defnes the relationship between the animal and the environment (Blau & Wagman, 2022; Gibson, 1979; Turvey, 2019). There are two key reasons for this. First, it is only at the ecological scale that everyday behaviors—such as picking up, stepping on, looking at, or gnawing on a fallen twig—are performed. These, and all other behaviors, require particular multifaceted relationships between the animal and the substances and surfaces that comprise its niche. Consequently, it is only at this scale that opportunities to perform everyday behaviors—afordances for picking up, stepping on, looking at, and gnawing on a fallen twig—emerge. Second, it is also only at this scale that the lawfully generated structure in ambient energy arrays that provides information about afordances emerges. The substances and surfaces that surround a given animal structure the ambient energy arrays that likewise surround that animal. This structuring generates unique patterns of energy at every location that an animal could be relative to those substances and surfaces. The patterns in

What and How Animals Know About Afordances

FIGURE 3.5

51

Describing the world, and what there is to be known about it, at the ecological scale allows for a description of knowing as the detection of lawfully generated information.

ambient energy arrays that an animal encounters as it performs everyday behaviors are lawfully determined by its relationships to those substances and surfaces.1 These relationships include the animal’s size, shape, and mass as well as how and where it moves relative to those substances and surfaces. Therefore, these patterns are informative about the indefnitely large set of opportunities for behavior that emerge from such relationships. This means that afordances can be perceived directly—rather than conceived indirectly—by detecting lawfully generated structure in ambient energy arrays that provide information about afordances (Figure 3.5). This has important implications for what animals can know about the world and how they can know those things. We discuss two such implications next. Animals Know About Afordances, Not About Properties

Patterns in ambient energy arrays that an animal encounters as it performs everyday behaviors are lawfully structured by its relationship to the surrounding environment,2 not by properties of either the animal or environment in isolation (such as size, shape, distance, angle, mass, or color). This means that animals can (and do) directly perceive afordances but cannot (and do not) directly perceive Euclidean or Newtonian properties. This, in turn, means that animals ought to primarily (if not exclusively) know about afordances, not properties—a hypothesis that has been investigated and supported by diverse bodies of empirical research. For example, research investigating what (and how) people know about hand-held objects has consistently shown how heavy an object feels is ambiguously related to its mass. Under some circumstances, two objects of the same mass can feel unequally heavy; under other circumstances, two objects of unequal mass can feel equally heavy. Researchers have

52

Jefrey B. Wagman, Tyler Dufrin, and Thomas A. Stofregen

traditionally attempted to understand how a property (mass) is translated and transformed into a mental experience (heaviness). However, over the past few decades, researchers have discovered that perception of heaviness is better understood as perception of moveableness (afordances for being moved). The afordances of a hand-held object depend on how it resists being moved in diferent directions. Accordingly, studies have shown how heavy an object feels is lawfully related to the forces required to move that object (Amazeen & Turvey, 1996; Shockley et al., 2001). Moreover, two objects of the same mass feel unequally heavy (and two objects of diferent masses will feel unequally heavy) to the extent that those objects are unequally (or equally) difcult to move. Rather than indirectly perceiving object heaviness, people perceive afordances for moving that object (Shockley et al., 2004; Streit et al., 2007; Wagman, 2015). As another example, research investigating what infants know about ground surfaces (and how they know those things) has consistently shown that while older infants tend to avoid the “deep side” of a visual clif, younger infants do not (Gibson & Walk, 1960). Researchers have traditionally attempted to understand such diferences as developmental changes in the ability to perceive depth. However, over the past few decades, researchers have discovered that such diferences are better understood in terms of developmental changes in the ability to perceive afordances (see Adolph, 2019; Adolph & Hoch, 2019). Afordances for crawling down from one surface to another depend on a very particular multifaceted relationship between the infant and those surfaces. Learning about such afordances and the lawfully generated structure in ambient energy arrays that provides information about such afordances takes weeks of crawling experience. Accordingly, studies have shown that infants who have become skilled in perceiving afordances for crawling down from one surface to another are no better at perceiving afordances for stepping down from one surface to another (see Adolph et al., 2014). Rather than learning to perceive isolated properties, infants are learning to perceive and act on afordances. Animals Know About Afordances, as Such

A given afordance emerges from a particular multifaceted relationship between the animal and the environment. The afordance is qualitatively diferent than—of higher order than—the properties of the animal or environment from which it emerges. Moreover, a given afordance is perceived directly—by detecting lawfully structured information about that afordance. This means that a given afordance ought to be perceived as such—as an emergent, higher-order “complex particular”—rather than a

What and How Animals Know About Afordances

53

combination of discrete lower-order properties of the animal or environment. This hypothesis has been investigated and supported by multiple bodies of empirical research (see Thomas et al., 2020). One such body of research has shown that perceived afordances for a given behavior are not reducible to an additive combination of perceived properties of the animal and the environment that are constituents of that afordance. For example, perceived afordances for overhead reaching are not reducible to an additive combination of perceived arm length plus perceived shoulder height (Wagman & Stofregen, 2020). Similarly, perceived afordances for overhead reaching with a tool are not reducible to an additive combination of perceived arm-plus-tool-length and perceived shoulder height (see Thomas & Riley, 2014). A related body of research has shown that improvements in the ability to perceive a given afordance are dissociable from improvements in the ability to perceive properties of the animal and the environment that are constituents of that afordance. For example, improvements in the ability to perceive afordances for sitting while wearing blocks on the feet are independent of improvements in the ability to perceive the height of the blocks (Mark, 1987; Mark et  al., 1990). Likewise, improvements in the ability to perceive afordances for passing through an opening are independent of improvements in the ability to perceive the width of the opening and the width of the body (Higuchi et al., 2011; Yasuda et al., 2014). Why Is Afordance Any Better Than Stimulus? What Does a Theory of Afordance Suggest That Stimuli Cannot; How Has It Moved the Needle Past Gibson 1960’s Recognition of How Little We Can Defne Stimuli?

Gibson (1960) pointed out that the concept of stimulus has a long history in psychology—especially in attempts to understand the successful performance of everyday behaviors. Yet, like many other concepts rooted in the Euclidean and Newtonian assumptions about the world, what animals potentially know about the world, and how animals know such things, the classical concept of the stimulus is inappropriate for a scientifc explanation of everyday behavior. The concept of the stimulus seems to be an attempt to rectify an incompatibility between two diferent centuries-old assumptions. In Newtonian physics, objects infuence each other only by discrete and local mechanical processes. At the same time, in Cartesian metaphysics, the world of “outside” animals is entirely separate from the world of “inside” animals. Therefore, the only way for an animal to know the outside world is for discrete and local processes to span this ontological gap and bring the world

54

Jefrey B. Wagman, Tyler Dufrin, and Thomas A. Stofregen

outside that animal into contact with the world inside that animal. “Stimulus”—therefore—is the name given to any number of entities (objects, properties, energies) that participate in the discrete, local mechanical processes that bring the world into contact with the animal (and therefore bring the animal into contact with the world; Figure 3.2). In this way, the classical concept of “stimulus” is an attempt at an epistemological solution for an ontological problem. Initially, stimulus was used to describe the direct application of electrical current to tissue. Galvani’s 1780 discovery that electricity could cause involuntary muscle contractions even in (recently) dead animals was astonishing and was the inspiration for Frankenstein (Montillo, 2013). Such direct electrical stimulation of bodily tissue typically results in seemingly simple and involuntary muscular contractions (refexes) or seemingly simple and meaningless mental experiences (sensations). Later, the scope of the term was broadened to include the various forms of transmitted energy (e.g., light sound) that indirectly stimulated the nervous system via a particular receptor surface—or even the objects or properties from which such energy is transmitted. Such stimuli were presumed to form the basis of more complex and voluntary behaviors (i.e., movements) and more meaningful mental experiences (i.e., perception). Eventually, stimulus became a stand-in for any variable that might reliably generate or infuence a particular response or mental experience. This more general use of the term was a foundation of behaviorist psychology, which dominated the 1950s and 1960s, and of its successor, cognitive psychology, which remains prominent. The concept of the stimulus has a set of fundamental characteristics (Table 3.1; Gibson, 1960). First, the stimulus exists in (or as part of) the environment. The twig itself, its properties, and the light transmitted from the twig are outside the lemur and, therefore, part of the environment (Figure 3.4, right). Though the light received by the lemur’s retina is on a part of the lemur, it is clearly not part of the lemur. Consequently, light is also part of the environment. Second, the stimulus is transmitted from the environment and then passively received by (some part of) the animal. Properties of the twig are encoded in the light that is transmitted from the twig and received by the retina (Figure 3.2). And therefore, third, like any data received in any transmission, the stimulus itself is meaningless—the twig and its properties are inherently meaningless to the lemur. They only become meaningful by means of association with inherently meaningful things. And the light transmitted from the twig (and received by the retina) is also meaningless because it is ambiguously related to the twig’s properties. It only becomes meaningful after it has been transformed into a mental experience of such properties (Figure 3.2). Fourth, the stimulus is objective (i.e., it is

What and How Animals Know About Afordances

55

TABLE 3.1 Fundamental characteristics of “stimulus” and “afordance.”

Stimulus

Afordance

Exists in (or as part of) the E

Exists in the relationship between A and E Discovered by A Meaningful relationships specifed by lawful information; meanings are discovered Dependent on a given (type of) animal and on a given set of circumstances Nebulous spatial or temporal boundaries Vary in whether, when, and how they are perceived or acted on Dependent on (the fexibility of) behavior; independent of (the rigidity of) anatomy

Transmitted from E; received by A Meaningless data represented by meaningless stimulation; meaning must be acquired Independent of any given animal and of any given set of circumstances Well-defned spatial and temporal boundaries Vary in efectiveness Dependent on (the rigidity of) anatomy; independent of (the fexibility of) behavior

independent of any given animal) and abstract (i.e., it is independent of any given set of circumstances). The properties of the twig are so by defnition. But the light transmitted from the twig is so as well, given its ambiguous relation to those properties. This ambiguity means that the light received by the retina could refer to any number of an indefnitely large set of properties, twigs, or objects. Fifth, the stimulus has well-defned spatial and temporal boundaries. The twig, its properties, and the light transmitted from the twig and received by the retina have a particular location, and the response generated by the stimulus also has a particular duration. Sixth, stimuli are defned by whether, when, and how they elicit a response. A transmission of light that regularly elicits a strong response from the lemur is an efective stimulus. A transmission of light that irregularly elicits a strong response (or regularly elicits a weak response) from the lemur is a less efective stimulus. A transmission of light that never reaches the lemur—say refecting from a twig on the other side of the forest and therefore cannot elicit a response—is not a stimulus. And—quite strangely—a transmission of light that reaches the lemur and causes a response but then ceases to cause a response (e.g., if the lemur closes its eyes or gazes elsewhere), temporarily ceases to be a stimulus. Seventh, stimuli are dependent on (the rigidity of) anatomy and independent of (the fexibility of) behavior. The degree to which a stimulus elicits a given response from a given animal depends on the anatomy of that

56 Jefrey B. Wagman, Tyler Dufrin, and Thomas A. Stofregen

animal—for example, the number and type of receptors embedded in its sense organs, the number and type of efectors attached to its body, and its various anatomical components are wired together. However, it does not depend on whether the response that is elicited (e.g., a muscle twitch, an eyeblink) is of any relevance whatsoever to the variety of everyday behaviors performed by the animal. Fundamental Characteristics of Afordances

Gibson realized that each of these characteristics was problematic for a scientifc explanation of the successful performance of everyday behaviors. Rather than persist in using the stimulus concept to overcome the fundamental separation of the animal and the environment, he developed the afordance concept to refect the fundamental relationship between the animal and the environment. Unlike stimuli, afordances are aspects of ontology. Afordances are facts that might (or might not) be known and exist regardless of whether they are perceived. The concept of afordance has a very diferent set of fundamental characteristics (Table 3.1). First, a given afordance of the twig exists neither in the animal nor in the environment. Instead, it emerges from the multifaceted relationship between the two (Figure 3.4, left). And the ambient energy arrays lawfully structured by this relationship are neither outside nor on the animal so much as they surround or envelop the animal. Second, afordances are neither transmitted by an object nor received by (any part of) the animal. Instead, they are specifed by patterns in ambient energy arrays at a given point of observation and are discovered by an animal (Figure 3.5). Animals explore these patterns and obtain information about afordances. Third, given that afordances are opportunities to perform everyday behaviors in an ecological niche, they are meaningful to animals—as are the patterns in ambient energy arrays that specify afordances. What the twig afords the lemur is what the twig means to the lemur. These meanings are discovered, not constructed in the head. Fourth, afordances are neither objective (in that they are independent of any given animal) nor abstract (in that they exist independently of any given set of circumstances). Instead, they are both relational and concrete. They emerge from the relationship between the animal and the environment under particular circumstances. And the patterns in ambient energy arrays encountered by the animal are informative about afordances that exist under this set of circumstances and the afordances that will or might exist under other sets of circumstances. Fifth, many afordances have nebulous spatial or temporal boundaries (Figure 3.3, right). Afordances emerge from particular relationships between animals and environments under particular circumstances. In many

What and How Animals Know About Afordances

57

cases, neither these relationships nor these circumstances have well-defned locations or durations. Therefore, in many cases, afordances have neither well-defned locations nor well-defned durations. Sixth, afordances do not vary in efectiveness, but they may vary in whether, when, and how they are perceived or acted on. When looking for food (e.g., a fallen piece of fruit), a lemur may be more likely to perceive and act on afordances for looking under or next to a twig. When scurrying across the forest foor, the very same lemur may be more likely to perceive and act on afordances for stepping over the twig. And—reassuringly—if the lemur does not (or cannot) perceive a given afordance, that afordance continues to exist. Seventh, afordances are largely dependent on (the fexibility of) behavior and independent of (the rigidity of) anatomy. An afordance emerges from a multifaceted relationship between the animal and the environment, but any given multifaceted relationship can be brought about in indefnitely many ways. A lemur might be able to pick up a twig with its forelimb, its hindlimb, its tail, or even its mouth—despite the anatomical diferences across these efectors. Moreover, the lawful structuring of ambient energy arrays by such relationships entails that information about a given afordance can be detected in indefnitely many ways. A lemur could perceive an afordance of a twig by actively exploring the structure that emerges across multiple energy arrays (Mantel et al., 2015; Stofregen et al., 2017; Stofregen & Bardy, 2001). What Is the Connection Between Afordances, Behavioral Scale, and Intention? How Distant or Reluctant Can a PerceivingActing System Be Before Afordances Come Into Play? Do Afordances Exert Their Role Only When the System Is Close or Willing Enough? One Candidate Defnition of Afordances

Although there are competing (and sometimes conficting) defnitions of afordance both inside and outside ecological psychology, one candidate defnition (in fact, the elaborated defnition given earlier) is as follows (Blau & Wagman, 2022; Chemero, 2003; Stofregen, 2003): Afordances are opportunities for behavior that emerge from the multifaceted relationship between the animal and the environment under a particular set of circumstances. There is a lot to like about this candidate’s defnition of afordances (Figure 3.4, right). For one thing, it respects the multiplicity of afordances. That

58

Jefrey B. Wagman, Tyler Dufrin, and Thomas A. Stofregen

is, it respects the fact that the number of afordances for a given animal is uncountably large, given (i) the number of animal components that contribute to afordances, (ii) the number of environmental components that contribute to afordances, and, consequently, (iii) the (even larger) number of multifaceted relationships between these sets. Although a given set of circumstances may reduce the number of possible multifaceted relationships, it will still be the case that under any given set of circumstances, there are always more afordances than can be perceived or acted on, and there are always multiple ways in which to perceive and act on any given afordance. Another thing to like about this defnition is that it respects that afordances are qualitatively diferent than—are of higher order than—the animal or environmental components from which they emerge, which by comparison are of lower order. That is, an afordance is no diferent than any other higher-order relation that lawfully emerges from (yet qualitatively difers from) lower-order components. Critically, however, the higher order relation and the (lower order) components from which it emerges have the same ontological status and are subject to the same natural laws expressed at diferent scales (Turvey et  al., 1981). Moreover, the higherorder relations between the animal and the environment yield a given afordance such that they lawfully structure ambient energy arrays. Thus, animals can know about a given afordance by detecting such structure and lawfully structured information about that afordance (Figure 3.5). Yet another thing to like about this candidate defnition is that it respects the goal-directedness, or intentionality, of perceiving and acting on afordances. Given the multiplicity of afordances, at any given moment, an animal must choose which afordances to perceive and act on as well as when and how to do so. Moreover, that animal must do so with respect to overarching goals and underlying circumstances. A perhaps implied—but critical—consequence of this candidate defnition is that afordances are not (and cannot be) obligations. Animals are not compelled to perceive or act on any afordances. Some afordances may be perceived or acted on more easily than others. For example, a lemur may more easily perceive and act on the afordances of a nearby twig than those of a faraway twig. But ease (or difculty) is not the same as an obligation. The lemur may choose not to act on the afordances of the nearby twig because it is aware of the afordances of the faraway twig. Alternatively, the lemur may not perceive any afordances relating to the twig when its current goals are unrelated to such afordances (e.g., when it is looking for a place to rest). Some afordances may also be perceived and acted on more purposefully than others. For example, a lemur may purposely perceive and act on afordances for climbing a tree but may accidentally act

What and How Animals Know About Afordances

59

on afordances for falling out of a tree (regardless of whether such afordances are perceived). But purpose (or lack thereof) is not the same as an obligation. However, for all that there is to like about the candidate defnition, it may ultimately be incomplete in that it may apply only to a particular subset of the indefnitely many afordances that (can) exist for a given animal in a given niche. In particular, the candidate defnition seems to apply quite well to the subset of afordances (i) that emerge from a relationship between an animal’s physical capabilities and the environment; (ii) for behaviors that are well-defned with respect to scale, scope, and duration; and (iii) that emerge under circumstances that apply to animals as individuals (such as metabolic costs, penalties for errors, and physical comfort). Such afordances—so-called “afordances for behaviors” (see Bruineberg et al., 2019)—include those for reaching, grasping, stepping over, and ftting through and are regularly encountered by members of most (if not all) animal species (Wagman et  al., 2019). It is perhaps no surprise that the bulk of empirical eforts has been directed at investigating perceiving and acting on afordances in this subset (see Wagman, 2020; Figure 3.3, left). However, the candidate defnition does not seem to apply as well to the subset of afordances (i) that emerge from a relationship between an animal’s cognitive capabilities and the environment; (ii) for behaviors that are not well-defned with respect to scale, scope, and duration; and (iii) that emerge under circumstances that apply to animals as members of a community or society (including those that emerge due to social norms, customs, or conventions). Such afordances—so-called afordances for “higher order cognition” (see Bruineberg et al., 2019)—include those for interacting with individuals of difering social status, completing a crossword puzzle, attending a museum exhibit, going on a date with a romantic partner, or obtaining a college degree. Afordances of this kind exist for fewer species and, in many cases, are less frequently encountered than the kinds of afordances central to the candidate defnition of afordance. Yet, such afordances exist and are perceived and acted on by members of at least some animal species (Figure 3.3, right). The fact that the consensus defnition seems to apply (more easily) to “afordances for behaviors” than to “afordances for higher order cognition” seems to be one of the primary motivations for the claims that (i) these are fundamentally distinct categories of afordances and (ii) whereas “afordances for behavior” can be directly perceived—by detecting lawfully structured information about such afordances (Figure 3.5), afordances for higher-order cognition must be indirectly conceived—by detecting (partly non-lawful) regularities that constrain possibilities (Bruineberg et al., 2019; Figure 3.6, left).

60 Jefrey B. Wagman, Tyler Dufrin, and Thomas A. Stofregen

FIGURE 3.6

In the consensus defnition of afordances, afordances for behavior can be known by detecting lawfully structured information, but afordances for higher-order cognition can only be known (in part) by doing so in the context of non-lawful, conventional constraints (left); in the amended defnition of afordances, all (both lower- and higher-order) afordances can be known by detecting lawfully structured information (right).

What and How Animals Know About Afordances

61

The Amended Defnition of Afordances

We believe that these distinctions (in both ontology and epistemology) are unnecessary and (more importantly) inconsistent with a general and thoroughgoing Ecological Approach capable of providing law-based explanations of the full range of phenomena that occur in animal-environment systems (Blau & Wagman, 2022; Turvey, 2019). We believe that only a modest amendment of the candidate defnition of afordances is required to eliminate these distinctions in ontology and epistemology and set the stage for such a general and thoroughgoing Ecological Approach. Our amended defnition of afordances is as follows: Afordances are emergent opportunities for behavior. Lower-order afordances emerge from the multifaceted relationship between the animal and the environment under a particular set of circumstances. Higherorder afordances emerge from the multifaceted relationship between afordances under a particular set of circumstances. In place of the two distinct categories of afordances—those for behavior and those for higher-order cognition—we propose a single category of afordances that difer only in their relation to other afordances (i.e., whether they are higher or lower order relative to other afordances). By defnition, a lower-order afordance is of higher order than the (lower order) animal and the environment components from which it emerges, and a higher-order afordance is of higher order than the lower-order afordances from which it emerges. These higher-order relations are no diferent than any other higher-order emergent relations in the natural world (such as friction emerging from a relation between surfaces). Critically, higher-order afordances and the lower-order afordances from which they emerge have the same ontological status and are subject to the same natural laws expressed at diferent scales (Turvey et  al., 1981). Moreover, like the higher-order relations (between components) that yield a given afordance, the higher-order relations (between lowerorder afordances) that yield a higher-order afordance lawfully structure ambient energy arrays. Given the nature of higher-order afordances (as higher-order relations between higher-order relations), information about higher-order afordances is likely to be higher-order as well, and developing sensitivity to that information may require specifc kinds of sociocultural experiences. Therefore, in place of diferent explanations as to how animals know about diferent categories of afordances, we propose a single explanation for how animals know about all afordances—by detecting lawfully

62

Jefrey B. Wagman, Tyler Dufrin, and Thomas A. Stofregen

structured patterns in ambient energy arrays that provide information about afordances (Figure 3.6, right). We turn to the implications of our amended defnition for the ontology and epistemology of afordances next. How Do Systems Engage With Afordances as They Move Among Tasks and Intentions?

In the candidate defnition, afordances emerge from the multifaceted relationship between the animal and the environment under a particular set of circumstances. In this defnition, a given afordance has a necessary relation to the animal and the environment but no necessary relation to other afordances. Therefore, while afordances are emergent higher-order relationships, each afordance (each higher-order relationship) exists as an entity unto itself. As an animal engages in daily life activities, it encounters a sequence or series of stand-alone entities, each independent from other afordances and the larger context in which the afordance emerges. Consequently, in the candidate defnition, only those afordances that emerge from very particular relationships and under very particular circumstances can lawfully structure ambient energy arrays. Bruineberg et al. (2019) have proposed that only afordances that emerge from relationships and under circumstances that are sufciently “locally present” (i.e., “afordances for behavior”) can lawfully structure ambient energy arrays. They argue that when an animal encounters afordances for sitting on a chair (as a stand-alone physical act), all of the relationships from which and circumstances under which that afordance emerges are locally present. Thus, these relationships and circumstances can (and do) lawfully structure ambient energy arrays (Figure 3.5). By contrast, they argue that afordances that emerge from relationships and under circumstances that are insufciently “locally present” (i.e., “afordances for higher order cognition”) cannot lawfully structure ambient energy arrays (or cannot do so completely). For example, when an animal encounters afordances for sitting on the chair on display in a museum exhibit, only some of the relationships from which and circumstances under which this afordance emerges are locally present. In particular, those relating to afordances for sitting on a chair (as a stand-alone physical act) are present, but those relating to the human sociocultural practices associated with visiting museums (and museum exhibits) are not (Figure 3.6, left). Therefore, animals can know about afordances for behavior—such as those for sitting on a chair—in the same way, that a lemur can know about afordances for picking up a twig—by detecting the lawfully structured patterns in ambient energy arrays that provide information about that afordance. But, animals can only know about afordances for higher order

What and How Animals Know About Afordances

63

cognition—such as those for sitting on a chair on display in a museum exhibit—by detecting a combination of (i) lawfully structured patterns in ambient energy arrays that provide information about that afordance and (ii) non-lawful regularities that constrain on (but do not provide lawful information about) afordances. Such regularities are provided by the larger context in which the afordance is encountered, including, among other things, human sociocultural practices (in this case, those associated with visiting a museum exhibit; Figure 3.6, left). One Kind of Afordance; One Way to Know About Afordances

In the amended defnition, afordances emerge from the multifaceted relationships between the animal and the environment and between other afordances under a particular set of circumstances. In this defnition, a given afordance has a necessary relation to the animal and the environment but also to other afordances. Therefore, afordances are emergent higher-order relationships that exist in relation to other higher-order relationships. Accordingly, as an animal engages in daily life activities, it encounters a set of relations among afordances, including the relations among lower-order afordances from which a given higher-order afordance emerges. In other words, we argue that the larger context in which afordance is embedded—including human sociocultural practices—is not a separate non-afordance factor that constraints a given afordance. Instead, the larger context in which afordance is embedded is how that afordance relates to other afordances. That is, the context is the set of relations among lowerorder afordances (e.g., afordances for sitting on chairs and for attending museum exhibits) that yield a higher-order afordance (e.g., afordances for sitting on a chair on display in a museum exhibit). Thus, afordances for sitting on a chair on display in a museum exhibit and afordances for sitting on a chair (perhaps in the museum cafeteria) are not examples of the same afordance in diferent contexts so much as they are examples of two diferent higher-order afordances. Moreover, a person who encounters a given afordance (including a higher-order afordance) does not (and can never) encounter that afordance outside of its relation to other afordances. For example, a person who encounters the afordances of a chair on display in a museum exhibit does not merely encounter the afordances of the chair (as if the person and the chair just so happen to be in the same place at the same time). Rather they do so under a set of circumstances that includes moving from a place where the chair is not (anywhere other than the museum exhibit) to the place where the chair is (the museum exhibit). In doing so, they encounter the afordances of the chair in relation to many other (comparatively)

64

Jefrey B. Wagman, Tyler Dufrin, and Thomas A. Stofregen

lower-order afordances, including those of the entrance to the museum, those of the museum lobby, those of the hallway leading to the exhibit, those of the exhibit itself, and those of the other museum-goers. And in doing so, they encounter lawfully structured patterns in ambient energy arrays that provide information about these (comparatively) lower-order afordances. Thus, when they (eventually) encounter the chair on display in the museum exhibit, they have encountered a continually unfolding, lawfully structured pattern in (and across) ambient energy arrays that provides information about how these lower-order afordances relate to one another, yielding the higher order afordances of the chair on display in the museum exhibit (see Heft et al., 2014). We propose that this sort of continuously unfolding higher-order lawfully structured pattern in ambient energy arrays (or something like it) provides information about higherorder afordances (Figure 3.6, right). Therefore, by the amended defnition, a person can know about higherorder afordances for sitting on a chair on display in a museum exhibit in the same way they can know about lower-order afordances for sitting on a chair and in the same way a lemur can know about afordances for picking up a twig—by detecting lawfully structured patterns in ambient energy arrays that provide information about such afordances. In the case of knowing about the afordances for sitting on a chair on display in a museum exhibit, however, these lawfully structured energy patterns provide information about the relationship between afordances for sitting on chairs and afordances for attending museum exhibits (Figure 3.6, right). It is important to appreciate that just as the person and the chair will never just so happen to be in the same place at the same time, a person does not just so happen to develop sensitivity to the higher-order afordances of the chair on display the museum exhibit (or any afordance for that matter). Rather, just like developing sensitivity to lower-order afordances, developing sensitivity to higher-order afordances is a (sometimes long-term) process of discovering (i) the (sometimes subtle) patterns in ambient energy arrays that provide information about such afordances and (ii) the (sometimes subtle) exploratory behaviors that serve to reveal such patterns. Moreover, this process is often facilitated (or even made possible) by specifc kinds of sociocultural experiences, including the actions of caregivers, peers, teachers, and coaches (e.g., Nonaka & Stofregen, 2020; O’Neal et al., 2021). The Bottom Line: Is Context Separate From Relations?

In both the candidate and our amended defnitions, afordances emerge from the multifaceted relationship between the animal and the environment under a particular set of circumstances. In other words, in both

What and How Animals Know About Afordances

65

defnitions, afordances are relational and context-dependent. The question is whether the relations from which a given afordance emerges and the context in which a given afordance emerges (including, e.g., human sociocultural practices) are meaningfully separable. Suppose the relations from which afordances emerge and context are separate and distinct. In that case, either context does not exist at the ecological scale, or the context otherwise cannot lawfully structure ambient energy arrays (Figure 3.6, left). Neither of these options is desirable. Moreover, both seem to imply that (i) some aspects of knowable reality lawfully structure ambient energy arrays while others do not, (ii) there are multiple ontological categories of knowables, and (iii) there can be contextfree afordances as well as afordance-free contexts. We do not believe that these implications can be sustained within a general and thoroughgoing Ecological Approach grounded in lawful relations and frst principles (Turvey, 2019; Turvey & Carello, 2012). However, suppose (as we propose), context consists of relations from which higher-order afordances emerge. In that case, contextual factors (including metabolic costs, penalties for errors, and physical comfort but also norms, customs, and conventions) are describable using the frameworks of Ecological physics and Ecological geometry as relations among afordances. In that case, (i) all aspects of knowable reality lawfully structure ambient energy distributions, (ii) there is only one ontological category of knowables, and (iii) there are neither contextfree afordances nor afordance-free contexts. These implications are consistent within a general and thoroughgoing Ecological Approach in which afordances and the laws that govern them exist in indefnitely rich contexts (Gibson, 1979; Turvey, 2019; Turvey et al., 1981; Turvey & Carello, 2012). Uniqueness Does Not Trump Lawfulness

One of the explicit motivations for the claim that are distinct kinds of afordances (“afordances for behavior” vs. “afordances for higher order cognition”) that require diferent kinds of explanations (detection of information vs. detection of regularities) is to capture the “variety of sociocultural practices within the human way of life” (Bruineberg et al., 2019, p. 5239). Clearly, visiting a museum exhibit is not among the sociocultural practices of lemurs. But, lemurs do have other sociocultural practices (Kendal et al., 2010; Kittler et al., 2015; Whiten, 2018) as do other non-human animals such as songbirds, dolphins, wolves, and insects. At the same time, not all humans (e.g., infants and toddlers) have the required understanding of sociocultural practices within the human way

66

Jefrey B. Wagman, Tyler Dufrin, and Thomas A. Stofregen

of life to perceive and act on higher-order afordances relating to social norms, customs, or conventions. However, we claim that higher-order afordances are routinely perceived by all humans and by many (if not most) species of non-humans, including those with relatively unsophisticated nervous systems (e.g., goldfsh, earthworms, mollusks, insects, and jellyfsh) and those without nervous systems of any kind (e.g., amoeba, and even plants; Carello et  al., 2012; Chittka & Rossi, 2022; Turvey, 2013; Wagman et  al., 2019). In short, we accept that many of the afordances that humans perceive and act on are unique to the human species and that the larger context in which such afordances are embedded includes sociocultural practices unique to humans. However, we take issue with the proposal that such uniqueness requires a unique set of explanatory processes that applies solely to humans. An afordance is of higher order than the components from which it emerges. Yet, the afordance and the components have the same ontological status and are subject to the same laws expressed at diferent scales. In the same way, we have argued that the set of afordances for and sociocultural practices of humans are of higher order than (or simply diferent from) those for and of other species. Yet, all afordances have the same ontological status and are subject to the same laws expressed at diferent scales. In our view, the complexity of afordances for and sociocultural practices of a species varies on a continuum with the complexity of that species’ niche. Interestingly, when the complexity of a given niche is artifcially modifed, species in that niche (e.g., rats, goldfsh) can often perceive and act on afordances and engage in sociocultural practices that they otherwise would not have, and can do so with minimal practice (e.g., driving a motorized vehicle; Crawford et  al., 2020; Givon et  al., 2022). We claim that studies like these raise questions not about the unique ways in which certain species of animals perceive and act on afordances but how relations among afordances are perceived and exploited by a wide range of species, such that a greater understanding of such relations can contribute to the development of theories of perception and action that are general across species (Fultot et al., 2019; Mangalam et al., 2022). Notes 1. Later, we will argue that this is a limiting case and that patterns in ambient energy arrays that an animal encounters as it performs everyday behaviors are lawfully determined not only by its relationships to surrounding substances and surfaces but also by (higher order) relations between afordances. 2. Again, later we will argue that these patterns are also structured by (higherorder) relations between afordances.

What and How Animals Know About Afordances

67

Reference list Adolph, K. E. (2019). An ecological approach to learning in (not and) development. Human Development, 63(3–4), 180–201. https://doi.org/: 10.1159/ 000503823 Adolph, K. E., & Hoch, J. E. (2019). Motor development: Embodied, embedded, enculturated, and enabling. Annual Review of Psychology, 70(1), 141–164. https://doi.org/10.1146/annurev-psych-010418-102836 Adolph, K. E., Kretch, K. S., & LoBue, V. (2014). Fear of heights in infants? Current Directions in Psychological Science, 23(1), 60–66. https://doi.org/10.1177/ 0963721413498895 Amazeen, E. L., & Turvey, M. T. (1996). Weight perception and the haptic size— Weight illusion are functions of the inertia tensor. Journal of Experimental Psychology: Human Perception and Performance, 22(1), 213–232. https://doi. org/10.1037/0096-1523.22.1.213 Blau, J. J. C., & Wagman, J. B. (2022). Introduction to Ecological Psychology: A Lawful Approach to Perceiving, Acting, and Cognizing. Routledge. Bruineberg, J., Chemero, A., & Rietveld, E. (2019). General ecological information supports engagement with afordances for “higher” cognition. Synthese, 196(12), 5231–5251. https://doi.org/10.1007/s11229-018-1716-9 Carello, C., Vaz, D., Blau, J. J. C., & Petrusz, S. (2012). Unnerving intelligence. Ecological Psychology, 24(3), 241–264. https://doi.org/10.1080/10407413.20 12.702628 Chemero, A. (2003). An outline of a theory of afordances. Ecological Psychology, 15(2), 181–195. https://doi.org/10.1207/S15326969ECO1502_5 Chittka, L., & Rossi, N. (2022). Social cognition in insects. Trends in Cognitive Sciences, 26(7), 578–592. https://doi.org/10.1016/j.tics.2022.04.001 Crawford, L. E., Knouse, L. E., Kent, M., Vavra, D., Harding, O., LeServe, D., Fox, N., Hu, X., Li, P., Glory, C., & Lambert, K. G. (2020). Enriched environment exposure accelerates rodent driving skills. Behavioural Brain Research, 378, 112309. https://doi.org/10.1016/j.bbr.2019.112309 Fultot, M., Adrian Frazier, P., Turvey, M. T., & Carello, C. (2019). What are nervous systems for? Ecological Psychology, 31(3), 218–234. https://doi.org/10.108 0/10407413.2019.1615205 Gibson, E. J., & Walk, R. D. (1960). The “visual clif.” Scientifc American, 202(4), 64–71. www.jstor.org/stable/24940447 Gibson, J. J. (1960). The concept of the stimulus in psychology. American Psychologist, 15(11), 694–703. https://doi.org/10.1037/h0047037 Gibson, J. J. (1966). The Senses Considered as Perceptual Systems. Houghton Mifin. Gibson, J. J. (1979). The Ecological Approach to Visual Perception. Houghton Mifin. Givon, S., Samina, M., Ben-Shahar, O., & Segev, R. (2022). From fsh out of water to new insights on navigation mechanisms in animals. Behavioural Brain Research, 419, 113711. https://doi.org/10.1016/j.bbr.2021.113711 Heft, H., Hoch, J., Edmunds, T., & Weeks, J. (2014). Can the identity of a behavior setting be perceived through patterns of joint action? An investigation of place. Behavioral Sciences, 4(4), 371–393. https://doi.org/10.3390/bs4040371

68

Jefrey B. Wagman, Tyler Dufrin, and Thomas A. Stofregen

Higuchi, T., Murai, G., Kijima, A., Seya, Y., Wagman, J. B., & Imanaka, K. (2011). Athletic experience infuences shoulder rotations when running through apertures. Human Movement Science, 30(3), 534–549. https://doi.org/10.1016/j. humov.2010.08.003 Kendal, R. L., Custance, D. M., Kendal, J. R., Vale, G., Stoinski, T. S., Rakotomalala, N. L., & Rasamimanana, H. (2010). Evidence for social learning in wild lemurs (Lemur catta). Learning & Behavior, 38(3), 220–234. https://doi. org/10.3758/LB.38.3.220 Kittler, K., Schnoell, A. V., & Fichtel, C. (2015). Cognition in ring-tailed lemurs. Folia Primatologica, 86(1–2), 106–116. https://doi.org/10.1159/000368669 Mangalam, M., Fragaszy, D. M., Wagman, J. B., Day, B. M., Kelty-Stephen, D. G., Bongers, R. M., Stout, D. W., & Osiurak, F. (2022). On the psychological origins of tool use. Neuroscience & Biobehavioral Reviews, 104521. https://doi. org/10.1016/j.neubiorev.2022.104521 Mantel, B., Stofregen, T. A., Campbell, A., & Bardy, B. G. (2015). Exploratory movement generates higher-order information that is sufcient for accurate perception of scaled egocentric distance. PLoS One, 10(4), e0120025. https://doi. org/10.1371/journal.pone.0120025 Mark, L. S. (1987). Eyeheight-scaled information about afordances: A study of sitting and stair climbing. Journal of Experimental Psychology: Human Perception and Performance, 13(3), 361–370. https://doi.org/10.1037/0096-1523.13.3.361 Mark, L. S., Balliett, J. A., Craver, K. D., Douglas, S. D., & Fox, T. (1990). What an actor must do in order to perceive the afordance for sitting. Ecological Psychology, 2(4), 325–366. https://doi.org/10.1207/s15326969eco0204_2 Montillo, R. (2013). The Lady and Her Monsters: A Tale of Dissections, RealLife Dr. Frankenstein, and the Creations of Mary Shelly’s Masterpiece. Harper Collins. Nonaka, T. (2020). The triad of medium, substance, and surfaces for the theory of further scrutiny. In J. B. Wagman & J. J. C. Blau (Eds.), Perception as Information Detection: Refections on Gibson’s Ecological Approach to Visual Perception (pp. 21–36). Routledge. Nonaka, T., & Stofregen, T. A. (2020). Social interaction in the emergence of toddler’s mealtime spoon use. Developmental Psychobiology, 62(8), 1124–1133. https://doi.org/10.1002/dev.21978 O’Neal, E. E., Zhou, S., Jiang, Y., Kearney, J. K., & Plumert, J. M. (2021). Let’s cross the next one: Parental scafolding of prospective control over movement. Child Development, 92(2), e173–e185. https://doi.org/10.1111/cdev.13457 Reed, E. S. (1988). James J. Gibson and the Psychology of Perception. Yale University Press. Reed, E. S. (1996). Encountering the World: Toward an Ecological Psychology. Oxford University Press. Shockley, K., Carello, C., & Turvey, M. T. (2004). Metamers in the haptic perception of heaviness and moveableness. Perception & Psychophysics, 66(5), 731–742. https://doi.org/10.3758/BF03194968 Shockley, K., Grocki, M., Carello, C., & Turvey, M. T. (2001). Somatosensory attunement to the rigid body laws. Experimental Brain Research, 136(1), 133– 137. https://doi.org/10.1007/s002210000589

What and How Animals Know About Afordances

69

Stofregen, T. A. (2003). Afordances are enough: Reply to Chemero et  al. (2003). Ecological Psychology, 15(1), 29–36. https://doi.org/10.1207/ S15326969ECO1501_03 Stofregen, T. A., & Bardy, B. G. (2001). On specifcation and the senses. Behavioral and Brain Sciences, 24(2), 195–213. https://doi.org/10.1017/ S0140525X01003946 Stofregen, T. A., Mantel, B., & Bardy, B. G. (2017). The senses considered as one perceptual system. Ecological Psychology, 29(3), 165–197. https://doi.org/10.1 080/10407413.2017.1331116 Streit, M., Shockley, K., & Riley, M. A. (2007). Rotational inertia and multimodal heaviness perception. Psychonomic Bulletin & Review, 14(5), 1001–1006. https://doi.org/10.3758/BF03194135 Thomas, B. J., & Riley, M. A. (2014). Remembered afordances refect the fundamentally action-relevant, context-specifc nature of visual perception. Journal of Experimental Psychology: Human Perception and Performance, 40(6), 2361–2371. https://doi.org/10.1037/xhp0000015 Thomas, B. J., Riley, M. A., & Wagman, J. B. (2020). Information and its detection: The consequences of Gibson’s theory of information pickup. In J. B. Wagman & J. J. C. Blau (Eds.), Perception as Information Detection: Refections on Gibson’s Ecological Approach to Visual Perception (pp.  237–252). Routledge. Turvey, M. T. (1992). Afordances and prospective control: An outline of the ontology. Ecological Psychology, 4(3), 173–187. https://doi.org/10.1207/ s15326969eco0403_3 Turvey, M. T. (2004). Space (and its perception): The frst and fnal frontier. Ecological Psychology, 16(1), 25–29. https://doi.org/10.1207/s15326969eco1601_3 Turvey, M. T. (2013). Ecological perspective on perception-action: What kind of science does it entail? In W. Prinz, M. Bessard, & A. Herwig (Eds.), Action Science: Foundations of an Emerging Discipline. MIT Press. Turvey, M. T. (2019). Lectures on Perception: An Ecological Perspective. Routledge. Turvey, M. T., & Carello, C. (2012). On intelligence from frst principles: Guidelines for inquiry into the hypothesis of Physical Intelligence (PI). Ecological Psychology, 24(1), 3–32. https://doi.org/10.1080/10407413.2012.645757 Turvey, M. T., Shaw, R. E., Reed, E. S., & Mace, W. M. (1981). Ecological laws of perceiving and acting: In reply to Fodor and Pylyshyn (1981). Cognition, 9(3), 237–304. Wagman, J. B. (2015). As easy to move as a feather: Perception of lightness as ease to move. Journal of Motor Behavior, 47(4), 340–342. https://doi.org/10.1080/ 00222895.2014.983451 Wagman, J. B. (2020). A guided tour of Gibson’s theory of afordances. In J. B. Wagman & J. J. C. Blau (Eds.), Perception as Information Detection: Refections on Gibson’s Ecological Approach to Visual Perception (pp.  130–148). Routledge. Wagman, J. B., Blau, J. J. C., & Dufrin, T. (2023). Perception by efortful touch and a lawful approach to (the evolution of) perceiving and acting. In E. Bruner (Ed.), Cognitive Archaeology, Body Cognition, and the Evolution of Visuospatial Perception. Academic Press.

70 Jefrey B. Wagman, Tyler Dufrin, and Thomas A. Stofregen

Wagman, J. B., Lozano, S., Jiménez, A., Covarrubias, P., & Cabrera, F. (2019). Perception of afordances in the animal kingdom and beyond. In I. Zepeda, J. Camacho, & E. Camacho (Eds.), Aproximaciones al estudio del comportamiento y sus aplicaciones (pp. 70–108). Universidad de Guadalajara. Wagman, J. B., & Stofregen, T. A. (2020). It doesn’t add up: Nested afordances for reaching are perceived as a complex particular. Attention, Perception, & Psychophysics, 82(8), 3832–3841. https://doi.org/10.3758/s13414-020-02108-w Whiten, A. (2018). Social dynamics: Knowledgeable lemurs gain status. Current Biology, 28(8), R344–R346. https://doi.org/10.1016/j.cub.2018.03.004 Yasuda, M., Wagman, J. B., & Higuchi, T. (2014). Can perception of aperture passability be improved immediately after practice in actual passage? Dissociation between walking and wheelchair use. Experimental Brain Research, 232(3), 753–764. https://doi.org/10.1007/s00221-013-3785-9

4 MENTAL ACTION AND THE SCOPE OF AFFORDANCE PERCEPTION Thomas McClelland and Max Jones

A key question in afordance research is how broad a range of afordances we can perceive. Most advocates of afordance perception agree on some well-trodden examples—afordances to walk, afordances to climb, afordances to eat—but how far does the list extend? For example, should we include afordances for joint action, such as perceiving a sofa as afording lifting by us? Should we include afordances for tool-assisted action, such as perceiving a tree as afording climbing with a ladder? Should we include afordances that depend on a wider cultural practice, such as perceiving a post-box as afording to send a letter in a society with a postal system? These questions have important implications for empirical work on afordance perception. Given that we want a research program that encompasses all forms of afordance perception, it is particularly important to determine what scope our afordance perception has. In this wide-reaching discussion, attention has recently turned to the possibility that we perceive afordances for mental action. Some of our actions— such as attending, imagining, or deliberating—are typically described as mental actions. But do we perceive our environment as afording such mental actions? McClelland (2020) labels the view that we do perceive such afordances the Mental Afordance Hypothesis. If this hypothesis is given credence, it opens up multiple new avenues for afordance research. But do we perceive afordances for mental action? And if so, what mental afordances do we perceive? These questions depend on how exactly the distinction between mental and non-mental action is drawn. This turns out to be something of a conceptual minefeld, particularly when it comes to the possibility of embodied and extended cognition. Without arguing for any particular DOI: 10.4324/9781003396536-6

72

Thomas McClelland and Max Jones

view, this chapter aims to map out three views of mental activity and explore what they would each mean for the Mental Afordance Hypothesis. Afordances and Afordance Perception

Afordances are possibilities for action in an agent’s environment. A stone, for example, afords throwing for an agent just in case the agent can throw it. The stone being throwable involves a complementarity between agent and object: a match between features intrinsic to the stone, such as its size and weight, and features intrinsic to the agent, such as their strength and skill. It is generally built into the defnition of afordances that the thing aforded is an action (Michaels, 2003), though there are exceptions. Actions are things that an agent does, and not all of our behaviors are actions. For example, throwing a stone is an action, but instinctively shutting your eyes as a stone hurtles toward you is not. In some cases, it might be unclear whether someone has agency over behavior and thus unclear whether that behavior qualifes as an action. In other cases, it might be clear that someone is acting agentially but unclear how far their agency extends. For example, throwing a stone is an action, but are fne-grained adjustments to the wrist as the stone is released actions? One common view is that something can only qualify as an action if one is aware of it, so to the extent that such fnemotor adjustments fall outside awareness, they will not qualify as actions. In another view, much of what we do is determined by processes outside our awareness, meaning the requirement would be too demanding. The questions of agency and awareness are particularly pressing when it comes to mental action. For example, for something to aford imagining, it must aford an act of imagination, but to what extent is imagining something you do rather than something that just happens? There has been considerable philosophical disagreement regarding how much agency we have over our mental lives (see, for an overview, Peacocke, 2021). Agency over one’s mental processes plausibly requires awareness of those processes, but how fne-grained awareness do we have of the various mental activities that contribute to, for example, a complex act of imagination? Again, this is an open question. If one adopted a conception of afordances that eschewed the action requirement, they might countenance afordances for mental activities that are not mental actions. It is one thing for something to aford an action for an agent, but quite another for the agent to perceive that afordance. For example, once you have learned how to juggle, numerous objects will aford juggling-with. But that does not guarantee that you will perceive them as afording juggling-with. The Mental Afordance Hypothesis thus depends not just on environments ofering possibilities for mental action but on the agent being perceptually attuned to those possibilities. Diferent views can be taken on

Mental Action and the Scope of Afordance Perception

73

the nature of afordance perception. In James Gibson’s view, an agent can extract the afordances available to them from the optic array—the structured arrangement of light that reaches the eye (Gibson, 1966). Afordance perception is then constituted by directly detecting these patterns with no need for any internal process of interpreting and representing the environment. However, in some more recent views, afordance perception is constituted by internal representations of the afordances in the environment and not by direct perception (Siegel, 2014). Which way one goes on, this question could well have implications for mental afordances. In a Gibsonian view, afordances for imaginative actions would have to be detectable from the optic array, whereas on a representational view, such afordances could be perceived through an internal interpretation of sensory signals. A fnal choice-point concerns the motivational power of afordances. On one view (again advocated by Gibson), our perception of a throwing afordance is neutral. Whether we are motivated to act on this afordance depends on our background goals, emotions, and desires. In another view, advocated by researchers such as Ridderinkhof et al. (2011), afordance perception is inherently motivational. Rather than neutrally seeing the stone as something that could be thrown, we see it as positively calling out to be thrown. This choice-point again has implications for mental afordances. Are we looking for cases where we simply perceive possibilities for mental action or for cases where we perceive our environment as demanding mental action? It should be noted that even though the Mental Afordance Hypothesis requires mental afordances to be perceptible, it need not require them to be perceived in just the same way as ordinary bodily afordances. This is captured neatly by Scarantino: There is no reason to assume that a theory of perception can explain in the same way how humans perceive, if they do, a number being divide-by-twoable (a mental afordance), a fying ball being catch-able (a basic physical afordance) . . . [A]fordance properties ought not be treated by default as a homogenous block by theories of perception. They inherit from their constitutive relations with kinds of doings and kinds of happenings a number of distinguishing properties that are potentially relevant to establish whether they are perceivable, and, if they are, how they are perceived. —Scarantino (2003, pp. 960–961) This view invites interesting empirical questions about whether we perceive mental afordances and how we perceive them. Our aim in this chapter is to explore how the concept of afordances can be extended to mental action. Because of this exploratory goal we will, as far as possible, avoid committing to any particular view of the nature of afordances. What do we understand by the term afordance? As stated earlier, we regard

74

Thomas McClelland and Max Jones

an afordance as a possibility for action but side-step some of the more fnegrained debates about how to defne the term. What role does afordance play in perception? Is it the entity organisms perceive or the means through which organisms perceive? Are afordances the only perceptual dependent variables? On all of this, we can remain relatively neutral. The question of whether we perceive afordances for mental actions cross-cuts disagreements about the status of afordance perception. Why is afordance any better than stimulus? What does a theory of afordance suggest that stimuli cannot? What is the connection between afordances, behavioral scale, and intention? How distant or reluctant can a perceiving-acting system be before afordances come into play? How do systems engage with afordances as they move among tasks and intentions? Again, we can remain relatively neutral on these important questions. Although we will discuss intentions and the motivational power of afordances, we will avoid taking a stand on these questions. We aim later in the chapter to pick up on those points of contention that make the most difference to the prospects of the Mental Afordance Hypothesis. Now that we are equipped with a fuller understanding of afordances and afordance perception, we can explore the notion of mental afordances more deeply. Mental afordances are perceptible possibilities for mental action, but how should we distinguish mental action and non-mental action? One view is that what makes an action mental is done covertly, that is, without overt bodily movement. As we will see, though, this view has difculty accommodating embodied and extended mental actions. Another view is that what makes an action mental is that it is done to produce a mental state; for example, trying to recall someone’s name is a mental action because it is done to bring about a mental state in which that name is remembered. This accommodates some of the cases of mental action precluded by the frst view but can still be accused of relying on an excessively internalist conception of the mental. A fnal view is that there is no real distinction between mental and non-mental action and thus no real distinction between mental and non-mental afordances. Without taking a stand on which view is most defensible, we explore the implications each view would have for whether we perceive mental afordances and, if so, what kinds of mental afordance we perceive. Mental Action as an Internal Process

Peacocke (2021) explains that “[m]ental actions are sometimes individuated among actions by distinction from bodily action, or by their ‘covert’ nature” (p. 3). In this view, bodily actions are actions you do with your body, and mental actions are actions you do with your mind. For example, throwing a stone is an overt bodily action, whereas imagining throwing a stone is a covert mental action. On this conception of mental action, mental afordances

Mental Action and the Scope of Afordance Perception

75

are afordances for covert non-bodily action. The limited existing literature on mental afordances is based on this conception of mental action. Scarantino (2003) contrasts mental with bodily action, and Metzinger (2017) and McClelland (2020) both explicitly defne mental action as covert action. Accordingly, the mental afordances they discuss are afordances for covert acts of counting, covert shifts of attention, and covert acts of imagination. Understanding mental action in terms of non-bodily action need not be dependent on a traditional neurocentric and representationalist conception of the mind, which would counter many of the motivations ecological psychologists had for positing afordances of any kind in the frst place. Even if one takes the boundaries of the mind to be somewhere other than the boundaries of the brain, a mental act might nevertheless be performed without ostensibly engaging in any bodily movements. Similarly, such covert processes could be understood in non-representational terms rather than in terms of internal mental representations. The troubles for this conception of mental action come in those cases where mental action does seem to involve overt bodily movement. Bruineberg and van den Herik (2021) argue that “the bodily/mental distinction does not neatly track the overt/covert distinction. Paradigmatic mental actions sometimes involve bodily movement” (p.  3). When one counts using one’s fngers or using an abacus, the action involves overt bodily movement but would still be regarded as a mental action. Similarly, when imagining throwing a stone, you might move your arm in mimicry of the throw. And shifting your attention to a distracting stimulus often involves overtly shifting your head and eyes toward the stimulus. In these cases, too, the involvement of overt bodily movement doesn’t seem to preclude them from being mental actions (Table 4.1). One option here is to bite the bullet and maintain that cases like the aforementioned are not mental actions. In this view, acts of counting, attending, or imagining are only mental if they are genuinely covert. Accordingly, TABLE 4.1 Examples of mental action that do not neatly line up with the covert/

overt divide, as well as examples of covert and overt versions of each kind of mental action. Mental action

Covert

Overt

Attention

Covert attention

Imagination

Internal simulation

Counting

Counting in one’s head

Overt attention (including saccades, gaze, and head movements) Pretending Practicing Finger-counting Abacus-counting

76

Thomas McClelland and Max Jones

mental afordances are exclusively possibilities for covert action. So, if an agent perceives something as afording counting with their fngers, they perceive a non-mental afordance. This kind of response leaves us with a strangely disjointed view of mental actions in which counting in one’s head and counting with one’s fngers are completely diferent actions. It also gives us a similarly disjointed view of afordances in which counting afordances are replaced with counting-inone’s-head afordances and counting-with-one’s-fngers afordances. Matters get worse for this view when we consider the possibility that most of our mental actions involve overt movement, even if such movements are often subtle and subconscious. Although many of our mental actions involve no movement, there is evidence to suggest that many such mental activities are systematically accompanied by slight adjustments to our body and subtle movements of our sensory receptors. For example, some claim that even socalled covert attention involves functionally signifcant microsaccades (Engbert & Kliegl, 2003; Martinez-Conde et al., 2013). Similarly, eye movements and pupil dilation are systematically related to imagined content (Spivey & Geng, 2001; Sulutvedt et al., 2018), and, as all philosophers know, people often grasp at their mouth and chin when they think. Many would be hesitant to call these “actions” since the agent is usually unaware that they occur. However, we have already seen that awareness is not always regarded as a prerequisite for action. You can be aware of throwing a stone without being aware of the fne-motor processes involved in gripping and releasing the stone. Perhaps by the same token, you can be aware of your thinking without being aware of the bodily movements that underwrite your thinking. Here we can begin to see a relationship emerging between the extent to which one takes mental afordances to be embodied and the grain and scale at which one is willing to posit afordances. If afordances are limited to large-scale deliberative actions, then mental afordances can be largely taken to aford actions that are performed internally, but if one accepts subtle changes to one’s body as legitimate forms of action then the boundary between mental and non-mental afordances may begin to break down. The preceding suggests that an action’s being covert is not necessary for it being mental. There are also reasons to doubt that being covert is sufcient for mentality. There seem to be cases of acting without movement that one might be hesitant to call distinctively mental actions. For example, when a prey animal freezes upon seeing a predator in order to try to avoid detection, it makes sense to describe this as a bodily action. Moreover, it can plausibly be understood in terms of the creature perceiving and acting on a freezing-afordance. However, the lack of bodily movement that ensues shouldn’t thereby render the act of freezing a mental act, nor would it render the freezing-afordance a mental afordance. This, too, suggests we need a diferent way of framing the distinction between mental and non-mental actions.

Mental Action and the Scope of Afordance Perception

77

Mental Action as an Internal Result

Another way of distinguishing what makes an action a mental action is to look to the goal of the action, that is, what the action aims to accomplish. Proust (2001), for example, defnes a mental action as an action with a goal “which has to do with mental events, mental states, or mental dispositions” (p. 107). For Proust, this is explained in terms of changes to mental representations, which is unlikely to be attractive to many fans of afordances who would seek to maintain an anti-representationalist approach. However, it may still be possible to defne mental afordances in terms of mental goals without committing either way on the issue of whether mental states are essentially representational. Proust also introduces a further condition (ibid.), which may be useful for our purposes, namely that mental actions must be self-directed, in the sense that the goal must be some kind of change to the agent’s mind. This helpfully rules out the myriad of actions that aim to infuence others, such as talking, gesturing, etc., which don’t intuitively qualify as mental actions. In this view, mental actions are distinguished from non-mental actions by their result rather than by the process intended to produce that result. Counting coins, for example, is a mental action because it aims to achieve the mental state of knowing how many coins there are. This then means that the process of counting qualifes as mental regardless of whether it is performed covertly or overtly. Likewise, counting on one’s fngers or an abacus would be a mental action because it has a mental result. Consequently, mental afordances are afordances for actions with a mental result. When coins aford to count for an agent, they aford discoveringhow-many-coins-there-are. Acting on that afordance could be overt or covert, but it’s an afordance for mental action either way. This characterization of mental action draws attention to exploratory actions. Exploratory actions aim to acquire new information for the agent rather than at any specifc practical goal. Since acquiring new information is a mental state, any exploratory action would qualify as a mental action on this view. When afordances are introduced in the literature, authors focus on afordances for actions with primarily practical goals. For example, it is implicit in the oft-used example of the graspability of the mug on my desk that the action of grasping the mug will be in service of the practical goal of drinking the delicious tea it contains. However, a signifcant proportion of the actions that we engage in don’t serve an immediate practical goal. Rather, they provide the organism with access to new information in the environment or to learn to enable or facilitate future actions with more practical goals. In short, we sometimes engage in a particular form of epistemic action or theoretical action (Kirsh, 1996; Kirsh & Maglio, 1994; Neth & Müller, 2008), that is, actions with cognitive or

78

Thomas McClelland and Max Jones

epistemic goals. Rather than just exploiting the opportunities for action in our immediate environment, much of our activity is dedicated to exploring the environment to reveal further afordances to be exploited. What does this mean for our understanding of mental afordances? Well, all exploratory afordances would qualify as mental afordances in this picture. This fts with some of the candidate mental afordances discussed already. Afordances for attention are always, in our sense, exploratory afordances. Attending to something aims to acquire new information about it, so attentional afordances are mental afordances. Arguably, attending can sometimes be done covertly (Posner et al., 1980), but in ecologically valid scenarios will often be accomplished by utilizing overt actions, such as (in the case of visual attention) shifts of the eye or head position. But this still qualifes as a mental action insofar as it aims to acquire information about the attended object. Many imaginative afordances can also be understood as exploratory afordances aimed at achieving some epistemic goal. For example, when confronted with a puzzle, such as the Tower of Hanoi puzzle, agents will sometimes move the pieces around, “acting to learn” about what can be done with them rather than directly solving the puzzle (Neth & Payne, 2019). In some cases, this will be achieved by overtly manipulating the puzzle pieces, but, in others, the agent may achieve the same epistemic goal by merely imagining or simulating moving the pieces. There may be some situations that positively aford imagining or simulating rather than more overt forms of exploratory action, such as cases where trying out an action is risky. For example, situations where it is hard to perceive whether a rock face is climbable or a fast river is jumpable may aford simulation of the relevant action. Yet, even in these cases of afordances for imagining, overt bodily actions may play a role; for example, the agent may rehearse some of the run-ups to the jump to aid their mental simulation. But all such cases qualify as mental actions insofar as they have the immediate goal of learning about something. But does this characterization of mental actions stand up to scrutiny? One objection is that action being aimed at achieving a mental state is not sufcient for it being a mental action. For example, going to check what is in the fridge is aimed at coming to know what is in the fridge, but few would describe this as a mental action. Similarly, when an agent sees the fridge as afording looking-inside, this exploratory afordance does not seem apt for description as a mental afordance. Another objection is that action being aimed at achieving a mental state is not necessary for it being a mental action. If you count coins to divide them out evenly between people, then your aim is not an internal state of knowing how many coins there are but the external state of achieving an equal distribution of the coins. Accordingly, the counting afordance plausibly qualifes as mental even if it does not aim to achieve some mental state. Cases like this indicate

Mental Action and the Scope of Afordance Perception

79

that the boundary between epistemic and practical goals is blurry, so a conception of mental action built on that distinction will be similarly blurry. Another problem with this approach is that it still presupposes an internalist view of the mental. The frst defnition of the mental got into diffculties with its presupposition that mental actions are internal covert processes. Although this second defnition allows mental processes to be embodied and extended, the result of that action must still be an internal state. But on a strong conception of extended cognition, one’s mental states need not be internal. If one counts a set of coins using an abacus, for example, one need not count the beads and form an internal representation of how many coins there are. Instead, the state of the abacus itself constitutes your belief that there are n coins. Here, it is not just the mental process but the mental result of that process external to the body. This stronger form of externalism is, of course, contentious. But if one accepts such cases, then the Proustian defnition of mental action will not be ft for purpose. Mental Action as a Broken Concept

The two views we have discussed so far reveal how difcult it is to fnd an adequate defnition of mental action to build an account of mental afordances. What can be done in the face of such difculties? One option is to search for a diferent defnition. Given that extended cognition has been hard to accommodate, it makes sense to turn to the literature on extended cognition for a defnition. The most prominent account of extended cognition comes from Clark and Chalmers’ (1998) seminal paper on the topic, where they defne extended cognition in terms of functional similarity with internal mental processes via their infamous “parity principle.” This provides little help in this context, though, as it merely forestalls the question, with the nature of internal mental processes still up for grabs. It also smuggles the signifcance of the overt/covert distinction by privileging internal processes as the paradigm of mentality. Instead, we need a characterization of mental action that is neutral regarding whether the action is purely internal or partly external and whether it involves overt/covert action. Rowlands and Mark (1999) and Menary (2007) provide just such a characterization through their “manipulation thesis,” according to which a particular process can be understood as a cognitive process if it consists of the manipulation of information-bearing structures, whether these be internal structures, such as neural states, or external structures, such as arrangements of objects in the environment or external symbols. However, as with Proust’s earlier defnition of mental actions, there is a danger that this formulation will be rejected by many fans of ecological psychology, as framing things in terms of the manipulation of informational structures may be deemed too representationalist.

80 Thomas McClelland and Max Jones

In light of these continued problems, one might doubt whether a distinction between mental and non-mental action should be made. Perhaps the mental/non-mental distinction implicitly relies on outdated divisions between mind and body and between body and world. The best way to accommodate embodied and extended cognition would be to scrap the mental/non-mental distinction entirely. The examples discussed so far are still actions, but there’s no appropriate way to divide them into mental and non-mental actions. What does this mean for the Mental Afordance Hypothesis? On the face of it, such skepticism is completely at odds with the hypothesis. If there are no such things as mental actions, then there are no such things as mental afordances. Although this is true to a point, there could be a way of defending something very close in spirit to the Mental Afordances Hypothesis. We started this chapter by asking whether afordance perception includes afordances for actions like attending, imagining, and counting. If it turns out that these actions should not be described as mental, the important question remains: do we perceive afordances to attend, imagine, or count? The Mental Afordance Hypothesis can then be reframed as the view that we perceive afordances for actions traditionally regarded as mental without committing to any real distinction between mental and non-mental action. Interestingly, at least one view in the literature does just this. Bruineberg et al. (2019) develop a notion of afordances for “higher cognition,” such as afordances for planning and imagining. They explain that “[t]his conceptual framework is important for radical embodied and enactive cognitive science, because it allows these increasingly infuential paradigms to extend their reach to forms of ‘higher’ cognition such as long-term planning and imagination” (p. 1). In their kind of framework, the distinction between mental and bodily actions is unhelpful, but the kinds of actions they are trying to bring into the scope of afordance perception are just the kinds we are talking about. Concluding Remarks

The hypothesis that we perceive afordances for mental action potentially has signifcant ramifcations for empirical work on afordances. At the most basic level, it invites inquiry into whether we do indeed perceive afordances for mental action. If we do, it invites inquiry into which mental afordances we perceive and how we perceive them. As we have seen, though, empirical work on these questions will have to navigate the conceptual minefeld of how to defne mental action. If mental processes can be neither embodied nor extended, then the Mental Afordance Hypothesis can be framed as afordances for covert action. If mental processes can be embodied or extended, they might be better framed in terms of afordances for actions to achieve internal mental states. And if mental states need not be internal to

Mental Action and the Scope of Affordance Perception

81

Are there mental affordances?

Yes

No

What are mental actions?

Covert actions

Neurocentric mental affordances FIGURE 4.1

Actions with internal goals

No mental/physical divide

Possibility of embodied and/or extended mental affordances

Decision tree showing different ways of understanding mental affordances in terms of mental action, and the implications for the compatibility of the Mental Affordance Hypothesis with Embodied Cognition and Extended Cognition accounts.

the agent, then an alternative definition of the mental/non-mental distinction might be needed. For an overview of the options discussed, see Figure 4.1. Crucially though, there is an extent to which empirical researchers can sidestep these difficult questions. Whether we perceive affordances for those actions that we normally classify as mental, needs to be investigated. Affordance researchers have developed a variety of valuable research paradigms for exploring our perception of ordinary affordances. By adjusting those paradigms, we may reveal our sensitivity to affordances for acts of attention, imagination, counting, exploration, and more. Reference list Bruineberg, J. P., Chemero, A., & Rietveld, E. (2019). General ecological information supports engagement with affordances for “higher” cognition. Synthese, 196(12), 5231–5251. https://doi.org/10.1007/s11229-018-1716-9 Bruineberg, J. P., & van den Herik, J. C. (2021). Embodying mental affordances. Inquiry, 1–21. https://doi.org/10.1080/0020174X.2021.1987316 Clark, A., & Chalmers, D. (1998). The extended mind. Analysis, 58(1), 7–19. www.jstor.org/stable/3328150

82

Thomas McClelland and Max Jones

Engbert, R., & Kliegl, R. (2003). Microsaccades uncover the orientation of covert attention. Vision Research, 43(9), 1035–1045. https://doi.org/10.1016/ S0042-6989(03)00084-1 Gibson, J. J. (1966). The Senses Considered as Perceptual Systems. Houghton Mifin. Kirsh, D. (1996). Adapting the environment instead of oneself. Adaptive Behavior, 4(3–4), 415–452. https://doi.org/10.1177/105971239600400307 Kirsh, D., & Maglio, P. (1994). On distinguishing epistemic from pragmatic action. Cognitive Science, 18(4), 513–549. https://doi.org/10.1016/0364-0213(94)90007-8 Martinez-Conde, S., Otero-Millan, J., & Macknik, S. L. (2013). The impact of microsaccades on vision: Towards a unifed theory of saccadic function. Nature Reviews Neuroscience, 14(2), 83–96. https://doi.org/10.1038/nrn3405 McClelland, T. (2020). The mental afordance hypothesis. Mind, 129(514), 401– 427. https://doi.org/10.1093/mind/fzz036 Menary, R. (2007). Cognitive Integration: Mind and Cognition Unbounded. Palgrave Macmillan. Metzinger, T. (2017). The problem of mental action—Predictive control without sensory sheets. In T. Metzinger & W. Wiese (Eds.), Philosophy and Predictive Processing (pp. 1–26). The MIND Group. Michaels, C. F. (2003). Afordances: Four points of debate. Ecological Psychology, 15(2), 135–148. https://doi.org/10.1207/S15326969ECO1502_3 Neth, H., & Müller, T. (2008). Thinking by doing and doing by thinking: A taxonomy of actions. 30th Annual Meeting of the Cognitive Science Society, 993–998. Neth, H., & Payne, S. J. (2019). Thinking by doing? Epistemic actions in the Tower of Hanoi. Proceedings of the Twenty-Fourth Annual Conference of the Cognitive Science Society, 691–696. Peacocke, A. (2021). Mental action. Philosophy Compass, 16(6), e12741. https:// doi.org/10.1111/phc3.12741 Posner, M. I., Snyder, C. R., & Davidson, B. J. (1980). Attention and the detection of signals. Journal of Experimental Psychology: General, 109(2), 160–174. https://doi.org/10.1037/0096-3445.109.2.160 Proust, J. (2001). A plea for mental acts. Synthese, 129(1), 105–128. https://doi. org/10.1023/A:1012651308747 Ridderinkhof, R. K., Forstmann, B. U., Wylie, S. A., Burle, B., & van den Wildenberg, W. P. M. (2011). Neurocognitive mechanisms of action control: Resisting the call of the Sirens. WIREs Cognitive Science, 2(2), 174–192. https://doi. org/10.1002/wcs.99 Rowlands, M., & Mark, R. (1999). The Body in Mind: Understanding Cognitive Processes. Cambridge University Press. Scarantino, A. (2003). Afordances explained. Philosophy of Science, 70(5), 949– 961. https://doi.org/10.1086/377380 Siegel, S. (2014). Afordances and the contents of perception. In B. Brogaard (Ed.), Does Perception Have Content? Oxford University Press. Spivey, M. J., & Geng, J. J. (2001). Oculomotor mechanisms activated by imagery and memory: Eye movements to absent objects. Psychological Research, 65(4), 235–241. https://doi.org/10.1007/s004260100059 Sulutvedt, U., Mannix, T. K., & Laeng, B. (2018). Gaze and the eye pupil adjust to imagined size and distance. Cognitive Science, 42(8), 3159–3176. https://doi. org/10.1111/cogs.12684

5 AN AFFORDANCE-BASED APPROACH TO THE ORIGINS OF CONCEPTS Manuel Heras-Escribano, David Travieso, and Lorena Lobo

Explaining the origins of concepts has been a challenging problem in the philosophy of mind since at least the origins of British empiricism. John Locke talked extensively about this issue in his famous An Essay Concerning Human Understanding (Locke, 1948) and established the Modern version of the problem. Since then, many authors have dealt with this problem. Since we do not aim to enumerate all approaches, we must summarize the current state of the art as follows: there are two main approaches to the problem with critical diferences, but also with shared commitments. First, according to the empiricist approach to the origins of concepts, our minds use the information gathered by perceptual states processed and enriched to build up concepts about ordinary things in the world. Under this view, concepts are built upon the sensations and perceptual states that come from experience as they undergo diferent processes in which the percepts are refned and enriched to form concepts. This perspective emphasizes the richness of stimuli and how the materials we gather from experience determine the kind of concepts can have in mind. Authors in the Empiricist tradition include John Locke, David Hume, and contemporary authors like Burrhus F. Skinner, Lawrence W. Barsalou, William James, and Jesse Prinz. Second, according to the nativist approach, mechanisms, capacities, and contents in mind allow us to have concepts about the world with minimal contribution from our experience. This perspective emphasizes the cognitive equipment we have as humans and how it determines how we think regardless of having diferent experiences. Authors in the nativist tradition do not restrict to Plato, René Descartes, or Gottfried W. Leibniz; infuential DOI: 10.4324/9781003396536-7

84

Manuel Heras-Escribano, David Travieso, and Lorena Lobo

thinkers nowadays like Noam Chomsky, Jerry A. Fodor, and Steven Pinker have also endorsed some version of nativism. According to some authors (Fodor, 2004; Prinz, 2005), there are diferences between empiricists and nativists regarding a concept and its origins. For nativists, concepts are like words—they are used to think about the world in terms of representations. In this sense, a concept is like a symbol. Furthermore, few objectives can be accomplished with symbols other than communicating. On the contrary, the so-called empiricist view on concepts conceives them as an ability or capacity by which we produce an internal model of an entity, allowing us to categorize that entity to interact with it. However, there is a critical aspect that the empiricist nativist views have in common. As Prinz wrote: [R]ationalists say that concepts are primarily in the business of representing, and opponents of rationalism say that concepts are primarily in the business of doing. This distinction should not be regarded as a disjoint dichotomy. Empiricists do not deny that concepts represent. Rather, they claim that concepts have other equally important functions. Empiricists claim that concepts must be able to represent things in a way that facilitates interaction with those things. Representing must be in the service of doing. —Prinz (2005, p. 681) Despite the diferences in focus between empiricism and nativism regarding the central role of concepts, both claim that concepts have a representational nature, although they understand the function of those representations diferently. Nativists claim that concepts are symbols that represent the world, whereas empiricists claim that they are representations of entities that allow us to interact with them. They also agree that perception and action are also based on representational states. What Do We Understand by the Term “Afordance”? Introducing Afordances as a Bedrock for Cognition

Here we sketch the main features or requirements needed to build a nonrepresentational, experience-based approach to the origins of concepts. The best way to avoid the problems that a representational approach to the origins of concepts has to face is to start from a non-representational approach to perception and action, such as ecological psychology, according to which, perception is primarily of afordances. So, if we defend that experience is formed, thanks to the ecological approach to perception and

An Afordance-Based Approach to the Origins of Concepts

85

action, it is not necessary to postulate representations to explain how perception and action work. Ecological psychology was born between the 1950s and the 1980s, thanks to James J. Gibson and Eleanor J. Gibson, and some colleagues and students who further developed this approach. Ecological psychology is based on some key ideas: (i) the main unit of analysis for explaining cognition is neither the brain nor the organism, but the organism-environment system; (ii) perception and action are two sides of the same continuous process; (iii) perception is direct, which means that it is based on the pick-up of perceptual information without appealing to inner processing or representations; and (iv) perception is of afordances (Chemero, 2009; HerasEscribano, 2019; Richardson et al., 2008). Afordances are the main objects of perception for ecological psychologists. They are the directly perceived opportunities for acting in the environment—for instance, doors are directly perceived as pass-throughable, doorknobs are directly perceived as graspable, and stairs are directly perceived as step-on-able. We fnd afordances as a suitable starting point to explain the origins of concepts. The following section summarizes the main aspects of both the nativist and the empiricist views on the origins of concepts. We then discuss how to build an afordance-based approach to the origins of concepts. We introduce embodied concepts, specifcally bodily skillful and non-discursive ways to patternize the world that works as a missing link between basic and discursive cognitive states. In this sense, we develop an afordancebased and non-representational way to form embodied concepts, which in turn become the bedrock to build regular concepts. Two Past Theories for the Origins of Concepts: An Ecological Assessment The Nativist Approach

Nativists claim that concepts are like words or symbols: they represent the world. According to Fodor (2004), concepts represent certain objects if they are reliably activated after being presented with the kind of object they are supposed to represent. In this sense, a concept X reliably activates when a worldly item x is presented before us. How can we explain that concepts activate in the presence of certain items? Fodor postulates an asymmetric dependency: concept X sometimes activates in the presence of y because it normally and reliably activates under the presence of x (and not the other way round). As such, the concept X contains key features that represent the item x in a general way, just like words represent things in the world.

86 Manuel Heras-Escribano, David Travieso, and Lorena Lobo

The diference is that concepts represent the things they stand for not in a natural language but in the language of thought. In this sense, there is a causal linkage for explaining this connection, but there is little elaboration regarding the sensory origins of concepts in this view (Prinz, 2005). The Empiricist Approach

As we have seen, the nativist view is based on the idea that concepts have fxed features, and they activate before the presence of objects that have certain features connected to the ones of concepts via some kind of causal linkage or dependence. In the empiricist view of concepts, there is no set of fxed features that activate every time we encounter an item x. Prinz (2005) claims that there is empirical evidence gathered by Barsalou (1987) that concepts are variable constructions in our working memory built-in context-sensitive ways from our long-term memory. In this sense, two worldly items x that are sufciently diferent would activate the same concept X not because they share the same fxed patterns but because they represent the same category in diferent ways, regarding diferent contexts, and having diferent goals in every situation. Prinz completes this view by saying that concepts are perceptually based in that they are “made up from representations that are indigenous to the senses” (Prinz, 2005, p. 686). And these representations include the features of every sense modality through which we form such a representation. Prinz calls this the modal specifcity hypothesis and claims it is based on the empiricist (in particular, Humean) claim that all our ideas are copies of our impressions. The empiricist view accepts two main ideas of nativism: frst, concepts have a representational nature; second, concepts are reliably or normally caused by the category they represent. The main diference is how these categories are formed, which is when the modal specifcity hypothesis comes into play. We have seen that, for nativism, there is a series of fxed features that reliably activate before certain items appear. But, according to empiricism, those features are formed by particular perceptual modalities. In this sense, “[i]f perceptual states are essential for getting concepts to represent, we can simply hypothesize that concepts are copies of those perceptual states” (Prinz, 2005, p.  687). So the features change depending on the modality through which the item is represented or categorized. As such, we categorize what we have before us as having diferent perceptual features as basic materials and use previous modality-dependent representations to activate the concepts. So, in this sense, the empiricist view retains the explanatory power of the nativist approach postulated with the causal linkage and the representational nature of concepts but adds the modal specifcity hypothesis, which places the origins of concepts not in innate and amodal symbols in

An Afordance-Based Approach to the Origins of Concepts

87

our minds but in our sense modalities. A minor change that has strong consequences regarding the origins of concepts, one of them being that it is more parsimonious. This is why Prinz concludes that “[i]f concepts are associated with perceptual representations, perhaps that’s all we need. Postulating a further class of representations (amodal symbols) is unnecessary” (Prinz, 2005, p. 688). Thus, empiricism agrees with nativism in that concepts are mechanisms by which we categorize the world by representing it. However, whereas nativism claims that this categorization occurs in a symbolic way (not emphasizing the causal, sensory basis), empiricism claims that this categorization occurs via specifc modality, emphasizing the role of the sensory systems in the origins of concepts. What Role Does Afordance Play in Perception? Is It the Entity Organisms Perceive or the Means Through Which Organisms Perceive? Are Afordances the Only Perceptual Dependent Variables?

We perceive afordances or opportunities for action since these opportunities are related to our bodily skills: things are graspable if we have opposable thumbs and the ability to grasp, and steps are climbable if we have big enough legs and the ability to walk. According to ecological psychology, we do not need to postulate representations to explain how we directly perceive afordances. Instead, we perceive afordances simply by picking up the informational variables available in the environment that specify them. A wide variety of experimental data gathered for more than fve decades strongly support this claim (see, e.g., the summary and analysis of the results in Chemero, 2009; Richardson et al., 2008; Segundo-Ortin et al., 2019). In this sense, ecological psychology ofers a variety of “do not need” arguments regarding empirically supported and evidence-based representations. Why Is Afordance Any Better Than Stimulus? What Does a Theory of Afordance Suggest That Stimuli Cannot; How Has It Moved the Needle Past Gibson 1960’s Recognition of How Little We Can Defne Stimuli? The Importance of Ecological Information for the Origins of Concepts

Ecological psychologists are skeptics regarding the modal specifcity hypothesis in the case of afordance perception. According to ecological psychology, perceptual information is amodal: it can be detected by diferent perceptual systems and allows for the same behavioral responses. This view is supported by experimental evidence gathered by studies in ecological sensory

88 Manuel Heras-Escribano, David Travieso, and Lorena Lobo

substitution, which show that participants pick up the same ecological information through vision and touch for doing the same task, and they elicit the same behavioral response of distally attributing the same environmental objects, surfaces, et cetera (Travieso et al., 2015). Also, similar experimental studies in ecological sensory substitution have proven that typical features of a sense modality, like visual expansions in vision, can be translated into vibration patterns, and participants can discriminate them as such with the same efciency as in the visual modality (Cancar et al., 2013; Lobo et al., 2018). Ecological psychologists accept that there are specifc features of different perceptual systems, but at the same time, ecological information for afordances is amodal (which is sufcient to counter the modal specifcity hypothesis according to which every modality has diferent features and there are no shared amodal features across sensory modalities). Building a story of the origins of concepts from an ecological or afordance-based approach does not need to appeal to symbols that work like words in a language of thought. Concepts, then, should help us to interact with the world just like afordances do, so the primary function of concepts should be that of helping us engage with the environment. We disagree with the shared commitment of both views that representations are needed to explain the origins of concepts. Regarding nativism, we disagree that concepts always have a symbolic, discursive-like nature. Regarding empiricism, we disagree with the modal specifcity hypothesis and support that we interact with the world via concepts. In sum, we ofer a third way beyond nativism and empiricism to explain the origins of concepts. In this view, perception of afordances is the starting point, so concepts from an ecological perspective are in continuity with perception and action. The next section describes the story of the origins of concepts from an ecological standpoint. An Ecological Perspective to the Origins of Concepts

We agree with empiricists that concepts derive from experience, but we have diferent ideas regarding what we understand by “experience.” We think neither that perception is passive nor that sensations constitute representations and, with them, concepts; although we agree with empiricists that concepts facilitate interactions with objects and events. An Ecological Starting Point

Ecological psychology relies on an afordance-based approach to perception and action. We propose to scale up this approach to the problem of the origins of concepts. In this view, perception and action are continuous

An Afordance-Based Approach to the Origins of Concepts

89

and cyclic, and experience is not the passive sensory reception of worldly impingements but the implicit, tacit embodied knowledge that comes from a history of interactions with the environment. As we act in the world to perceive and act, we learn to perceive more efciently and improve our mastery of perceiving afordances. In this sense, it is not only the ecological information available in the environment that helps us modulate our actions but also our experience, our implicit, tacit, embodied knowledge of what happens when we perceive and act upon afordances. Experience, from this perspective, is an embodied and situated know-how that serves the purpose of navigating the environment skillfully and is the product of our developmental history of interactions with the environment. Both actual information and experience, understood as know-how, are essential for explaining how we take afordances. Organisms with a previous experience based on a history of interactions have a sense of anticipation. This sense of anticipation is provided by the very structure of ecological information available in the environment for perceiving afordances, thanks to the lawful relationship between the organism’s movements and the structure of the energy array of ecological information. This lawful regularity is established by specifcity because a given variable lawfully or regularly leads to another moment or state if all other things remain unchanged. As we can see, the upcoming state is triggered by the very structure of the present state as discrete moments within a dynamic, continuous perceptual experience. This sense of anticipation generated by the very structure of information has been defned as a “current future,” and it is the capacity to anticipate the next perceptual state from the present one. This is how organisms can face new situations and perceive and act upon new afordances. What Is the Connection Between Afordances, Behavioral Scale, and Intention? How Distant or Reluctant Can a PerceivingActing System Be Before Afordances Come Into Play? Do Afordances Exert Their Role Only When the System Is Close or Willing Enough? From Afordances to Experience The Role of Experience and Tacit, Bodily Knowledge in Perception

We cannot leave aside that organisms do not perceive and act upon new afordances all the time: they often navigate familiar places. In this sense, when we perceive and take afordances, we also do that based on our previous experiences with similar afordances in diferent contexts. Thus, in

90

Manuel Heras-Escribano, David Travieso, and Lorena Lobo

our everyday dealing with the world, thanks to the perception and taking of afordances, we establish a history of interactions from which derives implicit know-how of the consequences of taking an afordance. And part of that tacit, implicit knowledge includes what we can expect and what can result from the perception and taking of afordances. For example, after turning a doorknob for the frst time, you know that there is a range of consequences (mainly that the door either opens forward or backward), so you can also anticipate the upcoming state partially based on your history of interactions with doorknobs. This should not be controversial, as ecological perceptual learning implicitly requires this embodied, tacit knowledge since we attribute expertise based on a previous experience of interactions in which novices learn to detect the most specifc informational variable. So, once we perceive and take a specifc afordance for the frst time, it is natural that the next time we will expect what will happen; this is, we expect that our action will have this or that consequence. If this happens for every afordance we take, it is reasonable to conclude that we develop an implicit bodily knowledge of the repertoire of consequences derived from taking diferent afordances in diferent contexts. Since we deal with entrances and apertures in diferent contexts and with diferent afordances, we have in mind a repertoire of what to expect when dealing with one of them: they can have either doorknobs or simply need to be pushed, they can be pushed or pulled, there are also sliding doors, etc. But usually, in familiar contexts, no doors fall under the ground, hide under the ceiling when pushing them, or simply disappear when we touch them. The range of consequences is somehow limited in our environments, which helps us establish something like a repertoire of consequences of what will happen. When confronting doors, we have a more or less general idea of the possibilities regarding how to interact with them because we know the consequences of dealing with doors. This range of consequences helps us build some expectations regarding what we will fnd, so consequences and expectations are tightly linked. A Dispositional Basis of Experience

As such, this repertoire of diferent consequences allows us to develop different dispositions when we take afordances. We have seen that when we perceive and take afordances, we must consider both the ecological information available in the environment and the previous experience that we have in the form of tacit, implicit, or bodily knowledge. This knowledge has been established, thanks to the history of interactions with previous afordances, from which we have learned a range of consequences that occur when taking them, and that helps us have diferent expectations

An Afordance-Based Approach to the Origins of Concepts

91

regarding what will happen when we take them. In this sense, if we focus on the kind of knowledge we develop based on expectations and consequences, it is easy to think that both aspects allow us to develop certain behavioral tendencies or dispositions toward certain afordances. So, if we know how to make afordances and what will happen, expecting that a range of particular situations will take place when I take an afordance means developing a certain tendency toward some opportunity for action. Dispositions are conceived as how our experience is embodied in us. Just like there is a range of consequences when taking an afordance, there is also a range of dispositions toward those afordances. Returning to the example of doorknobs, if we want to go to a place where doors are covering the apertures, we tend to have this or that bodily movement for opening the door depending on whether it has doorknobs or not; it is a sliding door or not, etc. This experiential knowledge is bodily, as these tendencies are purely corporal: they are not abstractly thought and then applied mechanically without considering the environment’s particularities. This implicit bodily knowledge in the form of dispositions includes a wide range of postures, orientations, and the like that serve the purpose of taking certain afordances, depending on our goals. Thus, we have an implicit, embodied corpus of tendencies or dispositions to act when we perceive and take afordances, which can be understood as the basis for certain habits in a Deweyan sense (Segundo-Ortin & Heras-Escribano, 2021). How Do Systems Engage With Afordances as They Move Among Tasks and Intentions? Habits and Experience

The notion of habit has a signifcant place in Dewey’s thought, from his critique of the concept of the refex arc in psychology to his work on Human Nature and Conduct. In the latter work, he did not make this claim explicitly but proposed a new ontology of the mind based on habits that could overcome the traditional one based on states, events, and representations. Dewey defned individuals as complex systems of habits: that is, as an embodied set of complex ways of feeling, thinking, and acting. But what are the main aspects of habits? First, habits are functional relations with the environment: “habits are life functions in many respects . . . especially in requiring the cooperation of organism and environment” (Dewey, 1922, p. 14). This idea is well developed in Dewey’s critique of the refex arc in psychology (Dewey, 1896). In his view, stimuli are not pre-given objects or events that trigger automatic responses, as Watsonian behaviorism claimed. In Dewey’s view, both stimulus and response are a

92 Manuel Heras-Escribano, David Travieso, and Lorena Lobo

temporal and functional distinction within a continuous sequence of acts that are built by the organism with regard to its previous experience and goals: “In calling one stimulus, another response, we mean nothing more than that such an orderly sequence of acts is taking place” (Dewey, 1896, p. 366). This view implies that stimulus and response cannot be ontologically distinguished, as a response shapes the stimuli just as the stimuli enable a response. “The so-called response is not merely to the stimulus; it is into it” (Dewey, 1896, p. 359). Stimulus and response afect each other in the continuous development of the perception-action cycle. This continuity of perception and action and the rejection of the stimulus-response scheme are entirely in line with the ideas of ecological psychology. Following this reasoning, a habit might be defned as a particular and wellestablished type of functional, dispositional relation between an organism and an object or event, serving as the basis to establish new functional relationships between organism and environment. This view is also shared by ecological psychology, as it focuses not on the organism alone but on the organism-environment (functional) system. Second, habits provide continuity and consistency to organismal action, without which the organism cannot adapt to the environment. In this sense, there is routinization of habits and automaticity, but it is the automaticity derived from organismal learning, fexibility, and efciency, not the automaticity derived from rigid and mechanical behavior. As Dewey wrote, “[H]abits involve mechanization . . . but mechanization is not of necessity all there is to habit” (Dewey, 1922, p. 42). So, habits provide stable patterns of actions based on dispositions that become the main features of an organism. Moreover, these habits shape the organism’s personality, providing it with a framework for acting depending on circumstances. In conclusion, organisms are not discoordinated bundles of habits but agents whose actions are well-structured and coordinated, thanks to habits. Finally, these habits are the basis for explaining intentional, purposeful actions in organisms. If habits are embodied and pre-refective coordinative structures, they can be understood as “active means, means that project themselves, energetic and dominant acts of acting” (Dewey, 1922, p. 25). Dewey claimed an instrumental nature in which habits and their projective nature are means to achieve ends, as when we develop a habit of smiling in certain situations. Taking habits as instrumental allows Dewey to introduce one of its two most important properties: consciousness and intelligent thinking. In Dewey’s view, consciousness and habits go hand in hand because the former arises when the latter meets certain obstacles. In a Heideggerian way, as long as there is a smooth organism-environment coupling or adjustment,

An Afordance-Based Approach to the Origins of Concepts

93

no consciousness or refection upon our actions is required. However, when disadjustments enter the scene, intelligent thinking intervenes: [T]he work of intelligence in observing consequences and in revising and readjusting habits, even the best of good habits, can never be foregone. Consequences reveal unexpected potentialities in our habits whenever these habits are exercised in a diferent environment from that in which they were formed. (Dewey, 1922, p. 51) In the example of grasping the doorknob, one has the habit of opening doors, including a general way of dealing with doorknobs and their afordances consistently. This consistency includes an expectation of what will happen based on one’s experience. Hence, when we grasp and turn a doorknob, we expect that the door will exert a yaw rotation instead of a pitch or roll rotation. However, if, for some strange reason, someday we will face the strange case of a door that rotates around its transverse or longitudinal axes instead of its vertical axis, then we will have to intelligently revise or readjust our habits regarding opening doors (Figure 5.1). There is something that should be emphasized: in the example of turning the doorknob, there is no ecological information available about how the door is going to rotate, yet we have a certain expectation regarding how it is going to rotate, and our bodies are disposed of in a certain way regarding that expectation. In this sense, there is no need to invoke mental representations to explain this expectation when we can simply appeal to the idea of habit, as these coordinative structures allow us to pattern the world. Introducing Embodied Concepts

As we have seen, the habits we form through interaction with afordances include an implicit, embodied knowledge of what we can do and expect. This knowledge is embodied and pre-refective, including the postures, orientations, and movements that the body enacts when the organism has a certain goal or experience, and it is confronted with particular afordances whose taking will satisfy its goals. For example, organisms that perceive afordances of graspability know what to expect and how to move and orient when grasping something, and they know what actions can and cannot be done after grasping something (e.g., they know that they can throw the object, and they know that if they open their hands, the object will fall). This repertoire of consequences is well-known, the organism anticipates it,

94 Manuel Heras-Escribano, David Travieso, and Lorena Lobo

FIGURE 5.1

Example of how experience and habits interrelate. Courtesy of Cato Benschop.

and it is used to satisfy the agents’ goals with certain habits. In this sense, we can claim that knowing how to behave when taking afordances (in an implicit, embodied way) includes knowing a repertoire of possible consequences, which in turn implies that organisms can have a general idea of what to expect when dealing with certain objects and situations: that it can be delimited, somehow, a certain range of outcomes that derive from their

An Afordance-Based Approach to the Origins of Concepts

95

actions. This knowledge of certain consequences that regularly happen and that is expected on the basis of previous experience implies that some regular pattern or general feature has been identifed in every context in which an afordance has been taken. Such a pattern is anticipated, meaning it is identifed and discriminated against even before it occurs. It is a regular pattern identifed and anticipated from a limited number of particular situations, just like concepts are generalizations from particular cases. For this reason, we think it is reasonable to postulate the existence of embodied concepts,1 or a set or repertoire of consequences or patterns that can be expected or anticipated from the active dealing with a certain object or situation in terms of taking available afordances. Embodied concepts are not linguistic, discursive, or explicit. This means that discursive features are not applied to them. For example, they are not conceptually articulated propositional contents. At the same time, what they have in common with linguistic concepts is that they are generalizations from particular cases and help us deal with the world more efciently. What’s their importance, then, and what do we gain by postulating their existence? As we see it, embodied concepts are important because, in the literature, we usually fnd a story in which we move from contentless states in basic cognition to contentful cognitive states flled with concepts, as in an all-or-nothing explanation in which we move from nothing (contentless states with reactive and mechanical responses) to everything (conceptually articulated propositional states). But we think it is easier to postulate intermediate states in which there is something in the middle, between mere contentless reaction and pure conceptual abstraction, that makes this transition between both kinds of states gentle and not abrupt. In this sense, embodied concepts are the implicit, pre-discursive, bodily knowledge of certain causal patterns in the world, formed by the set of known consequences and expectations of what we can do when taking the afordances of particular things and environments. Knowing these regularities is more than merely reacting in a mechanical, uncoordinated, and disembodied way because the organism is precisely aware of what will happen due to the previously mentioned experience that allows it to have expectations and anticipation. Acting in a purely reactive and mechanical way implies that the organism has no experience, no previous history of interactions in which some kind of embodied, pre-refective, pre-discursive knowledge has not been achieved. But we know that even the simplest organisms learn through perception, that there is fexibility and adaptivity regarding what information we have to detect when learning to take afordances efciently, and that this knowledge, just like Dreyfus claimed, is “stored,” “not as representations in mind, but as dispositions to respond to the solicitations of situations in the world” (Dreyfus, 2002, p. 367). It is not discursive or

96

Manuel Heras-Escribano, David Travieso, and Lorena Lobo

explicit knowledge but an implicit bodily know-how related to the expectations and anticipations of patterned situations and the postures, movements, and orientations our bodies develop when confronted with those expectations. In this sense, infants and some animals with enough memory and learning capacities have those embodied concepts, too, as these creatures implicitly know what to expect from certain situations and enact a repertoire of movements in such contexts. It is a kind of bodily knowledge that includes the awareness of certain patterns due to learning through their particular history of interactions with the world. So, in this sense, embodied concepts are the products of our experience: the products of developing certain habits that coordinate our activity, thanks to our history of interactions with the environment. These interactions with afordances allow us to discover some patterns in the environment that, in turn, help us to develop some expectations, then some dispositions, and then some habits that skillfully coordinate our activity. In this sense, pre-discursive and embodied concepts are the missing link between basic contentless states and discursive, propositional contents. Understanding embodied concepts as representations would be an error when there is no need to postulate such entities. Here we appeal to the work of Eleanor Gibson on perceptual learning and the notion of the education of attention. For example, Gibson and Pick (2000, p. 158) stated, Perceptual learning is not an association of elementary processes nor is it a construction from elements of any kind nor the formation of a representation. It is a process of diferentiation resulting in specifcation of information for an afordance, a relation of an animal and its environment. The structure of the light in a room can ideally specify that an object is approaching us, so we start the movement to avoid a collision before that collision happens. It is necessary to perceive that information specifc to the event of a collision to do appropriate actions on time, like moving away from the object’s trajectory, catching it, or hitting it. Moreover, even if there is a component of a future situation, this is not a case in which we need to form a representation of that future event because the ambient energy array is rich enough for us to pick a specifc variable that guides our performance (Turvey, 2018). In our understanding, the expectations and anticipation that emerge from embodied concepts are based on the same ecological principles and not on cognitivist or representational ones. If someone tries to postulate representational inferences in the case of Figure 5.1, it is due to the widespread cognitivist assumption that the (bodily) knowledge of what to expect must be stored as a representation in our

An Afordance-Based Approach to the Origins of Concepts

97

heads. Nevertheless, anticipation and expectations are fundamental in our everyday perception and performance. This knowledge related to perceptual learning is better understood as a case of diferentiation than a case of enrichment or representational inference of the available information in the organism-environment system. Embodied Concepts as Interfaces Between Experience and Discursivity

We have seen that the gap between contentless, basic cognitive states, and contentful discursive states can be elegantly flled in with the postulation of embodied, pre-discursive concepts. In this sense, the elements present at one stage are the materials from which we can build the next stage. First, when perceiving an afordance for the frst time, the detection of information and the taking of an afordance serve as the basis of the canon for learning how to do it more efciently the next time. The next time we have some expectation that is the product of a previous interaction with the environment. After taking the same afordance several times, consequences are stable, and expectations are established, allowing us to develop certain dispositions. The stable consequences or range of possible consequences allow us to pattern the environment (present and future) in specifc ways. Once we have patterned the consequences, we can pattern the anticipations, so we start developing dispositions toward the afordances of particular objects and events. With time and routine, these dispositions become habits that coordinate our ways of achieving specifc goals, enacting routinary movements and orientations depending on our goals, experience, and available afordances. Thus, when we have developed some habits that coordinate our behavior depending on specifc patterns in the world that generally appear when taking particular afordances, we can claim that we can form some embodied concepts that help us deal with the world more efciently. At the same time, embodied concepts are the materials for forming traditional, discursive concepts once language enters the scene and critical changes are produced in our cognitive world. In this sense, embodied concepts are the interfaces between actionperception processes and conceptual, discursive articulation. Conclusion

Our proposal ofers a third way for explaining the origins of concepts beyond nativism and empiricism. We have seen that nativism and empiricism share several aspects: the representational nature of concepts and the causal origins of concepts. The main diference is that while nativism claims that concepts are symbolic amodal structures, empiricism claims that

98 Manuel Heras-Escribano, David Travieso, and Lorena Lobo

features of sense modalities form them. Also, nativism claims that the main function of concepts is to represent worldly items just like words do, while empiricism claims that the main function is to coordinate with the world via some representational structure. Our proposal rejects the representational nature of concepts and aims to ofer a non-representational approach to their origins based on the main aspects of ecological psychology, which is the same as saying that it is an afordance-based approach to concepts. We did not aim to ofer a full-blown account of concepts but to illuminate the previous steps for building an afordance-based or ecological approach to their origins. Some steps should be taken frst, which is what we have done in the third section of this chapter. We have sketched a journey from how we perceive and take afordances from an ecological perspective to the formation of experience understood as bodily know-how that comes from the history of interactions with the environment. This experience allows us to have certain expectations of the consequences of our activity, which in turn allows us to develop certain dispositions, which are, in turn, the basis of habits. Habits and their experience allow us to pattern the world, and these patterns are derived from a limited number of particular situations, just like concepts are generalizations from particular cases. For this reason, we postulate the existence of embodied concepts, which are non-representational ways in which we pattern the world based on our habits and worldly consequences. These embodied concepts would be the interfaces or links between mere perception and action processes on the one side and full-blown conceptuality and abstract thinking on the other. Acknowledgments

We are thankful to Annemarie Kalis, Cato Benschop, Josephine Pascoe, and Miguel Segundo-Ortin for fruitful comments and discussions on an earlier draft of this chapter. We are also thankful to Cato Benschop for the wonderful illustration for Figure 5.1. The work for this research has been generously funded by the following sources: Juan de la CiervaIncorporación Postdoctoral Fellowship (Ministerio de Ciencia e Innovación, Spain), Proyecto de Consolidación Investigadora 2022 “Toward an Ecological Approach to the Natural Origins of Content: From Direct Perception to Social Norms (ECOCONTENT)” (Ministerio de Ciencia e Innovación, Spain), and the research project “De la experiencia a los conceptos: Una reformulación del problema de Molyneux a través de la sustitución sensorial ecológica (ECOCONCEPT)” funded by the Ayudas a Proyectos de Investigación Científca 2022 Program of the BBVA Foundation (Spain).

An Afordance-Based Approach to the Origins of Concepts

99

Note 1. We know that the term “embodied concept” has been introduced in the literature before us (see, e.g., Shapiro, 2019, Chapter 4, for a wonderful update on the issue). However, whereas this term has been used to explain how the cognitivist approach to concepts is rooted in embodied aspects, here we propose a diferent way of understanding the term that is not related to cognitivism.

Reference List Barsalou, L. W. (1987). The instability of graded structure: Implications for the nature of concepts. In U. Neisser (Ed.), Concepts and Conceptual Development: Ecological and Intellectual Factors in Categorization (Vol. 10139, pp.  101– 140). Cambridge University Press. Cancar, L., Díaz, A., Barrientos, A., Travieso, D., & Jacobs, D. M. (2013). Tactilesight: A sensory substitution device based on distance-related vibrotactile fow. International Journal of Advanced Robotic Systems, 10(6), 272. https://doi. org/10.5772/56235 Chemero, A. (2009). Radical Embodied Cognitive Science. MIT Press. Dewey, J. (1896). The refex arc concept in psychology. Psychological Review, 3(4), 357–370. https://doi.org/10.1037/h0070405 Dewey, J. (1922). Human Nature and Conduct. Henry Holt and Company. Dreyfus, H. L. (2002). Intelligence without representation—Merleau-Ponty’s critique of mental representation the relevance of phenomenology to scientifc explanation. Phenomenology and the Cognitive Sciences, 1(4), 367–383. https:// doi.org/10.1023/A:1021351606209 Fodor, J. (2004). Having concepts: A brief refutation of the twentieth century. Mind & Language, 19(1), 29–47. https://doi.org/10.1111/j.1468-0017.2004.00245.x Gibson, E. J., & Pick, A. D. (2000). An Ecological Approach to Perceptual Learning and Development. Oxford University Press. Heras-Escribano, M. (2019). The Philosophy of Afordances. Palgrave Macmillan. Lobo, L., Travieso, D., Jacobs, D. M., Rodger, M., & Craig, C. M. (2018). Sensory substitution: Using a vibrotactile device to orient and walk to targets. Journal of Experimental Psychology: Applied, 24, 108–124. https://doi.org/10.1037/ xap0000154 Locke, J. (1948). An essay concerning human understanding, 1690. In Readings in the History of Psychology (pp.  55–68). Appleton-Century-Crofts. https://doi. org/10.1037/11304-008 Prinz, J. J. (2005). The return of concept empiricism. In H. Cohen & C. B. T.-H. of C. in C. S. Lefebvre (Eds.), Handbook of Categorization in Cognitive Science (pp.  679–695). Elsevier Science Ltd. https://doi.org/10.1016/ B978-008044612-7/50085-8 Richardson, M. J., Shockley, K., Fajen, B. R., Riley, M. A., & Turvey, M. T. (2008). Ecological psychology: Six principles for an embodied—Embedded approach to behavior. In P. Calvo & A. B. T.-H. of C. S. Gomila (Eds.), Perspectives on Cognitive Science (pp. 159–187). Elsevier. https://doi.org/10.1016/ B978-0-08-046616-3.00009-8

100

Manuel Heras-Escribano, David Travieso, and Lorena Lobo

Segundo-Ortin, M., & Heras-Escribano, M. (2021). Neither mindful nor mindless, but minded: Habits, ecological psychology, and skilled performance. Synthese, 199(3), 10109–10133. https://doi.org/10.1007/s11229-021-03238-w Segundo-Ortin, M., Heras-Escribano, M., & Raja, V. (2019). Ecological psychology is radical enough: A reply to radical enactivists. Philosophical Psychology, 32(7), 1001–1023. https://doi.org/10.1080/09515089.2019.1668238 Shapiro, L. (2019). Embodied Cognition. Routledge. Travieso, D., Gómez-Jordana, L., Díaz, A., Lobo, L., & Jacobs, D. M. (2015). Body-scaled afordances in sensory substitution. Consciousness and Cognition, 38, 130–138. https://doi.org/10.1016/j.concog.2015.10.009 Turvey, M. T. (2018). Lectures on Perception: An Ecological Perspective. Routledge.

6 TOWARD AN ECOLOGICAL THEORY OF TIME Brandon J. Thomas

What Do We Understand by the Term Afordance?

James J. Gibson (1979) originally coined the term afordance. “The afordances of the environment are what it ofers the animal, what it provides or furnishes, either for good or ill” (p. 127). In other words, afordances are action capabilities that depend on organism-environment reciprocity. Afordances are also perceived directly by detecting information, which Gibson (1966a) defned as invariant spatiotemporal patterns in energetic media that lawfully relate to the perceivable entities that structure them. Afordances are ontological; they exist regardless of whether they are perceived or actualized. Afordance is perhaps Gibson’s most infuential and empirically successful concept. Research on afordances exploded in the 1980s and inspired a generation of researchers who studied not only afordances and afordance perception but a wide range of embodied phenomena (Leitan & Chafey, 2014). I will focus on how the concept challenges traditional notions of time in the study of perception and, more broadly, experience. What Role Does Afordance Play in Perception? Is It the Entity Organisms Perceive or the Means Through Which Organisms Perceive? Are Afordances the Only Perceptual Dependent Variables?

The perception and actualization of afordances (i.e., actions) is a universal activity of living things across phylogeny, including organisms that do not possess a nervous system (Reed, 1996). The dynamics of afordances DOI: 10.4324/9781003396536-8

102

Brandon J. Thomas

over time comprise the organism-environment, an object of experience for organisms. Furthermore, afordances defne the perception-action landscape of organisms within the ecological niche, and their perception and actualization are necessary and sufcient for adaptive, goal-directed behavior. In other words, afordances are the entity perceived, and perceptionaction is how perception is accomplished. Afordances may be the only objects of experience. Even though people can estimate classical physical properties like length and distance, they might approximate them with perceived afordances. For example, perceived heaviness by dynamic touch maps onto perceived move-ability (Shockley et al., 2004). Other research has indicated that perceived afordances do not necessarily relate to relevant classical properties of the agent or environment alone, like length or width (Mark, 1987; Thomas & Riley, 2014; Wagman et al., 2013, 2018). Furthermore, a large body of evidence in the embodied cognition literature demonstrates the role of action, intentions, emotions, and metabolic capacity in perceiving classical properties (see Morgado & Palluel-Germain 2015 for a review). Nonetheless, I will not decisively claim that afordances are the only perceptual dependent variable since a large body of evidence suggests that classical properties are validly and reliably reported. Also, non-afordance events are at least specifed by information and, therefore, perceivable. For example, agents can perceive the classical event of a ball rolling on a U-shaped surface, as Bingham (1995) described. This event is not an afordance. Nonetheless, all of its properties are specifed by information. In any case, it is reasonable to claim that measures of afordance perception are the only ecologically universal dependent variable, given their necessary role in the adaptive behavior of organisms. Therefore, afordances are epistemically primary, even if the perception of non-afordances is a special case for higher-level phylogenetic organisms. Why Is Afordance Any Better Than Stimulus? What Does a Theory of Afordance Suggest That Stimuli Cannot; How Has It Moved the Needle Past Gibson 1960’s Recognition of How Little We Can Defne Stimuli?

Gibson’s theory of afordances is fundamentally diferent from the concept of a stimulus. The term stimulus was initially used to denote the nervous system’s response to the presentation of an object or energy. The term has been used more broadly to describe any sensory, motor, or perceptual outputs resulting from presenting physical objects and energy as the life sciences and psychology, more specifcally, have evolved (Cassedy, 2008). As it pertains to perception and experience, the term stems from

Toward an Ecological Theory of Time

103

the assumption of organism-environment dualism that has dominated the study of experience for centuries: the notion that the phenomena in the environment are separate and fundamentally diferent from those experienced by the organism. The stimulus is the crossover between these realms, allowing scientists to understand how physical entities become psychological entities. On the other hand, afordances stem from Gibson’s assumption that the organism-environment relations are inseparable and fundamentally commensurate. The stimulus concept is unnecessary because there is only one class of phenomena within the ecological niche. Afordances do not require the activity of sensory systems to exist, and while the perception of afordances does require both energetic distributions and sensory apparatus, the individual responses of sensory systems are of less interest than the information that lawfully relates energetic distributions to the organismenvironment, which organisms have adapted diferent solutions to detect and exploit (Gibson, 1966a). There are many diferences between afordances and stimuli, but the focus in this chapter is on the relationship between each concept and time. Stimuli are short-lived phenomena. Often, stimuli exist on the timescale of milliseconds or even nanoseconds. This is an important scale of analysis, though it is far from representative of the scale of ordinary perceptionaction phenomena, which can occur on the timescale of minutes, hours, or even longer. Since they are defned on the scale of individual biological responses, the stimulus is here and gone. Its efect on the organism is thought to expire once the chain of biological responses terminates. For example, the presentation of light to the retina causes a chain of biological responses that travels from the retina through the central nervous system, and each neuronal response happens briefy and then dissipates. A storage system is required to carry the efect of the stimulus through time until the presentation of a similar stimulus can reactivate it. However, the timescale of the ecological niche is much larger than that of a stimulus, even when considering a stimulus’ journey through the whole nervous system, which is on the order of seconds. There are also larger timescales that are skirted by using storage as a construct. However, storage is only necessary if perception is considered time-limited. Rather than depending on the spatiotemporal scales of nervous system activity, afordances exist over larger and variable spatiotemporal scales on the order of ordinary perception-action. Nonetheless, they are also timelimited in a diferent sense. Afordances emerge and evaporate from existence as events continuously fow through time and space (see Chemero, 2003). The temporal constraints depend on the spatiotemporal scale of the organism-environment instead of the nervous system. While the concept

104

Brandon J. Thomas

of an afordance sidesteps some of the theoretical quandaries of a stimulus, scholars have yet to fully describe this temporal fow of afordances. For instance, an elementary school desk might have aforded sitting in 4th grade, but it has since been destroyed and no longer does. Still more, there are no satisfying accounts for the diference in experiencing afordances before they came into existence (e.g., before the chair was designed), while they exist (e.g., while standing in front of the chair in 4th grade), and after they are no longer possible (e.g., the chair no longer exists). In essence, ecological psychology lacks a description of the past, present, and future along with their epistemic counterparts: memory, perception, and anticipation. To understand afordances’ role in perception and experience more broadly, I will describe time at the ecological scale. Then, I will ofer an account of time that situates experience within the past, present, and future (i.e., the ecological frame). What Is the Connection Between Afordances, Behavioral Scale, and Intention? How Distant or Reluctant Can a PerceivingActing System Be Before Afordances Come Into Play? Do Afordances Exert Their Role Only When the System Is Close or Willing Enough?

Events at the ecological scale occur at a tiny proportion of the maximum speed of the universe (i.e., speed of light). Organisms are fxed in a stable gravitational feld amid our residential epoch. Thus, organisms on Earth do not encounter the relativity of spacetime (Einstein, 1915). Space and time are both linear and correlated at the ecological scale. We cannot teleport between locations, and locomoting farther takes more time. The important diference between space and time is that space is reversible, and time moves in the direction of entropy due to the second law of thermodynamics (Clausius, 1865). Thus, we can be in the same place again, but never simultaneously. As Newton, Einstein, and others have demonstrated, force, motion, and causality can be defned neatly within this spacetime (Einstein, 1915; Galilei, 1632/1967; Newton, 1687). The movement of most classical matter has a determinate past and future. Minkowski (1907) provided a geometric description of bodies moving through spacetime from the perspective of an observer (Figure 6.1). The past and future light cones expand from the present in a four-dimensional manifold (3 spatial and 1 temporal). The edges of the cones are determined by the speed of light, which dictates the potential for an observer to interact with another body (i.e., an event). The causal history of any observer is traced in a worldline, which is the actual history of events for an observer. Every event is represented as a segment of the worldline. Also, everything contained within the future and

Toward an Ecological Theory of Time

FIGURE 6.1

105

Minkowski space for an observer at a point in spacetime (collapsed from 4 to 3 dimensions: 2 spatial and 1 temporal for ease of demonstration). The observer is a point in spacetime. Time is on the y-axis. Space is on the x-axis. Observers have a causal past and future of events represented as a worldline (yellow). The present is a hyperspace and represents the spatial dimensions. The past and future light cones (here represented as triangles) contain all possible causes as constrained by the speed of light and the principle of locality.

past light cones could have interacted with the observer but didn’t. Most importantly, the present is a point. It exists for a moment that travels away immediately. Considering these realities, we should experience a “razor’s edge” of time with a past and future but no present (Gibson, 1966b). However, we do not experience time in this way. Our experience of time is inexplicable, given this description. We experience time in three distinct epochs: the past, present, and future. How can we experience a present if it comes and goes at the speed of light? Most extant theories suggest that organisms carry a representation of events from the past light cone with them. These theories suggest that organisms are time-restricted to the present, again based on common sense. This assumption is invalid. The nature of spacetime suggests that organisms expand in time. New laws are needed to address ecological time.

106

Brandon J. Thomas

Much work has been done on empirically validating the existence and perception of afordances. More recent work has focused on afordance perception dynamics, demonstrating that adaptive behavior depends on the ongoing, real-time detection of afordances. For example, Warren and colleagues ofered a theory of goal-directed perception-action known as information-based control (Fajen, 2007; Warren, 2006; Warren et  al., 2001), building on the work of Lee (1976). Warren depicts the organismenvironment as a pair of informationally and physically coupled dynamical systems that produce emergent behavior. These models have successfully explained various dynamic behavioral phenomena, including obstacle avoidance (Fajen & Warren, 2003) and crowd dynamics (Warren, 2018). Fajen (2007) reformulated Warren’s model into an afordance-based control model, expressing the information for perception-action as a function of action capabilities (i.e., the boundary between impossible and possible actions). This transforms arbitrary extrinsic units (properties specifed by information that is about just the environment but not about the organism) into meaningful intrinsic units. Thus, organisms behave to keep actions aforded. For example, braking to avoid collisions can be described intrinsically as braking to keep avoiding a collision possible. Once the boundary between possible and impossible actions is crossed, collision is inevitable, and the braking event is fxed in the past. Afordance-based control ofers an essential insight into the nature of events that is partially consistent with Chemero’s (2003) description of events as the changing layout of afordances. As time evolves, afordances emerge and dissolve from existence at the speed of action. However, there is no general account of the event structure of afordances and actions. Almost all work on the dynamics of afordances and afordance perception focuses on the present moment. How long is the present? Is it a constant period, or does it vary depending on the task? What are the constraints of afordances and actions that are spatiotemporally remote, whether in the past or future light cone? How do we remember past events? What are the infuences of events far into the future? These fundamental questions need to be reckoned with for the ecological approach to account for the whole of temporal physics and experience. How Do Systems Engage With Afordances as They Move Among Tasks and Intentions?

I begin with the following assumptions: Gibson described perception as extended in time (Gibson, 1966a, 1966b; Gibson et al., 1969). I will generalize this point and suggest that (i) all experience is extended through time; (ii) ecological time is a physical/natural theory of time at the ecological

Toward an Ecological Theory of Time

107

scale, and (iii) ecological time can be described using physical law, so time is linear, unidirectional, and irreversible. Space is linear, multidirectional, and reversible. These assumptions are not novel. From here, I introduce new laws and principles that draw on extant work and that apply to the special case of time at the ecological scale. The Spacetime Principles

The frst fve laws defne the spacetime of the ecological scale. Law #1: Ecological Events

Ecological time consists of spacetime events called ecological events (events). Events are reciprocal organism-environment interactions (i.e., perception-action) that are propertied. All these properties, along with the events themselves, are specifed and perceivable. This description draws heavily from Bingham and colleagues’ work, who provided well-specifed spacetime descriptions of ecological events (Bingham, 1995; Bingham et al., 1995). Events and event properties are specifed by information as described earlier (Gibson, 1979). Lee (1976) and, later, Runeson and colleagues also made signifcant contributions to the term (Runeson, 1994; Runeson & Frykholm, 1983). Events structure several energetic distributions that are detectable through multiple modalities simultaneously (Stofregen et  al., 2017; Stofregen & Bardy, 2001), including chemical and hormonal arrays in the nervous and endocrine systems (Witt & Riley, 2014). I will ofer a similar description of the time course of ecological events that is consistent with Shaw and Kinsella-Shaw’s (1988) depiction of ecological mechanics, which describes the energetic symmetry relations of information and action in goal-directed action. However, the focus here is on the spacetime geometry and how it relates to experience. Events correspond to the spacetime interval of actions (Eq. 1), which are defned by the reciprocal forces exchanged by the organism-environment in a defned epoch. Events begin when the organism and environment exchange energy (start of an action: As). Events end when energy has been conserved (end of an action: Ae). Ta = Ae – As,

(1)

where Ta is the actual time of an event, Ae is the end of an action/event, and As is the start of an action/event as measured by a standard clock. Eq. 1 places a new constraint on spacetime, which is not included in classical

108

Brandon J. Thomas

physics. The speed of light is not the speed limit at the ecological scale. The speed of light is replaced with the speed of action (Fajen, 2007). Instead of a light cone, ecological events have an action cone. Furthermore, ecological events have an extended presence that occurs several orders of magnitude slower than light. Still more, the present contains events slightly into the future that are superposed and specifed by information. Thus, the hyperspace of the present must be replaced by the hyperrectangle of the present. Finally, Law #10 dictates that the future action cone is indeterminate. Thus, we need an ecological Minkowski space (Figure 6.2), which is in line with Shaw and Kinsella-Shaw’s (1988) Ω cell.

FIGURE 6.2

Ecological Minkowski space. It is equivalent to Figure 6.1, except for the following. An organism replaces the observer. The variable of the speed of action determines the geometry of spacetime instead of the constant of the speed of light. Thus, it’s far narrower (not to scale) than cosmological spacetime. It is also variable over time, which produces a choppy spacetime. Also, the present becomes three-dimensional because it occurs at the speed of action. This produces a perceivable future of afordances specifed and described by the spacetime principles. It becomes a hyperrectangle, though its shape might difer. Finally, the action cone of the future is inherently indeterminate because the near future is solely specifed, and the forces of organisms trace a stochastic worldline. All of these hypotheses are testable.

Toward an Ecological Theory of Time

109

The geometry is determined by the actual speed of an event (Ta). The path an organism traces through the action cone is its worldline. The action cone includes all possible events that could have occurred at the speed of action, and the worldline is the actual history of events. The geometry of the action cone is an open empirical question, but the animal movement literature ofers predictions (Shaw, 2020). Organisms comprise nested efector systems (Bernstein, 1967; Turvey & Fonseca, 2014). Therefore, events are discrete, nested, and additive. This might seem counterintuitive since the action system is highly nonlinear (Riley & Turvey, 2002; Turvey, 1990). It also may seem strange to suggest that events are discrete, given that energy fows through the organism-environment are continuous. On the other hand, events can be considered discrete since energy is always conserved. However, time at the ecological scale is linear because the niche on Earth is in a stable gravitational feld, and ecological forces are not strong enough to afect time, though unstable gravitational felds would still ft into this description as the general rule. Each action is part of a chain of actions and sub-actions that evolve in spacetime. This is consistent with the work of Wagman and colleagues (Wagman et al., 2019; Wagman & Stofregen, 2020). The granularity of events is an open empirical question, as is the granularity of the perception of events and event properties. The nestedness of events is given by Eq. 2. Ta = Σti,

(2)

where ti is each sub-event within event Ta. Zacks and colleagues might ofer some clues to this puzzle (Zacks & Swallow, 2007). Law #2: The Ecological Present

Events are in the present, while perception-action occurs concurrently with Ta and slightly into the future since afordances are future actions and are specifed in the present. Because the present is fnite and the past has a fully specifed history, time is asymmetric. Organisms have a fxed worldline of events up to the present. Organisms have an indeterminate future that has yet to happen beyond the present. In essence, afordances are superposed nonlocal states, as Turvey (2015) described. It stands to reason that the information might exist, which would be an unclassifed force particle carried by low-energy, ambient distributions. These particles would have to (1) be carried by energy distributions that are structured by the environment and thus be ubiquitous, (2) interact with low energy distributions and perhaps phase transition at

110

Brandon J. Thomas

greater energetic magnitudes, (3) be massless since they must travel as fast as light (which is why distant stars are visible), (4) have a negative/repulsive charge, since perceptually guided action is repelled through a phase space by information like a magnet (Warren, 2006), and (4) collapse the wave function (Schrödinger, 1926) for the afordance that is actualized, perceived, and occurs in the relative past of the time-expanding self. Law #3: The Ecological Past

The past begins once an action ends (< As). That is, once the current action is no longer possible because of Law #1. The past is a fxed, unchangeable series of events that evolves and grows as the worldline stretches through time. Law #4: The Time-Expanding Self

The history of an organism is ever-evolving at the present edge of its worldline. Thus, the current organism is the history of its interactions with the environment. Organisms are expanding over time. The self rides atop an ever-expanding worldline. The self does not end with Ae. It had existed, so it continues to exist forever. It was also perceived, so it will always be perceived for the organism with sensory-motor adaptations. I will return to this in the agency principles. Law #5: The Ecological Future

The future is the set of possible events after the present (> Ae) and above the hyperrectangle of the present. The future is inherently open to any possible events within the action cone, which is ever-expanding. Because organisms produce stochastic forces, the future events become less predictable further a worldline expands into the future. Consequently, the future is indeterminate. The Agency Principles

The following principles refer to the experience of events and the principles therein. They could also be described as epistemological principles because they depict the origin of experience. However, I prefer the term agency because it highlights the role of organisms interacting with the environment. The experience of the world is a byproduct. I also do not mean to suggest that agency is non-physical. Every principle herein is about

Toward an Ecological Theory of Time

111

the physical universe. Agency/experience is perception-action in a timeexpanding organism. Law #6: Law of Direct Perception

Perception-action is the only means to become conscious of events. Any property of an ecological event is perceivable. This law encompasses Gibson’s (1966a) description of a perceptual system, which includes the organs and efectors used to continuously perceive to act and acts to perceive. Organisms continuously control their actions with respect to the boundary between possible and impossible actions (Fajen, 2007). As a result, the afordance boundary/limit (Fmax) is the spacetime limit of afordances.1 Law #7: Ecological Time

The time of events is perceivable. Perceived time of events is specifed and perceived as the maximum possible speed that an action can be completed. In other words, we perceive events with respect to Fmax. Tp = %Fmax × Ta,

(3)

where Tp is perceived time, %Fmax is the percentage of the fastest possible speed that an action can be completed by an organism. Since events normally occur at less than 100% of maximum speed (< Fmax), organisms should generally underestimate the time of events, unless they actualize the afordance at max speed, which isn’t common. Perception of nested sub-actions is additive so long as the action approaches FMax (Eq. 2). Thus, as %Fmax increases the relationship between Ta and Tp increases. The inverse of this principle explains a commonsense aspect of time perception. As we attend to more granular Ti, we will underestimate Ta more (Figure 6.3). Space co-emerges with time. This is clear from examining the action cone (Figure 6.2), so ecological space is non-linear. The perception of space is constrained by Law #7. In other words, spacetime is determined by Fmax. This might account for the observed phenomena within the action-specifc perception literature (Proftt & Linkenauger, 2013; Riley et al., 2007; Witt, 2011), which depicts the infuence of action ability on the perception of classical properties. Any constraints on Amax will constrain the present spacetime and thus explain the infuence of action potentiality on spatial perception. For example, it will take longer to travel a distance with an encumbrance than without because Fmax will be longer, so it should be perceived as farther.

112

Brandon J. Thomas

FIGURE 6.3

Visual depiction of the co-additivity of events. Ta is given by Eq. 1. Tp is given by Eq. 3. Each number represents the end of an actual event (solid) or perceived event (dotted). Since organisms rarely move at Fmax, events are normally perceived to be shorter than they are. Thus, perceived time is compounded by the proportion of the granularity of events and the length of the over-arching event in accordance with Eq. 2.

Law #8: Ecological Memory

Once FMax is crossed, and action is no longer possible (< Ae), an event becomes remembered. Our experience of events beyond FMax is ecological memory. This memory does not involve the storage and retrieval of information. Because we are time-expanding beings, our worldline is always perceivable. However, in the future, the action cone expands, and events become more difcult to discriminate from each other as well as from other possible events (i.e., perceived afordances that were not actualized) as expressed by Eq. 3. In other words, because multiple afordances were perceived in the present, events that might have happened in the past action cone can be remembered in addition to actual events on the worldline. As discussed earlier, contemporary theories of mind and brain assume a time-restricted organism. These theories suggest that memories are stored as traces for later retrieval, and other similar traces are more likely to interfere with them as time proceeds. On the other hand, a time-expanded organism suggests that events in the past action cone must be discriminated from past events or potential past events in real time. As the action cone—which includes an ever-increasing number of potential events—expands into the past, it becomes ever more difcult to discriminate past events from each other and potential past events. To put this another way, a worldline in the past is ever more confusable with itself and other events in its action cone. This theoretical approach is logically consistent with the predictions

Toward an Ecological Theory of Time

113

of the interference theory of memory (Underwood, 1957), which suggests that memories in a storage system interfere with each other over time. Both theories require that remembered events be discriminated, except I posit that all perceived events continue to exist on the worldline as part of the self, rendering storage unnecessary. Memory is a weak force of experience on perception-action, while development (see Law #9) is a strong force. Law #9: Ecological Development

Each event occurs after an ever-expanding series of events that trace the worldline. Eleanor J. Gibson and colleagues did pioneering work in development and learning (Gibson, 1969, 1988; Gibson & Pick, 2000; Thelen et  al., 1991; Thelen & Smith, 1996). These events trace out the everexpanding worldline of an organism due to Law #5. Events are stacked on top of each other in time. The present state of development is at the top of this stack. That is, the present state of development is the current state of an ever-expanding, perceiving-acting organism. Development is a strong force of experience since it fully determines the present state of the organism. The present moment is always the top of the worldline, so in a sense, the perception-action system is its current state of development. Throughout development, organisms continuously adjust to the dynamic fow of events that make up their worldline. Surprisingly, learning is a principle rather than a law because it obeys Law # 8 (Adolph, 2019). The state of learning in the present refects the current state of the ever-expanding self. This principle accommodates the theory of direct learning (Jacobs & Michaels, 2007), which depicts an organism’s path through a vector space of information variables and the dynamics of converging on specifying information over time and with experience. Law #10: Ecological Anticipation

The future is the time after the current event (> Ae), so it is entirely in the future action cone and beyond the hyperrectangle of the present. The future is weakly specifed. That is to say, the local physics of the present has potential worldlines that organisms are sensitive to within the hyperrectangle of the present (Figure 6.2). The future is indeterminate. It has yet to happen and is hitherto unknown. It is subject to unpredictable forces, as described by Law #5. The action cone very quickly expands so that the future is open with possibilities. Though organisms detect information in the present due to Law #7, future events are perceived, and perception-action is about these events.

114

Brandon J. Thomas

The prospective control of action depends on detecting information that specifes afordances. Thus, organisms detect information about the future during the present. The temporal extent that future events can be specifed is an open empirical question (see Matthis et al., 2017), though the taskspecifc nature of events suggests that this extent (Tp) is variable. Concluding Remarks

I have given an overview of afordance theory emphasizing the dynamics of perception-action. From there, I ofered an explanation of ecological time that rests on known physical principles and the basic assumption that perception-action is fundamental. The theory reorganizes extant theories and concepts regarding experience and introduces several new concepts that require empirical inquiry. Its details will require revision with empirical and theoretical scrutiny. Lastly, ecological time also ofers many new empirical questions to be addressed. These puzzles allow the opportunity to absorb the excellent work in multiple, separate disciplines and approaches while ofering numerous avenues for theory refnement and advancement. I hope that this discussion will begin a meaningful conversation about time. After all, the future is ours to determine. Acknowledgments

This chapter is a culmination of more than a decade of work, so there are many strong forces in my development to shout out! This project began ten years ago with a conversation after a conference with Jef Wagman when I was a master’s student at Illinois State University. Thanks to Jef for being a great mentor and friend throughout my career. Thanks to my Master’s advisor Dawn McBride for mentoring me in my early career in memory research and teaching me all the basics of experimental research. Thanks to Scott Jordan for all your hospitality, mentorship, and kindness. Thanks to all the folks in the CAP center and the PMD lab at the University of Cincinnati for fostering a collaborative, stimulating, and mad science-like environment. In particular, thanks to my Ph.D. advisor Mike Riley: who encouraged and supported me in my research while always challenging me to develop my skills as a scientist. Thanks to the folks of VSPC Lab at the University of Utah. In particular, thanks to my postdoc advisors, Jeanine Stefanucci, Sara Creem-Regehr, and Jon Butner, who warmly welcomed me into their group and helped me expand my research program as both mentors and collaborators. Finally, thanks to all the colleagues and students I have mentored and/or collaborated with. In particular, thanks to Matt Hawkins for your hard work, collaboration, and friendship.

Toward an Ecological Theory of Time

115

Note 1. It is important to note here that Fajen (2007) did not claim that maximum action capabilities are ubiquitously perceived. Fajen cited the possibility that afordances below the max are specifed and potentially perceived. The claim that FMax is perceived is a unique claim of this chapter.

Reference List Adolph, K. E. (2019). An ecological approach to learning in (not and) development. Human Development, 63(3–4), 180–201. https://doi.org/10.1159/000503823 Bernstein, N. A. (1967). The Co-Ordination and Regulation of Movements. Pergamon Press. Bingham, G. P. (1995). Dynamics and the problem of visual event recognition. In R. F. Port & T. van Gelder (Eds.), Mind as Motion: Explorations in the Dynamics of Cognition (pp. 403–448). MIT Press. Bingham, G. P., Schmidt, R. C., & Rosenblum, L. D. (1995). Dynamics and the orientation of kinematic forms in visual event recognition. Journal of Experimental Psychology: Human Perception and Performance, 21(6), 1473–1493. https:// doi.org/10.1037/0096-1523.21.6.1473 Cassedy, S. (2008). A history of the concept of the stimulus and the role it played in the neurosciences. Journal of the History of the Neurosciences, 17(4), 405–432. https://doi.org/10.1080/09647040701296861 Chemero, A. (2003). An outline of a theory of afordances. Ecological Psychology, 15(2), 181–195. https://doi.org/10.1207/S15326969ECO1502_5 Clausius, R. J. E. (1865). Vorlesungen über Mechanische Wärmetheorie. Niedersächsische Staats-und Universitätsbibliothek. Einstein, A. (1915). Die Feldgleichungen der Gravitation. Sitzung Der Physikalische-Mathematischen Klasse, 25, 844–847. Fajen, B. R. (2007). Afordance-based control of visually guided action. Ecological Psychology, 19(4), 383–410. https://doi.org/10.1080/10407410701557877 Fajen, B. R., & Warren, W. H. (2003). Behavioral dynamics of steering, obstacle avoidance, and route selection. Journal of Experimental Psychology: Human Perception and Performance, 29(2), 343–362. https://doi.org/10.1037/0096-1523.29.2.343 Galilei, G. (1967). Dialogue Concerning the Two Chief World Systems. University of California Press. Gibson, E. J. (1969). Principles of Perceptual Learning and Development. Appleton-Century-Crofts. Gibson, E. J. (1988). Exploratory behavior in the development of perceiving, acting, and the acquiring of knowledge. Annual Review of Psychology, 39, 1–42. https://doi.org/0.1146/annurev.ps.39.020188.000245 Gibson, E. J., & Pick, A. D. (2000). An Ecological Approach to Perceptual Learning and Development. Oxford University Press. Gibson, J. J. (1966a). The Senses Considered as Perceptual Systems. Houghton Mifin. Gibson, J. J. (1966b). The problem of temporal order in stimulation and perception. Journal of Psychology, 62(2), 141–149. https://doi.org/10.1080/0022398 0.1966.10543777

116

Brandon J. Thomas

Gibson, J. J. (1979). The Ecological Approach to Visual Perception. Houghton Mifin. Gibson, J. J., Kaplan, G. A., Reynolds, H. N., & Wheeler, K. (1969). The change from visible to invisible. Perception & Psychophysics, 5(2), 113–116. https:// doi.org/10.3758/BF03210533 Jacobs, D. M., & Michaels, C. F. (2007). Direct learning. Ecological Psychology, 19(4), 321–349. https://doi.org/10.1080/10407410701432337 Lee, D. N. (1976). A theory of visual control of braking based on information about time-to-collision. Perception, 5(4), 437–459. https://doi.org/10.1068/ p050437 Leitan, N. D., & Chafey, L. (2014). Embodied cognition and its applications: A brief review. Sensoria: A Journal of Mind, Brain & Culture, 10(1), 3–10. https:// doi.org/10.7790/sa.v10i1.384 Mark, L. S. (1987). Eyeheight-scaled information about afordances: A study of sitting and stair climbing. Journal of Experimental Psychology: Human Perception and Performance, 13(3), 361–370. https://doi.org/10.1037/0096-1523.13. 3.361 Matthis, S. J., Barton, S. L., & Fajen, B. R. (2017). The critical phase for visual control of human walking over complex terrain. Proceedings of the National Academy of Sciences, 114(32), E6720–E6729. https://doi.org/10.1073/ pnas.1611699114 Minkowski, H. (1907). Diophantische Approximationen: Eine Einführung in die Zahlentheorie (Vol. 2). Druck und Verlag von B. G. Tuebner. Morgado, N., & Palluel-Germain, R. (2015). How actions constrain the visual perception of space. In Y. Coello & M. H. Fischer (Eds.), Perceptual and Emotional Embodiment (pp. 175–188). Routledge. Newton, I. (1687). Philosophiæ Naturalis Principia Mathematica. Proftt, D. R., & Linkenauger, S. A. (2013). Perception viewed as a phenotypic expression. In W. Prinz, M. Beisert, & A. Herwig (Eds.), Action Science: Foundations of an Emerging Discipline (Vol. 171). MIT Press. Reed, E. S. (1996). Encountering the World: Toward an Ecological Psychology. Oxford University Press. Riley, M. A., & Turvey, M. T. (2002). Variability and determinism in motor behavior. Journal of Motor Behavior, 34(2), 99–125. https://doi.org/10.1080/ 00222890209601934 Riley, P. O., Paolini, G., Della Croce, U., Paylo, K. W., & Kerrigan, D. C. (2007). A kinematic and kinetic comparison of overground and treadmill walking in healthy subjects. Gait & Posture, 26(1), 17–24. https://doi.org/10.1016/j. gaitpost.2006.07.003 Runeson, S. (1994). Perception of biological motion: The KSD-principle and the implications of a distal versus proximal approach. In G. Jansson, S. S. Bergström, & W. Epstein (Eds.), Perceiving Events and Objects (pp. 383–405). Lawrence Erlbaum Associates, Publishers. Runeson, S., & Frykholm, G. (1983). Kinematic specifcation of dynamics as an informational basis for person-and-action perception: Expectation, gender recognition, and deceptive intention. Journal of Experimental Psychology: General, 112(4), 585–615. https://doi.org/10.1037/0096-3445.112.4.585

Toward an Ecological Theory of Time

117

Schrödinger, E. (1926). An undulatory theory of the mechanics of atoms and molecules. Physical Review, 28(6), 1049–1070. https://doi.org/10.1103/ PhysRev.28.1049 Shaw, A. K. (2020). Causes and consequences of individual variation in animal movement. Movement Ecology, 8(1), 12. https://doi.org/10.1186/s40462-020-0197-x Shaw, R., & Kinsella-Shaw, J. (1988). Ecological mechanics: A physical geometry for intentional constraints. Human Movement Science, 7(2), 155–200. https:// doi.org/10.1016/0167-9457(88)90011-5 Shockley, K., Carello, C., & Turvey, M. T. (2004). Metamers in the haptic perception of heaviness and moveableness. Perception & Psychophysics, 66(5), 731–742. https://doi.org/10.3758/BF03194968 Stofregen, T. A., & Bardy, B. G. (2001). On specifcation and the senses. Behavioral and Brain Sciences, 24(2), 195–213. https://doi.org/10.1017/ S0140525X01003946 Stofregen, T. A., Mantel, B., & Bardy, B. G. (2017). The senses considered as one perceptual system. Ecological Psychology, 29(3), 165–197. https://doi.org/10.1 080/10407413.2017.1331116 Thelen, E., & Smith, L. B. (1996). A Dynamic Systems Approach to the Development of Cognition and Action. MIT press. Thelen, E., Ulrich, B. D., & Wolf, P. H. (1991). Hidden skills: A dynamic systems analysis of treadmill stepping during the frst year. Monographs of the Society for Research in Child Development, 56(1), 1–103. https://doi.org/10.2307/1166099 Thomas, B. J., & Riley, M. A. (2014). Remembered afordances refect the fundamentally action-relevant, context-specifc nature of visual perception. Journal of Experimental Psychology: Human Perception and Performance, 40(6), 2361–2371. https://doi.org/10.1037/xhp0000015 Turvey, M. T. (1990). Coordination. American Psychologist, 45(8), 938–953. https://doi.org/10.1037/0003-066X.45.8.938 Turvey, M. T. (2015). Quantum-like issues at nature’s ecological scale (the scale of organisms and their environments). Mind and Matter, 13(1), 7–44. Turvey, M. T., & Fonseca, S. T. (2014). The medium of haptic perception: A tensegrity hypothesis. Journal of Motor Behavior, 46(3), 143–187. https://doi.org/10. 1080/00222895.2013.798252 Underwood, B. J. (1957). Interference and forgetting. Psychological Review, 64(1), 49–60. https://doi.org/10.1037/h0044616 Wagman, J. B., Lozano, S., Jiménez, A., Covarrubias, P., & Cabrera, F. (2019). Perception of afordances in the animal kingdom and beyond. In I. Zepeda, J. Camacho, & E. Camacho (Eds.), Aproximaciones al estudio del comportamiento y sus aplicaciones (pp. 70–108). Universidad de Guadalajara. Wagman, J. B., & Stofregen, T. A. (2020). It doesn’t add up: Nested afordances for reaching are perceived as a complex particular. Attention, Perception, & Psychophysics, 82(8), 3832–3841. https://doi.org/10.3758/s13414-020-02108-w Wagman, J. B., Thomas, B. J., & McBride, D. M. (2018). Perceiving and remembering afordances for others are continuous processes. Experimental Psychology, 65(6), 385–392. https://doi.org/10.1027/1618-3169/a000424 Wagman, J. B., Thomas, B. J., McBride, D. M., & Day, B. M. (2013). Perception of maximum reaching height when the means of reaching are no longer in view.

118

Brandon J. Thomas

Ecological Psychology, 25(1), 63–80. https://doi.org/10.1080/10407413.2013. 753810 Warren, W. H. (2006). The dynamics of perception and action. Psychological Review, 113(2), 358–389. https://doi.org/10.1037/0033-295X.113.2.358 Warren, W. H. (2018). Collective motion in human crowds.Current Directions in Psychological Science, 27(4), 232–240. https://doi.org/10.1177/0963721417746743 Warren, W. H., Kay, B. A., Zosh, W. D., Duchon, A. P., & Sahuc, S. (2001). Optic fow is used to control human walking. Nature Neuroscience, 4(2), 213–216. https://doi.org/10.1038/84054 Witt, J. K. (2011). Action’s efect on perception. Current Directions in Psychological Science, 20(3), 201–206. https://doi.org/10.1177/0963721411408770 Witt, J. K., & Riley, M. A. (2014). Discovering your inner Gibson: Reconciling action-specifc and ecological approaches to perception—Action. Psychonomic Bulletin & Review, 21(6), 1353–1370. https://doi.org/10.3758/ s13423-014-0623-4 Zacks, J. M., & Swallow, K. M. (2007). Event segmentation. Current Directions in Psychological Science, 16(2), 80–84. https://doi.org/10.1111/j.1467-8721. 2007.00480.x

PART II

The Role of Exploratory Activity and Scale in Perception of Afordances

7 THE DYNAMICS OF AFFORDANCE EMERGENCE AND PERCEPTION Matheus M. Pacheco and Ricardo Drews

The ecological psychology approach to perception and action promotes the concept of afordances as one of its key aspects, following James J. Gibson (1979). The statement that agents perceive the environment that surrounds them in terms of “possibilities for action” is of great importance for an approach that bases its ideas on an intertwined coupling of perception and action (see Turvey & Fonseca, 2014). In fact, afordances are a consequence of Gibson’s theory of direct perception: “Perhaps the composition and layout of surfaces constitute what they aford. If so, to perceive them is to perceive what they aford” (p. 119). Afordance is also a hard concept to defne as authors refer to it either in practical but not-so-precise terms or through precise but not-so-agreedon formalisms (see Adolph, 2019; Stofregen, 2003; Turvey, 1992). Maybe because of this dispute, how afordances emerge/disappear and are perceived through fast environmental changes to (slow) learning, development, and aging has received far less attention. In this chapter, considering some basic assumptions from motor behavior and evidence from motor learning and development, we will argue on divergences between afordances and their perception. If that is the case, these incongruencies between afordance existence and perception require further theorizing to be explained. From this, we develop a framework on afordance perception dynamics, expanding our current work on motor learning and development. This view borrows fndings (and views) from social-afective theories in psychology, a possibility that has yet to be considered. Despite its speculative nature, this view demonstrates the current DOI: 10.4324/9781003396536-10

122

Matheus M. Pacheco and Ricardo Drews

state of knowledge on afordance perception dynamics but might provide directions of inquiry tied to other domains more established in developmental (and social-afective) psychology. What Do We Understand by the Term Afordance? The Dynamic Nature of Afordances

It is commonplace to read that afordances are “dynamic” (e.g., Chemero, 2003; Fajen et  al., 2009). Let us accept Stofregen’s (2003) defnition of afordances (see also Chemero, 2009): afordances are “properties of the animal-environment system that determine what can be done” (p. 124). An afordance emerges from an animal’s ability to perform a given action and the environmental features that support such action. Whenever support or ability changes, an afordance might emerge or disappear. In a situation of a soccer player intending to pass the ball (see Araújo et al., 2006), assuming that the soccer player is skilled (can pass the ball to his teammates), the dynamics of his teammates and adversaries will modify how the environment supports passing. A sufcient gap between two adversaries (and an appropriate position of a teammate) allows passing through them; small diferences in the adversaries’ positions eliminate such possibility (see Corrêa et al., 2016). Another example, when waiting to cross a street, the cross-ableness of the street (without being hit by a car) changes as cars accelerate or break. Emergence and dissolution of afordances can also occur, given changes in individual capabilities at other timescales. Warm-up and fatigue (see Newell et al., 2009) are transient changes in behavior that usually result in performance change. While the former (improvement) relates to postural and movement adjustments to familiarization/attunement to the task requirements, the latter relates to decrements in performance given decreased attention and capacity to control movements given long exposure to the task. These diferent dynamics, by defnition, modify when an individual can perform an action. If an individual needs to jump over a large gap, run to catch a bus, or perform any activity that requires wellperformed movement patterns, a fatigued “body” limits (or modifes) the movements that one can perform (e.g., Côté et al., 2008) and might not ft what the environment supports. Motor learning and development are other processes that infuence the emergence and dissolution of afordances. As an individual learns how to perform a given action (e.g., jumping over the fence), an environment that supports such action to the given skill level (e.g., fences that have the appropriate height for the given individual) and this individual action capacity

The Dynamics of Afordance Emergence and Perception

123

constitute an afordance. That is, the underlying idea is that when an individual demonstrates a new movement pattern or increases the capacity to attend to task demands in a given movement pattern, he/she demonstrates an afordance. The possibility to learn a skill requires the possibility for an afordance to exist, and as learning occurs, an afordance is demonstrated to exist. The possibility to act is a function of structural and functional development constraints supporting such actions (Newell, 1986). An individual reaching elderly age, despite the continuous practice of a given action, will fnd the same environment unsupportive as strength, fexibility, and other physical attributes are modifed and decrease his/her action capabilities (see Konczak et al., 1992). What Role Does Afordance Play in Perception? Is It the Entity Organisms Perceive or the Means Through Which Organisms Perceive? Are Afordances the Only Perceptual Dependent Variables? Afordance Perception, Afordances, and Their Actualization

The dynamics of afordance emergence (and dissolution) is just a matter of the relationship between agent and environment supporting behavior. Nonetheless, there is more than what has been discussed so far between afordances and their actualization. In this section, we discuss some aspects of perception and action that must be addressed for afordances perception dynamics to be fully explained. The Stochastic Nature of Afordances

Let us frst contemplate the level of detail of an afordance. When an individual perceives an afordance, to which level of detail does the environment support specify the details of the action to be performed? Considering the relational view of afordance assumed here (Chemero, 2009; Stofregen, 2003), how detailed is the possible action in this relation? If environmental support is overly specifc to a given action to be performed, the perceived afordance might not be actualized as one will never reach such specifcity. For instance, let us say that one creates a situation in which the only way for an individual to, with an overarm throw, hit a target is for him/her to throw a ball with a release velocity of 5.9456, 5.253, and 0 m/s in the z, y, and x axes, respectively. Not considering the release position, an individual can hardly perform what we want with such a level of detail, even if he/she is able to throw in this exact manner in one trial or

124

Matheus M. Pacheco and Ricardo Drews

FIGURE 7.1

Demonstration of inherent variability of the motor system. When confronted with a task (a), what are the action counterparts considered a given environmental support that exist when perceiving an afordance? It cannot be in terms of specifc details of the action: (b) shows an example when a task is overly constraining, a single solution is allowed. (c) shows hypothetical data demonstrating the usual variability that individuals demonstrate in a throwing task: large variation along the goal space of possibilities but some variation orthogonal to it (a.k.a. a “task-synergy”). We argue (see text) that individuals learn enabling competencies that, through afordance overestimation, allow further enabling competencies to be learned: (d) shows the preparatory movement of the overarm throw which is necessary for individuals to learn the (e) trunk rotation (frst pelvis, then thorax) that allows further throwing distance.

another in a less constrained task. This is made impossible, given the inherent variability of the motor system (Figure 7.1a–c; see Newell & Corcos, 1993; Riley & Turvey, 2002). One solution might be to say that, in perceiving an afordance, individuals relate a family of solutions that ft the task and environmental constraints: a “task-synergy” (Figure 7.1c; Latash et al., 2007; Wilson et al., 2016). For instance, in throwing a ball to hit a target, one could, for each

The Dynamics of Afordance Emergence and Perception

125

throw, change the release velocities and position while maintaining the ball landing at the exact location. The fact that this does occur for an extensive range of tasks (e.g., Black et al., 2007; Cohen et al., 2012; Domkin et al., 2005) could be considered as evidence for afordances being a relation of the family of movements that reach a given goal and the environment that supports such family. This would mean that over-constraining a task (or having an overly specifc intention for action) would result in a lack of afordances. The individual is not able to perform such a task (even if he/ she might be able to perform movement patterns that vary “around” it). Note that even if we agree with perceiving a family of possibilities (that fulfll the given intention), throwing a ball to a target will always come with some “undesired” variability: variability that results in an error. The subsystems that compose the movement system are themselves quite variable (e.g., neural activity, muscle recruitment, feedback loops) and, despite large coupling (“enslavement,” see Kelso, 1995) and dissipation of variability from lower to higher levels of analysis, variability in the outcome remains. Then a question raised is whether this resulting variability in the outcome is “considered” in relation to the supportive environment. Do individuals encompass an allowed error term in perceiving afordance? If one intends to pass a ball to a teammate, am I looking for a pass-able situation that will result in an afordance of at least 80% success rate? This issue of variability in actions results in questions for characterizing the afordance itself. When there is no 100% success rate in an action considering a given “supportive” environment, is it to say that there is no ft? Is the ft probabilistic?1 An exciting point rarely discussed is whether afordance perception relates to time spent performing a given action. Consider for now the walking difculties that elderly individuals present; is it possible that they perceive shorter distances as walk-able while not perceiving longer distances as possible? This could relate to the issue of variability discussed earlier. That is, the system might be able to maintain activity as long variability does not push the system out of this given movement solution; after some time, it is better to stop movement and restart to maintain the initial movement pattern (see Brakke & Pacheco, 2019). It can also be that fatigue and ftness prevent the maintenance (or continuing) of a given activity; do individuals perceive the number of hits or repetitions that they can make (e.g., “I can perform ten push-ups but not 11”)? Would this be a judgment of a diferent kind? Despite seeming overkill, the questions raised in this section must be considered if motor behavior is genuinely considered in a theory of afordances (and their perception). More important, as motor learning and

126

Matheus M. Pacheco and Ricardo Drews

development include changes from missing to succeeding (more than not but less than always), with variability (functional or not) having a large role, such issues must be considered. Do We Ever Perceive an Afordance Existence?

Ecological psychology, in its beginnings, emphasized vision2 and, subsequently, visually driven information. This has led researchers to highly deemphasize the need for other perception modes for an afordance to be perceived. Further, researchers have put too much faith in how vision information specifes an afordance. An exciting series of experiments by Adolph and colleagues demonstrates the need for other sources of information (e.g., Adolph et al., 2010; Lo et  al., 1999). In their classical experiments on toddlers’ decisions to descend on slopes (of diferent degree magnitudes), they manipulated the friction of the ground to see whether children and adults appropriately perceived the afordance of walking down slopes. In all studies, children failed to avoid descending on slippery slopes and fell. This has been explained as children using vision as their primary source of information; they only explored through other perception systems (i.e., touch) when slopes were too slanted (even for high friction conditions) or when they had contingent information (distinct surface) (Adolph et al., 2010). Adults also failed to attend to correct information as they were highly biased toward visual information (e.g., ground shine; Joh et al., 2006, 2007). Environmental support in low/high friction conditions can be perceived by exploring the surface friction through touching (even if it is needed for every step). However, there are situations where environmental support is only guaranteed after the action was performed. Consider a bridge to be crossed, which looks fragile. Visual information of the bridge, touching, or even “testing” the stifness will not determine whether the bridge is crossable as none of the information that can be derived from these sources will fully specify environmental support. This bridge will be said to have demonstrated afordance after supporting someone crossing for “this attempt of this person to cross in this velocity with this amount of weight.” One cannot even be sure that the bridge will sustain the next crossing of the same person as perturbations to its structure on the “going” might have deteriorated the support for “coming back.” The frst question is whether it is ever possible to perceive the afordance “in full” (prospectively). If there are always aspects of the environment that cannot be perceived until after the action, then it is questionable that one could perceive all aspects that specify the environmental support and, thus, the afordance. If it is not true that in all situations, there are

The Dynamics of Afordance Emergence and Perception

127

aspects that cannot be perceived until after the action, then the second question is raised: do individuals perceive all aspects that specify environmental support? Adolph’s studies seem to demonstrate that individuals attend visually to given aspects of the environment and use these to guide action—even when they do not specify the environmental support for the action. The use of non-specifying information has been demonstrated (Jacobs & Michaels, 2007; Rop & Withagen, 2014; Withagen & Van Wermeskerken, 2009), but its implications were deemphasized in arguing for fast convergence to specifying information (Jacobs & Michaels, 2002). Adolph and colleagues’ studies on adults seem to argue against this fast convergence. The Incongruence Between Dynamics of Afordance Existence and Its Perception

If to act, we must perceive an afordance, then the literature3 demonstrates several situations in which individuals fail to do it properly. There are two ways in which the literature points to some misperception of afordances. The frst is the problem of acting (or judging action possible) when there is no supporting environment for action or action capacity is limited. The second is the opposite: when afordances exist, but individuals judge an action as impossible. Motor development literature, mainly in the works of Karen Adolph and colleagues (see Adolph, 2019, for review), has demonstrated the learning that occurs through experience and development. Infants who learn locomotion patterns become better in diferentiating environments that support their action over months of experience. This has been demonstrated in several behaviors (e.g., locomoting down/up slopes—Adolph et al., 1997; crossing gaps—Adolph, 2000; passing bridges—Berger et al., 2010). The important part for us here is that there are situations that, after a child learns a mode of locomotion, despite no supportive environment, the child still attempts to act. Adolph et al. (1997) demonstrated that even after ten weeks of crawling/walking, babies were still attempting (in half of the trials) to go down risky slopes (slopes beyond their maximum descending slope). This has been described as “infants do not perceive afordances when they frst acquire a new skill in development” (Adolph, 2019, p. 8). Only after a long period (weeks or months) the child starts to modify how he/she acts in non-supportive environments. Thus, there is a moment when there is no afordance, and a child still attempts to act. This overestimation of capabilities is more than just relegated to infants and toddlers. Older children and adults rate their skills above their own

128

Matheus M. Pacheco and Ricardo Drews

capacities in tasks in which they were not experts (e.g., Peker et al., 2021; Whitaker et al., 2020). For instance, in an experiment comparing gymnasts and non-athletes, in a seven-year to eight-year-old group, Peker and colleagues showed that the non-athlete children overestimated their capacity to jump vertically and horizontally while the same did not occur for the gymnast group. Similar overestimation occurred for college-aged students in stepping and leaping tasks (Day et al., 2015). Peker et  al. (2021) showed that young gymnasts needed to be bettertuned to their movement possibilities as well. Contrary to non-athletes, nonetheless, these children underestimate their capacities in both jumping tasks (horizontal and vertical). Interestingly, their underestimation occurred with a higher magnitude for the non-habitual task of jumping vertically to reach a ball. One could relate the underestimation to experience, as Whitaker et  al. (2020) showed a similar underestimation of movement capacities for more experienced climbers. However, the issue seems more complex. For instance, Cole et  al. (2013) found that college-aged students underestimated their maximum leaping capacities (while underestimating other tasks). This contrasts Day et al. (2015), who studied the same age participants in the same tasks and showed a tendency to overestimate. Indeed, fast adjustment of estimates is seen for studies that asked participants about their judged capacities before and after performing the action (e.g., Cole et  al., 2013; Franchak & Adolph, 2014). Some authors have even argued for almost instantaneous adaptation (no need for practice or feedback) when individuals have altered their body dimensions or action capabilities (see Fajen et al., 2009). Nonetheless, considering the previous studies discussed with toddlers, who take a long time to appropriately perceive the afordance, and experienced individuals who, despite continued practice, fail to be accurate in their estimates, we see the possibility for different time scales of change on estimates’ adjustment. How Do Systems Engage With Afordances as They Move Among Tasks and Intentions? Relating Afordance Perception in Learning4

In summary, afordance perception has diferent dynamics from afordance existence, and the disparity is observed over an individual’s age and experience in a task. What we aim to do in this last section is to speculate on a possible route to understand the issue considering the social-afective literature as a path.

The Dynamics of Afordance Emergence and Perception

129

Afordance Perception as a Function of Enabling Competencies

The frst thing is that there is an adaptive role for this afordance existenceperception incongruency. Exploring environment and action ft is the only way to experience whether there is a ft and fnd solutions to new problems (create a new ft)—as they often arise. Learning would involve attempts to act in unknown “territory.” As limits to action capabilities and identifcation of more helpful information to attend to are found, individuals would demonstrate more stable performance and be able to switch actions or avoid mistakes whenever the environment-agent capabilities ft is not there. We have been working on a general idea of how learning occurs in development, emphasizing the connections between early competencies to new ones.5 This idea (which is still underdeveloped) considers how learned perception-action couplings allow and base discoveries (new perception-action opportunities emerge; Figure 7.1d,e). This has been primarily focused on usual motor skill dependencies observed after seven years of age (see dos Santos et al., 2022; Pacheco et al., 2022; Seefeldt, 1980). For instance, a child who learns how to “use” elastic energy from stretching (storing) and contracting (releasing) muscle/ligaments to project a ball from their hands (e.g., throw) will excel in tasks requiring projection of other objects (e.g., a dart), and this might basis tasks that require projecting objects using an implement (striking in tennis, for instance) (see O’keefe et al., 2007). This is usually observed as the delayed proximal-to-distal action in overarm throw (frst, the pelvis rotates, then thorax, then you have arm adduction, humerus rotation, and, fnally, the ball throw (Roberton & Halverson, 1984). It could be that such use of elastic energy (or, as referred, whip-pattern) supports the learning of projecting objects with other body parts (kicking; see Southard, 2014). The idea is that learning a frst movement “component” of a given task will allow learning components of new tasks that are based on this frst component. We have called these components “enabling competencies.” One can envisage enabling competencies, opening possibilities for new acquisition as a branching process. The branching possibilities are constrained by the dynamic resources that the environment/body supports (see Holt et al., 2010): to use elastic energy, one needs ligaments, muscles, and joints that favor stretching and contracting correctly as well as a medium (e.g., air) that permits movements to be performed fast enough for elastic energy not to be dissipated into heat. Note that one also needs to have “discovered,” in the case of throwing, preparatory movements (moving the arm away from the direction of the throw beforehand) that are, in themselves, enabling competencies.6

130

Matheus M. Pacheco and Ricardo Drews

The discovery of the enabling competencies would occur as a process of search that starts from the current movement capacities. The “tree” of enabling competencies would serve as the initial condition for learning (branching). The search process (see Pacheco et  al., 2019), in summary, is the motion through action possibilities and task requirements guided by the information that emerges from the motion itself (see Jacobs & Michaels, 2007). The process of search, thus, would be constrained/guided by the initial status of the perception-action possibilities of the learner (the “tree” of enabling competencies) and would lead the process of increase in perception-action possibilities (branching of the tree). Returning to afordance perception, consider that a child has enough competency to throw using the whip pattern. This implies a supportive environment for this movement pattern to “frst” occur. If the child had never experienced any other environment, there is no reason to expect that the specifying information for the afordance has been discovered. The supporting environment where the child learned might ofer several other informational variables that, for this initial environment, are sufcient for throwing to be accomplished. This initial attendance to non-specifying information does not limit performance in this initial environment, and thus, the action might show signs of improvement after continued practice. When a new situation (new environment) appears, the capacity to move is present, but the perception of the actual informational variables that specify environmental support for the action is not attended to (and might not even be there). As individuals attend to non-specifying informational variables, they might act when such informational variable is present, and the afordance is not. Thus, individuals might act in terms of perceived afordances that are not there, so they fail in the task. This failure informs about the experienced relation between action capabilities and environmental support; the failure shows that the ft is not there or action adjustments are necessary. This can invoke exploration regarding action modifcations or changes on the informational variable being attended. Then, learning can occur. This process of errors and exploration—driven by afordance misperception—supports increased stability in the skill that encompasses the enabling competency and appropriate perception of supportive environments. This process, then, would support strengthening the enabling competency branch. Because of this continued exploration, new competencies can be acquired to support goal achievement in similar tasks. For instance, changing the medium, if the individual is in a swimming pool and attempts to throw a ball forcefully, the “full” whip pattern might not be correctly implemented as the pelvis and thorax rotating sequence is slowed down. Elastic energy is dissipated into heat. Then, through continued attempts, the child must further explore the possibilities of increasing throwing distance. In this case, either the child learns

The Dynamics of Afordance Emergence and Perception

131

to jump and coordinate the throw when the body is at peak height (see Verhoeven & Newell, 2016) or starts using “half” of the whip-like pattern (starting with the thorax). In this sense, afordance misperception creates possibilities for branching to occur: new enabling competencies are acquired, given the attempt to perform a skill in a non-supportive environment. What Is the Connection Between Afordances, Behavioral Scale, and Intention? How Distant or Reluctant Can a PerceivingActing System Be Before Afordances Come Into Play? Do Afordances Exert Their Role Only When the System Is Close or Willing Enough? Social-Afective Factors as “Forcings” to Branching

The above idea would provide a potential “adaptive role” for afordance misperception, but it is limited in explaining why individuals would be “tempted” to act beyond their capabilities. In some sense, having the possibility to act does not tempt an individual to do so or perceive the action as possible. What would be a factor that could explain this? The literature on motivation hints at why this might be the case. Harter (1978) showed that children, up to seven years of age, tend to overestimate their capabilities. This inaccurately perceived motor competence has been discussed as a “positive” feature that keeps children active during their childhood, supporting learning and further engagement in physical activities (Stodden et al., 2008). Children are also said to present higher intrinsic motivation for action (Ryan & Deci, 2000). This intrinsic motivation (and curiosity) has been used in several models to explain why children keep exploring new movement possibilities when a given function has already been acquired (see Gottlieb & Oudeyer, 2018). It could be that children overestimate their abilities “naturally” and are intrinsically motivated; these would be a fuel to maintain engagement in actions and, consequently, support motor skill acquisition. This “natural tendency” for action results in acting erroneously in a supportive environment or acting “correctly” in a non-supportive environment. These two motivational concepts could explain why overestimated capacities are seen in early development, but it is not enough to explain why over/underestimation occurs in other situations. Then, a third possibility to explain diferences between persons in afordance perception might come from self-efcacy (Bandura, 1977). Following Bandura’s (2006) guidelines, the self-efcacy scale results in an individual sigmoid curve demonstrating the confdence that a given performance can be achieved (similar to an afordance “curve,” see note 3). The diference between curves is that the afordance curve describes a “can-cannot” curve, while the self-efcacy

132

Matheus M. Pacheco and Ricardo Drews

curve takes “confdence” into account. Recent studies (Drews et al., 2021a, 2021b) have shown fast changes in self-efcacy from before to after a brief exposure to the task (Pacheco et al., 2021), similar to afordance “tuning” that occur after some task experience (Cole et al., 2013). The inclusion of confdence in self-efcacy might explain the diferences in afordance existence and afordance perception. Dividing the sigmoidal self-efcacy curve in two (low and high self-efcacy regions/performances), one could explain over and underestimates, considering how one is confdent in a “can” or “cannot” region. That is, the learner might be overly confdent in regions of low self-efcacy (implying a tendency to overestimate capacities) or underconfdent in regions of high self-efcacy (implying a tendency to underestimate capacities). This possible diference in “confdence” has not been studied much in ecological psychology. A question remains: if action experience leads to a better perception of afordances, why do experts (or experienced individuals) tend to underestimate their capabilities (Peker et al., 2021; Whitaker et al., 2020)? We believe that this might relate to self-efcacy and the variable nature of motor behavior discussed earlier. Consider that experienced individuals have higher chances to experience their limits and, to the same extent, the variability in outcome. The action limits (e.g., the best one can throw) are the frst to be afected in case of fatigue, lack of warm-up, and other intervenient variables. Variability in the outcome would increase as one approximates the action limits—in dynamical terms, this might refer to critical values where the given action loses stability (Kelso, 1995). This approximation to limits, then, would also refer to a decrease in overall performance which would, ultimately, afect self-efcacy and perceived motor competence. Thus, confdence in achieving the outcome would decrease around action limits and result in an underestimation of action possibilities. In some situations, this underestimation might come in handy. Climbers (as in Whitaker et al., 2020) might beneft from not performing close to their action capacity limits, acting in a “safe zone.” Provided the penalty for failure can be large in climbing (even with all the security measures), individuals might consider the low rate of success (or large variability in performance; see note 1). Variability in the outcome might be bigger than the benefts (the gain) of performing close to their limits. This can also be argued to be the case in many sports that induce large penalties for failing (e.g., gymnasts in Peker et al., 2021).7 The Afordance of “Learnability” and Concepts of Ability

There is another non-mutually exclusive possibility considering the socialafective literature. We have been relating motivational aspects (usually

The Dynamics of Afordance Emergence and Perception

133

used to explain engagement in activities) to the perception of action capacities. Inaccurate perceived motor competence and self-efcacy would relate to over/underestimating action capacities, altering whether individuals see activities as possible and favor/limiting engagement. The exciting possibility—and taking advantage of the speculative nature of the text until here—is that it might be that the perception of afordances does not need to be related only to current action possibilities but also to potential action possibilities. The relation that gives rise to afordances might not need to be considered in terms of an action that someone already demonstrated in the past but of the capacity to learn the action and, only then, perform that given action. If we consider the possibility of afordances extending in time—which is inevitably the case for higher-order goals (e.g., repairing the roof requires nested goals of climbing the stairs, using tools, and others; Wagman et al., 2016b, 2016a)—one can see the possibility of an afordance relating to making a task “learn-able.” What would be the relation of action and perception that could maintain such afordance? First, as considered earlier, one could assume that as individuals act and receive feedback on the task, there is an ongoing interaction between perception-action possibilities and task requirements. This interaction would provide information about this relationship (Newell et al., 1989; Pacheco et al., 2019). Such information guides change in behavior (some would call it information for learning; Jacobs & Michaels, 2007). Behavior changes (in some cases learning), then, depend on the individual’s capacity to change behavior according to this information. Thus, an environment (a task-environment context) in which the individual presents the capacity to efect change in behavior and attend to the information for change in behavior of that task would describe a “learnable” afordance of this given task. In terms of our “tree” model, this would be facilitated (if not determined) by the enabling competencies that allow interacting with the task “appropriately.” An individual unable to interact with the task requirements as it lacks sufcient “skill” will perceive the task as impossible to learn. For instance, if an individual fails to learn how to switch smoothly from lying down to standing on a surfboard, he/she will be unable to learn to “drive” the surfboard. Inappropriate switching may cause postural instabilities that prevent the learner from interacting with the task (and movement possibilities), limiting the capacity to change behavior according to the task demands. As all other concepts discussed so far, this “learn-able” afordance is not “free” of social-afective concepts (and developmental trends). The concept at stake here is of “conception of ability” (Dweck, 2002): individuals might believe that their abilities (e.g., intelligence, motor skills) are either

134

Matheus M. Pacheco and Ricardo Drews

malleable—being a matter of efort and learning—or fxed (“inherited”)— and being limited or too stable to change (Dweck & Leggett, 1988). Individuals in these two extremes would perform in terms of diferent “goals” (improve/outperform others) and might vary in their resilience to failures (Dweck, 2000). These conceptions can be modifed through experiences, and, as all other concepts described earlier, young children seem less prone to drawbacks from failures (Dweck, 2002). The Issue of Sufciency

The process of action (interaction of the individual with the task demands) leading to accurate perception here is primary as it is through this process that actions become more refned to a given environment and attendance to more useful informational variables occur. Nonetheless, as our ideas in search (and the presented framework) would predict, unless an individual has experience in several situations—with varied environmental support— and there is the need for high performance, this individual might never correctly perceive an afordance. To correctly perceive a given afordance, the specifying information must be attended to whenever an intention to perform a given action is present, and the given action is possible. As demonstrated, individuals need (longterm) practice to perceive the specifying information for several tasks. Variation in tasks might help to demonstrate that some informational variables are not useful for the performed action (Higueras-Herbada et  al., 2019; Huet et al., 2011)—although variation is not sufcient (considering motor learning literature; Pacheco & Newell, 2018; Van Rossum, 1990). Nonetheless, there are sets of tasks on which diferentiation of what is useful or not (in terms of information) is not as clear (see Rop & Withagen, 2014); attending to non-useful information does not result in large decrements in performance. Thus, if an individual learning to perform an action experiences variations in sets on which the diference between useful and non-useful information is not large, an individual might continue to attend non-useful information (Rop & Withagen, 2014; Withagen & Chemero, 2009; Withagen & Van Wermeskerken, 2009). This would occur because, given the individual’s experiences, the use of non-specifying information is sufcient to maintain the desired level of performance. The degree to which performance variation leads to changes in behavior is also an issue. Not considering athletes who must perform at the maximum capabilities in competitions, regular individuals would not maintain a search for better action possibilities and informational variables as the current performance is sufcient (see Simon, 1956). Our group has shown that individuals might stop searching when performance is

The Dynamics of Afordance Emergence and Perception

135

“sufcient,” which prevents the perception of better solutions (Pacheco et al., 2017, 2020). Thus, our rationale does not posit that afordance perception is ever in terms of specifying information. The studies on the friction of the ground from Adolph and colleagues demonstrate that even for a highly performed task (walking) adults might not perceive the necessary information for the activity (walking on a slippery foor) as, usually, this is not required. Possibly, only when most situations experienced require the perception of friction would individuals start attending to informational variables that specify it. “Why Is Afordance Any Better Than Stimulus? What Does a Theory of Afordance Suggest That Stimuli Cannot; How Has It Moved the Needle Past Gibson 1960’s Recognition of How Little We Can Defne Stimuli?”

We started this chapter by emphasizing that the dynamics of afordance perception seem to be of a diferent “nature” when compared to afordance existence. After highlighting conceptual and empirical aspects that a theory of afordance perception should encompass, we brought a tentative framework to explain the incongruence between afordance perception and afordance existence. This framework heavily relies on several socialafective concepts and theories and our current work on motor learning and development. Regarding the issues raised in this chapter, we do not see any as a problem for a theory of afordances. In fact, without a theory of afordances, one would be at sea trying to relate sensory stimuli to all “action-based” concepts that we consider in this text. It is even tempting to challenge some of the theories that base the mentioned social-afective concepts (see note 7) through an afordance-based view. However, a theory of afordances might need to encompass these features/issues if there is a desire to explain motor learning and development. This would be the case, given the inherent variability of motor actions and over/underestimates of action perception occur often, and these, as we hope to have demonstrated, require changes in how afordance perception is theorized. The theoretical advances in defning afordances and characterizing their existence are required, but they are still insufcient to explain the fundamental dynamics of afordance perception. We must note that we assumed a link between engagement and afordance perception in explaining how social-afective concepts could relate to over/underestimating action capacities. This is especially the case for perceived motor competence and intrinsic motivation. This assumption is

136

Matheus M. Pacheco and Ricardo Drews

fragile. There is a link between these, but a complete account must be worked out and tested, so our claims can be further considered. Provided the similarities in thinking (and measurement) between self-efcacy and afordance judgment, we see a more straightforward path of conciliation there. Our attempt to bring social-afective concepts to explain fndings in “domains” that initially have avoided (ignored) them is a path that others have also embraced in the motor domain (e.g., de Meester et  al., 2018; Wulf & Lewthwaite, 2016). Ecological psychology has already made signifcant steps in the social sphere (Borghi et al., 2022; Heft, 2020), but we feel this can be enhanced by considering social-afective concepts (following, for instance, Withagen, 2018). This integration is a natural course for a theory that directs eforts into the appropriate level of analysis of perception and action. The theory of afordances is a path that frees psychology (and other disciplines) from ideas of impoverished stimulation of the senses. Our ideas are in the direction of pushing further theorizing so that afordances are able to grasp all the complexity of behavior in perceiving in acting. Notes 1. One could argue that the allowed “failure” is not fxed. For instance, the Fosbury fop—the technique on which the athlete jumps backward above the bar in a high jump and lands on his/her back—could only be popularized after the advent of deep foam matting. In this situation, failure to perform the technique ideally is favored as the mat dissipates the impact. Nonetheless, such an example could be extended speculatively to other situations where fear of getting hurt is the limiting factor to attempt to perform a movement (e.g., standing on a slackline), and the number of successes to perform is non-zero but is small. 2. The titles of two classic texts make this point clear: “The Ecological Approach to Visual Perception” (Gibson, 1979) and “Preliminaries to a theory of action with reference to vision” (Turvey, 1977). 3. In this text, we are assuming that the means to measure afordance perception are appropriate. For most studies, afordances are measured by a participant’s judgment of the “possibility to act” when confronting the task requirements. Variations would include the usual procedure in motor development for early childhood. In these, what is measured is the attempt to act while the experimenter notes attempts and successes to defne afordance perception and afordance existence, respectively. Ultimately, the result is a sigmoidal curve that shows a clear-cut value at which the afordance ceases to exist. Some potential issues can be discussed. The possibility of verbal judgment and perception not being fully linked has been discussed elsewhere (see Jacobs & Michaels, 2002). Additionally, one could claim that in tasks where there is “social pressure” (parents, peers, experimenter), asking individuals to perform might induce errors (attempts where there should be none). Then, only “representative situations” (acting when no other peer-induced action occurs) would truly demonstrate when afordances are perceived.

The Dynamics of Afordance Emergence and Perception

137

4. We will use learning here in the same sense as Adolph (2019) describes “learning in (not and) development.” 5. Pacheco, M. M., Vasconcelos, M. O. F., Godoi Filho, J. R. M., Henrique, R. S., & Drews, R. The Motor Explorer: Rethinking Motor Competence for Engagement in Physical Activity. (Under review). 6. Enabling competencies are not actions. These can be thought of learned movement components that can be observed in a range of skills (actions). 7. Let us note that Bandura (Bandura, 1977) was not against representations of the self—which is critically at odds with a Gibsonian view. To avoid some criticisms and maintain the view within the ecological approach, by self-efcacy, we mean a long-term perception of its own efcacy in acting—similar to ideas of perception of the perceptual-motor workspace or information for learning (see Jacobs & Michaels, 2007; Pacheco et al., 2019). In this way, “rate of success” would not need to be calculated or maintained “stored” somewhere and confdence is a resultant feature of this perception.

Reference List Adolph, K. E. (2000). Specifcity of learning: Why infants fall over a veritable clif.Psychological Science, 11(4), 290–295. https://doi.org/10.1111/1467-9280.00258 Adolph, K. E. (2019). An ecological approach to learning in (not and) development. Human Development, 63(3–4), 180–201. https://doi.org/10.1159/000503823 Adolph, K. E., Bertenthal, B. I., Boker, S. M., Goldfeld, E. C., & Gibson, E. J. (1997). Learning in the development of infant locomotion. Monographs of the Society for Research in Child Development, 162, 1–58. https://doi. org/10.2307/1166199 Adolph, K. E., Joh, A. S., & Eppler, M. A. (2010). Infants’ perception of afordances of slopes under high- and low-friction conditions. Journal of Experimental Psychology: Human Perception and Performance, 36(4), 797–811. https:// doi.org/10.1037/a0017450 Araújo, D., Davids, K., & Hristovski, R. (2006). The ecological dynamics of decision making in sport. Psychology of Sport and Exercise, 7(6), 653–676. https:// doi.org/10.1016/j.psychsport.2006.07.002 Bandura, A. (1977). Self-efcacy: Toward a unifying theory of behavioral change. Psychological Review, 84, 191–215. https://doi.org/10.1037/0033-295X.84.2.191 Bandura, A. (2006). Guide for constructing self-efcacy scales. Self-Efcacy Beliefs of Adolescents, 5(1), 307–337. Berger, S. E., Adolph, K. E., & Kavookjian, A. E. (2010). Bridging the gap: Solving spatial means—Ends relations in a locomotor task. Child Development, 81(5), 1367–1375. https://doi.org/10.1111/j.1467-8624.2010.01478.x Black, D. P., Smith, B. A., Wu, J., & Ulrich, B. D. (2007). Uncontrolled manifold analysis of segmental angle variability during walking: Preadolescents with and without down syndrome. Experimental Brain Research, 183(4), 511–521. https://doi.org/10.1007/s00221-007-1066-1 Borghi, A. M., Osińska, A., Roepstorf, A., & Raczaszek-Leonardi, J. (2022). Concepts in interaction: Social engagement and inner experiences. Philosophical Transactions of the Royal Society B: Biological Sciences, 378(1870), 20210351. https://doi.org/10.1098/rstb.2021.0351

138

Matheus M. Pacheco and Ricardo Drews

Brakke, K., & Pacheco, M. M. (2019). The development of bimanual coordination across toddlerhood. Monographs of the Society for Research in Child Development, 84(2), 7–147. https://doi.org/10.1111/mono.12405 Chemero, A. (2003). An outline of a theory of afordances. Ecological Psychology, 15(2), 181–195. https://doi.org/10.1207/S15326969ECO1502_5 Chemero, A. (2009). Radical Embodied Cognitive Science. MIT Press. Cohen, E., Yemini, E., Schafer, W., Feitelson, D. G., & Treinin, M. (2012). Locomotion analysis identifes roles of mechanosensory neurons in governing locomotion dynamics of C. elegans. Journal of Experimental Biology, 215(20), 3639–3648. https://doi.org/10.1242/jeb.075416 Cole, W. G., Chan, G. L. Y., Vereijken, B., & Adolph, K. E. (2013). Perceiving afordances for diferent motor skills. Experimental Brain Research, 225(3), 309–319. https://doi.org/10.1007/s00221-012-3328-9 Corrêa, U. C., de Pinho, S. T., da Silva, S. L., Clavijo, F. A. R., Souza, T. de O., & Tani, G. (2016). Revealing the decision-making of dribbling in the sport of futsal. Journal of Sports Sciences, 34(24), 2321–2328. https://doi.org/10.1080/ 02640414.2016.1232488 Côté, J. N., Feldman, A. G., Mathieu, P. A., & Levin, M. F. (2008). Efects of fatigue on intermuscular coordination during repetitive hammering. Motor Control, 12(2), 79–92. https://doi.org/10.1123/mcj.12.2.79 Day, B. M., Wagman, J. B., & Smith, P. J. K. (2015). Perception of maximum stepping and leaping distance: Stepping afordances as a special case of leaping afordances. Acta Psychologica, 158, 26–35. https://doi.org/10.1016/j. actpsy.2015.03.010 de Meester, A., Stodden, D., Goodway, J., True, L., Brian, A., Ferkel, R., & Haerens, L. (2018). Identifying a motor profciency barrier for meeting physical activity guidelines in children. Journal of Science and Medicine in Sport, 21(1), 58–62. https://doi.org/10.1016/j.jsams.2017.05.007 Domkin, D., Laczko, J., Djupsjöbacka, M., Jaric, S., & Latash, M. L. (2005). Joint angle variability in 3D bimanual pointing: Uncontrolled manifold analysis. Experimental Brain Research, 163(1), 44–57. https://doi.org/10.1007/s00221-004-2137-1 dos Santos, F. G., Pacheco, M. M., Stodden, D., Tani, G., & Maia, J. A. (2022). Testing Seefeldt’s profciency barrier: A longitudinal study. International Journal of Environmental Research and Public Health, 19(12), 7184. https://doi. org/10.3390/ijerph19127184 Drews, R., Pacheco, M. M., Bastos, F. H., & Tani, G. (2021a). Efects of normative feedback on motor learning are dependent on the frequency of knowledge of results. Psychology of Sport and Exercise, 55, 101950. https://doi.org/10.1016/j. psychsport.2021.101950 Drews, R., Pacheco, M. M., Bastos, F. H., & Tani, G. (2021b). Knowledge of results do not afect self-efcacy and skill acquisition on an anticipatory timing task. Journal of Motor Behavior, 53(3), 275–286. https://doi.org/10.1080/002 22895.2020.1772711 Dweck, C. S. (2000). Self-Theories: Their Role in Motivation, Personality, and Development. Psychology Press. Dweck, C. S. (2002). The development of ability conceptions. In A. Wigfeld & J. Eccles (Eds.), Educational Psychology (pp. 57–88). Academic Press. https://doi. org/10.1016/B978-012750053-9/50005-X

The Dynamics of Afordance Emergence and Perception

139

Dweck, C. S., & Leggett, E. L. (1988). A social-cognitive approach to motivation and personality. Psychological Review, 95, 256–273. https://doi. org/10.1037/0033-295X.95.2.256 Fajen, B. R., Riley, M. A., & Turvey, M. T. (2009). Information, afordances, and the control of action in sport. International Journal of Sport Psychology, 40(1), 79–107. Franchak, J. M., & Adolph, K. E. (2014). Gut estimates: Pregnant women adapt to changing possibilities for squeezing through doorways. Attention, Perception, & Psychophysics, 76(2), 460–472. https://doi.org/10.3758/s13414-013-0578-y Gibson, J. J. (1979). The Ecological Approach to Visual Perception. Houghton Mifin. Gottlieb, J., & Oudeyer, P.-Y. (2018). Towards a neuroscience of active sampling and curiosity. Nature Reviews Neuroscience, 19(12), 758–770. https://doi. org/10.1038/s41583-018-0078-0 Harter, S. (1978). Efectance motivation reconsidered. Toward a developmental model. Human Development, 21(1), 34–64. https://doi.org/10.1159/000271574 Heft, H. (2020). Ecological psychology as social psychology? Theory & Psychology, 30(6), 813–826. https://doi.org/10.1177/0959354320934545 Higueras-Herbada, A., de Paz, C., Jacobs, D. M., Travieso, D., & Ibáñez-Gijón, J. (2019). The direct learning theory: A naturalistic approach to learning for the post-cognitivist era. Adaptive Behavior, 27(6), 389–403. https://doi. org/10.1177/1059712319847136 Holt, K. G., Wagenaar, R. O., & Saltzman, E. (2010). A dynamic systems: Constraints approach to rehabilitation. Brazilian Journal of Physical Therapy, 14, 446–463. https://doi.org/10.1590/S1413-35552010000600002 Huet, M., Jacobs, D. M., Camachon, C., Missenard, O., Gray, R., & Montagne, G. (2011). The education of attention as explanation of variability of practice efects: Learning the fnal approach phase in a fight simulator. Journal of Experimental Psychology: Human Perception and Performance, 37, 1841–1854. https://doi.org/10.1037/a0024386 Jacobs, D. M., & Michaels, C. F. (2002). On the apparent paradox of learning and realism. Ecological Psychology, 14(3), 127–139. https://doi.org/10.1207/ S15326969ECO1403_2 Jacobs, D. M., & Michaels, C. F. (2007). Direct learning. Ecological Psychology, 19(4), 321–349. https://doi.org/10.1080/10407410701432337 Joh, A. S., Adolph, K. E., Campbell, M. R., & Eppler, M. A. (2006). Why walkers slip: Shine is not a reliable cue for slippery ground. Perception & Psychophysics, 68(3), 339–352. https://doi.org/10.3758/BF03193681 Joh, A. S., Adolph, K. E., Narayanan, P. J., & Dietz, V. A. (2007). Gauging possibilities for action based on friction underfoot. Journal of Experimental Psychology: Human Perception and Performance, 33, 1145–1157. https://doi. org/10.1037/0096-1523.33.5.1145 Kelso, S. J. A. (1995). Dynamic Patterns: The Self Organization of Brain and Behaviour. MIT Press. Konczak, J., Meeuwsen, H. J., & Cress, M. E. (1992). Changing afordances in stair climbing: The perception of maximum climbability in young and older adults. Journal of Experimental Psychology: Human Perception and Performance, 18(3), 691–697. https://doi.org/10.1037/0096-1523.18.3.691

140

Matheus M. Pacheco and Ricardo Drews

Latash, M. L., Scholz, J. P., & Schöner, G. (2007). Toward a new theory of motor synergies. Motor Control, 11(3), 276–308. https://doi.org/10.1123/mcj.11.3.276 Lee, I.-C., Pacheco, M. M., & Newell, K. M. (2018). Constraints specifc infuences of vision, touch and surface compliance in postural dynamics. Gait & Posture, 59, 117–121. https://doi.org/10.1016/j.gaitpost.2017.09.014 Lo, T., Avolio, A. M., Massop, S. A., & Adolph, K. E. (1999). Why toddlers don’t perceive risky ground based on surface friction. In M. A. Grealy & J. A. Thomson (Eds.), Studies in Perception and Action V (pp. 231–235). Erlbaum. Newell, K. M. (1986). Constraints on the development of coordination. In M. G. Wade & H. T. Whiting (Eds.), Motor Development in Children: Aspects of Coordination and Control (pp. 341–360). Martinus Nijhof. Newell, K. M, & Corcos, D. M. (1993). Issues in variability and motor control. In K. M. Newell & D. M. Corcos (Eds.), Variability and Motor Control (pp. 1–12). Human Kinetics. Newell, K. M., Kugler, P. N., Van Emmerik, R. E. A., & McDonald, P. V. (1989). Search strategies and the acquisition of coordination. In S. A. B. T.-A. in P. Wallace (Ed.), Perspectives on the Coordination of Movement (Vol. 61, pp. 85–122). Elsevier. https://doi.org/10.1016/S0166-4115(08)60019-9 Newell, K. M., Mayer-Kress, G., Hong, S. L., & Liu, Y.-T. (2009). Adaptation and learning: Characteristic time scales of performance dynamics. Human Movement Science, 28(6), 655–687. https://doi.org/10.1016/j.humov.2009.07.001 O’keefe, S. L., Harrison, A. J., & Smyth, P. J. (2007). Transfer or specifcity? An applied investigation into the relationship between fundamental overarm throwing and related sport skills. Physical Education and Sport Pedagogy, 12(2), 89–102. https://doi.org/10.1080/17408980701281995 Pacheco, M. M., Bastos, F. H., & Drews, R. (2021). Motivation and information in motor performance: Modelling of self-efcacy and knowledge of results interaction in a timing task. Brazilian Journal of Motor Behavior, 15(3), 250–266. https://doi.org/0.20338/bjmb.v15i3.226 Pacheco, M. M., dos Santos, F. G., Marques, M. T. S. P., Maia, J. A. R., & Tani, G. (2022). Transitional movement skill dependence on fundamental movement skills: Testing Seefeldt’s profciency barrier. Research Quarterly for Exercise and Sport, 93(4), 718–727. https://doi.org/10.1080/02701367.2021.1912691 Pacheco, M. M., Hsieh, T.-Y., & Newell, K. M. (2017). Search strategies in practice: Movement variability afords perception of task dynamics. Ecological Psychology, 29(4), 243–258. https://doi.org/10.1080/10407413.2017.1368354 Pacheco, M. M., Lafe, C. W., & Newell, K. M. (2019). Search strategies in the perceptual-motor workspace and the acquisition of coordination, control, and skill. Frontiers in Psychology, 10, 1874. https://doi.org/10.3389/fpsyg.2019.01874 Pacheco, M. M., Lafe, C. W., & Newell, K. M. (2020). Search strategies in practice: Testing the efect of inherent variability on search patterns. Ecological Psychology, 32(2–3), 115–138. https://doi.org/10.1080/10407413.2020.1781536 Pacheco, M. M., & Newell, K. M. (2018). Search strategies in practice: Infuence of information and task constraints. Acta Psychologica, 182, 9–20. https://doi. org/10.1016/j.actpsy.2017.11.004 Peker, A. T., Erkmen, N., Kocaoglu, Y., Bayraktar, Y., Arguz, A., Wagman, J. B., & Stofregen, T. A. (2021). Perception of afordances for vertical and horizontal

The Dynamics of Afordance Emergence and Perception

141

jumping in children: Gymnasts versus non-athletes. Research Quarterly for Exercise and Sport, 92(4), 770–778. https://doi.org/10.1080/02701367.2020.1775768 Riley, M. A., & Turvey, M. T. (2002). Variability and determinism in motor behavior. Journal of Motor Behavior, 34(2), 99–125. https://doi. org/10.1080/00222890209601934 Roberton, M. A., & Halverson, L. E. (1984). Developing Children—Their Changing Movement: A Guide for Teachers. Lea & Febiger. Rop, G., & Withagen, R. (2014). Perceivers vary in their capacity to beneft from feedback in learning to perceive length by dynamic touch. Attention, Perception, & Psychophysics, 76(3), 864–876. https://doi.org/10.3758/s13414-013-0598-7 Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist, 55, 68–78. https://doi.org/10.1037/0003-066X.55.1.68 Seefeldt, V. (1980). Developmental motor patterns: Implications for elementary school physical education. In Psychology of Motor Behavior and Sport (pp. 314–323). https://doi.org/10.1007/s40279-020-01332-6 Simon, H. A. (1956). Rational choice and the structure of the environment. Psychological Review, 63, 129–138. https://doi.org/10.1037/h0042769 Southard, D. L. (2014). Changes in kicking pattern: Efect of experience, speed, accuracy, and efective striking mass. Research Quarterly for Exercise and Sport, 85(1), 107–116. https://doi.org/10.1080/02701367.2013.829383 Stodden, D. F., Goodway, J. D., Langendorfer, S. J., Roberton, M. A., Rudisill, M. E., Garcia, C., & Garcia, L. E. (2008). A developmental perspective on the role of motor skill competence in physical activity: An emergent relationship. Quest, 60(2), 290–306. https://doi.org/10.1080/00336297.2008.10483582 Stofregen, T. A. (2003). Afordances as properties of the animal-environment system. Ecological Psychology, 15(2), 115–134. https://doi.org/10.1207/ S15326969ECO1502_2 Turvey, M. T. (1977). Contrasting orientations to the theory of visual information processing. Psychological Review, 84(1), 67–88. https://doi. org/10.1037/0033-295X.84.1.67 Turvey, M. T. (1992). Afordances and prospective control: An outline of the ontology. Ecological Psychology, 4(3), 173–187. https://doi.org/10.1207/ s15326969eco0403_3 Turvey, M. T., & Fonseca, S. T. (2014). The medium of haptic perception: A tensegrity hypothesis. Journal of Motor Behavior, 46(3), 143–187. https://doi.org/10. 1080/00222895.2013.798252 Van Rossum, J. H. A. (1990). Schmidt’s schema theory: The empirical base of the variability of practice hypothesis: A critical analysis. Human Movement Science, 9(3), 387–435. https://doi.org/10.1016/0167-9457(90)90010-B Verhoeven, F. M., & Newell, K. M. (2016). Coordination and control of posture and ball release in basketball free-throw shooting. Human Movement Science, 49, 216–224. https://doi.org/10.1016/j.humov.2016.07.007 Wagman, J. B., Caputo, S. E., & Stofregen, T. A. (2016a). Hierarchical nesting of afordances in a tool use task. Journal of Experimental Psychology: Human Perception and Performance, 42(10), 1627–1642. https://doi.org/10.1037/ xhp0000251

142

Matheus M. Pacheco and Ricardo Drews

Wagman, J. B., Caputo, S. E., & Stofregen, T. A. (2016b). Sensitivity to hierarchical relations among afordances in the assembly of asymmetric tools. Experimental Brain Research, 234(10), 2923–2933. https://doi.org/10.1007/ s00221-016-4695-4 Whitaker, M. M., Pointon, G. D., Tarampi, M. R., & Rand, K. M. (2020). Expertise efects on the perceptual and cognitive tasks of indoor rock climbing. Memory & Cognition, 48(3), 494–510. https://doi.org/10.3758/s13421-019-00985-7 Wilson, A. D., Weightman, A., Bingham, G. P., & Zhu, Q. (2016). Using task dynamics to quantify the afordances of throwing for long distance and accuracy. Journal of Experimental Psychology: Human Perception and Performance, 42, 965–981. https://doi.org/10.1037/xhp0000199 Withagen, R. (2018). Towards an ecological approach to emotions and the individual diferences therein. New Ideas in Psychology, 51, 21–26. https://doi. org/10.1016/j.newideapsych.2018.04.004 Withagen, R., & Chemero, A. (2009). Naturalizing perception: Developing the Gibsonian approach to perception along evolutionary lines. Theory & Psychology, 19(3), 363–389. https://doi.org/10.1177/0959354309104159 Withagen, R., & Van Wermeskerken, M. (2009). Individual diferences in learning to perceive length by dynamic touch: Evidence for variation in perceptual learning capacities. Perception & Psychophysics, 71(1), 64–75. https://doi. org/10.3758/APP.71.1.64 Wulf, G., & Lewthwaite, R. (2016). Optimizing performance through intrinsic motivation and attention for learning: The OPTIMAL theory of motor learning. Psychonomic Bulletin & Review, 23(5), 1382–1414. https://doi.org/10.3758/ s13423-015-0999-9

8 DESCRIPTION OF THE WORLD THAT AGENTIAL SYSTEMS FIT INTO Tetsushi Nonaka

What Do We Understand by the Term Afordance?

Let us start with the inverse question: what do we not understand without the term afordances? Although he did not have the term afordances available, Edwin Holt (1916, p. 55), Gibson’s mentor at Princeton, foreshadowed the question: The organism is moving with reference to some object or fact of the environment. . . . in order to understand what the organism is doing, you will just miss the essential point if you look inside the organism. For the organism, while a very interesting mechanism in itself, is one whose movements turn on objects outside of itself, much as the orbit of the earth turns upon the sun; and these external, and sometimes very distant, objects are as much constituents of the behavior process as is the organism which does the turning. Holt argued that we do not understand what an organism is doing without the description of the environment that the actions of an organism are a constant function of. Moreover, those properties of the environment of which the organismic actions are a function, as Holt (1916, p. 163) put it, “are always open to empirical investigation.” The point Holt was making here is nowhere better illustrated than in Darwin’s (1892) study on the habits of earthworms to plug up the mouths of their burrows with leaves and other objects (see also Reed, 1996;

DOI: 10.4324/9781003396536-11

144

Tetsushi Nonaka

Sasaki & Nonaka, 2016). Hundreds of leaves collected by Darwin unambiguously showed that how the leaves were drawn into the burrows was a constant function of specifc properties of the environment, where the leaves of diferent shapes were dragged into the burrows in diferent manners. For example, in the case of lime leaves, which are very broad at the base, only 4% of the leaves had been drawn in by the base, while in the case of the laburnum leaves, which were not more pointed toward the apex than toward the base, a far larger proportion (27%) had been drawn in by the base. “If a man had to plug up a small cylindrical hole, with such objects as leaves, petioles or twigs,” Darwin (1892, p.  64) wrote, “he would drag or push them in by their pointed ends; but if these objects were very thin relatively to the size of the hole, he would probably insert some by their thicker or broader ends.” Darwin’s eldest son William collected 237 fallen Rhododendron leaves in his garden that vary very much in shape partly due to the curling in of their margins while drying after they have fallen of. William examined the “draw-in-ability” of Rhododendron leaves he collected and found that 65% of them could have been drawn by worms into their burrows more easily by the base or foot-stalk than by the tip partly due to the curling in of the margins and 27% could have been drawn in more easily by the tip than by the base. Then, in the same garden, he collected 91 Rhododendron leaves that the worms had drawn into their burrows. Of these, 66% had been drawn in by the base or foot-stalk and 34% by the tip, which closely mirrored the distribution of leaves with diferent opportunities, implying that the actions of the worms were related to the “draw-in-ability” of the leaves. Yet, at the same time, earthworms did not act fxedly in all cases. When the worms drew the petioles of Clematis montana into the burrows in the well-beaten gravel walk, the worms showed a high degree of selectivity in choice of the point at which they grasped leaves, where nearly fve times as many had been drawn in by pointed tip as by the thicker base. Yet, when the worms drew the same petioles into the burrows in the lawn and fowerbed where the soil was soft and yielded more easily, the worms were less selective, and the proportion of those drawn in by the pointed tip to those drawn in by the thicker base was less than three to one. The actions of the worms were not mechanistic responses to the objects but appeared to be a function of what the meeting between the objects and the burrows in diferent conditions of the ground would aford. Darwin further highlighted the very fexible nature of worms’ action by presenting a variety of objects that the worms used for plugging up their burrows, which included some fower peduncles, decayed twigs of trees, bits of paper, feathers, tufts of wool and horsehairs, and even small stones.

Description of the World That Agential Systems Fit Into

145

Darwin described what the earthworm was doing in terms of the properties of the environment that the actions of the worms discriminated for the purpose of plugging up the burrows (Sasaki & Nonaka, 2016). The set of properties of the environment that Darwin (1892) described would never be apparent if it were not for the actions of burrowing worms. Yet, they are real. Gibson might have called them afordances for plugging up the burrows or, more broadly, afordances for controlling the states of the burrows. Taking his cues from Holt, as a student of visual perception, Gibson began to wonder if the level of description of the world that our everyday visual perception or “useful vision” (Gibson, 1967) fts into is so unique that it deserves investigation in its own right. Consider, for example, the skill of distinguishing environmental substances, the layout of surfaces, and what things are good for one task or another. What diferent types of clay, rock, vegetation, bark, leaves, fur, feathers, or skin aford to us may not be distinguishable at frst glance. But they can be potentially identifed through the activity of perceiving—looking at, palpating, listening to, or snifng them. Unlike transient neural signals converted from stimulus energy impinging on receptors, the embodied activity of perceiving, involving adjustments of organs, is a systematic function of a set of meaningful properties of the environment that are selectively attended to by the active perceiver. Then, it naturally follows that understanding such functional skills of perceiving demands the description of the functional referent of perception. Otherwise, it would be like watching a tennis match with half the court occluded from view (Nonaka, 2020). In one of his unpublished notes entitled The Evolution of Locomotion and Manipulation, Gibson wrote: “the control of behavior in the environment, so-called ‘voluntary’ behavior, is often the controlling of encounters with the environment that aford beneft or injury. One behaves in order to control encounters” (Gibson, 1978a, italics original). In more general terms, a defning feature of agential systems, including animals, is that they strive to control encounters with the environment so as to obtain benefcial encounters while avoiding harmful encounters. “Perceiving the afordances of things is,” as Gibson put it, “actually perceiving what encounters with them would aford” (Gibson, 1978b, italics original). In so doing, goaldirected actions of agents are coordinated to particular scales of nature reciprocally, the unfolding of which being a constant function of what encounters with the environment aford. Now, let us go back to the question we started with. The answer to the question would be that we do not understand what agents are doing without addressing what their encounters with the substances, surfaces, places, objects, and events of the environment aford, that is, afordances.

146

Tetsushi Nonaka

The term afordances provides a principled way to frame the issue of how agential systems control the encounters with their environment, by carving up particular scales of reality in which meanings and problems reside (Smith, 2009). What Role Does Afordance Play in Perception? Is It the Entity Organisms Perceive or the Means Through Which Organisms Perceive? Are Afordances the Only Perceptual Dependent Variables?

There are things in the environment, to be sure, that provide opportunities for exploratory activities by organisms equipped with specifc perceptual systems. For example, a gaseous medium of the terrestrial environment that transmits and scatters light (which brings about the condition called illumination) and illuminated surfaces (which structure the ambient light) aford seeing for an agent equipped with vision who occupies a point of observation in the medium. The atmospheric medium of the terrestrial environment, which propagates pressure waves and difuses molecules, also afords hearing sounds and smelling the substance at a distance. What a thing afords an agent is determined by what it is—an invariant combination of properties of the thing at the ecological level taken with reference to the agent. “The invariant combinations of features that constitute the afordances of these realities are what the animals pay attention to” (Gibson, 1982/2020a, p.  566). The question above—what role afordance plays in perception or not—needs to be grounded in describing what exists at the ecological level of reality. There are things in the environment that aford perception and action for which an agent is equipped, and the agent controls an encounter with the environment by tuning into the information that specifes what the encounter with her environment afords. We strive to pick up the information that specifes what the encounters with the environment aford and often succeed in controlling the encounters. But we can also fail to do so. For the perception of the afordances of a thing, what matters is whether a perceiver succeeds in detecting the information in the ambient energy array that specifes what an encounter with the thing afords. Sometimes, selectively tuning into the information that specifes afordances requires an extended learning process involving more than one individual in a populated environment (Reed, 1996). A tool such as a spear may aford important activities involved in hunting to some individuals, but given to an individual who has never been encouraged to attend to or to learn about those hunting-specifc

Description of the World That Agential Systems Fit Into

147

properties of the spear, the individual would be no more capable of hunting than without it (Ingold, 1987; Reed, 1993). In the case of early stone tool making, a specifc layout of surfaces of the stone afords fracturing a sharp-edged fake (based on the principle of conchoidal fracture), which is observable but not easily attended to by inexperienced stone tool makers (Nonaka, 2020, 2023). Just as the light needs no eyes to exist, so does the information that specifes afordances not depend on perception for its existence. This is the fundamental fact that provides opportunities for perceiving and acting agents to change (in phylogenetic, ontogenetic, and behavioral time scales) in such a way as to tune into, detect, and share the information that specifes what the environment afords. As (Gibson, 1979/2016, p. 244) put it, Perceiving gets wider and fner and longer and richer and fuller as the observer explores the environment. The full awareness of surfaces includes their layout, their substances, their events, and their afordances. Note how this defnition includes within perception a part of memory, expectation, knowledge, and meaning—some part but not all of those mental processes in each case. I am not quite sure what is meant by whether afordances are the only perceptual dependent variables. If the question asks whether we can get to know what the encounters with the environment would aford only by means of perception and by no other means, then the answer is: No. There are diferent ways of knowing what the encounters with the environment would aford. “Perceiving is the simplest and best kind of knowing” (Gibson, 1979/2016, p 251), but there are also other kinds (knowing by means of instruments, maps, language, and so on), although the knowledge obtained by instruments, maps, or language may be diferent from those obtained from direct perception of the environment. Gibson distinguished perception of the environment from explicit knowledge put into words. But at the same time, he suggested that the theory of information pickup can close the supposed gap between perception and knowledge. Gibson (1979/2015, p. 246) addressed this point as follows: Knowing is an extension of perceiving. The child becomes aware of the world by looking around and looking at, by listening, feeling, smelling, and tasting, but then she begins to be made aware of the world as well. She is shown things, and told things, and given models and pictures of things, and then instruments and tools and books, and fnally rules and short cuts for fnding out more things.

148

Tetsushi Nonaka

Why Is Afordance Any Better Than Stimulus? What Does a Theory of Afordance Suggest That Stimuli Cannot; How Has It Moved the Needle Past Gibson 1960’s Recognition of How Little We Can Defne Stimuli?

Rather than asking whether afordance is better or worse than stimulus and why, the relation between afordance and stimulus would be better understood by the fact that stimulus that impinges on receptors and corresponding sensations cannot carry information about the meaningful properties of substances, surfaces, places, objects, and events of the environment, that is, afordances. In theories of visual perception, “stimulation by light and corresponding sensations of brightness are traditionally supposed to be the basis of visual perception” (Gibson, 1979/2016, p.  47). Yet, simultaneously, “it has long been known that the stimulation of receptors is not sufcient for perception . . . stimuli do not specify the sources in the environment from which they come” (Gibson, 1982/2020b, p. 475). A traditional approach to resolve this gap has been to postulate intermediate processes of construction that somehow go beyond the insufcient data or to suppose “the operations of the mind upon the deliverances of the senses” (Gibson, 1976, p. 234). For example, one of the handbooks of neuroscience characterized such an approach as follows: “Perception begins in receptor cells that are sensitive to one or another kind of stimulus energy. . . . But what is seen in the ‘mind’s eye’ goes far beyond what is presented in the input” (Kandel et al., 2013, pp. 445–446). The approach Gibson took to resolve the problem of visual perception was radically diferent from such a traditional approach. Gibson rejected the fundamental assumption that stimulation by light and corresponding sensations are the basis of perception. Thereby, Gibson made a sharp distinction between (1) light as a stimulus that stimulates photoreceptors in the retina and (2) light as information for perception that can activate a perceptual system. Along with this distinction, Gibson developed ecological optics to salvage the latter fact of light that had never been made explicit. In his opening address at the Workshop on Ecological Optics held at Cornell University in June 1970, Gibson listed fve difculties that traditional optics could not resolve and that pushed him toward ecological optics. First among them was “the difculty of interpreting the results of Metzger’s experiments on the Ganzfeld” (Gibson, 1982/2020c, p. 136)—a famous experiment on homogeneous visual stimulation. A series of experiments on the Ganzfeld provided evidence that active accommodation of an eye would become impossible in homogeneous ambient light (i.e., in the unusual circumstance where the light coming to the nodal point of the eye has no discontinuities of intensity in diferent directions); in consequence,

Description of the World That Agential Systems Fit Into

149

“the possessor of the eye could not fx it on anything, and the eye would drift aimlessly” (Gibson, 1979/2016, p. 47). Vision fails in homogeneous ambient light even with adequate stimulation at receptors. Then it follows that, although stimulation by light may be necessary for seeing, light as stimulus energy is not a sufcient condition for seeing. The light that stimulates passive receptors may be at a diferent level from the light that afords activity involved in visual perception that orients the organs of perception and explores an ambient array. Ecological optics is concerned with many-times-refected light between the terrestrial medium and surfaces (illumination), which is at a diferent level from the electromagnetic energy of radiant light. Illumination is a fact of a higher order than radiation, albeit not contradictory, which results in ambient light—light surrounding a point in the medium that is structured (i.e., is diferent in diferent directions) chiefy by the layout of terrestrial surfaces. Gibson used the term an ambient optic array to describe the structure of ambient light. A central hypothesis for ecological optics is that the variables of an ambient optic array may carry information about the composition and layout of surfaces in the environment from which the light comes. By the term information Gibson meant specifcations of the environment (Gibson, 1979/2016, p. 231). “There are almost certainly laws,” Gibson wrote, “by which some variables in the optic array specify some environmental facts” (Gibson, 1961, p. 260). As the point of observation moves in the medium, the invariants of the transforming ambient optic array specify the layout of the surfaces of the environment. The transformations come from displacements of a point of observation, and the invariants come from the layout of the environmental surfaces. Hence, the invariant structure specifes the layout; the perspective transformation specifes a path of observation. For example, let us consider the case of distinguishing an obstacle from an opening in a cluttered environment. Note that the layout of surfaces, substances, and events constitute what they aford (Gibson, 1979/2016, p. 119). Diferent layouts of surfaces, such as an obstacle and an opening, have diferent afordances: the former afords collision, whereas the latter afords passage. Although in both cases, there is a closed or nearly closed contour in the optic array at a stationary point of observation, a slight shift of the point of observation results in the transformation of the optic array that would reveal invariants that distinguish an obstacle from an opening: Loss (or gain) of structure outside a closed contour during the perceiver’s approach (or retreat) specifes an obstacle. Gain (or loss) of structure inside a closed contour during the perceiver’s approach (or retreat) specifes an opening (Gibson, 1979/2016, p. 219). By separating of these invariants, the perceiver can, in turn, guide and control the encounters with the layout

150

Tetsushi Nonaka

of surfaces with diferent afordances, that is, steering away from an obstacle or entering an opening (James J Gibson, 1998). Gibson once (Gibson, 1982/2020b, p. 476) wrote: It is perfectly legitimate to apply physical stimulation to an animal . . . , but one should not expect to learn about perception or behavior in this way. For the latter purpose the experimenter must provide or display information. It is one thing to try to understand how receptors work by applying physical stimuli, but it is quite another to consider sensations evoked by those physical stimuli as the basis of perception. Gibson rejected the latter assumption and sharply distinguished light as stimulus energy that is absorbed by photoreceptors and light as information for perception of the environment and its afordances. The notion of ambient information further led Gibson to redefne the act of perceiving, for the unfolding process of perceiving would difer between the cases where the information that specifes the environment and its afordances are available to perceiver or not. In the former case, one would seek more informative invariants that specify the facts about the environment. If necessary, one might retake a close look at what is there. But in the latter case, where no information is available that specifes the environment, one can only think and guess. Gibson reserved the term perception for the former process of knowing, which difers from the latter regarding the nature of the activity involved and the resources used. Gibson wrote, [P]erceiving is an achievement of the individual, not an appearance in the theater of his consciousness. It is a keeping-in-touch with the world, an experiencing of things rather than having experiences. It involves awareness instead of just awareness. It may be awareness of something in the environment or something in the observer or both at once, but there is no content of awareness independent of that of which one is aware. (Gibson, 1979/2016, p. 228) What Is the Connection Between Afordances, Behavioral Scale, and Intention? How Distant or Reluctant Can a PerceivingActing System Be Before Afordances Come Into Play? Do Afordances Exert Their Role Only When the System Is Close or Willing Enough?

An agent and its environment are to be considered as reciprocal terms. An agent and its environment imply each other over all scales, from cells and cellular collectives to the collective of individual animals. As biologist Michael Levin (Levin, 2022, p. 3) put it,

Description of the World That Agential Systems Fit Into

151

[T]he capacity to work toward goals (preferred future states) is ubiquitous across the biosphere and present at all scales of organization, from the planning capacities of primates to the abilities of cellular collectives to modify their activity to achieve a specifc embryonic anatomy despite perturbations. At each of these scales of organization, what things in the environment aford an agent is determined by what they are—invariant combinations of properties of things (their size, morphology, stifness, viscosity, and so on)—taken with reference to the agent, its action-systems, its anatomy, its size, and its behavioral scale. For example, at the scale of planktonic microswimmers, their aquatic environment is dominated by viscous forces, governed by physical rules that are far diferent from those of larger aquatic animals and sometimes counterintuitive (Guasto et al., 2011). When the larva of the annelid Platynereis dumerilii—a planktonic microswimmer—propels itself, there is a stagnation point in the fow feld where the local velocity of the fuid is zero around the anterior tip of the body (Jékely et al., 2021; Figure 8.1). Such stagnation points are an invariant feature of the fow felds around self-propelled microswimmers (Guasto et  al., 2010). In Platynereis larvae, some fow sensors are located just at this stagnation point, at the most anterior tip of the head (Jékely et al., 2021). This suggests that these fow sensors located at the stagnation point will not be exposed to self-induced fow and may help isolate the information about external fow from the selfinduced fow. Hydrodynamics at the scale of their body afords a unique opportunity for these microswimmers equipped with fow-sensing perceptual systems to extract the invariant features of external fow independent of self-induced fow, which in turn afords these organisms to control their orientation in external fow felds. This is one example of the connection between the behavioral scale of an agent and what the encounters with the environment aford the agent. Regarding the subsequent two questions, I am unsure what it means for a perceiving-acting system to be distant from or close to afordances. When is an agent close to or distant from an afordance? Again, afordances are invariant combinations of properties of things at the ecological level taken with reference to an agent (Gibson, 1982/2020d, p. 558). To paraphrase Bunge (1977, p. 16), there would be no afordances in themselves fying above concrete things, being close to or distant from a perceiving-acting system, and I found these questions difcult to answer. I am also not quite sure what it would mean for afordances to come into play or exert their role. Can afordances per se exert their role? Afordances do not cause behavior. As (Gibson, 1979/2016, p. 215) famously put it, locomotion and

152

Tetsushi Nonaka

FIGURE 8.1

Flow feld around P. dumerilii larva, beating with its locomotor cilia. Flow rates are very low at the stagnation point indicated by the white arrow. Red dots mark the position of the fow sensors. The fow was visualized by fuorescent microbeads. Figure adapted from Jékely et al., (2021).

manipulation are neither triggered nor commanded but controlled by information by seeing oneself in the environment. Perceiving the afordances is by an agent in its environment, who strives to detect the information that specifes what encounters with the environment would aford. Without this informational grounding, as Warren (2022, p.  412) rightly pointed out, “[A]fordance talk can descend into wordplay.” How Do Systems Engage With Afordances as They Move Among Tasks and Intentions?

In terms of visual perception, this question is related to the issue of how it is possible for animals to be “visually oriented to the surfaces of their environment, not merely to light as such” (Gibson, 1998, p. 161). How is it possible

Description of the World That Agential Systems Fit Into

153

for an agent not only to perceive how to get about things and what to or not to do with them but also to control encounters with the environment? In 1709, George Berkeley suggested that the chief end of vision was for animals “to foresee . . . the damage or beneft which is like to ensue upon the application of their bodies to this or that body which is at a distance: which foresight how necessary it is to the preservation of an animal” (Berkeley, 1934, pp.  38–39). In 1979, exactly 270 years later, Gibson (1979/2015, p. 221) wrote that “what the philosopher called foresight is what I call the perception of the afordance.” Gibson agreed with Berkeley about the utility of vision: perceiving what an encounter with the environment will aford. They disagreed over how animals see what encounters with things aford by means of light. Berkeley observed that “what we immediately and properly see are only lights . . . All which visible objects are only in mind; nor do they suggest aught external, whether distance or magnitude, otherwise than by habitual connection as words do things” (Berkeley, 1934, p. 49). Thereby, Berkeley had to suppose that for animals to see the damage or beneft upon encountering things requires “the experience they had what tangible ideas are connected with such and such visible ideas” (Berkeley, 1934, p.  38), that is to say, the prior experience of the encounters themselves. On the other hand, Gibson (1979/2015, p. 221), based on the distinction between stimulation for passive receptors and information for active perceptual systems, suggested that information is available in the ambient light, which specifes what an encounter with the thing afords the perceiver. When information of perception is conceived as available in the ambient energy fux in the medium, not as signals in a bundle of nerve fbers, and when perception is thought of as the activity of a perceptual system that hunts for information in the ambient optic array, instead of as processes that transform the stimulus energy that the receptors receive, a new theory of control of encounters becomes possible (Gibson, 1979/2016, p 251). One important consequence of ecological optics is that two kinds of information are always available—one about the environment and another about the self (Gibson, 1979/2016, p. 228). Control of encounter with the environment depends on (1) perceiving the persisting, invariant afordances of the surface and (2) co-perceiving the self in its environment (e.g., the changing degree to which a task is not yet completed by one’s action). Consider the case of a bee foraging on fowers. The bee who lands on a fower must perceive the fower and control his fight. He has to see an invariant environment in order to identify the fower, and to see himself moving through the environment in order to guide his locomotion. (Gibson, 1982/2020e, p. 533)

154

Tetsushi Nonaka

It is true, as Gibson suggested, that the extracting of information for the perception of the world and the extracting of information for the bodily control of performances are diferent processes, even if complementary. The perception of a goal, its afordance, controls locomotion in one way whereas the visual proprioception of the optical outfow controls locomotion in an entirely diferent way. (Gibson, 1982/2020e, p. 533) An agent needs information about her activities and the environment to attain an end or goal of the task in the environment (Gibson, 1982/2020f, p. 232). The principle of ecological optics that all perceiving involves coperception of environment and self, according to Neisser (1988, p.  43), applies also to the social environment and to the interpersonal self, i.e., the self that is established in these interactions. Just as the ecological self is specifed by the orientation and fow of optical texture, so the interpersonal self is specifed by the orientation and fow of the other individual’s expressive gestures; just as the ecological self is articulated and confrmed by the efects of our own physical actions, so the interpersonal self is developed and confrmed by the efects of our own expressive gestures on our partner. My colleagues and I observed the development of controlling encounters with the human mealtime environment in infants around the time when they started using a spoon (Kasuya & Nonaka, 2023; Nonaka & Goldfeld, 2018; Nonaka & Stofregen, 2020). In early encounters with a spoon, the afordances of spoon infants actualized include mouthing, running fngers over, banging against an object, bouncing one end on the table, twisting with fngers, passing from hand to hand, rolling on the table, shaking, dropping, throwing, touching own head, rhythmical tapping on the table, and so on, which changed meal to meal. In these early instances of spoon use during mealtime, instead of inhibiting the infant’s improper spoon use, the caregiver tends to organize the environment that such goal-irrelevant activities of the infant are tolerated (e.g., by moving a cup flled with liquid out of the infant’s reach) and keeps feeding the infant while she is playing with the spoon (Nonaka & Goldfeld, 2018). Then when the infant starts orienting the spoon to the food, the caregiver would gradually introduce specifc afordances for functional feeding encounters (e.g., by steadying the dish to help the infant to get food on the spoon) that are appropriate to the socially based practice of mealtime. During this process, we found two distinct streams of reciprocal informational coupling between the caregiver and the novice spoon feeder (Nonaka & Stofregen, 2020): (1) The caregiver uses or adjusts the afordances available on the table, and the infant actively looks at the caregiver’s

Description of the World That Agential Systems Fit Into

155

hand (or a utensil in hand) acting on the objects in the environment. (2) When the infant actualizes one or another afordance of a spoon (e.g., ingestion of food or playful behavior), the infant looks at the face of the caregiver more often than chance, as if to obtain the information about the interpersonal self. These observations indicate that infants are concerned about the situation they are engaged in, actively exploring the information about what matters in the mealtime situation (Nonaka, 2022). In addition, the fact that the fow of the infant’s attention is reciprocally coupled to the fow of the caregiver’s action implies that the interpersonal context provides the foothold for the infant’s attention, who is actively developing a “nose” for specifc afordances appropriate in communal practice (Rietveld, 2008). Another important consequence of Gibson’s information-based theory of perception is that there can be information in the present for the future. For example, the rate of magnifcation of a visual solid angle that corresponds to a surface in the optic array specifes the imminence of the collision to the surface in the near future (Gibson, 1979/2016; Lee & Reddish, 1981). Gibson (1978b, italics original) suggested that there are cases where “an event that ‘casts its shadow before’, i.e., perception ‘ahead of time’, where the fnal event is specifed by the subordinate events leading up to it.” “This holds true,” Gibson (1978b) continued, “when an outcome is implicit in the prior stages of a total event (a ‘temporal Gestalt’). The end exists throughout. The outcome is ‘inevitable’ unless the ‘course of events’ is altered, i.e., unless an extraneous event is introduced.” In another memo, Gibson (Gibson, 1978c, italics original) asked: Is it true that one can foresee the outcome of a course (of events; of action), i.e., perceive it not just imagine it? Is prediction automatic in some cases? What cases? Are there any cases of ‘causally connected’ events whose causes are understood? Can we learn to perceive causation? Or better, can one learn to perceive inevitability? One study, among others, examined an issue related to these questions. Nonaka et al. (2010) experimentally tested how stone knappers (craftspeople who make stone tools) foresee and control the outcome of fracture events resulting from their faking actions. Nonaka et  al.’s hypothesis was that experts could foresee and control outcomes by tuning their actions to the lawful regularities that exist in principle called a conchoidal fracture used in fracturing a stone. Previous conchoidal fracture experiments using a protocol in which a steel ball is dropped on plate glass documented an invariant relation among particular variables (Dibble & Pelcin, 1995). The size of the stone fragments (called fakes) depends on the combination of two variables—the

156

Tetsushi Nonaka

FIGURE 8.2

(a) Ventral surface of a fake detached by conchoidal fracture, as hit by a stone hammer in (b). Flake terminology is provided in (b). Courtesy of Jose-Manuel Benito Alvarez.

angle of the edge and the distance between that edge and the point of the percussion (Figure 8.2). The force with which the hammer strikes a block of stone (called core) does not afect the fake size once it is above the threshold of fracture initiation (i.e., the minimum amount of force needed to remove a fake of a given size). The threshold of fracture initiation depends on fake size, which in turn depends on the striking location. Although they are by no means the only variables that afect the outcome of conchoidal fracture, importantly, these variables are all under the direct control of the actors. Before the detachment of a fake, do knappers attend to such lawful relations to foresee and control the outcome of a strike given to a core? In the experiment, participants—including prominent replica craftspeople—were asked to draw the fracture path’s outline on the fint core’s surface expected to result from the blow they would deliver at the core (Nonaka et al., 2010). After outlining the fracture path, the participants were asked to proceed to fracture the stone as expected. The result of the experiment indicated that the task was very difcult. However, a few expert knappers

Description of the World That Agential Systems Fit Into

157

proved capable of controlling the fracture path almost exactly as they had expected. Confrming the hypothesis above, it turned out that the outline of the fracture path drawn prior to the fake removal by those experts who succeeded in the task already exhibited the lawful relation among the three variables—the angle of the exterior edge, the distance between the point of percussion to the exterior edge, and the fake dimensions that refect the constraints of conchoidal fracture (Figure 8.2). Furthermore, it was found that experts hit the core using the hammerstone with lower kinetic energy than nonexperts and that only experts controlled the kinetic energy of the hammerstone at impact in relation to the to-be-detached fake size (Nonaka et al., 2010). Experts were aware of the properties of the core and the mass of the hammerstone at hand in the sense that they detached fakes without excessively overshooting the required kinetic energy at impact. These results suggested that “foresight” involved in the control of stone faking depends on the continuous participation of the knapper’s behavior in the lawful regularity of conchoidal fracture. This participation, in turn, is made possible by the perceptual attunement of the knapper to discriminate a specifc layout of the surface of the core that has a consequence on the future fracture event (Nonaka et al., 2010). The path of fracture resulting in a fake is entailed by the natural unfolding of the system of which the behavior of the knapper is a part (cf. Stepp & Turvey, 2010). In other words, foresight in control of stone faking is a matter of focusing the perception and behavior of an actor on the informative structure that specifes the inevitability of the environmental event in which the actor can take part (Nonaka, 2012, 2020, 2023). This may be one piece of evidence that supports the idea that one can perceive the outcome of a course of action and that one can learn to perceive the inevitability of causally connected events to control encounters with the environment that aford beneft or cost. Acknowledgments

I thank Bill Mace for kindly sharing Gibson’s unpublished notes on the control of encounters. I am grateful to the Editors for afording me the opportunity of writing this chapter. This work was supported by the JSPS KAKENHI (grant numbers JP21H05823, JP21KK0182, and JP22H00988). Reference List Berkeley, G. (1934). A New Theory of Vision and Other Writings. JM Dent & Sons. Bunge, M. (1977). Treatise on Basic Philosophy: Ontology I: The Furniture of the World (Vol. 3). D. Reidel Publishing Company.

158

Tetsushi Nonaka

Darwin, C. (1892). The Formation of Vegetable Mould, Through the Action of Worms: With Observations on Their Habits. William Clowes Ltd. Dibble, H. L., & Pelcin, A. (1995). The efect of hammer mass and velocity on fake mass. Journal of Archaeological Science, 22(3), 429–439. https://doi. org/10.1006/jasc.1995.0042 Gibson, J. J. (1961). Ecological optics. Vision Research, 1(3), 253–262. https://doi. org/10.1016/0042-6989(61)90005-0 Gibson, J. J. (1967). For new book (“Useful vision”?). In Division of Rare and Manuscript Collections. Cornell University Library. Gibson, J. J. (1976). The myth of passive perception: A reply to Richards. Philosophy and Phenomenological Research, 37(2), 234–238. https://doi. org/10.2307/2107194 Gibson, J. J. (1978a). How do we control contacts with the environment?: The problem of foresight. In Division of Rare and Manuscript Collections. Cornell University Library. Gibson, J. J. (1978b). The evolution of locomotion and manipulation. In Division of Rare and Manuscript Collections. Cornell University Library. Gibson, J. J. (1978c). What is it to expect? In Division of Rare and Manuscript Collections. Cornell University Library. Gibson, J. J. (1979/2016). The Ecological Approach to Visual Perception. Psychology Press (Originally published in 1979). Gibson, J. J. (1982/2020a). A note on what exists at the ecological level of reality. In E. Reed & R. Jones (Eds.), Reasons for Realism: Selected Essays of James J. Gibson (pp. 566–568). Routledge (Originally published in 1982). Gibson, J. J. (1982/2020b). Note on the distinction between stimulation and stimulus information. In E. Reed & R. Jones (Eds.), Reasons for Realism: Selected Essays of James J. Gibson (pp. 477–487). Routledge (Originally published in 1982). Gibson, J. J. (1982/2020c). A history of the ideas behind ecological optics: Introductory remarks at the workshop on ecological optics. In E. Reed & R. Jones (Eds.), Reasons for Realism: Selected Essays of James J. Gibson (pp. 135–150). Routledge (Originally published in 1982). Gibson, J. J. (1982/2020d). Afordances and behavior. In E. Reed & R. Jones (Eds.), Reasons for Realism: Selected Essays of James J. Gibson (p.  558). Routledge (Originally published in 1982). Gibson, J. J. (1982/2020e). Notes for a tentative redefnition of behavior. In E. Reed & R. Jones (Eds.), Reasons for Realism: Selected Essays of James J. Gibson (pp. 528–533). Routledge (Originally published in 1982). Gibson, J. J. (1982/2020f). The use of proprioception and the detection of propriospecifc information. In E. Reed & R. Jones (Eds.), Reasons for Realism: Selected Essays of James J. Gibson (pp. 231–239). Routledge (Originally published in 1982). Gibson, J. J. (1998). Visually controlled locomotion and visual orientation in animals. Ecological Psychology, 10(3–4), 161–176. https://doi.org/10.1080/10407 413.1998.9652681 Guasto, J. S., Johnson, K. A., & Gollub, J. P. (2010). Oscillatory fows induced by microorganisms swimming in two dimensions. Physical Review Letters, 105(16), 168102. https://doi.org/10.1103/PhysRevLett.105.168102

Description of the World That Agential Systems Fit Into

159

Guasto, J. S., Rusconi, R., & Stocker, R. (2011). Fluid mechanics of planktonic microorganisms. Annual Review of Fluid Mechanics, 44(1), 373–400. https:// doi.org/10.1146/annurev-fuid-120710-101156 Holt, E. B. (1916). The Freudian Wish and Its Place in Ethics. Henry Holt and Company. Ingold, T. (1987). The Appropriation of Nature: Essays on Human Ecology and Social Relations. University of Iowa Press. Jékely, G., Godfrey-Smith, P., & Keijzer, F. (2021). Reaference and the origin of the self in early nervous system evolution. Philosophical Transactions of the Royal Society B: Biological Sciences, 376(1821), 20190764. https://doi.org/10.1098/ rstb.2019.0764 Kandel, E. R., Schwartz, J. H., Jessell, T. M., Jessell, D. of B. and M. B. T., Siegelbaum, S., & Hudspeth, A. J. (2013). Principles of Neural Science (5th ed., Vol. 4). McGraw-Hill. Kasuya, J., & Nonaka, T. (2023). When do toddlers point during mealtime?: Pointing in the second year of life in everyday situations. Frontiers in Psychology, 14, 1050975. https://doi.org/10.3389/fpsyg.2023.1050975 Lee, D. N., & Reddish, P. E. (1981). Plummeting gannets: A paradigm of ecological optics. Nature, 293(5830), 293–294. https://doi.org/10.1038/293293a0 Levin, M. (2022). Collective intelligence of morphogenesis as a teleonomic process. PsyArXiv. https://doi.org/10.31234/osf.io/hqc9b Neisser, U. (1988). Five kinds of self-knowledge. Philosophical Psychology, 1(1), 35–59. https://doi.org/10.1080/09515088808572924 Nonaka, T. (2012). What exists in the environment that motivates the emergence, transmission, and sophistication of tool use? Behavioral and Brain Sciences, 35(4), 31–32. https://doi.org/10.1017/S0140525X11002056 Nonaka, T. (2020). The triad of medium, substance, and surfaces for the theory of further scrutiny. In J. B. Wagman & J. J. C. Blau (Eds.), Perception as Information detection: Refections on Gibson’s Ecological Approach to Visual Perception (pp. 21–36). Routledge. Nonaka, T. (2022). Activation of stance by cues, or attunement to the invariants in a populated environment? Behavioral and Brain Sciences, 45, e263. https://doi. org/10.1017/S0140525X22001261 Nonaka, T. (2023). Towards an ecology of evolving skills. In T. Wynn, K. A. Overmann, & F. L. Coolidge (Eds.), Oxford Handbook of Cognitive Archaeology (pp. 00–00). Oxford University Press. Nonaka, T., Bril, B., & Rein, R. (2010). How do stone knappers predict and control the outcome of faking? Implications for understanding early stone tool technology. Journal of Human Evolution, 59(2), 155–167. https://doi.org/10.1016/j. jhevol.2010.04.006 Nonaka, T., & Goldfeld, E. C. (2018). Mother-infant interaction in the emergence of a tool-using skill at mealtime: A process of afordance selection. Ecological Psychology, 30(3), 1–21. https://doi.org/10.1080/10407413.2018.1438199 Nonaka, T., & Stofregen, T. A. (2020). Social interaction in the emergence of toddler’s mealtime spoon use. Developmental Psychobiology, 62(8), 1124–1133. https://doi.org/10.1002/dev.21978

160

Tetsushi Nonaka

Reed, E. S. (1993). The intention to use a specifc afordance: A conceptual framework for psychology. In R. H. Wozniak & K. W. Fischer (Eds.), Developing in Context: Acting and Thinking in Specifc Environments (pp. 45–76). Erlbaum. Reed, E. S. (1996). Encountering the World: Toward an Ecological Psychology. Oxford University Press. Rietveld, E. (2008). Situated normativity: The normative aspect of embodied cognition in unrefective action. Mind, 117(468), 973–1001. https://doi.org/10.1093/ mind/fzn050 Sasaki, M., & Nonaka, T. (2016). The reciprocity of environment and action in self-righting beetles: The textures of the ground and an object, and the claws. Ecological Psychology, 28(2), 78–107. https://doi.org/10.1080/10407413.201 6.1163983 Smith, B. (2009). Toward a realistic science of environments. Ecological Psychology, 21(2), 121–130. https://doi.org/10.1080/10407410902877090 Stepp, N., & Turvey, M. T. (2010). On strong anticipation. Cognitive Systems Research, 11(2), 148–164. https://doi.org/10.1016/j.cogsys.2009.03.003 Warren, W. H. (2022). Epilogue: The cartesian submariner learns to surf. In A.  Szokolszky, C. Read, & P. Zsolt (Eds.), Intellectual Journeys in Ecological Psychology: Interviews and Refections from Pioneers in the Field (pp. 410–430). Routledge.

9 THE ROLE OF EXPLORATORY ACTIVITY IN AFFORDANCE PERCEPTION Alen Hajnal

What Do We Understand by the Term Afordance?

Variability is an essential property of physical and biological systems. Physical events and living organisms’ behavior are expressions of variability fuctuations at multiple scales. Behaviors are made possible by active exploration and sampling of the environment. Afordances are bifurcations of behaviors that propel the organism-environment system through changes. A chair either afords sitting down or not, depending on the ft between organismic and environmental properties. The properties are detected via exploratory activity (Mark et al., 1990). Active exploration is key to ensure that changes are not random and become permanent skills. Active exploration is a means through which ambient energy arrays are detected and sampled under various transformations and spatiotemporal scales. The science of complex systems adopts the “strategic reductionism” method by acknowledging that not any scale is privileged over another and that behavior emerges from simultaneous interactions at multiple scales. The fow of information through the system exhibits itself as the complex marshaling of fuctuations and behavioral variability. As such, variability is treated as an integral part of biological systems, not random noise. Proponents of the ecological approach to perception and action explicitly state that variability is a necessary component of active exploratory activity that plays a part in sampling ambient energy arrays for information detection (Gibson, 1979). Complementary theories of motor behavior state that variability is an inherent and necessary feature of motor systems

DOI: 10.4324/9781003396536-12

162

Alen Hajnal

that exhibit context-conditioned variability (Bernstein, 1967; Turvey et al., 1982). Contrary to traditional information processing theories and classic engineering approaches, variability is part of the system just as much as the “signal” is part of the system. Time series recordings of movement patterns have been shown to exhibit self-correlation at multiple scales. Self-correlation of variability patterns at multiple scales is assessed and computed by multifractal algorithms. Multifractality globally describes a complex system by taking into account changes and interactions at all levels of the system. The present contribution conjectures that fuctuations (neuronal, behavioral, and environmental) measured in any part of the system should, in principle, be equally representative of the whole system. Traditionally, nonspecifc movement patterns, such as postural adjustments, tremors, and foraging activity, were treated as noise and eliminated from further analysis and consideration. In the next section, we review the growing literature that shows how seemingly nonspecifc movement patterns of the body and physiological fuctuations shape and predict behavior in functional tasks (such as the perception of afordances). What Role Does Afordance Play in Perception? Is It the Entity Organisms Perceive or the Means Through Which Organisms Perceive? Are Afordances the Only Perceptual Dependent Variables?

The behavior of living systems can be described as a series of bifurcations where the organism performs an action because it is possible to perform it and intends to do it. One action leads to the next by narrowing or widening the possibilities for subsequent actions, given the organism’s goals. Afordances are real, physical, embodied possibilities for action that can be perceived (although other entities are perceivable, too, like events). In terms of a trafc metaphor, an action is a particular road the organism takes when it arrives at a fork (i.e., a bifurcation) in the road. The fork in the road is the essence of any afordance. The goal of behavior is defned by the endpoint of the journey and the particular direction in which it unfolds. To stay on the proverbial “road,” the organism must continuously explore and be attentive to its body movements and the “road conditions.” This exploratory activity is an essential component of the action itself (actualization of the afordance) that enables the organism to “stay the course” while harnessing the fexibility of the body’s degrees of freedom and fexibly exploiting the task constraints that are defned by the behavioral goal. Exploration is best described as a complex patterning of energy measured by multifractality. There are a growing number of empirical

The Role of Exploratory Activity in Afordance Perception

163

accounts that connect the multifractality of motor behavior with psychological processes, such as perception (Stephen & Hajnal, 2011), intention (Palatinus et  al., 2014), and insight (Stephen et  al., 2009). Linking the fractality of movement to perception of postural stability is an emerging feld of investigation. Priplata et al. (2003) have shown that wearing vibrating shoe insoles prevents falls in the elderly only when the vibration pattern has a fractal signature instead of producing random vibrations. Nonlocomotory postural adjustments during functional tasks can serve as exploratory activity aiding information detection (Stofregen et  al., 2005). The multifractal structure of variability in postural sway is a good predictor of perceptual responses above and beyond standard measures of central tendency and variance in afordance tasks (Doyon et al., 2019; Hajnal et al., 2018, 2022; Masoner et al., 2020). Fractality of heart rhythms has been shown to predict heart attacks (Peng, Havlin, Hausdorf, et al., 1995), and fractality of locomotory behavior is diagnostic of the early onset of Parkinson’s disease (Hausdorf, 2007). Many classic cognitive and motor tasks are efcaciously described by measurements of multifractal fuctuations, such as the visual Fitts task (Eddy & Kelty-Stephen, 2015), visual recognition of human point-light displays (Palatinus et al., 2013), length perception, displacement of center-of-pressure during upright stance (Delignières et  al., 2003), human eye movements (Billock et al., 2001; Kelty-Stephen & Mirman, 2013; Yokoyama et  al., 1996), intention (Palatinus et  al., 2014), conversation dynamics (Ashenfelter et al., 2009), anxious phobic disorders (Dick et al., 2012), team coordination (Likens et al., 2014), decision making (Ross, 2014), executive function (Anastas et al., 2014), schizophrenia (Takahashi et al., 2009) and many others. Multifractality has also been introduced as a useful descriptor of animal behavior. Spatial and temporal measures of biological systems have often been found to exhibit fractal scaling. Animal vocalizations such as feeding clicks of yellow seahorse (Haris et  al., 2014), and the structure of songbird rhythms (Roeske et al., 2018) have been successfully characterized by multifractal measures. Albatross search patterns (Humphries et  al., 2012; Viswanathan et  al., 1996), wolf search paths (Bascompte & Vilà, 1997), mammalian social hierarchies (Hill et  al., 2008), copepod movement patterns (Schmitt & Seuront, 2001), bottlenose dolphin dive durations (Seuront & Cribb, 2011), and foraging behavior among wild Japanese macaques have been described by self-similarity measures and used to distinguish between healthy and diseased individuals (MacIntosh et al., 2011). Similarly, fractal measurements are indicative of compromised health among chimpanzees (Alados & Hufman, 2000), toxic

164

Alen Hajnal

substance exposure among fathead minnows (Alados & Weber, 1999), food stress among domestic chickens (María et  al., 2004), and aggressive social interactions among domestic pigs (Rutherford et al., 2006). A recent study by Gu et al. (2015) established a connection between physical ftness and the fractality of movement patterns in mice: exercise made the movement patterns more self-correlated. The applicability of fractal measures is not restricted to the animal kingdom as fractal scaling is also present in plant development, for example, tree growth (Zeide & Gresham, 1991). Physiological and anatomical characteristics, such as vascular structure (Hermán et al., 2001), DNA sequences (Peng, Havlin, Stanley, et al., 1995), and human respiration (Thamrin & Stern, 2010), have also been efectively described by these complex measures. The structure of sense organs such as the interior of the human cornea and the lens (most notably the spatial distribution of tissue elements) has also been shown to adhere to scale invariance (Clark, 2001). Importantly, multifractality describes the physical environment in which organisms live. Smooth fat surfaces with multifractal spatial topography are better suited for interactions with light and produce more energy if used as solar panels (Jurečka et al., 2014). Light, the visual perception medium, exhibits multifractal properties when scattered (Shayeganfar et al., 2009). Furthermore, photographic images of natural scenes such as rocky beaches also exhibit multifractal patterns when analyzed in terms of the fuctuation patterns of pixel intensity changes (Turiel & Parga, 2000). In summary, multifractality is present in the physical environment, in the patterns of the physical medium (light) that serve as the stimulus for visual perception, in the anatomical structure of sense organs such as the eye cornea, and in the behavior of living organisms as well. Multifractal measures can be used to (i) predict perceptual, and motor responses above and beyond standard measures of central tendency and standard measures of variability; (ii) diagnose certain neurological disorders; and (iii) treat certain disorders of motor behavior such as postural instability and fall risk. In more general terms, multifractality promises to have great explanatory power for human behavior, animal behavior, and the behavior of physical systems that are relevant to living creatures. There is also an empirical challenge to fnd the kind of measurement and description for complex behavior that allows us to gauge the role of interactivity in the apparent behavioral changes. Brain imaging and EEG measurements are based on average data (across trials and participants) to infer something about the localization of function. The measure we propose is based on raw data and treats individual diferences not as a random source of variability but as an essential component of behavior. Furthermore, the goal of time series analyses of various behavioral patterns

The Role of Exploratory Activity in Afordance Perception

165

(body movements) and physiological signals (heartbeat, skin conductance amplitudes, etc.) is not to localize a brain function; rather, it is to predict the occurrence and outcome of cognitive, perceptual, and motor behavior. Whereas the localization hypothesis assumes a single process that drives the system, a neural pattern responsible for the observed changes in behavior, we suspect the participation of many processes in and outside the central nervous system. Fractal analyses started as a search for a single parameter that describes a complex system’s behavior. Behaviors that can be captured with a single parameter are relatively rare in nature. The fexibility, intelligence, and context sensitivity of living systems hinge upon many loosely coupled processes that make up those systems. Fractal analysis is a kind of spectral analysis designed to work on noisy data and determine the properties of many participating processes. Multifractal analysis is designed to capture the heterogeneity in the emergence of behavior and assess the characteristic changes as the system reorganizes to address an environmental issue. Multifractal analysis is good at predicting and capturing transitions in behavior, such as fnding an item, fnding a solution, or recognizing something or someone. Wider multifractal spectra inform us that a multitude of cumulative processes force and yield to each other when creating the observed behavior. In other words, many degrees of freedom are being used. When the spectrum is narrow, it tells us that there are fewer main players, and the system is becoming more deterministic and relies on fewer degrees of freedom. Multifractal analysis helps us describe and predict the behavior of a system in ways it was not possible before. The general notion of context sensitivity provides the theoretical underpinning of spectral analyses. Context sensitivity means that a complex system includes the organism, the environment, and the interactions between the two that shape the system’s behavior. A more specifc motivation for the usefulness of spectral analyses for the science of complex systems lies in embodied cognition, the idea that bodily experiences and environmental infuences are both essential components of complex cognitive systems. Gottlieb’s notion of probabilistic epigenesis (Gottlieb, 2007) is the biological instantiation of a complex, multiply embedded, embodied system in which information fows through multiple levels and in multiple directions. Behavior unfolds over time due to the abundance of multiscale interactions that usher the organism through its various stages of behavior on the short-time scale and through developmental stages along the long-time scale. The most generic proposition of the present project is that complex systems adhering to the principles of probabilistic epigenesis are best described by spectral analyses such as multifractal analysis.

166

Alen Hajnal

Why Is Afordance Any Better Than Stimulus? What Does a Theory of Afordance Suggest That Stimuli Cannot; How Has It Moved the Needle Past Gibson 1960’s Recognition of How Little We Can Defne Stimuli?

Fluctuations are the “currency” by which behaviors are assembled, sustained, and detected. Detection is at the core of the defnition of perception. The implicit assumption of traditional perception science is that detecting information (i.e., perception) is a conscious activity. Consciousness enables agency and separates perception from passive states such as sleep. A further corollary of this description is that a minimum amount of energy pattern can be detected, called a threshold. This assumes that detecting the information below the threshold is not meaningful and does not contribute to behavior. However, our proposition is that detection is possible below thresholds, even without conscious awareness and explicit attention. As a concept at best, threshold is probabilistic and meaningless at worst. Behavior can be afected by information detected without conscious awareness or overt attention. Just as physical instruments exist that can amplify a weak but real energy array, subthreshold signals should, in principle, permit the detection of information. Davis et  al. (2014) have demonstrated that video footage of a bag of chips can be used to reconstruct environmental sounds that are heard in the video by analyzing minute displacements of the bag as sound waves bounce of its surface. Although Davis’ algorithm did not use multifractal analysis, our current proposition shares a common goal with theirs: to extract information about behavior and events from minute spatiotemporal changes of any aspect or element of the situation. Davis’ demonstration highlights an important aspect of complex systems: energy arrays (visual, acoustic, haptic) present in a behavioral context permeate every aspect of the task environment and are equally available for a perceiver. The notion of stimulus energy arrays has been formally introduced to perceptual psychology by James J. Gibson (1979, 1960). The essential features of optic, acoustic, and haptic arrays are gradients of change that remain stable over spatiotemporal transformations. According to Gibson, perceivers can extract information about events and behaviors by attending to any part of the energy array at any location for any period. This includes any surface, sound bite, or light pattern available for the perceiver to detect. These patterns of change specify unfolding behaviors and events. Multifractal analysis and other analyses of variability are the best ways to analyze these arrays precisely because their focus is on building an inventory of the amount and type of changes that complex systems undergo during each bout of activity, be it cognitive or physical activity. The changes in energy arrays are termed energy fows and are a function of unfolding behavior. For example, optic

The Role of Exploratory Activity in Afordance Perception

167

fow is generated by the patterning of ambient light as the organism moves. Acoustic fows are generated similarly as a function of behavior. The third challenge faced by our approach to complex cognitive systems concerns the design of a viable experimental paradigm to test the theoretical hypothesis that behavioral and physiological fuctuations of various kinds are just as useful for explaining typical cognitive processes as standard neuroscientifc measures are. If we treat the system as truly integrated and complex, movement patterns of the body should be as closely mapped onto cognitive processes as changes in oxygenation levels in parts of the brain captured by various brain imaging techniques. Consequently, no theoretical or logical argument would claim the primacy of the latter measurement over the former in terms of explanatory power. Typical measurements obtained in behavioral neuroscience experiments may place cognitive processes at the narrow end of the multifractal spectrum width. On the other hand, global measures of bodily movements, not confned to fuctuations in the brain, may produce a wider multifractal spectrum width. Three classes of experiments are suitable to investigate the relevance of fuctuations in explaining cognitive processes: (i) the efect of environmental fuctuations, (ii) the efect of behavioral fuctuations, and (iii) the efect of the interaction of environmental and behavioral fuctuations. The motivation for studying the efect of environmental fuctuations is based on the observation that visual perception takes place in a constantly changing environment that is interfaced with the fuctuating medium of light registered by an eye that is itself in constant motion (O’Regan, 1992). Static environments, static stimuli, and static sense organs are misleading simplifcations because, with them, researchers are forced to disregard all the rich variability that is essential to normal functioning. A typical problem in vigilance tasks, such as the job of air trafc controllers, is the lack of changes in the visual task environment. We propose that adding subtle visual noise patterns to the visual stimulus would actually help maintain sustained attention and perceptual performance. Complex noise patterns could be used to guide attention as a sort of visual “prosthetic device” in the same way that vibrating shoe insoles aid in the maintenance of stable posture in certain patient populations (Priplata et  al., 2003). By adding visual noise to artifcial stimuli (such as radar images), we are reconstructing the natural complexity of natural stimuli (Turiel & Parga, 2000). The motivation for studying the efect of behavioral fuctuations is based on recent perception and action research fndings. One demonstrative example is by Stephen and Hajnal (2011), who have shown that nonstationary free wielding of handheld objects exhibits fractal patterns of variability. This fractal pattern predicts perceptual length judgments about wielded objects above and beyond standard central tendency and

168

Alen Hajnal

variability measures. There are a growing number of studies in which it was demonstrated that the deep structure of variability could predict perceptual responses (i.e., multifractality) of body movements during exposure to visual and haptic stimuli (Doyon et al., 2019, 2021; Hajnal et al., 2018; Palatinus et al., 2014). Finally, a thoroughgoing analysis of fuctuations in stimulus energy arrays is necessary as they contain “traces” of events and activity in the immediate surroundings. A suitable medium to study would be the scattering of light. Similarly to the analysis of sound patterns done by Davis et al. (2014) the proposal is to analyze video footage of a seemingly uniform surface that refects light and creates a Ganzfeld pattern of stimulation. The question is whether this Ganzfeld pattern of scattered light contains information about events and activity nearby that takes place of-camera. High-speed cameras can extract object shapes from refected and scattered light (Faccio et al., 2020; O’Toole et al., 2018). Without apparent shadows on the surface, the only information is the complex scattering pattern of light. We hypothesize that multifractal analysis of the video footage would be quantifably specifc to the of-camera activity. For example, if two people are having a conversation in the room just of-camera, the variability patterns in the video footage should refect this activity as compared to conditions where the two partners are sitting quietly without talking. Should this link between scattered light and nearby activity be proven, it would open new avenues for research into the very nature of stimulus energy arrays and fow patterns that are taken for granted by standard psychophysics and perceptual psychology. What Is the Connection Between Afordances, Behavioral Scale, and Intention? How Distant or Reluctant Can a PerceivingActing System Be Before Afordances Come Into Play? Do Afordances Exert Their Role Only When the System Is Close or Willing Enough?

Developing a new kind of psychophysics based on multiscale interactions is needed. Traditional psychophysics asks how a physical property is translated into sensations and percepts and assumes that the sensory apparatus merely registers basic physical properties of the surrounding energy distribution. Under this view, it is assumed that (1) the sensory system is a passive receiver of stimulation, (2) only low-level primitives are registered, and the nervous system must create a coherent perception from those primitives via a series of computations embodied in brain mechanisms. We wish to avoid making either of these two assumptions. As our literature review showed, there are good reasons to think that the sensory apparatus (and its constituents down to the neural level) participates in multiscale fuctuations that

The Role of Exploratory Activity in Afordance Perception

169

shape perception, action, and cognition. More importantly, we hypothesize that there is a continuous, unbroken fow of interactions between the physical, biological, neural, and psychological events according to the principles of Gottlieb’s theory of probabilistic epigenesis. All these levels are active and interconnected, and their co-fuctuations are traceable with existing analytical techniques. What has been demonstrated in the past decade suggests that signifcant discoveries are within reach; a new chapter is already being written about how nature creates living, perceiving, and acting systems. Our review aimed to demonstrate that fuctuations of biological systems embedded in dynamically changing physical environments produce behaviors that support perception, action, and cognition of equivalent complexity. The job of complex science is to uncover the connections among the many instantiations of these fuctuation patterns at all scales. The proposed approach based on treating variability as a central “connective tissue” of complex systems has consequences for two infuential concepts of experimental psychology: threshold and stimulus. The consequence of assuming that all scales contribute to efcacious behavior is that the notion of a psychophysical threshold becomes obsolete. Behavior is just as afected by variability below a psychophysical threshold as it is by a threshold of apprehension for a given task. To paraphrase Gibson’s adage of “behavior is regular without being regulated” (Gibson, 1979), we can stake our claim that behavior is shaped at multiple scales (large and small) without being constrained by a single, privileged scale. Likewise, the notion of a stimulus also becomes meaningless. Under the assumption of a highly interconnected and interactive system, it does not make sense to posit the existence of a static, image-like, modular, context-independent snapshot of the dynamically unfolding behavior as a useful description. The concepts of the threshold and stimulus have been valuable metaphors under the mechanistic description of active systems that cashed in on the apparent power of representational theories of cognition and behavior. The hubris of these theories has been to overtly anthropomorphize all organisms and to relegate all the material world to passive matter. This false dualism has prevented scientists from noticing the immense richness of fuctuations as a hallmark feature of complex dynamical systems that drives intentional activity. The hallmark characteristic of complex systems is their interconnectedness at multiple spatial and temporal scales. The consequence of such interconnectedness is the existence of long-range correlations among distal component parts. This is why measurements at diferent scales and component parts can be considered equivalent, that is, equally informative about the state of the system, its functions, and performance. In this sense, categories such as far and close, willingness or reluctance to act, cannot be defnitively determined. Boundaries and ownership are also not

170

Alen Hajnal

clearly defned. This is not considered to be a shortcoming of the system. Instead, scientists should focus on defning spatiotemporal scales’ relevance and pragmatic values. In this theoretical framework, afordances serve as a pragmatic scafolding upon which behavior is built. The process of assembling behavior is one of efciently marshaling the fow of energy via the exploratory activity of perceptual systems and performative activity of the motor systems. This can be accomplished by functional specifcity of afordances (Stephen & Hajnal, 2011; Surber et al., 2022; Wagman & Hajnal, 2014) by performing the same task using diferent component parts, or inversely, by performing multiple tasks using the same component parts.

How Do Systems Engage With Afordances as They Move Among Tasks and Intentions?

An intention can be described as a goal-oriented potential action. The realization of intentions requires efort on the part of the system, that is, a plan to temporarily recruit component parts to expend energy in executing a skillful action. Efort is a behavioral amalgamation of intention and attention, that is, embodied orientedness toward a task. Efort (Bills, 1927, 1937; Hajnal et al., 2014, 2016) stabilizes behavior by enforcing the connections among softly assembled component parts to perform a given task. In sports, the “quiet eye” (Jacobson et al., 2021; Vickers, 2011) is an example of efort to focus one’s attention by stabilizing one’s posture and gait to accurately perform a behavioral task. Perception of afordances is a function of efort exhibited as the marshaling of fuctuations and variability in ambient energy arrays. The environment and task constraints shape the marshaling of fuctuations. Efort measures physical, neural, biological, and cognitive resistance required to overcome said constraints. This resistance is expressed as oscillation and variability at multiple scales. To the extent that behavior is treated as an enabling constraint (Raja & Anderson, 2021) on the organism’s activity at multiple scales, efort describes the cumulative resistance at diferent scales to constraints that shape the fow of energy through the system to accomplish a task. Systems engage with diferent functional tasks (i.e., afordances) by redistributing efort at multiple scales diferently as task demands dictate and change. The essence of the agency is efortful functional behavior. Reference List Alados, C. L., & Hufman, M. A. (2000). Fractal long-range correlations in behavioural sequences of wild chimpanzees: A non-invasive analytical tool for the evaluation of health. Ethology, 106(2), 105–116. https://doi. org/10.1046/j.1439-0310.2000.00497.x

The Role of Exploratory Activity in Afordance Perception

171

Alados, C. L., & Weber, D. N. (1999). Lead efects on the predictability of reproductive behavior in fathead minnows (Pimephales promelas): A mathematical model. Environmental Toxicology and Chemistry, 18(10), 2392–2399. https:// doi.org/10.1002/etc.5620181038 Anastas, J. R., Kelty-Stephen, D. G., & Dixon, J. A. (2014). Executive function as an interaction-dominant process. Ecological Psychology, 26(4), 262–282. https://doi.org/10.1080/10407413.2014.957985 Ashenfelter, K. T., Boker, S. M., Waddell, J. R., & Vitanov, N. (2009). Spatiotemporal symmetry and multifractal structure of head movements during dyadic conversation. Journal of Experimental Psychology: Human Perception and Performance, 35(4), 1072–1091. https://doi.org/10.1037/a0015017 Bascompte, J., & Vilà, C. (1997). Fractals and search paths in mammals. Landscape Ecology, 12(4), 213–221. https://doi.org/10.1023/A:1007990703139 Bernstein, N. A. (1967). The Co-Ordination and Regulation of Movements. Pergamon Press. Billock, V. A., de Guzman, G. C., & Scott Kelso, J. A. (2001). Fractal time and 1/f spectra in dynamic images and human vision. Physica D: Nonlinear Phenomena, 148(1), 136–146. https://doi.org/10.1016/S0167-2789(00)00174-3 Bills, A. G. (1927). The infuence of muscular tension on the efciency of mental work. American Journal of Psychology, 38(2), 227–251. https://doi. org/10.2307/1415202 Bills, A. G. (1937). Fatigue in mental work. Physiological Reviews, 17(3), 436– 453. https://doi.org/10.1152/physrev.1937.17.3.436 Clark, J. I. (2001). Fourier and power law analysis of structural complexity in cornea and lens. Micron, 32(3), 239–249. https://doi.org/10.1016/S0968-4328 (00)00044-5 Davis, A., Rubinstein, M., Wadhwa, N., Mysore, G. J., Durand, F., & Freeman, W. T. (2014). The visual microphone: Passive recovery of sound from video. ACM Transactions on Graphics, 33(4), 1–10. https://doi.org/10.1145/2601097.2601119 Delignières, D., Deschamps, T., Legros, A., & Caillou, N. (2003). A methodological note on nonlinear time series analysis: Is the open-and closed-loop model of Collins and De Luca (1993) a statistical artifact? Journal of Motor Behavior, 35(1), 86–96. https://doi.org/10.1080/00222890309602124 Dick, O. E., Svyatogor, I. A., Ishinova, V. A., & Nozdrachev, A. D. (2012). Fractal characteristics of the functional state of the brain in patients with anxiuos phobic disorders. Human Physiology, 38(3), 249–254. https://doi.org/10.1134/ S036211971202003X Doyon, J. K., Clark, J. D., Hajnal, A., & Legradi, G. (2021). Efects of surface luminance and texture discontinuities on reachableness in virtual reality. Ecological Psychology, 33(1), 1–30. https://doi.org/10.1080/10407413.2020.1820336 Doyon, J. K., Hajnal, A., Surber, T., Clark, J. D., & Kelty-Stephen, D. G. (2019). Multifractality of posture modulates multisensory perception of stand-on-ability. PLoS One, 14(2), e0212220. https://doi.org/10.1371/journal.pone.0212220 Eddy, C. L., & Kelty-Stephen, D. G. (2015). Nesting of focal within peripheral vision promotes interactions across nested time scales in head sway: Multifractal evidence from accelerometry during manual and walking-based ftts tasks. Ecological Psychology, 27(1), 43–67. https://doi.org/10.1080/10407413.2015. 991663

172

Alen Hajnal

Faccio, D., Velten, A., & Wetzstein, G. (2020). Non-line-of-sight imaging. Nature Reviews Physics, 2(6), 318–327. https://doi.org/10.1038/s42254-020-0174-8 Gibson, J. J. (1960). The concept of the stimulus in psychology. American Psychologist, 15(11), 694–703. https://doi.org/10.1037/h0047037 Gibson, J. J. (1979). The Ecological Approach to Visual Perception. Houghton Mifin. Gottlieb, G. (2007). Probabilistic epigenesis. Developmental Science, 10(1), 1–11. https://doi.org/10.1111/j.1467-7687.2007.00556.x Gu, C., Coomans, C. P., Hu, K., Scheer, F. A. J. L., Stanley, H. E., & Meijer, J. H. (2015). Lack of exercise leads to signifcant and reversible loss of scale invariance in both aged and young mice. Proceedings of the National Academy of Sciences, 112(8), 2320–2324. https://doi.org/10.1073/pnas.1424706112 Hajnal, A., Bunch, D. A., & Kelty-Stephen, D. G. (2014). Going for distance and going for speed: Efort and optical variables shape information for distance perception from observation to response. Attention, Perception, & Psychophysics, 76(4), 1015–1035. https://doi.org/10.3758/s13414-014-0629-z Hajnal, A., Bunch, D. A., & Kelty-Stephen, D. G. (2016). Pulling out all the stops to make the distance: Efects of efort and optical information in distance perception responses made by rope pulling. Attention, Perception, & Psychophysics, 78(2), 685–699. https://doi.org/10.3758/s13414-015-1035-x Hajnal, A., Clark, J. D., Doyon, J. K., & Kelty-Stephen, D. G. (2018). Fractality of body movements predicts perception of afordances: Evidence from stand-onability judgments about slopes. Journal of Experimental Psychology: Human Perception and Performance, 44(6), 836–841. https://doi.org/10.1037/xhp0000510 Hajnal, A., Surber, T., Overstreet, T., Masoner, H., Dowell, C., Funkhouser, A., Shelley-Tremblay, J., & Samu, K. (2022). Complex posturals sway is related to perception of stand-on-ability. Ecological Psychology, 34(1–2), 1–18. https:// doi.org/10.1080/10407413.2022.2035225 Haris, K., Chakraborty, B., Menezes, A., Sreepada, R. A., & Fernandes, W. A. (2014). Multifractal detrended fuctuation analysis to characterize phase couplings in seahorse (Hippocampus kuda) feeding clicks. Journal of the Acoustical Society of America, 136(4), 1972–1981. https://doi.org/10.1121/1.4895713 Hausdorf, J. M. (2007). Gait dynamics, fractals and falls: Finding meaning in the stride-to-stride fuctuations of human walking. Human Movement Science, 26(4), 555–589. https://doi.org/10.1016/j.humov.2007.05.003 Hermán, P., Kocsis, L., & Eke, A. (2001). Fractal branching pattern in the pial vasculature in the cat. Journal of Cerebral Blood Flow & Metabolism, 21(6), 741–753. https://doi.org/10.1097/00004647-200106000-00012 Hill, R. A., Bentley, R. A., & Dunbar, R. I. M. (2008). Network scaling reveals consistent fractal pattern in hierarchical mammalian societies. Biology Letters, 4(6), 748–751. https://doi.org/10.1098/rsbl.2008.0393 Humphries, N. E., Weimerskirch, H., Queiroz, N., Southall, E. J., & Sims, D. W. (2012). Foraging success of biological Lévy fights recorded in situ. Proceedings of the National Academy of Sciences USA, 109(19), 7169–7174. https://doi. org/10.1073/pnas.1121201109 Jacobson, N., Berleman-Paul, Q., Mangalam, M., Kelty-Stephen, D. G., & Ralston, C. (2021). Multifractality in postural sway supports quiet eye training in aiming

The Role of Exploratory Activity in Afordance Perception

173

tasks: A study of golf putting. Human Movement Science, 76, 102752. https:// doi.org/10.1016/j.humov.2020.102752 Jurečka, S., Angermann, H., Kobayashi, H., Takahashi, M., & Pinčík, E. (2014). Multifractal analysis of textured silicon surfaces. Applied Surface Science, 301, 46–50. https://doi.org/10.1016/j.apsusc.2014.02.102 Kelty-Stephen, D. G., & Mirman, D. (2013). Gaze fuctuations are not additively decomposable: Reply to Bogartz and Staub. Cognition, 126(1), 128–134. https:// doi.org/10.1016/j.cognition.2012.09.002 Likens, A. D., Amazeen, P. G., Stevens, R., Galloway, T., & Gorman, J. C. (2014). Neural signatures of team coordination are revealed by multifractal analysis. Social Neuroscience, 9(3), 219–234. https://doi.org/10.1080/17470919.2014.882861 MacIntosh, A. J. J., Alados, C. L., & Hufman, M. A. (2011). Fractal analysis of behaviour in a wild primate: Behavioural complexity in health and disease. Journal of the Royal Society Interface, 8(63), 1497–1509. https://doi.org/10.1098/ rsif.2011.0049 María, G. A., Escós, J., & Alados, C. L. (2004). Complexity of behavioural sequences and their relation to stress conditions in chickens (Gallus gallus domesticus): A non-invasive technique to evaluate animal welfare. Applied Animal Behaviour Science, 86(1), 93–104. https://doi.org/10.1016/j.applanim.2003.11.012 Mark, L. S., Balliett, J. A., Craver, K. D., Douglas, S. D., & Fox, T. (1990). What an actor must do in order to perceive the afordance for sitting. Ecological Psychology, 2(4), 325–366. https://doi.org/10.1207/s15326969eco0204_2 Masoner, H., Hajnal, A., Clark, J. D., Dowell, C., Surber, T., Funkhouser, A., Doyon, J., Legradi, G., Samu, K., & Wagman, J. B. (2020). Complexity of postural sway afects afordance perception of reachability in virtual reality. Quarterly Journal of Experimental Psychology, 73(12), 2362–2375. https://doi. org/10.1177/1747021820943757 O’Regan, J. K. (1992). Solving the “real” mysteries of visual perception: The world as an outside memory. Canadian Journal of Psychology, 46(3), 461–488. https:// doi.org/10.1037/h0084327 O’Toole, M., Lindell, D. B., & Wetzstein, G. (2018). Confocal non-line-of-sight imaging based on the light-cone transform. Nature, 555(7696), 338–341. https://doi.org/10.1038/nature25489 Palatinus, Z., Dixon, J. A., & Kelty-Stephen, D. G. (2013). Fractal fuctuations in quiet standing predict the use of mechanical information for haptic perception. Annals of Biomedical Engineering, 41(8), 1625–1634. https://doi.org/10.1007/ s10439-012-0706-1 Palatinus, Z., Kelty-Stephen, D. G., Kinsella-Shaw, J., Carello, C., & Turvey, M. T. (2014). Haptic perceptual intent in quiet standing afects multifractal scaling of postural fuctuations. Journal of Experimental Psychology: Human Perception and Performance, 40(5), 1808–1818. https://doi.org/10.1037/a0037247 Peng, C.-K., Havlin, S., Hausdorf, J. M., Mietus, J. E., Stanley, H. E., & Goldberger, A. L. (1995). Fractal mechanisms and heart rate dynamics: Long-range correlations and their breakdown with disease. Journal of Electrocardiology, 28, 59–65. https://doi.org/10.1016/S0022-0736(95)80017-4 Peng, C.-K., Havlin, S., Stanley, H. E., & Goldberger, A. L. (1995). Quantifcation of scaling exponents and crossover phenomena in nonstationary heartbeat time

174

Alen Hajnal

series. Chaos: An Interdisciplinary Journal of Nonlinear Science, 5(1), 82–87. https://doi.org/10.1063/1.166141 Priplata, A. A., Niemi, J. B., Harry, J. D., Lipsitz, L. A., & Collins, J. J. (2003). Vibrating insoles and balance control in elderly people. The Lancet, 362(9390), 1123–1124. https://doi.org/10.1016/S0140-6736(03)14470-4 Raja, V., & Anderson, M. L. (2021). Behavior considered as an enabling constraint. In F. Calzavarini & M. Viola (Eds.), Neural Mechanisms: New Challenges in the Philosophy of Neuroscience (pp. 209–232). Springer International Publishing. https://doi.org/10.1007/978-3-030-54092-0_10 Roeske, T. C., Kelty-Stephen, D., & Wallot, S. (2018). Multifractal analysis reveals music-like dynamic structure in songbird rhythms. Scientifc Reports, 8(1), 4570. https://doi.org/10.1038/s41598-018-22933-2 Ross, S. N. (2014). Fractal model of nonlinear hierarchical complexity: Measuring transition dynamics as fractals of themselves. Behavioral Development Bulletin, 19, 28–32. https://doi.org/10.1037/h0100587 Rutherford, K. M. D., Haskell, M. J., Glasbey, C., & Lawrence, A. B. (2006). The responses of growing pigs to a chronic-intermittent stress treatment. Physiology & Behavior, 89(5), 670–680. https://doi.org/10.1016/j.physbeh.2006.08.006 Schmitt, F. G., & Seuront, L. (2001). Multifractal random walk in copepod behavior. Physica A: Statistical Mechanics and Its Applications, 301(1), 375–396. https://doi.org/10.1016/S0378-4371(01)00429-0 Seuront, L., & Cribb, N. (2011). Fractal analysis reveals pernicious stress levels related to boat presence and type in the Indo—Pacifc bottlenose dolphin, Tursiops aduncus. Physica A: Statistical Mechanics and Its Applications, 390(12), 2333–2339. https://doi.org/10.1016/j.physa.2011.02.015 Shayeganfar, F., Jabbari-Farouji, S., Movahed, M. S., Jafari, G. R., & Tabar, M. R. R. (2009). Multifractal analysis of light scattering-intensity fuctuations. Physical Review E, 80(6), 61126. https://doi.org/10.1103/PhysRevE.80.061126 Stephen, D. G., Boncoddo, R. A., Magnuson, J. S., & Dixon, J. A. (2009). The dynamics of insight: Mathematical discovery as a phase transition. Memory & Cognition, 37(8), 1132–1149. https://doi.org/10.3758/MC.37.8.1132 Stephen, D. G., & Hajnal, A. (2011). Transfer of calibration between hand and foot: Functional equivalence and fractal fuctuations. Attention, Perception, & Psychophysics, 73(5), 1302–1328. https://doi.org/10.3758/s13414-011-0142-6 Stofregen, T. A., Yang, C.-M., & Bardy, B. G. (2005). Afordance judgments and nonlocomotor body movement. Ecological Psychology, 17(2), 75–104. https:// doi.org/10.1207/s15326969eco1702_2 Surber, T., Overstreet, T., Masoner, H., Dowell, C., & Hajnal, A. (2022). Functional specifcity of the afordance of reaching. Experimental Psychology, 69(1), 23–39. https://doi.org/10.1027/1618-3169/a000544 Takahashi, T., Kosaka, H., Murata, T., Omori, M., Narita, K., Mitsuya, H., Takahashi, K., Kimura, H., & Wada, Y. (2009). Application of a multifractal analysis to study brain white matter abnormalities of schizophrenia on T2-weighted magnetic resonance imaging. Psychiatry Research: Neuroimaging, 171(3), 177– 188. https://doi.org/10.1016/j.pscychresns.2008.03.009 Thamrin, C., & Stern, G. (2010). New methods: What do they tell us? Fluctuation analysis of lung function. European Respiratory Monogram, 47, 310–324.

The Role of Exploratory Activity in Afordance Perception

175

Turiel, A., & Parga, N. (2000). The multifractal structure of contrast changes in natural images: From sharp edges to textures. Neural Computation, 12(4), 763– 793. https://doi.org/10.1162/089976600300015583 Turvey, M. T., Fitch, H. L., & Tuller, B. (1982). The Bernstein perspective: I. The problems of degrees of freedom and context-conditioned variability. In J. A. S. Kelso (Ed.), Human Motor Behavior: An Introduction (pp. 239–252). Lawrence Erlbaum. Vickers, J. N. (2011). Mind over muscle: The role of gaze control, spatial cognition, and the quiet eye in motor expertise. Cognitive Processing, 12(3), 219–222. https://doi.org/10.1007/s10339-011-0411-2 Viswanathan, G. M., Afanasyev, V., Buldyrev, S. V., Murphy, E. J., Prince, P. A., & Stanley, H. E. (1996). Lévy fight search patterns of wandering albatrosses. Nature, 381, 413–415. https://doi.org/10.1038/381413a0 Wagman, J. B., & Hajnal, A. (2014). Task specifcity and anatomical independence in perception of properties by means of a wielded object. Journal of Experimental Psychology: Human Perception and Performance, 40(6), 2372–2391. https:// doi.org/10.1037/xhp0000014 Yokoyama, H., Niwa, S., Itoh, K., & Mazuka, R. (1996). Fractal property of eye movements in schizophrenia. Biological Cybernetics, 75(2), 137–140. https:// doi.org/10.1007/s004220050281 Zeide, B., & Gresham, C. A. (1991). Fractal dimensions of tree crowns in three loblolly pine plantations of coastal South Carolina. Canadian Journal of Forest Research, 21(8), 1208–1212. https://doi.org/10.1139/x91-169

10 SCALING UP Lawfulness of Afordances Requires Independence From Any Single “Scale of Behavior” Damian G. Kelty-Stephen

What Do We Understand by the Term Afordance?

My understanding is that afordances are, broadly construed, opportunities for action (Gibson, 1979). Afordances are the organizations of stimulation into patterns that defne the possibilities for how an organism can engage with the world to pursue goals (Heft, 2003, 2017). A looming issue in my view is the scale (or scales) at which we can defne or identify afordances (Michaels, 2003). The relationship of afordance to scale has a couple of intriguing entailments that I consider subsequently. What Role Does Afordance Play in Perception? Is It the Entity Organisms Perceive or the Means Through Which Organisms Perceive? Are Afordances the Only Perceptual Dependent Variables?

We can perceive afordances. To say less is to deny ecological psychology’s empirical confrmation of the reality and importance of organism as a player in perception and action (Burton, 1992; Jiang & Mark, 1994; Mark, 1987; Mark et al., 1990; Warren, 1984; Warren & Whang, 1987). The question of whether afordances are all we perceive breaks down into two smaller questions: (i) whether any non-afordances populate our subjective experience and (ii) whether experiencing non-afordances requires either indirect perception or non-perceptual modes. For instance, Gibson (1979) remarked that “to perceive an afordance is not to classify an object” (p. 134). Gibson might have answered the (i) question positively, and his DOI: 10.4324/9781003396536-13

Scaling Up

177

admission that some direct perception might not cover non-afordance items suggests a negative answer to (ii). This position persists today, hinging on a premise of locality, that is, immediacy anchored in the specifc scales defned by the anatomical span of an organism’s reach in the present moment (Golonka & Wilson, 2019; Heft, 2003, 2017). According to this perspective, the organism may engage with ambient energy arrays that span long scales in space and time, but the organism’s perceptual experience is founded in the strictly local contact, for instance, within arms’ reach, with those arrays. This perspective is open to the possibility that organisms make contact with afordances in diferent senses, that is, in relatively abstract, context-general terms as well as in relatively specifc, context-specifc terms (e.g., Costall, 1997; Shaw et al., 2019). Traversals to other levels strike these local-contact perspectives as requiring steps that depart from what ecological psychology is all about: for example, an indirect sort of perception (e.g., Heft, 2003), perhaps even depending on “neural activity” caused by ecological information but then somehow “decoupled” from it—so directness and nesting appear exchangeable for indirectness and decoupling beyond the seeming limits of an afordance (Golonka & Wilson, 2019). Other perspectives appear ready to envision afordances as inhabiting more scales than the most local, immediate present. Allowing that ecological nestings extend across multiple scales might allow the direct perception a much longer leash. Our ability to classify objects might be compatible with direct perception, requiring no break with afordances and no spells of indirect perception (e.g., Withagen & Chemero, 2012). And if classifying objects can be a direct-perceptual faculty, I, for one, would be open to defnitions of afordances rooted in multiple scales. At that point, I realize I might be in the heretical position of faulting Gibson and considering that afordance perception might be no diferent from classifying objects. Heresy may not be so bad. As a great ecological psychologist once said, “It is instructive to be delinquent (occasionally) in one’s faith” (Turvey, 1977, p. 83). Indeed, Gibson (1979, 1966b, 1950) was nothing if not ready to admit failures and try something new. So, there may be no great sin in fostering more elaborate concepts of direct perception and afordance with new knowledge about nesting relationships across scales. Certainly, Gibson’s professional trajectory suggests he began to see a progressively broader scope for direct perception. Who is to say that ecological psychology should not continue to extend the logic over broader scales? Ecological psychology excited me because it was ofering to replace inferential mediators with nesting relationships—wholesale, I thought. I confess I did

178

Damian G. Kelty-Stephen

not come to ecological psychology to say, “Well, okay, just a little bit of indirect perception beyond the local contact!” But this is also my bias: I am sooner on the side of “radical” embedded, embodied approaches to perception and action—and perhaps I am not qualifed to see the blessings of letting indirect perception coexist with direct perception. Why Is Afordance Any Better Than Stimulus? What Does a Theory of Afordance Suggest That Stimuli Cannot; How Has It Moved the Needle Past Gibson 1960’s Recognition of How Little We Can Defne Stimuli?

The afordance concept has been a foundational step for theories of perception and action. It has taken us beyond the concept of molecular, punctate, impoverished stimuli and into a new way of understanding the world as a populated with events at many scales that refect and inform the activity of organisms, again, at many scales. The afordance has, in efect, been the vindication of the organism—a persistently elusive and murky concept in all social and life sciences (Baedke, 2019)—a critical player in the drama of perception and action. We cannot overstate the genius of the seminal work documenting afordance perception (Burton, 1992; Jiang & Mark, 1994; Mark, 1987; Mark et al., 1990; Warren, 1984; Warren & Whang, 1987). It portrayed the following gem of wisdom: whatever photons or onboard physiology might make a strictly computational explanation of goal-directed vision difcult, organisms can see at least some aspects of the visual world in terms of their muscle-and-bone capacity to act in the world. And this specifcity of afordance to action capabilities is not just written simply at the level of description of kinematic cartoons, for instance, of a spatial issue of whether the organism will bump its head on the top of the doorway. No, it was specifc to the kinetics and energetics of actually enacting the movement in interacting with the feature in question. In this way, the foundational afordance research was positively prescient, laying the groundwork for opening our eyes to understanding the rapid, prospective, full-bodied engagement of the organism with light and touch at scales far beyond relatively fne-scale thresholds (e.g., beyond viscosity subrange or Planck’s constant; Federle & Endlein, 2004; Kiely & Collins, 2016; Marsden et al., 1983; Moreno et al., 2011; Turvey & Fonseca, 2014). The accuracy with which afordance judgments aligned with kinetic constraints on goal-directed movement—and rapidly worked to reassert this alignment even with subtle adjustments of enabling constraints—was groundbreaking. In efect, it vindicated the relevance of the organism to theories of perception and action as more than just an amalgam of servo processes atop a balance of sensors and

Scaling Up

179

actuators. The immediacy and availability of action-relevant patterns of stimulation to perception was a clear point in favor of ecological psychology and all attempts to fnd perception and cognition in the body beyond strictly neural responses. But the Reality of Organisms Should Not Require Burning Bridges to Other Scales

This seminal empirical work had the great insight to anchor afordances, these real possibilities for action, in the dimensionless, scale-independent formalism appropriate to the law (e.g., Kugler et al., 1982; Kugler & Turvey, 1987). Hence, the elaboration of afordance as a key variable in perception and action let us climb above the dust of photons and pixels and pinpricks and action potentials, and it gave ecological psychology a way of speaking about organisms as more than just a sum of equally dustlike, molecular components to process molecular inputs. So we owe Gibson (1979) a great debt for showing us where to look for molar features that would let the organism play a viable role in its own right—rather than leaving the organism to be so much handwaving. My concern is that, having clambered from small scale up to the larger scale where (more) organisms can be defned, we do not pull the ladder up and make afordance a prohibition on crossing scales. Gibson (1979) made these pioneering steps while explicitly warning his readers away from physics, anatomy, and physiology (p. xiii). I do understand that overemphasis on the molecular rather than the molar has long been a sin of all these scientifc disciplines, and I also understand that Gibson’s contemporaries in psychologists likely burned away his patience by bringing only the most molecular aspects of physics, anatomy, and physiology to bear on psychology. However, in his broad-brushing dismissal of these felds, Gibson may have ignored that he was coming to this interest in an organism and in a system-wide process over the same decades that these other felds were doing the same (Lewontin, 1982; Mandelbrot, 1974; Nicolis & Prigogine, 1977; von Bertalanfy, 1986). However, I try to charitably assume that his denial that physics, anatomy, and physiology could help ecological psychology was sooner a rebuke to psychologists using the most molecular, most impoverished aspects of these felds—sooner than a refusal that events on those scales could not be relevant. The scale-independence of natural law has always been a theoretical reason I have had for disagreeing with this one feature of Gibson’s (1979) perspective. But I believe that too-normative readings of Gibson’s (1979) disavowal of these felds have risked sealing the afordance up on a scale of its own and threatening ecological psychology’s best chance at the lawful explanation of perception and action.

180

Damian G. Kelty-Stephen

The disavowal of physics, anatomy, and physiology always trips me up. Ecological psychology disavowed these felds—until it did not. That is, Gibson (1979) disavowed physics, anatomy, and physiology, but the next wave of ecological psychologists found less ofensive systems-theoretical approaches to these scientifc disciplines (e.g., Kugler et al., 1982; Kugler & Turvey, 1987) from the likes of Soodak and Iberall (1978), Yates (1982), and Pattee (1977). In what seems like a brief glimmer of insight, the afordances blossomed as a dimensionless and scale-free law. This development was a beautiful courtship between Gibsonian theory and the physics, anatomy, and physiology that was sufciently molar and not just molecular to inform ecological theory. But then this firtation seemed to peter out. And then we came to a point where afordances took on a native scale, and the scale-dependent afordance took hold alongside a reinstatement of Gibsonian mistrust for physics, anatomy, and physiology. What Is the Connection Between Afordances, Behavioral Scale, and Intention? How Distant or Reluctant Can a PerceivingActing System Be Before Afordances Come Into Play? Do Afordances Exert Their Role Only When the System Is Close or Willing Enough? Scale Matters for Discerning Lawfulness

Honestly, the scale seems such a dismal little afair compared to the romance of afordances. It might seem peripheral to afordances’ more central role of furnishing opportunities to act. But I think it is quite important to any approach that aims to replace inferential mediators between perception and action with lawful nesting relationships. In particular, ecological psychologists expect “afordances [to] embody laws” governing perception and action (Turvey, 1992, p.  178), particularly from an understanding of “law” as a “real possibility” (Turvey, 1992, p. 177; Turvey et al., 1981). This understanding of law as a possibility—and neither determiner nor enforcer—is standard in the physical sciences (e.g., Pattee, 2013). The diference between possibility and enforcement may cover some of what Raja and Chemero (2020) had in mind when proposing a pluralistic normativity where the lawfulness amounted to diferent specifcations under diferent constraints. Though to be clear, laws do not require nesting, and it was Gibson and the ecological perspectives he inspired that expected the lawfulness supporting perception and action to unfold through nesting relationships. In any event, laws are scale-independent. This abstract point becomes clear and concrete in, for instance, the bodyscaled ratios defning step-on-ability relating leg lengths or eye heights to

Scaling Up

181

riser heights (Warren, 1984; Warren & Whang, 1987). The alternative, that is, of anchoring afordances in a limited set of scales, could be disastrous for lawful description. One More Time With Feeling: Afordances as Scale-Dependent?

But yes, I have heard that afordances can belong to a specifc scale and not to others. For example, Reed (1996) caught a wide swath of the ecological-psychological community’s imagination with the proposal that “afordances are resources for an animal at the scale of behavior” (p. 38). And sure enough, Reed understood well enough that there was not a single scale of behavior but instead meant that behavior was limited to a range of (i.e., plural) behavioral “scales” (p. 187). But the diference between one scale or a limited range of scales has been collapsed in the retelling—today, ecological psychologists talk about afordances defnable at “the behavioral level” or the “behavioral scale” (Araújo & Davids, 2016; Gastelum, 2020; Golonka & Wilson, 2019; Heft, 1989; Posteraro, 2014; Withagen & van Wermeskerken, 2010; Zipoli Caiani, 2021) or even “ecological scale” (Segundo-Ortin et al., 2019; R. Shaw & Kinsella-Shaw, 1988). None of these authors would disagree with the notion of a range of scales for behavior/ activity. However, the point here is that afordances have often appeared as a coarse-grained feature saving perceptual theory from the task of piecing perception together from minuscule, punctate, feeting, impoverished stimuli—as well as the developmental and phylogenetic or sociocultural scales extending beyond day-to-day goal-directed behavior (Heft, 2003; Reed, 1996). Crucially, the limitation of scale is intended as a blessing because there is no (or less) integration or inference necessary if the action-relevant features of the world are macroscale, written at the same level of precision as an organism’s goal statements. Certainly, if the afordances are opportunities for action, they should inhabit a scale on which the organism can act. And if we believe that the organism acts due to an intentional relationship to its surroundings, it is tempting to situate our notion of afordances at the scales most commonly addressed with, say, organism self-reports of their goals. For instance, if the organism reports, “I would like to take a drink from this teacup,” we might readily conclude that the afordances include the graspability of a vessel and the sippability of a hot liquid. These features have a coarse grain that fts snugly within the string of symbols like “would like,” “take a drink,” and “this teacup.” Certainly, these approachable, accessible phrases are as graspable to the interlocutor as the cup on the table at hand. The organism would not report a goal of stirring up a range of olfactory and tactile action potentials, and sipping tea need not hinge on conditions

182

Damian G. Kelty-Stephen

unfolding over a lifespan or evolutionary time scales. Unsurprisingly, afordances inspire their own new words, for instance, “step-on-ability.” So, yes, I understand well: attending less to the molecular details is a very efective way to give credence to the larger-scale events involving organism experience. And so, if we treat “intentionality” only as a linguistically expressible goal, then we force the intended action to appear to us as a string of symbols. And so, yes, it follows: the high-level content of a goal statement could be a better cue to the organization of stimulus information and its perceptual consequences—better certainly than sifting through the dust of action potentials and photons. Heresy or What Nesting May Entail: Hierarchical Afordances Spanning Multiple Scales

Then again, actions might easily wander across a wider range of scales than available on a tabletop demonstration (Van Orden, 2010; Van Orden et al., 2012, 2003). What is more, the intentionality of the organism doing the perceiving and acting is more general than what a symbol string can carry. The verbally describable goal of a protagonist organism navigating its surroundings is only a fne sliver of intentionality. The intention is not only what fts into a string of symbols, scale-dependent bits of language. It has much to do with the “aboutness” of perception and action, that is, an organism’s perpetual investment in the context (Freeman, 1997; Shaw, 2001)—whether or not that organism entertains an explicit goal before lighting out on its journey. In this case, intentionality will likely extend beyond the strictures of symbol-string goal statements. Likewise, we may wish to imagine afordances as features that are not bound at the scale of what is linguistically reportable as the goal. So, either curiously for some or maybe completely sensibly for others, ecological psychology also holds out a possibility that afordances distribute themselves hierarchically at many nested scales, from the fne scales, for instance, ftting in the grasping hand, to the longer and larger scales (Kimmel & Rogler, 2018; Wagman et al., 2016a, 2016b, 2017; Wagman & Stofregen, 2020)—sometimes across scales as vast of culture and language (Chemero, 2009; Rietveld & Kiverstein, 2014), and sometimes across levels of generality, for example, from relative concrete, graspable tokens to relatively abstract types (Costall, 1997; Shaw et al., 2019). This portrayal refects the nuance that actions are not limited to the scale of the body or particular objects but can unfold more slowly, over greater spatial extents, and across a wide variety of tokens (van Dijk & Withagen, 2015). Ecological psychologists emphasizing the rhetorically singular “scale of activity” might be open to the idea that there are hierarchies within what is a range

Scaling Up

183

of scales. But ecological psychologists keen on reining the afordance into more limited (range of) scale(s) of activity reserve a skeptical view that these larger/longer scales contain true afordances. This latter skepticism about larger/longer scales refects a premise that direct-perceptual contact with afordances has an immediacy, a locality to the current position and present moment (Golonka & Wilson, 2019; Heft, 2003). So, Do We Want Afordances to Be Laws or Constraints? They Cannot Be Both

The stakes are high for a lawful account of perception and action emphasizing nesting relationships rather than inference. If afordances embody laws, and if laws are independent of scale, then they cannot be defned as belonging to a specifc scale—not even at the so-called scale of behavior (Reed, 1996, p. 38). In this light, a hierarchical organization of afordances may be more compatible with lawful explanation, that is, because hierarchical organizations are defnable across a range of scales. There are most certainly scale-dependent factors helping to shape the expression of afordances on perception and action, but these scale-dependent factors are constraints, as in the constraints existing either in context or in anatomy. We can proftably discuss constraints as helping to shape perception and action, and we can even proftably discuss constraints at each of a wide range of scales (Anderson, 2014; Baggs et  al., 2020; Raja & Anderson, 2021). But crucially, constraints are not laws: whereas laws are scale-independent regularities that depend on rate, constraints are rate-independent factors that depend on scale (Pattee, 2013). Confusing laws with constraints can happen to the best of us, but it usually has the consequence of confusing our readers and colleagues (Pattee, 2007). Describing afordances as scale-dependent (i.e., dependent on the “behavioral (range of) scale(s)”) and rendering them as constraint has two major entailments for theories of perception and action. First, for ecological psychology, treating afordances as constraints muddles the theory and ignores the capacity for a hierarchy of afordances to carry the lawful nesting relations thought important to Gibsonian perspectives. Constraints often include symbols, logical rules, and all other Shannon information-theoretic ingredients that harness fuid, lawful dynamics for computation and formal inference (Pattee, 2013). Symbols and logical rules are precisely what those familiar, accessible goal statements were. So, suddenly, the ft of the afordance language into the symbol string of the goal statement might get slightly constrictive. What if marrying our theory to defning goals as “verbally reportable wishes” locks the door and throws out the key? None of these symbolic, logical items are typically welcome in traditional ecological

184

Damian G. Kelty-Stephen

and psychological approaches—Gibson (1979) famously eschewed inference and computation as necessary conditions for perception and action. What if the scale dependence of afordance locks direct perception inside a cage of computer code? The second liability of the scale-dependent description of afordance is that it leaves the door open for what I think of as a computational gloss of Gibsonian theory. For instance, “afordance” has become a wildly popular concept in felds well beyond ecological theory—perhaps beyond what some Gibson scholars see as appropriate usage of the term (Chong & Proctor, 2020). Ironically, proponents of computational psychology have been among the most frequent consumers of afordances. But, then again, is it ironic? Ecological psychology fnds itself in the curious business, to this day, of denying that it reduces to computational psychology after all (Segundo-Ortin et  al., 2019). It gets much worse: computational approaches take the scale-dependent aspect of afordances to mean that afordances are as symbolic, computable, and inferable as any other scale-dependent constraint. And the viral spread of the word afordance results in computational approaches to perception and action amounts to little more than assuming Gibson meant computational anyway and, adding insult to injury, proposing computational models in which afordance perception runs on a convolutional neural network (Gu et al., 2021; Qian et al., 2020; Wang & Tarr, 2018). So, computational psychologists heard the bit in ecological psychology about hierarchy and nesting relationships, but they latched on to the idea of afordance as a feature defned at a specifc scale, the “scale of activity.” And so they decide to make a hierarchical inference engine—as if inference were not the problem to start with. But if you give computational psychologists a scale-dependent constraint, we have seen that they can use it. The co-opting of afordance as a scale-independent constraint is much of what little is new in computational psychology’s bemused response to ecological psychology. Little else has changed here: there are no laws in computational psychology—it remains the exact Bayesian inference today (de Lange et al., 2018; Petzschner et al., 2017, 2021) that Turing revived for his frst computing machines (Hodges, 1983). And as Vinson et  al. (2016) aptly pointed out, the diagnosis of nothing-but-computation to see here (Fodor & Pylyshyn, 1981) is just plain recycled anew without much change for a younger generation (see Firestone & Scholl, 2016). The problem is that even when ecological psychology reasserts that it is not computational psychology all over again, it still reasserts the scale-dependence of afordance and ecological-psychology explanation, for example, at “the ecological scale” (e.g., Segundo-Ortin et al., 2019). Let me repeat this in sparser terms. When ecological psychologists refer to afordance as a scale-dependent item, they efectively give computational

Scaling Up

185

psychologists precisely what they need to extend a computational model of perception and action. When computational psychology folds a scaledependent constraint efortlessly into a logical inference, ecological psychologists can say, “Oh, but we told them not to infer!” But here is the problem: if afordances only reside on one scale or a limited range of scales, there is little else to do with them except infer and compute. So then, it is easy to see how our ecological psychological colleagues go from the premise of scale-dependent afordance to conceding sometimes indirect, sometimes decoupled perception (Golonka & Wilson, 2019; Heft, 2003). And yes, you can say that direct perception of afordances requires an immediacy or a locality in the present moment. But that ignores what Gibson (1966b) said about distinctions between past, present, and future being spurious (Stepp & Turvey, 2015). In fact, Gibson (1966b) may not have pinned direct perception on the present: “Resonance to information, that is, contact with the environment, has nothing to do with the present” (p. 276). On the other hand, if afordances are permitted more widely to exist beyond anatomical span and across many scales, then we have a shot at a lawful explanation without so much interruption of direct perception—not to mention without providing more ammunition to our computational-theoretical colleagues. How Do Systems Engage With Afordances as They Move Among Tasks and Intentions?

As wiser minds than I have already ably described (Chemero, 2009; Costall, 1997; Kimmel & Rogler, 2018; Rietveld & Kiverstein, 2014; Shaw et al., 2019; Wagman et al., 2016a, 2016b, 2017; Wagman & Stofregen, 2020), afordances distribute across multiple scales. And despite fears that no mechanisms exist that would allow an organism to make continuous, concerted use of opportunities for action at all scales, cascades are a class of mechanism that generates scale-invariant organizations to beft lawful independence from any single scale (Lovejoy & Schertzer, 2018; Mandelbrot, 1974). Cascades generate structures at all scales (e.g., eddies or vortices) that contain and shape structures they contain while supporting the structures containing them. Cascades are the reason for all of my fractal and, more generally, multifractal modeling of movement variability: multifractal geometry is the only analytical framework I know for estimating evidence of nonlinear interactions across scales—in organism behavior (e.g., Kelty-Stephen & Mirman, 2013) and in task context (Stephen & Dixon, 2011). In contrast, scale dependence is much more compatible with Euclidean geometries, such as the “blockworlds” that were stock in trade for computational psychology

186

Damian G. Kelty-Stephen

(Winograd, 1972). The nonlinearity of the interaction means that it is not simply the addition of scale-dependent afordances, for instance, of blocks of real possibility at a diferent scale. Instead, the Warren-style (1984) phrasing of afordance aims to put the organism-environment relationship in center stage. Multifractal fuctuations may characterize the richly textured and maddeningly fuid boundaries between/among organisms and environments—much in the way Mandelbrot (1967) characterizes actual geophysical coastlines between landmass and grinding tides (Mangalam et  al., 2022). Hence, if neologisms like “step-on-ability” are symbolic, scale-dependent constraints, sometimes phrased in ways native to a (single or limited range of) “scale(s) of activity/behavior/ecology,” then multifractal arrangements that extend fuidly across multiple scales may be like lawful backbones that can spool out and/or bend according to scale-dependent constraints, for example, the eddies or efuvia that precipitate out of all known fows. The multifractality of behavior puts the organism in touch with a vast range of (maybe all available?) scales of ambient arrays. This cosmopolitan openness across scales seems helpful for allowing perceiving-acting systems to educate their attention (Gibson, 1966a, 1979; Jacobs & Michaels, 2007) and in turn to maintain ongoing, robustly dexterous movements (Profeta & Turvey, 2018). We fnd this multifractal form billowing through movements of hand (Booth et al., 2018; Kelty-Stephen et al., 2016; Stephen et al., 2012), foot (Stephen & Hajnal, 2011), eye (Kelty-Stephen & Mirman, 2013; Stephen et al., 2009; Wallot et al., 2015), head (Bell et al., 2019; Carver et al., 2017; Kelty-Stephen & Dixon, 2014), torso (Eddy & KeltyStephen, 2015; Teng et al., 2016), and postural center of pressure (Mangalam, Chen, et  al., 2020; Mangalam & Kelty-Stephen, 2020; Palatinus et al., 2014)—not to mention through the objects we interact with (Avelar et al., 2019; Harrison et al., 2014). In all of the foregoing cases, multifractal estimates from measured behavior predict how organisms make use of the information most immediately available to those body parts—and, in some of these cases, even predict afordance perception and confdence in afordance judgments (Doyon et al., 2019; Hajnal et al., 2018). What is more, the exchange of multifractal fuctuations across the organism’s whole body, that is, among the just-listed degrees of freedom, can predict perceptual outcomes (Mangalam, Carver, et  al., 2020a, 2020b). And at the larger scale of multiple organisms, it appears that multifractal fuctuations may support the cohesion of swarming (Carver et al., 2017; Dixon & Kelty-Stephen, 2012), that is, the building of scale-dependent constraints at the social scales. Here is a real possibility: (i) Systems might engage with scale-independent lawfulness by embodying cascades, characterized by nonlinear

Scaling Up

187

interactions across scales generating fractal and multifractal geometries. (ii) The organism’s nested architecture might generate its scale-invariant patterning of fractal and multifractal geometry in a multifractal world. (iii) Scale-invariant form might give organisms their ability to take hold of the similarly fractal and multifractal geometry of events in the context— providing room for dexterity among unpredictable events (Stephen et al., 2008; Stephen & Dixon, 2011). Identifying objects may not be perceiving afordances, after all. However, even if not, the object and any label could be helpful to constraints shaping how afordances will appear at another scale. Constraints at one scale set up the landscape for laws to spill across to another scale (Balagué et  al., 2019). Constraints (e.g., objects, labels, ability to identify either) may be the products of lawful processes constrained at other scales; in fact, there is no other way for them to occur (Pattee, 2001). In that sense, the necessity of the distinction between laws and constraints is not the sign that the world itself is composed of two types of things—rather this distinction stresses the reality and necessity of interactions across multiple scales, in which a frozen constraint at one scale is a fuid law-driven process at another. Perceiving afordances could be the means for identifying an object, with afordances underwriting steps toward using an object or its label. The dressing up of afordances in particular constraints converts “real possibility” to a specifed response (Raja & Chemero, 2020). If we compare organism behavior to fuid fow, the power laws governing fuid fow describe the many possible ways fuid could fow. The exact behavior depends on how these possibilities become constrained by the shape of the terrain (i.e., basin, the sources, and sinks) over potentially very large scales and by the shape of the component molecules (i.e., permitting translation, rotation, and vibration) at the small scales. The lawful possibility is the cascades’ capacity to generate various multifractal trajectories. The specifcity unfolds as a complementary handshake between constraints and laws across multiple scales, though: for the short term, molecular and terrain-based constraints shape the multifractal pattern of the measured behavior (Mazzi & Vassilicos, 2004), and in the long term, the multifractal patterns of the fow can themselves chip away at and erode the terrain or break or press the molecular bonds, for example, as coal-fred or warmblooded exhalations of carbon dioxide might calve icebergs from ice sheets or dissolve the rigid snowfake symmetry. Specifcation lives at none of the individual scales. However, it relies on all of them interacting—relying particularly on the capacity of some of the lawful dynamics unfolding at many scales, allowing medium-scale dynamics to exploit the impermanence and the persistence of shorter- and longer-scale dynamics, respectively.

188

Damian G. Kelty-Stephen

If afordances refect lawful scale-independent, fuid, cascade relationships, I doubt we need to resort to a strictly inferential process for explaining experiences beyond a so-called present. I am comfortable with objects not being afordances, but direct-perceptual experiences likely hinge on encounters with a blend of lawful afordances with constraints. This possibility may push us to understand how fuid, lawful systems learn to build and twiddle the scale-dependent parts that compose all of the lovely machines that build themselves, break themselves, and build themselves from the older wreckage. Reference list Anderson, M. L. (2014). Beyond componential constitution in the brain: Starburst amacrine cells and enabling constraints. In T. Metzinger & J. M. Windt (Eds.), Open MIND. MIND Group. Araújo, D., & Davids, K. (2016). Team synergies in sport: Theory and measures. Frontiers in Psychology, 7, 1449. https://doi.org/10.3389/fpsyg.2016.01449 Avelar, B. S., Mancini, M. C., Fonseca, S. T., Kelty-Stephen, D. G., de Miranda, D. M., Romano-Silva, M. A., de Araújo, P. A., & Silva, P. L. (2019). Fractal fuctuations in exploratory movements predict diferences in dynamic touch capabilities between children with attention-defcit hyperactivity disorder and typical development. PLoS One, 14(5), e0217200. https://doi.org/10.1371/journal.pone.0217200 Baedke, J. (2019). O organism, where art thou? Old and new challenges for organism-centered biology. Journal of the History of Biology, 52(2), 293–324. https:// doi.org/10.1007/s10739-018-9549-4 Baggs, E., Raja, V., & Anderson, M. L. (2020). Extended skill learning. Frontiers in Psychology, 11, 1956. https://doi.org/10.3389/fpsyg.2020.01956 Balagué, N., Pol, R., Torrents, C., Ric, A., & Hristovski, R. (2019). On the relatedness and nestedness of constraints. Sports Medicine, 5(1), 6. https://doi. org/10.1186/s40798-019-0178-z Bell, C., Carver, N., Zbaracki, J., & Kelty-Stephen, D. (2019). Nonlinear amplifcation of variability through interaction across scales supports greater accuracy in manual aiming: Evidence from a multifractal analysis with comparisons to linear surrogates in the Fitts task. Frontiers in Physiology, 10, 998. https://doi. org/10.3389/fphys.2019.00998 Booth, C. R., Brown, H. L., Eason, E. G., Wallot, S., & Kelty-Stephen, D. G. (2018). Expectations on hierarchical scales of discourse: Multifractality predicts both short- and long-range efects of violating gender expectations in text reading. Discourse Processes, 55(1), 12–30. https://doi.org/10.1080/0163853X.2016.1197811 Burton, G. (1992). Nonvisual judgment of the crossability of path gaps. Journal of Experimental Psychology: Human Perception and Performance, 18(3), 698– 713. https://doi.org/10.1037/0096-1523.18.3.698 Carver, N. S., Bojovic, D., & Kelty-Stephen, D. G. (2017). Multifractal foundations of visually-guided aiming and adaptation to prismatic perturbation. Human Movement Science, 55, 61–72. https://doi.org/10.1016/j.humov.2017.07.005

Scaling Up

189

Chemero, A. (2009). Radical Embodied Cognitive Science. MIT Press. Chong, I., & Proctor, R. W. (2020). On the evolution of a radical concept: Afordances according to Gibson and their subsequent use and development. Perspectives on Psychological Science, 15(1), 117–132. https://doi. org/10.1177/1745691619868207 Costall, A. (1997). The meaning of things. Social Analysis: The International Journal of Social and Cultural Practice, 41(1), 76–85. www.jstor.org/stable/23171733 de Lange, F. P., Heilbron, M., & Kok, P. (2018). How do expectations shape perception? Trends in Cognitive Sciences, 22(9), 764–779. https://doi.org/10.1016/j. tics.2018.06.002 Dixon, J. A., & Kelty-Stephen, D. G. (2012). Multi-scale interactions in Dictyostelium discoideum aggregation. Physica A: Statistical Mechanics and Its Applications, 391(24), 6470–6483. https://doi.org/10.1016/j.physa.2012.07.001 Doyon, J. K., Hajnal, A., Surber, T., Clark, J. D., & Kelty-Stephen, D. G. (2019). Multifractality of posture modulates multisensory perception of stand-on-ability. PLoS One, 14(2), e0212220. https://doi.org/10.1371/journal.pone.0212220 Eddy, C. L., & Kelty-Stephen, D. G. (2015). Nesting of focal within peripheral vision promotes interactions across nested time scales in head sway: Multifractal evidence from accelerometry during manual and walking-based Fitts tasks. Ecological Psychology, 27(1), 43–67. https://doi.org/10.1080/10407413.2015. 991663 Federle, W., & Endlein, T. (2004). Locomotion and adhesion: Dynamic control of adhesive surface contact in ants. Arthropod Structure & Development, 33(1), 67–75. https://doi.org/10.1016/j.asd.2003.11.001 Firestone, C., & Scholl, B. J. (2016). Cognition does not afect perception: Evaluating the evidence for “top-down” efects. Behavioral and Brain Sciences, 39, e229. https://doi.org/10.1017/S0140525X15000965 Fodor, J. A., & Pylyshyn, Z. W. (1981). How direct is visual perception? Some refections on Gibson’s “ecological approach.” Cognition, 9(2), 139–196. Elsevier Science. https://doi.org/10.1016/0010-0277(81)90009-3 Freeman, W. J. (1997). Nonlinear neurodynamics of intentionality. Journal of Mind and Behavior, 18(2–3), 291–304. www.jstor.org/stable/43853825 Gastelum, M. (2020). Scale matters: Temporality in the perception of afordances. Frontiers in Psychology, 11, 1188. https://doi.org/10.3389/fpsyg.2020.01188 Gibson, J. J. (1950). The Perception of the Visual World. Houghton Mifin. Gibson, J. J. (1966a). The Senses Considered as Perceptual Systems. Houghton Mifin. Gibson, J. J. (1966b). The problem of temporal order in stimulation and perception. Journal of Psychology, 62(2), 141–149. https://doi.org/10.1080/0022398 0.1966.10543777 Gibson, J. J. (1979). The Ecological Approach to Visual Perception. Houghton Mifin. Golonka, S., & Wilson, A. D. (2019). Ecological representations. Ecological Psychology, 31(3), 235–253. https://doi.org/10.1080/10407413.2019.1615224 Gu, Q., Su, J., & Yuan, L. (2021). Visual afordance detection using an efcient attention convolutional neural network. Neurocomputing, 440, 36–44. https:// doi.org/10.1016/j.neucom.2021.01.018

190

Damian G. Kelty-Stephen

Hajnal, A., Clark, J. D., Doyon, J. K., & Kelty-Stephen, D. G. (2018). Fractality of body movements predicts perception of afordances: Evidence from stand-on-ability judgments about slopes. Journal of Experimental Psychology: Human Perception and Performance, 44(6), 836–841. https://doi.org/10.1037/ xhp0000510 Harrison, H. S., Kelty-Stephen, D. G., Vaz, D. V, & Michaels, C. F. (2014). Multiplicative-cascade dynamics in pole balancing. Physical Review E, 89(6), 60903. https://doi.org/10.1103/PhysRevE.89.060903 Heft, H. (1989). Afordances and the body: An intentional analysis of Gibson’s ecological approach to visual perception. Journal for the Theory of Social Behaviour, 19(1), 1–30. https://doi.org/10.1111/j.1468-5914.1989.tb00133.x Heft, H. (2003). Afordances, dynamic experience, and the challenge of reifcation. Ecological Psychology, 15(2), 149–180. https://doi.org/10.1207/ S15326969ECO1502_4 Heft, H. (2017). Perceptual information of “an entirely diferent order”: The “cultural environment” in the senses considered as perceptual systems. Ecological Psychology, 29(2), 122–145. https://doi.org/10.1080/10407413.2017.1297187 Hodges, A. (1983). Alan Turing: The Enigma. Princeton University Press. Jacobs, D. M., & Michaels, C. F. (2007). Direct learning. Ecological Psychology, 19(4), 321–349. https://doi.org/10.1080/10407410701432337 Jiang, Y., & Mark, L. S. (1994). The efect of gap depth on the perception of whether a gap is crossable. Perception & Psychophysics, 56(6), 691–700. https://doi.org/10.3758/BF03208362 Kelty-Stephen, D. G., & Dixon, J. A. (2014). Interwoven fuctuations during intermodal perception: Fractality in head sway supports the use of visual feedback in haptic perceptual judgments by manual wielding. Journal of Experimental Psychology: Human Perception and Performance, 40(6), 2289–2309. https:// doi.org/10.1037/a0038159 Kelty-Stephen, D. G., & Mirman, D. (2013). Gaze fuctuations are not additively decomposable: Reply to Bogartz and Staub. Cognition, 126(1), 128–134. https:// doi.org/10.1016/j.cognition.2012.09.002 Kelty-Stephen, D. G., Stirling, L. A., & Lipsitz, L. A. (2016). Multifractal temporal correlations in circle-tracing behaviors are associated with the executive function of rule-switching assessed by the trail making test. Psychological Assessment, 28(2), 171–180. https://doi.org/10.1037/pas0000177 Kiely, J., & Collins, D. J. (2016). Uniqueness of human running coordination: The integration of modern and ancient evolutionary innovations. Frontiers in Psychology, 7, 262. https://doi.org/10.3389/fpsyg.2016.00262 Kimmel, M., & Rogler, C. R. (2018). Afordances in interaction—The case of Aikido. Ecological Psychology, 0. https://doi.org/10.1080/10407413.2018.14 38198 Kugler, P. N., Kelso, J. S., & Turvey, M. T. (1982). On the control and coordination of naturally developing systems. In J. A. S. Kelso & J. E. Clark (Eds.), The Development of Movement Control and Coordination (pp. 5–78). John Wiley & Sons. Kugler, P. N., & Turvey, M. T. (1987). Information, Natural Law, and the SelfAssembly of Rhythmic Movement. Lawrence Erlbaum.

Scaling Up

191

Lewontin, R. C. (1982). Organism and environment. In H. C. Plotkin (Ed.), Learning, Development, and Culture (pp. 151–170). Wiley. Lovejoy, S., & Schertzer, D. (2018). The Weather and Climate: Emergent Laws and Multifractal Cascades. Cambridge University Press. Mandelbrot, B. B. (1967). How long is the coast of Britain? Statistical self-similarity and fractional dimension. Science, 156(3775), 636–638. https://doi. org/10.1126/science.156.3775.636 Mandelbrot, B. B. (1974). Intermittent turbulence in self-similar cascades: Divergence of high moments and dimension of the carrier. Journal of Fluid Mechanics, 62(2), 331–358. https://doi.org/10.1017/S0022112074000711 Mangalam, M., Carver, N. S., & Kelty-Stephen, D. G. (2020a). Global broadcasting of local fractal fuctuations in a bodywide distributed system supports perception via efortful touch. Chaos, Solitons & Fractals, 135, 109740. https://doi. org/10.1016/j.chaos.2020.109740 Mangalam, M., Carver, N. S., & Kelty-Stephen, D. G. (2020b). Multifractal signatures of perceptual processing on anatomical sleeves of the human body. Journal of The Royal Society Interface, 17(168), 20200328. https://doi.org/10.1098/ rsif.2020.0328 Mangalam, M., Chen, R., McHugh, T. R., Singh, T., & Kelty-Stephen, D. G. (2020). Bodywide fuctuations support manual exploration: Fractal fuctuations in posture predict perception of heaviness and length via efortful touch by the hand. Human Movement Science, 69, 102543. https://doi.org/10.1016/j. humov.2019.102543 Mangalam, M., Fragaszy, D. M., Wagman, J. B., Day, B. M., Kelty-Stephen, D. G., Bongers, R. M., Stout, D. W., & Osiurak, F. (2022). On the psychological origins of tool use. Neuroscience & Biobehavioral Reviews, 104521. https://doi. org/10.1016/j.neubiorev.2022.104521 Mangalam, M., & Kelty-Stephen, D. G. (2020). Multiplicative-cascade dynamics supports whole-body coordination for perception via efortful touch. Human Movement Science, 70, 102595. https://doi.org/10.1016/j.humov.2020.102595 Mark, L. S. (1987). Eyeheight-scaled information about afordances: A study of sitting and stair climbing. Journal of Experimental Psychology: Human Perception and Performance, 13(3), 361–370. https://doi.org/10.1037/0096-1523. 13.3.361 Mark, L. S., Balliett, J. A., Craver, K. D., Douglas, S. D., & Fox, T. (1990). What an actor must do in order to perceive the afordance for sitting. Ecological Psychology, 2(4), 325–366. https://doi.org/10.1207/s15326969eco0204_2 Marsden, C. D., Merton, P. A., & Morton, H. B. (1983). Rapid postural reactions to mechanical displacement of the hand in man. Advances in Neurology, 39, 645–659. http://europepmc.org/abstract/MED/6660114 Mazzi, B., & Vassilicos, J. C. (2004). Fractal-generated turbulence. Journal of Fluid Mechanics, 502, 65–87. https://doi.org/10.1017/S0022112003007249 Michaels, C. F. (2003). Afordances: Four points of debate. Ecological Psychology, 15(2), 135–148. https://doi.org/10.1207/S15326969ECO1502_3 Moreno, M. A., Stepp, N., & Turvey, M. T. (2011). Whole body lexical decision. Neuroscience Letters, 490(2), 126–129. https://doi.org/10.1016/j.neulet.2010. 12.041

192

Damian G. Kelty-Stephen

Nicolis, G., & Prigogine, I. (1977). Self-Organization in Nonequilibrium Systems. Wiley. Palatinus, Z., Kelty-Stephen, D. G., Kinsella-Shaw, J., Carello, C., & Turvey, M. T. (2014). Haptic perceptual intent in quiet standing afects multifractal scaling of postural fuctuations. Journal of Experimental Psychology: Human Perception and Performance, 40(5), 1808–1818. https://doi.org/10.1037/a0037247 Pattee, H. H. (1977). Dynamic and linguistic modes of complex systems. International Journal of General Systems, 3(4), 259–266. https://doi. org/10.1080/03081077708934771 Pattee, H. H. (2001). The physics of symbols: Bridging the epistemic cut. Biosystems, 60(1), 5–21. https://doi.org/10.1016/S0303-2647(01)00104-6 Pattee, H. H. (2007). Laws, constraints, and the modeling relation—History and interpretations. Chemistry & Biodiversity, 4(10), 2272–2295. https://doi. org/10.1002/cbdv.200790187 Pattee, H. H. (2013). Epistemic, evolutionary, and physical conditions for biological information. Biosemiotics, 6(1), 9–31. https://doi.org/10.1007/s12304-0129150-8 Petzschner, F. H., Garfnkel, S. N., Paulus, M. P., Koch, C., & Khalsa, S. S. (2021). Computational models of interoception and body regulation. Trends in Neurosciences, 44(1), 63–76. https://doi.org/10.1016/j.tins.2020.09.012 Petzschner, F. H., Weber, L. A. E., Gard, T., & Stephan, K. E. (2017). Computational psychosomatics and computational psychiatry: Toward a joint framework for diferential diagnosis. Biological Psychiatry, 82(6), 421–430. https:// doi.org/10.1016/j.biopsych.2017.05.012 Posteraro, T. S. (2014). On the utility of virtuality for relating abilities and afordances. Ecological Psychology, 26(4), 353–367. https://doi.org/10.1080/10407 413.2014.958039 Profeta, V. L. S., & Turvey, M. T. (2018). Bernstein’s levels of movement construction: A contemporary perspective. Human Movement Science, 57, 111–133. https://doi.org/10.1016/j.humov.2017.11.013 Qian, K., Jing, X., Duan, Y., Zhou, B., Fang, F., Xia, J., & Ma, X. (2020). Grasp pose detection with afordance-based task constraint learning in single-view point clouds. Journal of Intelligent & Robotic Systems, 100(1), 145–163. https://doi.org/10.1007/s10846-020-01202-3 Raja, V., & Anderson, M. L. (2021). Behavior considered as an enabling constraint. In F. Calzavarini & M. Viola (Eds.), Neural Mechanisms: New Challenges in the Philosophy of Neuroscience (pp. 209–232). Springer International Publishing. https://doi.org/10.1007/978-3-030-54092-0_10 Raja, V., & Chemero, A. P. (2020). In favor of impropriety. Constructivist Foundations, 15(3), 213–216. https://constructivist.info/15/3/213.raja Reed, E. S. (1996). Encountering the World: Toward an Ecological Psychology. Oxford University Press. Rietveld, E., & Kiverstein, J. (2014). A rich landscape of afordances. Ecological Psychology, 26(4), 325–352. https://doi.org/10.1080/10407413.2014.958035 Segundo-Ortin, M., Heras-Escribano, M., & Raja, V. (2019). Ecological psychology is radical enough: A reply to radical enactivists. Philosophical Psychology, 32(7), 1001–1023. https://doi.org/10.1080/09515089.2019.1668238

Scaling Up

193

Shaw, R. E. (2001). Processes, acts, and experiences: Three stances on the problem of intentionality. Ecological Psychology, 13(4), 275–314. https://doi. org/10.1207/S15326969ECO1304_02 Shaw, R. E., & Kinsella-Shaw, J. M. (1988). Ecological mechanics: A physical geometry for intentional constraints. Human Movement Science, 7(2), 155–200. https://doi.org/10.1016/0167-9457(88)90011-5 Shaw, R. E., Kinsella-Shaw, J. M., & Mace, W. M. (2019). Afordance types and afordance tokens: Are Gibson’s afordances trustworthy? Ecological Psychology, 31(1), 49–75. https://doi.org/10.1080/10407413.2018.1508353 Soodak, H., & Iberall, A. (1978). Homeokinetics: A physical science for complex systems. Science, 201(4356), 579–582. https://doi.org/10.1126/science.201. 4356.579 Stephen, D. G., Anastas, J., & Dixon, J. (2012). Scaling in cognitive performance refects multiplicative multifractal cascade dynamics. Frontiers in Physiology, 3, 102. https://doi.org/10.3389/fphys.2012.00102 Stephen, D. G., Boncoddo, R. A., Magnuson, J. S., & Dixon, J. A. (2009). The dynamics of insight: Mathematical discovery as a phase transition. Memory & Cognition, 37(8), 1132–1149. https://doi.org/10.3758/MC.37.8.1132 Stephen, D. G., & Dixon, J. A. (2011). Strong anticipation: Multifractal cascade dynamics modulate scaling in synchronization behaviors. Chaos, Solitons & Fractals, 44(1), 160–168. https://doi.org/10.1016/j.chaos.2011.01.005 Stephen, D. G., & Hajnal, A. (2011). Transfer of calibration between hand and foot: Functional equivalence and fractal fuctuations. Attention, Perception, & Psychophysics, 73(5), 1302–1328. https://doi.org/10.3758/s13414-0110142-6 Stephen, D. G., Stepp, N., Dixon, J. A., & Turvey, M. T. (2008). Strong anticipation: Sensitivity to long-range correlations in synchronization behavior. Physica A: Statistical Mechanics and Its Applications, 387(21), 5271–5278. https://doi. org/10.1016/j.physa.2008.05.015 Stepp, N., & Turvey, M. T. (2015). The muddle of anticipation. Ecological Psychology, 27(2), 103–126. https://doi.org/10.1080/10407413.2015.1027123 Teng, D. W., Eddy, C. L., & Kelty-Stephen, D. G. (2016). Non-visually-guided distance perception depends on matching torso fuctuations between training and test. Attention, Perception, & Psychophysics, 78(8), 2320–2328. https:// doi.org/10.3758/s13414-016-1213-5 Turvey, M. T. (1977). Contrasting orientations to the theory of visual information processing. Psychological Review, 84(1), 67–88. https://doi.org/10.1037/ 0033-295X.84.1.67 Turvey, M. T. (1992). Afordances and prospective control: An outline of the ontology. Ecological Psychology, 4(3), 173–187. https://doi.org/10.1207/ s15326969eco0403_3 Turvey, M. T, & Fonseca, S. T. (2014). The medium of haptic perception: A tensegrity hypothesis. Journal of Motor Behavior, 46(3), 143–187. https://doi.org/10. 1080/00222895.2013.798252 Turvey, M. T, Shaw, R. E., Reed, E. S., & Mace, W. M. (1981). Ecological laws of perceiving and acting: In reply to Fodor and Pylyshyn (1981). Cognition, 9(3), 237–304.

194

Damian G. Kelty-Stephen

van Dijk, L., & Withagen, R. (2015). Temporalizing agency: Moving beyond on-and ofine cognition. Theory & Psychology, 26(1), 5–26. https://doi. org/10.1177/0959354315596080 Van Orden, G. C. (2010). Voluntary performance. Medicina, 46(9), 581. https:// doi.org/10.3390/medicina46090083 Van Orden, G. C., Holden, J. G., & Turvey, M. T. (2003). Self-organization of cognitive performance. Journal of Experimental Psychology: General, 132(3), 331–350. https://doi.org/10.1037/0096-3445.132.3.331 Van Orden, G. C., Hollis, G., & Wallot, S. (2012). The blue-collar brain. Frontiers in Physiology, 3, 207. https://doi.org/10.3389/fphys.2012.00207 Vinson, D. W., Abney, D. H., Amso, D., Chemero, A., Cutting, J. E., Dale, R., Freeman, J. B., Feldman, L. B., Friston, K. J., & Gallagher, S. (2016). Perception, as you make it. Behavioral and Brain Sciences, 39, e260. https://doi.org/10.1017/ S0140525X15002678 von Bertalanfy, L. (1986). General System Theory: Foundations, Development, Applications. Braziller. Wagman, J. B., Caputo, S. E., & Stofregen, T. A. (2016a). Hierarchical nesting of afordances in a tool use task. Journal of Experimental Psychology: Human Perception and Performance, 42(10), 1627–1642. https://doi.org/10.1037/ xhp0000251 Wagman, J. B., Caputo, S. E., & Stofregen, T. A. (2016b). Sensitivity to hierarchical relations among afordances in the assembly of asymmetric tools. Experimental Brain Research, 234(10), 2923–2933. https://doi.org/10.1007/ s00221-016-4695-4 Wagman, J. B., & Stofregen, T. A. (2020). It doesn’t add up: Nested afordances for reaching are perceived as a complex particular. Attention, Perception, & Psychophysics, 82(8), 3832–3841. https://doi.org/10.3758/s13414-020-02108-w Wagman, J. B., Stofregen, T. A., Bai, J., & Schloesser, D. S. (2017). Perceiving nested afordances for another person’s actions. Quarterly Journal of Experimental Psychology, 71(3), 790–799. https://doi.org/10.1080/17470218.2016.1 277249 Wallot, S., O’Brien, B., Coey, C. A., & Kelty-Stephen, D. (2015). Power-law fuctuations in eye movements predict text comprehension during connected text reading. In D. C. Noelle, R. Dale, A. S. Warlaumont, J. Yoshimi, T. Matlock, & C. D. Jennings (Eds.), Proceedings of the 37th Annual Meeting of the Cognitive Science Society (pp. 2583–2588). Cognitive Science Society. Wang, A., & Tarr, M. (2018). Learning intermediate features of afordances with a convolutional neural network. Journal of Vision, 18(10), 1267. https://doi. org/10.1167/18.10.1267 Warren, W. H. (1984). Perceiving afordances: Visual guidance of stair climbing. Journal of Experimental Psychology: Human Perception and Performance, 10(5), 683–703. https://doi.org/10.1037/0096-1523.10.5.683 Warren, W. H., & Whang, S. (1987). Visual guidance of walking through apertures: Body-scaled information for afordances. Journal of Experimental Psychology: Human Perception and Performance, 13(3), 371–383. https://doi. org/10.1037/0096-1523.13.3.371

Scaling Up

195

Winograd, T. (1972). Understanding natural language. Cognitive Psychology, 3(1), 1–191. https://doi.org/10.1016/0010-0285(72)90002-3 Withagen, R., & Chemero, A. (2012). Afordances and classifcation: On the signifcance of a sidebar in James Gibson’s last book. Philosophical Psychology, 25(4), 521–537. https://doi.org/10.1080/09515089.2011.579424 Withagen, R., & van Wermeskerken, M. (2010). The role of afordances in the evolutionary process reconsidered: A niche construction perspective. Theory & Psychology, 20(4), 489–510. https://doi.org/10.1177/0959354310361405 Yates, F. E. (1982). Systems analysis of hormone action. In R. F. Goldberger & K. R. Yamamoto (Eds.), Biological Regulation and Development: Hormone Action (pp. 25–97). Springer. https://doi.org/10.1007/978-1-4684-1125-6_2 Zipoli Caiani, S. (2021). Intensional biases in afordance perception: An explanatory issue for radical enactivism. Synthese, 198(17), 4183–4203. https://doi. org/10.1007/s11229-018-02049-w

PART III

Afordances Through the Lens of Neuroscience

11 AFFORDANCE SWITCHING IN SELF-ORGANIZING BRAIN-BODYENVIRONMENT SYSTEMS Vicente Raja and Matthieu M. de Wit

In this chapter, we develop answers to the fve questions posed to each of the contributors of this volume on the modern legacy of James J. Gibson’s afordance concept. We devote special attention to the important but relatively unexplored question of how organisms switch from responding from one to another afordance, focusing in particular on understanding the dynamics at the neural scale within the brain-body-environment system during this process. What Do We Understand by the Term Afordance?

We understand afordances as opportunities for action that organisms fnd in their environment. These opportunities for action are things like the grab-ability of a mug, the pass-ability of a door, the catch-ability of a ball, and the cross-ability of a street or a river. The presence of an afordance for an organism in a given situation can be arbitrarily complex. For instance, the grab-ability of a mug depends on the size and shape of the mug but also on the anatomy and skills of the organism. And the case of the cross-ability of a street is even more complicated as, at least for humans, it involves social norms and constraints on top of the environmental layout and the body and abilities of the street crosser. While acknowledging this complexity, ecological psychologists claim organisms can directly perceive afordances because there’s enough environmental information as to that to be the case (see the next section for brief further discussion). Direct perception of afordances and the environmental availability of information to do so are the only two constraints of our understanding of the concept. DOI: 10.4324/9781003396536-15

200

Vicente Raja and Matthieu M. de Wit

This means we will not engage in defnitional debates regarding the ontological status of afordances. There are several options out there: afordances as dispositions (Turvey, 1992; Turvey et al., 1981), afordances as nonfactual dispositions (Heras-Escribano, 2019), afordances as relations (Chemero, 2003; Stofregen, 2003), and afordances as their own thing (Barrett, 2020), to name a few. If you push either of us, we would likely choose one of these ontological characterizations of afordances for one or another reason. However, we think this is not relevant for our current purposes. The analysis we provide in the following sections should be immune to these ontological considerations. All the diferent ontological positions should be compatible with the idea of afordances as directly perceivable (via information) opportunities for action. What Role Does Afordance Play in Perception? Is It the Entity Organisms Perceive or the Means Through Which Organisms Perceive? Are Afordances the Only Perceptual Dependent Variables?

Afordances are objects of perception. Other theories of perception, including the classical ones, take colors, distances, or shapes, for instance, to be the objects of perception. The novelty introduced by Gibson in ecological psychology was to include opportunities for action in the list of the possible objects of perception—along with removing some of the classical ones from that same list. In this context, afordances are not the means through which organisms perceive. If anything, the means through which organisms perceive afordances in their environment have to do with the detection of information available in the energy arrays (e.g., light, chemicals) that surround them. The way such information specifes the available afordances and the way organisms are able to detect them are open empirical questions on which ecological psychologists keep working (see, e.g., Lee, 2009; Segundo-Ortin et al., 2019; Turvey, 2018). But that there is (ecological) information that specifes afordances and that such information is detectable by organisms is perhaps the core ecological hypothesis: [I]f there’s information in light for the perception of surfaces, is there information for the perception of what they aford? Perhaps the composition and layout of surfaces constitute what they aford. If so, to perceive them is to perceive what they aford. This is a radical hypothesis, for it implies that the “values” and “meanings” of things in the environment can be directly perceived. —Gibson (1979, p. 119, emphasis in the original)

Afordance Switching in Brain-Body-Environment Systems

201

There are two dimensions to this quote. First, it restates the hypothesis of afordances as objects of perception. The information in light—Gibson was focused on vision—is, perhaps, for afordances. This is a way to say ecological information specifes afordances. And second, it seems to clearly suggest afordances are not the only object of perception. Surfaces are perceivable as well, or at least that’s also explicitly claimed. And events are generally understood as objects of perception as well in the ecological literature (see Shaw & Cutting, 1980; Stofregen, 2000). Also, places themselves have been proposed as perceivable (Heft, 2018). An open question is also suggested in the quote above: are surfaces and their afordances the same? If that were the case, afordances could be in some sense the only object of perception. We would only need to equate not only surfaces but also events and places, with afordances, and we would have just one perceptual object to bind them all. To the best of our knowledge, however, this kind of thesis has not been properly defended in the literature. So, pending a convincing argument that defends otherwise, we hold a pluralist position regarding the objects of perception. Why Is Afordance Any Better Than Stimulus? What Does a Theory of Afordance Suggest That Stimuli Cannot; How Has It Moved the Needle Past Gibson 1960’s Recognition of How Little We Can Defne Stimuli?

We want to begin this section with a cautionary note: comparing the notions of afordance and stimulus can be misleading. The role afordances fulfll in the ecological theory is not completely akin to the role stimulus fulflls in other theories. As noted earlier, afordances are usually understood as one of the objects of perception, that is, one of those aspects of the environment organisms perceive by detecting ecological information. The way stimuli are characterized in the cognitive sciences writ large is more complicated than this. As James Gibson (1960) pointed out, the diferent uses of the notion of stimulus in the literature are not always compatible and, more generally, do not ofer a coherent understanding of what stimulation is and how it is related to perception and action. Raja (2022) reviews a bit of contemporary literature in which very diferent entities such as the input of retinal ganglion cells (Sayood, 2018), the vibration of rats’ whiskers (Ince et al., 2010), the structural properties of pictures of faces (Hashemi et  al., 2019), or the movie Memento (Kauttonen et  al., 2018) are labeled as “stimulus.” Some of these entities seem to refer to whatever triggers the activation of a physiological receptor (e.g., the input of retinal ganglion cells), while some others seem to refer to whatever our perceptual experience—psychological experience, more generally—of a given sensory

202

Vicente Raja and Matthieu M. de Wit

input is (e.g., the movie Memento). This variety of conceptions of stimulation reinforces Gibson’s claims in the 1960s and shows that the past few decades have not witnessed a serious improvement on the issue. Moreover, it highlights the mismatch between the notions of afordance and stimulus. The disconnection between afordances and stimuli is clear when the latter are understood in physiological terms. What triggers the activation of a physiological receptor is not the object of perception. In the mainstream theory, whatever inputs our sensory receptors get are not enough to provide perceptual knowledge of the environment, and perceptual states are only possible after the sensory input is processed by the brain. Perceptual states are built up within the brain in the form of perceptual models or perceptual representations (see the next section for further discussion of explanation via intrinsic mechanisms). And the objects of perception that feature in those representations are not stimuli but things, distances, scenes, etc. The physiological notion of stimuli has been historically applied tout court into psychology and has resulted in the idea of atomic stimulation. Put simply, atomic stimulation in psychology is physiological stimulation with sensory correlates—for example, correlates are not activations of receptors but simple sensations such as “cold” or “yellow” or “C sharp.” Gibson followed a tradition that can be traced back to William James and his defense of the richness and complexity of stimulation. Stimulation can be rich (i.e., non-atomic), and it can actually provide perceptual knowledge about the environment without the need for internal processing or a combination of simple sensations. The ecological notion of stimulus information is a culmination of this kind of thinking. So the story goes, some structures of the arrays of stimulation that surround us and other organisms are informative of the environment and our (and their) relation with it. For instance, together with other sources of information (e.g., inertial), a global centrifugal fow in our optical feld tells us we are engaging in forward locomotion, while a global centripetal fow tells us we are locomoting backward. These (higher-order) structures are ecological information. For ecological psychologists, ecological information provides organisms with perceptual contact to their environment by specifying the opportunities for interaction the latter ofers to them. These opportunities for interaction are, of course, afordances. Ecological information, as a form of stimulus information, substitutes the classical notion of atomic stimuli in the context of ecological psychology and provides a naturalistic way to account for perception without recourse to neural processes of construction, combination, or enrichment. But what about the other psychological notion of stimulus? Aren’t afordances like the objects of perception we experience when we watch Memento, for instance? In a way, afordances are just a diferent kind of

Afordance Switching in Brain-Body-Environment Systems

203

object. Therefore, if Memento is a stimulus, afordances must somehow be a stimulus as well. According to Gibson (1960), this confates the notion of stimulus with the notion of object of perception or object of experience. It is just a diferent uncareful application of the physiological notion of stimulus in psychology, this time not focusing on its atomistic nature but on its dependence on the response triggered—for example, the movie Memento is a stimulus because the experience of watching Memento is triggered by it. Despite its problems, and even if we do not endorse it, this is the accepted status quo in the feld. In a strict sense, afordances as objects of perception can be regarded as “stimuli” as much as any other object of perception. Afordances, nevertheless, are still diferent from those other objects of perception that mainstream cognitive sciences label as stimuli: while all the mainstream objects of perception achieve their perceptual status by a process of mental construction or enrichment, afordances are objects of perception because they are specifed by ecological information. This, again, is the core thesis of ecological realism. Afordances are properties of the organism-environment system that are meaningful to the organism without the need for a subjective attribution of meaning. Afordances provide a theory of meaning that takes meaning from “inside here,” that is, from the minds (or brains) of perceivers, and puts it “out there,” that is, in the organism-environment relationship. This is a revolution in psychology that would be impossible without abandoning the classical notion of stimulus. What Is the Connection Between Afordances, Behavioral Scale, and Intention? How Distant or Reluctant Can a PerceivingActing System Be Before Afordances Come Into Play? Do Afordances Exert Their Role Only When the System Is Close or Willing Enough?

When organisms act in their environment, they perceptually guide their own action with respect to afordances. In this context, a complete description of the behavioral scale includes the afordances involved in the described behavior. At least according to ecological psychologists, afordances are therefore tightly connected to behavior and to the behavioral scale of analysis (cf. de Wit et al., 2017). The relationship between afordances and intentions is not as straightforward as the one between afordances and behavior. The most important reason for this is perhaps that intentions are difcult to operationalize. Simply defned, intentions are the goals or aims of a cognitive system. They are a name for all those psychological states related to the willingness to do something. In a sense, intentions are triggers of behavior: we walk, run, grab a mug, kick a ball, write a poem, and so on, because we intend to do

204

Vicente Raja and Matthieu M. de Wit

so. It is relatively easy to agree with such a simple defnition when in a colloquial environment. However, it is not that easy to make scientifc sense of it. This difculty is likely due to the dual character of intentions that is revealed after a deeper scrutiny of the concept. On the one hand, intentions seem to be intrinsic to the cognitive system. Cognitive systems seem to be the origin of their own intentions and, therefore, of their own behavior. On the other hand, intentions seem to transcend cognitive systems. At least sometimes, intentions are about things other than cognitive systems—such aboutness is usually called “intentionality” by philosophers—and we recognize them as produced by extrinsic factors. In this sense, intentions seem to belong to the cognitive system and, at the same time, they seem to point to other things outside that system. Therefore, the main question is: how can we make sense of this duality scientifcally? It is not possible to provide a general answer to this question in the rest of this chapter. A possible way to narrow its scope into a manageable project is to address intentions from the point of view of task switching. How do we change from performing one task to performing another? Imagine you take the tram and decide to remain standing up despite the availability of free seats. After a couple of stops, you decide to sit down on one of those free seats. What is going on for that change to occur? Or imagine that, while playing a football game, you are running with the ball and then pass the ball to another member of your team. What happened for you to change from running to passing at that very moment? What is going on in these two situations for you to switch from one task (standing; running) to a diferent one (sitting; passing)? It seems that, maybe along with other cognitive and emotional processes, these are events in which cognitive systems change or develop their intentions. In this context, one possibility is to describe a fully intrinsic mechanism to explain task switching. Such a mechanism would entail the complete intrinsicality of intentions. This is a common strategy in cognitive neuroscience. For instance, Menon and Uddin (2010) have proposed the insula as a multitasking hub in the brain in charge of diferent activities, including task switching. This strategy is pervasive in the feld and is not diferent from attributing aspects of memory to the hippocampus or language production to Broca’s area. Beyond the well-known, general problems faced by the localization of psychological functions in particular brain areas (see Anderson, 2014), the search for a concrete task-switching mechanism in the brain has met with mixed results. For instance, a meta-analysis by Lenartowicz et al. (2010) found no unique neural signature for task switching. Neither of these competing results is of course defnitive, but we think their disparity is an invitation to fnd alternative solutions and interpretations for the phenomenon of task switching and, consequently, for intentions.

Afordance Switching in Brain-Body-Environment Systems

205

One of these alternatives is a genuinely ecological approach to task switching: task switching does not happen in the brain but within the organism-environment system. In other words, a proper characterization of task switching cannot be provided by the sole appeal to neural resources. Nonneural resources, like (non-neural) bodily and environmental factors— including afordances and enabling constraints (see Raja & Anderson, 2021)—are necessary to make full sense of this and other psychological phenomena. In the context of such an ecological (or at least distributed) strategy, Proftt and Linkenauger (2013) have coined the phrase “phenotypic reorganization” to describe the way in which organisms organize themselves for various tasks: walking, running, eating, grasping, throwing, etc. This self-organization creates temporary, reproducible synergies in both the body and brain to achieve the task at hand. Motor synergies in the body and neural synergies in the brain (like TALoNS; see Anderson, 2014) but also synergies that cut across the boundaries of the brain-bodyenvironment system (e.g., Baggs et al., 2020; Dotov et al., 2010) support the performance of the diferent tasks cognitive systems face in their daily life (see also Bingham, 1988). One aspect of this “phenotypic” organization is that it can be re-organized when diferent conditions are met. A cognitive system organized in this way for a particular task is robust enough to handle diferent perturbations within the task. For instance, a runner is able to keep running even when several conditions of the ground (e.g., shape, consistency) change. This is due to the redundancy of bodily and neural resources of the system that, when combined with the constraints of the environment, allow for a stable “running” organization to emerge even if when looking at little details of the system (i.e., some conditions of the ground or the particular activation of a set of muscles) nothing remains equal and constant variability is the rule. The system, though robust, is however not resilient to all possible perturbations and some of them do afect it. When they occur, they can cause the organism to switch tasks, say from grazing to running from a predator or from running to jumping in the face of an obstacle on the ground. It is also, at least in some cases, possible for an organism to resist responding to a perturbation, depending on the intentions of the organism and the particularities of the environmental situation (Withagen et  al., 2017). It is likely that, within this general process that cuts across the boundaries of the brain-body-environment system, the insula, for instance, registers and resonates to what changes in the landscape, but this is not the same as saying that insula causes the switch, or is the switch, even in cases when it is an important part of the whole causal chain. Thus, in this conceptualization of task switching, no clear distinction can be made between perception, action, or cognitive (e.g., switching) “parts.” There is no privileged

206

Vicente Raja and Matthieu M. de Wit

(e.g., neural) scale (Carello et al., 2012), and, it may be hard if not impossible to disassemble the system into stable functional parts that stay within the boundary of brain or skin (de Wit & Matheson, 2022). On the contrary, what relevant perturbations do is to change a given phenotypic organization by disrupting the (enabling) constraints that defne its softly assembled synergetic state, allowing for (or in some cases forcing) a phenotypic reorganization into a new synergy. This can occur in many diferent systems, some of them non-cognitive. For instance, if you somehow break the toilet bowl during a fush, the constraints imposed by the bowl and drain disappear, and gravity becomes the dominant force, spilling the water onto the bathroom foor. But the process also occurs in the case of cognitive systems executing diferent tasks. For example, if the blacksmith drops the hot iron he is molding, he ceases to be a “hammerer,” and becomes a “hot horseshoe avoider.” The carpenter’s broken nail will similarly disrupt the hammering while he goes in search of his pliers to remove the shard. As illustrated in these examples, afordance perception and actualization— and switches between them—are typically dynamic. Although many afordance judgment paradigms might imply otherwise, afordances are very commonly responded to (through action or inaction) “in the moment.” We have discussed situations in which task switching occurs and can be explained not in terms of an intrinsic mechanism but in terms of the selforganization of the brain-body-environment system, including changes at all scales of that system when it is disrupted. Some of the possible changes are, of course, changes in afordances. And these changes at diferent scales can be investigated empirically as cognitive systems engage with diferent tasks and exhibit diferent intentions. How Do Systems Engage With Afordances as They Move Among Tasks and Intentions?

As argued in the previous section, a complete answer to the question of how systems switch from responding from one to another afordance requires an analysis of the system at all scales (de Wit & Withagen, 2019). So far, ecological psychology has largely focused on the behavioral scale and hence there is a paucity of research on the contributions of the nervous system in brain-body-environment systems when organisms respond to afordances. However, interest in the neural scale is growing in the ecological community under the header of ecological neuroscience (e.g., Anderson, 2014; de Wit et al., 2017; Falandays et al., 2023; Favela, 2024; Jirsa et al., 2019; Raja, 2018, 2020; van der Weel et al., 2019). Moreover, there is a substantial ecological psychological literature that has focused on characterizing (strategy) switching within various cognitive tasks at the

Afordance Switching in Brain-Body-Environment Systems

207

behavioral scale which, although not focused on afordances, is relevant to this question and which we will discuss here (Anastas et al., 2011; Bruineberg et al., 2021; Dixon et al., 2014; Dotov et al., 2010; Favela et al., 2021; Kello & Van Orden, 2009; Nalepka et al., 2017, 2019, 2021; Patil et  al., 2020; Stephen et  al., 2009; Van Orden et  al., 2011; Wallot et  al., 2019). We will end the chapter by discussing ongoing research in the lab of one of us that characterizes the neural scale while participants dynamically respond to afordances. Anastas et  al. (2011) presented participants with a modifed version of the Wisconsin card sorting test, which is traditionally used to test children’s ability to fexibly switch among sorting rules. In this modifed version, young adults were instructed to induce the correct sorting rule on the basis of feedback about their card placement performance. In the language introduced previously, in the early phase of this task, as participants were attempting to discover the rule, their phenotypic organization might be described as a “rule-searching card sorter.” Then, following induction of the correct sorting rule, they can be seen as having reorganized into a “rule-following cart sorter.” The question is how to understand this switch. Is it best conceptualized intrinsically, as a switch controlled by activity at the neural scale (cf. Menon & Uddin, 2010), or as a switch between two self-organizing brain and body-spanning synergies? Results suggest the latter. As participants grasped and placed cards, their hand movements were tracked at a 60 Hz sampling rate and with millimeter precision. These time series were then submitted to detrended fuctuation analysis (DFA; Peng et  al., 1994). DFA is used to estimate Hurst exponents, which characterize the degree to which the dynamics of fuctuations within time series are scale-free or “fractal,” meaning that the statistics of fuctuations at any scale resemble those of fuctuations at all other available scales. This is indicative of interactions across scales and, with that, the absence of a privileged scale. Furthermore, an increase in Hurst exponent toward a value of 1 indicates loosening of constraints within a system (such as a “card sorter”), allowing for reorganization of that system, while a decrease in the direction of 0 indicates increased stability in the system (Hardstone et al., 2012) and thus presumably less openness to perturbation. Results showed that the Hurst exponent derived, recall, from participants’ hand motions showed a sharp rise and then fall as participants induced the rule, suggesting, frst, that they transitioned from one stable phenotypic organization into another and, second, interactions across scales. These results simultaneously argue against both an intrinsic view of switching (in this case between strategies) and a clean localization of action and cognitive processes onto specialized parts within scales (Dixon et al., 2014).

208

Vicente Raja and Matthieu M. de Wit

Thus, there is evidence for phenotypic organization and reorganization in the non-neural body (e.g., Anastas et  al., 2011; Stephen et  al., 2009). Similar results have been obtained when looking at reorganization of synergies involving not only the body but also aspects of the physical environment (i.e., tools; Dotov et al., 2010; Favela et al., 2021) and the social environment (Nalepka et al., 2017, 2019, 2021; Patil et al., 2020). There is also a large literature looking at fexible reorganization in the brain (e.g., Cocchi et al., 2017; Kelso, 2012). Though, taken together, all of the aforementioned implies that reorganization involves soft-assembly of synergies that span the neural and behavioral scales, very little research has attempted to characterize the fractality of time series simultaneously at each of these diferent scales (cf. Kello & Van Orden, 2009). Furthermore, to our knowledge no one has investigated such organization and reorganization while participants dynamically respond to afordances. We are currently developing a project that attempts to do just this. In an augmented reality interception task, participants are presented with expanding dot patterns that create the impression of a looming ball superimposed on a large response button attached to the wall in front of them. Balls approach the participant at a wide range of randomized speeds and the participant attempts to intercept the ball by hitting the button at the perceived time of impact of the ball with the wall. In an initial training phase, participants are instructed to always attempt to intercept the ball. Successful interceptions are defned as hitting the response button within a window of 140 ms around time-to-contact (TTC) of the ball relative to the wall of zero. Participants receive performance feedback, allowing them to discover which balls they can and cannot intercept successfully, that is, their afordance boundaries. In a subsequent afordance perception and actualization phase in which they no longer receive feedback, participants are instructed to “only go for balls that you can successfully intercept.” This paradigm allows us to track participants as they switch from being capable to incapable “interceptors,” and the behavioral and neural characteristics that are associated with sensitivity to this boundary. We are using functional near-infrared spectroscopy (fNIRS) to characterize the neural scale during the afordance perception and actualization phase. fNIRS allows for movement in participants (Pinti et al., 2020) making it a suitable method for measuring neural activity in dynamic afordance tasks. Our montage is set up to capture signal from “sensorimotor” cortex extending into the frontal cortex, given arguments for the involvement of those regions in afordance perception and actualization (e.g., Pezzulo & Cisek, 2016). We predict an increase in the Hurst exponent of the fNIRS time series as participants approach their afordance boundaries, since this is where

Afordance Switching in Brain-Body-Environment Systems

209

we would expect changes in the stability of the brain-body-environment system as it reorganizes. Changes in information use (i.e., toward TTCspecifying information) as well as changes in interception skill and afordance boundary sensitivity can also be conceptualized as phenotypic reorganizations, and so we also predict changes in Hurst exponents as a function of those (cf. Hajnal et al., 2018; Mangalam et al., 2020; Palatinus et al., 2013; Stephen et al., 2010). Tests of some of these predictions are currently underway. Future work will characterize scale-free fuctuations simultaneously at the behavioral and neural scales by including analyses of arm, head, and eye movements, in addition to fNIRS time series. Such a fnding would support the conceptualization of brains—not as intrinsic controllers of switches—but as components of larger self-organizing brain and body spanning soft-assembled synergies that reorganize as a function of changes in the afordances available to them, as outlined in this chapter. Finally, we briefy note that we have not discussed a deeper question underlying the question of why we expect scale-free dynamics at the neural scale: why are brains present at all in many organisms, given that transitions between diferent behavioral states are also possible in systems without nervous systems (Fultot et  al., 2019). One start of an answer to this question might be that neurons and neural networks are an excellent substrate for the complex, dynamic interactions between elements of a system that are required for fexible self-organization and reorganization (Cocchi et al., 2017; Kelso, 2012); another, that dynamics at the neural scale enable functional organization and reorganization at very short time scales (Fultot et al., 2019), which arguably constitutes an adaptive advantage to many forms of life. Reference List Anastas, J. R., Stephen, D. G., & Dixon, J. A. (2011). The scaling behavior of hand motions reveals self-organization during an executive function task. Physica A: Statistical Mechanics and Its Applications, 390(9), 1539–1545. https://doi. org/10.1016/j.physa.2010.11.038 Anderson, M. L. (2014). After Phrenology: Neural Reuse and the Interactive Brain. MIT Press. Baggs, E., Raja, V., & Anderson, M. L. (2020). Extended skill learning. Frontiers in Psychology, 11, 1956. https://doi.org/10.3389/fpsyg.2020.01956 Barrett, L. (2020). What’s wrong with afordances? Constructivist Foundations, 15(3), 229–230. Bingham, G. P. (1988). Task-specifc devices and the perceptual bottleneck. Human Movement Science, 7(2), 225–264. https://doi.org/10.1016/0167-9457 (88)90013-9 Bruineberg, J., Seifert, L., Rietveld, E., & Kiverstein, J. (2021). Metastable attunement and real-life skilled behavior. Synthese, 199(5), 12819–12842. https://doi. org/10.1007/s11229-021-03355-6

210

Vicente Raja and Matthieu M. de Wit

Carello, C., Vaz, D., Blau, J. J. C., & Petrusz, S. (2012). Unnerving intelligence. Ecological Psychology, 24(3), 241–264. https://doi.org/10.1080/10407413.20 12.702628 Chemero, A. (2003). An outline of a theory of afordances. Ecological Psychology, 15(2), 181–195. https://doi.org/10.1207/S15326969ECO1502_5 Cocchi, L., Gollo, L. L., Zalesky, A., & Breakspear, M. (2017). Criticality in the brain: A synthesis of neurobiology, models and cognition. Progress in Neurobiology, 158, 132–152. https://doi.org/10.1016/j.pneurobio.2017.07.002 de Wit, M. M., de Vries, S., van der Kamp, J., & Withagen, R. (2017). Afordances and neuroscience: Steps towards a successful marriage. Neuroscience & Biobehavioral Reviews, 80, 622–629. https://doi.org/10.1016/j.neubiorev. 2017.07.008 de Wit, M. M., & Matheson, H. E. (2022). Context-sensitive computational mechanistic explanation in cognitive neuroscience. Frontiers in Psychology, 13, 903960. https://doi.org/10.3389/fpsyg.2022.903960 de Wit, M. M., & Withagen, R. (2019). What should a “Gibsonian neuroscience” look like? Introduction to the special issue. Ecological Psychology, 31(3), 147– 151. https://doi.org/10.1080/10407413.2019.1615203 Dixon, J. A., Kelty-Stephen, D. G., & Anastas, J. (2014). The embodied dynamics of problem solving. In The Routledge Handbook of Embodied Cognition. Routledge. https://doi.org/10.4324/9781315775845.ch16 Dotov, D. G., Nie, L., & Chemero, A. (2010). A demonstration of the transition from ready-to-hand to unready-to-hand. PLoS One, 5(3), e9433. https://doi. org/10.1371/journal.pone.0009433 Falandays, J. B., Yoshimi, J., Warren, W., & Spivey, M. (2023). A potential mechanism for Gibsonian resonance: Behavioral entrainment emerges from local homeostasis in an unsupervised reservoir network. Cognitive Neurodynamics. https://doi.org/10.1007/s11571-023-09988-2 Favela, L. H. (2024). The Ecological Brain: Unifying the Sciences of Brain, Body, and Environment. Routledge. Favela, L. H., Amon, M. J., Lobo, L., & Chemero, A. (2021). Empirical evidence for extended cognitive systems. Cognitive Science, 45(11), e13060. https://doi. org/10.1111/cogs.13060 Fultot, M., Adrian Frazier, P., Turvey, M. T., & Carello, C. (2019). What are nervous systems for? Ecological Psychology, 31(3), 218–234. https://doi.org/10.108 0/10407413.2019.1615205 Gibson, J. J. (1960). The concept of the stimulus in psychology. American Psychologist, 15(11), 694–703. https://doi.org/10.1037/h0047037 Gibson, J. J. (1979). The Ecological Approach to Visual Perception. Houghton Mifin. Hajnal, A., Clark, J. D., Doyon, J. K., & Kelty-Stephen, D. G. (2018). Fractality of body movements predicts perception of afordances: Evidence from stand-on-ability judgments about slopes. Journal of Experimental Psychology: Human Perception and Performance, 44(6), 836–841. https://doi.org/10.1037/ xhp0000510 Hardstone, R., Poil, S.-S., Schiavone, G., Jansen, R., Nikulin, V., Mansvelder, H., & Linkenkaer-Hansen, K. (2012). Detrended fuctuation analysis: A scale-free

Afordance Switching in Brain-Body-Environment Systems

211

view on neuronal oscillations. Frontiers in Physiology, 3, 450. https://doi. org/10.3389/fphys.2012.00450 Hashemi, A., Pachai, M. V., Bennett, P. J., & Sekuler, A. B. (2019). The role of horizontal facial structure on the N170 and N250. Vision Research, 157, 12–23. https://doi.org/10.1016/j.visres.2018.02.006 Heft, H. (2018). Places: Widening the scope of an ecological approach to perception—Action with an emphasis on child development. Ecological Psychology, 30(1), 99–123. https://doi.org/10.1080/10407413.2018.1410045 Heras-Escribano, M. (2019). The Philosophy of Afordances. Palgrave Macmillan. Ince, R. A. A., Senatore, R., Arabzadeh, E., Montani, F., Diamond, M. E., & Panzeri, S. (2010). Information-theoretic methods for studying population codes. Neural Networks, 23(6), 713–727. https://doi.org/10.1016/j.neunet.2010.05.008 Jirsa, V. K., McIntosh, A. R., & Huys, R. (2019). Grand unifed theories of the brain need better understanding of behavior: The two-tiered emergence of function. Ecological Psychology, 31(3), 152–165. https://doi.org/10.1080/1040741 3.2019.1615207 Kauttonen, J., Hlushchuk, Y., Jääskeläinen, I. P., & Tikka, P. (2018). Brain mechanisms underlying cue-based memorizing during free viewing of movie Memento. NeuroImage, 172, 313–325. https://doi.org/10.1016/j.neuroimage.2018.01.068 Kello, C. T., & Van Orden, G. C. (2009). Soft-assembly of sensorimotor function. Nonlinear Dynamics, Psychology, and Life Sciences, 13(1), 57–78. Kelso, J. A. S. (2012). Multistability and metastability: Understanding dynamic coordination in the brain. Philosophical Transactions of the Royal Society B: Biological Sciences, 367(1591), 906–918. https://doi.org/10.1098/rstb.2011.0351 Lee, D. N. (2009). General tau theory: Evolution to date. Perception, 38(6), 837–850. Lenartowicz, A., Kalar, D. J., Congdon, E., & Poldrack, R. A. (2010). Towards an ontology of cognitive control. Topics in Cognitive Science, 2(4), 678–692. https://doi.org/10.1111/j.1756-8765.2010.01100.x Mangalam, M., Chen, R., McHugh, T. R., Singh, T., & Kelty-Stephen, D. G. (2020). Bodywide fuctuations support manual exploration: Fractal fuctuations in posture predict perception of heaviness and length via efortful touch by the hand. Human Movement Science, 69, 102543. https://doi.org/10.1016/j. humov.2019.102543 Menon, V., & Uddin, L. Q. (2010). Saliency, switching, attention and control: A network model of insula function. Brain Structure and Function, 214(5), 655– 667. https://doi.org/10.1007/s00429-010-0262-0 Nalepka, P., Kallen, R. W., Chemero, A., Saltzman, E., & Richardson, M. J. (2017). Herd those sheep: Emergent multiagent coordination and behavioral-mode switching. Psychological Science, 28(5), 630–650. https://doi. org/10.1177/0956797617692107 Nalepka, P., Lamb, M., Kallen, R. W., Shockley, K., Chemero, A., Saltzman, E., & Richardson, M. J. (2019). Human social motor solutions for human—Machine interaction in dynamical task contexts. Proceedings of the National Academy of Sciences, 116(4), 1437–1446. https://doi.org/10.1073/pnas.1813164116 Nalepka, P., Silva, P. L., Kallen, R. W., Shockley, K., Chemero, A., Saltzman, E., & Richardson, M. J. (2021). Task dynamics defne the contextual emergence

212

Vicente Raja and Matthieu M. de Wit

of human corralling behaviors. PLoS One, 16(11), e0260046. https://doi. org/10.1371/journal.pone.0260046 Palatinus, Z., Dixon, J. A., & Kelty-Stephen, D. G. (2013). Fractal fuctuations in quiet standing predict the use of mechanical information for haptic perception. Annals of Biomedical Engineering, 41(8), 1625–1634. https://doi.org/10.1007/ s10439-012-0706-1 Patil, G., Nalepka, P., Kallen, R. W., & Richardson, M. J. (2020). Hopf bifurcations in complex multiagent activity: The signature of discrete to rhythmic behavioral transitions. Brain Sciences, 10(8), 536. https://doi.org/10.3390/ brainsci10080536 Peng, C.-K., Buldyrev, S. V, Havlin, S., Simons, M., Stanley, H. E., & Goldberger, A. L. (1994). Mosaic organization of DNA nucleotides. Physical Review E, 49(2), 1685–1689. https://doi.org/10.1103/PhysRevE.49.1685 Pezzulo, G., & Cisek, P. (2016). Navigating the afordance landscape: Feedback control as a process model of behavior and cognition. Trends in Cognitive Sciences, 20(6), 414–424. https://doi.org/10.1016/j.tics.2016.03.013 Pinti, P., Tachtsidis, I., Hamilton, A., Hirsch, J., Aichelburg, C., Gilbert, S., & Burgess, P. W. (2020). The present and future use of functional Near-Infrared Spectroscopy (fNIRS) for cognitive neuroscience. Annals of the New York Academy of Sciences, 1464(1), 5–29. https://doi.org/10.1111/nyas.13948 Proftt, D. R., & Linkenauger, S. A. (2013). Perception viewed as a phenotypic expression. In W. Prinz, M. Beisert, & A. Herwig (Eds.), Action Science: Foundations of an Emerging Discipline (Vol. 171). MIT Press. Raja, V. (2018). A theory of resonance: Towards an ecological cognitive architecture. Minds and Machines, 28(1), 29–51. https://doi.org/10.1007/s11023-017-9431-8 Raja, V. (2020). Embodiment and cognitive neuroscience: The forgotten tales. Phenomenology and the Cognitive Sciences. https://doi.org/10.1007/ s11097-020-09711-0 Raja, V. (2022). Embodiment and cognitive neuroscience: The forgotten tales. Phenomenology and the Cognitive Sciences, 21(3), 603–623. https://doi. org/10.1007/s11097-020-09711-0 Raja, V., & Anderson, M. L. (2021). Behavior considered as an enabling constraint. In F. Calzavarini & M. Viola (Eds.), Neural Mechanisms: New Challenges in the Philosophy of Neuroscience (pp. 209–232). Springer International Publishing. https://doi.org/10.1007/978-3-030-54092-0_10 Sayood, K. (2018). Information theory and cognition: A Review. Entropy, 20(9), 706. https://doi.org/10.3390/e20090706 Segundo-Ortin, M., Heras-Escribano, M., & Raja, V. (2019). Ecological psychology is radical enough: A reply to radical enactivists. Philosophical Psychology, 32(7), 1001–1023. https://doi.org/10.1080/09515089.2019.1668238 Shaw, R. E., & Cutting, J. E. (1980). Clues from an ecological theory of event perception. In U. Bellugi & M. Studdert-Kennedy (Eds.), Signed and Spoken Language: Biological Constraints on Linguistic Form (pp. 57–84). Verlag Chemie GmbH. Stephen, D. G., Arzamarski, R., & Michaels, C. F. (2010). The role of fractality in perceptual learning: Exploration in dynamic touch. Journal of Experimental

Afordance Switching in Brain-Body-Environment Systems

213

Psychology: Human Perception and Performance, 36(5), 1161–1173. https:// doi.org/10.1037/a0019219 Stephen, D. G., Boncoddo, R. A., Magnuson, J. S., & Dixon, J. A. (2009). The dynamics of insight: Mathematical discovery as a phase transition. Memory & Cognition, 37(8), 1132–1149. https://doi.org/10.3758/MC.37.8.1132 Stofregen, T. A. (2000). Afordances and events: Theory and research. Ecological Psychology, 12(1), 1–28. https://doi.org/10.1207/S15326969ECO1201_1 Stofregen, T. A. (2003). Afordances as properties of the animal-environment system. Ecological Psychology, 15(2), 115–134. https://doi.org/10.1207/ S15326969ECO1502_2 Turvey, M. T. (1992). Afordances and prospective control: An outline of the ontology. Ecological Psychology, 4(3), 173–187. https://doi.org/10.1207/ s15326969eco0403_3 Turvey, M. T. (2018). Lectures on Perception: An Ecological Perspective. Routledge. Turvey, M. T., Shaw, R. E., Reed, E. S., & Mace, W. M. (1981). Ecological laws of perceiving and acting: In reply to Fodor and Pylyshyn (1981). Cognition, 9(3), 237–304. van der Weel, F. R., Agyei, S. B., & van der Meer, A. L. H. (2019). Infants’ brain responses to looming danger: Degeneracy of neural connectivity patterns. Ecological Psychology, 31(3), 182–197. https://doi.org/10.1080/10407413.2019.1 615210 Van Orden, G. C., Kloos, H., & Wallot, S. (2011). Living in the pink: Intentionality, wellbeing, and complexity. In C. Hooker (Ed.), Handbook of the Philosophy of Science (Vol. 10, pp.  629–672). Elsevier. https://doi.org/10.1016/ B978-0-444-52076-0.50022-5 Wallot, S., Lee, J. T., & Kelty-Stephen, D. G. (2019). Switching between reading tasks leads to phase-transitions in reading times in L1 and L2 readers. PLoS One, 14(2), e0211502. https://doi.org/10.1371/journal.pone.0211502 Withagen, R., Araújo, D., & de Poel, H. J. (2017). Inviting afordances and agency. New Ideas in Psychology, 45, 11–18. https://doi.org/10.1016/j. newideapsych.2016.12.002

12 WHAT IS NEXT FOR AFFORDANCES? TAKING BRAINS SERIOUSLY IN ORGANISM-ENVIRONMENT SYSTEMS Luis H. Favela

Ecological psychology is a research program and theoretical approach to perception and action originally developed by James J. Gibson in the midtwentieth century (e.g., Gibson, 1968). One of its roles in the history of psychology has been as an alternative to behaviorism and cognitivism. For some, behaviorism was far too focused on external overt action at the cost of giving any account of internal processing (Boden, 2006). For others, cognitivism was far too disembodied in treating cognition as an abstract informational process that is fundamentally computational and representational (cf. Neisser, 1967/2014), while ignoring the substantial roles that other features of the world play in cognition (e.g., body; Richardson et al., 2008). While initially a framework for perceptual psychology, Gibsonian ecological psychology has come to have a deep infuence on a wide range of felds, such as botany (Calvo, 2016), developmental psychology (Gibson, 1969; Thelen, 2000), human-machine interaction (Favela, 2019), neuroscience (Cisek, 2007), and virtual reality (Regia-Corte et al., 2013). What makes ecological psychology so appealing, especially as an alternative to behaviorism and cognitivism? Certainly, much of ecological psychology’s appeal stems from the theory of afordances. In brief, afordances are perceivable opportunities for behavior (Chemero, 2009; Fajen et al., 2009; Gibson, 1986/2015; HerasEscribano, 2019). To properly utilize afordances as part of the ecological psychology research program, one must understand the signifcance of both the organism and the environment in acts of perceiving and acting. Afordances are not just features of the external world (i.e., consistent with behaviorism) nor are they just features of the internal organism DOI: 10.4324/9781003396536-16

What Is NExT for Afordances?

215

(i.e., consistent with cognitivism). As Gibson put it, “An afordance is neither an objective property nor a subjective property; or it is both if you like. An afordance cuts across the dichotomy of subjective-objective. . . . It is equally a fact of the environment and a fact of behavior” (Gibson, 1986/2015). While that statement may sound cryptic, it makes a clear point: Explanations of afordances cannot prioritize or neglect the contributions of the organism or the environment. Embracing the perspective that the organism-environment system is the scale at which afordances must be explained has facilitated expansive knowledge in perceptual psychology. Early successful applications include catching objects (Oudejans et al., 1996), crossing gaps (Burton, 1992), passing through apertures (Warren & Whang, 1987), and sitting (Mark, 1987), to name a few. Sophisticated techniques have also been developed to empirically assess afordances. Afordances can be quantifed as a ratio of body-scaled and action-scaled measurements to measurements of relevant features of the environment. For example, the afordance of pass-through-able of apertures is defned as the apertureto-shoulder-width ratio (A/S; A: aperture width, S: shoulder width) that distinguishes the transition from passing through the aperture during typical locomotion to needing to turn shoulders to ft through (Favela et al., 2018; Warren & Whang, 1987). It should be clear that ecological psychology provides a scientifcally compelling research program for studying and explaining perception-action events at the scale of organism-environment systems. With all that said, ecological psychology has been haunted by a longstanding criticism; that is to say, the apparent absence of even a sketch of the contributions made by the brain to afordances.1 There should be no doubt that Gibsonian ecological psychology has made a forceful case for incorporating the multitude of roles the body, action, and environment play in causing and constituting perception. Indeed, seminal fgures in the cognitivist tradition have acknowledged Gibson and ecological psychology for these advances (e.g., Marr, 1982; Neisser, 1967/2014). Yet, Gibson’s specifc engagement with lessons from the neural sciences was mostly minimal beyond criticisms (e.g., “Neurophysiologists .  .  . are still under the infuence of dualism, however much they deny philosophizing;” Gibson, 1986/2015, p. 215) and vague metaphors (e.g., “the brain resonates;” Gibson, 1968, p.  260). This has led to some unfattering descriptions of Gibsonian ecological psychology: Gibson tends to leave the organism, if not empty, apparently stufed with foam rubber. —Pribram (1982, p. 370)

216

Luis H. Favela

J. J. Gibson’s theory of perception, for instance, seems to treat the whole visual system as a hunk of wonder tissue, for instance, resonating with marvelous sensitivity to a host of sophisticated “afordances.” —Dennett2 (1984, pp. 149–150) The message in these quotes is that if an account of the brain is not provided when explaining perception, then a researcher might as well believe that organisms are flled with “foam rubber.” Furthermore, when Gibsonians do talk about the brain, it seems to be in a way that conceives of it as a piece of “wonder tissue,” whose magical powers cannot be explained. While such quotes can be titillating, the fact is that the brain and nervous system have always been part of the ecological psychology theory of perception. One piece of evidence for this claim is that Gibsonians have always treated perception as occurring in perceptual systems. According to Gibson, visual perception is an action that occurs not just with the eyes but also with the movement of eyes on a face on a head on a neck on a torso on legs (Favela & Chemero, 2016; Gibson, 1968; Thomas et al., 2020). Along those lines, perceptual systems like humans have a brain and nervous system that partially constitute the organism part of organismenvironment systems. Be that as it may, critics may be on to something when they draw attention to the sparse work done by ecological psychologists to ofer explicit, experimentally supported connections between the brain and/or nervous system and afordances. The work that is out there tends to be centered on attempts to motivate potential congruence between ecological psychology and theories developed for other purposes, such as coordination dynamics (Fultot et  al., 2019), fractals (Raja, 2018), freeenergy principle (Bruineberg & Rietveld, 2019), Neural Darwinism (Reed, 1996), and neural reuse (Anderson, 2014). Research testing hypotheses directly about afordances and the nervous system are scant (e.g., van der Weel et al., 2022), with the majority arguably being about “afordances” in name only (e.g., Cisek, 2007), while not adhering to the core principles of ecological psychology (e.g., perception is direct). Here, I provide a path forward to understanding the brain’s specifc contribution to afordances: the NeuroEcological Nexus Theory (NExT). NExT hypothesizes that afordances emerge via systematic relationships between environmental (ecological) information and low-dimensional neural manifolds. This approach is motivated by recent neuroscience research demonstrating that neural population dynamics are preserved in low-dimensional manifolds within and across animals performing similar actions (e.g., Brennan & Proekt, 2019; Safaie et al., 2022). Accordingly, it is hypothesized that neural population dynamics map to particular afordance events with regularity. It is worth noting that NExT can be viewed

What Is NExT for Afordances?

217

as doing what was mentioned previously concerning prior work “merely” motivating potential congruence between ecological psychology and theories developed for other purposes, while not providing directly testable hypotheses and/or not observing ecological psychology’s core principles. In a sense, it is true that NExT appeals to methods and theories developed for other purposes. However, NExT is set apart by its providing an encompassing theory of the way afordance events are instantiated across organism-environment systems, with an explicit account of activity at the brain’s spatiotemporal scale. Moreover, it does so while maintaining the core principles of ecological psychology. In the next section, I present an overview of NExT. Then, I respond to a series of questions about afordances from the perspective of NExT, but more generally, from considerations of the relationship among afordances and brains. The reason for limiting the responses about the nature of afordances to the scope of NexT and brains is that I think there is much excellent work already out there that likely provides better answers than I could. However, there is limited discussion about afordances and brains, especially that does so in a manner that maintains ecological psychology’s core principles. NeuroEcological Nexus Theory (NexT)

The NeuroEcological Nexus Theory (NexT) is an investigative framework and a theory about the nature of minded organism-environment systems (Favela, 2024). Here, minded is applied broadly to be inclusive of cognition, goal-directed behavior, and intelligence. It aims to integrate ecological psychology and neuroscience. Accordingly, NExT is properly applied to the investigation of systems that have some form of a nervous system (Neuro-). Additionally, NExT must maintain the four primary principles at the core of ecological psychology (-Ecological; Favela, 2024; cf. Chemero, 2009; Gibson, 1986/2015): 1. Perception is direct: ecological information is rich enough to guide action. 2. Perception and action are continuous: perception guides action, and action enables perception. 3. Afordances: meaningful features of perception-action. 4. Organism-environment system: the proper spatiotemporal scale of inquiry for the previous three principles. NExT is committed to the position that minded organism-environment systems necessarily include organisms that are tightly coupled with or constituted by their environments. Here, the body (i.e., embodiment) and the

218

Luis H. Favela

world (i.e., embeddedness) are environments. The body’s contributions stem from combinations of both its proportions (e.g., height and weight) and its capabilities (e.g., postural sway and stride length). The world’s contributions stem from ecological information, which are “those patterns that uniquely specify properties of the world” (Gibson, 1986/2015, p. 177; Richardson et al., 2008, p. 67). Ecological information refers to the distributions of energy that surround an organism. They are higher-order properties in that they are necessarily spatiotemporal in nature, which means they exist in space over time (e.g., optic fow; Chemero, 2009; Fajen et  al., 2009; Gibson, 1968). That is the NeuroEcological part of NExT. The Nexus part refers to the idea that NExT is a “nexus” of convergence for ecological psychology and neuroscience. “Nexus” also refers to the primary causal and/or constitutive links or junctions that connect the various spatiotemporal scales of a minded organism-environment system. When attempting to integrate ecological psychology and neuroscience, the most signifcant nexus is that which connects brains, bodies, and environments. The afordance of pass-through-able provides an illustrative example of the application of NExT. NExT and the Afordance of Pass-through-able

The afordance of pass-through-able is the opportunity for an organism to move through an aperture. Take the case of a mouse foraging inside the walls of a house. It sees a light shining through a hole in the wall. As the mouse approaches the hole, it slows down, moves its head side-to-side in order to sense if the hole is wide enough to walk through. Since the hole is wide enough, it afords passing through for the mouse (Figure 12.1a). What is happening during this afordance event at relevant spatiotemporal scales of activity? First, to explain this afordance event (principle 3), NExT hypothesizes that the mouse-hole-in-wall system is the privileged spatiotemporal scale of description to understand the perceived and acted upon situation (principle 4). Consequently, ofering an explanation of pass-through-able for the mouse will be incomplete without accounting for causal and constitutive features of both mouse and environment. Second, neural spatiotemporal elements must be part of the explanation in order to include in NExT what is already in neuroscience-only approaches. Neural population dynamics are hypothesized as generating the relevant states for afordance events. Third, the neural population dynamics that contribute to the causes and constitution of the afordance of pass-through-able are principally low dimensional; specifcally, via the concept of manifold theory from topology. Topology is the mathematical study of the properties of objects that

219

The NeuroEcological Nexus Theory (NExT) applied to the afordance of pass-through-able. (a) The afordance pass-through-able is exhibited when a mouse walks through a hole and involves features of the environment (e.g., hole in wall), organism (e.g., shoulder width), and brain (e.g., sensorimotor neural populations). (b) Each row depicts a potential data source that contributes to a NExT-based explanation. Top: Time series from multielectrode recording of relevant neural populations. Middle: Motion tracking markers on mouse body and trace data. Bottom: Optic fow as example of relevant ecological information. (c) Each row depicts a phase space plot with state space generated from low-dimensional principal structure identifed from high-dimensional data recorded from sources such as those depicted in column (b). ((a) The author generated this image in part with DALL-E, a multimodal implementation of GPT-3, which is OpenAI’s large-scale language-generation model. Upon generating the image, the author reviewed, edited, and revised the image to their own liking and takes ultimate responsibility for the content of this publication. https://openai.com/product/dall-e; (b) Modifed and reprinted with permission from Niedringhaus et al. (2015), CC BY 4.0; Huang et al. (2021), CC BY 4.0; Matthis et al. (2022), CC BY 4.0.)

What Is NExT for Afordances?

FIGURE 12.1

220

Luis H. Favela

are maintained despite changing the object’s shape (e.g., twisting), without compromising the object’s integrity (e.g., cutting; Weisstein, 2022). In topology, a manifold is a mathematical object that looks locally like Euclidean space but globally may have a more complicated structure. Shapes can be classifed via their topological dimension, or n-dimensionality, for example, points: 0-dimension, curves: 1-dimension, surfaces: 2-dimension, and solids: 3-dimension (Edgar & Edgar, 2008). Classic examples of equal topological spaces are the cofee mug and donut, or torus. These shapes appear quite diferent (i.e., mug has a handle; donut has a hole that goes through). But they are equal with reference to their surface properties, such that they can be reshaped into each other without compromising their continuous integrity (e.g., without cutting them). This equivalence is due to their having the same topological dimension. Manifold theory is a branch of mathematics that studies the local and global properties of manifolds, such as curvature and diferentiability (Edgar & Edgar, 2008; Tu, 2011). The manifold hypothesis is the claim that very high dimensional datasets have much lower dimensional manifolds that capture their principal structure (Feferman et al., 2016). Here, “very high dimensional datasets” can be understood as referring to any large amount of recorded data. In this way, the manifold hypothesis can be understood as a form of dimensionality reduction, a data processing strategy for decreasing the number of a dataset’s features without losing valuable information. The neural manifold hypothesis claims that very high dimensional datasets—specifcally, in the form of neural population dynamics (Figure 12.1b, top)—have much lower dimensional manifolds that capture their principal structure (Figure 12.1c, top)—that generate specifc behaviors (i.e., neural modes; Chaudhuri et  al., 2019; Gallego et  al., 2017). Even though the activity of large numbers of neurons during any given task may seem to indicate high degrees of freedom, the subspace of relevant activity actually spans only a few variables. Moreover, those variables are interpretable via a limited number of manifold classes. In this way, the neural manifold hypothesis is a method for identifying neural modes that cause and constitute various behavioral and cognitive activities. While the mouse is deciding if the hole afords pass-through-ability, its body is performing a reciprocal perception-action activity. Specifcally, the mouse is detecting ecological information about the hole (e.g., optic fow), which informs its actions (i.e., head movement), which in turn alters the structure of the ecological information it is detecting, and so on (principle 2). What is happening at neural spatiotemporal scales is richly accounted for by the neural manifold hypothesis. If the research aim is to identify the neural populations most relevant to the afordance pass-through-able, then

What Is NExT for Afordances?

221

the neural population that causes and constitutes head movement will be one of the focuses because it plays a signifcant role in the mouse’s engagement with ecological information that guides the most applicable action. The activity of these particular neurons will be dependent on the mouse’s orientation in Euclidean space (Figure 12.2a). The neural manifold hypothesis claims that the principal structure of these population dynamics will be expressed by particular topological dimensions, such that head direction is one variable (i.e., 1-dimension topology) that fts on a low dimensional spline within the high dimensional cloud of data (Figure 12.2b). Prior research (e.g., Chaudhuri et  al., 2019; Gallego et  al., 2017) has demonstrated location equivalence in Euclidean space of the mouse’s real body (Figure 12.2a), location of activity along the low-dimensional spline (Figure 12.2b), and the abstracted torus (Figure 12.2c). Though there is an emphasis here on neural population activity, privileging the organism-environment system (principle 4) and the mutuality of perception-action (principle 2) are maintained. A crucial reason for this is that the activity of neural population dynamics does not hold a unidirectional causal relationship with the body or environment. The state of the body (i.e., head location; Figure 12.2a) and the environment (i.e., aperture edge; Figure 12.1a) inform and constrain the state of the neural activity. If the head cannot move any further to the left because it hits up against the hole’s edge, then the neural population will also not continue activity in

FIGURE 12.2

Neural manifold hypothesis methodology. (a) During the task of rotating the mouse’s head direction in Euclidean space (indicated by diamond shape-tipped arrow), activity from the relevant neural population is recorded. (b) Dimensionality reduction techniques are applied to isolate the neural activity, and then further analyzed to identify the latent variables via a generated spline. (c) Movement in Euclidean space and along the spline are equivalent to the location on the abstracted torus. ((a) Public domain; (b) Modifed and reprinted with permission from Xia et al. (2021), CC BY 4.0; (c) Public domain.)

222

Luis H. Favela

that direction, as exhibited by its location on the low dimensional manifold (Figure 12.2b,c). So too does the state of the neural activity inform and constrain the state of the body and environment, such that where the direction of the head points will provide the perspective from which the body (e.g., eyes) will detect ecological information (e.g., light refecting from the surface edges of the hole). In that way, direct perception (principle 1) is also maintained due to the fact that neural activity is informed and constrained by the direct engagement with ecological information. Fourth, the afordance of pass-through-able for the mouse fundamentally emerges at low-dimensional scales of organism (neural, body)-environment activity. Afordances do not exist alone in brains, bodies, or environments. Afordances are events that spread across organism-environment systems (Figure 12.1a). For the hole in the wall to aford pass-through-ability, the integration and coordination of spatiotemporal scales must occur. The principal structure of low-dimensional manifolds in neural populations (Figure 12.1b,c, top rows; Figure 12.2b) exists in a reciprocal relationship that both causes and constitutes the body’s structure and function, while also being caused and constituted by the body’s activities. The principal structure of low-dimensional confgurations of the body (Figure 12.1b,c, middle rows) exists in a reciprocal relationship that both causes and constitutes its relationship with the environment, while also being caused and constituted by the environment’s features. The ecological information of the environment (e.g., optical fow; Figure 12.1b, bottom row) exists in a reciprocal relationship that both causes and constitutes its relationship with the organism, while also being caused and constituted by the organism’s features. With this overview of the application of NExT to the afordance of pass-through-able complete, I now turn to a series of questions about afordances. What Do We Understand by the Term Afordance?

At its most basic, afordances are perceivable opportunities for behavior (e.g., Chemero, 2009; Fajen et al., 2009; Gibson, 1986/2015; HerasEscribano, 2019). They are a relational feature of organism-environment systems that emerge during perception-action activities. The afordance of pass-through-able exists at the spatiotemporal scale of, for example, a mouse’s body and the opening it can move through (Figure 12.1a). Hence, pass-through-ability emerges when a body’s proportions and its capabilities facilitate the organism’s capability to detect ecological information in such forms as optic arrays. The spatial and temporal structure of ecological information specifes afordances. It is, in part, because the mouse can see the shape of the opening by way of patterns in the optic array refecting

What Is NExT for Afordances?

223

from its surfaces that it knows it can move through. But it is also, in part, because of proprioception that the mouse knows its body proportions and movements will allow for passing through. It is in that way that afordances are equally a fact of the environment and a fact of behavior that cut across the subjective-objective dichotomy (Gibson, 1986/2015, p. 121). There is no contradiction in treating afordances as relational properties that emerge at the organism-environment scale and acknowledging that nervous systems play signifcant roles in what causes and constitutes afordances. This is true at least for mammals, which have sensorimotor systems that are partially constituted by a brain, spinal cord, and a system of other nervous tissue. Afordances have been investigated and explained at relatively larger spatiotemporal scales (e.g., body and limbs). Still, a more feshed-out story can be provided by complementary research aimed at smaller scale contributors like the nervous system. NExT provides a way to do that without compromising the critical role the theory of afordances plays in ecological psychology. What Role Does Afordance Play in Perception? Is It the Entity Organisms Perceive or the Means Through Which Organisms Perceive? Are Afordances the Only Perceptual Dependent Variables?

If we follow the line of thought through and take Gibson’s point seriously that afordances are equally a fact of the environment and a fact of behavior that cut across the subjective-objective dichotomy, then afordances do not play a role in perception—afordances are perception. An afordance is a perception-action event that encompasses the organism-environment system (Figure 12.1a). In that way, afordances are not entities to be perceived or means by which to perceive. The Gibsonian sense of afordance is more akin to a description of a system than a feature of the world. Thus, afordances are not means of perception. As a consequence, the perceptual system (cf. Gibson, 1968) is not the system that perceives afordances; it is the system that is the afordance event. As a result, the perceptual system is the brain, body, and environment, and an afordance is a description of the occurrence of a successful activity emerging during the perception-action loop. For the kinds of creatures that humans and mice are, brains and nervous systems play crucial roles in the causes and constitution of afordance events. NExT is an appropriate investigative framework for such organisms. However, the Gibsonian theory of afordances is not applicable only to mammals. It is a theory of the nature of perception, full stop. Any creature that perceives and acts in a world does so via the theory of afordances: their opportunities to act will

224

Luis H. Favela

be defned by their body proportions and capabilities and the world that shaped that body (e.g., evolution) and the ecological information that can be detected. Insofar as perception is understood along the lines of the Gibsonian theory of afordances, then afordances are the only perceptual dependent variable. This is true if all perception is for action and all action for perception. That seems to be the case when it comes to fundamental ways of being in the world; for example, “the four F’s: feeding, feeing, fghting, and reproduction” (Churchland, 1994, p.  31). However, it is unclear if the Gibsonian theory of afordances has room for other activities exhibited by organism-environment systems. For example, Gibson notes that “human awareness of clock-time, socialized time, is another matter” (Gibson, 1986/2015, p. 8) but later says in a discussion of social interaction that it “does not even begin to do justice to the power of the notion of afordances in social psychology” (Gibson, 1986/2015, p. 36). It is an open question whether other forms of perception fall into the afordance category—for example, moral (Jayawickreme & Chemero, 2008), religious (Barrett, 2014; Favela & Chemero, 2014), and social (Valenti & Gold, 1991)—or if they are distinct perceptual dependent variables. For humans, if afordances are perception, and if moral, religious, social, etc., experiences occur in part in brains, then a case can be made within NExT that afordances remain the only perceptual dependent variables. Why Is Afordance Any Better Than Stimulus? What Does a Theory of Afordance Suggest That Stimuli Cannot; How Has It Moved the Needle Past Gibson 1960’s Recognition of How Little We Can Defne Stimuli?

It is common to fnd in the ecological psychology literature discussion of John Dewey’s (1896) critique of the refex arc (e.g., Heft, 2001; HerasEscribano, 2019; Withagen, 2022). Dewey argued, “[T]he refex arc idea . . . is defective in that it assumes sensory stimulus and motor response as distinct psychical existences” (1896, p. 360). Dewey goes on to argue that psychological states involve “sensori-motor coordination” (1896, p. 361). Along those lines, Gibson’s theory of afordances is better than stimulus because it does not sufer the same dualistic defects pointed out by Dewey and has sensorimotor coordination at its core. Gibson’s theory of afordances imbues perceptual systems with meaning in a way that both behaviorism and cognitivism do not. The world of both the behaviorist and cognitivist is a meaningless and mechanical one, with organisms thrown into it (cf. Dasein, Heidegger, 1927/1962). To fnd

What Is NExT for Afordances?

225

meaning, the organism must draw it either from mental-less, mechanical associations with stimulus (behaviorism) or from a hyper-mental, mechanical process of transforming stimulus (cognitivism). It is funny that while the mental is absent from one and overrepresented in the other, both are dualistic in treating the organism as a passive and distinct entity from the environment. Afordances are meaningful precisely because there is no dualism of organism-environment or perception-action. Consequently, afordances are directly perceivable meaningful opportunities for behavior. As Gibson put it, Perhaps the composition and layout of surfaces constitute what they aford. If so, to perceive them is to perceive what they aford. This is a radical hypothesis, for it implies that the “values” and “meanings” of things in the environment can be directly perceived. Moreover, it would explain the sense in which values and meanings are external to the perceiver. —J. J. Gibson (1986/2015, p. 119; emphasis in the original) Gibson is drawing attention to a crucial consequence of his non-dualist conception of organism-environment systems exhibiting lawful perceptionaction properties. If ecological information specifes afordances, then ecological information also underlies what is meaningful to a perceiver. In other words, ecological information, such as patterns in ambient light, will specify in lawful ways what an organism can and cannot do. An aperture that is wide enough for a mouse to pass through is meaningful in a way that one that is too narrow is not. It follows then that afordances are a source of meaning for organisms because they are those features of the world they can do things with. Moreover, because there is no dualism separating organism-environment and because ecological information can be directly perceived, the brain is free from the burden of representing the environment and imbuing it with meaning (cf. cognitivism). By means of the mouse and pass-throughable example, the NExT framework demonstrates how the brain plays an essential role in mammalian perception-action but not to the point of “hyper-neural-izing” the phenomenon. For example, the head’s location (Figure  12.2a) and the hole’s edge (Figure 12.1a) inform and constrain neural activity, such that if the head cannot move because it hits the hole’s side, then the neural population will also not continue activity in that direction, as exhibited by its location on the low dimensional manifold (Figure 12.2b,c). Thus, the Gibsonian theory of afordances is maintained, and an account of brain activity is given without appeal to dualistic notions of stimulus.

226

Luis H. Favela

What Is the Connection Between Afordances, Behavioral Scale, and Intention? How Distant or Reluctant Can a PerceivingActing System Be Before Afordances Come Into Play? Do Afordances Exert Their Role Only When the System Is Close or Willing Enough?

Here, “behavioral scale” is understood in terms of the spatiotemporal scale of bodies acting in the world. Accordingly, and in line with the above discussion, the behavioral scale is one of the multiscale causal and constitutive contributors to afordance events, along with the neural scale and environmental scale (Figure 12.1b,c). Thus, afordances cannot be isolated as existing solely in a brain, body, or environment. They emerge at the perception-action scale of activity in organism-environment systems Accordingly, the afordance event of a mouse passing through a hole emerges from systematic relationships among brain, body, and environment. The connection between afordances and intention is not as challenging as it may seem. In the current context, intentions are goals and goals have success and failure conditions. The foraging mouse has the intention of obtaining food and nested within that intention is moving through the hole. If food is not found and/or the hole is not pass-through-able, then there is failure: the goal has not been met and the intention has not been satisfed. Depending on the nature of the neural system (e.g., memory capacity), body (e.g., nutrition), and environment (e.g., distractions), intentions can vary in their immediacy and reappearance. An aperture that was challenging to pass through one day may, due to memory capacity, remain a goal for another day for a monkey but not a dog, for a mouse but not a fruit fy, and so on. Even so, neural capacities for memory are not the only factor, as low nutrition could motivate or hinder a monkey’s goal or bad weather could halt an intention.3 How Do Systems Engage With Afordances as They Move Among Tasks and Intentions?

As argued via the NExT framework, afordance events emerge from the coordination of activity across the multiple spatiotemporal scales that cause and constitute organism-environment systems. One way to understand the organization of these multiscale activities is as nested structures (e.g., Gibson, 1986/2015, pp. 94, 103, 231; see also Favela, 2023; Heft, 2001; Mangalam et al., 2020; Wagman & Miller, 2003). Nested spatial structures include a body with organ systems comprised of cells, and so on. Nested temporal structures include the simultaneous timescales of body movement, neural network communication, single-neuron fring, synaptic chemical transmission, etc. These nested spatial and temporal structures coordinate across and

What Is NExT for Afordances?

227

within scales. A rich source of evidence for this is found in the vast literature demonstrating that the temporal characteristics of a wide range of brain and body activities are fractal in structure (e.g., Di Ieva, 2016; Holden et  al., 2013). In particular are those studies showing that multiple timescales enable and constrain each other during a variety of brain activities and body performances (for review, see Van Orden et al., 2012)? Along these lines, intentions and tasks are properly understood as being caused and constituted by nested multiscale structures. The mouse and afordance of pass-through-able provide a clear example of intentions (i.e., goals) being nested within each other. The mouse’s intention to pass through the hole is nested within its overarching intention of fnding food. As this event is happening, the timescales of the mouse’s neural activity are nested within each other (e.g., single-neuron dynamics within population dynamics), which are then nested within the body’s movement, which is nested within the environment. Concluding Remarks

Gibsonian ecological psychology has provided a rich alternative to behaviorism and cognitivism, and has inspired many felds outside of psychology as well. Much of its appeal and staying power stems from the theory of afordances. While ecological psychology provides a theoretically rich and empirically supported investigative framework, especially for perceptionaction phenomena, it is not without criticism. A common source of condemnation is the purported absence of a serious account of the brain’s role in perception-action. In the current work, I have provided a path forward to understanding the brain’s contribution to afordances: the NeuroEcological Nexus Theory (NExT). NExT hypothesizes that afordances emerge via systematic relationships between environmental (ecological) information and low-dimensional neural manifolds. Moreover, NExT ofers an account of the brain’s role without neglecting the four primary principles at the core of ecological psychology (e.g., perception is direct). Taken together, the theory of afordances successfully appealed to for decades by Gibsonians is complemented by the neural manifold hypothesis. If ecological psychologists accept NExT, then they will no longer be accused of believing in creatures flled with foam rubber and wonder tissue. Notes 1. The word haunted here is, admittedly, a bit dramatic. The fact is that James and Eleanor Gibson, as well as many (most?) of the Gibsonian tradition, likely do not think that is a criticism at all. As Eleanor Gibson wrote: Finally, what are we to think about the scene in experimental psychology today? I am not entirely happy about it, as my readers may detect. Some

228

Luis H. Favela

psychology departments have changed their names to “Department of Cognitive Science” or even “Department of Cognitive Neuroscience.” Prizes are going to psychologists who study neuroscience in relation to thinking or consciousness. There is certainly nothing wrong with studying thinking and neural processes, but I am not sure that the combination is going anywhere and I am sorry to see fashion desert the study of behavior, which I still consider the important subject matter for a psychologist. —E. J. Gibson (2002, p. 122) 2. In more recent work, Dennett (2017) appeals to the concept of afordances a great deal and seems to have warmed up to Gibson—but only ever so slightly. 3. This is the case for one of my dogs: no matter how much he has to go potty, if it is raining, then he will hold it to the bitter end—intention for relief or not!

Reference List Anderson, M. L. (2014). After Phrenology: Neural Reuse and the Interactive Brain. MIT Press. Barrett, N. F. (2014). The perception of religious meaning and value: An ecological approach. Religion, Brain & Behavior, 4(2), 127–146. https://doi.org/10.1080/ 2153599X.2013.816339 Boden, M. (2006). Mind as Machine: A History of Cognitive Science (Vol. 1–2). Oxford University Press. Brennan, C., & Proekt, A. (2019). A quantitative model of conserved macroscopic dynamics predicts future motor commands. ELife, 8, e46814. https://doi. org/10.7554/eLife.46814 Bruineberg, J., & Rietveld, E. (2019). What’s inside your head once you’ve fgured out what your head’s inside of. Ecological Psychology, 31(3), 198–217. https:// doi.org/10.1080/10407413.2019.1615204 Burton, G. (1992). Nonvisual judgment of the crossability of path gaps. Journal of Experimental Psychology: Human Perception and Performance, 18(3), 698– 713. https://doi.org/10.1037/0096-1523.18.3.698 Calvo, P. (2016). The philosophy of plant neurobiology: A manifesto. Synthese, 193(5), 1323–1343. https://doi.org/10.1007/s11229-016-1040-1 Chaudhuri, R., Gerçek, B., Pandey, B., Peyrache, A., & Fiete, I. (2019). The intrinsic attractor manifold and population dynamics of a canonical cognitive circuit across waking and sleep. Nature Neuroscience, 22(9), 1512–1520. https://doi. org/10.1038/s41593-019-0460-x Chemero, A. (2009). Radical Embodied Cognitive Science. MIT Press. Churchland, P. S. (1994). Can neurobiology teach us anything about consciousness? Proceedings and Addresses of the American Philosophical Association, 67(4), 23–40. https://doi.org/10.2307/3130741 Cisek, P. (2007). Cortical mechanisms of action selection: The afordance competition hypothesis. Philosophical Transactions of the Royal Society B: Biological Sciences, 362(1485), 1585–1599. https://doi.org/10.1098/rstb.2007.2054 Dennett, D. C. (1984). Cognitive wheels: The frame problem of AI. In C. Hookway (Ed.), Minds, Machines and Evolution (pp. 129–151). Cambridge University Press.

What Is NExT for Afordances?

229

Dennett, D. C. (2017). From Bacteria to Bach and Back: The Evolution of Minds. WW Norton & Company. Dewey, J. (1896). The refex arc concept in psychology. Psychological Review, 3(4), 357–370. https://doi.org/10.1037/h0070405 Di Ieva, A. (2016). The Fractal Geometry of the Brain. Springer. https://doi. org/10.1007/978-1-4939-3995-4 Edgar, G. A., & Edgar, G. A. (2008). Measure, Topology, and Fractal Geometry (Vol. 2). Springer. Fajen, B. R., Riley, M. A., & Turvey, M. T. (2009). Information, afordances, and the control of action in sport. International Journal of Sport Psychology, 40(1), 79–107. Favela, L. H. (2019). Soft-assembled human—Machine perceptual systems. Adaptive Behavior, 27(6), 423–437. https://doi.org/10.1177/1059712319847129 Favela, L. H. (2023). Nested dynamical modeling of neural systems: A strategy and some consequences. Journal of Multiscale Neuroscience, 2(1), 240–250. https:// doi.org/10.56280/1567939485 Favela, L. H. (2024). The Ecological Brain: Unifying the Sciences of Brain, Body, and Environment. Routledge. Favela, L. H., & Chemero, A. (2014). The value of afordances: Commentary on Barrett, the perception of religious meaning and value: An ecological approach. Religion, Brain & Behavior, 4(2), 147–149. https://doi.org/10.1080/21535 99X.2013.816343 Favela, L. H., & Chemero, A. (2016). The animal-environment system. In M. H. Fischer & Y. Coello (Eds.), Foundations of Embodied Cognition: Volume 1: Perceptual and Emotional Embodiment (pp. 59–74). Routledge. Favela, L. H., Riley, M. A., Shockley, K., & Chemero, A. (2018). Perceptually equivalent judgments made visually and via haptic sensory-substitution devices. Ecological Psychology, 30(4), 326–345. https://doi.org/10.1080/10407413.2018.1473712 Feferman, C., Mitter, S., & Narayanan, H. (2016). Testing the manifold hypothesis. Journal of the American Mathematical Society, 29(4), 983–1049. Fultot, M., Adrian Frazier, P., Turvey, M. T., & Carello, C. (2019). What are nervous systems for? Ecological Psychology, 31(3), 218–234. https://doi.org/10.108 0/10407413.2019.1615205 Gallego, J. A., Perich, M. G., Miller, L. E., & Solla, S. A. (2017). Neural manifolds for the control of movement. Neuron, 94(5), 978–984. https://doi.org/10.1016/j. neuron.2017.05.025 Gibson, E. J. (1969). Principles of Perceptual Learning and Development. Appleton-Century-Crofts. Gibson, E. J. (2002). Perceiving the Afordances: A Portrait of Two Psychologists. Lawrence Erlbaum Associates. Gibson, J. J. (1968). The Senses Considered as Perceptual Systems. Houghton Mifin. Gibson, J. J. (1986/2015). The Ecological Approach to Visual Perception. Psychology Press. Heft, H. (2001). Ecological Psychology in Context: James Gibson, Roger Barker, and the Legacy of William James’s Radical Empiricism. Psychology Press.

230

Luis H. Favela

Heidegger, M. (1927/1962). Being and Time. Harper & Row. Heras-Escribano, M. (2019). The Philosophy of Afordances. Palgrave Macmillan. Holden, J., Riley, M., Gao, J., & Torre, K. (2013). Fractal analyses: Statistical and methodological innovations and best practices. Frontiers in Physiology, 4, 97. https://doi.org/10.3389/fphys.2013.00097 Huang, K., Han, Y., Chen, K., Pan, H., Zhao, G., Yi, W., Li, X., Liu, S., Wei, P., & Wang, L. (2021). A hierarchical 3D-motion learning framework for animal spontaneous behavior mapping. Nature Communications, 12(1), 2784. https:// doi.org/10.1038/s41467-021-22970-y Jayawickreme, E., & Chemero, A. (2008). Ecological moral realism: An alternative theoretical framework for studying moral psychology. Review of General Psychology, 12(2), 118–126. https://doi.org/10.1037/1089-2680.12.2.118 Mangalam, M., Chen, R., McHugh, T. R., Singh, T., & Kelty-Stephen, D. G. (2020). Bodywide fuctuations support manual exploration: Fractal fuctuations in posture predict perception of heaviness and length via efortful touch by the hand. Human Movement Science, 69, 102543. https://doi.org/10.1016/j. humov.2019.102543 Mark, L. S. (1987). Eyeheight-scaled information about afordances: A study of sitting and stair climbing. Journal of Experimental Psychology: Human Perception and Performance, 13(3), 361–370. https://doi.org/10.1037/0096-1523.13.3.361 Marr, D. (1982/2010). Vision: A Computational Investigation Into the Human Representation and Processing of Visual Information. MIT Press. Matthis, J. S., Muller, K. S., Bonnen, K. L., & Hayhoe, M. M. (2022). Retinal optic fow during natural locomotion. PLoS Computational Biology, 18(2), e1009575. https://doi.org/10.1371/journal.pcbi.1009575 Neisser, U. (1967/2014). Cognitive Psychology: Classic Edition. Psychology Press. Niedringhaus, M., Chen, X., & Dzakpasu, R. (2015). Long-term dynamical constraints on pharmacologically evoked potentiation imply activity conservation within in vitro hippocampal networks. PLoS One, 10(6), e0129324. https://doi. org/10.1371/journal.pone.0129324 Oudejans, R. R. D., Michaels, C. F., Bakker, F. C., & Dolné, M. A. (1996). The relevance of action in perceiving afordances: Perception of catchableness of fy balls. Journal of Experimental Psychology: Human Perception and Performance, 22, 879–891. https://doi.org/10.1037/0096-1523.22.4.879 Pribram, K. H. (1982). Refections on the place of brain in the ecology of mind. In R. R. Hofman & D. S. Palermo (Eds.), Cognition and the Symbolic Processes (Vol. 2, pp. 361–381). Erlbaum. Raja, V. (2018). A theory of resonance: Towards an ecological cognitive architecture. Minds and Machines, 28(1), 29–51. https://doi.org/10.1007/s11023-017-9431-8 Reed, E. S. (1996). Encountering the World: Toward an Ecological Psychology. Oxford University Press. Regia-Corte, T., Marchal, M., Cirio, G., & Lécuyer, A. (2013). Perceiving afordances in virtual reality: Infuence of person and environmental properties in perception of standing on virtual grounds. Virtual Reality, 17(1), 17–28. https:// doi.org/10.1007/s10055-012-0216-3 Richardson, M. J., Shockley, K., Fajen, B. R., Riley, M. A., & Turvey, M. T. (2008). Ecological psychology: Six principles for an embodied—Embedded approach

What Is NExT for Afordances?

231

to behavior. In P. Calvo & A. B. T.-H. of C. S. Gomila (Eds.), Handbook of Cognitive Science: An Embodied Approach (pp. 159–187). Elsevier. https://doi. org/10.1016/B978-0-08-046616-3.00009-8 Safaie, M., Chang, J. C., Park, J., Miller, L. E., Dudman, J. T., Perich, M. G., & Gallego, J. A. (2022). Preserved neural population dynamics across animals performing similar behaviour. BioRxiv, 2022.09.26.509498. https://doi. org/10.1101/2022.09.26.509498 Thelen, E. (2000). Motor development as foundation and future of developmental psychology. International Journal of Behavioral Development, 24(4), 385–397. https://doi.org/10.1080/016502500750037937 Thomas, B. J., Riley, M. A., & Wagman, J. B. (2020). Information and its detection: The consequences of Gibson’s theory of information pickup. In J. B. Wagman & J. J. C. Blau (Eds.), Perception as Information Detection: Refections on Gibson’s Ecological Approach to Visual Perception (pp. 237–252). Routledge. Tu, L. W. (2011). Manifolds. In L. W. Tu (Ed.), An Introduction to Manifolds (pp. 47–83). Springer New York. https://doi.org/10.1007/978-1-4419-7400-6_3 Valenti, S. S., & Gold, J. M. M. (1991). Social afordances and interaction I: Introduction. Ecological Psychology, 3(2), 77–98. https://doi.org/10.1207/s15326969 eco0302_2 van der Weel, F. R. (Ruud), Sokolovskis, I., Raja, V., & van der Meer, A. L. H. (2022). Neural aspects of prospective control through resonating taus in an interceptive timing task. Brain Sciences, 12(12), 1737. https://doi.org/10.3390/ brainsci12121737 Van Orden, G., Hollis, G., & Wallot, S. (2012). The blue-collar brain. Frontiers in Physiology, 3, 207. https://doi.org/10.3389/fphys.2012.00207 Wagman, J. B., & Miller, D. B. (2003). Nested reciprocities: The organism—Environment system in perception—Action and development. Developmental Psychobiology, 42(4), 317–334. https://doi.org/10.1002/dev.10114 Warren, W. H., & Whang, S. (1987). Visual guidance of walking through apertures: Body-scaled information for afordances. Journal of Experimental Psychology: Human Perception and Performance, 13(3), 371–383. https://doi. org/10.1037/0096-1523.13.3.371 Weisstein, E. W. (2022). Topology: MathWorld—A Wolfram Web Resource. https:// mathworld.wolfram.com/Topology.html Withagen, R. (2022). Afective Gibsonian Psychology. Routledge. Xia, J., Marks, T. D., Goard, M. J., & Wessel, R. (2021). Stable representation of a naturalistic movie emerges from episodic activity with gain variability. Nature Communications, 12(1), 5170. https://doi.org/10.1038/s41467-021-25437-2

13 AFFORDANCE AND TOOL USE A Neurocognitive Approach François Osiurak and Giovanni Federico

More than 40 years have passed since Gibson (1977) coined the term “afordance.” Since then, the term has gained huge popularity, becoming ubiquitous in the literature on motor control. The success has also been very great in the neurocognition community interested in tool use, which falls within the scope of the present section. The term “afordance” is so common that tools are considered as afording everything, from motor actions (e.g., a knife afords grasping) to mechanical actions (e.g., a knife afords cutting) or intentions (e.g., a knife afords feeding). As afordance has become the solution to most problems, it has inevitably lost its specifcity, meaning everything and its opposite. Thus, in the neurocognition literature on tool use, the term “afordance” can refer to either action opportunities, online motor control, or semantic knowledge, to either hand-centered or toolcentered processes, or to either cognitive processes supported by dorsal or ventral brain structures (for a review, see Osiurak et al., 2017). To a lesser extent, the confusion that can sometimes exist in what afordances refer to is also present in the ecological psychology literature, from which the concept had initially emerged. This confusion already existed in Gibson’s writings. Yet, if we wish that the concept of afordance continues to be useful in the literature on tool use, there is a need to specify what it is and what it is not. This section discusses a theoretical framework that fulflls this purpose, which is the three action-system model (3AS), a framework that we developed some years ago (Osiurak et  al., 2017; Fig. 1). This model is a neurocognitive framework and certainly sufers from some limitations raised by ecological psychology. Nevertheless, it is not fully at odds with DOI: 10.4324/9781003396536-17

Afordance and Tool Use

233

the ecological approach, with which it agrees on several aspects. The core assumption is that, in humans, three neurocognitive systems (i.e., motor control/dorso-dorsal system, technical reasoning/ventro-dorsal system, and semantic knowledge/ventral system; Figure 13.1) are in charge of processing three diferent kinds of physical relationships (i.e., afordances, mechanical actions, and contextual relationships, respectively). Note that the question of semantic knowledge-contextual relationships will not be addressed here for the sake of clarity, our focus being only on the distinction between motor control-afordances versus technical reasoning-mechanical actions. The physical relationships refer to the description of the action possibilities ofered by the environment—a perspective very close to the ecological psychology approach. By contrast, neurocognitive systems refer to the ability of a specifc individual to take advantage of these possibilities. In the next sections, we will present this framework in more detail. However, it is necessary to defne some related concepts.

FIGURE 13.1

The three-action system (3AS) model. The key assumption of the 3AS model is that, in humans, three neurocognitive systems (i.e., motor control/dorso-dorsal system, technical reasoning/ventrodorsal system, and semantic knowledge/ventral system) are dedicated to the processing of three diferent and specifc kinds of physical relationships (i.e., afordances, mechanical actions, and contextual relationships, respectively).

234

François Osiurak and Giovanni Federico

As mentioned earlier, the present section will focus on the role of afordance in tool use. By identifying a tool as any external, unattached object held during the use or manipulated prior to the user modifying the form, position, or condition of another object or organism (including the user), Shumaker et al. (2011) ofered an operational defnition of tool use.1 Tool use, they argued, diverges from tool making, which refers to the structural modifcations a user can generate to transform an object into a tool, and from construction, which refers to the assemblage of two or more objects that are linked to make a functional semipermanent thing that is not held during its “use.” However, Beck (1980) stressed that these defnitions are fully operational as they do not presuppose that each behavior (i.e., tool use, tool making, construction) is associated with specifc cognitive skills, notably at the conception step. In other words, there is no bijection between these behavioral patterns and the underlying cognitive processes: Tool use does not necessarily refect, at the conception step, the manifestation of a cognitive process A, tool making the manifestation of a cognitive process B, and construction the manifestation of a cognitive process C (Osiurak & Heinke, 2018). In this section, we will mainly focus on the neurocognition literature on tool use simply because this literature generally overlooks the study of tool making and construction instead of placing a heavy emphasis on the manipulation component. However, we will consider that this literature is instructive in understanding how individuals—notable humans—can modify their environment to solve physical problems. We do not deny that tool use poses, in some cases, additional requirements regarding the manipulation. In some cases, tool use involves a distalization mechanism in that, once a tool is grasped appropriately to be used, the hand is no longer the end-efector (Arbib et al., 2009; Frey, 2007; for discussion on this aspect, see Osiurak & Federico, 2021). Instead, the end-efector becomes the active part of the tool, which implies an attentional shift from the hand to the active part of the tool. Yet, the hand is still needed to be controlled during the use. This distalization mechanism is not necessary for all the cases of tool use (e.g., throwing actions), as defned by Shumaker et  al. (2011). For this reason, the distalization mechanism could only apply when tools are manipulated during the use, thus referring to what Fragaszy and Mangalam (2018; see also Mangalam, 2016; Mangalam & Fragaszy, 2016) called tooling in opposition to tooluse cases in which there is no need to control the degrees of freedom of the body-plus-system diferently from the body-only-system. To sum up, even if we focus on the neurocognition literature on tool use, we will not consider that the cognitive processes involved in the conception of tool-use actions difer from the cognitive processes involved in tool making and

Afordance and Tool Use

235

construction. Nevertheless, we will keep in mind that some tool-use cases, tooling behavior, pose additional requirements as evidenced by the distalization mechanism. What Do We Understand by the Term “Afordance”?

As initially introduced by Gibson (1977), the term “afordance” refers to the action possibilities ofered to an animal by the environment with reference to the animal’s action capabilities. Although this defnition has been subject to intense debate by ecological psychologists (e.g., Chemero, 2003; Jones, 2003; Michaels, 2003; Stofregen, 2003), all agree that afordances are not representations—a prerequisite for their approach. By contrast, the notion of afordance has considerably evolved in the neurocognition domain, hence becoming part of the representation. For instance, Tucker and Ellis (1998) proposed that “the actions aforded by a visual object are intrinsic to its representation” (p. 844). Other proposals have been made in the same vein, through the notion of physical versus learned afordance (Norman, 2002), afordance versus functional afordance (Young, 2006), or through the idea that pure physical (Symes et al., 2007) or “structurerelated” (Buxbaum & Kalénine, 2010) afordances can be automatically activated when an object is seen. As these proposals, the 3AS model is a neurocognitive framework. Yet, it difers in two respects from them. The frst is concerned with the idea that afordances can be automatically perceived. We will come back to the question of automaticity later. The second is that we posit that afordances must not be confounded with representations. As advocated by Gibson and more broadly by the ecological approach, the 3AS model assumes that afordances refer to action possibilities that are both subjective and objective. Afordances are subjective because they correspond to the description of the action possibilities for a specifc animal according to its specifc capabilities. The entire description of these action possibilities (i.e., the set of afordances) for a specifc individual corresponds to the niche of that specifc individual (Gibson, 1979). Thus, the environment does not ofer the same possibilities for a dog as for a bird. A branch can aford perch-ability for a bird but not for a dog, whereas a large dead trunk laying on the ground can aford jump-ability for a dog but not for a bird. This is also true for individuals of the same species: A stair can be climb-able for an adult but not for a baby. Even the same stairs cannot aford the same action possibilities for two adults, which depends on their capabilities (e.g., based on leg length; Warren, 1984). Afordances are also objective in that they refer to what the environment ofers to an individual, irrespective of its intention or state (for a similar

236

François Osiurak and Giovanni Federico

viewpoint, see Shaw et al., 1982; we will come back later to this aspect). Importantly, afordances refer to the possible actions and must not be confounded with the currently feasible actions (Declerck, 2013). A chair is sit-able for an individual, irrespective of whether this individual stays at 20 cm or 100 km from the chair. A scientist interested in describing the niche of a specifc individual could do so if they manipulate a doll and test all the actions that the doll can perform in the environment (e.g., sitting, climbing by bending the doll’s knees, grasping an object with one or both hands). In the end, the description of all the action possibilities will correspond to the niche of the doll. The problem is that this way of proceeding, which is only based on the morphological features of an individual, is not completely adequate when the time comes to do so with a biological agent. The action possibilities described must also consider some biomechanical features, such as muscular forces. Although the consideration of these features tends to complexify the description that can be made, it remains “theoretically” feasible and is far less complicated than considering physiological features, which makes the problem of the description very difcult to solve. Indeed, a series of works has shown that the perception of the actions that can be performed on the environment can vary according to the individual’s physiological state (for a review, see Proftt, 2006; but see Durgin et  al., 2009; Firestone & Scholl, 2016). For example, people who are in declining health, with a low level of glucose, encumbered (i.e., wearing a heavy backpack), or physically fatigued tend to overestimate distances or hill slants (Bhalla & Proftt, 1999; Proftt et  al., 2003; Schnall et  al., 2010). Participants also perceive objects as farther when difcult to grasp (Linkenauger et al., 2009). Proftt (2006) suggested that these fndings demonstrate that perception is embodied and proposed the economy-of-action account. The function of perception is not to build an objective representation of the physical world but to anticipate the actions and their associated costs (see also Witt, 2011). There are at least three possible ways of defning what afordances are from these fndings. First, they are nothing more than describing action possibilities based on the individual’s morphological features (i.e., the scientist with their doll). This way of proceeding appears inappropriate for biological agents. So, afordances require taking into account the biomechanical features, which is the second possibility (i.e., morphological and biomechanical features). The third possibility is that afordances refer to action possibilities based on the conjunction of the morphological and biomechanical features and the individual’s physiological state. This third possibility may be the fnest way of describing an individual’s niche. However, it also suggests that the niche can vary over time, not only over short periods

Afordance and Tool Use

237

(e.g., being physically fatigued) but also over longer periods (e.g., aging), which makes the description difcult. The 3AS model is not equipped to tackle whether physiological states must be considered as a determinant in the description of the niche of an individual. However, it is sympathetic to the idea that physiological states may modulate the perception of our physical world. Here, for clarity, we will restrict our focus to the second possible way of defning afordances, as the description of action possibilities based on the individual’s morphological and biomechanical features. What Role Does Afordance Play in Perception? Is It the Entity Organisms Perceive or the Means Through Which Organisms Perceive? Are Afordances the Only Perceptual Dependent Variables?

The main interest of Gibson’s (1977) initial theory of afordance was to raise scientists’ awareness that representations are not mandatory for explaining how an individual acts in their environment. There are afordances. The individual can perceive them without building a representation of the object on which the action is directed. Nevertheless, this proposal is not at odds with the idea that extracting this information necessarily requires a brain or at least a nervous system. The 3AS model assumes that, in humans and our close primate relatives, a specifc brain network is in charge of processing this information dedicated to the execution of motor actions. This system is the motor-control system, which involves superior brain structures, including, among others, the superior parietal lobes and the intraparietal sulci and frontal lobe structures. Neurophysiological evidence has shown that neurons within this system can possess functional properties, depending on the motor actions performed (i.e., canonical neurons). Rizzolatti et  al. (1988) showed that, in macaque monkeys, some groups of neurons located in the rostral part of F5 within the premotor cortex discharged in response to the visual presentation of objects that require precision grips but not of objects requiring fnger prehension or whole hand prehension. Other groups of neurons in F5 were found to have the opposite patterns. Neurons with the same properties were also reported in macaques within the anterior intraparietal sulcus (Murata et al., 2000; Sakata & Taira, 1994; for reviews, see Rizzolatti & Luppino, 2001; Rizzolatti & Matelli, 2003). These fndings illustrate how the motor-control system can plan and execute motor actions based on objects’ physical properties (e.g., size, shape) without requiring any long-term representations about the objects. The 3AS model assumes that the human motorcontrol system shares the same functional properties, allowing humans to perceive the afordances useful for a specifc purpose. Here, the afordances

238

François Osiurak and Giovanni Federico

can be viewed as the action possibilities ofered by each object (e.g., precision grip-ability). The discovery that neurons can selectively discharge in response to external objects that ofer specifc actions has found echoes in the cognitive science literature, in which the concept of motor simulation has been developed. According to Jeannerod (2001), motor simulation refers to a “representation of the future, which includes the goal of the action, the means to reach it, and its consequences on the organism and the external world” (p. S103). Here we will use a more restricted defnition of motor simulation, assuming that it is a process by which an individual assesses its ability to perform an intended motor action by anticipating the outcome of his/her own motor actions and the energetic costs associated with them (Witt & Proftt, 2008). Witt and Proftt (2008) suggested that motor simulation is particularly useful by allowing the individual to explore potential motor actions with minimal costs and risks. As they stressed, “[L]ittle energy is wasted simulating various options compared with the energetic costs of trying all the possible actions before deciding which was best” (p. 1489). In a way, motor simulation is a process that allows the individual to solve Bernstein’s (1996) degree-of-freedom problem, which consists in selecting the most economical motor action among the wide range of possible motor actions. Motor simulation is not a conscious process, even if it can be. In this case, the term “mental imagery” is preferentially used. Evidence for the existence of motor simulation comes from studies in which it has been demonstrated that participants take longer to judge whether a diffcult grasp is difcult than an easy grasp is easy (Frak et  al., 2001) or that participants take as long to physically walk to a target as they do to imagine walking to it (Decety et al., 1989). To sum up, for the 3AS model, afordances describe the action possibilities ofered by the environment to a specifc animal in reference to its capabilities (i.e., physical level), and the motor-control system is in charge of processing them (i.e., neurocognitive level), without the need of possessing long-term representations about the objects on which the motor action is directed. There is no real distinction between afordance perception and motor simulation in this framework. Why Is Afordance Any Better Than Stimulus? What Does a Theory of Afordance Suggest That Stimuli Cannot; How Has It Moved the Needle Past Gibson 1960’s Recognition of How Little We Can Defne Stimuli?

One way of distinguishing afordances from stimuli is to consider that afordance perception does not involve any form of representation, whereas stimulus perception does. As explained, the 3AS model does not assume

Afordance and Tool Use

239

that long-term representations are needed to perceive afordances. Motor simulation can be done simply based on basic object physical properties. However, if we focus on the context of tool use, the 3AS model posits that in humans, tool use—as well as tool making, construction, etc. (see earlier)—also requires understanding mechanical actions, which must not be confounded with afordances. This is specifcally on this aspect that the 3AS model diverges from other neurocognitive and ecological psychology approaches. Indeed, there is a trend in the literature on afordance to overgeneralize the afordances. This trend was initiated by Gibson himself: An elongated object of moderate size and weight afords wielding. If used to hit or strike, it is a club or hammer. If used by a chimpanzee behind bars to pull in a banana beyond its reach, it is a sort of rake. In either case, it is an extension of the arm. A rigid staf also afords leverage and in that use is a lever. A pointed elongated object afords piercing—if large it is a spear, if small a needle or awl. . . . A rigid object with a sharp dihedral angle, an edge, afords cutting and scraping; it is a knife. It may be designed for both striking and cutting, and then it is an axe. —Gibson (1979, p. 133) In this quote, Gibson implicitly over-generalizes afordance by implying that even the actions performed by an external object toward another external object can be labeled as an afordance. This proposal violates the assumption of subjectivity that is fundamental to the concept of afordance. Indeed, afordances are subjective because they refer to the action possibilities ofered by the environment in reference to an individual’s capabilities, in the manner the scientist with the doll can do. However, if we start to describe all the actions that can be performed between external objects irrespective of the individual’s capabilities, then the concept of afordance loses its interest, simply becoming a description of all the possible actions of the world. In afordance terms, we can describe an object as grasp-able, for instance. However, considering that an object is cutting-with-able implies including an additional external object in the equation. There is another problem. An object can aford cutting if the target is an apple but not a wooden board. In this case, what determines the action possibility of “cutting” is not the individual’s capabilities but the relative object physical properties of each external object. For this reason, the 3AS model proposes to maintain the term “afordances” for describing all the action possibilities ofered by the environment to a specifc individual based on its capabilities. However, it also proposes that the term “mechanical actions” should describe the action possibilities that correspond to the physical interaction

240

François Osiurak and Giovanni Federico

between external objects. As suggested by Shumaker et al. (2011), the user can be an external object (e.g., brushing hair with a hairbrush). In this case, the user’s body is considered an external object that receives the action of another external object. The grasp-ability of the brush is the afordance, whereas the action of brushing is the mechanical action, which involves two external objects: the brush and the hair. The 3AS model suggests that technical reasoning is the critical ability that allows humans to reason about mechanical actions. This reasoning is both analogical and causal and is based on mechanical knowledge, which contains information about physical principles (for a review, see Osiurak, Lesourd, et al., 2020; Osiurak, Federico, et  al., 2020; Osiurak et  al., 2010; Osiurak & Reynaud, 2020). The rationale for distinguishing afordances from mechanical actions is based on fndings that have demonstrated that diferent neurocognitive systems are at work when people process afordances versus mechanical actions. Abundant literature has documented that the motor-control system is mainly supported in primates—including humans—by brain structures within the dorso-dorsal system (e.g., Rizzolatti & Matelli, 2003). By contrast, evidence indicates that the understanding of mechanical actions—supposed to be supported by technical reasoning—involves the ventro-dorsal system, particularly the area PF (i.e., Parietal area F) within the left inferior parietal lobe. Evidence for this comes from the neuropsychological studies of left brain-damaged patients with tool-use disorders, in which it has been shown that damage to the left area PF can generate difculties in selecting, making, and/or using familiar tools or novel tools to solve mechanical problems (e.g., extracting a target out from a box by bending a wire to create a hook; Goldenberg & Spatt, 2009; Martin et al., 2016; Salazar-López et al., 2016; for reviews, see Osiurak, Reynaud et al., 2021). Neuroimaging studies have also shown in healthy participants that this area is preferentially activated when participants focus on the mechanical actions between a tool and an object (e.g., is the tool correctly oriented to interact with the object?) whereas more superior brain structures, including the intraparietal sulcus is engaged when people focus on motor actions (e.g., is this hand posture correct to grasp this tool?; for review, see Reynaud et al., 2016). Interestingly, the same “tool-use” network has also been found in neuroimaging studies when people watch others using tools (Reynaud et al., 2019), suggesting that people also reason about mechanical actions in action observation. It has also been found that the left area PF is involved when participants watch physical events (e.g., a tower of bricks that can fall), even passively (Fischer et al., 2016), or that the cortical thickness of the left area PF predicts performance on psychotechnical tests (e.g., deciding the best confguration to pull an object with a rope;

Afordance and Tool Use

241

Federico, Reynaud et  al., 2023). These latter fndings confrm that the technical-reasoning network involving the left area PF is not simply a “tool-use” network but, more broadly, a “technical” network dedicated to understanding physical events. Interestingly recent evidence has demonstrated that technical-reasoning skills can bolster the evolution of technologies over generations, even when the information transmitted is incomplete or degraded (Osiurak et al., 2022; Osiurak, Lasserre, et al., 2021). These fndings together suggest that humans may possess a specifc network involved in the understanding of mechanical actions, which can also be engaged when people watch others using or making tools.2 What Is the Connection Between Afordances, Behavioral Scale, and Intention? How Distant or Reluctant Can a PerceivingActing System Be Before Afordances Come Into Play? Do Afordances Exert Their Role Only When the System Is Close or Willing Enough?

According to the 3AS model, afordances are directly but not automatically perceived, a perspective close to the ecological psychology approach. Indeed, afordance perception is direct because it does not require any kind of long-term representations of objects. However, afordances are not perceived automatically by the mere sight of objects. We do not perceive all the action possibilities that can be done with a given object but only those relevant according to the agent’s intention and state (Shaw et al., 1982). In the context of tool use, intentions are fulflled by realizing mechanical actions. For instance, the intention of feeding can be fulflled by the mechanical action of “cutting” (e.g., cutting a tomato). More recent developments of the 3AS model suggest that a cascade mechanism links technical reasoning and afordance perception (e.g., Osiurak & Federico, 2021; Osiurak & Reynaud, 2020), which can be briefy described as follows. Technical reasoning allows humans to generate technical solutions that can be useful for reaching a specifc goal. The outcome of this reasoning is the simulation of a specifc mechanical action between a tool and an object (e.g., a knife cutting a tomato). The role of the motor-control system, which is unaware of the goal of the action (e.g., object transport, tool use), is to select the most economical motor actions to realize this mechanical action in the physical world (e.g., grasping the knife with one hand and performing back-and-forth movement with the hand to allow the knife to perform its cutting action). Thus, there is a kind of cascade mechanism (Osiurak, Federico, et al., 2020): The representation of the technical solution biases the selection of appropriate motor actions. This representation also exerts a perceptual control on the motor-control system over the

242

François Osiurak and Giovanni Federico

activity, sometimes leading the agent to modify their motor actions. If a technical problem occurs (e.g., the knife does not cut at all), the agent can reason again to generate new technical solutions (e.g., selecting another knife). This cascade mechanism is supported by evidence indicating that people give priority to mechanical actions over motor actions when watching others using tools or when they simply scrutinize tools (Decroix et al., 2020; Decroix & Kalénine, 2019; Federico, Osiurak, et al., 2023; Federico et al., 2021; Federico, Osiurak, Reynaud, et al., 2021; Federico & Brandimonte, 2019, 2020; Massen & Prinz, 2007; Osiurak & Badets, 2014; Tamaki et al., 2020; van Elk et al., 2008). How Do Systems Engage With Afordances as They Move Among Tasks and Intentions?

As explained in the previous section, the 3AS model posits that afordance perception is in service of other cognitive processes. Individuals have intentions, that is, high-order goals such as feeding. In some cases, these intentions require solving physical problems by realizing mechanical actions between external objects. The selection of a specifc mechanical action eventually biases the perception of relevant afordances—and the process can reiterate, as explained earlier, with the cascade mechanism. In this context, afordances are nothing more than the description of motor action possibilities. As a result, when an individual moves among tasks and intentions, the afordances do not change because they correspond to the description of motor action possibilities irrespective of the individual’s intentions and state. The motor-control system perceives only those afordances that are relevant for a specifc purpose. Concluding Remarks

In sum, the 3AS model capitalizes on the initial conception of afordance by Gibson (1979, 1977) by assuming that long-term representations about objects are not needed to carry out motor actions on the environment (i.e., direct perception). We can grasp and manipulate objects, and so on, without these long-term representations. We can also use our motor-control system to explore objects’ physical properties. However, our motor actions are guided by perceptual efects that we generate. For example, we imagine a hammer pounding a nail, and this conception step biases the perception of appropriate afordances to realize, in the physical world, the perceptual efect generated (for discussion on this aspect, see Osiurak & Badets, 2016). In this respect, the 3AS model is a hybrid model. It is embodied because it acknowledges that afordances do not need representations to

Afordance and Tool Use

243

be perceived. However, it is also disembodied because it does not consider that afordance perception is the level at which a mechanical action is generated. We would like to conclude by illustrating this last point with the elegant study conducted by Wagman and Carello (2001). In this study, Wagman and Carello (2001) found that participants were relatively accurate in making judgments about whether an object was hammer-with-able based only on dynamic touch—they also explored poke-with-ability. Dynamic touch corresponds to the manipulation of an object through muscular efort. It is particularly useful to extract information about the physical properties of objects and, notably, mass distribution. The authors interpreted these fndings as evidence for “tool-use” afordance, suggesting that afordances can also be extended to tool use (i.e., hammer-with-ability). However, as explained earlier, an object is not intrinsically hammer-with-able. It is hammer-with-able only relatively to a specifc object so that it can be hammer-with-able for a given object but not another one. In this study, the participants were previously instructed to make their judgments based on a vertically oriented wooden rod frmly situated in a wooden block. In other words, their fndings do not demonstrate that people directly perceive that an object is hammer-with-able. Participants must have been instructed to judge the hammer-with-ability of the object handled without any target to do so. Of course, this is not possible, which justifes why the authors presented the participants with a wooden rod before the task. Therefore, participants’ judgment of hammerwith-ability involved additional cognitive processes based on the representation of the wooden rod’s physical properties (i.e., technical reasoning). This study illustrates how afordance perception can be in service of technical reasoning, although it cannot allow individuals to directly perceive mechanical action possibilities. Funding

This work was supported by grants from Région Auvergne-RhôneAlpes (NUMERICOG-2017-900-EA 3082 EMC-R-2075; FO) and the French National Research Agency (ANR; Project TECHNITION: ANR21-CE28-0023-01; FO). Notes 1. By tool use, we limit our focus on the use of physical tools (e.g., stone tool, hammer), which requires the selection of appropriate object physical properties to be appropriately used and oriented during the use. This does not concern the use of more recent interface-based tools (e.g., calculator, Smartphone), which requires learning arbitrary procedures to use them (i.e., pressing a button to

244

François Osiurak and Giovanni Federico

generate an arbitrary efect; for more discussion on this aspect, see Osiurak & Heinke, 2018; see also Fournier et al., 2021). 2. The idea that technical reasoning might be preferentially engaged when humans process mechanical actions does not imply that all our tool-use interactions are guided by technical reasoning. For example, as discussed earlier, using interfacebased tools might be done without technical reasoning simply by learning the arbitrary actions that generate an expected efect. In the same way, nonhumans might use other cognitive processes to learn mechanical actions and reproduce them appropriately.

Reference List Arbib, M. A., Bonaiuto, J. B., Jacobs, S., & Frey, S. H. (2009). Tool use and the distalization of the end-efector. Psychological Research, 73(4), 441–462. https:// doi.org/10.1007/s00426-009-0242-2 Beck, B. B. (1980). Animal Tool Behavior. Garland STPM Press. Bernstein, N. A., Latash, M. L., & Turvey, M. T. (1996). Dexterity and Its Development. Lawrence Erlbaum. Bhalla, M., & Proftt, D. R. (1999). Visual—Motor recalibration in geographical slant perception. Journal of Experimental Psychology: Human Perception and Performance, 25(4), 1076–1096. https://doi.org/10.1037/0096-1523.25.4.1076 Buxbaum, L. J., & Kalénine, S. (2010). Action knowledge, visuomotor activation, and embodiment in the two action systems. Annals of the New York Academy of Sciences, 1191(1), 201–218. https://doi.org/10.1111/j.1749-6632. 2010.05447.x Chemero, A. (2003). An outline of a theory of afordances. Ecological Psychology, 15(2), 181–195. https://doi.org/10.1207/S15326969ECO1502_5 Decety, J., Jeannerod, M., & Prablanc, C. (1989). The timing of mentally represented actions. Behavioural Brain Research, 34(1), 35–42. https://doi.org/10.1016/ S0166-4328(89)80088-9 Declerck, G. (2013). Why motor simulation cannot explain afordance perception. Adaptive Behavior,21(4), 286–298.https://doi.org/10.1177/1059712313488424 Decroix, J., Borgomaneri, S., Kalénine, S., & Avenanti, A. (2020). State-dependent TMS of inferior frontal and parietal cortices highlights integration of grip confguration and functional goals during action recognition. Cortex, 132, 51–62. https://doi.org/10.1016/j.cortex.2020.08.004 Decroix, J., & Kalénine, S. (2019). What frst drives visual attention during the recognition of object-directed actions? The role of kinematics and goal information. Attention, Perception, & Psychophysics, 81(7), 2400–2409. https://doi. org/10.3758/s13414-019-01784-7 Durgin, F. H., Baird, J. A., Greenburg, M., Russell, R., Shaughnessy, K., & Waymouth, S. (2009). Who is being deceived? The experimental demands of wearing a backpack. Psychonomic Bulletin & Review, 16(5), 964–969. https://doi. org/10.3758/PBR.16.5.964 Federico, G., Osiurak, F., Brandimonte, M. A., Salvatore, M., & Cavaliere, C. (2023). The visual encoding of graspable unfamiliar objects. Psychological Research, 87, 452–461. https://doi.org/10.1007/s00426-022-01673-z

Afordance and Tool Use

245

Federico, G., Reynaud, E., Navarro, J., Lesourd, M., Gaujoux, V., Lamberton, F., Ibarrola, D., Cavaliere, C., Alfano, V., Aiello, M., Seguin, P., Schnebelen, D., Brandimonte, M. A., Rossetti, Y., & Osiurak, F. (2023). The cortical thickness of the area PF of the left inferior parietal cortex mediates technical-reasoning skills. Scientifc Reports, 12, 11840. https://doi.org/10.1038/s41598-022-15587-8 Federico, G., & Brandimonte, M. A. (2019). Tool and object afordances: An ecological eye-tracking study. Brain and Cognition, 135, 103582. https://doi. org/10.1016/j.bandc.2019.103582 Federico, G., & Brandimonte, M. A. (2020). Looking to recognise: The pre-eminence of semantic over sensorimotor processing in human tool use. Scientifc Reports, 10(1), 6157. https://doi.org/10.1038/s41598-020-63045-0 Federico, G., Osiurak, F., & Brandimonte, M. A. (2021). Hazardous tools: The emergence of reasoning in human tool use. Psychological Research, 85(8), 3108–3118. https://doi.org/10.1007/s00426-020-01466-2 Federico, G., Osiurak, F., Reynaud, E., & Brandimonte, M. A. (2021). Semantic congruency efects of prime words on tool visual exploration. Brain and Cognition, 152, 105758. https://doi.org/10.1016/j.bandc.2021.105758 Firestone, C., & Scholl, B. J. (2016). Cognition does not afect perception: Evaluating the evidence for “top-down” efects. Behavioral and Brain Sciences, 39, e229. https://doi.org/10.1017/S0140525X15000965 Fischer, J., Mikhael, J. G., Tenenbaum, J. B., & Kanwisher, N. (2016). Functional neuroanatomy of intuitive physical inference. Proceedings of the National Academy of Sciences, 113(34), E5072–E5081. https://doi.org/10.1073/pnas. 1610344113 Fournier, I., Beck, S. R., Droit-Volet, S., Brogniart, J., & Osiurak, F. (2021). Learning versus reasoning to use tools in children. Journal of Experimental Child Psychology, 211, 105232. https://doi.org/10.1016/j.jecp.2021.105232 Fragaszy, D. M., & Mangalam, M. (2018). Tooling. Advances in the Study of Behavior, 50, 177–241. https://doi.org/10.1016/bs.asb.2018.01.001 Frak, V., Paulignan, Y., & Jeannerod, M. (2001). Orientation of the opposition axis in mentally simulated grasping. Experimental Brain Research, 136(1), 120–127. https://doi.org/10.1007/s002210000583 Frey, S. H. (2007). What puts the how in where? Tool use and the divided visual streams hypothesis. Cortex, 43(3), 368–375. https://doi.org/10.1016/ S0010-9452(08)70462-3 Gibson, J. J. (1977). The theory of afordances. In R. E. Shaw & J. Bransford (Eds.), Perceiving, Acting, and Knowing: Toward an Ecological Psychology (pp. 67–82). Lawrence Erlbaum. Gibson, J. J. (1979). The Ecological Approach to Visual Perception. Houghton Mifin. Goldenberg, G., & Spatt, J. (2009). The neural basis of tool use. Brain, 132(6), 1645–1655. https://doi.org/10.1093/brain/awp080 Jeannerod, M. (2001). Neural simulation of action: A unifying mechanism for motor cognition. NeuroImage, 14(1), S103–S109. https://doi.org/10.1006/nimg. 2001.0832 Jones, K. S. (2003). What is an afordance? Ecological Psychology, 15(2), 107–114. https://doi.org/10.1207/S15326969ECO1502_1

246

François Osiurak and Giovanni Federico

Linkenauger, S. A., Witt, J. K., Stefanucci, J. K., Bakdash, J. Z., & Proftt, D. R. (2009). The efects of handedness and reachability on perceived distance. Journal of Experimental Psychology: Human Perception and Performance, 35(6), 1649–1660. https://doi.org/10.1037/a0016875 Mangalam, M. (2016). What makes a tool. In V. Weekes-Shackelford, T. K. Shackelford, & V. A. Weekes-Shackelford (Eds.), Encyclopedia of Evolutionary Psychological Science (pp.  1–5). Springer. https://doi. org/10.1007/978-3-319-16999-6_3153-1 Mangalam, M., & Fragaszy, D. M. (2016). Transforming the body-only system into the body-plus-tool system. Animal Behaviour, 117, 115–122. https://doi. org/10.1016/j.anbehav.2016.04.016 Martin, M., Nitschke, K., Beume, L., Dressing, A., Bühler, L. E., Ludwig, V. M., Mader, I., Rijntjes, M., Kaller, C. P., & Weiller, C. (2016). Brain activity underlying tool-related and imitative skills after major left hemisphere stroke. Brain, 139(5), 1497–1516. https://doi.org/10.1093/brain/aww035 Massen, C., & Prinz, W. (2007). Activation of action rules in action observation. Journal of Experimental Psychology: Learning, Memory, and Cognition, 33(6), 1118–1130. https://doi.org/10.1037/0278-7393.33.6.1118 Michaels, C. F. (2003). Afordances: Four points of debate. Ecological Psychology, 15(2), 135–148. https://doi.org/10.1207/S15326969ECO1502_3 Murata, A., Gallese, V., Luppino, G., Kaseda, M., & Sakata, H. (2000). Selectivity for the shape, size, and orientation of objects for grasping in neurons of monkey parietal area AIP. Journal of Neurophysiology, 83(5), 2580–2601. https://doi. org/10.1152/jn.2000.83.5.2580 Norman, J. (2002). Two visual systems and two theories of perception: An attempt to reconcile the constructivist and ecological approaches. Behavioral and Brain Sciences, 25(1), 73–96. https://doi.org/10.1017/S0140525X0200002X Osiurak, F., & Badets, A. (2014). Pliers, not fngers: Tool-action efect in a motor intention paradigm. Cognition, 130(1), 66–73. https://doi.org/10.1016/j.cognition. 2013.09.005 Osiurak, F., & Badets, A. (2016). Tool use and afordance: Manipulation-based versus reasoning-based approaches. Psychological Review, 123(5), 534–568. https://doi.org/10.1037/rev0000027 Osiurak, F., Claidière, N., Bluet, A., Brogniart, J., Lasserre, S., Bonhoure, T., Di Rollo, L., Gorry, N., Polette, Y., Saude, A., Federico, G., Uomini, N., & Reynaud, E. (2022). Technical reasoning bolsters cumulative technological culture through convergent transformations. Science Advances, 8(9), eabl7446. https:// doi.org/10.1126/sciadv.abl7446 Osiurak, F., & Federico, G. (2021). Four ways of (mis-)conceiving embodiment in tool use. Synthese, 199(1), 3853–3879. https://doi.org/10.1007/s11229020-02960-1 Osiurak, F., Federico, G., Brandimonte, M. A., Reynaud, E., & Lesourd, M. (2020). On the temporal dynamics of tool use. Frontiers in Human Neuroscience, 14, 579378. https://doi.org/10.3389/fnhum.2020.579378 Osiurak, F., & Heinke, D. (2018). Looking for intoolligence: A unifed framework for the cognitive study of human tool use and technology. American Psychologist, 73(2), 169–185. https://doi.org/10.1037/amp0000162

Afordance and Tool Use

247

Osiurak, F., Jarry, C., & Le Gall, D. (2010). Grasping the afordances, understanding the reasoning: Toward a dialectical theory of human tool use. Psychological Review, 117(2), 517–540. https://doi.org/10.1037/a0019004 Osiurak, F., Lasserre, S., Arbanti, J., Brogniart, J., Bluet, A., Navarro, J., & Reynaud, E. (2021). Technical reasoning is important for cumulative technological culture. Nature Human Behaviour, 5(12), 1643–1651. https://doi.org/10.1038/ s41562-021-01159-9 Osiurak, F., Lesourd, M., Navarro, J., & Reynaud, E. (2020). Technition: When tools come out of the closet. Perspectives on Psychological Science, 15(4), 880– 897. https://doi.org/10.1177/1745691620902145 Osiurak, F., & Reynaud, E. (2020). The elephant in the China shop: When technical reasoning meets cumulative technological culture. Behavioral and Brain Sciences, 43. https://doi.org/10.1017/S0140525X20000291 Osiurak, F., Reynaud, E., Baumard, J., Rossetti, Y., Bartolo, A., & Lesourd, M. (2021). Pantomime of tool use: Looking beyond apraxia. Brain Communications, 3(4), fcab263. https://doi.org/10.1093/braincomms/fcab263 Osiurak, F., Rossetti, Y., & Badets, A. (2017). What is an afordance? 40 years later. Neuroscience & Biobehavioral Reviews, 77, 403–417. https://doi.org/10.1016/j. neubiorev.2017.04.014 Proftt, D. R. (2006). Embodied perception and the economy of action. Perspectives on Psychological Science, 1(2), 110–122. https://doi.org/10.1111/ j.1745-6916.2006.00008.x Proftt, D. R., Stefanucci, J., Banton, T., & Epstein, W. (2003). The role of efort in perceiving distance. Psychological Science, 14(2), 106–112. https://doi.org/ 10.1111/1467-9280.t01-1-01427 Reynaud, E., Lesourd, M., Navarro, J., & Osiurak, F. (2016). On the neurocognitive origins of human tool use: A critical review of neuroimaging data. Neuroscience & Biobehavioral Reviews, 64, 421–437. https://doi.org/10.1016/j. neubiorev.2016.03.009 Reynaud, E., Navarro, J., Lesourd, M., & Osiurak, F. (2019). To watch is to work: A review of neuroimaging data on tool use observation network. Neuropsychology Review, 29(4), 484–497. https://doi.org/10.1007/s11065-019-09418-3 Rizzolatti, G., Camarda, R., Fogassi, L., Gentilucci, M., Luppino, G., & Matelli, M. (1988). Functional organization of inferior area 6 in the macaque monkey. II: Area F5 and the control of distal movements. Experimental Brain Research, 71(3), 491–507. https://doi.org/10.1007/BF00248742 Rizzolatti, G., & Luppino, G. (2001). The cortical motor system. Neuron, 31(6), 889–901. https://doi.org/10.1016/S0896-6273(01)00423-8 Rizzolatti, G., & Matelli, M. (2003). Two diferent streams form the dorsal visual system: Anatomy and functions. Experimental Brain Research, 153(2), 146– 157. https://doi.org/10.1007/s00221-003-1588-0 Sakata, H., & Taira, M. (1994). Parietal control of hand action. Current Opinion in Neurobiology, 4(6), 847–856. https://doi.org/10.1016/0959-4388(94) 90133-3 Salazar-López, E., Schwaiger, B. J., & Hermsdörfer, J. (2016). Lesion correlates of impairments in actual tool use following unilateral brain damage. Neuropsychologia, 84, 167–180. https://doi.org/10.1016/j.neuropsychologia.2016.02.007

248

François Osiurak and Giovanni Federico

Schnall, S., Zadra, J. R., & Proftt, D. R. (2010). Direct evidence for the economy of action: Glucose and the perception of geographical slant. Perception, 39(4), 464–482. https://doi.org/10.1068/p6445 Shaw, R., Turvey, M. T., & Mace, W. (1982). Ecological psychology: The consequence of a commitment to realism. In W. Weimer & D. Palermo (Eds.), Cognition and the Symbolic Process (pp. 159–226). Erlbaum. Shumaker, R. W., Walkup, K. R., & Beck, B. (2011). Animal Tool Behavior: The Use and Manufacture of Tools by Animals. Johns Hopkins University Press. Stofregen, T. A. (2003). Afordances as properties of the animal-environment system. Ecological Psychology, 15(2), 115–134. https://doi.org/10.1207/S15326969 ECO1502_2 Symes, E., Ellis, R., & Tucker, M. (2007). Visual object afordances: Object orientation. Acta Psychologica, 124(2), 238–255. https://doi.org/10.1016/j. actpsy.2006.03.005 Tamaki, Y., Nobusako, S., Takamura, Y., Miyawaki, Y., Terada, M., & Morioka, S. (2020). Efects of tool novelty and action demands on gaze searching during tool observation. Frontiers in Psychology, 11. https://doi.org/10.3389/ fpsyg.2020.587270 Tucker, M., & Ellis, R. (1998). On the relations between seen objects and components of potential actions. Journal of Experimental Psychology: Human Perception and Performance, 24(3), 830–846. American Psychological Association. https://doi.org/10.1037/0096-1523.24.3.830 van Elk, M., van Schie, H. T., & Bekkering, H. (2008). Conceptual knowledge for understanding other’s actions is organized primarily around action goals. Experimental Brain Research, 189(1), 99–107. https://doi.org/10.1007/s00221008-1408-7 Wagman, J. B., & Carello, C. (2001). Afordances and inertial constraints on tool use. Ecological Psychology, 13(3), 173–195. https://doi.org/10.1207/S153269 69ECO1303_1 Warren, W. H. (1984). Perceiving afordances: Visual guidance of stair climbing. Journal of Experimental Psychology: Human Perception and Performance, 10(5), 683–703. https://doi.org/10.1037/0096-1523.10.5.683 Witt, J. K. (2011). Action’s efect on perception. Current Directions in Psychological Science, 20(3), 201–206. https://doi.org/10.1177/0963721411408770 Witt, J. K., & Proftt, D. R. (2008). Action-specifc infuences on distance perception: A role for motor simulation. Journal of Experimental Psychology: Human Perception and Performance, 34(6), 1479–1492. https://doi.org/10.1037/a0010781 Young, G. (2006). Are diferent afordances subserved by diferent neural pathways? Brain and Cognition, 62(2), 134–142. https://doi.org/10.1016/j.bandc. 2006.04.002

14 FROM TURING TO GIBSON Implications of Afordances for the Sciences of Organisms Madhur Mangalam, Louise Barrett, and Dorothy M. Fragaszy

What Do We Understand by the Term “Afordance”?

Afordances, as conceived by James J. Gibson, are the potential for actions that an object or environment ofers to an organism; they are the directly perceived opportunities for interaction or engagement with the surroundings relative to the organism’s action capabilities, intentions, and needs. Afordances determine what an organism can do in a given environmental setting and so determine the signifcance or meaning of that setting for the organism. In this view, then, perception is both non-inferential and intimately linked to action: via their constant and ongoing activity, organisms generate information from their surroundings and use this to guide further action, and they discover meaning through their ongoing exploratory activity, rather than meaning being added to the stimulus by an internal cognitive process (Gibson, 1979). Perception is thus an active process of “keeping in touch with the environment” (Gibson, 1979, p. 239), not any mental state in the head. The theory of direct perception of afordances (and therefore, the rejection of “mental gymnastics;” Chemero, 2009) has profound implications for mind sciences—it conficts with the prevalent metaphors of the “brain as a computer,” “perception as sensory input,” and “cognition as information processing,” leading to “action (behavior) as output.” These informationtheoretic metaphors invoke representations (processes, or computations, occurring entirely within the nervous system) to mediate organisms’ interactions with their environments. These ideas trace back to John von Neumann, who developed the ENIAC (Electronic Numerical Integrator and DOI: 10.4324/9781003396536-18

250

Madhur Mangalam, Louise Barrett, and Dorothy M. Fragaszy

Computer) in the late 1940s. von Neumann’s visionary design encompassed the foundational elements found in modern computers: a central processing unit, main memory, peripherals, and external storage, drawing parallels between this computational architecture and the intricate workings of the brain. According to von Neumann, the CPU mirrored associative neurons, while input and output devices functioned like sensory and motor neurons. This infuential “von Neumann architecture” shaped the trajectory of artifcial intelligence, and his language has formed the bedrock of our metaphors of the computational architecture of the mind. von Neumann drew on the work of Alan Turing, the acclaimed mathematician and “father” of computer science. Turing revolutionized computing with his concept of (so-called) Turing machines, which he frst set out in the 1930s (Turing, 1950, 1936). During the First and Second World Wars, “computers” were not machines but people charged with generating map grids, surveying aids, and navigation and artillery tables based on strictly set rules. Turing aimed to automate this laborious computational process to increase efciency. He sought to create a machine that could perform calculations like a human computer using paper and pencil. Therefore, the abstract machine he developed was an efort to encompass the overall environment in which human computers undertook their tasks. The moveable head of the machine represented the person, and the infnite paper tape that served as a medium for reading and printing symbols represented the paper and pencil on which the human-computer worked out their sums. The moveable head of the machine would thus traverse the tape, allowing for the manipulation of symbols. Turing’s aim here was entirely practical and mathematical, not psychological: his goal was to automatize calculation, not provide a comprehensive analysis of human behavior or cognition. However, his ideas inspired speculation that human thought might operate algorithmically like a universal Turing machine that can simulate an arbitrary Turing machine on arbitrary input. Thus, von Neuman and others developed the metaphor of the brain as a universal Turing machine, which opened doors for modeling cognition with algorithm-dependent digital computers. However, as Andrew Wells points out in his book Rethinking Cognitive Computation: Turing and the Science of the Mind (Wells, 2017), the development of these ideas included a crucial misstep, whereby the Turing machine’s symbol manipulations were reconceived as a model of internal brain-based cognitive processes. The paper tape was seen as an analog of human memory. That is, early proponents of a computational approach to mind placed the entire Turing machine into the head of the human-computer, whereas, as we have made clear earlier, the infnite tape (representing ample paper) and the symbols manipulated by a Turing machine (representing the

From Turing to Gibson

251

calculations made on the paper) are both external to the mind and part of the environment. Even the machine-head is not to be seen as a model of the mind alone but of the entire person, equipped with paper and pencil, operating in the environment (Wells refers to this set-up as a “mini-mind” to get across the idea that this could be a complete description of an agent with a very simple mind, or a partial description of a more complex one). Turing machines are thus ecological contraptions in a Gibsonian sense— calculations result from an interaction between the agent (person) and the environment (paper and pencil), and the manipulated symbols sit outside the mind and are not part of it. The mistaken conceptualization of Turing machines as “inside the head” inevitably (and fatally) minimized the role of bodies and the environment in cognition, enabling the belief that computer intelligence could rival human intelligence despite lacking a physical embodiment. In this computational viewpoint, psychological processes became associated with “information processing.” Sensory input entered the cognitive system, symbols were manipulated as in a computer, and the output of these cognitive computations guided actions. This conception casts perception, cognition, and action as separate, sequential processes. Understanding the mind now centered on the processing between input and output, with the brain being considered hardware and cognitive processes as software. It is crucial to delve into the history of, and subsequent misconception surrounding, the universal Turing machine to better understand how the theory of afordances (and direct perception, more generally) aligns with the biology of the body, including the nervous system and avoids the intractability of information-theoretic metaphors of neural function. For instance, the brain operates at multiple levels of organization, from individual neurons to neural networks, brain regions, and global dynamics. Each level has unique physiological properties, which makes it challenging to fnd a unifed information-theoretic framework. Information-theoretic measures rely on assumptions about information representation and interpretation, but these assumptions may not capture the complexity and fuidity of neural processing. Finally, the brain is dynamic and functions in a contextdependent manner; that is, it handles information based on the system’s state and goals. Information theory models, in contrast, are static, which is a strong limitation of these models. Hence, the information-theoretic interpretation of the Turing machine is of limited value in understanding the organization and role of the nervous system and how an organism engages with its surroundings. A more accurate understanding of Turing machines as systems comprising an agent embedded in an environment makes a more ecological view of cognitive computation possible. It potentially reconciles what

252

Madhur Mangalam, Louise Barrett, and Dorothy M. Fragaszy

would appear to be incommensurate paradigms (Barrett, 2011). Specifically, Wells (Wells, 2017, 2002) proposes that if we combine Gibson’s theory with Turing’s computational theory, generating a formal model of afordances as confgurations of a Turing machine becomes possible. The integration of Turing’s and Gibson’s theories undercuts the isolated “brain as a computer” metaphor by bringing to the fore the complementary relationship between internal organism structure and external environmental structure (Wells, 2017, 2002). This treatment of afordances makes them facts of the environment (the tape and symbols = environment and ambient energy arrays), the behavior (movements of the machine’s head = movements of the organism), and the environment and the organism determinately linked to ongoing behavior via the organism’s capabilities (manipulation of symbols based on the strictly set rules = neural attunement). A confguration of a Turing machine is defned by an ordered triple (x, q, k) ∈ Σ × K × N, where x represents the symbol on the tape, q represents the machine’s current state, and k represents the machine’s position on the tape (Turing, 1950). This comprehensive confguration determines the machine’s future behavior; efectively predicting the future behavior of the machine hinges solely on taking into account the machine’s current state and the symbol on the tape. Hence, the theory of computation, pioneered by Turing, establishes a direct relationship between a symbol, the state of a machine, and the symbol’s position on the tape, enabling precise manipulation of that symbol. Similarly, the theory of afordances connects objects’ inherent properties to the range of potential actions available to an organism possessing distinct capabilities (Wells, 2017, 2002). What Role Does Afordance Play in Perception? Is It the Entity Organisms Perceive or the Means Through Which Organisms Perceive? Are Afordances the Only Perceptual Dependent Variables?

We hold that organisms perceive afordances—that is, afordances are perceived entities. This view vitiates “perception through some entity” models. For example, let us assert for argument’s sake that organisms perceive afordances through some entity “X.” The question immediately following this assertion is: “How is ‘X’ perceived?” and if one claims that “X” is perceived through another entity “Y,” then the question immediately following is: “How is ‘Y’ perceived?,” invoking an intractable sequence of mediators for perception. A logical solution to the impossibility of infnite regress is that perception of afordances is “direct,” without any representation or mediating entity “X.” The ecological program of research has shown that

From Turing to Gibson

253

information in the environment is sufcient to specify afordances directly as the confguration of a Turing machine (coupling the facts of the environment, the organism’s behavior, and the link between the organism and the environment). As Gibson noted, Perhaps the composition and layout of surfaces constitute what they aford. If so, to perceive them is to perceive what they aford. This is a radical hypothesis because it implies that the ‘values’ and ‘meanings’ of things in the environment can be directly perceived. (Gibson, 1979, p. 129) Modern ecological psychologists frequently regard information-as-specifcation as the main principle of Gibson’s ecological philosophy. Consequently, empirical investigations focus on identifying specifc energy patterns within the surrounding stimulus array to support the theory of direct perception. Nevertheless, organisms can rely on non-specifying variables in certain circumstances until sufcient perceptual learning occurs to enable a shift to the specifying variables (Fajen, 2005; Jacobs & Michaels, 2002). For example, humans react to looming objects on a collision course with their faces by blinking their eyes as a defensive reaction. In experimental settings, newborns blink their eyes when a looming virtual object reaches a threshold visual angle or angular velocity, strategies that work adequately to blink their eyes in advance of virtual contact so long as the object is not accelerating. By six months, infants have switched to a time-to-contact strategy (used by adults), efective for all looming objects, including those accelerating on approach (Kayed & van der Meer, 2007). Adult novices rely more than experts on task variables lacking specifcity in tasks, for instance, in which participants perceive masses of colliding balls based on changes in optical patterns (Jacobs et al., 2000; Runeson, 1988). These anomalies seem to challenge the ecological theory unless one accepts that the specifcation of information is not central to direct perception (Withagen & Chemero, 2012) and that perceptual learning is a normal part of development and accommodation in animals of all ages, especially among the young. In a similar vein, it would be interesting to examine perception and neural organization in amphibians before and after metamorphosis (such as from an aquatic tadpole to an air-breathing frog with four limbs). Metamorphosis entails a wholesale change in body shape and function, diet, locomotion, and more; changes in afordances should accompany such physical and ecological changes and, thus, changes in relevant specifying variables. In this way, newly metamorphosed frogs can be considered novices with respect to their new bodies and lives. Hence, they are a good target for studies of

254

Madhur Mangalam, Louise Barrett, and Dorothy M. Fragaszy

perceptual attunement to specifying variables (e.g., about the movement of mobile prey). Historically, perception of class membership has been treated as a challenge to direct perception because if classes do not have determinate boundaries and, thus, do not have physical properties in common, then there are no patterns in the ambient light that specify class membership (Withagen & Chemero, 2012). In this situation, a class refers to entities based on family resemblances. Nevertheless, class members cannot always be specifed precisely (as in the class of “tree”—the boundary between bush and tree is permeable, and likewise between vine and tree). This challenge rests on the assumption that a linguistic entity (the notion of “class”) is also a perceptual entity, meaning it specifes an afordance. This assumption is faulty. Human language results from an inherently constructive process (Arbib et al., 2023). While language can refer to afordances, such as asking one to report if one can step over a barrier or will need a ladder, we should not assume that all linguistic constructions refer transparently to afordances nor that all afordances are accessible to language. Experimental procedures that require participants to use linguistic constructs (such as the notion of class membership) to frame their responses to experimenters’ queries add a layer of translation on top of perception; such procedures ask about information removed from afordances. Thus, direct perception of class membership should not be expected. In our view, the historical conundrum of perception of class membership by direct perception can be considered irrelevant to evaluating the validity of the theory of direct perception. In summary, perception is intricately tied to the organism’s precise location and cannot be separated from it. This reality undermines the concept of perception mediated by external entities, suggesting instead that perception is direct and centered on afordances. Although challenges to the theory of direct perception arise, such as reliance on non-specifc variables or, historically, identifying members of a class, a broader understanding of information and the relation between perception and language enables one to handle these challenges as compatible with direct perception. Why Is Afordance Any Better Than Stimulus? What Does a Theory of Afordance Suggest That Stimuli Cannot; How Has It Moved the Needle Past Gibson 1960’s Recognition of How Little We Can Defne Stimuli?

A stimulus refers to a physical event or object that can stimulate our senses by transducing energy (chemical, electromagnetic, or mechanical) via the sensory receptors into electrochemical transduction through neurons. A

From Turing to Gibson

255

stimulus exists independently of an organism’s abilities or intentions. Conversely, afordance refers to the arrangement of stimuli in structured patterns that delineate an organism’s potential to interact with its environment to pursue its objectives (Heft, 2017, 2018). They depend on the relationship between the organism’s abilities and intentions and the properties of the environment. Afordances are context-dependent and relative to the organism’s capabilities and needs. For instance, an apple afords grasping and eating, a certain noise afords covering the ears or to investigating its source, and sunlight afords basking or adjusting one’s position for more warmth. To summarize, stimuli impose discrete sensory inputs on organisms, while afordances are perceptions of opportunities for action based on an organism’s abilities and intentions. Afordances go beyond the stimulation of sensory receptors and involve the organism’s potential environmental engagement. Gibson showed that organisms could not simply rely on properties of light defned by classical physics and geometry as relevant descriptors for biological vision. Instead, they must turn to the global patterns of stimulation to which organisms have become attuned through both evolution and development. It is also worth noting that, as Gibson (1976) pointed out, animals in their natural environments encounter a “fowing sea of stimulus energy” (p. 398) to which organisms have become fnely attuned through evolution and from which animals can generate information for their use; only in the laboratory can we impose discrete stimuli that target particular sensory receptors in isolation. How does the organization of the central nervous system support the perception of afordances? The pioneering work of Michael Graziano ofers one answer to this question. In 2002, Graziano and colleagues discovered that the motor cortex of rhesus monkeys exhibited an arrangement distinct from that previously known: namely, functional zones linked to various action categories (Graziano et al., 2002), which they referred to as “ethological action maps.” The theological action map encompassed a range of complex, functionally relevant actions, including hand-to-mouth movements, defensive movements safeguarding the body, reaching and grasping motions, and other behaviors typical of macaques. The research team could elicit these complex and well-coordinated behaviors by applying electrical stimulation to specifc regions within the cortical action zones. These ethological action maps in rhesus monkeys have thus challenged conventional understandings of the motor cortex (Graziano, 2006, 2016; Graziano & Afalo, 2007). Since these initial studies, Graziano and colleagues have substantiated these fndings using various methodologies such as electrical and optogenetic stimulation, chemical manipulation, lesions, single neuron recording, functional imaging, anatomical tract tracing, behavioral analysis, and computational modeling (Afalo & Graziano, 2006a, 2006b,

256

Madhur Mangalam, Louise Barrett, and Dorothy M. Fragaszy

2007; Cooke et al., 2003; Cooke & Graziano, 2003, 2004a, 2004b; Graziano et  al., 2004; Graziano & Cooke, 2006). Furthermore, ethological action maps have been investigated in rhesus monkeys by others (Overduin et al., 2012), and extensively in other species, including rats (Brecht et al., 2004; Brown & Teskey, 2014), mice (Harrison et al., 2012), galagos (Stepniewska et al., 2005, 2009, 2011), squirrels (Cooke et al., 2012), cats (Ethier et  al., 2006), and humans (Desmurget et  al., 2014). Such work suggests that the ethological action map may be a fundamental organizing principle within the motor cortex of mammalian species. This, in turn, has profound implications for comprehending the neural basis of perceiving and acting upon afordances. On the one hand, stimulus-induced rhythmic movements, such as chewing or whisking observed in mice and rats, involve the active engagement of subcortical structures, like the facial nucleus (Berg & Kleinfeld, 2003; Deschênes et al., 2012). On the other hand, actions like reaching or bringing the hand to the mouth prioritize the coordinated activation of multiple muscles simultaneously rather than relying on precise temporal sequencing. These latter movements exhibit a higher degree of fexibility regarding the contribution of individual muscles or joints, while the overall movement remains relatively consistent. Moreover, particular actions can be realized in multiple ways; that is, diferent combinations of muscles can produce similar functional outcomes (d’Avella et al., 2003; d’Avella & Bizzi, 2005). These fndings have led to the intriguing suggestion that perception and action operate at the level of synergies (Cuadra & Latash, 2019; Profeta & Turvey, 2018; Turvey, 2007). In light of these developments, the integration of behavioral research with neuroscience has emerged as an increasingly pressing need, as highlighted by recent calls in neuroscience and the broader feld of mind sciences (Krakauer et  al., 2017; Parada & Rossi, 2018; Tytell et al., 2011). In this context, afordances, as bridges between perception and action, ofer a more promising path toward this objective than a stimulus-centric view of perception. What Is the Connection Between Afordances, Behavioral Scale, and Intention? How Distant or Reluctant Can a PerceivingActing System Be Before Afordances Come Into Play? Do Afordances Exert Their Role Only When the System Is Close or Willing Enough?

Concepts of afordances, behavioral scale, and intention are reciprocally related. Afordances ofer individuals potential actions. The individual’s behavioral scale, representing the range of actions they can undertake, impacts how they perceive afordances. Intention infuences the perception

From Turing to Gibson

257

and utilization of afordances. If an object afords myriad behaviors, afordances do not constrain behavior. The multiplicity of afordances associated with an object in a given setting is entirely consistent with the theme of intentionality central to the ecological approach, as noted by Heft (1989): “the afordances of an object are realized in relation to some intentional act in the individual’s behavior repertoire” (p. 21). Organisms are nonfractionable wholes, comprising genes, cells, organs, and limbs, and, therefore, explanations for organismal behaviors must align with the scale at which those behaviors emerge—at the ecological level at which organisms perceive and act (Gibson, 1979). Examples for humans include body-scaled ratios, which defne the “step-on-ability” of stairs by establishing relationships between leg lengths, eye heights, and riser heights (Konczak et al., 1992; Warren, 1984; Warren & Whang, 1987). It is also the case that while afordances and behavioral scales may align, intention can span scales of space and time that are not necessarily relevant at the ecological level. For example, an apple afords grasping and eating to an individual with a low blood glucose level but may not be perceived as such by a satiated individual. Likewise, the same apple may aford grasping to an individual who is not hungry but wishes to take it along for an upcoming road trip. The physiological processes observed in these examples thus extend to the scale of metabolism. At the same time, we also have to be aware of the potential infuence of intentional long-term planning behavior (i.e., the acquisition of an afordance broadens the range of internal states, enhancing behavioral capabilities across extended time scales). The far-reaching and infuential efect of intention spanning multiple scales of space and time is most notably observed in developing organisms, as in human infants who persist in walking despite the costs of unsteady steps, frequent falls, and challenges in perceiving afordances in their newfound upright posture, and as their bodies are changing rapidly, altering afordances on a daily basis (Adolph, 2008; Adolph & Tamis-LeMonda, 2014). Confning afordances to a limited range of space and time scales would undermine the attainment of a comprehensive and systematic explanation of the connections among afordances, behavioral scale, and intention. How Do Systems Engage With Afordances as They Move Among Tasks and Intentions?

Returning to Wells’ ecological computation model, it is apparent that internal states, functioning as components of Turing machines, actively direct attention to the dynamics of the environment (Wells, 2017, 2002). This phenomenon becomes apparent as diferent confgurations gain prominence while the machine moves along the tape during computation. Just as

258

Madhur Mangalam, Louise Barrett, and Dorothy M. Fragaszy

locomotion across the tape is vital for Turing machines, movement through the environment is fundamental to organisms’ perception of their environments. However, this requirement does not necessitate an explicit model of the environment within the “brains” of Turing machines or organisms’ brains. As we noted earlier, in an ecological view, perception is the activity by which organisms keep in touch with their environments, not the means by which they construct them. The analogy here is resonating with, or tuning into, the environment; rather than a linear process by which organisms sense, represent, plan, and act, we have a loopy process of ongoing and continual sensorimotor coordination in which the brain and nervous system help ensure appropriate behavior in specifc environmental contexts. That is, as Gibson pointed out: “The theory of afordances implies that to see things is to see how to get about among them and what to do or not do with them. If this is true, visual perception serves behavior, and perception controls behavior” (Gibson, 1979, p. 223). The discoveries regarding the primacy of movement in afordance perception go beyond establishing a connection between an object’s physical attributes, such as rotational inertia, and perceptual experiences through manipulating objects in hand, like perceiving heaviness and length-related afordances (Turvey & Carello, 2011). Through skillful navigation of their intentions, human participants can obtain a wide range of information from the same physical events involving grasping and manipulating an object. For example, they can perceive a handheld object’s length, width, and shape from the same exploratory motion with that object (Burton et al., 1990; Mangalam, Conners, Kelty-Stephen, et al., 2019; Mangalam, Chen, et al., 2020; Turvey et al., 1998). They can also adapt their approach to suit specifc aspects of their surrounding environment. For instance, they can perceive the length of a handheld object by probing a distant surface and the distance of that surface (Carello et  al., 1992). Furthermore, diferent exploration methods reveal distinct information, such as static moment versus rotational inertia (Kingma et al., 2004). Notably, the intention behind these exploratory movements shapes their execution. In one study, participants wielded unseen, rectangular parallelepipeds of various sizes and made judgments about their length or width. With feedback, participants gradually shifted their attention toward the optimal variable, demonstrating the education of attention (also known as perceptual learning). When asked to report a new property, participants made signifcant attentional adjustments that helped place their judgments in the ballpark of the optimal variable for the new property (Arzamarski et al., 2010). Similarly, visual feedback infuences judgments of object length by wielding in the hand, leading to refned exploratory movements and the utilization of informative variables in alignment with the received feedback (Michaels

From Turing to Gibson

259

et  al., 2008; Wagman et  al., 2008). Despite the fact that all these studies used a standard set of objects, participants used variable exploratory movements to perceive diferent afordances of those objects. Turing’s infuence extends beyond the logical prowess of Turing machines. The symbols in a Turing machine refect rate-independent processes (Pattee, 2001). This kind of “rate-independence” means that the symbols persist even during interruptions and reappearances on the screen or page. However, the utility of symbols and logic depends on the support of physics for processing and interpreting logical constraints. At a smaller scale, electrons facilitate digit processes in computers, while at a larger scale, movements (such as keypresses and scrolls) contribute to our comprehension and meaning extraction from computers. Specifcally, the usefulness of digitally displayed symbols emerges in conjunction with rate-dependent dynamic elements, such as electric currents and force felds, which sustain the path of these symbols (Pattee, 2013). To understand how these rate-dependent processes complement logic, Turing introduced another, albeit less widely recognized, concept to the behavioral and cognitive sciences realm—the notion of “cascades” (Kelty-Stephen & Mangalam, 2022)—a theoretical framework used in fuid dynamics to describe the transfer of energy from large-scale motions to smaller scales, leading to a cascade-like efect. Turing astutely recognized the role of instability in biological pattern formation, exemplifed by the striking “rosette” spots on a jaguar, and he underscored its signifcance in fostering creative and intelligent thinking (Turing, 1952). While some interpretations separate Turing’s focus on pattern formation from his development of logical machinery to mimic intelligence (Dawes, 2016), we propose that both domains of thought were integral to his outlook. Recent research argues that Turing’s concept of cascading instability aligns with a geometric framework governed by power laws (Kelty-Stephen & Mangalam, 2022). Applying these geometrical frameworks to neuronal activity data has revealed that the brain operates in a state of criticality, a state or condition in complex systems with a delicate balance between order and chaos. Rather than resembling random fuctuations occurring independently, cortical columns generate spreading activity extending between neurons and large-scale resting-state networks, operating simultaneously across multiple scales. Applying the same geometric framework to behavioral data has revealed that cascade instability manifests in various aspects of perceptual functioning, including executive functioning, postural control, and efortful perception (Kelty-Stephen & Mangalam, 2022). This perspective emphasizes the complex multiscale patterning of physiological activity that enables the attunement of organisms’ perceptual systems with environmental information.

260

Madhur Mangalam, Louise Barrett, and Dorothy M. Fragaszy

The Turing machine model illustrates that organismal structure complements environmental structure. Nevertheless, sufcient environmental structure for unambiguous perception does not negate the necessity and use of internal structure to facilitate perception and drive behavior. Hence, whether a particular behavior arises from the environmental or organismal structure cannot be determined a priori. We have operated under this assumption to investigate how movements support the haptic perception of object properties through wielding an object in hand. In this case, it seems reasonable to narrow our focus to the point of contact (e.g., mechanoreceptors in the skin). However, we must exercise caution to avoid becoming excessively fxated on these experimental details to the extent that we neglect the broader context. Perception entails information beyond the hand alone; it permeates the entire organism, originating from and infuencing other areas. It is a holistic reaction that encompasses the entire body. By occluding the grasped object, we do not isolate distinct perceptual subsystems (e.g., visual and haptic); instead, we disrupt a comprehensive cascade encompassing various motor functions. Intriguingly, when participants, blindfolded and standing on a force plate, employ hand movements to perceive properties of an object haptically, the temporal structure of their postural center of pressure (CoP) predicts their judgments of object heaviness and length (Mangalam, Chen, et al., 2020; Mangalam & KeltyStephen, 2020). Despite the object being held in the hand and not underfoot, the relationship between the feet and the ground surface infuences the efortful touch of the hand. Even in cases where the participant remains stationary, the temporal structure of the CoP holds direct implications for perceiving objects supported by the shoulders, with diferences arising depending on whether the person focuses on the entire object or just a part of it (Palatinus et al., 2013, 2014). These fndings invite contemplation of the intriguing connection between the grasping hand and the standing feet. Similarly, bearded capuchin monkeys, standing bipedally and using stones weighing half their body mass or more to strike palm nuts placed on stone anvils, show more stringent control of the position of their feet and lower legs during their strike than of the hands and arms (Mangalam, Pacheco, et al., 2018; Mangalam, Rein, et al., 2018). Maintenance of postural stability is likely an organizing factor in much human activity, evident in what we term “suprapostural dexterity” (Kelty-Stephen et al., 2021). Meticulous investigations conducted by Mangalam, Kelty-Stephen, and colleagues have uncovered the signifcant role played by various physiological manifestations spread throughout the body in perceiving the afordances of handheld objects. Notably, the observed interactions across diferent scales during hand-wielding, stemming from muscular activity, have proven to be more predictive of perceived length than the measurement of

From Turing to Gibson

261

muscular activity alone (Mangalam, Conners, & Singh, 2019; Mangalam, Conners, Kelty-Stephen, et al., 2019). Additionally, participants’ heads display subtle yet cascading swaying movements that critically impact their ability to integrate visual feedback (Kelty-Stephen & Dixon, 2014). These fndings impel us to adopt a swarm perspective when considering the relevance of postural sway in hand-wielding—considering the body to be a veritable swarm of quasi-autonomous parts falling in and out of cooperation (Kelty-Stephen & Mangalam, 2022). Assuming that efortful perception solely revolves around the hand’s exploration of an object leads to the assumption that other anatomical parts do not provide any additional valuable information except for downstream harmonics generated by the resonance of hand-wielding throughout the body. However, our previous examination of pairs or triples involving the head, hand, and torso has now expanded to encompass the entire body’s movement (Mangalam, Carver, et al., 2020a, 2020b). The wielded object and arm interact across scales, mutually enhancing each other’s contribution. These movement patterns propagate from the forearm to the object and vice versa, infuencing the upper arm. Moreover, beyond these interconnected relationships within the upper body, an increase in multifractality within the postural CoP predicts subsequent improvements in perceptual accuracy. These fndings collectively exemplify why it is crucial to envision the body-object complex as constantly adapting its confguration, akin to a Turing machine as conceived ecologically, and to consider the waxing and waning of cascading movement patterns implied by Turing’s cascade instability to facilitate the perception of afordances. Turing’s profound insights into both theoretical Turing machines (Turing, 1950) and cascade instability in physical systems (Turing, 1952) provide valuable lessons for comprehending the interplay between logical and physical processes that underlie organism behavior. Turing’s analysis suggests that the interpretation of symbols and logic heavily relies on the support provided by physical processes. The interdependence of logical and physical processes is evident through cascading dynamics, as demonstrated by empirical fndings in various domains of perceptual functioning, such as executive functioning, postural control, and efortful perception (Kelty-Stephen & Mangalam, 2022). This perspective highlights the inseparable relationship between the [logical] event of arriving at a perceptual judgment and the [physical] process of arriving at that judgment. Further exploration into the role of cascade instability in organizing the complex dynamics of neuronal activity underlying the perception of afordances holds great promise. Empirical inquiries into these cascading mechanisms will yield novel insights into how organisms engage with afordances as they navigate tasks and intentions. By investigating the intricate interplay

262

Madhur Mangalam, Louise Barrett, and Dorothy M. Fragaszy

between physical and cognitive processes, we can deepen our understanding of how organisms perceive and interact with their environment dynamically and adaptively. Reference List Adolph, K. E. (2008). Learning to move. Current Directions in Psychological Science, 17(3), 213–218. https://doi.org/10.1111/j.1467-8721.2008.00577.x Adolph, K. E., & Tamis-LeMonda, C. S. (2014). The costs and benefts of development: The transition from crawling to walking. Child Development Perspectives, 8(4), 187–192. https://doi.org/10.1111/cdep.12085 Afalo, T. N., & Graziano, M. S. A. (2006a). Partial tuning of motor cortex neurons to fnal posture in a free-moving paradigm. Proceedings of the National Academy of Sciences, 103(8), 2909–2914. https://doi.org/10.1073/pnas.0511139103 Afalo, T. N., & Graziano, M. S. A. (2006b). Possible origins of the complex topographic organization of motor cortex: Reduction of a multidimensional space onto a two-dimensional array. Journal of Neuroscience, 26(23), 6288–6297. https://doi.org/10.1523/JNEUROSCI.0768-06.2006 Afalo, T. N., & Graziano, M. S. A. (2007). Relationship between unconstrained arm movements and single-neuron fring in the macaque motor cortex. Journal of Neuroscience, 27(11), 2760–2780. https://doi.org/10.1523/ JNEUROSCI.3147-06.2007 Arbib, M. A., Fragaszy, D. M., Healy, S. D., & Stout, D. (2023). Tooling and construction: From nut-cracking and stone-tool making to bird nests and language. Current Research in Behavioral Sciences, 100121. https://doi.org/10.1016/j. crbeha.2023.100121 Arzamarski, R., Isenhower, R. W., Kay, B. A., Turvey, M. T., & Michaels, C. F. (2010). Efects of intention and learning on attention to information in dynamic touch. Attention, Perception, & Psychophysics, 72(3), 721–735. https://doi. org/10.3758/APP.72.3.721 Barrett, L. (2011). Beyond the Brain: How Body and Environment Shape Animal and Human Minds. Princeton University Press. Berg, R. W., & Kleinfeld, D. (2003). Rhythmic whisking by rat: Retraction as well as protraction of the vibrissae is under active muscular control. Journal of Neurophysiology, 89(1), 104–117. https://doi.org/10.1152/jn.00600.2002 Brecht, M., Schneider, M., Sakmann, B., & Margrie, T. W. (2004). Whisker movements evoked by stimulation of single pyramidal cells in rat motor cortex. Nature, 427(6976), 704–710. https://doi.org/10.1038/nature02266 Brown, A. R., & Teskey, G. C. (2014). Motor cortex is functionally organized as a set of spatially distinct representations for complex movements. Journal of Neuroscience, 34(41), 13574–13585. https://doi.org/10.1523/ JNEUROSCI.2500-14.2014 Burton, G., Turvey, M. T., & Solomon, H. Y. (1990). Can shape be perceived by dynamic touch? Perception & Psychophysics, 48(5), 477–487. https://doi. org/10.3758/BF03211592

From Turing to Gibson

263

Carello, C., Fitzpatrick, P., & Turvey, M. T. (1992). Haptic probing: Perceiving the length of a probe and the distance of a surface probed. Perception & Psychophysics, 51(6), 580–598. https://doi.org/10.3758/BF03211655 Chemero, A. (2009). Radical Embodied Cognitive Science. MIT Press. Cooke, D. F., & Graziano, M. S. A. (2003). Defensive movements evoked by air puf in monkeys. Journal of Neurophysiology, 90(5), 3317–3329. https://doi. org/10.1152/jn.00513.2003 Cooke, D. F., & Graziano, M. S. A. (2004a). Super-flinchers and nerves of steel: Defensive movements altered by chemical manipulation of a cortical motor area. Neuron, 43(4), 585–593. https://doi.org/10.1016/j.neuron. 2004.07.029 Cooke, D. F., & Graziano, M. S. A. (2004b). Sensorimotor integration in the precentral gyrus: Polysensory neurons and defensive movements. Journal of Neurophysiology, 91(4), 1648–1660. https://doi.org/10.1152/jn.00955.2003 Cooke, D. F., Padberg, J., Zahner, T., & Krubitzer, L. (2012). The functional organization and cortical connections of motor cortex in squirrels. Cerebral Cortex, 22(9), 1959–1978. https://doi.org/10.1093/cercor/bhr228 Cooke, D. F., Taylor, C. S. R., Moore, T., & Graziano, M. S. A. (2003). Complex movements evoked by microstimulation of the ventral intraparietal area. Proceedings of the National Academy of Sciences, 100(10), 6163–6168. https://doi. org/10.1073/pnas.1031751100 Cuadra, C., & Latash, M. L. (2019). Exploring the concept of Iso-Perceptual Manifold (IPM): A study of fnger force-matching tasks. Neuroscience, 401, 130–141. https://doi.org/10.1016/j.neuroscience.2019.01.016 d’Avella, A., & Bizzi, E. (2005). Shared and specifc muscle synergies in natural motor behaviors. Proceedings of the National Academy of Sciences USA, 102(8), 3076–3081. https://doi.org/10.1073/pnas.0500199102 d’Avella, A., Saltiel, P., & Bizzi, E. (2003). Combinations of muscle synergies in the construction of a natural motor behavior. Nature Neuroscience, 6(3), 300–308. https://doi.org/10.1038/nn1010 Dawes, J. H. P. (2016). After 1952: The later development of Alan Turing’s ideas on the mathematics of pattern formation. Historia Mathematica, 43(1), 49–64. https://doi.org/10.1016/j.hm.2015.03.003 Deschênes, M., Moore, J., & Kleinfeld, D. (2012). Snifng and whisking in rodents. Current Opinion in Neurobiology, 22(2), 243–250. https://doi.org/10.1016/j. conb.2011.11.013 Desmurget, M., Richard, N., Harquel, S., Baraduc, P., Szathmari, A., Mottolese, C., & Sirigu, A. (2014). Neural representations of ethologically relevant hand/mouth synergies in the human precentral gyrus. Proceedings of the National Academy of Sciences, 111(15), 5718–5722. https://doi.org/10.1073/pnas.1321909111 Ethier, C., Brizzi, L., Darling, W. G., & Capaday, C. (2006). Linear summation of cat motor cortex outputs. Journal of Neuroscience, 26(20), 5574–5581. https:// doi.org/10.1523/JNEUROSCI.5332-05.2006 Fajen, B. R. (2005). Perceiving possibilities for action: On the necessity of calibration and perceptual learning for the visual guidance of action. Perception, 34(6), 717–740. https://doi.org/10.1068/p5405

264

Madhur Mangalam, Louise Barrett, and Dorothy M. Fragaszy

Gibson, J. J. (1976). The myth of passive perception: A reply to Richards. Philosophy and Phenomenological Research, 37(2), 234–238. https://doi. org/10.2307/2107194 Gibson, J. J. (1979). The Ecological Approach to Visual Perception. Houghton Mifin. Graziano, M. S. A. (2006). The organization of behavioral repertoire in motor cortex. Annual Review of Neuroscience, 29(1), 105–134. https://doi.org/10.1146/ annurev.neuro.29.051605.112924 Graziano, M. S. A. (2016). Ethological action maps: A paradigm shift for the motor cortex. Trends in Cognitive Sciences, 20(2), 121–132. https://doi.org/10.1016/j. tics.2015.10.008 Graziano, M. S. A., & Afalo, T. N. (2007). Mapping behavioral repertoire onto the cortex. Neuron, 56(2), 239–251. https://doi.org/10.1016/j.neuron.2007.09.013 Graziano, M. S. A., & Cooke, D. F. (2006). Parieto-frontal interactions, personal space, and defensive behavior. Neuropsychologia, 44(6), 845–859. https://doi. org/10.1016/j.neuropsychologia.2005.09.009 Graziano, M. S. A., Cooke, D. F., Taylor, C. S. R., & Moore, T. (2004). Distribution of hand location in monkeys during spontaneous behavior. Experimental Brain Research, 155(1), 30–36. https://doi.org/10.1007/s00221-003-1701-4 Graziano, M. S. A., Taylor, C. S. R., & Moore, T. (2002). Complex movements evoked by microstimulation of precentral cortex. Neuron, 34(5), 841–851. https://doi.org/10.1016/S0896-6273(02)00698-0 Harrison, T. C., Ayling, O. G. S., & Murphy, T. H. (2012). Distinct cortical circuit mechanisms for complex forelimb movement and motor map topography. Neuron, 74(2), 397–409. https://doi.org/10.1016/j.neuron.2012.02.028 Heft, H. (1989). Afordances and the body: An intentional analysis of Gibson’s ecological approach to visual perception. Journal for the Theory of Social Behaviour, 19(1), 1–30. https://doi.org/10.1111/j.1468-5914.1989.tb00133.x Heft, H. (2017). Perceptual information of “an entirely diferent order”: The “cultural environment” in The Senses Considered as Perceptual Systems: Ecological Psychology, 29(2), 122–145. https://doi.org/10.1080/10407413.2017.1297187 Heft, H. (2018). Places: Widening the scope of an ecological approach to perception—Action with an emphasis on child development. Ecological Psychology, 30(1), 99–123. https://doi.org/10.1080/10407413.2018.1410045 Jacobs, D. M., & Michaels, C. F. (2002). On the apparent paradox of learning and realism. Ecological Psychology, 14(3), 127–139. https://doi.org/10.1207/ S15326969ECO1403_2 Jacobs, D. M., Michaels, C. F., & Runeson, S. (2000). Learning to perceive the relative mass of colliding balls: The efects of ratio scaling and feedback. Perception & Psychophysics, 62(7), 1332–1340. https://doi.org/10.3758/BF03212135 Kayed, N. S., & van der Meer, A. L. H. (2007). Infants’ timing strategies to optical collisions: A longitudinal study. Infant Behavior and Development, 30(1), 50–59. https://doi.org/10.1016/j.infbeh.2006.11.001 Kelty-Stephen, D. G., & Dixon, J. A. (2014). Interwoven fuctuations during intermodal perception: Fractality in head sway supports the use of visual feedback in haptic perceptual judgments by manual wielding. Journal of Experimental

From Turing to Gibson

265

Psychology: Human Perception and Performance, 40(6), 2289–2309. https:// doi.org/10.1037/a0038159 Kelty-Stephen, D. G., Lee, I. C., Carver, N. S., Newell, K. M., & Mangalam, M. (2021). Multifractal roots of suprapostural dexterity. Human Movement Science, 76, 102771. https://doi.org/10.1016/j.humov.2021.102771 Kelty-Stephen, D. G., & Mangalam, M. (2022). Turing’s cascade instability supports the coordination of the mind, brain, and behavior. Neuroscience & Biobehavioral Reviews, 141, 104810. https://doi.org/10.1016/j.neubiorev.2022.104810 Kingma, I., van de Langenberg, R., & Beek, P. J. (2004). Which mechanical invariants are associated with the perception of length and heaviness of a nonvisible handheld rod? Testing the inertia tensor hypothesis. Journal of Experimental Psychology: Human Perception and Performance, 30(2), 346–354. https://doi. org/10.1037/0096-1523.30.2.346 Konczak, J., Meeuwsen, H. J., & Cress, M. E. (1992). Changing afordances in stair climbing: The perception of maximum climbability in young and older adults. Journal of Experimental Psychology: Human Perception and Performance, 18(3), 691–697. https://doi.org/10.1037/0096-1523.18.3.691 Krakauer, J. W., Ghazanfar, A. A., Gomez-Marin, A., MacIver, M. A., & Poeppel, D. (2017). Neuroscience needs behavior: Correcting a reductionist bias. Neuron, 93(3), 480–490. https://doi.org/10.1016/j.neuron.2016.12.041 Mangalam, M., Carver, N. S., & Kelty-Stephen, D. G. (2020a). Global broadcasting of local fractal fuctuations in a bodywide distributed system supports perception via efortful touch. Chaos, Solitons & Fractals, 135, 109740. https://doi. org/10.1016/j.chaos.2020.109740 Mangalam, M., Carver, N. S., & Kelty-Stephen, D. G. (2020b). Multifractal signatures of perceptual processing on anatomical sleeves of the human body. Journal of the Royal Society Interface, 17(168), 20200328. https://doi.org/10.1098/ rsif.2020.0328 Mangalam, M., Chen, R., McHugh, T. R., Singh, T., & Kelty-Stephen, D. G. (2020). Bodywide fuctuations support manual exploration: Fractal fuctuations in posture predict perception of heaviness and length via efortful touch by the hand. Human Movement Science, 69, 102543. https://doi.org/10.1016/j. humov.2019.102543 Mangalam, M., Conners, J. D., Kelty-Stephen, D. G., & Singh, T. (2019). Fractal fuctuations in muscular activity contribute to judgments of length but not heaviness via dynamic touch. Experimental Brain Research, 237(5), 1213–1216. https://doi.org/10.1007/s00221-019-05505-2 Mangalam, M., Conners, J. D., & Singh, T. (2019). Muscular efort diferentially mediates perception of heaviness and length via dynamic touch. Experimental Brain Research, 237(1), 237–246. https://doi.org/10.1007/s00221-018-5421-1 Mangalam, M., & Kelty-Stephen, D. G. (2020). Multiplicative-cascade dynamics supports whole-body coordination for perception via efortful touch. Human Movement Science, 70, 102595. https://doi.org/10.1016/j.humov.2020.102595 Mangalam, M., Pacheco, M. M., Izar, P., Visalberghi, E., & Fragaszy, D. M. (2018). Unique perceptuomotor control of stone hammers in wild monkeys. Biology Letters, 14(1), 20170587. https://doi.org/10.1098/rsbl.2017.0587

266

Madhur Mangalam, Louise Barrett, and Dorothy M. Fragaszy

Mangalam, M., Rein, R., & Fragaszy, D. M. (2018). Bearded capuchin monkeys use joint synergies to stabilize the hammer trajectory while cracking nuts in bipedal stance. Proceedings of the Royal Society B: Biological Sciences, 285(1889), 20181797. https://doi.org/10.1098/rspb.2018.1797 Michaels, C. F., Arzamarski, R., Isenhower, R. W., & Jacobs, D. M. (2008). Direct learning in dynamic touch. Journal of Experimental Psychology: Human Perception and Performance, 34(4), 944–957. https://doi. org/10.1037/0096-1523.34.4.944 Overduin, S. A., D’Avella, A., Carmena, J. M., & Bizzi, E. (2012). Microstimulation activates a handful of muscle synergies. Neuron, 76(6), 1071–1077. https:// doi.org/10.1016/j.neuron.2012.10.018 Palatinus, Z., Dixon, J. A., & Kelty-Stephen, D. G. (2013). Fractal fuctuations in quiet standing predict the use of mechanical information for haptic perception. Annals of Biomedical Engineering, 41(8), 1625–1634. https://doi.org/10.1007/ s10439-012-0706-1 Palatinus, Z., Kelty-Stephen, D. G., Kinsella-Shaw, J., Carello, C., & Turvey, M. T. (2014). Haptic perceptual intent in quiet standing afects multifractal scaling of postural fuctuations. Journal of Experimental Psychology: Human Perception and Performance, 40(5), 1808–1818. https://doi.org/10.1037/a0037247 Parada, F. J., & Rossi, A. (2018). If neuroscience needs behavior, what does psychology need? Frontiers in Psychology, 9, 433. https://doi.org/10.3389/ fpsyg.2018.00433 Pattee, H. H. (2001). The physics of symbols: Bridging the epistemic cut. Biosystems, 60(1), 5–21. https://doi.org/10.1016/S0303-2647(01)00104-6 Pattee, H. H. (2013). Epistemic, evolutionary, and physical conditions for biological information. Biosemiotics, 6(1), 9–31. https://doi.org/10.1007/ s12304-012-9150-8 Profeta, V. L. S., & Turvey, M. T. (2018). Bernstein’s levels of movement construction: A contemporary perspective. Human Movement Science, 57, 111–133. https://doi.org/10.1016/j.humov.2017.11.013 Runeson, S. (1988). The distorted room illusion, equivalent confgurations, and the specifcity of static optic arrays. Journal of Experimental Psychology: Human Perception and Performance, 14(2), 295–304. https://doi. org/10.1037/0096-1523.14.2.295 Stepniewska, I., Fang, P.-C., & Kaas, J. H. (2005). Microstimulation reveals specialized subregions for diferent complex movements in posterior parietal cortex of prosimian galagos. Proceedings of the National Academy of Sciences, 102(13), 4878–4883. https://doi.org/10.1073/pnas.0501048102 Stepniewska, I., Fang, P.-C. Y., & Kaas, J. H. (2009). Organization of the posterior parietal cortex in galagos: I. Functional zones identifed by microstimulation. Journal of Comparative Neurology, 517(6), 765–782. https://doi.org/10.1002/ cne.22181 Stepniewska, I., Friedman, R. M., Gharbawie, O. A., Cerkevich, C. M., Roe, A. W., & Kaas, J. H. (2011). Optical imaging in galagos reveals parietal—Frontal circuits underlying motor behavior. Proceedings of the National Academy of Sciences, 108(37), E725–E732. https://doi.org/10.1073/pnas.1109925108

From Turing to Gibson

267

Turing, A. M. (1936). On computable numbers, with an application to the entscheidungsproblem. Proceedings of the London Mathematical Society, 2(42), 345–363. Turing, A. M. (1950). I.—Computing machinery and intelligence. Mind, 59(236), 433–460. Turing, A M. (1952). The chemical basis of morphogenesis. Philosophical Transactions of the Royal Society B: Biological Sciences, 237(641), 37–72. https://doi. org/10.1007/BF02459572 Turvey, M. T. (2007). Action and perception at the level of synergies. Human Movement Science, 26(4), 657–697. https://doi.org/10.1016/j.humov.2007.04.002 Turvey, M. T., Burton, G., Amazeen, E. L., Butwill, M., & Carello, C. (1998). Perceiving the width and height of a hand-held object by dynamic touch. Journal of Experimental Psychology: Human Perception and Performance, 24(1), 35–48. https://doi.org/10.1037/0096-1523.24.1.35 Turvey, M. T., & Carello, C. (2011). Obtaining information by dynamic (efortful) touching. Philosophical Transactions of the Royal Society of London B: Biological Sciences, 366(1581), 3123–3132. https://doi.org/10.1098/rstb.2011.0159 Tytell, E. D., Holmes, P., & Cohen, A. H. (2011). Spikes alone do not make behavior: Why neuroscience needs biomechanics. Current Opinion in Neurobiology, 21(5), 816–822. https://doi.org/10.1016/j.conb.2011.05.017 Wagman, J. B., McBride, D. M., & Trefzger, A. J. (2008). Perceptual experience and posttest improvements in perceptual accuracy and consistency. Perception & Psychophysics, 70(6), 1060–1067. https://doi.org/10.3758/PP.70.6.1060 Warren, W. H. (1984). Perceiving afordances: Visual guidance of stair climbing. Journal of Experimental Psychology: Human Perception and Performance, 10(5), 683–703. https://doi.org/10.1037/0096-1523.10.5.683 Warren, W. H., & Whang, S. (1987). Visual guidance of walking through apertures: Body-scaled information for afordances. Journal of Experimental Psychology: Human Perception and Performance, 13(3), 371–383. https://doi. org/10.1037/0096-1523.13.3.371 Wells, A. J. (2002). Gibson’s afordances and Turing’s theory of computation. Ecological Psychology, 14(3), 140–180. https://doi.org/10.1207/ S15326969ECO1403_3 Wells, A. J. (2017). Rethinking Cognitive Computation: Turing and the Science of the Mind. Bloomsbury. Withagen, R., & Chemero, A. (2012). Afordances and classifcation: On the signifcance of a sidebar in James Gibson’s last book. Philosophical Psychology, 25(4), 521–537. https://doi.org/10.1080/09515089.2011.579424

PART IV

Applications of the Ecological Theory of Afordances

15 UNDERSTANDING SKILLED ADAPTIVE BEHAVIOR The Role of Action, Perception, and Cognition in an Ecological Dynamics Perspective Ludovic Seifert, Duarte Araújo, and Keith Davids

What Do We Understand by the Term Afordance?

Gibson (1979) proposed that humans perceive action possibilities or afordances ofered by the environment. His ideas imply that, in sports, performers perceive objects, surfaces, or events by what they may ofer an athlete in terms of opportunities for action. In ecological psychology, afordance is not an independent entity but a property of the environment related to the organism that results in the emergence of behaviors. For example, environmental surfaces ofer diferent actions for diferent people due to, among other constraints, their distinct physical properties, such as limb lengths (see Fajen et al., 2009, for critical features of afordances discussed in the context of sports). For instance, does a surface aford stepping on with the feet or climbing on with the legs and arms (in the case of a young child), given the key physical properties of the individual? Perhaps the composition and layout of surfaces constitute what they aford. If so, to perceive them is to perceive what they aford. This is a radical hypothesis, for it implies that the “values” and “meanings” of things in the environment can be directly perceived. —James J. Gibson (1979, p. 127) For Gibson (1979), afordance is a central concept of psychology, which captures the complementary relationship of an organism with its environment. Gibson pointed out a fundamental criticism of indirect theories of perception, stating that “psychologists assume that objects are composed DOI: 10.4324/9781003396536-20

272

Ludovic Seifert, Duarte Araújo, and Keith Davids

of their qualities . . . what we perceive when we look at objects are their afordances, not their qualities” (p.  134). Perceiving the environment in terms of afordances leaves dispensable those intermediate processes required to transform action-independent perceptions into action-oriented perceptions. In the process of (direct) perception, there is no integration and combination of cues involved. The primary objects of an individual’s perceptual experience are the action possibilities that the environment afords. Those action possibilities ofered by the environment are relative to an individual’s properties (e.g., body characteristics) and abilities. For example, Warren (1984) found that the maximum riser height of a step that was perceived to be climbable was 0.88 proportion of adult participants’ leg length and that this ratio was invariant between tall and short adults. Warren, therefore, concluded that the ability to perform an action such as stepping onto an obstacle is body-scaled. Further to the seminal work of Warren, the study of body-scaled afordances has been shown to form the basis of a range of behaviors, including reaching (Carello et al., 1989), sitting (Mark et  al., 1990), and aperture perception (Warren & Whang, 1987). Building on the concept of body-scaled afordances, Warren (1984) proposed a more functional conceptualization of afordances being actionscaled. This more functional proposal signifes that afordances are dynamically predicated on the action capabilities of an individual relative to the environment in which an action is performed (Fajen, 2005). For instance, Konczak et  al. (1992) emphasized that afordance perception for step height is best understood as scaling relative to leg fexibility and strength (i.e., action-scaled relative to an individual at any moment) rather than leg length (i.e., body-scaled). Given that afordances refect the relational nature of many properties of organisms and many properties of environments, they are proposed to be nested in the context of other afordances (Wagman & Stofregen, 2020). According to Wagman et al. (2016), nested afordances refer to the multiple afordances that are available any given performance context, with a consequent obligation to choose among them or order them. Afordances are nested over several diferent spatial and temporal scales (Wagman et al., 2016; Wagman & Morgan, 2010) and can be exploited sequentially, simultaneously, or in parallel (i.e., simultaneously but independently of one another; Mark et al., 2015). Contemporary perspectives have subsequently proposed that skilled athletic experience infuences the perception of nested afordances (Higuchi et al., 2011; Hove et al., 2006; Weast et al., 2011). van Dijk and Rietveld (2021) suggested that nested afordances invite skilled performers to act further as one situation develops into the next via these embedded invitations. For instance, in a climbing task, Seifert et al. (2021a) showed that skilled climbers use a handhold as a transition

Understanding Skilled Adaptive Behavior

273

(to chain movements) rather than as an isolated event. In skilled climbers, multiple holds are collectively perceived as a single nested climbing opportunity to transit vertically (Boschker et al., 2002; Seifert et al., 2017, 2021b). Indeed, evidence shows that expert climbers better perceive the functional properties of holds than less experienced climbers. Conversely, less experienced tend to exploit the structural properties of individual holds, leading to an emerging hold-by-hold climbing sequence instead of an interconnected chain of movements (Boschker et al., 2002; Boschker & Bakker, 2002). To exemplify, when beginners performed a climbing task in a learning program of 13 sessions, results showed that the route aforded more learning opportunities as their visual-motor exploration became more efcient (Hacques et al., 2021). However, when skill transfer was assessed on three new routes (varying either in distance between holds, i.e., between 15 and 20 cm; hold orientation, i.e., holds were turned by 90°; or varied in shape; Figure 15.1), results suggested that learners transferred their skill to the route with an increased distance between handholds, but not to the other two routes. In fact, the “distance”-manipulated route induced lowerorder behavioral changes as learners needed to recalibrate a similar hand

FIGURE 15.1

Illustration of the handholds characteristics for the control route of the learning protocol and for the three routes of the transfer test. The arrows indicate the preferential grasping enabled by the handhold. Reproduced with permission from Hacques et al. (2021).

274

Ludovic Seifert, Duarte Araújo, and Keith Davids

grasp. In contrast, the “orientation”- and “shape”-manipulated routes induced higher-order behavioral changes, since those routes required different grasping patterns and postural orientation, suggesting that learners did not transfer their calibration and that calibration is posture-specifc (as observed in motor development studies by Adolph et al., 2008; Kretch and Adolph, 2013). By avoiding a mechanistic conceptualization of afordances (i.e., afordances can constrain behaviors but do not cause them; Gibson, 1979), ecological psychology emphasizes the important role of individuals to self-regulate their behaviors, with respect to the environment, in achieving intended task goals. These ideas shape our understanding of individual diferences in afordances that people may use and explain why people can accept or reject afordances available to them invitationally (Withagen et al., 2012). What Role Does Afordance Play in Perception? Is It the Entity Organisms Perceive or the Means Through Which Organisms Perceive? Are Afordances the Only Perceptual Dependent Variables?

Since the ecological dynamics perspective construes goal-directed behaviors in sports as emanating from the hard-assembled (physical) and softassembled (informational) links between performers and their performance environments (Kugler & Turvey, 1987), processes of cognition, perception, and action can be captured by eco-physical variables (Araújo & Davids, 2018). These variables express the ft between the physics of the environment and a performer’s continuous behavioral adaptations. Environmental properties directly inform what an individual can and cannot do (Withagen et al., 2012, 2017). Therefore, the emphasis on eco-physical variables avoids a representational tendency to search for variables inside the organism that are deemed to mechanistically “cause” behaviors. It sidesteps the organismic asymmetry needed in explaining the brain driving cognition, perception, and action through neural or symbolic activity (Davids & Araújo, 2010). Afordances not only are perceptual variables but entail action components, emphasizing that perception and action cannot be separated. Perception-action coupling obliges mutuality and reciprocity between an individual and the environment, with the individual-environment system being the most appropriate unit of analysis captured by eco-physical variables. An in situ example is the eco-physical variable GDD (goal-directed displacement) index developed to study dyadic system behaviors in tennis rallies (Carvalho et al., 2014). Using eco-physical variables in research

Understanding Skilled Adaptive Behavior

275

and practice enables understanding of how cognitive processes might be predicated on continuous, emergent, performer-environment interactions in sports. In particular, athlete decision-making can be analyzed with ecophysical variables, which entails that behaviors can be understood as selforganizing, emerging from the continuous interactions of environmental, individual, and task constraints. From the performer’s point of view, the task is to exploit surrounding physical (e.g., feld characteristics as determined by sports rules) and informational (e.g., the movement of the ball and other players) constraints to continuously (re)organize behaviors. Constraints reduce the number of confgurations available to a dynamical movement system at any time. In a performance environment, behavior patterns emerge to satisfy constraints as less functional organization states are dissipated. Changes in performance constraints can lead a system toward bifurcation points where choices emerge as more specifc information becomes available, ever constraining the environment-athlete system to switch to a more and more functional path of behavior (such as running with the ball into a larger gap on the feld, rather than another which is smaller; Araújo et al., 2006). An account of afordances inspired by the philosophical ideas of Wittgenstein views them as embedded in socio-cultural practices of an organization, club, community, or society, for example, with performers being historically dependent on socio-cultural skills to use specifc afordances that are always available but selectively used (Rietveld & Kiverstein, 2014). Thus, afordances exist in a performance environment but are selectively used, dependent on one’s expertise. Therefore, expert cognitive and perceptual activity can be considered selective engagement with an afordance available in a rich landscape. This idea can be exemplifed by footballers being adaptive enough to pick up and utilize afordances for action in different performance contexts. When a performer changes from one action mode in team games (running with the ball) to another (passing the ball when a defender is approaching to tackle), a transition in the course of action is expressed. Transitions among stable behavioral states (i.e., action modes) emerge due to dynamic instabilities, providing a universal decisionmaking process for switching between distinct patterns (Kelso, 1995). Such stabilities and instabilities in a performance environment do not exist a priori in the structure of the player or the surrounds. However, they are co-determined by the confuence of constraints and information, exemplifying how control lies in the performer—environment system. Emergent behavioral patterns can be formally modeled using diferential equations and potential functions to describe the long-range dynamics underlying interactions of system components. From this viewpoint, athletes search in a sports performance context to arrive at a stable, functional solution. A

276

Ludovic Seifert, Duarte Araújo, and Keith Davids

viable option selected is the strongest attractor for an individual at a given moment, with other options having less strength of attraction. This selection only emerges from the continuous interactions of an individual and a performance environment. Ignoring other options is a dynamical consequence since if a system relaxes to one attractor, it concomitantly ignores the remaining options (attractors). The fact that there is a stronger attractor does not eliminate the infuence of other attractors in the sports context (e.g., Araújo et al., 2013). Under dynamic performance conditions, other attractors (i.e., other options) may emerge and exert their attraction in the system’s dynamics, emphasizing the performer-environment reciprocity. On pages 138–139 of his book, Gibson (1979) considered choice behavior more broadly as the environment consisting of afordances of objects, surfaces, events, and other people that have value or not for each individual (and this can change over time for the same individual. A great example is given of a letterbox having an attraction for an individual when he/she has a letter to post and lives in an environment where there is a postal system, but not when there is no letter to be posted. In emphasizing the value and meaning of things, afordances can repel or invite actions. These ideas form the basis of the person-environment interactions that afordances underpin. The metaphor of a feld is helpful here: some afordances stand out more than others in a whole landscape of afordances that are available to be utilized. Bruineberg and Rietveld (2014) distinguished a feld from a landscape of afordances: A landscape of afordances corresponds to the “afordances available in an ecological niche” (p. 2), which relate to the whole spectrum of abilities available in our socio-cultural practices. A feld of afordances relates to the “afordances that stand out as relevant for a particular individual in a particular situation; i.e., the multiplicity of afordances that solicit the individual” (Bruineberg & Rietveld, 2014, p. 2). Some afordances are experienced as soliciting immediately, others are experienced as soliciting on the horizon, and still, others are entirely ignored (only the latter leave us cold). We can distinguish between an afordance, that is, a possibility for action available at a specifc location, and a solicitation. A solicitation is an afordance that stands out as relevant in a specifc situation lived by an animal. A solicitation is the experiential equivalent of a bodily action readiness: the readiness of the afordance-related ability (Rietveld, 2008). In deriving this conceptualization, Gibson (1979) drew upon Kurt Lewin’s concept of valences as pushing an agent either toward or away from an object, event, or another person. The value of an object perceived by an individual agent is changed by the experience and a person’s needs. The afordance is not changed, but the value or meaning changes for each person-environment relationship. The value and meaning of an afordance will change as a function of an individual’s efectivities and needs. So, skills,

Understanding Skilled Adaptive Behavior

277

capacities, and experience will infuence whether an afordance is more or less soliciting for an individual, especially when understood with respect to their needs in dynamic performance contexts like sports. Relatedly, Withagen et al. (2017) sketched a dynamical model of the agent-environment relationship where the agency is conceptualized as the capacity to modulate the coupling strength with the environment. This model explained how the agent could intentionally infuence how they are infuenced by the diferent afordances, which may be more or less soliciting. By modulating the coupling strength, the agent simply alters the dynamics of the performer-environment system, thus shaping the behavior that emerges. This model opens to ecological dynamics the challenge of understanding how a performer’s changes (e.g., psychological, biological) infuence their intentions and the modulation of the coupling strength with the environment. Following the same logic, it opens the possibility to understand how immediate changes in an environment (e.g., social, normative, rule-based, task, implements) also exert constraints on the modulation of the coupling strength with the performer through diferences in the solicitation. Why Is Afordance Any Better Than Stimulus? What Does a Theory of Afordance Suggest That Stimuli Cannot; How Has It Moved the Needle Past Gibson 1960’s Recognition of How Little We Can Defne Stimuli?

A theory of “stimulus” implies that the stimulus conditions a response. This relation implies automaticity through conditioning, whereas an afordance implies an invitation (Withagen et al., 2012), highlighting the importance of agency incorporating decision-making and problem-solving, not separately from action, and framed by intentionality. To explain how humans and other animals establish perceptual contact with their environment, Gibson (1966) introduced a radical new theory of the “stimulus information available to the receptors.” He argued that the assumption of impoverished stimulus information, on which most inferential theories of perception are based, is fallacious. Traditional information-processing theories argue that, for perceivers to be informed about sources of stimulation, they must store a large set of alternatives in memory that are selected according to those stimulating conditions. In other words, stimulation of the receptors does not directly inform about its sources. For example, one cannot know from the raw energy of the light ray itself whether its source is near or distant. Receptors in an eye stimulated by a ray of light can only be informed of the presence of the ray of light. For stimulation to act informatively, a perceiver may be said to have some means of interpreting or decoding such stimulation according to a limited set of memorized possibilities.

278

Ludovic Seifert, Duarte Araújo, and Keith Davids

Contrary to this classical viewpoint, Gibson argued for a very diferent conceptualization of information that he considered was a more appropriate way to understand perception and cognition among living organisms. From this ecological view, the structure of light, for example, could be intrinsically informative. Gibson’s theory implies that sensitivity to information structure must exist in a perceiver but that there does not have to be an added interpretation process (Richardson et al., 2008). For Gibson (1979, 1966), perceptual information resides in the ambient energy arrays and fows and can directly inform an individual about the environmental properties of our world. Notably, Gibson also stressed the importance of an individual’s movement in detecting and discovering specifying information. In his view, perceivers are rarely passive: to detect information; perceptual systems scan the ambient arrays to detect information actively. Furthermore, by moving, performers can create spatiotemporal energy patterns specifc to environmental properties. The moving body of an (embodied) observer contributes to the structure of an array at any observed place. Because performers exploit spatiotemporal energy patterns specifc to the to-be-perceived or to-be-acted-upon properties of the environment, inferential processes are dispensable, and direct perceptual contact with a performance environment is possible. Since Gibson wanted to understand perception of the environment, instead of working with primitives such as points, lines, planes, and projections, he began with the ambient optic array. The ambient optic array is structured light surrounding a point of observation. It consists of multiple refected light rays flling a medium (in this case, the air). This physical characterization means that there must be light sources, refecting surfaces, and a medium. The same set of refecting surfaces that make the lightflled medium possible also accounts for diferences in intensity in diferent directions from any point of observation. These diferences exist by diferences in arrangement (layout) relative to one another and the illumination, diferences in texture, and diferences in pigment structure. In the case of light, the optic array (i.e., information) just exists; it is not coming to the eye. Gibson (1979) referred to the fact that animals with compound eyes, as well as animals with chambered eyes, show visually guided behavior (such as avoidance behaviors in the presence of an expanding shadow specifying a looming object) as evidence that they are designed to use optical information in the array, even though they have no retinal images. Thus, retinal images are not necessary for vision, but the information is (Mace, 1986). Gibson maintained that such optical changes (variants) and non-changes (invariants) in the surrounding array are used by perceivers to regulate their movements relative to the dynamics of the environment. As a structure surrounding a performer, patterned ambient energy (e.g.,

Understanding Skilled Adaptive Behavior

279

optic array, sound waves) can be explored or observed actively. When this occurs, the patterned energy fow changes in some ways, but not all. For example, an important way that the optical array of an idealized frozen environment changes occurs when an observer moves. Everything in the array fows, and there are regular exchanges of array components revealed and hidden. But within the fow, some relations of components to one another stay invariant. These invariants specify (i.e., inform an individual about) stable features of the environment. The changes, or variants, specify the observer’s movement relative to the stable environment. Because invariants are defned only with respect to variants, it follows that change is necessary to reveal non-change. It also follows that coordinated movements of an observer can be specifed only relative to the invariant structure of the surrounding energy arrays (Mace, 1986). Importantly, variants and invariants of the ambient optic array allow for the separation of that which belongs to the environment from that which belongs to the performer. As the performer’s point of view changes, one can perceive when the point of view changes and when the environment is changing. When the point of view changes in a stable environment, the persistence of that environment is specifed by invariants (Gibson, 1979). A consequence of the optics of occlusion is that as one uncovers new surfaces by exploration, one is extending the amount of connected, concurrently existing surfaces that one has detected. One of the most closely studied invariants in the informational array is an optic variable known as tau, which has been proposed to specify the time to contact (Tc) information (Lee, 2009). Since its conceptualization and operationalization by David Lee in the 1970s to the 1980s, it has generated a signifcant amount of research and debate in the literature. When a non-deforming object, such as a ball, moves toward an observer, the contours of the ball form a solid visual angle. This optic fow is perceived over time as a nested hierarchy of solid angles that expands symmetrically on approach to the observer. The optical information specifed by the relative rate of expansion of the solid visual angle enclosed by these contours is tau. Tau provides the individual with direct information about the remaining time Tc between the point of observation and an approaching object (Lee, 1976). Assuming that an approaching object has a constant velocity and is traveling toward the point of observation, there are considerable advantages of being able to perceive Tc. For instance, the individual does not need to indirectly process ambiguous cues on distance, object size, and velocity and perform mental calculations to compute the object’s time of arrival. Not needing to rely on indirect processing of cues is particularly benefcial when performing dynamic activities such as playing tennis. In these activities, performers may have little time to react and need not spend

280

Ludovic Seifert, Duarte Araújo, and Keith Davids

time calculating the time of arrival of an object or surface while interpreting events in the environment. Travassos et  al. (2012) developed an interesting application of the tau concept to study team sports performance. They studied how Futsal defenders intercept the trajectory of a ball (passed between attackers) by comparing how they regulated their actions using critical information sources, such as time to intercept the ball, in successful and unsuccessful interceptions of a pass. Time to ball interception was measured via the diference between the time of each defender to reach an interception point in ball trajectory and the time of the ball’s arrival at the same interception point. Results showed positive values of time-to-ball interception when passes were not intercepted and negative to zero values when passes were intercepted. Furthermore, analysis of defenders’ adaptations to the visual information available in the environment revealed that continuous changes in participants’ movement velocities toward the ball constrained their exploitation of these informational variables and the success of their interceptive actions. What Is the Connection Between Afordances, Behavioral Scale, and Intention? How Distant or Reluctant Can a PerceivingActing System Be Before Afordances Come Into Play? Do Afordances Exert Their Role Only When the System Is Close or Willing Enough?

Afordances are inviting or soliciting but do not compel individuals to behave in a certain way. Behavior not only is composed of the action we observe but also incorporates perception, emotions, cognitions, intentions, and social dimensions (e.g., shared afordances in collective task performance). Intentions are not causes of action but constraints on the action; they are not mental as opposed to physical but are embodied in the kinds of performance behaviors typically observed in performers. Reed (1993) argued that these intentional organizational patterns emerge in situations where diferent afordances can be utilized to enhance performance in contexts like sports. This performer-environment basis of conceptualizing behavior indicates that afordances can be used, motivating an organism to act, but they are not to be viewed as causes of behavior because a person may not act on a perceived afordance (Withagen et al., 2017). Adaptive behavior can emerge continuously from the confuence of constraints under the boundary conditions of intentions embedded within task goals. Perceiving is an activity of the whole body acting on and in the environment to obtain information (Gibson, 1966; Reed, 1996) and requires coordinated movement. For example, the legs function by bringing one perceptually

Understanding Skilled Adaptive Behavior

281

guided way to a place where nested adjustments of head turning, eye movements, lens accommodation, hand positioning, fnger movement, and so forth can perform their functions in a coordinated act. Perceiving is guided by the practical requirements of a person’s intended goals, achievements, and circumstances. A person has to perceive enough of the environment to accomplish intentional goals, but that’s all. There are no right or wrong perceptions. Moreover, perceiving can improve. If there is always more structure that can be clarifed with more exploration, then the possibility for enhanced perceiving is always present. Perceiving involves guiding the complex adjustments of the body and performing goal-directed activities in the environment (Mace, 1986). Perception, like action, is a coordinated activity, an achievement of the performer-environment system (Guignard et al., 2017). Swimming is an excellent example to consider, as any displacement in the water is highly dependent on fuid characteristics. Swimmers use these properties to propel themselves through the water in diferent ways. Traditionally, biomechanical research has focused on swimmers’ actions without exploring perturbations to fuid fows. Conversely, fuid mechanics research has sought to record fuid behaviors, isolated from how the swimmers cope with environmental constraints (e.g., in some experiments, fuid fows are passively studied on mannequins or robot efectors). When a swimmer moves in the water, they create continuous specifc changes in fuid motions, providing action information. When swimming in outdoor aquatic environments (such as oceans, rivers, and lakes), swimmers can gain information on fuid motion when interacting with waves, rips, and currents to decide how to exploit aquatic fows to move in context (Guignard et  al., 2020). For Gibson (1979), movement generates information which, in turn, supports further movements, leading to a cyclical relationship between information and movement. The interdependency between the detection of information and the generation of movement implies that perception and action cannot be studied separately (Araújo et al., 2006). Only in practice contexts where perception and action processes are expressed within representative environments may we observe the smartness of evolutionary-designed perceptual mechanisms (Davids et  al., 2012). In this context, smartness refers to the dedicated function for which a perceptual mechanism has evolved. This understanding of intentionality can be extended to team cognition and decision-making in a team sport. During competitive performance, the organization of action by perceiving surrounding informational constraints is expressed in “knowledge” of the environment. This idea emphasizes that the perception of shared afordances underpins the main communication channel between team members during team coordination tasks (Araújo &

282

Ludovic Seifert, Duarte Araújo, and Keith Davids

Davids, 2016). Predicated on the key ideas of Reed (1996), we argued that afordances are collective environmental resources that have existed prior to the upskilling of the individuals who came to perceive and use them. Ecological dynamics predict that the presence of others extends action possibilities that are realizable by individuals to action possibilities realizable by groups. Indeed, Gibson (1979) argued that “behavior afords behavior” (p. 135) and that it was important to note that the “richest and most elaborate afordances” (p.  135) of all are afordances of other people in social contexts. The suggestion is that afordances can be perceived by a group of individuals trained to become perceptually attuned to them (Silva et al., 2013). Through practice, players can become perceptually attuned to the afordances of others and afordances for others during the competitive performance. They can refne their actions (Fajen et al., 2009) by adjusting behaviors to functionally adapt to those of other teammates and opponents. Moreover, individuals in a team can act in a way to create competitive circumstances (the afordances) that are favorable to them. For example, by pressuring opponents in team games to play more in one zone of a playing area, this strategy can create afordances for attacking play by freeing other areas of the feld. These processes enable a group of players to act synergistically with respect to specifc performance circumstances. An important feature of a team synergy is the capacity of one individual (e.g., a player in a team) to infuence the behaviors of others (Araújo & Davids, 2016; Riley et al., 2011). Decisions and actions of players forming a localized synergy should not be viewed as independent, explaining how multiple players synchronize activities per dynamic performance environments in fractions of a second (Silva et al., 2016). The coupling of players, as independent degrees of freedom, into local synergies, is based upon perception-action systems functioning in a social context, supported by the collective perception of shared afordances (Silva et al., 2013). Ecological dynamics analyses of team sports have attempted to explain how interactions between players and information from the performance environment constrain patterns of stability, variability, and transitions in organizational states of such team synergies. Research has demonstrated that inherent degeneracy (i.e., fexible behavior, like “many” structures to “one” function relationship; Mason, 2010) in perception-action systems provides the basis for the diversity of actions required to negotiate information-rich, dynamic social environments toward a task goal (Seifert et al., 2016). For instance, the goal for icefall climbers is to anchor their ice axes to the ice surface, which could be achieved by “swinging” ice axes against the icefall to create specifc anchorages or by “hooking” the blade of the ice axes into existing holes in the icefall, supporting that expert climbers utilize

Understanding Skilled Adaptive Behavior

283

functionally equivalent movements (i.e., the degeneracy of perceptualmotor system) during the realization of afordances (Seifert et al., 2014). Research has demonstrated that inherent degeneracy (i.e., fexible behavior, like “many” structures to “one” function relationship; Mason, 2010) in perception-action systems provides the basis for the diversity of actions required to negotiate information-rich, dynamic social environments toward a task goal (Seifert et al., 2016). For instance, the goal for icefall climbers is to anchor their ice axes in the ice surface, which could be achieved by “swinging” ice axes against the icefall to create specifc anchorages or by “hooking” the blade of the ice axes into existing holes in the icefall, supporting that expert climbers utilize functionally equivalent movements (i.e., degeneracy of perceptual-motor system) during the realization of afordances (Seifert et al., 2014). How Do Systems Engage With Afordances as They Move Among Tasks and Intentions?

Tasks and intentions are not separated; in other words, tasks are not external, and intentions are not internal to an individual but strongly nested. Thus, considering an organism-environment system, afordances are not excluded from this system but are deeply part of it. The ecological dynamics approach stresses the primacy of performer-environment couplings by means of which problems, such as moving about, deciding with whom to create attachments, competing with adversaries, and so on, can be resolved in a performance environment. In this view, any system capable of performing such feats of behavior is a cognitive system. These problems can range from tasks derived from biology (e.g., goal-path decisions in a school of fshes) to abstract reasoning (e.g., deciding on the next move in a chess match), as long as it involves making a decision that solves the problem in an intelligent (functionally adaptive) way (Turvey & Carello, 2012). In the traditional view, the behavioral expression of those decisions is not at the heart of cognition because behavior is assumed to be a simple implementation of a mental plan. However, the regulation of behavior, as an expression of a cognitive process, should not be attributed to one part of the animal-environment system (i.e., the individual). Decision-making in activities like sports is an example of cognitive processes, which are embodied, situated, and dynamic. For ecological dynamics, decision-making behavior is defned as transitions in the course of action and is a central issue for intelligent performance (Araújo et al., 2006). Thus, a signifcant challenge is to understand the ability of each individual to perceive the surrounding layout of the performance environment on the scale of their body and action capabilities (Fajen et al., 2009). As a performer moves with respect

284

Ludovic Seifert, Duarte Araújo, and Keith Davids

to their surroundings, opportunities for action persist, emerge, and dissolve, even though the surroundings are analyzed as objects, and the relations among them remain stable. Subtle changes of action can give rise to multiple and marked variations in opportunities for subsequent actions. The dynamic process implied in the perception of afordances provides the basis by which a performer can control their behaviors prospectively. To interact meaningfully with the environment, performers must have complex interiors, an on-board (metabolic) potential capable of biogenic forces that may be used to cancel, modulate, or delay their immediate reaction to an external force (Araújo et  al., 2006; Kugler & Turvey, 1987). This way, the agency is the capacity to modulate the coupling strength with the environment because an agent can infuence to what extent they are infuenced by diferent afordances in a performance landscape (Withagen et  al., 2017). From this perspective, distinctive cognitive capabilities are what they are by laws and general principles (Profeta & Turvey, 2018), where dynamics (involving laws of motion and change) and dynamical systems (involving time evolution of observable quantities according to law) can ofer the tools to understand cognitive processes. For example, research about attacker-defender dyads in team sports has revealed that these agent interactions could also be described in low-dimensional order parameter dynamics (Araújo et al., 2006). Previous work by Schmidt et al. (1990) proposed that two individuals might be considered a single system. Using the same gradient equation as the model developed by Haken et al. (1985), this work has suggested that a dyadic synergy can reveal nonlinear properties, namely entrainment and sustained periodic behavior, and specifc modes of inter-personal coordination, which emerge under contextual, personal, and task constraints. Following this line of thinking, the study of the dynamics of attacker-defender dyads in team sports has shown that decision-making emerges from continuous interactions of team players (e.g., Araújo et al., 2013; Passos et al., 2008). In these social systems, attackers and defenders form closely interacting dyads in which individuals do not deliberately seek to coordinate actions since an attacker must avoid a stable dyadic organization with a marking defender. Cooperating and competing players do not share a standard neuronal system, so emergent coordination and transition phases are solely based on the task constraints present in specifc performance environments. Part of the attractiveness of such dynamic models is derived from the fact that they can explain diferent decisions using the same underlying process of originating and decaying attractors and without the need to use mental representations (see Araújo et al., 2013; Araújo & Davids, 2016). The second key idea is that tasks and intentions reveal afordances infuencing behaviors at diferent time scales because the task an individual

Understanding Skilled Adaptive Behavior

285

performs at the moment is temporary and nested in other tasks performed in parallel, feeding future tasks and intentions. Afordances favor certain behaviors and select against others (Withagen et al., 2012). The constraints underlying the tendency for favored behaviors to be realized multiply and thus are not only derived from the performer. However, the characteristics of the performer, such as their skills, are of paramount importance. Kiverstein and Rietveld (2015) defne skilled intentionality as “the individual’s selective openness and responsiveness to a rich landscape of afordances” (p. 701; Figure 15.2). This notion indicates that the everyday environment ofers a range of more or less inviting afordances (Withagen et al., 2012). However, these afordances are only accessible to individuals with the necessary skills to act on them. For example, where one tennis player with an excellent backhand shot may perceive an opportunity to penetrate an opponent’s court when using it, another player who is highly skilled at volleying may perceive every ball as an opportunity to approach the net. Thus, athletes interact with a surrounding environment through skilled engagement with the afordances that a performance environment ofers them because of

FIGURE 15.2

Afordances exist in a landscape surrounding individuals and soliciting their actions, as long as they have relevant efectivities that are functional for interacting with a specifc performance environment. Note that individuals can interact with diferent afordances (here captured by diferent shapes and sizes of symbols) in unique and individualized ways.

286

Ludovic Seifert, Duarte Araújo, and Keith Davids

their unique skills. From this viewpoint, the process of perceptual attunement brings an “openness” to afordances that, without skill, would not be accessible since it is a skill that opens possibilities for action to an individual. Importantly, subsequent actions are modulated by the individual, exerting their agency in the performer-environment system dynamics at appropriate points to yield a performance trajectory (Withagen et al., 2017). In turn, these dynamics are coupled to larger-scale dynamics, guiding the formation of the behavioral trajectory over longer time scales. Reciprocally, the longer-term dynamics could infuence the short-term interactions (thus highlighting specifc afordances), for example, by altering environmental conditions (Araújo et al., 2019). Because a behavioral trajectory is assembled anew on each occasion, the action sequence is historically contingent and variable, allowing for the fexibility observed in ordinary action sequences. As Profeta and Turvey (2018) rightly argued, the continuity of the ambient optic array and the prospective nature of perception are essential assumptions that would allow the performance of a sequence of movements to be accomplished with the task specifed in the act of perceiving. Wagman et al. (2016) showed empirical evidence for means–ends relations among nested afordances, where information arrangements about possible future states of afairs, once detected, constrain how states of afairs evolve. Acknowledgments

This project received the support of the French National Agency of Research (ID: ANR-17-CE38-0006 DynACEV and ANR-19-STHP-0004 NePTUNE). Reference List Adolph, K. E., Tamis-LeMonda, C. S., Ishak, S., Karasik, L. B., & Lobo, S. A. (2008). Locomotor experience and use of social information are posture specifc. Developmental Psychology, 44(6), 1705–1714. https://doi.org/10.1037/ a0013852 Araújo, D., & Davids, K. (2016). Team synergies in sport: Theory and measures. Frontiers in Psychology, 7, 1449. https://doi.org/10.3389/fpsyg.2016.01449 Araújo, D., & Davids, K. (2018). The (sport) performer-environment system as the base unit in explanations of expert performance. Journal of Expertise, 1, 3. Araújo, D., Davids, K., & Hristovski, R. (2006). The ecological dynamics of decision making in sport. Psychology of Sport and Exercise, 7(6), 653–676. https:// doi.org/10.1016/j.psychsport.2006.07.002 Araújo, D., Dicks, M., & Davids, K. (2019). Selecting among afordances: A basis for channeling expertise in sport. In M. Cappuccio (Ed.), Handbook of Embodied Cognition and Sport Psychology (pp. 537–556). MIT Press.

Understanding Skilled Adaptive Behavior

287

Araújo, D., Diniz, A., Passos, P., & Davids, K. (2013). Decision making in social neurobiological systems modeled as transitions in dynamic pattern formation. Adaptive Behavior, 22(1), 21–30. https://doi.org/10.1177/10597123 13497370 Boschker, M. S. J., & Bakker, F. C. (2002). Inexperienced sport climbers might perceive and utilize new opportunities for action by merely observing a mode. Perceptual and Motor Skills, 95(1), 3–9. https://doi.org/10.2466/pms.2002.95.1.3 Boschker, M. S. J., Bakker, F. C., & Michaels, C. F. (2002). Memory for the functional characteristics of climbing walls: Perceiving afordances. Journal of Motor Behavior, 34(1), 25–36. https://doi.org/10.1080/00222890209601928 Bruineberg, J., & Rietveld, E. (2014). Self-organization, free energy minimization, and optimal grip on a feld of afordances. Frontiers in Human Neuroscience, 8, 599. https://doi.org/10.3389/fnhum.2014.00599 Carello, C., Grosofsky, A., Reichel, F. D., Solomon, H. Y., & Turvey, M. T. (1989). Visually perceiving what is reachable. Ecological Psychology, 1(1), 27–54. https://doi.org/10.1207/s15326969eco0101_3 Carvalho, J., Araújo, D., Travassos, B., Fernandes, O., Pereira, F., & Davids, K. (2014). Interpersonal dynamics in baseline rallies in tennis. International Journal of Sports Science & Coaching, 9(5), 1043–1056. https://doi. org/10.1260/1747-9541.9.5.1043 Davids, K., & Araújo, D. (2010). The concept of “organismic asymmetry” in sport science. Journal of Science and Medicine in Sport, 13(6), 633–640. https://doi. org/10.1016/j.jsams.2010.05.002 Davids, K., Araújo, D., Hristovski, R., Passos, P., & Chow, J. Y. (2012). Ecological dynamics and motor learning design in sport. In N. J. Hodges & A. M. Williams (Eds.), Skill Acquisition in Sport: Research, Theory and Practice (pp. 112–130). Routledge. Fajen, B. R. (2005). Perceiving possibilities for action: On the necessity of calibration and perceptual learning for the visual guidance of action. Perception, 34(6), 717–740. https://doi.org/10.1068/p5405 Fajen, B. R., Riley, M. A., & Turvey, M. T. (2009). Information, afordances, and the control of action in sport. International Journal of Sport Psychology, 40(1), 79–107. Gibson, J. J. (1966). The Senses Considered as Perceptual Systems. Houghton Mifin. Gibson, J. J. (1979). The Ecological Approach to Visual Perception. Houghton Mifin. Guignard, B., Button, C., Davids, K., & Seifert, L. (2020). Education and transfer of water competencies: An ecological dynamics approach. European Physical Education Review, 26(4), 938–953. https://doi.org/10.1177/1356336X20902172 Guignard, B., Rouard, A., Chollet, D., Hart, J., Davids, K., & Seifert, L. (2017). Individual—Environment interactions in swimming: The smallest unit for analysing the emergence of coordination dynamics in performance? Sports Medicine, 47(8), 1543–1554. https://doi.org/10.1007/s40279-017-0684-4 Hacques, G., Komar, J., & Seifert, L. (2021). Learning and transfer of perceptual-motor skill: Relationship with gaze and behavioral exploration. Attention, Perception, & Psychophysics, 83(5), 2303–2319. https://doi.org/10.3758/ s13414-021-02288-z

288

Ludovic Seifert, Duarte Araújo, and Keith Davids

Haken, H., Kelso, J. A. S., & Bunz, H. (1985). A theoretical model of phase transitions in human hand movements. Biological Cybernetics, 51(5), 347–356. https://doi.org/10.1007/BF00336922 Higuchi, T., Murai, G., Kijima, A., Seya, Y., Wagman, J. B., & Imanaka, K. (2011). Athletic experience infuences shoulder rotations when running through apertures. Human Movement Science, 30(3), 534–549. https://doi.org/10.1016/j. humov.2010.08.003 Hove, P., Riley, M. A., & Shockley, K. (2006). Perceiving afordances of hockey sticks by dynamic touch. Ecological Psychology, 18(3), 163–189. https://doi. org/10.1207/s15326969eco1803_2 Kelso, S. J. A. (1995). Dynamic Patterns: The Self Organization of Brain and Behaviour. MIT Press. Kiverstein, J., & Rietveld, E. (2015). The primacy of skilled intentionality: On Hutto & Satne’s the natural origins of content. Philosophia, 43(3), 701–721. https://doi.org/10.1007/s11406-015-9645-z Konczak, J., Meeuwsen, H. J., & Cress, M. E. (1992). Changing afordances in stair climbing: The perception of maximum climbability in young and older adults. Journal of Experimental Psychology: Human Perception and Performance, 18(3), 691–697. https://doi.org/10.1037/0096-1523.18.3.691 Kretch, K. S., & Adolph, K. E. (2013). Clif or step? Posture-specifc learning at the edge of a drop-of. Child Development, 84(1), 226–240. https://doi. org/10.1111/j.1467-8624.2012.01842.x Kugler, P. N., & Turvey, M. T. (1987). Information, Natural Law, and the SelfAssembly of Rhythmic Movement. Lawrence Erlbaum. Lee, D. N. (1976). A theory of visual control of braking based on information about time-to-collision. Perception, 5(4), 437–459. https://doi.org/10.1068/p050437 Lee, D. N. (2009). General tau theory: Evolution to date. Perception, 38(6), 837–850. Mace, W. M. (1986). JJ Gibson’s ecological theory of information pickup: Cognition from the ground up. In T. J. Knapp & L. C. Robertson (Eds.), Approaches to Cognition: Contrasts and Controversies (pp. 137–157). Erlbaum. Mark, L. S., Balliett, J. A., Craver, K. D., Douglas, S. D., & Fox, T. (1990). What an actor must do in order to perceive the afordance for sitting. Ecological Psychology, 2(4), 325–366. https://doi.org/10.1207/s15326969eco0204_2 Mark, L. S., Ye, L., & Smart, L. J. (2015). Perceiving the nesting of afordances for complex goal-directed actions. In The Cambridge Handbook of Applied Perception Research, Vol. I. (pp.  547–567). Cambridge University Press. https://doi. org/10.1017/CBO9780511973017.034 Mason, P. H. (2010). Degeneracy at multiple levels of complexity. Biological Theory, 5(3), 277–288. https://doi.org/10.1162/BIOT_a_00041 Passos, P., Araújo, D., Davids, K., Gouveia, L., Milho, J., & Serpa, S. (2008). Information-governing dynamics of attacker—Defender interactions in youth rugby union. Journal of Sports Sciences, 26(13), 1421–1429. https://doi. org/10.1080/02640410802208986 Profeta, V. L. S., & Turvey, M. T. (2018). Bernstein’s levels of movement construction: A contemporary perspective. Human Movement Science, 57, 111–133. https://doi.org/10.1016/j.humov.2017.11.013

Understanding Skilled Adaptive Behavior

289

Reed, E. S. (1993). The intention to use a specifc afordance: A conceptual framework for psychology. In R. H. Wozniak & K. W. Fischer (Eds.), Developing in Context: Acting and Thinking in Specifc Environments (pp. 45–76). Erlbaum. Reed, E. S. (1996). Encountering the World: Toward an Ecological Psychology. Oxford University Press. Richardson, M. J., Shockley, K., Fajen, B. R., Riley, M. A., & Turvey, M. T. (2008). Ecological psychology: Six principles for an embodied—Embedded approach to behavior. In P. Calvo & A. B. T.-H. of C. S. Gomila (Eds.), Perspectives on Cognitive Science (pp. 159–187). Elsevier. https://doi.org/10.1016/ B978-0-08-046616-3.00009-8 Rietveld, E. (2008). Situated normativity: The normative aspect of embodied cognition in unrefective action. Mind, 117(468), 973–1001. https://doi.org/10.1093/ mind/fzn050 Rietveld, E., & Kiverstein, J. (2014). A rich landscape of afordances. Ecological Psychology, 26, 325–352. https://doi.org/10.1080/10407413.2014.958035 Riley, M., Richardson, M., Shockley, K., & Ramenzoni, V. (2011). Interpersonal synergies. Frontiers in Psychology, 2, 38. https://doi.org/10.3389/ fpsyg.2011.00038 Schmidt, R. C., Carello, C., & Turvey, M. T. (1990). Phase transitions and critical fuctuations in the visual coordination of rhythmic movements between people. Journal of Experimental Psychology: Human Perception and Performance, 16(2), 227–247. https://doi.org/10.1037/0096-1523.16.2.227 Seifert, L., Cordier, R., Orth, D., Courtine, Y., & Croft, J. L. (2017). Role of route previewing strategies on climbing fuency and exploratory movements. PLoS One, 12(4), e0176306. https://doi.org/10.1371/journal.pone.0176306 Seifert, L., Dicks, M., Wittmann, F., & Wolf, P. (2021a). The infuence of skill and task complexity on perception of nested afordances. Attention, Perception, & Psychophysics, 83(8), 3240–3249. https://doi.org/10.3758/s13414-021-02355-5 Seifert, L., Dicks, M., Wittmann, F., & Wolf, P. (2021b). The perception of nested afordances: An examination of expert climbers. Psychology of Sport and Exercise, 52, 101843. https://doi.org/10.1016/j.psychsport.2020.101843 Seifert, L., Komar, J., Araújo, D., & Davids, K. (2016). Neurobiological degeneracy: A key property for functional adaptations of perception and action to constraints. Neuroscience & Biobehavioral Reviews, 69, 159–165. https://doi. org/10.1016/j.neubiorev.2016.08.006 Seifert, L., Wattebled, L., Herault, R., Poizat, G., Adé, D., Gal-Petitfaux, N., & Davids, K. (2014). Neurobiological degeneracy and afordance perception support functional intra-individual variability of inter-limb coordination during ice climbing. PLoS One, 9(2), e89865. https://doi.org/10.1371/journal. pone.0089865 Silva, P., Chung, D., Carvalho, T., Cardoso, T., Davids, K., Araújo, D., & Garganta, J. (2016). Practice efects on intra-team synergies in football teams. Human Movement Science, 46, 39–51. https://doi.org/10.1016/j.humov.2015.11.017 Silva, P., Garganta, J., Araújo, D., Davids, K., & Aguiar, P. (2013). Shared knowledge or shared afordances? Insights from an ecological dynamics approach to team coordination in sports. Sports Medicine, 43(9), 765–772. https://doi. org/10.1007/s40279-013-0070-9

290

Ludovic Seifert, Duarte Araújo, and Keith Davids

Travassos, B., Araújo, D., Davids, K., Vilar, L., Esteves, P., & Vanda, C. (2012). Informational constraints shape emergent functional behaviours during performance of interceptive actions in team sports. Psychology of Sport and Exercise, 13(2), 216–223. https://doi.org/10.1016/j.psychsport.2011.11.009 Turvey, M. T., & Carello, C. (2012). On intelligence from frst principles: Guidelines for inquiry into the hypothesis of Physical Intelligence (PI). Ecological Psychology, 24(1), 3–32. https://doi.org/10.1080/10407413.2012.645757 van Dijk, L., & Rietveld, E. (2021). Situated anticipation. Synthese, 198(1), 349– 371. https://doi.org/10.1007/s11229-018-02013-8 Wagman, J. B., Caputo, S. E., & Stofregen, T. A. (2016). Hierarchical nesting of afordances in a tool use task. Journal of Experimental Psychology: Human Perception and Performance, 42(10), 1627–1642. https://doi.org/10.1037/ xhp0000251 Wagman, J. B., & Morgan, L. L. (2010). Nested prospectivity in perception: Perceived maximum reaching height refects anticipated changes in reaching ability. Psychonomic Bulletin & Review, 17(6), 905–909. https://doi.org/10.3758/ PBR.17.6.905 Wagman, J. B., & Stofregen, T. A. (2020). It doesn’t add up: Nested afordances for reaching are perceived as a complex particular. Attention, Perception, & Psychophysics, 82(8), 3832–3841. https://doi.org/10.3758/s13414-020-02108-w Warren, W. H. (1984). Perceiving afordances: Visual guidance of stair climbing. Journal of Experimental Psychology: Human Perception and Performance, 10(5), 683–703. https://doi.org/10.1037/0096-1523.10.5.683 Warren, W. H., & Whang, S. (1987). Visual guidance of walking through apertures: Body-scaled information for afordances. Journal of Experimental Psychology: Human Perception and Performance, 13(3), 371–383. https://doi. org/10.1037/0096-1523.13.3.371 Weast, J. A., Shockley, K., & Riley, M. A. (2011). The infuence of athletic experience and kinematic information on skill-relevant afordance perception. Quarterly Journal of Experimental Psychology, 64(4), 689–706. https://doi.org/10.1 080/17470218.2010.523474 Withagen, R., Araújo, D., & de Poel, H. J. (2017). Inviting afordances and agency. New Ideas in Psychology, 45, 11–18. https://doi.org/10.1016/j. newideapsych.2016.12.002 Withagen, R., de Poel, H. J., Araújo, D., & Pepping, G.-J. (2012). Afordances can invite behavior: Reconsidering the relationship between afordances and agency. New Ideas in Psychology, 30(2), 250–258. https://doi.org/10.1016/j. newideapsych.2011.12.003

16 DISABILITY THROUGH THE LENS OF AFFORDANCES A Promising Pathway for Transforming Physical Therapy Practice Paula L. Silva and Sarah M. Schwab

There are currently one in seven adults living with movement-related disability, a number that is expected to grow as the world population ages (Okoro et  al., 2018). The onset of this type of disability is associated with a variety of medical conditions that limit a person’s mobility. Some of these conditions are developmental or present at birth, such as cerebral palsy and spina bifda, and some are acquired, such as non-specifc chronic pain, arthritis, stroke, multiple sclerosis, and Parkinson’s disease, among others. For individuals who live with movement-related disability, everyday, often taken-for-granted activities (e.g., using utensils to eat, opening a jar, grasping and lifting a cup of cofee and putting it back on the table, standing up from a sofa, crossing a street, and writing and speaking) can be challenging if not impossible to perform. Physical therapists are health professionals who are tasked with helping disabled individuals1 manage these movement-related challenges through interventions that aim to develop, preserve, and expand their movement possibilities. Unfortunately, many individuals who participate in physical therapy perceive standard interventions—which often focus on normalization of body structures and functions—as disconnected from, rather than supportive of, their functional goals (Angeli et  al., 2019; Gibson, 2016; Nicholls, 2022; Roush & Sharby, 2011). Worse still, some people with disability characterize standard interventions as oppressive because they take as “ideal reference” the bodies of individuals without disability (Abberley, 1995; Finklestein, 2013; Gibson, 2016; Imrie, 2000; Moll & Cott, 2013; Oliver, 1990; Oliver, 1993; Shogan, 1998). DOI: 10.4324/9781003396536-21

292

Paula L. Silva and Sarah M. Schwab

This set of circumstances reveals a critical need for a diferent model of physical therapy practice, one that centers the experiences and goals of those whom it serves (Nicholls, 2017, 2022). The vision of the disability community is for rehabilitation to be understood as part of the broader process of social transformation required to enable those whose bodies difer from established norms (Stein et al., 2007). In our view, a model of physical therapy practice that is responsive to this vision must be oriented by (and support) the unique strategies that individuals with disability use to create enabling conditions for themselves in a world that is often insensitive to their bodies and skill sets. Our goal for this chapter is to show that the concept of afordance—when properly construed to capture the experience of disabled individuals—can provide the right level of description for these strategies and guide the desired paradigm shift in physical therapy practice. To achieve this goal, we will draw heavily on the work of Dr. Arseli Dokumaci (Dokumaci, 2016, 2017, 2019, 2023). Dokumaci is a disability scholar who lives with disability associated with rheumatoid arthritis. Along with other disability scholars and disability rights activists (e.g., Bones et al., 2022; Lonsdale, 1990; Sherry, 2013), she documents the challenges that individuals with disability experience as a result of living in a world that (too often) does not readily complement their bodily capabilities and skills. Importantly, for present purposes, Dokumaci’s work also captures “how disabled people improvise more habitable worlds” by using their bodily performances to create new, previously unimagined afordances out of the available material features of the environment (Dokumaci, 2023). She qualifes these afordances as “activist” because they are a product of creative, deliberate, and often efortful action. We aim to show how the concept of activist afordances situate the day-to-day bodily performances of individuals with disability in the broader, social process of world (re)making and, in doing so, invites an understanding of disability as an experience that is both embodied and embedded. We will propose a new paradigm for physical therapy practice inspired by this understanding that we believe aligns with the vision of the disability community for health rehabilitation. We will end with the lessons we learned for Ecological theory and research in the form of brief answers to the questions that unite the various chapters of this book. Prior to moving forward, we feel it is important to disclose that we, the authors, are two female, non-disabled ecological psychologists and rehabilitation scientists. The proposition we ofer for the paradigm shift in physical therapy is inspired by the work of disability scholars and disability rights activists, including many who identify as disabled. We fully recognize that our reading of this work is necessarily fltered by our own ableist

Disability Through the Lens of Afordances

293

biases and thus we stand ready to revise (or fully revamp) our proposition as we engage with criticisms to it from the disability community. Moving Through the World With Ease: The Enabling Role of Afordances I look where the goal is, I am drawn by it, and the whole bodily machine does what must be done for me to get there. —Merleau-Ponty (1973, p. 77)

Picture this scenario: You are tired and thirsty from running errands on a warm day in a busy urban area and decide to go to a local pub across the street to take a break and have a refreshing drink. You approach the crosswalk, move toward the pub, and enter. Upon entry, you notice a sign that says “seat yourself” and spot one empty table available on the second level. You swiftly move toward and secure the table to rest and have a cold glass of water (or beer), as intended. This scenario brings out a sense of seamless continuity between the emergence of a goal (or intention) and the situated actions that give expression to it. The complex coordinated movements of the body that support these actions are implied but not foregrounded in our description—just like these movements are often not foregrounded in the minds of those who, like the authors, are currently abled-bodied. In the words of Merleau-Ponty: Our bodily machine simply “does what must be done.” For millions among us with movement-related disability, this experience is not as prevalent. At issue is why that is. Reports on the lived experience of disabled individuals suggest that for a person to move through the world with ease, mindlessly drawn by their goals and desires, they must readily fnd, in their surrounding environment, material conditions that enable actions that bring about desired outcomes (Dokumaci, 2013; Leder, 1990; Sobchack, 2005). Put diferently, the experience of goal-action continuity implicates readily accessible environmental afordances. The scenario that opened this section has many likely candidates: A trafc light that, by virtue of pausing trafc, periodically creates an open path that afords crossing the busy street to approach the pub; a fight of stairs in the pub that afords a means for getting to the second foor to secure the table; a chair that afords sitting down to rest, among others. Notice that we listed the afordances in shorthand, that is, without mentioning the movement capabilities or skills that specify them: The capability to reliably achieve a walking speed compatible with the time a stop light allots for crossing, the capability to step up and onto a series of steps, the capability to achieve and comfortably sustain a sitting posture. In

294

Paula L. Silva and Sarah M. Schwab

other words, we presupposed—as is common practice when environmental settings are designed—the capabilities and skills that non-disabled adults reliably develop and habitually express. It is important to note, however, that the very same features we listed as supportive of activity in the hypothetical scenario that opened this section would likely be perceived as barriers by disabled individuals whose capabilities and skills difer from what we assume to be the “norm” (Davis, 1995; Sobchack, 2005). In such cases, the experience of the body is foregrounded and the described sense of free and spontaneous movement is disrupted (Leder, 1990; Sobchack, 2005). Any attempt to reduce this experience to a problem within the individual (i.e., as the sole consequence of a biologic pathological process) ignores the historical and social processes that create environmental conditions that support habitual (or normative) skills often in detriment of other possible ways of perceiving and acting in the world.

On the Mutual Relations Between Afordances and “Normative” Skills Other people fgure in the afordances of things . . . not just in constructing them, but in defning, explaining, and ‘policing’ their use. Objects do not simply exist, they are ‘maintained.’ —Costall (1995, p. 473)

Gibson coined the term afordance to capture the functional signifcance of the environment for a particular organism at a particular moment (Gibson, 1979, 1982). Organisms of the same species, by virtue of expressing common perceptual and motor capabilities and being exposed to speciestypical practices, usually become attuned to, and repeatedly realize a set of similar environmental afordances in service of their livelihood (Costall, 1995; Heft, 2001, 2007; Lewontin, 1983; Withagen & van Wermeskerken, 2010). In other words, while environmental objects and surfaces in the environment could potentially ofer many possibilities for action, we do not perceive or realize them with equal probability (Costall, 1995, 2006; Heft, 2003). For instance, a chair in an unoccupied table at a restaurant is likely to be perceived as a place to sit down. Even though its material properties could also ofer a place to stand on, we usually do not realize this afordance of chairs, at least not in usual circumstances. We are socialized not to. Case in point: Things have usual (or canonical) afordances that have been chosen by other humans born before us (Heft, 2003). Through the infuence of people who are already skilled in perceiving and realizing these canonical afordances, we learn the functional signifcance of the

Disability Through the Lens of Afordances

295

material and social features in our environment—what they are meant to aford. For those of us born without a condition that impairs or limits the body and its functions, the process of development and socialization reliably and organically scafolds the perceptual-motor skills that specify canonical afordances (Heft, 1989; Ingold, 2000). Over time, we readily and habitually express these skills and realize canonical afordances in the service of our goals. Take walking as an example. The usual pendular patterns of gait emerge when toddlers learn to produce force, at the right time, to efectively push of with the foot against the ground to swing the leg and propel the body forward (Holt et al., 2006). In other words, they learn to actualize the push-on-ability afordance ofered by most surfaces encountered in our day-to-day lives—surfaces with sufcient friction to create the reaction force required to propel the body. Practice of this skill is often encouraged by caregivers, who guide and support their initial attempts. The learning process is always situated, and thus, perceptual-motor skills can never be separated from the contextual conditions that enable them (Ingold, 2000). They are products of particular individual-environment relations shaped by our bodily performances but not reduceable to them (Ingold, 2000). For instance, the prototypical “normal” pattern of walking—which requires one to push of against the ground with the foot—cannot be expressed on a slippery surface, such as an icy sidewalk or wet foors. In such situations, individuals must create diferent relations between their body and the surface to get to the desired location safely. Instead of pushing of against the surface with the foot to take each step, one may choose instead to raise and lower the leg, using the hip muscles, to carefully and just slightly advance forward. This improvised performance is a way of “making” a non-ideal, slippery surface aford the required support for locomotion (Figure 16.1). The process of making afordances out of less-than-ideal material features requires some level of deliberation and calculated efort for the intended action to be expressed (Leder, 1990; Sobchack, 2005). Not surprisingly, humans build surfaces (inside and outside) that support our usual means of locomotion and engage in practices to minimize threats to it: We remove ice from sidewalks after a snowstorm, cleaning personnel often put out signs identifying wet foors, and so on. In fact, afordances that assume “typical” patterns of walking have been incorporated into our environmental landscape in the shape of stairs, sidewalks with high curbs, timing of stoplights, etc. These ubiquitous material features then sustain walking as “the normative” form of locomotion and make other forms of locomotion (e.g., body swinging around crutches, wheeling, among others) less efective (Ingold, 2000).

296 Paula L. Silva and Sarah M. Schwab

FIGURE 16.1

Illustrative example of perceptual-motor skills emerging from individual-environmental relations.

Disability Through the Lens of Afordances

297

The general lesson here is that canonical afordances and “normative” skills—that is, skills habitually expressed by non-disabled individuals at diferent stages of their development—are co-selected and mutually maintained (Withagen & van Wermeskerken, 2010). Perceiving the canonical afordances that can serve a particular goal directly facilitates the (mindless) expression of the skills that realize them. Repeatedly (and unconsciously) realized, canonical afordances fall into routine (Costall, 1995, 2006; Heft, 2001). As a result, those of us who are non-disabled become rather oblivious to their inherent contribution to our habitual ways of functioning in the world; we stop noticing the assistance we receive from our environment. As a result, the expression of a normative skill is often conceived as a product of capacities that can be located within the individual. In line with Ecological theory and related afordance research, we argue that these capacities are best described and conceptualized in reference to the physical and social environment (Heft, 2007; Ingold, 2000; Mark, 1987; Oudejans et al., 1996; van der Kamp et al., 1998; Warren, 1984). In sum, while development and socialization scafold the habitual skills that enhance our individual ft to the surrounding environment, our collective social-cultural practices maintain and transform—albeit at a slower timescale—our environment’s materiality in ways that often presuppose these skills (Costall, 1995; Heft, 2007; Ingold, 2000; Lewontin, 1983; Withagen & van Wermeskerken, 2010). Thus, the individual-environment ft, often taken as the starting point in most analyses of afordances and their role in behavior, is neither a given nor incidental; it emerges from our individual and collective embodied actions in the world. In light of this premise, it is important to inquire whose livelihood is enabled and whose might be restricted or limited by the afordances selected, refned, and maintained by these actions. Afordances of Our Evolving Human Niches: Enabling for Whom? People with disability live in a world designed to ft other people but not them. It’s like trying to walk in shoes three sizes too large for you . . . In these shoes, I am clumsy and I have blisters, but would not have either problem if I could only get shoes in my size. Unfortunately, they do not make shoes my size. —Richardson (2022, p. 60)

The furnishings of our existing human niches ofer optimal fts for and, hence, afordances that primarily enable non-disabled individuals and their

298

Paula L. Silva and Sarah M. Schwab

habitual ways of moving. Disabled individuals, however, often experience disruptions in their ft to the physical and social environments in which they perform their activities. In her video ethnographies, Dokumaci (cf. here) captured many instances of such disruptions as she observed people with chronic pain engage in a number of goal-directed functional activities (e.g., eating a meal, standing up from a sofa, opening a jar). The following are descriptions of observed individual-environment relations that are clearly marked by misft rather than ft: [T]he strength with which a knife needs to be used against the plasticity of cooked chicken and someone’s painful hands that cannot exert that pressure; the less than knee-high height of the surface of a sofa, the softness of its cushions, the slipperiness of their texture, the height of its armrest and someone’s knees, arms, and elbows that are in pain; the rigidity of a metal cap, its resistance to deformation, and its round shape that necessitates an efort that the participants’ swollen hands cannot exert. —Dokumaci (2017, p. 9) Experiences of individual-environment “misft” are not unique to those whose movements are restricted by pain. It shows up often and in multivarious forms for anyone whose bodily dispositions and skills do not fall within the range that our human environments can readily accommodate (cf. Bones et  al., 2022; Schwab et  al., 2022). For instance, a child with cerebral palsy who cannot speak or write and wishes to communicate will experience a misft with people who are only attuned to standard forms of oral and written communication (Sargent et al., 2013). A person with a lower extremity amputation who needs to hop from the bed to the bathroom in the middle of the night experiences a misft with a large, unfamiliar hotel room (Sobchack, 2005). A person walking with a prosthetic limb and a cane may experience misft with crosswalks equipped with trafc lights whose mode of operation hinges on a gait speed she cannot reliably achieve (Sobchack, 2005). We could fll pages and pages with examples, but the ones we laid out sufce to show that our current human environments have not evolved to be sufciently sensitive to the widely diverse embodiments and skillsets of individuals with disability. Further, the examples show that the causes of individual-environment misft inhere neither in a person’s body (however atypical) nor in the environment (no matter what material and social features it has or lacks) but rather in their mutually constraining entanglement in context of a goal-directed activity at a particular moment in time (Heft, 1989; Schwab et al., 2022; Toro et al., 2020; Vaz et al., 2017).

Disability Through the Lens of Afordances

299

Because humans are active, moving beings, their relationship with the surrounding environment is always in fux, and so are the consequences of their encounters. Thus, misft can be experienced at one moment and (potentially) resolved at some later time. We now turn our attention to these dialectic dynamics, which are evidenced in Dokumaci’s rich descriptions of the engagement of disabled individuals with their environment. This dynamic is marked by temporally extended instances of individualenvironment tension that evolve to resolutions. To be precise, we use the term tension to capture the idea that the misft between particular features of the environment and a person’s movement capabilities is not necessarily permanent, but potentially temporary. We use the term resolution to capture a reconfguration of individual-environment relations in the face of tension that gives rise to new forms of ft and the discovery of new (“nonnormative”) capacities for goal attainment. On Instances of Individual-Environment Tension and Their Consequences A landscape is viewed not as a feld of opportunities, but difculties to negotiate. Etymologically, ‘ease’ comes from the French word ‘aise’, originally meaning ‘elbow room’ or ‘opportunity’. This experience of world-as-opportunity is precisely what dis-ease calls into question. —Leder (1990, p. 81)

Instances of individual-environment tension, while varied in detail, have a common consequence: A disruption in the experience of goal-action continuity. Reports from individuals with disability suggest that the gap between the two is flled with worries about whether the desired action is possible and deliberation about how it might be executed (cf. Schwab et al., 2022). In that space, there is an increased awareness of the materiality of the body and surrounding environment and of the boundary between the two (Dokumaci, 2017, 2023; Schwab et  al., 2022; Sobchack, 2005; Toro et  al., 2020). First-person reports of individuals with disability are telling in this respect. Here is a quote from Dr. Sobchack—an American cinema and media theorist and cultural critic—extracted from an essay in which she describes her experience navigating the world after amputation: [A]lthough I am mobile and directed across the street toward the transcendent achievement of whatever my real goal is, I am, for what seems the most elongated moment, also redirected to my bodily immanence to worry about, whether or not I can traverse the street in the normative time the trafc light allots—and experience tells me that it doesn’t

300

Paula L. Silva and Sarah M. Schwab

allot sufcient time for the disabled or, indeed, the elderly to cross without anxiety. Thus, my intentionality—the streaming forth and outerdirectedness away from awareness of my body and toward my worldly projects—is suddenly forestalled. —Sobchack (2005, p. 8) Dr. Sobchack’s goal in situations such as this is to get to a place across a busy street, a goal that is not immediately expressed by her “bodily machine.” Her goal (or intention) to cross the street is forestalled by her encounter with the environmental features that constitute our usual crosswalks. This encounter is certainly best characterized by tension—a momentary misft—between the period allotted for crossing and the speed she can reliably achieve when locomoting with her prosthetic leg and cane. As a result of this tension, the temporary open path created by the red light does not readily aford crossing to her. Rather, the open path presents as a challenge to be overcome, a barrier to goal achievement, forcing Dr. Sobchack to grapple with the often emotionally charged experience of “I cannot.” Challenges to goal-directed action arising from individual-environment tension is a common thread in reports of individuals with various forms of movement-related disability (Bones et al., 2022; Dokumaci, 2017, 2023; Schwab et al., 2022; Sherry, 2013). Going back to the instances highlighted in Dokumaci’s ethnographies: The goal of eating chicken is forestalled by the impossibility of applying the amount of pressure to a knife required to cut it; the goal of standing up from a low sofa is forestalled by the impossibility of leveraging its surfaces to extend the leg and pull the body upright; the goal of opening a jar is forestalled by the impossibility of twisting and unscrewing the tight metal cap with the hand; the goal of communicating is forestalled by the impossibility of speaking and writing; the goal of hopping from the bed to the bathroom is forestalled by the long path with no places for support. The lesson to generalize from these examples is that the encounter of an individual with their surrounding environment specifes the presence— or absence—of usual afordances implicated by that individual’s goal and, perforce, determines whether they readily and mindlessly express appropriate goal-directed actions or experience a barrier to do so. Most afordance-based analyses of behavior stop at this juncture, highlighting either possibility or impossibility of action at the timescale of the individual-environment encounter (e.g., Alt et al., 2021; Fajen & Matthis, 2011; Mark, 1987; Warren, 1984). A critical, yet underexplored question is what happens once immediate action is impeded by (an always potentially temporary) experience of individual-environment misft?

Disability Through the Lens of Afordances

301

From Individual-Environment Tension to Resolution: The Body and Its Movements as the Medium for Afordance Creation Being aware of the absence of reciprocities that are readily available within the order of the material world, we move, sense, act, and behave in such peculiar ways that our movements, actions, and behaviours forge another kind of ‘ftting’, another kind of reciprocity—a reciprocity, with which our pain is minimised, and ourselves and the environment are more rhythmic with each other. —Dokumaci (2017, p. 10)

Disabled individuals do not stay passive in the face of tension with their surrounding environment. While they do experience the impossibility for their activity in certain circumstances, frst-person reports and ethnographic work suggest that many people with disability remain “open” to explore, and through individual and collective strategies, fnd new possibilities for achieving their goals (Dokumaci, 2013, 2017, 2023; Rahlin et al., 2019; Sobchack, 2005). For instance, the Disability Movement organizes collective strategies that seek to continuously amplify accessibility and participation of disabled individuals in our society (Watermeyer et al., 2006). Such collective strategies are critical and have been instrumental in promoting changes to expand the ft of our physical-social-institutional environment to a greater range of bodies, skills, and ways of living. Our focus here, however, will not be on these highly organized, collective strategies. Rather, we will focus on the improvised (though equally critical) strategies that people with disability engage in to create enabling conditions for their own day-to-day activities. Dokumaci’s ethnographic work focuses on these strategies and highlights the critical role they play when disabled individuals experience tension between their bodies and the surrounding environment (2013, 2017, 2023). Here are some examples: When a participant’s goal of eating chicken was forestalled by a tension between the knife and her painful hands, she decided to use her teeth, which also aford cutting. When another participant’s goal of standing up was forestalled by a tension between the sofa and her painful limbs, she choreographed a unique series of shoulder, elbow, and knee movements that carved out, in succession, the sorts of relations with the surfaces of the furniture that aforded the support she needed to stand up. When yet another participant’s goal of opening a jar was forestalled by a tension between the metal cap and her painful hand, she knew to use hot water to expand the metal and loosen up the cap enough such that it became twistable.

302

Paula L. Silva and Sarah M. Schwab

In her essay “Choreography for One, Two, and Three Legs (A phenomenological meditation in movements),” Dr. Sobchack did not share how she usually resolves the tension she experiences with trafc lights and crosswalks. However, she recounts in detail how she choreographs her movements, using the support of surrounding surfaces and objects, to efciently get her chores done at home (Sobchack, 2005). When she must move around in unfamiliar spaces, careful planning is required. Here is how she anticipates tension and resolution when she stays in hotels: When I arrive (and before I take of my leg to get ready for bed), I have learned to ‘case the joint’: if the room is small and the bed not far from the bathroom, I calculate the distance in hops—but have to factor in the placement of furniture, doors, walls and moldings I might hold onto for stability and safety; if the room is large, I also have learned to move the available chairs around in advance of my one-leggedness. —Sobchack (2005, p. 59) The experience of disability implicates the need to “hold in abeyance the canonical afordances of things” (Dokumaci, 2017, p. 14) and uncover the hidden possibilities that the substances and surfaces of the environment can provide their disabled bodies. The quote above illustrates how Dr. Sobchack views the chairs in the room not only as places to sit (a canonical afordance) but also as objects that, if positioned on the path from bed to bathroom ahead of time, can serve as support for her preferred mode of locomotion during the night. Could her labor be minimized by architectural changes in the environment? Absolutely. But when conditions are not ideal, as is often the case, individuals with disability create, through their bodily performances, new afordances for the material features available in the surrounding environment in the service of their goals. Often the creation of afordances requires the collaboration of others. The story of Alice and her mother Mariana Rosa, both Brazilians, is rich with examples of joint afordance creation. Alice lacks the motor coordination to speak or use a standard pencil to write. Together, mother and daughter added meaning to the movements she can do. For instance, Alice learned to let her head “fall” forward when she wants to say “yes” in response to a question and learned to hold her head still when she means “no.” From this foundation and the help of her mom (and other intimate partners and educators), Alice learned to use her “yes” and “no” signals to choose letters and other linguistic signals to build words, then sentences, and then her stories. Today, Alice is nine and publishes insightful critical summaries of books she reads at home and at school. In her summary of a book that teaches children about the hardships of dictatorships, Alice

Disability Through the Lens of Afordances

303

reminds us that authoritative behavior happens in democracies, too—like when people do things to and for her without asking permission. All of these examples illustrate that when faced with impossibility for immediate, mindless, goal-directed action, individuals with disability often deliberately use their bodies as a resource to ofset what is functionally lacking in our current human environments. That is, they create, with their embodied, embedded actions, new possibilities for achieving their goals, which Dokumaci termed activist afordances. There is good reason for the qualifcation. These are afordances that individuals with disability, often deliberately and efortfully, bring to life to resolve the tension they experience with the functional features of our environment that are insensitive to their bodies. In Dokumaci’s words: Individuals with disability “make up” and “make real” conditions for goal achievement that were not readily given, by “making do with” what is available (Dokumaci, 2023, p. 9). That is, they transform—through bodily performances—their relations with the surrounding physical and social environment. In doing so, they materialize new possibilities, even if only ephemerally, for their own activities. The concept of activist afordances celebrates the agency of individuals with disability in the process of creating enabling conditions for their dayto-day activities. It is critical to emphasize, however, that there is a physical, cognitive, and emotional cost when one has to perform afordances that are not readily available in the environment. One might reasonably hypothesize that this cost is proportional to the “remoteness” of the ft between individual and environment as currently presented. Thus, the concept of activist afordance is not meant, in any way, to negate the social responsibility to transform our physical and social environment in ways that amplify its sensitivity to the diverse embodiments of people with disability. To the contrary, activist afordances center people with disability in this process of social transformation. These afordances bring to life, even if only ephemerally, the vision of a disabled person for what the environment could ofer them through new accessible features, services, assistive tools, adaptive equipment, new forms of collaboration with other people, and so on. In sum, disabled bodies in action both highlight environmental barriers and reveal more accessible futures for themselves. Predominant models of disability, however, do not accommodate this view of disabled bodies as active, creative constructors of new possibilities not in spite of, but because of, their singularities. In what follows, we will review models and the impact of their limited conception of the disabled body in physical therapy practice. We will then describe an Ecological model of disability that overcomes their limitations and show how

304

Paula L. Silva and Sarah M. Schwab

it can catalyze the desired paradigm shift in physical therapy practice— one that situates physical therapy as part of a broader process of social transformation. The Disabled Body in Prominent Models of Disability: Implications for Physical Therapy Practice We live with particular social and physical struggles that are partly consequences of the conditions of our bodies and partly consequences of the structures and expectations of our societies, but they are struggles which only people with bodies like ours experience. —Wendell (1989, p. 117)

As noted in the introduction of this chapter, physical therapists are health professionals trained to assist individuals in achieving their functional goals through interventions aimed to develop, preserve, and expand their movement possibilities. Movement is by its very nature embodied. Thus, physical therapy practice is directly infuenced by theoretical assumptions about the body and how it relates to functioning and disability, even if these are rarely explicitly articulated. We argue—in good company (Feldner et al., 2022; Gibson, 2016; Nicholls, 2022; Nicholls & Gibson, 2010)—for the need to bring such assumptions to light because they explain the disconnect between disability studies and rehabilitation sciences and the limited impact of physical therapy practice in advancing the goals of people with disability. Physical therapy has been historically rooted in Cartesian materialism (Table 16.1), which views the body as a simple machine whose overall functioning (or malfunctioning) can be understood through analysis of its component parts (Martínez-Pernía et  al., 2017; Nicholls, 2017; Nicholls & Gibson, 2010) models of disability most closely resonate with Cartesian materialism. According to these medical models (Hogan, 2019; Marks, 1997)—any functional challenge experienced by disabled people can be reduced to body malfunctions called impairments. Impairments are operationalized, in turn, as deviations in body structures and functions from “norms” that center on idealized, able-bodies (Paterson & Hughes, 2000). Historically shaped by the medical model and its Cartesian view of the body, physical therapy practice largely emerged oriented by goals defned by the therapists (achievement of “normative” skills) through normalization of identifed impairments. The following quote from Finkelstein (1993) expresses the way this ideology of normality has played out during his rehabilitation experience after a spinal injury:

Disability Through the Lens of Afordances

305

TABLE 16.1 Summary of prominent models of disability.

Medical model

Social model

ICF model

Disability result of pathology within the individual

Disability result of Disability result of disabling word (physical synthesis of biological, and attitudinal barriers) individual, and social characteristics Disabled body sole source Disability is an expression Consistent with biopsychosocial of limitation of diversity to be celebrated and supported perspective by more inclusive environments Consistent with Cartesian Consistent with social Considers diferent levels constructionism materialism of disability: body, whole person, whole person in a context Continues to be Does not account for Disabled body reduced implemented like to body structure and how bodily impairment medical model shapes disability function impairments   experience

The aim of returning the individual to normality is the central foundation stone upon which the whole rehabilitation machine is constructed. If, as happened to me following spinal injury, the disability cannot be cured, normative assumptions are not abandoned. . . . The rehabilitation aim becomes to assist the individual to be as ‘normal as possible’. The result, for me, was endless soul-destroying hours at Stoke Mandeville Hospital trying to approximate able-bodied standards by ‘walking’ with callipers and crutches. —Finkelstein (1993, cited by Oliver, 1993, p. 16) The medical model and the practices it promotes have been criticized by disability scholars, including scholars with disability (Campbell et  al., 1996; Charlton, 1998; Shakespeare, 2006) and by critical rehabilitation researchers (Beaudry, 2016; Bunbury, 2019; Gibson, 2016; Haegele & Hodge, 2016) because the medical model refects a narrative of disabled bodies as pathological or defective—something to be “fxed.” The disabled body is thus viewed solely as a source of limitation, never for the potential of its singular characteristics to create new (previously unimagined) possibilities for action. It follows then that the usual skills demonstrated by non-disabled individuals (like walking or speaking) are viewed as expressions of “optimal functioning.” Relatedly, alternative skills (like wheeling

306

Paula L. Silva and Sarah M. Schwab

or using hand and/or head movements to communicate) are viewed as inherently suboptimal (Gibson, 2016; Nicholls & Gibson, 2010), never an expression of creative action to be promoted and supported by an inclusive environment. The social view of disability (Table 16.1) has been proposed to replace the strictly biological view embraced by the medical model. This view has been framed diferently in diferent places (cf. Gustavsson, 2008; Hahn, 1988) but found its most prominent formalization in the social model of disability. The social model was the product of intellectual and political discussions of members of the Union of Physically Impaired Against Segregation (UPIAS), in Britain (cf. Shakespeare, 2006). It conceptualizes disability as something imposed on top of impairments in body structures and functions by environmental barriers—both physical and attitudinal. Here is a quote from the UPIAS document that captures the intent of the social model to more comprehensively address environmental barriers: We fnd ourselves isolated and excluded by such things as fights of steps, inadequate public and personal transport, unsuitable housing, rigid work routines in factories and ofces, and a lack of up-to-date aids and equipment. —UPIAS, cited in Shakespeare (2006, p. 197) The social model is rooted in social constructionism (Siebers, 2001), a theory of knowledge that holds that characteristics of the body typically thought to be immutable and solely biological—like ability—are determined by cultural and historical contexts. With a strictly social view of disability, the relation between body and environment is unidirectional, such that the environment provides the disabled body with meaning but not the other way around. It follows that disability is a social (rather than biological) disadvantage, which implicates advances in social justice (rather than medicine or physical therapy) as the means to promote the well-being of individuals with disability. By identifying the environmental barriers (physical, attitudinal, and institutional) that need to be removed or altered, the social model has been important for the inclusion of individuals with disability in society. Implicit in the social model is a very diferent narrative of the disabled body: one that characterizes disability as an expression of diversity to be celebrated and supported by more inclusive environments. The social model has ofered physical therapists a non-medicalized view of disability that has certainly shaped their discourse and attitudes, but it has not ofered a concrete avenue for changing actual practice (Roush & Sharby, 2011).

Disability Through the Lens of Afordances

307

Despite its many positive contributions, the social model has been criticized by disability scholars, led by many in the feminist tradition, because it lacks constructs to capture how an individual’s lived experiences are shaped in meaningful ways by bodily impairments (French, 1993) (Crow, 1992; Shakespeare, 2006; Sherry, 2013; Siebers, 2001). As individuals, most of us simply cannot pretend with any conviction that our impairments are irrelevant because they infuence every aspect of our lives. We must fnd a way to integrate them into our whole experience and identity for the sake of our physical and emotional well-being, and subsequently, for our capacity to work against disability. —Crow (1992, p. 7) The call from these scholars is for a relational understanding of disability, that is, a model that conceptualizes disability as the outcome of interactions between a person with bodily impairments and the contexts in which they conduct their lives. The relational model that has most infuenced rehabilitation (in general) and physical therapy (in particular) is the International Classifcation of Functioning, Disability, and Health (ICF), developed by the World Health Organization (WHO). The ICF model (Table 16.1) catalyzed many desirable changes in physical therapy research and practice. One notable example of this change is how it prompted research to better understand the impact of health conditions and interventions on functioning at diferent levels (e.g., Brugnaro et  al., 2022; dos Santos et  al., 2012; Ferguson et  al., 2014): the level of the body (integrity of its structures and functions), the level of the whole person (activity performance), and the level of the whole person in a social context (or participation). An important lesson from this work is that interventions focused on addressing impairments in body structures and functions do not always translate to improvements in activity and participation (e.g., Dodd et al., 2002; Morris et al., 2004; Reedman et al., 2017; Ross et al., 2016). Relatedly, interventions that target activity practice within a supportive environmental context can promote functioning at the level of activity and participation even when bodily impairments are not directly addressed (e.g., Reedman et  al., 2019; Rensink et  al., 2009; Schwellnus et  al., 2020). This type of understanding has prompted some promising changes in physical therapy practice. For instance, the rehabilitation of individuals with spinal cord injuries often focuses on promoting possibilities for locomotion in the new paralyzed state (e.g., training to locomote on wheelchair), guided by the understanding that this mode of locomotion can be efective under appropriate environmental conditions (Shakespeare et al., 2018).

308

Paula L. Silva and Sarah M. Schwab

Despite its promises, the ICF has proven both slow and limited in catalyzing a more profound and general transformation in physical therapy practice, which continues to be broadly predicated on normalization principles (Gibson, 2016; Guccione et al., 2019; Imrie, 2000). Examples include interventions to increase symmetry in the patterns of walking expressed by individuals with hemiplegia following stroke (Meder et  al., 2022), to correct gait mechanics in individuals with chronic knee pain (Segal et al., 2015), to increase joint mobility and fexibility in children with cerebral palsy (Wiart et al., 2008), and to teach children and adults with neurological impairments how to use “proper” postures and movements in diferent activities (Johnson, 2009; Michielsen et  al., 2019). One reason for this is that under the ICF framework, individual and environmental factors continue to be defned independently from each other, refecting dualistic thinking consistent with Cartesian materialism. It follows that the ICF shares with medical models the use deviations from statistical norms to describe impairments in the disabled body: [I]mpairment is a loss or deviation from certain generally accepted population standards in the biomedical status of the body and its functions. . . . Abnormality here is used strictly to refer to a signifcant variation from established statistical norms . . . and should be used only in this sense. —World Health Organization (2007, p. 213) That is, impairments continue to be operationalized as “undesirable” deviations of the disabled body from non-disabled norms. The addition of the clause “only in this sense” to the defnition of impairment in the ICF is certainly an attempt to remove the value-judgment from such deviations. Nonetheless, it roughly equates the singularities expressed in disabled bodies with loss, without consideration of whether and how these impact a person’s experience, given their way of living. In doing so, the ICF reproduces the medicalized view of the disabled body as pathological or defective. This view, even if unarticulated and implicit (perhaps even unknown), continues to conceptualize the disabled body as a source of constraint and difculty with the only option being attempts to normalize it. The work of Moll and Cott (2013) is particularly insightful in this respect. They interviewed nine individuals with cerebral palsy who reported having experienced intensive rehabilitation focused on “normalizing” movements, particularly, their pattern of walking. Participants reported that once they reached adolescence, they were deemed by the medical team to have achieved their potential, meaning that they were as close as they would be to normative ideals. With this very limited view of potential in

Disability Through the Lens of Afordances

309

mind, rehabilitation professionals conclude there is “nothing else to be done.” This pervasive idea within physical therapy practice that there is nothing else to be done beyond normalization is consistent with a narrow, Cartesian understanding of the disabled body. A fundamentally diferent understanding and view of the disabled body is needed to reimagine physical therapy practice so that it can more closely align with the vision, goals, and expectations of the disability community and disability scholars and be a more impactful resource to those whom it serves. An Ecological Approach to Disability: Reconceptualizing the Disabled Body Lived, or phenomenal space, is thus not abstract, or geometric or thought; rather it arises out of motility, and lived relations of space are generated by the capacities of the body’s motion and the intentional relations that motion constitutes. —Sobchack (2005, p. 54)

Models of disability inspired in the Ecological approach—which we will jointly refer to as ecological models—have been proposed independently by Dokumaci (2017, 2023) and by Toro, Kiverstein, and Rietveld (Toro et  al., 2020). Ecological models difer in their details (Table 16.2), but they converge on the proposition that disability is experienced when the usual afordances in our current human environments are pervasively insufcient to support individuals’ activities and ways of living. Dokumaci (Dokumaci, 2023) uses the term shrinkage to refer to the reduced functional space available for disabled individuals to go about their business in the world.

TABLE 16.2 Summary of ecological models of disability.

Disability experienced when the usual (canonical) afordances are pervasively insufcient to support individuals’ activities and ways of living “Shrinkage,” or reduced functional space and opportunities, creates individualenvironment “misft” Individual-environment tension can more frequently create feelings of “I cannot” Activist afordances emerge in response to limited ability to exploit canonical afordances Dynamical, bidirectional relation between disabled body and environment Consistent with radical or complex perspectives of the body from phenomenology Body is described as it is experienced and not as deviations from “norms,” which allows for an appreciation of the unique capacities of disabled bodies

310

Paula L. Silva and Sarah M. Schwab

Shrinkage occurs, as we have discussed, due to historical and cultural processes that have structured human environments with features and practices that are insensitive to and, hence, create misfts for disabled bodies (Dokumaci, 2017, 2023; Toro et  al., 2020). In this sense, ecological models reinforce the lesson of social models that the challenges experienced by disabled people implicate a social disadvantage. However, proponents of these models also recognize that the feld of afordances is reconfgured depending on the extent to which individuals become subjectively attuned to their bodies’ potential to perform the accessible features that the environment lacks. As we have shown, activist afordances resolve some of the tension disabled individuals experience with their environment, making activity possible. Activist afordances are often precarious and emerge because disabled people cannot exploit the usual (canonical) functionalities of environmental features (Dokumaci, 2017, 2023). Activist afordances, thus, are an expression of necessity not abundance. Nonetheless, activist afordances highlight the agency of disabled individuals as bodies in action. It is evidence that disabled bodies, because of their singular characteristics, can shape the environment and expand its possibilities. Diferently from the social model, a bidirectional, dynamic relation between the disabled body and environment is implicated. Ecological models, thus, resonate with a so-called radical (Chemero, 2009; Di Paolo et al., 2017; van Gelder, 1998) or complex (Siebers, 2001, 2013) perspective on the body, rooted in phenomenology (cf. Käufer & Chemero, 2021). Phenomenology as a tradition denies the idea that the mind (or self) is separate from the body (MerleauPonty, 1945). Accordingly, scholars from this tradition focus on the subjective experience of being a self as a body rather than a self that has a body. This distinction is crucial because it requires a conceptualization of the body as it is experienced. Two modes of experience—“body as subject” and “body as object”— have been described in prior work. Arguably, the experience of body as subject is prominent when we readily encounter afordances to support our activities. Under such circumstances, we experience the world, through our bodies, as an expansive feld of possibilities, as the perfect stage for our projects (Merleau-Ponty, 1945). Awareness of our body’s physicality is at a minimum, and goal-directed movements are expressed freely and spontaneously. In this mode, the body “does what must be done” (Merleau-Ponty, 1973). In instances of individual-environment tension (when afordances are lacking or not readily presented to our perceptual systems), the experience of body as subject gives way to the experience of body as object. In these situations, the space “contracts toward the body” and is experienced as a set of obstacles to manage, and one is “forced to think

Disability Through the Lens of Afordances

311

about motility rather live it” (Sobchack, 2005, p. 58). The body’s motility must be deliberately planned, and when viable, efortfully produced. As we discussed, instances of individual-environment tension—which are substantially more prevalent for disabled individuals—can evolve to resolution when new forms of engagement with the world are discovered. Thus, the body as it is experienced—referred to as lived body—is best described as an ensemble of capacities that mediates our access to the world in service of our goals and needs (Sherry, 2013, following MerleauPonty, 1945). The impact of these capacities on behavior (in general) and on disability (in particular) cannot be understood by looking at the body as a set of strictly physical-biological mechanisms. Ecological models are, thus, in line with ICF in that they propose that disability can only be understood if we consider the dynamic interactions between an individual with a particular body and their environment. However, due to their grounding on the ecological concept of afordances and on radical or complex perspectives of the body, ecological models avoid the pitfalls of the ICF that arise from treating the individual and environment as two independent systems that interact. Rather than assuming individual-environment dualism, proponents of ecological models implicate the individual-environment system as the irreducible unit of analysis in explanations of what it means to be disabled. Taking the individual-environment system as the unit of analysis goes beyond asserting that both individual and environmental factors contribute to a phenomenon. It means that no one constituent of this system can be adequately specifed in the explanation of a phenomenon (here the experience of disability) without the specifcation of the other (Dewey & Bentley, 1960). For instance, a ramp leading up to the entrance of a building cannot be specifed as afording access or as a barrier to access without consideration of a person interested in entering the building. The ramp expands the feld of afordances for a person on a wheelchair but can potentially shrink it for person walking with a prosthetic leg and a cane if other options (stairs) are not available. The experience may also change over time depending on their experience negotiating ramps. As Heft (1989) has proposed, over the course of many interactions, the meanings of particular environmental features may be transformed over time. The practical lesson here is that an analysis of disability in terms of afordances cannot be equated with an analysis of environmental features without carefully considering who is engaging with them, with what set of goals, and under what social-cultural contexts (Hutchby, 2001). Neither should such analysis be based on momentary individual-environment encounters. As we have seen, an experience of tension in such encounters can be transmuted into new opportunities for goal achievement.

312

Paula L. Silva and Sarah M. Schwab

Understanding the experience of disability in terms of afordances requires instead an analysis of the ever-changing relations between disabled people and the surrounding physical and social environment. This exercise is particularly crucial when considering a person’s response to so-called accessibility features and assistive technologies (Bloomfeld et al., 2010). The fip side is that the impact of individual factors (in particular bodily impairments) on disability can also only be apprehended if described in reference to a person’s activities within a physical-social-institutional environment. Consider, as illustration, a child with cerebral palsy who exhibits reduced head control (an impairment), such that when sitting on their wheelchair the head “falls” forward if displaced from the head support. Without any reference to a situated activity, a seemingly uncontrolled “head-fall” might be construed as an expression of pathological embodiment, something to be corrected to help with the child’s functioning (e.g., by use of rigid head support to impede the head from moving). Now imagine that the child without head control is Alice—the Brazilian junior writer we referred to earlier. As you might recall, Alice intentionally lets her head fall when a collaborator sweeping through a range of letters points to the one she needs to construct her words, sentences, and stories. In this context, Alice does not experience her “head-falls” as pathological. Rather, she uses it to direct a collaborator in her unique process of communicating her ideas. There is no question that impairments restrict movement possibilities, but within the range of possibilities available, afordances can be (and often are) created. Thus, diferent from the ICF, ecological models invite descriptions of the body as it is experienced, that is, in terms of its impact in the feld of afordances supporting a person’s day-to-day living and not in terms of deviations from norms. As we will show next, this conceptualization allows one to appreciate the unique capacities and potential of disabled bodies which physical therapists, as movement specialists, are uniquely positioned to support. A Paradigm-Shift in Physical Therapy Practice: Expanding Movement Possibilities for Afordance Creation The challenge is to function. I use this word advisedly and am prepared to fnd another if it ofends. People with disabilities want to be able to function: to live with their disability, to come to know their body, to accept what it can do, and to keep doing what they can for as long as they can. They do not want to feel dominated by the people on whom they depend for help, and they want to be able to imagine themselves in the world without feeling ashamed. —Siebers (2010, p. 69)

Disability Through the Lens of Afordances

313

The paradigm shift in physical therapy practice we are about to propose is predicated on the idea that freedom or liberation can come from embodiment, that is, from a person’s capacity for movement (Dokumaci, 2017; Grosz, 2010). This idea is made concrete in descriptions of activist afordances. These afordances are evidence that disabled people, as bodies in action (hands that see and talk, mouths that color, uncontrolled head movements that write, monopeds that perform virtuous pirouettes on stage, bodies on wheels that play basketball, and so on) have been establishing diferent relations with the surrounding environment, and in doing so creating conditions for new forms of living. The guiding principle for physical therapy practice we propose is to support disabled people in this creative process of “niche construction.” A physical therapy practice aligned with this principle would be defned by interventions designed to assist individuals with disability (1) to enhance their attunement and control over the ensemble of capacities that defne their lived bodies and (2) to develop and maintain the capacities required to achieve their functional goals. The nature of specifc interventions will vary depending on the goals and needs of the recipient of care and, most importantly, on their evolving experience as bodies in action. That is, we must understand how they have been “performing” the afordances that support their activity and collaboratively determine how to facilitate that performance. Providing space for disabled individuals and their families to exchange experiences could be a fruitful avenue for re-imagining embodied-embedded strategies to achieve functional goals that can be supported by physical therapy interventions. From these collaborations, one can also begin to imagine ways to identify assistive equipment, technology, and accommodations that can more readily enable performance of a particular individual within particular contexts. The approach to physical therapy practice we propose respects insights from the few available studies that have interrogated disabled individuals about what they believe physical therapy could do for them (Bezmez, 2016; Clifton, 2014; Moll & Cott, 2013; Papadimitriou, 2008; Van de Velde et al., 2012). Most particularly, it proposes an emphasis on agency rather than autonomy and focuses on attunement and development of capacity for meaningful activities rather than normalization. Additionally, it values a collaborative approach that includes space for disabled individuals undergoing physical therapy to exchange experiences, including experiences of grief, which can assist them in managing the challenges of living with impairments. Our approach is also in line with innovative clinical-reasoning strategies that call upon therapists to move beyond the third-person perspective and give equal consideration to the frst- and second-person perspectives (Øberg et al., 2015). These shifts in clinical reasoning can minimize focus

314

Paula L. Silva and Sarah M. Schwab

on the mechanical body (“body as object”) and increase emphasis on the dynamics of lived bodily engagement between the client and therapist (“body as subject;” Øberg et al., 2015; Schwab et al., 2023). Under a second-person perspective, a therapist can expand knowledge of a client’s motor impairments, while a client can also gain more insight (through the frst- and second-person perspectives) into their own possibilities for improvement (Normann et  al., 2013). Involving people with disability in the clinical-reasoning process in this way creates “ecological niches,” where afordances can be increased by clinician infuence on clients, and vice versa, which can ultimately strengthen the therapeutic alliance (Shaw et al., 2022). The area of adaptive sports ofers examples of specifc training approaches that fully aligns with the proposed guiding principle for physical therapy practice (Hutzler, 2007). Consider for instance the case of a 16-year-old teenager (Eric; pseudonym) interested in joining a swimming practice. Eric presents with hemiplegia associated with cerebral palsy who at that point was already able to swim independently. The frst step taken by the coach was to inquire what Eric’s goal was: swimming as a sport (e.g., swimming for speed) or swimming for promotion of bodily capacity (e.g., swimming as exercise to improve strength and mobility of the impaired arm and leg). Eric was interested in practicing swimming as a sport. With this goal in mind, the coach analyzed Eric’s swim stroke and determined that the impaired arm was creating increased drag and limiting speed. Rather than trying to “normalize” the movements of the arm and preserve “standard stroke patterns,” the coach guided Eric to experiment with a diferent stroke pattern altogether: resting the impaired arm near the chest, allowing the opposite arm to develop full speed. Eric was also instructed to try breathing with a full body roll on the longitudinal axis, which increased the length of his stroke. The swimming mechanics Eric discovered and practiced along with appropriate equipment adjustments (e.g., wearing a buoyant suit to decrease leg sinking) accomplished a reduction in drag that a “normal stroke” could not have accomplished. Eric (with the assistance of his coach) made the water aford him the opportunity to swim at a fast pace. That is, the goal to swim competitively was achieved through increases in capacity (i.e., better individual-environment ft) not normalization. An example of intervention aligned with the proposed guiding principle within the rehabilitation context was reported by Darrah et al. (2011). The authors presented the case of a child whose goal was to fnger-feed himself Cheerios independently. The usual strategy in rehabilitation would be to teach and enforce the use of a pincer grasp that the child could not perform. With a clear focus on enabling the desired function, rather than normalizing movement, the therapist experimented with putting peanut butter

Disability Through the Lens of Afordances

315

on the fngertips of the child so that the food could stick to it. That is, an activist afordance was invented, which allowed the child to accomplish fnger feeding in one session. This therapeutic choice immediately provides the child opportunity to experience eating Cheerios in a diferent way, and the assumption of the therapist was that her active engagement with this particular task can scafold new capabilities and possibilities. Something to be investigated by future longitudinal studies designed to empirically validate this specifc intervention and others based on the principle of supporting disabled individuals in the creation of activist afordances and related accommodations when needed. It is important to note that physical therapy practice is not disconnected from social attitudes toward disability, which is a force toward maintenance of its status quo. Paralympic games have changed attitudes toward adaptive sports, which provides an important context for the successful application of Ecological principles in the training of disabled athletes. To be precise, “alternative” ways of swimming, running, playing basketball, tennis, and so on are currently celebrated in many societies for the achievements that they are. There is no commitment to “usual” ways of practicing each of the sports. A variety of ways of doing things are encouraged, accommodated, and celebrated. A similar attitude toward activist afordances that emerge in the context of everyday, mundane activities is critical for the proposed model of physical therapy to reach its full potential in supporting the bodily creativity of individuals with disability. Activist Afordances, Disability, and Physical Therapy: Implications for Afordance Theory and Research [W]hen the world’s surfaces become most unresponsive to the impairments, diseases, and pains they live with . . . the imaginary space of performance opens up. In this space, disabled people make up and make real action possibilities as if those missing world’s counterparts were present. . . . Activist afordances bend the seemingly fxed forms, sand the hard edges, and give movement to the rigid layering of the world AS IF it were habitable, in as yet unimagined and undreamed ways. —Dokumaci (2023, p. 8)

What Do We Understand by the Term Afordance?

We understand afordances as possibilities for action specifed by the mutuality (or ft) between properties of the environment and an individual’s lived body, that is, their ensemble of capacities or movement potentialities for accessing the environment in the service of their goals and needs. Our

316

Paula L. Silva and Sarah M. Schwab

understanding is thus in alignment with Heft’s idea that an afordance is specifed and thus perceived in relation to goal-directed (intentional) acts, not solely the body’s physical dimensions (Heft, 1989). What Role Does Afordance Play in Perception? Is It the Entity Organisms Perceive or the Means Through Which Organisms Perceive? Are Afordances the Only Perceptual Dependent Variables?

We take afordances as environmental features that organisms actively seek, perceive, and realize in serving their goals and plans. As we highlighted in this chapter, the process of development and socialization scaffolds the habitual skills that enhance an individual’s ft to the environment, and our collective social-cultural practices maintain and transform— albeit at a slower timescale—our environment’s materiality in ways that often presuppose these skills (Costall, 1995; Heft, 2007; Ingold, 2000; Lewontin, 1983; Withagen & van Wermeskerken, 2010). Thus, the individual-environment ft that specifes afordances and underlies their perception is neither a given nor incidental; it emerges from our individual and collective embodied actions in the world.

Why Is Afordance Any Better Than Stimulus? What Does a Theory of Afordance Suggest That Stimuli Cannot; How Has It Moved the Needle Past Gibson 1960’s Recognition of How Little We Can Defne Stimuli?

Afordances are descriptions of the environment in terms of what organisms can do with it. If afordances can be perceived—which a large body of research suggests is the case—then they can (diferently from the stimulus) be considered a source of embodied and embedded knowledge about what actions are possible, given desired functional outcomes within a given environment. Notably, a theory of afordances suggests that the lived body and its agency (rather than strictly mental or physiological processes) should be centered on explaining a person’s subjective experiences of the environment. Notably, it invites us to think of a person’s experience of the environment not as something that lives in their mind but as something natural, palpable, generated by the capacities of the body in motion and by the relations that this motion can and cannot create in the context of intentional acts. This understanding grounds ecological models of disability, which invites us to conceptualize it as a particular mode of engaging with a world of restricted possibilities. We exemplifed this mode of engagement in the bi-directional, dialectic dynamics of individuals with a

Disability Through the Lens of Afordances

317

disability with the surrounding environment. These dynamics explain the challenges disabled individuals experience and how they actively resolve them through their imaginative performances that create new, unique possibilities for their being in the world. Notably, the ecological models of disability we described—with important practical implications for rehabilitation practice—would certainly not arise from behavior theories grounded on stimulus-response. What Is the Connection Between Afordances, Behavioral Scale, and Intention? How Distant or Reluctant Can a Perceiving-Acting System Be Before Afordances Come Into Play? Do Afordances Exert Their Role Only When the System Is Close or Willing Enough?

We resonate with Heft’s claim that afordances and intentions are coimplicated (Heft, 1989). A person’s intentions drive their action relative to the environment in search of afordances that support further action leading to desired functional outcomes. Individual-environment encounters can be described at the spatial-temporal scale of behavior. The experience of disability shows us that whether such encounters result in ft (and consequently goal-action continuity), tension (and consequently a disruption in goal-action continuity), or tension-resolution (new routes that reestablishes goal-action continuity after an experienced disruption) depends critically on the history of interactions of a particular individual with their environment. Notably, this history is embedded in relatively slow-changing social-cultural practices that have shaped the material properties of human environments to reciprocate (and thus provide readily accessible afordances for) abled-bodied individuals. The implication we wish to highlight is that usual (or canonical) afordances of the environment—afordances that have been selected and sustained through relatively stable practices—infuence the nature of interactions any individual has with their environments (willingly or not). We do not perceive all potential functions of environmental features with equal probability. We are socialized not to do so. When canonical afordances do not reciprocate a person’s evolving body (e.g., due to impairments), they are more likely to experience tension in their encounters with the environment and have their plans and projects forestalled. The experience of tension of an individual with a particular environment feature can be transformed over time as the individual learns to relate to it in novel ways through their unique bodily performances. Performance (which itself implicates processes evolving at a faster pace than behavior itself) is the medium for changing the possibilities for action that a person can access from a given situation for goal achievement.

318

Paula L. Silva and Sarah M. Schwab

How Do Systems Engage With Afordances as They Move Among Tasks and Intentions?

Material features of the environment are crucially relevant to understanding how individuals engage with afordances as they move through tasks and intentions. Nevertheless, more is needed. The material environment functions much like a stage does for actors in a play. That is, actors must operate within the possibilities allowed by the physical properties of surfaces and objects that populate the stage. Nevertheless, by moving through that space—by performing—an actor realizes particular meanings of these surfaces, objects, and co-actors relevant to their unique participation in the plot. The theory of afordances, thus, pushes us to think of meaning as generated by the capacities of a body in motion and by the “fts” that this motion can and cannot create in the context of intentional acts. When the ft between an individual’s lived body and the surrounding environment is not readily present but remote—as is often the case for disabled individuals—there is a shrinking in the feld of afordances with related restrictions to bring about desired functional outcomes. Disabled people are not passive when faced with such shrinking. Both collectively (through social movements) and individually (through their unique bodily movements) “enact and bring into being the worlds that are not available to them.” The experience of disability thus highlights the active participation of the perceiver in bringing into existence novel functionalities (activist afordances) for environmental features to support and expand their repertoire of intentional acts. Dokumaci’s work on activist afordances lets us see that perceiving and actualizing usual afordances in the service of task goals (i.e., afordances that have been historically explored) and afordances that have yet to be utilized (i.e., afordances that are too remote and, thus, unlikely to be perceived) are not the same thing. The latter requires additional, creative, and efortful work that the former does not (just like a poorly designed stage requires more imaginative and efortful performances from actors). Dokumaci’s theory of activist afordances names, recognizes, and invites us to better understand the processes supporting this additional work that otherwise has remained unaccounted for. Future Directions

Dokumaci’s concept of activist afordances invites a more central role for the temporality of performance (i.e., of embodied, embedded action) in afordance theory, which opens up new, underexplored avenues for afordance research. An important frst step is to characterize the perceptual and motor performances of individuals of diferent ages with and without disability in the process of invention of new afordances under conditions of

Disability Through the Lens of Afordances

319

individual-environment tension that precludes immediate achievement of task goals. Instances of individual-environment tension may be identifed by disabled individuals in “real life” or created in “virtual reality” so that the process of resolution culminating in the creation of new afordances can be tracked, characterized, and conceptualized. This line of work invites a number of new questions. For instance, what characteristics of perceptual activity and bodily performances forecast transitions from individual-environment tension to resolution culminating in the emergence of activist afordances? How might other people (physical therapists, intimate partners, strangers) facilitate these transitions? Does a contextual intervention that resolves a particular instance of individualenvironment tension facilitate resolution in novel instances of individual-environment tension not directly addressed? What characteristics of interpersonal dynamics map on to successful joint action or collaborations between individuals with diferent capabilities for a given task? This list of questions is obviously not exhaustive. These simply represent directions we wish to explore with our own work moving forward to guide and support the design of physical therapy interventions in line with the principle of practice we have proposed. Note 1. Individuals within the disability community have varied preferences about identity-frst and person-frst language, including diferences in the language they use to describe themselves and the language they prefer to be used to describe them (Andrews et al., 2022). In this chapter, we use both. We wish to emphasize however that disability language is continuously evolving and that each person’s disability language preferences should always be respected and supported.

Reference List Abberley, P. (1995). Disabling ideology in health and welfare—The case of occupational therapy. Disability & Society, 10(2), 221–232. https://doi. org/10.1080/09687599550023660 Alt, J. M., Kiefer, A. W., MacPherson, R., Davis, T. J., & Silva, P. L. (2021). The efect of navigation demand on decision making in a dynamic, sport-inspired virtual environment. Journal of Sport & Exercise Psychology, 43(5), 375–386. https://doi.org/10.1123/jsep.2020-0320 Andrews, E. E., Powell, R. M., & Ayers, K. (2022). The evolution of disability language: Choosing terms to describe disability. Disability and Health Journal, 15(3), 101328. https://doi.org/10.1016/j.dhjo.2022.101328 Angeli, J. M., Harpster, K., Huijs, L., Seid, M., Sheehan, A., & Schwab, S. M. (2019). Patient-centered goal setting in developmental therapy: Discordance between documented goals and caregiver-perceived goals. Pediatric Quality & Safety, 4(4), e199. https://doi.org/10.1097/pq9.0000000000000199

320

Paula L. Silva and Sarah M. Schwab

Beaudry, J.-S. (2016). Beyond (models of) disability? The Journal of Medicine and Philosophy: A Forum for Bioethics and Philosophy of Medicine, 41(2), 210– 228. https://doi.org/10.1093/jmp/jhv063 Bezmez, D. (2016). Looking for a “cure”: Negotiating “walking” in a Turkish rehabilitation hospital. Disability & Society, 31(3), 389–405. https://doi.org/10 .1080/09687599.2016.1167670 Bloomfeld, B. P., Latham, Y., & Vurdubakis, T. (2010). Bodies, technologies and action possibilities: When is an afordance? Sociology, 44(3), 415–433. https:// doi.org/10.1177/0038038510362469 Bones, P. D. C., Gullion, J. S., & Barber, D. (2022). Redefning Disability. Brill. https://doi.org/10.1163/9789004512702 Brugnaro, B. H., Oliveira, M. F. P., de Campos, A. C., Pavão, S. L., & Rocha, N. A. C. F. (2022). Postural control in down syndrome and relationships with the dimensions of the International classifcation of functioning, disability and health—A systematic review. Disability and Rehabilitation, 44(11), 2207–2222. https://doi.org/10.1080/09638288.2020.1830439 Bunbury, S. (2019). Unconscious bias and the medical model: How the social model may hold the key to transformative thinking about disability discrimination. International Journal of Discrimination and the Law, 19(1), 26–47. https://doi. org/10.1177/1358229118820742 Campbell, J., Oliver, M., & Oliver, M. (1996). Disability Politics: Understanding Our Past, Changing Our Future. Routledge. Charlton, J. I. (1998). Nothing About Us Without Us: Disability Oppression and Empowerment. Univ of California Press. Chemero, A. (2009). Radical Embodied Cognitive Science. MIT Press. Clifton, S. (2014). Grieving my broken body: An autoethnographic account of spinal cord injury as an experience of grief. Disability and Rehabilitation, 36(21), 1823–1829. https://doi.org/10.3109/09638288.2013.872202 Costall, A. (1995). Socializing afordances. Theory & Psychology, 5(4), 467–481. https://doi.org/10.1177/0959354395054001 Costall, A. (2006). On being the right size: Afordances and the meaning of scale. In G. Lock & B. L. Molyneaux (Eds.), Confronting Scale in Archaeology: Issues of Theory and Practice (pp.  15–26). Springer. https://doi. org/10.1007/0-387-32773-8_2 Crow, L. (1992). Renewing the Social Model of Disability. University of Leeds. Darrah, J., Law, M. C., Pollock, N., Wilson, B., Russel, D. J., Walter, S. D., Rosenbaum, P., & Galuppi, B. (2011). Context therapy: A new intervention approach for children with cerebral palsy. Developmental Medicine & Child Neurology, 53(7), 615–620. https://doi.org/10.1111/j.1469-8749.2011.03959.x Davis, L. J. (1995). Enforcing Normalcy: Disability, Deafness, and the Body. Verso. Dewey, J., & Bentley, A. F. (1960). Knowing and the Known (Vol. 111). Beacon Press. Di Paolo, E., Buhrmann, T., & Barandiaran, X. (2017). Sensorimotor Life: An Enactive Proposal. Oxford University Press. Dodd, K. J., Taylor, N. F., & Damiano, D. L. (2002). A systematic review of the efectiveness of strength-training programs for people with cerebral palsy. Archives of Physical Medicine and Rehabilitation, 83(8), 1157–1164. https:// doi.org/10.1053/apmr.2002.34286

Disability Through the Lens of Afordances

321

Dokumaci, A. (2013). On falling III. Performance Research, 18(4), 107–115. https://doi.org/10.1080/13528165.2013.814334 Dokumaci, A. (2016). Micro-activist afordances of disability: Transformative potential of participation. In A. Ganzert, I. Otto, M. Denecke & R. Stock (Eds.), ReClaiming Participation: Technology, Mediation, Collectivity (pp.  67–84). Transcript Publishing. Dokumaci, A. (2017). Vital afordances, occupying niches: An ecological approach to disability and performance. Research in Drama Education: The Journal of Applied Theatre and Performance, 22(3), 393–412. https://doi.org/10.1080/13 569783.2017.1326808 Dokumaci, A. (2019). People as afordances: Building disability worlds through care intimacy. Current Anthropology, 61(S21), S97–S108. https://doi. org/10.1086/705783 Dokumaci, A. (2023). Activist Afordances: How Disabled People Improvise More Habitable Worlds. Duke University Press. dos Santos, A. N., Pavão, S. L., de Campos, A. C., & Rocha, N. A. C. F. (2012). International classifcation of functioning, disability and health in children with cerebral palsy. Disability and Rehabilitation, 34(12), 1053–1058. https://doi. org/10.3109/09638288.2011.631678 Fajen, B. R., & Matthis, J. S. (2011). Direct perception of action-scaled afordances: The shrinking gap problem. Journal of Experimental Psychology: Human Perception and Performance, 37(5), 1442–1457. https://doi.org/10.1037/ a0023510 Feldner, H. A., Evans, H. D., Chamblin, K., Ellis, L. M., Harniss, M. K., Lee, D., & Woiak, J. (2022). Infusing disability equity within rehabilitation education and practice: A qualitative study of lived experiences of ableism, allyship, and healthcare partnership. Frontiers in Rehabilitation Sciences, 3, 947592. https:// doi.org/10.3389/fresc.2022.947592 Ferguson, G. D., Jelsma, J., Versfeld, P., & Smits-Engelsman, B. C. M. (2014). Using the ICF framework to explore the multiple interacting factors associated with developmental coordination disorder. Current Developmental Disorders Reports, 1(2), 86–101. https://doi.org/10.1007/s40474-014-0013-7 Finklestein, V. (1993). Representing disability. In J. Swain, C. Thomas, C. Barnes, & S. French (Eds.), Disabling Barriers—Enabling Environments. Sage. French, S. (1993). Disability, impairment or something in between? In J. Swain, V. Finkelstein, S. French, & M. Oliver (Eds.), Disabling Barriers—Enabling Environments (pp. 17–25). Sage. Gibson, B. (2016). Rehabilitation: A Post-critical Approach. CRC Press. Gibson, J. J. (1979). The Ecological Approach to Visual Perception. Houghton Mifin. Gibson, J. J. (1982). Afordances and behavior. In E. Reed & R. Jones (Eds.), Reasons for Realism: Selected Essays of James J. Gibson (p. 558). Lawrence Erlbaum Associates, Inc. Grosz, E. (2010). Feminism, materialism, and freedom. In D. Coole & S. Frost (Eds.), New Materialisms: Ontology, Agency, and Politics. Duke University Press. Guccione, A. A., Neville, B. T., & George, S. Z. (2019). Optimization of movement: A dynamical systems approach to movement systems as emergent phenomena. Physical Therapy, 99(1), 3–9. https://doi.org/10.1093/ptj/pzy116

322

Paula L. Silva and Sarah M. Schwab

Gustavsson, A. (2008). Resistance, Refection and Change: Nordic Disability Research. Studentlitteratur. Haegele, J. A., & Hodge, S. (2016). Disability discourse: Overview and critiques of the medical and social models. Quest, 68(2), 193–206. https://doi.org/10.1080/ 00336297.2016.1143849 Hahn, H. (1988). The politics of physical diferences: Disability and discrimination. Journal of Social Issues, 44(1), 39–47. https://doi.org/10.1111/j.1540-4560.1988. tb02047.x Heft, H. (1989). Afordances and the body: An intentional analysis of Gibson’s ecological approach to visual perception. Journal for the Theory of Social Behaviour, 19(1), 1–30. https://doi.org/10.1111/j.1468-5914.1989.tb00133.x Heft, H. (2001). Ecological Psychology in Context: James Gibson, Roger Barker, and the Legacy of William James’s Radical Empiricism. Psychology Press. Heft, H. (2003). Afordances, dynamic experience, and the challenge of reifcation. Ecological Psychology, 15(2), 149–180. https://doi.org/10.1207/S15326 969ECO1502_4 Heft, H. (2007). The social constitution of perceiver-environment reciprocity. Ecological Psychology, 19(2), 85–105. https://doi.org/10.1080/10407410701331934 Hogan, A. J. (2019). Moving away from the “medical model”: The development and revision of the world health organization’s classifcation of disability. Bulletin of the History of Medicine, 93(2), 241–269. Holt, K. G., Saltzman, E., Ho, C.-L., Kubo, M., & Ulrich, B. D. (2006). Discovery of the pendulum and spring dynamics in the early stages of walking. Journal of Motor Behavior, 38(3), 206–218. https://doi.org/10.3200/JMBR.38.3.206-218 Hutchby, I. (2001). Technologies, texts and afordances. Sociology, 35(2), 441– 456. https://doi.org/10.1017/S0038038501000219 Hutzler, Y. (2007). A systematic ecological model for adapting physical activities: Theoretical foundations and practical examples. Adapted Physical Activity Quarterly, 24(4), 287–304. https://doi.org/10.1123/apaq.24.4.287 Imrie, R. (2000). Disability and discourses of mobility and movement. Environment and Planning a: Economy and Space, 32(9), 1641–1656. https://doi. org/10.1068/a331 Ingold, T. (2000). The Perception of the Environment: Essays on Livelihood, Dwelling and Skill. Psychology Press. Johnson, P. (2009). Assessment and clinical reasoning in the Bobath concept. Bobath Concept, 43. Käufer, S., & Chemero, A. (2021). Phenomenology: An Introduction (2nd ed.). Polity Press. Leder, D. (1990). The Absent Body. University of Chicago Press. Lewontin, R. C. (1983). Gene, organism and environment. In D. S. Bendall (Ed.), Evolution from Molecules to Men (pp. 273–286). Cambridge University Press. Lonsdale, S. (1990). Women and Disability: The Experience of Physical Disability among Women. Springer. Mark, L. S. (1987). Eyeheight-scaled information about afordances: A study of sitting and stair climbing. Journal of Experimental Psychology: Human Perception and Performance, 13(3), 361–370. https://doi.org/10.1037/0096-1523.13.3.361 Marks, D. (1997). Models of disability. Disability and Rehabilitation, 19(3), 85–91. https://doi.org/10.3109/09638289709166831

Disability Through the Lens of Afordances

323

Martínez-Pernía, D., González-Castán, Ó., & Huepe, D. (2017). From ancient Greece to the cognitive revolution: A comprehensive view of physical rehabilitation sciences. Physiotherapy Theory and Practice, 33(2), 89–102. https://doi.org /10.1080/09593985.2016.1266720 Meder, K. G., LoJacono, C. T., & Rhea, C. K. (2022). A systematic review of non-pharmacological interventions to improve gait asymmetries in neurological populations. Symmetry, 14(2), 281. https://doi.org/10.3390/sym14020281 Merleau-Ponty, M. (1945). Phenomenology of Perception. Duquesne University Press. Merleau-Ponty, M. (1973). The Prose of the World. Northwestern University Press. Michielsen, M., Vaughan-Graham, J., Holland, A., Magri, A., & Suzuki, M. (2019). The Bobath concept—A model to illustrate clinical practice. Disability and Rehabilitation, 41(17), 2080–2092. https://doi.org/10.1080/09638288.20 17.1417496 Moll, L. R., & Cott, C. A. (2013). The paradox of normalization through rehabilitation: Growing up and growing older with cerebral palsy. Disability and Rehabilitation, 35(15), 1276–1283. https://doi.org/10.3109/09638288.2012.7 26689 Morris, S. L., Dodd, K. J., & Morris, M. E. (2004). Outcomes of progressive resistance strength training following stroke: A systematic review. Clinical Rehabilitation, 18(1), 27–39. https://doi.org/10.1191/0269215504cr699oa Nicholls, D. A. (2017). The End of Physiotherapy. Routledge. Nicholls, D. A. (2022). Physiotherapy Otherwise. Tuwhera Open Books. Nicholls, D. A., & Gibson, B. E. (2010). The body and physiotherapy. Physiotherapy Theory and Practice, 26(8), 497–509. https://doi.org/10.3109/0959398 1003710316 Normann, B., Sørgaard, K. W., Salvesen, R., & Moe, S. (2013). Contextualized perceptions of movement as a source of expanded insight: People with multiple sclerosis’ experience with physiotherapy. Physiotherapy Theory and Practice, 29(1), 19–30. https://doi.org/10.3109/09593985.2012.698717 Øberg, G. K., Normann, B., & Gallagher, S. (2015). Embodied-enactive clinical reasoning in physical therapy. Physiotherapy Theory and Practice, 31(4), 244– 252. https://doi.org/10.3109/09593985.2014.1002873 Okoro, C. A., Hollis, N. D., Cyrus, A. C., & Grifn-Blake, S. (2018). Prevalence of disabilities and health care access by disability status and type among adults— United States, 2016. Morbidity and Mortality Weekly Report, 67(32), 882–887. https://doi.org/10.15585%2Fmmwr.mm6732a3 Oliver, M. (1990). The politics of disablement—New social movements. In M. Oliver (Ed.), The Politics of Disablement (pp. 112–131). Palgrave Macmillan. https://doi.org/10.1007/978-1-349-20895-1_8 Oliver, M. (1993). What’s so wonderful about walking. In Inaugural Professional Lecturer. University of Greenwich. Oudejans, R. R., Michaels, C. F., van Dort, B., & Frissen, E. J. P. (1996). To cross or not to cross: The efect of locomotion on street-crossing behavior. Ecological Psychology, 8(3), 259–267. https://doi.org/10.1207/s15326969eco0803_4 Papadimitriou, C. (2008). Becoming en-wheeled: The situated accomplishment of re-embodiment as a wheelchair user after spinal cord injury. Disability & Society, 23(7), 691–704. https://doi.org/10.1080/09687590802469420

324

Paula L. Silva and Sarah M. Schwab

Paterson, K., & Hughes, B. (2000). Disabled bodies. In P. Hancock, B. Hughes, E. Jagger, K. Paterson, R. Russell, E. Tulle-Winton, & M. Tyler (Eds.), The Body, Culture and Society (pp. 29–44). Open University Press. Rahlin, M., Barnett, J., Becker, E., & Fregosi, C. M. (2019). Development through the lens of a perception-action-cognition connection: Recognizing the need for a paradigm shift in clinical reasoning. Physical Therapy, 99(6), 748–760. https:// doi.org/10.1093/ptj/pzz026 Reedman, S. E., Boyd, R. N., & Sakzewski, L. (2017). The efcacy of interventions to increase physical activity participation of children with cerebral palsy: A systematic review and meta-analysis. Developmental Medicine & Child Neurology, 59(10), 1011–1018. https://doi.org/10.1111/dmcn.13413 Reedman, S. E., Boyd, R. N., Trost, S. G., Elliott, C., & Sakzewski, L. (2019). Efcacy of participation-focused therapy on performance of physical activity participation goals and habitual physical activity in children with cerebral palsy: A randomized controlled trial. Archives of Physical Medicine and Rehabilitation, 100(4), 676–686. https://doi.org/10.1016/j.apmr.2018.11.012 Rensink, M., Schuurmans, M., Lindeman, E., & Hafsteinsdóttir, T. (2009). Task-oriented training in rehabilitation after stroke: Systematic review. Journal of Advanced Nursing, 65(4), 737–754. https://doi.org/10.1111/j.1365-2648.2008.04925.x Richardson, J. (2022). Finding my way in a society where I don’t ft. In P. D. C. Bones, J. S. Gullion, & D. Barber (Eds.), Redefning Disability (pp. 60–71). Brill. https://doi.org/10.1163/9789004512702_009 Ross, S. M., MacDonald, M., & Bigouette, J. P. (2016). Efects of strength training on mobility in adults with cerebral palsy: A systematic review. Disability and Health Journal, 9(3), 375–384. https://doi.org/10.1016/j.dhjo.2016.04.005 Roush, S. E., & Sharby, N. (2011). Disability reconsidered: The paradox of physical therapy. Physical Therapy, 91(12), 1715–1727. https://doi.org/10.2522/ ptj.20100389 Sargent, J., Clarke, M., Price, K., Grifths, T., & Swettenham, J. (2013). Use of eyepointing by children with cerebral palsy: What are we looking at? International Journal of Language & Communication Disorders, 48(5), 477–485. https://doi. org/10.1111/1460-6984.12026 Schwab, S. M., Andrade, V., Santos Moreira, T., Cavanaugh, J. T., Vaz, D. V, & Silva, P. L. (2023). Narrowing the physiotherapy knowledge-practice gap: Faculty training beyond the health sciences. Physiotherapy Theory and Practice, 39(4), 675–689. https://doi.org/10.1080/09593985.2022.2027585 Schwab, S. M., Spencer, C., Carver, N. S., Andrade, V., Dugan, S., Greve, K., & Silva, P. L. (2022). Personal factors understood through the ecological-enactive model of disability and implications for rehabilitation research. Frontiers in Rehabilitation Sciences, 3, 954061. https://doi.org/10.3389/fresc.2022.954061 Schwellnus, H., King, G., Baldwin, P., Keenan, S., & Hartman, L. R. (2020). A solution-focused coaching intervention with children and youth with cerebral palsy to achieve participation-oriented goals. Physical & Occupational Therapy in Pediatrics, 40(4), 423–440. https://doi.org/10.1080/01942638.2020.1711841 Segal, N. A., Glass, N. A., Teran-Yengle, P., Singh, B., Wallace, R. B., & Yack, H. J. (2015). Intensive gait training for older adults with symptomatic knee osteoarthritis. American Journal of Physical Medicine & Rehabilitation, 94(10S), 848–858. https://doi.org/10.1097/PHM.0000000000000264

Disability Through the Lens of Afordances

325

Shakespeare, T. (2006). The social model of disability. The Disability Studies Readerity Studies Reader, 2, 197–204. Shakespeare, T., Cooper, H., Bezmez, D., & Poland, F. (2018). Rehabilitation as a disability equality issue: A conceptual shift for disability studies? Social Inclusion, 6(1), 61–72. Shaw, R., Abbey, H., Casals-Gutiérrez, S., & Maretic, S. (2022). Reconceptualizing the therapeutic alliance in osteopathic practice: Integrating insights from phenomenology, psychology and enactive inference. International Journal of Osteopathic Medicine, 46, 36–44. https://doi.org/10.1016/j.ijosm.2022.06.003 Sherry, M. (2013). If I Only Had a Brain: Deconstructing Brain Injury. Routledge. Shogan, D. (1998). The social construction of disability: The impact of statistics and technology. Adapted Physical Activity Quarterly, 15(3), 269–277. https:// doi.org/10.1123/apaq.15.3.269 Siebers, T. (2001). Disability in theory: From social constructionism to the new realism of the body. American Literary History, 13(4), 737–754. www.jstor. org/stable/3054594 Siebers, T. (2010). Disability Aesthetics. University of Michigan Press. Siebers, T. (2013). Disability and the theory of complex embodiment—For identity politics in a new register. The Disability Studies Reader, 4, 278–297. Sobchack, V. (2005). “Choreography for one, two, and three legs” (a phenomenological meditation in movements). Topoi, 24(1), 55–66. https://doi.org/10.1007/ s11245-004-4161-y Stein, M. A., Stein, P., Weiss, D., & Lang, R. (2007). Convention on the rights of persons with disabilities. European Journal of Health Law, 14(3), 281–298. Toro, J., Kiverstein, J., & Rietveld, E. (2020). The ecological-enactive model of disability: Why disability does not entail pathological embodiment. Frontiers in Psychology, 11, 1162. https://doi.org/10.3389/fpsyg.2020.01162 Van de Velde, D., Bracke, P., Van Hove, G., Josephsson, S., Devisch, I., & Vanderstraeten, G. (2012). The illusion and the paradox of being autonomous, experiences from persons with spinal cord injury in their transition period from hospital to home. Disability and Rehabilitation, 34(6), 491–502. https://doi.org/ 10.3109/09638288.2011.608149 van der Kamp, J., Savelsbergh, G. J. P., & Davis, W. E. (1998). Body-scaled ratio as a control parameter for prehension in 5- to 9-year-old children. Developmental Psychobiology, 33(4), 351–361. https://doi.org/10.1002/(SICI)1098-2302 (199812)33:43.0.CO;2-P van Gelder, T. (1998). The dynamical hypothesis in cognitive science. Behavioral and Brain Sciences, 21(5), 615–628. https://doi.org/10.1017/S0140525X98001733 Vaz, D. V., Silva, P. L., Mancini, M. C., Carello, C., & Kinsella-Shaw, J. (2017). Towards an ecologically grounded functional practice in rehabilitation. Human Movement Science, 52, 117–132. https://doi.org/10.1016/j.humov.2017.01.010 Warren, W. H. (1984). Perceiving afordances: Visual guidance of stair climbing. Journal of Experimental Psychology: Human Perception and Performance, 10(5), 683–703. https://doi.org/10.1037/0096-1523.10.5.683 Watermeyer, B., Lorenzo, T., Priestley, M., & Schneider, M. (2006). Disability and Social Change: A South African Agenda. HSRC Press. Wendell, S. (1989). Toward a feminist theory of disability. Hypatia, 4(2), 104–124. https://doi.org/10.1111/j.1527-2001.1989.tb00576.x

326

Paula L. Silva and Sarah M. Schwab

Wiart, L., Darrah, J., & Kembhavi, G. (2008). Stretching with children with cerebral palsy: What do we know and where are we going? Pediatric Physical Therapy, 20(2), 173–178. https://doi.org/10.1097/PEP.0b013e3181728a8c Withagen, R., & van Wermeskerken, M. (2010). The role of afordances in the evolutionary process reconsidered: A niche construction perspective. Theory & Psychology, 20(4), 489–510. https://doi.org/10.1177/0959354310361405 World Health Organization (2007). International Classifcation of Functioning, Disability, and Health: Children & Youth Version: ICF-CY. World Health Organization.

17 “I GOT ALL I NEED TO KNOW ABOUT AFFORDANCES FROM NORMAN” What Engineers, Designers, and Architects Need to Know About Afordances Balagopal Raveendranath, Elenah Rosopa, and Christopher C. Pagano

Human factors psychology, also known as engineering psychology or ergonomics, is an interdisciplinary feld that focuses on improving human interactions with their environments and the devices that they use. By understanding human capabilities and limitations, human factors researchers strive to improve human performance, efciency, and safety, and train individuals to acquire skills efectively. Expertise in human factors and ergonomics is essential in domains such as healthcare, industry, aviation, and transportation, where human-human and human-machine interactions are ubiquitous. Thus, unsurprisingly, the concept of afordances is ingrained in the feld of human factors (Dainof, 2008; Dainof & Mark, 1995). Throughout this chapter, we will use the terms “designers” and “design” to encompass human factors and ergonomics, as well as related felds including design, engineering, engineering design, product design, architecture, occupational safety, human-computer interaction, and others, whose goal is the creation of artifacts that are efective, safe, and easy to use by diverse populations of people and/or artifcial autonomous systems. As discussed by Norman (2015), these disciplines often use the concept of afordances diferently, with each often acting in an isolated manner within its own disciplinary “silo.” Although its roots go as far back as Gibson and Crooks (1938), James Gibson created the concept of afordances in the 1970s as an integral part of a larger theory of direct, non-mediated, non-representational perception, and motor control. This theory is referred to as the “Ecological Approach” (Gibson, 1979), and it can be contrasted with traditional approaches to perception that trace their roots at least as DOI: 10.4324/9781003396536-22

328

Balagopal Raveendranath et al.

far back as Descartes’ seventeenth-century division of the mind from the body (Blau & Wagman, 2022; Lombardo, 1987). Traditional approaches to psychology and motor control follow an epistemological dualism. The mind resides within the interior of the head and receives stimulation from the senses that is missing information required for accurate perception. Vision, for example, is via two-dimensional retinal images that do not faithfully represent the three-dimensional world. The mind must use prior knowledge and mental operations to create a copy of the world, and thus perception is of this internal mental representation, not of the world itself. The various theories and approaches referred to as “traditional” all difer in the details of what the prior knowledge, mental operations, and internal representations are like, but they all agree on the requirement of all or most of them in some form (Blau & Wagman, 2022; Chemero, 2009; Warren, 2022). Textbooks on design follow the traditional approach by referring to elaborate information processing models to explain behavior and to elucidate design solutions (McCormick & Sanders, 1982; Norman, 2013; Proctor & Van Zandt, 2018; Wickens et al., 2015). According to such models, humans follow a series of mental operations or information-processing steps that help us achieve our goals. These models follow the computer metaphor in compartmentalizing behavior into separate modules, starting with the senses processing input from the environment while holding it in short-term memory. This information is then combined with top-down information stored in long-term memory to form perception, which is seen as the act of using inference to impose meaning onto the stimulation received by the senses. What Do We Understand by the Term Afordance?

The Ecological Approach proposes that the senses do in fact provide veridical information about the world and that the senses evolved to put the animal in contact with the world in such a way that perception is of the world itself, rather than a mental reconstruction of it (Gibson, 1966, 1979). Afordances exist separate from mental activity, and they structure perceptual information that puts users in contact with them (Turvey et al., 1981). Much of the empirical work performed within the Ecological approach involves analytically uncovering the presence of such information in ambient energy distributions (light, sound, etc.) and empirically confrming whether experience allows users to come to employ such information. Ecological Psychology proposes a similar strategy for design, with the additional steps of frst uncovering the task-relevant elements of the system or work domain—including its afordances, determining what

“I Got All I Need to Know About Afordances From Norman”

329

information about them should be presented to the user, and designing ways to ensure that such information is presented in a usable form (Bennett & Flach, 2019; Pagano et al., 2021; Vicente & Rasmussen, 1990). The Ecological Approach provides great promise for the design felds because, if direct perception is possible, and if it can be implemented with artifacts that designers are interested in creating, then the use of such artifacts can be made safe, efective, and inclusive while minimizing cognitive burdens on the user during both training and use. Ample research has demonstrated that this sort of design is possible. According to Gibson, The afordances of the environment are what it ofers the animal, what it provides or furnishes, either for good or ill. The verb to aford is found in the dictionary, but the noun afordance is not. I have made it up. I mean by it something that refers to both the environment and the animal in a way that no existing term does. It implies the complementarity of the animal and the environment. —Gibson (1979, p. 119; emphases in the original) A designer recently ofered this very useful defnition, An afordance is a relationship between two (or more) interacting systems that describes a potential behavior that neither system can exhibit alone. —Maier (2022, p. 206) While these defnitions seem similar, this similarity has resulted in much confusion because, as described in the next section, the defnitions have been used within very diferent theoretical approaches, ecological and traditional, that have resulted in very diferent design methodologies. Historical Background to the Use of Afordances in Design

The most prominent modern instantiation of the traditional approaches is cognitive psychology. We will describe it and its history relative to the concept of afordances in a necessarily simplifed and general manner. During the middle of the twentieth century, the feld of psychology by and large underwent a “cognitive revolution” that corresponded with the development of modern digital computers and the emergence of computer science (Gardner, 1987). The relationship between psychology and computer science was symbiotic; the computer provided psychology with a real-world metaphor and model of what the mind might be like (Kelty-Stephen et al., 2022) and psychology provided computer science with an understanding

330

Balagopal Raveendranath et al.

of how the human mind works that could be used to inform the development of computing systems (e.g., Newell et al., 1958). In practice, however, computer science would not follow biological models but would instead fnd engineering solutions to making artifcial systems capable of detecting, manipulating, utilizing, and storing information in ways that were most useful within specifc applications. Psychology, however, continues to predominantly follow the computer metaphor. Like computer science, and unlike the Ecological Approach, cognitive science tends to follow engineering-like theories of the mind instead of biological theories (Blau & Wagman, 2022; Chemero, 2009). A prominent player in the cognitive revolution of the mid-twentieth century was Donald Norman (Gardner, 1987). He earned his Ph.D. in psychology after earning B.S. and M.S. degrees in electrical engineering. He authored infuential textbooks that helped defne the emergent feld of cognitive psychology (Lindsay & Norman, 1972; Norman, 1976). Later, Norman shifted his career from psychology to product design (Norman, 2013). This shift is evident in both the publication of his book entitled The Psychology of Everyday Things and its reissue under the new title The Design of Everyday Things (Norman, 1990, 1988, emphasis added). In that book, he introduced his own version of afordances. Following Gibson, Norman saw afordances as the actions made possible to users by the surfaces and objects of their environment (e.g., a chair can be sat on, a curb can be tripped over), but according to Norman, “ afordances result from the mental interpretations of things, based on our past knowledge and experience applied to our perception of the things about us” (Norman, 1988, p. 219). The concept that Norman labeled as “afordances” was very diferent than that used by Ecological Psychology. He used the term to indicate classes of internal mental models built by users within their minds via indirect perception—the very antithesis of what afordances are. “I was really talking about perceived afordances, which are not at all the same thing as real ones” (Norman, 1999, p. 38). Thus, Norman altered the concept of afordances to ft within traditional approaches to psychology, and it was largely this version that was adopted by designers (Chong & Proctor, 2020). The concept of afordances, as described by Norman, has caused much confusion within the design community (Burlamaqui & Dong, 2015). Later, Norman introduced a new term, “signifer,” that he hoped would clear this confusion (Norman, 2013). A signifer is a cue that highlights an afordance, such as a curb painted yellow or a button to be clicked on in a computer interface. A signifer alerts the user to the presence of an afordance, but it requires cognitive deliberations or cultural knowledge to be understood, and it is diferent from the afordance itself. Although afordances can exist without a signifer, the role of a signifer is to alert the user

“I Got All I Need to Know About Afordances From Norman”

331

to the presence of afordances. By introducing signifers and afordances as two distinct components of design, Norman expects that “[s]ignifers are of far more importance to designers than are afordances” (Norman, 2013, p. xv). Another related term that Norman discusses in the context of design is metaphors. A metaphor refers to something that is familiar to the user and that can help them understand a novel design. An example is the “save” icon seen in computer applications. It is a stylized image of a foppy disk, which was a popular device in the late twentieth century for storing digital information. A skull and crossbones symbol to indicate danger is another example of a metaphor. Norman’s Legacy

While there are a few notable exceptions, the engineering and design communities have adopted Norman’s cognitive approach to afordances. They typically cite Gibson, as did Norman, but they ignore or seem unaware of either Gibson’s larger theory or of the work that has been done on afordances within psychology and the human movement sciences during the 40-plus years since the publication of Gibson’s (1979) book. When attempting to describe Ecological theory to an engineer or designer, one is often met with a quote like the one which inspired the title of this chapter (which is an actual quote from a designer who has published on afordances), or with a similar comment that theory in general is “academic.” In Norman’s words, “this internal debate within modern psychology is of little relevance here” (1988, p. 219). This dismissal has led to missed opportunities and blatant theoretical contradictions in the work done by designers. For example, Maier and Fadel (Maier & Fadel, 2009, 2003) created a systematic design method that provides an important extension of the afordance concept, and which has proved useful in many applications. However, they explicitly rejected Ecological theory and embraced Norman’s cognitive approach (Maier & Fadel, 2003; Masoudi et al., 2019). They believe that they extended the concept of afordances to be applicable not only to humans interacting with objects but also to inanimate objects interacting with each other, such as autonomous robots and gears meshing. Their term “artifact-artifact afordance” (AAA) is novel, and their insights provide many new and invaluable contributions. However, an integral part of the Ecological Approach has always been that human perception and action follow natural laws and that human perception and action are continuous with the behavior of both organisms that lack nervous systems and the interactions between complex inanimate systems (e.g., Carello et  al., 2012; Dixon et  al., 2016; Kelty-Stephen et  al., 2022; Shaw et  al., 1995; Turvey, 1990, 2019). The Ecological Approach provides a methodology

332

Balagopal Raveendranath et al.

for how AAAs can be realized to achieve complex self-organizing behaviors and how they come into being naturally with or without human design (see Blau & Wagman, 2022). This methodology is lacking in the work of Maier and Fadel, and the failure to employ the Ecological Approach to create a more complete understanding of the power of AAAs for design has been a missed opportunity. While the Ecological Approach achieves a law-based account of interactions between people and things, Norman, Maier, and others within the design communities follow traditional cognitive psychology in treating human behavior as similar to that of modern robots. With robots, the functions of Descartes’ ghost in the machine (the internal mind) are performed by a central processing unit and indirect perception is seen as a necessary result. This dualism is also the predominant theory of the human mind within cognitive psychology, and this shared stance follows naturally from cognitive psychology and computer science’s shared history and assumptions regarding indirect perception (by humans and machines). It is often difcult to inform engineers about the Ecological Approach because they already understand, and are more comfortable with, its opposite—cognitive psychology as an equivalent to computer engineering. Norman (1999; 2013), for example, writes that he has “never understood” the Ecological approach. However, if Ecological theory is not the correct theory to apply to AAAs, then when two gears mesh, which one has the mental representations necessary for Norman’s afordances to be realized? Clearly, this is not what Maier and Fadel intended, but such are the contradictions caused by ignoring theory. They attempt to resolve this contradiction by acknowledging that afordances exist even when (1) they are not perceived, and (2) when no mental representations are present (Maier & Fadel, 2009, 2003). They apply the frst to biological creatures (such as humans) and robots, and the second to non-robotic inanimate objects, though a method for distinguishing between these classes is unspecifed. Thus, for them afordances may be realized by meshing gears in a necessarily direct manner, but afordances may not be directly perceived or acted upon by animate organisms or robots. One can trip over an unseen step, but a human or robot must have an internal mental representation to interact with a step properly. The frst contradiction is replaced by a second where simple mechanical devices are seen as capable of directly interacting with their environments while highly complex dynamical systems such as humans cannot, despite the lengthy evolution of their species and/or the learning and expertise occurring within the individual.1 Robots are not necessarily entities that must perceive and act indirectly, designers have chosen to construct them that way. Similarly, humans do not necessarily perceive and act indirectly; designers have often chosen to construct environments and displays that force people to behave that way.

“I Got All I Need to Know About Afordances From Norman”

333

The Ecological Approach resolves the above contradictions and provides a methodology for designing for direct interaction between users and elements of their environments, as well as for AAAs. Thus, it provides a preferred theory for design. For example, consider a ball rolling on a fat horizontal surface that meets a “step” (a short vertical surface and then a second horizontal surface that is higher than the one that the ball is currently rolling on). The ball and the step can be considered an AAA. One outcome is that the rolling ball hits the step and bounces backward. Alternatively, if the radius of the ball is large relative to the height of the step, then the ball could roll up onto the step and continue to move in its present direction. Therefore, the AAA for the ball-stair system is constrained by factors like the size of the ball relative to the height of the step. If a very small step were to be made progressively taller, then a “critical point” would be reached where the afordance changes and the step is no longer “roll-up-onto-able.” The ball-stair AAA is similar to afordances for humans where a taller step is not step-onto-able (Warren, 1995; Figure 17.1). In addition, just as there is an optimal stair height for efcient human locomotion, if a ball’s bouncing up into the air is important then

FIGURE 17.1

If a change in elevation is short relative to the size of an “actor,” then when the actor encounters the “step” the actor will be able to go up onto the step and continue in their current direction of travel (left column). This is the case whether the actor is animate (top row) or inanimate (bottom row). If the change in elevation is tall relative to the size of an actor, then a diferent mode of behavior will result when it is encountered (right column).

334

Balagopal Raveendranath et al.

there is likely an “optimal” step height that results in the highest vertical bounce when the ball strikes the step. Just as afordances for humans are determined by both the person’s geometry, such as leg length, and their dynamical abilities, such as strength and fexibility (Cesari, 2005), the afordances for the ball are similarly determined by its size, elasticity, internal pressure, etc. Shaw et al. (1995) present a similar discussion of optimal and critical points associated with AAAs for fuids of diferent viscosities fowing through pipes of various diameters. In each of these cases—the ball, the fuid, and the human—the afordances are quantifed by dimensionless intrinsic measures (Shaw et al., 1995; Warren, 1995), and none of these cases require mental representations to explain changes in behavior. The human, the ball, and the fuid can all be considered “actors” that are acting upon their environments. Maier and Fadel (2009, 2003) discuss complex AAAs where the two systems are mutually acting upon each other. Ecological psychology provides the tools for understanding such “interpersonal” or “social” afordances and coordination in a non-representational manner for both animate and inanimate systems (Schmidt & Richardson, 2008; Turvey, 1990; see Blau & Wagman, 2022, for review). To summarize, it was predominantly Norman’s concept of afordances as requiring indirect perception that caught on within the design communities, and this created the frst big division, or bifurcation, of the concept (McGrenere & Ho, 2000). As described by Norman (2015), designers and psychologists have been acting in parallel by using two diferent concepts, each largely unaware of the other. Norman has repeatedly chastised designers for using afordances inappropriately (Norman, 1999; 2013), but he never describes the usage of the concept as employed by Ecological Psychology. We will briefy do so in the remainder of this chapter. However, the main purpose of this chapter is to inspire designers to go beyond the current usage of afordances in their domains and seek out a fuller understanding of the Ecological Approach. For more comprehensive descriptions of afordances and Ecological theory the reader is directed to Blau and Wagman (2022), Heft (2001), and to the other chapters in this volume. What Role Does Afordance Play in Perception? Is It the Entity Organisms Perceive or the Means Through Which Organisms Perceive? Are Afordances the Only Perceptual Dependent Variables? Designing for the Direct Perception of Afordances

The goal of design is to create products and environmental features that provide afordances that are safe and usable. Many designs, however,

“I Got All I Need to Know About Afordances From Norman”

335

contain hazards that coexist with desired functions, such as a stair that can be fallen from or a kitchen tool that can cut the user. Undesired afordances come into being when a design inadvertently creates them and/or fails to remove them (Brown & Blessing, 2005). For desired afordances to be usable and for unwanted afordances to be avoided, they must be perceptible. Desirable afordances that are not perceptible are not usable and undesirable afordances that are not perceived are hazardous. Thus, safety is determined by both the number of hazardous afordances present and how perceptible they are. Johnson (1998), for example, describes a “disappearing step,” where a bottom step in a fight of stairs appears to be part of the foor, causing people descending the stairs to fail to perceive its afordance for falling, resulting in injuries (see also Blau & Wagman, 2022). A stair that is inadvertently rendered “invisible” is hazardous because the afordance for falling exists even when it is not perceived. Finding solutions for invisible afordances can be complex. Logan (2012) describes a tragedy that occurred when wheat and barley grains treated with alkylmercury fungicide were imported to Iraq from North America so that farmers could grow their own crops. Since these treated grains were unsuitable for consumption, they were dyed red in color and a skull and crossbones image was applied to the packaging to indicate danger. However, to the Kurdish farmers this image was just artwork; they did not understand the skull and crossbones metaphor or the red signifer. To them the grains ofered at least two afordances: planting to grow crops but also eating after washing of the red dye and cooking. The exploitation of the second afordance resulted in tragedy. The use of signifers and metaphors did not render information regarding the negative afordance in usable form. In the context of food, the most relevant invariant information for humans tends to be through smell and taste, with vision only being useful in certain contexts (Garcia et  al., 2021; Rosenblum, 2011). Perhaps adding some pungent smell to the grains could have made the afordance of inedibility more directly perceivable. In another example, natural gas is a common fuel used in residential and commercial buildings. Leaks are hazardous, but the presence of leaking gas is imperceptible without artifcially added information. Any number of odors could be used to make gas perceptible, but in an example of good human factors the gas is typically given the smell of rotten eggs, naturally leading to avoidance and/or corrective action. Fuel hydrogen, which spontaneously catches fre when it leaks under high pressure, is typically imperceptible to vision or olfaction, even when burning. This led NASA designers to devise ways of artifcially adding information, and thus replacing their original “broom method,” whereby “workers would take a broom and walk around with the head stretched out in front of them. If the head began to burn, there

336

Balagopal Raveendranath et al.

was a leak” (Granath, 2015). Imagine how dangerous human-made environments would be if all fres were equally imperceptible, or how diferent the emergence of human civilization might have been without perceptible fre, or how our senses might have evolved diferently! Why Is Afordance Any Better Than Stimulus? What Does a Theory of Afordance Suggest That Stimuli Cannot; How Has It Moved the Needle Past Gibson 1960’s Recognition of How Little We Can Defne Stimuli? The Active Observer

The presence of task-relevant information does not guarantee the perception of an afordance. Gibson (1966, 1979) argues that perception is information-based and not sensation-based. As per sensation-based (i.e., traditional) theories, perception begins with a static stimulus activating receptor cells in a sense organ. There is rarely adequate information in a static stimulus, which requires additional mental interpretations to achieve meaningful perception. Focusing on the stimulus has resulted in the traditional approach’s strategy of studying perception as a passive phenomenon. In contrast, the Ecological theory postulates that perception begins even before sensations are aroused by stimuli. The agent actively explores the available invariant information over time and even creates such information via ongoing activity and purposive explorations, such as with manual explorations for haptic information and movements that create visual optic fow (Hartman et al., 2016; Mantel et al., 2015). The organs of perception and action orient toward such information, rather than conveying information to the brain passively (Gibson, 1966). One could study perception by focusing on a stationary observer judging distances, size, weight, etc. However, the role played by the information available in the environment, and the context is ignored in such studies (Philbeck & Loomis, 1997). An alternative is to focus on the relationship between an observer and their environment by using action-based response measures (Napieralski et  al., 2011; Pagano & Isenhower, 2008). This is exactly what afordances ofer. Afordances are the primary dependent variables for perception. For example, let us consider the afordance of passability through a gap. A person should judge a gap as passable only if the horizontal width of the gap is at least 1.3 times the body width of the person. If not, the person will have to rotate their shoulder to pass through (Corbetta & Snapp-Childs, 2009; Warren & Whang, 1987). As such, this afordance is perceived directly by the agent involved, by means of an invariant information over time, the ratio of the width of the gap to the shoulder width of

“I Got All I Need to Know About Afordances From Norman”

337

the agent as specifed by optical eye height (Warren, 1995). The agent does not have to mentally compute this ratio. Such perception becomes difcult when one is perceiving afordances for another agent, such as tele-operated robotic device (Jones et al., 2011; Mantel et al., 2012; Moore et al., 2009), or when afordances change due to changes in one’s own abilities. Proftt and Linkenauger (2013) extended the idea of the observer body to include not only morphology but also physiology, or physiological processes within the body such as energy expenditure and conservation. In other words, information within the environment is scaled by bioenergetic resources needed to perform a given action. As an example, a person who works in an industrial plant may perform a repetitive task that becomes more difcult as the energy used to complete the task exceeds the energy available to the person (Proctor & Van Zandt, 2018). By taking this into consideration in its understanding of the perception-action process, the ecological approach simplifes design and improves performance, as compared to the traditional approach. What Is the Connection Between Afordances, Behavioral Scale, and Intention? How Distant or Reluctant Can a PerceivingActing System Be Before Afordances Come Into Play? Do Afordances Exert Their Role Only When the System Is Close or Willing Enough? Synergies in Design

Oftentimes, objects ofer multiple afordances. For example, a claw hammer afords both driving nails with one side of its head and removing nails with its claw. Sometimes designers will obscure some information, thereby constraining behavior. In other words, an object or surface that afords multiple actions can favor just one action that the designer feels is most necessary for a given context. Another key factor in design is the expected observers’ intention. A carpenter who generally uses a claw hammer could prospectively perceive both the afordances of driving and removing nails. However, when the goal is to drive nails, the carpenter actualizes that afordance. Not only can information about an object or surface ofer multiple afordances to a person, but also multiple invariants could be available to perceive the same afordance. In a study conducted by Raveendranath et al. (2023), participants were faster and more accurate to perceive the axis of rotation of a rotating panel when multiple invariants specifying the pivot axis were available. The participants’ probability of picking up at least one of the many invariants available increases in this case. Having multiple information sources available to the user, both within and between modalities, is an efcient design solution for many applications.

338

Balagopal Raveendranath et al.

The fact that a given object or constellation of environmental features provides users with multiple afordances presents the designer with a problem like that faced by Nikolai Bernstein in trying to understand how the motor system copes with its many degrees of freedom. Bernstein’s degrees of freedom problem pertains to the multitude of units to be controlled by the central nervous system to perform even very simple motor tasks (Bernstein, 1967; Bernstein et al., 1996; Tuller et al., 1982). There are multiple limbs, joints, muscles, etc., to be controlled. Even within a single afordance, such as the sit-ability of an ergonomic chair, there can be so many individual components to be separately adjusted that the creation of an optimal overall chair confguration may be too complex for the typical user to accomplish (Dainof, 2008). Simply put, the degrees of freedom problem can render an ergonomic chair, or any other product, unusable. For biology, the solution to the degrees of freedom problem is synergies; yoking separate anatomical components into functional units that work together. Synergies are groups of muscle and joint linkages that work together to act in coordination, such as in the diferent gaits of a horse or the synchronization of two limbs during drumming (Bernstein, 1967; Bernstein et  al., 1996; Riley et  al., 2011; Trefner & Turvey, 1996; Turvey et  al., 2022). They combine separate anatomical units to produce complex motions with fewer degrees of freedom to be controlled. Thus, the brain does not have to individually control the many degrees of freedom of the motor control system (see Blau and Wagman, 2022). The OctArm project is a good example of a robotic design where the concept of synergies was utilized (Moore et al., 2006; Walker et al., 2005). The OctArm was a teleoperated robotic “continuum” arm with many degrees of freedom modeled after octopus limbs and elephant trunks (Figure 17.2). To control its movement, the operator must be continuously in touch with the current state of the robot’s many degrees of freedom (position, velocity, etc.), and their relation to the environment (distances, properties of materials handled by the robot arm, etc.). The operator faces the degrees of freedom problem, where they have an abundance of degrees of freedom to control. The Ecological solution is to introduce synergies into the interface that combine separate units into fewer degrees of freedom that allow the operator to intuitively control the system’s many degrees of freedom (Moore et al., 2006; Tuller et al., 1982; Walker et al., 2005). Synergies can also be employed for the design of input devices that increase usability for process control (Hajdukiewicz & Vicente, 2004). Synergies span the separate anatomical units of the motor system and organize them together to achieve goals. It is important to note that when we use an external tool, the synergy spans this tool along with the individual parts of the body (Mangalam et al., 2022). The tool becomes a part

“I Got All I Need to Know About Afordances From Norman”

FIGURE 17.2

339

The OctArm robotic limb attached to a tracked teleoperated robot.

of the body, and it expands the number of afordances available to the user (Gibson, 1979; Maravita & Iriki, 2004; Pagano & Day, 2020; Pagano & Turvey, 1998). Hence, while designing devices to augment human capabilities and training people to improve their skills in utilizing tools and other implements, it is important to consider how synergies are formed and how design can help the user take advantage of such synergies. Calibration is critical to this process (Day et al., 2017). How Do Systems Engage With Afordances as They Move Among Tasks and Intentions? Calibration

To improve user performance, a product can be redesigned, or the user can be retrained. To understand how users learn to use a product, we should focus on three aspects—attunement, calibration, and exploration (Wagman et al., 2001). When people use tools, including prosthetic limbs, they receive proprioceptive information about the afordances available. For example, when we heft and wield an object with our hands, the resistance of the object against rotation helps us perceive the properties of the object like its length, weight et cetera (Amazeen & Turvey, 1996; Pagano et al., 1993). Thus, when we learn to use a tool, we are learning to attune to the goal relevant invariant information available. After we get attuned and calibrated to the information, we perceive such tools to be a part of our own body (Day et  al., 2017; Mangalam et  al., 2022; Pagano & Day, 2020).

340

Balagopal Raveendranath et al.

With a new tool, new possibilities of action become available to the user. Perception of these afordances requires the user to scale the available information to their own action capabilities. When baseball batters warm up before games, they sometimes add weight to their bats. This greatly increases the inertia of the bat. To adjust to this alteration, the batter produces increased muscular forces. Following this routine, when they remove the weight and bat during the game, it helps them generate enough force to hit the ball out of the park for a home run. However, calibrating to the added weight afects the timing of their swing, since sometimes they end up swinging the bat faster than they are used to. Scott and Gray (2010) showed that it takes batters several attempts to become calibrated to the change in bat weight. When the batters were allowed to wield and dynamically explore the bat in their hands (without swinging it) before hitting, the number of attempts required to calibrate was reduced signifcantly. For any organism, exploratory behavior is essential to attune to taskrelevant information and calibrate to one’s action capabilities. Recent studies uncovered how the intention to perceive diferent properties of an object afects how people wield the object to seek information (Arzamarski et al., 2010; Mangalam et al., 2019; Riley et al., 2002). In other words, a person who intends to perceive length exhibits a diferent exploratory behavior compared to when they intend to perceive weight. Given our ability to explore, even if our action capabilities change, or if the perceptual information changes, we can recalibrate easily, thereby improving performance. Therefore, product designers should encourage exploratory behavior in users, allowing them suffcient time to calibrate to the available perceptual information. For example, opportunities to calibrate allow users of immersive virtual reality to adjust afordance perceptions when their end efectors are represented by changing avatars (Day et al., 2019; Venkatakrishnan et al., 2023). Perception within immersive virtual reality tends to be distorted, with visual depths being compressed (Napieralski et al., 2011). Designers address this problem by altering the nature of virtual reality systems, making them higher fdelity, more complex, more expensive, etc. Work in our laboratory has shown that a more efcient strategy is to present the user with an opportunity to calibrate to the virtual environment (Ebrahimi et al., 2016; Kohm et al., 2022). Exploration, attunement, and calibration are necessary for a user to diferentiate possible afordances via feedback and practice. The degree to which someone is given the opportunity to explore the action space, along with the amount of feedback received, determines the time required to calibrate (van Andel et al., 2017). In a simulation of a laparoscopic surgery task, distance-to-break (DTB) has been identifed as the task-relevant information to which participants can attune to for judging whether they would break a tissue while manipulating it (Altenhof et al.,

“I Got All I Need to Know About Afordances From Norman”

341

2017; Hartman et al., 2016; Long et al., 2014). Like time-to-contact (an optical invariant that specifes when we might collide with an approaching surface), DTB is a haptic invariant that specifes when a tissue will break if the laparoscopic tool is pushed further. It took only about 15 minutes to train participants who had no experience with laparoscopic surgery to stop within a few millimeters of a target breakage point. Even expert surgeons showed signifcant improvement after brief training. Thus, we suggest that Ecological theory provides a radical new approach to training for expertise. Rather than observing the performance of experts and then training novices to emulate them, begin by determining analytically what information should be detected by the user to perform the required task, design the display such that the afordance is perceptible, and train both experts and novices to attune and calibrate to this information. Note that the traditional approach does not improve the experts, while this approach does. With the traditional approach, the experts may be utilizing fawed displays, and recording how they do so and then teaching that to novices only perpetuates inferior methods. The traditional approach focuses on training for the addition of knowledge and mental operations in the head (e.g., cue integration, mental models, situation awareness), while the Ecological approach puts the user in better contact with the domain (Efken et  al., 1997). Users were trained to use DTB, and thus perform a simulated surgical task successfully, without ever being explicitly taught what DTB is. Concluding Remarks

The concept of afordances has been widely used by designers, but often in a way that conficts with Gibson’s theory of direct perception. This has led to two opposing views of afordances. A strong theoretical foundation can help in making sense of data and in developing efcient design solutions. The utilization of conficting theories (or no theory!) results in diferent end designs. We urge designers to go beyond the conceptualization of afordances that has been ofered by Norman and others and apply Ecological theory. The employment of Ecological theory within a number of applications has already resulted in designs that are more usable, safer, and that require less time and mental efort for both their training and their use. Note 1. Plants, single-celled organisms, and other lower organisms necessarily have direct perception, because they lack the neural structures required to form mental representations or to utilize cues (Carello et al., 2012; Turvey, 2019). If perception in higher organisms is necessarily indirect, as contended by traditional theories, then at what point during evolution was direct perception lost? What

342

Balagopal Raveendranath et al.

caused it to be lost? Why would more complex neural structures, including more sophisticated perception and action systems, evolve in such a way as to put organisms in worse contact with their environments? The evolution of more sophisticated perception-action systems should be expected to put organisms in better contact with the world. Perhaps direct perception and action were never lost, and traditional theories are fawed. To the extent that mental representations exist, it is possible that they can be functionally transparent in the same sense that artifcial displays can be made to be functionally transparent (see Pagano & Day, 2020; Vicente & Rasmussen, 1990, for discussions of functional transparency). Thus, the existence of representations might not automatically entail indirect perception. Just as light, retinas, and artifcial displays can each be a medium for direct perception, perhaps mental representations can be a medium for direct perception as well.

Reference List Altenhof, B. M., Pagano, C. C., Kil, I., & Burg, T. C. (2017). Learning to perceive haptic distance-to-break in the presence of friction. Journal of Experimental Psychology: Human Perception and Performance, 43(2), 231–244. https://doi. org/10.1037/xhp0000298 Amazeen, E. L., & Turvey, M. T. (1996). Weight perception and the haptic size— Weight illusion are functions of the inertia tensor. Journal of Experimental Psychology: Human Perception and Performance, 22(1), 213–232. https://doi. org/10.1037/0096-1523.22.1.213 Arzamarski, R., Isenhower, R. W., Kay, B. A., Turvey, M. T., & Michaels, C. F. (2010). Efects of intention and learning on attention to information in dynamic touch. Attention, Perception, & Psychophysics, 72(3), 721–735. https://doi. org/10.3758/APP.72.3.721 Bennett, K. B., & Flach, J. (2019). Ecological interface design: Thirty-plus years of refnement, progress, and potential. Human Factors, 61(4), 513–525. https:// doi.org/10.1177/0018720819835990 Bernstein, N. A. (1967). The Co-ordination and Regulation of Movements. Pergamon Press. Bernstein, N. A., Latash, M. L., & Turvey, M. T. (1996). Dexterity and Its Development. Lawrence Erlbaum. Blau, J. J. C., & Wagman, J. B. (2022). Introduction to Ecological Psychology: A Lawful Approach to Perceiving, Acting, and Cognizing. Routledge. Brown, D. C., & Blessing, L. (2005). The relationship between function and afordance. 17th International Conference on Design Theory and Methodology, 155–160. https://doi.org/10.1115/DETC2005-85017 Burlamaqui, L., & Dong, A. (2015). The use and misuse of the concept of afordance. In J. S. Gero & S. Hanna (Eds.), Design Computing and Cognition (pp. 295–311). Springer. https://doi.org/10.1007/978-3-319-14956-1_17 Carello, C., Vaz, D., Blau, J. J. C., & Petrusz, S. (2012). Unnerving intelligence. Ecological Psychology, 24(3), 241–264. https://doi.org/10.1080/10407413.2012.702628 Cesari, P. (2005). An invariant guiding stair descent by young and old adults. Experimental Aging Research, 31(4), 441–455. https://doi.org/10.1080/03610 730500206840

“I Got All I Need to Know About Afordances From Norman”

343

Chemero, A. (2009). Radical Embodied Cognitive Science. MIT Press. Chong, I., & Proctor, R. W. (2020). On the evolution of a radical concept: Afordances according to Gibson and their subsequent use and development. Perspectives on Psychological Science, 15(1), 117–132. https://doi. org/10.1177/1745691619868207 Corbetta, D., & Snapp-Childs, W. (2009). Seeing and touching: The role of sensory-motor experience on the development of infant reaching. Infant Behavior and Development, 32(1), 44–58. https://doi.org/10.1016/j.infbeh.2008.10.004 Dainof, M. J. (2008). Ecological ergonomics. In O. Y. Chebykin, G. Bedny, & W. Karwowski (Eds.), Ergonomics and Psychology (pp. 21–46). CRC Press. Dainof, M. J., & Mark, L. S. (1995). Use of a means-end abstraction hierarchy to conceptualize the ergonomic design of workplaces. In M. J. Dainof & L. S. Mark (Eds.), Global Perspectives on the Ecology of Human-Machine Systems (pp. 273–292). CRC Press. Day, B., Ebrahimi, E., Hartman, L. S., Pagano, C. C., & Babu, S. V. (2017). Calibration to tool use during visually-guided reaching. Acta Psychologica, 181, 27–39. https://doi.org/10.1016/j.actpsy.2017.09.014 Day, B., Ebrahimi, E., Hartman, L. S., Pagano, C. C., Robb, A. C., & Babu, S. V. (2019). Examining the efects of altered avatars on perception-action in virtual reality. Journal of Experimental Psychology: Applied, 25(1), 1–24. https://doi. org/10.1037/xap0000192 Dixon, J. A., Kay, B. A., Davis, T. J., & Kondepudi, D. (2016). End-directedness and context in nonliving dissipative systems. In E. Dzhafarov, S. Jordan, R. Zhang, & V. Cervantes (Eds.), Contextuality from Quantum Physics to Psychology (pp. 185–208). World Scientifc. Ebrahimi, E., Babu, S. V, Pagano, C. C., & Jörg, S. (2016). An empirical evaluation of visuo-haptic feedback on physical reaching behaviors during 3D interaction in real and immersive virtual environments. ACM Transactions on Applied Perception, 13(4), 1–21. https://doi.org/10.1145/2947617 Efken, J. A., Kim, N.-G., & Shaw, R. E. (1997). Making the constraints visible: Testing the ecological approach to interface design. Ergonomics, 40(1), 1–27. https://doi.org/10.1080/001401397188341 Garcia, J., Rusiniak, K. W., & Brett, L. P. (2021). Conditioning food—Illness aversions in wild animals: Caveant canonici. In H. Davis & H. M. B. Hurwitz (Eds.), Operant-Pavlovian Interactions (pp. 273–316). Routledge. Gardner, H. (1987). The Mind’s New Science: A History of the Cognitive Revolution. Basic Books. Gibson, J. J. (1966). The Senses Considered as Perceptual Systems. Houghton Mifin. Gibson, J. J. (1979). The Ecological Approach to Visual Perception. Houghton Mifin. Gibson, J. J., & Crooks, L. E. (1938). A theoretical feld-analysis of automobile-driving. American Journal of Psychology, 51(3), 453–471. https://doi. org/10.2307/1416145 Granath, B. (2015). Hydrogen Detection Tape Saves Time, Lives. www.nasa.gov/ feature/hydrogen-detection-tape-saves-time-lives Hajdukiewicz, J. R., & Vicente, K. J. (2004). What does computer-mediated control of a thermal-hydraulic system have to do with moving your jaw to speak?

344

Balagopal Raveendranath et al.

Evidence for synergies in process control. Ecological Psychology, 16(4), 255– 285. https://doi.org/10.1207/s15326969eco1604_1 Hartman, L. S., Kil, I., Pagano, C. C., & Burg, T. (2016). Investigating haptic distance-to-break using linear and nonlinear materials in a simulated minimally invasive surgery task. Ergonomics, 59(9), 1171–1181. https://doi.org/10.1080/ 00140139.2015.1127429 Heft, H. (2001). Ecological Psychology in Context: James Gibson, Roger Barker, and the Legacy of William James’s Radical Empiricism. Psychology Press. Johnson, D. A. (1998). New stairway—Old problems. Ergonomics in Design, 6(4), 7–10. https://doi.org/10.1177/106480469800600405 Jones, K. S., Johnson, B. R., & Schmidlin, E. A. (2011). Teleoperation through apertures: Passability versus driveability. Journal of Cognitive Engineering and Decision Making, 5(1), 10–28. https://doi.org/10.1177/1555343411399074 Kelty-Stephen, D. G., Cisek, P. E., De Bari, B., Dixon, J., Favela, L. H., Hasselman, F., Keijzer, F., Raja, V., Wagman, J. B., Thomas, B. J., & Mangalam, M. (2022). In search for an alternative to the computer metaphor of the mind and brain. ArXiv Preprint ArXiv: 2206.04603. Kohm, K., Babu, S. V, Pagano, C., & Robb, A. (2022). Objects may be farther than they appear: Depth compression diminishes over time with repeated calibration in virtual reality. IEEE Transactions on Visualization and Computer Graphics, 28(11), 3907–3916. https://doi.org/10.1109/TVCG.2022.3203112 Lindsay, P. H., & Norman, D. A. (1972). Human Information Processing: An Introduction to Psychology. Academic Press. Logan, M. K. (2012). Set phasers on stun and other true tales of design, technology, and human error. Biomedical Instrumentation & Technology, 46(6), 466. https://doi.org/10.2345/0899-8205-46.6.466 Lombardo, T. J. (1987). The Reciprocity of Perceiver and Environment: The Evolution of James J. Gibson’s Ecological Psychology (Vol. 18). Routledge. Long, L. O., Hartman, L. S., Pagano, C. C., Kil, I., Singapogu, J., & Burg, T. C. (2014). Haptic perception of distance-to-break for compliant tissues in a surgical simulator. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 58(1), 1984–1988. https://doi.org/10.1177/154193121458141 Maier, J. R. A. (2022). Refections on 20 years of afordance-based design. In Z. Djebbara (Ed.), Afordances in Everyday Life: A Multidisciplinary Collection of Essays (pp. 203–209). Springer. https://doi.org/10.1007/978-3-031-08629-8_19 Maier, J. R. A., & Fadel, G. M. (2003). Afordance-based methods for design. 15th International Conference on Design Theory and Methodology, 785–794. https://doi.org/10.1115/DETC2003/DTM-48673 Maier, J. R. A., & Fadel, G. M. (2009). Afordance based design: A relational theory for design. Research in Engineering Design, 20(1), 13–27. https://doi. org/10.1007/s00163-008-0060-3 Mangalam, M., Conners, J. D., & Singh, T. (2019). Muscular efort diferentially mediates perception of heaviness and length via dynamic touch. Experimental Brain Research, 237(1), 237–246. https://doi.org/10.1007/s00221-018-5421-1 Mangalam, M., Fragaszy, D. M., Wagman, J. B., Day, B. M., Kelty-Stephen, D. G., Bongers, R. M., Stout, D. W., & Osiurak, F. (2022). On the psychological origins of tool use. Neuroscience & Biobehavioral Reviews, 104521. https://doi. org/10.1016/j.neubiorev.2022.104521

“I Got All I Need to Know About Afordances From Norman”

345

Mantel, B, Hoppenot, P., & Colle, E. (2012). Perceiving for acting with teleoperated robots: Ecological principles to human—Robot interaction design. IEEE Transactions on Systems, Man, and Cybernetics—Part A: Systems and Humans, 42(6), 1460–1475. https://doi.org/10.1109/TSMCA.2012.2190400 Mantel, B., Stofregen, T. A., Campbell, A., & Bardy, B. G. (2015). Exploratory movement generates higher-order information that is sufcient for accurate perception of scaled egocentric distance. PLoS One, 10(4), e0120025. https://doi. org/10.1371/journal.pone.0120025 Maravita, A., & Iriki, A. (2004). Tools for the body (schema). Trends in Cognitive Sciences, 8(2), 79–86. https://doi.org/10.1016/j.tics.2003.12.008 Masoudi, N., Fadel, G. M., Pagano, C. C., & Elena, M. V. (2019). A review of afordances and afordance-based design to address usability. Proceedings of the Design Society: International Conference on Engineering Design, 1(1), 1353– 1362. https://doi.org/10.1017/dsi.2019.141 McCormick, E. J., & Sanders, M. S. (1982). Human Factors in Engineering and Design. McGraw-Hill. McGrenere, J., & Ho, W. (2000). Afordances: Clarifying and evolving a concept. Graphics Interface, 2000, 179–186. Moore, K. S., Gomer, J. A., Pagano, C. C., & Moore, D. D. (2009). Perception of robot passability with direct line of sight and teleoperation. Human Factors, 51(4), 557–570. https://doi.org/10.1177/0018720809341959 Moore, K. S., Rodes, W. M., Csencsits, M. A., Kwoka, M. J., Gomer, J. A., & Pagano, C. C. (2006). Interface evaluation for soft robotic manipulators. Unmanned Systems Technology VIII, 6230, 460–470. https://doi.org/10.1117/12.666040 Napieralski, P. E., Altenhof, B. M., Bertrand, J. W., Long, L. O., Babu, S. V, Pagano, C. C., Kern, J., & Davis, T. A. (2011). Near-feld distance perception in real and virtual environments using both verbal and action responses. ACM Transactions on Applied Perception, 8(3), 1–19. https://doi.org/10.1145/2010325.2010328 Newell, A., Shaw, J. C., & Simon, H. A. (1958). Chess-playing programs and the problem of complexity. IBM Journal of Research and Development, 2(4), 320– 335. https://doi.org/10.1147/rd.24.0320 Norman, D. A. (1976). Memory and Attention: An Introduction to Human Information Processing. John Wiley & Sons. Norman, D. A. (1988). The Psychology of Everyday Things. Basic Books. Norman, D. A. (1990). The Design of Everyday Things. Basic Books. Norman, D. A. (1999). Afordance, conventions, and design. Interactions, 6(3), 38–43. https://doi.org/10.1145/301153.301168 Norman, D. A. (2013). The Design of Everyday Things: Revised and Expanded Edition. Basic Books. Norman, D. A. (2015). Afordances: Commentary on the special issue of AI EDAM. AI EDAM, 29(3), 235–238. https://doi.org/10.1017/S0890060415000232 Pagano, C. C., & Day, B. (2020). Ecological interface design inspired by “the meaningful environment.” In J. B. Wagman & J. J. C. Blau (Eds.), Perception as Information Detection: Refections on Gibson’s Ecological Approach to Visual Perception (pp. 37–50). Routledge. Pagano, C. C., Day, B., & Hartman, L. S. (2021). An argument framework for ecological psychology and architecture design. Technology|Architecture + Design, 5(1), 31–36. https://doi.org/10.1080/24751448.2021.1863665

346

Balagopal Raveendranath et al.

Pagano, C. C., Fitzpatrick, P., & Turvey, M. T. (1993). Tensorial basis to the constancy of perceived object extent over variations of dynamic touch. Perception & Psychophysics, 54(1), 43–54. https://doi.org/10.3758/BF03206936 Pagano, C. C., & Isenhower, R. W. (2008). Expectation afects verbal judgments but not reaches to visually perceived egocentric distances. Psychonomic Bulletin & Review, 15(2), 437–442. https://doi.org/10.3758/PBR.15.2.437 Pagano, C. C., & Turvey, M. T. (1998). Eigenvectors of the inertia tensor and perceiving the orientations of limbs and objects. Journal of Applied Biomechanics, 14(4), 331–359. https://doi.org/10.1123/jab.14.4.331 Philbeck, J. W., & Loomis, J. M. (1997). Comparison of two indicators of perceived egocentric distance under full-cue and reduced-cue conditions. Journal of Experimental Psychology: Human Perception and Performance, 23, 72–85. https://doi.org/10.1037/0096-1523.23.1.72 Proctor, R. W., & Van Zandt, T. (2018). Human Factors in Simple and Complex Systems. CRC press. Proftt, D. R., & Linkenauger, S. A. (2013). Perception viewed as a phenotypic expression. In W. Prinz, M. Beisert, & A. Herwig (Eds.), Action Science: Foundations of an Emerging Discipline (Vol. 171). MIT Press. Raveendranath, B., Pagano, C. C., Nasiri, M., Robb, A. C., & Babu, S. V. (2023). Efect of texture on the perception of axis of rotation of rotating panels. Ecological Psychology, 35(1–2), 1–30. https://doi.org/10.1080/10407413.2022.21 47838 Riley, M. A., Richardson, M., Shockley, K., & Ramenzoni, V. (2011). Interpersonal synergies. Frontiers in Psychology, 2, 38. https://doi.org/10.3389/ fpsyg.2011.00038 Riley, M. A., Wagman, J. B., Santana, M.-V., Carello, C., & Turvey, M. T. (2002). Perceptual behavior: Recurrence analysis of a haptic exploratory procedure. Perception, 31(4), 481–510. https://doi.org/10.1068/p3176 Rosenblum, L. D. (2011). See What I’m Saying: The Extraordinary Powers of Our Five Senses. WW Norton & Company. Schmidt, R. C., & Richardson, M. J. (2008). Dynamics of interpersonal coordination. In A. Fuchs & V. K. Jirsa (Eds.), Coordination: Neural, Behavioral and Social Dynamics (pp.  281–308). Springer Berlin Heidelberg. https://doi. org/10.1007/978-3-540-74479-5_14 Scott, S., & Gray, R. (2010). Switching tools: Perceptual-motor recalibration to weight changes. Experimental Brain Research, 201(2), 177–189. https://doi. org/10.1007/s00221-009-2022-z Shaw, R. E., Flascher, O. M., & Kadar, E. E. (1995). Dimensionless invariants for intentional systems: Measuring the ft of vehicular activities to environmental layout. In J. M. Flach, P. A. Hancock, J. Caird, & K. J. Vicente (Eds.), Global Perspectives on the Ecology of Human-Machine Systems (pp. 293–357). Lawrence Erlbaum. Trefner, P. J., & Turvey, M. T. (1996). Symmetry, broken symmetry, and handedness in bimanual coordination dynamics. Experimental Brain Research, 107(3), 463–478. https://doi.org/10.1007/BF00230426 Tuller, B., Turvey, M. T., & Fitch, H. L. (1982). The Bernstein perspective: II: The concept of muscle linkage or coordinative structure. In J. A. S. Kelso (Ed.), Human Motor Behavior: An Introduction (pp. 253–270). Lawrence Erlbaum.

“I Got All I Need to Know About Afordances From Norman”

347

Turvey, M. T. (1990). Coordination. American Psychologist, 45(8), 938–953. https://doi.org/10.1037/0003-066X.45.8.938 Turvey, M. T. (2019). Lectures on Perception: An Ecological Perspective. Routledge. Turvey, M. T., Shaw, R. E., & Mace, W. (2022). Issues in the theory of action: Degrees of freedom, coordinative structures and coalitions. In J. Requin (Ed.), Attention and Performance VII (pp. 557–595). Routledge. Turvey, M. T., Shaw, R. E., Reed, E. S., & Mace, W. M. (1981). Ecological laws of perceiving and acting: In reply to Fodor and Pylyshyn (1981). Cognition, 9(3), 237–304. van Andel, S., Cole, M. H., & Pepping, G.-J. (2017). A systematic review on perceptual-motor calibration to changes in action capabilities. Human Movement Science, 51, 59–71. https://doi.org/10.1016/j.humov.2016.11.004 Venkatakrishnan, R., Venkatakrishnan, R., Raveendranath, B., Pagano, C. C., Robb, A. C., Lin, W.-C., & Babu, S. V. (2023). How virtual hand representations afect the perceptions of dynamic afordances in virtual reality. IEEE Transactions on Visualization and Computer Graphics, 29(5), 2258–2268. https://doi.org/10.1109/TVCG.2023.3247041 Vicente, K. J., & Rasmussen, J. (1990). The ecology of human-machine systems II: Mediating “direct perception” in complex work domains. Ecological Psychology, 2(3), 207–249. https://doi.org/10.1207/s15326969eco0203_2 Wagman, J. B., Shockley, K., Riley, M. A., & Turvey, M. T. (2001). Attunement, calibration, and exploration in fast haptic perceptual learning. Journal of Motor Behavior, 33(4), 323–327. https://doi.org/10.1080/00222890109601917 Walker, I. D., Dawson, D. M., Flash, T., Grasso, F. W., Hanlon, R. T., Hochner, B., Kier, W. M., Pagano, C. C., Rahn, C. D., & Zhang, Q. M. (2005). Continuum robot arms inspired by cephalopods. Unmanned Ground Vehicle Technology VII, 5804, 303–314. Warren, W. H. (1995). Constructing an econiche. In J. M. Flach, P. A. Hancock, J. Caird, & K. J. Vicente (Eds.), Global Perspectives on the Ecology of HumanMachine Systems (pp. 210–237). CRC Press. Warren, W. H. (2022). Epilogue: The cartesian submariner learns to surf. In A. Szokolszky, C. Read, & P. Zsolt (Eds.), Intellectual Journeys in Ecological Psychology: Interviews and Refections from Pioneers in the Field (pp. 410–430). Routledge. Warren, W. H., & Whang, S. (1987). Visual guidance of walking through apertures: Body-scaled information for afordances. Journal of Experimental Psychology: Human Perception and Performance, 13(3), 371–383. https://doi. org/10.1037/0096-1523.13.3.371 Wickens, C. D., Hollands, J. G., Banbury, S., & Parasuraman, R. (2015). Engineering Psychology and Human Performance. Psychology Press.

18 PERCEPTION OF AFFORDANCES IN INTERACTION WITH AUTONOMOUS SYSTEMS Tri Nguyen, Corey Magaldino, Jayci Landfair, Matt Langley, and Polemnia G. Amazeen

The advancement and increasing prevalence of autonomous systems have not only resulted in renewed interest in research on our interactions with this new technology but also prompted a need to reexamine the role of afordance in facilitating meaningful behavior. Afordances have been examined in several technological contexts, among them language (Hodges & Fowler, 2010), robotics (Chemero & Turvey, 2007), virtual and augmented reality (Stefen et al., 2019), video games (Deterding et al., 2011), screen layout (Piolat et  al., 1997), and technologies in interface design (Gaver, 1991). In this chapter, we focus on developments of our understanding of afordances from the perspective of research on human-autonomy interactions and complex object manipulation. In particular, we will discuss several fndings from our recent study on human-autonomy teaming to highlight the applicability of afordances in realistic interaction with a modern autonomous system (Figure 18.1). In this study, participants were tasked with controlling a vehicle with ondemand automated driving assistance while avoiding obstacles and maintaining distance from a lead vehicle. We examined participants’ driving patterns and decisions to utilize automation in the context of changing task difculty within a trial and automation quality across trials. Dynamical systems analyses of the variability in driving patterns revealed that participants were attuned to the capabilities of the autonomous vehicle and made decisions to use or opt not to use this functionality in response to specifc environmental cues. In the current chapter, we discuss the role of afordance in the complex interaction between the driver, the autonomous vehicle, and the environment. DOI: 10.4324/9781003396536-23

Afordances in Interaction With Autonomous Systems

FIGURE 18.1

349

The driving simulator. (a) A participant sitting in the simulation testbed; (b) the simulated driving track; and (c) the simulation display. Modifed from Gremillion et al. (2016).

What Do We Understand by the Term Afordance?

Gibson (1979) defned afordances as opportunities for actions specifc to the perceiving agent’s ecological needs. A possibility for action, an afordance, is formed when there is a match between an agent’s physiology and capability, called efectivities, and the interactive properties within the environment (Chemero, 2003; Gibson, 1979; Turvey, 1992). Afordances reside not within agents nor as properties of the environment but in the dispositional relationship between the two (Stofregen, 2003; Stofregen et al., 2006). The properties that specify an afordance are inherently lawful and are invariantly structured in the ambient perceptual—most commonly, visual, haptic, and acoustic—arrays of the environment (Turvey et  al., 1981). Therefore, the roles of perception and action are inseparable. Jointly, they embody the attunement to and exploitation of this ecological information to fulfll an intended behavioral outcome (Hommel et al., 2001; O’Regan & Noë, 2001; Rizzolatti & Craighero, 2004; Warren, 1990). Afordances ft naturally within the framework of dynamical systems theory, which captures behaviors as quantifable interactions between an agent and its physical-social environment (Mechsner et al., 2001; Smith & Thelen, 1993; Tschacher & Dauwalder, 2003). An essential characteristic of coordination is the strength of coupling between its components (Amazeen et al., 1998; Fitzpatrick et al., 1994; Kelso, 1984, 1995; Mark, 1987; Wagman et al., 2018; Warren, 1984). Changes in coupling strength afect the observed behavior and its stability. One example of coupling strength is connectedness to our tools. Efective utilization of tools depends on the attunement and exploitation of the

350

Tri Nguyen et al.

tool’s purpose. Detection of that purpose is dependent on the physical and functional properties of the tool and is not necessarily tied strictly to the purpose for which it was designed. In most studies on tool use, simple tools such as a hand rake or a claw grabber are studied. The behavior of those tools, which efectively extend the reaching capabilities of the agent using the tool, is readily apparent (Cardinali et al., 2009; Maravita & Iriki, 2004). However, even with simple tools, attunement and exploitation are not always immediate. For example, Bril et al. (2010) examined the kinematics pattern between novice and expert stone knappers. While both groups could adapt their behaviors in response to diferent task demands, only experts showed invariant striking force regardless of the hammer’s weight, indicating an attunement to the relationship between tool and environment. Studies in dynamical systems theory extended the concept of afordance by showing that invariant structure in variability also plays a crucial role in predicting behavioral outcomes. Here, the invariants we refer to are patterns of fuctuations in behavioral signals that repeat over time and frequencies. Such self-similar fuctuations come from nested interacting processes (such as the respiratory cycle or the day/night cycle) extending across scales (Amazeen, 2018; Van Orden et al., 2003; Wagenmakers et al., 2004). As a product of the coordination between processes within and without the agent, invariants in variability necessarily encompass the agent-environment coupling. For example, Stephen et  al. (2009) asked participants to solve gear system problems. Whereas most participants initially solve these problems by manually tracing the turn direction of each gear until they reach the fnal gear, a new strategy is eventually discovered in which turn direction can be determined by grouping all odd or gears together. Structured variability in participants’ gaze, as quantifed by entropy, predicted the moment the new strategy was discovered and new behavior emerged. Here, we can see that invariants also accompanied the attunement and exploitation of invariant information in the agent-environment system in the variability of behaviors. As technology progresses, tools include artifcial intelligence (AI), autonomous vehicles, smart devices, synthetic teammates, etc., all exhibiting varying degrees of autonomy and complexity (Graziano et al., 2005). With modern technology, complex tools such as autonomous vehicles pose a challenge to afordances because their invariant behavior may be opaque and unpredictable. For instance, most accidents involving autonomous vehicles were from other vehicles crashing into the autonomous vehicle from the rear (NHTSA report, June 2022). This suggested that autonomous vehicles were behaving in an unexpected manner by normal drivers.

Afordances in Interaction With Autonomous Systems

351

Research in adversarial AI also repeatedly demonstrated that the features used by AI to detect objects, signs, or faces are entirely diferent from those used by human drivers (Evtimov et al., 2017; Sharif et al., 2016). As such, it is much harder to ascertain the invariant behaviors of autonomous systems. Even so, recent fndings demonstrated that the same principle of attunement and exploitation of invariant information could be applied to interactions with these novel systems. In our recent research on driving behavior with on-demand autonomous assistance, we examined the variability structure of participants’ driving patterns in the context of varying degrees of automation quality and environmental challenges (Nguyen et al., 2022). We found that participants engaged in drastically diferent driving patterns when driving through a simple straight path versus when they had to negotiate a tight S-curve. These patterns of driving behaviors corresponded to fundamental shifts in the invariant structures of variability in the driving data. Further dynamical systems analysis looking at the relationship between fuctuations in driving data and environmental cues such as road curvature revealed a tight coupling that was also infuenced by the quality of the autonomous assistive system. Participants were attuned to the reliable (invariant) performance of high-quality automation and exploited that during interactions with the environment. Recent eforts in understanding human interaction with complex objects also highlighted the importance of invariant information in successfully controlling such unpredictable systems. One such system is the fuid-ina-cup system, which can potentially exhibit chaotic behavior due to the unpredictable nature of force interaction with the fuid. The complexity of such systems allows for multiple successful control strategies. For instance, people can oscillate the cup in sync (in-phase) or counter (anti-phase) to the movement of the fuid to establish stable control of the system. Researchers modeling interactions with this system found that people gravitate toward more predictable control solutions rather than energy-efcient solutions (Bazzi & Sternad, 2020; Maurice et al., 2018). Once again, these fndings suggested that the natural strategy in interaction with complex objects is to attune to invariant behavior patterns, then exploit that information for predictable control. In conclusion, complex objects and autonomous systems may require more time and efort to attune to. However, the information detected follows lawful properties captured by dynamical systems analysis and is otherwise consistent with the characteristics of afordances as formulated initially by Gibson.

352

Tri Nguyen et al.

What Role Does Afordance Play in Perception? Is It the Entity Organisms Perceive or the Means Through Which Organisms Perceive? Are Afordances the Only Perceptual Dependent Variables?

The classic view of afordances, and one that we support, is that they are perceived or detected directly without the need for inference. Even the coupling strength in complex, unpredictable, and opaque systems is directly perceived in the context of action. Afordances dynamically change as the context for actions evolves. Environmental changes lead to changes in invariant information, and the agent’s active exploration of diferent solutions in diferent contexts also contributes to changes in afordances. One well-known example of direct perception is the rate of optical expansion tau. Lee (1976) demonstrated that the rate at which an image on the retina expands or contracts contains information about the time to collide with that object. It was ftting for the current chapter that the original work was done with respect to timing the braking of a car to avoid a collision. Think of the common scenario where we must maintain distance from a vehicle in front of us. If the leading vehicle suddenly decelerates, this information in the environment causes a change in the rate of expansion tau of that vehicle on our retina. This sudden expansion indicates imminent collision. The value of tau relates to the magnitude of impact. It is important to note that changes in tau are not solely dependent on the environment but also on our actions. If we suddenly accelerate, then we will see the exact change in the rate of expansion tau. Tau dynamically changes as a function of both changes in the environment and the agent’s behavior. In our work on human-autonomy teaming in driving, participants could accurately identify the quality of the autonomous driving system under a high cognitive workload and adjust their frequency and timing of usage accordingly. Participants were more likely to use automation when they detected that it was reliable and frequently adjusted usage of automation depending on whether they were driving a simple straight path or about to enter a difcult S-curve. The decision to engage in automation depends on evaluating the relationship between the autonomous system’s reliability, the driver’s propensity to trust in automation, and the current environmental constraints. Through dynamical analyses, we demonstrated that participants’ usage of automation corresponded to invariant structures in their driving behavior, which suggested that this complex relational term was managed using the same principle as traditional examples of afordances and direct perception. In the fuid-in-a-cup system, efective control requires participants to test diferent movement patterns and evaluate whether that pattern is stable. Furthermore, participants must maintain or reproduce that pattern even

Afordances in Interaction With Autonomous Systems

353

after fnding a viable strategy. Despite the struggles of computational models in modeling and simulating control strategies for this system, Maurice et  al. (2018) found that people were able to consistently fnd, evaluate, execute, and maintain both in-phase and anti-phase movement patterns, which allowed them to control the system stably. In both cases, modeling such complex systems, exploring solutions, evaluating performance under varying conditions, and shifting behavior is not trivial. Nevertheless, our ability to consistently excel at this task supports the notion that even complex coupling strength and invariant structure can be perceived directly. In our frst example, Lee (1976) argued that the task of avoiding collision with a vehicle can be better explained by relying on the relationship between the lead vehicle and the agent’s action, as quantifed by the rate of optical expansion. In the context of autonomous vehicles, we argued that the difcult decision of when to use automation can be better understood by quantifying the relationship between the agent’s ability, the automation’s quality, and the task constraints. Similarly, with the fuid-in-a-cup system, we showed that the difcult task of fnding and maintaining stable control of a chaotic system can be best characterized by looking at the relative phase relationship between the vessel and the liquid. Overall, these recent fndings suggested that the classic view of afordance as being directly perceived can be extended to include higher-order relational terms that characterize interactions with modern autonomous systems. Why Is Afordance Any Better Than Stimulus? What Does a Theory of Afordance Suggest That Stimuli Cannot; How Has It Moved the Needle Past Gibson 1960’s Recognition of How Little We Can Defne Stimuli?

Gibson (1960) pointed out the difculties and discrepancies in defning stimuli in psychology. One of the issues that we want to focus on concerns whether stimuli can be defned strictly in physical terms or exist in the context of a perceiver. Gibson (1966) distinguished them with the labels available stimuli and efective stimuli. The physical existence of a stimulus does not mean that it will be efective. That depends on the presence of an acting and perceiving observer. For example, the horizon line is specifed by the eye height of the observer. This is why ship lookouts are positioned on the crow’s nest; their horizons are further away than that of observers on the deck. While the information carried by light in the environment exists regardless of the observer, such stimuli only become efective when an observer is present. The major advantage of afordances over stimuli is that afordances focus on the relational properties of an agent-environment system instead of treating stimuli in the environment and responses from the agent as

354

Tri Nguyen et al.

independent entities that need to be combined and matched. However, as researchers turn their attention to more complex behaviors, the diferences between these two approaches become more and more obvious. Consider the workload of a driver engaged in a complex driving scenario. By defnition, workload is a relational quality between the driver and the surrounding environment. Workload does not solely depend on the number and type of obstacles in the environment. After all, the same number of obstacles may be stressful to a novice driver but may not even draw the attention of a seasoned driver. Therefore, workload can only be defned as the relative match between environmental conditions and the driver’s experience and capability (Drnec & Metcalfe, 2016). In studies of human-autonomy teaming, workload, which is already a relational construct, is often the building block for even higher-order relational properties such as trust or reliance on automation (Demir & Cooke, 2022; Nguyen et al., 2022). A driver’s decision to use automation depends on the relationship among the quality of the autonomous system, the driver’s innate driving ability, their attitude toward automation, and their current workload. As we can see, understanding this complex behavior relies on relations nested within relations. Let us look at another example. Gibson (1966) described communication as structured information in the environment, for example, the structure of light refected from shapes and edges of symbols on a page or the congruence among patterns of sound, vocal, facial expressions, and gestures. Today’s research looks at the structure of these structured constructs. For instance, Gorman et al. (2010) quantifed the structure of the communicative pattern between teammates in a simulated command and control task. Instead of considering communication from each team member as a separate variable, they derived a relational parameter called kappa that captured coordination among team members based on the timing of essential interactions. The evolving dynamics of kappa were used to understand how coordination at the team level changed throughout a task. Task performance was successfully predicted by kappa, where team performance was best when the team remained fexibly coordinated in various contexts. Here, we see Gibson’s principle extended to examine the structure of structured information in a team structure that extends across individuals. Since Gibson’s (1960) critical treatment of the term stimulus, researchers have continually built more complex explanations of behaviors by building on relationships and structures. We frequently look at the relationship between relations and structure of structures to help us model ever more complex systems (Gao & Lee, 2006; Smith & Thelen, 1993; Stevens et al., 2013). In many cases, by setting the level of analysis to the relational level (e.g., tau; Lee, 1976) rather than the level of constituent components (speed, distance, acceleration in the Lee example), we may

Afordances in Interaction With Autonomous Systems

355

begin identifying the parameters that control the system of interest, and, in turn, the behavior of interest (e.g., kappa, Gorman et al., 2010). Modeling behavior at the relational level also sidesteps the excessive number of required conditionals to account for all possible context-dependent variables. These reasons highlight the advantage of emphasizing relational units. The alternative would require independent stimulus units within the environment or the organism and rules for matching the two. Not only would a stimulus-response equivalent require excessive numbers of conditionals to account for all the context-dependent variables; it simply makes more sense to treat relational qualities as the building blocks rather than treating physical properties of the environment and responses from an agent as building blocks and attempt to list out rules for matching them. Most importantly, modeling systems using relational terms allow us to preserve the system’s dynamic even as we examine higher-order patterns. What Is the Connection Between Afordances, Behavioral Scale, and Intention? How Distant or Reluctant Can a PerceivingActing System Be Before Afordances Come Into Play? Do Afordances Exert Their Role Only When the System Is Close or Willing Enough?

Afordances guide actions at multiple behavioral scales, some of which can be very distant. For example, it is well-known that the rate of expansion of the retinal image can provide information on the potential and timing of collisions (Lee, 1976). In this instance, the invariant information that informs action can be derived from a short window preceding the current action. However, we have also shown evidence that afordances guide action in complex task environments. For example, drivers must decide whether to engage automation when approaching a difcult S-curve. That is, their perception of the afordance of drivability for this road section guides their decision to use automation. In this case, the evidence accumulated on the capability of their autonomous system occurred over an extended period preceding the action. Indeed, when we examined the data during S-curves across all automation quality, we detected a high correlation between the variability in driving data and variability in course features across nested frequency bands ranging between 1-second and 16-second cycles. However, when participants had access to highly capable automation, this correlation between driving behavior and course features also unfolded over extended 60-second cycles. In other words, the evidence indicated that the decision to engage with automation might utilize information accumulated over an extensive time span. Afordance should not be limited to any specifc scale of behavior.

356

Tri Nguyen et al.

Intention is an integral component of afordance. Gibson (1979) described intention, or need, as an invitation or demand for action. Intention is the anchor on the part of the agent that specifes an afordance. Invariant information is always present in the environment, so intention is the mechanism with which an agent selects which afordance to attend to. For example, an apple always afords eating. However, if the agent’s need is not to fnd sustenance, this afordance will simply not be picked up. Clearly, to Gibson (1979), the theory of afordance cannot function without intention. When we examine the concept of intention in interaction with complex and autonomous systems, intention is still just as important, if not more so. Every time a participant turns on automation or decides on a movement pattern, they need to evaluate the outcome of said action in the context of their intended goal, whether to avoid collision with obstacles or fnd a stable control pattern. However, because the invariant information to be attuned in complex or autonomous systems is opaque, the evaluation of behavior outcomes becomes even more critical. If the participant does not have a clear goal in mind, it becomes impossible to judge whether a use case for automation or a new movement pattern successfully moves the participant closer to their desired outcome. In other words, without intention, the attunement process cannot be completed. Consequently, perception of the afordances of complex and autonomous systems also cannot happen. In modeling terms, intention can be operationalized as the model’s ftness function. Without a ftness function, the performance of a set of parameters cannot be evaluated. Without this evaluation, we cannot fnd a set of parameters to generate higher performance. Indeed, numerous ftness functions can be selected in modeling the fuid-in-a-cup system, each corresponding to a diferent movement goal, such as maximizing predictability or minimizing force expenditure. Similar to a shift in intention in classic examples of afordance, such a shift in the ftness function would lead to drastically diferent movement solutions (Bazzi & Sternad, 2020). Likewise, in the discussion of invariant variability structures in complex systems, selective pressure is repeatedly raised (intent and constraints from the environment, Van Orden et  al., 2003; the need to exert control on behavior, Likens et al., 2015). These considerations all point to the importance of intention in the perception of afordances. How Do Systems Engage With Afordances as They Move Among Tasks and Intentions?

As context-dependent relations between agent and environment, afordances necessarily change in response to changes in the system’s confguration, whether those changes occur in the environment, in intention, or

Afordances in Interaction With Autonomous Systems

357

the agent’s capabilities (Konczak et al., 1992). When the retinal image of an object in your feld of view suddenly starts expanding, that object now afords collision (Lee, 1976; Oudejans et al., 1996). When information in the environment changes, new afordances become available. A cofee table afords placing an object when looking for a place to put down your groceries but may aford sitting if you want to sit down (Heras-Escribano, 2019). Afordances of the same environment change as intention shifts. A chair may aford crawling under when you are still a small child but may aford only sitting when you are an adult. As the agent’s capabilities change, so do afordances. These are well-known examples of how afordances fexibly shift whenever there is a change in the system’s confguration. The role of tools in perception of afordances is an important question due to their dual roles both as an alteration of the agent’s capabilities and as a part of the information in the environment (Bingham et  al., 1989; Chemero, 2009). For example, with a hammer in hand, a stone core suddenly afords knapping (Bril et al., 2010). However, how should this afordance be understood? Does the agent now perceive the stone as knapp-able, considering the newfound hardness of the agent-tool system? Or does the agent perceive the stone as knapp-able through the graspable hammer? To further complicate matters, how do autonomous systems ft in with this framework of afordances (Figure 18.2)? Should they be viewed simply as one of the many unpredictable parts of the environment? Or should they be considered an additional source of intentionality with their afordances, to be considered whenever afordances are perceived and actualized (Stoffregen et  al., 1999) Or should they be subsumed into the agent’s now expanded and extended capability? We do not believe that autonomous

FIGURE 18.2

The relationship between the agent and the environment is complicated by the addition of automation. Should automation be treated as another part of the environment, as a part of the agent’s capabilities, or as its own entity?

358

Tri Nguyen et al.

systems can be fully described with just one of these viewpoints. The various forms and diverse contexts with which interactions with autonomous systems may unfold, require us to treat each instance on a case-by-case basis or adopt a more generalized view. For example, if we encounter a self-driving car or delivery robot on the road, they would most likely be considered an unpredictable part of the environment due to our limited understanding of their behavior and inability to control their actions or communicate our intentions. That is, the autonomous system itself would aford un-interactable. On the other hand, if we were considering whether to hand over the task of driving to an autonomous vehicle, that system would more likely be considered an embodied, extended part of the agent’s action capability due to our attunement to the system’s behavior and ability to engage or disengage at will. This view is similar to an embodied tool perspective (Fragaszy & Mangalam, 2018; Martel et al., 2016; Wagman & Carello, 2003). If the autonomous system is another agent or teammate, it may be best to view them as an independent source of intention. We cannot predict their behavior well nor disengage with them as needed. Instead, we can only communicate our intention to them, then interact with the system’s emergent behavior. In short, the specifcs of each autonomous system and the context for action dictate how afordances are engaged. Alternatively, we can view perceiving and actualizing afordances with autonomous systems more broadly. From a system view, autonomous systems are embedded in the agent-environment system and are simply a part of the overall structure—yet another process adding its unique variability to the greater dynamical landscape. This view does not contradict the characterization of afordances presented in our examples. Rather, it recontextualizes each instance simply as a variation of the same cohesive system. Overall, we argue that afordances should be viewed as the relational term specifying the relationship between the environment, the agent’s intention, and their capabilities. That is, when any element of the system changes, the system does not engage with a diferent afordance. Rather, a diferent part of the perceptual array specifes the system’s new confguration. Concluding Remarks

In this chapter, we advocate that afordance is invariant information specifc to the relationship between an agent and the environment. Perceiving and acting with afordances are attuning and exploiting this information to fulfll an intentional outcome. As more complex and unpredictable autonomous systems are introduced into our interactions with the world, attuning to the invariant properties of such systems may be more challenging. However, the process is not fundamentally diferent from any other interactions. Understanding

Afordances in Interaction With Autonomous Systems

359

interactions with autonomous systems as nested, invariant relations has major advantages over the alternative, which requires defning stimuli and responses with extensive conditionals to refect the complexity of realistic behaviors. In contrast, focusing on relational terms allows us to preserve the dynamics at smaller scales and extend our explanations of the system’s behavior to include more complex interactions at larger scales. In conclusion, the perspective presented here is an essential framework going forward, especially as we tackle behaviors in ever more complex and realistic settings. Reference List Amazeen, P. G. (2018). From physics to social interactions: Scientifc unifcation via dynamics. Cognitive Systems Research, 52, 640–657. https://doi.org/10.1016/j. cogsys.2018.07.033 Amazeen, P. G., Amazeen, E. L., & Turvey, M. T. (1998). Breaking the refectional symmetry of interlimb coordination dynamics. Journal of Motor Behavior, 30(3), 199–216. https://doi.org/10.1080/00222899809601337 Bazzi, S., & Sternad, D. (2020). Human control of complex objects: Towards more dexterous robots. Advanced Robotics, 34(17), 1137–1155. https://doi.org/10.1 080/01691864.2020.1777198 Bingham, G. P., Schmidt, R. C., & Rosenblum, L. D. (1989). Hefting for a maximum distance throw: A smart perceptual mechanism. Journal of Experimental Psychology: Human Perception and Performance, 15, 507–528. https://doi. org/10.1037/0096-1523.15.3.507 Bril, B., Rein, R., Nonaka, T., Wenban-Smith, F., & Dietrich, G. (2010). The role of expertise in tool use: Skill diferences in functional action adaptations to task constraints. Journal of Experimental Psychology: Human Perception and Performance, 36(4), 825–839. https://doi.org/10.1037/a0018171 Cardinali, L., Frassinetti, F., Brozzoli, C., Urquizar, C., Roy, A. C., & Farnè, A. (2009). Tool-use induces morphological updating of the body schema. Current Biology, 19(12), R478–R479. https://doi.org/10.1016/j.cub.2009.05.009 Chemero, A. (2003). An outline of a theory of afordances. Ecological Psychology, 15(2), 181–195. https://doi.org/10.1207/S15326969ECO1502_5 Chemero, A. (2009). Radical Embodied Cognitive Science. MIT Press. Chemero, A., & Turvey, M. T. (2007). Gibsonian afordances for roboticists. Adaptive Behavior, 15(4), 473–480. https://doi.org/10.1177/1059712307085098 Demir, M., & Cooke, N. J. (2022). Interpersonal coordination and asynchronization in human-machine teams. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 66(1), 2041. Deterding, S., Dixon, D., Khaled, R., & Nacke, L. (2011). From game design elements to gamefulness: Defning “gamifcation.” MindTrek ’11: Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments, 9–15. https://doi.org/10.1145/2181037.2181040 Drnec, K., & Metcalfe, J. S. (2016). Paradigm development for identifying and validating indicators of trust in automation in the operational environment of human automation integration. In D. D. Schmorrow & C. M. Fidopiastis (Eds.),

360

Tri Nguyen et al.

Foundations of Augmented Cognition: Neuroergonomics and Operational Neuroscience (pp. 157–167). Springer. Evtimov, I., Eykholt, K., Fernandes, E., Kohno, T., Li, B., Prakash, A., Rahmati, A., & Song, D. (2017). Robust physical-world attacks on machine learning models. ArXiv Preprint ArXiv: 1707.08945, 2(3), 4. Fitzpatrick, P., Carello, C., Schmidt, R. C., & Corey, D. (1994). Haptic and visual perception of an afordance for upright posture. Ecological Psychology, 6(4), 265–287. https://doi.org/10.1207/s15326969eco0604_2 Fragaszy, D. M., & Mangalam, M. (2018). Tooling. Advances in the Study of Behavior, 50, 177–241. https://doi.org/10.1016/bs.asb.2018.01.001 Gao, J., & Lee, J. D. (2006). Extending the decision feld theory to model operators’ reliance on automation in supervisory control situations. IEEE Transactions on Systems, Man, and Cybernetics—Part A: Systems and Humans, 36(5), 943–959. https://doi.org/10.1109/TSMCA.2005.855783 Gaver, W. W. (1991). Technology afordances. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 79–84. https://doi. org/10.1145/108844.108856 Gibson, J. J. (1960). The concept of the stimulus in psychology. American Psychologist, 15(11), 694–703. https://doi.org/10.1037/h0047037 Gibson, J. J. (1966). The Senses Considered as Perceptual Systems. Houghton Mifin. Gibson, J. J. (1979). The Ecological Approach to Visual Perception. Houghton Mifin. Gorman, J. C., Amazeen, P. G., & Cooke, N. J. (2010). Team coordination dynamics. Nonlinear Dynamics, Psychology, and Life Sciences, 14, 265–289. Graziano, M. S. A., Afalo, T. N. S., & Cooke, D. F. (2005). Arm movements evoked by electrical stimulation in the motor cortex of monkeys. Journal of Neurophysiology, 94(6), 4209–4223. https://doi.org/10.1152/jn.01303.2004 Gremillion, G. M., Metcalfe, J. S., Marathe, A. R., Paul, V. J., Christensen, J., Drnec, K., Haynes, B., & Atwater, C. (2016). Analysis of trust in autonomy for convoy operations. Proc. SPIE 9836, Micro-and Nanotechnology Sensors, Systems, and Applications VIII, 98361Z. https://doi.org/10.1117/12.2224009 Heras-Escribano, M. (2019). The Philosophy of Afordances. Palgrave Macmillan. Hodges, B. H., & Fowler, C. A. (2010). New afordances for language: Distributed, dynamical, and dialogical resources. Ecological Psychology, 22(4), 239–253. https://doi.org/10.1080/10407413.2010.517086 Hommel, B., Müsseler, J., Aschersleben, G., & Prinz, W. (2001). The Theory of Event Coding (TEC): A framework for perception and action planning. Behavioral and Brain Sciences, 24(5), 849–878. https://doi.org/10.1017/S0140525X01000103 Kelso, S. J. A. (1984). Phase transitions and critical behavior in human bimanual coordination. American Journal of Physiology-Regulatory, Integrative and Comparative Physiology, 246(6), R1000–R1004. https://doi.org/10.1152/ ajpregu.1984.246.6.R1000 Kelso, S. J. A. (1995). Dynamic Patterns: The Self Organization of Brain and Behaviour. MIT Press. Konczak, J., Meeuwsen, H. J., & Cress, M. E. (1992). Changing afordances in stair climbing: The perception of maximum climbability in young and older adults. Journal of Experimental Psychology: Human Perception and Performance, 18(3), 691–697. https://doi.org/10.1037/0096-1523.18.3.691

Afordances in Interaction With Autonomous Systems

361

Lee, D. N. (1976). A theory of visual control of braking based on information about time-to-collision. Perception, 5(4), 437–459. https://doi.org/10.1068/p050437 Likens, A. D., Fine, J. M., Amazeen, E. L., & Amazeen, P. G. (2015). Experimental control of scaling behavior: What is not fractal? Experimental Brain Research, 233(10), 2813–2821. https://doi.org/10.1007/s00221-015-4351-4 Maravita, A., & Iriki, A. (2004). Tools for the body (schema). Trends in Cognitive Sciences, 8(2), 79–86. https://doi.org/10.1016/j.tics.2003.12.008 Mark, L. S. (1987). Eyeheight-scaled information about afordances: A study of sitting and stair climbing. Journal of Experimental Psychology: Human Perception and Performance, 13(3), 361–370. https://doi.org/10.1037/0096-1523.13.3.361 Martel, M., Cardinali, L., Roy, A. C., & Farnè, A. (2016). Tool-use: An open window into body representation and its plasticity. Cognitive Neuropsychology, 33(1–2), 82–101. https://doi.org/10.1080/02643294.2016.1167678 Maurice, P., Hogan, N., & Sternad, D. (2018). Predictability, force, and (anti)resonance in complex object control. Journal of Neurophysiology, 120(2), 765–780. https://doi.org/10.1152/jn.00918.2017 Mechsner, F., Kerzel, D., Knoblich, G., & Prinz, W. (2001). Perceptual basis of bimanual coordination. Nature, 414(6859), 69–73. https://doi.org/10.1038/35102060 National Highway Trafc Safety Administration (2022). Summary Report: Standing General Order on Crash Reporting for Automated Driving Systems (DOT HS 813 324). U.S. Department of Transportation. https://www.nhtsa.gov/sites/ nhtsa.gov/fles/2022-06/ADS-SGO-Report-June-2022.pdf Nguyen, T., Magaldino, C., Landfair, J., Demir, M., Amazeen, P. G., & Kang, Y. (2022). Distinguishing driving behavior using the dynamical systems analysis (DSA) toolbox: Implications for trust in automation. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 66(1), 822. https://doi. org/10.1177/1071181322661469 O’Regan, J. K., & Noë, A. (2001). A sensorimotor account of vision and visual consciousness. Behavioral and Brain Sciences, 24(5), 939–973. https://doi. org/10.1017/S0140525X01000115 Oudejans, R. R. D., Michaels, C. F., Bakker, F. C., & Dolné, M. A. (1996). The relevance of action in perceiving afordances: Perception of catchableness of fy balls. Journal of Experimental Psychology: Human Perception and Performance, 22, 879–891. https://doi.org/10.1037/0096-1523.22.4.879 Piolat, A., Roussey, J.-Y., & Thunin, O. (1997). Efects of screen presentation on text reading and revising. International Journal of Human-Computer Studies, 47(4), 565–589. https://doi.org/10.1006/ijhc.1997.0145 Rizzolatti, G., & Craighero, L. (2004). The Mirror-Neuron System. John Wiley & Sons. Sharif, M., Bhagavatula, S., Bauer, L., & Reiter, M. K. (2016). Accessorize to a crime: Real and stealthy attacks on state-of-the-art face recognition. CCS ’16: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, 1528–1540. https://doi.org/10.1145/2976749.2978392 Smith, L. B., & Thelen, E. (1993). A Dynamic Systems Approach to the Development of Cognition and Action. MIT Press. Stefen, J. H., Gaskin, J. E., Meservy, T. O., Jenkins, J. L., & Wolman, I. (2019). Framework of afordances for virtual reality and augmented reality. Journal of

362

Tri Nguyen et al.

Management Information Systems, 36(3), 683–729. https://doi.org/10.1080/07 421222.2019.1628877 Stephen, D. G., Dixon, J. A., & Isenhower, R. W. (2009). Dynamics of representational change: Entropy, action, and cognition. Journal of Experimental Psychology: Human Perception and Performance, 35(6), 1811–1832. https://doi. org/10.1037/a0014510 Stevens, R., Gorman, J. C., Amazeen, P., Likens, A., & Galloway, T. (2013). The organizational neurodynamics of teams. Nonlinear Dynamics in Psychology and Life Sciences, Psychology, and Life Sciences, 17(1), 67–86. Stofregen, T. A. (2003). Afordances as properties of the animal-environment system. Ecological Psychology, 15(2), 115–134. https://doi.org/10.1207/S1532 6969ECO1502_2 Stofregen, T. A., Bardy, B. G., & Mantel, B. (2006). Afordances in the design of enactive systems. Virtual Reality, 10(1), 4–10. https://doi.org/10.1007/ s10055-006-0025-7 Stofregen, T. A., Gorday, K. M., Sheng, Y.-Y., & Flynn, S. B. (1999). Perceiving afordances for another person’s actions. Journal of Experimental Psychology: Human Perception and Performance, 25, 120–136. https://doi.org/10.1037/ 0096-1523.25.1.120 Tschacher, W., & Dauwalder, J.-P. (2003). Dynamical Systems Approach to Cognition, the: Concepts and Empirical Paradigms Based on Self-Organization, Embodiment, and Coordination Dynamics (Vol. 10). World Scientifc. Turvey, M. T. (1992). Afordances and prospective control: An outline of the ontology. Ecological Psychology, 4(3), 173–187. https://doi.org/10.1207/s153269 69eco0403_3 Turvey, M. T., Shaw, R. E., Reed, E. S., & Mace, W. M. (1981). Ecological laws of perceiving and acting: In reply to Fodor and Pylyshyn (1981). Cognition, 9(3), 237–304. Van Orden, G. C., Holden, J. G., & Turvey, M. T. (2003). Self-organization of cognitive performance. Journal of Experimental Psychology: General, 132(3), 331–350. https://doi.org/10.1037/0096-3445.132.3.331 Wagenmakers, E.-J., Farrell, S., & Ratclif, R. (2004). Estimation and interpretation of 1/fα noise in human cognition. Psychonomic Bulletin & Review, 11(4), 579–615. https://doi.org/10.3758/BF03196615 Wagman, J. B., & Carello, C. (2003). Haptically creating afordances: The user-tool interface. Journal of Experimental Psychology: Applied, 9(3), 175–186. https:// doi.org/10.1037/1076-898X.9.3.175 Wagman, J. B., Langley, M. D., & Farmer-Dougan, V. (2018). Carrying their own weight: Dogs perceive changing afordances for reaching. Quarterly Journal of Experimental Psychology, 71(5), 1040–1044. https://doi.org/10.1080/1747021 8.2017.1322990 Warren, W. H. (1984). Perceiving afordances: Visual guidance of stair climbing. Journal of Experimental Psychology: Human Perception and Performance, 10(5), 683–703. https://doi.org/10.1037/0096-1523.10.5.683 Warren, W. H. (1990). The perception—Action coupling. In B. Bloch & B. I. Bertenthal (Eds.), Sensory-Motor Organizations and Development in Infancy and Early Childhood (pp. 23–37). Springer.

19 ON AFFORDANCES AND THEIR ENTAILMENT FOR AUTONOMOUS ROBOTIC SYSTEMS Mihai Andries, Lorenzo Jamone, Justus H. Piater, and Erol Sahin

In robotics, afordances endow robots with autonomy by allowing them to identify action possibilities in their environment and predict their efects, making action planning possible. The robotics community is transitioning from a perspective where afordances (and robotic abilities, in general) are considered fxed (innate) (Do et  al., 2018; Myers et  al., 2015) to a perspective where afordances are considered learnable (acquired) (Akbulut & Ugur, 2020; Montesano et al., 2018). This learnable approach is championed by cognitive robotics in general, and developmental robotics in particular (Min et  al., 2016). Here, robotic autonomy involves discovering and learning new afordances through sensorimotor exploration. In this perspective, when a robot is presented with an object, afordances allow us to understand how it can be used, that is, which functions it can provide. Thus, a robot is expected to recognize afordances available to it as well as to other agents in the environment. A robot aware of its perceptual/cognitive/actuation limitations (for instance, not being able to open a door), in a human-robot interaction setting, may ask for a service from a human who is able to provide it (Rosenthal et al., 2010). What do we understand by afordances in robotics? How are they linked to intentions and action planning? These questions will be explored in this chapter, which is an edited version of answers given by four roboticists working on the topic of afordances, amalgamated together into a coherent essay.

DOI: 10.4324/9781003396536-24

364

Mihai Andries, Lorenzo Jamone, Justus H. Piater, and Erol Sahin

What Do We Understand by the Term Afordance?

At its core, the notion of afordance promotes the idea that an agent’s perception abilities are shaped by its own action abilities, rather than being action-agnostic and generic. Coupled with the goal of the agent, this perspective allows the agent to perceive only the properties of the environment that are relevant for actions toward achieving its goal. Meanwhile, the same environment may ofer diferent action possibilities to diferent agents, depending on the sensorimotor capabilities of the specifc agent. For example, a stable surface afords traversing to an agent that is able to locomote and whose body dimensions ft the size of the surface borders; a stone afords hammering to an agent who is able to pick it and who has enough force to lift it. Formally, this entails representing afordances as a relationship between an object and an agent, via an action, a resulting efect, all happening within an environment allowing this interaction. Describing objects, agents, actions, efects, and environments using their features (physical or logical) allows generic descriptions through pattern matching. What Role Does Afordance Play in Perception? Is It the Entity Organisms Perceive or the Means Through Which Organisms Perceive? Are Afordances the Only Perceptual Dependent Variables? Role of Afordances in Perception

Perception is the process of making sense of sensory signals. Afordances are entities that organisms perceive and discover through interaction with their environment. Although afordances have traditionally been associated to visual perception (Gibson, 1966), the concept of afordances can be associated to any of the other senses. Are Afordances Means or Ends of Perception?

Making this distinction may not be particularly useful, since perception is not generally a unidirectional processing pipeline. At least in higher animals, perception at its core involves reconciling an internal model with sensory signals. Nevertheless, organizing the perceptual world into afordances recruits them as means (in addition to ends) of perception. Perception gives rise to behavior, and information currently not perceived is flled in by memory or inference. Internal models can be placed along a continuum, from highly reactive visuomotor behavior (like when playing tennis) to remarkably detailed internal models (like those of blind people).

On Afordances for Autonomous Robotic Systems

365

This coexistence of diferent processing scales has historically inhibited discourse by falsely pitting reactive against refective behavior. Perceptual Economy

The means through which an agent perceives afordances are those of ecological perception, powerfully illustrated by Eleanor J. Gibson (1994, 2000, 2003): “narrowing down from a vast manifold of (perceptual) information to the minimal, optimal information that specifes the afordance of an event, object, or layout.” For this perceptual process to take place, the agent requires previous learning, that is, learning what is the minimal information to be picked: this happens through both evolution and development, leveraging the sensorimotor exploration of the environment by physical interaction. The notion of afordance promotes “perceptual economy” by computing only the perceptual cues that are relevant to the agent’s goals and action capabilities. Otherwise, action-agnostic perception would require an infnite amount of computational power. Through such minimal perception, it becomes possible to link the perceptual computation to the actions of the agent. Such minimal and action-linked perception is usually referred to as direct perception. Coupling this linking with the predicted efect of the action is usually referred to as perceiving the world through the action possibilities aforded by the environment. Are Afordances the Entity Organisms Perceive, or the Means Through Which Organisms Perceive?

If afordances were the only (primitive) means through which organisms perceive, this would imply that an organism has an innate set of perceivable afordances, which cannot be infnite in size due to physical limitations of our bodies. Adding a new afordance to this fnite set would require frst discovering the afordance through perception, perceiving it using more primitive features. Hence, it could be argued that afordances are perceived and are not the only means through which organisms perceive. For example, humans are good at visual object recognition (i.e., telling the name of an object they see), and this is a useful perceptual task that has little to do with afordances (though some of the representations might be shared between afordance perception and object recognition, as suggested by neuroscience (Kruger et al., 2013) and explored in robotics (Dehban et al., 2022). Conversely, nothing precludes an agent from later using afordances as a means for perception in order to attribute semantic meaning to objects in his environment (Do et al., 2018). To resume, we perceive more than just afordances, including objects that ofer no afordances (e.g., due to their distance or physical constitution,

366

Mihai Andries, Lorenzo Jamone, Justus H. Piater, and Erol Sahin

such as clouds; or abstract concepts, such as beauty). This being said, afordance perception, and more broadly ecological perception, is a central cognitive process of many (if not all) living organisms, from primates (Cisek, 2007) to cockroaches (Sponberg & Full, 2008), and maybe even plants (Frazier et al., 2020). In the primate brain, this has given rise to multiple visual processing streams, as evidenced by patients with brain lesions, who can recognize but not handle certain objects or, conversely, can handle objects without being able to name them or even without being consciously aware of them (Milner & Goodale, 1995). Why Is Afordance Any Better Than Stimulus? What Does a Theory of Afordance Suggest That Stimuli Cannot; How Has It Moved the Needle Past Gibson 1960’s Recognition of How Little We Can Defne Stimuli?

Stimulus is an entity (object, event, etc.) in the environment that triggers the agent’s perceptual processes to generate a response. Although it is used broadly in Psychology, the distinction on whether it refers to the actual entity in the environment, or perceptual processes triggered within the agent usually remains vague. Nevertheless, the concept of a stimulus shares some of the properties of afordance: frst, it is minimal, meaning that it refers to the minimal set of perceptions that can elicit a particular response. Second, it is directly linked to action, meaning that perceptions that do not elicit responses from the agent are not considered stimuli. The main diference between a stimulus and an afordance is that a stimulus automatically elicits a response without taking into account the conscious goal of the agent. The stimulus is part of a reactive control loop and would at least trigger the diversion of attention onto itself. An afordance, on the other hand, is not a perceptual process that is linked with an automatic response. It allows the agent to observe directly (automatically, unconsciously) what the environment ofers, but then use this to achieve its conscious goals. Afordance Versus Stimulus

An afordance can be seen as a tuple combining fve elements: the object, agent, action, efect, and environment, whereas a stimulus describes only the object/event that generates a reaction from the agent. Hence, a stimulus is directly linked to action: the action component is automatically triggered (it is reactive) and does not take into account the conscious goal of the agent. Gibson (1960) essentially points out that the term stimulus is heavily overloaded and is used by psychologists to refer to just about any sensory signal controlled by an experimenter, across a wide range of complexity,

On Afordances for Autonomous Robotic Systems

367

from delivering an electric pulse to a single sensory neuron to phrases of speech. In analyzing the use of the term, he tries to specify what types of stimuli are useful for what types of psychological experiments. To study mechanisms and roles of afordance, the notion of sensory signals seems more useful. Sensory signals are the signals produced by sensory neurons. Perception is the process of making sense of these signals. Our body constantly produces a food of sensory signals, most of which are irrelevant and are ignored by perceptual processes most of the time. Perception flters and processes sensory information to reconcile an internal model. Part of what perception produces is afordances. The perception of afordances starts from the sensing of specifc stimuli. Not all stimuli that are available: only those that are relevant to the agent, given the sensorimotor capabilities of the specifc agent; only the minimal set that the agent has learned to pick, from a stream of several stimuli. Then, the added value of afordances is that these stimuli are automatically translated into actions, or more precisely, action possibilities: the stimulus itself is not really perceived (e.g., represented, stored, memorized, reasoned upon), but instead it is directly converted (i.e., direct perception) into an action representation. It should be noted, at this point, that while Gibson was defning afordance perception from a theoretical point of view (he was a theoretical psychologist, after all), those who are interested in using the notion of afordances to improve autonomous systems (e.g., robots) might not fnd the word direct very useful. For example, in the case of robotics, visual stimuli are picked by a camera and translated into an electronic signal and then a matrix of numbers; to generate the action representations, some computation is nevertheless required, whether it is by direct perception or by representing the stimuli and then reasoning upon them. However, the useful part of the notion of afordances is that this computation can be minimal and fast; to obtain these useful characteristics, the computation has to assume the sensorimotor capabilities of the agent as a prior, and this is typically obtained by previous learning (i.e., by exploring the environments using its own sensorimotor abilities, the agent has learned what is the minimal amount of information to pick from the visual stream, and therefore the generation of an action representation from such picked information can be fast and efective). Fast Afordance Detection

It is not evident how useful it is to insist that afordance detection be minimal or fast. No action opportunity pops abruptly into existence; they are part of the agents’ environment, of which the agent maintains a continually evolving internal model (at fuctuating levels of detail, precision, accuracy, and awareness). At what level of computational cost of detection

368

Mihai Andries, Lorenzo Jamone, Justus H. Piater, and Erol Sahin

does an afordance cease to be an afordance? Moreover, robustness can be expected to be ecologically more important than (and at odds with) minimality of sensory signals involved. By the same argument, Gibson’s notion of direct perception does not seem crucially relevant to afordances. Directness of perception mostly refers to no reliance on memory, inference, or learning. This largely limits the level of sophistication and abstraction of direct perception to shape-from-X, which plausibly includes the detection of some but not all afordances. Afordance theory provides several benefts: • It allows to create semantic maps of the environment, by labeling objects with afordances they provide. • It allows to identify a missing element within an afordance tuple, based on its experience (in the form of known afordances). For example, it allows to identify an appropriate action on a given object to obtain a desired efect (Awaad et al., 2013). • Since afordances incorporate an object’s physical description, they can be used in object design (Andries et al., 2020). Conceptually, objects could be designed on the basis of a list of afordances they must provide, using knowledge of corresponding physical shapes providing those afordances. • It allows to defne object interaction interfaces in the object space frame, which are generic with respect to the robot model. For instance, an afordance may be available to any robot capable of performing a specifc action (described in the object space frame) on the object. The robot can compute the corresponding action (in its own body frame) using inverse kinematics. Pragmatic work in robotics, compared to Gibson’s intentionally vague defnitions of afordances, moved on to formal defnitions of afordances that allow unambiguous software implementation on the robot (Chavez-Garcia et al., 2017; Krüger et al., 2011). What Is the Connection Between Afordances, Behavioral Scale, and Intention? How Distant or Reluctant Can a PerceivingActing System Be Before Afordances Come Into Play? Do Afordances Exert Their Role Only When the System Is Close or Willing Enough? Afordances and Intentions

Although there is no need for intentions in order to perceive afordances, intentions are useful for economical perception of the environment. In the presence of intentions, afordances are possibilities that can be acted

On Afordances for Autonomous Robotic Systems

369

upon. It can be argued that an agent always has active implicit or explicit intentions/goals. Because perceiving afordances is fast and requires minimal computation, it is not a burden for an agent to perceive afordances that might not be used; however, having the action representations ready can then speed up action execution if the intention (i.e., the need for action) comes to be. Similarly, afordances ofered by the immediate environment are perceived faster than afordances that are ofered in the future after the execution of a series of actions. In addition, the perception of an action possibility could in fact suggest to the agent that a useful action that the agent did not think about (i.e., an action for which there was no intention at all) is available. This can be extremely useful for robotic systems as well. In fact, most formalizations of afordances in robotics rely on representations that include not only the action but also the efects (or in other terms, the goal) of the action (Dehban et al., 2016, 2017; Gonçalves et al., 2014; Krüger et al., 2011; Montesano et  al., 2008; Sarathy & Scheutz, 2018; Stramandinoli et al., 2018; Ugur & Piater, 2017). Therefore, the perceived possibilities for actions (and for achieving certain efects) can be used for action planning, leading to problem solving. If intention can be formulated as a goal state with specifc properties, and if afordances aford actions performing transitions between states (with specifc preconditions and efects), then we can perform symbolic planning to identify a sequence of applied afordances that brings us to the desired goal state. In this sense, afordance perception can be seen as a semantic layer of perception. Afordances represent the possible actions that were identifed by the agent in the environment, and which are available for planning purposes. When an agent interacts with its environment, it perceptually parses the environment into relevant afordances. The perceived afordances are necessarily fltered by the agent’s behavior or intent, as the contrary would require a fnite set of types and instances of afordances to exist within the agent’s perceived surroundings. For example, when strolling through a forest among shrubs and branches, there is an infnity of grasp afordances almost everywhere, but they are all useless to the strolling agent. It is wasteful and in fact impossible for a fnite brain to process all of them all the time. What the agent cares about is afordances for foot placement, obstacles, etc. However, when the agent contemplates climbing a tree, the world is perceived very diferently, and reachable grasp afordances become perceptual building blocks. Ideally, perceiving afordances should be implemented as a rapid and computationally efcient process, enabling an agent to perceive afordances that may be utilized immediately or later on. Having readily available

370

Mihai Andries, Lorenzo Jamone, Justus H. Piater, and Erol Sahin

action representations can expedite action execution when the intention or need for action arises. In this context, afordances ofered by the immediate environment are perceived more swiftly than those that arise in the future following a series of executed actions. Simply perceiving the possibility of action can alert the agent to a helpful action that had not been previously considered or intended. This aspect holds signifcant value, particularly for robotic systems. In robotics, most formalizations of afordances encompass not only the action itself but also its efects. This comprehensive representation enables the perception of potential actions and associated efects, which can be leveraged by robotic systems to engage in efcient action planning (e.g., enabling them to navigate cluttered environments and solve complex tasks efectively without being overwhelmed by the multitude of possible actions). The computationally fast perception of afordances (comprehending actions and their efects) contributes to improving robotic systems’ problem-solving capabilities and overall performance. How Do Systems Engage With Afordances as They Move Among Tasks and Intentions? Afordances and Tasks in Cognitive Robotics

When the concept of afordances is used in autonomous systems (e.g., robots), it becomes evident that afordance perception is only a part of a larger cognitive architecture. Robotic approaches that relied on afordances and ecological perception alone did achieve interesting reactive behaviors (Arkin, 1998; Brooks, 1990; Duchon et  al., 1998; Murphy, 1999) but could not scale to more complex tasks. However, the integration of afordance perception with other cognitive processes (e.g., object recognition, attention, semantic reasoning, inhibition, action planning) within larger cognitive architectures (Antunes et al., 2016; Franklin et al., 2014; Kruger et al., 2009; Sun, 2007; Vernon et al., 2011) has enabled robots to realize complex problem solving in unstructured environments. Observation and Inference of Afordances

Afordances ofer themselves to the agent; the agent flters them by tasks and intentions, and may perceive them consciously, subconsciously, or not at all. The agent may actively look for them, or even infer them based on indirect evidence (such as observing another climber holding onto something invisible from the current vantage point).

On Afordances for Autonomous Robotic Systems

371

Planning Using Afordances

Since actions are characterized by their efects, afordances can serve as planning operators. In addition to planning over a fxed set of perceived afordances, problem solving may prompt the agent to actively look for specifc afordances to form a feasible or improved plan. The goal of the agent (defned by the tasks and/or intentions) requires the agent to plan (either through forward chaining or other mechanisms) a sequence of actions that would take the agent from the current environment to a desired state of the environment. Such “planning” requires the agent to start with the current environment, which is immediately perceivable to pan out and evaluate the possible courses of action that would bring the agent closer to its goal. Afordances support this process in two ways: frst, the afordances ofered by the environment (such as objects) can be used to defne and limit the action set for planning. Second, the goals (or subgoals) can be used to flter the set of afordances to be sought after in the environment preventing the agent to be “drowning” in afordances. In this planning context, an action described by an afordance, with its preconditions (object descriptions allowing the action, agent’s physical and logical capabilities) and its postconditions (description of efects of the applied action), can be seen as components of a planning domain. Thus, classical planning methods (McDermott, 1998) can be used to plan action sequences for executing tasks and reaching intended goals (Ahmetoglu et al., 2022). Guiding Learning Using Afordances

Conversely, afordances ofer opportunities for exploration. If the efect of an action is insufciently known, the agent may just try it out. This may lead to the discovery of new afordances. Concluding Remarks

The consensus is that an afordance is an opportunity for action ofered by an object to an agent. Formal defnitions present it as a relationship between features of an object, an agent, an action, an efect, and an environment. Afordances are discovered through sensorimotor exploration of the environment. Perceiving afordances allows to attribute semantic meaning to objects in the environment. An argument was provided suggesting afordance perception is not the only type of perception. In the domain of robotics, where multiple types of sensorial processing are commonplace (object detection, segmentation, recognition, etc.), this is defnitely true.

372

Mihai Andries, Lorenzo Jamone, Justus H. Piater, and Erol Sahin

The concept of afordance is more complex than that of “stimulus.” In a sense, an afordance provides a direct link between a stimulus and an action possibility. It is debatable whether afordance perception requires the presence of goals/intentions. On one hand, discovering afordances present in the environment does not require a specifc goal (other than the goal of learning how the environment behaves). On the other hand, goals help to flter afordances into useless and useful ones (for reaching a specifc goal). Intentionally scanning the environment only for specifc afordances allows for economic perception. Integrating afordance perception with cognitive processes like semantic reasoning and action planning endows robots with the ability to compute action sequences for reaching their intended goals. Like reinforcement learning and developmental (epigenetic) robotics, afordance is yet another concept originating in psychology that has found practical applications in robotics. Autonomous cognitive robots being still an immature technology, the evolution of the use of afordances in robotics is worth following closely. Reference List Ahmetoglu, A., Seker, M. Y., Piater, J., Oztop, E., & Ugur, E. (2022). Deepsym: Deep symbol generation and rule learning for planning from unsupervised robot interaction. Journal of Artifcial Intelligence Research, 75, 709–745. https://doi. org/10.1613/jair.1.13754 Akbulut, M. T., & Ugur, E. (2020). Learning object afordances from sensorymotor interaction via Bayesian networks with auto-encoder features. International Journal of Intelligent Systems and Applications in Engineering, 8(2), 52–59. https://doi.org/10.18201/ijisae.2020261584 Andries, M., Dehban, A., & Santos-Victor, J. (2020). Automatic generation of object shapes with desired afordances using voxelgrid representation. Frontiers in Neurorobotics, 14, 22. https://doi.org/10.3389/fnbot.2020.00022 Antunes, A., Jamone, L., Saponaro, G., Bernardino, A., & Ventura, R. (2016). From human instructions to robot actions: Formulation of goals, afordances and probabilistic planning. 2016 IEEE International Conference on Robotics and Automation (ICRA), 5449–5454. https://doi.org/10.1109/ICRA.2016.7487757 Arkin, R. C. (1998). Behavior-based Robotics. MIT Press. Awaad, I., Kraetzschmar, G. K., & Hertzberg, J. (2013). Afordance-based reasoning in robot task planning. Planning and Robotics (PlanRob) Workshop ICAPS-2013. Brooks, R. A. (1990). Elephants don’t play chess. Robotics and Autonomous Systems, 6(1), 3–15. https://doi.org/10.1016/S0921-8890(05)80025-9 Chavez-Garcia, R. O., Andries, M., Luce-Vayrac, P., & Chatila, R. (2017). Discovering and manipulating afordances. In D. Kulić, Y. Nakamura, O.

On Afordances for Autonomous Robotic Systems

373

Khatib, & G. Venture (Eds.), 2016 International Symposium on Experimental Robotics (pp.  679–691). Springer International Publishing. https://doi. org/10.1007/978-3-319-50115-4_59 Cisek, P. (2007). Cortical mechanisms of action selection: The afordance competition hypothesis. Philosophical Transactions of the Royal Society B: Biological Sciences, 362(1485), 1585–1599. https://doi.org/10.1098/rstb.2007.2054 Dehban, A., Jamone, L., Kampf, A. R., & Santos-Victor, J. (2016). Denoising autoencoders for learning of objects and tools afordances in continuous space. 2016 IEEE International Conference on Robotics and Automation (ICRA), 4866– 4871. https://doi.org/10.1109/ICRA.2016.7487691 Dehban, A., Jamone, L., Kampf, A. R., & Santos-Victor, J. (2017). A deep probabilistic framework for heterogeneous self-supervised learning of afordances. 2017 IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids), 476–483. https://doi.org/10.1109/HUMANOIDS.2017.8246915 Dehban, A., Zhang, S., Cauli, N., Jamone, L., & Santos-Victor, J. (2022). Learning deep features for robotic inference from physical interactions. IEEE Transactions on Cognitive and Developmental Systems, 1. https://doi.org/10.1109/ TCDS.2022.3152383 Do, T.-T., Nguyen, A., & Reid, I. (2018). AfordanceNet: An end-to-end deep learning approach for object afordance detection. 2018 IEEE International Conference on Robotics and Automation (ICRA), 5882–5889. https://doi. org/10.1109/ICRA.2018.8460902 Duchon, A. P., Kaelbling, L. P., & Warren, W. H. (1998). Ecological robotics. Adaptive Behavior, 6(3–4), 473–507. https://doi.org/10.1177/105971239800600306 Franklin, S., Madl, T., D’Mello, S., & Snaider, J. (2014). LIDA: A systems-level architecture for cognition, emotion, and learning. IEEE Transactions on Autonomous Mental Development, 6(1), 19–41. https://doi.org/10.1109/ TAMD.2013.2277589 Frazier, P. A., Jamone, L., Althoefer, K., & Calvo, P. (2020). Plant bioinspired ecological robotics. Frontiers in Robotics and AI, 7, 79. https://doi.org/10.3389/ frobt.2020.00079 Gibson, E. J. (1994). An Odyssey in Learning and Perception. MIT Press. Gibson, E. J. (2000). Perceptual learning in development: Some basic concepts. Ecological Psychology, 12(4), 295–302. https://doi.org/10.1207/ S15326969ECO1204_04 Gibson, E. J. (2003). The world is so full of a number of things: On specifcation and perceptual learning. Ecological Psychology, 15(4), 283–287. https://doi. org/10.1207/s15326969eco1504_3 Gibson, J. J. (1960). The concept of the stimulus in psychology. American Psychologist, 15(11), 694–703. https://doi.org/10.1037/h0047037 Gibson, J. J. (1966). The Senses Considered as Perceptual Systems. Houghton Mifin. Gonçalves, A., Abrantes, J., Saponaro, G., Jamone, L., & Bernardino, A. (2014). Learning intermediate object afordances: Towards the development of a tool concept. 4th International Conference on Development and Learning and on Epigenetic Robotics, 482–488. https://doi.org/10.1109/ DEVLRN.2014.6983027

374

Mihai Andries, Lorenzo Jamone, Justus H. Piater, and Erol Sahin

Krüger, N., Geib, C., Piater, J., Petrick, R., Steedman, M., Wörgötter, F., Ude, A., Asfour, T., Kraft, D., Omrčen, D., Agostini, A., & Dillmann, R. (2011). Object— Action complexes: Grounded abstractions of sensory—Motor processes. Robotics and Autonomous Systems, 59(10), 740–757. https://doi.org/10.1016/j. robot.2011.05.009 Kruger, N., Janssen, P., Kalkan, S., Lappe, M., Leonardis, A., Piater, J., RodriguezSanchez, A. J., & Wiskott, L. (2013). Deep hierarchies in the primate visual cortex: What can we learn for computer vision? IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8), 1847–1871. https://doi.org/10.1109/ TPAMI.2012.272 Kruger, N., Piater, J., Worgotter, F., Geib, C., Petrick, R., Steedman, M., Asfour, T., Kraft, D., Hommel, B., Agostini, A., Kragic, D., Eklundh, J.-O., Kruger, V., Torras, C., & Dillmann, R. (2009). A formal defnition of object-action complexes and examples at diferent levels of the processing hierarchy. Computer and Information Science, 1–39. McDermott, D., Ghallab, M., Howe, A., Knoblock, C., Ram, A., Veloso, M., Weld, D., & Wilkins, D. (1998). PDDL—The planning domain defnition language. Technical Report CVC TR-98–003/DCS TR-1165, Yale Center for Computational Vision and Control. Milner, A. D., & Goodale, M. A. (1995). The Visual Brain in Action. Oxford University Press. Min, H., Yi, C., Luo, R., Zhu, J., & Bi, S. (2016). Afordance research in developmental robotics: A survey. IEEE Transactions on Cognitive and Developmental Systems, 8(4), 237–255. https://doi.org/10.1109/TCDS.2016.2614992 Montesano, G., Crabb, D. P., Jones, P. R., Fogagnolo, P., Digiuni, M., & Rossetti, L. M. (2018). Evidence for alterations in fxational eye movements in glaucoma. BMC Ophthalmology, 18(1), 191. https://doi.org/10.1186/s12886-018-0870-7 Montesano, L., Lopes, M., Bernardino, A., & Santos-Victor, J. (2008). Learning object afordances: From sensory--Motor coordination to imitation. IEEE Transactions on Robotics, 24(1), 15–26. https://doi.org/10.1109/TRO.2007.914848 Murphy, R. R. (1999). Case studies of applying Gibson’s ecological approach to mobile robots. IEEE Transactions on Systems, Man, and Cybernetics—Part A: Systems and Humans, 29(1), 105–111. https://doi.org/10.1109/3468.736365 Myers, A., Teo, C. L., Fermüller, C., & Aloimonos, Y. (2015). Afordance detection of tool parts from geometric features. 2015 IEEE International Conference on Robotics and Automation (ICRA), 1374–1381. https://doi.org/10.1109/ ICRA.2015.7139369 Rosenthal, S., Biswas, J., & Veloso, M. M. (2010). An efective personal mobile robot agent through symbiotic human-robot interaction. AAMAS ’10: The Ninth International Conference on Autonomous Agents and Multiagent Systems, 10, 915–922. Sarathy, V., & Scheutz, M. (2018). A logic-based computational framework for inferring cognitive afordances. IEEE Transactions on Cognitive and Developmental Systems, 10(1), 26–43. https://doi.org/10.1109/TCDS.2016.2615326 Sponberg, S., & Full, R. J. (2008). Neuromechanical response of musculo-skeletal structures in cockroaches during rapid running on rough terrain. Journal of Experimental Biology, 211(3), 433–446. https://doi.org/10.1242/jeb.012385

On Afordances for Autonomous Robotic Systems

375

Stramandinoli, F., Tikhanof, V., Pattacini, U., & Nori, F. (2018). Heteroscedastic regression and active learning for modeling afordances in humanoids. IEEE Transactions on Cognitive and Developmental Systems, 10(2), 455–468. https:// doi.org/10.1109/TCDS.2017.2700207 Sun, R. (2007). The importance of cognitive architectures: An analysis based on CLARION. Journal of Experimental & Theoretical Artifcial Intelligence, 19(2), 159–193. https://doi.org/10.1080/09528130701191560 Ugur, E., & Piater, J. (2017). Emergent structuring of interdependent afordance learning tasks using intrinsic motivation and empirical feature selection. IEEE Transactions on Cognitive and Developmental Systems, 9(4), 328–340. https:// doi.org/10.1109/TCDS.2016.2581307 Vernon, D., Von Hofsten, C., & Fadiga, L. (2011). A Roadmap for Cognitive Development in Humanoid Robots. Springer Science & Business Media.

INDEX

abilities 4, 7, 8, 12–15, 19, 20, 27, 28, 34, 131, 133, 151, 192, 199, 255, 272, 276, 334, 337, 364; ability 14, 15, 18–20, 22, 28, 34, 45, 48, 49, 52, 53, 84, 87, 111, 122, 132, 133, 177, 187, 207, 233, 238, 240, 261, 272, 276, 283, 306, 309, 340, 353, 354, 358, 372; able-bodied 293, 305, 317; able-bodiedness 22; able-bodies 304 acoustic 166, 167, 349 adapt 7, 13, 29, 33, 92, 258, 282, 350; adaptations 110, 128, 274, 280; adapting 261; adaptive 102, 106, 129, 131, 209, 275, 280, 283, 303, 314, 315 agency 17, 18, 72, 110, 111, 166, 170, 277, 284, 303, 310, 313, 316 anatomical 5, 56, 57, 164, 177, 185, 261, 338; anatomy 55, 57, 151, 179, 180, 183, 199 anticipate 89, 90, 236; anticipates 93, 302; anticipating 238; anticipation 41, 89, 95–97, 104, 113; anticipations 96, 97 array 8, 73, 89, 96, 146, 149, 153, 155, 166, 222, 253, 262, 278, 279, 286, 358; arrays 6, 46, 48, 50–52, 56–58, 61–65, 107, 161, 166, 168, 170, 177, 186,

200, 202, 222, 252, 266, 278, 279, 349 artifcial 2, 27, 167, 250, 327, 330, 342; artifcial intelligence 350 assistive 303, 312, 313, 351 asymmetric 85, 109, 142, 194; asymmetries 323; asymmetry 18, 22, 274, 287 athlete 271, 275; athletes 7, 134, 275, 285, 315; athletic 272 attention 5, 15, 34, 71, 75–78, 81, 96, 121, 122, 155, 166, 167, 170, 186, 199, 216, 257, 258, 366, 370; attentional 258; attentive 162 attractor 228, 276; attractors 276, 284 attune 17, 339–341, 351; attuned 72, 255, 282, 294, 298, 310, 339, 348, 351, 356; attunement 8, 35, 122, 157, 209, 252, 254, 259, 313, 339, 340, 349–351, 356, 358 augment 339; augmented 208, 360, 361 automatic 91, 155, 366; automatically 235, 241, 342, 366, 367; automaticity 92, 235, 277; automation 348, 351–357; automatize 250; autonomous 2, 8, 325, 327, 331, 348, 350–359, 363, 367, 370, 372;

Index

autonomy 8, 313, 350, 360, 363 avoidance 106, 278, 335 aware 34, 58, 72, 76, 95, 147, 150, 157, 257, 301, 363, 366; awareness 72, 76, 96, 147, 150, 166, 224, 237, 299, 300, 310, 341, 367 barrier 254, 300, 311; barriers 45, 294, 303, 305, 306 Bayesian 184 behaviorism 91, 214, 224, 225; behaviorist 38, 54, 224 bifurcation 162, 275, 334; bifurcations 162 body-scaled 215, 272 brain 2, 6, 7, 20, 75, 85, 112, 164, 165, 167, 168, 202, 204–209, 215, 216, 219, 223, 225–227, 237, 240, 244–252, 258, 259, 274, 336, 338, 346, 366, 369; brain-body-environment 6, 199, 205, 206, 209; brain-damaged 240; brains 203, 209, 217, 218, 222–224, 258 braking 106, 116, 288, 352 branching 129–131 calibrate 17, 340, 341; calibrating 340; calibration 174, 193, 263, 274, 287, 339, 340, 343, 344, 347 capabilities 7, 8, 29, 46, 59, 101, 106, 115, 122, 123, 127–132, 134, 178, 188, 218, 222, 224, 235, 238, 239, 249, 252, 255, 257, 272, 283, 284, 293, 294, 299, 315, 319, 327, 340, 348, 350, 357, 358, 364, 365, 367, 370; capability 222, 293, 349, 354, 355, 357, 358; capable 61, 147, 157, 208, 283, 284, 330, 332, 355, 368 capacities 83, 96, 128, 130–133, 135, 142, 151, 226, 277, 297, 299, 309, 311–313, 315, 316, 318; capacity 18, 84, 89, 102, 122, 123, 127, 128, 130, 132, 133, 141, 151, 178, 183, 187, 226, 277, 282, 284, 307, 313, 314 caregiver 154, 155; caregiver-perceived 319; caregivers 34, 64, 295

377

cartesian 50, 53, 304, 305, 308, 309 cascade 188, 241, 259–261; cascadelike 259; cascades 185–187, 259; cascading 6, 259, 261 catch 33, 122; catch-ability 199; catchable 73; catching 33, 96, 215 categories 59, 61, 65, 169, 255; categorization 87; categorize 84, 86, 87; category 4, 20, 46, 48, 61, 65, 86, 224 causality 104; causation 36, 37, 155 cells 148, 150, 201, 226, 257, 336; cellular 150, 151 central nervous system 103, 255, 338 chaos 174, 191, 193, 259, 265; chaotic 351, 353 child 34, 127, 129, 130, 147, 298, 312, 314, 315, 357; childhood 131; children 19, 27, 34, 126–128, 131, 134, 302, 308, 320 chimpanzee 239; chimpanzees 163 choice 141, 144, 276, 315; choicepoint 73; choices 275 clinical 313; clinical-reasoning 313, 314; clinician 314 coach 314; coaches 64 cognition 4, 46, 59–63, 65, 71, 79, 80, 84, 95, 102, 165, 169, 179, 249–251, 271, 274, 278, 281, 283; cognitions 280; cognitive 4, 5, 8, 34, 46, 54, 59, 77, 79, 80, 85, 95, 97, 163, 165–167, 170, 201, 203–207, 220, 234, 238, 242, 243, 249–251, 262, 275, 283, 284, 329–332, 352, 363, 370, 372; cognitivism 214, 215, 224, 225, 227; cognitivist 14, 96, 215, 224 collision 96, 106, 149, 253, 352, 353, 356, 357; collisions 106, 355 communicating 84, 300, 312; communication 226, 281, 298, 354 communities 26, 27, 332; community 15, 26, 59, 206, 232, 275, 292, 293, 309, 319, 330, 363 complexities 4; complexity 5, 66, 136, 167, 169, 202, 251, 350, 351, 359, 366 component 8, 42, 96, 129, 169, 170, 187, 304; components 48, 56, 58, 61, 66, 129, 165, 179, 209,

378

Index

248, 257, 275, 279, 331, 338, 349, 354, 371 computable 184; computation 6, 7, 183, 184, 250–252, 257, 365, 367; computational 6, 178, 184, 185, 210, 214, 250–252, 255, 353, 365, 367; computationally 369, 370; computational-theoretical 185; computations 168, 249, 251; compute 185, 279, 337, 368, 372; computer 184, 249–252, 328–332, 344; computers 250, 259, 329; computing 184, 330 conscious 111, 166, 238, 366; consciously 366, 370; consciousness 14, 22, 92, 93, 150, 166, 228 constrain 33, 59, 63, 111, 221, 222, 225, 227, 257, 274, 282; constrained 22, 105, 124, 130, 169, 187, 280; constraining 124, 275, 298, 337 constrains 34; constraint 22, 107, 170, 183–185, 187; constraints 2, 6, 7, 46, 63, 103, 106, 111, 123, 157, 162, 170, 178, 180, 183, 186–188, 199, 205–207, 259, 271, 275, 277, 280, 281, 284, 285, 352, 353, 356 constructionism 305, 306, 325; constructivist 192, 209, 246 context 7, 8, 14, 22, 63, 64, 95, 133, 155, 165, 166, 185, 187, 272, 275, 276, 281, 305, 307, 314–316, 318, 336, 351–353, 356, 358, 370, 371; contextconditioned 162, 175; contextdependent 44, 46, 65, 255, 355, 356; context-general 177; context-independent 169; contexts 4, 14, 15, 21, 34, 63, 65, 86, 89, 90, 96, 258, 275, 277, 282, 311, 348, 352, 354; context-sensitive 86, 210; context-specifc 177; contextual 6, 65, 233, 295, 319 coordinate 32, 33, 96–98, 131, 226, 284; coordinated 92, 145, 256, 279–281, 293, 354; coordination 163, 216, 222, 224, 226, 258, 265, 281, 284,

334, 338, 349, 350, 354; coordinative 92, 93 correlated 104; correlates 202; correlation 355; correlations 169 cortex 7, 43, 208, 237, 255, 256; cortical 255, 259 coupled 18, 106, 155, 165, 217, 286, 364; coupling 18, 20, 92, 121, 125, 154, 253, 274, 277, 282, 284, 349–353, 365; couplings 129, 283 cultural 3, 12, 13, 16, 21, 27, 31, 34, 46, 71, 299, 306, 310, 330; culture 13, 19, 182 decision 81, 163, 352–355; decisionmaking 138, 275, 277, 281, 283, 284; decisions 126, 282–284, 348 detect 6, 90, 95, 103, 113, 114, 147, 152, 166, 200, 222, 278, 351; detectable 73, 107, 200; detected 57, 87, 161, 166, 224, 279, 286, 341, 351, 352, 355; detecting 46, 48, 51, 52, 58–64, 73, 101, 114, 146, 166, 201, 220, 278; detection 5, 51, 65, 76, 97, 106, 163, 166, 200, 281, 350, 367, 368, 371 determinism 116, 141; deterministic 165 development 5, 31, 36–38, 66, 92, 113, 114, 121, 122, 126, 127, 129, 131, 135, 154, 164, 180, 250, 253, 255, 259, 274, 295, 297, 313, 316, 365; developmental 31, 36, 39, 52, 89, 133, 165, 181, 214, 291, 363, 372 dexterity 187, 260, 265; dexterous 6, 186 diferentiate 340; diferentiating 127; diferentiation 96, 97, 134 dimension 15, 220; dimensional 218, 220–222, 225; dimensionality 220, 221; dimensionless 6, 179, 180; dimensions 3, 13, 16, 20, 105, 128, 157, 175, 201, 221, 280, 316, 364 disabilities 7, 312; disability 7, 291–293, 297–307, 309, 311–319; disabled 291–294,

Index

298–306, 308–313, 315, 317–319; disabling 305; nondisabled 292, 294, 297, 305, 308 dispositional 28–30, 37, 92, 349; dispositions 29, 30, 36, 90–92, 95–98, 200, 298 dissipated 129, 130, 275; dissipates 103; dissipation 125 drivability 355; drive 7, 8, 133, 260, 317, 337; driving 66, 67, 337, 348, 349, 351, 352, 354, 355, 358 dual 204, 357; dualism 103, 169, 215, 225, 311, 328, 332; dualistic 224, 225; dualities 3; duality 18, 204 dynamic 32, 33, 37, 89, 102, 106, 113, 122, 129, 206, 208, 209, 243, 251, 259, 262, 275–277, 279, 282–284, 299, 310, 311, 355; dynamical 20, 22, 106, 169, 211, 229, 230, 275–277, 284, 309, 332, 334, 348–352, 358; dynamically 8, 169, 207, 208, 262, 272, 340, 352 dynamics 5–7, 20, 35, 70, 101, 106, 113, 121–123, 127, 128, 135, 163, 183, 187, 199, 207, 209, 216, 218, 220, 221, 227, 228, 231, 246, 251, 257, 259, 261, 274–278, 282–284, 299, 314, 316, 317, 319, 354 earth 36, 104, 109, 143 edge 156, 225; edges 104, 222, 354 efectivities 28, 276, 285, 349 efector 48, 109; efectors 56, 57, 111, 281, 340; end-efector 234 embedded 7, 14, 56, 63, 66, 99, 165, 169, 178, 251, 272, 275, 280, 292, 303, 316–318, 358; embeddedness 218 embodied 4, 6–8, 71, 74, 76, 79–81, 85, 89–99, 101, 102, 145, 162, 165, 168, 170, 178, 236, 242, 278, 280, 283, 292, 297, 303, 304, 316, 318, 358; embodiedembedded 313; embodiment 217, 251; embodiments 298, 303; embody 21, 180, 183, 349; embodying 186

379

emergence 7, 121–123, 159, 165, 271, 293, 319, 329, 336; emergent 48, 52, 61–63, 106, 275, 284, 330, 358 enabling 178, 205, 292, 301, 303 enactive 20, 80; enactivism 20; enactivist 3, 18 encounter 63, 64, 86, 104, 146, 153, 255, 300, 310, 358; encountered 56, 63, 64, 295, 333; encounters 5, 18, 51, 62, 63, 145–147, 149, 151–154, 157, 188, 299, 311, 317, 333 energy 6, 43, 46, 48, 50–52, 54, 56–58, 61–66, 89, 96, 102, 107, 109, 129, 130, 145, 146, 149, 150, 153, 157, 161, 162, 164, 166, 168, 170, 177, 200, 218, 238, 252–255, 259, 277–279, 328, 337 entropy 104, 350 environment 4, 5, 7, 8, 11–18, 20, 26–31, 44–46, 48–54, 56–59, 61–63, 71–73, 77–79, 82, 85, 87–93, 96–98, 101–103, 106, 107, 109, 110, 114, 121–123, 125–127, 129–131, 133, 134, 143–155, 157, 164, 165, 167, 170, 185, 199–205, 208, 210, 214, 215, 218, 219, 221–223, 225–227, 229, 233–239, 242, 249, 251–253, 255, 257, 258, 262, 271, 272, 274–285, 292–295, 297–303, 306, 309–313, 315–319, 328–330, 336–338, 340, 348–358, 363– 372; environment-agent 129; environmental 3, 4, 8, 13, 16, 19, 21, 58, 88, 121–124, 126, 127, 130, 134, 138, 145, 149, 157, 161, 162, 165–167, 199, 205, 216, 226, 227, 230, 249, 252, 258–260, 271, 274, 275, 278, 281, 282, 286, 293–295, 300, 303, 306–308, 311, 316–318, 334, 338, 348, 351, 352, 354 environments 7, 12, 21, 27, 56, 72, 90, 95, 127, 130, 167, 169, 186, 217, 218, 222, 249, 255, 258, 272, 274, 281–284, 298, 303, 305, 306, 309, 310, 317,

380

Index

327, 332–334, 336, 355, 364, 367, 370 epigenesis 165, 169 epigenetic 372 ergonomic 338; ergonomics 327 event 92, 96, 104, 106, 107, 109, 111–113, 115, 155, 180, 212, 218, 223, 226, 227, 254, 261, 273, 276, 360, 365, 366 events 1, 34, 77, 88, 91, 97, 102–114, 145, 147–149, 155, 157, 161, 162, 166, 168, 169, 178, 179, 182, 187, 201, 204, 213, 215– 218, 222, 223, 226, 240, 241, 258, 271, 276, 280 evolution 5, 12, 45, 145, 189, 224, 241, 255, 284, 332, 365; evolutionary 12, 182; evolutionary-designed 281; evolve 12, 109, 286, 299, 311; evolved 102, 235, 281, 298, 328, 336; evolves 106, 110, 352; evolving 5, 313, 317, 319, 354, 367 executive 163, 261 expectation 93, 97, 116, 147; expectations 90, 91, 95–98, 188, 304, 309 experience 1, 5, 12, 13, 18, 20, 27, 28, 32, 37, 44, 45, 50, 52, 54, 83, 84, 88–95, 97, 98, 101–107, 110–114, 127–129, 132, 134, 141, 153, 176, 177, 182, 190, 201–203, 267, 272, 276, 277, 292–294, 298–305, 308–313, 315–318, 320, 322, 328, 330, 341, 354, 368; experience-based 84; experienced 21, 29, 128, 130, 132, 135, 273, 276, 299, 304, 308–312, 317; experiences 3, 7, 20–22, 32, 54, 61, 64, 83, 89, 134, 150, 165, 188, 224, 258, 292, 298, 302, 307, 313, 316, 321; experiencing 4, 27, 104, 150, 176; experiential 4, 91, 276 expert 35, 40, 41, 156, 273, 275, 282, 283, 341, 350; expertise 14, 275, 327, 332; experts 34, 35, 128, 132, 155, 157, 253, 341, 350

exploration 130, 161, 162, 258, 261, 273, 279, 281, 339, 340, 352, 363, 365, 371; explorations 336; exploratory 2, 5, 64, 73, 77, 78, 146, 161–163, 170, 249, 258, 259, 340 eye 73, 76, 78, 148, 149, 163, 164, 167, 170, 175, 180, 186, 209, 257, 277, 281, 337, 353; eyes 55, 72, 75, 147, 178, 216, 222, 253, 278 fatigue 122, 125; fatigued 122, 236, 237 feminism 21; feminist 22, 307 fnger 237, 281, 315; fngers 75–77, 154; fngertips 315 fexibility 55, 57, 92, 95, 123, 162, 165, 256, 272, 286, 308, 334; fexible 144, 208, 209, 282, 283; fexibly 207, 354, 357 fow 32, 103, 104, 113, 118, 151, 152, 154, 155, 161, 167–170, 187, 202, 218–220, 222, 230, 279, 336; fows 158, 165, 166, 167, 186, 278, 279, 281 fuctuating 167, 367; fuctuation 164, 169, 174, 207; fuctuations 161–163, 166–170, 186, 188, 207, 209, 259, 350, 351 fuid 151, 186–188, 259, 281, 334, 351; fuidity 251; fuidly 186; fuids 334 foot 186, 295, 369 foraging 153, 162, 163, 218, 226 fractal 163–165, 167, 185, 187, 227; fractal-generated 191; fractality 163, 164, 208 geometric 45, 104, 259, 309; geometries 185, 187; geometry 42, 44, 45, 65, 107–109, 185, 187, 255, 334 gestures 154, 354; gesturing 77 goal 42, 73, 77, 78, 93, 124, 125, 130, 154, 162, 164, 166, 181–183, 226, 238, 241, 250, 282, 283, 292, 293, 297, 299–301, 303, 311, 314, 317, 319, 327, 334, 337, 339, 356, 364, 366, 369, 371, 372; goal-directed 102, 106, 107, 145, 178, 181, 217, 274, 281, 288, 298, 300, 303,

Index

310, 316; goal-directedness 58; goal-oriented 170; goals 35, 48, 58, 73, 77–79, 81, 86, 91–94, 97, 133, 134, 151, 162, 176, 181, 183, 203, 226, 227, 242, 251, 274, 280, 281, 291–293, 295, 301–304, 309, 311, 313, 315, 316, 318, 319, 328, 338, 365, 366, 369, 371, 372 grasp 76, 87, 93, 236, 238, 240, 242, 274, 314, 369; grasp-ability 240; graspability 77, 93, 181; grasp-able 239; graspable 85, 87, 181, 182, 357; grasping 46, 47, 59, 77, 93, 182, 205, 232, 236, 241, 255, 257, 258, 260, 274, 291 hammer 156, 239, 242, 337, 357; hammering 206, 364; hammerstone 157; hammerwith-ability 243; hammer-withable 243 hand 55, 182, 186, 207, 234, 237, 240, 241, 256, 260, 273, 281, 298, 300, 301, 306, 313, 357; handheld 167; hand-held 51, 52; hands 93, 129, 236, 260, 298, 301, 313, 339, 340; handwielding 261 haptic 166, 168, 260, 336, 341; haptically 260 hierarchical 182–184; hierarchically 182; hierarchies 163, 182; hierarchy 183, 184, 279 human 3, 6, 11, 13, 16, 18, 26–28, 34, 38, 39, 62, 63, 65, 66, 83, 91, 154, 163, 164, 224, 237, 250, 251, 254, 257, 258, 260, 297, 298, 303, 309, 310, 317, 327, 330–336, 339, 344–347, 351, 363; humanautonomy 348, 352, 354; human-computer 250, 327; human-machine 214, 327; human-robot 363; humans 6, 13–15, 17, 20, 21, 26, 27, 31, 65, 66, 73, 83, 199, 216, 223, 224, 233, 234, 237, 239–241, 244, 253, 256, 257, 271, 277, 294, 295, 299, 328, 331–335, 345, 365

381

impoverished 1, 136, 178, 179, 181, 277 indeterminacy 33; indeterminate 33, 36, 37, 108–110, 113 infant 52, 137, 154, 155; infants 15, 27, 31, 52, 65, 96, 127, 154, 155, 253, 257 information 5–8, 17, 39, 46, 48, 50–52, 55–64, 77, 78, 83, 85, 87–90, 93, 95–97, 101–103, 106–110, 113, 114, 126, 127, 129, 130, 133–135, 146–155, 161–163, 165, 166, 168, 172, 177, 182, 186, 199–203, 209, 216–222, 224, 225, 227, 237, 240, 241, 249, 251, 253–254, 258–262, 264, 266, 274, 275, 277–282, 286, 328–330, 335– 337, 339–341, 349–358, 361, 364, 365, 367; informational 79, 87, 90, 130, 134, 135, 152, 154, 214, 274, 275, 279–281; informationally 106; information-based 106, 155, 336; information-bearing 79; information-processing 277, 328; information-rich 282, 283; information-theoretic 183, 251; informative 51, 150, 157, 202, 258, 278; informatively 277; informed 222, 277; informing 35; informs 220, 355 innate 15, 86, 354, 363, 365 intelligence 93, 133, 165, 217, 250, 251, 259, 350; intelligent 50, 92, 93, 259, 283 intentionality 58, 182, 204, 257, 277, 281, 285, 300, 357; intentionally 277, 312, 368, 372 interpersonal 3, 13, 21, 154, 155, 319 invariant 5, 101, 146, 149, 151, 153, 155, 272, 279, 335, 336, 339, 341, 350–353, 355, 356, 358, 359; invariantly 5, 349; invariants 149, 150, 278, 279, 337, 350 knowledge 6, 43, 45, 89–91, 93, 95–97, 122, 147, 177, 201, 202, 208, 215, 232, 233, 240,

382

Index

248, 281, 289, 306, 314, 316, 328, 330, 341, 368 landscape 3, 7, 21, 24, 29, 31–33, 35, 37, 40, 187, 205, 275, 276, 284, 285, 295, 299, 358 language 2, 13, 86, 88, 97, 147, 182, 183, 204, 250, 254, 262, 348 law 104, 107–113, 179, 180, 288; lawbased 45, 61, 332; law-driven 187; lawful 5, 6, 46, 55, 57, 65, 67, 69, 89, 155–157, 179–181, 183, 185–188, 225, 349, 351; lawfully 46, 48, 50–52, 58–66, 89, 101, 103; lawfulness 5, 65, 176, 180, 186; laws 4, 5, 42, 58, 61, 65, 66, 68, 105, 107, 149, 180, 183, 184, 187, 259, 284, 331; non-lawful 46, 59, 60, 63 learning 5, 8, 15–17, 19, 52, 67, 90, 92, 95–97, 113, 121–123, 125, 127, 129–131, 133–135, 146, 253, 273, 295, 332, 339, 360, 363, 365, 367, 368, 371, 372 length 42, 53, 102, 112, 163, 167, 218, 235, 258, 260, 272, 334, 339, 340 light 54, 55, 73, 80, 96, 103–106, 108, 110, 146–150, 164, 166–168, 174, 178, 200, 201, 218, 222, 225, 254, 255, 277, 278, 293, 297, 299, 300, 304, 328, 342, 353, 354 limb 271, 298, 339; limbs 223, 253, 257, 301, 338, 339 linear 104, 107, 109, 258, 263 linguistic 32, 95, 254, 302; linguistically 182 local 33, 36, 37, 42, 43, 50, 53, 54, 113, 151, 177, 178, 191, 210, 220, 265, 282; locality 105, 177, 183, 185; localization 164, 165, 204; localize 165; localized 282; locally 62, 220. locomote 307, 364; locomoting 104, 127, 202; locomotion 44, 127, 145, 151, 153, 154, 202, 215, 253, 258, 295, 302, 307 locomotor 137, 152; locomotory 163; nonlocomotory 163 looming 208, 213, 253, 278

look 272, 370, 371; looking 11, 48, 50, 57, 58, 125, 145, 147, 357 macaque 237; macaques 163, 237, 255 machine 6, 8, 211, 250–253, 257, 259–261, 293, 300, 304, 305, 332; machinery 259; machines 7, 184, 188, 250, 251, 257–259, 261, 332 magnitude 48, 108, 128, 153, 352; magnitudes 110, 126 manifold 6, 104, 218, 220–222, 225, 227, 365; manifolds 6, 216, 220, 222, 227 manipulate 236, 242; manipulated 126, 234, 250, 251, 273, 274; manipulating 44, 78, 258, 330, 340; manipulation 79, 145, 152, 234, 243, 250, 252, 255, 348; manipulations 250 material 4, 14–16, 19, 26, 27, 32, 36, 37, 169, 292–295, 298, 301, 302, 317, 318; materialism 304, 305, 308; materiality 297, 299, 316; materials 28, 83, 86, 97, 338 mechanism 143, 185, 204, 206, 234, 241, 242, 245, 281, 356, 359; mechanisms 83, 87, 168, 185, 202, 261, 281, 311, 367, 371 memories 112, 113; memorized 277, 367; memory 82, 86, 96, 104, 112, 113, 147, 226, 250, 277, 328, 364, 368 mental 4, 11, 34, 44, 45, 50, 52, 54, 71–82, 93, 99, 147, 203, 225, 238, 249, 279, 280, 283, 284, 316, 328, 330, 332, 334, 336, 341, 342; mentality 76, 79; mentally 337 metabolic 59, 65, 102, 284; metabolism 257 metaphor 162, 250, 252, 276, 328–331, 335; metaphors 169, 215, 249–251, 331, 335 metaphysical 27, 30, 33; metaphysics 26, 53 mind 74, 75, 77, 80, 83, 90, 91, 95, 112, 148, 153, 180, 250, 251, 256, 328–330, 332; minds 83, 87, 330 molecular 178–180, 182, 187; molecules 49, 146, 187, 322

Index

monkey 226; monkeys 237, 255, 256, 260 motivate 2, 216, 226; motivated 131; motivating 217, 280; motivation 131, 135, 165, 167, 375; motivational 73, 74, 131, 132 motivations 59, 65, 75 multifractal 162–168, 185–187; multifractality 5, 162–164, 168, 186, 261 muscle 54, 56, 125, 129, 338; muscleand-bone 178; muscles 205, 256, 295, 338; muscular 50, 54, 243, 260, 261, 340 nativism 84, 86–88, 97, 98; nativist 83–86; nativists 84, 85 natural 5, 8, 12, 27, 31, 45, 49, 58, 61, 86, 90, 98, 106, 157, 164, 167, 175, 179, 255, 316, 351; nature 5, 14, 20, 22, 27, 61, 73, 74, 79, 86, 91, 92, 97, 98, 105, 106, 114, 122, 123, 132, 133, 135, 144, 145, 150, 159, 168, 169, 203, 217 navigate 21, 80, 89, 261, 370; navigating 1, 89, 182, 299; navigation 250, 258 nervous 2, 54, 66, 101–103, 107, 165, 168, 216, 217, 223, 237, 249, 251, 255, 258, 331, 338 nested 109, 111, 133, 182, 187, 189, 226, 227, 273, 279, 281, 283, 285, 286, 350, 354, 355, 359; nesting 141, 177, 180, 182–184 network 184, 209, 226, 237, 240, 241; networks 209, 251, 259 neural 2, 6, 7, 43, 44, 50, 79, 125, 145, 165, 168–170, 177, 179, 184, 199, 202, 204–209, 215, 216, 218–222, 225–227, 248, 251–253, 256, 263, 274; neurocognition 232, 234, 235; neurocognitive 6, 232, 233, 235, 238–240, 247; neuron 255, 367; neuronal 103, 162, 259, 261, 284; neurons 138, 209, 220, 221, 237, 238, 246, 250, 251, 254, 259, 367 Newton 104; Newtonian 42, 44, 45, 49, 51, 53

383

niche 3, 13, 16, 17, 19, 21, 22, 32, 45, 46, 48, 50, 56, 59, 66, 102, 103, 109, 235–237, 276, 313; niches 13, 14, 19, 43, 45, 46, 297, 314 noise 161, 162, 167, 255; noisy 165 nonlinear 7, 185; non-linear 111; nonlinearity 186 normative 15, 16, 24, 30, 40, 138, 277, 289, 294, 295, 297, 299, 304, 305, 308; normatively 26, 27; normativity 13–15, 18, 26, 180 optic 73, 149, 153, 155, 166, 218– 220, 222, 278, 279, 286, 336; optical 154, 172, 202, 222, 253, 264, 278, 279, 337, 341, 352, 353; optics 148, 149, 153, 154, 279 organism 15–19, 77, 85, 91–93, 95, 103, 106–113, 143, 162, 165, 167, 176–179, 181, 182, 185–187, 199, 203, 205, 214–216, 218, 219, 222, 225, 234, 238, 249, 251–253, 260, 261, 271, 280, 294, 340, 355, 365; organismal 92, 257, 260; organism-environment 13, 27, 85, 92, 97, 101–103, 106, 107, 109, 161, 186, 203, 215, 216, 218, 221–226, 283; organismic 143, 161, 274; organisms 1–4, 6, 14, 16, 17, 20, 28, 30, 38, 74, 89, 92–95, 101–106, 108– 114, 117, 146, 151, 161, 164, 169, 177–179, 186, 187, 199– 203, 205, 206, 209, 216, 217, 223–225, 249, 252, 253, 255, 257, 258, 261, 262, 272, 278, 294, 316, 331, 332, 364–366 orient 336; orientation 151, 154, 221, 273, 274; orientations 91, 96, 97 overestimate 128, 131, 132, 236; overestimation 124, 127, 128 pass-through-ability 220, 222 pass-through-able 215, 218–220, 222, 226, 227 pathological 20, 294, 305, 308, 312; pathology 305

384

Index

pattern 36, 43, 64, 93, 95, 97, 98, 123, 125, 130, 131, 163, 165–168, 172, 187, 259, 295, 308, 314, 350, 352, 354, 356, 364; patterned 96, 97, 278, 279; patterning 162, 167, 187, 259; patterns 4, 5, 17, 26, 33, 34, 36, 37, 46, 50, 51, 56, 62–64, 66, 73, 86, 88, 92, 95–98, 101, 122, 125, 127, 162–164, 166–169, 175, 176, 179, 187, 208, 218, 222, 225, 234, 237, 253, 254, 261, 274, 275, 278, 280, 282, 295, 308, 314, 348, 350–355 perturbation 205, 207; perturbations 126, 151, 205, 206, 281 phase 109, 110, 207, 208, 353; antiphase 351, 353 phenomenological 13, 20–22; phenomenologists 21, 22; phenomenology 3, 20, 22, 309, 310 phenotypic 206–209; phenotypical 30 physically 106, 236–238, 306 plant 164, 337; planted 36; plants 366 postural 133, 162–164, 186, 218, 259– 261, 274; posture 167, 170, 240, 257, 293; postures 91, 93, 96, 308; posture-specifc 274 psychophysical 5, 169; psychophysics 168 reach 5, 128, 154, 177, 238, 239; reachable 287, 369; reaching 46, 47, 53, 59, 117, 241, 255, 256, 272, 343, 350 reaference 159 realism 14, 203 reciprocities 301; reciprocity 32, 101, 274, 276, 301 regularities 59, 63, 65, 95, 155, 183; regularity 89, 157, 216 rehabilitation 2, 292, 304, 305, 307–309, 314 relational 3, 14, 16–20, 22, 28, 29, 37, 38, 44–46, 56, 65, 123, 222, 223, 272, 307, 352–355, 358, 359 representation 7, 79, 86, 96, 99, 235–238, 241, 251, 252, 328, 370; anti-representationalist 77; non-representational 75, 84, 85,

98, 327, 334; representational 73, 84, 86, 96–98, 169, 214, 274; representations 48, 73, 75, 77, 84–88, 91, 93, 95, 96, 202, 235, 237–239, 241, 242, 249, 262, 263, 284, 328, 332, 334, 365, 367, 369, 370 resonance 185, 261; resonate 304, 310; resonates 205, 215; resonating 216, 258 robot 281, 332, 338, 339, 358, 363, 368; robotic 2, 8, 337–339, 363, 369, 370; robotics 8, 348, 363, 365, 367–372; robots 8, 331, 332, 363, 367, 370, 372 rotate 93, 336; rotates 93, 129; rotating 130, 221, 337; rotation 93, 124, 129, 187, 337, 339; rotational 258 running 154, 204, 205, 275, 293 scafold 34, 297, 315; scafolding 15, 170; scafolds 295 scale 2–6, 19, 20, 46, 50, 51, 59, 65, 74, 76, 88, 103, 104, 107–109, 117, 131, 151, 161, 164, 165, 168, 169, 176, 179–187, 199, 203, 206–209, 215, 217, 218, 222, 223, 226, 256, 257, 259, 283, 317, 340, 370; scaled 68; scale-dependence 184; scale invariance 164 scale-dependent 6, 180–186, 188; scale-free 180, 207, 209; scale-independence 179; scaleindependent 179, 180, 184, 188; scale-invariant 6, 185, 187; scales 5, 19, 58, 61, 103, 128, 140, 145–147, 150, 151, 161, 162, 169–171, 176–179, 181–183, 185–189, 206–209, 218, 220, 222, 223, 226, 227, 257, 259–261, 272, 284, 286, 350, 355, 359, 365; scaling 163, 164, 272 sensations 54, 83, 88, 148, 150, 168, 202, 336 sense 15, 32, 56, 76, 86–89, 98, 218, 294, 301, 336; senses 2, 86, 136, 148, 254, 328, 336, 364 sensitive 34, 86, 113, 148, 298; insensitive 303, 310; sensitivity

Index

34, 35, 61, 64, 81, 165, 208, 209, 216, 278, 303 sensorimotor 208, 219, 223, 224, 258, 363–365, 367, 371; sensory 32, 73, 76, 86–89, 102, 103, 135, 168, 201, 202, 224, 249–251, 254, 255, 366–368 sequence 62, 92, 130, 252, 273, 286, 369, 371; sequences 42, 164, 304, 371, 372; sequencing 256 shape 51, 108, 187, 189, 199, 205, 220, 222, 237, 246, 262, 273, 274, 295; shapes 144, 168, 220, 285, 354, 368, 372 shoulder 53, 215, 219, 301, 336; shoulders 215, 260 simulate 250; simulated 245, 341, 349, 354; simulating 78, 238, 353; simulation 75, 78, 238, 239, 241, 340, 349; simulator 349 sit 33, 204, 251, 294, 302, 357; sitability 27; sit-ability 338; sitable 236; sitting 33, 53, 62–64, 104, 116, 168, 204, 236, 272, 293, 312, 349, 357, 361 skill 15, 72, 122, 123, 127, 129–131, 133, 145, 209, 273, 286, 292, 295, 297, 322; skilled 7, 15, 34, 41, 52, 100, 122, 271–273, 285, 294; skillful 85, 170, 258; skillfully 34, 89, 96; skills 14, 32, 87, 114, 127, 133, 145, 161, 199, 234, 241, 275, 276, 285, 286, 292–298, 301, 304, 305, 316, 327, 339 smell 335; smelling 146, 147 social 3, 4, 11–13, 16–18, 21, 22, 26–28, 30–32, 34, 36–39, 46, 59, 66, 136, 154, 163, 164, 178, 186, 199, 208, 224, 277, 280, 282–284, 292, 294, 295, 297, 298, 303–307, 312, 315, 318, 334, 346 social-afective 5, 121, 122, 128, 131–133, 135, 136 social-cultural 297, 311, 316, 317; socialization 295, 297, 316; socialized 224, 294, 317; socializing 33; socially 26, 34, 154 societies 304, 315; society 59, 71, 82, 275, 301, 306

385

sociocultural 13–16, 19–22, 61–66, 181; socioculturally 13; sociology 13; sociomaterial 4, 15, 26–29, 31, 32, 36, 37; sociomateriality 27 soft-assembled 209; soft-assembly 208 space 4, 18, 19, 69, 103–105, 107, 108, 110, 111, 113, 124, 177, 218–221, 257, 309, 310, 313, 315, 318, 340, 368; spaces 2, 220, 302; spacetime 4, 104, 105, 107–109, 111 spatial 55, 56, 104, 105, 111, 163, 164, 169, 175, 178, 182, 222, 226, 272; spatial-temporal 317; spatiotemporal 101, 103, 161, 166, 170, 171, 217, 218, 220, 222, 223, 226, 278; spatiotemporally 106 species 12, 16, 17, 19, 21, 29, 30, 45, 46, 59, 66, 235, 256, 294, 332; species-typical 294 specifcation 27, 187, 253, 311; nonspecifying 127, 130, 134, 253; specifcations 149; specifying 113, 127, 130, 134, 135, 202, 209, 253, 254, 278, 337, 358 specifcity 86–89, 123, 170, 178, 187, 232, 253 spectra 165, 171; spectral 165; spectrum 165, 167, 276 spinal 223, 304, 305, 307 stabilities 37, 275; stability 130, 132, 163, 207, 209, 282, 302, 349; stabilizes 170; stabilizing 170 stair 235, 335; stairs 28, 85, 133, 235, 257, 293, 295, 311, 335 standing 104, 133, 204, 260, 291, 298, 300, 301 step-on-ability 180, 182, 186, 257; step-on-able 85; step-onto-able 333; stepping 46–48, 50, 52, 59, 117, 128, 271, 272; steps 87, 98 stimulate 254; stimulated 54, 277; stimulates 148; stimulating 114, 277; stimulation 20, 54, 55, 136, 148–150, 153, 168, 176, 179, 202, 255, 262, 277, 328 stimuli 1, 3, 17, 19, 30, 54–56, 74, 83, 91, 92, 103, 135, 148, 150, 167, 168, 178, 201–203,

386

Index

238, 255, 336, 353, 359, 366, 367; stimulus 1–3, 17–19, 32, 53–55, 74, 75, 91, 92, 102, 103, 115, 145, 148–150, 153, 164, 166–169, 182, 201–203, 224, 225, 238, 249, 253–255, 277, 316, 336, 353–355, 366, 367, 372; stimulus-response 92, 317, 355 structural 123, 201, 273 structure 17, 19, 33, 36, 41, 48, 51, 52, 57, 58, 61, 62, 65, 89, 96, 98, 101, 106, 107, 126, 146, 149, 157, 163, 164, 168, 219–222, 227, 252, 260, 275, 278, 279, 281, 305, 350, 351, 353, 354, 358; structured 8, 51, 52, 56, 58–60, 62–64, 73, 109, 255, 278, 310, 349, 354; structures 18, 26, 32, 36, 37, 79, 93, 185, 202, 226, 227, 232, 237, 240, 256, 282, 283, 291, 304, 306, 307, 347, 351, 352, 354, 356; structuring 50, 57 symbol 182, 183, 250, 252, 331, 372; symbolic 87, 88, 97, 183, 184, 186, 274, 369; symbols 79, 84–88, 181–183, 192, 250–252, 259, 261, 285, 354; symbolstring 182 symmetry 107, 308 synergies 6, 205, 207–209, 256, 282, 286, 337–339; synergy 206, 282, 284, 338 task 35, 88, 106, 122–125, 128–130, 132–135, 145, 153, 154, 156, 157, 162, 163, 166, 169, 170, 181, 185, 188, 204–208, 220, 221, 243, 253, 272–275, 277, 280, 282–284, 286, 289, 315, 318, 319, 337, 340, 341, 348, 350, 353–355, 358, 359; taskrelevant 328, 336, 340; tasks 3, 21, 22, 35, 125, 128, 129, 130, 134, 162, 163, 167, 170, 205, 206, 208, 227, 242, 250, 253, 281, 283–285, 318, 338, 370, 371 team 255, 275, 280–282, 284, 289, 290, 308, 354; teaming 348, 352; teammate 122, 125, 358; teammates 122, 282, 350, 354

technological 3, 348; technologies 241, 312, 320, 348; technology 313, 348, 350, 372 temporal 19–22, 36, 55, 56, 92, 103– 106, 114, 115, 155, 163, 169, 189, 222, 226, 227, 246, 256, 260, 272; temporality 20, 318 tensegrity 117, 141, 193 tension 171, 299–303 therapy 7, 291, 292, 303, 304, 306–309, 312–315, 319 threshold 156, 166, 169, 253; thresholds 166, 178 throw 72, 93, 123–125, 129–132; throwable 72; throwing 16, 72–76, 124, 125, 129, 130, 140, 154, 234 time 4, 19, 101–114, 125, 208, 218, 257, 280, 299, 340, 352 timescale 103, 297, 300, 316; timescales 39, 103, 122, 226, 227 timing 138, 295, 340, 352, 354, 355 tissue 54, 164, 169, 216, 223, 340, 341; tissues 2, 344 toddlerhood 138; toddlers 65, 126–128, 140, 159, 295 tool 6, 8, 53, 68, 141, 146, 147, 159, 170, 191, 194, 232, 234, 239–241, 243–248, 290, 335, 338–341, 343, 344, 350, 358, 359, 373, 374; tooling 234, 235; tools 133, 155, 208, 232, 234, 240–242, 303, 339, 345, 349, 350, 357; tool-use 234, 235, 240, 241, 243, 244 topography 164 topological 220, 221 topology 218, 220, 221 touch 69, 88, 90, 102, 140, 150, 178, 188, 191, 211, 230, 243, 260, 265 touching 126, 154 transform 153, 234, 272, 303 transformation 50, 149, 292, 303, 304 unpredictable 36, 113, 187, 350–352, 357, 358 value 15, 276, 367; value-judgment 308; value-neutral 32; values 23, 30, 170, 200, 225, 271 variability 124–126, 132, 135, 161–164, 166–170, 185, 205, 282, 348, 350, 351, 355, 356, 358

Index

variable 54, 89, 96, 132, 179, 258, 259, 274, 279; variables 87, 113, 130, 132, 134, 135, 149, 155– 157, 172, 220, 224, 237, 253, 254, 258, 274, 275, 280, 355 variants 149, 150, 279, 337, 350 vehicle 66, 347, 348, 350, 352, 353, 358 vehicles 350, 353 velocity 123, 126, 253, 279, 338 visible 110, 116, 153, 343 vision 7, 88, 126, 136, 146, 149, 153, 178, 201, 255, 278, 328, 335 visual 8, 52, 78, 88, 112, 126, 145, 148, 149, 152, 154, 155, 163, 164, 166–168, 178, 216, 235, 237, 253, 258, 260, 261, 279, 280, 336, 340, 349, 364–367

387

visually 115, 126, 127, 152, 158, 229, 278, 287, 346 visual-motor 273 visuomotor 364 walk 218, 238; walk-able 125; walking 126, 127, 135, 205, 238, 257, 293, 295, 298, 308, 311 waves 146, 166, 279, 281 weight 72, 218, 239, 336, 339, 340, 350 whole-body coordination 191, 265 width 53, 102, 215, 267, 336 wield 339, 340 wielding 167, 190, 258, 260; hand-wielding 261 words 84, 85, 88, 98, 153, 182, 302