Mind, Body, and Digital Brains 9783031583629, 9783031583636

Citation preview

Integrated Science 20

Flavia Santoianni Gianluca Giannini Alessandro Ciasullo Editors

Mind, Body, and Digital Brains

Integrated Science Volume 20 Editor-in-Chief Nima Rezaei, Tehran University of Medical Sciences, Tehran, Iran

The Integrated Science Series aims to publish the most relevant and novel research in all areas of Formal Sciences, Physical and Chemical Sciences, Biological Sciences, Medical Sciences, and Social Sciences. We are especially focused on the research involving the integration of two of more academic fields offering an innovative view, which is one of the main focuses of Universal Scientific Education and Research Network (USERN), science without borders. Integrated Science is committed to upholding the integrity of the scientific record and will follow the Committee on Publication Ethics (COPE) guidelines on how to deal with potential acts of misconduct and correcting the literature.

Flavia Santoianni · Gianluca Giannini · Alessandro Ciasullo Editors

Mind, Body, and Digital Brains

Editors Flavia Santoianni Department of Humanistic Studies University of Naples Federico II Naples, Italy

Gianluca Giannini Department of Humanistic Studies University of Naples Federico II Naples, Italy

Alessandro Ciasullo Department of Humanistic Studies University of Naples Federico II Naples, Italy

ISSN 2662-9461 ISSN 2662-947X (electronic) Integrated Science ISBN 978-3-031-58362-9 ISBN 978-3-031-58363-6 (eBook) https://doi.org/10.1007/978-3-031-58363-6 © The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland If disposing of this product, please recycle the paper.

Preface

An unprecedented systemic revolution in all fields of humans is being generated by the transformations introduced and induced by artificial intelligence and neurotechnology. These transformative tensions have a profound effect on the complex and adaptive nature of actual society. Many questions arise, concerning which opportunities and risks are associated with new scenarios, what idea of humanity is emerging from the increasingly widespread use of artificial intelligence technologies, and what idea of integrated science we should promote to accompany these ongoing transformations. This volume focuses on both theoretical and empirical issues and joins contributions from different disciplines, concepts, and sensibilities, bringing together scholars from fields that at first glance may appear different—neuroscience and cognitive neuroscience; robotics, informatics, human-computer interaction, artificial intelligence, and information processing systems; education, philosophy, law, psychobiology, and psychology. All these research fields are held together by the very object to be discussed: a broad, articulate, and polyphonic reflection on the status of theories and fields of application of digital technologies and artificial intelligence, seen from the perspective of the digital mind, digital body, and digital brain. Scientific and humanistic issues are considered through an interdisciplinary point of view, with the purpose of deepening emerging trends in various disciplines. In the introductory part Digital Humanism, the authors show how new scenarios have been raising from the ever-increasing interaction between sapiens, digital technologies, and artificial intelligence systems, in a loop that imposes a reconsideration of themes, issues, and interpretative categories related to the artificial era and the incoming digital humanism. What idea of humanity is emerging from the increasingly widespread use of artificial intelligence technologies? The fragility of society’s structure makes it susceptible to naive attitudes which often lead to polarizing and irrational views. On the other hand, technological advances can let us foresee multiple situations, which requires ethical considerations. Adaptive texture which substantiates the biological matrix of humans becomes even more significant when it leverages biomimetic approaches to education. Mind, body, and brain relation in the digital era is seen as the new digital humanism, v

vi

Preface

where an anthropocentric approach is reaffirmed in a future scenario in which humans and machines will be integrated. In the future scenario, generative artificial intelligence introduces a paradigm shift in the artificial intelligence field of research, whose emerging frontiers encompass quantum-inspired generative models, human and artificial intelligence collaboration, and ethical generative artificial intelligence. The focus revolves around adaptability, online learning, and meta-learning, which augment generative artificial intelligence real-world relevance and introduce the part Digital Mind that explores consciousness, self-identity, and self-regulated learning supported by technologies, examining the role of artificial intelligence and extended reality to understand these processes. Augmented reality is considered through learners’ beliefs to encourage personalized learning, whose features are implemented by adaptive instructional systems. Operational strategies for artificial intelligence integration and democratization in education encompass cognitive, biometric, physical, and spatial dimensions. Natural and artificial, human and machine, real and virtual interweave, expanding boundaries and providing new tools to improve intelligent adaptive systems. The interplay and overlapping in digital evolution between artificial intelligence, body, and brain, and the intertwining of digital brains’ activities are analyzed in the part Digital Body and Digital Brains, where digital brains—brains in relation with digital technologies—and brain activity are studied through brain-computer interfaces and non-invasive brain stimulation techniques. The brain structure is shaped by environmental enrichment, which can be implemented through digital technologies reproducing complex virtual environments. There is a close evolutionary link between brains and the practices of the living body: the dynamic landscape of systems designed for the learning of motor skills and sensor-based technologies contribute to integrate technology with human-centered elements. Digital brains live in digital worlds and digital societies. Will our moral agency be compromised by the ever more widespread diffusion of artificial agents and digital technologies? It is not a question of assuming optimistic or pessimistic postures, nor of supporting an instrumentalist, determinist, or constructivist view of technology, but rather of preparing and developing strategies to govern the processes underway and avoid becoming passive spectators of a transformation that is already driving us in the direction of an overall redefinition of the human being and its prerogatives. What governance tools should be promoted to prevent our collective forms of living from being compromised? What opportunities and risks are associated with the new revolution? The final part Digital Revolution raises several intriguing questions, as cyberwarfare, which emerges as a new and pivotal front in global conflict scenarios into the digital era. Artificial intelligence systems are demonstrating their full development in legal systems, expanding technical possibilities, and making possible new forms of conflict treatment. The widespread diffusion of artificial intelligence and the use of algorithms in decision-making processes related to rights and citizenship status introduce

Preface

vii

the theme of algorithmic citizenship, shaped by digital interactions mediated by algorithms. The public narrative about artificial intelligence concerns some critical points and involves implications of it which require philosophical aspects as human and artificial intelligence collaboration, societal impact assessment, and intergenerational responsibility. Education is indeed responsible for designing and leveraging meaningful strategies to cope with digital systemic revolution, to identify what change means and can mean. This volume offers a framework of different perspectives and, at the same time, a platform for discussion aimed not only at experts, but also at a non-specialist public interested in the topics. Naples, Italy

Flavia Santoianni Gianluca Giannini Alessandro Ciasullo

Contents

Part I Digital Humanism: Technologies, Artificial Intelligence, and Sapiens 1

Sapiens’ Future in the Age of Artificial World Picture . . . . . . . . . . . . . . Gianluca Giannini

3

2

Biomimetic Learning Design in the Artificial Era . . . . . . . . . . . . . . . . . . . Flavia Santoianni

17

3

Robots and Humans . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Bruno Siciliano and Daniela Passariello

29

4

Exploring the Current State and Future Potential of Generative Artificial Intelligence Using a Generative Artificial Intelligence . . . . Antonio Pescapè

37

Part II Digital Mind: Self-identity, Self-regulated Learning, and Intelligent Adaptive Systems 5

Supporting Learners’ Metacognition and Meta-Affect . . . . . . . . . . . . . Jessica White and Benedict du Boulay

6

Understanding Consciousness in the Age of AI and XR: Altered States, Emerging Realities, and the Digital Self . . . . . . . . . . . . . Nicola De Pisapia

81

Learner Modeling Interpretability and Explainability in Intelligent Adaptive Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Diego Zapata-Rivera and Burcu Arslan

95

7

59

8

Deep Learning in Educational Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 Alessandro Ciasullo

9

Augmented Reality in Higher Education an Exploratory Study on the Beliefs of Medical Students . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125 Massimo Marcuccio, Lucia Manzoli, Irene Neri, Laura Cercenelli, Giovanni Badiali, Maria Elena Tassinari, Gustavo Marfia, Emanuela Marcelli, and Stefano Ratti ix

x

Contents

Part III Digital Body and Digital Brains: Brain-Computer Interactions, Neuroergonomics, and Psychomotor Learning 10 Brain-Computer Interaction and Neuroergonomics . . . . . . . . . . . . . . . . 141 Fabien Lotte and Camille Jeunet-Kelway 11 Non-invasive Modulation of Brain Activity During Human-Machine Interactions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157 Stefania C. Ficarella 12 Environmental Enrichment in Real and Virtual Realms Fosters Neural Plasticity Related to Learning and Memory Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173 Giulia Torromino 13 AI-Powered Psychomotor Learning Through Basketball Practice: Opportunities and Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193 Miguel Portaz, Raúl Cabestrero, Pilar Quirós, and Olga C. Santos 14 The Intelligence of the Hands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217 Maria Teresa Catena Part IV Digital Revolution: Ethical, Philosophical, and Technological Cooperation 15 Cyber Warfare and Ethical Frontiers: Elevating Conflict to the Digital Frontline of Global Struggles . . . . . . . . . . . . . . . . . . . . . . . . . 231 Simon Pietro Romano 16 Cooperation, Law and Artificial Intelligence Technologies . . . . . . . . . . 253 Francesco Romeo 17 Algorithmic Citizenship: Fostering Democracy, Inclusion and Explainability in the Era of Artificial Intelligence . . . . . . . . . . . . . . 265 Pio Alfredo Di Tore, Fabrizio Schiavo, Monica Di Domenico, and Giuseppina Rita Mangione 18 Life-on-Life: Humanism Facing the Challenges of Artificial Intelligence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277 Mario Cosenza

Part I Digital Humanism: Technologies, Artificial Intelligence, and Sapiens

1

Sapiens’ Future in the Age of Artificial World Picture Gianluca Giannini

How can one control what cannot be controlled? Perhaps one should create “antagonistic” machines that would control one another (i.e., control the outcomes of their actions)? But what should we do if they present contradictory results at output? (Stanisław Lem, Doubts and Antinomies)

Abstract

AI, in its various forms and articulations, represents a range of new opportunities and strategies that our species can use to improve its condition. Often the scenarios presented in association with AI speak of a humanity threatened by machines or, conversely, of a utopian future in which man and machine will be fully hybridized harmoniously. In this article, I try to put into shape some of the key questions emerging in the field of Artificial Intelligence, avoiding sensationalist exaggerations on the one hand and, on the other, pointing out that the real challenge posed by intelligent machines lies in the new ways in which humans will be able to rethink and reinvent themselves and, therefore, their own possibilities for existence in what, even now, connotes itself as an Artificial World.

G. Giannini (B) Department of Humanistic Studies, University of Naples Federico II, Via Porta di Massa 1, 80133 Naples, Italy e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 F. Santoianni et al. (eds.), Mind, Body, and Digital Brains, Integrated Science 20, https://doi.org/10.1007/978-3-031-58363-6_1

3

4

G. Giannini

Joanna Joy Bryson recently noted that: For many decades, artificial intelligence (AI) has been a schizophrenic field pursuing two different goals: an improved understanding of computer science through the use of the psychological sciences; and an improved understanding of the psychological sciences through the use of computer science. Although apparently orthogonal, these goals have been seen as complementary since progress on one often informs or even advances the other [1]: 3. And still: Intelligence is the capacity to do the right thing at the right time. It is the ability to respond to the opportunities and challenges presented by a context. This simple definition is important because it demystifies intelligence, and through it AI. It clarifies both intelligence’s limits and our own social responsibilities in two ways. First, note that intelligence is a process, one that operates at a place and in a moment. It is a special case of computation, which is the physical transformation of information. Information is not an abstraction. It is physically manifested in energy (light or sound), or materials. Computation and intelligence are therefore also not abstractions. They require time, space, and energy. This is why—when you get down to it—no one is ever that smart [1]: 4. From here, from these premises, perhaps it’s time to pause and be cautious about the cheap use of all-encompassing labels such as Artificial Intelligence and begin to make two brief context considerations: 1. Our generations are experiencing a profound, in some ways radical, shift. It is a real revolution already entailing the reshaping of every aspect of our lives, opening great new perspectives but raising disturbing questions. 2. We are in a New Era, and all of us are critical players in a cultural, technological, value, political, legal, economic, and health transformation of society. We are living in a time in history in which culture and change appear to be genuinely synonymous and in which, for the first time in human history, technologies of our civilization are, perhaps, getting out of hand or, at any rate, totally transfiguring our anthropic horizons. The label that accompanies it now everywhere, Artificial Intelligence, is itself a kind of oxymoron: intelligence (something natural) conjugated to artificial (which is literally non-natural). It’s therefore propaedeutic and fundamental to ask what we mean by AI and, more importantly, based on this, what use I will make of it in the following pages. In fact, if we refer to the beginnings of making a machine act in ways that would be called intelligent if a human behaved similarly [2], we are already in a problematic recess. Artificial Intelligence is a counterfactual expression because it first has nothing to do with thought but with behavior. Suppose the human being behaved-it would be called intelligent. It does not mean that the machine is intelligent or that it is even thinking therefore... (the eventual judgment on intelligence pertains to action) that is: behavior as the way an individual acts, especially in certain situations, about the environment and with the people with whom he is in contact.

1 Sapiens’ Future in the Age of Artificial World Picture

5

We have yet to say a great deal relative to AI. Let us start with a minimal data set, and it is worth stating, still preliminarily, several things. Every new technology since the days of Galileo Galilei’s Telescope raises doubts, concerns, resistance, disbelief, and projections of all kinds. Often, instead of shaping and addressing the issues that might discern them, we have been (and still are) swayed by a kind of natural distrust, especially for changes that might appear too rapid. There is no doubt that, especially those that have been underway for more than a quarter of a century (but in fact, we could pre-date by a few decades even if we think of Bios-Technologies), technical and technological revolutions have complex and controversial implications, foreshadowing forthcoming, and not-too-distant scenarios of radical changes. However, one must be stressed: technical and technological revolutions are always related to the anthropic context in which they are determined and developed. They are not only the result of it but even interact with it profoundly and redetermine it because, in some capacity, they respond to a demand for redetermination. At this first level of consideration, it can be said that technical and technological revolutions/evolutions are never neutral, indifferent, and impartial. Why is there so much interest in these new technologies that fall in the groove of Artificial Intelligence? Indeed, they have the potential for interaction, transformation, and context redetermination that may be unique in Sapiens’ history. So, and again, what do we mean by Artificial Intelligence? According to Kate Crawford: Each way of defining artificial intelligence is doing work, setting a frame for how it will be understood, measured, valued, and governed. […] I argue that AI is neither artificial nor intelligent. Rather, artificial intelligence is both embodied and material, made from natural resources, fuel, human labor, infrastructures, logistics, histories, and classifications. AI systems are not autonomous, rational, or able to discern anything without extensive, computationally intensive training with large datasets or predefined rules and rewards. In fact, artificial intelligence as we know it depends entirely on a much wider set of political and social structures. […] Once we connect AI within these broader structures and social systems, we can escape the notion that artificial intelligence is a purely technical domain. At a fundamental level, AI is technical and social practices, institutions and infrastructures, politics and culture. Computational reason and embodied work are deeply interlinked: AI systems both reflect and produce social relations and understandings of the world [3]: 7, 8.

Precisely because all of this is to be kept in mind as well, my proposal, first and foremost, is that when we talk about Artificial Intelligence, we must assume not a specific technical-application datum and/or a separate discipline but rather identify a (moving) perimeter that holds it all together. It would be preferable to assume AI as a synthetic ‘locution’ account for all of this. In other words, when we use the term ‘Artificial Intelligence,’ we are talking about a very heterogeneous set of things that affects, by now, the most disparate areas of human doing and acting. We are primarily talking about a specific historical era (ours) that realizes a specific and articulated World Picture. And when we talk about the World Picture, especially if we move within a reflection meant to be primarily philosophical, it is impossible not to start again from the meaning given by a philosopher like Martin Heidegger to World Picture.

6

G. Giannini

And this, as will be seen below, to disassociate and, therefore, also distance from several contemporary philosophical discourses in Digital Philosophy and Ethics. We shall see, including Heidegger and all those still inspired by him today. In the famous lecture delivered on June 9, 1938, in Freiburg, The Foundation of the Modern Picture of the World by Metaphysics, later published in Off the Beaten Track in 1950 under the title The Age of the World Picture, Heidegger is of a solar clarity unfolding, in a few passages, a very complex architecture and one on which he had been working immediately after the publication of Being and Time, that is, once he had ascertained the risks of anthropomorphism and the investigative limits inherent in that type of approach specifically to Seinsfrage pivoting on Da-Sein: One of the essential phenomena of the modern age is its science. A phenomenon of no less importance is machine technology. We must not, however, misinterpret that technology as the mere application of modern mathematical physical science to praxis. Machine technology is itself an autonomous transformation of praxis, a type of transformation wherein praxis first demands the employment of mathematical physical science. Machine technology remains up to now the most visible outgrowth of the essence of modern technology, which is identical with the essence of modem metaphysics [4]: 57.

Having posited the incipit by virtue of which the load-bearing manifestation of the Modern lies in the flowering of the sciences, somehow it becomes necessary to grasp its essential load-bearing factor. For Heidegger, the prerogative of modern science is that by reason of which investigation moves within a fundamental plan of nature, the reason why To set up an experiment means to represent or conceive the conditions under which a specific series of motions can be made susceptible of being followed in its necessary progression, i.e., of being controlled in advance by calculation [4]: 61.

Given this premise and specified that here World is saying about the totality of being, immediately it’s interesting to note what is the way in which the approach to taking place in the groove of scientific research since Heidegger makes it clear that research requires or instead demands that the World has been opened and that is, designed already in a certain way so that it—as research, as scientific research precisely—can confirm a posteriori this plan of nature, pre-arranged, preestablished, a priori. It is the plan of nature, its overall representation, and its picture, therefore, that makes a natural phenomenon visible: one sees, in fact, what is presented as visible. And that phenomenon is not seen randomly, but in how a priori that plan of nature is pre-disposed, which is configured here as a picture. Essentially, modern science expresses, in the most extreme form, the absorption of being in subjectivity. Thought based on such a representation understands, therefore, Being as something stable, unchanging, and ever-present, that is, as a fact, not as an event, as something that the subject itself can dispose of, thus ignoring its eventual and

1 Sapiens’ Future in the Age of Artificial World Picture

7

never-final character, its multiplicity and mutability. The thing stands as we (pre-) view it: the matter itself stands in the way it stands to us, before us. To “put oneself in the picture” about something means: to place the being itself before one just as things are with it, and, as so placed, to keep it permanently before one [4]: 67.

Therefore, the World Picture does not depict or place anything in a picture. It is the placing that man operates on the world; it is the placing of the world before man as the object of his rap-presentation. And this happens only because there is something like a vision of the world that requires, even claims for itself, something to be seen: a picture, precisely. The picture that thus rises to the figure of the domain of the calculating and pre-disposing logos; logos that therefore puts such a World-Picture before itself to exploit and dominate it. The peculiarity of the modern world is thus to inaugurate an unprecedented form of subjectivism: the World Picture derives from the activity of a subject who imagines, that is, who represents. This means that this specific genesis that of the modern subject imposes a reduction of entities to objects and, more generally, of the world to rap-presentation. In turn, this reduction cannot but accompany and imply, in a kind of seamless circularity, something like the permanent leavening of the modern subject as the dominus of the entity in its totality. Beings as a whole are now taken in such a way that a being is first and only in being insofar as it is set in place by representing-producing humanity. […] Man sets himself forth as the scene in which, henceforth, beings must set-themselves-before, present themselves— be, that is to say, in the picture. Man becomes the representative of beings in the sense of the objective [4]: 67–68, 69.

However, far from being an act of freedom, man’s self-positioning as subject (and the consequent reduction of entities to objects) is the essential destiny of historical humanity in its relating to and approaching a Being that it now perceives as absent. Undoubtedly, modern man produces his position and poses himself as subject, but this is done in the wake of the original forgetfulness of Being. The decision on the fate of modern man thus springs from the forgetting of Being and its truth. It is the renunciation of the possibility of discovering and thinking it in its inaugural-inaugural trait as an event. This peculiar interpretive device, amended by the now outdated (since it is in turn metaphysical) dispute concerning Being, has animated and continues to animate much philosophy of technology and philosophy of (alleged) science, as well as ethical strands that try to extricate themselves from the mortally ineffective grip of contemporary pragmatism, all engaged with the problematic of the multilevel understanding—and thus, epistemological, contemplative, interpretive and, precisely, ethical-normative—of the digital—or, better yet, of the Digital Era. Then, this is the scenario on which I would like to propose some brief considerations. Starting with a kind of direct question that in principle might sound as follows: in the Digital Era, absolute artificial world, does this subject-object

8

G. Giannini

dynamic work that sees Sapiens establish itself as the only syntagmatic actor however much, according to Heidegger, necessitated by the epochal mode of unveiling Being, if ever there is a Being? As stated of Seinsfrage itself, if it is true as much philosophy of technology and philosophy of (alleged) science, as well as ethical strands that try to extricate themselves from the deathly ineffective grip of contemporary pragmatism hold, namely, that the peculiarity of the modern and contemporary world has been and still is to focus on a form of extreme subjectivism so much so that the accurate picture of the world derives from the activity of this same imagining subject and which imposes a reduction of entities to objects and, more generally, of the world to representation, are we sure that we have identified the true interpretive key to access the specific of the Digital Era as an absolute artificial world? To be specific—is it useless to go around too much—of the AI Era? That is, of that artificium, which somehow constitutes the maximum fall point of mechanical technology as an autonomous transformation of praxis, such as to import the use of the mathematical science of nature, in the supreme assumption of computational thinking, binary language. A falling point that implies, to the extreme, that Only that which becomes object in. this way is—is considered to be in being. We first arrive at science as research when the Being of whatever is, is sought in such objectiveness. This objectifying of whatever is, is accomplished in a setting-before, a representing, that aims at bringing each particular being before it in such a way that man who calculates can be sure, and that means be certain, of that being [4]: 66.

It is quite clear that our generations are experiencing, in an extremely rapid manner, profound turning points that are not simply destined to... but are already totally reconfiguring not only patterns of life but, in a way, identity structures. The extraordinary advances in technology in recent times have made the relationship between the digital and the human increasingly imperative, and, in addition to the certainties these raise, there are not a few open questions about managing the transition to increasingly complex forms of interaction and integration between human and non-human. We live in a new era in which we are all protagonists, consciously or unconsciously, of a radical mutation in terms of culture, values, politics, economics, health etc. We live in a time in which Culture and Change now seem to have become synonymous and in which, in a completely unprecedented way, technologies seem to constitute irreversibly and, in the round, our new habitat, internal and external. Not simply within this framework, but as a driver so that this framework has been and is being determined there is no doubt that Artificial Intelligence is at the heart of these very transformations. Exposure to new forms of Artificial Intelligence is already changing at its roots the experience of individuals in reciprocity with increasingly technological objects and contexts. In the now almost present future, humans will in essence be subjected to a decisive pressure that will propel them toward their own ultimate transcendence. An overall transfiguration that enjoins as of now to think about what the future human condition will be.

1 Sapiens’ Future in the Age of Artificial World Picture

9

If indeed this is the horizon not simply speculative but theoretical-practical, it is quite clear that at least two contextual considerations are necessary. One is apparently more comprehensive and starts with a basic question: why so much interest in AI? Better yet: are we certain of this centrality of AI? The answer, even for the use of skeptics and the distracted, is there for all to see: the term-umbrella AI literally conceals a pluriverse. AI indeed has a potential for interaction, transformation, and context redetermination that may be, maybe, unique in Sapiens’ history. In fact, AI programs are already mutating and transforming broad fields of human endeavor, from the economy to the more mundane of everyday life. One thinks of voice assistants (increasingly prevalent in electronic devices) of ANNs (Artificial Neural Networks) or artificial neural networks that sift Big Data in search of patterns that will serve to predict trends, tastes, and desires; of the large-scale explosion of chatbots; and of the imminent invasion of the Metaverse. Indeed, perhaps it can be said, with a good deal of certainty, that the all-encompassing umbrella underlying the acronym AI, at present and in principle, concerns application forms of automatic intelligence limited to well-defined tasks in well-defined domains that proceed, however, in multiple directions to the point of investing the entire sphere of human action. But at least two contextual considerations were mentioned. The second pertains to the very terms in question. AI, Artificial Intelligence, is a kind of oxymoron. Intelligence is, that is, something natural conjugated to artificial, which is literally non-natural. What, then, is meant by AI? Already, if one refers to the Thuringian beginnings, namely the possibility of conceiving an artificium capable of acting in ways that would be called intelligent if a human behaved the same way, one finds oneself in a decidedly problematic recess. One runs into, in fact, a counterfactual expression. Moreover, it is clear that what is being aimed at has nothing to do with thought but, instead, with behavior and, therefore, with action. If the human being behaved ... it would be called intelligent: it does not mean that the machine is intelligent or even thinking. In this, a few definitions help narrow it down. It is possible to refer to two defining guidelines and then arrive at a third one that might act as a glue somehow. And so, drawing from the now official one proposed years ago by Marco Somalvico, a first theoretical-disciplinary definition, whereby AI is that discipline, belonging to computer science, that studies the theoretical foundations, methodologies and techniques that enable the design of digital systems (hardware) and systems of programs (software) capable of providing the electronic processor with performances that, to a common observer, would appear to be the exclusive domain of human intelligence [5]: 12,

hence, the key term for resolving the behavior is reassuring (?) performance. Then, secondly, a possibility of application definition, whereby “the goal is not to simulate human intelligence” but rather “to emulate human intelligence, since there is no a priori reason why certain performances of human intelligence cannot

10

G. Giannini

also be provided by a machine” [5]: 13. It is pretty clear that in the case of emulation, intelligent performance is achieved by using the machine’s mechanisms, different from those conceivable for humans, but precisely such as to provide functions/outcomes qualitatively equivalent and quantitatively superior to those of humans. Here, in the interweaving of reproduction-performance, emulationfunctions-results, the spillover of application order is played out since the goal of AI, as knowledge and discipline first and foremost, is to build intelligent entities capable-of . That is machines that can compute/compute how to act effectively and safely in various situations from a potentially unlimited experiential background— obviously, acting effectively and safely, directly and/or indirectly, having humans as the goal. However, here, it is possible to insert what could be considered a third definition—a philosophical definition of AI. If what we can circumscribe as the standard model of AI is about rational acting, in other words, the goal is to set up an entity, an intelligent agent subject that undertakes in every situation the best (the adjective is merely descriptive) of possible actions; we have already laid the groundwork for something unprecedented. That is, we have set our hand to a peculiar and different artificium that bases its singularity, somewhat emulative of that of man, in a kind of autonomous creativity supported by a constitutive engagement-experience-learning-doing paradigm typical of the living and, nevertheless, different from the living. In all respects, this peculiar artificium would be an otherness that, escaping past interpretative dynamics concerning the traditional canons of machine and automation, would stand out for man as a true interlocutor-other. And this should be made clear without giving in to the temptations of technophobes and/or, on the contrary, techno enthusiasts. First, one must start from an assumption unwelcome to most: if one decides in any way to understand the universe of AI, it is necessary to start from the observation of a kind of separation, even conceptual, between the historical and every time historicized concept of intelligence and that of the capacity of acting. In this sense, AI qualifies as a new capacity to act and not, as common sense would have it, as a mere reproduction of human intelligence. In this sense, the category of Absolute Other has been introduced. The capacity to act is indefectibly the prerogative of the artificial that is designed and implemented as the drop point of a set of computational techniques inspired by humans use their nervous systems and bodies to feel, learn, reason, and act. It is quite evident, and precisely through the use of computational models, that the artificium, starting from its own specific constitutive language-which, it may have initially had to do with a pre-written design of nature-and from its own material dissimilarity, will operate—and it is not by chance that I use this verb to emphasize once again that this is a matter related to ag˘ere and not to intelleg˘ere—in a manner quite different from humans. What is surprising is that with AI tools, delegation takes on completely new degrees, because a novel feature comes into play, one that other tools lack: functional autonomy. Functional autonomy refers to the inherent ability of a system to perform a task or perform a function without requiring constant human user intervention or supervision. [...] Thanks

1 Sapiens’ Future in the Age of Artificial World Picture

11

to the possibility of designing systems capable not only of processing structured representations of both the data and the purpose to be achieved, but also of modifying their operation according to analyses of the processes they perform, we can now count on tools capable of functioning on their own; freeing us from the inconvenience of devoting all our energy and attention to the task delegated to them [6]: 78, 79.

It is this that, in an entirely a-evaluative way, prompts one to say that the space of a correlational other opens up here. And even in an abstractly unprecedented way, such as to clothe an order of absoluteness that configures an apposition of an Other not among many, but a syntagmatic other actor at the top of the operational chain. The eventual appeal-which is then a game of hide-and-seek-to the distinction between Strong AI, which envisions an automaton conscious of its intelligence and thus capable of being able to act independently, and Weak AI, which instead envisions an automaton limited to a specific task for which it is designed with no possibility of autonomous expansion-and we know that nowadays because of the difficulty of building true Strong AIs, with total autonomy and adaptability, more credence has been given to the Weak approach, which accounts for most of the AI we deal with on a day-to-day basis-it is not decisive notably to this aspect highlighted above. So much so that probing is, it would be, another type of question: what expectation undergirded and underlies this other Sapiens enterprise that led to such an extreme form of delegation as to configure such a type of functional autonomy? One thing must be noted and emphasized. Otherwise, every discourse, already from its premises, falls into a common-sense chatter that one tries to ennoble with some Heidegger-like tutelary deity: technical and technological revolutions are always familiar to the anthropic context in which they determine and develop. Not only are they the result of it, even they interact with it profoundly, to the point of transfiguring and redetermining the context itself and, therefore, as always and as always, the human itself, which not only has never been something given once and for all but, on the contrary, has found its most proper persistent possibility in the constant transfiguration of itself. In fact, it can be said that technical and technological revolutions/evolutions are never neutral, indifferent, or impartial precisely because each time, they precipitate the true point of fall of the redetermined historical inflection within the persistent strategy of Sapiens. There is no doubt that the artificium that goes under the label umbrella of AI represents, in all respects, concerning the topography of this situational historical inflection of Sapiens’ persistent strategy, a third-level manipulation, i.e., a technology that succeeds, in an autonomous, functionally autonomous manner, in bringing into communication artifacts produced by Sapiens himself, without the latter having to be involved, to any extent, in their operation and, even, in their management and coordination. This is already bringing about a marginalization of Sapiens and its refluxing within a dimension of artificial agents that no longer have Sapiens himself as terminus ad quem, precisely in the literal sense, as the boundary, first and last, at which something is determined. After all, it should be clear to everyone, at least those from other regions of knowledge approach the question, that:

12

G. Giannini We call ourselves Homo sapiens—man the wise—because our intelligence is so important to us. For thousands of years, we have tried to understand how we think; that is, how a mere handful of matter can perceive, understand, predict, and manipulate a world far larger and more complicated than itself. The field of artificial intelligence, or AI, goes further still: it attempts not just to understand but also to build intelligent entities [7].

So, if the goal of AI is to build intelligent entities, that is, machines that can calculate how to act effectively and safely in a wide variety of situations, we must also take note that: AI currently encompasses a huge variety of subfields, ranging from general (learning and protection) to the specific, such as playing chess, proving mathematical theorems, writing poetry, driving a car on a crowded street, and diagnosing diseases. AI is relevant to any intellectual task; it is truly a universal field [7]: 1.

AI can be applied to every sphere of human thought; it is a universal field… Undoubtedly, the ethical macro-question underlying any kind of reflection (even and primarily philosophical) revolves precisely around this. That is, around the question: could such powerful, intelligent, and functionally autonomous entities, in turn, design machines even more intelligent than themselves and Sapiens, reasoning that Sapiens would find themselves sharing the planet, its old anthropic space such as One World, with a new (artificial) species that to such a degree would be dominant that it would redefine the margins of its World-Owner as the actual World-Artificium? Now, beyond more or less science-fiction retro-thoughts, the question is not trivial, so much so that numerous personalities from the worlds of science, technology and all-round research have expressed caution and championed an open letter, Research Priorities for Robust and Beneficial Artificial Intelligence: an Open Letter [8], in which, among other things, AI research is supported and strongly warned, however and contextually, about the non-negotiable need for AI to do what we ask it to do. That is, “because of the great potential of AI, it is important to research how to harness its benefits while avoiding potential pitfalls”. Essentially, and as relaunched in the EU, AI research and development must still be human-centric, such that AI’s power is put at the service of human progress [9]. Put this way, the problem seems to be solved, and after all, the ethical color patch seems to serve as a good palliative. The question, which, among other things, revolves around man’s place not so much only in the Loop with AI but on what is and will be man’s place in the World, here and now but also and especially tomorrow, even if it is the World understood as the totality of being, seems to me to present itself as epochal, to the point of deflating traditional approaches: both speculative (as precisely mentioned concerning Heidegger) and ethical. So much so that the human-centrism formula seems to solve everything but is absolutely empty. Let us start again from the problem that is raised repeatedly and which, precisely, the supposedly ethically absolute goal of human-centrism only solves in premise if it is filled with content. The question that, as such, is capital is to

1 Sapiens’ Future in the Age of Artificial World Picture

13

reach an agreement between our actual preferences and the goal posed to/by the machine, knowing full well at this point that Artificial Intelligence, in practice, does not behave ethically. It doesn’t even behave unethically. It really has no idea what ethics is. But we who observe its predictions can evaluate whether the results are aligned or conflicting with our ethical principles. [...] Can we let predictions be made that a human would judge to be wrong? Would that be ethical behavior? Or should we correct the data or the algorithms that make use of the statistical models generated from the data? The ethics of Artificial Intelligence lurks in the folds of statistical models [6]: 74, 75.

It’s clear: the problem facing us is not trivially means-ends but, more radically, the alignment of values. Therefore, the values and goals entrusted to the machine must be aligned with those of man. And this is, fundamentally, a philosophical question because Sapiens’ values and goals are at stake. Present and near venturous. The problem is that even before we fantasize about the potential of the artificium, we should be aware that the values are historical and not absolute and metaphysical. Where is the point, then? Finding a solution in the face of nonlinear issues is genuine dilemmas—a decision problem between two moral imperatives, neither of which is entirely preferable—specifically to the conjugation of practice and value. Concerning this, in addition to the encompassing epistemological, contemplative, and interpretive order put up by much philosophy of technique and philosophy of (alleged) science, even the approach of certain ethical strands that try to extricate themselves from the deadly ineffective grip of contemporary pragmatism, and therefore traditional solutions, literally no longer hold up, because they are outdated and utterly unequal to the scope of the challenge. On the one hand, the vertical approach—out of space-time since about five centuries, that is, since, as Heidegger himself ascertained, it becomes “Through this, whatever is comes to a stand as object and in that way alone receives the seal of Being. That the world becomes picture is the same event with the event of man’s becoming subiecturn in the midst of that which is” [4]: 69—refers to metaphysically grounded, thus prescriptive and unquestionable codes of values, in which value finds its legitimacy in an extra-physical instance (God, the world of ideas and so on), on the other hand, the horizontal one, seems to present aspects of greater effectiveness and efficiency only in appearance. Indeed, it does not aim to impose eternal values and is usually attentive to human needs considering historical conditions and transformations. This includes, for example, so-called Engineering Ethics, or the Ethics of New Technologies, and, in any case, any reflexive platform that accompanies a specification complement to the word ‘Ethics.’ This is the recess where the most severe and problematic debate is played out, as well as mentioned, and Bentham and Kant are the two reference champions. That is, consequentialism/utilitarianism is on the grounds that what is useful is that which results in the greatest happiness of the most significant number of people and tends to make ethics even an exact science on a par with mathematics. On the other hand, precisely, de-ontologism is regarding which moral act would be suitable for any person under circumstances similar to those in which a subject finds himself at the

14

G. Giannini

moment of performing it. If then all this is even undergirded by the formulations of the categorical imperative, namely, from operating in such a way that the maxim of the will can always hold at all times as the principle of universal legislation to acting in such a way as to treat humanity both in your person and in the person of every other person always as an end and never as a means, it seems that we have given content to the human-centrism mentioned earlier. However, even in the wake of what was said earlier about AI as an Absolute Other, there are several complications and difficulties that cause even the most generous speculative endeavors to overflow into the realm of triviality every time. Hence, first problem: no philosophy, or ethical theory, is assumable in all circumstances; at best we can speak of guiding criteria. Second problem: values are historical. Value is the activity of valuation that is always in a spatiotemporal location. It’s a tension to do-value, but value is what here and now is worth through the action from which and for which it actually is, worth in terms of the packaging/orientation of a specific form of concrete acting. What is value for us today may be disvalue tomorrow and, more importantly, disjointed from the action of its actual exercise, may rise to nothing more than rhetorical statements of intent. Third problem: ethics is always an ethics of the situation. Fourth problem (the one framed earlier): the AI is an Other, another Syntagmatic Actor. Therefore, if relative to the AI macro-question, one seeks a closed and even universal answer, as is quite evident, not only is the expectation placed in the question wrong but, also and above all, one derogates and fails to understand the situation in its absolute specificity and singularity. The question itself always remains open, and it could not be otherwise, even more so when at stake is, as in the case of AI, an Absolute Other. The closed and even universal answer, which is what is often sought in view of a pre-programming of the artificium, falls miserably in the presence of the imponderable that the Other is by its mere fact of being. It has been said that the expectation reposed in the question that is out of focus; in the end it all flows back only into a problem of an Ethical Instruction Manual, into a label ethics, which will always attempt to move, woefully inconclusively, on the variable arrangements of the two criteria mentioned earlier within the horizontal approach. That is: either ethics precedes, or we run for cover later in the terms of the cost-benefit device. Or, again: the computable cost-benefit device is even prodrome in such a way as to give ethics a syncretistic foundation between consequentialism/utilitarianism and de-ontologism. The issue, as we tried to hint already in the opening, is different and is played out on this side of good and evil: today, in the age of AI, man is under a decisive pressure that propels him toward his own, further (and perhaps final) transcendence. Because of the biotechnological and information technology revolutions, it’s required that man project himself beyond himself, toward something else. An overall transfiguration that enjoins a rethinking of what is the past, present and future human condition. The problem of choice, of ethical choice, if there is one, is afterwards, as always. Contrary to what is commonly thought, the question about AI is not a question about the specific form of unveiling of Being and therefore about what

1 Sapiens’ Future in the Age of Artificial World Picture

15

makes this historical juncture epochal. Nor is the question about AI not and simply a technical and technological question, easily filled by a first-hand acquisition of knowledge even by philosophy, which, among other things, delights in third- and fourth-hand arrangements. Nor, finally, is it the disguise behind an epistemological screen that is evidently a necessary and yet never sufficient condition for fully understanding the situation, this time-here in all its infinite conjunctural combinations. The question about AI is first and foremost a question about man. On the man who has put his hand to AI; on the man who enjoys and will enjoy it; and, above all, on the man of tomorrow who in the disorienting and disoriented terms of mere use risks rediscovering himself even used. As has been tried to be said from the very first lines, there is no doubt that Artificial Intelligence is one of the greatest promises of humanity; thanks to its developments, current and around the corner, we will probably be able to do things that would be unthinkable today: just think of quantum AI and therefore quantum computers. We will live better, and perhaps longer and happier lives. And yet it is also not possible not to grasp to the end what could be, but in nuce already are, the implications associated with this kind of technology that will reach and then greatly exceed the finesse and pliability of what we consider the best of human traits. Talking about AI, under these conditions and according to these coordinates, implies talking about technology and philosophy, about machines and humans, about natural and artificial, in entirely new terms. It means increasingly developing the ability to know the human being with his needs and developing the ability to integrate with technological innovation without getting lost in it. And in this the co-participation in a unified project of philosophers and scientists is decisive. We must get our hands on a flexible and multifunctional Lògos and structure a common language as a basis for comparison that comes out of both the jargons of scientific language and some of the complexities of philosophical language. This constitutes the heuristic scaffolding for the building of a New Humanism, a new condition of the human that restarts from a newfound scientific-philosophical intimacy that is now inescapable and vital precisely for the human to come. After all, we must start again from the awareness that, every time, our stories begin from an end. From the end of an idea of ourselves about ourselves that has accompanied us productively even for centuries: the Nietzschean vital metaphors, even in the terms of our self-narratives that re-insert us in our relationship with the outside world, even to make sense of it. Since we have taken to constructing and telling them, however, the theme has always been the desire to emancipate ourselves from ourselves to become something different and something else to continue being. As human beings we live in the ruins of an imagined splendor, never attained and never attainable. We have always had a much higher idea of our destiny. We are and continue to be, no more and no less than in the same way as other living things, starting with the connotations of our peculiar persistent strategy as a species. Certainly complex, very complex, and AI is a figure of that. But this is not that it makes us a special living being and/or one that enjoys a privileged ontological status.

16

G. Giannini

It simply tells of the fact that, in a seamless circularity, in making ourselves in relation to the entity, even the artificial entity created by us and which becomes an autonomous syntagmatic actor, we find ways to continue to be and, therefore, to redo ourselves, in the terms also of self-narrative hypotheses that constantly surpass themselves. And in the end, with respect to AI, with respect to the Age of Artificial World Picture, if the real question might be: is it worth it?, to assume in full that to some degree an answer has already been given. So much so that it would at most be a matter of redefining the question further and, just as an example, begin to consider that: the most notable thing about AI is not AI itself, let alone its “intelligence,” but its capacity for reshaping how we live, particularly through its ability to exacerbate certain human behaviors, turning them into undesirable or problematic tendencies [10]: 27.

But it is only the beginning. Just the beginning, especially if you understand, once and for all, that what is at stake here is not Hardware and/or Software, but Humansware. Humansware capital issue that we can thus condense into the following question: “how is AI redefining the key forms in which we express our humanity?” [10]: 2.

References 1. Bryson JJ (2020) The artificial intelligence of the ethics of artificial intelligence: an introductionary overview for law and regulation. In: Dubber MD, Pasquale F, Das S (eds) The Oxford handbook of ethics of AI. Ofxord University Press, New York, pp 3–25 2. Turing AM (1950) Computing machinery and intelligence. Mind LIX, pp 433–460 3. Crawford K (2021) Atlas of AI. Power, politics, and the planetary costs of artificial intelligence. Yale University Press, New Haven and London 4. Heidegger M (2002) The age of the world picture (1938). In: Young J, Haynes K (eds) Off the beaten track. Cambridge University Press, Cambridge 5. Somalvico M (1987) L’Intelligenza Artificiale. Rusconi, Milano 6. Quintarelli S (ed) (2020) Intelligenza Artificiale. Cos’è davvero, come funziona, che effetti avrà. Bollati Boringhieri, Torino 7. Russell SJ, Norvig P (2010) Artificial intelligence. A modern approach, Third edn. Pearson Education Inc, Upper Saddle River 8. https://futureoflife.org/open-letter/ai-open-letter/ 9. https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52018DC0237 10. Chamorro-Premuzic T (2023) I, human. AI, automation, and the quest to reclaim what makes us unique. Harvard Business Review Press, Boston

2

Biomimetic Learning Design in the Artificial Era Flavia Santoianni

HCI is a strange and wonderful field. Chignell et al. 2023

Abstract

The development of brain-based digital technology is being driven by technology innovation and neuroscience, with potential applications in education. The fields of educational neuroscience, neuro-education, and brain-based education have emerged to explore the role of the brain in teaching and learning. This collaboration between education and neuroscience has been further enhanced by the emergence of Critical Neuroscience, which examines the sociocultural and contextual aspects of scientific research. Technology is seen as a related discipline that can contribute to the exploration of brain-based learning. The collaboration between neuroscience and education is being reshaped by the emergence of bio-educational technology field of research. The integration of technology into education needs to be carefully regulated to ensure it is studentcentered and guided by educational strategies. The principles of biomimetic learning design, which include personal differentiation, adaptive modifiability, developmental discontinuity, interaction and integration, and implicit support, can be applied to digital and artificial learning environments. This approach aims to personalize and adapt learning experiences, support self-regulation, and integrate explicit and implicit learning.

F. Santoianni (B) Department of Humanistic Studies, University of Naples Federico II, Naples, Italy e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 F. Santoianni et al. (eds.), Mind, Body, and Digital Brains, Integrated Science 20, https://doi.org/10.1007/978-3-031-58363-6_2

17

18

1

F. Santoianni

Human Computer Interaction and Digital Generations

At the beginning of human computer interaction in the 1960s, human operators collaborated with computers to perform tasks, involving the concept of human augmentation and human machine symbiosis in a non-discretionary way in the task-oriented human–machine interface, focusing on guidelines, standards, and training. In the 1980s, after the introduction of personal computing—and later of personal devices—the spreading to the mass market has led to the concept of usability and user experience, and to technology-driven and design-oriented approaches, which have substituted the idea of efficiency. Humans became users with more discretion over computers, e.g. through social media and gaming; at the same time, the actual trend towards inclusive design and Human-Artificial Intelligence automation is getting back into the game human factors. Human augmentation nowadays means to distinguish diverse kind of augmentation, such as the diversity of levels of interaction emerging from the diversity of age, skills, and personal preferences. Inclusive, social, and collaborative aspects of computing are now considered [1]. Since 1950s, there has been a continuous transformation of computers’ interfaces from an initial starting as hardware, only usable by engineers, passing through interface as software designed by programmers in the 1960s and 1970s, up to 1990s, when interfaces were reconsidered as terminal for end users and human factors began to play again a role in the human computer interaction. Interfaces became more dialogic and gradually converged toward computer supported cooperative work in the 1980s and 1990s, first involving end users and later end users’ social groups [2]. The Digital Natives or Millennials and Generation Y—born between 1981 and 1996—have grown up with social media, while Generation Z or Zoomers— born between 1997 and 2012—have grown up as an already net generation and share with the previous generation the technological adaptation, and the skills to be interactive, team-oriented, and participatory in social contexts. The incoming Generation Alpha or Screenagers—born between the beginning of 2010 and the middle of 2020—are used to live with and “inside” devices’ screens of personal computers, tablets, and cell phones.1 Generation Z digital behavior has been studied to evaluate their psychological well-being. Results showed high levels of stress, increased anxiety, depression, and perceived loneliness [3]. From a cognitive point of view, heavy use of social media is leading to decreased attention, hyperactivity, and lack of empathy [4].

1 According to recent global statistics, nowadays the average person spends 2 h and 24 min on social media every day. In 2021, the daily average time spent by 16–23 users on social media was 3 h, while all age groups have a consume of 2 h and 25 min. This range overcomes the average time spent by social media users to eat, to drink, and to socialize [3]. In 2022, 95% of USA teenagers use smartphone, while only 23% in 2011. In UK, under 14 spend more time with their devices than in conversation with their families [4].

2 Biomimetic Learning Design in the Artificial Era

19

Their teen brain is still plastic and can be heavily influenced by passive or active use of technology.2 According to the technology adoption debate [5], if cognitive system is supported by external devices which can substitute it, digital generations’ development lies in the interaction with various devices, e.g. mobile devices, which may deeply influence its growth. On the other hand, brain-based technologies are now coming into the classroom [6] promoting cognitive enhancement. Over the past twenty years, cognitive enhancement has been the focus of an interdisciplinary debate concerning its individual, social, and ethical implications [7]. Cognitive enhancement refers to the increase of cognitive functions, such as attention, perception, learning, and memory, but also reasoning, planning, and problem solving. Even if discussed in its implication, the core meaning of cognitive enhancement is the improvement of mind capacities through internal or external processing systems.3 In cognitive enhancement debate, any artificial aid given to the support of the cognitive system may be seen as potentially problematic. Artificial Intelligence (AI) is a research field implemented in the digital age, which is quickly developing, and sometimes it may appear as an autonomous agent—not under control—and, for this reason, potentially problematic. Like rapidly evolving complex phenomena, it is still lacking an overall interpretative framework which could analyze its dynamics with the temporal distance needed to objectively look at historically relevant phenomena. Another phenomenon is converging with the contingency of relentless technology development of Artificial Intelligence, which is the emotion of nostalgia. Nostalgia is considered to promote resistance and skepticism of users toward future-directed technology, but some models interpret it as a dual property emotion, because it encourages social connectedness and, on this basis, it could also sustain technological innovation [8]. During phylogenetic evolution, humans—with personal and social connected intelligence, but natural intelligence—have often coped with environmental needs. Environment itself has represented a sort of alternative intelligence, just because nature is not always a controllable agent and, although it isn’t an intentional agent, it can behave as an autonomous agent, whose causality is not always predictable and may be indeed opposite to the adaptive attempts of humans. Humans adapt to nature, in a positive way, when it’s possible to manage and transform it; at the same time, when it becomes risky, disruptive, or devastating, when it gets out of hand, adaptation requires a greater effort.

2

Active or passive use of technology is supposed to be the shift between a positive use of technology or not. For instance, social media active users comment posts, share information, and participate to the virtual community. Passive users instead do not directly communicate or interact with others. Active or passive use of social media, if associated with social media intensity of use, can influence social life and well-being [3]. 3 Mind is aided by neurotechnological and pharmacological strategies or non-pharmacological ones as mnemonic rules or meditation [7].

20

F. Santoianni

Artificial Intelligence—in particular Artificial General Intelligence (AGI)—is nowadays considered a long-term risk, as it can be seen as a non-human mind which potential performances should be contained [9]. While nature is an intelligent autonomous agent which belongs as humans to the world environment, artificial intelligence is a human construction, as education. For this reason, as education again, the field of AI safety should be socially connected and prioritized for human adaptation and integration to this technological frontier. Human–computer integration involve moral and ethical issues which should be considered to encourage computational and educational approaches which leave to human agency the full comprehension and the understanding control of AI enabled devices [10]. From an educational point of view, is needed to co-create interpretative analysis paths which may contribute to let the emerging phenomenon of Artificial Intelligence—and more in general, the incoming brain-based digital technology be increasingly decodable, by deepening its possible effects on humans and its interactions and therefore to let it be more manageable in an adaptive way.

2

Biomimetic Learning and Critical Neuro-Education

Technology innovation is promoting brain-based digital technology according to neuroscience and applied to education. Emerging neurocomputation and biomimetic learning systems may play a role in future education of digital brains, in relation to their cognitive development and evolution, considering digital brain as a human which approach of technology adoption lies within the digital world. An educational research question can then be highlighted about how education should change accordingly. Since 1980s, the idea to intertwine research on brain, education, and neuroscience has been launched as a research challenge and quickly developed worldwide.4 The emerging disciplinary fields, educational neuroscience [14]— neuro-education [15] and brain-based education—were designed to deepen the role of the brain in the teaching and learning processes [16], discovering its potential in enhancing active knowledge building through the integrated model of brain-based learning, which studies holistic learners’ experiences within multimodal learning environments [17, 18]. Beyond this research aim, there is the educational objective

4

By the American Education Research Association (AERA). In the 1990s, during the Decade of the Brain, the Harvard Graduate School of Education (HGSE) in the USA introduced the research on Mind, Brain, and Behavior (MBB) in 1993, which was at the roots of the interdisciplinary research on Mind, Brain, and Education (MBE) started by Kurt Fischer in 2004 as a program of the newborn International Mind, Brain and Education Society (IMBES) [11]. At the same time, in France, Bruno della Chiesa was carrying on the project Learning Sciences and Brain Research for the Council on Educational Research and Innovation of the Organization for Economic Cooperation and Development in Paris; in Italy, Elisa Frauenfelder was introducing BioPedagogy following in the footsteps of the French Bio-Pédagogie, developed by Flavia Santoianni as Bio-Educational Sciences (BES) in Naples; while in Japan Hideaki Koizumi was working on the activities of the Baby Science Society in Tokyo [12, 13].

2 Biomimetic Learning Design in the Artificial Era

21

to train education engineers o neuro-educators [19] which are prepared to apply neuroscientific knowledge in the classroom practice. Even if this incoming collaboration—a two-way collaboration [11] rather than a one-way route between education and neuroscience or vice versa [20]—has been critically discussed, since the hypothesized bridge could be considered too far [21] or too much directly related to school learning [22], this relationship has been consolidated by the emergence of Critical Neuroscience. Critical Neuroscience is an emerging field of study which deepen how sociocultural and contextual aspects of science may influence its approach to experiments. Neuro-disciplines and neuro-cultures coexist in the idea of Critical Neuroscience as a reflexive scientific practice intertwined with history, philosophy, anthropology, and sociology. The study of the brain, even if objective, is the result of collaborative work of communities of scientists in a specific time and space [23]. Starting from the first age of interest in joining brain and education research in the 1980s [24, 25], the neologism neuro-education and its correlates emerged between the 1990s and 2000s to reflect on brain-based learning or brain-compatible learning according to the Critical Neuroscience manifesto [6]. The core idea of plasticity become a metaphor of adaptation, development, and change [26]. Cognitive neuroscience field was expanded to include social aspects, as in social cognitive neuroscience, and lifelong learning, studying adult patterns of neural activity, as in developmental neuroscience [27]. Anyway, linking education and neuroscience may be as to walk on a rough terrain, since neuroscience has a grounded realm of enquiry, while education is a social invention [27] which aim is often not to understand phenomena but to design approaches to cope with phenomena and its field of research always requires contextual application. A question is if knowledge and expertise from humanities may affect neuroscientific practice co-creating the social and cultural brain or instead giving non correct representations of its findings. Vice versa neuroscience has contributed to the paradigmatic revolution of re-considering information processing a large-scale parallel processing rather than a sequential process as suggested by computational research at a first stage. Misinterpretation of neuroscience is shaped as neuromyths, which have been opposed to neuro-facts because they are inaccurate brain-based translations from neuroscience to education [6]. Neuromyths are implicitly learned from cultures and contribute to spread misconceptions in science5 when it meets humanities and education, while a more

5

Examples of neuromyths [28, 29] are popular and unrealistic beliefs about neuroscience and education such as the sharp differentiation between the right and the left hemisphere of the brain, while there are numerous interhemispheric connections in brain functioning [30, 31]. Hemispheric specialization is not divided in centers or single locations, because brain develops complex networks [32]. Other neuromyths claim that only the 10% of the brain is available for use and that brain plasticity is active only during critical periods of development while learning is a lifelong process [30, 33]. From the point of view of neuromyths, neuroplasticity allows to believe that there is a narrow relationship between the listening of classical music and cognitive abilities development [6].

22

F. Santoianni

accurate translation from neuroscientific findings is needed [34] and a rigorous foundation for teaching and learning should rely on carefully grounded research on brain-based learning aspects [32]. According to this reflexive turn, which highlights the interplay of several disciplines rotating around neuroscience, and around its relationship with education, technology becomes one of the possible related disciplines which may concur to explore brain-based learning.

3

Technology Integration and Digital Resilience

Technology has deeply implemented neuroscience activities through in vivo studies. Information processing and behavioral organization became at first readable by brain imaging [32] and brain scanning, such as functional magnetic resonance imaging (fMRI) to localize specific regions of the brain [1] and then implemented in Artificial Intelligence and neurocomputational technologies of machine learning and deep learning. Brain-based technology (BBT) has been intended as a technology which takes into consideration human cognition through hardware, software, procedures, and workflows; a technology for data analysis and interpretation driven by cognitive criteria. BBT is also an entanglement of computational approaches inspired to the functioning of the brain and to biology as a brain and computer interface (BCI), a technology based on a direct influence between brain and external devices. While hard computing is binary, sequential, precise, and accurate, soft computing approaches are computational techniques based on Artificial Intelligence which can simulate brain activity. For instance, adaptive neural networks (ANNs) refer to the connections between artificial neurons belonging to different layers of a system through a behavior miming biological neural networks, recently evolved into deep learning, algorithms which train networks with multiple hidden layers. ANNs can be applied to pattern recognition and classification through matching and clustering, while fuzzy logic is used in complex contexts because it is sensitive to uncertainty and tolerant of imprecision and approximation. Fuzzy logic algorithms are indeed based on human cognition and do not allow only true or false binary choices but admit partial truth, expressed by any real number between 0 and 1 as considered variables. Genetic algorithms are influenced by biology and use instead natural selection to develop evolving sets of solutions for search problems and optimization problems in machine learning [35]. An educational research question is if AI biomimetic systems may reshape not only learning environments but also learners’ cognitive, emotional, and perceptual identities, since they grow up in brain-based technology grounded educational

Some studies highlight the role of neuromyth of learning styles in discouraging the use of all the possible learning modalities, and other studies hypothesize that the differences between male and female brain are related to biology rather than to cognitive psychology.

2 Biomimetic Learning Design in the Artificial Era

23

environments, developing as digital brains.6 The discussion revolves around the concepts of personal identity, individual autonomy, safety, authenticity of the experiences, and risks-benefits relation. From an educational point of view the main reason to speak about digital brains is the ubiquitous presence of innovative and continuously changing technology within learners’ lives, which may concur to the development and the evolution of cognitive functions and may contribute to influence and reshape learners’ identities. The last Generation Alpha or Screenagers are integrated with their own technology—personal computers, tablets, and cell phones—and, since biomimetic technologies are situated as a pervasive presence in learners’ everyday life, intriguing questions could arise. If Artificial Intelligence, cognitive computing systems, genetic algorithms of machine learning, and neural networks of deep learning underlie many online services and power search engines or social media platforms, neurotechnology could really be able to influence and modulate learners’ cognitive behaviors through ever changing brain-computer interfaces and quickly developing digital cognitive tools [26]. Human–computer interaction is nowadays seen as a human–computer integration, that is a symbiotic relationship in which humans and soft computing can be interpreted as a continuum and their emerging patterns of behavior are to be considered in a holistic way [1]. The HC-Integration approach (HInt) is based on the idea of mixed agency of human and machine/software agents which cooperate and interact at multiple levels, including societal levels [40]. The emergence of Artificial Intelligence in education (AIEd) has been considered as a social phenomenon which opens hybrid educational spaces where human and nonhuman agents may coexist [41]. The combination of human natural intelligence and increasing levels of automation of AI systems needs cognitive compatibility, which means that humans could always foresee systems’ behavior, because inputs and tasks are designed and performed by humans [1]. Artificial Intelligence can both substitute and complement human cognition, but human factors remain the core source of advantage [42] when they are combined and integrated with neurotechnology. Neurotechnology reshape educational spaces and extend human cognition through the translation of biological neural networks into artificial neural networks developing biomimetic learning systems which may become more and more effective in modeling learners’ behaviors. Nevertheless, the intertwining with digital technologies is not merely changing behaviors, but it could be reshaping brains, according to digital media saturation due to living constantly in a digital world.

6 Digital brain is a wide term including many interpretations within it, comprehending meanings shared by science and neuroscience, all including simulations of brain in different disciplines, ranging for instance from spatiotemporal atlases of early brain development of infants through high-resolution MRI [36], to neuronal network models to study mental disorders as brain disorders [37], or to test and evaluate image analysis methods through a digital brain phantom [38]. Intelligent assistants in the digital world maybe also considered as a digital brain [39].

24

F. Santoianni

Digital brains are continuously asked to cope with incoming stream of information rather than to reflect on their own actions [4, 43]. They are becoming used to change while surfing the net, to frequently interrupt cognitive activities, to be shallow in learning, and so on. Effects of digital media saturation could be longlasting, since the teen brain is developing, and it is constantly open to learning [44]. The use of digital technologies can be adaptive according to several variables, including learners’ age and personal characteristics intertwined with the type of media and its use [45]. A positive adaptation to a risk experience can be defined resilience, a dynamic process depending on general promotive factors, as the family context, and protective factors, as personal identities or developmental contexts which may soften the impact of negative experiences on individuals. Online resilience e.g. is a strategy of well-being that implies social support through communication and proactive behaviors, along with digital literacy, which allows to be related to the digital world in a critical way and in a creative manner [46]. Digital resilience is considered an adaptive outcome that involves digital literacy, digital skills, and critical thinking. Digital acculturation regulates the interactions between real and digital worlds, mediated by digital competencies, which empower the awareness of risks and opportunities related to digital usability [47]. If applied to Artificial Intelligence and brain-based technologies, human factors make the difference in human computer interaction and can play a key role in supporting digital resilience development.

4

Bio-educational Technology and Biomimetic Learning Design

As well as neuroscience has addressed the educational rethinking of learning processes through educational neuroscience, neuro-education, and brain-based education, the same way neurotechnology is nowadays enhancing a reconsideration of human cognition and its education which may be differently shaped by brain-based technologies and Artificial Intelligence. The two-way collaboration between neuroscience and education is to be rethought through a new field of research, bio-educational technology. Nowadays technology is the medium, the shared factor which allows the development of both fields towards innovative challenges. Digital technologies inspired by neuroscience and brain-based neurocomputation are becoming biosocial technologies applied to education. Learning environments are re-designed as biosocial environments, which brain-like activities are computationally performed. The previous collaboration leveraged a reciprocal influence, in which neuroscience has gradually acquired a critical interpretation by absorbing sociocultural and contextual points of view, while education has seen the continuous integration of digital technologies inspired by neuroscience and brain-based neurocomputation into teaching and learning.

2 Biomimetic Learning Design in the Artificial Era

25

The interplay between neurotechnology and learners needs adaptation processes regulated by educational strategies. The relationship between technology and teaching and learning has been analyzed highlighting some key points which refer to innovative teaching as a student-centered approach, based both on guidance, individual experiences, and co-working, formal and informal, linked to sociocultural contextual backgrounds, with interactive, creativity oriented content, and user friendly resources [5]. Nevertheless, when actual learning copes with artificial intelligence it may be not always intertwined with teaching activities and so it can become a not supervised process, autonomous, and uncontrolled—from an educational point of view, a risky process. According to all disciplines connecting education and neuroscience—educational neuroscience, neuro-education, brain-based education, and so on—learning processes should be scaffolded by a bio-educational approach, which means to reuse biological models of processes and systems to reshape educational framework in relation to them, to regulate individual adaptation to technology. Human factors are involved in focusing on the core ideas which may influence the encounter between natural intelligence and artificial intelligence in an adaptive way, to manage the incoming change, and to maintain human identity safe through the imitation of nature in a biomimetic way. As neurotechnology has simulated brain functioning to implement technological brain-like networks, as well bio-educational technology has identified five principles of biomimetic learning design which can be applied to digital and artificial autonomous learning and unsupervised learning situations related to technology oriented environments [5, 16, 48]: 1. Principle of personal differentiation states that every kind of learning environment should be personalized, that is self-paced, tailored on individual cognitive differences, and customized according to personal interests because in nature each learner is a unique adaptive system, to be considered in a holistic way a cognitive, emotional, and organismic entanglement. Within technological intelligent learning environments, her/his activities should be based on individual discovery, personal experiences, and openness to generative ideas. 2. Principle of adaptive modifiability focuses on self-efficacy empowerment which in nature regulates the levels of disclosure and resistance of learners to experiences and may support their adaptation to contexts. In this interpretative framework, within technological intelligent learning environments, digital literacy and skills play the role of the acknowledgement of possible technological interactions and contribute to evaluate eventual risky situations. Interdependent relationships with formal and informal educational environments should be context-aware, gradually organized, and interactive, which means involving online and real-world tasks shaped by users. 3. Principle of developmental discontinuity recognizes that in nature every adaptive system is in a variable and discontinuous individual pathway. Each learner may give different feedback to a learning environment from time to time,

26

F. Santoianni

in relation to her/his real-life authentic previous experiences. In technological intelligent learning environments, learning pathways should be shattered in small chunks of micro-content, to allow learners to go forwards and backwards in a flexible way, retracing one’s steps or proceeding in a disorderly manner through time slots of micro-moments. 4. Principle of interaction and integration follows the personal history of learners along epigenesis in nature, during which cognitive systems are structured and organized in function of the interaction with internal and external inputs that are gradually self-regulated and synergistically integrated within the cognitive system to co-create personal identity. Self-regulation of cognitive strategies is an achievement reachable through the distinction between natural and acquired strategies and the acknowledgement of their dynamic ongoing balance. Technological intelligent learning environments should have intuitive layouts based on personal preferences. 5. Principle of implicit support highlights implicit levels which underlie explicit cognition and collaborate with it like a support on demand. Explicit and implicit levels may collaborate in cognitive development, because in nature implicit is constantly present in human perception and processing of any learning environment. If applied to technological intelligent learning environments, implicit can be intuitively expressed and represented by spatial dimension, i.e. through basic logic,7 a prototypical level of cognitive processing which supports its development [49]. Learning from nature in a biomimetic way, the relationship between learners and technological intelligent learning environments can be characterized by personalized and adaptive learning, discontinuous and self-regulated, explicit and implicit learning. Biomimetic learning design encounters Artificial Intelligence and neurotechnology co-constructing ubiquitous, formal and informal, situated and connected networks [51].

References 1. Chignell M, Wang L, Zare A, Li J (2023) The evolution of HCI and human factors: Integrating human and artificial intelligence. ACM Trans Comput-Hum Interact 30(2):17–30 2. Grudin J (1990) The computer reaches out: the historical continuity of interface design. In: Proceedings of the conference on human factors in computing systems. Association for Computing Machinery, pp 261–268 3. Roberts JA, David ME (2023) On the outside looking in: social media intensity, social connection, and user well-being: the moderating role of passive social media use. Can J Behav Sci 55(3):240–252

7

Basic Logic Theory hypothesizes six prototypical functions intermediary between implicit and explicit – integration (add), consequence (chain), individuation (each), comparison (compare), derivation ( focus), correlation (link) – underlying both perceptual and high-order levels of cognition [49, 50].

2 Biomimetic Learning Design in the Artificial Era

27

4. Fey M (2023) The digital brain: challenges for nurse educators. J Prof Nurs 48:8755–7223 5. Santoianni F (2021) Key aspects of mobile digital education. In: Santoianni F, Petrucco C, Ciasullo A, Agostini D (eds) Teaching and mobile learning. interactive educational design. CRC Press, Taylor & Francis, Boca Raton FL, pp 3–28 6. Wannyn W, Choudhury S (2022) Politics of plasticity: implications of the new science of the ‘“teen brain”’ for education. Cult Med Psychiatry 46:31–58 7. Hildt E (2013) Cognitive enhancement—a critical look at the recent debate. In: Hildt E, Franke AG (eds) Cognitive enhancement. An interdisciplinary perspective. Springer, Dordrecht, pp 1– 14 8. Dang J, Sedikides C, Wildschut T, Liu L (2023) More than a barrier: nostalgia inhibits, but also promotes, favorable responses to innovative technology. J Pers Soc Psychol. Advance online publication. https://doi.org/10.1037/pspa0000368 9. Wissing BG, Reinhard MA (2018) Individual differences in risk perception of artificial intelligence. Swiss J Psychol 77(4):149–157 10. Shneiderman B (2022) Human-centered AI. Oxford University Press, Oxford 11. Schwartz M (2015) Mind, brain, and education: a decade of evolution. Mind Brain Educ 9(2):64–71 12. Fischer KW (2009) Building a scientific groundwork for learning and teaching. Mind Brain Educ 3(1):165–169 13. Santoianni F (2019) Brain education cognition. La ricerca pedagogica italiana. RTH Res Trends Humanit 6:44–52 14. Aldrich R (2013) Neuroscience, education, and the evolution of the human brain. Hist Educ 42(3):396–410 15. Carew TJ, Magsamen SH (2010) Neuroscience and Education: an ideal partnership for producing evidence-based solutions to guide 21st century learning. Neuron 67(9):685–688 16. Santoianni F, Ciasullo A (2022) Milestones of bioeducational approach in mind, brain, and education research. In: Rezaei N (ed) Integrated education and learning. Springer, Cham ZG, pp 297–318 17. McBrien JL, Brandt RS (1997) The language of learning: a guide to education terms. Association for Supervision and Curriculum Development, Alexandria 18. Solomon M, Hendren RL (2003) A critical look at brain-based education. Middle Matters 12(1):1–3 19. Gardner H (2008) Quandaries for neuroeducators. Mind Brain Educ 2:165–169 20. Byrnes JP, Fox NA (1998) Minds, brains, and education: Part II. Responding to the commentaries. Educ Psychol Rev 10(4):431–439 21. Bruer JT (1997) Education and the brain: a bridge too far. Educ Res 26(8):4–16 22. Bowers JS (2016) The practical and principled problems with educational neuroscience. Psychol Rev 123:600–612 23. Choudhury S, Nagel SK, Slaby J (2009) Critical neuroscience: linking neuroscience and society through critical practice. Bio Societies 4(1):61–77 24. Cruickshank WM (1981) A new perspective in teacher education: the neuroeducator. J Learn Disabil 14(6):337–341 25. Fuller JK, Glendening JG (1985) The neuroeducator: professional of the future. Theory Pract 24(2):135–137 26. Williamson B, Pykett J, Nemorin S (2018) Biosocial spaces and neurocomputational governance: brain-based and brain-targeted technologies in education. Discourse: Stud Cult Polit Educ 39(2):258–275 27. Tolmie A (2015) Neuroscience of education. University of London, London 28. OECD (2002) Understanding the brain: towards a new learning science. OECD, Paris 29. OECD (2007) Understanding the brain: the birth of a learning science. OECD, Paris 30. Goswami U (2004) Neuroscience and Education. Br J Educ Psychol 74:1–14 31. Beauchamp C, Beauchamp MH (2013) Boundary as bridge: an analysis of the educational neuroscience literature from a boundary perspective. Educ Psychol Rev 25:47–67

28

F. Santoianni

32. Fischer KW, Goswami U, Geake J, The Task Force on the Future of Educational Neuroscience (2010) The future of educational neuroscience. Mind Brain Educ 4(2):68–80 33. Feilera JB, Stabio ME (2018) Three pillars of educational neuroscience from three decades of literature. Trends Neurosci Educ 13:17–25 34. Pincham HL, Matejko A, Obersteiner A, Killikelly C, Abrahao KP, Benavides-Varela S, Gabriel FC, Rato JR, Vuillier L (2014) Forging a new path for educational neuroscience: an international young-researcher perspective on combining neuroscience and educational practices. Trends Neurosci Educ 3:28–31 35. Dell’Aversana P (2017) Neurobiological background of exploration geosciences. Elsevier Science & Technology, United States 36. Nature (2023) Digital brain atlases reveal postnatal development to 2 years of age in human infants. Nat Methods 20:38–39 37. Peled A (2021) Reconceptualising the DSM: neuroanalysis and digital brain profiling. Eur Psychiatry 64(1):17–18 38. Aubert-Broche B, Evans AC, Collins L (2006) A new improved version of the realistic digital brain phantom. Neuroimage 32(1):138–145 39. Wongchoosuk C (ed) (2018) Intelligent system. InTech, London 40. Shneiderman B, Plaisant C, Cohen M, Jacobs S, Elmqvist N, Diakopoulos N (2016) Grand challenges for HCI researchers. Interactions 23(5):24–25 41. Williamson B (2017) Computing brains: learning algorithms and neurocomputation in the smart city. Inf Commun Soc 20(1):81–99 42. Krakowski S, Luger J, Raisch S (2023) Artificial intelligence and the changing sources of competitive advantage. Strateg Manag J 44:1425–1452 43. Turkle S (2016) Reclaiming conversation: the power of talk in a digital age. Penguin, Mexico 44. Alho K, Moisala M, Salmela-Aro K (2022) Effects of media multitasking and video gaming on cognitive functions and their neural bases in adolescents and young adults. Eur Psychol 27(2):131–140 45. Salmela-Aro K, Motti-Stefanidi F (2022) Digital revolution and youth. Consequences for their development and education. Eur Psychol 27(2):73–75 46. Vissenberg J, d’Haenens L, Livingstone S (2022) Digital literacy and online resilience as facilitators of young people’s well-being? A systematic review. Eur Psychol 27(2):76–85 47. Stavropoulos V, Motti-Stefanidi F, Griffiths MD (2022) Risks and opportunities for youth in the digital era: a cyber developmental approach to mental health. Eur Psychol 27(2):86–101 48. Santoianni F (2020) Brain based education. La ricerca bioeducativa sperimentale. RTH Res Trends Humanit 7:28–33 49. Santoianni F (2014) Modelli di studio. Apprendere con la teoria delle logiche elementari. Erickson, Trento 50. Santoianni F (2011) Educational models of knowledge prototypes development. Connecting text comprehension to spatial recognition in primary school. Mind Soc 10:103–129 51. Shengquan Y, Yu L (2021) An introduction to artificial intelligence in education. Springer Nature, Singapore

3

Robots and Humans

Bruno Siciliano and Daniela Passariello

Abstract

Previously separated from humans behind a fence, the new advanced robots (or cobots) are sharing our workspace and collaborating with us. The perception of robotics technology is improving, as we experience more ways it can positively affect our lives. In this scenario, the terms Artificial Intelligence (AI) and Robotics are liberally used, and frequently interchanged today. However, the physical nature of a robotic system distinguishes it from the pure abstraction of AI. We are experiencing a transition from Information Technology (IT) to InterAction Technology (IAT). The paradigm of convergence between physical entity and digital twin (phygital twin) will guide with 5G the development of new robotic applications making a natural transition from the Internet-of-Things (IoT) to the Internet-ofSkills (IoS). Armed with these skills, robots can be controlled dynamically in real time and be connected to people and machines locally and globally. In a future scenario in which humans and machines are called to coexist, the socialization process of robotics will conform to an anthropocentric approach, for the benefit of the community, towards a technological and digital humanism that can help us reaffirm the least artificial feature of our world: our humanity.

B. Siciliano (B) · D. Passariello Department of Electrical Engineering and Information Technology, University of Naples Federico II, Via Claudio 21, 80125 Napoli, Italy e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 F. Santoianni et al. (eds.), Mind, Body, and Digital Brains, Integrated Science 20, https://doi.org/10.1007/978-3-031-58363-6_3

29

30

B. Siciliano and D. Passariello

1 Information Technology The increasingly rapid diffusion of digital technologies in all human activities is the phenomenon that best characterizes the 21st century. The computer, born as a calculation tool, has become, thanks to the Internet, a communication tool. In 1982, Time magazine dedicated its cover to the computer for its “great influence in our daily lives” by awarding its subject of the year title to a “machine” instead of a person for the first time [1]. With that iconographic image, technological progress—even before the advent of the Internet—promised in some way to revolutionise the life of the individual and of the community, a prediction in fact realized and enhanced by the development of AI (Artificial Intelligence). With the convergence and integration between digital tools and data processing techniques, we have moved on to a new technological era, that of IT (Information Technology), defined as those technologies used in information processing, i.e. in processes of acquiring, processing, storing and sending information. The term IT can be incorporated within the term ICT (Information and Communication Technology) with which we indicate the fusion of information technologies with communication technologies: information is not processed by stand-alone machines—systems capable of functioning by itself or independently of other systems with which they may interact—but is processed in multiple locations and exchanged within the network. ICT defines the set of technologies that allow information to be processed and communicated through digital means. This area includes the study, design, development, creation, support and management of information and telecommunications systems, also with attention to software applications and the hardware components that host them. The ultimate goal of ICT is the manipulation of information data through the conversion, storage, protection, transmission and secure retrieval of information. ICT has come to shape society and the planet, becoming part of the challenges of the Anthropocene.

2 Automation and Robotics Digital technologies entered industrial design and production in the 1960s. These are numerically controlled machine tools, in which software guides the processing that the intelligent machine performs to obtain the required product. The goal of industrial automation is to create a new way of conceiving and developing production, integrating manufacturing with the management information system and other company functions, such as the adaptation of the product to customer requests. Robotics is a branch of industrial automation. The first robots have been widely used in industry since that decade. The first automatons dot an ancient history, from the Christian and Arab Middle Ages to the Italian Renaissance, with the ingenious designs of Leonardo da Vinci, up to the flourishing in the eighteenth century in Europe and Asia of creations such as Jaquet-Droz’s family of androids and karakuriningyo mechanical dolls. The playful dimension of the automatons was rooted in a

3 Robots and Humans

31

frontier technical knowledge for the time. In the industrial period, with the prevalence of the concept of utility of machines over any other function, the marvels of these automatons to be considered as the parents of modern robots become a memory of past eras [2]. The term robot, of Slavic origin and synonymous with subordinate work, comes ˇ from the pages of the Czech writer Karel Capek in the drama R.U.R. (Rossum’s Universal Robots) of 1920 to indicate an anthropomorphic machine designed and built with organic material to relieve the fatigues of humans. Twenty years later, we are in 1940, the image of the robot changes becoming a mechanical artifact with the Russian writer Isaac Asimov [2]. The main factors that have determined its diffusion in the manufacturing industry, especially the automotive industry, have been the reduction of production costs, the increase in productivity, the improvement of product quality standards and, last but not least, the possibility to eliminate harmful or repetitive tasks for the operator. Compared to the past, today the robot’s actions are no longer a pre-established sequence of movements, but they are performed automatically thanks to a control system that governs the motion in relation to what is happening in the environment. Hence, the commonly accepted definition since 1980 of Robotics as the “intelligent connection between perception and action” with a cognitive dimension, in relation to the possibility of decision and planning of the actions to be carried out; a sensory dimension, understood as knowledge of reality through the analysis of data; finally, an implementation dimension with the actions to be taken to achieve the desired goal. The action is offered by a mechanical system equipped with locomotion organs to move (wheels, tracks, mechanical legs) and/or manipulation organs to intervene on the objects present in the surrounding environment (mechanical arms, artificial hands, tools). Perception is entrusted to a sensory system capable of acquiring information on the mechanical system and on the environment (position sensors, cameras, force and tactile sensors). The intelligent connection is entrusted to a control system that governs the motion in relation to what happens in the environment, according to the same principle of feedback that regulates the functions of the human body [3].

3 InterAction Technology Digital is not merely a tool but a mediator between humans and their environment. It not only redesigns the latter—think of modern factories, and those of the future in which the massive presence of robots will completely redesign spaces—but it reinvents the way we work, study, and socialize. We are the protagonists of a technological revolution in which Robotics is destined to play a driving role for a new generation of autonomous devices which, through the ability to learn, will be able to collaborate with human and interact with the external environment. Hence the neologism IAT (InterAction Technology) introduced to explain how the convergence between Robotics and AI will project us towards a

32

B. Siciliano and D. Passariello

new generation of intelligent devices that will be the missing link between the digital world and the physical one [4]. In this sense, IAT represents the natural evolution of ICT and lays the foundations for a real symbiosis between human and machine thanks to an increasingly intuitive technology, which will make it possible to use robots with the same ease with which we use today common devices. If previously robots were confined for safety reasons to spaces far from humans, in the new generation factories they have effectively become cobots, which cooperate together with the worker in a safe and reliable way, or they are equipped with autonomy to move and work even in presence of uncertainty and variability of the environment. We will have a technology capable of revolutionizing not only the production approach, but also our daily life. Reduction of risks and work fatigue, improvement of the production processes of material goods and their sustainability, safety, efficiency and reduction of the environmental impact due to the transport of people and goods, physical assistance to the disabled or elderly, progress in diagnostic techniques and surgeries are all examples in which the new technologies of interaction can be a tool at the service of humans.

4 From Internet-of-Things to Internet-of-Skills 5G is expected to contribute to promoting the efficiency of robotic systems, which will pave the way for a new generation of robots controlled via wireless communication and at the same time equipped with new processing and data storage resources via the cloud. Robots operating in environments co-inhabited by humans in the near future will need to have the ability to interact, make decisions and react flexibly to unexpected events. To do this, a robot must be able to probe the environment and gain awareness of its surroundings. Improving sensory skills and processing of information from the outside is essential and this will be possible through the use of exteroceptive sensors such as distance, vision and contact sensors which will have to be increasingly sophisticated. Deep learning-based image and sound processing techniques have increased the need for high computational resources. Even if the computing capacity and the miniaturisation of processors has improved in recent years, one cannot think of housing all the necessary computing capacity in a robot. Being able to connect a robot via a wireless connection with one or more external computers represents an enormous wealth that can open the door to a new generation of robots with unprecedented characteristics of autonomy, safety and reliability. That said, in order for a robot to work correctly, the connection must allow for the transfer of a large number of data per second and that the time taken to transfer the data be sufficiently small and above all predictable, i.e. with constant latency. With 5G, robots will eventually be able to be controlled dynamically in real time and be connected with people and machines both locally and globally. It is therefore understandable how IoT (Internet-of-Things) can be overcome by IoS (Internet-ofSkills), a “tactile internet” to allow a remote physical experience through haptic

3 Robots and Humans

33

devices that combine with the skills, for example, of the drone operator or the surgeon dealing with an operation performed using a remote robotic system. Ultimately, the new paradigm of interpenetration between the emulation of the digital twin and the operation of physical reality (phygital twin) redesigns in an extraordinary way not only the industrial field but also promises to have an impact in other fields of application: agri-food, medical-health, urban mobility, hostile or poorly structured environments [5].

5 Roboethics What has been described belongs to a future dimension towards which the world of research is heading. But the promise of the pervasiveness of robots and intelligent machines in our society cannot fail to refer to a different range of ethical problems and moral dilemmas on which we all have to reflect, as scientists and as humans. What will be the social impact in terms of the labor market if the use of the progress of Robotics and AI further concentrates power and wealth in the hands of a few? In psychological terms, what will be the repercussions that could derive from human– robot relationships? How can we deal with addiction to such tools? In legal terms, we should consider whether robots deserve to be recognized as “people” and what the legal and moral implications of this choice are. There are many questions and they must be asked starting from the assumption that human is a “technical subject” and has always equipped himself/herself with tools of freedom and liberation with which he has been able to evolve. In the future Robotics and AI will be two tools with a high gradient of development, and therefore it goes without saying that a profound reflection must be opened on the constraints that will have to be placed on progress, so that technology does not become dangerous and alienating but, like politics and economy, take charge and always have in view the needs and the centrality of human in his becoming. The anthropisation of robots cannot ignore the resolution of those ethical, legal, social, economic (ELSE) problems which have so far slowed down their diffusion in our society. Italian robotics has been the cradle of reflections on these problems and in Italy the term Roboethics was born to indicate ethics applied to robotics, on the occasion of an international symposium in which philosophers, jurists, sociologists, economists together with the robotics community, laid the foundations of an ethics in the design, construction and use of robots. An opportunity that proved to be decisive for creating awareness of the need for such ethics and which in the future is essential for coping with the digital transition. Roboethics primarily deals with a series of issues related to the growing diffusion of robots in society, in particular the dual use of technology (virtuous or harmful), the impact on the labor market, on people’s psychology, on the environment, the digital divide between rich and poor regions of the world, the problem of dependence on technology, understood as personal dependence and social dependence. Roboethics

34

B. Siciliano and D. Passariello

is in this sense a human ethics that treats robots as machines and discusses their social role [6]. At a later level and in a more strictly human–robot interaction context, the concept of autonomy and responsibility for the actions of the robotic system comes into play. The growing ability of robots to perform autonomous actions and complex tasks raises issues of liability and acceptability in a wide range of applications. In some military, industrial and service fields, critical analyzes of these problems are mostly oriented towards the development of ethical policies that require substantial MHC (Meaningful Human Control) over autonomous robots whereby humans, and not machines and their algorithms, should ultimately maintain control, and therefore moral responsibility, of the relevant decisions that impact humans. More generally, we can say that developing the growing autonomy of robotic systems in harmony with the moral autonomy and assumption of responsibility of human beings is one of the great technological and, at the same time, ethical challenges of our time [7].

6 Towards a Technological Humanism The new generation of robots will actively cooperate with humans; beyond agility, necessary for effective manipulation, some strategic capabilities appear to be safety, reliability and aesthetics, necessary for effective human–machine interaction. Safety is of paramount importance so that the robot can react and learn from the stimuli coming from the environment it interacts with. The first and obvious concern with respect to robot safety is the possible harm from an unwanted collision between human and machine. Damage usually means physical injury or damage to human health, or even to property or the environment—where property includes the robot itself. The quantification of the damage, and therefore of the safety of a robotic system, passes through appropriate biomechanical analyzes on human–robot contact. Reliability is defined as the ability to perform a task in a manner that can legitimately be trusted. To prevent failures from being more frequent and serious than admissible, reliability proposes four tools: prevention, removal, error prediction and error tolerance. Regarding aesthetics, the appearance of a robot and its way of interacting with humans is of paramount importance. Until a few years ago, the asymmetry between the usually excellent performance of industrial robots and their ugly and disharmonious bodies, with rough manners and potentially dangerous movements for the human environment, was clearly evident. Now that robots are starting to be an integral part of our lives, they need to be aesthetically pleasing. A robot designed in collaboration with artists, designers and architects can be as harmonious and beautiful as a biological machine—through an electronic microscope even a gnat shows its extraordinary symmetry and beauty—and can be as harmonious and beautiful as a work of plastic art, which in turn may be as good as the nature that inspired it, if not more so [8]. In this scenario, the key role of design with respect to robotic technologies should

3 Robots and Humans

35

be clear in their becoming part of our daily life and in essentially modifying them in responsible and beneficial ways. It is designers who shape the interfaces between humans and machines and, as a result, help make robots as widespread as computers and smartphones. We are the protagonists of a technological revolution that brings with it an extraordinary connectivity between humans and machines from which derive new languages, ways of knowing, working and participating in collective life, and which can turn into a new push: that of affirming the least artificial characteristic of our world: our humanity [2].

References 1. Garante per la Protezione dei Dati Personali. Uomini e Macchine. Protezione Dati per un’Etica del Digitale. Atti del Convegno, Roma (2018). https://www.garanteprivacy.it/web/guest/home/ docweb/-/docweb-display/docweb/8987028 2. Siciliano B (2020) Robotica. In: Atlante Treccani. https://www.treccani.it/magazine/atlante/ cultura/Robotica.html 3. Siciliano B, Sciavicco L, Villani L, Oriolo G (2009) Robotics: modelling, planning and control, 2nd edn. Springer, Berlin, Heidelberg 4. Bicchi A, Siciliano B (2021) Robotics for interaction technology: Italy’s key role in the next revolution. In: Nature Italy. https://www.nature.com/articles/d43978-021-00124-4 5. Siciliano B (2022) Così cambierà il nostro modo di interagire con i robot. In: Innovation Post. Robot, 5G e l’Internet of Skills. https://www.innovationpost.it/tecnologie/robotica/dallinternet-of-things-all-internet-of-skills-come-cambiera-il-nostro-modo-di-interagire-con-irobot/ 6. Siciliano B, Khatib O (2016) Springer handbook of robotics, 2nd ed, Chap 80. Springer, Berlin, Heidelberg 7. Siciliano B, Tamburrini G (2018) Dai robot chirurgici ai robot pizzaioli (e ritorno). In: La Stampa, Origami. https://www.origamisettimanale.it/2018/05/16/speciali/dai-robot-chirurgiciai-robot-pizzaioli-e-ritorno-HarMsdQB7YCOC53n3p578H/pagina.html 8. Bonifati N, Siciliano B (2014) Dalla pizza alla cura dell’uomo. Le abilità di RoDyMan. In: Scienza & Filosofia. http://www.scienzaefilosofia.com/2018/03/19/dalla-pizza-alla-curadelluomo-le-abilita-di-rodyman/

4

Exploring the Current State and Future Potential of Generative Artificial Intelligence Using a Generative Artificial Intelligence Antonio Pescapè

Abstract

Generative Artificial Intelligence (GAI) represents a paradigm shift within artificial intelligence, evolving beyond discriminative tasks to autonomously create novel and coherent content. Models like Variational Autoencoders (VAEs) and Generative Adversarial Networks (GANs) exemplify this capability across domains. In natural language processing, GPT-3 excels in text generation, while StyleGAN revolutionizes visual content creation. GAI’s impact extends to healthcare, personalized content recommendation, and various creative domains. Large Language Models (LLMs), including ChatGPT 3.5 and 4, epitomize GAI’s transformative potential. LLMs crack language complexity, enabling autonomous language understanding and creativity. The difference lies in ChatGPT 4’s data-to-text focus and improved intelligence, albeit with prompt limitations. Transfer Learning and Domain Adaptation enhance GAI’s generalization. Pre-training on diverse data and fine-tuning for specific tasks address data scarcity, maintaining adaptability. Domain Adaptation minimizes performance degradation in novel contexts, crucial for real-world robustness. Interactive and Conditional Generation showcases GAI’s dynamism. ChatGPT 3.5’s conversational abilities and conditional GANs allow content synthesis with precise control. Challenges include mode collapse and conditioning information quality. Continual Learning ensures GAI adapts over time without catastrophic

Antonio Pescapè—Prompting ChatGPT3.5 and ChatGPT4. This paper represents my first attempt to study the ability to write an article discussing GAI using a GAI (ChatGPT) as a tool. A. Pescapè (B) Department of Electrical Engineering and Information Technologies, Via Claudio 21, 80127 Naples, Italy e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 F. Santoianni et al. (eds.), Mind, Body, and Digital Brains, Integrated Science 20, https://doi.org/10.1007/978-3-031-58363-6_4

37

38

A. Pescapè

forgetting. Techniques like replay mechanisms and memory augmentation mitigate challenges. Adaptability, online learning, and meta-learning augment GAI’s real-world relevance. Ethical and Responsible GAI combines technical measures with philosophical considerations. Bias mitigation, explainability, and privacy-preserving techniques address technical challenges. Philosophical aspects encompass value alignment, human-AI collaboration, fairness, and societal impact assessment. Emerging frontiers include Quantum-Inspired Generative Models, Human-AI Collaboration, and Ethical GAI. Technical emphasis includes bias mitigation, explainability, privacy preservation, robustness, and multimodal fusion. Philosophical emphasis involves value alignment, informed consent, fairness, societal impact, intergenerational responsibility, and digital ethics. In the intersection of GAI and intellectual property rights (IPRs), technical challenges include autonomous creation, data training sets, attribution, and fair use. Philosophical considerations frame AI as a tool, emphasize human oversight, balance innovation and IP protection, and recognize cultural impact. In conclusion, ethical GAI encompasses technical excellence and philosophical insights. Navigating GAI’s intersection with IPRs requires collaborative efforts for a framework that fosters innovation while upholding ethical and legal dimensions.

1

Method

This paper represents an experiment in which we aim at showing how the current state and future potential of generative artificial intelligence are described by a Generative Artificial Intelligence itself. For doing this we have prompted each specific section with the title of the section and with a request for providing an answer with more emphasis on technical details and then with much more emphasis on philosophical aspects and details. The first reply of ChatGPT to each request has been revised many times by asking to improve the answer. We didn’t edit the output because the idea was to underline the capability of the platform of talking about itself. Then, there are no section written by a human; the human just provided all prompts (approx. 250 prompts).

2

Introduction

In recent years, the landscape of artificial intelligence (AI) has witnessed a transformative shift with the emergence of Generative Artificial Intelligence (GAI). This paradigm shift represents a significant milestone in AI research and development, marking the transition from discriminative tasks to the autonomous creation of novel and coherent content. GAI encompasses a diverse array of advanced techniques and models, including Variational Autoencoders (VAEs), Generative Adversarial Networks (GANs), and large language models (LLMs) like GPT-3 and

4 Exploring the Current State and Future Potential of Generative Artificial …

39

StyleGAN, each pushing the boundaries of what is possible in content generation across various domains. At the core of GAI lies the ability to autonomously generate content that exhibits human-like creativity and coherence. This remarkable capability has unlocked new possibilities in fields such as natural language processing, computer vision, music composition, and creative arts, revolutionizing the way we interact with technology and consume digital content. From generating lifelike images and videos to composing music and crafting compelling narratives, GAI has transcended traditional boundaries, blurring the line between human and machine creativity. Transfer Learning and Domain Adaptation play pivotal roles in enhancing the generalization and adaptability of GAI models. By leveraging pre-trained representations and fine-tuning them for specific tasks or domains, GAI models can effectively address data scarcity and domain shift challenges, ensuring robust performance across diverse contexts. Techniques such as Unsupervised Domain Adaptation (UDA) and adaptive representations enable GAI models to learn from diverse data sources and adapt to changing environments, laying the foundation for real-world deployment and scalability. Interactive and Conditional Generation further exemplify the dynamism and versatility of GAI models. Through interactive dialogue systems and conditional generation techniques, GAI models can engage in meaningful conversations, understand contextual nuances, and synthesize content with precise control over attributes and styles. Whether it’s generating personalized recommendations, crafting tailored responses, or co-creating content with users in real-time, GAI models are reshaping the dynamics of human–computer interaction and collaboration. Continual Learning and Adaptability are essential principles that underpin the long-term evolution and sustainability of GAI models. By incorporating techniques such as continual learning, catastrophic forgetting mitigation, and meta-learning, GAI models can adapt and improve over time without losing previously acquired knowledge or skills. This adaptive capability is crucial for ensuring the relevance and performance of GAI models in dynamic and evolving environments, where new challenges and opportunities emerge incessantly. Ethical and Responsible GAI represents a critical imperative in the development and deployment of GAI technologies. As GAI models become increasingly capable and pervasive, it is essential to address ethical considerations such as bias mitigation, explainability, privacy preservation, and societal impact assessment. Philosophical inquiries into the nature of human-AI collaboration, value alignment, and digital ethics further underscore the importance of fostering a holistic and responsible approach to GAI development and governance. In conclusion, the current state and future potential of Generative Artificial Intelligence represent a remarkable journey of exploration, innovation, and ethical reflection. As we navigate the complexities and opportunities presented by GAI, it is imperative to strike a balance between technological advancement and ethical stewardship, ensuring that GAI serves as a force for positive transformation and human flourishing in the digital age.

40

3

A. Pescapè

Generative Artificial Intelligence (GAI)

Generative Artificial Intelligence (GAI) is a category of artificial intelligence that can create new text, images, video, audio, or code. It is capable of generating new outputs based on the data it has been trained on. Generative Artificial Intelligence constitutes a paradigm within the broader realm of artificial intelligence, wherein models are designed not merely for discriminative tasks but possess the inherent ability to generate novel and coherent content autonomously. At its core, GAI aims to simulate creative and decisionmaking processes akin to human cognition, thus transcending traditional AI models. Generative models, exemplified by architectures like Variational Autoencoders (VAEs) and Generative Adversarial Networks (GANs), exhibit the distinct capability to synthesize data, be it textual, visual, or multimodal, through a learned understanding of underlying patterns within training data. These models encapsulate probabilistic frameworks, allowing for the generation of diverse and contextually relevant outputs. The versatility of GAI finds expression across myriad domains. In natural language processing, models such as GPT-3 excel in generating coherent and contextually nuanced text, demonstrating prowess in language understanding and composition. In the visual domain, models like StyleGAN facilitate the creation of high-fidelity images with varying artistic styles, revolutionizing content creation and digital artistry. The impact of GAI on the ambient landscape is profound. The introduction of generative models has not only catalyzed breakthroughs in creative endeavors but has also extended its reach into domains such as healthcare, where synthetic data generation aids in privacypreserving research. Moreover, GAI’s influence resonates in personalized content recommendation systems, augmenting user experiences in digital platforms. Large Language Models (LLMs) are a type of generative AI and a type of foundation model. LLMs are notable for their ability to achieve general-purpose language understanding and generation. They learn statistical relationships from text documents during a computationally intensive self-supervised and semisupervised training process. The LLMs behind ChatGPT mark a significant turning point and milestone in artificial intelligence. Two things make LLMs game changing. First, they’ve cracked the code on language complexity. Now, for the first time, machines can learn language, context and intent and be independently generative and creative. Second, after being pre-trained on vast quantities of data (text, images or audio), these models can be adapted or fine-tuned for a wide range of tasks. This allows them to be reused or repurposed in many different ways. ChatGPT is an AI chatbot that uses a Generative Pre-trained Transformer (GPT), a type of large language model (LLM), to process user inputs and produce human-like text as responses in a conversational way. It can identify context and keep track of user prompts, which enables it to provide conversational experiences with a natural flow. The main difference between ChatGPT 3.5 and ChatGPT 4 is that while ChatGPT 3.5 is a text-to-text model, ChatGPT 4 is more of a data-to-text model.

4 Exploring the Current State and Future Potential of Generative Artificial …

41

ChatGPT 4 is more intelligent than ChatGPT 3.5, can handle longer prompts and conversations, and makes fewer factual mistakes. ChatGPT 3.5, on the other hand, generates responses more quickly and does not have the hourly prompt limitations that ChatGPT 4 has.

4

Advanced Concepts in GAI

Moving beyond foundational generative models, contemporary research delves into advanced concepts such as Transformers, leveraging attention mechanisms for improved context understanding. The integration of reinforcement learning principles introduces a dynamic element to generative models, enabling adaptive and goal-oriented content creation.

4.1

Transfer Learning and Domain Adaptation

In the pursuit of enhancing model generalization, researchers are actively engaged in exploring transfer learning techniques within the GAI paradigm. The ability to pre-train models on large, diverse datasets and fine-tune them for specific tasks or domains showcases promise in addressing data scarcity issues and adapting to novel contexts.

4.1.1 Transfer Learning Transfer Learning involves pre-training a model on a large dataset and subsequently fine-tuning it for a specific task or domain. In the GAI domain, this process is particularly impactful, given the scarcity of labeled data for certain tasks and the desire to leverage knowledge gained from diverse datasets. Pre-training Phase

During the pre-training phase, a generative model, often based on architectures like Transformers, is exposed to a vast and diverse dataset. This dataset could encompass a wide range of textual, visual, or multimodal information, enabling the model to learn general features, representations, and patterns. Popular pre-trained models in GAI include OpenAI’s GPT series for text and StyleGAN for images. These models become adept at capturing hierarchical structures, semantics, and stylistic elements present in the diverse training data. Fine-Tuning Phase

Once pre-trained, the model is fine-tuned on a target task or domain with a smaller, task-specific dataset. This facilitates the adaptation of the learned features to the nuances of the specific task, improving performance and generalization. Fine-tuning allows the model to specialize without starting from scratch, significantly reducing the amount of labeled task-specific data required. The depth of

42

A. Pescapè

the pre-trained model ensures a transfer of knowledge, enhancing its effectiveness in a new context.

4.1.2 Domain Adaptation Domain Adaptation addresses the challenge of model performance degradation when applied to a target domain different from the one it was trained on. In GAI, where models encounter diverse real-world scenarios, domain adaptation becomes crucial for maintaining robustness and efficacy. Source and Target Domains

The source domain refers to the domain on which the model is pre-trained, while the target domain is the novel context or dataset where the model is intended to perform. Discrepancy Minimization

Domain Adaptation often involves minimizing the distributional difference between the source and target domains. Discrepancy minimization methods, such as adversarial training, aim to align feature distributions, enabling the model to generalize effectively across domains. Adaptive Representations

Generative models trained with domain adaptation strategies learn adaptive representations that encapsulate domain-invariant features. This adaptation minimizes the impact of domain shift, ensuring the model’s performance remains robust even in previously unseen contexts. Unsupervised Domain Adaptation (UDA)

UDA methods are particularly relevant when labeled data in the target domain is scarce. By leveraging unsupervised learning principles, these methods enable the model to adapt to the target domain without relying on labeled target-domain data during adaptation. In the dynamic landscape of GAI, the judicious application of Transfer Learning and Domain Adaptation techniques contributes to the model’s adaptability, scalability, and efficiency in real-world scenarios, ensuring its continued relevance across diverse contexts. Researchers continue to refine and advance these methodologies, pushing the boundaries of what generative models can achieve in varying domains.

4.2

Interactive and Conditional Generation

Recent strides in GAI manifest in models capable of interactive and conditional generation. Techniques such as OpenAI’s ChatGPT 3.5 showcase conversational capabilities, while conditional GANs permit the synthesis of content based on specific constraints, enabling precise control over generated outputs.

4 Exploring the Current State and Future Potential of Generative Artificial …

43

4.2.1 Interactive Generation Interactive generation refers to the ability of generative models to engage in dynamic and responsive interactions, often in real-time, with users or other systems. This capability extends beyond static content creation and enables the model to actively participate in a back-and-forth exchange of information. Dialog Systems

GAI models designed for interactive generation often incorporate advanced dialog systems. These systems leverage architectures like Transformers, enabling the model to understand context, track conversation history, and generate coherent responses. Notable examples include OpenAI’s ChatGPT series. Contextual Understanding

The model’s ability to comprehend and respond contextually is crucial for interactive generation. Techniques like attention mechanisms enable the model to focus on relevant parts of the conversation, maintaining coherence and relevance in responses. User Intent Recognition

Advanced interactive models incorporate user intent recognition mechanisms. This involves identifying the user’s goals, requests, or queries within the ongoing conversation, allowing the model to generate responses that align with the user’s objectives. Real-Time Adaptation

Interactive generation models often possess real-time adaptation capabilities. As the conversation evolves, the model adjusts its understanding and response generation dynamically, ensuring that the generated content remains contextually accurate and coherent. Conditional Generation

Conditional generation involves generating content based on specific conditions or constraints imposed during the generation process. This introduces a higher degree of control over the output, allowing users or systems to dictate certain aspects of the generated content. Conditional GANs

Generative Adversarial Networks (GANs) are often employed for conditional generation tasks. Conditional GANs take additional input, referred to as conditioning information, which guides the generation process. For example, in image synthesis, the condition might be a textual description.

44

A. Pescapè

Embedding Conditions

Conditions can be embedded in various ways, such as through additional input layers, concatenation with the input data, or as part of the latent space representation. This ensures that the model incorporates the conditioning information while generating content. Controlled Attributes

Conditional generation allows for the control of specific attributes or features in the generated output. In text generation, conditions might include sentiment or style preferences, while in image generation, conditions could specify visual characteristics like color or style. Interactive Conditional Image Synthesis

In certain applications, such as design or creative arts, interactive conditional generation facilitates user-guided content creation. Users can provide conditions or constraints, influencing the generated output in a collaborative and creative manner.

4.2.2

Challenges and Considerations

Mode Collapse

Conditional generation models may face challenges like mode collapse, where the model focuses excessively on generating a narrow range of outputs. Techniques like diversity-promoting losses are employed to mitigate this issue. Conditioning Information Quality

The quality and informativeness of the conditioning information are crucial. Incomplete or ambiguous conditions can lead to undesired variations in generated content. Dynamic Adaptation in Interactive Scenarios

Ensuring seamless and dynamic adaptation in interactive scenarios requires robust mechanisms for context handling, user intent recognition, and real-time adjustments in the generation process. By advancing techniques in interactive and conditional generation, the GAI community aims to empower users with more nuanced control over the content creation process, fostering collaborative and user-centric applications. Ongoing research focuses on refining these capabilities for improved interactivity, adaptability, and creativity in generative models.

4.3

Continual Learning and Adaptability

Efforts to instill continual learning mechanisms within generative models are gaining traction. The ability of models to adapt over time, assimilating new knowledge without catastrophic forgetting, is a pursuit that aligns with the evolving nature of data and user preferences.

4 Exploring the Current State and Future Potential of Generative Artificial …

45

4.3.1 Continual Learning Continual Learning (CL) is a paradigm that focuses on enabling AI models to learn and adapt continuously over time, incorporating new information while retaining knowledge acquired from previous experiences. In the context of GAI, continual learning is crucial for keeping models relevant and effective in dynamic and evolving environments. Catastrophic Forgetting Mitigation

One of the primary challenges in continual learning is mitigating catastrophic forgetting, where the model tends to lose information about previously learned tasks when adapting to new ones. Techniques like replay mechanisms, which involve periodically revisiting and training on past data, help alleviate this issue. Regularization Techniques

Various regularization techniques are employed to encourage model parameters to adapt gradually rather than abruptly. Elastic Weight Consolidation (EWC) and Variational Continual Learning (VCL) are examples of methods that introduce penalty terms during training to preserve important parameters for previously learned tasks. Memory Augmentation

Memory-augmented networks or external memory modules are incorporated to enhance a model’s ability to retain information. These modules enable the model to store and retrieve relevant knowledge, facilitating continual learning without significant degradation in performance on earlier tasks.

4.3.2 Adaptability Adaptability in the context of GAI refers to a model’s capacity to adjust its behavior, representations, or parameters in response to changes in the environment, data distribution, or task requirements. Online Learning

Online learning strategies enable models to adapt to new data on-the-fly. This involves updating the model’s parameters incrementally as new data becomes available, ensuring that the model remains current and responsive to changes. Transfer Learning Across Modalities

GAI models are often required to generate content across different modalities (e.g., text, images, and sound). Adaptability in this context involves leveraging knowledge gained in one modality to enhance performance in another. Techniques like multimodal pre-training facilitate such cross-modal adaptability. Adaptation to Concept Drift

In real-world scenarios, data distributions can change over time, leading to concept drift. GAI models must adapt to such shifts to maintain performance. Techniques

46

A. Pescapè

like domain adaptation, which minimizes the distributional difference between the source and target domains, are applied to address this challenge. Meta-Learning

Meta-learning approaches aim to enable models to rapidly adapt to new tasks with minimal data. The model is trained on a variety of tasks, facilitating quick adaptation to new, unseen tasks during deployment. Model-agnostic Meta-Learning (MAML) is an example of a meta-learning framework.

4.3.3

Challenges and Considerations.

Task Interference

Continual learning may face challenges when tasks interfere with each other. Strategies to mitigate interference include architectural modifications, such as compartmentalization or modularity, that isolate knowledge related to different tasks. Dynamic Data Distribution

In continually evolving environments, the dynamic nature of data distribution poses challenges. Techniques to adapt to these changes, such as ensemble methods or adaptive learning rates, are essential for maintaining model effectiveness. Resource Allocation

Continual learning may require significant computational resources, especially when dealing with large-scale models. Efficient strategies for resource allocation and model compression are explored to make continual learning more scalable. The ongoing research in Continual Learning and Adaptability within GAI seeks to create models that not only learn from new experiences but also gracefully adapt to changing circumstances, ensuring their relevance and effectiveness in dynamic and evolving scenarios.

4.4 4.4.1

Some Applications of GAI

How is Generative AI Being Used in Creative Fields Such as Art and Music? GAI is revolutionizing several creative fields such as art and music. In music production, generative AI is used to create novel musical materials for advertisements or other creative purposes. Similarly, in art, generative AI is being used to create AI-generated artworks, opening up new avenues for creativity and self-expression. Moreover, AI algorithms can create new content or data that is similar to humangenerated examples in these fields. This has led to the creation of new content, including images, music, audio, and videos. AI systems learn from a large set of existing data in these fields and use that knowledge to generate new, original content that resembles the learned material. These advancements have significantly altered the jobs of content creators in creative fields. In fact, workers in creative

4 Exploring the Current State and Future Potential of Generative Artificial …

47

fields will need to become content editors as a result of generative AI. Content editing requires a different set of skills than content creation, which means that the workforce will need to adapt to these new changes. Despite the benefits of generative AI in creative fields, copyright infringement caused by the inclusion of copyrighted artwork in training data is an obstacle that needs to be overcome. This issue poses a significant challenge to those involved in the development of generative AI systems in these fields. Nonetheless, generative AI has the potential to create new content that revolutionizes these industries.

4.4.2

In What Ways is Generative AI Being Applied in Content Generation and Natural Language Processing? GAI is being applied in content generation and natural language processing, with the ability to learn patterns and relationships in a dataset of human-created content. This technique applies the learned patterns to generate new content, which has a variety of applications, including copywriting, product descriptions, social media posts, and even entire articles. One of the most significant advantages of generative AI is its ability to derive real, in-context value from vast stores of content, using various techniques for creating new documents from existing content. In copywriting, generative AI produces high-quality content at a faster rate and can be trained to imitate a specific writing style or brand voice. Natural language processing algorithms help to ensure that the generated copy is grammatically correct and coherent. GAN-based techniques can create high-quality copies of medical documents and archives that are too expensive to store in a high-resolution format, while super-resolution GANs can create high-resolution image renderings. Generative AI models are useful for creating voice assistants with human-like speech that can read out text or convert text into speech. Supervised learning is the most common method used to train generative AI models in content generation and natural language processing, where they are given a set of human-created content and corresponding labels to learn how to generate similar content with the same labels. Generative AI is revolutionizing how media companies help consumers find relevant content to satisfy their in-the-moment needs by providing highly personalized content and service recommendations through conversational interactions, including music, video, and blogs. Overall, generative AI is highly proficient at creating a wide range of artifacts quickly and at scale, augmenting workers’ ability to draft and edit text and other media, generating, translating, and verifying software code, summarizing, simplifying, and classifying content, and improving chatbot performance. 4.4.3

What are the Current Limitations and Challenges in the Use of Generative AI in Various Industries? The use of GAI in various industries faces current limitations and challenges, despite its promising future. Companies utilizing such models must be aware of their limitations and capabilities and avoid relying on them for tasks they cannot perform. While generative AI models have varying limitations and capabilities, managing expectations is crucial for companies employing them. Implementing

48

A. Pescapè

generative AI comes with major limitations such as the need for substantial computational resources and training data. Furthermore, ethical concerns regarding the potential misuse of generative AI, particularly in creating deepfakes or other forms of misinformation, should not be ignored. The technology is complex and challenging to implement, and companies must address issues of value accrual, profitability, and retention while using generative AI. The rapid growth of generative AI applications is not enough to build durable software companies; growth must be profitable, and users need to generate profits once they sign up and stick around for a long time. While the future of generative AI is promising, addressing these challenges and limitations is necessary for its successful integration into various industries.

4.5

Emerging Research Frontiers

As researchers, our collective pursuit involves exploring novel architectures, refining training methodologies, and navigating the uncharted waters of unsupervised and self-supervised learning paradigms. The interdisciplinary nature of GAI beckons collaboration across fields to unlock its full potential, with forays into multimodal generative models and the integration of domain-specific expertise.

4.5.1 Quantum-Inspired Generative Models At the intersection of quantum computing and GAI lies a nascent yet promising area of research. Quantum-inspired generative models harness quantum principles to potentially outperform classical counterparts, ushering in a new era of computational efficiency and expanded generative capabilities. 4.5.2 Human-AI Collaboration The frontier of Human-AI collaboration is a burgeoning area of exploration. Researchers are investigating models that complement human creativity, acting as collaborative tools rather than standalone entities. This collaborative paradigm envisions a synergy wherein AI augments human capabilities in creative endeavors, decision-making, and problem-solving. 4.5.3 Interdisciplinary Collaborations The interdisciplinary nature of GAI encourages collaborations with experts from fields such as philosophy, psychology, sociology, and the arts. Such partnerships enrich the contextual understanding of generative models, ensuring their alignment with human values and cultural sensitivities.

4 Exploring the Current State and Future Potential of Generative Artificial …

4.6

49

Ethical and Responsible GAI

The ethical dimensions of GAI persist as a focal point of research and discourse. Mitigating biases, ensuring fairness, and addressing issues of interpretability continue to be critical considerations. Researchers are actively engaged in developing techniques to audit and quantify biases within generative models, fostering responsible AI practices. Notwithstanding its strides, GAI is not devoid of limitations. Ethical concerns surrounding bias in training data and the potential propagation of societal biases merit meticulous consideration. The interpretability of generative models remains a challenge, necessitating concerted efforts to enhance explainability and transparency to foster trust in AI systems.

4.6.1

Technical Emphasis

Bias Mitigation

Technical measures include designing algorithms that actively identify and mitigate biases in training data. Adversarial training, fairness-aware loss functions, and techniques like re-weighting or re-sampling data points contribute to reducing bias in generative models. Explainability and Transparency

Developing interpretable models is crucial for understanding how and why a model makes specific decisions. Attention mechanisms, layer-wise relevance propagation, and saliency maps are technical methods used to enhance model explainability, fostering transparency and accountability. Privacy-Preserving Techniques

To address privacy concerns, techniques like federated learning, differential privacy, and homomorphic encryption are applied. These methods allow models to learn from decentralized data sources without compromising individual privacy. Robustness Against Adversarial Attacks

Generative models must be robust against adversarial attacks. Techniques such as adversarial training, input preprocessing, and incorporating robust optimization methods help mitigate vulnerabilities and enhance the model’s resilience. Multimodal Fusion Techniques

In applications involving multiple modalities, integrating information from different sources requires advanced fusion techniques. Methods like cross-modal attention mechanisms and joint embedding spaces enable generative models to effectively synthesize diverse content.

4.6.2

Philosophical Emphasis

Value Alignment

The philosophical foundation of ethical GAI often centers around aligning AI values with human values. Engaging in ethical AI design involves introspection on

50

A. Pescapè

societal values, cultural sensitivities, and the potential impact of AI decisions on diverse communities. Informed Consent and Autonomy

Philosophically, GAI should respect principles of informed consent and user autonomy. Users should be aware of AI interactions and have control over the generated content. Ethical considerations extend to empowering users with the ability to shape and guide AI-generated outcomes. Human-AI Collaboration

A philosophical perspective underscores the collaborative nature of GAI, where AI is viewed as a tool that complements human creativity rather than a substitute. Ethical GAI should enhance human capabilities, encourage creativity, and avoid the erosion of human agency. Fairness and Justice

Philosophical considerations emphasize the importance of fairness and justice. Ethical GAI should strive to mitigate biases and avoid perpetuating existing societal inequalities. Decision-making processes within AI models should be scrutinized to ensure equitable outcomes. Societal Impact Assessment

Ethical GAI involves ongoing societal impact assessments. Philosophically, researchers and developers should contemplate the broader implications of AI applications, including their potential to influence social structures, employment, and power dynamics. Intergenerational Responsibility

A philosophical perspective on ethical GAI involves considering the long-term impact on future generations. Responsible AI development should account for the potential consequences of current decisions on the well-being and autonomy of future societies. Digital Ethics and Cultural Considerations

Philosophical discussions extend to digital ethics, recognizing the cultural and ethical diversity across the globe. Responsible GAI should be sensitive to cultural nuances, avoiding the imposition of a singular set of values and ethical standards. Generative Artificial Intelligence (GAI) holds significant potential for addressing social and cultural issues, but it also presents challenges that must be carefully navigated. From a technical standpoint, GAI should strive to mitigate biases and avoid perpetuating existing societal inequalities. Decision-making processes within AI models should be scrutinized to ensure equitable outcomes. Additionally, ongoing societal impact assessments are essential to contemplate the broader implications of AI applications, including their potential to influence social structures, employment, and power dynamics. Philosophically, researchers and

4 Exploring the Current State and Future Potential of Generative Artificial …

51

developers should consider the long-term impact on future generations, recognize digital ethics, and be sensitive to cultural nuances, avoiding the imposition of a singular set of values and ethical standards. In summary, the responsible deployment of GAI involves technical measures to mitigate biases and ensure equitable outcomes, as well as philosophical considerations that encompass the broader societal impact and cultural sensitivity. By integrating these aspects, GAI can be utilized to promote diversity and inclusion in society, contributing to the creation of more representative and accessible content.

4.7

Addressing Social and Cultural Issues with Generative Artificial Intelligence (GAI)

4.7.1 Diverse Content Creation Media Representation: GAI can challenge stereotypes and promote representation by generating diverse and inclusive content in various forms, including images, videos, and written content. Narrative Creation: GAI can contribute to a more inclusive storytelling landscape by assisting in the creation of narratives that reflect diverse perspectives, experiences, and cultures. 4.7.2 Language and Communication Translation and Multilingual Support: GAI-powered language translation tools can foster cross-cultural understanding and communication across diverse linguistic communities, breaking down language barriers. Accessible Communication: GAI can aid in creating communication tools that cater to individuals with different abilities, ensuring accessibility for a wider range of people. 4.7.3 Bias Mitigation Fair Decision-Making: GAI models can be fine-tuned to reduce biases in decisionmaking processes, promoting fairness and equality in areas such as hiring, lending, and law enforcement. Algorithmic Transparency: Ensuring transparency in GAI algorithms helps identify and address biases, promoting accountability and fairness in their applications. 4.7.4 Education and Training Diverse Learning Resources: GAI can foster an inclusive learning environment by generating educational content that represents diverse perspectives, histories, and cultures. Training for Bias Recognition: GAI tools can be utilized to train individuals in recognizing and mitigating biases, promoting awareness and sensitivity.

52

A. Pescapè

4.7.5 Healthcare Equity Personalized Healthcare: GAI can contribute to personalized healthcare solutions, considering diverse genetic backgrounds, cultural preferences, and individual needs for more equitable healthcare delivery. Mental Health Support: GAI-powered virtual assistants can provide culturally sensitive mental health support, acknowledging and addressing diverse cultural contexts. 4.7.6 Inclusive Design and Accessibility Universal Design: GAI can assist in the development of products and services with universal design principles, ensuring accessibility for individuals with diverse abilities and needs. Inclusive Technology: GAI-powered technologies can be designed to cater to a broad range of users, considering factors such as age, gender, and cultural background. 4.7.7 Community Engagement Feedback and Co-creation: GAI can facilitate community engagement by collecting feedback and insights from diverse communities, ensuring that technologies are designed with their needs and values in mind. Virtual Communities: GAI can contribute to the creation of virtual spaces that promote inclusivity and collaboration among diverse groups of people. 4.7.8 Workplace Diversity and Inclusion Generative Artificial Intelligence (GAI) has the potential to promote diversity and inclusion in the workplace through various applications. Here are some examples. Recruitment and Hiring

GAI-powered tools can be used to analyze job descriptions and identify language biases that may discourage diverse candidates. By suggesting inclusive language, these tools help create job postings that attract a broader range of applicants. GAI algorithms can assist in resume screening, ensuring that biases are minimized in the initial stages of the recruitment process. This promotes a more diverse pool of candidates being considered for interviews. Interviewing and Assessment

Virtual interview platforms powered by GAI can be designed to minimize biases in the interview process. These platforms can analyze verbal and non-verbal cues to provide objective feedback, reducing the impact of unconscious biases. GAI tools can assess candidates’ skills through simulations and scenarios, focusing on objective performance metrics rather than subjective judgments, thus contributing to fairer evaluations.

4 Exploring the Current State and Future Potential of Generative Artificial …

53

Training and Development

GAI-driven training programs can be tailored to address unconscious biases and promote diversity and inclusion awareness among employees. These programs can adapt based on individual learning styles and preferences. Virtual reality (VR) simulations powered by GAI can provide immersive diversity training experiences, allowing employees to better understand perspectives different from their own. Inclusive Communication

GAI can assist in developing communication tools that cater to diverse linguistic abilities, ensuring that all employees can effectively engage in workplace discussions. Language translation tools powered by GAI can facilitate communication across multicultural teams, breaking down language barriers and fostering a more inclusive work environment. Performance Evaluation

GAI algorithms can be used to assess performance in a more objective manner, minimizing biases that might be present in subjective evaluations. This ensures that promotions and rewards are based on merit and not influenced by unconscious biases. Employee Well-Being

GAI-driven virtual assistants can provide mental health support, considering diverse cultural contexts and sensitivities. These assistants can offer resources and guidance to employees, promoting well-being across diverse demographics. Tools powered by GAI can analyze workplace data to identify potential sources of stress or bias, enabling organizations to proactively address issues that may affect employee inclusion and job satisfaction. Employee Resource Groups (ERGs)

GAI can be utilized to support ERGs by providing data-driven insights into the needs and preferences of diverse employee groups. This information can help organizations tailor initiatives that foster a more inclusive workplace culture. Leadership Development

GAI tools can assist in identifying potential biases in leadership decisions and recommend strategies for inclusive leadership development. This can help organizations cultivate a leadership team that reflects diversity and embraces inclusive practices. It’s important to note that the ethical deployment of GAI in the workplace is crucial to prevent reinforcing existing biases. Continuous monitoring, transparency, and collaboration with diverse stakeholders are essential to harness the positive potential of GAI for promoting diversity and inclusion in professional environments.

54

4.8

A. Pescapè

Intersection of GAI and IPRs

Navigating the intricate terrain where Generative Artificial Intelligence (GAI) intersects with intellectual property rights requires a nuanced exploration, delving into the technical and philosophical underpinnings. In the spirit of key figures like Ian Goodfellow, Yann LeCun, and Sam Altman, let’s embark on this journey.

4.8.1

Technical Considerations

Autonomous Creation and Ownership

The technical intricacies of GAI pose a fundamental question: when a generative model autonomously creates content, who holds ownership rights? The absence of direct human authorship challenges traditional copyright frameworks, necessitating a paradigm shift in our understanding of intellectual property. Data Training Sets and Derivative Works

GAI’s reliance on extensive data training sets introduces complexities regarding derivative works. The model’s ability to blend, remix, and reinterpret data raises questions about the originality of the output and the rights associated with the training data. Attribution and Authorship Identification

The technical challenge of attributing authorship in AI-generated content underscores the need for innovative solutions. Developing robust algorithms for traceability and watermarking could be crucial in establishing the provenance of content. Fair Use and Transformative Works

The technical implementation of fair use principles in AI-generated content requires defining the thresholds for transformative works. Developing algorithms capable of assessing the degree of transformation and creativity in generated outputs becomes a technical imperative.

4.8.2

Philosophical Considerations

AI as Tool and Co-creator

Philosophically, framing AI as a tool and co-creator rather than a traditional author prompts a reevaluation of intellectual property paradigms. This perspective aligns with the collaborative vision where AI augments human creativity rather than usurping it. Human Oversight and Accountability

The philosophical framework underscores the importance of human oversight in AI systems. Discussing the role of human creators, curators, and the accountability they bear for the outputs of generative models becomes pivotal in shaping ethical guidelines.

4 Exploring the Current State and Future Potential of Generative Artificial …

55

Balancing Innovation and IP Protection

Philosophically, striking a delicate balance between fostering innovation and preserving intellectual property rights emerges as a guiding principle. Encouraging a landscape where creators feel incentivized while ensuring the broader societal benefit requires thoughtful philosophical considerations. Cultural Impact and Global Perspectives

Embracing a global perspective, philosophical discussions extend to the cultural impact of AI-generated content. Recognizing diverse cultural nuances and avoiding the imposition of a singular set of values encapsulates the ethical fabric underpinning intellectual property considerations.

5

Conclusion

In conclusion, the exploration of the current state and future potential of Generative Artificial Intelligence (GAI) reveals a landscape rich with innovation, challenges, and opportunities. Across various domains, from advanced concepts in GAI to interactive and conditional generation, and from continual learning to ethical considerations, it is evident that GAI is poised to reshape industries, transform human–computer interaction, and drive societal change. As we reflect on the emerging research frontiers and the intersection of GAI with intellectual property rights (IPRs), it becomes apparent that collaboration and ethical stewardship are paramount. The technical advancements in GAI must be accompanied by a deep commitment to bias mitigation, transparency, privacy preservation, and societal impact assessment. Philosophical considerations underscore the need for value alignment, human oversight, and fairness in the development and deployment of GAI technologies. In navigating the complexities of GAI, it is imperative to strike a delicate balance between innovation and responsibility, fostering a culture of inclusivity, accountability, and ethical conduct. By embracing these principles, we can harness the transformative potential of GAI to foster creativity, drive progress, and enhance human well-being in the digital age. As we embark on this journey of exploration and discovery, let us remain steadfast in our commitment to shaping a future where Generative Artificial Intelligence serves as a catalyst for positive change, empowering individuals, organizations, and societies to thrive in a rapidly evolving world.

5.1

Note of the Author

In this section we report a personal conclusion of the author in form of three questions. What are my personal conclusions for this experiment?

56

A. Pescapè

My personal conclusion is that ChatGPT, when prompted correctly, is able to describe itself and its future landscape. The degree of precision and depth that ChatGPT can achieve depends on the prompt, but after several attempts, it eventually reaches an acceptable version, although certainly not profound. Am I satisfied with the results? Yes, I am satisfied with the result as I wanted to understand and demonstrate to what depth ChatGPT knew itself, and I succeeded. Did you reached your objectives? Yes, I reached the objectives. We are still far from a text written by an expert and a scientist.

Part II Digital Mind: Self-identity, Self-regulated Learning, and Intelligent Adaptive Systems

5

Supporting Learners’ Metacognition and Meta-Affect Jessica White and Benedict du Boulay

Abstract

This chapter describes self-regulated learning and its support by educational technology systems using artificial intelligence (AI). Self-regulated learning is explained through concepts such as judgements of learning, dialogic teaching, self and co-regulation, as well as epistemic development. This chapter covers the historical development of systems to support both the metacognitive and meta-affective aspects of self-regulated learning as well as their current use. A contemporary, adaptive learning platform Area9 Rhapsode™ (https://area9l yceum.com/ Accessed 1 Dec 2023) and Adapt© from Collins, fully based on an adaptive learning platform Area9 Rhapsode™; Area9 Lyceum™ https://area9l yceum.com/. Accessed 1 Dec 2023; https://www.collinsadapt.co.uk/ Accessed 26 Dec 2023 are used as running examples throughout the chapter.

1

Introduction

An important goal of schools, colleges and universities is to equip students to be competent members of society. But society is not static and each student’s future journey within it is unpredictable. So, a crucial area of formal education is to develop students’ understanding of, and skill in, the nature of their own learning, in order that later they can more readily acquire new understandings and develop new skills. This area would cover issues such as understanding their

J. White · B. du Boulay (B) University of Sussex, Brighton, UK e-mail: [email protected] J. White e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 F. Santoianni et al. (eds.), Mind, Body, and Digital Brains, Integrated Science 20, https://doi.org/10.1007/978-3-031-58363-6_5

59

60

J. White and B. du Boulay

own personal strengths and weaknesses as learners, as well as acquiring strategies to deal with the predictable barriers to learning that include lack of self-efficacy, over-confidence, bafflement, frustration and sometimes boredom. Broadly this area of education is called “learning to learn” or “self-regulated learning” and covers understanding and regulation of both cognition and feelings. Artificial Intelligence has played an ongoing role in helping learners improve their skills in learning [see, e.g. 3] from the early days in the 1970s but the recent interest in generative AI, and in ChatGPT in particular, has led to renewed interest in this use of AI in education. There has always been a greater focus on the cognitive rather than on the affective side of learning but that is slowly changing. The goal of this chapter is to introduce self-regulated learning and its support by educational technology systems using artificial intelligence (AI). This introduction covers both the historical development of such systems as well as their contemporary use. A particular commercial, adaptive learning platform, Area9 Rhapsode™ is used as a running example throughout the chapter. This chapter is in 5 sections. The next section provides a theoretical account of the of nature self-regulated learning and its related concepts such as self-efficacy and motivation. Section 3 outlines the role that AI has played in fostering selfregulated learning. It includes examples of very early work in the field as well as more recent work to provide some historical context. Section 4 changes tack and focuses specifically on issues around learning to understand and regulate the affective aspects of learning. Finally, Sect. 5 offers some concluding remarks.

2

Metacognition and Self-Regulated Learning

The purpose of this section is to give the reader an understanding of ‘self–regulated learning’ and ‘metacognition’. This is within the context of digital education, whilst briefly addressing the epistemological basis within which these concepts have been constructed. This section extends some of these conceptual approaches by applying them to the blended learning environments of the adaptive learning platform Area9 Rhapsode™. The purpose is to synthesise applied examples with theoretical debates, so that educators, researchers and practitioners can relate to exemplars, to apply these within their own blended learning contexts for best practice. In the adaptive learning platform Area9 Rhapsode™, an important part of the multimodal learning and instructional design is to make metacognition visible to learners and teachers as well as instructors. This contributes to their critical thinking, leading to mastery. Metacognition was a term coined by Flavell [4] to be “knowledge and cognition about cognitive phenomena” (p. 906) which has more generally been described as the overarching concept of thinking about one’s thinking. ‘Self-regulation’ emerged from the term metacognition [5] and has focused on monitoring cognition [6]. This includes planning, monitoring and evaluation [7– 11]. ‘Self-Regulated Learning’ emanated from this domain study that addresses the interaction between contextual, structural, cognitive and motivational factors

5 Supporting Learners’ Metacognition and Meta-Affect

61

[6] that had a precedent in ‘Self-Regulation’ (SR) that was explored by Bandura [12–14]. The relational aspects of these interactions have had less attention and will be predominantly the focus of this section, applied to practical examples from the adaptive learning platform Area9 Rhapsode™. Existing research literature indicates that the improvement of metacognition [4, 7, 9–11, 15] accelerates learning. Existing educational and policy structures have often given the responsibility for learning to the students themselves. However, the burden to create optimal circumstances for learning has often rested with teachers. This can lead to stress and overwhelm, particularly when socio-pedagogical needs are so diverse. Rather than observing isolated entities it is possible, taking examples from Area9 Rhapsode™ to observe how the ‘dialogic’ [16–18] and ‘multimodal’ [19–21] relational interactions between learners, teachers and technology, can develop and improve metacognition. It is these interactions that can support self-regulated learning [22, 23] which is a part of metacognitive control [7, 9–11]; as well as facilitating group metacognition and co-regulation in blended learning environments. Furthermore, this can lead to development of meta-affective states for both learners and teachers. The contribution of meta-affect to metacognition and self-regulated learning has been explored extensively, see Boekaerts and Cascallar for how educators could help students develop meta-affect awareness and Loyens et al. for motivational aspects [24–28]. Furthermore, Bandura [13, 14] expanded his research on self-efficacy to include emotional regulation. Further applied examples of meta-affect will be explored in the following sections of this chapter. Learners’ (students, employees or trainees) self-regulated learning patterns have been made visible within Area9 EDUCATOR™ tool that has also been applied in Adapt© fully based on the adaptive learning platform Area9 Rhapsode™, as seen in the example below with ‘student metacognition levels’ (see Fig. 1) that guides teachers to understand how learners’ ‘meta-states’ dynamically change as they progress through the module. This summary can also be anonymised and made visible to learners with an ‘Open Learner Model’ [29]. This is where teachers can guide students to further consolidate their metacognitive knowledge and regulation, by supporting them to discuss, through dialogue, their planning, monitoring and evaluation strategies. This can be facilitated in learner pairs with peers first and foremost, to create a safe and supportive space to discuss mistakes and misconceptions. Once comfortable with seeing learning as a process, where they are as adaptable as the digital platform itself, learners tend to feel less daunted to continue through dialogue, to discuss their misconceptions and mistakes as a group, in class. This method in practice has enabled an equalising of the teacher-student relationship, so that learning and teaching are more evidently part of the same process. This dialogic approach will be discussed further in

62

J. White and B. du Boulay

Fig. 1 Metacognition levels. Adapt© fully based on Area9 Rhapsode™

Sect. 2. In Area9 Rhapsode™ learners are asked to self-evaluate1 their prior knowledge before starting a module. As learners progress through the module they make continuous ‘judgements of learning’ or JOLS. Dunlosky and Metcalf [30] have authored extensive reviews of JOLS on a question-by-question basis. These JOLS are also used to check that learners have understood the ‘multimodal’ [19] reading materials, before moving on to an associated probe or learning objective. This enables Area9 Rhapsode™ to adapt to their appropriate knowledge and confidence level by either consolidating their knowledge further or providing challenging questions that are corresponding and relevant to learners’ level. This scaffolding [31] and continuous self-regulated learning [6] facilitates metacognitive monitoring [11] and ‘visible learning’ [22, 32].2 Area9 Rhapsode™ embedded metacognition relates to learners’ conscious and unconscious awareness of their knowledge. ‘The conscious and unconscious are so intimately related that no mental life is possible without them both. However, they are different in nature. The first refers to the mental representations, and the second to the processes, predominately associative, acting on these representations’ [33]. In Area9 Rhapsode™, as learners transition between different metacognitive states or ‘meta-states’ in relation to their comprehension of knowledge, this symbiotic relationship to knowledge [34] not only supports Area9 Rhapsode™’s adaptive

1 This is specifically for adult learners in Area9 Rhapsode™. For the K12 platform Adapt, teachers’ prime students’ knowledge of the specific subject area such as Biology in Science before learners learn with Adapt© which is Area9 Rhapsode™ platform transposed to age-appropriate learning design with the platform Adapt© (Home - Collins Adapt). 2 Under associated concepts that include those listed: self-reported grades, cognitive task analysis, conceptual change programs, self-efficacy, and strategy to integrate with prior knowledge.

5 Supporting Learners’ Metacognition and Meta-Affect

63

response but also scaffolds [31] learners to adapt so that they are in alignment with their ‘meta-states’. This relational interaction enables learners’ self-regulated learning and metacognitive development.

2.1

Multimodality, Metacognition and Self-Regulated Learning

Multimodal literacy is a term that originates in social semiotics and refers to the study of language that includes images, as well as written/spoken word that combines two or more modes of meaning [35]. Multimodality refers to the dynamic association of various representational modes that are in relationship with each other to create meaning [19] Multimodality has been defined as “a socially and culturally shaped resource for making meaning” [36]. In Area9 Rhapsode™ this multimodal instructional design helps to facilitate learners’ understanding. Area9 Lyceum™s Learning Engineers collaborate with teachers to break down complex knowledge by co-creating multimodal learning objectives. This is so that learners can understand domain knowledge through diverse modalities. For example, for a mathematics or science module learners can understand problems with pictorial and iconic representations of numerical concepts including diagrams; an audio coach and interactive written text, where the learners interact with Area9 Rhapsode™’s interface to sequence or categorize information, combined with embedded self-regulatory measures, such as judgements of learning [30] that help to define learners’ meta-states, that are dynamic and not fixed within the adaptive learning processes. Learners, through continuous deliberate practice with Area9 Rhapsode™’s multimodal learning objectives, are able to self-regulate by engaging with learning strategies such as spaced repetition, retrieval practice and interleaving that are metacognitive regulation strategies that support student’s learning and metacognitive development, embedded within Area9 Rhapsode™’s instructional design. Furthermore, this engagement facilitates learners’ understanding by adapting to their gradual mastery of knowledge in relation to their internal as well as external, socially mediated metacognitive states. There is a circular dynamic between learner’s cognition and metacognition mediated by Area9 Rhapsode™. Within this dynamic, learners’ metacognitive knowledge, that includes beliefs about their competencies, in relation to the domain knowledge in Area9 Rhapsode™, can be reflected upon through this critical thinking process with Area9 Rhapsode™ by engaging with the adaptive, multimodal learning objectives. This relational interaction is as dynamic and adaptable as the learners themselves, rather than being static, linear, or hierarchical. Area9 Rhapsode™’s adaptive, multimodal literacies [20] contribute to learners’ mastery of the domain knowledge and teachers’ instructional content. This adaptive, multimodality supports learners’ metacognitive development as learners are able to gain distance from one mode by focusing on another. For example, within the platform, learners are able to listen to audio to further comprehend

64

J. White and B. du Boulay

written text whilst being able gain holistic comprehension by associating the two another example of this self-regulated learning can be observed by deepening learners’ conceptual understanding not only in text mode, but also in pictorial mode. Giving students a combination of modes to interact with as they learn, can give space for learners’ metacognition to emerge and develop in combination with their understanding of the domain knowledge. As learners engage with multimodal learning with Area9 Rhapsode™ learners activate ‘dual processing’ [37]. Learners make meaningful cognitive and metacognitive associations [38] by understanding the relationships between the different modes that free up ‘cognitive load’ [39] by giving learners multimodal possibilities for comprehension and meta-comprehension. ‘Gestalt’ understanding [40] is possible through these relational interactions. Despite online digital platforms valuing cognitive load theory [39] when developing learning and instructional design, there is also the value of a guided, metacognitive approach to support students further. For example, by embedding and making visible the intentional, self-directed understanding of how learners monitor and regulate tasks, this can provide possibilities to learn metacognitive strategies, so that students have agency to continue to self-regulate themselves and others in a variety of future educational contexts. When, these opportunities for learners to think metacognitively are lacking in online curricula, it is a factor that limits success of transferability in some programs [41]. Area9 Rhapsode™ combines these metacognitive self and shared regulatory strategies within the embedded multimodal, instructional learning design to support learning that leads to mastery. The terms cognitive ‘processing’, ‘encoding’ ‘or ‘reading’, to some extent have implied that the ‘message’ lies within the different modes such as the image, text, or audio or, that self-regulated learning is only happening inside learners’ brains on a cognitive level. Whilst these factors do have associative importance, it is rather the combined relational interactions mediated by these modes and adaptive learning, through teachers and learners’ social relations [42] that develops metacognition. So, it is arguably the dialogical [17, 18] mediation [43] of these of multimodal literacies [20] that may be contributing to the shaping of learner’s metacognition.

2.2

Dialogic Teaching, Self-regulation and Shared Metacognitive Regulation

‘Dialogic Teaching’ [44] can support ‘self-regulation’ and ‘shared metacognitive regulation’ in digital and blended learning environments such as observed with Area9 Rhapsode™. Exploring classical Greek interpretation, ‘Dia’ means through and ‘logos’ means word or discourse. Robin Alexander’s extensive work on Dialogic Teaching [44, 45] explores how to engage multiple voices where teachers and learners, through dialogue can co-construct knowledge that is fluid and dynamic. This dialogic interaction can develop both learners and teachers’ metacognition that can mitigate fixed dogmatic perspectives in relation to domain knowledge.

5 Supporting Learners’ Metacognition and Meta-Affect

65

Fig. 2 Meta triangle adaptive learning platform

This dialogic approach, particularly within the context of Area9 Rhapsode™, when combined with multimodal, adaptive learning and embedded self-regulation strategies that develops learner’s metacognition, can facilitate personalised access points and learning paths where students can feel that they are not always dictated by a hierarchical directive; rather they can gain agency by understanding challenging subject matter, that may have seemed impenetrable or alienating if misunderstood (see Fig. 2). Dialogic engagement with domain knowledge through interactive, multimodal learning objectives as exemplified in Area9 Rhapsode™ enables further mediated dialogic engagement with learners, teachers, and peers. This can enable selfregulation as well as shared metacognitive regulation. Learners consistently engage in deliberate practice with those multimodal learning objectives with interactive questions, that include: ranking, multiple choice, fill-in-the-blank, categorisation and labelling together with judgements of learning, [30] and embedded selfregulated learning strategies such as interleaving, spaced repetition and retrieval practice. These support learners with metacognitive regulation by becoming consciously aware of their gaps in knowledge as well as their own biases and misconceptions in relation to knowledge. This enables learners to become more self-directed by correcting their own mistakes, whilst remaining open to others’ pluralistic perspectives when learning within the classroom and lifelong learning. As learners become comfortable with understanding that learning is not only about outcomes, rather it is a process that is dynamic and subject to change, learners are able to be more aware of others’ different perspectives that are also dynamic and not fixed states [46]. In class, or with adult learners, in diverse teams, having open and constructive dialogue with teachers and peers flattens the hierarchy, bringing collective, intergenerational insights that benefit from varying levels of expertise. This important pluralistic and diverse dialogue [46] can foster collaboration, within and between different perspectives. This dialogic approach facilitates shared metacognitive regulation as well as group metacognition by making visible [32]

66

J. White and B. du Boulay

conceptually negotiated meanings that are not fixed, but dynamic and shaped with the multiple learning and teaching relationships mediated by technology such as Area9 Rhapsode™ and extended into a variety of educational contexts.

2.3

Judgements of Learning

‘Judgements of learning’ have been a part of research on metacognitive monitoring that has been seen to support learners’ self-regulation as they learn [e.g. 47, 48–50]. Due to the extensiveness of these studies, JOLs have been one of the more prominent concepts to understand how learners monitor their developing understanding. Extensive reviews [30] have discussed how judgements of learning are better done retrospectively rather than prospectively [51]. This may relate to local and global JOLs. There is less exploration of the learners’ self-concept about the JOLs. Research within this field has focused on item-by-item decisions rather than exploring the relationships with the broader social, contextual learning goals situated within online and blended environments, which is the focus of this section with Area9 Rhapsode™ and Adapt© fully based on Area9 Rhapsode™ as an exemplar. Learners are able to interact with teachers/instructors ‘mediated’ [43] by Area9 Rhapsode™ multimodal probes, who question like a patient, personal tutor that knows the level of student’s knowledge and anticipates their next appropriate learning resource or question depending on how they answer and is facilitated by the platform’s adaptive capabilities. Learners are provided with a judgement of learning (see Fig. 3) capability that is learner’s self-report, confidence rating relating to the module’s domain knowledge. This is for every explanatory learning resource and question within Area9 Rhapsode™. This contributes to how learners’ metacognition is made visible in the Area9 EDUCATOR™ reports and the Student Metacognition Levels in Adapt© fully based on Area9 Rhapsode™. Furthermore, the judgements of learning give learners the opportunity to reflect upon their answer. This continuous metacognitive monitoring [43] that refers to learners’ awareness of comprehension and task performance and is typically difficult to make visible in traditional learning environments. This continuous judgement of learning in Area9 Rhapsode™ activates learners’ awareness and accuracy so learners can effectively develop their self-efficacy [12]. Accurate metacognitive monitoring with these judgements of learning support learners’ continuous self-regulation [49] throughout the Area9 Rhapsode™ modules. As learners judge their own understanding in a closely relatable way to the domain knowledge within the Area9 Rhapsode™ module, they gain confidence in metacognitive knowledge and regulation that is specific to that subject area, which has been seen to be more effective than just teaching metacognition alone. This means that students learn best when they are taught to think about and regulate their own learning in relation to the subject that they are studying and the context that it is situated within. As questions and learning resources in Area9

5 Supporting Learners’ Metacognition and Meta-Affect

67

Fig. 3 Judgements of learning. Adapt© fully based on Area9 Rhapsode™

Rhapsode™ are co-constructed with teachers, teaching of both the subject matter and metacognition is mediated with and through Area9 Rhapsode™ facilitated by a ‘dialogical approach’ between teacher and student [16, 18] than enables self and shared metacognitive regulation. Area9 Rhapsode™’s adaptive capabilities enable learners to interact with questions and multimodal learning resources that help to explain and clarify understanding or misconceptions if the learner needs support or is struggling. These interactions develop symbiotically with Area9 Rhapsode™’s adaptive capabilities to create a personalised learning path for each learner. Each learner is able to access the same core learning objectives but taught dialogically to their level. If a learner is competent, learners engage in further ‘dialogic’ questions from teachers/ instructors that are mediated [43] through the constructed ‘multimodal instructional design’ [52] and ‘multimodal literacies’ [20] as well as through Area9 Rhapsode™’s adaptive learning capabilities, including the metacognitive layer, to support participation in learning more challenging aspects of the learning objective, or new learning objectives. This mediated learning can support self-regulation, as these dialogic interactions are like a tutorial with a patient and knowledgeable tutor, who gently understands what the learner needs by asking them questions, or if they are struggling, providing didactic learning resources or less difficult questions. Through this process learners understanding develops combined with their judgements of learning, that cultivates student’s self-regulated learning and metacognition.

2.4

Epistemic Development and Self-regulation

Making epistemic understanding of domain knowledge explicit in school and higher education, as well as work-based learning and development, can support

68

J. White and B. du Boulay

learners’ self-regulation and shared metacognitive-regulation. As learners become familiar with what it means to know a specific domain, such as how to approach the discipline and construct new knowledge within those foundations and principles, learners can become knowledgeable about how to calibrate their learning in relationship to those guiding principles of a particular discipline. Furthermore, by grasping the underlying epistemology of a domain, learners can understand that, like their own epistemic development these guiding principles underlying knowledge are fluid, dynamic and subject to change through their own participation in that domain knowledge, of which they become a part. Knowing how to actively participate in this knowledge construction by calibrating with the established epistemology of a domain, learners participate in epistemic development. This occurs by learners thinking metacognitively about these underlying foundations of specific domains and disciplines, and how they can self-regulate, by calibrating with domain epistemology. That self-regulation continues as learners gain confidence through understanding, to further build upon discipline. Self-regulation that is part of metacognitive regulation [53] is embedded in Area9 Rhapsode™ through a combination of the judgements of learning with self-rating confidence measures, as well as the way in which the questions and learning resources are constructed. Learners engage with multimodal probes (questions) and learning resources, that are interleaved with each other through spaced practice powered by Area9 Rhapsode™’s adaptive capabilities, that activates learners’ retrieval of their prior knowledge with specific language of thinking words [54]. These often include revised Blooms taxonomy words that are linked to the specific domain language with the module, that learners will have learnt and continue to learn by engaging with that specific knowledge domain through Area9 Rhapsode™. These language of thinking words, can build upon learners’ epistemic development, whilst concurrently supporting learners’ self-regulation and metacognition. This can be done by learners becoming more familiar with specific language that enables domain knowledge construction, so that learners can synthesize it with their own lexicon and schema. Developing metacognition requires the concept and its language to be taught explicitly [53, 55].

3

The Role of AI in Fostering Self-Regulated Learning and Learning to Learn

This section builds on the example of Area9 Rhapsode™, given above, to illustrate how educational technology, particularly that which is based on AI, has played a crucial role in gathering data about self-regulated learning behaviour as well as providing tools to support learner development in this area. This section is not a comprehensive review but offers some snapshots of the development of systems over time. For a broad overview of this area see, for example, Azevedo and Aleven [3]. An early account of such tools can be found under the title “The computer as a Tool for Learning through Reflection” [56]. Since then, this support has taken

5 Supporting Learners’ Metacognition and Meta-Affect

69

three broad forms. First, there are tutoring systems that engage with learners both at the subject domain level and also at the metacognitive level [see, e.g. 57]. Such systems aim to support both metacognitive reflection as well as metacognitive regulation. Second there are systems that invite the learner to engage in “learning by teaching” or “learning by being questioned” in order to help them reflect on the quality of what they know and understand [see, e.g. 58]. Such systems mainly focus on assisting metacognitive reflection. Third, there are dashboards, also known as Open Learner Models, that reflect back to learners their learning trajectories with a view to assisting both reflection and regulation [see, e.g. 59]. They are called open learner models because most learner-focussed AI systems create and update a normally “hidden” model of the learner so that the system can personalise the choice of task, the scaffolding or the feedback to that learner. Opening up that otherwise hidden model so that the learner can view it enables many possibilities for reflection by the learner. In some cases, the learner can challenge the model generated by the system [60]. It is important to note that reflecting and regulating, particularly with the aim of improving self-regulated learning capability requires extra effort from the learner and thus extra motivation as compared to simply learning the subject matter. Self-regulated learning is dependent upon students being motivated to regulate cognition, strategies, and effort (Pintrich and De Groot, 1990, Zimmerman, 1990) and to respond to feedback (Zimmerman, 1990) [my emphasis]. Feedback is inherent to self-regulated learning processes, with external feedback providing essential information about potential discrepancies between current and desired performance (Butler & Winne, 1995). Learning analytics provides a method of providing external feedback to students about their learning, with dashboards (and by extension learning analytics alerts) providing a cue to potential discrepancies (Kim, Jo, & Park, 2016). However, as Jivet et al. (2017) caution in relation to learning analytics dashboards, “making learners aware is not enough” (p. 93); self-regulated learning also requires self-reflection and action. This involves understanding, internalising and using the feedback to improve academic performance (Nicol & Macfarlane-Dick, 2006), and this process may be impacted by affective responses to the feedback [my emphasis]. [61, page 9]

3.1

Tutoring Systems and Metacognition

A well-rehearsed but mistaken critique of intelligent tutoring systems is that they prioritise mastery of a domain learning over development of self-regulated learning [62]. While this may be true of some systems, several systems have been developed that combine a focus on both goals [57]. In addition, the very precision with which tutoring systems need to specify and record their interactions with the user enables detailed data to be collected and comparative experiments run to determine learning gains, improvement in metacognitive capability or the effects of contingent variables such as emotion and goal orientation. One example of such empirical work has been conducted by [63]. They used BioWorld, a system to teach diagnosis to medical students to explore the detailed “interplay between affect and self-regulation”. BioWorld does

70

J. White and B. du Boulay

not directly train metacognitive capability but has proved an excellent platform to explore the relation of metacognition to other contingent issues. For example, Lajoie and her colleagues found that “High performers exhibited marginally more monitoring SRL behaviours, while low performers tended to orient and reorient throughout the task.” In terms of tutoring to improve metacognition, help seeking has been one of the metacognitive skills addressed in the field of artificial intelligence in education. This issue has been chosen because many computer-based tutors provide a “help button” and this can be misused either through overuse, not being used when it might have been beneficial, or dishonestly used to “game the system” [64]. Two examples of systems aiming to assist the learner better gauge what they do and do not understand (judgements of learning) and consequently their use of the help system are described as follows. The Help Tutor aimed to help students become better learners by teaching them how to make more effective use of the help system built into the Cognitive Tutors [65]. It used the same theoretical understanding of learning and the same technology as the tutoring system, namely modelling the student via a rule-base and providing immediate feedback, to detect and provide feedback on poor quality use of the help button. The system was evaluated in real classrooms and achieved some “partial successes in attempts to produce lasting improvements in metacognitive behaviors” [66]. In a similar vein, [67] used Ecolab, an system for teaching ecology to children to explore the contingent issue of the children’s goal orientation. They showed that the children’s goal orientation, whether mastery or performance, was a strong factor in how they approached their help-seeking and thus needed to be factored into the design of effective scaffolding. MetaTutor is the most sophisticated example in this subsection. It provides much more rounded training in both metacognitive reflection and metacognitive regulation than the examples above [57]. It is designed to develop self-regulated learning rather than just help-seeking. “MetaTutor is a hypermedia-based ITS that teaches challenging STEM content (e.g., human circulatory system)...Over the years, the design of the STEM content has included experts in several fields of STEM and biomedical sciences.” It contains four virtual personal assistants that monitor the student’s activity and intervene when appropriate with advice and feedback based on self-regulated learning theory. (1) Gavin the Guide helps students to navigate through the system and orient the students about the task; (2) Pam the Planner guides students in setting appropriate sub-goals byactivating their prior knowledge and coordinating sub-goals; (3) Mary the Monitor helps students to monitor their progress toward achieving their subgoals by prompting and scaffolding several metacognitive processes such as FOKs [Feelings of Knowing], JOLs [Judgements of Learning], and CEs [Content Evaluations]; and

5 Supporting Learners’ Metacognition and Meta-Affect

71

(4) Sam the Strategizer helps students deploy SRL learning strategies, such as summarizing and note-taking, making inferences, re-reading, and generating hypotheses. Learners can interact with these PAs and enact specific SRL learning processes by selecting any feature of the SRL palette displayed at the right-hand side of the interface during the learning session.” [57 page 6]

With regard to point 4 above, an evaluation of MetaTutor for a medical domain found that it improved the note-taking of low prior-knowledge students [68].

3.2

Learning by Being Questioned or by Teaching

A very early example of learning by being questioned is found in the system Why [69]. This implemented a “Socratic” dialogue in which the system questioned the student about their beliefs and understanding of the processes involved in rainfall, posing questions and counterexamples to the student’s answers to help them gradually refine their understanding. It is well attested that learning by teaching someone else is an excellent way of reflecting on what one fully or only partially understands [see, e.g. 70]. Several early AIED systems involved adding a virtual learning companion whom the human student attempted to teach [see, e.g. 71, 72]. More recently, the system Betty’s Brain has become the classic example of this approach [58, 73, 74]. Using this system, the human learner reads online materials about an ecosystem and subsequently creates a concept map to explain the various processes involved. The learner can submit the concept map for checking and gets advice as to which parts of it are correct, wrong or inadequate. Via the reflective creation of a sequence of increasingly accurate concept maps the learner gradually achieves a much greater understanding of the domain. An important factor is that the order in which the learner builds this concept map is completely under their control. Due to the open-ended nature of the learner’s interaction, more recent work with Betty’s Brain has explored the value of adaptive scaffolding to help learners’ self-regulation skills and thus their progress in a productive manner [75].

3.3

Open Learner Models and Metacognition

Bull and Kay [59] provided an early account of “the similarities between the goals of supporting and encouraging metacognition in intelligent tutoring systems and learning in general, and the benefits of opening the learner model to the learner and to the teacher.” A more up to date account of how open learner models can assist self-regulated learning can be found in [76]. She provides several examples of how OLMs have been shown to be effective in helping learners master domain knowledge or skill than when the OLM was not available. More interestingly for this chapter, she

72

J. White and B. du Boulay

cites an example where the OLM improved self-assessment accuracy (judgement of learning) in primary school children. The vastly increased role of data in AIED systems and the consequential development of learning analytics has enabled many developments in dashboards aiming to assist self-regulated learning. However, the results so far are limited in terms of improving learners’ self-regulation. For example, a review of the uses of learning analytics to support self-regulated learning by Viberg et al. [77] examined 54 empirical studies but found limited evidence of improving learners’ self-regulated learning skills, though they did find some evidence of improved learning gains. In another review, Matcha et al. [78] came up with similar results with w.r.t self-regulated learning skills. Perhaps one way that progress could be made in improving self-regulation skills is by getting the learners themselves engaged in the design of the dashboards [79]. In another direction involving AI, Molennar describes a prototype of a Hybrid Human-AI Regulation (HHAIR) system: HHAIR positions hybrid regulation as a collaborative task of the learner and the AI which is gradually transferred from AI-regulation to self-regulation. Learners will increasingly regulate their own learning progressing through different degrees of hybrid regulation. In this way HHAIR supports optimized learning and the transfer and development of SRL skills for lifelong learning (future learning). [80 page 1]

4

The Role of AI in Fostering the Affective Side of Self-Regulated Learning

Much of the preceding part of this chapter has focused on the cognitive aspects of self-regulation. This section is concerned with the affective dimension of selfregulation. The cognitive and the affective aspects are intertwined [see, e.g. 81] not least because attempting regulative activities such as planning, goal setting and reflecting can trigger, or be triggered by, both enabling and disabling feelings. Indeed, the cognitive interpretations of, and affective reactions to, these feelings can themselves be enabling or disabling, and can have various effects on cognitive processing [82]. The regulation of the affective aspects of learning are especially important in such subjects as mathematics [83], second language learning [84] and in sports training [85]. Useful overviews of theories of affect, meta-affect and affective pedagogy from the point of view of artificial intelligence in education can be found in [86] and in D’Mello et al. [87]. Becoming fully self-regulating as a learner requires being both metacognitively and meta-affectively aware, as well as being able to deploy effective strategies to regulate both the cognitive and the affective aspects of learning. For example, in the case of mathematics, [88] argues: a curriculum in which affect is considered may include generating problems from the curiosity of students, in order to develop their sense that intense feelings are appropriate, and teaching them to apply heuristics when these feelings occur. Anxiety, fear, and despair

5 Supporting Learners’ Metacognition and Meta-Affect

73

(but not puzzlement, bewilderment, and frustration) maybe regarded as essentially undesirable affective states; yet we need to provide appropriate, domain-specific ways for students to handle the negative affect when it (inevitably) occurs. We also need to provide productive experiences with and uses of desirable affect, including the affect of frustration. Students should occasionally experience frustration--not too much, of course--and then be brought back from it, guided to make progress in the problem with processes suggested by the frustration, and feel subsequent pleasure, elation, and satisfaction that is heightened by the earlier experience of frustration. We give students too little experience with intensely positive affective states, rarely connecting them with their own, genuine mathematical achievements. [88 page 217–8]

Let us continue with the example of learning mathematics. The feeling of panic when faced with what looks like an incomprehensible mathematical problem and the consequent recollection of previous such unhappy episodes can both narrow the learner’s mental field of view as well as promote the “fight or flight” response, often in favour of flight. Various systems have been developed to support the affective dimension of the learner’s self-regulation. Some of these infer the learner’s affective state indirectly from their progress on the learning task. Others instrument or observe the learner in various ways to get less indirect data about the learner’s affective state. In terms of developing the learner’s affective self-regulation, systems fall into three broad kinds: Purely Reactive systems try to recognise the learner’s affective state and react to it in a way that is hoped to be helpful: for example, by reinforcing a presumed, positive state through praise (say) or by empathising with a presumed, negative state [89]. Such systems will have a limited effect on the learner’s affective selfregulatory capability. One way for systems to react to learner affect is to use an animated pedagogical agent whose demeanour, facial expression and gestures can indicate its feelings in response to the learner’s progress. One such system, Passenger, assisted collaborative work among university students [90]. The pedagogical agent in this system showed a happy or a sad face and/or a confused gesture in response to the learners’ progress or lack of it. Explicit systems both react to and name the inferred affective state of the learner. Identifying the affective state to the learner and suggesting how to deal with it during learning should develop the learners’ accuracy in identifying and managing such states. Arroyo et al. [91] describe a system for school-level mathematics that adapts its task difficulty, scaffolding and feedback in terms of the learner’s affective, cognitive and metacognitive state. In terms of affective self-regulation, where appropriate, the system explicitly praises effort rather than outcomes and deemphasizes the importance of immediate success. In another example, Supportive AutoTutor used a variety of methods to infer the affective state of the learner including eye-tracking, body posture and text analysis and then reacted in motivationally supportive ways [92]. These included

74

J. White and B. du Boulay

adjusting the facial expression and speech modulation of an on-screen pedagogical agent and adjusting the content of the next dialogue move with the student. The researchers compared Supportive AutoTutor with the regular AutoTutor and found that Supportive Autotutor enabled improved learning gains, particularly for students with low initial domain knowledge. In addition, Supportive AutoTutor attempts to help the learner regulate negative emotions by explicitly attributing the source of the learners’ named emotion to the material or the tutor instead of to the learners themselves: So the Affective AutoTutor might respond to mild boredom with ‘This stuff can be kind of dull sometimes, so I’m gonna try and help you get through it. Let’s go.’ A response to confusion would include attributing the source of the confusion to the material (‘Some of this material can be confusing. Just keep going and I am sure you will get it’) or the tutor itself (‘I know I do not always convey things clearly. I am always happy to repeat myself if you need it. Try this one’). [87]

Pro-active systems specifically engage learners in affectively focused activities during learning to help them learn to manage their affective states both negative and positive. As far as we know there is no tutoring system yet that provides the kind of guided tutoring in managing feelings during learning, as described by [88] (above. On a much-reduced scale, [93] developed a system that engaged the students in relaxation activities at various points during their learning e.g. after they had failed to complete a task successfully.

5

Conclusions

This chapter has described AI-based educational systems that support learners’ metacognitive and meta-affective development. These include intelligent tutoring systems, systems for questioning the learner or enabling the learner to engage in teaching, and open learner models. A contemporary, commercial system, Rhapsode™, was used to illustrate support for both self-regulation and co-regulation. This system adapts its pedagogic strategy on an individual basis, dependent on the explicit judgements of learning of the learners and on its own implicit inferences as to how these are to be interpreted. The complexities of the nature of knowing, including its highly individualised and contextualised aspects indicate that dialogue between learners and between learners and teachers is crucial. AI-based learning systems can only mimic the subtle interactions of highly skilled human tutors needed to help learners in this way to a limited extent, though progress is being made. Support for system-based affective self-regulation is much less developed, though there are many systems that react to what they infer their learners are feeling. These fall into three categories: purely reactive, explicit and pro-active. There is no pro-active system yet available, as far as we know, that embodies the kind of managed provocation of both negative and positive feelings in learners as envisaged by Goldin [88].

5 Supporting Learners’ Metacognition and Meta-Affect

75

References 1. Collins Adapt©. http://www.collins.co.uk/page/Collins+Adapt. Accessed 26 Dec 2023 2. Area9 Lyceum Rhapsode™ and Rhapsode™ Area9 Lyceum. https://area9lyceum.com/the-pla tform/rhapsode-learner/. Accessed 1 Dec 2023 3. Azevedo R, Aleven V (eds) (2013) International handbook of metacognition and learning technologies. Springer international handbooks of education. Springer, New York 4. Flavell JH (1979) Metacognition and cognitive monitoring: a new area of cognitive–developmental inquiry. Am Psychol 34(10):906–911. https://doi.org/10.1037/0003-066X.34.10.906 5. Baker L, Brown AL (1984) Metacognitive skills and reading. In: Pearson PD et al (eds) Handbook of reading research. Longman, New York, pp 353–394 6. Lajoie SP (2008) Metacognition, self regulation, and self-regulated learning: a rose by any other name? Educ Psychol Rev 20:469–475. https://doi.org/10.1007/s10648-008-9088-1 7. Cross DR, Paris SG (1988) Developmental and instructional analyses of children’s metacognition and reading comprehension. J Educ Psychol 80(2):131–142. https://doi.org/10.1037/00220663.80.2.131 8. Paris SG, Winograd P (1990) How metacognition can promote academic learning and instruction. In: Jones BF, Idol L (eds) Dimensions of thinking and cognitive instruction. Lawrence Erlbaum Associates, Inc., pp 15–51 9. Schraw G, Moshman D (1995) Metacognitive theories. Educ Psychol Rev 7:351–371. https:// doi.org/10.1007/BF02212307 10. Whitebread D et al (2009) The development of two observational tools for assessing metacognition and self-regulated learning in young children. Metacogn Learn 4:63–85. https://doi.org/ 10.1007/s11409-008-9033-1 11. Schraw G (2009) A conceptual analysis of five measures of metacognitive monitoring. Metacogn Learn 4:33–45. https://doi.org/10.1007/s11409-008-9031-3 12. Bandura A (1977) Self-efficacy: toward a unifying theory of behavioral change. Psychol Rev 84(2):191–215. https://doi.org/10.1037/0033-295X.84.2.191 13. Bandura A (1982) Self-efficacy mechanism in human agency. American Psycologist 37(2):122–147. https://doi.org/10.1037/0003-066X.37.2.122 14. Bandura A (1986) Social foundations of thought and action. In: Marks DF (ed) The helath psychology reader. Englewood Cliffs NJ, Prentice-Hall 15. Kuhn D, Dean D Jr (2004) Metacognition: a bridge between cognitive psychology and educational practice. Theory Pract 43(4):268–273.https://doi.org/10.1207/s15430421tip4304_4 16. Mercer N, Howe C (2012) Explaining the dialogic processes of teaching and learning: the value and potential of sociocultural theory. Learn Cult Soc Interact 1(1):12–21. https://doi.org/ 10.1016/j.lcsi.2012.03.001 17. Soong B, Mercer N (2010) Improving students’ revision of physics concepts through ICTbased co-construction and prescriptive tutoring. Int J Sci Educ 33(8):1055–1078. https://doi. org/10.1080/09500693.2010.489586 18. Freire P (1970) Pedagogy of the oppressed. Continuum, New York 19. Brandt S, Kress Gvan Leeuwen T (2001) Multimodal discourse: the modes and media of contemporary communication. Arnold, Londin 20. Jewitt C, Kress G (2003) A multimodal approach to research in education in language, literacy and education: a reader. Goodman S et al (eds) Trentham books: Stoke-on-Trent, UK, pp 277– 292 21. Jewitt C (2008) Multimodality and literacy in school classrooms. Rev Res Educ 32(1):241– 267. https://doi.org/10.3102/0091732x07310586 22. Zimmerman BJ, Schunk DH (Eds) (1989) Self-regulated learning and academic achievement: theory, research, and practice. Springer. 23. Schunk DH (1994) Self-regulation of self-efficacy and attributions in academic settings. In Schunk DH, Zimmerman BJ (eds) Self-regulation of learning and performance: issues and educational applications. Lawrence Erlbaum Associates, Inc., pp 78–99

76

J. White and B. du Boulay

24. Boekaerts M, Cascallar E (2006) How far have we moved toward the integration of theory and practice in self-regulation? Educ Psychol Rev 18:199–210. https://doi.org/10.1007/s10 648-006-9013-4 25. Pintrich PR (2004) A conceptual framework for assessing motivation and self-regulated learning in college students. Educ Psychol Rev 16(4). https://doi.org/10.1007/s10648-004-0006-x 26. Winne PH, Perry NE (2000) Measuring self-regulated learning. In: Boekaerts M, Pintrich P, Zeidner M (eds) Handbook of self-regulation. Academic Press 27. Witherspoon AM, Azevedo R, D’Mello S (2008) The dynamics of self-regulatory processes within self-and externally regulated learning episodes during complex science learning with hypermedia. In: Woolf BP et al (eds) Intelligent tutoring systems. ITS 2008. Springer, Berlin, Heidelberg 28. Loyens SMM, Magda J, Rikers RMJP (2008) Self-directed learning in problem-based learning and its relationships with self-regulated learning. Educ Psychol Rev 20(4):411–427https://doi. org/10.1007/s10648-008-9082-7 29. Bull S, Kay J (2010) Open learner models. In: Nkambou R, Bourdeau J, Mizoguchi R (eds) Advances in intelligent tutoring systems. Studies in computational intelligence. Springer, Belin, Heidelberg, pp 301–322 30. Dunlosky J, Metcalfe J (2009) Metacognition. Sage Publications 31. Wood D,Bruner J, SRoss G (1976) The role of tutoring in problem solving. J Child Psychol Psychiatry 17(2):89–100.https://doi.org/10.1111/j.1469-7610.1976.tb00381.x 32. Hattie J (2008) Visible learning: a synthesis of over 800 meta-analyses relating to achievement. Routledge, London 33. Perruchet P, Vinter A (2003) The self-organizing consciousness as an alternative model of the mind. Behav Brain Sci 25(3):360–380. https://doi.org/10.1017/s0140525x02550068 34. Young M (2020) Knowledge and the sociology of education. Acta Paedagog Vilnensia 44:10– 17. https://doi.org/10.15388/ActPaed.44.1 35. Mills KA, Unsworth L (2017) Multimodal literacy. In: Oxford research encyclopedia of education, pp 1–32 36. Kress GR (2010) Multimodality: a social semiotic approach to contemporary communication. Routledge, London & New York 37. Clark JM, Paivio A (1991) Dual coding theory and education. Educ Psychol Rev 3(3):149–210. https://doi.org/10.1007/BF01320076 38. Hebb DO (1949) The organization of behavior; a neuropsychological theory. Wiley 39. Sweller J, Chandler P (1991) Evidence for cognitive load theory. Cogn Instr 8(4):351–362. https://doi.org/10.1207/s1532690xci0804_5 40. Wertheimer M (1944) Gestalt theory. Soc Res 11:78–99 41. Ellerton P (2022) On critical thinking and content knowledge: a critique of the assumptions of cognitive load theory. Think Ski Creat 43. https://doi.org/10.1016/j.tsc.2021.100975 42. Banks M, Ruby J (2011) Made to be seen: perspectives on the history of visual anthropology. University of Chicago Press 43. Wertsch JV (2007) Mediation. In: Daniels H, Cole M, Wertsch JV (eds) The Cambridge companion to Vygotsky. Cambridge University Press, pp 178–192 44. Alexander R (2020) A dialogic teaching companion. Routledge, London 45. Alexander RJ (2008) Towards dialogic teaching: rethinking classroom talk. Dialogos, York 46. Bohm D (1996) On dialogue. Routledge, London 47. Dunlosky JA, Rawson K (2012) Overconfidence produces underachievement: inaccurate self evaluations undermine students’ learning and retention. Learn Instr 22(4):271–280.https://doi. org/10.1016/j.learninstruc.2011.08.003 48. Kornell N, Metcalfe J (2006) Study efficacy and the region of proximal learning framework. J Exp Psychol Learn Mem Cogn 32(3):609–622. https://doi.org/10.1037/0278-7393.32.3.609 49. Thiede KW, Dunlosky J (1999) Toward a general model of self-regulated study: an analysis of selection of items for study and self-paced study time. J Exp Psychol Learn Mem Cogn 25(4):1024–1037. https://doi.org/10.1037/0278-7393.25.4.1024

5 Supporting Learners’ Metacognition and Meta-Affect

77

50. Thiede KW, Anderson MCM (2003) Summarizing can improve metacomprehension accuracy. Contemp Educ Psychol 28(2):129–160. https://doi.org/10.1016/S0361-476X(02)00011-5 51. Yeung N, Summerfield C (2012) Metacognition in human decision-making: confidence and error monitoring. Philos Trans: Biol Sci 367:1310–1321. https://doi.org/10.1098/rstb.2011. 0416 52. Selander S, Kress G (2010) Designs for learning–a multimodal perspective. Norstedts, Stockholm 53. Pintrich PR, Zusho A (2002) The development of academic self-regulation: the role of cognitive and motivational factors. In: Wigfield A, Eccles JS (eds) Development of achievement motivation: a volume in educational psychology. Academic Press, pp 249–284 54. Tishman S, Perkins D (1997) The language of thinking. Phi Delta Kappan 78(5):368–374 55. Tanner KD (2012) Promoting student metacognition. CBE—Life Sci Educ 11(2):113–120. https://doi.org/10.1187/cbe.12-03-0033 56. Collins A, Brown JS (1988) The computer as a tool for learning through reflection. In: Mandl H, Lesgold A (eds) Learning issues for intelligent tutoring systems. Springer, New York, pp 1–18 57. Azevedo R et al (2022) Lessons learned and future directions of MetaTutor: leveraging multichannel data to scaffold self-regulated learning with an intelligent tutoring system. Front Psychol 13. https://doi.org/10.3389/fpsyg.2022.813632 58. Biswas G, Segedy JR, Bunchongchit K (2016) From design to implementation to practice a learning by teaching system: Betty’s Brain. Int J Artif Intell Educ 26(1):350–364.https://doi. org/10.1007/s40593-015-0057-9 59. Bull S, Kay J (2008) Metacognition and open learner models. In: The 3rd workshop on metacognition and self-regulated learning, at ITS 2008 60. Dimitrova V (2003) STyLE-OLM: interactive open learner modelling. Int J Artif Intell Educ 13(1):35–78 61. Howell JA, Roberts LD, Mancini VO (2018) Learning analytics messages: impact of grade, sender, comparative information and message style on student affect and academic resilience. Comput Hum Behav 89:8–15.https://doi.org/10.1016/j.chb.2018.07.021 62. Watters A (2017) Dunce’s App: How Silicon Valley’s brand of behaviorism has entered the classroom. In: The Baffler 63. Lajoie SP et al (2021) Examining the interplay of affect and self regulation in the context of clinical reasoning. Learn Instr 72. https://doi.org/10.1016/j.learninstruc.2019.101219 64. Baker R et al (2008) Why students engage in “Gaming the System” behaviours in interactive learning environments. J Interact Learn Res 19(2):185–224 65. Roll I et al (2007) Designing for metacognition—applying cognitive tutor principles to the tutoring of help seeking. Metacogn Learn 2(2–3):125–140. https://doi.org/10.1007/s11409007-9010-0 66. Koedinger K et al (2009) In Vivo experiments on whether supporting metacognition in intelligent tutoring systems yields robust learning. In: Hacker DJ, Dunlosky J, Graesser AC (eds) Handbook of metacognition in education. Routledge, New York, pp 383–412 67. Harris A et al (2009) Scaffolding effective help-seeking behaviour in mastery and performance oriented learners. In: Proceedings of 14th international conference on artificial intelligence in education 68. Trevors G, Duffy M, Azevedo R (2014) Note-taking within MetaTutor: interactions between an intelligent tutoring system and prior knowledge on note-taking and learning. Educ Technol Res Dev 62(5):507–528. https://doi.org/10.1007/s11423-014-9343-8 69. Stevens AL, Collins A (1977) The goal structure of a socratic tutor. In: ACM ‘77 proceedings of the 1977 annual conference 70. Duran D (2016) Learning-by-teaching. Evidence and implications as a pedagogical mechanism. Innov Educ Teach Int 54(5):476–484. https://doi.org/10.1080/14703297.2016.1156011 71. Chan T-W Chou C-Y (1997) Exploring the design of computer supports for reciprocal tutoring. Int J Artif Intell Educ 8(1):1–29

78

J. White and B. du Boulay

72. Uresti JAR, du Boulay B (2004) Expertise, motivation and teaching in learning companion systems. Int J Artif Intell Educ 14(2):193–231 73. Biswas G et al (2005) Learning by teaching: a new agent paradigm for educational software. Appl Artif Intell 19(3–4):363–392. https://doi.org/10.1080/08839510590910200 74. Leelawong K, Biswas G (2008) Designing learning by teaching agents: the Betty’s Brain system. Int J Artif Intell Educ 18(3):181–208 75. Munshi A et al. (2022) Analysing adaptive scaffolds that help students develop self-regulated learning behaviours. J Comput Assist Learn. https://doi.org/10.1111/jcal.12761 76. Bull S (2020) There are open learner models about! IEEE Trans Learn Technol 13(2):425–448. https://doi.org/10.1109/tlt.2020.2978473 77. Viberg O, Khalil M, Baars M (2020) Self-regulated learning and learning analytics in online learning environments. In: Proceedings of the tenth international conference on learning analytics & knowledge, pp 524–533 78. Matcha W et al (2020) A Systematic review of empirical studies on learning analytics dashboards: a self-regulated learning perspective. IEEE Trans Learn Technol 13(2):226–245. https://doi.org/10.1109/tlt.2019.2916802 79. Molenaar I et al (2019) Designing dashboards to support learners’ self-regulated learning. In: Companion proceedings 9th international conference on learning analytics & knowledge (LAK19) 80. Molenaar I (2022) The concept of hybrid human-AI regulation: exemplifying how to support young learners’ self-regulated learning. Comput Educ: Artif Intell 3.https://doi.org/10.1016/j. caeai.2022.100070 81. Rusyati L et al (2021) The interconnection between students’ meta-affective, meta-cognitive and achievement in science learning. In: 5th Asian education symposium 2020 (AES 2020) 82. Boekaerts M (2007) Understanding students’ affective processes in the classroom. In: Schutz PA, Pekrun R (eds) Emotion in education. Acadmic Press, Burlington, MA, pp 37–56 83. Rebolledo-Mendez G et al (2021) Meta-affective behaviour within an intelligent tutoring system for mathematics. Int J Artif Intell Educ 32:174–195. https://doi.org/10.1007/s40593-02100247-1 84. Viberg O, Kukulska-Hulme A, Peeters W (2023) Affective support for self-regulation in mobile-assisted language learning. Int J Mob Blended Learn 15(2):1–15.https://doi.org/10. 4018/ijmbl.318226 85. Wagstaff CR (2014) Emotion regulation and sport performance. J Sport Exerc Psychol 36(4):401–412. https://doi.org/10.1123/jsep.2013-0257 86. Arroyo I, Porayska-Pomsta K, Muldner K (2023) Theories of affect, meta-affect, and affective pedagogy. In: du Boulay B, Mitrovic A, Yacef K (eds) Handbook of artificial intelligence in education. Edward Elgar Publishing, Cheltenham 87. D’Mello SK et al (2013) Affect, meta-affect, and affect regulation during complex learning. In: Azevedo R, Aleven V (eds) International handbook of metacognition and learning technologies. Springer, New York, pp 669–681 88. Goldin GA (2000) Affective pathways and representation in mathematical problem solving. Math Think Learn 2(3):209–219. https://doi.org/10.1207/S15327833MTL0203_3 89. Dias J et al (2006) Empathic characters in computer-based personal and social education. In: Pivec M (ed) Affective and emotional aspects of human-computer interaction: game-based and innovative learning approaches. IOS Press, Amsterdam, pp 246–254 90. Marin BF, Hunger A, Werner S (2006) Corroborating emotion theory with role theory and agent technology: a framework for designing emotional agents as tutoring entities. J Netw 1(4):29–40 91. Arroyo I et al (2014) A multimedia adaptive tutoring system for mathematics that addresses cognition, metacognition and affect. Int J Artif Intell Educ 24(4):387–426. https://doi.org/10. 1007/s40593-014-0023-y 92. D’Mello SK, Lehman B, Graesser A (2011) A motivationally supportive affect-sensitive auto tutor. In: Calvo RA, D’Mello SK (eds) New perpectives on affect and learning technologies. Springer, New York, pp 113–126

5 Supporting Learners’ Metacognition and Meta-Affect

79

93. Yussof MZ, du Boulay B (2010) A tutoring system using an emotion-focused strategy to support learners. In: Proceedings of the 18th international conference on computers in education (ICCE 2010). Putrajaya, Malaysia

6

Understanding Consciousness in the Age of AI and XR: Altered States, Emerging Realities, and the Digital Self Nicola De Pisapia

Abstract

This chapter explores the impact of Artificial Intelligence (AI) and Extended Reality (XR) on our understanding of consciousness and self-identity. It examines the role of these technologies in extending the dynamic nature of identity as shaped by cognitive processes, subjective experiences and their underling neural processes. The concept of the self is reevaluated in the context of digital interactions, particularly focusing on how interfacing with AI and virtual experiences contribute to identity formation. The chapter also addresses the potential of AI and XR to mimic and induce altered states of consciousness, offering new insights into the plasticity and richness of human experience. Finally, it discusses the implications of these technologies for brain organization and the perception of reality, highlighting the necessity of balancing technological advancements with deep ethical considerations.

1

Consciousness, the Sense of Self and Brain Organization

Understanding consciousness, traditionally a focal point in cognitive neuroscience, psychology, and philosophy, has become increasingly pertinent with the emergence of advanced digital technologies like Artificial Intelligence (AI) and Extended Reality (XR), encompassing Virtual Reality (VR), Augmented reality (AR), and Mixed Reality (MR). These technologies provide new dimensions to our understanding of consciousness, incorporating novel phenomena and methods of exploration [1].

N. De Pisapia (B) Department of Psychology and Cognitive Science, University of Trento, Trento, Italy e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 F. Santoianni et al. (eds.), Mind, Body, and Digital Brains, Integrated Science 20, https://doi.org/10.1007/978-3-031-58363-6_6

81

82

N. De Pisapia

Consciousness can be defined as the state or quality of awareness, or of being aware of an external object or something within oneself. It refers to the subjective experience of the mind and the world. This includes a sense of self-awareness, the experience of sensations, thoughts, and emotions, and the ability to perceive and interact with one’s environment. The term consciousness is well-articulated in the works of David Chalmers [2], who adds depth to this understanding by distinguishing the easy and hard problems of consciousness. The easy problems, despite their name, are by no means trivial. They encompass the ways in which the brain processes information, resulting in various cognitive functions and behaviors. These problems are deemed easy because they are amenable to standard scientific methods. Researchers can, for instance, observe and measure the brain’s response to stimuli, or examine the neural correlates of specific cognitive tasks, allowing for theories to be tested against observable phenomena. In this context, the comprehensive review by Seth and Bayne [3] offers further insights into the diverse theoretical approaches to understanding these aspects of consciousness. In contrast, the hard problem of consciousness delves into a more profound and, in many ways, more perplexing area. It concerns the subjective experience, often referred to as qualia—the introspective, first-person aspects of conscious experience. Why and how does processing of information in the brain lead to an inner, subjective experience? Why does it feel like something to see a color or experience pain? This problem is hard not only because it resists a straightforward empirical approach, but also because it raises deep philosophical questions about the nature of experience and the mind-body relationship. It challenges us to explain why certain physical processes in the brain give rise to conscious experience at all (the explanatory gap). The scientific understanding of identity and self is intricately linked to the study of consciousness. Identity and self are dynamic constructs, continuously shaped by cognitive processes, memories, and subjective experiences. In neuroscience, the self is seen through functions like autobiographical memory, which helps construct a personal narrative, and executive functions, essential for maintaining a consistent sense of self through the integration of diverse information and adaptation to new situations. Consciousness provides the backdrop against which the self is drawn, encompassing not only awareness of the external world but also the internal narrative of thoughts, feelings, beliefs, and intentions. This narrative forms the core of our subjective experience of identity [4]. Gallagher [5] presents the self as a multi-layered construct, extending from minimal self-awareness to a complex narrative self-concept, deeply intertwined with consciousness and reliant on autobiographical memory and future planning. The sense of identity, a pivotal aspect of human consciousness, is characterized by several core properties grounded in psychological research. Firstly, identity is multifaceted; it encompasses personal identity (our self-concept, including traits and values) and social identity (our identification with groups), as elucidated by Erikson’s theory of psychosocial development [6]. Secondly, identity is dynamic and malleable, continuously shaped and reshaped by experiences, social interactions,

6 Understanding Consciousness in the Age of AI and XR: Altered States …

83

and personal introspection, a concept central to James Marcia’s identity status theory [7]. Thirdly, identity involves a narrative component, where individuals construct a cohesive life story or narrative identity, integrating past experiences and future aspirations, as described by McAdams and McLean [8]. Lastly, the sense of identity incorporates the dimension of continuity and coherence, as people perceive a persistent self across time despite various changes, a phenomenon explored by Lifton [9] and his idea of a protean self . These properties underscore the complexity of identity as a construct, highlighting its fluid yet structured nature, deeply interwoven with cognitive and social processes. In the seminal work by Belk [10], the concept of the extended self was introduced, positing that our belongings significantly contribute to and reflect our identities. This perspective suggests that possessions are not merely material assets, but integral to our self-perception, and crucial in forming and expressing our identities. This idea has profound implications for understanding consumer behavior, as it underscores the deep-seated connection between our belongings and our sense of self. The concept of self-extension is observable in various aspects of our lives. Tools, musical instruments, clothing, vehicles, homes, and less tangible elements like the type of places we frequent, the people we associate with, and the literature we engage with, all play significant roles in defining our identity. These items and associations are not mere possessions or acquaintances; they are essential components that constitute our identity. Building on this idea, an intriguing question arises: in an increasingly digital world, how are advanced digital interactions, such as social media, VR, and other forms of digital engagement, further extending our sense of self? How do these virtual experiences and digital footprints contribute to the mosaic of our identity, and what implications does this have for our understanding of the sense of self in the digital age? Furthermore, the exploration of consciousness and identity traditionally focuses on ordinary states, but there are important additional dimensions to this inquiry: Altered States of Consciousness (ASC). These states, which vary significantly from our normal waking experience, are characterized by distinct changes in perception, emotion, cognition, and sense of self. Commonly experienced by everyone in various forms such as dreams, dreamless sleep, hypnotic trance, meditative states, or under the influence of psychedelic substances, and even through technological means (such as sound or video stimulations), ASC are pivotal in understanding consciousness and the construction of identity [11, 12]. The significance of ASC lies in their ability to offer a broader perspective on the plasticity and malleability of consciousness and identity, illustrating that our usual perception of self is but one aspect of a much more dynamic and multifaceted spectrum of human experience. AI and XR technologies hold the potential to not only mimic, but also induce ASC, opening new avenues for research and raising intriguing questions about their effects on ordinary users. They enable controlled simulations of altered perceptual and cognitive states, providing researchers with tools to systematically investigate ASC in a manner that was previously unattainable. This raises the question: how

84

N. De Pisapia

effectively can AI and XR be employed to replicate, induce, or study ASC? Moreover, what insights might these technologies offer into the plasticity and variability of human consciousness? In the context of ordinary consumers, the widespread availability and use of these technologies necessitate an examination of their potential impact. How do AI-driven interfaces and XR environments affect the everyday user’s perception, cognition, and sense of self? Are these technologies merely extending the realm of virtual experiences, or are they inducing subtle changes in the user’s state of consciousness, akin to those observed in known ASC? And importantly, what are the implications of these technologically induced altered states for our understanding of identity and self in an increasingly digital world? These questions into the technological augmentation of human experience serve as bridges to a more foundational aspect of consciousness studies: the role of brain organization in shaping our conscious experience and sense of self. Neuroscientific research establishes that consciousness and the sense of self are deeply intertwined with brain organization. It is a basic assumption of neuroscience that every sensation, perception, and subjective experience is rooted in the brain’s neural networks. Functional neuroimaging studies, such as those by Dehaene and Changeux [13], have implicated a network of brain regions, including the prefrontal cortex, posterior parietal cortex, and thalamus, in generating conscious experience. The precise mechanisms and neural circuits underlying conscious awareness are topics of ongoing investigation [3], and mostly focused on the easy problems of consciousness, while leaving the hard problems aside. Concerning the neural basis of the self and the sense of identity, the defaultmode network, as originally characterized by Raichle et al. [14], is interpreted as playing a crucial role in maintaining the sense of self. This brain network, comprising regions like the medial prefrontal cortex and posterior cingulate cortex [15], is active during rest and involved in self-referential thinking and mind-wandering, essential aspects of self and identity processing [16]. ASC are often explained as variations in the brain’s level of order in such networks. The Entropic Brain Hypothesis, proposed by Carhart-Harris et al. [17], describes the richness of conscious states as depending on the system’s entropy. In this model, higher entropy states, such as those experienced during psychedelic drug use, are associated with a broader range of potential neural configurations and altered perceptions of consciousness. This hypothesis, extended by recent studies, presents a continuum of conscious states from highly ordered (low entropy) to highly disordered (high entropy), offering a neurobiological basis for understanding ASC. The advent of AI and XR technologies poses critical questions about their impact on these fundamental aspects of the human mind and brain networks’ organization. Do these technologies affect consciousness, interact with or modify the sense of identity, and induce ASC, affecting brain’s functioning? We propose affirmative responses to these questions, as discussed in the rest of the chapter,

6 Understanding Consciousness in the Age of AI and XR: Altered States …

85

and indicate that—as these technological devices become more and more sophisticated—a profound shift is taking place in our understanding and experience of consciousness and identity in the digital age.

2

The Potential of AI and XR Technologies

The contemporary exploration of consciousness has the potential to enter in a new era with the advent of AI and XR technologies. These innovations are not merely technological advancements; they can represent a paradigm shift in how we understand, interact with, and experience consciousness and self-identity [18]. AI, particularly with its recent strides towards Artificial General Intelligence (AGI)—a form of AI capable of understanding, learning, and applying its intelligence broadly and flexibly, akin to human cognitive abilities—challenges our conventional understanding of consciousness. Unlike humans, we can safely assume that current AI operates without subjective experiences, yet it exhibits behaviors that mimic human cognitive and neural processes. The apparent neural mimicry, as explored in Hassabis et al. [19], raises profound questions about the nature and requirements of consciousness and its neural bases. AI’s capabilities, especially in natural language processing and decision-making, as exemplified in systems like GPT [20], are increasingly sophisticated, blurring the distinction between human and machine intelligence, and possibly also human and machine consciousness [21]. The influence of digital on human identity is—in this technological context— multifaceted and deeply transformative. Turkle [22] argues that our interactions with digital systems reshape our self-perception, creating a new form of digital identity. This digital identity as defined here emerges from the personalization algorithms of AI, which tailor interactions and experiences, leading individuals to perceive these digital entities as extensions of their own self. Such a phenomenon suggests a redefinition of identity in the digital age, where the self is increasingly a blend of organic and digital experiences. The merging of AI and XR presents groundbreaking possibilities in consciousness research. AI’s potential to simulate complex cognitive processes, combined with XR’s immersive capabilities, could lead to the creation of hyper-realistic simulations of conscious experiences. This convergence offers a novel approach to studying consciousness, allowing researchers to replicate and examine conscious experiences in controlled environments. An intriguing application of this convergence is in the study of phenomena like lucid dreaming. Lucid dreaming, an ASC where one is aware of dreaming and can exert control over the dream [23], offers a unique window into consciousness. The integration of AI’s cognitive simulation with XR’s immersive environments could simulate lucid dreaming experiences, providing researchers a novel platform to study this complex phenomenon, for example increasing the ability of dreamers to acquire lucidity during dreaming. Such simulations could offer insights into the

86

N. De Pisapia

mechanisms of consciousness and the boundaries between waking and dreaming states. In the context of understanding consciousness and reality, the works of the cognitive scientist Donald Hoffman offer a pivotal insight. Hoffman posits that what we perceive as reality is not an objective truth, but rather a subjective construct created by our nervous system for survival purposes [24]. This perspective aligns with the broader neuroscience consensus that our senses and brain processes do not reproduce the external world as it is, but rather filter and interpret it to form a functional reality conducive to survival. In the context of current AI and XR technologies, this concept takes on a new dimension. These advanced systems, with their ability to augment, simulate, and even manipulate sensory inputs, challenge and expand our traditional notions of reality. They create immersive experiences that, while perceptually convincing, are distinct from the physical world. This raises compelling questions about the evolution of our sense of reality in the digital age. How will our nervous system and consciousness adapt to and integrate these artificially constructed realities? And more importantly, how will this affect our understanding and interaction with what we consider the “real” world? These technologies, as Zolla [25] already recognized, have the potential to be more than mere tools for entertainment or escapism. They could be transformative in enhancing our understanding and experience of the world, potentially leading to a deeper realization of the nature of reality and the self. The integration of AI and XR technologies into our sensory and cognitive processes has the potential to redefine our perception of reality, blurring the lines between the physical and the digitally constructed worlds, and challenging our evolutionary-developed mechanisms of perception.

3

The Emergence of the Extended Digital Self

The rapid advancements in AI and XR technologies are not only reshaping our understanding of consciousness, but also strengthening the digital self as a novel and more expanded form of identity. This emerging concept reflects how our interactions with digital technologies continuously shape and reshape our identities, profoundly influencing our consciousness and, consequently, our perception of reality. Digital interactions and interfaces have evolved radically through the years, with a transition from basic input methods to advanced digital personas enabled by AI. Digital humans through enhanced AI models personalize responses and learn individual preferences, leading to personal assistants that can increasingly take on cognitive tasks on behalf of users. In the current phase, digital interactions transcend single-channel communications, like text or speech, by integrating richer omnichannel models. This involves combining speech and textual input with gestures, facial expressions, tonal evaluation, sentiment analysis, context evaluation, and intent extraction, which provide systems with a more integrated understanding of human interactions beyond mere words. These inputs are processed through

6 Understanding Consciousness in the Age of AI and XR: Altered States …

87

AI models and applications to generate highly personalized, contextual responses for each user, effectively extending their capabilities into the digital realm. Lifelike digital humans, with synthesized voices indistinguishable from human voices, coupled with realistic facial or full-body avatars, create immersive experiences. This technology finds significant application in the gaming industry, where recreating and giving agency to digital humans aims to replicate physical presence and express identity in virtual worlds [26]. The extended digital self is therefore emerging, as an individual’s online persona, an identity sculpted through interactions with digital technologies, ranging from social media profiles to avatars in virtual worlds. But this self is not just confined to the digital world, whereas it evolves in a sense of identity that coexists and integrates with our physical being. These digital identities are dynamic, continually influenced by our digital experiences and interactions. They can even evolve independently of our physical selves, presenting unique challenges and opportunities in terms of privacy, authenticity, and identity construction, as discussed by Turkle [22]. The technology’s ability to blur the lines between real and synthetic behavior raises concerns about decision-making processes and the actions of these digital entities, as long a psychological body-centered self is still active in the nervous system. This dynamic nature is particularly evident in the context of AI and XR technologies. For instance, AI can create dynamic avatars that adapt and evolve based on interactions within their digital environment. Such advancements are highlighted in the work of Bailenson [27], who explores how these AI-generated entities can exhibit complex behaviors and personalities, further blurring the line between digital and physical identities. Belk [28] delves into the concept of the extended self in the context of the digital age. He explores how our identities have expanded beyond the physical world to include digital realms. Key themes include the dematerialization of possessions into digital forms, the re-embodiment of self through avatars and online representations, and the co-construction of self, where our digital identity is shaped by interactions with others online. According to him, our use of digital media, such as social media, email, blogging, online dating, and digital video games, leads to a form of disembodiment compared to face-to-face interactions. However, in many of these digital contexts, we are re-embodied through avatars, photos, or videos. Belk notes that these online representations do not always accurately or honestly reflect our physical selves. There is a tendency to present idealized or fantasized versions of ourselves rather than our actual selves. Despite this, activities on social networking sites can reveal true personality characteristics. The use of pseudonyms online also makes it easier for individuals to explore and express new aspects of their identity, such as gender, before doing so offline. The digital era has brought about two key changes in the notion of the extended self: the core self is no longer seen as singular, and the physical body’s role is diminished through the use of avatars and other visual re-embodiments. Belk discusses the different perspectives in virtual worlds and video games, noting that controlling an avatar can create a sense of embodiment and telepresence.

88

4

N. De Pisapia

Extended Digital Selves, States of Consciousness and Brains

Our interactions with digital technologies continuously shape and reshape our identities, profoundly influencing our consciousness and, as a consequence, our perception of reality. At the forefront of this evolution are the intricate changes occurring in our brain, particularly within the already mentioned default-mode network. This brain network, including key areas such as the medial prefrontal cortex and posterior cingulate cortex, is known to be active during restful introspection and is closely linked to the construction of our selves [15, 16]. The engagement with digital interfaces and environments has the potential to drive cognition and neural adaptations, thus influencing the way our brains process information and perceive reality, as suggested by research in brain plasticity. For example, in recent studies the Internet’s impact on cognitive behavior has been linked to changes in brain structure and function. Specifically, the shift towards shallow learning, characterized by rapid scanning and minimal deep thought and influenced by easy online information access and hypertext use, is thought to potentially affect brain circuitry involved in deep reading and contemplation. Neural adaptations in response to Internet use, especially in relation to attempts to multitasking and media-multitasking, may influence executive control and attention networks, and therefore prefrontal cortex areas. Internet addiction has been associated with altered brain networks related to self-control and reward-processing [29]. Additionally, the integration of the brain with computational systems through Brain-Computer Interfaces (BCIs) is a significant leap toward a more digitally interconnected existence. BCIs, facilitating direct communication pathways between the brain and external devices, are not only promising for therapeutic applications, such as restoring functions in cases of stroke [30], but also point towards a future where human cognition might be deeply integrated with AI. This integration holds the potential to expand our cognitive abilities and alter our consciousness in the direction of creating fully functioning cyborgs [31]. Further, AI and XR technologies are uniquely positioned to induce ASC. These technologies, through immersive environments, have the capacity to significantly impact perception, emotion, and cognition. Studies have shown that VR, for instance, can induce states akin to meditation or hypnosis, thereby offering therapeutic benefits like stress reduction, and enhancing personal and clinical change [32]. Such experiences, facilitated by AI and XR, could potentially expand our sense of identity, providing controlled environments for therapeutic and mind-expanding purposes. In one of our studies [33], we provided empirical evidence of how VR can induce changes in perception, emotion, and cognition, akin to naturally occurring ASC. We investigated the impact of DeepDream-generated hallucinatory VR videos on cognitive flexibility. We found that exposure to altered perceptual phenomenology via these simulated psychedelic experiences enhances cognitive flexibility. This is demonstrated through changes in participants’ decision-making

6 Understanding Consciousness in the Age of AI and XR: Altered States …

89

processes, characterized by reduced reliance on automatic responses and more chaotic dynamics. The study suggests that such virtual experiences reorganize cognitive dynamics, encouraging exploration of novel decision strategies and inhibiting automated choices. This implies that VR-induced altered states can positively affect cognitive adaptability in a way similar to the ASC induced by actual psychedelic drugs. Concerning AI and ASC, recent advancements have opened new possibilities. AI’s ability to analyze and interpret neural patterns allows for the de-codification and creation of personalized mental states. This can be achieved through AI-driven biofeedback systems, which monitor physiological markers (like brain waves and heart rate) and provide real-time adjustments to induce specific states of consciousness. Additionally, AI’s role in creating immersive, responsive environments, such as those used in meditation apps, can guide users into targeted ASC. These technologies, leveraging AI’s predictive and adaptive capabilities, create unique opportunities for exploring and enhancing human consciousness. In one such recent study [34], the authors developed a metric called the Explainable Consciousness Indicator (ECI) using deep learning to quantify arousal and awareness in ASC. This approach utilized convolutional neural networks to process time-series Electroencephalography (EEG) data, allowing the differentiation between various states of consciousness under physiological, pharmacological, and pathological conditions. Notably, it was effective in distinguishing between states like Rapid Eye Movement (REM) sleep and general anesthesia, highlighting the parietal regions of the brain as key areas for quantifying awareness and arousal in ASC. The ECI could be used to create personalized neurofeedback systems. By monitoring an individual’s brain activity in real time, AI-driven systems could provide feedback or adjust stimuli to guide the user into specific ASC. This could be used for therapeutic purposes, such as in treating mental health conditions, or for enhancing relaxation, sleep, meditative states or cognitive functions like creativity. Furthermore, combining ECI with VR technologies could lead to immersive experiences that dynamically adjust based on the user’s state of consciousness. This could revolutionize entertainment, therapy, and training simulations by providing highly personalized and responsive environments. The convergence of AI, XR, and brain sciences suggests a not-too-distant future where the boundaries of self and identity may expand beyond traditional limits. By blending our physical and digital experiences through the biology of our nervous system, these technologies offer novel avenues for exploring and understanding consciousness. While this expansion offers therapeutic benefits, it also poses profound philosophical questions about the nature of selfhood and human identity in a digitally integrated world.

90

5

N. De Pisapia

Looking Ahead: Future Implications and Developments

The integration of AI and XR technologies into the fabric of human consciousness and identity is leading us into a future rich with possibilities but also fraught with complex ethical considerations. This integration signifies a true paradigm shift, affecting both our scientific comprehension of consciousness and the direct experience of self and reality by each user of these technologies. AI and XR technologies have the potential to bring profound benefits across various aspects of human life. AI’s capacity to enhance cognitive abilities and support decision-making processes can revolutionize learning, working, and communication. XR’s immersive experiences offer new ways to transcend physical limitations, as reviewed in Slater and Sanchez-Vives [35]. These technologies, applied in the domain of consciousness, could lead to novel subjective experiences, expanding the horizons of our understanding of reality. However, alongside these advancements, significant challenges emerge. One of the primary concerns is the authenticity of digital experiences. As we delve deeper into virtual worlds and interact more with AI-driven entities, distinguishing between real and virtual experiences becomes increasingly complex. This blurring of lines raises questions about the nature of experience and reality itself. Experiences in virtual worlds are as valid and genuine as in physical reality, not merely illusory or secondary, challenging conventional ideas about what constitutes reality [21]. Privacy and security of personal data in the digital realm is another critical concern. With the increasing sophistication of AI and XR, personal data becomes more vulnerable to misuse. The ethical considerations surrounding the use of these technologies, as discussed for example by Brey [36] concerning VR, are becoming more crucial. These issues encompass not just the privacy and security of data but also broader ethical dilemmas around the influence of these technologies on human behavior and society. In the exploration of ethical considerations surrounding AI and XR technologies, we need to examine the psychological and behavioral impacts, highlighting the potential to profoundly manipulate and alter human psychological states in unexplored ways. For example, the immersive and influential nature of VR raises significant ethical concerns, particularly in relation to the user’s psychological well-being and autonomy. The prevention of long-term psychological harm and the potential for VR to exploit the vulnerabilities of the human psyche are actual risks, and it is the responsibility of VR developers and researchers to ensure that these technologies do not inadvertently cause psychological distress or manipulate users in unethical ways [37]. Among the pathological undesirable consequence on mental health of identity associated with XR is Depersonalization/Derealization Disorder. This disorder involves a persistent feeling of detachment from one’s body and thoughts (depersonalization) and a sense of unreality about the external world (derealization) [38]. The concern is that prolonged or intense use of VR technology could exacerbate

6 Understanding Consciousness in the Age of AI and XR: Altered States …

91

or contribute to these conditions, given VR’s immersive nature and its capacity to blur the lines between the physical and digital reality [39]. As we navigate these emerging landscapes, a multidisciplinary approach becomes essential. Cognitive neuroscience offers insights into how AI and XR technologies impact our brains and consciousness, providing a scientific basis for understanding their effects. At the same time, philosophy and ethics are vital in guiding the responsible use of these technologies. Furthermore, as we venture into uncharted territories of the possibility of digital consciousness [21], ethical frameworks grounded in human values such as human connection, self-preservation, and nature preservation become imperative, while still allowing technological progress. Societal engagement and policy-making play a crucial role in shaping the future of AI and XR technologies. As these technologies become more integrated into everyday life, public discourse and policy decisions must reflect a balance between technological advancement and ethical responsibility. This balance is crucial to ensure that these technologies enhance human experiences without compromising fundamental human values. The implications of AI and XR on human consciousness, identity, and reality perception are starting to be profound. We are entering an era characterized by a dynamic interplay among human consciousness, AI and XR technologies, and the evolving concept of an extended digital self. This interaction is redefining our perception and understanding of reality. Embracing this intersection offers a promising pathway to not only understand, but also shape our evolving realities. The public document “Digital Futures Final Report” by the European Commission’s DG CONNECT, published in 2016, explored long-term technological impacts and policy challenges for Europe up to 2050. Regarding identity, the report states that human enhancement technologies challenge traditional notions of human identity and what it means to be human—in accord with trans-humanism [40]. The document raises concerns about creating a new normative view of humans, potentially leading to the stigmatization of those who do not conform to this norm. Eight years after the publication of that report, we are well into that process of transformation. Our hope is that such trajectory takes place venturing into the realm of eudemonia, namely as a pursuit of wisdom and a fulfilling life. The fusion of AI and XR with human consciousness has the potential of not only reshaping our understanding of what it means to be human, but also guiding us towards a positive and profound exploration of our inner selves, as advocated in the so-called positive technology [41]. This wisdom should not only deepen our understanding of sensory experiences and the nature of reality, but also enrich our spiritual and existential insights. It invites us to ponder profoundly on our identity, our place within the cosmos, and the intricate dynamics between consciousness and reality. However, it is crucial to acknowledge that the path towards harnessing this rapidly evolving technological landscape for the enhancement of human wellbeing is neither inevitable nor straightforward. The future might unfold in unexpected ways due to unforeseen

92

N. De Pisapia

events or deliberate choices. Conversely, to tilt the scales towards a favorable outcome, it is imperative that we collectively embrace a eudemonic vision of personal growth. This approach should not be seen merely as one among many alternatives, but as a vital trajectory that harmoniously integrates technological progress with universal prosperity, environmental stewardship, and the safeguarding of our fundamental human essence.

References 1. Metzinger TK (2018) Why is virtual reality interesting for philosophers? Fronti Robot AI 5:101 2. Chalmers DJ (1997). The conscious mind: In search of a fundamental theory. Oxford Paperbacks 3. Seth AK, Bayne T (2022) Theories of consciousness. Nat Rev Neurosci 23(7):439–452 4. Liu J, Perry J (Eds) (2011) Consciousness and the self: New essays Cambridge University Press 5. Gallagher S (2000) Philosophical conceptions of the self: implications for cognitive science. Trends Cogn Sci 4(1):14–21 6. Erikson EH (1968) Identity youth and crisis. WW Norton & Company 7. Marcia JE (1980) Identity in adolescence. Handbook of Adolescent Psychol 9(11):159–187 8. McAdams DP, McLean KC (2013) Narrative identity. Curr Dir Psychol Sci 22(3):233–238 9. Lifton RJ (1999) The protean self: Human resilience in an age of fragmentation. University of Chicago Press 10. Belk RW (1988) Possessions and the extended self. J Consumer Res 15(2):139–168 11. Fromm E (1977) An ego-psychological theory of altered states of consciousness. Int J Clin Exp Hypn 25(4):372–387 12. Vaitl D, Birbaumer N, Gruzelier J, Jamieson G, Kotchoubey B, Kübler A, Weiss T (2005) Psychobiology of altered states of consciousness. Psychol Bull 131(1):98–127 13. Dehaene S, Changeux J-P (2011) Experimental and theoretical approaches to conscious processing. Neuron 70(2):200–227 14. Raichle ME, MacLeod AM, Snyder AZ, Powers WJ, Gusnard DA, Shulman GL (2001) A default mode of brain function. Proc Natl Acad Sci 98(2):676–682 15. De Pisapia N, Barchiesi G, Jovicich J, Cattaneo L (2019) The role of medial prefrontal cortex in processing emotional self-referential information: a combined TMS/fMRI study. Brain Imaging Behav 13:603–614 16. De Pisapia N (2010) Spontaneous brain activity and the self. Proc Italian Assoc Cognit Sci (AISC) 2010:256–260 17. Carhart-Harris RL, Leech R, Hellyer PJ, Shanahan M, Feilding A, Tagliazucchi E, Nutt DJ (2014) The entropic brain: a theory of conscious states informed by neuroimaging research with psychedelic drugs. Front Hum Neurosci 8:20 18. Floridi L (2015) The fourth revolution in our self-understanding. In: Philosophy, computing and information science, pp 19–27. Routledge 19. Hassabis D, Kumaran D, Summerfield C, Botvinick M (2017) Neurosc Inspired Artif Intell Neuron 95(2):245–258 20. Brown T, Mann B, Ryder N, Subbiah M, Kaplan JD, Dhariwal P, Amodei D (2020) Language models are few-shot learners. Adv Neural Inf Process Syst 33:1877–1901 21. Chalmers DJ (2023). Could a large language model be conscious? arXiv preprint arXiv:230307103 22. Turkle S (2011) Alone together: why we expect more from technology and less from each other. Basic Books

6 Understanding Consciousness in the Age of AI and XR: Altered States …

93

23. De Pisapia N (2021) Lucid dreaming: from phenomenology to the neurobiological research. G Ital Psicol 48(1):187–218 24. Hoffman D (2019). The case against reality: why evolution hid the truth from our eyes. WW Norton & Company 25. Zolla E (1995) Uscite dal mondo. Adelphi Ed 26. Campbell M, Jovanovic M (2022) Digital self: the next evolution of the digital human. Computer 55(04):82–86 27. Bailenson J (2018). Experience on demand: What virtual reality is, how it works, and what it can do. WW Norton & Company 28. Belk R (2016) Extended self and the digital world. Curr Opin Psychol 10:50–54 29. Loh KK, Kanai R (2016) How has the Internet reshaped human cognition? Neuroscientist 22(5):506–520 30. Mane R, Chouhan T, Guan C (2020) BCI for stroke rehabilitation: motor and beyond. J Neural Eng 17(4):041001 31. Barfield W, Williams A (2017) Cyborgs and enhancement technology. Philosophies 2(1):4 32. Riva G, Baños RM, Botella C, Mantovani F, Gaggioli A (2016) Transforming experience: the potential of augmented reality and virtual reality for enhancing personal and clinical change. Front Psych 7:164 33. Rastelli C, Greco A, Kenett YN, Finocchiaro C, De Pisapia N (2022) Simulated visual hallucinations in virtual reality enhance cognitive flexibility. Sci Rep 12(1):4027 34. Lee M, Sanz LR, Barra A, Wolff A, Nieminen JO, Boly M, Lee SW (2022) Quantifying arousal and awareness in altered states of consciousness using interpretable deep learning. Nat Commun 13(1):1064 35. Slater M, Sanchez-Vives MV (2016) Enhancing our lives with immersive virtual reality. Front Robot AI 3:74 36. Brey P (2018) The ethics of emerging virtual reality technologies. Oxford University Press, In Oxford Handbook of Ethics of AI 37. Madary M, Metzinger TK (2016) Real virtuality: a code of ethical conduct. Recommendations for good scientific practice and the consumers of VR-technology. Front Robot AI 3: 3 38. Peckmann C, Kannen K, Pensel MC, Lux S, Philipsen A, Braun N (2022) Virtual reality induces symptoms of depersonalization and derealization: a longitudinal randomised control trial. Comput Hum Behav 131:107233 39. Spiegel JS (2018) The ethics of virtual reality technology: social hazards and public policy recommendations. Sci Eng Ethics 24(5):1537–1550 40. Bostrom N (2005) A history of transhumanist thought. J Evolut Technol 14(1) 41. Riva G, Baños RM, Botella C, Wiederhold BK, Gaggioli A (2012) Positive technology: using interactive technologies to promote positive functioning. Cyberpsychol Behav Soc Netw 15(2):69–77

7

Learner Modeling Interpretability and Explainability in Intelligent Adaptive Systems Diego Zapata-Rivera and Burcu Arslan

Abstract

Learner models are used to support the implementation of personalization features in Adaptive Instructional Systems (AISs; e.g., adaptive sequencing of activities, adaptive feedback), which are important aspects of Intelligent Adaptive Systems. With the increased computational power, more advanced methodologies, and more available data, learner models include a variety of Artificial Intelligence techniques. These techniques have different levels of complexity, which influence interpretability and explainability of learner models. Interpretable and explainable learner models can facilitate appropriate use of the learner modeling information in AISs, their adoption, and scalability. This chapter elaborates on the definitions of interpretability and explainability, describes interpretability and explainability levels of different models, elaborates on the levels of explainability to produce needed information for teachers and learners, and discusses implications and future work in this aera.

1

Introduction

Learner models are representations of learners’ knowledge, skills and other attributes that are used by Adaptive Instructional Systems (AIS) to support personalization. Personalization features may involve adaptive selection of activities and adaptive feedback. Learner models can include information about learners’

D. Zapata-Rivera (B) ETS, Princeton, NJ 08541, USA e-mail: [email protected] B. Arslan ETS Global, 1077 XX Amsterdam, The Netherlands © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 F. Santoianni et al. (eds.), Mind, Body, and Digital Brains, Integrated Science 20, https://doi.org/10.1007/978-3-031-58363-6_7

95

96

D. Zapata-Rivera and B. Arslan

cognitive and non-cognitive skills [28, 62]. Learner models can be produced based on information gathered before learner’s interactions with the AIS and refined as more information about the learner is collected as learners interact with the AIS [62]. Learners and other educational stakeholders can have access to learner model information [15, 76]. Types of interaction with the learner model and purposes for accessing have been studied in the area of Open Learner Modeling (OLM) [77]. OLM approaches include the design and use of interfaces to facilitate interaction with learner model information. These interactions can include guidance mechanisms (e.g., interaction scripts, negotiation approaches) and guidance provided by pedagogical agents or other humans. Communicating learning model information to teachers and learners requires knowing their needs for assessment information, and the evidence available to support learner model claims. A learner model that supports the generation of explanations for various types of end users can facilitate this process. Different types of learner modeling approaches (i.e., top-down, bottom-up, hybrid) exhibit different affordances and challenges for the generation of explanations for various types of end users [78]. Interpretability and explainability of learner models are key concerns in learning and assessment contexts, especially considering that making inferences from learning and assessment data is challenging due to different sources of noise in learning and assessment data [37]. For example, teachers and students may want to know how information maintained by the system is used to support student learning. Also, students may be interested in knowing more about how the system calculates their progress. Interpretability and explainability can facilitate the adoption of AISs since trust in these systems may increase as teachers and learners better understand why recommendations and decisions by the system are made [19, 37, 39, 71, 79]. For example, a learner model that supports interpretability and explainability can be deployed to justify the suggested adaptive sequencing of activities and/or the information presented in dashboards for teachers and learners as well as to explain the underlying mechanisms of pedagogical agents’ interactions (e.g., triggering interactions based on the status of cognitive and non-cognitive skills). In this chapter, we define interpretability and explainability with respect to the transparency of the models and the explanation generated to the end users (e.g., learners and teachers), classify the models in terms of their interpretability and explainability, and discuss explainability to teachers and learners. We conclude with discussing the implications and future directions.

2

Interpretability and Explainability

With the increased computational power, more advanced methodologies, and more available data, more Adaptive Instructional Systems (AISs) can now make use of an AI technique called Machine Learning (ML) to make predictions, decisionmaking, and personalization in addition to the symbolic, rule-based Artificial Intelligence (AI) techniques [17, 40, 42, 43, 50, 59, 63, 72]. ML algorithms can

7 Learner Modeling Interpretability and Explainability in Intelligent Adaptive …

97

help with that creation of models learned from (big) data and use these models to support decision making by making predictions and identifying hidden relationships and patterns in the data. In general, creating ML models can be efficient in terms of human-labor and the relative high accuracy that ML models may have. However, ML models have a couple of high-risk drawbacks that require careful and thorough processes to ensure that their applications do not harm end users. The first drawback is related to the quality of the data that are used to train ML models. Because the ML models are learning from the data, inaccurate, incomplete, or incompatible datasets (i.e., data biases) give rise to biased decisions and predictions [60]. Therefore, to lower the risks to the end users, it is important to: (a) assess the quality of the data, (b) collect data from diverse groups, and (d) be transparent about the content and characteristics of training data [2, 37, 60]. The second high-risk drawback is related to the complexity of the algorithms, which is the focus of this chapter. A group of ML models that leverages more complex ML algorithms such as deep neural networks or deep learning (DL) and large language models (LLMs). DL utilizes artificial neural networks, which are algorithms inspired by the structure and function of the human brain at a very high level. The input data is processed through multiple layers, where each layer extracts and amplifies specific features of the data. Different from other ML algorithms, DL can better handle unstructured data and can perform feature extraction automatically with minimal domain knowledge and human effort and with high predictive accuracy [4]. However, even if the complex ML models’ prediction accuracy is high, these models have a potential to make their decisions based on the correlations between irrelevant features and the outcome variable (e.g., see [12] for an ML model that classifies husky vs wolf images based on the pixels related to snow rather than the feature of the canine). LLMs work by analyzing and processing vast amounts of text data and use DL to understand and generate human-like text predicting the most likely next word or phrase based on their training data without necessarily understanding the meaning of the text and without necessarily generating output based on facts (i.e., hallucinations) [32]. These models are trained on diverse datasets from books, websites, and other written materials to learn language patterns, grammar, and context. As a result, LLMs can perform tasks like having dialogs with humans, translating languages, summarizing texts, and creating content. LLMs are being employed in different educational contexts, such as creating (conversational) intelligent tutoring systems [17, 59], having personalized educational dialogs with students [66], classification of algebra errors [43]. Despite their advantages, to perform these complex tasks, DL and LLM models include hundreds to billions of parameters and involve complex computations, which makes it harder to interpret the models’ inner decision-making process. The biases included in the training data combined with a lack of understanding how the AISs make their decisions may lead to unreliable, thus untrustworthy systems. Finally, there is work on the use of neural-symbolic approaches [11, 65, 73] aimed at leveraging the advantages and mitigating the disadvantages of both rule-based symbolic and sub-symbolic ML approaches.

98

D. Zapata-Rivera and B. Arslan

Although the decisions are not as high stake as in the use case of AI in medical decision-making, in the context of education, it is important that learners and teachers have adequate and valid explanations about the AIS’s decisions so that they: (a) trust the system’s decisions, (b) have agency to take an appropriate action when they detect an inaccurate or biased prediction, decision, or recommendation (see also [37]). The interpretability and explainability of AI systems are the central focus of the Explainable AI (XAI) field (for a historical perspective see [21]). These two terms are closely related and there is no consensus on how they are defined [1, 2, 6, 9]. Most ML researchers use these terms interchangeably to refer to the degree to which an AI algorithm’s output can be understood by humans (e.g., [1, 38, 48]) although there are differences between these two terms as different psychological constructs from the perspective of cognitive science (see [13]). In the scope of this chapter, we use interpretability of a model as a notion attached to the model’s inner decision-making transparency in relation to its expert user (e.g., ML engineers, data scientists). If a model’s inner decision-making processes are transparent in a way that experts (e.g., ML engineers, data scientists) can understand how the model works, the model’s interpretability is considered as high (see Fig. 1). These types of ML models are classified as inherently interpretable, “glass-box”, or “white-box” models. On the other hand, if a model’s inner decision-making processes are hard to comprehend by the experts because of their complexity, it is referred as a “black-box” model. The ML models that fall in between these two categories are called “grey-box” models [2, 3, 10]. On the other hand, we use explainability as a term that encompasses two notions: explainability of a model and explainability to the end user. Explainability of a model, which is closely related to the interpretability of a model, is a process to

Fig. 1 A figure depicting the notions of interpretability, explainability, and explanation in relation to the model transparency and end user. The three dots between “glass-box” and “black-box” models represent “grey-box” models. The bidirectional arrow between the end user and explanation depicts that end user can reject or modify the explanation depending on the context, which in turn feed into to the model

7 Learner Modeling Interpretability and Explainability in Intelligent Adaptive …

99

apply methods to understand why the ML models make their decisions [2]. While interpretability of a model is a more static construct, explainability of a model can be a more dynamic construct. For example, “glass-box” models do not necessarily require researchers to apply an additional method to understand why they make their decision since how they work is transparent. Thus, these types of models have both high model interpretability and explainability (see Fig. 1). However, “black-box” models are initially have low interpretability and they require applying additional methods (i.e., post-hoc methods, see [48]) to be able to increase their model explainability [2, 48, 55]. Taking a human-centered approach, in addition to the explainability of a model, in this chapter, we discuss another notion under explainability, which is explainability to the end user. Unlike explainability of the model which is related to an ML model’s inner workings and the methods to understand its decision-making processes, explainability to the end user is a process to generate explanations about the model’s decision-making process through external representations (e.g., a graph) and/or natural language for end users who are experts or non-experts (e.g., teachers or learners) based on their needs so that they can comprehend the explanations and take action to ensure agency. These explanations can support human–machine interactions in AISs, better accountability to increase trust in these systems (see [37] for a comprehensive XAI-ED framework; see also [15] for an open learner modeling framework). Explainability to the end user includes two-way interactions with the Explanation Engine in which end users (e.g., learner or teacher) can interact with it to elicit more information, provide context, or reject the explanation provided. With the help of the Explanation Engine, the information provided by the end user used as feedback to the model (see two-way arrow between the end user and the Explanation Engine and the arrows from the Explanation Engine to the model in Fig. 1). Explanation Engine is also responsible for not only presenting explanations but also presenting them at the right time (see also [18] for lessons learned from ITSs for XAI). Moreover, it may allow end users to have an option to turn-on and turn-off the explanations based on their needs. In the next section, we provide more information on the interpretability and explainability of different types of learner models in AISs.

3

Interpretability and Explainability of Different Types of Models

Historically, a variety of approaches have been implemented for modeling learners (see [79]). In addition to differences in variables chosen to depict the learner’s knowledge and skills and the context in which they were applied, these approaches may include different types of models that have different levels of interpretability and explainability. We first present some types of models and their levels of interpretability (see Table 1). Subsequently, we provide a brief overview about the methods related to increasing the explainability of the “black-box” and “glass-box” models.

100

D. Zapata-Rivera and B. Arslan

Table 1 Different types of models and their levels of interpretability Model types

Model subtypes

Interpretability level

Model label

Deep learning

Deep Knowledge Tracing (e.g., [23, 51]), Graph Neural Networks (e.g., [70]), Large Language Models (e.g., [50])

Low

“Black-box”

Machine learning I (Ensemble Methods)

Random Forest Decision Tree (e.g., [72]), XGBoost (e.g., [64]), AdaBoost (e.g., [26])

Low

“Black-box”

Neural-symbolic learning

Knowledge Enhanced Graph Neural Networks (e.g., [52]), Temporal Learner Modeling (e.g., [31])

Medium

“Grey-box”

Machine learning II (Fuzzy and Probabilistic Methods)

Fuzzy Logic (e.g., [27]), Bayesian Knowledge Tracing (e.g., [74]), Naïve Bayes (e.g., [44]), Bayesian Learner Models (e.g., [20, 53, 84]); Knowledge Spaces (e.g., [24, 25])

Medium

“Grey-box”

Machine learning III

Linear and Logistic Regression (e.g., High [67]), Generalized Additive Models (e.g., [22]), Decision Trees (e.g.,[75])

“Glass-box”

Symbolic

Cognitive modeling (e.g., [5, 7, 8]), Rule-based systems (e.g., [33]), Constrained-Based Learner Modelsa (e.g., [46])

“Glass-box”

High

a Constrained-Based

Learner Models have other versions that can be classified as probabilistic or Deep Learning (e.g., [47])

As we mentioned above, interpretability and explainability of learner models is essential for teachers and learners to better understand why recommendations and decisions are made by the AIS. While rule-based, symbolic AI approaches make decisions in a transparent way, the level of human effort and content knowledge required to infuse knowledge into these models is extensive. On the other hand, although ML models can learn from the data without human involvement in the learning process, human effort and expertise required is also extensive to develop ML algorithms, to provide labels for the training data in supervised ML models, and to make sure the ML models learn the correct representation (see [49] for human-in-the-loop ML). For the “glass-box” ML models, the Explanation Engine can generate global explanations about how different features or variables contribute to model’s decisions (see the row Machine Learning III in Table 1) without necessarily applying post hoc methods [37]. Therefore, the fidelity of explanations is considered high. Although these type of “glass-box” models are considered as inherently interpretable, when the number of features is high, it might get harder for humans to understand the decision process. On the other hand, the models that have

7 Learner Modeling Interpretability and Explainability in Intelligent Adaptive …

101

low (i.e., “black-box”) or medium (i.e., “grey-box”) interpretability require applying additional methods to move on the explainability axis from low to medium or high (see Fig. 1). These different types of methods generate different types of explanations (see [48] for a comprehensive list of methods). Model-agnostic global methods such as surrogate models generate explanations about how different features or variables affect model’s overall behavior by learning another interpretable ML model to approximate the outcome of the “black-box” or “greybox” model resulting in low fidelity explanations [2, 37, 48]. In contrast to model-agnostic global methods, model-agnostic local methods, such as LIME (Local Interpretable Model-agnostic Explanations [54]) or SHAP (SHapley Additive exPlanations [41]), focus on explaining individual predictions, which might be particularly useful in AISs where learner-level explanations are necessary and the AISs does not include a “glass-box” model. In addition to these types of methods, there are example-based explainability methods such as counterfactual explanations [68]. Although different explainability methods have been introduced to make the “black-box” models more explainable, it is important to emphasize that most of these approaches are based on an approximation of the model’s behavior; thus, they do not offer high fidelity explanations as inherently interpretable models do, and should be used with caution (see also [56–58] for critiques of using post hoc methods to make explanations for “black-box” models). In the next section, we describe the types of information needs of teachers and learners and elaborate on the explainability required to meet those needs.

4

Explainability to Teachers and Learners

Teachers and learners interacting with AISs have different types of assessment information needs. Learner models can provide the information needed to support learning and teaching processes. As we discussed above, Explainability Engines, for example, can be used to generate explanations required to respond to teachers’ and learners’ questions (see Fig. 1). A variety of external representations can be used to provide users with responses to their questions. Researchers in the area of Open Learner Modeling (OLM) have explored various types of external representations such as graphical representations, interactive reports, dashboards, and the use of pedagogical agents that make use of learner model information to provide guidance to users in the exploration of learner models [15, 34, 80, 83]. Table 2 summarizes some of the most common assessment information needs of teachers and students and identifies the level of explainability required by the learner model to provide such information. Teacher questions related to student performance at the individual, sub-group, and class levels, such as “What are my students’ strengths and weaknesses?”, or related to information that can help inform future teaching, such as “How difficult were the tasks for my students?” may require low-medium levels of learner model explainability depending on whether the teacher is interested in digging

102

D. Zapata-Rivera and B. Arslan

Table 2 Assessment information needs of teachers and learners and required level of model explainabilitya End user Assessment information needs

Required level of model explainability

Teachers Student performance at the individual, sub-group, and class levels • What are my students’ strengths and weaknesses? • How did the class perform on a task or a group of tasks? • How does a student’s performance compare to that of other students? • Progress information at the individual, subgroup, and class levels • How much progress have my students made towards mastery? Information that can help inform future teaching • How difficult were the tasks for my students? • What were the most frequent errors and misconceptions? Information that helps understand current performance • Were my students engaged in the task(s)? • Did my students try to game the system? • How reliable are the knowledge and engagement estimates calculated by the system? Instructional recommendations • What should I do next to help an individual student or the class as a whole?

Low-Medium Low-Medium Medium–High Medium–High

Learners Actionable feedback that they can use to guide their learning • What are my strengths and weaknesses? • How can I improve? Progress and performance information • How much progress have I made towards mastery? • How does my performance compare to that of other students? Evidence supporting assessment claims • What type of information was used to calculate my knowledge levels? • Can I provide additional evidence to update my knowledge levels in the system?

Low-Medium Low-Medium Medium–High

a First

two columns are adapted from “Supporting Human Inspection of Adaptive Instructional Systems”, by [76]. Copyright by Educational Testing Service, 2019 All rights reserved

deeper into the evidence used to answer these questions. Other questions related to information that helps teachers to understand current learner performance, such as “Were the students engaged in the task?” or “How reliable are the knowledge and engagement estimates calculated by the system?” may require learner models that support medium–high explainability levels since there could be a variety of aspects influencing these estimates. Questions that are related to instructional recommendations, such as “What should I do next to help an individual student or the class as a whole?” may require predictive models that make use of evidence

7 Learner Modeling Interpretability and Explainability in Intelligent Adaptive …

103

from various sources. Explanations generated using these types of predictive models may require additional user support to help users understand how the data are used to make predictions and the limitations of these models. Similarly, in the case of learners, questions related to receiving actionable feedback that learners can use to guide their learning, such as “What are my strengths and weaknesses?”, or related to their progress and performance, such as “How much progress have I made towards mastery?” may require learner models that support low-medium explainability levels based on the amount of supporting evidence required by learners. However, questions related to evidence supporting assessment claims, such as “What type of information was used to calculate my knowledge levels?” or “Can I provide additional evidence to update my knowledge levels in the system?” require medium–high explainability levels since they require additional evidence and more sophisticated explanations. Learner model explainability can benefit from a clear structure connecting claims to supporting evidence which may include process and response data. The implementation of an evidence layer can facilitate the generation of explanations through external representations, and interaction mechanisms that make use of learner model information to support learning and teaching [80]. The implementation of such evidence layer can be facilitated by using top-down and hybrid approaches that combine top-down and bottom-up approaches to designing learner models with different levels of interpretability and explainability (e.g., models that have been created by leveraging Evidence-Centered Design principles [45] together with several psychometric models and “big data” processes [82]). However, different techniques are explored to improve the explainability of “blackbox” learner models resulting from the application of bottom-up approaches (e.g., Chain-of-Thought prompt engineering; [69]) and neural-symbolic or neurosymbolic approaches for AI models; [29, 30]). These approaches may require a considerable amount of human effort to both creating the evidence managing mechanisms and validating the results produced by the model [61, 69]. This evidence layer can support the implementation of the Explanation Engine, which can offer explainability services in an instructional ecosystem, making a positive impact in terms of scalability. Finally, the evidence layer could be conceptually placed to the right of the Explanation Engine, between the output of the models feed and the Explanation Engine in Fig. 1.

5

Conclusions and Discussion

As new advances in ML become available and applications of these technologies extend, it is important to emphasize the need for interpretable and explainable models in education settings. Below we conclude by discussing the implications of improving interpretability and explainability of learner models in the context of AISs.

104

D. Zapata-Rivera and B. Arslan

• An appropriate level of learner model interpretability and explainability is required to support trust and adoption of AISs. Understanding how AISs support teaching and learning is an important first step in making sure that teachers’ and learners’ expectations are met. A general understanding of how adaptive components of the AIS are implemented and how they are intended to support teaching and learning may have a positive effect in adoption of these systems. Different levels of interaction with learner models should be supported to answer different types of user questions [35, 36]. These levels of interaction require models that support the generation of appropriate explanations. • The amount of human effort required to create explainable learner models that can respond to the needs for information of educational stakeholders can vary. The amount of data required to support learner model claims and the mechanisms for evidence identification and aggregation can also vary depending on the type of learner modeling approach used [79] and the data available to create those models. We expect that as new advances in AI become available, learner models will become more useful in supporting human decision making. Privacy, data security, and evaluation of learner models in supporting appropriate decision making will continue to be areas of interest. Modeling approaches should support the generation of explanations that consider various levels of uncertainty associated with different types of evidence sources and the nature of evidence aggregation and accumulation processes. AISs should consider maintaining different views of the learner model to capture teachers’ and learners’ perspectives. These perspectives can contribute to interesting negotiation and reflection processes that can have positive instructional value (e.g., knowledge awareness, self-reflection and self-regulation [14, 15]). In fact, human-in-the-loop approaches can reduce diagnostic complexity and provide immediate confirmation when levels of uncertainty are high. Teachers value flexibility when interacting with AISs. They appreciate the system handling common cases but be alerted on particular cases that may require their attention, so they have the opportunity to override suggestions made by the AIS based on additional information about the learner and the learning context that they may have [16, 81].

6

Future Work

Future work involves continue advancing in the development and evaluation of modeling approaches that support appropriate use of learner modeling information. Improvements in interpretability and explainability of these models contributes to achieving this goal. As more data (e.g., multimodal data) and AI technologies to create innovative learner models become available, additional opportunities for personalization in education contexts will arise (e.g., through the use of AISs). It is paramount that AI systems are designed taking into account the need for user understanding of the benefits and limitations of these technologies. We expect that

7 Learner Modeling Interpretability and Explainability in Intelligent Adaptive …

105

additional work will be done in areas such as human-centered AI, data privacy, and data security to support the responsible use of AI. Acknowledgements This material is based upon work supported by the National Science Foundation and the Institute of Education Sciences under Grant #2229612. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation or the U.S. Department of Education.

References 1. Adadi A, Berrada M (2018) Peeking inside the black-box: a survey on explainable Artificial Intelligence (XAI). IEEE Access 6:52138–52160 2. Ali S, Abuhmed T, El-Sappagh S, Muhammad K, Alonso-Moral JM, Confalonieri R, Herrera F (2023) Explainable artificial intelligence (XAI): what we know and what is left to attain trustworthy artificial intelligence. Inf Fusion 99:101805 3. Alonso JM, Castiello C, Mencar C (2015) Interpretability of fuzzy systems: current research trends and prospects. Springer Handbook of Computational Intelligence 4. Alzubaidi L, Zhang J, Humaidi AJ, Al-Dujaili A, Duan Y, Al-Shamma O, Santamaría J, Fadhel MA, Al-Amidie M, Farhan L (2021) Review of deep learning: concepts, CNN architectures, challenges, applications, future directions. J Big Data 8:53. https://doi.org/10.1186/s40537021-00444-8 5. Anderson JR (2005) Human symbol manipulation within an integrated cognitive architecture. Cogn Sci 29:313–341 6. Arrieta AB, Díaz-Rodríguez N, Del Ser J, Bennetot A, Tabik S, Barbado A, Herrera F (2020) Explainable Artificial Intelligence (XAI): concepts, taxonomies, opportunities and challenges toward responsible AI. Information Fusion 58:82–115 7. Arslan B, Taatgen NA, Verbrugge R (2017) Five-year-olds’ systematic errors in second-order false belief tasks are due to first-order theory of mind strategy selection: a computational modeling study. Front Psychol 8.https://doi.org/10.3389/fpsyg.2017.00275 8. Arslan B, Verbrugge R, Taatgen N (2017) Cognitive control explains the mutual transfer between dimensional change card sorting and first-order false belief understanding: a computational modeling study on transfer of skills. Biol Inspired Cognit Archit 20:10–20. https://doi. org/10.1016/j.bica.2017.03.001 9. Benchekroun O, Rahimi A, Zhang Q, Kodliuk T (2020) The need for standardized explainability. arXiv:201011273 10. Bennetot A, Franchi G, Del Ser J, Chatila R, Diaz-Rodriguez N (2022) Greybox XAI: a neuralsymbolic learning framework to produce interpretable predictions for image classification. Knowl-Based Syst 258:109947 11. Besold TR, Kühnberger KU (2015) Towards integrated neural–symbolic systems for humanlevel AI: two research programs helping to bridge the gaps. Biologically Inspired Cognitive Archit 14:97–110 12. Besse P, Castets-Renard C, Garivier A, Loubes JM (2019) Can everyday AI be ethical? Machine Learning algorithm fairness. Statistiques et Société 6 13. Broniatowski DA (2021) Psychological foundations of explainability and interpretability in artificial intelligence 14. Bull S (2020) There are open learner models about! IEEE Trans Learn Technol 13:425–448 15. Bull S, Kay J (2016) SMILI☺: A framework for interfaces to learning data in open learner models, learning analytics and related fields. Int J Artif Intell Educ 26:293–331

106

D. Zapata-Rivera and B. Arslan

16. Cardona MA, Rodríguez RJ, Ishmael K (2023) Artificial intelligence and future of teaching and learning: insights and recommendations. US Department of Education, Office of Educational Technology 17. Chen Y, Ding N, Zheng HT, Liu Z, Sun M, Zhou B (2023) Empowering private tutoring by chaining large language models. arXiv preprint arXiv:230908112 18. Clancey WJ, Hoffman RR (2021) Methods and standards for research on explainable artificial intelligence: Lessons from intelligent tutoring systems. Appl AI Lett 2:53 19. Conati C, Barral O, Putnam V, Rieger L (2021) Toward personalized XAI: a case study in intelligent tutoring systems. Artif Intell 298:10350 20. Conati C, Gertner A, Vanlehn K (2002) Using Bayesian networks to manage uncertainty in student modeling. User Model User-Adap Inter 12:371–417 21. Confalonieri R, Coba L, Wagner B, Besold TR (2021) A historical perspective of explainable artificial intelligence. Wiley Interdisciplinary Rev Data Mining Knowl Discovery 11:1391 22. Dikaya LA, Avanesian G, Dikiy IS, Kirik VA, Egorova VA (2021) How personality traits are related to the attitudes toward forced remote learning during Covid-19: predictive analysis using generalized additive modeling. Front Educ 6:108 23. Ding X, Larson EC (2021) On the interpretability of deep learning based models for knowledge tracing. arXiv preprint arXiv:210111335 24. Falmagne JC, Albert D, Doble C, Eppstein D (2013) Knowledge spaces: applications in education. Springer Science & Business Media 25. Falmagne JC, Koppen M, Villano M, Doignon JP, Johannesen L (1990) Introduction to knowledge spaces: how to build, test, and search them. Psychol Rev 97:201 26. Forbes-Riley K, Litman D (2004) Predicting emotion in spoken dialogue from multiple knowledge sources. In: Proceedings of the human language technology conference of the North American chapter of the association for computational linguistics: HLT-NAACL 2004, pp 201–208 27. Gagan G, Lalle S, Luengo V (2012) Fuzzy logic representation for student modelling. In: ITS 2012–11th international conference on intelligent tutoring systems-co-adaptation in learning. Springer, Heidelberg, pp 428–433 28. Greer J, McCalla G (1994) Student models: the key to individualized educational systems. Springer, New York, NY 29. Hammond K, Leake D (2023) Large language models need symbolic AI. In: Proceedings of the 17th international workshop on neural-symbolic reasoning and learning, CEUR workshop proceedings, Siena, Italy. pp 3–5 30. Hitzler P, Eberhart A, Ebrahimi M, Sarker MK, Zhou L (2022) Neuro-symbolic approaches in artificial intelligence. Natl Sci Rev 9:035 31. Hooshyar D (2023) Temporal learner modelling through integration of neural and symbolic architectures. Educ Inf Technol.https://doi.org/10.1007/s10639-023-12334-y 32. Huang L, Yu W, Ma W, Zhong W, Feng Z, Wang H, Liu T (2023) A survey on hallucination in large language models: principles, taxonomy, challenges, and open questions 33. Jaques PA, Seffrin H, Rubi G, Morais F, Ghilardi C, Bittencourt II, Isotani S (2013) Rule-based expert systems to support step-by-step guidance in algebraic problem solving: the case of the tutor PAT2Math. Expert Syst Appl 40:5456–5465 34. Kay J (2021) Scrutability, control and learner models: foundations for learner-centered design in AIED. In: Roll I, McNamara D, Sosnovsky S, Luckin R, Dimitrova V (eds) Artificial intelligence in education. AIED 2021. Lecture Notes in Computer Science. Springer, Cham 35. Kay J, Kummerfeld B, Conati C, Porayska-Pomsta K, Holstein K (2023) Scrutable AIED. In: Handbook of artificial intelligence in education, p 101 36. Kay J, Zapata-Rivera D, Conati C (2020) The GIFT of scrutable learner models: why and how. In: Ra M, Sinatra AC, Graesser X, Hu B, Goldberg JA, Hampton (eds)—Data visualization. U.S. Army CCDC—Soldier Center, Orlando, FL, pp 25–40 37. Khosravi H, Shum SB, Chen G, Conati C, Tsai YS, Kay J, Gaševi´c D (2022) Explainable artificial intelligence in education. Comput Educ Artif Intell 3:100074

7 Learner Modeling Interpretability and Explainability in Intelligent Adaptive …

107

38. Koh PW, Liang P (2017) Understanding black-box predictions via influence functions. In: International conference on machine learning. PMLR, pp 1885–1894 39. Leichtmann B, Humer C, Hinterreiter A, Streit M, Mara M (2023) Effects of explainable artificial intelligence on trust and human behavior in a high-risk decision task. Comput Hum Behav 139:107539 40. Lin CC, Huang AYQ, Lu OHT (2023) Artificial intelligence in intelligent tutoring systems toward sustainable education: a systematic review. Smart Learn Environ 10:41. https://doi.org/ 10.1186/s40561-023-00260-y 41. Lundberg SM, Lee SI (2017) A unified approach to interpreting model predictions. Adv Neural Inf Process Syst, 30 42. McNamara DS, Arner T, Butterfuss R, Fang Y, Watanabe M, Newton N, Roscoe RD (2023) ISTART: adaptive comprehension strategy training and stealth literacy assessment. Int J Human-Comput Interact 39:2239–2252 43. McNichols H, Zhang M, Lan A (2023) Algebra error classification with large language models. In: International conference on artificial intelligence in education. Springer Nature Switzerland, Cham, pp 365–376 44. McQuiggan SW, Mott BW, Lester JC (2008) Modeling self-efficacy in intelligent tutoring systems: an inductive approach. User Model User-Adap Inter 18:81–123 45. Mislevy RJ, Almond RG, Lukas JF (2003) A brief introduction to evidence-centered design. ETS Res Report Series 2003:1–29 46. Mitrovic A, Martin B, Suraweera P (2007) Intelligent tutors for all: constraint-based modeling methodology, systems and authoring. IEEE Intell Syst 22:38–45 47. Mitrovic A, Ohlsson S (2016) Implementing CBM: SQL-Tutor after fifteen years. Int J Artif Intell Educ 26:150–159 48. Molnar C (2022) Interpretable machine learning: a guide for making black box models explainable, 2nd, ed 49. Mosqueira-Rey E, Hernández-Pereira E, Alonso-Ríos D, Bobes-Bascarán J, Fernández-Leal Á (2023) Human-in-the-loop machine learning: a state of the art. Artif Intell Rev 56:3005–3054. https://doi.org/10.1007/s10462-022-10246-w 50. Ouyang F, Wu M, Zheng L, Zhang L, Jiao P (2023) Integration of artificial intelligence performance prediction and learning analytics to improve student learning in online engineering course. Int J Educ Technol High Educ 20:4 51. Piech C, Spencer J, Huang J, Ganguli S, Sahami M, Guibas L, Sohl-Dickstein J (2015) Deep knowledge tracing. arXiv preprint arXiv:150605908 52. Raj K (2023) A neuro-symbolic approach to enhance interpretability of graph neural network through the integration of external knowledge. In: Proceedings of the 32nd ACM international conference on information and knowledge management, pp 5177–5180 53. Reye J (2004) Student modelling based on belief networks. Int J Artif Intell Educ 14:63–96 54. Ribeiro MT, Singh S, Guestrin C (2016) Why should I trust you?”: Explaining the predictions of any classifier. In: Proceedings of the 22nd SIGKDD international conference on knowledge discovery and data mining, pp 1135–1144 55. Rizzo M, Veneri A, Albarelli A, Lucchese C, Conati C (2023) A theoretical framework for AI models explainability with application in biomedicine. In: IEEE conference on computational intelligence in bioinformatics and computational biology (CIBCB), pp 1–9 56. Rosé CP, McLaughlin EA, Liu R, Koedinger KR (2019) Explanatory learner models: Why machine learning (alone) is not the answer. Br J Edu Technol 50:2943–2958 57. Rudin C (2019) Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nat Mach Intell 1:206–215 58. Rudin C, Radin J (2019) Why are we using black box models in AI when we don’t need to? A lesson from an explainable AI competition. Harv Data Sci Rev 1:1–9 59. Schmucker R, Xia M, Azaria A, Mitchell T (2023) Ruffle&Riley: towards the automated induction of conversational tutoring systems. arXiv preprint arXiv:231001420

108

D. Zapata-Rivera and B. Arslan

60. Schramowski P, Turan C, Andersen N, Rothkopf CA, Kersting K (2022) Large pre-trained language models contain human-like biases of what is right and wrong to do. Nat Mach Intell 4:258–268 61. Shum K, Diao S, Zhang T (2023) Automatic prompt augmentation and selection with chain-ofthought from labeled data. arXiv preprint arXiv:230212822. http://arxiv.org/abs/2302.12822 62. Shute VJ, Zapata-Rivera D (2012) Adaptive educational systems. In: Durlach P (ed) Adaptive technologies for training and education. Cambridge University Press, New York, pp 7–27 63. Singh N, Gunjan VK, Mishra AK, Mishra RK, Nawaz N (2022) Seistutor: a custom-tailored intelligent tutoring system and sustainable education. Sustainability (Switzerland) 14:4167 64. Su W, Jiang F, Shi C, Wu D, Liu L, Li S, Shi J (2023) An XGBoost-based knowledge tracing model. Int J Comput Intell Syst 16:13 65. Sun R, Bookman LA (1994) Computational architectures integrating neural and symbolic processes: a perspective on the state of the art. Kluwer, Norwell, MA 66. Tack A, Piech C (2022) The AI teacher test: measuring the pedagogical ability of blender and GPT-3 in educational dialogues. arXiv preprint arXiv:220507540 67. Vaessen BE, Prins FJ, Jeuring J (2014) University students’ achievement goals and helpseeking strategies in an intelligent tutoring system. Comput Educ 72:196–208 68. Wachter S, Mittelstadt B, Russell C (2017) Counterfactual explanations without opening the black box: automated decisions and the GDPR. Harv JL & Tech 31:841 69. Wei J, Wang X, Schuurmans D, Bosma M, Ichter B, Xia F, Chi E, Le Q, Zhou D (2022) Chainof-thought prompting elicits reasoning in large language models. Adv Neural Inf Process Syst 35:24824–24837 70. Xia Z, Dong N, Wu J, Ma C (2023) Multi-variate knowledge tracking based on graph neural network in ASSISTments. IEEE Trans Learn Technol 71. Xu W (2019) Toward human-centered AI: a perspective from human-computer interaction. Interactions 26:42–46 72. Yang C, Chiang FK, Cheng Q, Ji J (2021) Machine learning-based student modeling methodology for intelligent tutoring systems. J Educ Comput Res 59:1015–1035 73. Yu D, Yang B, Liu D, Wang H, Pan S (2023) A survey on neural-symbolic learning systems. Neural Networks 74. Yudelson MV, Koedinger KR, Gordon GJ (2013) Individualized Bayesian knowledge tracing models. Artificial Intelligence in Education: 16th International Conference, AIED 2013. Springer, Memphis, TN, USA, pp 171–180 75. Zacharis NZ (2018) Classification and regression trees (CART) for predictive modeling in blended learning. IJ Intell Syst Appl 3:9 76. Zapata-Rivera D (2019) Supporting human inspection of adaptive instructional systems. Adaptive Instructional Systems: First International Conference, AIS 2019, Held as Part of the 21st HCI International Conference, HCII 2019. Springer International Publishing, Orlando, FL, USA, pp 482–490 77. Zapata-Rivera D (2020) Open student modeling research and its connections to educational assessment. Int J Artif Intell Educ. https://doi.org/10.1007/s40593-020-00206-2 78. Zapata-Rivera D, Arslan B (2021) Enhancing personalization by integrating top-down and bottom-up approaches to learner modeling. In: R. S, J S (eds) adaptive instructional systems. Adaptation strategies and methods. HCII 2021. Lecture Notes in Computer Science. Springer, Cham, pp 234–246 79. Zapata-Rivera D, Arslan B (2021) Enhancing personalization by integrating top-down and bottom-up approaches to learner modeling BT—adaptive instructional systems. Adaptation strategies and methods. In: Sottilare RA, Schwarz J (eds). Springer International Publishing, Cham, pp 234–246 80. Zapata-Rivera D, Brawner K, Jackson GT, Katz IR (2017) Reusing evidence in assessment and intelligent tutors. In: Sottilare R, Graesser A, Hu X, Goodwin G (eds)—Assessment methods. U.S. Army Research Laboratory, Orlando, FL, pp 125–136 81. Zapata-Rivera D, Hansen EG, Shute VJ, Underwood JS, Bauer MI (2007) Evidence-based approach to interacting with open student models. Int J Artif Intell Educ 17:273–303

7 Learner Modeling Interpretability and Explainability in Intelligent Adaptive …

109

82. Zapata-Rivera D, Liu L, Chen L, Hao J, Davier A (2016) Assessing science inquiry skills in immersive, conversation-based systems. In: Daniel BK (ed) Big Data and learning analytics in higher education. Springer International Publishing, pp 237–252 83. Zapata-Rivera JD, Greer J (2002) Exploring various guidance mechanisms to support interaction with inspectable learner models. Proc Intell Tutoring Syst ITS 2002:442–452 84. Zapata-Rivera JD, Greer JE (2004) Interacting with inspectable Bayesian student models. Int J Artif Intell Educ 14:127–163

8

Deep Learning in Educational Scenario Alessandro Ciasullo

“You take the blue pill... the story ends, you wake up in your bed and believe whatever you want to believe. You take the red pill... you stay in Wonderland, and I show you how deep the rabbit hole goes” Morpheus (The Matrix)

Abstract

The relationship between deep learning and human neural networks is speculative due to limited understanding of brain processes. Deep Learning employs artificial neural networks to predict patterns in data, resembling biological brain functions. This connection extends to education, where AI models simulate learning processes. The overlap involves the dynamics of human learning and the attempt to replicate them through automated structures. Deep Knowledge Tracing, presented at NeurIPS 2015, utilizes recurrent neural networks to predict student performance based on previous data. In education, this approach predicts personalized training needs efficiently, fostering a significant human–machine relationship. The ASSISTments project exemplifies collaboration between researchers and teachers, offering a platform for interactive

A. Ciasullo (B) Department of Humanistic Studies, University of Naples Federico II, Naples, Italy e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 F. Santoianni et al. (eds.), Mind, Body, and Digital Brains, Integrated Science 20, https://doi.org/10.1007/978-3-031-58363-6_8

111

112

A. Ciasullo

learning. The success is attributed to continuous repetition in science-related modes and teacher involvement. This reflects two educational data mining approaches: organic interaction between computers and humans and a more automated AI-mediated synthesis. Balancing AI advantages with human-centric education is crucial, emphasizing the need for a control policy and literacy strategy for intelligent technologies. The challenge is to develop intensive AI education forms in the coming months. This contribution explores operational strategies for implementing AI education, addressing challenges such as diverse Deep Learning algorithms, non-open datasets, theoretical hesitations, and BigTech interests. A focus on AI in education (AIED) involves training in Computational Thinking. The Korean experiment demonstrates successful Deep Learning education for children. The challenge lies in balancing technological literacy with ontological perspectives. Doleck et al. emphasize explaining rather than just predicting AI processes. Holmes et al. propose AIED applications like collaborative learning, student forum monitoring, continuous assessment, AI learning companions, and AI teaching assistants. The democratization of AI in education requires expanding datasets, broadening data sources, integrating real contexts, clarifying algorithms, and enhancing AI competence in curricula. Educational Data Mining (EDM) plays a crucial role in predicting student achievement. The article suggests a conceptual framework for AI integration, encompassing cognitive, biometric, physical, and spatial dimensions, alongside algorithmic, educational dataset, and subjective feedback considerations. Openended conclusions emphasize the need for a comprehensive, curriculum-based, and critical approach to AI in education, focusing on digital literacy, dataset understanding, and the impact of AI on subjective experiences.

1

Introduction

The relationship between deep learning and the biological functioning of human neural networks is ‘vaguely’ specular, taking into account that the processes underlying the functioning of the brain are in their entirety still only partially understood. In this perspective, so-called Deep Learning appears as a series of specific predictive methods in which computers are able to read recursive sequences and patterns within a certain multitude of data. This is done through the use of artificial neural networks that can roughly mimic the functioning of the biological brain. Once a deep learning application has learnt the recursiveness of certain patterns within a data set used for its training, it is subsequently able to predict the same patterns when encountering new situations within other data sets [1]. Attempting to describe and identify the existing links between Deep Learning and education also involves describing the specific terms and vocabulary that link AI and its computerized neural structures to educational processes. This link, therefore, is determined in a ‘natural’ continuity: the one hand, Deep Learning models

8 Deep Learning in Educational Scenario

113

simulate biological neural networks; on the other hand, artificial neural networks can be trained to anticipate, support and simulate the functions of human behaviour and learning. Learning is the fundamental construct for realizing this relationship. Furthermore, we see a further dual relationship: on the one hand, the psychodynamics and neuronal characteristics of learning linked to the neurophysiological processes of the human learning dimension, and on the other, an intricate network of automated structures that attempt to reproduce—by means of cybernetic processes—the typical mechanisms of the human brain and mind structures. The close relationship between learning, education and deep learning is based on this complex dynamic of the relationship between what the brain and mind are and how it is possible to recreate their dynamics. During the twenty-ninth NeurIPS (Neural Information Processing Systems) conference held in Montreal, Canada in 2015, one of the papers that stood out was the one by Piech et al. [2] entitled Deep Knowledge Tracing. In that proposal, they describe the complexity of drawing—in computersupervised instruction—models of human learning functioning that can represent the active processes and neural mechanisms of a student while following a course of study. To obtain a more reliable probabilistic view of student learning modeling, they aim to create a group of recurrent neural networks (RNNs). The methodological process of Deep Knowledge Tracing, understood as a model aimed at predicting students’ performance in future interactions, through the reading of previous performances catalogued in Datasets was able to simulate neural networks what the subsequent educational performances would be. In the educational field, this possibility, by overcoming the need to define a specific learning domain in computer systems and then simulate it through artificial neural networks, is able to predict very effectively what future personalized training needs students invested in a learning process may have. Providing sources of information according to individual needs; structuring customized paths methodologically in line with subjective needs; having evaluations and feedback much more in line with the learning dynamics of each individual; seem to suggest that the relationship between artificial deep learning models and human learning is being the most interesting field of the human–machine relationship. Heffernan and Heffernan’s implementation and description of the ASSISTments project is highly interesting [3]. Relationships between researchers and teachers are structured in this ecosystem. They can work together on a platform hosted by the Worcester Polytechnic Institute that allows teachers to write individual ASSISTments (a set of repositories with questions, answers, tutorials and video tutorials) or to use ready-made ASSISTments made by others, grouping them into a set of activities to be assigned to students. Both teachers and students can receive direct feedback on their performance through this system, which is based on the contracted form of the words ‘assistance’ and ‘assessment’. By observing how these materials are used, researchers can both implicitly and explicitly infer additional data.

114

A. Ciasullo

The authors’ realization and deduction through the ASSISTments platform can be traced back to two main issues: 1. The platform was a success because it enabled continuous repetition using science-related learning modes and did not restrict learning to a single topic; 2. Computerized learning and feedback systems are best suited for direct management by teachers within the classroom routine, not just by computers [3]. The two cases described above, help us to define at least two main approaches related to educational data mining and intelligent tutoring systems: • on the one hand, an approach that pushes towards an organic interaction between computers and humans, • and on the other hand, a set of operational hypotheses that push more decisively in the direction of automation processes mediated AI neural synthesis. Both positions, in my view, cannot overcome a certain epistemological tension that continuously holds together the advantages offered by AI systems and their predictive and generative capacities, with the need for educational processes capable of involving and giving meaning to the human in its transformative and emancipative dimension. AI ‘for the human’ and never ‘beyond the human’. An effective control policy for neural synthesis systems is necessary, along with a major literacy strategy for the critical, conscious, and supportive use of intelligent generative technologies. In other words, the major challenge for the coming months is to develop intensive AI education forms.

2

Operational Plans for AI in Education

This contribution intends to fit precisely into the groove of this perspective, and to do so, it aims to analyze what possibilities we have for defining operational strategies to put AI education into practice. In order to do so, however, we must refer to some crucial questions concerning the construction of data analysis systems to be used for integrative and/or predictive educational purposes: 1. many Deep Learning-based analysis systems do not share similar algorithmic and training modes of operation, which makes it quite difficult to identify a common, holistic functional strategy with respect to these systems; 2. many of the datasets with which AIs are trained are not open and shared, which means that the contents are not interchangeable, implementable and, above all, knowable in their underlying constitution, with major ethical implications also with regard to the resulting output and biases; 3. there is a certain reluctance to use deep learning systems and predictive semantic analysis in education in certain theoretical assumptions;

8 Deep Learning in Educational Scenario

115

4. there is a problem of pronounced interest on the part of BigTech, which invest copious resources in the development of AI-based technologies; this leads one to think that the interests arising from these do not always aim at the simple evolution of human development, but rather at the consolidation of positions of technological dominance. 5. These series of statements, which call to mind numerous other pressing questions, are just some of the possible implications to be acted upon in order to interpret, read and also direct processes of ‘intelligent use of intelligent systems’ in a conscious and effectively educational manner. AIED (Educational AI) refers to the part of educational reflection that pertains to AI [1]. Some training models and methods aimed at improving AI literacy seem to be shifting towards methods that are centered around the CT model (Computational Thinking) [4, 5], a process involving problem solving, system design, understanding human behavior of the main concepts of computer science [6, 7]. In this regard, a Korean experiment [8] that aims to teach Deep Learning concepts to primary school and early secondary school children is intriguing. Using Computational Thinking methodologies and the development of nine different educational programmes, the development of the artificial intelligence algorithm for recognizing CNN (Convutional Neural Network) images [9] was tackled as a subject with excellent learning results. This, along with another long series of experiments, highlights the extent to which a constructivist approach to AI education is supported and utilized. The crucial point on which we should reflect, however, is that educating on the conscious use of a technology does not only go through technology, but involves deep-rooted subjective learning processes that pertain to the social, value-based, evolutionary, cultural, sentimental and emotional character of the learner. The way to indicate trajectories of critical and conscious use of Deep Learning technologies, and thus of AI, is that of a profound mixture between literacy in technological functioning (enabling the subject to understand the algorithmic dynamics underlying the functioning of the systems) and, on the other hand, an ontological vision entrusted to meta-level educational processes (centered on the existential, trans and intra-human function that such a complex transformation can mean). Interesting is the perspective of Doleck et al. [10] who advocate the idea that the real challenge in optimizing and using predictive models related to neural networks and Deep Learning is not so much in ‘predicting’ what might happen as in ‘explaining’ why certain processes occur. In this sense, the usefulness of pedagogy and especially of the bio-educational sciences [11] is precisely that of providing interpretative frameworks for phenomena that reproduce learning mechanisms, are able to foresee processes but do not grasp their underlying evolutionary formative principles. The potential integration of artificial intelligence technology into educational systems and practices is described by Holmes, Bialik, and Fadel in the chapter ‘Artificial Intelligence in Education’ [12]. Their hypotheses, which are backed by

116

A. Ciasullo

an educational approach based on the so-called AIEDs, fall into a few macrocategories: • Collaborative Learning, i.e. the possibility of creating peer support networks not focused on generic interests but based on the suggestion of specific learning skills and interests of each individual student or group of students; • Student Forum Monitoring, the ability to intelligently and quickly manage, organize, address and moderate the large number of posts generated within learning groups collaborating online. These advantages are all the more significant the more complex the network of relationships contained in the posts becomes, which is why a virtual assistant or active facilitation tool can improve monitoring and mentoring within these channels; • Continuous Assessment, one of the problems of learning dynamics is that generalizing the results of a single test or examination as an overall element of the students’ actual preparation. A process of continuous analysis, managed by intelligent systems of biometric and cognitive reading of learning processes, can stimulate a process of continuous authentic assessment, capable of suggesting strategies, alternative paths, strengths or weaknesses to enhance academic performance; • AI Learning Companions, suggest that learning processes are more engaging when supported by assistance systems that help the learner. Smart guidance implemented by an AI that focuses on voice assistants and typical mobile learning technologies (smartphones, tablets, etc.) can provide direct assistance and help to learners. One could also include within these solutions the colloquial practice in foreign language learning that would allow anyone to overcome the difficulties, fears and shyness of having to converse with a human being for foreign language training; • AI Teaching Assistant, rethinking the role of teachers means giving them new functions especially at the time of the so-called AIED. These are not systems that support the teacher’s cognitive structures, nor ways to reduce the relational and dialogical possibilities typical of the profession, but systems that can reduce the bureaucratic mass of activities attributed to teachers and support the methodological understanding of the various types of students they deal with. The ability to structure programs that are tailored to the specific needs of each individual student would become more rapid and mature due to all of this. [12]. The analysis of various operational hypotheses related to AI applied to education suggests that personalized and customizable teaching–learning is possible. Although the concepts of personalization are not new within the pedagogical debate from an inclusive perspective, the further possibility is that of creating dynamic clouds that are constantly changing, capable of representing everyone’s learning conditions moment by moment. One would thus move away from the concept of personalization and individualization (typical of an inclusive approach

8 Deep Learning in Educational Scenario

117

that goes beyond the concept of norm and normality), in favor of a new epistemological and ontological formulation of the learning subject: a pedagogy for the subject in continuous transformation. In such a hypothesis, the concept of static knowledge, of standard educational models, of teaching practices entrusted to the sole experience of teachers cannot be contemplated, but the prospect of a teacher who is an agent of transformation opens up: technology expert, capable of continually remodeling his or her own action, capable of structuring processes of reworking and reading data.

3

Deep Learning’s Functional Foundations

Artificial neural network architectures that reproduce certain functions of the human brain through algorithms are known as deep learning systems. They are determined as an evolution of Machine Learning, which is an initial complex learning architecture for machines but which requires greater control on the part of those who train it, with a consequent increase in time and work to correct, adjust and ratify the responses necessary for machine learning. Deep learning systems have the ability to train themselves by continuously reading data organized in datasets, which is a characteristic. It progressively manages to realize more and more refined answers and organizations through a layered data filtering mechanism; each layer extracts data from a lower level of representation and then passes it on to the next level. With this work of layer-by-layer analysis, it is possible to reconstruct hierarchical representations that are then reassembled to obtain the complete characteristics of the object, sentences, figures, sounds in their entirety [13]. Training a neural network can be done using different functional approaches, such as supervised learning, unsupervised learning, and hybrid learning. • Supervised. According to this mode, the algorithm’s prediction functions are incentivized to make the outputs of certain responses from specific inputs more consistent. Refinement of the algorithm’s recognition and the most appropriate response is what the function aims to achieve. Supervised training is completed when all the responses are consistent with the inputs provided. • Unsupervised training-learning, aims to recognize certain occurrences, invariances, and patterns in the data, so as to create a pattern or distribution according to certain constraints. The algorithms are instructed to search for the structures that comprise the data quantities by this method. Hierarchical organization (clustering) of data can be solved by using this learning mode, which involves placing certain parts of data together for association and homogeneity. • Reinforcement learning is a training mode that adopts the typical features of behaviorist learning, utilizing positive and negative reinforcement mechanisms. In fact, there is a tendency to reward the algorithm for correct associations and penalize it for incorrect ones, thus tending to construct a series of refinements in the responses that lead towards a system of correct answers.

118

A. Ciasullo

• Hybrid learning is supported by architectures that utilize the two dimensions mentioned above, specific non-supervised and supervised learning. In the case of the unsupervised component, we are in the presence of generative algorithms capable of recognizing structural coherences in the data, coupled with supervised algorithms capable of discriminating the coherence between input and output in depth. The construction of multi-level algorithms typical of Deep Learning is widely achieved by combining the two modes. Among the most interesting possibilities on the horizon of the use of Deep Learning processes in education are undoubtedly the Convolutional Learning Networks (CNNs), which exert their interpretative capacity especially in the recognition of images, voices or audio. This type of system works in layers, i.e. it acts as a series of filters capable of recognizing certain elements from the input, superimposing them, reducing their characteristics by synthesizing their main aspects, activating connections and thus producing output responses. Convolutional, nonlinearity, pooling, and fully connected layers are the types of layers that make up a convoluted network. Each of these algorithmic filters simulates biological neural activation functions, and does so through the assignment of weights and biases, i.e. features that regulate the coherence between what the input proposes and how the machine—through a dense network of layers—produces or proposes. The output of a neuron is expressed by the formula: output = inputs × weights + bias. To verify if there are elements of concordance in their superposition, convolution is a mathematical operation that combined two functions. This functional model is used to outline elements for the genesis of new functions by multiplying the points common to the two functions through mathematical procedures. This solution is particularly effective in the realization of automatic learning and neural networks, especially in applications dealing with image classification data, computer vision, and natural language processing (NLP) [14] (Fig. 1). By comprehending these complex functional dynamics, it is possible to guide an optimization process to integrate CNNs into teaching–learning processes. The hypotheses for the development of an Educational Deep Learning network are

Fig. 1 Functioning of a convolutional neural network [15]

8 Deep Learning in Educational Scenario

119

based on the implementations formulas of pedagogical practices, which seem to be the guiding perspective.

4

Deep Learning Through Education Scenarios

The rush to try to bend both scientifically and operationally AI to the various specific domains of knowledge seems to lead many to reduce the complexity of reality to the functional matrices of a few ready-to-use solutions. To address the complexity we are facing, we need structured approaches that can comprehend and rewrite the approach to using technologies in different domains [16]. In the case of education, several fundamental issues need to be addressed: the lack of aptitude for change that some established practices assume within the complex universe of education; the cognitive difficulty in restructuring one’s approach to knowledge mediated by integration technologies such as AI; hypothesizing new rules and new possible applications in historically structured and situated contexts; the tendency to attribute the blame of the social world to new technologies; the tendency to attribute almost esoteric values to something whose functioning is not fully understood. The use of these ‘attribute spaces’, which are logical varieties that intertwine reality and the reality mediated by digital technologies, leads to a polarization of their use, even in education, that is inevitable and a-critical. The reason for this is that aiding in the clarification of the processes that govern the revolutionary horizon of AI applied to education is crucial. The point is also to allow AI to be trained through datasets capable of overcoming the bias produced by approaches centered on massively produced quantities of data that are predominantly produced in advanced cultural contexts and managed through processes that are predominantly produced by the use of cognitive artefacts in the hands of a few technological giants. It is possible that in the future, education will take the shapes established by the rules of only a few large technology corporations. This is why the democratization of AI, as an element that implicitly and explicitly shapes and educates future humanity, passes through the quality with which we are able to train, make learning and educate AI itself. Wanting to broaden the discourse, we should imagine that the solution to the risk of a dictatorship of the processes of utilization and cataloguing of big data is not to be overcome by reducing the use of AI but by creating open protocols of creation and generation of multiple forms of AI. This means: • Extending datasets a wide range of cultures allows for regulation of data acquisition and use. • Broadening the constitution of datasets by proceeding to digitise everything that is not digital (books, monuments, works, audio, music) avoiding that the constitution of the data mainly consists of results produced within the digital;

120

A. Ciasullo

• To bring into the processes of educational analysis, of suggestion, of customized didactic hypotheses, of tailored educational assessment, the experiences gathered within the real contexts analyzed, avoiding to trace back to too general parameters deeply rooted situations; • Clarify which algorithms and technologies are used to understand how to construct the data. • Increase overall competence in AI by restructuring all training curricula currently. The literature surrounding Educational Data Mining (EDM) suggests that there are significant steps that can be identified. EDM is a sophisticated system that analyzes recurrences deduced from educational data to serve various purposes, including predicting student achievement [17]. Educational processes, personalization, individualization, and AI-mediated implementation seem to be connected through the concept of prediction. [10, 17, 18]. Thus, one has a perspective of reading the interaction of AI with subjectivity for educational purposes of calculating the space between reading the subjective, cognitive and biological data, Data Analysis/Data Mining of comparison with large amounts of data, and predictions of upcoming future subjective learning possibilities. The likelihood of predicting the subject matter in the ways and forms most appropriate for the individual subject depends on the ability to verify the subject’s knowledge and co-genitive skills in a given domain, add to this analysis the subject’s biometric, physical, and spatial feedback during a series of lessons, compare them with a series of educational datasets to obtain indications, reports, automated structuring processes of personalized content. However, in order to carry out a process of logical representation of such a relationship, so that we can define a hypothetical order of magnitude with respect to the degree of productivity, we would have to refer to a system of indicators i.e., an indirect measurement system for describing complex phenomena. The indicators will be semantically oriented, requiring the description of the elements, behaviors, biometric data, and cognitive feedback that is not directly linked to numerical values. It would be a tool for complex evaluation of a phenomenon to which a link is attributed to a conceptual reference/reference that is part of a research model (a priori) or an interpretive scheme (a posteriori). We should basically move from a conceptual representation related to the realization of abstract concepts (constructs), and then define them into their empirically verifiable representations capable of reflecting the nature of the phenomena under consideration, after which we proceed to the internal identification of specific indicators. A genuine organization of predictive and trend indicators is required to identify methodological-contextual strategies that are appropriate for the learning subject [19].

8 Deep Learning in Educational Scenario Fig. 2 Table of concepts and indicators

121

Domains

Conceptual elements

Possible indicators to be identified

Cognitive knowledge and skills

Level of understanding of the topics, ability to apply knowledge, specific skills in the field of study.

Results in standardized tests, training assessments, active participation in learning activities.

Biometric feedback

Cardiac frequency, skin measMonitoring of the subject's urements (galvanic), brain acphysiological responses tivity (if available), detection of during the lessons. movement (if available).

Physical and spatial feedback

Movement sensors, posture analysis through wearable techPhysical involvement, pos- nologies or cameras, physical ture, space movements interactions with the environduring lessons. ment during learning.

Educational set

Representativeness of the data used for the prediction.

Diversity of the topics covered in datasets, degree of adaptation to standard educational curriculums, inclusiveness of different learning styles.

Algorithms of au- Methods used to process data and formulate foretomatic analysis casts. and learning

Precision of predictions, ability to adapt to new data, resistance to anomalies.

Measurement of how much the predictions related to the collected data Predictive validity reflect on future learning.

Comparison between the forecasts and the actual results of future learning, evaluation of consistency over time.

Perception of one's learning by the subject.

Questionnaires, interviews, written or oral feedback on understanding, interests and preferences.

Subjective feedback

External factors that can influence learning (e.g. enAnalysis of control vironmental conditions, level of stress). variables

Monitoring and control of external variables to isolate the effect of the variables considered in research.

Update of algorithms based on new data, review of the methodology in response to new disIteration and con- The research should be flexible and adaptable over coveries or changes in learning tinuous adaptatime. conditions. tion

In order to generate a productivity index, the analysis process should go through these conceptual representations and analyze specific indicators displayed in the Fig. 2.

5

Open-Ended Conclusions

Numerous and diverse educational perspectives pertain to the introduction of AI. An integrative and mutually respectful approach is necessary to achieve a meaningful relationship between education and AI. To do this, the need to identify the conceptual matrices to be addressed in defining this relationship and subsequently the identification of indicators and variables that can be observed and measured in processes of active experimentation with AI-powered technologies appears increasingly sharp. Overcoming the current tendency to focus all attention on individual apps, software, and generative intelligence technologies can be achieved through an organic

122

A. Ciasullo

review of conceptual frameworks. Instead, we should examine how we redesign human, learning subjectivity, and personalization processes with AIs. To foster an open but critical approach to controlled availability and the epistemological definition of the complex relationship being formed, it is necessary to work on: • specific curricula that meaningfully involve learning about the digital processes underlying AI. • understanding the composition of datasets and the processes that can help users detect and reduce cognitive and cross-cultural bias can help users. • a curriculum that focuses on understanding the influence of AI on the generation of subjective products. • curriculum for protecting and informed management of one’s data and privacy. All of this can happen if the run-up to the use of AI and in the reorganization of Deep Learning processes are clear, or at least clarified by the Bigtechs, and especially if we have the strength to first understand what function it can exert in learning and then integrate it within all learning processes. The challenge is daunting; the possibilities are limitless. It’s up to us to determine its significance.

References 1. Perrotta C, Selwyn N (2020) Deep learning goes to school: toward a relational understanding of AI in education. Learn Media Technol 45:251–269. https://doi.org/10.1080/17439884.2020. 1686017 2. Piech C, Bassen J, Huang J et al (2015) Deep knowledge tracing. In: Cortes C, Lawrence N, Lee D et al (eds) Advances in neural information processing systems. Curran Associates, Inc. 3. Heffernan NT, Heffernan CL (2014) The ASSISTments ecosystem: building a platform that brings scientists and teachers together for minimally invasive research on human learning and teaching. Int J Artif Intell Educ 24:470–497. https://doi.org/10.1007/s40593-014-0024-x 4. Lodi M, Martini S (2021) Computational thinking, between Papert and Wing. Sci Educ 30:883–908. https://doi.org/10.1007/s11191-021-00202-5 5. Vonèche JJ (1983) Mindstorms: Children, computers and powerful ideas: by Seymour Papert, Basic Books, New York (1980). New Ideas Psychol 1:87. https://doi.org/10.1016/0732-118 X(83)90034-X 6. Tikva C, Tambouris E (2021) Mapping computational thinking through programming in K12 education: a conceptual model based on a systematic literature review. Comput Educ 162:104083. https://doi.org/10.1016/j.compedu.2020.104083 7. Wing JM, Stanzione D (2016) Progress in computational thinking, and expanding the HPC community. Commun ACM 59:10–11. https://doi.org/10.1145/2933410 8. Ryu M, Han S (2019) AI education programs for deep-learning concepts. J Korean Assoc Inf Educ 23:583–590. https://doi.org/10.14352/jkaie.2019.23.6.583 9. Chen L, Li S, Bai Q et al (2021) Review of image classification algorithms based on convolutional neural networks. Remote Sens 13:4712. https://doi.org/10.3390/rs13224712 10. Doleck T, Lemay DJ, Basnet RB, Bazelais P (2020) Predictive analytics in education: a comparison of deep learning frameworks. Educ Inf Technol 25:1951–1963. https://doi.org/10. 1007/s10639-019-10068-4

8 Deep Learning in Educational Scenario

123

11. Frauenfelder E, Santoianni F, Ciasullo A (2018) Implicito bioeducativo. Emozioni e cognizione. RELAdEI Rev Latinoam Educ Infant 7:42–51 12. Holmes W, Bialik M, Fadel C (2023) Artificial intelligence in education. In: Data ethics: building trust: how digital technologies can serve humanity. Globethics Publications, pp 621–653 13. Mathew A, Amudha P, Sivakumari S (2021) Deep learning techniques: an overview. In: Hassanien AE, Bhatnagar R, Darwish A (eds) Advanced machine learning technologies and applications. Springer, Singapore, pp 599–608 14. Understanding of a convolutional neural network|IEEE Conference Publication|IEEE Xplore. https://ieeexplore.ieee.org/abstract/document/8308186. Accessed 21 Dec 2023 15. Aphex34—Creative Commons Attribution-ShareAlike 4.0 International. https://commons.m. wikimedia.org/wiki/File:Typical_cnn.png#mw-jump-to-license 16. Frauenfelder E, Santoianni F (1997) Nuove frontiere della ricerca pedagogica tra bioscienze e cibernetica. Ed. Scientifiche Italiane, Napoli 17. Akour M, Alsghaier H, Alqasem O (2020) The effectiveness of using deep learning algorithms in predicting students achievements. Indones J Electr Eng Comput Sci 14. https://doi.org/10. 11591/ijeecs.v19.i1.pp388-394 18. Li S, Liu T (2021) Performance prediction for higher education students using deep learning. Complexity 2021:e9958203. https://doi.org/10.1155/2021/9958203 19. Maggino F (2005), L’analisi dei dati nell’indagine statistica. Firenze University Press.

9

Augmented Reality in Higher Education an Exploratory Study on the Beliefs of Medical Students Massimo Marcuccio, Lucia Manzoli, Irene Neri, Laura Cercenelli, Giovanni Badiali, Maria Elena Tassinari, Gustavo Marfia, Emanuela Marcelli, and Stefano Ratti

Abstract

Augmented reality (AR) is changing the field of university teaching, including teaching human anatomy, which is one of the fundamental areas in medical education. Within this process, in the academic years 2021–22 and 2022–23 at the International School of Medicine and Surgery of the University of Bologna, a teaching laboratory was activated using the AEducAR 2.0 prototype developed by the authors. It is an educational tool integrating augmented reality and 3D printed objects developed by an interdisciplinary team of anatomists, maxillofacial surgeons, biomedical engineers and educational scientists. One hundred and thirty second-year medical students in the human anatomy course voluntarily joined the lab. After a didactic tutor presented the tool’s functionality, they were asked to study a new topic through a guided pathway run by software loaded on a tablet and divided into two activities (one exploratory and one interactive) with the support of a skull processed by a 3D printer. After each activity, students had to take a test to assess learning outcomes. During the lab, the behaviour of 12 students was videotaped, as well as their activity on the tablet. At the end of the workshop, all students filled out a questionnaire, and 10 of them voluntarily participated in a semistructured interview to survey their views about the relationship between the prototype and other teaching methods and the function of assessment tests within AEducAR 2.0. This paper presents the outcomes of the latter research activity. Students stated the effectiveness of the prototype in terms of engagement. They emphasized its positive aspects related primarily to the manipulative and active possibility that is also functional to the knowledge memorization process. The hypothesis of integration between the laboratory experience and the activity in the septic room was also

M. Marcuccio (B) · L. Manzoli · I. Neri · L. Cercenelli · G. Badiali · M. E. Tassinari · G. Marfia · E. Marcelli · S. Ratti Alma Mater Studiorum - University of Bologna, Bologna, Italy e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 F. Santoianni et al. (eds.), Mind, Body, and Digital Brains, Integrated Science 20, https://doi.org/10.1007/978-3-031-58363-6_9

125

126

M. Marcuccio et al.

advanced. Regarding evaluation, the choice of having timely feedback on the given answers was considered positive.

1

Introduction

The patho-biological knowledge of anatomical regions and their interconnections guarantees future physicians’ better surgical and clinical performance [1]. Therefore, developing an effective teaching method for human anatomy has been a long-standing goal. Cadaver dissection remains the gold standard teaching method for macroscopic anatomy [2]. Nonetheless, there has been a notable transformation in the didactical approach to this subject in the past few years. A paradigm shift is underway as modern technological tools advance, enabling students to delve into threedimensional anatomical structures. This opportunity facilitates immersive and interactive learning experiences that prove challenging to emulate in conventional academic environments [3]. However, introducing technological innovation processes in education has repercussions on both technical-instrumental aspects and other dimensions of teaching practice: the subjects involved (students and teachers), training curricula and university organisations. The interaction of new technologies with students’ characteristics (beliefs, attitudes, expectations, motivations) constitutes an essential element to be analysed as it influences choices related to teaching/ learning processes [4, 5]. This research area was developed in human anatomy education [6], but only partially in relation to didactic innovation connected to new technologies [7]. For this reason, with the following research we present some outcomes in this new field. Our goal has been to carry out an initial exploration on university students’ beliefs about the introduction of an optimised version of a human anatomy teaching laboratory named AEducAR [8] in which augmented reality was integrated with 3D printing.

2

Theoretical Framework

Augmented reality (AR) is a technology that facilitates the integration of digital content into the real world. AR enables interaction with diverse anatomical structures in medical anatomy by overlaying additional information onto tangible 3D anatomical models [9–11]. Furthermore, 3D printing plays a role in medical education and surgical training [12–15]. In addition, in the field of three-dimensional virtual reality, the introduction of new features in digital anatomical atlases with positive outcomes on learning is under continuous development [16–18].

9 Augmented Reality in Higher Education an Exploratory Study on the Beliefs …

127

The healthcare community advocates integrating emerging technologies into medical training [19]. Notably, augmented reality has emerged as a beneficial tool to supplement cadaver dissection. However, it is crucial to emphasize that technology should not entirely replace cadaver dissection, as the latter provides a direct and irreplaceable approach to understanding the human body [20, 21]. Introducing AR into teaching human anatomy represents a strategic solution to address challenges associated with cadaver dissection, such as the limited availability of bodies or heightened emotional involvement [22, 23]. Moreover, AR can enhance medical students’ educational experience by providing an immersive and interactive learning experience [24–27]. Numerous research endeavours have been undertaken to assess the efficacy of augmented reality by gauging learning outcomes and showcasing its overarching potential [28, 29]. In contrast, few studies analyse other aspects that may influence the introduction of teaching innovations to improve the teaching/learning process [7]. However, the analysis of the socio-cultural background, beliefs, learning styles, students’ teaching choices, their emotional reactions, and the physical environment becomes a critical moment to fully understand the innovation process [6, 30–32]. Indeed, these factors can influence learning outcomes and foster the design of new experiences. For example, in the learning experience with the AEducAR prototype [8], in which virtual reality was integrated with 3D object printing,1 the analysis of medical students’ perceptions collected using a questionnaire revealed that it is not only perceived as a valid didactic tool that fosters the development of anatomical knowledge comparable to traditional textbooks but also as a didactic situation capable of increasing motivation and involvement. Based on the latest research results, an optimised version of the tool—AEducAR 2.0—was developed to increase its interactive aspects and was first tested with first-year medical students.

3

The AEducAR 2.0 Teaching Lab

To carry out the AEducAR 2.0 anatomy course activity, the students had a skull created with a 3D printer, a set of plastic models of facial muscles, and a tablet with software to visualise the virtual images in AR (Fig. 1). The experiment developed in two macro phases (learning and assessment). In the first (Track and Explore), the student explored nerves and eye muscles, eye

1

This is a device that sought to integrate the principles of Mayer’s cognitive theory of multimedia learning (CTML) [43] with Kolb’s experiential learning theory [45]. In this regard, in Sommerauer & Müller’s systematic review outcomes [46, p. 5], it is stated that “the creation of an effective learning experience may require the incorporation of ideas from more than one learning theory”.

128

M. Marcuccio et al.

Fig. 1 The tools available to students

Fig. 2 Track and explore: example of “Nerves and Eye Muscles” exploration

movements, and facial bones2 (Fig. 2) and then completed a Quiz consisting of five multiple-choice questions (Figs. 3, 4 and 5).3 The second macro-phase (Place and Check) involves two moments. The first consisted of two phases: exploration, in which the student explored the facial muscles, making them appear on the screen also in a recursive manner (Fig. 6); place and check, in which the student placed the plastic models of the eye muscles on the skull, sequentially, and subsequently checked their correct positioning using the application (Fig. 7).

2

The “SHOW/HIDE target” button allows the display of the virtual skull model and the overlay of the virtual content. 3 The Check button provided immediate feedback by highlighting the correct answer in green.

9 Augmented Reality in Higher Education an Exploratory Study on the Beliefs …

129

Fig. 3 Track and explore: example of “Eye Movement” exploration

Fig. 4 Track and explore: “Facial Bones” exploration example

Fig. 5 Track and explore: example of a quiz question

At the end of these two phases, there was a second quiz consisting of three multiple-choice questions (Fig. 8).

130

M. Marcuccio et al.

Fig. 6 Place and Check: example of an exploratory phase

Fig. 7 Place and Check phase example

Fig. 8 Track and Explore: Quiz example

4

Objectives and Research Questions

The objectives of the research were as follows: – detect students’ beliefs about the experience; – detect students’ learning levels; – describe the behaviour of students and tutors during the workshop session;

9 Augmented Reality in Higher Education an Exploratory Study on the Beliefs …

131

– detect aspects of AEducAR to be improved. Specifically, the research questions were as follows: – What were the students’ reactions to the experience? Was the experience perceived helpful in comparison with the dissection of cadavers in the dissecting room and digital atlases? – which learning level was achieved by the students at the end of the activity? – What were the behaviours of students and tutors during the experience? Which behaviours were functional to the learning process? – What are the functionalities of AEducAR that can be improved?

5

Method

5.1

Participants

During the second semester of the academic years 2021/2022 and 2022/2023, 130 students (56% female; 61% Italian; 85% aged 19–23) in the second year of the Bologna International School of Medicine (BOMS) voluntarily enrolled in the Anatomy laboratory of the nervous system and sensory organs.

5.2

Data Collection

Data were collected using four instruments. At the end of the workshop, researchers administered an anonymous questionnaire structured in two parts: the first (3 questions) on socio-demographic information; the second (9 items in Likert format and two open-ended questions) aimed at gathering information about previous experiences with AR, the study method used, enjoyment, usefulness of the activity for medical training, and general comments. Learning data was collected through the two quizzes. The external behaviour of 12 student volunteers was video-recorded via video cameras, and at the same time, the moves made within the activity were videorecorded via tablet. Finally, ten student volunteers (four females and six males) between the ages of 20 and 25 were involved in a semi-structured interview to detect their beliefs about the novelty of the experience; the comparison between AEducAR 2.0, cadaver dissection and digital atlases and learning assessment tests. We will examine only the outcomes of the latter data collection activity in this contribution.

132

5.3

M. Marcuccio et al.

Procedures and Tools for Conducting Interviews and Analysing Transcripts

Given the exploratory nature, researchers conducted conversationally ‘semistructured’ interviews [33–35] inspired by Lumbelli’s indications on non-directive interviewing [36], including also probing questions to avoid possible response effects [37], primarily social desirability. With the consent of the students, the interviews were audio-recorded. The audio recordings were transcribed verbatim, and the transcripts were analysed using a qualitative content analysis with MAXQDA software. The codebook was constructed using a top-down and bottom-up approach [38]: the main content categories were defined a priori while being supplemented with new categories that emerged during the analysis of the transcripts. For the restitution of the elements, fragments of the transcripts were reorganised into an organic text around the main themes of interest, bringing out similarities and differences between the student’s statements.

6

Results

The first area of enquiry concerned the students’ activity about using IT tools for study and the experience’s novelty level. The student’s previous experience in the use of IT tools for the study was reasonably widespread, and their judgements were overwhelmingly positive. For example, online software (3D online atlases)4 was used. They are in some cases preferred to paper ones because they “are very convenient, they are fast, intuitive” (Stud01), and they allow “to have just the depth, the mutual positions: in two-dimensional space, it is more difficult” (Stud01). The complementarity of online tools with traditional ones was also emphasised: ‘Some [paper] atlases are very clear, clearer than three-dimensional ones. However, sometimes we cannot understand a concept well, to fix it well: then we use the [online] ones that are faster […] than an atlas that you have to scroll: they are complementary in my opinion’ (Stud01). In light of this, all interviewees underlined the novelty of the experience; they also added a perception of surprise at the precision of the technical aspects. Another area investigated was the relationship between the didactic strategy of AR (AEducAR 2.0) and other strategies used to foster the development of learning related to the study of anatomy. The comparison concerned the observation and tactile exploration of the cadaver in the septic room and the use of anatomical atlases in paper and computerised form (offline and online software). The comparison revealed an initial set of aspects of AEducAR 2.0 that offer added value to the learning process and some of its limitations.

4

For example, Anatomy 3D Atlas (https://anatomy3datlas.com/). Stud04 made explicit reference to Complete Anatomy (https://3d4medical.com) for a fee.

9 Augmented Reality in Higher Education an Exploratory Study on the Beliefs …

133

Specific aspects perceived as the added value of AEducAR 2.0 compared to the dissection room include, for example, the possibility of observing the exact position of muscles moving them as required. Furthermore, it is possible to manipulate the structure of the different components at a tactile level, understanding their shape and size. The critical points that emerged mainly concerned the texture of the different anatomical parts, which ‘can never be the same as that of a real body’ (Stud04) since they are made of plastic as they were designed for teaching purposes. In the light of the pros and cons that emerged from the comparison between AEducAR 2.0 and observation and exploration in the septum room, some students propose an “integration” (Stud10) and declare a “complementarity” (Stud01; Stud03) between the two teaching methods: “both provide realistic help” (Stud03). Furthermore, in one case, it emerged that the dissection room and AEducAR 2.0 equally succeed in providing opportunities to overcome a difficulty in studying anatomy, namely that of allowing one to depict through the imagination aspects of which it is challenging to have direct experience. Regarding the comparison with digital anatomical atlases, the added value of AEducAR 2.0 is mainly noticeable in the second phase of the experience, namely Place and Check. The positive aspects concern, for example, the possibility of fully reconstructing the arrangement of the muscles, moving them and placing them in the correct positions. In addition, memorising the different anatomical parts took on great relevance ‘because first you look at them, then you try to memorise them, then you place them, also doing an active part, which in my opinion remains much more in mind’ (Stud01). About memorisation, Stud04 made explicit reference to ‘active recalling’ techniques and the colouring of the different parts of the skull, emphasising their effectiveness for mnemonic purposes. In addition, the importance of the appearance of explanation windows at the selected muscle was emphasised. The analysis of the students’ words revealed that the main limitation of AEducAR 2.0 in comparison to computerised atlases concerns the issue of perspective and depth of vision: “The depth was a little bit more complicated to handle especially […] superimposed, without really understanding the depth at which it was […] my problem is when you see a structure, but you cannot place it well in space” (Stud01); “you could not have much perspective, you had to stand frontally and to distinguish other structures you had to see them from a different perspective” (Stud01). A structural part of the AEducAR 2.0 experience consisted of the assessment of learning through ‘quizzes’. At the outset, all students regarded the quizzes as moments of ‘usefulness’, ‘importance’ and ‘convenience’, citing immediate feedback on their learning as the main reason. Among the main positive aspects of quizzes, students emphasised the memorisation process; for example: “In my opinion, it helps to fix the names of things in my mind, to see, to correlate them immediately” (Stud03). Stud02 also confirms the positive aspect of the immediacy of the feedback on the memorisation

134

M. Marcuccio et al.

process: “It is useful to have immediate feedback”. The frequency of the assessment is also recognised as an essential factor in fostering learning: “The fact of having a continuous quiz, a continuous test stimulates me more to remember the various topics […] I am a person who needs to be tested continuously” (Stud02). Another aspect that emerged is the fact that the students considered the assessment moment of AEducAR 2.0 to be consistent with the learning self-regulation strategies of self-assessment or self-interrogation: “In my study method, I make myself flashcards with questions rather than reading and repeating, which is useless to me!” Students also emphasised that the moment of the ‘intermediate tests’ (Stud03) reinforce learning, whether the outcome is positive or negative. In this way, the theme of the learning value of error also emerges explicitly: “I prefer it if I get it wrong [the answer] because if I get it wrong, I remember why I got it wrong” (Stud02). Finally, Stud04 considered the presence of quizzes as an element in favour of AEducAR 2.0 over digital atlases.

7

Discussion

The analysis of the interview transcripts shows how the students overcame the “fascination effect”—or “wow effect”—that constitutes one of the main biases in the research results in applying virtual and augmented reality. The aspects considered positive were balanced by identifying several limitations interpreted as indications for the development of technical and didactic aspects of the AEducAR 2.0 experience. These elements led some students to hypothesise an integration between the laboratory experience and the activity in the dissecting room. This result complements the findings of Codd [39] and Lim [40] research, where the researchers only advanced the didactic integration between cadaver dissection and other strategies as a potential development of the innovation introduced. The students’ perceptions showed how, compared to digital atlases, the AEducAR 2.0 experience acquired value above all for its experiential-manipulative component, which, as Morris confirms [41], constitutes a motivating and supportive element in the process of memorising knowledge [42]. Students recognised the positive value in the different type of content and their structuring—especially in the second macro-phase of the activity—which, in sequential form, were organised first in visual form and then in visual-verbal form (pop-ups appearing) in order not to overload the students’ cognitive load, confirming in this one of the principles of Mayer’s multimedia learning theory [43]. Regarding learning assessment, the students’ perceptions partly confirm the findings of Jacobs et al. [44], who state that “feedback is necessary for the development of non-technical and technical skills, but cannot be completely derived from computer-supplied scores alone”. This result points to how the use of the device confirmed students’ “misconceptions” about assessment.

9 Augmented Reality in Higher Education an Exploratory Study on the Beliefs …

8

135

Conclusions

Starting from the considerations that emerged from the analysis and discussion of the data and the intrinsic limitations of this research related to the exploratory nature and the smallness of the sample, it is possible to identify some patterns of action for developing the AEducAR 2.0 laboratory. Firstly, the students suggested that it might be helpful to broaden the laboratory experience’s anatomical scope, extend the same laboratory format to other subjects, and set up a stable augmented reality anatomical laboratory that students can freely access in a regulated form. From a technical point of view, the tool could be improved by providing ways to rotate the structures to have different visual perspectives. About the assessment component within the device, a form of feedback could be designed that integrates the simple message about the correctness of the answer with additional elements also provided, for example, by the tutors who accompany the students during the laboratory experience.

References 1. Arráez-Aybar L, Sánchez-Montesinos I, Mirapeix R (2010) Relevance of human anatomy in daily clinical practice. Ann Anat 192(6):341–348 2. Ghosh S (2017) Cadaveric dissection as an educational tool for anatomical sciences in the 21st century. Anat Sci Educ 10(3):286–299 3. Patra A, Asghar A, Chaudhary P et al (2022) Integration of innovative educational technologies in anatomy teaching: new normal in anatomy education. Surg Radiol Anat 44(1):25–32 4. Dalim CSC, Kadhim H, Sunar MS et al (2017) Factors influencing the acceptance of augmented reality in education: a review of the literature. J Comput Sci 13(11):581–589 5. Cao W, Yu Z (2023) The impact of augmented reality on student attitudes, motivation, and learning achievements—a meta-analysis (2016–2023). Humanit Soc Sci Commun 10(1):1–12 6. Meyer AJ, Armson A, Losco CD et al (2015) Factors influencing student performance on the carpal bone test as a preliminary evaluation of anatomical knowledge retention. Anat Sci Educ 8(2):133–139 7. Backhouse S, Taylor D, Armitage J (2019) Is this mine to keep? Three-dimensional printing enables active, personalized learning in anatomy. Anat Sci Educ 12(5):518–528 8. Cercenelli L, De Stefano A, Billi AM et al (2022) Anatomical education in augmented reality: a pilot experience of an innovative educational tool combining ar technology and 3d printing. Int J Environ Res Public Health 19(3):1024 9. Sutherland J, Belec J, Sheikh A et al (2019) Applying modern virtual and augmented reality technologies to medical images and models. J Digit Imaging 32(1):38–53 10. Tang K, Cheng D, Mi E et al (2020) Augmented reality in medical education: a systematic review. Can Med Educ J 11(1):e81–e96 11. Venkatesan M, Mohan H, Ryan J et al (2021) Virtual and augmented reality for biomedical applications. Cell Rep Med 2(7):100348 12. Ganguli A, Pagan-Diaz G, Grant L et al (2018) 3d printing for preoperative planning and surgical training: a review. Biomed Microdevices 20(3):1–24 13. Leung G, Pickett A, Bartellas M et al (2022) Systematic review and meta-analysis of 3dprinting in otolaryngology education. Int J Pediatr Otorhinolaryngol 155:11083 14. Maglara E, Angelis S, Solia E et al (2022) Three-dimensional (3d) printing in orthopedics education. J Long-Term Eff Med Implants 30(4):255–258

136

M. Marcuccio et al.

15. Molinari G, Emiliani N, Cercenelli L et al (2022) Assessment of a novel patient-specific 3d printed multi-material simulator for endoscopic sinus surgery. Front Bioeng Biotechnol 10:974021 16. Smit N, Bruckner S (2019) Towards advanced interactive visualization for virtual atlases. In: Rea PM (ed) biomedical visualisation. Advances in experimental medicine and biology, vol 3, Springer, Cham, p 85–96 17. Schwartzman G, Ramamurti P (2021) Visible body human anatomy atlas: innovative anatomy learning. J Digit Imaging 34(5):1328–1330 18. Gloy K, Weyhe P, Nerenz E et al (2022) Immersive anatomy atlas: learning factual medical knowledge in a virtual reality environment. Anat Sci Educ 15(2):360–368 19. Tang Y, Chau K, Pak K et al (2022) A systematic review of immersive technology applications for medical practice and education: trends, application areas, recipients, teaching contents, evaluation methods, and performance. Educ Res Rev 35:100429 20. Arráez-Aybar L, García-Mata R, Murillo-González J et al (2021) Physicians’ viewpoints on faculty anatomists and dissection of human bodies in the undergraduate medical studies. Ann Anat 238:151786 21. Moro C, Štromberga Z, Raikos A et al (2017) The effectiveness of virtual and augmented reality in health sciences and medical anatomy. Anat Sci Educ 10(6):549–559 22. Winkelmann A (2016) Consent and consensus-ethical perspectives on obtaining bodies for anatomical dissection. Clin Anat 29(1):70–77 23. Wisenden P, Budke K, Klemetson C et al (2018) Emotional response of undergraduates to cadaver dissection. Clin Anat 31(2):224–230 24. Dhar P, Rocks T, Samarasinghe R et al (2021) Augmented reality in medical education: students’ experiences and learning outcomes. Med Educ Online 26(1):1953953 25. Huang K, Ball C, Francis J et al (2019) Augmented versus virtual reality in education: an exploratory study examining science knowledge retention when using augmented reality/ virtual reality mobile applications. Cyberpsychol Behav Soc Netw 22(2):105–110 26. Parsons D, MacCallum K (2021) Current perspectives on augmented reality in medical education: applications, affordances and limitations. Adv Med Educ Pract 12:77–91 27. Sandrone S, Carlson CE (2021) Future of neurology & technology: virtual and augmented reality in neurology and neuroscience education: applications and curricular strategies. Neurology 97(15):740–744 28. Moro C, Birt J, Stromberga Z et al (2021) Virtual and augmented reality enhancements to medical and science student physiology and anatomy test performance: a systematic review and meta-analysis. Anat Sci Educ 14(3):368–373 29. Chytas D, Johnson E, Piagkou M et al (2020) The role of augmented reality in anatomical education: an overview. Ann Anat 229:151463 30. Al-Mohrej OA, Al-Ayedh N, Masuadi E et al (2017) Learning methods and strategies of anatomy among medical students in two different Institutions in Riyadh, Saudi Arabia. Med Teach 39:S15–S21 31. Chiou RJ, Tsai PF, Han DY (2015) Impacts of a gross anatomy laboratory course on medical students’ emotional reactions in Taiwan: the role of high-level emotions. BMC Med Educ 21(1):1–13 32. Sheffler P, Rodriguez T, Cheung C et al (2022) Cognitive and metacognitive, motivational, and resource considerations for learning new skills across the lifespan. Wiley Interdiscip Rev Cogn Sci 13(2):e1585 33. Corbetta P (1999) Metodologia e tecniche della ricerca sociale. il Mulino, Bologna 34. Lucisano P, Salerni A (2002) Metodologia della ricerca in educazione e formazione. Carocci, Roma 35. Bichi R (2007) La conduzione delle interviste nela ricerca sociale. Carocci, Roma 36. Lumbelli L (1984) Qualità e quantità nella ricerca empirica in pedagogia. In: Becchi E, Vertecchi B (eds) Manuale critico della sperimentazione e della ricerca educativa. Franco Angeli, Milano, pp 101–133 37. Zammuner V (1988) Tecniche dell’intervista e del questionario. il Mulino, Bologna

9 Augmented Reality in Higher Education an Exploratory Study on the Beliefs …

137

38. Lucidi F, Alivernini F, Pedon A (2008) Metodologia della ricerca qualitativa. il Mulino, Bologna 39. Codd AM, Choudhury B (2011) Virtual reality anatomy: Is it comparable with traditional methods in the teaching of human forearm musculoskeletal anatomy? Anat Sci Educ 4(3):119– 125 40. Lim KHA, Loo ZY, Goldie SJ et al (2016) Use of 3D printed models in medical education: a randomized control trial comparing 3D prints versus cadaveric materials for learning external cardiac anatomy. Anat Sci Educ 9(3):213–221 41. Morris TH (2020) Experiential learning–a systematic review and revision of Kolb’s model. Interact Learn Environ 28(8):1064–1077 42. Augustin M (2014) How to learn effectively in medical school: test yourself, learn actively, and repeat in intervals. Yale J Biol Med 87(2):207–212 43. Mayer RE (2005) Cognitive theory of multimedia learning. In: Mayer RE (ed) The Cambridge handbook of multimedia learning. Cambridge University Press, Cambridge, pp 31–48 44. Jacobs C, Foote G, Joiner R et al (2022) A narrative review of immersive technology enhanced learning in healthcare education. Int Med Educ 1(2):43–72 45. Kolb DA (2015) Experiential learning: experience as the source of learning and development, 2nd ed. Pearson, Upper Saddle River 46. Sommerauer P. Müller O (2018) Augmented reality for teaching and learning: a literature review on theoretical and empirical foundations. In: 26th European conference on information systems: beyond digitization—facets of socio-technical change, ECIS 2018, Portsmouth, UK, June 23–28, 2018

Part III Digital Body and Digital Brains: Brain-Computer Interactions, Neuroergonomics, and Psychomotor Learning

10

Brain-Computer Interaction and Neuroergonomics

Fabien Lotte and Camille Jeunet-Kelway

Abstract

When studying human brains in relation with digital technologies, or digital brains, a relatively recent technology may prove particularly promising to do so: BrainComputer Interfaces (BCI). Indeed, BCI can decode measures of users’ brain activity in real-time, in order to enable direct control of computers via brain activity or to monitor users’ mental states when interacting with technologies (so-called neuroergonomics). This chapter presents an introductory overview of this technology, i.e., it describes its motivations, brief history, components, principles of operation and various applications, e.g. for assistive technologies, neurorehabilitation or safety, performance and user experience assessment and optimisation. It also touches on the various current limitations of this technology, which makes it rather different from the science-fiction-like representations it may evoke. Altogether, we hope this chapter can offer a brief but clear glimpse into what BCI can and cannot do, and motivate readers to possibly consider them in their future research and/or developments.

Fabien Lotte and Camille Jeunet-Kelway have contributed equally to this work. F. Lotte (B) Inria Center at the University of Bordeaux/LaBRI, 200 avenue de la vieille tour, 33405 Talence Cedex, France e-mail: [email protected] C. Jeunet-Kelway CNRS, EPHE, INCIA, UMR5287, Université de Bordeaux, Bât. 2A- 2ème étage, 146 rue Léo Saignat, 33076 Bordeaux Cedex, France e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 F. Santoianni et al. (eds.), Mind, Body, and Digital Brains, Integrated Science 20, https://doi.org/10.1007/978-3-031-58363-6_10

141

142

F. Lotte and C. Jeunet-Kelway

1 Introduction Digital technologies, and especially neurotechnologies, provide breakthrough opportunities (i) to enhance one’s control over their body and (ii) to understand their “mind” (in terms of cognitive, motivational and emotional states). On the one hand, thanks to neurotechnologies, the brain can be used to send direct commands to digital applications. Indeed, it is now possible to interact with our environment without moving: the control intent being directly extracted from the brain activity and translated into a command for an external device. Thus, mobility and communication can be restored in paralysed patients, and even enabled or enhanced for all for instance for navigation in a “metaverse”, or in contexts of distant interaction. A whole research field is dedicated to the optimisation of those systems called Brain-Computer Interfaces (BCIs). Among BCIs, two types, named “active” and “reactive” BCIs, are especially relevant for enhancing or restoring communication and control. We provide details on their functioning and association challenges hereinafter. On the other hand, neurotechnologies can be used to detect, directly from one’s brain, cognitive, motivational or emotional states. This information can then be used to optimise the human-technology interaction, and thereby one’s well-being and safety. Thus, for instance, pilots’ attention levels can be assessed in real-time during critical situations, and the information provided in the cockpit adapted accordingly so as to optimise decision-making. Again, a whole research field is dedicated to the optimisation of this approach, named neuroergonomics (NE), which is based on so-called “passive” BCIs. We also provide more details on the latter in the following sections of this chapter. We will first introduce the BCI technology: their historical background, their functioning, the different types (active, reactive, passive BCIs). Then, we will provide use-case examples and prospects for (i) active/reactive BCIs for communication and control, and (ii) passive BCIs for neuroergonomics.

2 Brain-Computer Interaction: From History to Principles 2.1 Historical Background BCIs have been defined in 2002 by Wolpaw et al. [1] as “a communication system in which messages or commands that an individual sends to the external world do not pass through the brain’s normal output pathways of peripheral nerves and muscles”. But the history of BCIs dates back to the end of the 19th century, with Dr. R. Caton, a neurologist from Liverpool, who was the first person to record the electrical activity produced by the brain. Those recordings were performed on dogs [2]. It is in 1924 that Dr. H. Berger (German neurologist and psychiatrist) first succeeded in applying this method to humans: the electroencephalography (EEG) was born [3]. In a nutshell, EEG consists in placing electrodes on the scalp to measure the electrical

10 Brain-Computer Interaction and Neuroergonomics

143

signals produced by the cortical neurons. H. Berger documented an increased of amplitude of this electrical activity during sleep and a disappearing when people would die. From the nineteen-thirties, after the method was validated by engineers, EEG increasingly developed together with the identification of different types of brain activities called “rhythms” (characterised by the associated frequency band and location) and of their functional roles. In the late fifties, J. Kamiya, psychologist and researcher at the University of Chicago, demonstrated that patients suffering from anxiety disorders could learn to self-regulate alpha rhythms and that this selfregulation ability correlated with an improvement of their clinical symptoms. This is one of the first protocols of neurofeedback. Neurofeedback consists in training people to voluntarily self-regulate specific brain patterns in order to improve or restore associated abilities. Since then, clinical neurofeedback has developed, especially for psychiatric (e.g., anxiety disorders, obsessive compulsive disorders) and neurological (e.g., epilepsy, Parkinson’s disease). In 1974, with the empowerment of informatics, Dr. J. Vidal, computer scientist at UCLA, suggested for the first time to use people’s ability to self-regulate brain patterns in order to send commands to external devices [4]. This is the birth of brain-computer interaction that, since then, have exponentially developed in terms of number of academic specialists and industry, and also in terms of applications (that range from video games to assistive technologies) [5]. Hereinafter is introduced the functioning of BCIs.

2.2 Overview of the Brain-Computer Interface (BCI) Loop Designing a BCI system requires setting up a communication loop—an interface— between the user’s brain and the computer, see Fig. 1. This so-called BCI loop [6], can be described as comprising five main elements, briefly presented below, and detailed in the subsequent sections: 1. Neuromarker production: For users, the first step to control a BCI is to “produce” a specific pattern of brain activity—a so-called neuromarker—that reflects their intentions or mental states. For instance, users can imagine a left or right hand movement, which will change their brain activity in their right or left sensorimotor cortex respectively. 2. Measurement of brain activity: Once the user has “produced” this neuromarker, the BCI will measure the user’s brain activity to later be able to identify which neuromarker can be recognised from it. Various neuroimaging sensors can be used to do so, the most used being EEG. 3. Brain signal processing: The brain activity being measured, the recorded signals will then be processed in order to identify which neuromarker is present, if any. For instance, this brain signal processing will try to identify whether the users’ EEG signals contain the neuromarker corresponding to a left or right hand imagined movement. To do so, various signal processing and machine learning algorithms are typically used.

144

F. Lotte and C. Jeunet-Kelway

Fig. 1 Schematic representation of the BCI loop, describing the main components enabling interaction between the brain and the computer

4. Decision: Once the neuromarker has been identified, a specific decision can be taken, usually associated to an action, e.g., the computer cursor can be moved towards the left if the neuromarker of a left-hand imagined movement was identified. 5. Feedback: Finally, the BCI loop can be closed by proposing a feedback to users, informing them about what neuromarker was identified, possibly together with an indicator of the confidence of the BCI system in this recognition. This feedback is essential to enable users to learn to control a BCI. BCI control is indeed a skill that needs to be learned and mastered, and such a learning is only possible with feedback. The following section will provide more details about these components.

2.2.1 Brain Activity Recording Techniques Various neuroimaging modalities are available to measure brain activity in BCIs [7]. They can be categorised as non-invasive and invasive, depending on whether the sensors are placed below the skull (for the latter) or on or around the skull (for the former). Non-invasive techniques include ElectroEncephaloGraphy (EEG), functional Near-InfraRed Spectroscopy (fNIRS), MagnetoEncephaloGraphy (MEG) and functional Magnetic Resonance Imagining (fMRI). EEG and MEG measure the brain electrical and magnetic activity respectively, while fNIRS and fMRI measure the blood concentration in oxygen in the brain, which reflects which brain areas are the most active. Among these techniques, fMRI and MEG have the best spatial resolution—i.e., they can measure the brain activity of the smallest and deepest brain regions with most precision. Current MEG and fMRI systems are however

10 Brain-Computer Interaction and Neuroergonomics

145

very bulky, cannot be transported, and very costly (at least around a million euros each). In contrast, EEG and fNIRS are portable and relatively cheap. Note that both fMRI and fNIRS suffer from a low temporal resolution, i.e., they can acquire brain activity measures only once or a few times per second, contrary to EEG and MEG which can do so typically hundreds of time per second. Invasive techniques include mainly ElectroCorticoGraphy (ECoG) and Micro-Electro Array (MEA). ECoG are sensors measuring the brain electrical activity below the skull, on top of the cortex (the external layer of the brain), while MEA measures neuron spiking activity, including possibly single neuron activity, using electrodes penetrating into the brain. Invasive techniques usually provide a much better signal-to-noise-ratio and a much better spatial resolution than non-invasive methods (with MEA having the best spatial resolution), at the cost of a limited coverage of brain activity (only a few or even a single brain area can be measured) and the need for surgery to implant the sensors. Currently, there is no non-invasive brain activity recording technique that can measure brain activity in the whole brain, with both high spatial and temporal resolution. Thus, currently, EEG is by far the most used techniques (followed by fNIRS), as it is relatively cheap, portable, non-invasive and with a high temporal resolution. In the remainder of this chapter, we will thus focus on EEG-based BCI applications for communication, control and neuroergonomics.

2.2.2 Neuromarkers Once measured, the brain activity is to be analysed to identify neuromarkers. In EEG, two main families of neuromarkers are typically analysed: (1) Event-Related Potentials (ERPs), which are brain responses to specific events or stimuli, with a specific time course, and (2) oscillatory activity, which corresponds to ongoing changes in EEG oscillations, i.e., changes in the EEG signal power in various frequency bands. The amplitude of the ERPs at different latencies following a stimulus and the power of specific EEG rhythms at different sensor locations have been associated with different mental states or intentions, and can thus be used to decode the latter. In addition to these two most common neuromarker types, additional ones can be used, such as measures of brain signal complexities or measures of synchronisation between the signals from different brain areas (so-called functional connectivity). For an overview of the different neuromarkers that can be exploited in BCIs and the associated BCI paradigms, the interested reader can refer to [8]. We will describe more specific neuromarkers later in this chapter, when describing different BCI types.

2.2.3 Signal Processing and Decision In order to automatically infer BCI users’ intentions and mental states from their EEG signals, signal processing and machine learning algorithms are typically used. Usually, this signal processing pipeline starts with a preprocessing step, to clean and denoise EEG signals, which are typically affected by various sources of noise and artifacts [9]. Such artefacts include for instance muscle artefacts (electromyography— e.g., from facial muscles) that are also recorded by EEG sensors but are not originating

146

F. Lotte and C. Jeunet-Kelway

from the brain and thus degrade the EEG signal quality. Then, a step called feature extraction is performed in order to describe EEG by (ideally a few) relevant values, called features, for instance by computing the power of EEG signals in various frequency bands and sensors, when aiming at identifying specific oscillatory activity neuromarkers [10]. Finally, machine learning algorithms are used to automatically identify which features correspond to which neuromarker, for instance, whether the estimated power in various frequency bands and sensors correspond to a left or right hand imagined movement. Typically, when a BCI is being used for the first time, there is a need for a calibration phase, during which the user will “produce” known neuromarkers upon instructions, e.g., by repeatedly imagining a specific hand movement at a specific time. The EEG signals collected during that phase will be used as labelled training data (i.e., labelled with the known intention or mental state in which the user was) to train machine learning algorithms. Once calibrated, the machine learning algorithm will be able to recognise the users’ intention/mental state from features extracted from ongoing (and unlabelled) EEG signals. Typical algorithms used for this purpose include linear classification algorithms such as Linear Discriminant Analysis (LDA) or Support Vector Machine (SVM) [11]. Recently, advanced non-linear classification algorithms such as Riemannian geometry classifiers or Deep Learning are increasingly more explored [12]. Once the users’ intention/state has been estimated by these algorithms, they can be associated with a specific command for the applications controlled by the BCI, e.g., making a wheelchair turn left when an imagined left-hand movement has been recognised.

2.2.4 Feedback The last component of the BCI loop is the feedback the users are provided with to help them learn to control a BCI. The most typical feedback type is a visual gauge, e.g., a gauge extending towards the left or right to indicate that the BCI has recognised a left or right imagined hand movement respectively, with the length of the gauge representing the confidence of the BCI in its estimation. However, recent research has shown that different types of feedback can help BCI users learn better and faster, e.g., richer and more realistic feedback (e.g., seeing a virtual hand moving, possibly in immersive virtual reality, when an imagined movement is detected) [13], multimodal feedback combining visual and tactile or audio feedback (the former being usually more effective than the later) [14], or even social feedback with artificial learning companions providing advice and guidance to BCI trainees based on their performance and progression [15]. For an overview of BCI feedback research, the interested reader can refer to [16]. This feedback learning process can enable BCI users to learn BCI control and thus to increase their proficiency: their mental commands can be increasingly more accurately recognised by the BCI with such feedback training.

10 Brain-Computer Interaction and Neuroergonomics

147

2.3 Three Main Types of BCIs Zander and Kothe [17] have suggested a classification of BCIs into 3 main categories characterised by (i) specificities in the BCI loop and (ii) the applications they are the most relevant for.

2.3.1 Active BCIs Objective. The objective of active BCIs is to send voluntary commands to external applications. Functionning. The specificity of active BCIs is that they allow asynchronous control. In other terms, users decide when they want to send a command— the later being most often emitted through mental imagery. Indeed, as mentioned earlier, mental imagery tasks such as mental calculation, mental navigation or motor imagery (i.e. imagination of movements) are associated with modulations of specific brain oscillations that can be detected using the EEG. Hence, during the calibration of the BCI, users are asked to perform mental imagery tasks (for instance imagining left and right hand motor imagery). The associated brain patterns (here, theoretically, modulations of sensorimotor rhythms over the right and left sensorimotor cortices, respectively) are measured and associated with specific commands, e.g., making a wheelchair turn left and right respectively. After some training, that includes informative (neuro)feedback, users are supposedly able to control the wheelchair through the voluntary, and asynchronous (self-paced), imagination of left and right hand movements. Challenges. The asynchronous control of active BCIs assumes a continuous recording of the brain activity and the ability to detect intentional commands and only intentional commands. Yet, in real-life settings, when people interact with their environment, a lot of modulations occur in their brains thus making “false positive detection” most likely. A first challenge thus consists in identifying paradigms that minimise the false positives (command sent while not intended) and also false negatives (no command detected while there was an intention to). A second major challenge for this kind of BCIs is the user training. Indeed, users have to learn to selfregulate specific brain activities to produce patterns that are (i) stable (the same as the ones produced during the calibration) so that they can be recognised and translated into commands, and also (ii) distinct between the commands so that this translation is reliable. Many factors seem to impact those self-regulation abilities, including individual characteristics (e.g., personality, cognitive profile) and the interface and training design (e.g., instructions, feedback) [16]. A main challenge thus consists in adapting the training procedures (and especially the feedback) so that a reliable and efficient BCI control is accessible to as many users as possible.

2.3.2 Reactive BCIs Objective. The objective of reactive BCIs is also to send voluntary commands to external applications. Functioning. Contrary to active BCIs, reactive BCIs mostly enable synchronous control. Indeed, reactive BCIs get their name from the fact that commands are sent through the brain patterns generated automatically as a reaction

148

F. Lotte and C. Jeunet-Kelway

to stimuli of interest produced by the system, and to which the users pay attention to. Those brain patterns are called event-related potentials (ERPs). Many different ERPs exist. Among them, the P300 and SSVEPs (for steady-state visual evoked potentials) are currently the most used in brain-computer interaction due to the relative ease to measure them using EEG. The P300 is a positive (P) potential occurring around 300 ms after the appearance of stimulus of interest on which one focuses their attention. In the P300 speller application for instance, which has been designed to enable severely paralysed patients to communicate, letters/symbols flash the ones after the others. Each time the symbol the patient wants to select flashes, a P300 is triggered. Several flashes are required for smoothing the brain signal (increasing the signalto-noise-ratio) and enabling a reliable detection of this P300 and a translation into a command. SSVEP-based BCIs rely on the fact that observing a light flashing at a certain frequency will induce an increase of the amplitude of the brain activity at the same frequency in the visual cortex. Thus, equipping for instance different domestic appliances with LED lights flashing at different frequencies could enable switching them on/off simply by looking at them. Despite the constraint of enabling only synchronous control, reactive BCIs have a main advantage being that they are very reliable with performances around 90% of good detection. Challenges. The main challenge with reactive BCIs is to prevent the visual fatigue generated by the stimuli, for instance by using other sensory modalities (auditory and tactile stimuli can be used).

2.3.3 Passive BCIs Objective. The objective of passive BCIs is to measure the mental states of a person in order to adapt an interactive system accordingly, in real-time. Mental states encompasses cognitive (e.g., attention), motivational and emotional (e.g., frustration) states. Functioning. Passive BCIs rely on our capacity to identify brain activity patterns (oscillations or ERPs) that specifically underlie mental states of interest and that can be reliably detected using the brain recording method used (i.e., most often, the EEG). Indeed, based on those patterns, it is then possible to infer the mental state of a person and to adapt the system with which they are interacting accordingly, in real-time. For instance, the difficulty of training exercises can be reduced when high mental workload levels are detected. One kind of ERP is especially relevant for passive BCIs, the so-called error potentials that are triggered when one notices an error in the interaction they are involved in. For instance, when manipulating a mouse, if the cursor stops moving unexpectedly, an error potential will likely be elicited. Error potentials can be used for instance to evaluate user experience: if error potentials are elicited when people click on a button because the page displayed is not the one they expected to see, then the application interface should be revised. Challenges. The main challenge here consists in identifying neuromarkers (i.e., brain activity patterns) that are both reliable and specific. Brain activity variability is far from being fully understood, and identifying invariants representing specific mental states whatever the mental/motor task performed is a real challenge, what is more due to the

10 Brain-Computer Interaction and Neuroergonomics

149

fact that such mental states usually involve multiple brain areas, both at the cortical and sub-cortical levels (the latter being difficult to assess using the EEG).

3 BCIs for Communication and Control: Applications, Limitations and Prospects Now that BCI principles are exposed, the following sections will present their main application areas, starting by using them for communication and control, which can be notably useful for assistive technologies, entertainment or neurorehabilitation.

3.1 Control of Assistive Technologies for Communication and Mobility Applications. Assistive technologies benefit a lot from neurotechnologies and BCIs in particular as the latter enable severely paralysed persons to control external devices, without moving. Two main application areas are developed: mobility and communication [18]. Smart wheelchairs, prosthetics and exoskeletons are certainly the most emblematic applications of active BCIs. Regarding communication, the most relevant paradigm seems to be reactive BCIs with, for instance, the P300 speller introduced earlier that enables a digital spelling. Limitations. As mentioned earlier in this chapter, active control through mental imagery is not very reliable yet and requires a lot of cognitive resources. Therefore, most systems are coupled with other paradigms to increase reliability. For instance, BCI-controlled wheelchairs might be equipped with infrared sensors to detect and avoid obstacles and thereby rely on a shared-control paradigm [19]. Active (mental imagery) and passive (error potentials) BCIs can also be combined so that an erroneous recognition of a mental command can be detected thanks to the elicitation of an error potential and corrected. Prospects. The reactive BCI-based communication applications are very reliable and increasingly usable thanks to a reduction of the time required to select letters/symbols and thus communicate. Regarding active BCI-based mobility applications, current research and development directions suggest that for them to be sufficiently reliable, EEG should be replaced by electrocorticography or intra-cortical electrode arrays as the latter provide a better signal-to-noise ratio and more stable data [20]. With those invasive methods, relying on a surgical operation and continuous recordings of brain activity, come multiple ethical concerns including questions of acceptability, accessibility, safety and benefits/risk ratio.

3.2 Enhancing Interaction for Entertainment Applications Applications. BCIs can also be used, for healthy users and motor impaired users alike, as an alternative or additional control modality for entertainment applications,

150

F. Lotte and C. Jeunet-Kelway

typically video games and virtual reality [21,22]. Typically active BCIs can be used to navigate virtual environments by using mental imagery, while reactive BCIs, notably P300-BCIs or SSVEP-BCIs, can be used to select virtual objects or to activate virtual buttons that are flashing or flickering. Such BCI-based commands can either replace classical mice/game pads, to provide a new and original control modality, or be used as a complementary command to classical game pads. For the former, various video game prototypes have been proposed, in which virtual avatars are fully controlled by a BCI [21,22], whereas for the later, the avatar can be controlled by a game pad and additional in-game super-power can be triggered using the BCI, see, e.g., [23,24]. Some of such applications are even commercialised now, see, e.g., the NextMind system that is based on SSVEP-like neuromarkers for gaming and virtual reality. Limitations. While all such applications are particularly appealing for the general public, it should be mentioned that current BCIs are not reliable, and make frequent mistakes in the mental command recognised. As such, fully controlling a video game using a BCI can be rapidly frustrating and demotivating. While SSVEP-related BCIs are more reliable than active BCIs and thus possibly more useful for this type of application, it should be stressed that game pads and computer mice are still by far the most efficient and effective control devices for video games. Prospects. Future research in BCI for entertainment thus needs to either vastly improve the reliability of BCIs, or find new ways to use BCIs for entertainment that are less reliant on reliable control. The former is less likely in the short term. For the latter, it may seem more promising to use passive BCIs (rather than active or reactive ones) for dynamically adapting the game content and/or parameter (e.g., difficulty) to the players’ mental states.

3.3 Neurorehabilitation Applications. For neurological and psychiatric pathologies associated with wellidentified pathological brain patterns, active BCIs and neurofeedback can be used to enhance (neuro)rehabilitation procedures and favour motor and cognitive recovery. For instance, in patients who underwent a stroke and experience motor aftereffects, standard rehabilitation procedures involve mental imagery: patients are asked to imagine movements in order to stimulate their sensorimotor cortex, foster synaptic plasticity (i.e., the reconstruction of connections between the neurons) and thereby favour motor recovery. The main limitation of this approach is that therapists do not know (i) when exactly the patient imagines the movement, which results in the impossibility to provide a synchronised feedback, and (ii) if the patient actually triggers their sensorimotor cortex (or other brain areas, in which case the impact on motor recovery would be reduced). With BCIs, it is possible to detect modulations of brain activity in the sensorimotor cortex and thereby (i) to provide a real-time feedback, synchronised with the imagination, thus closing the sensorimotor loop and fostering synaptic plasticity, and (ii) to inform the therapist so that they can guide the patients to identify the most efficient strategies. Beyond stroke rehabilitation, the number of clinical studies investigating the efficiency of BCIs to enhance the

10 Brain-Computer Interaction and Neuroergonomics

151

efficiency of neurorehabilitation procedures for patients with Parkinson’s disease, epilepsy, post-traumatic stress disorders, attention deficits and many others has significantly increased in the last years. Limitations. While theoretically more efficient than standard approaches (especially for attention deficits [25] and stroke rehabilitation [26]), BCI-based approach remain barely used in clinical settings. This can be explained, at least in part, by acceptability factors: it is not enough to design a technology, the latter then has to be usable by the end-users [27]. Therapists should be trained, hospitals should allocate specific funding, the population (including the clinicians, patients and their family) should be properly informed. Prospects. BCI technologies will become increasingly affordable. They are also increasingly publicised. Therefore, if inderdisciplinary and intersectoral collaborations arise in the coming years, and if acceptability and organisational constraints are considered, BCIs for neurorehabilitation should develop a lot in the near future.

4 BCIs for Neuroergonomics: Applications, Limitations and Prospects In addition to using BCIs for controlling various interactive systems, BCIs can be used for Neuroergonomics, to study the brain in the wild, notably to assess and optimise safety, performance, learning or user experience directly through brain activity [28]. Hereafter we present three families of BCI-based neuroergonomics applications: (1) for safety critical systems such as aeronautics and transportation; (2) for training and education and (3) for user experience optimisation.

4.1 Safety and Performance in Aeronautics and Transportation Applications. In many critical systems with a human-in-the-loop, such as in aeronautics and transportation, there is a need to ensure the safety and performance of the operators across time and situations. Doing so notably requires monitoring those operators’ states (e.g., their fatigue or vigilance), to assess the impact of the latter on the critical system operation, and possibly optimise it dynamically so as to improve safety and performance. Interestingly enough, these operators’ mental states can be monitored using passive BCIs [29,30]. For instance, a number of research works have demonstrated the possibility to estimate mental states such as fatigue, attention, vigilance or mental workload in both plane pilots and/or car drivers [29,31]. In turns, these mental state estimates can be used to dynamically adapt the interaction with the system, e.g., changing the amount of information displayed in a plane cockpit to avoid pilot overload, or to increase the level of automation proposed in autonomous cars if the driver is too tired or not paying attention. The later type of technology would be called a NeuroAdaptive Technology (NAT), i.e., a system adapting its properties based on neural information [32]. Limitations. The main limitation of this neuroergonomic approach to safety and performance in aeronautics

152

F. Lotte and C. Jeunet-Kelway

and transportation lies in the specificity of the neuromarkers used to estimate the operators’ mental states mentioned above. Indeed, while many proof-of-concepts have been made, they were often made in a single context and for studying a single operator’s mental state. As such, it is unclear whether the identified neuromarkers and associated BCIs to identify them would still work if used when other mental states are also varying (e.g., does fatigue also impact the identified neuromarker of mental workload?). In other words, it is still unclear how specific these neuromarkers are, and how much they can be confounded by other states. Prospects. Future research in this area will thus need to study the specificity of the various neuromarkers identified, and possibly identify new neuromarkers, representing mental states that can be useful to also monitor in this context (e.g., information overload or attention tunnelling). Estimating such neuromarkers and mental states from EEG is also not enough to ensure useful NATs. Indeed, there is also a need to build models (typically computational models) describing how changes in the monitored mental states affect the operation of the critical system targeted, so as to be able to provide suitable and timely interventions (e.g., when to increase or decrease the system autonomy?), in order to optimise safety and performance.

4.2 Intelligent Tutoring Systems for Training and Education Applications. As mentioned earlier, passive BCIs can also be used to facilitate/improve learning. Intelligent tutoring systems [33] consist in computerised tutors that adapt in real-time in order to optimise users’ learning. To do so, they include a “student model” that makes use of behavioural (posture, number of clicks, …) and sometimes physiological (eye pupil size, skin conductance) to infer the mental state of the student (cognitive workload, frustration, attention, …). With BCIs, it is now possible to design “neuroadaptive” tutoring systems that make use directly of the student’s brain activity to infer their mental states. Brain activity information might be more specific than physiological data. For instance, variations of skin conductance can be related to emotions, but also to room temperature of circadian rhythm. Limitations. While promising, it should be noted that we are still far from the identification of stable and fully reliable neuromarkers of humans’ cognitive states. Given the complexity of brain activity and the low signal-to-noise ratio of EEG activity, we should all be careful when claiming that we can infer one’s mental state. Prospects. The most relevant approach seems to combine different indicators (self-reported, behavioural, physiological, neurophysiological) in order to define models of mental state indicators that are as precise and reliable as possible. Then, a compromise should be determined as precision (using multiple sensors) should not come at a cost of usability and acceptability. For those systems to be useful, they first have to be used. And multiple, expensive sensors might discourage stakeholders to adopt those promising technologies.

10 Brain-Computer Interaction and Neuroergonomics

153

4.3 Measuring and Improving User Experience in Interactive Set-Ups Applications. Beyond aeronautics, transportation and education/training, and maybe more generally, BCI and Neuroergonomics can be used to assess User eXperience (UX) with interactive systems in general. Typically, passive BCIs can be used to monitor UX with various input devices (e.g., mice, haptic or tactile devices)—to assess and compare these devices—but also to monitor UX with output devices (e.g., various displays), to monitor users’ perception [34,35]. For instance, it has been shown that mental workload could be monitored from EEG during 3D object manipulation tasks or 3D navigation tasks (input devices) [36]—to assess when and where an input can be too cognitively demanding to use; while EEG can also be used to assess visual comfort with stereoscopic displays [37] or to detect users’ perception with system errors displayed in virtual reality [38] (output devices). These types of assessment can be used in complement to traditional UX assessment methods (e.g., questionnaires or interview), to obtain better insights into the UX with a given system, and thus to design a better system. Like for aeronautics and transportation, such BCIbased UX assessments can be used to design NATs, that will use these assessments to dynamically reconfigure the user-interface, in order to optimise UX on the fly, e.g., by correcting errors detected in the users’ EEG (see [39] for a review). Limitations. The limitations here would be similar to that of safety-critical systems (although to a lesser extent as safety is usually less an issue in the present contexts) and to intelligent tutoring systems, i.e., the neuromarkers identified for various UX metrics may not be very specific, and thus the UX assessment should be considered carefully. Moreover, while EEG is suitable to be used in the design and testing phase of interactive devices with testers in the lab, it is currently not so usable and comfortable for everyday prolonged use. Prospects. Future works will thus need to focus on assessing the specificity of UX neuromarkers in complex real-life situations, and possibly to design EEG sensors that are more transparent and wearable, and passive BCIs that are more usable and acceptable altogether, so that NATs optimising UX could be used outside the lab.

5 Conclusion Overall, this chapter presented a brief and introductory overview of the current state of BCI technology and neuroergonomics. It presented the motivations for BCI research and how they work, notably regarding their main components that are neuromarkers, brain imaging modalities, signal processing and feedback. It also described what are the main applications and usages of BCIs, either for communication and control, which includes assistive technologies, entertainment and neurorehabilitation, or for Neuroergonomics, notably for assessing and optimising safety and performance in aeronautics and transportation, for education and training, or for optimising user experience with interactive systems. Altogether BCIs and Neuroergonomics are thus very promising to study human brains in relation with digital technologies, i.e., dig-

154

F. Lotte and C. Jeunet-Kelway

ital brains, and possibly to enhance this relationship using NATs. However, beyond science fiction and beyond unfortunately too-often exaggerated media representations, BCIs are (fortunately) far from being able to read minds, and currently still suffer from various limitations. There are notably not reliable enough, may lack specificity in the mental states they can recognise, and are uncomfortable, tedious and inconvenient to use in real-life, among other [40]. Consequently, there is still a lot of research needed before they can be used in daily life outside laboratories. The research community is working hard in that direction.

References 1. Wolpaw JR, Birbaumer N, McFarland DJ, Pfurtscheller G, Vaughan TM (2002) Brain– computer interfaces for communication and control. Clin Neurophysiol 113(6):767–791 2. Caton R (1875) Electrical currents of the brain. J Nerv Ment Dis 2(4):610 3. Berger H (1929) Über das elektroenkephalogramm des menschen. Arch Psychiatr Nervenkr 87(1):527–570 4. Vidal JJ (1973) Toward direct brain-computer communication. Ann Rev Biophys Bioeng 2(1):157–180 5. Nam CS, Nijholt A, Lotte F (eds) (2018) Brain-computer interfaces handbook: technological and theoretical advance. Taylor & Francis 6. Van Gerven M, Farquhar J, Schaefer R, Vlek R, Geuze J, Nijholt A, Ramsey N, Haselager P, Vuurpijl L, Gielen S et al (2009) The brain-computer interface cycle. J Neural Eng 6(4):041001 7. Wolpaw JR, Loeb GE, Allison BZ, Donchin E, do Nascimento OF, Heetderks WJ, Nijboer F, Shain WG, Turner JN. BCI meeting 2005-workshop on signals and recording methods. IEEE Trans Neural Syst Rehabil Eng 14(2):138–141 8. Abiri R, Borhani S, Sellers EW, Jiang Y, Zhao X (2019) A comprehensive review of EEG-based brain–computer interface paradigms. J Neural Eng 16(1):011001 9. Kafiul Islam Md, Rastegarnia A, Yang Z (2016) Methods for artifact detection and removal from scalp EEG: a review. Neurophysiol Clin/Clin Neurophysiol 46(4–5):287–305 10. Bashashati A, Fatourechi M, Ward RK, Birch GE (2007) A survey of signal processing algorithms in brain–computer interfaces based on electrical brain signals. J Neural Eng 4(2):R32 11. Lotte F, Congedo M, Lécuyer A, Lamarche F, Arnaldi B (2007) A review of classification algorithms for EEG-based brain-computer interfaces. J Neural Eng 4(2):R1 12. Lotte F, Bougrain L, Cichocki A, Clerc M, Congedo M, Rakotomamonjy A, Yger F (2018) A review of classification algorithms for EEG-based brain-computer interfaces: a 10 year update. J Neural Eng 15(3):031005 13. Alimardani M, Nishio S, Ishiguro H (2018) Brain-computer interface and motor imagery training: the role of visual feedback and embodiment. In: Evolving BCI therapy-engaging brain state dynamics 2(64) 14. Pillette L, N’kaoua B, Sabau R, Glize B, Lotte F (2021) Multi-session influence of two modalities of feedback and their order of presentation on MI-BCI user training. Multimod Technol Interact 5(3):12 15. Pillette L, Jeunet C, Mansencal B, N’kambou R, N’Kaoua B, Lotte F (2020) A physical learning companion for mental-imagery BCI user training. Int J Human-Comput Stud 136:102380 16. Roc A, Pillette L, Mladenovic J, Benaroch C, N’Kaoua B, Jeunet C, Lotte F (2021) A review of user training methods in brain computer interfaces based on mental tasks. J Neural Eng 18(1):011002

10 Brain-Computer Interaction and Neuroergonomics

155

17. Zander TO, Kothe C (2011) Towards passive brain–computer interfaces: applying brain– computer interface technology to human–machine systems in general. J Neural Eng 8(2):025005 18. del R Millán J, Rupp R, Mueller-Putz G, Murray-Smith R, Giugliemma C, Tangermann M, Vidaurre C, Cincotti F, Kubler A, Leeb R et al (2010) Combining brain–computer interfaces and assistive technologies: state-of-the-art and challenges. Front Neurosci 161 19. Carlson T, Monnard G, del R Millán J (2011) Vision-based shared control for a BCI wheelchair. Int J Bioelectromagn 13:20–21 20. Moly A, Costecalde T, Martel F, Martin M, Larzabal C, Karakas S, Verney A, Charvet G, Chabardes S, Benabid AL et al (2022) An adaptive closed-loop ECoG decoder for long-term and stable bimanual control of an exoskeleton by a tetraplegic. J Neural Eng 19(2):026021 21. Kerous B, Skola F, Liarokapis F (2018) EEG-based BCI and video games: a progress report. Virtual Real 22:119–135 22. Lécuyer A, Lotte F, Reilly RB, Leeb R, Hirose M, Slater M (2008) Brain-computer interfaces, virtual reality, and videogames. Computer 41(10):66–72 23. Nijholt A, Plass-Oude Bos D, Reuderink B (2009) Turning shortcomings into challenges: brain–computer interfaces for games. Entertain Comput 1(2):85–94 24. Klug M (2022) Real virtual magic–modifying a VR game with a BCI to enhance immersion. In: Proceedings of the neuro adaptive technologies (NAT) conference 25. Saif MGM (2023) Clinical efficacy of neurofeedback protocols in treatment of attention deficit/hyperactivity disorder (ADHD): a systematic review. Psychiatr Res Neuroimaging 111723 26. Bai Z, Fong KNK, Zhang JJ, Chan J, Ting KH (2020) Immediate and long-term effects of BCIbased rehabilitation of the upper extremity after stroke: a systematic review and meta-analysis. J Neuroeng Rehabil 17:1–20 27. Grevet E, Forge K, Tadiello S, Izac M, Amadieu F, Brunel L, Pillette L, Py J, Gasq D, JeunetKelway C (2023) Modeling the acceptability of BCIs for motor rehabilitation after stroke: a large scale study on the general public. Front Neuroergon 3:1082901 28. Ayaz H, Dehais F (2019) Neuroergonomics: the brain at work and in everyday life. Academic Press 29. Borghini G, Astolfi L, Vecchiato G, Mattia D, Babiloni F (2014) Measuring neurophysiological signals in aircraft pilots and car drivers for the assessment of mental workload, fatigue and drowsiness. Neurosci Biobehav Rev 44:58–75 30. Lotte F, Roy RN (2019) Brain–computer interface contributions to neuroergonomics. In: Neuroergonomics. Elsevier, pp 43–48 31. Dehais F, Duprès A, Blum S, Drougard N, Scannella S, Roy RN, Lotte F (2019) Monitoring pilot’s mental workload using ERPs and spectral power with a six-dry-electrode EEG system in real flight conditions. Sensors 19(6):1324 32. Fairclough SH, Zander TO (2021) Current research in neuroadaptive technology. Academic Press 33. Nkambou R, Mizoguchi R, Bourdeau J (2010) Advances in intelligent tutoring systems, vol 308. Springer Science & Business Media 34. Frey J, Mühl C, Lotte F, Hachet M (2014) Review of the use of electroencephalography as an evaluation method for human-computer interaction. In: International conference on physiological computing systems, vol 2. SCITEPRESS, pp 214–223 35. Frey J, Hachet M, Lotte F (2017) EEG-based neuroergonomics for 3d user interfaces: opportunities and challenges. Le Travail Humain 80(1):73–92 36. Frey J, Daniel M, Castet J, Hachet M, Lotte F (2016) Framework for electroencephalographybased evaluation of user experience. In: Proceedings of the 2016 CHI conference on human factors in computing systems, pp 2283–2294 37. Frey J, Appriou A, Lotte F, Hachet M (2016) Classifying EEG signals during stereoscopic visualization to estimate visual comfort. Comput Intell Neurosci 7–7:2016

156

F. Lotte and C. Jeunet-Kelway

38. Si-Mohammed H, Lopes-Dias C, Duarte M, Argelaguet F, Jeunet C, Casiez G, Müller-Putz GR, Lécuyer A, Scherer R (2020) Detecting system errors in virtual reality using EEG through error-related potentials. In: 2020 IEEE conference on virtual reality and 3D user interfaces (VR). IEEE, pp 653–661 39. Solovey ET, Putze F et al (2021) Improving HCI with brain input: review, trends, and outlook. Found Trends Human–Comput Interact 13(4):298–379 40. Fairclough SH, Lotte F (2020) Grand challenges in neurotechnology and system neuroergonomics. Front Neuroergon 2

11

Non-invasive Modulation of Brain Activity During Human-Machine Interactions Stefania C. Ficarella

Abstract

Cognitive Neuroscience investigates how the brain generates cognitive functions such as language, memory, vision, that allow an organism to efficiently interact with the environment. In recent years, the understanding of the mechanisms underlying cognitive functions has been boosted by the development of noninvasive brain stimulation techniques (NIBS). Such techniques use technologies that temporarily and painlessly modulate brain activity, without needing surgical intervention, thus providing a safe and accessible mean to explore and influence brain functions. The purpose of this chapter is to briefly present NIBS techniques and to discuss examples of how healthy human brain functions can be modulated in order to alter, positively or negatively, typical behavior. Coupling NIBS with neuroimaging techniques that allow to non-invasively record brain activity can be used to perform closed-loop stimulations, adapting stimulation parameters to the individual, time-varying brain activity. Present and future NIBS applications to human-machine interactions in various domains are presented and discussed.

Humans represent the only animal species, as far as we know, that spends energy and resources to study and understand itself. Everything we think, feel or do comes from brain functions. Cognitive Neuroscience allows, through the use of specific techniques, to non-invasively access and study cognitive functions (e.g. memory, language, inhibitory control) in awake, healthy humans. Neuroimaging techniques, such as functional resonance imaging (fMRI) and electro/magneto encephalography (EEG/MEG), allow to non-invasively register brain activity while volunteers

S. C. Ficarella (B) Information Processing and Systems, ONERA, Base Aérienne 701, Salon de Provence, France e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 F. Santoianni et al. (eds.), Mind, Body, and Digital Brains, Integrated Science 20, https://doi.org/10.1007/978-3-031-58363-6_11

157

158

S. C. Ficarella

perform a cognitive task. They inform on which brain circuits might be involved in the cognitive functions necessary to perform the task itself and, for EEG/MEG measures, the temporal dynamics of their activation. These techniques, however, only allow to draw correlational results, without demonstrating the causal role of specific brain areas or circuits in the cognitive functions under investigation. Moreover, they do not allow to interfere with ongoing brain activity, which, aside from the obviously useful clinical applications, can be used to enhance cognitive functions in the healthy population. Non-invasive brain stimulation techniques (NIBS) overcome both these limitations. The next paragraph contains a brief description of NIBS techniques and examples of their possible use in healthy participants.

1

Non-invasive Brain Stimulation Techniques

These techniques typically use electrical or magnetic impulses to alter the excitability of neuronal populations in the cortex, without needing surgical intervention [1]. Transcranial magnetic stimulation (TMS) allows, through a coil placed on the participant’s head, the delivering of a brief (~100µs in duration) magnetic pulse that reaches the brain virtually unchanged and generates, through electromagnetic induction, an electrical current in the underlying cortical neurons [2]. Depending on the brain area that is stimulated and the stimulation parameters, net excitatory or inhibitory effects can be induced (Fig. 1a). For example, low frequency (1-5Hz) and high frequency (10-20Hz) repetitive TMS are known to generate inhibitory and excitatory effects, respectively (but see [3, 4]). Inhibitory effects in healthy participants are usually sought to investigate the causal role of a specific brain area in a date cognitive function. For instance, low frequency repetitive TMS (1Hz) was used by Ficarella and Battelli [5] to temporarily inhibit activity in two brain areas involved in action inhibition: the right inferior frontal gyrus (rIFG) and the left dorsal fronto-median cortex (dFMC). Action inhibition can be triggered by external events, such as a red traffic light instructing a driver to stop pressing the accelerator pedal, or by internal factors, consciously perceived as a “change of mind”. Repetitive TMS allows, by comparing performance before and after stimulation, to demonstrate that the stimulated brain area is part of a circuit responsible for the cognitive function under study. In this specific case, inhibiting the activity of a brain area involved in action inhibition would result, behaviorally, in failed inhibitions; therefore participants would respond more often. In order to investigate the causal role of these two brain areas in externally-triggered and internally-generated action inhibition, the authors applied repetitive TMS to rIFG and dFMC (as well as a control site), on separate sessions. After each stimulation, participants performed an ad-hoc prepared cognitive task, a variant of the Go/No-Go paradigm, requiring participants to sometimes inhibit a planned motor response (button press). Critically, while stimulus’ color instructed participants to either execute (green) or inhibit (magenta) the action in the externally-triggered condition, a psychometrically-defined ambiguous color,

Fig. 1 Principal non-invasive brain stimulation techniques. Transcranial magnetic stimulation (a) can be used in a repetitive, paired pulse and single pulse fashion. Transcranial electrical stimulation (b) can be bifocal or multifocal. Depending on the stimulation parameters, direct current (tDCS), alternating current (tACS) and random noise (tRNS) stimulations are possible. NMDA and AMPA are Glutamate receptors that mediate Long-term Potentiation (LTP) NIBS effects. The GABA system is mainly responsible for Long-Term Depression (LTD) effects. BDNF: Brain-derived neurotrophic factor. Reproduced with permission from Sprugnoli and colleagues [11].

Title Suppressed Due to Excessive Length 159

160

S. C. Ficarella

which each participant viewed as green on half of the trials in a baseline test, was used in the internally-generated Go/No-Go condition. Since the stimulus’ color was perfectly ambiguous for each participant, it could not influence their decision and they had to voluntarily decide whether to act or inhibit the planned action. TMS-induced behavioral change is typically observed right after the end of the stimulation, for a duration that generally matches the stimulation duration, although longer effects have been found, such as in Ficarella and Battelli [5], suggesting that longer-lasting plasticity effects might take place [6]. Compared to pre-stimulation (baseline) performance, repetitive TMS to the rIFG and dFMC significantly enhanced the number of provided button presses (failed inhibitions) to the externally-triggered and internally-generated conditions, respectively. These results are in line with the notion of at least partially separated brain circuits for externally-triggered and internally-generated action inhibition and demonstrate the causal role of these two brain areas in these cognitive functions. It should be noted, however, that inducing inhibitory effects at the neuronal level does not necessarily induce performance worsening. Cognitive functions arise from activation patterns in neuronal circuits, composed of multiple cortical and subcortical areas. The behavioral effects of NIBS, therefore, depend on: interindividual neuroanatomical differences, the resting activity of the stimulated area at the moment of the stimulation, how the stimulated brain area is linked to the rest of the circuit and how performance is measured. Some cognitive functions are lateralized in the brain, such as speech, motion and, to some extent, visual attention. For instance, the left primary motor cortex (M1) is responsible for movements in the right side of the body, and vice versa. However, inhibiting, using TMS, the left (right) M1 enhances, through trans-callosal connections, the contralateral M1, as well as other connected brain areas, thus preserving motor functions [7, 8]. Excitatory effects of repetitive TMS can, on the other hand, be sought to enhance cognitive functions in healthy participants. The term “neuroenhancement” can be conceptualized as “a variety of interventions and technologies aiming to improve human performance above the subject’s normal performance, beyond what is considered ‘physiologically normal’” [9]. The use of NIBS for neuroenhancement has been somewhat limited up to now. Nonetheless, the dorsolateral prefrontal cortex, for instance, has been targeted using high-frequency repetitive TMS in numerous studies, effectively enhancing functions such as response inhibition, verbal fluency and episodic memory [10]. Non-invasive transcranial electrical stimulation (tES) is generally delivered using two electrodes, one positive (anode) and one negative (cathode), placed on the volunteer’s scalp. A weak (generally, under 2mA) electrical current flows from the anode to the cathode, following the path of least resistance inside the cortex. More recently, multifocal or high-definition tES has been developed, in order to increase the spatial resolution of the technique [12]. Depending on the stimulation parameters, specifically the current waveform shape, three main tES type can be distinguished [13] (Fig. 1b). Transcranial Direct Current stimulation (tDCS) employs, once the desired amplitude is reached through a ramp-up period of a few

Title Suppressed Due to Excessive Length

161

tens of seconds, a constant current. Generally, inhibitory effects are found at the anode site and excitatory effects are found at the cathode site. Conversely, transcranial Alternating Current stimulation (tACS) uses an oscillatory current, typically a sine wave (but see [14]), within a pre-defined frequency range, that induces regular polarity shifts. Therefore, the electrodes acting as anode and cathode alternate every half cycle. While the application of tDCS (inhibitory vs. excitatory) is more intuitive and similar to repetitive TMS, tACS aims at modulating the frequency band-specific communication between brain areas, called “neuronal oscillations”. Neuronal oscillations are “rhythmic fluctuations generated by the activity of local neuron populations or neuron assemblies across brain areas” [15] and were first described by Hans Berger (1873–1941) through electroencephalographic (EEG) measures 100 years ago. It has long been known that a, perhaps causal, link exists between oscillatory communicating networks and cognitive functions [16]. Brain oscillations are typically divided in five frequency bands: delta (