Rationality and Ethics in Artificial Intelligence [1 ed.] 1527594416, 9781527594418

Citation preview

Rationality and Ethics in Artificial Intelligence

Rationality and Ethics in Artificial Intelligence Edited by

Boris D. Grozdanoff, Zdravko Popov and Silviya Serafimova

Rationality and Ethics in Artificial Intelligence Edited by Boris D. Grozdanoff, Zdravko Popov and Silviya Serafimova This book first published 2023 Cambridge Scholars Publishing Lady Stephenson Library, Newcastle upon Tyne, NE6 2PA, UK British Library Cataloguing in Publication Data A catalogue record for this book is available from the British Library Copyright © 2023 by Boris D. Grozdanoff, Zdravko Popov, Silviya Serafimova and contributors All rights for this book reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, without the prior permission of the copyright owner. ISBN (10): 1-5275-9441-6 ISBN (13): 978-1-5275-9441-8

TABLE OF CONTENTS

Acknowledgements .................................................................................. vii Chapter I ..................................................................................................... 1 A Brief History of Computer Ethics and how it is Connected to AI Ethics Mariana Todorova Chapter II .................................................................................................. 10 Ethical Clashes in the Prospects for Autonomous Vehicles Silviya Serafimova Chapter III ................................................................................................ 35 Ethical Challenges to Artificial Intelligence in the Context of Pandemic and Afterwards Iva Georgieva Chapter IV ................................................................................................ 52 The AI-Run Economy: Some Ethical Issues Anton Gerunov Chapter V ................................................................................................. 71 Scratch My Back & I Will Scratch Yours: Putting Game Theory and Artificial Intelligence on a Converging Path beyond Computational Capabilities Boris Gurov Chapter VI ................................................................................................ 93 Artificial Intelligence in Defence Todor Dimitrov Chapter VII ............................................................................................. 116 Process Mining with Machine Learning Nikola Sotirov

vi

Table of Contents

Chapter VIII ........................................................................................... 131 Approaching the Advanced Artificial Intelligence Alexander Lazarov Chapter IX .............................................................................................. 152 The Structure of Artificial Rationality Boris Grozdanoff Chapter X ............................................................................................... 199 Discriminator of Well-Formed Formulae as a Component of Artificial Human Rationality Dimitar Popov

ACKNOWLEDGEMENTS This volume includes a selection of papers, written on the basis of talks delivered at three high-level conferences on Artificial Intelligence (AI), held in Sofia in 2018 and 2019. The topics cover a broad spectrum of AI related themes, among which approaches to artificial general intelligence (AGI) and human-like reasoning models, the problem of ethical AI, modern implementations of AI in the economy and defense sectors. The editors want to thank cordially to Prof. Anastas Gerdjikov, rector of the University of Sofia, Academician Julian Revalski, chairman of the Bulgarian Academy of Sciences, Gen Maj. Grudi Angelov, rector of the National Defense Academy “Georgi Sava Rakovski”, the Japanese Ambassador to the Republic of Bulgaria H.E. Masato Watanabe, Dr. Ivo Trayanov, chairman of the Defense and International Security Institute (DISI), Prof. Zdravko Popov, in his quality of the chairman of the Public Policy Institute (PPI), Dr. Karina Angelieva, deputy minister of science and education, Dr. Hiroshi Yamakawa, the Director of the Dwango AI Laboratory and Chairperson of the Whole Brain Architecture Initiative (WBAI), and former Chief Editor of the Japanese Society for Artificial Intelligence (JSAI) and Dr. Momtchil Karpouzanov from the American University in Bulgaria (AUBG). The editors also wish to cordially thank Dr. Dimitar Popov for his invaluable help in proofreading the manuscript and editing the formalism.

CHAPTER I A BRIEF HISTORY OF COMPUTER ETHICS AND HOW IT IS CONNECTED TO AI ETHICS MARIANA TODOROVA In his article “A very short history of computer ethics”,1 the author Terrell Bynum defines computer ethics as a scientific field that emerged in the first years of the outbreak of World War II, beginning with Massachusetts Institute of Technology professor Norbert Wiener. He defined the new field while participating in the development of an anti-aircraft weapon, the purpose of which was to intercept and track an enemy aircraft, then calculate its probable trajectory and inform the other parts of the weapon to activate the projectile. The emerging challenges for engineers, according to Bynum, led to the creation of a new branch of science by Wiener and his colleagues, which they called cybernetics, the science of information feedback systems. It was cybernetics combined with the digital computers created at the time that motivated the inventor to draw several important ethical conclusions. In 1950, he (Wiener) published the first book of its kind on computer ethics (though he nowhere explicitly calls his reasoning that way): The Human Use of Human Beings,2 where he spoke of the benefits and risks of automation and the use of computers. The text sounds like a come true (self-fulfilling prediction), as Wiener predicts that computers will enhance human capabilities, free people from repetitive manual labor, but also allow for processes of dehumanization and subordination of the human species. The 1

Bynum,”A Very Short History of Computer Ethics”, accessed July 8, 2022, https://web.archive.org/web/20080418122849/http:/www.southernct.edu/organizati ons/rccs/resources/research/introduction/bynum_shrt_hist.html. 2 Norbert Wiener, The Human Use of Human Beings. Cybernetics and Society (New York: Doubleday Anchor Book, 1950).

2

Chapter I

author warns us not to accept computers as available entities, but to always keep in mind that they are trained (something that actually happens with machine learning) and can go beyond our control. Such a development is a prerequisite for complete dependence and even for control over humanity. The danger, he said, comes from the fact that computers cannot think abstractly and therefore cannot comprehend and evaluate human values. Wiener adds that the invention of digital computers will lead to a second industrial revolution, which will have multi-layered dimensions, will unfold for decades and will lead to radical changes. For these reasons, he explicitly warns in the chapter, “Someone Communication Machines and Their Future,” as well as throughout the book, that workers must adapt to changes in their jobs. Governments need to draft new laws and regulations. Industries and businesses need to create new policies and practices. Professional organizations need to prepare new codes for their members. Sociologists and psychologists need to study new phenomena, and philosophers need to rethink and redefine outdated social and ethical concepts. Norbert Wiener made valuable recommendations nearly 70 years ago, which unfortunately do not find systematic application to this day. The changes he describes are under way, the exponential development of technology is accelerating, and although there are many projects of universities and research centers on the machine ethics of artificial intelligence, there is still a lack of serious discussion and consensus on key issues that touch. Walter Manner was the researcher who formalized the term “computer ethics”, defining it as part of the applied (ethics) in his work: Starter Kit on Teaching Computer Ethics3 in 1976. He devoted his subsequent work to efforts to emancipate this title as a separate scientific field. The intention to strictly distinguish computer ethics from fundamental ethical issues is implicit.

3

Walter Maner, Starter Kit on Teaching Computer Ethics (Self-published in 1978. Republished in 1980 by Helvetia Press in cooperation with the National Information and Resource Center for Teaching Philosophy).

A Brief History of Computer Ethics and how it is Connected to AI Ethics

3

James Moore, also known by his law of the same name, also dedicated an article4 on this issue. He believes that the ambiguity surrounding computer ethics arises because there is a political vacuum over how to use computer technology. Through computers, we acquire new abilities that provide us with new opportunities and choices for action. Very often, according to him, there are no political measures for such situations, and if there are any, they are inadequate. For Moore, a central task in computer ethics is to determine what we need to do in specific computer-related situations, such as formulating policies and action guides. Computer ethics, according to him, must take into account both individual and social rights and policies. Therefore, it identifies four areas of computer ethics: 1) identifying the computer-generated policy vacuum; 2) clarification of conceptual ambiguities; 3) formulation of policies for the use of computer technologies 4) ethical justifications. Moore correctly outlines the steps that should be taken to fill the ethical and then legal and regulatory gaps, but fails to point out that this task is too ambitious to be executed. Philosophers, anthropologists, psychologists and neuroscientists must take the lead in such a task, but they must work alongside representatives of labor and social systems, education, medicine, security and military technology. That is, with experts from all fields who will be influenced by computer technology and artificial intelligence. Deborah Johnson also contributed to this issue. In her article, “Computer Ethics”,5 she defines it as a field that explores how computers provoke new varieties of classical moral problems and how they worsen and deepen when we apply common moral norms to emerging spheres. She does not share the opinion that this should be a new part of ethics, but simply that a new perspective is set for problems concerning property, power, personal space and responsibility, etc.

4

Moor, James H. “What Is Computer Ethics?” In Computers and Ethics, ed. Terrell Ward Bynum (Basil Blackwell, 1985), 266 – 275. 5 Deborah Johnson, “Computer Ethics”, Prentice-Hall, reprinted by Metaphilosophy, Vol.16, No. 4 (October 1985): 319-322.

4

Chapter I

Back in 1995, Krystyna Górniak-Kocikowska predicted in her article6 that computer ethics, then considered still part of applied ethics, would evolve into a system of global ethics applicable to every culture in the world. She associates it with the times of the print media, arguing that Bentham and Kant developed their ethical theories in response to this discovery, and believes that the same will be repeated with computer ethics, which must respond to the computer-induced revolution. According to her, the nature of the expected phenomenon (computer revolution) is such that the ethics of the future will have a global character, in the sense that it will also address the integrity of human actions and relationships. Because computers know no boundaries and operate globally, they will set the stage for a new global universal ethic that must be universal to all human beings. The author adds that all local ethics (considering both individual areas and cultural and regional features (of Asia, Africa, Europe, America, etc.) may grow into a global ethic, inheriting computer ethics in information era. Johnson (1985) also talks about global ethics, but uses a different meaning. For her, it will belong to new kinds of extensive moral problems. Inherited and contemporary ethical theories will continue to be fundamental, and this will not lead to a revolution in ethics. With the help of Terrell Bynum, two opposite concepts of computer ethics are revealed to us. On the one hand, there is the thesis of Wiener, Maner and Gorniyak-Kosikovska about a revolution in ethics and an obligation of humanity to rethink its very foundations, as well as of human life. On the other hand, Johnson's more conservative view is presented, which defends the position that ethics will remain “untouched” and that these are the same old ethical issues in a new reading, which in turn will make computer ethics meaningless as a separate part. In the debate thus outlined, we can agree in part with statements from both theses. Ethics that address information and computer technology, as well as artificial intelligence, will be global and universal, as responses to the

6

Górniak-Kocikowska, Krystyna. “The Computer Revolution and the Problem of Global Ethics.” In Global Information Ethics, eds. Terrell Ward Bynum and Simon Rogerson (Opragen Publications, 1996), 177–190.

A Brief History of Computer Ethics and how it is Connected to AI Ethics

5

consequences of interacting with artificial intelligence will not be a regional problem or only within nation states. We can assume nuances in the attitude and legal regulations towards the narrow, specialized artificial intelligence, which is not yet competitive, in terms of human brain capacity and awareness. For some cultures, nationalities, or global companies, it will probably be permissible for the personal assistant, artificial intelligence, to have a level of trust with which to perform delegated functions and decisions. For others, this will not be equally valid. However, when addressing general or superartificial intelligence, it will be necessary for ethical aspects to be universally valid on a planetary level, addressed to all mankind. Will the code of ethics for artificial intelligence be objective if the discussion is dominated by catastrophic or overly optimistic scenarios? Mara Hvistendahl, a scientific correspondent for the Guardian, in her article “Can we stop AI outsmarting humanity”7 considers (only) the emergence of artificial intelligence after about 30 years as the end of human evolution, began 50,000 years ago with the erection of the species Homo Sapiens and 5,000 years ago with the emergence of the phenomenon of civilization, citing the scientist Jean Tallinn. She is a co-founder of Skype and is strongly influenced by the scientist Elizer Yudkowski, according to whom artificial intelligence can hypothetically destroy humanity. On this occasion, Tallinn became a regular donor to the organization of Yudkowski, which dedicates its work to a project for the so-called “friendly artificial intelligence”. The category of “friendly,” according to Hvischendal, does not mean a reduction to skills such as dialogue with people or that it will be guided only by love and altruism. According to her, “friendly” can be defined as having human motivation, impulses and values. That is, it should not be useful for the machines of the future to arrive at the conclusion that they must erase us in order to achieve their goals. For these reasons, Tallinn founded the Center for the Study of Existential Risk in Cambridge.

7 Mara Hvistendahl, “Can We Stop Outsmarting Humanity?”, The Guardian, Match 28, 2019.

6

Chapter I

The concept behind Tallinn is that software does not need to be programmed to destroy humanity, but can “decide” so along the course of its existence. As we have already noted, this might be the product of a small error, software bug, etc. The scientist also refers to the example of Bostrom, who reveals that artificial intelligence could decide that the atoms in the human body are a good raw material and can be used in another way as a resource. Objections to such arguments come from the Technology Guild, which says it is too early to seek a solution against hostility. They recommend shifting the focus to current problems, such as the fact that most of the algorithms were created by white men and that this fact has given rise to the biases that accompany them. Stuart Armstrong of the Future of humanity institute at the Oxford Institute also deals with these issues. Armstrong even goes further and suggests that purely physical intelligence be confined to a container and limited to answering questions. Its strategy is to protect people from possible manipulation. It also proposes to have a mechanism for disconnection from people or self-exclusion from the software itself under certain conditions. However, Armstrong fears that these conditions may be sufficient as a measure because artificial intelligence can learn to protect itself or at least develop “curiosity”, as there is such an option. In this context, Hvistendahl reports on a programmer, Tom Murphy VII, who invented a program that could teach itself to play Nintendo computer games. In practice, this software is invincible and the only way to stop it is not to play (ibid.). Tallinn, on the other hand, believes that even if the power button is masked and not of interest to artificial intelligence, there is still no solution to the problem of potential threat, as artificial intelligence may have secretly replicated itself hundreds of thousands of times on the web. For these reasons, researchers and practitioners are united around the idea of artificial intelligence being taught to recognize and study human values. That is, according to Tallinn, he must learn to evaluate people outside the canons of strict logic. For example, that we often say one thing but think another, that we enter conflicts or think differently when we are drunk, and so on. a state of irrationality (ibid.).

A Brief History of Computer Ethics and how it is Connected to AI Ethics

7

Tallinn takes as its best formula the statement of the Cambridge philosopher Hugh Price, who defines that artificial intelligence in ethical and cognitive aspects should be like a “superman”. Other questions arise - if we do not want artificial intelligence to dominate us, then should we surpass it. And these questions again inevitably lead us to the presence of consciousness and free will in artificial intelligence. Boris Grozdanoff in his article “Prospects for a Computational Approach to Savulescu’s Artificial Moral Advisor”8 proposes the creation of a universally valid homogeneous human ethic that can be codified so that artificial intelligence can study it and to create “Artificial moral agent” (AMA). According to him, ethics is also a product of human evolution, which was developed by science and, in particular by philosophy, to create a set of rules to bring order to society through which it can survive and prosper. Grozdanoff launched the thesis of circumstantial normativity in the form of an ethical system. It has to be formalized and axiomatized, which seems like a rather complicated and almost impossible task. In addition to being unified, encircled, and translatable into languages that AI can understand, it must also survive its incarnation in a general/general AI program. For Grozdanoff, the solution lies in the construction of a semantic engine (AMA) that can deliver and handle epistemological processes. The formula that Hugh Price, Boris Grozdanoff and other scientists offer is correct, but much work is needed to precede it. Today, we are witnessing a resurgent wave of neoconservatism, which sharply criticizes liberal theories such as multiculturalism, globalism, etc. In parallel with these processes, we can predict a resurgence of nationalism, hardening of the concepts of “nation states” and “identities”. This context would certainly prevent attempts to seek a universally valid formula for human ethics that could eventually be applied as a matrix for the training of general artificial intelligence. Cultural diversity and different human civilizational norms do not share the same views on the categories of “good” and “bad”, human

8

Boris D. Grozdanoff, “Prospects for a Computational Approach to Savulescu’s Artificial Moral Advisor,” ȿɬɢɱɟɫɤɢ ɢɡɫɥɟɞɜɚɧɢɹ, ɛɪ.5, No. 3 (December 2020): 107-120.

8

Chapter I

rights, etc. their definition, which mostly fit world-class institutions such as the UN. There is a call from the European Commission to companies, when creating new software involving artificial intelligence, to use the integration of ethical rules as a competitive advantage. In a document published on April 7, 2019, Europe stated that it would invent and propose global standards to guide all other players in this field. Such requests provoked sharp comments, including from Daniel Castro, vice president of the Information Technology and Innovation Foundation (ITIF). Undoubtedly, it is crucial that there is a global strategic factor in the face of the European Commission in particular and the European Union in general, which is concerned and will try to impose and take precedence in ethical standards and frameworks for artificial intelligence. The problem is that this position will most likely be peripheral and non-binding. UNESCO is also developing a solid and intergovernmental document - an ethical framework for artificial intelligence, which will also provide important and meaningful recommendations. From now on, it is important that all scientific papers and articles “meet” with political documents' positions to find the best solution for the ethical development of artificial intelligence.

References Bynum, Terrell Ward. “A Very Short History of Computer Ethics”. Accessed July 8, 2022. https://web.archive.org/web/20080418122849/http:/www.southernct.edu/o rganizations/rccs/resources/research/introduction/bynum_shrt_hist.html. Górniak-Kocikowska, Krystyna. “The Computer Revolution and the Problem of Global Ethics.” In Global Information Ethics, edited by Terrell Ward Bynum and Simon Rogerson, 177–190. (Guildford: Opragen Publications, 1996). Grozdanoff, Boris D. “Prospects for a Computational Approach to Savulescu’s Artificial Moral Advisor.” ȿɬɢɱɟɫɤɢ ɢɡɫɥɟɞɜɚɧɢɹ. ɛɪ. 5.,

A Brief History of Computer Ethics and how it is Connected to AI Ethics

9

No. 3 (2020): 107-120. https://jesbg.com/eticheski-izsledvania-br-5kn3-2020/ Hvistendahl, Mara. “Can We Stop Outsmarting Humanity?” The Guardian, March 28, 2019. https://www.theguardian.com/technology/2019/mar/28/can-we-stoprobots-outsmarting-humanity-artificial-intelligence-singularity. Johnson, Deborah. “Computer Ethics”, Prentice-Hall, reprinted by Metaphilosophy, Vol.16, No. 4 (1985): 319-322. Maner, Walter. 1978. Starter Kit on Teaching Computer Ethics (selfpublished in 1978. Republished in 1980 by Helvetia Press in cooperation with the National Information and Resource Center on Teaching Philosophy). Moor, James H. “What Is Computer Ethics?” In Computers and Ethics, edited by Terrell Ward Bynum, 266–275. (Basil Blackwell, 1985). Wiener, Norbert. 1950. The Human Use of Human Beings. Cybernetics and Society. New York: Doubleday Anchor Book.

CHAPTER II ETHICAL CLASHES IN THE PROSPECTS FOR AUTONOMOUS VEHICLES SILVIYA SERAFIMOVA Introduction Clarifications The moral issues regarding the use of autonomous vehicles (AVs) are not a new phenomenon in the ethical discourse.1 Some of them date back to the concerns about the so-called trolley dilemma, introduced by Philippa Foot in 1967. The dilemma is a thought experiment according to which a fictitious onlooker can choose to save five people in danger of being hit by a trolley, by diverting the trolley to kill just one person.2 At first sight, the trolley dilemma looks like a utilitarian moral dilemma, which is based upon calculating the maximization of well-being for more representatives at the expense of the suffering of the few. If that were the case, there would be no dilemma whatsoever. The solution would be one to switch the trolley so that the five people can be saved. However, such a decision is an act utilitarian decision par excellence. In turn, the trolley dilemma is modified within so-called moral design problem, which addresses the moral challenges in building AVs.3 In this 1

For the challenges in relating the trolley cases to the ethics of AVs, see Geoff Keeling,”The Ethics of Automated Vehicles,” (PhD thesis, University of Bristol, 2020), 45-68. 2 Philippa Foot, “The Problem of Abortion and the Doctrine of the Double Effect,” Oxford Review, No. 5 (1967): 5-15. 3 Geoff Keeling, “Commentary: Using Virtual Reality to Assess Ethical Decisions in Road Traffic Scenarios: Applicability of Value-of-Life-Based Models and

Ethical Clashes in the Prospects for Autonomous Vehicles

11

context, moral programming can be examined as displaying adaptations of the trolley dilemma, with the difference that the AVs are preprogrammed to make such decisions.4 Automated vehicle technologies are “the computer systems that assist human drivers by automating aspects of vehicle control” including a wide range of capabilities such as antilock brakes and forward collision warning, adaptive cruise control and lane keeping, as well as fully automated driving.5 The moral concerns derive from the findings of theoretical research robotics, which show that crash-free environment is unrealistic.6 This means that if a crash is unavoidable, “a computer can quickly calculate the best way to crash on the basis of a combination of safety, the likelihood of the outcome, and certainty in measurements much faster and with greater precision than a human can”.7 Current projects for AVs aim at building partially autonomous vehicles, assuming that drivers can take back control of the vehicle under given circumstances. Such AVs are considered as artificial moral agents (AMAs) belonging to Levels 3 and 4 of NHTSA’s classification.8 According to the more “optimistic” projects of fully autonomous AVs, one should build AVs as artificial autonomous moral agents (AAMAs) belonging to Level 5 of the same classification. The moral challenges in building AVs concern the strive of the engineers for developing a “universally accepted moral code Influences of Time Pressure,” Front. Behav. Neurosci, No. 11 (December 2017): 247; Keeling, Geoff. “Against Leben’s Rawlsian Collision Algorithm for Autonomous Vehicles.” In Philosophy and Theory of Artificial Intelligence 2017 (Studies in Applied Philosophy, Epistemology and Rational Ethics. 44), ed. Vincent C. Müller (Springer 2018), 259-272. https://doi.org/10.1007/978-3-319-96448-5_29. I refer to the online version of this publication. 4 Darius-Aurel Frank, Polymeros Chrysochou, Panagiotis Mitkidis and Dan Ariely, “Human Decision-Making Biases in the Moral Dilemmas of Autonomous Vehicles,” Sci Rep, No 9, 13080 (September 2019): 2. 5 Noah Goodall, “Ethical Decision Making during Automated Vehicle Crashes,” Transp. Res. Record J., Transp. Res. Board 2424, No.1 (January 2014): 58. 6 Goodall, “Ethical Decision Making during Automated Vehicle Crashes,” 59. 7 Goodall, “Ethical Decision Making during Automated Vehicle Crashes,” 60. 8 NHTSA’s classification of AVs includes the following levels: no automation (Level 0), driver assistance (Level 1), partial automation (Level 2), conditional automation (Level 3), high automation (Level 4) and full automation (Level 5).

12

Chapter II

that could guide the machines’ behavior”.9 The largest project of how one can “train” AI morality in such situations is so-called Moral Machine experiment. It is an online experimental platform that is designed to examine the moral dilemmas faced by AVs. It collects data of millions of moral decisions for the purposes of training machine-learning algorithms.10 Similar to the methodological pitfalls in objectifying the moral outcomes of the trolley dilemma, these regarding the Moral Machine experiment depend upon who decides for whom under what circumstances. Therefore, the moral outcomes can be evaluated by taking into account the plurality of the intersecting perspectives (passenger, pedestrian, observer), as well as the decision-making modes (deliberate, intuitive).11 The aforementioned specifications show that if one wants to find objective and morally justifiable solutions to the AV scenarios, one should constructively evaluate the role of different human predispositions in the process of moral decision-making. Recognizing the role of biases is of crucial importance for the AV manufacturers since “sourcing people’s moral preferences on a large scale requires developing a standardized and reliable instrument that actually controls for participants’ perspective and decision-making mode”.12

Structure The main objective of this paper is to demonstrate why finding some potential solutions to the moral design problem and Moral Machine experiment requires one to recognize the challenges in building AVs as moral rather than purely computational challenges. For the purposes of exemplifying the latter, I tackle the benefits and disadvantages of two types of AVs projects, viz. the Rawlsian collision algorithm, which displays a

9

Frank, Chrysochou, Mitkidis and Ariely, “Human Decision-Making Biases in the Moral Dilemmas of Autonomous Vehicles,” 1. 10 Frank, Chrysochou, Mitkidis and Ariely, “Human Decision-Making Biases in the Moral Dilemmas of Autonomous Vehicles,” 1. 11 Frank, Chrysochou, Mitkidis and Ariely, “Human Decision-Making Biases in the Moral Dilemmas of Autonomous Vehicles,” 2. 12 Frank, Chrysochou, Mitkidis and Ariely, “Human Decision-Making Biases in the Moral Dilemmas of Autonomous Vehicles,” 17.

Ethical Clashes in the Prospects for Autonomous Vehicles

13

contractualist project and some of Frank et al.’s thought experiments which represent utilitarian and deontological scenarios.13 The Section “A contractualist prospect for AVs” is devoted to the role of the Rawlsian collision algorithm. In addition to the investigation of Keeling’s criticism of this algorithm, in Section “Rawlsian collision algorithm” I analyze why the way in which Leben elaborates upon Rawls’ theory of the original position and the maximin rule necessitates one to rethink the role of value-of-life heuristics for the worst-off people. I also explore how such an analysis can contribute to revealing the diversity of core values: specifically, the values of survival probability and survival. These values are examined as related to the computation of life and death in AV accidents, as represented in Keeling’s three scenarios. The scenarios are discussed in Section “Exemplifying Rawlsian collision algorithm”. In Section “The role of biases”, I aim to explore the impact of people’s biased moral preferences upon the evaluation of survival probabilities. In Section “Building “utilitarian” AVs”, I analyze some Frank et al.’s thought experiments regarding the complicated use of AVs. In Section “The role of experimental ethics”, special attention is paid to the challenges posed by evaluating the results through the methods of experimental ethics. Consequently, in Section “Some “hybrid” moral explanations”, I investigate why the difficulties in justifying moral autonomy of AVs are driven by the way in which Green et al.’s exploration makes room for two triplets–these of emotions–intuitive decision-making–deontological decision-making and cognitive knowledge–deliberate decision-making–utilitarian decision-making. The objective is to demonstrate why the exaggerated trust in the triplets may trigger the misrecognition of a solution to one-versus-many case as a utilitarian solution, while it can be driven by purely deontological motivation. For the purposes of revealing the reasons behind the conflation of deontological and utilitarian decisions within the AV scenarios, I also examine the role of what I call utilitarian explanatory bias.

13

The choice of projects demonstrates how both theoretical ethical approaches (these adopted in Rawlsian algorithm) and empirical ethical approaches (these incorporated into Frank et al.’s experiments) require further elaboration.

14

Chapter II

A Contractualist Prospect for AVs Rawlsian Collision Algorithm The moral design problem inherits some of the moral concerns about the trolley dilemma, when examined from a utilitarian perspective. That is why one should look to adopt another approach. An illuminative example of such an approach is found in Rawls’ theory of justice. It is elaborated upon by Leben into so-called Rawlsian collision algorithm. Leben’s “contractualist” answer to the moral design problem is grounded into two main ideas borrowed from Rawls–these of the original position and the maximin rule.14 The original position is “a hypothetical situation in which representative citizens decide on principles of justice to regulate the basic structure of society from a list of alternatives”.15 Each party represents the interests of a sub-group of citizens and all citizens have a representative.16 The parties in the original position are supposed to decide from the perspective of so-called veil of ignorance. This is a situation where “no one knows his place in society, his class, position, or social status; nor does he know his fortune in the distribution of natural assets, his strength, intelligence, and the like”.17 For the purposes of maintaining the objective foundations of justice, Rawls introduces so-called maximin decision procedure. The gist of the latter concerns the selection of principles, which provide “the greatest allocation of primary goods to the worst-off citizens”.18 The point is some minimal set

14 Keeling, “Against Leben’s Rawlsian Collision Algorithm for Autonomous Vehicles,” 4. 15 John Rawls, Theory of Justice (Cambridge, Mass.: The Belknap Press of Harvard University Press, 1971), 122. Keeling, “Against Leben’s Rawlsian Collision Algorithm for Autonomous Vehicles,” 2. 16 Keeling, “Against Leben’s Rawlsian Collision Algorithm for Autonomous Vehicles.”, 2. 17 Rawls, Theory of Justice, 137. Keeling, ”Against Leben’s Rawlsian Collision Algorithm for Autonomous Vehicles,” 2. 18 Rawls, Theory of Justice, 150-161. Keeling, ”Against Leben’s Rawlsian Collision Algorithm for Autonomous Vehicles,” 3.

Ethical Clashes in the Prospects for Autonomous Vehicles

15

of rights, liberties and opportunities for the worst-off citizens to be guaranteed.19 Regarding the maximin rule, Leben modifies it due to the specificity of survival probabilities. As Keeling points out, the “iterated form of maximin described by Leben is called leximin”.20 The leximin rule resembles the maximin rule, while comparing the survival probabilities of the worst-off person on each alternative. However, the leximin can randomize between two or more alternatives having identical profiles of survival probabilities.21 In other words, the leximin can compare the second-lowest survival probabilities with the remaining alternatives. By doing so, it can select the highest survival probability to the second worst-off person. Such a conceptualization raises some significant challenges because the algorithms do not take into account the moral agents’ value-of-life heuristics. These agents are recognized as belonging to the group of the worst-off people just because their life is at stake. Furthermore, such algorithms rely upon one formalized in a moral sense presumption, namely, that survival is the highest good for all group members. Certainly, no one questions the fulfilled probability for survival as being high good in itself. However, the issue is that there might be representatives of the worst-off group who prefer to die rather than surviving with debilitating injuries.22 Leben himself is aware of this challenge, admitting that some non-fatal injuries might be evaluated as equivalent or worse than fatal injuries.23 The problem is that when a lifelong debilitating injury is set versus fatal injury

19 Keeling, ”Against Leben’s Rawlsian Vehicles,” 3. 20 Keeling, ”Against Leben’s Rawlsian Vehicles,”, 4, Note 3. 21 Keeling, ”Against Leben’s Rawlsian Vehicles,” 5. 22 Keeling, ”Against Leben’s Rawlsian Vehicles,” 10. 23 Derek Leben, “A Rawlsian Algorithm Technol 19, No. 2 (March 2017): 111.

Collision Algorithm for Autonomous Collision Algorithm for Autonomous Collision Algorithm for Autonomous Collision Algorithm for Autonomous for Autonomous Vehicles,” Ethics Inf

16

Chapter II

within the Rawlsian collision algorithm, the fatal injury is given priority in making the corresponding decision. The strive to expand the scope of the maximin rule by introducing the leximin rule requires a reevaluation of Rawls’ idea of life project. The leximin rule makes room for comparing the second-lowest survival probabilities on the remaining alternatives, but does not shed light upon whether it might be “more just” for the worst-off person or people to die than to suffer debilitating injuries. In turn, this specification necessitates one to reconsider Rawls’ original position. One should also analyze how the veil of ignorance should be elaborated upon so that the graduation of justice can meet the requirements of an internally differentiated group of worst-off people. As Keeling points out, Rawls does not assume the application of the maximin rule as universally valid.24 He clearly denies its application as a general principle of rational decisions in the case of risk and uncertainty. This clarification puts in question the extrapolation of the maximin rule to that of the leximin, when survival probabilities are at stake. One of the reasons is that when such probabilities are tested, the parties reaching an agreement should not be indifferent to their own and others’ life projects. A specification that contradicts the requirements set by the veil of ignorance. Generally speaking, the moral challenge is to reveal why Rawlsian collision algorithm can only address the moral design problem, but cannot solve it. Keeling describes the gist of the problem by saying that Leben’s answer concerns not a set of moral principles, but how one builds an algorithm based upon some principles.25 Certainly, an algorithm grounded in contractualist principles resists some objections against potential utilitarian collision algorithms.26 However, I would argue that in the strive for avoiding utilitarian relativism, one revives some crucial moral concerns. 24

Keeling, ”Against Leben’s Rawlsian Collision Algorithm for Autonomous Vehicles,” 8. 25 Keeling, ”Against Leben’s Rawlsian Collision Algorithm for Autonomous Vehicles,” 5. 26 Keeling, ”Against Leben’s Rawlsian Collision Algorithm for Autonomous Vehicles,” 5.

Ethical Clashes in the Prospects for Autonomous Vehicles

17

Correspondingly to the pitfalls of utilitarian moral math, one may face a questionable contractualist moral math, which “threatens” moral evaluation with its formal regulations.

Exemplifying Rawlsian collision algorithm Keeling describes three challenges which should be overcome if Leben’s answer to the moral design problem is proved as satisfactory.27 Scenario 1 The AV can swerve left or right. If the AV swerves left, there is a 0% chance that its passenger will sustain a fatal injury and a 100% chance that its passenger will sustain a lifelong debilitating injury. If the AV swerves right, there is a 1% chance that its passenger will sustain a fatal injury and a 99% chance that its passenger will remain unharmed. According to Keeling, Leben’s algorithm chooses to swerve left because it gives the passenger the greatest survival probability.28 Certainly, dividing rational preferences into strict and weak preferences necessitates the definition of the preferences to survival as strict preferences and those to non-fatal injuries–as weak preferences. However, regardless of the fact that the preference for survival is considered a strict preference in logical terms, it may turn out that it is a weak preference in moral terms. Extrapolating Keeling’s concern about the selection of an alternative, which is not in the passenger’s rational self-interest,29 I would argue that the more serious problem is when the programming is not in the passenger’s moral selfinterest either.

27

Keeling, ”Against Leben’s Rawlsian Collision Algorithm for Autonomous Vehicles,” 9. 28 Keeling, ”Against Leben’s Rawlsian Collision Algorithm for Autonomous Vehicles,” 10. 29 Keeling, ”Against Leben’s Rawlsian Collision Algorithm for Autonomous Vehicles,” 10-11.

18

Chapter II

Scenario 2 Keeling’s main concern about the second scenario is that the maximin rule gives “undue weight to the moral claims of the worst-off”.30 The AV can swerve left or right. If the AV swerves left, there is a 100% chance that its passenger will die, and twenty nearby pedestrians will be unharmed. If the driverless car swerves right, there is a 99% chance that its passenger will die, and a 100% chance that twenty nearby pedestrians will receive lifelong debilitating injuries. Rawlsian algorithm selects the right swerve regardless of how many pedestrians will receive lifelong debilitating injuries.31 Leben argues that he would always prefer to be one of the injured pedestrians claiming that such scenarios are unlikely to arise.32 This argument is relevantly criticized by Keeling who claims that the low probability does not make the moral concerns of the pedestrians less important.33 Going back to Rawls’ theory of the original position, it is apparent that Leben’s assumption is grounded in the misinterpretation of the veil of ignorance. In the second scenario, the parties do not choose a principle of justice because it can provide an objective moral treatment to the group of the worst-off people, but because they want to be fairly treated if/when they occasionally fall into that group. Thus, the requirements of having objective knowledge in the original position and the maximin rule are not fulfilled. In addition to Keeling’s well-formulated concern that a personal preference to a non-fatal but debilitating injury is “not a good moral reason” to inflict a large number of injuries to prevent a single death,34 one should take into 30 Keeling, ”Against Leben’s Rawlsian Collision Algorithm for Autonomous Vehicles,” 11. 31 Keeling, ”Against Leben’s Rawlsian Collision Algorithm for Autonomous Vehicles,” 11. 32 Leben, “A Rawlsian Algorithm for Autonomous Vehicles,” 114; Keeling, “Against Leben’s Rawlsian Collision Algorithm for Autonomous Vehicles,” 11. 33 Keeling, ”Against Leben’s Rawlsian Collision Algorithm for Autonomous Vehicles,” 12. 34 Keeling, ”Against Leben’s Rawlsian Collision Algorithm for Autonomous Vehicles,” 12.

Ethical Clashes in the Prospects for Autonomous Vehicles

19

consideration the problems of moral decisions’ quantification. Swerving right, as Leben suggests, is not necessarily a morally acceptable option. This is possible only if both the decision-makers and those belonging to the worst-off group evaluate the survival as being the highest good. That is why I would argue that the problems with the second scenario do not derive from “the undue weight” to the moral claims of the worst-off people, but rather from the fact that the due weight is not relevantly graduated in moral terms. The lack of such a graduation affects the way in which the group of the worst-off people is determined. Scenario 3 Keeling points out that there is a scenario which includes an algorithm that assigns a higher survival probability to the worst-off people than Leben’s algorithm. This is the greatest equal chance algorithm.35 The AV can swerve left or swerve right. If the AV swerves left, there is a 0% chance that Anne will survive, and a 70% chance that Bob will survive. If the AV swerves right, there is a 1% chance that Bob will survive, and a 60% chance that Anne will survive. Leben’s algorithm programs the AV to swerve right because it assigns a survival probability of 1% to the worst-off party.36 The third scenario brings us back to the moral concerns about the one-versus-one case. When we have to decide who should die, taking into account that there are only two persons involved, the decision cannot be made on the basis of the people’s number. One should know who these people are, as well as what their life projects look like. Otherwise, one cannot make an informed decision in moral terms. This difficulty is not overcome by Keeling’s algorithm of the greatest equal chances either. Even if the AV is programmed to construct a “weighted lottery between the alternatives, where the weightings are fixed to ensure

35 Keeling, ”Against Leben’s Rawlsian Collision Algorithm for Autonomous Vehicles,” 12. 36 Keeling, ”Against Leben’s Rawlsian Collision Algorithm for Autonomous Vehicles,” 12.

20

Chapter II

that the affected parties receive the greatest equal survival probabilities”,37 the following problem occurs. Precising the survival probability of 32.6%, which is certainly greater than 1%, is only the first step in resolving the moral dilemma. The precision of survival probabilities in programming the greatest equal chances does not trigger unquestionable moral consequences for the affected parties. For instance, computing the greatest equal chances for survival does not shed light upon the case when Anne, who could be a mother of three kids, or when Benn, who could be a researcher able to find a cure for cancer, should be sacrificed.38 Therefore, even if the most precise algorithm is elaborated upon, this algorithm does not make the moral concerns less significant. They derive not from the computation of life and death, but from life and death as such.

The Role of Biases The analysis of the Rawlsian collision algorithm shows that the challenges in building AVs derive from people’s moral decisions. The crucial role of biases is evident in the way in which the respondents give preference to saving the life of the pedestrian or that of the passenger(s) depending on whether their personal perspective is made salient or not.39 While analyzing and extrapolating the findings of the Moral Machine experiment, one should keep in mind that the Rawlsian collision algorithm explicitly avoids the recognition of the personal perspectives of the passenger, the pedestrian and the observer for the sake of achieving an

37 Keeling, ”Against Leben’s Rawlsian Collision Algorithm for Autonomous Vehicles,” 12. 38 Having argued that there are some collisions in which the greatest equal chances algorithm is preferred to Leben’s algorithm” (Keeling, ”Against Leben’s Rawlsian Collision Algorithm for Autonomous Vehicles,” 13), Keeling elaborates upon his view saying that the greatest equal chances algorithm is “not great” either (Keeling, “The Ethics of Automated Vehicles,” 108). The reason is that it depends upon the ties between the affected parties, as well as assuming that each person has an equal moral claim to be saved (Keeling, “The Ethics of Automated Vehicles,” 108). These conditions are not satisfied in AV collisions. 39 Frank, Chrysochou, Mitkidis and Ariely, “Human Decision-Making Biases in the Moral Dilemmas of Autonomous Vehicles,” 1.

Ethical Clashes in the Prospects for Autonomous Vehicles

21

optimal objectivity by applying the veil of ignorance. The idea is that the self-preserving intentions should be reduced to a minimum so that the group of the worst-off people can be determined in the most objective manner. In this context, a detailed examination of the human decision-making biases within the Rawlsian collision algorithm can contribute to firstly, limiting the role of the parties’ dominating self-preservation attitudes and secondly, demonstrating why neither the group of the decision-makers nor that of the addressees of the decisions are homogenous groups of moral agents. The second clarification, which concerns value-of-life heuristics, can make room for evaluating the heuristics in question in positive terms as well. Regarding the future development of the Rawlsian collision algorithm within the framework of the Moral Machine experiment, one may investigate its effects if a graduated social norm violation is examined as a part of the decision-making process. Frank et al. provide a thought experiment (Study 6) assuming that the pedestrian’s social norm violation results in significant likelihood of being sacrificed.40 The graduation of the violated norm varying from a low norm violation (when the pedestrian walked in a street with no signs, traffic signals, or crosswalks), going through a control condition (when the pedestrian walked in the crosswalk) and ending up with a high norm violation (when the pedestrian jaywalked at a red light)41 sets the question of coupling the issue of responsibility with that of guilt.42 If the pedestrians in the Rawlsian collision algorithm are placed in the worstoff group due to the objectivity of the high norm violation, does it mean that we can deprive them of the right to survive? Furthermore, do we have the

40

Frank, Chrysochou, Mitkidis and Ariely, “Human Decision-Making Biases in the Moral Dilemmas of Autonomous Vehicles,” 11. 41 Frank, Chrysochou, Mitkidis and Ariely, “Human Decision-Making Biases in the Moral Dilemmas of Autonomous Vehicles,” 11. 42 For the different evaluations of whether or not the jaywalker’s awareness of undertaking a risk of death or a serious harm assumes that they tacitly consent to being harmed or killed, see Keeling, “The Ethics of Automated Vehicles,” 127-134.

22

Chapter II

right to deprive them of survival probability because we assume that it is their fault to fall into that group? Elaborating upon this approach, I argue that Rawls’ original position should be modified. If the parties agree to sacrifice themselves, in case they are norm violators, denying the survival probability by default is the only just decision. Certainly, such a line of thought reaches a dead end not only with respect to Rawls’ theory of justice. On the other hand, if the parties agree to sacrifice the pedestrians because they believe that the pedestrians are guilty, such an agreement hardly can be called moral at all. It questions the moral design problem by compromising the initial status of the pedestrians. The moral gist of the dilemma is who has the moral right to decide on behalf of others for their own sake so that one can avoid the implications of moral arbitrariness.

Building “Utilitarian” AVs The Role of Experimental Ethics Regarding the implications of moral agents’ preferences, Bonnefon, Shariff and Rahwan43 argue that the participants in the Moral Machine experiment favor “a utilitarian moral doctrine that minimizes the total casualties in potentially fatal accidents, but they simultaneously report preferring an autonomous vehicle that is preprogrammed to protect themselves and their families over the lives of others”.44 When the participants in the thought experiments think about the results of the dilemmas for the greater good of society, they seem to employ a utilitarian moral doctrine. Consequently, when they consider themselves and their loved ones, the participants show

43Jean-François

Bonnefon, Azim Shariff and Iyad Rahwan, “The Social Dilemma of Autonomous Vehicles,” Science, No. 352 (June 2016): 1573-1576. 44 Azim Shariff, Jean-François Bonnefon and Iyad Rahwan, “Psychological Roadblocks to the Adoption of Self-driving Vehicles,” Nature Human Behavior, No. 1 (September 2017): 694-696.. Frank, Chrysochou, Mitkidis and Ariely, “Human Decision-Making Biases in the Moral Dilemmas of Autonomous Vehicles,” 1.

Ethical Clashes in the Prospects for Autonomous Vehicles

23

preferences towards a deontological moral doctrine that rejects the idea of sacrificing the passengers.45 Frank et al. elaborate upon the aforementioned correlations by conducting some experiments in which they trace the relations between the choice of perspectives (passenger, pedestrian, observer), the decision-making mode (deliberate, intuitive) and the different conditions, which can affect the participants’ biases (such as whether or not there is a child among the passengers, whether or not the passenger sits in the front sit etc.). The clarification of these conditions is of crucial importance when the dilemma is not one-versus-many, as in the classical trolley dilemma, but one-versusone. Before tackling the reasons behind the preference to a utilitarian over deontological moral doctrine or vice versa, one should keep in mind that such a preference is justified within the framework of experimental ethics whose results are limited in terms of the number of respondents and target groups. For instance, the approval of utilitarian AVs is achieved by conducting six online MTurk studies. As Bonnefon et al. argue, the studies in question are considered as largely reliable, although the respondents are “not necessarily representative of the US population”.46 The theoretical outcome of such investigations is that one should carefully examine the results of experimental ethics since it sets some particular objectives, which are not necessarily representative for the general diversity of moral practice, nor are they representative for the moral motivation of all moral agents. Even if a given group of respondents gives preference to utilitarian decisions when cognitive resources are actively used, it does not follow firstly, that the decisions are utilitarian only due to the reference to 45

David Rand, “Cooperation, Fast and Slow: Meta-Analytic Evidence for a Theory of Social Heuristics and Self-Interested Deliberation,” Psychological Science, 27, No. 9 (September 2016): 1192-1206. Frank, Chrysochou, Mitkidis and Ariely, “Human Decision-Making Biases in the Moral Dilemmas of Autonomous Vehicles,” 1. 46 Bonnefon,Shariff and Rahwan, “The Social Dilemma of Autonomous Vehicles,” 1575.

24

Chapter II

the resources in question and secondly, that every single time when cognitive resources are mobilized in making decisions, these decisions are necessarily utilitarian.

Some “Hybrid” Moral Explanations Frank et al. conduct a study (Study 3)47 on the biased moral preferences to a utilitarian or deontological moral doctrine. The hypothesis of prioritizing moral utilitarianism is tested by adding a second passenger, while maintaining the number of pedestrians (one) in the crosswalk. If people employ a utilitarian doctrine, they should favor the saving of the two passengers at the expense of the pedestrian. In case of no change, the result would suggest that people employ a deontological doctrine due to the assumption that it is not morally acceptable to sacrifice the pedestrian.48 The findings support the hypothesis that people employ a utilitarian moral doctrine in a deliberate decision-making mode, while people in an intuitive decision-making mode rely upon the more accessible deontological doctrine.49 This means that when the participants in the thought experiment react spontaneously, they decide to save the pedestrian. As a reason for that Frank et al. point out the role of a culturally embedded bias, namely, that US citizens are taught that pedestrians on public roads may not be hurt by drivers.50 In turn, when the participants carefully think about the situation, they prefer to sacrifice the pedestrian for the sake of the two passengers since thus they can maximize the well-being of the majority. However, the general trend, as demonstrated by Study 3, is that the prevalent choice of the participants remains in favor of the single pedestrian. In this context, Frank et al. ask the question of what degree of utility trade-off would be necessary 47

Frank, Chrysochou, Mitkidis and Ariely, “Human Decision-Making Biases in the Moral Dilemmas of Autonomous Vehicles,” 7. 48 Frank, Chrysochou, Mitkidis and Ariely, “Human Decision-Making Biases in the Moral Dilemmas of Autonomous Vehicles,” 7. 49 Frank, Chrysochou, Mitkidis and Ariely, “Human Decision-Making Biases in the Moral Dilemmas of Autonomous Vehicles,” 7. See also Joshua D. Greene et al., “Cognitive Load Selectively Interferes with Utilitarian Moral Judgment, “ Cognition, 107, No. 3 (June 2008): 1144-1154. 50 Frank, Chrysochou, Mitkidis and Ariely, “Human Decision-Making Biases in the Moral Dilemmas of Autonomous Vehicles,” 7.

Ethical Clashes in the Prospects for Autonomous Vehicles

25

for people to prefer that an AV harms an innocent pedestrian, as well as whether there will be a difference between the deliberate and intuitive decisions.51 In Study 4, one more control variable is added, namely, that of age. Its role is exemplified by the specification that one of the passengers is a child sitting in the back sit. Frank et al. clarified that this variable concerns the value-of-life heuristics, which is typical for Western cultures. The life of a younger person is valued over that of an older person.52 Surprisingly, participants’ decisions are almost identical to these in Study 3–there is no significant increase in the likelihood of sacrificing the pedestrian. However, the difference becomes obvious, when comparing the passenger and the pedestrian perspectives. The likelihood of people’s intuitive decisions to sacrifice the pedestrian in the passenger perspective is four times higher than in the pedestrian perspective condition.53 The investigation shows that people are less protective of the child in the pedestrian condition than in the passenger and control conditions.54 In this context, I would argue that the main problem with the ethical explanatory instrumentarium concerns firstly, the reasons behind relating deliberate decision-making to utilitarian decisions and, consequently, intuitive decision-making to deontological decisions. Secondly, one should keep in mind the specification that intuitivism and deontology are mutually exclusive in moral terms. That is why I raise a hypothesis that the biggest complications regarding people’s biased moral preferences derive from narrowing the role of emotions to intuitive and deontological decision-

51

Frank, Chrysochou, Mitkidis and Ariely, “Human Decision-Making Biases in the Moral Dilemmas of Autonomous Vehicles,” 7 52 Frank, Chrysochou, Mitkidis and Ariely, “Human Decision-Making Biases in the Moral Dilemmas of Autonomous Vehicles,” 8. 53 Frank, Chrysochou, Mitkidis and Ariely, “Human Decision-Making Biases in the Moral Dilemmas of Autonomous Vehicles,” 9. 54 Frank, Chrysochou, Mitkidis and Ariely, “Human Decision-Making Biases in the Moral Dilemmas of Autonomous Vehicles,” 9.

26

Chapter II

making processes, while ascribing the use of cognitive knowledge to deliberate decision-making as a matter of utilitarian decision-making. Moral utilitarianism does not exclude the role of moral feelings. The latter can play not only a negative role, as is in the case with hedonism, but also a positive one. Moral feelings such as empathy and sympathy contribute to the maximization of the collective well-being. In turn, the role of emotions is explicitly neglected in deontological ethical projects such as Kant’s ethics. Judging by the results of Studies 3 and 4, I argue that that there are two questionable points in favoring the role of deontological ethical reasons, when one shows a high likelihood to avoid the sacrifice of the passenger. Firstly, emotions, which make the Self identify with the passenger as other, triggering the self-projective bias in Self’s favor, can hardly be called a reason for a deontological ethical stance. In other words, the result seems deontologically acceptable due to the fact that one survival probability is ascribed a higher moral value. However, the motivation for that, which is intuitively triggered by the self-preservation, could be egoistic, and nothing to do with deontological ethics. Secondly, if an intuitive decision-making based upon emotions is considered as typical for a deontological decision-making, it would mean that moral intuitivism will be wrongly reduced to a formalist approach of a Kantian type. Consequently, it would mean that issues concerning the maximization of well-being and that of survival probabilities in the field of AVs are deprived of any emotional moral engagement whatsoever. What are the particular consequences of the aforementioned specifications to the utilitarian AV scenarios? Study 3 also shows that the number of the sacrificed pedestrians increases with the number of the passengers.55 There is nothing unusual, except for the tendency from the pedestrian perspective. For the first time (compared to Studies 1 and 2), the focus is shifted toward

55

Frank, Chrysochou, Mitkidis and Ariely, “Human Decision-Making Biases in the Moral Dilemmas of Autonomous Vehicles,” 7.

Ethical Clashes in the Prospects for Autonomous Vehicles

27

sparing the passengers.56 In the intuitive condition, only 4.3% of the respondents chose to sacrifice the pedestrian, while in the deliberate condition, that percentage increased up to 60%.57 The simplest explanation is that the more respondents think, the more they are inclined to think altruistically. The respondents are supposed to be more zealous of sacrificing themselves for the sake of preserving the survival probability for the rest. Such an explanation, however, raises some methodological concerns. Certainly, the act of self-sacrifice is emotionally entangled, which means that it cannot be examined as a use of more cognitive resources.58 It concerns the participants’ self-projection ability to put themselves into others’ shoes and feel empathy for them by evaluating their value-of-life heuristics. On the other hand, the percentage increase can be interpreted differently. The data of 60% can be refracted through the lens of Rawlsian collision algorithm if Rawls’ original position is taken into account. Then, the high percentage can be examined as a result of one’s willingness to provide a just decision for the worst-off group of people, which, in this case, is recognized as that of the passengers (having a more complicated status). Therefore, oneversus-many case is not necessarily subjectable to moral utilitarian interpretations since the number prevalence can be a factor for moral evaluations within both utilitarian and contractualist theories. Practically speaking, this means that from the perspective of Rawlsian collision algorithm, the number of the passengers also plays a role, but the reasons are different. While in the utilitarian mode, the quantity is recognized as a criterion for the maximization of the survival probabilities,

56

Frank, Chrysochou, Mitkidis and Ariely, “Human Decision-Making Biases in the Moral Dilemmas of Autonomous Vehicles,” 7. 57 Frank, Chrysochou, Mitkidis and Ariely, “Human Decision-Making Biases in the Moral Dilemmas of Autonomous Vehicles,” 7. 58 Frank, Chrysochou, Mitkidis and Ariely, “Human Decision-Making Biases in the Moral Dilemmas of Autonomous Vehicles,” 2.

28

Chapter II

in the contractualist mode, it is used for delineating the group of the worstoff people.59 Some of Frank et al.’s findings also support the thesis that sacrificing one person for the sake of saving many is not necessarily a moral utilitarian outcome. This is well-demonstrated by the increase in the number of control variables in Study 7. The results illustrate that the possession of a driver’s license, higher education, knowledge about AVs etc. increases the likelihood of sacrificing the pedestrian.60 The attitudes towards sacrificing a single person for the sake of saving many will influence the significant increase in the likelihood of sacrificing the pedestrian.61 However, this is not necessarily the main reason for such a sacrifice. Otherwise, it would have meant that highly-educated people, with a high economic standard, are more inclined to promote utilitarian moral values. Drawing such a conclusion is problematic in many respects. For instance, the respondents may simply identify “more easily”62 with the passengers than with the pedestrian due to their own background of being active car users. The “easier” self-projection is theoretically explained by the similarities between the passenger and the control conditions. If the observers’ selfprojection is similar to that of the passengers, the results are expectedly in favor of the passengers. Thus, the maximization of the collective well-being is encouraged when the passengers represent the social majority. This can happen regardless of the observers’ motivation, which may be driven not by utilitarian, but by some other values. If the similarities in the experience and the corresponding self-projective biases were a necessary and sufficient condition of solving the utilitarian AV dilemmas, it would have meant that those who take the pedestrian’s stance are egotists giving priority to their

59 This interpretation supports Keeling’s concerns that Bonnefon et al.’s research can solve the moral design problem by using empirical methods (Keeling, “The Ethics of Automated Vehicles”, 35). 60 Frank, Chrysochou, Mitkidis and Ariely, “Human Decision-Making Biases in the Moral Dilemmas of Autonomous Vehicles,” 15. 61 Frank, Chrysochou, Mitkidis and Ariely, “Human Decision-Making Biases in the Moral Dilemmas of Autonomous Vehicles,” 15. 62 Frank, Chrysochou, Mitkidis and Ariely, “Human Decision-Making Biases in the Moral Dilemmas of Autonomous Vehicles,” 15.

Ethical Clashes in the Prospects for Autonomous Vehicles

29

own individual survival. Drawing such a conclusion is highly exaggerated as well. Based upon the aforementioned investigations, I argue that respondents’ preference for a utilitarian explanation of the pedestrian’s sacrifice displays a strong predisposition to a utilitarian explanatory bias. If the observers, who more easily identify with the passengers due to their deliberate decision of maximizing the collective survival probabilities, are guided merely by utilitarian morality, one cannot explain why there is such a strong attitude against the sacrifice of the pedestrian. Certainly, this attitude has much to do with the strong normative validity of the idea of the innocent pedestrian. However, since the AVs in the experiments are supposed to work without passengers’ intervention, the presumption of the passengers’ innocence should remain untouched as well.

Conclusion By examining the moral challenges in building AVs, I argue that the pitfalls deriving from the utilitarian implications of the one-versus-many case gain new strength. For the purposes of demonstrating the origin of some problems with the design of a universal moral code for AVs, I have examined two types of projects. The first one is a contractualist project, namely, Rawlsian collision algorithm formulated by Leben, while the second project is underlined by the findings of some utilitarian and deontological AV thought experiments, as conducted by Frank et al. Regarding the Rawlsian collision algorithm, I draw the conclusion that in addition to the necessity of critically rethinking Leben’s reasons behind borrowing Rawls’ concepts of the original position, veil of ignorance and maximin rule, one should also focus upon the neglected value-of-life heuristics. This suggestion is based upon the assumption that only thus can one understand the complexity of moral decisions about survival probabilities. Concerning the elaboration of the original position and the maximin rule, I argue that tackling their reception is of crucial importance for understanding the problematic aspects of Rawlsian collision algorithm. As such a

30

Chapter II

questionable issue, I point out the lack of some knowledge about value-oflife heuristics in the original position, which is initially denied by both Rawls and Leben. However, such heuristics affects the application of the leximin rule. Specifically, if one does not know whether the worst-off person prefers to die rather than suffering debilitating injuries, one cannot make a just decision for that other. The impact of the blurred boundaries between the values of survival probabilities and survival upon the leximin rule shows that the latter might be no longer considered as just. That is why I argue that elaborating upon the potential moral efficiency of Rawlsian collision algorithm requires Rawls’ idea of the veil of ignorance to be modified. It should address the enrichment of knowledge about the life projects of all parties. Having analyzed the three examples, which Keeling sets as a test for Leben’s collision algorithm, I clarify why they face some limitations in justifying the values of survival probabilities. Regarding the first scenario, I point out the complexity of the logical distinction between the preferences to survival as strict preferences and those to non-fatal injuries as weak preferences. However, providing a logical distinction is insufficient for cases when a rational preference to non-fatal injuries can be considered as a strict preference in moral terms. In addition to Keeling’s criticism of Leben’s use of the maximin rule, which puts undue weight to the moral claims of the worst-off people, I also argue that Leben’s own preference to be in the shoes of the injured pedestrian is grounded in the misinterpretation of the veil of ignorance. In this second scenario, it is demonstrated how the parties choose a principle as just because they want to be fairly treated if they occasionally fall into the group of the worst-off people. The third scenario, which represents Keeling’s own suggestion for the greatest equal chance algorithm, contributes to increasing the survival rate, but does not provide moral criteria of differentiation for cases such as that of one-versus-one. In this context, I argue that an alternative in revealing the different moral values ascribed to survival probabilities and survival as such can be found

Ethical Clashes in the Prospects for Autonomous Vehicles

31

by examining the role of people’s biased moral preferences. A detailed examination of the impact of biases within Rawlsian collision algorithm can contribute to firstly, limiting the role of the parties’ self-projective biases (specifically, their self-preservation attitudes) and secondly, demonstrating why neither the group of the decision-makers nor those of the addressees of the decisions are homogenous groups of moral agents. If one examines the impact of the pedestrian’s social norm violation upon the personal perspective of the observer within Rawlsian collision algorithm, there are two significant problems, at least. If the deciding parties agree to sacrifice themselves, in case they are violators, they should initially deny the survival probability for themselves as the only just decision. On the other hand, if the parties agree to sacrifice the pedestrians because they believe that the pedestrians are guilty, such an agreement cannot be called moral at all. The moral gist of the dilemma is who has the moral right to decide on behalf of others for their own sake so that one can avoid the implications of moral arbitrariness. Regardless of the fact that the potential solutions to the trolley dilemma go beyond the utilitarian explanatory framework, utilitarian decision-making plays a crucial role in understanding the challenges in building a universal moral code for AVs. While tackling the implications of the decision-making in question, one should keep in mind that it is determined as such due to the limited methods of experimental ethics, which may encourage the recognition of some ungrounded ethical generalizations. Analyzing Frank et al.’s findings, as displayed in Studies 3 and 4, I reach the conclusion that the main problem with the ethical explanatory instrumentarium concerns firstly, the reasons behind relating deliberate decision-making to utilitarian moral decisions and consequently, intuitive decision-making to deontological moral decisions. Secondly, one should reconsider the relation between intuitivism and deontology, because they are mutually exclusive in moral terms. The analysis of Frank et al.’s findings shows that even if there are no formal reasons to reject the statement that the high likelihood of sacrificing the passengers is driven by deontological ethical arguments, the emotion of self-

32

Chapter II

preservation cannot be recognized as a positive moral feeling, which grounds a deontological decision-making. I draw the conclusion that the relevant search for a universal moral code for AVs requires the reconsideration of the triplet of emotions–intuitive decision-making–deontological decision-making and that of cognitive knowledge–deliberate decision-making–utilitarian decision-making. Such a clarification reveals why the sacrifice of one person for the sake of many is not necessarily a moral utilitarian outcome, although it can meet the formal utilitarian requirements of solving the one-versus-many case. If an intuitive decision-making based upon emotions is considered as typical of deontological decision-making, it would mean that moral intuitivism is paradoxically reduced to a formalist approach from a Kantian type. Consequently, it would mean that issues concerning the maximization of well-being and that of survival probabilities in the field of AVs are initially unrelated to the process of emotional moral engagement. The respondents’ tendency to give preference to a utilitarian over the deontological interpretation of the one-versus-many case can be explained with a stronger influence of a utilitarian explanatory bias. The impact of this bias is clearly demonstrated when the thought experiments assume that the pedestrian is guilty of the potential AV accident. However, both the passengers and pedestrians are equally innocent in a sense, because the thought experiments assume that the vehicle is designed to be autonomous. Comparing and contrasting the application of deontological and utilitarian decision-making principles to the AV design shows that some “hybrid” moral models should be adopted. Elaborating upon such a line of thought means that one can avoid the recognition of the universal moral code as an absolute code by giving preference to a set of codes, which are mutually complementary. Adopting this approach would contribute to improving the options of AVs’ moral self-update by regulating the balance between their autonomy and morality, as much as possible.

Ethical Clashes in the Prospects for Autonomous Vehicles

33

References Bonnefon, Jean-François, Azim Shariff, and Iyad Rahwan. “The Social Dilemma of Autonomous Vehicles.” Science, No. 352 (2016): 15731576. https://doi. org/ 10.1126/science.aaf2654. Foot, Philippa. “The Problem of Abortion and the Doctrine of the Double Effect.” Oxford Review, No. 5 (1967): 5-15. Frank, Darius-Aurel, Polymeros Chrysochou, Panagiotis Mitkidis, and Dan Ariely. “Human Decision-Making Biases in the Moral Dilemmas of Autonomous Vehicles.” Sci Rep, No. 9, 13080 (2019). https://doi.org/10.1038/s41598-019-49411-7. Goodall, Noah. “Ethical Decision Making during Automated Vehicle Crashes.” Transp. Res. Record J., Transp. Res. Board 2424, No. 1 (2014): 58-65. https://doi.org/10.3141/2424-07. Greene, Joshua D., Silviya A. Morelli, Kelly Lowenberg, Leigh E. Nystrom, and Jonathan D. Cohen. “Cognitive Load Selectively Interferes with Utilitarian Moral Judgment.” Cognition, 107, No. 3 (2008): 1144-1154. https://doi.org/10.1016/j.cognition.2007.11.004. Keeling, Geoff. “Commentary: Using Virtual Reality to Assess Ethical Decisions in Road Traffic Scenarios: Applicability of Value-of-LifeBased Models and Influences of Time Pressure.” Front. Behav. Neurosci, 11, 247 (2017). https://doi.org/10.3389/fnbeh.2017.00247. PMID: 29311864; PMCID: PMC5733039. Keeling, Geoff. 2018. “Against Leben’s Rawlsian Collision Algorithm for Autonomous Vehicles.” In Philosophy and Theory of Artificial Intelligence 2017 (Studies in Applied Philosophy, Epistemology and Rational Ethics. 44), edited by Vincent C. Müller, 259-272. (Leeds: Springer 2018). https://doi.org/10.1007/978-3-319-96448-5_29. Keeling, Geoff. “The Ethics of Automated Vehicles.” PhD thesis., University of Bristol, 2020. Leben, Derek. “A Rawlsian Algorithm for Autonomous Vehicles.” Ethics Inf Technol 19, No. 2 (2017): 107-115. https://doi.org/10.1007/s10676017-9419-3.

34

Chapter II

Rand, David G. “Cooperation, Fast and Slow: Meta-Analytic Evidence for a Theory of Social Heuristics and Self-Interested Deliberation.” Psychological Science, 27, No. 9 (2016): 1192-1206. https://doi.org/10.1177/0956797616654455. Rawls, John. 1971. Theory of Justice. Cambridge, Mass.: The Belknap Press of Harvard University Press. Shariff, Azim, Jean-François Bonnefon, and Iyad Rahwan. “Psychological Roadblocks to the Adoption of Self-driving Vehicles.” Nature Human Behavior No. 1 (2017): 694-696. https://doi.org./10.1038/s41562-0170202-6.

CHAPTER III ETHICAL CHALLENGES TO ARTIFICIAL INTELLIGENCE IN THE CONTEXT OF PANDEMIC AND AFTERWARDS IVA GEORGIEVA Introduction The ethical consequences of the work done in the field of Artificial Intelligence (AI) in the current pandemic world differ extensively from those that were discussed in the previous times before the outbreak of COVID-19. The focus on the nature of the events happening worldwide due to the outbreak of the virus led to various ripple effects on the notions directed toward science and technology. It also, however, provokes the occurrence of extreme phenomena such as conspiracy theories, fake news, anti-vaccination movements, deep fakes, and similar ones, just to name a few. It is now more than ever important to restore the trust in science and deploy the means of technology in service of people’s health and wellbeing and to prove the ethical foundations in any actions taken toward serving these goals with the available and new technological endeavors. However, exactly at this point, science, technology, and AI in particular might seem even more threatening as one of the strongest traumatic results of the pandemic is the one that questions the ability of humankind to survive and undermines the innate ability to believe that we are not so existentially vulnerable and it is important not to surrender to ideas that we might be a subject to extinction. Even though technology is not directly held responsible for the current state of affairs, the theories about the origin of the pandemic, as well as fears of malicious use of various technological means cause increased distrust in the

36

Chapter III

deployment of sophisticated means to fight the virus. In this way, fears of uncontrollable forces, including such ones as AI and robots, might seem taken out of science fiction, and still fit too well into the irrational beliefs of the exhausted people amid the constant fight as a trait of human nature. That is why, the necessity for ethical and social acceptance and the ensuring of the reliability of the technological means meet those standards in the recent uses of technology and the defending of its particular means is an important work of the scientists1 along with other concrete scientific achievements that aim to help and clear rather than further mystify the role of technology in the fight with the coronavirus.

AI Before and After The image of technology and AI and robotics in particular from the prepandemic era was more of a result from a mixture of science-fiction dreams and creative endeavors of advanced societies such as the Japanese one for example.2 However, the needs of the current times are different and require fast and focused response to secure areas of life that are directly threatened by the pandemic – such as medicine, work, and education, and affect lifestyle in general by deepening the need for more sophisticated applications of technology, and AI in particular, in healthcare.3 For example, before the COVID-19 times it was consequently discussed issue to utilize AI in full replacement of doctors, who raised various ethical issues and fears of depersonalization of medical work4 that of course, would not 1

Iva Georgieva, Elisabeth Beaunoyer and Matthieu J. Guitton. “Ensuring Social Acceptability of Technological Tracking in the COVID-19 Context,” Computers in Human Behavior 116 (March 2021): 106639. 2 Peter Kahn et al., “Human Creativity Can Be Facilitated through Interacting with a Social Robot.” In 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI), published by IEEE (Christchurch, New Zeeland, March 7-10 2016), 173-180. 3 Amelia Fiske et al., “Your Robot Therapist Will See You Now: Ethical Implications of Embodied Artificial Intelligence in Psychiatry, Psychology, and Psychotherapy,” Journal of Medical Internet Research 21, No. 5 (May 2019): e13216. 4 Mark Arnold and Ian Kerridge, “Accelerating the De-personalization of Medicine: The Ethical Toxicities of COVID-19,”Journal of Bioethical Inquiry 17, No. 4 (August 2020): 815-821.

Ethical Challenges to Artificial Intelligence in the Context of Pandemic and Afterwards

37

dissolve in the current situation,5 as now it would be much better justifiable to consider more than aiding and rather to accept generally replacing medical personnel in dangerous environments due to the pandemic.6 Many concerns have shifted in a similar way as a result of the changed way of living and perception of technology. However, bridging the scientific rationale with the common-sense concerns is crucial as this new type of normal way of living requires acceptance of a state of affairs that is different from those imagined and discussed before. Areas of AI application directly affected and rapidly changed due to the pandemic are for example telemedicine, tracking technology, vaccine development, and digital health in general 7. In particular, such tools as health chatbots (e.g. Babylon Health) 8would be a solution for the needs of communication when face-to-face contact is avoided, and more and more artificial agents and serious games would become means to address issues that were otherwise given to practitioners and preferred as traditional inperson practices.9 AI creation aims to disrupt healthcare in order to redesign it and now the chance to do so is more than evident.10 Besides obvious 5

Taoran Liu et al., “Patients’ Preferences for Artificial Intelligence Applications versus Clinicians in Disease Diagnosis During the SARS-CoV-2 Pandemic in China: Discrete Choice Experiment.” Journal of Medical Internet Research 23, No. 2 (February 2021): e22841. 6 Sonu Bhaskar et al., “Designing Futuristic Telemedicine Using Artificial Intelligence and Robotics in the COVID-19 Era.” Frontiers in Public Health 8 (November 2020): 708. 7 Deepak Jakhar and Ishmeet Kaur, “Current Applications of Artificial Intelligence for COVIDဨ19,” Dermatologic Therapy (May 2020). Tim Robbins et al., “COVID19: A New Digital Dawn?” Digital Health (April 2020). Abu Sufian et al., “Insights of Artificial Intelligence to Stop Spread of Covid-19.” In Big Data Analytics and Artificial Intelligence against COVID-19: Innovation Vision and Approach, eds. Aboul-Ella Hassainen, Nilanjan Dey and Sally Elghamrawy. (Springer, Cham: Springer, 2020), 177-190. 8 Babylon Health, accessed June 12, 2022, https://www.babylonhealth.com. 9 John McGreevy et al., “Clinical, Legal, and Ethical Aspects of Artificial Intelligence–assisted Conversational Agents in Health Care,” Jama 324, No. 6 (August 2020): 552-553. 10 Julia M. Puaschunder et al., “The Future of Artificial Intelligence in International Healthcare: An Index.” In Proceedings of the 17th International RAIS Conference on Social Sciences and Humanities (Scientia Moralitas Research Institute, 2020), 1936.

38

Chapter III

answers to the already discussed questions such as what is the artificial agent that finds updated meaning in the new context of the pandemic, the issue of heightened cyber-security threats in the midst of a worldwide situation challenged with the sense of authenticity, should change the way these perceptions are analyzed by specialists and accepted by society.11 Moreover, digital communication leads to information overload and phenomena such as “Zoom fatigue”12 that lead to increased necessity for artificial agents to replace or aid the human-computer interaction (HCI) and the technologically aided communication means. However, this also leads to issues about surveillance and lack of privacy as we choose our own trackers from the ones enforced by the pandemic to the ones we have already accepted with not much questioning, such as the health-related apps for example – and it is difficult to see the true difference and the transparency of the changing processes behind these phenomena of shifting needs for technological support.13 All these current effects put enormous pressure and toll on mental health, as well as pose ethical questions that are difficult to address while events continue to unfold and decisions are yet hard to make and, at the same time, already delayed. The difficult balance between technology and humanity has always been an issue with the advance in this field, and now more than ever it seems we are facing a shift in the necessity for technological presence in the world.14

Shifts in AI Ethics due to the Pandemic Looking through the development of technologies including AI to address the pandemic might give some preliminary answers to questions, such as 11

Menaka Muthuppalaniappan and Kerrie Stevenson, “Healthcare Cyber-Attacks and the COVID-19 Pandemic: An Urgent Threat to Global Health, “International Journal for Quality in Health Care” 33, No. 1 (February 2021). 12 Jeremy N. Bailenson, “Nonverbal Overload: A Theoretical Argument for the Causes of Zoom Fatigue, “Technology, Mind, and Behavior” 2, No. 1 (February 2021). 13 David Leslie, “Tackling COVID-19 through Responsible AI Innovation: Five Steps in the Right Direction,” Harvard Data Science Review (April 2020). 14 Paul M. Leonardi, “COVIDဨ19 and the New Technologies of Organizing: Digital Exhaust, Digital Footprints, and Artificial Intelligence in the Wake of Remote Work,” Journal of Management Studies (October 2020).

Ethical Challenges to Artificial Intelligence in the Context of Pandemic and Afterwards

39

which areas are most important to clarify. For example, interactive monitoring maps were the first applications of AI during the COVID outbreak, later new technologies for faster detection and vaccine development were included while ethical issues were not sufficiently considered as quick response was the main goal.15 AI and robots were introduced to fight fatigue in hospitals,16 while people were developing various adverse symptoms due to prolonged social distancing and effects such as phobias began to emerge that obviously would need separate tools to address.17 Now, as much more time passed than initially expected, the situation has not changed much, and the settling of the “new normal” requires new tools to address long-term and delayed rather than the initial and quickly dissolved problems that the pandemic introduced to our lives. It is still unclear in what way the main pillars of life such as healthcare, work and education would have to change, and in what way people will be able to cope with the necessity for permanent change or for the acceptance of the constant change of affairs, which in fact is worse as a result and would require a profound understanding of the ethical consequences of the technological innovation related to this.18 The “infodemic” that also resulted exponentially from the specifics of the pandemic could be addressed with the means of AI. This term refers to the widespread misinformation that circulates through social media and creates divisions between groups of people, nations and cultures and consequently deepens the negative effects of the pandemic. Issues related to the ethical and social acceptability of tracking technologies are just one side of the irrational fears about the technological aids, on the other side there is a real 15

Asaf Tzachor, Jess Whittlestone and Lalitha Sundaram, “Artificial Intelligence in a Crisis Needs Ethics with Urgency,” Nature Machine Intelligence 2, No. 7 (June 2020): 365-366. 16 Ajmal Zemmar, Andres M. Lozano and Bradley J. Nelson, “The Rise of Robots in Surgical Environments during COVID-19,” Nature Machine Intelligence 2, No. 10 (October 2020): 566-572. 17 Gopi Battineni, Nalin Chintalapudi and Francesco Amenta, “AI Chatbot Design during an Epidemic Like the Novel Coronavirus,” Healthcare, vol. 8, No. 2 (October 2020): 154. 18 Sara Gerke, Timo Minssen and Glenn Cohen., “Ethical and Legal Challenges of Artificial Intelligence-driven Healthcare,” Artificial Intelligence in Healthcare (June 2020): 295-336.

40

Chapter III

danger of privacy breaches, online hoaxes and malicious software that might significantly endanger the users while providing untrue information.19 Detecting these will again require technological means and possibly relying on AI, and that is why trust becomes such a twofold phenomenon now – a problem and a solution, and even more complex than that.20 Concerns about the realization the such methods, which help mitigating the problems resulting from technology require technology itself, are real and create necessity for a separate response in relation to the current state of affairs of ethics of technology and the increasingly leading role of AI in its application in the context of the pandemic. However, AI in connection with virtual reality (VR) for example, can become the real disruptive technology in healthcare. From creating intelligent virtual environments to building intelligent artificial agents,21 the freedom of giving new possibilities pairs with the responsibility for offering greater ethical impact 22 and so can build trust in the users. In relation to the consideration of the double-edged impact of technology, defending the idea that AI cannot fully replace medical personnel where high skills like empathy and ethical considerations are needed, would rather propose the use of the technology in the mundane and mechanical tasks. Still, the possibility of creating a completely artificial counterpart in communication where human beings might fail seems considerably promising and necessary, especially in the current context of the pandemic, as exhaustion and lack of motivation take their toll. When it is possible to rely on a well-

19

Harbind Kharod and Israel Simmons, “How to Fight an Infodemic: The Four Pillars of Infodemic Management,” Journal of Medical Internet Research 22, No. 6 (2020): e21820. 20 Muhammad Mustafa Kamal, “The Triple-edged Sword of COVID-19: Understanding the Use of Digital Technologies and the Impact of Productive, Disruptive, and Destructive Nature of the Pandemic,” Information Systems Management 37, No. 4 (September 2020): 310-317. 21 Michael Luck and Ruth Aylett, “Applying Artificial Intelligence to Virtual Reality: Intelligent Virtual Environments,” Applied Artificial Intelligence 14, No. 1 (November 2000): 3-32. Vladimir M. Petroviü, “Artificial Intelligence and Virtual Worlds–Toward Human-level AI Agents,” IEEE Access 6 (July 2018): 3997639988. 22 Ben Kenwright, “Virtual Reality: Ethical Challenges and Dangers [Opinion],” IEEE Technology and Society Magazine 37, No. 4 (December 2018): 20-25.

Ethical Challenges to Artificial Intelligence in the Context of Pandemic and Afterwards

41

developed virtual partner that serves its function, being it the care of an elderly person who lacks contact with relatives, or a helper of kids who need to learn from a different source than their teachers or overly busy parents in the new era of virtual meetings and online education, seems like a refreshing and promising alternative.23 Nevertheless, rethinking the way things function became necessity and redesigning the means of achieving our goals is necessary to answer this challenge and be prepared for the future. It is without a doubt that digital health assistants and medical chatbots, e.g. from sophisticated ones meant for visually impaired persons to offer more freedom and better life to more common ones such as the period tracking apps that also remind of birth control pills to women, they can all provide significant changes in the facilitation of everyday tasks, especially when people are overwhelmed with the response to more daunting necessities such as prevention and treatment of a viral infection. Of course, the most common application of such smart advisors will be the detection and solution of various symptoms, related or not to the pandemic, that might challenge the integrity of people’s lifestyles and threaten their health.24 Other areas of life that suffer strongly from the pandemic are more subtle, namely the psychological changes resulting in decreased motivation and creativity 25that can be addressed again with the help of AI and VR in areas that are inevitably changed due to the pandemic, such as tourism for example.26 The possibility of offering gamification in addition to the intelligent assistants turns the interest toward serious games that again can

23

Hui Luan et al., “Challenges and Future Directions of Big Data and Artificial Intelligence in Education,” Frontiers in Psychology 11 (October 2020). 24 Jingwen Zhang et al., “Artificial Intelligence Chatbot Behavior Change Model for Designing Artificial Intelligence Chatbots to Promote Physical Activity and a Healthy Diet,” Journal of Medical Internet Research Vol. 22, No. 9 (September 2020): e22845. Nicola Luigi Bragazzi et al., “How Big Data and Artificial Intelligence Can Help Better Manage the COVID-19 Pandemic,” International Journal of Environmental Research and Public Health 17, No. 9 (May 2020): 3176. 25 Giuseppe Riva et al., “Surviving COVID-19: The Neuroscience of Smart Working and Distance Learning,” Cyberpsychology, Behavior, and Social Networking 24, No. 2 (February 2021): 79-85. 26 Andrei O.J. Kwok and Sharon G.M. Koh, “COVID-19 and Extended Reality (XR),” Current Issues in Tourism 24, No. 14 (July 2020): 1-6.

42

Chapter III

connect AI and VR and offer a new platform for treating patients with different kinds of suffering (both purely physical and subtly psychological) and serve as a training tool for education, prevention and building of resilience in the context of the increasingly challenging contemporary life.27 Nevertheless, the above applications of AI technology go deep into personal lives and hence create stronger concerns in relation to privacy, data collection and overall fears for possible control of human life. Decisions about why and how the personal information is used and stored are still taken by the specific commercial entities rather than governments or health agencies. It is necessary to consider these issues before accepting technology on such a level and still the time is insufficient to extensively consider all the factors and make the necessary decision in a well-thought and debated ethical context.28 Again, here the human factor is so important as no technology can offer a better solution than the complex consideration of historical, political and even more, philosophical premises on the possibility of giving such power to technological artefacts. That is why even though the situation is so confusing, acting toward common decisions and goals in relation to AI, healthcare and wellbeing in the context of the pandemic world is more than necessary now. From IoT solutions29 that are needed more than ever in the everyday life now to other specific for this time issues such as the analysis of antivaccination movements,30 the detection of misinformation and panicinduced/inducing information online,31 as well as more global problems

27

Giuseppe Riva et al., “Positive Technology and COVID-19,” Cyberpsychology, Behavior, and Social Networking 23, No. 9 (September 2020): 581-587. 28 Stephen Cave et al., “Using AI Ethically to Tackle Covid-19,” BMJ 372 (March 2021). 29 Musa Ndiaye et al., “IoT in the Wake of COVID-19: A Survey on Contributions, Challenges and Evolution,” IEEE Access 8 (October 2020): 186821-186839. 30 Amir Hussain et al., “Artificial Intelligence–enabled Analysis of Public Attitudes on Facebook and Twitter toward COVID-19 Vaccines in the United Kingdom and the United States: Observational Study,” Journal of Medical Internet Research 23, No. 4 (April 2021): e26627. 31 Jim Samuel et al., “Covid-19 Public Sentiment Insights and Machine Learning for Tweets Classification,” Information 11, No. 6 (June 2020): 314.

Ethical Challenges to Artificial Intelligence in the Context of Pandemic and Afterwards

43

such as the “infodemic” turning into a “digital pandemic”,32 these tasks can all be handled by AI. It is a matter of collective responsibility33 to also realize that there are the previously mentioned here long-term effects and the general double-edged sword effect of technology, along with more subtle ones that actually affect populations on a very deep level, e.g., the problem with digital inequalities 34that accumulate and create a pandemic of separate pandemic-related issues.

The Future Is Now, and Still Yet to Come AI can become the main tool for the future prevention of pandemics and related to them global issues35 but its role should be clarified and defended as well as precisely regulated. Moreover, the ideas of how AI can be utilized are somehow confined in strictly non-human types of activities and lack a sense of freedom and imagination. AI should not be the cold and threatening force that we can put to work in the desired areas, it should be perceived as our own creation that is aiding force standing next to human efforts in areas that need development. For example, as studies show a steep decline in creativity due to technology impact from an early age, it is the same research that offers that AI can be a “companion” of man for more creative endeavors.36 The pandemic has shown how creative industries such as theater and basic foundations of life such as education might be completely transformed with digital, interactive

32

Charalambos Tsekeris and Yannis Mastrogeorgiou, “Contextualising COVID-19 as a Digital Pandemic,” Homo Virtualis, 3(2) (December 2020): 1–14. 33 Seumas Miller and Marcus Smith, “Ethics, Public Health and Technology Responses to COVIDဨ19,” Bioethics Vo. 35, No. 4 (February 2021): 366-371. 34 Marco Marabelli, Emmanuelle Vaast and Lydia Li, “Preventing the Digital Scars of COVID,” European Journal of Information Systems (2021). 35 Puaschunder, Julia. “The Future of Artificial Intelligence in International Healthcare: An Index.” In Proceedings of the 17th International RAIS Conference on Social Sciences and Humanities (Scientia Moralitas Research Institute, 2020), 1936. 36 Zaccolo, Sandro. “Artificial Intelligence as a Creativity Companion,” accessed July 8, 2022, http://openresearch.ocadu.ca/id/eprint/3167/.

44

Chapter III

and more creative methods of communication.37 It has been a long-term effort to export creative media in VR, for example, and now as the physical distancing and other factors keep performers away, it is the virtual medium that can bridge entities and actors together and even spice them up with the help of AI. To illustrate this, a connection between theater, VR and AI in the context of cognitive science can work to provide “sense making” between the subject and the system.38 VR can be aided by AI in utilizing virtual environments for creating flexible to the desired treatment results mediums that match the purpose of a psychological therapy.39 The notion that AI in VR can produce powerful immersive stories that are aided by virtual characters designed to specific needs is not new.40 Embodiment of avatars in them might bring about necessary results that can help people achieve things otherwise impossible due to many restrictions, and these are now increased in number by the overall pandemic situation worldwide. While VR application of AI might seem so compelling, it might also sound scary and even escapist solution to an overly digitalized world.41 The subtle balance between answering needs and leading to issues is a truly important task and that is where the requirement of ethically considered and approved means should come.

37

Mariana-Daniela González-Zamar and Emilio Abad-Segura, “Implications of Virtual Reality in Arts Education: Research Analysis in the Context of Higher Education,” Education Sciences 10, No. 9 (August 2020): 225. 38 Pierre DeLoor et al., “Connecting Theater and Virtual Reality with Cognitive Sciences.” In Virtual Reality International Conference (Laval, France, Apr 2010), 221-225. 39 Iva Georgieva and Georgi V. Georgiev, “Reconstructing Personal Stories in Virtual Reality as a Mechanism to Recover the Self,” International Journal of Environmental Research and Public Health 17, No. 1 (March 2020): 26. 40 McLellan, Hilary. “Magical Stories: Blending Virtual Reality and Artificial Intelligence.” In Imagery and Visual Literacy: Selected Readings from the Annual Conference of the International Visual Literacy Association (Tempre, Arizona, October 12–16, 1994). 41 Rana Saeed Al-Maroof et al., “Fear from COVID-19 and Technology Adoption: The Impact of Google Meet during Coronavirus Pandemic,” Interactive Learning Environments (October 2020): 1-16.

Ethical Challenges to Artificial Intelligence in the Context of Pandemic and Afterwards

45

As stressed here before, it is the technology that intrinsically holds the notion of seeming threatening creation and altogether an ultimate solution. In the era of challenges to the human health, it seems that what was made by the man can be the cure for the suffering caused as well.42 The toll of the prolonged exposure to stress that has now becoming habituated as a “new normal” way of living is causing a pandemic of other challenging effects on human nature that are yet to be detected. It is important to address this lingering avalanche of plethora of problems that might surface when life starts to get back to normal or as it seems, as they never truly return as they were before, to remain as safe and well as possible afterwards. The possibility to seek help from technological innovation and AI in connection with VR in particular, might help the person to experience new ways of processing adverse events and seek growth among them.43 Seeing events in such a new light might offer answers to how to exit a situation that has lasted for too long already.

Conclusion The aftermath of the pandemic is still unpredictable, but the new world after it will and at the same time will not be so globalized or that much depending on technology. Divisions and contradictions become stronger, however the need for smarter solutions also emerges above previous needs. It is a matter of devoted scientific work to determine which is important to stay and hold and which is no longer serving, but it is undoubtedly sure that the preservation of what is humane and what is ensuring the wellbeing of humankind remains the greater goal of them all.

42

Krešimir ûosiü et al., “Impact of Human Disasters and COVID-19 Pandemic on Mental Health: Potential of Digital Psychiatry,” Psychiatria Danubina 32, No. 1 (Spring 2020): 25-31. 43 Iva Georgieva and Georgi V. Georgiev. “Redesign Me: Virtual Reality Experience of the Line of Life and Its Connection to a Healthier Self, “ Behavioral Sciences 9, no. 11 (November: 2019): 111.

46

Chapter III

References Al-Maroof, Rana Saeed, Said A Salloum, Aboul Ella Hassanien, and Khaled Shaalan. “Fear from COVID-19 and Technology Adoption: The Impact of Google Meet during Coronavirus Pandemic.” Interactive Learning Environments (2020): 1-16. https://www.tandfonline.com/doi/full/10.1080/10494820.2020.1830121. Arnold, Mark, and Ian Kerridge. “Accelerating the De-personalization of Medicine: The Ethical Toxicities of COVID-19.” Journal of Bioethical Inquiry 17, No. 4 (2020): 815-821. https://link.springer.com/article/10.1007/s11673-020-10026-7. Babylon Health. Accessed June 12, 2022. https://www.babylonhealth.com. Bailenson, Jeremy N. “Nonverbal Overload: A Theoretical Argument for the Causes of Zoom Fatigue.” Technology, Mind, and Behavior” 2, No. 1 (2021). https://tmb.apaopen.org/pub/nonverbal-overload/release/2 Battineni, Gopi, Nalini Chintalapudi, and Francesco Amenta. “AI Chatbot Design during an Epidemic Like the Novel Coronavirus.” Healthcare, vol. 8, No. 2 (2020): 154. https://doi.org/10.3390/healthcare8020154. Bhaskar, Sonu, Sian Bradley, Sateesh Sakhamuri, Sebastian Moguilner, Vijay Kumar Chattu, Shawna Pandya, Starr Schroeder et al. “Designing Futuristic Telemedicine Using Artificial Intelligence and Robotics in the COVID-19 Era.” Frontiers in Public Health 8 (2020): 708. https://org.doi/10.3389/fpubh.2020.556789. Bragazzi, Nicola Luigi, Haijiang Dai, Giovanni Damiani, Masoud Behzadifar, Mariano Martini, and Jianhong Wu. “How Big Data and Artificial Intelligence Can Help Better Manage the COVID-19 Pandemic.” International Journal of Environmental Research and Public Health 17, No. 9 (2020): 3176. https://pubmed.ncbi.nlm.nih.gov/32370204/. Cave, Stephen, Jess Whittlestone, Rune Nyrup, and Rafael A. Calvo. “Using AI Ethically to Tackle Covid-19.” BMJ 372 (2021). https:/doi:org/https://doi.org/10.1136/bmj.n364. ûosiü, Krešimir, Siniša Popoviü, Marko Šarlija, and Ivan Kesedžiü. “Impact of Human Disasters and COVID-19 Pandemic on Mental Health: Potential of Digital Psychiatry.” Psychiatria Danubina 32, No. 1 (2020): 25-31. https://pubmed.ncbi.nlm.nih.gov/32303026/

Ethical Challenges to Artificial Intelligence in the Context of Pandemic and Afterwards

47

De Loor, Pierre, Charlie Windelschmidt, Karen Martinaud, and Vincent Cabioch. “Connecting Theater and Virtual Reality with Cognitive Sciences.” In Virtual Reality International Conference, 221-225. (Laval, France, Apr 2010). Fiske, Amelia, Peter Henningsen, and Alena Buyx. “Your Robot Therapist Will See You Now: Ethical Implications of Embodied Artificial Intelligence in Psychiatry, Psychology, and Psychotherapy.” Journal of Medical Internet Research 21, No. 5 (2019): e13216. https://www.jmir.org/2019/5/e13216/. Georgieva, Iva, and Georgi V. Georgiev. “Redesign Me: Virtual Reality Experience of the Line of Life and Its Connection to a Healthier Self.” Behavioral Sciences 9, No. 11 (2019): 111. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6912767/. Georgieva, Iva, and Georgi V. Georgiev. “Reconstructing Personal Stories in Virtual Reality as a Mechanism to Recover the Self.” International Journal of Environmental Research and Public Health 17, No. 1 (2020): 26. https://doi.org/10.3390/ijerph17010026. Georgieva, Iva, Elisabeth Beaunoyer, and Matthieu J. Guitton. “Ensuring Social Acceptability of Technological Tracking in the COVID-19 Context.” Computers in Human Behavior 116 (2021): 106639. https://www.sciencedirect.com/science/article/pii/S0747563220303861 Gerke, Sara, Timo Minssen, and Glenn Cohen. “Ethical and Legal Challenges of Artificial Intelligence-driven Healthcare.” Artificial Intelligence in Healthcare (2020): 295-336. https://doi.org/10.1016/B978-0-12-818438-7.00012-5. González-Zamar, Mariana-Daniela, and Emilio Abad-Segura. “Implications of Virtual Reality in Arts Education: Research Analysis in the Context of Higher Education.” Education Sciences 10, No. 9 (2020): 225. https://www.mdpi.com/2227-7102/10/9/225. Hussain, Amir, Ahsen Tahir, Zain Hussain, Zakariya Sheikh, Mandar Gogate, Kia Dashtipour, Azhar Ali, and Aziz Sheikh. “Artificial Intelligence–enabled Analysis of Public Attitudes on Facebook and Twitter toward COVID-19 Vaccines in the United Kingdom and the United States: Observational Study.” Journal of Medical Internet Research 23, No. 4 (2021): e26627. https://pubmed.ncbi.nlm.nih.gov/33724919/.

48

Chapter III

Jakhar, Deepak, and Ishmeet Kaur. “Current Applications of Artificial Intelligence for COVIDဨ19.” Dermatologic Therapy (2020). https://onlinelibrary.wiley.com/doi/full/10.1111/dth.13654. Kahn, Peter H., Takayuki Kanda, Hiroshi Ishiguro, Brian T. Gill, Solace Shen, Jolina H. Ruckert, and Heather E. Gary. “Human Creativity Can Be Facilitated through Interacting with a Social Robot.” In 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI), published by IEEE, 173-180 (Christchurch, New Zeeland, March 7-10 2016). Kamal, Muhammad Mustafa. “The Triple-edged Sword of COVID-19: Understanding the Use of Digital Technologies and the Impact of Productive, Disruptive, and Destructive Nature of the Pandemic.” Information Systems Management 37, No. 4 (2020): 310-317. https://doi.org/10.1080/10580530.2020.1820634. Kenwright, Ben. “Virtual Reality: Ethical Challenges and Dangers [Opinion].” IEEE Technology and Society Magazine 37, No. 4 (2018): 20-25. https://ieeexplore.ieee.org/document/8558774. Kharod, Harbind, and Israel Simmons. “How to Fight an Infodemic: The Four Pillars of Infodemic Management.” Journal of Medical Internet Research 22, No. 6 (2020): e21820. Kwok, Andrei O.J., and Sharon G.M. Koh. “COVID-19 and Extended Reality (XR).” Current Issues in Tourism 24, No. 14 (2020): 1-6. https://doi.org/10.1080/13683500.2020.1798896. Leonardi, Paul M. “COVIDဨ19 and the New Technologies of Organizing: Digital Exhaust, Digital Footprints, and Artificial Intelligence in the Wake of Remote Work.” Journal of Management Studies (2020). https://onlinelibrary.wiley.com/doi/10.1111/joms.12648. Leslie, David. “Tackling COVID-19 through Responsible AI Innovation: Five Steps in the Right Direction.” Harvard Data Science Review (2020). https://hdsr.mitpress.mit.edu/pub/as1p81um/release/4. Liu, Taoran, Winghei Tsang, Fengqiu Huang, Oi Ying Lau, Yanhui Chen, Jie Sheng, Yiwei Guo, Babatunde Akinwunmi et al. “Patients’ Preferences for Artificial Intelligence Applications versus Clinicians in Disease Diagnosis during the SARS-CoV-2 Pandemic in China: Discrete Choice Experiment.” Journal of Medical Internet Research 23, No. 2 (2021): e22841. https://www.jmir.org/2021/2/e22841/.

Ethical Challenges to Artificial Intelligence in the Context of Pandemic and Afterwards

49

Luan, Hui, Peter Geczy, Hollis Lai, Janice Gobert, Stephen J.H. Yang, Hiroaki Ogata, Jacky Baltes et al. “Challenges and Future Directions of Big Data and Artificial Intelligence in Education.” Frontiers in Psychology 11 (2020). https://pubmed.ncbi.nlm.nih.gov/33192896/. Luck, Michael, and Ruth Aylett. “Applying Artificial Intelligence to Virtual Reality: Intelligent Virtual Environments.” Applied Artificial Intelligence 14, No. 1 (2000): 3-32. https://doi.org/10.1080/088395100117142. Marabelli, Marco, Emmanuelle Vaast, and Lydia Li. “Preventing the Digital Scars of COVID.” European Journal of Information Systems (2021). https://dx.doi.org/10.2139/ssrn.3755050. McGreevey, John D., C. William Hanson, and Ross Koppel. “Clinical, Legal, and Ethical Aspects of Artificial Intelligence–assisted Conversational Agents in Health Care.” Jama 324, No. 6 (2020): 552553. https://pubmed.ncbi.nlm.nih.gov/32706386/. McLellan, Hilary. 1995. “Magical Stories: Blending Virtual Reality and Artificial Intelligence.” In Imagery and Visual Literacy: Selected Readings from the Annual Conference of the International Visual Literacy Association (Tempre, Arizona, October 12–16, 1994). Miller, Seumas, and Marcus Smith. “Ethics, Public Health and Technology Responses to COVIDဨ19.” Bioethics Vo. 35, No. 4 (2021): 366-371. https://doi.org/10.1111/bioe.12856. Muthuppalaniappan, Menaka, and Kerrie Stevenson. “Healthcare CyberAttacks and the COVID-19 Pandemic: An Urgent Threat to Global Health.” International Journal for Quality in Health Care” 33, No. 1 (2021). https://pubmed.ncbi.nlm.nih.gov/33351134/. Ndiaye, Musa, Stephen S Oyewobi, Adnan M. Abu-Mahfouz, Gerhard P. Hancke, Anish M, Kurien, and Karim Djouani. “IoT in the Wake of COVID-19: A Survey on Contributions, Challenges and Evolution.” IEEE Access 8 (2020): 186821-186839. https://ieeexplore.ieee.org/document/9220109. Petroviü, Vladimir M. “Artificial Intelligence and Virtual Worlds–Toward Human-level AI Agents.” IEEE Access 6 (2018): 39976-39988. https://ieeexplore.ieee.org/document/8410872. Puaschunder, Julia M. “The Future of Artificial Intelligence in International Healthcare: An Index.” In Proceedings of the 17th International RAIS

50

Chapter III

Conference on Social Sciences and Humanities, 19-36. (Scientia Moralitas Research Institute, 2020). Puaschunder, Julia M., Josef Mantl, and Bernd Plank. “Medicine of the Future: The Power of Artificial Intelligence (AI) and Big Data in Healthcare.” RAIS Journal for Social Sciences, Vol. 4, No. 1 (2020): 18. http:/doi.org/10.2139/ssrn.3662010. Riva, Giuseppe, Fabrizia Mantovani, and Brenda K. Wiederhold. “Positive Technology and COVID-19.” Cyberpsychology, Behavior, and Social Networking 23, No. 9 (2020): 581-587. https://doi.org/10.1089/cyber.2020.29194.gri. Riva, Giuseppe, Brenda K Wiederhold, and Fabrizia Mantovani. “Surviving COVID-19: The Neuroscience of Smart Working and Distance Learning.” Cyberpsychology, Behavior, and Social Networking 24, No. 2 (2021): 79-85. https://doi.org/10.1089/cyber.2021.0009. Robbins, Tim, Sarah Hudson, Pijush Ray, Sailesh Sankar, Kiran Patel, Harpal Randeva, and Theodoros N. Arvanitis. “COVID-19: A New Digital Dawn?” Digital Health (2020). https://doi.org/10.1177/2055207620920083. Samuel, Jim, G. G. Ali, Md Rahman, Ek Esawi, and Yana Samuel. “Covid19 Public Sentiment Insights and Machine Learning for Tweets Classification.” Information 11, No. 6 (2020): 314. https://doi.org/10.3390/info11060314. Sufian, Abu, Dharm Singh Jat, and Anuradha Banerjee. “Insights of Artificial Intelligence to Stop Spread of Covid-19.” In Big Data Analytics and Artificial Intelligence against COVID-19: Innovation Vision and Approach, edited by Aboul-Ella Hassainen, Nilanjan Dey and Sally Elghamrawy, 177-190. (Springer, Cham: Springer, 2020). Tsekeris, Charalambos, and Yannis Mastrogeorgiou. “Contextualising COVID-19 as a Digital Pandemic.” Homo Virtualis, 3(2) (2020): 1–14. https://doi.org/10.12681/homvir.25445. Tzachor, Asaf, Jess Whittlestone, and Lalitha Sundaram. “Artificial Intelligence in a Crisis Needs Ethics with Urgency.” Nature Machine Intelligence 2, No. 7 (2020): 365-366. https://www.nature.com/articles/s42256-020-0195-0.

Ethical Challenges to Artificial Intelligence in the Context of Pandemic and Afterwards

51

Zaccolo, Sandro. “Artificial Intelligence as a Creativity Companion.” Accessed July 8, 2022. http://openresearch.ocadu.ca/id/eprint/3167/. Zemmar, Ajmal, Andres M., Lozano, and Bradley J. Nelson. “The Rise of Robots in Surgical Environments during COVID-19.” Nature Machine Intelligence 2, No. 10 (2020): 566-572. https://www.nature.com/articles/s42256-020-00238-2. Zhang, Jingwen, Yoo Jung Oh, Patrick Lange, Zhou Yu, and Yoshimi Fukuoka. “Artificial Intelligence Chatbot Behavior Change Model for Designing Artificial Intelligence Chatbots to Promote Physical Activity and a Healthy Diet.” Journal of Medical Internet Research Vol. 22, No. 9 (2020): e22845. https://www.jmir.org/2020/9/e22845/.

CHAPTER IV THE AI-RUN ECONOMY: SOME ETHICAL ISSUES ANTON GERUNOV Introduction The increased automation of human activity essentially means that a plethora of activities that used to be performed by sapient and conscious human beings are now gradually transferred to machines, or more precisely, to artificial intelligence agents. The most visible examples of such automation are phenomena like self-driving cars and autonomous weapons systems. It is, therefore, little surprise that the ethical issues connected to those specific applications are gaining increasing prominence. Those concerns are clearly exemplified in the journal Nature’s specific ethics of AI invited commentaries. There Stuart Russell1 strongly warns of the Lethal Autonomous Weapons Systems (LAWS), while Veloso2 urges to find complementarities and synergies between humans and AI. These specific instances of AI applications may, however, be misleading as to the overall ethical implications of this emerging technology. This paper will thus take a broader perspective and try to elucidate the implications of a full-fledged transformation of the economy by AI. The trends for ever larger use of automated business decision-making leveraging big data are

1

Stuart Russell, “Take a Stand on AI Weapons,” Nature, 521 (7553) (May 2015): 415-416. 2 Veloso, Manuela, “Embrace a Robot-Human World,” Nature, 521 (7553) (May 2015): 416-418.

The AI-Run Economy

53

quite obvious at this point3 but the key question remains as to what would be the implications of marginalizing or even almost completely eliminating human actors from a majority of economic choices. Apart from the obvious individual consequences, such a move is likely to generate systemic consequences that can be aptly explored via means of comprehensive simulations. Our research aims to outline such an approach by constructing two very simple model economies–a human-run and an AI-run one and comparing and contrasting their performance. Based on this simulation experiment, we further analyze some ethical issues that will arise as business automation and AI-driven decision-making become ubiquitous.

Literature Review The increasing abilities to process potentially sensitive information together with the explosion of data availability and the rapidly decreasing costs of computing have enabled unprecedented mass data processing with potentially large implications for humans. Such data processing – either on its own, or as enabling AI creates a plethora of ethical issues that have increased in complexity over the past decades. In the era before autonomous decision-making by machines (so-called domain AI) ethical issues revolved mainly around data and included issues such as the privacy, accuracy, property, and accessibility of information.4 A key issue here was how personal information is protected in such a way so as not to cause harm to individuals, and what its overall governance (and thus power) structure is. This early thinking on data ethics presupposes human agency in the use of information and so the key question is who possesses the data and has the right to profit from it. A natural early question of data ethics is about the locus of ownership and the power asymmetries surrounding it. This is then still a human-centric version of ethics.

3

Hsinchun Chen, Roger H. Chiang and Veda C. Storey, “Business Intelligence and Analytics: From Big Data to Big Impact,” MIS Quarterly, 36(4) (December 2012): 1165-1188. 4 Mason, Richard O. “Four Ethical Issues of the Information Age.” In Computer Ethics, ed. John Weckert (London: Routledge, 2017), 41-48.

Chapter IV

54

Technological developments meant that the agency in decision-making has been slowly moving from humans to machines, or AI. Following Bryson and Winfield,5 we define Artificial Intelligence (AI) as any digital artefact displaying one or more of the following properties: Ɣ Ɣ Ɣ

The capacity to perceive contexts for actions; The capacity to act; The capacity to associate context with action.

A note of clarification is in order here. When using the term “AI” the usual connotation entails domain AI, where a decision-making algorithm can perform domain-specific activities, replacing or augmenting the human agent. The broader term “Artificial General Intelligence”, or AGI, has come to describe the “real” all-encompassing multi-domain intelligence that will make machines both sentient and sapient.6 The former is already a fact, the latter is in the research phase and will likely need several decades before it is fully developed. Rogozea7 spells a number of necessary characteristics for such AI services. They need to be available, affordable, accountable, compatible, comprehensible and comprehensive. Those characteristics can be viewed as being the foundations for ethical AI, but they neither describe it, nor can be used to judge whether decisions made were indeed ethical in their very nature. Those features merely enable an ethical inspection of the AI system. The introduction of actual machines with agency and potentially destructive capability accelerated the ethical discussion. In 2005, the project Euronet

5 Joanna Bryson and Alan F.T. Winfield, “Standardizing Ethical Design for Artificial

Intelligence and Autonomous Systems,” Computer, 50 (5) (May 2017): 116-119. 6 Nick Boström and Eliezer Yudkowsky, “The Ethics of Artificial Intelligence.” In The Cambridge Handbook of Artificial Intelligence, eds. Keith Frankish and William M. Ramsey (New York: Cambridge University Press, 2014), 316-334. 7 Rogozea, Liliana. “Towards Ethical Aspects on Artificial Intelligence.” In Proceedings of the 8th WSEAS International Conference on Artificial Intelligence, Knowledge Engineering and Data Bases (World Scientific and Engineering Academy and Society (WSEAS), 2009), 507-512.

The AI-Run Economy

55

Roboethics Atelier8 aimed to create the first comprehensive ethical guidance for users, manufacturers and policy-makers in the domain. Some of the major recommendations include Ɣ Safety – the robot needs to be safe by design but also include an override mechanism by a human operator; Ɣ Security – the level of information and physical security of the robot needs to be sufficiently high to avoid malicious attacks; Ɣ Traceability – the robot will need to have a logging and audit system to record their activities; Ɣ Identifiability – the robots should have unique identifiers; Ɣ Privacy – sensitive personal data of humans need to be protected. Research in this domain continues activity.9 Another major spur for the ethics discussions was the development and introduction of self-driving cars and their potential deleterious impact on human beings.10 In this context Etzioni and Etzioni11 underline that there are two major ways to introduce ethical aspects to AI-driven decision-makers: top-down and bottom-up. Top-down approaches include “teaching” a system of ethics to the machine, while bottom-up approaches involve learning from a large number of actual human decisions in such a way that the AI can reconstruct a system of ethics. Etzioni and Etzioni12 argue that both approaches are unfeasible and probably unnecessary as the locus of ethics needs to remain in humans either through their preferences or through legislated norms.

8

Veruggio, Gianmarco. “The Euron Roboethics Roadmap.” In 2006 6th IEEE-RAS International Conference on Humanoid Robots (Genoa, December 4-6, 2006), 612617. 9 Wendell Wallach and Colin Allen, Moral Machines: Teaching Robots Right from Wrong (Oxford: Oxford University Press, 2008). Patrick Lin, Keith Abney and George A. Bekey, eds. Robot Ethics. The Ethical and Social Implications of Robotics (Cambridge, Massachusetts. London, England: The MIT Press, 2012). 10 Jean-François Bonnefon, Azim Shariff and Iyad Rahwan, “The Social Dilemma of Autonomous Vehicles,” Science, No. 352 (June 2016): 1573-1576. 11 Amitai Etzioni and Oren Etzioni, “Incorporating Ethics into Artificial Intelligence,” The Journal of Ethics, 21(4) (March 2017): 403-418. 12 Etzioni and Etzioni, “Incorporating Ethics into Artificial Intelligence”.

56

Chapter IV

While this may well be true in well-regulated domains, it seems unrealistic in others, such as defense and security. Russell13 strongly warns of the technological sophistication of autonomous lethal weapons systems and their ability to target human beings without explicit operator control. This wave of technological innovation has further spurred important ethical debates. Boström and Yudkowski14 argue for a more comprehensive approach to AI ethics, claiming that AI needs to be subjected to the same criteria that are used for human actors performing social functions. More precisely those are Ɣ Responsibility – the AI needs to act in a judicious way so as to fairly make decisions for the benefit of the greater good; Ɣ Transparency – the AI needs to make decisions in such a way that they can be understood by human agents, or even rationalized by humans’ Ɣ Auditability – all activities of the AI need to leave clear traces (e.g. logs) so that they can be reviewed and audited if deemed necessary by human agents; Ɣ Incorruptibility – the algorithms need to be designed in such a way that they cannot be easily compromised by careless or malicious operators or other actors; Ɣ Predictability – the AI needs to produce reliable results, i.e., to produce similar output given a set of inputs in order to ensure stability of decision-making; Ɣ Avoidance of harm to innocent humans – the aspect that has naturally attracted most attention is the safety of AI. It needs to be designed in such a way that no harm or unintended negative effects are brought to bear on humans in contact with the AI. Torresen15 underlines the important point that even under strict rules for AI, some resultant harm may be difficult to predict or avoid, thus further

13

Russell, “Take a Stand on AI Weapons”. Boström and Yudkowsky, “The Ethics of Artificial Intelligence”. 15 Jim Torresen, “A Review of Future and Ethical Perspectives of Robotics and AI,” Frontiers in Robotics and AI, 4, 75 (January 2018). 14

The AI-Run Economy

57

complicating the ethical analysis of AI decision-making. In this paper we aim to show how those overarching principles of ethics can be applied to a hypothetical centralized AI-driven economic decision-maker.

Simulation Results Two simulations are run to better compare and contrast two types of economies. The former is a realistic human-driven economy, where a large number of heterogeneous human agents with bounded rationality make sometimes suboptimal decisions. Those agents are modeled in a behaviorally realistic fashion by drawing major insights from the fruitful research in the field of behavioral economics. While this economy does exhibit a tendency towards full resource utilization, it is hampered by imperfections in markets and individual decisions (inertia, habit formation, imperfect markets, suboptimal decisions, etc.) The latter economy assumes a single centralized, perfectly rational optimizing AI taking care of the entire economy. We assume that the AI economy is programmed in such a way that it exhibits economically rational optimizing behavior so that all productive assets are fully utilized at any given time. This means that the difference between the optimal or potential production (so-called output gap) tends to converge to zero. This section presents the overall structure of the two economies, presents key assumptions in their modeling, and outlines the main results from 10,000 simulated periods of each.

Overall Economic Structure First, we model the overall economic structure by separately identifying the behavior of two aggregated groups of actors – the producers (i.e. the aggregate supply), and the consumers (i.e. the aggregate demand). We use a standard set of equations to model economic dynamics by taking recourse to the canonical New Keynesian DSGE model.16 The economic is thus

16

Jordi Gali, Monetary Policy, Inflation and the Business Cycle (Princeton: Princeton University Press, 2008). Carl Walsh, Monetary Theory and Policy (US: MIT Press, 2003).

58

Chapter IV

described by forward-looking aggregate demand (equation 1) and aggregate supply (equation 2):

The aggregate demand equation (1) shows how demand depends on previous income (yt-1), on the levels of expected prices (ʌt+1), on interest rates (r), and on future expectations of growth (yt+1). In addition, there is a random shock component (İt) taking part of unexpected endogenous or exogenous events. The forward-looking components of this equation (expectations) are the key behavioral input into the determination of production. Similarly, the aggregate supply equation (2) is modeled as dependent on current levels of income (yt), previous price level (ʌt-1), and future expectations about pricing (ʌt+1). And the error term (Șt) is added to account for potential unexpected shocks to the supply side of the economy. Again, the forward-looking components are the main behavioral input into the equation. To close the system, we postulate a process for the interest rates, following a standard Taylor rule.17 Essentially, this rule states that the Central Bank sets the economy-wide interest rates taking reference to the current prices (ʌt) in order to meet its inflation targets, to the current input (yt) in order to avoid excessive harm to the real economy, and is bound by previous interest (rt-1) rates both through decisions and through efforts to implement interest rate smoothing. This equation has no clear behavioral component and can be implemented by both human actors (as it is currently the case) or by algorithms (in a possible future development):

Despite being quite simple, those three key equations present the mainstay of the modeled economy and provide for relatively realistic overall dynamics.

17

John B. Taylor, Macroeconomic Policy in a World Economy: From Econometric Design to Practical Operation (New York: W.W. Norton, 1993).

The AI-Run Economy

59

A Decentralized Economy with Heterogeneous Irrational Agents The first simulation revolves around modeling a number of autonomous heterogeneous agents that show signs of irrationality in their decisionmaking patterns. Previous research18 has shown that humans follow a number of predictable behavior patterns when confronted with economic decisions. Most notably, they default to habit, try to maximize their utility, or mimic others’ decisions (herd behavior). In addition to that, they tend to evolutionary change their behavior in response to previous mistakes, but this adjustment remains imperfect and is bound by habit and inertia. We thus model human expectations using a fixed set of the following heuristics:

In this case H1 corresponds to rationality, whereby humans try to make the best possible forecast; H2–to habit default, where human merely adopt their previous forecasts as the new expectation; and H–to herding, where humans just adopt the expectation that is most prevalent in the economy. Furthermore, we model the instability of human behavior by allowing expectations to vary, thus letting individuals switch between heuristics with a given probability (p) using a logit model. Equation (7) presents the logit switching model for output:

18

Gerunov, Anton. “Experimental Study of Economic Expectations.” In Yearbook of St Kliment Ohridski University of Sofia, 14(1) (Sofia: Sofia University Press, Faculty of Economics and Business Administration, 2016), 103-130.

60

Chapter IV

In order to model even more realistically the effects of avoidance of cognitive dissonance and bounded rationality,19 we also include a lagged term in the switch mechanism, thus reaching:

Concluding, equation 8 gives the probabilities of agents choosing either of the three main heuristics for expectation formation. The overall economywide expectations are thus formed as a weighted average of individual expectations in accordance to individual choice probabilities, or:

This system of equations (4) through (9) aims to realistically model uniquely human behavior and allow us to see how it influences observed economic dynamics. This is done in exactly the same fashion for inflation as it is shown for output in eq. (4)-(9). In short, here we model economic expectations and decisions as dependent on irrational human impulses and show how this irrationality spills into economy-wide inertia and suboptimality.

A Centralized Economy with a Single Rational Decision-Maker The polar opposite of having a decentralize human-run economy is to have a centralized machine-run one. In this case, we expect decisions to be taken rationally, following the tenets of utility, profit, and welfare maximization across the system of interconnected markets. This means that expectations will be set rationally, and current decisions will be based on two key components: the totality of currently available information (Ĭt,1), and an accurate and unbiased forecast of the future. We model this by setting the following expectation formation mechanism for the rational AI decisionmakers (for output gap and inflation, respectively):

19

Daniel Kahneman, Thinking, Fast and Slow (New York: Farrar, Straus and Giroux, 2011).

The AI-Run Economy

61

We hypothesize that the AI is not bound by irrational habits and will have no incentive to switch away from the optimal decision-making. The fact that we are dealing with a single unified decision-maker also means that there is no need for an aggregation mechanism for disparate individual expectations. Due to the long-term trend of the economy to drift toward its potential production, the rational expectations for output and inflation would be set to zero plus a random shock due to unforeseen circumstances.

Simulation Results To simulate the two model economies, we use standard numeric values for the parameters as per the literature.20 Those are presented in Table IV.1 and can be shown to generate realistic economic dynamics.21 Each economy is then simulated over 10,000 periods and its overall statistical properties are shown and investigated. Detailed results for inflation and output are presented in Table IV.2. Table IV.1: Model Parametrization

Both simulated economies are designed in such a way that they tend to converge to full utilization of production factors (or potential output). Thus, it is not surprising that both of them have a mean output gap that is not significantly different from zero. This is also true for the corresponding

20

Gali, Monetary Policy, Inflation and the Business Cycle, 58. Anton Gerunov, Macroeconomic Modeling: Modern Approaches (Sofia: Sofia University Press, 2015). 21

62

Chapter IV

aggregate of prices. Over the long run, both human and AI-run economies tend to stabilize in the absence of large external shock. Table IV.2: Statistical Properties of the Two Simulated Economies

However, the key factor to note here is that the AI economy produces very different dynamics in this process. The human-run economy has both pronounced peaks and deep recessions in growth, with a total range over 15% between the highest and the lowest points. This is the typical economic cycle of boom and bust. In contrast, the AI-run economy has an extremely limited range of the gap–only 5.1%. This is also reflected in the standard deviations–the human-run economy features standard deviations for output that are three times as large as those generated by the simulated rational decision-maker. The proportions are similar when it comes to the price dynamics. The range of the human-run economy is about 2.5 times larger than that for the AI one, and the standard deviations are two times greater. Those results can be easily followed in Figure IV.1.

Figure IV.1: Output Gap in Two Simulated Economies It is straightforward to identify the differences in economic dynamics by following the output gap over time (see Figure IV.2). In the AI-run economy, there are virtually no dramatic drops in activity, and there are no upsurges. The trend in production is smooth, thus eliminating transition and adaptation costs for different automatic and human economic agents. Those

The AI-Run Economy

63

fluctuations persist in the human-run economy and impose possibly large overall costs on social welfare. The trends in inflation follow very similar dynamics.

Figure IV.2: Output Gap Dynamics in Two Simulated Economies In short, the AI-run economy features much lower volatility, greater shortrun stability, and has virtually eliminated violent economic crises and possible hyperinflation risks. We will use those economic simulations as starting points for investigating possible ethical issues in case such a transformation takes place.

Ethical Issues Leveraging the Boström and Yudkowski’s framework,22 we investigate the ethical aspects of transforming the economy from a decentralized boundedly rational system, run by humans into a centralized rational system, run by 22

Boström and Yudkowsky, “The Ethics of Artificial Intelligence”.

64

Chapter IV

AI. The following aspects are evaluated toward this end: responsibility, transparency, auditability, incorruptibility, predictability, avoidance of harm. The responsibility of the system refers to its ability to make fair and responsible decisions in pursuit of a beneficial goal. Depending on how the AI is implemented, we may observe different facets. In case the AI decisionmaker is programmed top-down with some pre-defined objective functions that are to be optimized, the AI can only be expected to take limited responsibility for its narrowly defined objectives. If the AI is instead trained in a bottom-up fashion, it should have a holistic understanding of its responsibility. At any rate, the ultimate rationale for the AI decisions and actions needs to be clear and will likely converge to some type of maximization of individual and group welfare. The end deliberation of whether the AI is responsible may need to be taken after extensive analysis is conducted. The transparency of the algorithm will again depend on its implementation. Detailed pre-programming or recourse to simple machine learning methods such as decision trees or even random forests will greatly enhance the transparency of decision-making. On the other hand, in the more likely scenario-sophisticated methods such as deep neural networks are used, then the transparency of the algorithm will likely be low and difficult to comprehend even by domain experts. This outlines a major trade-off in leveraging AI for business automation – transparency might have to be sacrificed to obtain efficiency. Such issues are unlikely to be solved computationally by identifying some optimal trade-off. The auditability of the AI is yet another issue at hand. With the automation of decisions and under current best practices and extant trends for logging, this seems to be a relatively minor issue. It is highly likely that a large part (if not all) of the actions and decisions of the AI are to be meticulously documented and available for further investigation. A possible concern here may be tampering with those logs, but some emerging technologies such as the blockchain can prevent that as well. Overall, auditability seems to be of limited concern.

The AI-Run Economy

65

The incorruptibility of the system is intimately connected to its transparency. The main goal here is to prevent the misuse of the system through its internal actions or through the action of outside agents. The incorruptibility is difficult to assess as there is hardly a way to decide whether a series of decisions is the result of the original process or a corrupted one. This is particularly true in our current case for a series of complex economic decisions that aim to align a set of incentives in such a way that the medium-term growth trend targets the elimination of a synthetically calculated gap. Thus, the incorruptibility can hardly be ascertained through the AI’s outcome but rather needs to be checked by investigating the process that generated it. Hence, transparency is needed. The predictability of the AI is another pivotal point to consider. This key ethical issue supposes that the stability needed for a complex social or economic system to function must be predicated on a certain degree of consistency, or predictability, of decision-making. In the case of the AI-run economy, we conjecture that there may be a high degree of predictability of outcomes, and a significantly lower degree of predictability of the process. Regardless of whether the AI’s objective is pre-programmed in a top-down approach or learned through bottom-up, it is highly likely that its resulting actions will tend to maximize profit for firms, utility for consumers, and overall social welfare within some given constraints. Moreover, we expect a certain stabilization behavior as shown in the simulations presented here, as extreme volatility tends to impose costs that are rationally avoided. In terms of means of achieving all of this, the AI will probably exhibit less predictability given the domain complexity. The final, and probably most controversial, ethical issue connected to a potential AI-run economy is the algorithm’s ability to inflict harm upon innocent humans. While this is usually connected to inflicting easily detectable physical harm, e.g., in the case of self-driving cars of LAWS, the AI-driven economic decision-maker has the capacity to inflict more insidious damage. As a result of some production or supply decisions, humans may be left with insufficient goods or services, thus endangering their health, physiological or emotional well-being. This subtle damage can be amplified at scale, leading to a serious decrease in social welfare. More directly, the economic AI may inflict damage by supplying goods that are

66

Chapter IV

demanded by consumers but are detrimental to them, such as weapons, drugs, or excessive amounts of alcohol. The AI-driven economy has yet another subtle property. It serves to greatly smoothen the economic cycle, thus depriving the human agents of both the spur of difficulty in downturns and the elation of exuberance in economic upturns. This realignment of incentives may lead to critical repercussions on the consumer behavior and the capacity for innovation. While those effects are barely studied, Sidhu and Deletraz23 show that excessive comfort has negative implications on entrepreneurial attitudes and risk tolerance. This may easily translate into a lower capacity for innovation. The harm in automating decision-making may thus not be physical pain but rather the stifling of human creativity.

Discussion The two simulated economies presented here serve as useful vantage points for applying a set of ethical criteria toward a hypothetical AI economic decision-maker. It seems that algorithmic responsibility and transparency will crucially depend on the specific implementation choices. In the example presented, both conditions were satisfied as responsibility was ensured through clear optimization behavior, while transparency stemmed from a nearly defined set of equations used by the AI. In a more realistic setting that leverages complex machine learning algorithms such as deep neural networks, those two properties are far from guaranteed. The auditability property of AI seems to be the least concern of all since extensive logging allows for a clear audit trail enabling ex-post forensics (if needed). AI’s incorruptibility, on the other hand, seems to be particularly difficult to assess in situations where the AI decision-maker is responsible for complex decisions in a non-linear uncertain environment such as the economy. Such cases call for process inspection rather than for evaluation of outcomes. This property is tightly connected to auditability and transparency. If the AI

23

Sidhu, Ikhlaq and Paris Deletraz. “Effect of Comfort Zone on Entrepreneurship Potential, Innovation Culture, and Career Satisfaction.” In 122nd ASEE Annual Conference & Exposition. Making Value of Society (Seattle, WA, June 14-17, 2015), 1770-1783.

The AI-Run Economy

67

algorithm is both transparent and auditable, then the human agent will find it much easier to ascertain whether the AI has been corrupted or is malfunctioning. Predictability also relies heavily on transparency, but it seems that outcomes will be easier to predict while the process used to reach them will not be. This is particularly true if the AI is trained using sophisticated black-box methods. Finally, the possibility of inflicting unforeseen and possibly undetectable damage is quite high in the case of economic decision-making by AI. Given the high complexity of the environment and since there are no discernible human losses, a possible suboptimal, malicious, or malfunctioning AI may remain undetected for an extended period of time. This holds particularly true for intangible damages, such as incentive misalignment and decreasing innovative and creative potential. It is clear that while some of the ethical requirements align neatly with economic imperatives (e.g. responsibility and effectiveness), others diverge quite dramatically (e.g. transparency and efficiency). Thus, the automation of economic and business decision-making will pose substantive moral and ethical questions and will necessitate significant tradeoffs between the desirable properties of the AI. We venture to propose that in order to achieve an optimal balance between objective constraints, large-scale and high-risk Artificial Intelligence algorithms need to go through a formal assessment and approval procedure. This may be in the form of an AI Impact Assessment that is conducted to answer the complex and interconnected ethical questions posed by complex AI algorithms. This Impact Assessment needs to balance the economic and business needs of producers and consumers against the safety and ethical concerns of broader society. While such an assessment does not need to be conducted by a state authority and can be relegated to alternative providers, its results may have to achieve a minimum amount of consensus among stakeholders before the evaluated AI is put into production. Such a rigorous multi-stakeholder approach will enable humankind to ensure that AI will end up creating more benefits than inflicting unsuspected harm.

68

Chapter IV

Conclusion This paper focused its attention on an area of AI decision-making that is still not substantially researched–the ethical issues stemming from the automation of economic and business decisions. To this end, we have presented and contrasted two model economies and shown how an AIdriven economy dramatically differs from the current one. Its new properties raise a set of ethical questions that were formally addressed by using the framework topic, as summarized by Boström and Yudkowski.24 While some of the issues necessitate concrete implementation to be fully resolved, we reached some preliminary conclusion that outline that a number of difficult tradeoffs may need to be made when AIs are comprehensively put into production and service. Our proposal is to define a rigorous assessment process with large-scale stakeholder involvement that ensures the beneficial utilization of the upcoming Artificial Intelligence algorithms.

References Bonnefon, Jean-François, Azim Shariff, and Iyad Rahwan. “The Social Dilemma of Autonomous Vehicles.” Science, No. 352 (2016): 15731576. https://doi. org/ 10.1126/science.aaf2654. Boström, Nick, and Eliezer Yudkowsky. “The Ethics of Artificial Intelligence.” In The Cambridge Handbook of Artificial Intelligence, edited by Keith Frankish and William M. Ramsey, 316-334. (New York: Cambridge University Press, 2014). Bryson, Joanna, and Alan F.T. Winfield. “Standardizing Ethical Design for Artificial Intelligence and Autonomous Systems.” Computer, 50 (5) (2017): 116-119. http://doi.org/10.1109/MC.2017.154. Chen, Hsinchun, Roger H. Chiang, and Veda C. Storey. “Business Intelligence and Analytics: From Big Data to Big Impact.” MIS Quarterly, 36(4) (2012): 1165-1188. Etzioni, Amitai, and Oren Etzioni. “Incorporating Ethics into Artificial Intelligence.” The Journal of Ethics, 21(4) (2017): 403-418. https://doi.org/10.1007/s10892-017-9252-2.

24

Boström and Yudkowsky, “The Ethics of Artificial Intelligence”.

The AI-Run Economy

69

Gali, Jordi. 2008. Monetary Policy, Inflation and the Business Cycle. Princeton: Princeton University Press. Gerunov, Anton. 2015. Macroeconomic Modeling: Modern Approaches. Sofia: Sofia University Press. Gerunov, Anton. “Experimental Study of Economic Expectations.” In Yearbook of St Kliment Ohridski University of Sofia, 14(1), 103-130 (Sofia: Sofia University Press, Faculty of Economics and Business Administration, 2016). Kahneman, Daniel. 2011. Thinking, Fast and Slow. New York: Farrar, Straus and Giroux. Lin, Patrick, Keith Abney, and George A. Bekey, eds. 2012. Robot Ethics. The Ethical and Social Implications of Robotics. Cambridge, Massachusetts. London, England: The MIT Press. Mason, Richard O. “Four Ethical Issues of the Information Age.” In Computer Ethics, edited by John Weckert, 41-48. (London: Routledge, 2017). Rogozea, Liliana. “Towards Ethical Aspects on Artificial Intelligence.” In Proceedings of the 8th WSEAS International Conference on Artificial Intelligence, Knowledge Engineering and Data Bases, 507-512. (World Scientific and Engineering Academy and Society (WSEAS), 2009). Russell, Stuart. “Take a Stand on AI Weapons.” Nature, 521 (7553) (2015): 415-416. Sidhu, Ikhlaq, and Paris Deletraz. “Effect of Comfort Zone on Entrepreneurship Potential, Innovation Culture, and Career Satisfaction.” In 122nd ASEE Annual Conference and Exposition. Making Value of Society, 1770-1783 (Seattle, WA, June 14-17, 2015). Taylor, John B. 1993. Macroeconomic Policy in a World Economy: From Econometric Design to Practical Operation. New York: W.W. Norton. Torresen, Jim. “A Review of Future and Ethical Perspectives of Robotics and AI.” Frontiers in Robotics and AI, 4, 75 (2018). https://doi/org/10.3389/frobt.2017.00075. Veloso, Manuela. “Embrace a Robot-Human World.” Nature, 521 (7553) (2015): 416-418.

70

Chapter IV

Veruggio, Gianmarco. “The Euron Roboethics Roadmap.” In 2006 6th IEEE-RAS International Conference on Humanoid Robots, 612-617. (Genoa, December 4-6, 2006). Accessed July 7, 2022. https://www3.nd.edu/~rbarger/ethics-roadmap.pdf. Wallach, Wendell, and Colin Allen. 2008. Moral Machines: Teaching Robots Right from Wrong. Oxford: Oxford University Press. Walsh, Carl. 2003. Monetary Theory and Policy. US: MIT Press.

CHAPTER V SCRATCH MY BACK & I WILL SCRATCH YOURS: PUTTING GAME THEORY AND ARTIFICIAL INTELLIGENCE ON A CONVERGING PATH BEYOND COMPUTATIONAL CAPABILITIES BORIS GUROV Game Theory Theorization and Possible Practical Applications and Solutions What are the options for optimizing the use of congested highways to the seaside and back during weekends? Is there a way to limit increased individual rational spending during periods of inflation, which triggers the aggravation of the phenomenon and paves the way for hyperinflation? How do two concurrent firms overcome the competition problem and collaborate instead in the creation of a new technical standard that will ensure their dominant position in the market? All of the above, without the intervention of a centralized authority. Those are some questions that game theory seeks answers to. Why would this type of problematic and analytical focus be of use for the development of AI? And vice versa. It is very tempting to declare that game theory was once a very promising research field, which reached its analytical limits and provided some good insight into decision making under uncertainty, and gave some theoretical awareness of the complex links between autonomous and free individual decision-making and the harmonious functioning of the collective, or the group. The problem with such a position lies in the fact that game theory is the sole analytical tool available to us for bridging the gap between

72

Chapter V

individual strategical behavior and reasoning in terms of group and group effects. Game theory might be highly formalized and offer complex and counterintuitive solutions, but the fact is that it represents the only path towards thinking and alleviating the tension between individual rationality (agent optimization) and collectively rational outcomes (social optimality). What is the best possible approach for a fleet of military ships to engage an enemy fleet depending on its goals (destroy the enemy, inflict heavy damage, hold them off, limit its own losses)?1 How do ships coordinate themselves and how do they cooperate with each other? Is it better that there is a centralized top-down decision-making mechanism in place, or the overall performance will gain from different ships in the fleet benefitting from some form of autonomy? There is a myriad of problems and layers of overlapping complex non-linear effects to be considered even before starting to answer such a question. Even if we eliminate the idea that we do have battleships with human crews and we are dealing with drones instead, which will reduce all ethical issues to resource management, the complexity of the problem remains impressive. What game theory provides for proposing a practical solution for such problems are exquisite models and quantification of this type of interrelations/interaction, quantifications that can produce a virtually unlimited amount of input for AI data hungry algorithms. Another path to be explored is that, by definition, those models are rooted in a decentralized decision-making optics. Given the recent advances of AI toward decentralization, for example, in the field of advanced architecture, GT could prove an extremely useful tool for shifting the focus from individually optimizing and adapting algorithms (deep learning on the individual algorithmic level) towards the creation of smart networks where different algorithms cooperate and coordinate their answers to systemic changes and disequilibria via common knowledge. Pushing the research agenda in the domain of AI in this direction may be an ambitious endeavor, but the

1

The history of Naval warfare testifies for the very heterogeneous goals of fleet commanders in different contexts. Of course, classical game theory is unable to explore the interdependency of such heterogeneous logics. Generally, it will assume the goal to be full-scale maximum damage to the opponent type of interaction.

Scratch My Back & I Will Scratch Yours

73

potential benefits are obvious. Last, but not least, exploring gametheoretical models via AI has the potential, if it goes beyond the usual omission of theoretical discussions, characteristic of the field (a tendency, which was rather present even in the 1980s and 1990s in the domain of evolutionary GT), to reconceptualize our understanding and analytical procedures of rationality. Bluntly said, engineers should start consulting more often the literature of economists.

Individual Optimization, Pareto-optimality, Common Knowledge. General Features of Static Gamer Theory– Historical Perspective Historically, GT was confronted with a decisive crossroad from its beginnings. From 1921 onward (1921-1928), French mathematician Emile Borel published a series of articles on the ways to formalize and study social situations and interactions in the form of games. What is crucial about Borel is that he considered that game analysis should be carried out from the standpoint of players.2 It is not a position that has gained popularity. Most specialists consider John Von Neumann as the real founding father of GT. He had a drastically different conception of how to abord games. In his 1928, “Theory of Parlor Games”,3 he profoundly marked the future development of GT by defining the task of the analyst as one of finding a mathematical solution to the game. The important point here is that Von Neuman’s formalist-theoretical vision aimed at a perfect mathematical solution4 prevailed over the psychological-empiricist search for individual 2

Emile Borel,”La théorie des jeux et les équations à noyau symétrique gauche.” In Comptes rendus de l’Académie des sciences, Vol. 173 (Paris: Académie des Sciences Morales et Politiques), 1304–1308. 3 There are two separate translations of the original text from German (“Zur Theorie der Gesellschaftsspiele,” Matematische Annalen, Vol. 100 (1928): 295-320). “Theory of Parlor Games” is in our humble opinion the better title, but the fact is that among Anglo-Saxon specialists it is the other version that is by far the dominant reference: John von Neuman, “On the Theory of Games of Strategy.” In Contributions to the Theory of Games, Vol. 4, eds. Albert Tucker and Duncan Luce (Princeton: Princeton University Press, 1959), 13-42. 4 Christian Schmidt makes a very convincing case for the unintended consequences for GT of the impact that Von Neuman had on its research program by subordinating

74

Chapter V

strategy formation of Borel. Von Neuman started with the analysis of bilateral games (interactions) and forged a considerable part of the conceptual tools of modern GT (strategy, outcome, payoff, player, etc.). Game theory took a sharp turn toward economics with the publication of Theory of Games and Economic Behavior (1944) in which Von Neuman collaborated with Oscar Montgerstern and they laid the foundations for expected utility theory and anchored in a definitive manner GT as a powerful hypothetico-deductive tool. The subsequent development of the domain was solely based on formalized mathematical procedures using as premises individual self-interested rationality of agents as a theoretical definition and exploring the effects on the aggregated level of the system of players. Probably the most famous mixed-motive (there are possibilities both for cooperation and conflict) bilateral game is the Prisoner’s dilemma. Below is its classical matrix of payments.

Fig. 1. Prisoner’s dilemma

axiomatically the rationality of players to the solution of the game, which in turn becomes defining for the definition of the rationality of players. Cf. Schmidt, Christian “From the ‘Standards of Behaving’ to the ‘Theory of Social Situations’. A Contribution of Game Theory to the Understanding of Institutions.” In Knowledge, Social Institutions, and the Division of Labor, eds. Pier Luigi Porta, Roberto Scazzieri and Andrew Skinner (Cheltenham: Edward Elgar Publishing, 2001), 153168.

Scratch My Back & I Will Scratch Yours

75

Prisoner’s dilemma (PD) erupted on the front scene of decision-making in the 1950s5 and is the most famous and discussed strategic interaction, with thousands of publications exclusively dedicated to it. In its classical version, the PD puts in an interaction with two outlaws arrested by the police. They are presented before a judge who can charge them with minor offenses (each one will go to prison for 2 years for illegal arms possession) or try to convict them of a major crime (armed robbery) for which the prosecution lacks evidence, however. The judge uses a demonic logic: each detainee is given the choice of either testifying against his/her accomplice (defect towards him/her), with the explanation that if they both denounce each other, they will go for 5 years in prison, and in the event of one of them confessing and the other not, the witness for the prosecution is set free and the other will serve the maximum possible sentence: 10 years in prison. The problem with the PD is that whatever the other party does, it is preferable for ego to defect. In this sense, from the standpoint of individual rationality, there is no dilemma whatsoever. Communication and coordination do not change anything: whatever the other party does it is still preferable for each player to confess and defect. The decision-making process being simultaneous or sequential, also does not affect the outcome: in the PD, defection is a strictly dominant strategy. The force of attraction of the PD comes from the extreme simplicity with which it puts in perspective the obstacles for selfinterested, instrumentally rational, egoistic individuals to benefit from mutually advantageous cooperation. PD became the ensign for the tension between individual and group rationality.6 Among the 78 (2 X 2) strategically non-equivalent games (2 players and 2 strategies per player), 5

It is the Rand Corporation who decided to finance the pioneers Flood and Dresher to test “Nash solutions” for non-cooperative games to support the US nuclear policy and doctrine. Cf Marvin Flood, Some Experimental Games. Research Memorandum RM–789 (Santa Monica, Cal.: Rand Corporation, 1952). Flood and Dresher are the first who have formalized the PD. Often it is the name of Albert Tucker, which is mistakenly associated with the primacy as to putting the PD in the centre of interest of GT. It is somewhat ironic that Tucker, who is a mathematician and who was the Ph.D. advisor of non-other than John Nash (popularized by actor Russell Crowe in the biographical drama film A Beautiful Mind) is credited with putting together the dramatic literary narrative that accompanies the model. 6 Rapoport, Anatol. “Prisoner’s Dilemma-Recollections and Observations.” In Game Theory as a Theory of Conflict Resolution, ed. Anatol Rapoport (Dordrecht: D. Reidel Publishing Company, 1974), 17-34.

76

Chapter V

which are present in Rapoport and Guyer’s classification,7 the PD is unique in its characteristics. It is the only one, which has a unique and stable equilibrium,8 which is Pareto-deficient (or Pareto-non optimal)9. Mutual defection is the only equilibrium in both pure and mixed strategies and defection is the dominant strategy for both players. Simultaneous and sequential playing both lead to the collectively unacceptable outcome. If it is a one-shot game, there is no room for cooperation. A point of special importance, especially in the light of bringing together GT and AI, resides in the direction that game theorists chose in their efforts to deal with the complexity of the problems posed by the interaction of players. It is a hypothesis (complete information)10 emitted in a rudimentary form for the first time by Von Neumann and Montgerstern,11 refined by John Nash12 and after some empirical tests and experimentation by Thomas Schelling,13 it became one of the pillars of David Lewis’s theory of social

7

Anatol Rapoport and Melvin J. Guyer, “A Taxonomy of 2 X 2 Games,” General Systems, Vol. 11 (1966): 203-214. Cf. also Anatol Rapoport and Melvin J. Guyer and David Gordon, The 2 X 2 Game (Ann Arbor: The University of Michigan Press, 1976). 8 Nash equilibrium (John Nash is Noble Laureate in economics in 1994 and Abel Prize winner in 2015) represents the outcome of the game in which the strategy of each player is the best possible response to the strategy chosen by the other player. It is also called “no-regret situation” given the fact that after the game is played no ulterior modification of any player’s strategy could provide him/her with a better payoff. 9 In contemporary economics a situation is Pareto-optimal if there is no other social situation theoretically conceivable that can ameliorate the resource situation of at least one individual, without deteriorating that of another one. The process of switching from a non-optimal to optimal situation being called Pareto-amelioration. 10 The concept of complete information considers that game situations are such that “each player is fully aware of the rules of the game and the utility functions of each of the players” [Duncan R. Luce and Howard Raiffa, Games and Decisions: Introduction and Critical Survey (Wiley, 1957)]. 11 John von Neuman and Oscar Montgerstern, Theory of Games and Economic Behavior (Princeton: Princeton University Press, 1944). 12 John F. Nash, “Equilibrium Point in N-person Games,” Proceedings of the National Academy of Science, No. 36 (1950): 48-49. 13 Thomas Schelling, The Strategy of Conflict (Cambridge, MA: Harvard University Press, 1960).

Scratch My Back & I Will Scratch Yours

77

conventions.14 Lewis developed the first systematic account of the concept of common knowledge,15 which is crucial for the correct understanding of orthodox game theory It is through common knowledge that game theorists chose to overcome the indeterminacy problem. Generally, common knowledge is explained as an assumption for the complete specification of the model, completing the notion of complete information (each player knows the rule of the game and each other utility functions) by making players additionally required to be aware of this fact; players must be aware of the awareness of the other players regarding the rules and each other’s utility functions. Furthermore, each player must be aware that each player is aware that all other players are aware, and so on. To put it in a nutshell, the common knowledge serves to smash the vicious circle of rational anticipation. The principle is especially useful in games (unlike the PD) in which there is no equilibrium (the assurance game) or there are multiple concurrent equilibria (the dating game), and its main purpose is coordination between players and avoiding sub-optimal outcomes. Given the fact that players are rational, they anticipate other players being rational and on the collective level, they all opt for the solution of the game: the Nash equilibrium, which is the demonstration of their rationality. In fact, it is not the rationality of players that leads to the solution of the game, but the solution of the game that brings out the rationality of players. Common knowledge assures the perfect uniformization of the point of view of players and the game theorists and as Schmidt16 and Hedoin17 rightly point out, there are enough elements in the theoretical and analytical backbone of game theory to strongly question its automatic affiliation with micro/individualistic explanations. Both authors point to common knowledge as a vivid illustration that the logic, which operates game-theoretical models, implies 14 David K. Lewis, Convention: A Philosophical Study (Oxford: Blackwell Publishing, 1969). 15 Cyril Hedoin, “A Framework for Community-based Salience: Common Knowledge, Common Understanding and Community Membership,” Economics and Philosophy, 30 (2014): 366. 16 Schmidt, “From the ‘Standards of Behaving’ to the ‘Theory of Social Situations’. A Contribution of Game Theory to the Understanding of Institutions”. 17 Cyril Hedoin, “Linking Institutions to Economic Performance: The Role of Macro-Structures in Micro-Explanations,” Journal of Institutional Economics, 8(3) (2012): 327-349.

78

Chapter V

to a very large extent the pre-existence of macrostructures that are directing the interaction between individual agents. Common knowledge is interesting from our perspective because it provides a way to vastly improve the interactional capacities of isolated agents (or strategies, or players, or algorithms). In fact, in many game-theoretical models, the postulate of common knowledge is a bridge between individual rationality and common rationality. Even if in its classical version, the universe of common knowledge is an outdated and exhausted analytically hypothetico-deductive common space of interactional logic, there are no obstacles to transposing this interactional model to have meaning for smart, heterogenic, and adaptative strategies/players/algorithms/ or even individual single-agent neural networks and why not, even it is probably overambitious, multiagent deep neural network.18 Two more remarks on the PD are of order: When the PD is played like a one-shot game (interaction is unique), there is no room for cooperation. However, if the game (interaction) is iterated, then there is the possibility of switching toward the Pareto-optimal outcome and cooperation. Isn’t bilateral optics especially constraining regarding the use of GT as a tool to collaborate with AI? Bilateral games in GT are by far more renowned, and when future interactions and heterogenic strategical populations have introduced a myriad of endless bilateral interactions generate an astonishing amount of data and potential input for the datahungry algorithms. Moreover, the classical problems of collective action (the n-person prisoner’s dilemma) are treated in GT with much of the same conceptual framework and even though there are some differences in the hypothetico-deductive treatment of models, those divergencies fade away 18

When doing the preliminary research for this far-fetched idea I stumbled onto a piece by Elena Nisioty, “In need of evolution: game theory and AI” MAY 12, 2018/#MACHINE LEARNING [“In Need of Evolution: Game Theory and AI”, accessed June 1, 2021, https://www.freecodecamp.org/news/game-theory-and-aiwhere-it-all-started-and-where-it-should-all-stop-82f7bd53a3b4/ “MAY 12, 2018/ #MACHINE LEARNING], which was a factor to pursue in such direction and where very similar ideas, concerning the opportunities for the development of supraindividual neural networks are put to the front.

Scratch My Back & I Will Scratch Yours

79

in the optics of adaptation, learning and especially reinforcement learning algorithms.

Evolutionary Game Theory, Computer Assisted Simulations, Reinforcement Learning, Deep Learning, Machine Learning What computer-assisted simulations (CAS) as an early precursor to the introduction of AI brought to the field of GT was the possibility of the omniscient, omni-calculating rational, but somewhat dorkish economic agent (ultimately repeating eternally the same choice in the same circumstance)19 with evolving, adapting and learning players (strategies/algorithms). That shifted almost immediately the analytical focus from exploring the faith of individual strategies to investigating the outcome and processes on the level of the system of players. That could be players playing in a multitude of bilateral PDs or players involved in a collective action problem (or coalition-building problems) in an n-person PD. It also shifted the balance from optimal individual strategies to strategies that perform well (satisfaction logic replaces individual maximization, limited rationality replaces omniscient economic rationality) in different strategical environments and with different sets of other strategies. Interactional capacities, which were not a topic of interest in deductive GT,20 became a center of attention. The shift of interest toward evolutionary game theory (EGT) started as a process at the end of the 1960s with the sustained work in economics and experimental psychology of pioneers like Herbert Simon and James March21 who were trying to formulate an alternative to the strict economic definition

19 As Vilfredo Pareto put it already in 1900: “From the point of view of choices, the Homo oeconomicus becomes a machine to make these choices, which given the circumstance, always makes the same choice”, cited by Bruni and Guala. Cf. Bruni, Luigino and Francesco Gualo. “Pareto’s Theory of Choice from the Cours to the Traittato. Utility, Idealization and Concrete Deductive Method.” In Pareto aujourd’hui, coll. Sociologies, ed. Bouvier Alban. (Paris: P.U.F., 1999), 118. 20 Except from experimental psychologists who used GT models (mostly PD) and conducted a massive, but somewhat chaotic research on the topic. 21 James G. March and Herbert Simon, Organizations (New York: Dover Publications, Inc., 1958).

80

Chapter V

of rationality. They jointly formulated the theory of bounded rationality (as early as 1958) unsatisfied with the empirical irrelevance of the orthodox economic agent and started working on finding analytical procedures that could reflect this theoretical reorientation. Very quickly, over the next years, they realized that eliminating the consequential program of strict economic rationality comes at a heavy cost: ipso facto it implies the abolition of the perfect analytical toolkit of the economic model and the use of mathematical deduction. Without going into a detailed historical account, it suffices to say that from that point onward, the search for such an alternative had a profound modifying impact on the research in the field of decision-making analysis. And it paved the way for the search for an alternative analytical logic: computer-assisted simulations, the use of Bayesian rationality, probabilistic strategies, and, at the end of the process, AI made its entrance into the field of GT. The most important consequence, however, was that giving up on strict economic rationality became a legitimate act, even to some extent among economists. It is a short 1973 article (just over 3 pages)22 published in “Nature” by the British-American duo John Maynard Smith (evolutionary biologist and geneticist) and George R. Price (population geneticist) that marks the abrupt switch of analytical optics and is associated with being the trigger of the boom of evolutionary GT. It is this paper that is unanimously considered the basis for the effervescence of EGT. The player and his strive for optimality are no longer the centers of attention. It is the collective stability of strategies and their long run fitness in changing and dynamic environments that become the main preoccupation. This shift originated in the conceptual frame of evolutionary biology and introduced a whole new conceptual area in GT bringing to the front concepts like evolutionary stable strategy, collective stability, evolutionary equilibrium selection, strategic selection, strategic replication, random mutations, etc.

22 John M. Smith and George R. Price, “The Logic of Animal Conflict,” Nature 246 (1973): 15–18.

Scratch My Back & I Will Scratch Yours

81

Fig. 2 Principles of the use of replicator dynamics in iterated bilateral games with heterogeneous strategical population Despite some resistance from mainstream game-theorist, this turn was welcomed and relatively quickly adopted by the research community. Pure imports for biology with a Darwinist perspective were very quickly introduced and ecological (replicator dynamics – operating at the populational level) and evolution-based (genetic algorithm – operating at the individual level) simulations became common practice in the field. From an initial set of agents (strategies) the strategic interactions are simulated in a process that favors agents/strategies that procure relatively better results and penalizes those that perform badly Despite the exact parameters chosen by the researcher, the process closely resembles the logic of the survival of the fittest. With each new generation of agents/strategies, the replicator rules determine the composition of the new population (see Fig. 2 above). Controlled or random mutations can also be introduced and thrown into the mix. The center of attention is now shifted toward both the evolution of individually successful strategies and the collective faith of the population and its strategic composition. And their mutual dependency.

82

Chapter V

The major novelties for GT from the introduction of CAS were: ৼ GT shifted from a hypothetico-deductive to a much more experimental analytical logic. ৼ A dynamic, changing environment replaced the static framework of classical GT. ৼ The static equilibrium optic was replaced with analysis in terms of processes, patterns, and mechanisms. ৼ CAS simulations put the spotlight on the feedback loop between individual strategic characteristics (composition of the strategic population) on the micro-level and/or the emerging/emergent structural factors and effects on the macro level. ৼ Classic GT models were definitively anchored in interactions between strategically homogeneous actors (rational agents). CAS introduced the possibility to run and simulate interactions between players with different decisional logics (strategical heterogeneity) and explore the systemic consequences. ৼ Multi-agent modeling, which is intimately linked to the progress of AI was an inextricable part of the spectacular development of evolutive GT in the 1980’s. ৼ CAS simulations introduced the possibility for researchers to manipulate the experimental setting and introduce controlled manipulations of the environment, time, events. etc. This is an important shift, especially in the optics of adapting, evolving, and learning strategies. ৼ CAS simulations greatly improved the empiric relevance of GT. The spectacularly accrued capabilities for specifying the experimental conditions offered the possibility to formulate conditional interactional patterns of the type that if the agents engaged in a specific configuration of their interactions behave in a certain manner, we could expect the subsequent effects on the individual level and the following emerging effects on the systemic level. ৼ CAS also had implications for Rational Choice Theory. It showed the discrepancy between the premises of the analytical model of perfect rationality (perfect information, unlimited capacities for information

Scratch My Back & I Will Scratch Yours

83

treatment, stable and hierarchized preferences, etc.)23 and the real conditions for decision making. The first modification consisted in the fact that several authors opted to think of strategies as a form of intelligent adaptation to context and not as a constant optimization.24 The second, and even more important one, was to shift the focus towards learning processes and capacities (the exploration of Bayesian rationality in the first stages),25 and the subsequent introduction of the genetic algorithm (GA),26 which introduced the optics of adaptation.

23 The economic agent must execute the optimal choice in any theoretically conceivable situation, thus maximising his/her utility function. 24 Robert Axelrod and Michael D. Cohen, “Coping with Complexity: The Adaptive Value of Changing Utility,” The American Economic Review, Vol. 74, No. 1 (March 1974): 30-42. John H. Holland, Adaptation in Natural and Artificial Systems (Michigan: The University of Michigan Press, 1975). Richard H. Day and Theodore Groves, eds. Adaptive Economic Models (New York: Academic Press, 1975). Patrick D. Nolan, “External Selection and Adaptive Change: Alternative Models of Sociocultural Evolution,” Sociological Theory, Vol. 2, (1984): 117-139. Robert Axelrod, The Complexity of Cooperation: Agent-Based Models of Competition and Collaboration (Princeton, N.J.: Princeton University Press, 1997). 25 Richard M. Cyert and Morris DeGroot, “Bayesian Analysis and Duopoly Theory,” The Journal of Political Economy Vol. 78, No. 5 (September–October 1970): 11681184. Richard M. Cyert and Morris DeGroot, “An Analysis of Cooperation and Learning in a Duopoly Context,” The American Economic Review, Vol. 63, No. 1 (March 1973): 24-37. Richard M. Cyert and Morris DeGroot, “Rational Expectations and Bayesian Analysis,” The Journal of Political Economy, Vol. 82, No. 3 (May–June 1974): 521-536. Robert Denhardt and Philip Jeffers, “Social Learning and Economic Behavior: The Process of Economic Socialization,” The American Journal of Economics and Sociology, Vol. 30, No 2 (April: 1971): 113125. Day and Groves, Adaptive Economic Models. John Harsanyi, “Bayesian Theory and Utilitarian Ethics,” The American Economic Review. Papers and Proceedings of the Ninetieth Meeting of the American Economic Association, Vol. 68, No. 2 (May 1978): 223-228. 26 The GA, very roughly, is functioning by safeguarding interactional information that procured relative success to the strategy and discarding information that didn’t for determining better strategic behavior in future interactions (Cf. Holland, Adaptation in Natural and Artificial Systems. John H. Holland, “Adaptive Algorithms for Discovering and Using General Patterns in Growing Knowledge Bases,” International Journal of Policy Analysis and Information Systems, No. 4 (1980): 245-268. John H. Holland, “Genetic Algorithms,” Scientific American, 267, No. 7 (July 1992): 44-50. David. E. Goldberg, Genetic Algorithms in Search, Optimization, and Machine Learning (Reading, Mass.: Addison-Wesley Pub. Co., 1989). Rick L. Riolo, “The Amateur Scientist: Survival of the Fittest Bits,” Scientific

84

Chapter V

The EGT even produced a best-seller and a star author. Robert Axelrod’s The Evolution of Cooperation (1983)27 became one of the most read, translated (20 + languages), and quoted scientific books. It is an excellent starting point for everybody interested in the problemɚtic. It showed in a rather convincing manner that from the moment the interaction in a PD has a future (iterated games) and there is room for a decision-making autonomy outside of the box (strategical heterogeneity), mutual general defection is no longer a pole of universal attraction. Nevertheless, a warning regarding Axelrod’s conclusions is of order. The hasty supposition of Axelrod that the simple reciprocal imitation of the strategy of the other player (the TIT-FORTAT strategy)28 is the best possible strategy, and that, gradually it becomes an evolutionary stable outcome on the level of the population (that the population will evolve towards an all TFT composition and no one will have the incentive to deviate from that collective equilibrium) is flawed. Kenneth Binmore, one of the leading game-theorists during the last 40 years, went so far as to devote a whole chapter to understanding the impact of Axelrod’s work. He labeled the whole interest generated by the work of Axelrod and the hasty acceptance of his conclusions the “TIT FOR TAT bubble”.29 Binmore is critical of the impact of Axelrod’s work and especially of his method of merging easily the borders between classical GT, CAS and quasiexperimentation, theory construction, and the empiric reality. The term is also an important reminder of the effects that taking methodological shortcuts has in the domain of the use of AI: by accepting Axelrod’s conclusions as firm theoretical foundations and proven decision-making patterns number of researchers in the field started integrating his theory of cooperative behavior in their own experiments. This conclusion is valid even to this day and in our understanding, is one of the determining factors for grounding the use of AI in the optic of individual learning/adaptation and the definition of the “work” of agents as understanding, mapping, and American, Vol. 267 (1) (1992): 114-117. This process is coupled with the maintenance of a general strategical line, which is rectified via the input of new interactional information. 27 Robert Axelrod, The Evolution of Cooperation (New York: Basic Books, 1984). 28 Cooperate when the other party cooperates, defect to defection and never start an interaction by a defection. 29 Kenneth Binmore, Playing Fair: Game Theory and the Social Contract, Vol. 1 (Cambridge, Mass.: MIT Press, 1994), 194.

Scratch My Back & I Will Scratch Yours

85

detecting the properties of the environment (its strategical composition and structural properties) and the formulation of a corresponding line of strategic behavior. Learning is a tool to reach coordination with others in multiagent environments.30 There is a variety of multiagent learning techniques proposed with the goal to coordinate interactions on various solutions and outcomes (e.g., Nash equilibrium, specific collective equilibria, Pareto-optimality, etc.) in different settings, e.g., minimax Qlearning,31 Nash Q-learning,32 and Conditional Joint Action Learning,33 quantum computation conditional strategies,34 to name just a few. Game theory is currently investing in the exploration of settings where multiple players learn through reinforcement, an area called multi-agent reinforcement learning (MARL). At the price of a considerable simplification, we can speculate that multi-agent learning is a rather complexified form of single agent learning in which additional neural networks are made available and “plugged” into the agent. The progress in this field is undeniable; but what is important is that ontologically speaking MARL is still a form of complex individual learning. A very complex and still complexifying form of learning, but clearly individual learning. Yet we need more if our choice is to extract more lessons from GT. We need more than strictly individual optimization for comprehending how to prevent pervasive collective effects and how to efficiently channel individual contributions toward the realization of common goals. Enough of rationality! Let’s talk some ethics … Promoting “The Collective” (if we use the Borg terminology from Star Trek) can be seen as a vicious form of totalitarian menace aiming to subdue individuality to the good of the whole. There are four major practical 30

Peter Stone and Manuela Veloso, “Multiagent Systems: A Survey from a Machine Learning Perspective,” Autonomous Robots, 8(3) (2000): 345–383. 31 Littman, Michael. “Markov Games as a Framework for Multi-agent Reinforcement Learning.” In Proceedings of the Eleventh International Conference on Machine Learning (San Francisco: Morgan Kaufmann, July 1994), 157–163. 32 Junling Hu and Michael P. Wellman, “Nash Q-learning for General-sum Stochastic Games,” Journal of Machine Learning Research, Vol. 4 (Nov 2003): 1039–1069. 33 Dipyaman Banerjee and Sandip Sen, “Reaching Pareto-optimality in Prisoner’s Dilemma Using Conditional Joint Action Learning,” Autonomous Agents and MultiAgent Systems 15(1) (2007): 91–108. 34 Konstantinos Giannakis et al., “Quantum Conditional Strategies and Automata for Prisoners’Dilemmata under the EWL Scheme,” Appl. Sci. 9(13) (2019): 2635.

Chapter V

86

consequences that we foresee from pursuing the type of research agenda we promoted here, and we will briefly examine their ethical implications: 1. Upgrading the multi-agent reinforcement learning at the procedural level via the use of GT models (for machines): there are no true ethical issues here. Machine learning is constantly refining and improving itself as a process, regardless of its collaboration with GT. GT is a research and refining path amongst others. 2. Upgrading and refining the analytical capacities in the exploration of GT models and increased understanding of the implications for the empirical world (both for human and machines): again, no ethical issues here. Learning GT and using its practical implications is not an ethical issue per se. Moreover, if I am responsible for negotiating an important peace treaty ending a bloody conflict, I will strongly prefer that the representative of the other party knows GT too. 3. Increased capacity to boost and/or prevent cooperation (for machines and humans): This is a classical ethical problem that could be found in the literature surrounding cooperation. If we go back to authors like Hobbes, Hume, Rousseau, and Smith (and engineers testing algorithms via GT also should do the same) we will discover that the ethical dimensions of the contradicting realities of a harmonious social order and individual liberty and autonomy are present in social and political philosophy from its debuts. Authors like Kenneth Arrow,35 Radiyna Braithwaite,36 Garry Runciman and Amartya Sen,37 Amartya Sen,38 John Rawls,39 Robert Nozick,40

35 Kenneth J. Arrow,

Social Choice and Individual Values (New York: Wiley, 1951). Radiyna B. Braithwaite, Theory of Games as a Tool for the Moral Philosopher (Cambridge: Cambridge University Press, 1955). 37 Garry W. Runciman and Amartya K. Sen, “Games, Justice and the General Will,” Mind, New Series, Vol. 74, No. 296 (October 1965): 554-562. 38 Amartya K. Sen, Ethique et Economie (Paris: Presses Universitaires de France, 2001). 39 John. Rawls, “Justice as Fairness,” Philosophical Review, Vol. 67 (April 1958): 164-194. John. Rawls, A Theory of Justice (Cambridge, Mass.: Belknap Press of Harvard University Press, 1971). 40 Robert Nozik. Anarchy, State and Utopia (New York: Basic Books, 1974). 36

Scratch My Back & I Will Scratch Yours

87

David Gauthier,41 and Gregory Kavka42 continued with that problematic discussing the ethical dimensions of the capacity for promoting/preventing cooperation but none of them declared that such knowledge could be unethical. At the end of the day, understanding those processes is beneficial for autonomous free individuals. Ethical issues arise when we are dealing with a central authority that promotes/prevent cooperation. It becomes, yet again, a question of good or bad governance. Ignoring purposefully those mechanisms is not really a safeguard against bad governance. 4. AI capacity to analyze, detect and prevent negative “unintended consequences” (perverted effects) in complex interactions – (machines and humans): the fact that people often, acting in accordance with rational individual reasons, could trigger collective consequences and phenomena that are unintended, undesirable and negative and non-rational for everybody is a theme that is present in social thought from the times of Weber and Simmel and explicitly formulated research agenda by Popper and von Hayek and later by Raymond Boudon.43 All those authors consider individual freedom as a cornerstone value, yet all of them strived to limit and counter “perverted effects”. There is nothing paradoxical in such a position. As thinkers that openly promote individual freedom, they are perfectly aware of one of the major consequences of its central stage place and growing importance in our societies: the risk of accrued undesirable and unintended negative social effects. As you might already have suspected the author of this article is not an expert in AI (despite a Ph.D. involving CAS), but rather in decision-making. The present text is an effort to persuade specialists in AI to consider the possibility that even though GT proved to be an extremely fruitful lab field

41 David Gauthier, “David Hume, Contractarian,” The Philosophical Review, Vol. 88, No. 1, (1979): 3-38. David Gauthier, Morals by Agreement (Oxford: Clarendon, 1986). 42 Gregory Kavka, Hobbesian Moral and Political Theory (Princeton: Princeton University Press, 1986). 43 Raymond Boudon, Effets pervers et ordre social (Paris: Presses Universitaire de France, 1977).

88

Chapter V

for the progress of different aspects of AI (algorithm refinement, learning procedures, deep learning, etc.) it is grant time to decide if the area of GT is something more than a test tube for specific algorithms and a tool for optimization of resources under constraints. Since GT is much more for numerous researchers in economics, sociology, political science, philosophy, decision making, biology and many other fields of knowledge, the question of if and how much AI is going to help affront the paradoxes of rationality is unavoidable. For us, that haven’t given up on exploring the effects of rationality (though its individual analytical effects and implications in its classical hypothetic-deductive form maybe on the board of exhaustion) the possible shift of optics from individually optimizing and adapting algorithms (deep learning on the individual algorithmic level) toward the creation of smart networks where different algorithms cooperate and coordinate their answers to systemic changes and disequilibria via a pool of common knowledge (yes …. “collective” common knowledge)44 is a logical and pressingly needed next step. Pushing the research agenda in the domain in this direction may be an ambitious and overcomplex endeavor, but the potential benefits are obvious, and they are far more reaching than the strict local advances of AI.

References Arrow, Kenneth J. 1951. Social Choice and Individual Values. New York: Wiley. Axelrod, Robert. 1984. The Evolution of Cooperation. New York: Basic Books. Axelrod, Robert. 1997. The Complexity of Cooperation: Agent-Based Models of Competition and Collaboration. Princeton, N.J.: Princeton University Press. Axelrod, Robert, and Michael D. Cohen. “Coping with Complexity: The Adaptive Value of Changing Utility.” The American Economic Review, Vol. 74, No. 1 (1974): 30-42.

44

Cf. Nisioti, “In Need of Evolution: Game Theory and AI”.

Scratch My Back & I Will Scratch Yours

89

Banerjee, Dipyaman, and Sandip Sen. “Reaching Pareto-optimality in Prisoner’s Dilemma Using Conditional Joint Action Learning.” Autonomous Agents and Multi-Agent Systems 15(1) (2007): 91–108. Binmore, Kenneth. 1994. Playing Fair: Game Theory and the Social Contract, Vol. 1. Cambridge, Mass.: MIT Press. Borel, Emile. “La théorie des jeux et les équations à noyau symétrique gauche.” In Comptes rendus de l’Académie des sciences, Vol. 173, 1304–1308. (Paris: Académie des Sciences Morales et Politiques). Boudon, Raymond. 1977. Effets pervers et ordre social. Paris: Presses Universitaire de France. Braithwaite, Radiyna B. 1955. Theory of Games as a Tool for the Moral Philosopher. Cambridge: Cambridge University Press. Bruni, Luigino, and Francesco Gualo. “Pareto’s Theory of Choice from the Cours to the Traittato. Utility, Idealization and Concrete Deductive Method.” In Pareto aujourd’hui, coll. Sociologies, edited by Bouvier Alban, 111-126. (Paris: P.U.F., 1999). Cyert, Richard M., and Morris DeGroot. “Bayesian Analysis and Duopoly Theory.” The Journal of Political Economy Vol. 78, No. 5 (1970): 11681184. Cyert, Richard M., and Morris DeGroot. “An Analysis of Cooperation and Learning in a Duopoly Context.” The American Economic Review, Vol. 63, No. 1 (1973): 24-37. Cyert, Richard M., and Morris DeGroot. “Rational Expectations and Bayesian Analysis.” The Journal of Political Economy, Vol. 82, No. 3 (1974): 521-536. Day, Richard H., and Theodore Groves, eds. 1975. Adaptive Economic Models. New York: Academic Press. Denhardt, Robert, and Philip Jeffers. “Social Learning and Economic Behavior: The Process of Economic Socialization.” The American Journal of Economics and Sociology Vol. 30, No 2 (1971): 113-125. Flood, Marvin. 1952. Some Experimental Games. Research Memorandum RM–789. Santa Monica, Cal.: Rand Corporation. Gauthier, David. “David Hume, Contractarian.” The Philosophical Review, Vol. 88, No. 1 (1979): 3-38. Gauthier, David. 1986. Morals by Agreement. Oxford: Clarendon.

90

Chapter V

Giannakis, Konstantinos, Georigia Theocharoplou, Christos Papalitsas, Sofia Fanarioti, and Theodore Andronikos. “Quantum Conditional Strategies and Automata for Prisoners’ Dilemmata under the EWL Scheme.” Appl. Sci. 9(13) (2019): 2635. https://doi.org/10.3390/app9132635. Goldberg, David. E. 1989. Genetic Algorithms in Search, Optimization, and Machine Learning. Reading, Mass.: Addison-Wesley Pub. Co. Harsanyi, John. “Bayesian Theory and Utilitarian Ethics.” The American Economic Review. Papers and Proceedings of the Ninetieth Meeting of the American Economic Association, Vol. 68, No. 2 (1978): 223-228. Hedoin, Cyril. “Linking Institutions to Economic Performance: The Role of Macro-Structures in Micro-Explanations.” Journal of Institutional Economics, 8(3) (2012): 327-349. Hedoin, Cyril. “A Framework for Community-based Salience: Common Knowledge, Common Understanding and Community Membership.” Economics and Philosophy, 30 (2014): 365-395. Holland, John H. 1975. Adaptation in Natural and Artificial Systems. Michigan: The University of Michigan Press. Holland, John H. “Adaptive Algorithms for Discovering and Using General Patterns in Growing Knowledge Bases.” International Journal of Policy Analysis and Information Systems, No. 4 (1980): 245-268. Holland, John H. 1992. “Genetic Algorithms.” Scientific American, 267, No. 7 (1992): 44-50. Hu, Junling, and Michael P. Wellman. “Nash Q-learning for General-sum Stochastic Games.” Journal of Machine Learning Research, Vol. 4 (2003): 1039–1069. Kavka, Gregory. 1986. Hobbesian Moral and Political Theory. Princeton: Princeton University Press. Lewis, David K. 1969 [2002]. Convention: A Philosophical Study. Oxford: Blackwell Publishing. Littman, Michael. “Markov Games as a Framework for Multi-agent Reinforcement Learning”. In Proceedings of the Eleventh International Conference on Machine Learning, 157–163. (San Francisco: Morgan Kaufmann, July 1994). Luce, Duncan R., and Howard Raiffa. 1957. Games and Decisions: Introduction and Critical Survey. New York: Dover Publications, Inc.

Scratch My Back & I Will Scratch Yours

91

March, James G. and Herbert Simon. 1958. Organizations. New York: Dover Publications, Inc. Merton, Robert K. “The Unanticipated Consequences of Purposive Social Action.” American Sociological Review, 1 (6) (1936). Nash, John F. “Equilibrium Point in N-person Games.” Proceedings of the National Academy of Science, No. 36 (1950): 48-49. Nisioti, Elena. “In Need of Evolution: Game Theory and AI”. Accessed June 1, 2021. https://www.freecodecamp.org/news/game-theory-and-aiwhere-it-all-started-and-where-it-should-all-stop-82f7bd53a3b4/ “MAY 12, 2018/#MACHINE LEARNING Nolan, Patrick D. “External Selection and Adaptive Change: Alternative Models of Sociocultural Evolution.” Sociological Theory, Vol. 2 (1984): 117-139. Nozik, Robert. 1974. Anarchy, State and Utopia. New York: Basic Books. Rapoport, Anatol. “Prisoner’s Dilemma-Recollections and Observations.” In Game Theory as a Theory of Conflict Resolution, edited by Anatol Rapoport, 17-34. (Dordrecht: D. Reidel Publishing Company, 1974). Rapoport, Anatol, and Melvin J. Guyer. “A Taxonomy of 2 X 2 Games.” General Systems, Vol. 11 (1966): 203-214. Rapoport, Anatol, Melvin J. Guyer, and David Gordon. 1976. The 2 X 2 Game. Ann Arbor: The University of Michigan Press. Rawls, John. “Justice as Fairness.” Philosophical Review, Vol. 67 (1958): 164-194. Rawls, John. 1971. A Theory of Justice. Cambridge, Mass.: Belknap Press of Harvard University Press. Riolo, Rick L. “The Amateur Scientist: Survival of the Fittest Bits.” Scientific American, Vol. 267 (1) (1992): 114-117. Runciman, Garry W., and Amartya K. Sen. “Games, Justice and the General Will.” Mind, New Series, Vol. 74, No. 296 (1965): 554-562. Schelling, Thomas. 1960. The Strategy of Conflict. Cambridge, MA: Harvard University Press. Schmidt, Christian. “From the ‘Standards of Behaving’ to the ‘Theory of Social Situations’. A Contribution of Game Theory to the Understanding of Institutions.” In Knowledge, Social Institutions, and the Division of Labor, edited by Pier Luigi Porta, Roberto Scazzieri and Andrew Skinner, 153-168. (Cheltenham: Edward Elgar Publishing, 2001).

92

Chapter V

Sen, Amartya, K. 2001. Ethique et Economie. Paris: Presses Universitaires de France. Smith, John M., and George R. Price. “The Logic of Animal Conflict.” Nature 246 (1973): 15–18. Stone, Peter and Manuela Veloso. “Multiagent Systems: A Survey from a Machine Learning Perspective.” Autonomous Robots, 8(3) (2000): 345– 383. https://doi.org/10.1023/A:1008942012299. Von Neuman, John. “Zur Theorie der Gesellschaftsspiele.” Matematische Annalen, 100 (1928): 295-320. Von Neuman, John. “On the Theory of Games of Strategy.” In Contributions to the Theory of Games, Vol. 4, edited by Albert Tucker and Duncan Luce, 13-42. (Princeton: Princeton University Press, 1959). Von Neuman, John, and Oscar Montgerstern. 1944. Theory of Games and Economic Behavior. Princeton: Princeton University Press.

CHAPTER VI ARTIFICIAL INTELLIGENCE IN DEFENCE TODOR DIMITROV Introduction Today, geopolitical competition is global and the race of technological adoption is extremely important. One of the main issues is the acceptance, integration and use of new technology in society. From Artificial Intelligence to quantum and everything in between, governments are in a race to leverage these technologies at scale and speed. The first adopter advantage for emerging disruptive technology could not be more prevalent in the world of geopolitics and deterrence. It is quite possible that the nations that win this race may be those with the most flexible bureaucracy rather than those with the best technology. Analyzing the processes of transformation in technology and taking into account the advantage of the military sphere to take priority of new developments first, it is necessary to change the psychology of military operations and issues of defense and security in general. The new dimensions of the combat space, the philosophy of intelligent warfare and the experience of military operations require the acquisition and use of a complex, broad spectrum of capabilities. The actuality of the theme comes from dynamics of the security environment and the increasingly widespread use of AI in the area of defense. The rapid development of intelligent machines in recent decades has shaped their path and their increasing use in the field of defense. The powerful economies allocate significant budgets to finance projects for the development of AI in the military field, and their goal is to establish

94

Chapter VI

leadership in this area. The competition is certainly not advertised, but to a large extent, an analogy can be made with the Cold War nuclear race. The topic of AI has been widely discussed in the last few decades, but the world has evolved, and nowadays there is a rapid technological development in all areas of our society. Some of them constantly involve new applications in the military and security affairs developing different systems, based on AI. It is quite important to pay attention to the issues of implementation of that new technology for defense capabilities because they could impact the entire future world. The emerging and disrupting technologies are expected to have a tremendous impact on the military affairs and not exclusively for creating killing robots. These technologies even do not have to be used only in lethal systems. According to the specialists the main issues for defense sector that AI technologies could improve are management, operational speed, precision, logistic, procurement, command and control, communications, intelligence, situational awareness, electronic warfare dominance, cybersecurity, sensors, integration and interoperability, different types of analysis and voice recognition algorithms, military medicine, etc. Also, it is critical that other factors such as ethics, legality, privacy and human control be considered. Before considering the different applications of Artificial Intelligence in defense, it is necessary to analyze the capabilities and differences between defense systems based on Narrow and General AI.

Narrow Artificial Intelligence in Defense Modern opinions say that the Narrow or called by Kenneth Payne Tactical AI, has relatively limited applications in a strictly profiled field.1 It can be concluded that the advantages of this type of AI in the defense, especially in the speed of decision-making, are likely to be decisive and will make a large part of the systems currently used unnecessary. Furthermore, the balance of power can shift dramatically depending on the pace of development of AI. Offensive strategy is likely to dominate, as the speed of 1

Kenneth Payne, Strategy Evolution and War–From Apes to Artificial Intelligence (Washington DC: Georgetown University Press, 2018).

Artificial Intelligence in Defence

95

decision-making will provide a huge advantage to the first strike. This is likely to lead to a reconsideration of some current concepts of automation in military operations. For example, in his study, Frank Barnaby claims that technology favors defensive action because remote-controlled weapon systems are more effective at destroying attack platforms.2 Additionally, if the defense itself is decentralized, it will be a difficult target for attack. But with the development of technology, the ability to recognize objects on modern artificial neural networks gives them a better chance to detect and recognize defense systems and then attack them with concentrated fire or other actions. It is likely that this development of technology will increase the prospects for success in the preventive preemptive strike and will change the current perception of the defensive dominance of nuclear weapons. The ability to concentrate force quickly and accurately revives the offensive logic mathematically based on the Lanchester square law. Control in this area will become increasingly difficult given the speed with which tactical AI decides how best to optimize its functions. AI raises concerns about the relative balance of power, especially with regard to offensive scenarios. The result could be that the dilemma of maintaining the status quo would fuel an AI arms race and increase pressure on the leading countries and companies. It is important to take this into account before discussing the broad speculation about the extent to which AI systems themselves form key areas of international relations - for example, through the need for collective solutions to major social changes in employment or through the impact on the boundary between the public and private spheres. Both raise significant controversy over the appropriate powers of states to monitor and control their citizens. And all this only if at this stage we are talking about narrow AI or tactical weapon systems with AI. 3 In order to better account for the nature of narrow AI, it is appropriate to make some generalizations about it:

2

Frank Barnaby, The Automated Battlefield: New Technology in Modern Warfare (Oxford: Oxford University Press, 1987). 3 Martin Ford, Rise of the Robots: Technology and the Threat of a Jobless Future (New York: Basic Books, 2019), 229-248.

96

Chapter VI

Ɣ At this stage, there are mainly theoretical dimensions of that type of AI. Intelligent machines are evolving rapidly, and even if broad parameters of this speed are visible, such as image recognition, in many other areas these parameters remain unclear and hazy; Ɣ When using narrow AI, the logic of combat effectiveness is likely to surpass the effects of traditional conflicts because the impact through strike force is likely to be significantly larger and more short-term. AI does not necessarily have immediate success in all areas of combat. For example, land conflicts are much more difficult to regulate and use AI than maneuvering in the space, air, or at sea. This is due to a more complex and diverse environment in terms of human presence and physical geography. Simultaneously, as long as war remains a human endeavor, the morale of the personnel, who have occasionally reversed the course of hostilities, has been greatly reduced, and less well-equipped forces have succeeded against nominally more powerful adversaries. Here, scenarios could be complicated if this type of weapon is used by malicious non-state groups. Nevertheless, the changes that will take place in the short and medium term are likely to be significant; Ɣ The application of AI in society changes a large number of existing institutions and practices, and the Armed Forces system does not stay away from this development. It is highly probable that AI will lead to even more significant changes in Defense than nuclear weapons. Autonomous, remote-controlled, nano- and other related technologies will significantly decrease the number of personnel in the battlefield and will require new skills from it. They will change the traditional organization of the Armed Forces into separate areas of land, sea, air and space because they will integrate platforms operating in all these areas. Alternative staff differences may arise - perhaps organized by combat security, such as intelligence, logistics, and others, or perhaps by technical functions - engineer, computer specialist, communications specialist, and others. They are likely to change existing perceptions of the military profession, the requirements for which will change significantly due to the increasing availability of automated and remote-controlled platforms.

Artificial Intelligence in Defence

97

Ɣ There are certainly speculations on which countries will produce the best narrow AI in Defense. The progress observed requires ongoing research and the allocation of large budgets, as well as other incentives to attract the best researchers, often foreign specialists. The usual structural incentives for intellectual creativity are applied - personal freedom, rule of law and guaranteed intellectual property rights. AI research is concentrated in countries that already have military power: United States, China, Russia, Great Britain, France and others. The United States are projected to invest more than $6 billion in AI-related research-and-development projects in 2021, while contract obligations are on pace to grow nearly 50 percent, to $3 billion, relative to 2020, according to the forecast.4 China is also betting on AI to strengthen its defense capabilities, and forecasts suggest that the country will become a world leader in this field by 2030. The analysis shows that the market share of AI in the defense industry is expected to reach $ 18.82 billion by 2025, with annual growth of 14.75% from 2020 to 2025.5 Numerous limitations slow down the spread of AI technologies at this stage. These are primarily hardware that could advance even faster with the development of specialized architectures, such as neuromorphic chips or quantum computing. Another limitation is the engineering expertise for training AI with working algorithms. Intelligent machines have a lack of motivation and for this training planned options are to develop an analogy of “genetic algorithms” that can be combined to create a more efficient result similar to human DNA.

4 Jon Harper, “Federal AI Spending to Top $6 Billion,” National Defense Magazine, October 2, 2021, https://www.nationaldefensemagazine.org/articles/2021/2/10/federal-ai-spendingto-top-$6-billion. 5 Sebastian-Gabriel Popescu, “Artificial Intelligence in Naval Operations” (paper presented at the International Scientific Conference Strategies XXI: Technologies — Military Applications, Simulations and Resources, NDU „Carol I”, Bucharest, 2019).https://docplayer.net/145260428-National-defense-university-commandand-staff-college-program-technologies-military-applications-simulation-andresources.html.

98

Chapter VI

A skeptical view of narrow AI shows that it can be seen as similar to earlier information processing technologies. This is a good way to reduce, but not eliminate, the available “fog of war”. The nature of war continues to be filled with a great deal of uncertainty. The development of combat narrow AI could further reduce friction on the battlefield without eliminating it altogether. Taking into account the main qualities of AI for independence and ability to improve performance through learning, it can be summarized that the information processing will be improved significantly. The main difference here is that AI has the ability to make technological decisions, not just easier decisions. Another skeptical view predicts that AI will never achieve the ability to deal in complex situations like continental land conflicts, where the military and civilians mix in a single space. In such environments, humans are certainly imperfect and unpredictable; also, the specific situation is highly dependent on technological progress. AI, used as a tactical weapon system, is able to transform the current psychological basis for making quick decisions when using violence. Intelligent machines will not be psychologically shocked by a surprise enemy shelling or rapid maneuvers. They will not experience fatigue or stress, as a ship's commander after a long watch or pressed by circumstances headquarters. They will neither assess risk based on prospect theory by calculating benefits and losses, nor make a subjective decision based on anger, failure, or perceived insult, nor any of the multiple cognitive principles people use in such a situation. But they will maneuver and concentrate much faster than human controlled systems. Despite skepticism and the security dilemma based on its unpredictability, the growing involvement of AI in the armed struggle given its essential advantages is inevitable. Regulation or banning of the development of AI combat systems will be extremely complicated because AI is a decision-making technology, not a specific weapon technology. Also, intelligent machines enter a wide range of activities outside the field of defense, which makes them extremely difficult to be limited with restrictive measures.

Artificial General Intelligence in the Field of Defense Artificial General Intelligence (AGI) is capable of having a revolutionary impact on military affairs. AGI has a more flexible and powerful intelligence

Artificial Intelligence in Defence

99

in each above aspect of narrow AI. This further expands the possibility for discrepancies between human designers and the behavior of the final product. This tremendous change will affect the balance of power between states by increasing the relative strength of the attack, challenging the ability of human commanders to exercise control after the start of hostilities and behaving in ways that are in mutual conflict, go beyond human boundaries and are difficult to predict. The problems could be exacerbated when it comes to escalating military conflict, deterrence, or violence in AI use. These aspects have so far been measured only by human psychology and, eventually, in the future, a novel discipline, “machine psychology,” will attempt a similar analysis of decision making, justified and performed by machines.6 Tactical AI, as we have seen, will have a significant impact on the strategy, even if it uses only the limited technologies available today. This type of narrow AI is much more “basic” than the scheme outlined before the AGI. It lacks the degree of cognitive flexibility to easily switch between tasks and adapt quickly to new circumstances. It can be combined with another domain-specific AI to create complex knowledge, as in the intelligent poker machine or similar to the group “swarm intelligence”. In contrast to this model, the AGI would be an integrated package capable of managing the individual discrete tasks of composite AI and many other elements. In its military form, this type of AI could integrate the activities of multiple nets into a distributed, swarm-like system. It could concentrate power at all levels of intensity. Of course, this AGI will retain the basic qualities of narrow AI, such as pattern recognition, memory, and speed, which are easily transformed into military power. Additionally, it is assumed that the overall AI will be able to generate distinctive motivation that is not provided by its designers. Combined with speed and the ability to escalate your efforts sharply, this makes it dangerous and unpredictable. The AGI will have various attributes: its nets will maneuver at speed, and it will concentrate its efforts quickly, outrunning any adversary known to date in military science. Additionally, it will be able to coordinate efforts across

6

Payne, Strategy Evolution and War–From Apes to Artificial Intelligence, 193.

100

Chapter VI

its full range of capabilities, the main impact of which would be the management of existing parameters for escalation, violence and deterrence. When considering the military aspects of AI, much attention is paid to the synergy between humans and machine. With the increasing speed of processes, people will not have the ability to stay in the cycle due to the lack of speed. In such cases, it is good that they can at least remain in the management chain and be able to intervene and stop actions that are already under way, especially escalation actions. The AGI does not need to be aware of what it is doing in a military conflict. He qualifies as a flexible and adaptive intelligence, able to navigate in a complex environment and achieve its goals. These qualities are especially useful when it comes to combat. The integration of different types of weapon systems with AGI will allow the machine to coordinate not only at the tactical level, but also at the operational and possibly strategic level. While tactical AI decisions may have operational and strategic effects in certain situations, combat AGI will deliberately modulate its behavior to prevail at higher levels of combat operations.

Aspects and Trends for the Use of AI in the Defense Field Researchers consider two macro-revolutions in the development of defense strategy. The first is related to evolutionary development, which gives humans a different type of social knowledge. One of the key features of prehistoric existence was the need for close cooperation in single-family groups for surviving. This environment probably led to the development of language. The abstract mathematical connection presented by Frederick Lanchester suggests that larger groups win proportionately in battle, where it is possible to concentrate power when all other elements are equal. Dominic Johnson and Niall McKay use Lanchester's law to view and justify human evolution as a product of conflict.7

7

Dominic Johnson and Niall MacKay, “Fight the Power: Lanchester’s Laws of Combat in Human Evolution,” Journal Evolution and Human Behavior, Vol.36, No. 2 (March 2015): 152-163.

Artificial Intelligence in Defence

101

The second macro-revolution in strategy is currently underway and happening as a result of the accelerated development of information processing technologies. It challenges the biological basis of decisionmaking. In human history, many other technologies and numerous weapon systems have been described as revolutionary. They dramatically changed the nature of the war, but these innovations and weapons require careful thinking in every context about how better to use them in a strategy. They certainly complicate the extraction of useful and lasting strategic concepts throughout history. But at the most fundamental level, such an abstraction is possible precisely because strategy is primarily a psychological activity, and human psychology is evolving very slowly. Given this, Clausewitz's triumph is relatively long in time. Later, when game theory came into play, Thomas Schelling's analysis recalled that human psychology was not limited to modeling rational choice theory. Emotion and heuristics are products of our psychology, and while misperceptions and calculations can be minimized, it is difficult to avoid situations such as the Cuban Missile Crisis, which could drag humanity into a crisis of devastating proportions. In the course of human evolution, a technology has already changed to some extent the developed psychological basis of strategy, and that is writing. According to current forecasts, AI will lead to an even more radical transformation. The connection between the two is that both technologies are changing our developed psychological ways of making decisions. The development of writing, independently several times in human history, provides a way to extract knowledge and expand information in time and space, thus facilitating specialization. Writing allows for reflection on strategy and catalyzes technological innovation. Information technology was relatively limited until very recently in human history. As they accelerated the development of computers, information technology created an expectation for a radical change in strategy. This, however, was premature. The information revolution in military affairs has changed the parameters of the strategy in multiple respects. It strengthened situational awareness and allowed greater speed, accuracy and concentration of impact. The countries that have developed it have significantly increased their fighting power. An example in this regard is the United States in Operation Desert Storm in 1990-91 against the Iraqi army. But it is also important to

102

Chapter VI

take into account the fact that there are limitations, both in terms of available technology and at the conceptual level, where the strategy, at this stage, remains undoubtedly psychological. All the world's smart munitions and network sensors cannot determine how effective an operation in Afghanistan against insurgents and the radical Taliban would be. At this stage in the development of military affairs, unequivocally, every automated technique needed a man for selecting the target or carry out some other decisive intervention. On the one hand, the development of AI can be seen as a logical continuation of this ongoing revolution in information processing. In this way, AI could be seen as another tool for processing information about people. Claims for military revolutions should be analyzed with skepticism because often the conditions set for their introduction are too insignificant. Examples in this regard are concepts that have come and gone at high speed in recent years, such as hybrid warfare, the indirect approach, the comprehensive approach, the fourth-generation war, effect-based operations, and much more. There is a certain possibility that the revolution in strategy, as a consequence of AI, will pass quickly and unnoticed. But it is important to stress that AI is more than just another means of processing information. The friction of war and luck will remain key elements of the strategy, but AI has enormous potential to bring about a radical change in strategic issues. By making strategic decisions on behalf of people, and even more so by performing them on their own, based on calculations that are not within human limits, intelligent machines can have far-reaching consequences. This could lead to the transformation of the societies that rule them, to a change in the relative power between them and to the evolution of the overall nature of future conflicts. Although the current technologies have made great changes, AI is much more radical because it changes the very basis of strategic decisions. Machines do not use the cognitive heuristics that inform human decisions, much less the conscious reflection we use to imagine the future or to predict what the adversary may aim for. In such a situation, when the intelligent machine wants to act at all costs to achieve the goals set by man, it is very likely that part of the communication will be “lost in translation”. Additionally, in the near future, AI actions will remain largely at the tactical level, but the development of technology and competitive relationships will

Artificial Intelligence in Defence

103

lead to the introduction of machine intelligence at ever higher levels of decision-making. AI itself is not a weapon technology, but above all a decision-making process. The course of evolution will inevitably bring the influence of AI to a strategic level. The strategy has evolved over the years thanks to human evolution, but in the future, the main factor for it will probably be increasingly machines and decresingly people. Machine learning, artificial neural networks, biosynthetic hybrid elements combining computers and living brain tissue are leading to more and more scenarios for the future development of AI. This trend in technology promises to blur the line between human and computer knowledge. Even more dramatic changes are possible. Such as “mind fusion”, where thought processes from one biological brain are transmitted to another, as this has already been demonstrated in rats.8 These synthetic approaches to modified human intelligence can be combined with others, such as “optogenetics”, where light elicits reactions in genetically engineered neurons. Biotechnology, including CRISPR (clustered regularly interspaced short palindromic repeats) gene editing9, raises additional opportunities to change evolutionary knowledge, arguing that it will be able to shape the ability to pay attention or recall. The goal of the individual human mind to act in accordance with external intervention is already obvious, and the way in which this can be realized has not yet been specified. The technologies described above are extremely suitable for creating a military force executing hierarchical commands, but they ask ethical questions. They also challenge the essence of what it means to be a human and even what it means to be an individual.

Applications and Perspectives in the Use of Artificial Intelligence in Defense There is a trend in the last few decades to incorporate more robotics and autonomous systems into military forces world-wide. Artificial intelligence and machine learning will allow these systems to tackle more challenging

8

Miguel Pais-Vieira et al., „A Brain-to-Brain Interface for Real-Time Sharing of Sensorimotor Information,” Sci Rep 3, 1319 (February 2013). 9 Michael Le Page, “What is CRISPR? A Technology that Can Be Used to Edit Genes,” New Scientist, November 12, 2020.

104

Chapter VI

tasks in a wider range of environments. Because of the comprehensive nature of AI technology, non-state groups and individuals will also be able to harness and use this technology. In combat operations, robots, swarms, and autonomous systems have the potential to increase the pace of combat. This is particularly the case for domains of machine-to machine interaction, such as in cyberspace or the electromagnetic spectrum. AI could be used not only to create more intelligent robotics, but also to power more advanced sensors, communications, and other key enablers. Following the development of AI in recent decades, it can be concluded that its superiority over human intelligence in the field of a number of logic games, such as chess, Asian Go and others, is obvious. Success in this type of task, which is very similar to the structure of the battlefield, shows how successful AI can be in such conditions. In games, the machine dominates because people think much slower and make mistakes. Even the best of us are not as consistent as intelligent machines. The collaboration between human minds and AI can increase the potential of a combat and other military systems to perform tasks more successfully. AI continues to develop, presenting and increasing its applications in a growing number of diverse defense systems. One of the key factors for success in this direction will be the reassessment of the goals of the algorithms for replacing human analysts and their activities. At this stage, it is still obvious the necessity to have human intervention in the cycle in order to realize the continuous evaluation and to guarantee the quality of the output of the advanced models and algorithms. But according to experts, it is not far away the moment when AI-led systems with some operational autonomy will take part in hostilities. Probably maritime environment and some other areas like space are particularly favorable for the initial deployment of this type of system, as it identifies targets relatively easily and without risk, as well as the relatively small presence of civilians. AI is defined as a critical component of combat operations in modern theory of combat operations. Compared to conventional systems, military platforms and systems equipped with AI are capable to manage a significantly larger volume of data, which leads to increased efficiency and speed of decision-making. Research and development in this area are

Artificial Intelligence in Defence

105

growing in line with increased funding. This, in turn, leads to easier and broader implementation of AI in the defense sector. This type of intelligent machine is already in service in a number of hightech military services around the world. They and the resulting capabilities can build and maintain combat space with software applications in different types of operations, as well as build networks of interconnected military platforms and systems. The contribution of AI to military operations is present and will become increasingly prevalent in all five military environments: land, air, sea, space and cyberspace. In the medium term, the commands of military operations and traditional defense systems are expected to be replaced by intelligent systems that can be deployed online and updated through communication links during the mission. In analyzing global trends, available AI-based investments and programs several main applications in the defense sector could be distinguished, where this type of intelligent systems tends to prove its importance in the medium term:

Battle Platforms Armed forces from around the world are increasingly using AI in weapons and other systems based on land, air, sea and space platforms. They perform a variety of tasks: surveillance and reconnaissance, barrier patrol for defense and maintenance of space/base, protection of forces, countering the mine threat, anti-submarine warfare, reconnaissance, hydrographic surveys, communication, information and navigation services, cargo delivery, information operations, performing high-precision synchronized strikes and others. Remote underwater vehicles have been widely used in recent decades to solve a wide range of tasks: surveillance and reconnaissance, mine search, hull inspection, data collection on hydrometeorological conditions and more. The tendency is for them to become more multifunctional and with greater autonomy. For example, the US Department of Defense's Defense

106

Chapter VI

Research Agency (DARPA)10 is funding the development of a robotic underwater system that is expected to be used to solve a wide range of tasks, ranging from detecting underwater mines to conducting underwater operations.11

Threat Monitoring and Situational Awareness Situational awareness and combat space monitoring in operations depend to a large extent on Intelligence, Surveillance and Reconnaissance (ISR) systems. These systems are based on the principle of comparisons of embedded algorithms and are used mainly for the acquisition and processing of information for the maintenance of a number of combat activities and decision-making. Unmanned platforms used for surveillance and reconnaissance are increasingly based on AI. At this stage, they are controlled remotely or sent on a predetermined route. Equipping these systems with analytical machines assists defense personnel in monitoring threats and improves situational awareness. Unmanned aerial and land vehicles, surface platforms and submarines with integrated AI can patrol large areas, identify potential threats and automatically transmit information about these threats to the response forces. The use of remote systems increases the security of the bases and guarded objects, as well as the safety and efficiency of the personnel in a combat situation or during a transition. Small robotic sensors could be used to collect information, and AI-enabled sensors and processing could help make better sense of that information. Deep neural networks are already being used for image classification for drone video feeds as part of the US Department of Defense’s Project Maven, in order to help human operators to process the large volumes of data being collected. While current AI methods lack the ability to translate this into an understanding of the broader context, AI systems could be used to fuse data from multiple intelligence sources and cue humans to items of interest. AI 10 DARPA is a government agency of the United States Department of Defense that deals with the development of new technologies in the US military 11 “Selects Performers to Advance Unmanned Underwater Vehicle Project,” DARPA, accessed February 05, 2021, https://www.darpa.mil/news-events/2021-02-05a.

Artificial Intelligence in Defence

107

systems can also be used to generate tailored spoofing attacks to counter such sensors and processors.12

Data Processing AI is particularly useful for quickly and efficiently processing large databases in retrieving valuable information. Intelligent machines can assist in the collection and analysis of information from different data sets, as well as in the retrieval and processing of information from various sources. This advanced analysis enables military personnel to recognize patterns and derive correlations, significantly reducing the timing of individual stages. For collection, the explosion of data that is occurring because of smart devices, the Internet of Things, and human internet activity is a tremendous source of potential information. This information would be impossible for humans to manually process and understand, but AI tools can help analyze connections between data, flag suspicious activity, spot trends, fuse disparate elements of data, map networks, and predict future behavior.

Force Management Starting with combat information and control systems, the defense programs of a number of countries expand their scope in the planning and implementing of AI in the command and control systems. These systems are used by platforms on land, sea, air, space, as well as in the cyber domain. The use of these types of systems leads to improved interaction between the individual components and the functioning of the military system as a whole. Simultaneously, they require significantly more limited maintenance and minimal human intervention. AI will facilitate the management and configuration of autonomous and high-speed platforms and weapons in the implementation of joint attacks and automatic control. A typical example in this regard is the use of swarm tactics, in which a large number of platforms

12 Cheryl Pellerin, “Project Maven to Deploy Computer Algorithms to War Zone by Year’s End,” Defense Government, accessed July 21, 2017, https://www.defense.gov/Explore/News/Article/Article/1254719/project-maven-todeploy-computer-algorithms-to-war-zone-by-years-end/.

108

Chapter VI

at relatively low cost, can attack an important and well-guarded enemy target. As the pace of battle accelerates and the volume and speed of information eclipse the ability of human warfighters, AI will become increasingly important for command and control. Autonomous systems that have been delegated authority for certain actions can react at machine speed at the battlefield’s edge without waiting for human approval. AI can also help commanders process information faster, allowing them to better understand a rapidly changing battlespace. Through automation, commanders can then relay their orders to their forces – human or machine – faster and more precisely. AI systems can also aid the military in a range of non-combat support functions. One use of AI will be to help defense leaders better understand their own forces. By analyzing large amounts of data, AI systems may be able to predict stress on the force in various components: when equipment requires maintenance; when programs are likely to face cost overruns or schedule delays; and when service members are likely to suffer degraded performance or physical or psychological injuries. Overall, AI has tremendous potential to help defense leaders improve the readiness of their own forces by assembling and fusing data and performing predictive analysis so that problems can be addressed before they become critical. AI is also is ripe for transforming traditional business processes within military and other government organizations.13

Logistics AI systems could play a crucial role in military logistics. The efficient supply and transportation of goods, ammunition, weapons and personnel is an essential factor for success in modern operations. The purpose of providing what is needed, where and when you need it, can be accomplished with the help of intelligent machines that synchronize the request, the

13 Singh Manvendra, “Is Predictive Analytics the Future? How AI Will Take It to the Next Level”, accessed September 14, 2020, https://levelup.gitconnected.com/is-predictive-analytics-the-future-how-ai-willtake-it-to-the-next-level-4ee90a240f9b.

Artificial Intelligence in Defence

109

database and the chain provider with an automated design of systems for loading and sending items. Probabilistic modeling can reduce forecasting errors by 20 to 50 percent. There are too many variables in a combat operation with which human operators can predict real-time levels to some extent. AI can solve these inaccuracies by making combat systems many times more effective.14

Targeting Some AI systems are developed with the task of increasing the accuracy in defining and prioritizing targets or target systems, as well as to determine the adequate impact on them in a complex combat environment. They allow the Defence Forces to gain an in-depth understanding of potential areas of work by analyzing reports, documents, information channels and other forms of unstructured information. In addition, the presence of AI in target recognition systems improves the ability of these systems to identify the position of their objects. The capabilities of activated AI recognition systems include predictions of enemy behavior based on probabilities, aggregation of weather and environmental conditions, anticipation and signaling of potential supply line difficulties or vulnerabilities, assessments of mission approaches, and proposing actions to reduce the risk. The machine learning of these types of systems is also used to detect, track and investigate targets from the data received from the sensors. For example, DARPA's Target Recognition and Adaptation in Contested Environments (TRACE) program uses machine learning techniques to

14

Harald Bauer and Peter Breuer, “Smartening up with Artificial Intelligence (AI) — What’s in It for Germany and Its Industrial Sector?”, accessed April 01, 2017, https://www.mckinsey.com/~/media/mckinsey/industries/ semiconductors/our%20insights/smartening%20up%20with%20artificial%20intelli gence/smartening-up-with-artificial-intelligence.ashx.

110

Chapter VI

automatically locate and identify targets and images using SyntheticAperture Radar - SAR).15

Combat Simulations and Training Simulation and training are a multidisciplinary field that connects systems engineering, software engineering and computer science to build computerized models to train personnel with the multilateral combat and support systems used during various types of operations. Investments in this type of application have increased significantly in recent years. AI can actively participate in such systems to analyze procedures and act in accordance with the identified adversary. In this way, the personnel are trained in conditions close to combat without endangering their lives and health. These types of systems allow you to easily redefine the tasks and through the use of AI to build a more difficult combat environment, with the ultimate goal to improve training without endangering people. Evolutionary and reinforcement learning methods could be used to generate new tactics in simulated environments, finding surprising solutions as they have in other settings.

Electronic Warfare AI systems could be used for electromagnetic spectrum dominance. They have ability to generate novel methods of jamming and communications through self-play, like AlphaGo Zero improving its game by playing itself. For example, one AI system could try to send signals through a contested electromagnetic environment while another system attempts to jam the signal. Through these adversarial approaches, both systems could learn and improve. In 2014 an US government agency DARPA held a Spectrum Challenge with human players competing to send radio signals in a contested environment. DARPA is now using machine learning to aid in

15

Ke Wang and Gong Zhang, “SAR Target Recognition via Meta-Learning and Amortized Variational Inference”, accessed October 21, 2020, https://www.mdpi.com/1424-8220/20/20/5966/pdf.

Artificial Intelligence in Defence

111

radio spectrum allocation, but this concept could also be applied to jamming and creating jam-resistant signals.16

Decoys and Camouflages Generative adversarial networks could be used to create militarily relevant deep fakes for camouflage and decoys, and small robotic systems could be used as expendable decoys. As the military incorporate more AI-enabled sensors for data classification, spoofing attacks against such systems will be increasingly relevant as well.

Cybersecurity Defense systems are often vulnerable to cyberattacks, which can lead to loss of classified information and damage to combat systems. This is why significant efforts are being made to systems equipped with AI to autonomously protect networks, computers, programs and data from any kind of unauthorized access. In addition, supported AI systems for web security can record the pattern of cyberattacks and develop counterattack tools to deal with them. The applications of AI in military operations are diverse and their number will increase with the increase of investments in the field and the practical imposition of this type of system in the defense sector. There is an ongoing debate about the ethical aspect of the introduction of intelligent systems in the armed struggle. The topic raises a variety of questions, such as: What will happen if an adversary or terrorist organization takes control of our remote platforms? Or if we do not produce these types of systems ourselves, but buy them and the manufacturer has a hidden ability to control them? We will find the answers to these questions in the future, but at this stage the dynamic development of AI sector and the new horizons in front of it are visible from a broad perspective.

16

DARPA, “The Radio Frequency Spectrum + Machine Learning=A New Wave in Radio Technology,” accessed August 11, 2017, https://www.darpa.mil/newsevents/2017-08-11a.

112

Chapter VI

Conclusion In conclusion, the following key remarks can be made: 1. The applications of AI accelerate decision-making processes, minimize errors, rapidly concentrate force and precision, could find their place in more and more defense sectors, leading to the evolution of the overall nature of future conflicts. 2. Human control in the military sphere will become increasingly difficult given the speed with which intelligent machines optimize their functions. 3. There is concern about the possibility of a rapid change in the relative balance of power, especially with regard to offensive scenarios. Combined with the likelihood that the lethal outcome will be delegated entirely to AI or that such technology falls into unscrupulous hands, it raises ethical objections to the uncontrolled development of intelligent machines. 4. Maintaining the status quo fuels a race in AI armaments and increases the pressure on leading nations. Overall, artificial intelligence can help the military improve understanding, predict behavior, develop novel solutions to problems, and execute tasks. Some applications, such as the use of AI to enable autonomous weapons, raise difficult legal, ethical, operational, and strategic questions. The potential for automation to increase the pace of combat operations to the point where humans have less control over the conduct of war raises profound questions about humanity’s relationship with war, and even the nature of war itself. The importance of AI is due not so much to the technology itself as to the processes and transformation of a number of aspects of defense. In this context, the three main challenges arising from AI's entry are: how AI can help achieve security and defense goals; how AI technologies can be protected from attacks; how to ensure protection against malicious use. The latter inevitably raises a number of serious issues of international law and ethics. The importance of the informal discussions on emerging technologies in the field of deadly autonomous weapons systems.

Artificial Intelligence in Defence

113

Expanding the role of AI can increase the benefits of the military system. Not only does the integration of greater AI autonomy reduce casualty rates of personnel, but such systems can adopt riskier tactics; target with greater accuracy; and operate with greater endurance, range, and speed while retaining a greater level of flexibility and mobility. There is a need for regulation of the legal and ethical aspects that AI in the defense sector will inevitably pose in the near future. In the process of developing international humanitarian law, it is essential that states ensure human control in decision making for the use of autonomous lethal weapons systems. Our new tools can make us smarter and can enable us to better understand the military battlefield, our world and ourselves. Deep Blue didn’t understand chess, or even know it was playing chess, but it played it very well. We may not comprehend all the rules our machines invent, but we will benefit from them nonetheless. Synergy between the human mind and AI has tremendous potential to increase our defense capabilities and at this moment, everything depends on us. The thought from the President of the Future of Life Institute Max Tegmark, is exactly in that direction: “Everything we love about civilization is a product of intelligence, so amplifying our human intelligence with artificial intelligence has the potential of helping civilization flourish like never before–as long as we manage to keep the technology beneficial.”17

References Barnaby, Frank. 1987. The Automated Battlefield: New Technology in Modern Warfare. Oxford: Oxford University Press. Bauer, Harald, and Peter Breuer. “Smartening up with Artificial Intelligence (AI)—What’s in It for Germany and Its Industrial Sector?”. Accessed April 01, 2017.

17

“Benefits and Risks of Artificial Intelligence”, Future of Life Institute, accessed February 12, 2022, https://futureoflife.org/background/benefits-risks-of-artificialintelligence/.

114

Chapter VI

https://www.mckinsey.com/~/media/mckinsey/industries/semiconducto rs/our%20insights/smartening%20up%20with%20artificial%20intellig ence/smartening-up-with-artificial-intelligence.ashx. DARPA. “The Radio Frequency Spectrum + Machine Learning=A New Wave in Radio Technology”. Accessed August 11, 2017. https://www.darpa.mil/news-events/2017-08-11a. DARPA. “Selects Performers to Advance Unmanned Underwater Vehicle Project”. Accessed February 05, 2021. https://www.darpa.mil/newsevents/2021-02-05a Ford, Martin. 2019. Rise of the Robots: Technology and the Threat of a Jobless Future. New York: Basic Books. Future of Life Institute. “Benefits and Risks of Artificial Intelligence”. Accessed February 12, 2022. https://futureoflife.org/background/benefits-risks-of-artificialintelligence/. Harper, Jon. “Federal AI Spending to Top $6 Billion.” National Defense Magazine, October 2, 2021. https://www.nationaldefensemagazine.org/articles/2021/2/10/federalai-spending-to-top-$6-billion. Johnson, Dominic, and Niall MacKay. “Fight the Power: Lanchester’s Laws of Combat in Human Evolution.” Journal Evolution and Human Behavior, Vol.36, No. 2 (2015): 152-163. Le Page, Michael. “What is CRISPR? A Technology that Can Be Used to Edit Genes.” New Scientist, November 12, 2020. https://www.newscientist. com/term/what-is-crispr/. Pais-Vieira, Miguel, Mikhail Lebedev, Carolina Kunicki, Jing Wang, and Miguiel A. L. Nicolelis. „A Brain-to-Brain Interface for Real-Time Sharing of Sensorimotor Information.” Sci Rep 3, 1319 (2013). https://doi.org/10.1038/srep01319. Payne, Kenneth. 2018. Strategy Evolution and War–From Apes to Artificial Intelligence. Washington: Georgetown University Press. Pellerin, Cheryl. “Project Maven to Deploy Computer Algorithms to War Zone by Year’s End”. Accessed July 21, 2017. https://www.defense.gov/Explore/News/Article/Article/1254719/proje ct-maven-to-deploy-computer-algorithms-to-war-zone-by-years-end/.

Artificial Intelligence in Defence

115

Popescu, Sebastian-Gabriel.”Artificial Intelligence in Naval Operations.” Paper presented at the International Scientific Conference Strategies XXI: Technologies—Military Applications, Simulations and Resources, NDU „Carol I”, Bucharest, 2019. https://docplayer.net/145260428-National-defense-universitycommand-and-staff-college-program-technologies-militaryapplications-simulation-and-resources.html. Singh, Manvendra. “Is Predictive Analytics the Future? How AI Will Take It to the Next Level”. Accessed September 14, 2020. https://levelup.gitconnected.com/is-predictive-analytics-the-futurehow-ai-will-take-it-to-the-next-level-4ee90a240f9b Wang, Ke, and Gong Zhang. “SAR Target Recognition via Meta-Learning and Amortized Variational Inference”. Accessed October 21, 2020. https://www.mdpi.com/1424-8220/20/20/5966/pdf.

CHAPTER VII PROCESS MINING WITH MACHINE LEARNING NIKOLA SOTIROV Introduction The problem of predicting processes has been arising in frequency in recent times due to many companies attempting to digitally transform and describe the process traces within their systems optimally. The purpose of this project is to tackle the problem by creation and comparison of different machine learning techniques.1 As event logging has been part of the majority of the industry sectors for many years, enough data can be collected and used to create and test mathematical models, which will be trained to perform pattern recognition, event and time of event predictors.2

Methodology The project was developed and implemented by a team of five researchers. The workflow of the project was segregated into the following subdivisions: 1. Exploratory data analysis on the dataset and generation of visualizations of the general process flow in the context of the system 2. Event and time prediction strategy layout 3. Feature extraction for each model from the dataset 1

“Your First Steps into the World of Process Mining”, Software AG, accessed May 25, 2021, https://www.softwareag.com/en_corporate/resources/asset/ebook/business-processtransformation/first-steps-processmining.html?utm_source=bing&utm_medium= cpc&utm_campaign=bt_aris_process-mining&utm_region=hq&utm_subcampaign =stg1&utm_content=ebook_your-first-steps-into-process-mining_ stage1&msclkid=1a8e9abfd556137ca255f8c70261ec49. 2 Software AG, “Your First Steps into the World of Process Mining”.

Process Mining with Machine Learning

117

4. Training and testing the models 5. Plotting the results of each model and comparing them The dataset used in the project is the process traces collection from 2012 in the BPI challenge. The columns extracted and used from the dataset are the following: case (a number, which indicates which trace the event belongs to), event (the name of the process), startTime (the timestamp on which the process occurred), completeTime (the timestamp of completion of the event).3 lifecycle: transition

Describes the event state transition, based on that point of time in its lifecycle.

concept: name

The name of the event.

time: timestamp

The time at which the event occurred.

Case: AMOUNT REQ

The amount of money that was requested at the particular event. The amount of money that was requested at the particular event.

Figure 1: Data dictionary for the dataset BPI Challenge 2012 The dataset has the following columns in it: lifecycle: transition, concept: name, time: timestamp, case: REG DATE, case: concept: name, case: AMOUNT REQ. Each step will be explained in detail in this section. The emphasis of the exploratory analysis was to visualize and understand the overall process flow of the dataset and how much a process matters (via looking at the transitions from process to process).

3

“Nikola Sotirov. Github Project Repository”, accessed June 24, 2021, https://github.com/ NickSot/process%5C_mining.

Chapter VII

118

Preliminary Analysis and Visualization of the Data Set

Figure 2: States and transitions 4 Figure 2 denotes the types of processes that occur in the system, and the respective transitions from and to each other. The more highlighted the transitions are in Figure 2, the larger their numbers of occurrences are. The goal is to create and use algorithms that learn the patterns of transitions between the processes and forecast where the next transition will lead towards and at what timestamp that particular transition will occur in the future with reasonably small errors. In Figure 3 it is showcased what the frequency of each event type present in the dataset is. This gives a notion of how skewed the dataset is towards certain data points. In this case, the dataset is imbalanced, since it contains some events much more than others. Fortunately, this does not introduce problems in terms of the prediction, since the event types are static, and follow the same flow patterns.

4

“Nikola Sotirov. Github Project Repository”.

Process Mining with Machine Learning

119

Figure 3: Event occurrence frequencies 5

Event and Time Prediction Models The project was segregated into two main parts-implementation of predictors for the next event/process prediction, and prediction of the time until the next process occurs. As to the event prediction, two models were used and evaluated a random forest classifier, and a neural network classifier. For each classifier, the confusion matrices were plotted and compared. Accuracy has been the used as the primary metric for both models. See section “Results”. The time until the next event prediction was implemented by using a neural network with a singular regression output neuron. R2 score and relative absolute errors were used to evaluate the performance. See section “Results”.

5

“Nikola Sotirov. Github Project Repository”

120

Chapter VII

Random Forest Classifier A random forest is a set of decision trees, where all trees are run on random samples of the data. In the end, the best viable solution is selected by voting.6

Figure 4: Random forest example 7 In this project, such a model is utilized to classify what the next event is based on the previous event as input. Figure 4 showcases an example random forest and its generalized workflow.8

A Neural Network for Next Process Prediction A neural network is a type of data structure that consists of layers of nodes and connections between those layers, where every node from one layer is connected to all nodes in the next layer. The connections between the nodes/neurons are altered mathematically with the purpose of reaching the result values in the output nodes/neurons with minimized error with respect to an expected value.9

6 “Workflow of the Random Forest Classifier”, accessed May 18, 2022, https://www.researchgate.net/figure/Workflow-random-forest-classifier_ fig3_343627992. 7 “Workflow of the Random Forest Classifier” 8 Workflow of the Random Forest Classifier”. 9 “Artificial Neural Networks”, accessed June 4, 2022, https://iq.opengenus.org/ artificial-neural-networks/.

Process Mining with Machine Learning

121

Figure 5 represents a neural network with four hidden layers, one input and one output layer.

Figure 5: Neural network example 10 A neural network was used to predict the next event, given an arbitrary state of the system. The network has four layers, each with the following sequence of neuron numbers in each layer: (128, 64, 48, 24) and respective activation functions: (’relu’, ’relu’, ’relu’, ’softmax’). The optimizer for adjusting the learning rate used is ADAM (Adaptive Moment Estimation). The loss function used is Binary Cross-entropy.

A Neural Network for Time until Next Event Prediction For the time until the next event prediction, a neural network with a continuous output, which indicates the time until the next event occurs in seconds, was used. This model contains four hidden layers with the following sizes and activation functions: (128, 48, 48), (’relu’, ’relu’, ’relu’). The output layer contains one neuron in it with linear activation since the value has to be continuous (a timestamp, represented as a float). The loss function is the mean squared error of the expected and the actual output of the network.

10

“Artificial Neural Networks”.

Chapter VII

122

Data Set Preprocessing and Feature Extraction for the Models Each model requires a set of attributes from the dataset in order to predict the time of the next event or the next event itself. In this section it will be described how the input data for each model is prepared.

Preprocessing for the Random Forest Model An essential aspect to take into consideration when inputting categorical independent variables in any model is to convert those inputs to one-hot encoded vectors. This ensures that the model does not get confused as to the statistical nature of the input. For example, the mean between Saturday and Monday is not Thursday, which would have been the case if Monday was labeled as ’1’, and Saturday as ’6’, assuming that the result is rounded up.

Preprocessing for the Neural Network Classifier of the Next Event The preprocessing required feature engineering in order to encapsulate the correlations between the features and the dependent variable - the next event type. The following features were used as inputs for the neural network: concept: name, previous event, lifecycle: transition, pp event, ppp event, p lifecycle: transition, where previous event, pp event, ppp event, p lifecycle: transition are the engineered features from the dataset. previous event

The event that occurred before the current event.

pp event

The event type of the process that occurred before the previous event.

ppp event

The event type of the process that occurred before the pp event.

p lifecycle: transition

The state in which the previous event was at that point of time in its life cycle.

Figure 6: Engineered feature dictionary for the neural network classifier

Process Mining with Machine Learning

123

The input data are the data frame of those columns, converted to a matrix (or tensor), which is then input to the model. The output layer [See Section “Neural network for next process prediction”] contains 24 neurons, and the activation is softmax, meaning that each neuron has a value between 0 and 1. The neuron where the value is the largest indicates which position in the one-hot encoding of the event type contains a ’1’. Therefore, this is chosen as the predicted event type of the next process.

Preprocessing for the Neural Network Regressor for the Time until the Next Event Previous-event

The event that occurred before the current event.

pp event

The event type of the process that occurred before the previous event.

weekday

The day of the week in which the process executes

p lifecycle: transition

The state in which the previous event was at that point of time in its life cycle.

Figure 7: Engineered input features for the The data used as inputs for the time of event prediction model consist of the following columns in the data set: concept: name, previous event, lifecycle: transition, pp event, p lifecycle: transition, weekday. The subset of these features that contains only the engineered ones is previous event, pp event, p lifecycle: transition, weekday. Analogously, as in the case of the model in Section “Preprocessing for the neural network classifier of the next event”, the input data frame is converted to a matrix tensor that is then processed by the network. The last layer’s neuron contains the timestamp value of the result. That is then compared to the expected result, and the mean squared error is calculated in order to minimize the error.11

11

See the Section “Results”.

124

Chapter VII

Training and Testing of the Models The data had to be segregated into training and testing data, where 70 percent of it is training the models, and 30 percent is for testing the models’ performances. Since the data is temporal, the following approach was used in order to divide the dataset without using future data to predict the past. The dataset is ordered by the first event’s timestamp in every trace in an ascending manner. Then a timestamp threshold is chosen to divide the dataset into 70 and 30 percent for training and testing data, respectively. Once it is split, separate .csv files are generated for each model, since all models have their own input features. Each file contains the base features of the dataset and the engineered ones.

Random Forest The random forest is trained by preprocessing the data from the default data set and then passing it to the model. Since the model cannot see the meaning behind a string enumeration value, the training becomes much more difficult, and inefficient. The training occurs in the following steps: 1. Separate the event columns into separate binary columns, where the name of the column is the name of the concrete event, and the value is whether the event is that or not by placing ’1’ and ’0’ respectively. 2. Drop the rows where the values are null. 3. Divide the set of dependent and independent variables - next event and event, lifecycle: transition, respectively. 4. Train the model with the data. The training step is implemented with the default method fit in the sklearn library. The fitting is done by the bagging method, which means that the dataset is fed to the forest in a random manner each time.

Process Mining with Machine Learning

125

Neural Network Classifier Training the neural network classifier was implemented by preprocessing the data, and passing it to the model in five epochs, with a batch size of 512. The split between training and testing data sets was 90 to 10 percent, respectively. The reason for the test data reduction is to increase the amount of training samples for the model, since the accuracy, while using only 70 percent as training data, was insufficient. The process of training and testing the model was the following: 1. Subdivide the event columns into separate binary columns, where the name of the column is the name of the concrete event, and the value is whether the event is that or not by placing ’1’ and ’0’, respectively. 2. Fill the values of the places where null values are present. 3. Train the model with the training data. 4. Test the model with the testing data. The training procedure consists of the following: • Forward propagation of errors Inputting the training data as batches of tensors to the input layer, and producing the results in the last layer, where each neuron contains the results of the previous hidden layers, passed through a softmax activation. This means that every neuron in the output layer contains a probability of being active relatively to the others. So the highest probability neuron becomes ’1’, and the rest - ’0’. This layer of neurons linearized in to a 1-dimensional vector, corresponds to the one-hot encoding of the event type that is predicted. The binary cross-entropy function is used to calculate the deviation from the actual result, produced by the network (aka the predicted event) and the expected predicted event. • Backward propagation of error Knowing the error from the forward propagation phase, the neural network can adjust the weights (the connections between the layers), such that the error is minimized. The learning rate is variable as this model utilizes the

126

Chapter VII

ADAM algorithm for momentum gain of the learning rate. As soon as the values of the weights are recalculated, the training procedure repeats until a reasonably small error is reached.

Neural Network Regressor The deep learning regression model was trained and tested in a similar fashion to the classifier. The differences are primarily in the loss function and the activation function. In the forward propagation, the final layer contains one neuron with a linear activation, hence making it appropriate to use a MSE (Mean Squared Error) activation function. Then the error is backwards propagated in the same manner, the only difference being the adjustment of weights, since the learning rate adaptation is now not achieved using the ADAM algorithm, but FTRL as it reduces the loss much faster. In this case, FTRL does not introduce issues regarding stability since the model is relatively shallow, and each layer contains a relatively large enough number of neurons.

Results This section explains the results of each model’s performance and opens a discussion as to which model behaves best by comparing the metrics.

Random Forest To describe the results of the random forest, a confusion matrix was generated and plotted to describe the different metrics used to validate the classifier. From Figure 8, one can conclude that for some of the events the random forest generates a relatively high number of true positives, however, for some the true positives are low in comparison to the false negatives and the false positives.

Process Mining with Machine Learning

127

Figure 8: Confusion matrix for the random forest’s predictions 12 The calculated accuracy for the random forest was 77.6%. The method for deriving results for the neural network classifier was identical to the random forest - a confusion matrix was generated. From Figure 9, it can be seen that there is a much higher percentage of true positives in the neural network predictions than in that of the random forest.

12

“Nikola Sotirov. Github Project Repository”.

128

Chapter VII

Neural Network Classifier

Figure 9: Confusion matrix for the neural network classifier’s predictions13 An interesting observation to note is that the matrix is quite sparse, meaning that the model is either predicting with high precision, but in some cases, it is thoroughly convinced that the predicted event type is correct, although the expected result is different. The calculated accuracy of the neural network next event type classifier is 83%. That is a relatively high percentage, considering the shallow structure of the neural network - it contains only four hidden layers. However, the high accuracy could be explained by the fact that the dataset contains patterns that are easily predictable.

13

“Nikola Sotirov. Github Project Repository”.

Process Mining with Machine Learning

129

The neural network predictor of the next event type is better in comparison to the random forest, as seen from 9 and by having compared the accuracy of both models.

Neural Network Regressor

Figure 10: Timedelta prediction occurrences against real timedelta occurrences.14 As the predictions of the regressor are not binary, but rather continuous, one cannot generate a confusion matrix to visualize the accuracy and precision of the model. Another method was used to visualize the model performance. From Figure 10, one can infer that the predictions somewhat align with the real timedeltas with some deviations, especially in the 0.5 to 2 seconds prediction interval. That is due to the fact that the model is biased. However, the predictions stabilize at the 3 to 5-second interval. The plot was done by implementing a logarithmic scale in order to fit all the bars in one plot. The calculated R2 score of the neural network timedelta predictor was 0.64. This 14

“Nikola Sotirov. Github Project Repository”

130

Chapter VII

means that the model is relatively accurate (predicting correctly in more than 60 percent of the cases), but definitely not as precise as the event type predictor.

Conclusion In conclusion, the results indicate that for classifying the next event type, the neural network prevails in performance, and in the context of prediction of the time until the next event occurrence, a reasonable R2 score was reached, even though the deep learning model was shallow.

Discussion Although reasonable results were achieved, the models could further be improved to fit the data better with increased accuracy and precision. For instance, the deep learning model that classifies the next event type can be improved by adding at least one LSTM layer that would capture the temporal aspect of the data. As to the timedelta predictor, more layers could be added with various activations. The number of epochs for both models could be increased, which could mean improvements in the results.

References “Artificial Neural Networks”. Accessed June 4, 2022. https://iq.opengenus.org/ artificial-neural-networks/. “Nikola Sotirov. Github Project Repository”. Accessed June 24, 2021. https://github.com/ NickSot/process%5C_mining. Software AG. “Your First Steps into the World of Process Mining”. Accessed May 25, 2021. https://www.softwareag.com/en_corporate/ resources/asset/ebook/business-process-transformation/first-steps processmining.html?utm_source=bing&utm_medium=cpc&utm_camp aign=bt_aris_process-mining&utm_region=hq&utm_subcampaign=stg 1&utm_content=ebook_your-first-steps-into-process-mining_stage1 &msclkid=1a8e9abfd556137ca255f8c70261ec49. “Workflow of the Random Forest Classifier”. Accessed May 18, 2022. https://www.researchgate.net/figure/Workflow-random-forestclassifier_ fig3_343627992.

CHAPTER VIII APPROACHING THE ADVANCED ARTIFICIAL INTELLIGENCE ALEXANDER LAZAROV Introduction A philosophical discussion of how to behave when communicating with autonomous, Advanced Artificial Intelligence (AAI) objects requires consideration of the following questions: What do intelligence, understanding, and autonomy mean? What is the difference between data and information? What is the philosophical essence of any intelligent procedure? What is real and what is virtual in a data exchange process? What are the differences between AI and robots? What is AAI, and where does it fall within the framework of narrow (specialized) AI and general artificial intelligence (AGI)? Which issues should be considered in cases of human to AAI and AAI to AAI communication? Answers to the above questions will offer a discourse for both theoretical analysis and future AI development. This paper will conclude that the progress of AI’s analytic and synthetic capabilities—in some perspectives, already close to human mental faculties—involve applied philosophy with a never previously experienced framework and range. Therefore, in designing the next generation AI—one intended to correspond to social expectations—thinkers of all philosophical traditions can no longer draw a strict borderline separating their wisdom from what science and engineering have already demonstrated.

132

Chapter VIII

Intelligence, Understanding, Autonomy Both psychology and philosophy have offered many descriptions of intelligence. Most of these have met severe criticism, and a consensus has never been reached. Thus, in addition to the constructive challenges, AI designers also answer fundamental philosophical queries as they go about their work. Perhaps, and amazing to many philosophers, some AI inventors were very successful in managing this situation. Among these narratives on intelligence, I see David Hanson’s concept as best serving AI development and one that makes possible a bridge between humans and machines in a philosophical discourse.1 Hanson claims that intelligence is a set of the following: An autonomous capacity to produce predictions about future events. An openness to the external world that allows detection and exploration of what is occurring. Developing and conducting an appropriate reaction to oncoming changes of state. According to Hanson, understanding of events and processes emerges whenever the intelligent bodies’ autonomously generated forecasts happen indeed. Evidently, successful predictions numbers and complexity can serve as criteria to assess and compare the characteristics of both biological and automata intelligent performance. However, this analysis requires an awareness of what autonomy means. As Luis Moniz Pereira outlines, although there are many arguable descriptions of autonomy (as there are with intelligence), today most experts would claim that autonomy simply means the opposite of externally driven.2 Based on these notions, intelligence is a capacity that belongs to living organisms, but also to some human-made equipment, which by itself manages data identification, collection, and processing that results in an independent informational 1

David Hanson, “Expanding the Design Domain of Humanoid Robot” (paper presented at the ICCS Cognitive Science Conference, Special Session on Android Science, Vancouver, USA, 2016). 2 Luis Moniz Pareira, “AI & Machine Ethics and Its Role in Society” (keynote speech at the International Conference of Artificial Intelligence and Information, Porto, Portugal, Dec. 5–7 2017).

Approaching the Advanced Artificial Intelligence

133

production. Comparing Hanson’s concept of AI to older ones illustrates that the research path has shifted over several decades. In 1955, Allan Turing foresaw AI’s emergence and suggested his famous Turing Test, thus, offering a first attempt at describing the workspace of AI exploration,3 Later, John McCarthy, et al. suggested what became the classic definition of AI.4 Positioning Mc Carthy’s claim next to Hanson’s illustrates that the current conceptualization offers a more pragmatic perspective. This is because the 1955 statement insisted that the core of the AI problem was to make machines behave in ways that would be called intelligent if humans acted in the same manner in similar circumstances, while the 2016 narrative posits philosophical guidelines which can be applied in practice. Alexander Wissner-Gross enhances Hanson’s view by stating that when dealing with probability (the prediction of various scenarios), intelligent bodies act to maximize the number of possible options that might occur in the future.5 He holds that intelligence involves keeping the freedom of action and producing choices of opportunities. Stuart J. Russell goes further underscoring that contemporary AI is already successful in dealing with predictions that it judges as uncertain at the time of their generation.6 Floridi and Sanders also highlight that AI is a growing source of interactive, autonomous, and self-learning actions that are informed, smart, autonomous, and are able to perform morally relevant actions independently.7 According to them, this is the fundamental ethical challenge posed by AI. In my opinion, a crucial factor to add to these analyses is that the highest intelligent performance is expressed not only in the passive estimation of what is about to happen and to react appropriately, but it also includes a 3

Alan Turing, “Computing Machinery and Intelligence,” Mind: A Quarterly Review of Psychology and Philosophy (October 1950): 433–460. 4 John McCarthy et al., “A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence, August 31, 1955,” AI Magazine 27, No. 4 (Winter 2006): 12. 5 Alex Wissner-Gross, “A New Equation for Intelligence.” TEDx-Beacon Street Talk. November 2013, accessed July 3, 2022, https://www.ted.com/talks/alex_wissner_gross_a_new_equation_for_intelligence. 6 Stewart J. Russell and Peter Norvig, Artificial Intelligence: A Modern Approach, 4th edition (Hoboken, N.J.: Pearson, 2021). 7 Luciano Floridi and J. W. Sanders, “On the Morality of Artificial Agents,” Minds and Machines 14, No.3 (August 2004): 349–379.

134

Chapter VIII

focused activity intended to influence the future in a pre-targeted perspective. All of us plan for tomorrow, next month, next year. We also seek objectives like affecting global climate change, prolonging human life, designing AI and our co-existence with it, etc. Therefore, if we have created and implemented high-quality AI systems, we must assume that they will perform a predictive function together with their manipulation of processes that will impact their future too. In case of their success in this, a great challenge immediately arises as to how our autonomously modeled future and AI’s will relate to a common physical and social environment. This new factor will strongly affect our traditional practices due to the encounter of our interests with those of the new human-made intelligent bodies. Moreover, we shall have to assess all ongoing events while considering what the innovative automation is about to demonstrate. AI will do the same regarding our actions.

The Difference between Data and Information Both data and information have a binary nature that humans and digital machines apply when examining the universe. The binary system is a clear and logical mathematical method that has a philosophical background, just as a mathematic equation considered from a philosophical standpoint is a matter of bringing diverse concepts together.8 Contemporary philosophy claims that, unlike an inanimate object, a living organism is capable of perceiving and investigating changes of state in diverse ways. The higher an organism’s level of nervous system development, the greater the variety of action assessments and option choices it can conduct. However, without exception, the exploration of what is external to the intelligent body results in an inclusive context that defines the entity’s status in a single from a pair of polar contrast states—front or back, big or small, static or moving, dangerous or not. Thus, all of us employ

8 Sylvain Duranton, “How Humans and AI Can Work Together to Create Better Businesses,” accessed May 7, 2020, https://www.ted.com/talks/sylvain_duranton_how_humans_and_ai_can_work_tog ether_to_create_better_businesses?language=en.

Approaching the Advanced Artificial Intelligence

135

binary opposites to sustain a picture and figure out its meaning no matter if considering physical entities or abstract social issues like good and evil. A peculiarity of our binary approach to the universe is that all of us find it difficult to define a meaning unless we recognize the possible opposite. If we cannot do that, confusion occurs and formulating a comprehensive explanation for such problems often lasts for centuries without a conclusion. For example, one can study the discussions of what happiness is, but since the description of being happy cannot be considered as exactly the opposite of unhappy, the matter remains unsettled. It is similar to harmony and disharmony, etc. In other words, we simultaneously negate that what things are cannot define what they are indeed. Perhaps, this mode of our mental activity is predefined by, and relevant to our male/female nature. Some thinkers hold that the physical and social reality is something quite different from the representations that humans produce by applying the binary approach. However, this issue falls beyond this text’s focus. The binary system’s inventor is Leibniz, and a few centuries ago he discussed it in The Monadology that experts highlight as a historic root of all recent philosophical-informational analyses.9 For Leibniz, Monads are the elementary building units of the universe and can only be found through pure logic. He aimed to prove that all knowledge is composed of these logical atoms or building blocks, and his claim has inspired many philosophers. The Leibnizian school has influenced digital equipment development as it stated that a mathematical system of symbolic logic allows one to better explore the laws of human thought and reasoning compared to spoken and written language. Apparently, this view has already been confirmed, at least for writing and storing human ideas, because any text editing computer program is doing what Leibniz proposed—it turns thoughts into digital, mathematical, codes. Moreover, the newest data and informational research imply the following consistent, logical concepts:

9 Gottfried Leibniz, The Monadology. Trans. Robert Latta (Oxford: Clarendon Press, 1898).

136

Chapter VIII

Whenever humans and AI operating bodies interact with the environment, with one another or among themselves, a change of state occurs. Each distinguishable state is a bit of data. The binary symbols “1” and “0” have just two states. They encode either a positive or negative answer to any assessment: “1” signifies “yes” while “0” signifies “no” in reply to a specific question. Each distinguishable state is a bit of data. The binary symbols “1” and “0” have just two states. They encode either a positive or negative answer to any assessment: “1” signifies “yes” while “0” signifies “no” in reply to a specific question. The binary system is useful because it is the simplest data encoding method as a binary “1” can represent any entity, while “0” indicates its absence. The number of distinguishable states that a system identifies and memorizes is the amount of data that it can encode. Binary system results are easy to conserve in memory for indefinitely long periods. Moreover, any code can be easily transferred from one place to another, carried by signals, generated in energetic circuits where each pulse serves as “1” and its absence as “0.” Significantly, a data bit is an elementary particle that can have only one of two possible values and thus, it is one which can be used in digital processing. I am not claiming that the same process happens in the human brain. However, I do not exclude such an option. There is a similarity because humans, and animals with higher nervous systems, and AI, have the capability to memorize large quantities of data, as well as to generate numerous links and references among these data. The higher the intelligent body’s memory and data processing capability, the greater the number of produced data links and references. The philosophical essence of the digital bit is obvious: it allows the intelligent actor external investigation and the translation of results into

Approaching the Advanced Artificial Intelligence

137

digital codes, which can undergo computations. As Floridi assumes, data are durable, reusable, quickly transportable, easily duplicable, and simultaneously shareable without end.10 A few years ago, the theory of Quantum Computing introduced the qubit, which made possible a compressed “yes+no” processing up to the moment when a necessity arises to provide decompression and have either the particular “yes” or “no” involved in an issue’s conclusion. The qubit technology brought a tremendous increase in processing speeds, described as thousands of times faster than classical computation. However, it did not change the bits’ essence in a philosophical discourse, which highlights that: There must be an intelligent body involved to formulate and ask a question, and then to detect or assess whether the answer is a positive or a negative one. Material-energetic objects or processes, as well as any form of social communication, are data carriers. In other words, each intelligent body, no matter of its biological or mechanical origin, can explore and synchronously synthesize new data. Most often, data, and information do not refer to a single bit, but to digital bit configurations (patterns) that emerge due to simultaneous asking and answering large-scale sets of various questions. Thus, the highest intelligence approaches difficult to understand and complex issues. All intelligent bodies can extract the data from any of its carriers, and further deal with it by modeling. Regarding humans, this activity relates to most mental faculties, but is most brightly expressed in perception and imagination, while machines perform computer modeling.

10

Luciano Floridi, “What the Near Future of Artificial Intelligence Could Be,” Philosophy & Technology 32 (March 2019): 19, https://link.springer.com/article/10.1007/s13347-019-00345-y.

138

Chapter VIII

Single digital bits as well as sets of bits move from sensors to brains and artificial processing units and undergo grouping, configuration, and structuring, which results in the generation of new digital constructions. These results are conserved in memory in a hierarchical architecture. Unlike the polar “yes” or “no” of a single data bit, a targeted set of questions can lead to an unlimited variety of digital patterns. They signify much more complex answers that can shift the pure positive or negative statements to deep multi-layered conclusions. To illustrate, applying digital bit calculation any computer-drawing program can suggest a color palette that is richer in nuances than that which the human eye can discern. Importantly, the higher the level of intelligence, the more frequent the inquiries and recognition of the environment via pattern applications. The intelligent actor’s capacity, and experience to examine and learn predefine the range and deepness of what he or it could achieve via an investigative analytic approach. In other words, as Michelangelo has claimed, every block of stone has a statue inside it, and it is the task of the sculptor to discover it. At a later stage, the conserved bits and patterns can be recalled from memory whenever needed and used for new analytic or synthetic processing. This in turn leads to partial or total restructuring, reorganization, and rearrangement as well as placing them in a set with new incoming data flows. Simultaneously, depending on its memory and processing capacity, an intelligent system may record a great number of both old and new digital constructions (codes) combined with various links and co-relationships. Importantly, these codes present a large variety of facts and dynamic events, but also synthesize data such as conclusions, assessments, judgments, and others that may affect future activities. Without exception and based on the processing of what is available in its memory that corresponds to the new incoming data flow, every intelligent body (whether biological or human-made) can provide targeting and goalorientation in reaction to the observed environment. Such bodies can also autonomously plan their activities within the environment. This ability illustrates the difference between the philosophical essence of data and that

Approaching the Advanced Artificial Intelligence

139

of information. To summarize, information is an intentional goal-oriented data bit pattern that belongs to its intelligent creator. That creator can share it by publishing it (making it accessible to other intelligent bodies) or memorize it and store it internally. Significantly, if published, information flows to other intelligent bodies) as data, and it is necessary to underscore that there is a nonstop data-information-data pulsation, which is essential and typical for all intelligent bodies’ communication. Expert analyses in this field recognize this, but frequently, non-experts apply “information” to characterize events that are only “data.”

The Essence of Intelligent Procedures No matter what type of intelligent actor (human, animal, or machine) the intelligent procedure in play is the same and follows four-steps: Step 1 involves external data identification, acquisition from the carrier, transfer to a processing unit, that encodes it, assesses it, and references it. All this activity results in the production of data patterns and placing them in memory as digitally recorded frameworks linked or related to others. Importantly, these new data constructions belong only to their creator. When discussing biological intelligent bodies, this intelligent procedure step is termed perception and presentation building, while for machines, this is data input. In either case, this phase results in creating a digital twin of what has been explored. Step 2 is the analysis of data records, followed by data synthesis—decision making on how to act accordingly to circumstances recognized in advance. Briefly, this is the stage of generating intentions and goal orientations. In other words, this is the phase of creativity when entirely new and purely private data patterns are born to drive subsequent intelligent actions. Step 3 is the information production phase. This is the process of structuring an approximate or precise plan of consecutive moves regarding how to reach the target goals. First, this intelligent act requires stored data recall from memory and an assessment of whether it is of sufficient quantity and quality to serve the wanted informational production. Then, most often, a targeted additional foreign data search is conducted to fill in any data gaps that were

140

Chapter VIII

found. Once the intelligent body recognizes that the new data acquisition from the physical and social environment is enough and appropriate, the new data search stops. Then, the data bits rearrangement, reconfiguration, and reconstruction occur combined with re-judgement and re-assessment of the new product’s applicative potential. The generative process ends when a representation/model is built on what is expected to happen in case the plan is put to practice. Thus, the in-form is created. To illustrate steps one to three: most people know that Frankfurt, Paris, and London are among the largest civil aviation centers in Europe. This is data they have probably captured in an act of non-focused memorization. Imagine that, at a moment, someone intends to fly from Sydney to Sofia and he discovers that there are no direct flights between those cities. Then, he recalls which are the biggest European airports and decides to check for connecting flights. This is a targeted additional data search to fill in an evident gap. Once a connection is found, the waiting time between flights is assessed as non-risky short or not inconveniently long, and the ticket price is judged as acceptable, the plan is recognized as ready to apply. The inform becomes available, and significantly, it belongs only to its author. AI follows a similar path. Once the in-form is produced, the intelligent procedure has two options: First, the intelligent body decides to keep the in-form in privacy. The human or the machine puts the plan in memory and never presents it to others. If this option is chosen, the intelligent procedure ends here. However, this happens more rarely than the second option – step 4. Step 4 relates to making the in-form accessible to other intelligent bodies by either performing it, or by conducting transformations.

The In-form, Per-form, Trans-form Triad Making the in-form accessible to other intelligent bodies can occur by either per-form or trans-form. Per-form means providing communication that involves just information to data exchange. The in-form creator shares ideas, concepts, assessments, opinions, impressions, desires, feelings, attitudes, plans, as data patterns that undergo data collection by others. This

Approaching the Advanced Artificial Intelligence

141

may happen through speech, writing, gesture, drawing, etc. The per-form mode may be either direct or encrypted, and data identification and acquisition may occur immediately or in the future. At first glance, it seems illogical to think about the in-form reaching others in advance of its author willingness to present it. Significantly, there is a tendency to develop AI to read human’s thoughts, even when one has no intention to publish them. In a philosophical discourse, this is a critical aspect of the human-to-AI interface that may change human subjectivity. However, I want to underscore two features that can appear and characterize any in-form performance. Demonstrating the in-form by performing it never brings considerable direct changes of state within the physical and social environment. Those may only happen later if the intelligent procedure has provoked some activity. Simultaneously, without exception, the per-form requires another intelligent body’s involvement. Any type of in-form sharing between two or more intelligent actors is what contemporary philosophy accepts as virtual communication. Trans-form, unlike the in-form performance, denotes actions focused on causing a change of state within the external environment driven by the intelligent body targets. Transforming is a specific expression of his or its informational product. In such cases, another intelligent body in play is possible, but not obligatory. In other words, conducting transformations may serve communication but they are not mandatorily tied to it. For example, I am at home alone and I decide to have a cup of coffee. I make one and drink it. All that action was the result of my informational production that focused my trans-forms to reach the goal. Someone else might have watched me and read my in-form contents or not. Importantly, in any event, providing transformations inevitably brings changes of state to the environment and they can be examined by both the information creator as well as by other intelligent counterparts. The totality of all evident changes of state are what contemporary philosophy considers as real. However, sharing abstract informational products like assessments, judgments, desires, impressions, attitudes, feelings, etc. is mainly possible through virtual performance.

142

Chapter VIII

Research on the in-form, per-form, and trans-form triad must also consider the de-form phenomenon. Deformations occur once the in-form is presented, no matter of whether the last appears as performance or transformation. In cases of per-form, the de-form is expressed as some misunderstanding between what the in-form author made accessible, and what its counterpart has understood during the communication process. There are many reasons that cause de-forms, but their analysis falls beyond this paper’s scope. In cases of trans-form production, the de-form is the difference between the info-product creator’s intended goal and what was successfully put into practice. One can imagine various reasons that cause such deviations, ranging from incorrect planning of activities to some general impossibilities due to physical or social circumstances. To illustrate de-form in the case of human-to-human in-form performance: I have a Bulgarian friend who worked as an architect in Africa in the late 1960ies. One day, he gave a servant a few US dollars and asked him to buy a pack of Marlboros from a neighboring shop. The servant asked how to proceed if Marlboros were unavailable, and the instruction was to buy something else. Most humans would assume that the order’s intent was to purchase some other brand of cigarettes. However, the servant returned bringing a hot-dog and described how excited and busy he was to decide what to buy instead of the unavailable Marlboros. This type of misunderstanding seems possible when communicating with intelligent machines if they literally copy a request. Here, future human-to-AI interface de-forms will be a challenging issue. To summarize, the important fact to bear in mind is that deformation is typical for any information to data exchange. Its level may vary from insignificant to entirely spoiling the mission.

The Difference Between AI and Robots Following the intelligent procedure analytic line that describes in-form generation and decision-making about whether and how to publish, the borderline between AI and robots is clear. AI is the information producer, while the robot is a peripheral tool that provides the in-form visibility in a manner corresponding to the targeted goal. Therefore, if the AI running

Approaching the Advanced Artificial Intelligence

143

system is dedicated only to performance, it needs only a network, a display, or even only load-speakers to serve it, so there are no robots needed. For example, an artificial chess-player or music composer needs no mechanical apparatus. Simultaneously, depending on their purpose, robots vary considerably in construction: humanoid, flying drones, small, crawling, and insect-like, etc. They may or may not have AI installed to guarantee their balance and operation. However, in any event, it is possible to connect them to an external AI system characterized by a high level of intelligence. Then, robots become engaged to act as AI trans-form instrumentation. Significantly, depending on the set of memory and data processing speed capacity, a single AI-operating machine can conduct many parallel multigoals in-form generations and drive multiple robots.

What is Advanced AI (AAI)? One can observe a variety of AI applications, designed for various activities in our lives: machine language recognition and interpretation, face and other image recognition, navigation, driverless vehicles, medical diagnostics, gaming, etc. Many of these AI solutions perform successful non-conserved big data streams identification and examination, applying deep algorithmic learning via neural networks parallel to the classic mathematical logicgrounded computer processing. Thus, they discover data patterns that are related to a variety of running interactions, links, and co-relations. In this perspective, contemporary AI systems’ analytic/synthetic capacities strongly exceed human thinking and imagination in particular areas of research. Here it is useful to define advanced AI (AAI) and clarify where it falls within the framework of narrow (specialized) AI and artificial general intelligence (AGI). To better understand this issue, one should know the three essential characteristics of big data: First, it is an enormous volume of data – a volume that no human could memorize and explore. Second, these are unstructured, real-time, and rapidly changing data flows.

Chapter VIII

144

Third, the diverse data streams to investigate have heterogeneous origins. Because being unemotional, AI operating machines can ask a single question as many times as they need without becoming annoyed or dysfunctional. Moreover, humans, being unable to deal with big data, experience a blind witnessing as to what machines do during data input and data processing. Also, we cannot foresee the computers’ activity based on studying all the rules they follow in their processing because, as Duranton outlines, AI constantly learns from its experience and thus, it continually produces and applies new rules.11 In other words, humans have no choice but to wait passively until AI intelligently generates results. Further, this black box phenomenon drives the conclusion that we can never be sure whether the AI has demonstrated the totality of its intelligent production or just a part of it. To study this issue, laboratory experimentation with AI systems practices a periodic data processing switch-off so that experts can survey retrospectively the many lengthy lines of logged mathematical activity. Although the success of this research is questionable, it seems that AI systems memorize more in-form than they publish. It has also been shown that AI deep learning retains some of its activity results for application in the future. Therefore, we may never be aware of all processing experiences, which it has accumulated. With this as background, the philosophical concept on AI development, which a few decades ago was seen as evolving from slow to similar to human mental faculties, and in some far future to AGI, is now accepted as obsolete. The contemporary argument states that the AI’s successful data processing, which overpowers our human abilities, does not make it smarter than we are. Therefore, the AI classification was oriented to narrowspecialized AI, designed as a human-centered and entirely humandependent assisting instrument, and AGI as something that will emerge in the future as a self-managing non-supervised actor. Pessimists predict its occurrence in some hundred years ahead, while optimists insist that it will

11

Duranton, “Humans and AI.”

Approaching the Advanced Artificial Intelligence

145

happen much sooner because of the exponential growth of the machines’ data memory and processing capacities. I will not enter this discussion of timing, although it is impressive that the focus on AGI emergence falls on when rather than whether. However, from my perspective, the crucial question is also how it will arise. Most texts, dedicated to AGI issues focus either on its differences with human intelligence (often claiming that machines just simulate intelligence), or draw a picture in which suddenly, a new revolutionary AGI performing automation is invented. Perhaps, thinkers are inspired by publications about the Deep Mind algorithms involved in demonstrations with the Japanese board game Go.12 Arguably, this is the hardest human intellectual game. Initially, the AlphaGo deep learning algorithm won the World Championship in Go by studying the entire human history of the game. Subsequently, AlphaZero achieved brighter success having been provided with just the game rules and having played for six weeks against itself. It studied its own experience, entirely ignoring the human practice history. Logically, as this AI managed learning Go in a short period, it was expected later to run into new spheres. Thus, radically new AGI expectations emerged. However, there are no publications to confirm success in such a trend. Although not positioning myself on the list of the AGI revolution path followers, I do not exclude it. Simultaneously, I can also draw two distinct scenarios for the narrow AI’s development: First, any narrow AI Deep Learning algorithm progress could follow a step-by-step intensifying specialization in new investigative fields, so that the particular AI operating system adds new capabilities. Thus, it will gradually expand its narrow competence in analytic and synthetic data processing. In this way, it would follow human educational practice. AI will become more and more applicable to various specific explorations and activities, but it will never become completely general. At the same time, a second creative option also seems possible. A new type of narrow AI could be developed, which could coordinate the 12 For more detailed information, see https://deepmind.com/research/case-studies/alphago-the-story-so-far .

146

Chapter VIII

activity of several or even all existing narrow intelligent systems so that limited universality arises by conducting synchronized multispecialization in various spheres. I recognize that these paths for narrow AI evolution are easier to design compared to AGI and thus they could emerge more rapidly. I call this eventual technology Advanced AI (AAI). In my opinion, it will be the apparent narrow AI next generation and as it will bring considerable new circumstances, due to having innovative intelligent bodies in play, our contemporary society should get prepared to meet it in short terms.

Advanced AI Communication and Networking The essential mission of any type of communication is sharing content and discovering meanings (thoughts, viewpoints, attitudes, feelings) to achieve mutual understanding, ergo, producing common successful predictions about future events even if they refer to issues retrospectively. In this fashion, mutual understanding among intelligent bodies means an agreement on what is about to happen combined with a common standpoint on how to react effectively to the estimated changes of state. Moreover, communication operates within a framework defined by the range of the inform production that every intelligent actor decides to present. All of us traditionally communicate among ourselves in a human-to-human fashion, and over a long time we have experienced the resulting benefits and disappointment. Everyone has their own approach, which includes consideration of the goal of the communication and the identity of the counterpart. However, the human-AI and AI-AI interfaces are different. Our experience here is based on just two decades of practice and its theoretical analysis. Further, human-to-AAI communication assessment is just prognostic because AAI is not yet available. Therefore, I will focus first on what seems common to us and them and what immediately appears is the intelligence procedure, which both humans and machines employ. Here, the issues to highlight are the following: As evident from many specialized AI actions, humans appear generally slower in data processing compared to intelligent machines, and this will

Approaching the Advanced Artificial Intelligence

147

not change. Moreover, in any event, the intelligent data input to in-form output process will suffer delays in relation to the examined changes of state because of the necessary period to process the acquired data and then determine follow-up moves on behalf of the intelligent actor. In many cases, the shorter the intelligent processing period is, the fewer the number of prediction errors. In this trend, it seems that machines will overpower humans. No matter the data volume, due to technological capacity when copying mathematical code, in any event AI-to-AI network data flow exchange always runs faster compared to human-to-human or human-to-AI communication. From this perspective, AI exceeds the human mental faculty, and in the future, this feature will be typical for AAI as well. The de-form level in cases of AI-to-AI data exchange is much lower compared to human-to-human, or human-to-AI communication. This is due to the direct mathematical code copying that skips the conversion to language, image, or another mode of the in-form signification, connotation etc. Advanced AI versions will keep this characteristic. Simultaneously, (at least at present) it will most often interpret a single meaning as embodied in a particular code in a one-to-one fashion, rather than denoting some multilayer approaches typical to humans. This issue needs special attention and care to avoid various misunderstandings with the machines. In my opinion, AAI, which arises as an autonomous intelligent body that can drive peripherals, will also add the following specific components that we will have to consider: First, we must expect AAI to focus on shaping its future together with us. Therefore, we shall have to assess all ongoing physical and social processes based on a prediction of what the innovative automation is about to demonstrate. Simultaneously, AAI will do the same about us. By investigating big data, AAI will apply deep learning comparative analyses to real-time data streams with heterogeneous origins, something that humans cannot do. Bearing in mind that any intelligent

148

Chapter VIII

procedure deals with only a partial data input of what is potentially available, which in turn depends on the intelligent body’s sensors, memory, and data processing capacity, in many cases, a prediction’s success is related to what has been involved in the in-form’s creativity. Therefore, compared with humans, AAI has a better chance for accurate predictivity, and we will need to add this assessment in communicating with it. Moreover, we will run our AAI-to-human communication never knowing if it shares with us all its informational production or just a part of it. The same is true when evaluating the next generation of AI-to-AI network informational exchange. Additionally, any AAI informational product that is presented to us will require an analysis of what kind and what amount of data shaped its inform generation. We should also consider the AAI type in play because unlike AGI, AAI is about to be just a set of synchronous specialized AI operations. Therefore, it is necessary to continually judge the level of its universality because distinct specialized sets may produce different outputs for similar inputs. The above list of issues may not cover all the variations typical for humanto-AI and AI-to-AI communications if compared to our human-to-human practices. However, I see them as a comprehensive background that sets out a framework for future research and analysis. I believe, many other issues will emerge. However, regarding human-to-human communication, many thinkers note that effective communication is less about talking and more about listening to reach a common understanding. I will not discuss if this is the best behavior among humans, but I think that it would work perfectly in our communication with advanced and general intelligence computers. I do not claim that AAI is smarter than we are—there are many arguments against such a statement—but without doubt, AAI will be smart enough and completely foreign to us. Thus, at least at the first stage, many of its conclusions will seem surprising both as to content and analytic perspective. However, many of them will be correct.

Approaching the Advanced Artificial Intelligence

149

Conclusion Today, our civilization is at the dawn of a new era. We are about to meet an unknown innovative autonomous intelligent actor—AAI operating machines that surpass us in some areas of our mental faculty. Moreover, AAI memorizes its deep learning investigation results for future application. Therefore, in the future, when analyzing a particular event or process, a group of similar machines all running similar hardware and software solutions, will generate diverse in-forms depending on what they have explored in the past. Although this is not an expression of human subjectivity, no doubt, this is a step toward AI individualization. Therefore, in addition to our human-to-human face-to-face communication traditions, we must develop a new human-to-AI and vice-versa interface that should not follow a pure subjective-objective path but should recognize the following new circumstances: Advanced AI systems can act in our favor, but also as our competitors and opponents. Moreover, to avoid encounters with them, we will need to predict AAI’s reaction in various situations while keeping in mind all its peculiarities, including new ones that we discover in future research. In cases of our synchronous interface with AAI solutions, we will need to recognize their ability to run in a parallel, multi-task mode that can result in the driving of multiple robots. Thus, we will need to be aware of whether regarding diverse activities we are facing a single system, or different intelligent machines. Finally, as AAI is about to become a cloud-based technology, we will need to develop the habit of communicating with nonsingular intelligent bodies, which not only borrow from common data sources including the deep learning results experiences, but also might share and exchange parts of their expertise to serve parallel missions. Therefore, contemporary progress in AI analytic and synthetic capacities, which seem close to the human mental faculties, requires a philosophical approach with a framework and range that we have never experienced.

150

Chapter VIII

Moreover, it needs to not only direct AI development, but also be prepared to meet the new challenges that will arise as AAI emerges. In my opinion, this task is so complex that philosophers of all schools and traditions can no longer draw a strict borderline separating their wisdom from what science and engineering demonstrate. First, they should focus on how to model the technological progress to protect human nature from AI targeted breakthroughs of our subjective firewalls and disclosure of in-forms that one does not want to publish. At the same time, we must develop a philosophical concept for our transition from human-to-human face-to-face communication to human-to-AI and vice-versa interface and interaction that will avoid mutual encounter and conflict.

References AlphaGo. Accessed July 1, 2022. https://deepmind.com/research/casestudies/alphago-the-story-so-far. Duranton, Sylvain. How Humans and AI Can Work Together to Create Better Businesses. Accessed May 7, 2020. https://www.ted.com/talks/sylvain_duranton_how_humans_and_ai_ca n_work_together_to_create_better_businesses?language=en. Floridi, Luciano. “What the Near Future of Artificial Intelligence Could Be.” Philosophy & Technology 32 (2019): 1–15. https://link.springer.com/article/10.1007/s13347-019-00345-y. Floridi, Luciano, and J. W. Sanders. “On the Morality of Artificial Agents.” Minds and Machines 14, No. 3 (2004): 349–379. Hanson, David. “Expanding the Design Domain of Humanoid Robot.” Paper presented at the ICCS Cognitive Science Conference, Special Session on Android Science, Vancouver, USA, 2016. Leibniz, Gottfried. The Monadology. Translated by Robert Latta. Oxford: Clarendon Press, 1898. McCarthy, John, Marvin L. Minsky, Nathaniel Rochester, and Claude E. Shannon. “A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence, August 31, 1955.” AI Magazine 27, No. 4 (2006): 12–14.

Approaching the Advanced Artificial Intelligence

151

Pareira, Luis Moniz. “AI & Machine Ethics and Its Role in Society.” Keynote speech at the International Conference of Artificial Intelligence and Information, Porto, Portugal, Dec. 5–7 2017. Russell, Stewart J., and Peter Norvig. 2016. Artificial Intelligence: A Modern Approach, 4th edition. London: Pearson Education Ltd. Turing, Alan. “Computing Machinery and Intelligence.” Mind: A Quarterly Review of Psychology and Philosophy (October: 1950): 433–460. Wissner-Gross, Alex. “A New Equation for Intelligence.” Accessed September 7, 2016. https://www.ted.com/talks/alex_wissner_gross_a_new_equation_for_i ntelligence .

CHAPTER IX THE STRUCTURE OF ARTIFICIAL RATIONALITY BORIS GROZDANOFF Abstract Deep Reinforcement Learning (DRL) is at the forefront of modern artificial general intelligence (AGI) research and formalizes AI tasks in terms of agent, environment, state, action, policy and reward and also harnesses the function approximation power of the other leading AI instrument, the artificial neural networks (ANNs). One of the main challenges contemporary efforts to build artificial general intelligence (AGI) systems face, in a broad Turing-test framed setting, is the artificial emulation of key components of human language-based reasoning, like meaning of singular terms and expressions but also semantically enabled logical reasoning. Here I envision a high-level AGI system, RAISON, that could be driven by a DRL architecture. I suggest that we can use Frege’s influential distinction between sense and reference in order to emulate linguistic meaning as found in real world human practice. For that purpose, I propose that a semantic graph space (SGS) can be trained on available NL datasets to deliver artificial semantic ability. I also suggest that the formal abilities of syntax manipulation and rules-based logical reasoning can harness the expressive power of SGS and thus enable knowledge-based «educational» training of an emerging general AI model.

Introduction In May 2022, the lead research scientist at Google’s DeepMind, arguably the world leading company in the development of Artificial General Intelligence systems, Nando De Freitas, provocatively claimed that

The Structure of Artificial Rationality

153

“It’s all about scale now! The Game is Over! It’s about making these models bigger, safer, compute efficient, faster at sampling, smarter memory, more modalities, INNOVATIVE DATA, on/offline. Solving these scaling challenges is what will deliver AGI. Research focused on these problems, eg S4 for greater memory, is needed. Philosophy about symbols isn’t. Symbols are tools in the world and big nets have no issue creating them and manipulating them”. 1

This statement, although not officially sanctioned as a DeepMind's position, created an intense discussion by some of the most influential AGI researchers and developers in the field. 2In its crux, it contains several key theses: 1. That there is a well-founded conception of an AGI (implicit) and 2. That AGI is achievable 3. The multimodal deep artificial neural networks (DANNs) and the deep reinforcement learning (DRL) architectures of AI are by and large sufficient for AGI 4. That «philosophy about symbols» cannot achieve AGI De Freitas’ position is characteristic of the prevailing view of AGI, which derives much of its influence from its success in narrow AI systems. This does not come from a won dominant position in the heated ongoing scientific debates in AI and related fields, like computer science, mathematics, cognitive science, neurophysiology, logic and analytic philosophy. In De Freitas’ statement we see very clearly the affirmative thesis that the general computational structure of A(G)I is already available and the (undisputed, sic!) gap between available (narrow, sic!) models and accepted A(G)I is a mere matter of mathematical and engineering improvements, that should focus on volumes of data, types of data, sizes of models and related engineering improvements like data storage (better memory) and computational processing. The negating thesis that

1

https://twitter.com/NandoDF/status/1525397036325019649 For a brief summary of the discussion see Sparkes, Matthew article in “Is DeepMind's Gato AI really a human-level intelligence breakthrough?” in New Scientist, 19.05.2022. https://www.newscientist.com/article/2320823-is-deepmindsgato-ai-really-a-human-level-intelligence-breakthrough/

2

154

Chapter IX

complements it is the one that while the above can deliver AGI, «philosophy of symbols» cannot. I will discuss both theses in more depth, but two points deserve mentioning before that: first, the term «more modalities» in De Freitas' claim and second, the apparent derogatory formulation of «philosophy of symbols». Under modalities De Freitas clearly has in mind modalities of multi-modal systems like GATO,3 where each modality has a distinct function and nature: playing a certain game, leading a conversation, image recognition and the like. On historical grounds, we can identify those modalities with different human abilities and in particular, such that contribute to human intelligence. Thus, the desired greater number of modalities effectively translates as «we need AI systems that have greater number human-like abilities». The «symbol philosophy» is easily decoded as pointing, mainly backwards, toward the now largely considered as unsuccessful early AI systems, founded on expert systems,4 which used symbolic manipulation and not ANN or DRL foundation. The implicit attack, however, is much deeper and is directed against the choice of formal syntax-based models of AI as opposed to current and argued as generally successful ANN and DRL-based systems. Unpacked, it translates as the following: «AI systems, built on any syntactic structure, are AI inferior to ANNs and other non-syntactic models». We can summarize that in De Freitas' thesis, the computational ANN and DRL architectures can deliver A(G)I whereas syntax architectures, expressing logical and other formalisms, cannot. The obvious assumption behind this is that the nature of human intelligence (HI) is best modeled via ANNs and DRL models, and not via syntactic models. We see that digital brain models are taken to (be able to) directly lead to human-like AI (HLAI). Or, to formulate it more clearly: proper brain function exhausts human intelligence. I will completely ignore the theoretical deficiencies of this clearly physicalist assumption, which identifies rationality with physical 3

For the GATO system see DeepMind’s paper: Scott Reed et al. (2022) “A Generalist Agent” in arXiv:2205.06175v2 [cs.AI]. 4 Puppe, F. (1993) “Characterization and History of Expert Systems” In: Systematic Introduction to Expert Systems. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-77971-8_1

The Structure of Artificial Rationality

155

processes, since my purpose here is not to develop a philosophical argument but instead to offer a practical suggestion, that, eventually, might even happen to inform, backwards, the philosophical debates on the nature of human rationality. Marcus5 and LeCun6 were among the first to dispute De Freitas, on different grounds and in different directions. LeCun disagrees about AGI on a fundamental level and argues that AGI is not and could not be well formulated, let alone achieved technically. The next best thing, according to him, is HLAI or human-like AI and our actual progress is only to be expected in the direction of HLAI, not AGI. The main challenges before HLAI are the lack of generalized self-supervised learning, no learning paradigm for machines to learn how “the world works”. I agree with him that it is best first to attempt HLAI, but I also believe, unlike LeCun, that HLAI can lead to AGI. The argument behind this is fairly trivial: if a true HLAI is artificially emulated, it would be trained on virtually all human knowledge and there is no reason whatsoever to think that it would not learn superhuman abilities in knowledge but also human-like rationality. From DRL systems like AlphaGo, we already know that AI can find better “moves” than humans have managed to for millennia. There is no obvious reason why the DRL model that manages to reach HLAI would not do the same with rational human abilities. Marcus, following his long defended positions, that step on the shoulders of AI giants, in his own words, like MacCarthy, Minsky and Simon, among others, directly argued against the thesis (3). Following upon his longstanding view that understanding the mind even at a high level is a necessary prerequisite for success in AGI and the impressive results of artificial neural networks (including in deep reinforcement learning models with neural networks for value and policy learning functions) could not alone achieve this goal, but should also be complemented with what he calls, following a

5 Markus’ position is developed here: https://garymarcus.substack.com/p/the-newscience-of-alt-intelligence?s=r 6 Matsuoa, LeCun et Al. (2022) “Deep learning, reinforcement learning, and world models” in Neural Networks, Volume 152, August 2022, Pages 267-275.

156

Chapter IX

tradition from some of AI's founding fathers, «symbol manipulation».7 He rejects the claim that what we take as artificial intelligence in its, arguably, more modest human-like AI form, versus the disputed immodest version as artificial general intelligence, is now solved and reduced to the effectively technical issue of scaling and related technical quantitative problems: «Alt Intelligence isn’t about building machines that solve problems in ways that have to do with human intelligence. It’s about using massive amounts of data – often derived from human behavior – as a substitute for intelligence. Right now, the predominant strand of work within Alt Intelligence is the idea of scaling. The notion that the bigger the system, the closer we come to true intelligence, maybe even consciousness.» 8

Marcus argues that not just De Freitas provocative statement that AI in its HLAI form is being reduced to the above alternative intelligence, but also that our best and most successful systems, like the recent GATO and Flamingo,9 express nothing else but alternatively intelligent capabilities and not at all human-like intelligent capabilities, least of all, generally intelligent capabilities. The core of his argument can be unpacked in the following form: 1. Artificial (General) Intelligence should mirror human intelligence (HI) 2. Key for HI is the (human rational) ability to solve problems 3. Current DANNs and DRL systems do not mirror human ability in (2) 4. Therefore, they are not and could not be human-like A(G)I The «alternative» intelligence systems, no matter how successful in practical tasks, all share the notorious problem of unexplainability, where any functional success is driven by a computational black-box. And not just on the parameter level, for example, with concrete optimized values of ANNs weights and biases, but on the general architectural level, which 7

Marcus, G. F. (2001). The Algebraic Mind: Integrating Connectionism and Cognitive Science. Cambridge, MA: MIT Press. 8 In https://garymarcus.substack.com/p/the-new-science-of-alt-intelligence?s=r 9 Jean-Baptiste Alayrac, Jeff Donahue et al. (2002) “Tackling multiple tasks with a single visual language model” in https://arxiv.org/abs/2204.14198

The Structure of Artificial Rationality

157

concerns the very structure of the AI model: the kind of networks chosen, the type of their integration, the learning algorithm, the activation function, etc. If a model like GATO or subsequent more successful and more powerful multimodal «generalizing» agents pretend human-like general intelligence they need to prove that the intelligent functionality of the model mirrors intelligence abilities of humans. Short of that, I am afraid that even Markus' alternative intelligence term would be too beneficial for such systems, since our only scientific criteria for intelligence are our criteria for human intelligence. Systems pretending general human-like intelligence would not only have failed to prove that they are human-like intelligent, they would have failed to prove that they are intelligent at all and thus they would be at best, only considered «alternative». Here I suggest that the leading ANN and DRL technologies are not to be opposed with syntactic (but also semantic and reasoning AI models) but, on the opposite, their power can be harnessed to drive such models in order to learn human-like abilities of syntax proficiency, semantic proficiency, reasoning proficiency and knowledge proficiency or all the kinds we find in intelligent human practice based on natural language (NL).

“Solving Intelligence” Approaches Intelligence, historically and scientifically, has always been related mainly to the human ability to understand (as clear in the term from Latin verb intelligere – to understand, to comprehend). While this ability certainly does not exhaust the semantic load of the term we cannot and we should not ignore it when attempting an A(G)I architecture. Another notion, implicit in intelligence, but quite present in its semantic explication, is the notion of reasoning, the ability to draw conclusions from knowledge, to use the principles of logic to further it and thus to justify real life choices. Together, arguably, they can deliver the content of the meaning of the term rationality, as found and as sought for by AI in the expression human rationality. Thus, for a proper human-like AI to be intelligence at all it would need to demonstrate the ability to understand, both expressions formed in natural language and real world situations (as opposed to simulated virtual situations); that is, to identify and artificially comprehend the meaning of expressions, texts and real life conversations. The already notorious

158

Chapter IX

LaMDA10 model ability to intelligently respond to such expressions and to adequately participate in such conversations is not an identical ability. The AI model should also be capable to reason on the basis of comprehended meaning, to draw analogies and inferences, and to perform all reasoning actions humans do, such as drawing logical inferences and following logical laws and rules of introduction and discharge of logical connectives, as found in predicate logic, a powerful system that can formalize with rigor expressions in natural language. It should also be able to reason with respect to making a decision and taking an action, that is, to found decision and action on reasoning. Such human-like ability would justify the behavior of the model as truly rational and intelligent as opposed to random, chaotic and unjustified. The black-box behavior of available models exhibits no intelligent reason to justify an action before all alternatives in a certain available set of actions. What they have instead is unexplained computational functionality that cannot be rationally understood, that is, in the light of the intelligere term, it cannot be accepted as intelligent. The AI model would also need to reason causally, among other things. It would need to understand the usage of the human terms cause and effect and to learn to use them in the same way. And at the end, it would need to unite all those abilities in a coherent unity, such as we find in a human's conscious rational mind. Only then would we have grounds to assess it as a humanlike artificial intelligence, if not in structure, at least in function. And that would be considered a success by the interdisciplinary community that slowly erects the complex theoretical foundation of human intelligence, which could only serve as a basis for its artificial counterpart, unlike the black-box model. The alternative would be to formulate virtually ex novo a novel, non-human intelligence and to the best of my knowledge, science knows much less about such intelligences than it does about the human one. All of the above considerations historically and scientifically fall under De Freitas', sadly, derogatory expression «philosophy of symbols». Here, I believe, we see a mistake of a category type. To accept that computational

10

Thoppilan, R. De Freitas et al. (2022) “LaMDA: Language Models for Dialog Applications”, in https://arxiv.org/abs/2201.08239v3

The Structure of Artificial Rationality

159

models, run on classical or, increasingly, quantum computers, can emulate the biophysical processes in the human brain that bring about what we observe as intelligent abilities and intelligent behavior is one thing, being limited physicalism as it is. But to identify the former with the latter would be the same as to affirming that what we observe as intelligent abilities and behavior are a chosen set of brain processes. This, naturally, would be absurd, besides being plainly false, for it is a plain fact of life that humans not merely do not observe such processes, let alone during communication, but never take them as the intelligent ability or the intelligent behavior themselves. What we observe is the intelligent behavior that runs on top and in some, still unknown, relation to those processes. It is therefore a much better justified model to attempt to model human intelligence not on the driving electro-chemical, that is, physical processes, effectively emulated by the function approximation of neural networks, but on the acting functionality of human rational behavior, much of which is self-conscious and not physical per se. The oldest and, arguably, the richest scientific probe into the nature and structure of human intelligence, as comprised by understanding, reasoning and self-awareness, is the western civilization's endeavor of philosophy that has developed immensely complex and critically evolved debates on the problems of semantics, reasoning, rationality and consciousness. This paper follows in the steps of this tradition, which, I believe, can be immensely useful for the practical purposes of arriving at a functioning general artificial intelligence. This approach, to structure an AGI model, based on philosophical doctrines, is surely unusual and highly unpopular in today's exclusively computer science based projects, which loosely take mathematical models of human neurons (like the ANNs) and models of human-environment interaction (like the reinforcement learning model) as sufficient to emulate the yet «unsolved» riddle of human intelligence. At the end of the day, however, all approaches would reach the same judge and the same field of assessment: the one that works first or best would undoubtedly be considered a valuable if not the valuable approach. Thus, the philosophy, mostly in its analytic line, faces an opportunity that is rarely if ever present, to prove success in a practically measurable setting, such as the AI and AGI race.

160

Chapter IX

Symbolism Unpacked The generally quoted «symbolism», however, both in pure mathematics and in logic, but also in written natural language, always comes in two parts: syntax and semantics.11 Computation, in its forms of, say, functional analysis, linear algebra or tensor calculus, is impossible without syntax and equally impossible without the semantics that interprets expressions in this syntax as well formed, as true or false, but also as meaningful. If the last seems like too hard a requirement for a mere purely mathematical expression of the form f(x)=y let us compare it with an alternative ()fyx=, which contains the exact same symbols and the same number of them. Where the two «expressions» differ syntactically is only their order. The «symbols», to use De Freitas preferred term, are the same. They are just not ordered in the same way in the two symbol strings. And yet the difference is highly non-trivial, for their meaning, and for their ability to be interpreted in truth-value. For the first is a well-formed formula of a function and as such is mathematical, whereas the latter is not; but is a meaningless and uninterpretable string of the same symbols not even capable to be approached with truth or falsity. And computations, such as those we use in ANN and DRL models, can only use the former kind, whereas the latter is certainly of no use not just for computer science but also for the whole of mathematics; symbolic gibberish can only figure with some valuable significance in science fiction or in a modernist poem. What is of key importance here, is that the syntax and its interpretation are inseparably intertwined, even if we decide to take a slight historical detour on Hilbert's formalism school of thought. From Hilbert, we know that any choice of syntax, of symbols and order assigned to them, could be, of course, arbitrary.12 But once chosen, it ceases to be arbitrary and begins to follow formal rules and principles and to require ultimate rigor in its writing in order to be semantically interpretable, that is, to be formally functional and thus significant and meaningful. The semantic interpretation of any

11 Van Fraassen, Bas C. (1971) Formal Semantics and Logic, The Macmillan Company, New York. 12 David Hilbert (1984) “On the infinite” in Philosophy of Mathematics, Edited by Paul Benacerraf and Hilary Putnam, pp. 183 – 201, Cambridge University Press.

The Structure of Artificial Rationality

161

syntactically introduced in a formal system symbol produces its functionality; the functionality enables the emergence of significance, meaning and truth. They enable the ability, familiar from logic of natural language, to reason with semantically interpreted syntactic expressions. Once humans have proper syntactic terms, like terms in English, they can interpret them semantically, that is, they can understand their meaning and they can judge their truth. On the basis of these interpretations they can attempt logical reasoning with them, like drawing an inference from a set of previously available semantically interpreted linguistic expressions. This ability, characteristic of human rational practice, is a conditio sine qua non in any artificial model that pretends human-like intelligence. If we cannot interpret the numeral 9 as the integer number «nine» we have no rational access to the object of the term, the number, if we assume «9» is the only symbol in the system reserved for it. Thus, the dynamic unity of properly interpreted good syntax is essential for all formal systems, the whole of mathematics included. Mathematical formulae have value only in virtue of their understanding. Mathematical proofs, that elevate conjectures to theorems, require an utmost rigor of understanding the formulae, the relations that hold among them as well as the logical rules, including the rules of inference. Those rules allow the rigorous transition from one formula to another in a manner that would furnish the totality of the steps where at the end we reach a theorem, which emerges in virtue of the mathematical proof and not in virtue of arbitrarily concatenated symbols, let alone uninterpreted ones. The «symbol manipulation» covers both syntax and semantics and is highly non-trivial for the most rigorous domains of sciences, mathematics and logic, and the fact that neural network-based computation simply vanishes without both does not help much to oppose Markus' point. In what follows, I will unpack my reading on what is needed behind the «symbol manipulation» to reach a structure of neural network differentiation, both at high, functional level but also on computationally – architectural level. Symbols can only provide a material for order, which needs to be semantically interpreted, in truth-value and in meaning, so that it can be logically reasoned with in order to reach inferential power. Arguments can be formally valid but, upon actual world data training, also

162

Chapter IX

materially sound. Therefore, much differentiation is needed beyond mere symbol manipulation, for only a set of distinct syntax, semantics and reasoning modules, functional after successful training, can begin mirroring the rational practice of humans, as found in everyday life. The two main characteristics of the environment of operation of human rational agents, most generally defined and devoid of any content, are its immense complexity and its notorious resistance to knowability. Combined, they render the modern state of the art approaches in AGI research defocused and ineffective from the stage of very beginning, which is the formulation of the task for which AI algorithms need to be proposed, trained and assessed. It is safe to note that even the most successful results of world leading AGI teams, such as DeepMind and Open AI, focus initially on very specific tasks, such as chess or Go superhuman abilities, with the expectation that the algorithms would eventually generalize to much different and broader tasks, eventually reaching an AGI scale generality. Perhaps the most successful system along this path has been DeepMind’s recent MuZero algorithm,13 which harnesses the learning capabilities of deep reinforcement learning in an unprecedented way not just for specific task supremacy, but with demonstrated potential for practical implementation and success retention in general domains as industry and, eventually, AGI. The world leading researchers and developing teams, however, do not share the same understanding of what would be a successful structure for an AGI system and what are the most adequate methods to approach it. Perhaps the best illustration of this is the recent debate between De Freitas and Markus, who defined very clearly the two leading stances.

Modern AI Technologies that Approach Human-Like Intelligence The nature of the tasks before the AI model determines the scope of technologies available for implementation to successfully perform the task. The modern field of Deep Learning (DL) is highly successful in acquiring the ability to learn from enormous amounts of data, mimicking in a very 13 Schrittwieser, J., Antonoglou, I., Hubert, T. et al. (2020) “Mastering Atari, Go, chess and shogi by planning with a learned model” in Nature 588, 604–609.

The Structure of Artificial Rationality

163

restrained way how the human brain functions. What is characteristic of the DL technology is that it is able to be fed with large volumes of previously unstructured data or largely unstructured data, and then process it via a dynamically structured architecture of neural networks, with input layer, which receives the data, hidden (or deep) layers, that process the data sent from the input layer and an output layer that classifies or predicts the desired output. DL is able to classify and suggest predictions with enormous accuracy and precision. The supervised learning (SL) technology essentially manages to arrive at a complex function that maps input data to output data after a process of training that is performed on categorized and previously labeled data. The technology is enormously successful in classification tasks. The technology of unsupervised learning (UL) is a different instrument that can be used to discover patterns and ordered volumes of information within large amounts of completely unstructured data. The UL is integrable with both the SL and RL technologies as of particular interest is the integration of UL with the environment layer in RL, where UL can be used to enhance and update in virtually real time the environment dataset, which plays a key role for the RL success. Thus, in a general RL system where the Environment, observed by the agent, constantly develops and changes, functionality that the AI architect would very much desire is the environment to evolve novel features and agents, previously unfamiliar to it. The novelty and unfamiliarity of such features can be introduced by the simultaneous work of an UL system serving this purpose. The system could be devised to observe actual environment and discover novel patterns in it, to categorize them, as environment features or as novel agents, and then introduce them into the RL environment layer, thus making them susceptible to be observed by the RL agent, functioning in a Markov Decision Process (MDP)14 setting.

14

MDP analizes tasks in terms of agent, environment, observations and actions, for an in-depth discussion see Olivier Sigaud and Olivier Buffet (Eds.) (2010) Markov Decision Processes in Artificial Intelligence, Wiley.

164

Chapter IX

Finally, the technology of Reinforcement Learning (RL) uses the MDP architecture in order to approach, structure and perform its tasks. The key modules in MDP are environment, agent, states of environment, actions of the agent and rewards. The RL technology is extremely successful in a number of famous cases, by and large goal-oriented tasks, as illustrated by the supremacy of RL systems in game-like settings and tasks. What is of particular interest for the AI architect here is that most real life situations that intelligent humans function in are very much akin to game-like scenarios. Thus, the success of RL up to date is very informative and suggestive of further success of RL in next stage complexity tasks, approaching real life scenarios of humans in everyday human functioning environments.

Deep Reinforcement Learning Model for Human-Like Rationality (Raison)15 The enormously successful approach of DRL analyzes the task within the MDP framework of an agent, operating via observing states of an environment and taking actions with quantified success toward a final predefined task; the model is driven by reward feedback. Eventually, the agent develops an improved policy of actions, which can be generally understood as the strategy to determine better next actions in order to reach the final task. The policy is a mapping of observed environment states to an agent’s actions, which provides the best “justified” option for a highest reward stimulus. The integration of deep neural networks into reinforcement learning proved immensely fruitful and particularly in scenarios where the spaces of the agent’s actions and the states of the environment are too large to be operated effectively within by a pure, non-neural reinforcement learning system. In DRL, neural networks can approximate both value and policy functions in RL and typically receive at input layer both value related data and agent’s policy-related data.

15

The RAISON name of the model is devised to accent on the English word reason, as one of the key components of the suggested model of human-like intelligence, the other being understanding, and contains the AI abbreviation for obvious purposes.

The Structure of Artificial Rationality

165

The RL and neural network modules have been successfully integrated with two additional AI methods: self-play (SP) and tabula rasa (TR) training, that is, training with no previously provided data sets. The potential of DRL systems integrating SP and TR (DRL-SPTR) was first demonstrated by DeepMind’s algorithms AlphaGo Zero, AlphaZero and most recently MuZero and LaMDA. What is truly remarkable in MuZero is the novel ability of the system to learn the rules of the game by itself and then to develop the ability of superhuman performance in the recognized game. The unprecedented success of DRL-SPTR systems is by itself a reason to investigate their prospects for human-like intelligent tasks, such as natural language comprehension and natural language based logical reasoning. But it is the functionality framework of DRL-SPTR that reveals an AGI potential given the characteristic nature of human-like AI tasks, like machine emulation of NL-based comprehension, reasoning, communication and problem solving, among others. Here I suggest that a DRL-SPTR system has the specific potential to formalize syntax tasks, semantic tasks and logical reasoning tasks. The reasons to expect that a DRL-SPTR system would have an AGI potential are much broader than the mere specific task success demonstrated by DeepMind’s and OPEN AI’s algorithms. First, the general RL setting, where an agent operates in an environment, is effectively isomorphic to the real life setting where a rational human agent operates in a real world environment. The agent-environment and the respective observation, states, and actions reflect very adequately the rational process where a rational human observes the world, attempts to comprehend it, attempts to reason about it, on the basis of comprehended observed information about it, formulates actions and decides to perform them on the basis of reasoning, which provides justification for both the type of the action (drink water) and the reasons to undertake it (dehydration needs to be eliminated out of self preservation). The reward stimulus is exceptionally evolutionary in its structural functionality. And the self-play method of learning, while uncharacteristic of human rational agents, is extremely productive in simulating human rational agent interaction with the environment.

166

Chapter IX

While the domain of artificial intelligence is highly interdisciplinary and encompasses fields such as mathematics, computer science, neuroscience, cognitive science and psychology, the history of “solving intelligence” is much older and richer than the history of all of these scientific disciplines, with the exception of pure mathematics. My stance in approaching the task, today described as “solving intelligence”, comes from the tradition of analytic philosophy, where the problems of natural language based comprehension, logical reasoning and normative justification have an intense, deep and very rich history. It is an assumption, both in analytic philosophy and in this work, that human rational activity flows at their level of functionality through the framework of human natural language, such as English. Within this tradition, human intelligence is generally grasped as the integrated set of necessary conditions of human rational abilities: NL mastery, NL understanding, both in syntax and semantics (that operate successfully in real life scenarios, including problem solving and NL communication with other human agents), NL based reasoning, founded on logical rules, such as rules of introduction and discharge of logical operators, as well as logical laws such as modus ponens and modus tollens, syllogistic reasoning and use of acquired knowledge in reasoning.

Abilities: Syntax, Semantics, Understanding, Reasoning and Rationality The AGI tasks, due to their distinctive nature, can be addressed by different, ad hoc implementations of the DRL-SPTR systems. Each task would emulate a distinct rational ability that is accepted as a component of humanlike intelligence and would have a dedicated DRL-SPTR system, which learns it. This ability would be acquired by the agent in the DRL-SPTR system after DRL training. The syntax ability (SYA) should consist of syntax structures for recognition and classification and it would need to be integrable with the semantic and the reasoning systems. To illustrate, if we want the syntax ability to be trained in 2nd order predicate logic, it would need to recognize 2nd order well formed formulae and differentiate them from non-wwfs in the same logic. This ability, albeit formal, is essential for human-like NL abilities and would

The Structure of Artificial Rationality

167

allow systems for semantic interpretation and logical reasoning to be trained. The semantic ability (SA) is essential for natural language comprehension and communication. For our purposes, we would differentiate between semantic interpretation in truth-value (SIT), which provides the ability to interpret NL expressions with respect to truth and falsity, and semantic interpretation in meaning (SIM), which provides the ability to interpret NL expressions according to meaning. For the latter, I choose Gottlob Frege’s influential distinction between sense (semantic interpretation in sense - SIS) and reference (semantic interpretation in reference - SIR) as functionally powerful enough to solve the task in real life scenarios. The reasoning ability (RA) will represent human’s real life practice where we reason, for our concrete purposes and along the main thesis of analytic tradition, within the natural language.16 Generally, this is the ability to operate with syntactically well-formed linguistic expressions, to interpret them semantically in truth-value, in sense and in reference, and on the basis of these to introduce and discharge connectives for well-formed formulae (atomic and molecular) as well as to concatenate them into larger sets of expressions. The RA system would learn to construct ordered chains of expressions following the logical forms found in human practice: the logical laws, such as modus ponens and modus tollens and many others. Most of all, the system would acquire the ability to reason syllogistically, to draw logically valid conclusions from available premises and to assess logical arguments as valid, invalid, sound or not sound. Human intelligence, when approached within the general framework of indo-European natural languages, limited in RAISON to English, necessarily contains the ability to comprehend (AC) the meaning and references of well-formed linguistic expressions and the ability to reason with them (RA). Comprehension and reasoning are distinct, albeit dynamically intertwined abilities, that we cannot fail to find in rational 16

Central thesis of analytic philosophy accepts that all reasoning occurs within NL – for a discussion see Martinich, A. P. and Sosa, David (eds.). 2001a: Analytic Philosophy: An Anthology, Blackwell Publishers, and Martinich, A. P. and Sosa, David (eds.). 2001b: A Companion to Analytic Philosophy, Blackwell Publishers.

168

Chapter IX

human behavior, be it on the level of its description or on the levels of its explanation and justification. Therefore, I accept that human-like intelligence contains as necessary components NL understanding and NL reasoning. The fourth necessary condition of human-like intelligence could best be defined as rational ability (RAA), which endows SYA, SA and RA with actual knowledge (AK) about the real world. Thus, in the hypothetical case where an integrated system (SYA+SA+RA or SSR for short) functions without training in actual knowledge 17about the different domains of the actual world,18 it would not be able to introduce true premises into its arguments. Thus, the arguments would at best be valid arguments, but not sound ones19 and the comprehension and reasoning abilities of the system would be exclusively limited to the form of the expression and not to any practically useful semantic load that is necessary for real world comprehension, real world reasoning, communication and problem solving, as found in human practice. Therefore, in order to obtain human-like rational ability (RAA) we must train an SSR system onto actual world knowledge (@WK) and here I strongly differentiate between mere data and knowledge, where data is raw unstructured information with no or virtually no practically useful connections of order between its elements. For knowledge, I see no better theory of epistemology,20 both for expressing the nature of human

17

Using the symbol “@” for “actual” as in @W or “actual world” Such as scientific facts from biology, physics, geography, history, etc. as well as behavior facts about history and possible explanations of human behavior, provided by psychology, cognitive science and most of all, non-rigorous common sense, found much more frequently in real life scenarios than scientifically rigorous facts. 19 Sound arguments are valid arguments with true premisses and possess the highly non-trivial value to transfer not mere logical validity from premises to the conclusion, but to transfer truth of premises onto truth of conclusion, thus making them immensely important for scientific purposes much transcending the formal rigor of pure mathematics and logic. For an overview see Etchemendy, J. (1999) The Concept of Logical Consequence, Stanford: CSLI Publications. 20 Epistemology is one of the oldest disciplines of philosophy and it deals with all problems of knowledge: its nature, its kinds, criteria for, etc. It is my assumption that analytic theories of knowledge are both the most promising ones and most suitable for the practical purposes of building AGI models. For a high-level 18

The Structure of Artificial Rationality

169

knowledge and as a basis for practical AI success in emulating human-like knowledge into artificially intelligent systems, than the leading analytic theory of knowledge of knowledge as justified true belief + x (or JTB+). In JTB+, the cognitive agent knows that P (the NL expression) if and only if he is justified to hold that P (justification or J), he actually is in a state of holding it (artificial belief or AB), and P is true as guaranteed by our SSR system. The exceptions, denoted by the “+”, and proven by Gettier cases 21, showed that the J, T and B conditions are necessary but not sufficient conditions for knowledge. Non-Gettier cases of knowledge, statistically dominating human practice, have very high survivability against counterfactual attacks, but still need to be supplemented with additional conditions where different positions in the debate offer different candidates.22 For our practical purposes here, I will ignore the “+” condition and I will accept that modeling RAISON on the JTB set of necessary conditions for knowledge is still a very significant success. A model that would emulate the sufficient “+” condition, could be further developed, once analytic debates in epistemology establish an accepted “winner” that solves the Gettier cases. In this way, I hope it is clear that only a @WK trained SSR system can become artificially rational and truly artificially intelligent. The SSRR system (syntax, semantics, reasoning and rationally trained system or SR2) would be an integrated AI system that exhibits human-like abilities to comprehend expressions, reasons using them and which knows, like humans, via @WK “educational” training in AI training datasets, like the WiKi Corpus, facts about the real world. But most of all, it would be able discussion see Ernest Sosa (2018) Epistemology (Princeton Foundations of Contemporary Philosophy, 14) Princeton University Press. 21 Gettier famously demonstrated that the JTB formula only provides the set of necessary conditions of knowledge and not the complete set of conditions for knowledge by constructing, statistically rare but epistemologically significant Gettier cases where cognitive agents satisfy J, T and B conditions and yet fail to possess knowledge, where their state is generally attributed to the so called “epistemic luck”. The original argument is presented in Gettier, Edmund (1963). “Is Justified True Belief Knowledge?”. Analysis. 23 (6): pp. 121–123. 22 For a discussion on Gettier solutions see Lycan, W. G. (2006). “On the Gettier Problem Problem.” In Epistemology Futures, (ed.) S. Hetherington, Oxford University Press.

170

Chapter IX

to form sound logical arguments and communicate via them. The AI ability in SR2 would represent a successful operation of human-like intelligent abilities in the actual world. Such ability is arguably the best candidate to be a decisive criterion to judge the intelligence of humans - it could not be mere communication between humans, as presented in the classical Turing test setting, but it should be the evolutionary environment of successful functioning, survival, adaptation and evolution of rational humans in the real world. A fully functional artificial agent, like RAISON, acting in this environment without being identified by humans as an artificial vs. human agent would be a much stronger version of the Turing test than the original one. Only recently in history has humanity discovered (or developed, or both) the domains of pure mathematics, logic, human natural science, art and ethics. They are an essential part of the actual world humans operate in since millennia and even if perhaps inaccessible to other living species, like plants and animals, or other intelligent species, like animals and AI run robots, they further and enrich the evolutionary environment of homo sapiens and therefore they need to be included in the @WK training of the SR2 system. The data for the DRL system training would need to allow its successful training and this is a non-trivial task. It would need to be mathematically encoded as n-dimensional tensors, as popular ML systems of language encoding of the word2vec 23kind would lack expressive power and sufficiently powerful integrability with semantic spaces like SGS. Without these properties, however, traditional encodings would be unable to serve their AGI purpose; the necessary mathematical expressive power can be delivered only by general n-dimensional tensors and thus the word-level NL encoding would be a word2tensor (or w2t) encoding. The necessary encoding for the DRL-SPTR system would need, on the one hand, to provide the specific encoding for each system (SYA, SA, RA and RAA): syntax2tensor (SY2T), semantics2tensor (SE2T), logic2tensor (L2T) and actualknowledge2tensor (@2T), but on the other, the dimensionality and the 23

Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean (2013) “Efficient Estimation of Word Representations in Vector Space”, in https://arxiv.org/pdf/1301.3781.pdf

The Structure of Artificial Rationality

171

shapes of each tensor should provide mathematical integrability between each encoding. The SY2T would encode the syntax well formed formulae of words and expressions. The SE2T would encode the graph content of terms and expressions that are the results of the three semantic interpretations, in sense, in reference and in truth-value, that comprise the semantic interpretation in meaning. The L2T would assemble semantically interpreted SY2T formulae into argument premises according to the rules of introduction and rules of discharge of logical connectives. The @2T would encode the interpretation of expressions with respect to actual world knowledge. The semantic space SGS that would enable the SA system but that would also host the @WK training data would best be formalized as a highly expressive graph space with operanda, syntactic and semantic, as nodes and operations as edges, where the 2nd order logical functionality would be introduced by allowing edge to edge connections that would represent predication of predicates, but would also allow for the natural formal expression of universal and existential quantifiers. Expressions of all kinds in the SR2 system, linguistic, syntactic, and semantically interpreted, would be graphs and graph-chains that would be attributed a number of indices, which would express the different systems’ properties and roles. In SGS, we introduce a syntactic index for well-formedness, possessed by graph g and not, say, graph e, a semantic index that expresses sense and another semantic index, that represents reference (pointing to the graphs that express the reference and negating a number of graphs that oppose this reference interpretations), logical index for a graph that is a valid premise in a valid argument A and a logical index for a graph that is a premise in a sound argument S. Sound arguments thus would be ordered sets of connected graphs in SGS and sound reasoning would be an ordered set of graphs where every graph represents either a true premise or a true conclusion. In the suggested DRL HL-AGI model RAISON, all of the above abilities would be abilities of the agent who would operate in an environment segmented in syntactic layer, semantic layers, reasoning layers and actual world data trained or educated layer. The model needs to learn a high-level structure that emulates the function. The elements of RAISON are

172

Chapter IX

integrated in the general space of the DRL model, which consists of an agent, environment, states of environment, actions of the agent and a reward signal. We also highly desire the agent to be capable of learning from selfplay, just like the immensely successful DRL models of AlphaZero and MuZero demonstrated.24

The DRL Structure of the Syntax Ability In reinforcement learning the deep neural network (DNN) integration can be extremely useful in cases when either the state space of the environment is very large, or when the action space of all available to the agent actions is very large; or both. For our concrete purposes here we need first to identify how the syntax ability could be modeled in the DRL setting and, in particular, how the deep neural network can serve in this task. In the general case, the DNN can be learned to map environment states to values, in the role of value function approximator; or it can learn to map state-action pairs to quality or Q values. In the case of the syntax ability we need first to identify what would be the syntax environment of the agent, what would be the states of the environment and what be the actions of the agent in it. The very agent would be a syntax agent and his agency role in the general DRL model of the HL rational agent would effectively be a one of a syntactic avatar which is integrated with the general agent and mediates between it and the syntactic environment. The environment would be the complete space of available ordered syntactic elements, like well-formed formulae and non-well formed formulae. However, since the syntactic function is inseparable in the integrated functionality of the AGI-DRL agent, and since in real life scenario humans never face a syntactic only environment, but face a naturally linguistic environment of expressions, in order for the syntactic agent to contribute to the DRL agent in a human-like way he would need to

24

See David Silver, Thomas Hubert et al. (2017) “Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm” in https://arxiv.org/pdf/1712.01815 and Schrittwieser, J., Antonoglou, I., Hubert, T. et al. (2020) “Mastering Atari, Go, chess and shogi by planning with a learned model” in Nature 588, 604–609.

The Structure of Artificial Rationality

173

face the same linguistic environment as the one the DRL faces. Because its task is only syntactic, upon observation of a fully linguistic environment (FLE – an environment of all available NL expressions that follow syntactic rules, receive semantic interpretations and participate in reasoning algorithms), that is, an environment of expressions such as the one humans face in real-time dialogues, speeches, or readouts, the syntactic agent would need to extract only syntactic features and structures from it. The approach to construct a text environment within a deep reinforcement learning model is certainly not novel. Ji He and al. used a similar approach for constructing the novel deep reinforcement relevance network (DRRN) for evaluating its performance in text games.25 They note the immensely large size of the state space times action space table for “vanilla” Q-learning recursion for any such pair of large spaces. He at al. use two DNNs, one for “state text embedding” and the other for “action text embedding”. We see that in He’s setting, both the states of the environment and the actions of the DRL agent are represented as text expressions in a natural language. The agent-environment interaction in their DRRN model is performed by the sequence of observed text states and executing an action text-state that changes the text-structured environment. The agent needs to learn the most relevant responses, action-texts, which, set in a text-game model, are generally analogous to a human-computer interaction, including the one we see in a classical Turing test setting. I find this approach generally applicable to our technical task of embedding in a syntactic DRL setting, where we will embed logical formulae as states and actions. If we aim to model an episode of syntactic agent interaction with the text environment in a text-game-like scenario where the DRL agent listens to and talks to another agent, human or artificial, the environment should consist of all linguistic expressions rendered during this episode and the “rules of the game” would be the totality of syntactic, semantic and reasoning rules. For the syntactic goals of the agent, the syntax of the logical

25 Ji He, Jianshu Chen, Xiaodong He, Jianfeng Gao, Lihong Li, Li Deng and Mari Ostendorf (2016) Deep Reinforcement Learning with a Natural Language Action Space in: arXiv:1511.04636v5 [cs.AI] https://doi.org/10.48550/arXiv.1511.04636 , last accessed 17.06.2022.

174

Chapter IX

form of the expression-state would need to be extracted by formalization. This task contains the trained ability to recognize the form, which could be performed by the syntactic DNN. The input of the network would be the observed state-expression of the environment and the output would be its logical form. The logical form would thus be available to the general DRL agent for integration with semantic interpretations and reasoning operations. The ability of a syntactic agent would be as syntactically human-like as practically possible to emulate. It would need to be able to recognize wffs from non-wffs, it would need to recognize the type of a concrete wff, for example JLJ, as found in the linguistic expression “John loves Jane”. Besides this, the syntactic agent would need to deliver points of integration of the syntactic output with the semantic environment and the reasoning environment, with respect to the semantic agents and the reasoning agents, to be defined below. Thus, the agent would observe and act in a syntactic environment that orders all syntactic material, available for the game: a communication session, Turing set dialogue, or a history of such dialogues, analogous to the history of played games in the same way AlphaGo played with itself, via self-play, an enormous number of games. Thus, the syntactic material would form the state space where the agent would observe one state per observation and would act upon it via his action, that would change the syntactic environment and would provide the next syntactic state for observation. The syntax ability would be modeled by a deep neural network that would take as an input naturally linguistic expressions and would learn to recognize its logically formalized form solely on the basis of its syntax. In the DRRN model of He and al. they use two separate DNNs, one for the states of the text environment and one for the text actions, due to the expected significant difference between the two inputs in terms of complexity and length. In our syntactic model, we do not need to account for such differences and we can therefore use one DNN that would take both states and actions as inputs. Our initial choice for syntax would be 2nd order predicate logic, which provides immense expressivity, sufficient for formalizing scientific expressions but also, and more importantly, it mirrors human linguistic practice more naturally than other logical formalisms because it allows quantification and predication over predicates. Again, the

The Structure of Artificial Rationality

175

rigor of the system would be available to the DRL model, but on the surface, in its mirroring human behavior, expressivity would need to dominate over rigor.

The DRL System: Environment, States, Actions The space of states of the general RL environment would actually be ordered expressions. The order expresses the three kinds of relations that hold in NL expressions: the syntactic, the semantic and the logical. Inferences would be sub-states of selected by the reasoning engine premises that can or cannot contribute to conclusions, where falling short of reaching a conclusion on the basis of a given set of expressions, taken as premises candidates, would be substituted by a different but yet following conclusion. The conclusion would be of the qualificatory kind “given premises [p1, p2, … pn] I cannot conclude that x” or “given premises [p1, p2, … pn] I can conclude that y”. The environment in our setting would be the initially unordered set of all available text expressions to the agent. The DRL system would use a deep neural network with an input layer that would take as input data the state of the environment, which the agent would “observe”. In our case, this would be the equivalent to the “move” of the computer in this “game”: in a Turing set dialogue, the computer would utter an expression and that would be the move of the machine in a gamified setting. The move would be a novel and last expression, which appears in the environment and which would change its state. The agent would take the move within the general state of the environment as an input and would employ the neural network to output a tensor of “move” probabilities for his actions and a scalar value for the expected “outcome” from the new state of the environment which would include its action. The success of the agent response would be the adequacy between the agents’ response and the sequence of the observed state-response pairs. This adequacy would be optimized for syntax adequacy, semantic adequacy and logical adequacy, by the corresponding layers of the model. The action of the agent is also an expression and is analogous to the movement of the DRL agent in a typical game setting. The task of the agent

176

Chapter IX

would be to construct a response expression. The choice of the construction would be from the available options, but the response would be integration from the three layers: syntactic, semantic and reasoning. Thus, from the syntactic layer, the agent would have, say, a thousand options of well formed logical formulae. The best chosen one would be the one that receives the highest reward for syntactic response success. The semantic choice in sense would be a set of options derived from the semantic graph space with a special accent on the sub-space defined by the response of the machine. The reference choice would be from a set of possible options from the semantic space that would fix the references of the singular terms and the reference of the expression.26 The reasoning choice would be to construct the agent’s response as the one logically best related to the response of the machine response.

The Structure of Semantic Ability: Frege’s Model of Sense and Reference In the case of the semantic ability we need first to identify what would be the semantic environment of the agent, what would be the states of the environment and what be the actions of the agent in it. The agent here would be the semantic avatar of the general DRL agent and his agency role in the general DRL model would be semantic. Its output would be integrated with the other layers’ outputs by the general agent and it would mediate between the syntactic environment and the reasoning environment. The semantic environment would be the complete space of available ordered semantic elements, which would be loaded with the complex and highly responsible obligation to deliver the elements of natural language semantics and to execute the semantic interpretations of both text-states and text-actions in the general DRL model. An important qualification is due here. As it is evident across several scientific domains let alone across historical traditions, in particular the

26 Frege’s reference of an expression can best be formalized in DRL as a (sub)state of the environment. An English translation can be found in Gottlob Frege (1948) “Sense and Reference” in The Philosophical Review, Vol. 57, No. 3 pp. 209-230 (22 pages), Duke University Press.

The Structure of Artificial Rationality

177

tradition of philosophy, which deals with key semantics as meaning, significance and truth, there is no predominantly accepted theory of meaning and truth. No theory that has been both accepted by the research community in any involved discipline and successfully implemented into an AI system that even loosely resembles a semantically capable machine functionality. Instead, what we see everywhere, is an intense and heated inter-disciplinary and, much more rarely, a cross-disciplinary debate on the true theory of truth and meaning of expressions in natural language. Thus, an AGI architect can either choose to devote his entire academic life to participate actively in this debate, which would certainly prevent him from building an AGI system whatsoever, or attempt a much more practical approach and see what works best in implementation and use it as a justificatory, albeit practical, foundation to rise the AI implemented theory of truth and meaning in rank among the leading theoretical rivals, which, still fall short of delivering a successful AI implementation. To sum it up: if a certain theory of truth and meaning demonstrably performs on a humanlike level in a certain AGI system with Turing test potential, this could be used as a side argument to claim primacy among rival theories until they do not demonstrate equal or better results in a competitive system. In my research field of analytic philosophy of language, I chose Frege’s influential theory of sense and reference 27as having a high practically implementable potential in an AGI system. Here I would avoid completely any stance in the ongoing debates on it, which would have been a conditio sine qua non if the purposes of this paper were theoretical. My purposes here are, however, entirely practical: the semantic success of the actual model, when developed in code and trained on data, would become measurable. If the system proves successful, this could expectedly be used as a novel argument in the modern philosophy of language debates on truth and meaning.

27

For my stance on Frege’s theory of sense and reference I am very much indebted to my doctoral supervisor and mentor, Nenad Miscevic, as well as to Katalin Farkas and Howard Robinson from the CEU, and, later, to Gabriel Uzquiano and Ofra Magidor from the University of Oxford. I am also indebted to the late Sir Michael Dummett, whom I had the chance to listen to on a number of seminars at Oxford just a year before his passing.

178

Chapter IX

The semantic agent would interact with a semantic environment, where the states of the environment and the agent-actions are semantic. In order for the semantic environment to be as human-like as practically possible, it would need to provide recognition of semantic truth and semantic meaning. Human linguistic practice is devoid of truth determinacy and meaning determinacy because we almost never find in real life human practice truthrigorous expressions and meaning-rigorous expressions. In Frege’s theory of sense and reference, meaning of linguistic expressions can be analyzed in terms of sense and reference and this dynamic duality exhibits the potential of expressing a significant volume of real life human language interactions while having the potential for actual rigor in truth and meaning. The semantics in this layer is present in the observed text-state of the environment and to illustrate this let us choose a well-worn example from Frege’s discussion on identity statements: “Hesperus is Phosphorus”28 (ST). If we take this expression with a semantic focus, our semantic agent would need to perform the following semantic tasks: 1. To recognize the semantics of the individual terms in ST (Hesperus, is, Phosphorus) 2. To recognize the semantics of the statement in ST 3. To relate (1) and (2) to the semantics of the history of the environment (the dialogue so far or the actual world data “educated” part of the larger environment) 4. To relate (1) and (2) to the semantics of the history of the environment (the relevant semantic contexts of the intended usage of the terms and the semantic intention of the statement) These tasks are key to human-like reasoning via language and the first acts that build up the human understanding ability. In this case, the semantic observation of the text-state by the semantic agent would need to understand the meaning of the terms and then to understand the meaning of the intended statement of ST. The understanding of the terms would consist in identifying in our semantic graph-modeled space (SGS) the concepts of the terms and 28

Gottlob Frege (1948) “Sense and Reference” in The Philosophical Review, Vol. 57, No. 3 pp. 209-230 (22 pages), Duke University Press.

The Structure of Artificial Rationality

179

the connections with other concepts in other expressions that convey best the meaning they have in ST. Thus, the meaning of every term in SGS would be the node of the term (the node for “Hesperus” or the edge of “is”, as operandae are modeled on graph nodes and operatii are modeled on graph edges) and the graphs, which employ it in other expressions. The totality of actual expressions with the node-term would represent the totality of the human practice of the term. The sub-space of the usage of the term in typical contexts, which form a certain meaning of the term (any term can have numerous meanings, some of the very distinct and remote from others), are represented by the connected graphs of the expressions, where graphs are expressions in the form aRb where a and b, as operandae are nodes and R, as an operatio, is an edge. The SGS graph space would represent the semantic source available to the environment, in our DRL model, but also to the general RL agent. In order for any semantic agent to be able to perform a semantic interpretation, it would need access to the SGS. To give a cursory illustration of the meaning of the term, the name “Hesperus” in SGS, we can assume that SGS already has a trained structure and provides us, or the agent, upon a query, with the following list: 1. “Hesperus” – node, operandum, name 2. Total connected edges for graphs – a natural number n 3. Definition graphs: star, planet, physical, radiating light, rises in morning, sets in evening, etc. 4. Expressions: a natural number m On the basis of these expressions, the agent can retrieve the actual meaning of the term in human practice (we assume that SGS has been trained on actual human practice). Graph selection can serve to form the sense of the term and a ST or the sense of an expression. It can also serve to fix the reference of the expressions. After the semantic interpretations in sense and reference, the agent can perform the semantic interpretation in truth-value.

180

Chapter IX

Frege defines the sense as the content of the cognitive result upon understanding the linguistic expression by the human agent.29 The sense can be exchanged with a fellow conversant in the same language or translated into a different language. The sense determines the reference (the object or state of affairs) to which the linguistic expression applies in a desirably unique way. To use Frege’s own illustration with the identity statements, which have a simple yet immensely significant structure of two names or descriptions being connected by a single linguistic identity operator like “is” or “are”: if we consider the two different expressions “Venus is the morning star” (Vm) and “Venus is Venus” (VV) we observe that they differ significantly in their informative content even if they actually refer to the same object, the planet Venus. This and many similar cases raise a puzzle about meaning, which requires an explanation by a successful theory of meaning. Frege’s explanation distinguishes between two elements of the meaning of the expressions. The first one is the sense of the meaning, which, in this puzzle, carries the load of the difference in informational content, evident from the different syntactic structure of the expressions, which differ in syntactic content and order. The second one is the reference of the expression, which in the puzzle happens to be one and the same object. The identity in the meanings of both expressions is explained by the identity in reference, while the difference in the meaning of the expressions is explained by their different senses. The sense, if it manages to determine the reference successfully so that the act of referring is performed, would satisfy a necessary condition for the referring; but not a sufficient condition. There is nothing in the language practice that requires the conversant to formulate the sense in a way that it would require or guarantee the reference as an accomplished act. To illustrate, we might speak meaningfully, that is, with a well-defined sense, of quantum particles like the graviton, included in the standard model of quantum mechanics, and yet we will fail to fix the reference and thus to accomplish the act of referring, since science still does not know if gravitons exist at all. The act of sense formulation only manages to form a beam of semantic light that begins to search for its defined reference, in a process

29

Ibid.

The Structure of Artificial Rationality

181

that sometimes captures it, as in expressions like “John loves Jane” and sometimes fails, as in “The car’s engine released a lot of phlogiston” where there is no real object “phlogiston”, referred to by the term. In mathematical cases like “5+7=12” or “5+7=10”, which mathematicians do find nontrivially distinct, in terms of truth and meaning, the inquiry into the referents of purely mathematical expressions continues the heated debates even today, as philosophers of mathematics clash their preferred views within the frames of Platonism, structuralism, formalism, fictionalism and other influential stances. We need to distinguish between an objectively occurring reference, that is, a reference that has happened as an act and an intended reference, which is the reference the conversant has in mind, when she formulates the expression and encodes it, within her linguistic abilities, in the expression. The two are evidently not identical. An actual act of reference might happen despite the sense, due to an ambiguous encoding, for example. And the intended in the sense reference might land on a different, unintended object. For example, the conversant might say “The balloon is growing in size” intending to refer to an actual latex inflatable balloon. But if in a written context, for example, written next to a financial column in the newspaper, a reader might mistakenly understand it as referring to an inflation balloon. In these and similar cases the act of understanding of the reference by the listener furnishes the act of the referring of the expression and the term. This happens according to Frege’s context principle, according to which every single term receives its sense only within an expression.30 The listener takes the reference as an inflation balloon, whereas the speaker intends the term in the expression to refer to a latex balloon. We have one expression, one intention, and yet two references, one of intention and one of comprehension. Both are essential for successful communication, but only the second is a communicated accomplished referring act. We can observe that the sense can be formalized as a recipe for an operation and thus as the initial step of a semantic algorithm, where in a finite number 30

An illuminating analysis on Frege’s context principle and the closely related principle of compositionality can be found in Linnebo, Øystein (2008) “Compositionality and Frege’s Context Principle” in Philosophy.

182

Chapter IX

of distinct steps the semantic definition, interpretation and operation execution are performed. We can generally define the semantic algorithm as having the following form:

Semantic Algorithm 1. Intention of the speaker 2. NL encoding of intention following a syntactic selection of symbols and an order that holds between them with a semantic load of the terms 3. Formation of expression 4. Formation of intended reference 5. Utterance 6. Acquisition by a listener (or the computer) 7. Decoding of the symbolic structure 8. Semantic interpretation in meaning: decoding the acquired sense 9. Semantic interpretation in reference: attempting to comprehend the intended reference 10. Act of referring after (9) 11. Semantic interpretation in truth value: the listener is able to assess truth of the expression on the basis of (8) and (9) 12. End of communication cycle (1 - 11) We see that in the SA, the steps contain both operandae and operatii. This structure makes it quantifiable for the purposes of the DRL model. From the perspective of implementation, it is significant to observe that the Fregean sense of an expression can be expressed as a semantic graph in a semantic space where nodes express concepts-operandae and edges express concepts-operatii. Such formal expression can be particularly useful in the task of training the semantic agent, but also, to provide the formal structure, which formally expresses a non-trivial graph isomorphism between natural language and mathematical expressions. This space can also be especially useful for an agent's memory, as a training database and as a knowledge base, where it can serve for the actual world data training of the DRL agent. The semantic graph space can provide the expressibility, the necessary rigor

The Structure of Artificial Rationality

183

and the integrability with syntax, reasoning and rationality necessary for the “education”. Frege’s theory of sense and reference can systematically resolve ambiguity cases through its power to identify the reference of a term, the reference of an expression and an order that holds between the references of the primitive terms in the expression and across expressions. The theory is also capable of identifying well-formedness of syntax and semantics via sense formulation and via its figuring in an expression. It is also capable of identifying the act of referring as a separate stage process with distinct steps with measurable states: definition of reference, intention of reference, identifying the object of reference, interpretation of reference, incomplete act, complete act, wrong reference and successful or unsuccessful communication of the reference to a conversant. Each step and state of the term or expression can be identified and assessed. These capabilities provide a powerful semantic toolkit for artificially modeling the semantic behavior of humans.

Atoms of Reasoning The first and minimal syntactic act of reasoning is, formally and syntactically, the predicate attribution to a bearer of a predicate, a subject, or, in 2nd order predicate logic, a predicate as well. This predication is also the first and minimal expression. The syntactic consideration of a single syntactic element, of any kind (subject, predicate, relation or quantifier) is not sufficient to be accepted as the first and minimal syntactic act of reasoning because it receives a good syntacticity or a syntactic wellformedness only at the general level of predicate or relation attribution to an x that is not itself the predicate or the relation and yet can accept it and carry it. The availability of a single element allows for the first act of quantification, since we can define both existential and universal quantifiers over it and we cannot do it without the availability of an element. In order to be able to form the quantification “There is an x” we need the x to be present, for otherwise the expression could only take the form “There is”, denoted syntactically by the existential quantifier symbol and this is not a well-formed expression in logical formalism, which also, for that very reason, fails to be able to carry any meaningful semantic load. The

184

Chapter IX

introduction of quantifiers thus becomes first possible only after our syntax provides us with an x the quantifiers can range over. After the quantification, the element, however, is not a single element anymore: we have two elements, a quantifier and an x, connected in a syntactically proper way which allows their concatenation to be regarded as the formal structure of an expression. The expression represents a formal act of quantification, which, when considered as a schema, has the same formally general form as the schema of predication of a predicate to a subject. The first and minimal semantic act of reasoning is grasping (comprehending, understanding) the meaning of a linguistic term. In our model, based on Frege’s theory of meaning, that amounts to grasping the sense of the term and grasping its reference, or, rather, attempting to grasp the sense and attempting to grasp the reference, since nothing requires or guarantees that either would be successful and finished. The second semantic act of reasoning is grasping the meaning of an expression; and again in our model, this amounts to attempting to grasp the sense and the reference of the expression. But at the level of semantic interpretation of an expression, a novel possibility emerges, unavailable so far: an interpretation of the expression with respect to its truth-value. Again, this can only figure as an attempt, for in a real life linguistic situation, a truth-value interpretation can be any of true, false, undetermined, or unavailable as a procedure. The third and last act of semantic reasoning is the semantic ordering of an expression, along with its constituent terms and their senses and references, with respect to other expressions and their terms, senses and references. This order, in a text body of any kind, acquires its own sense and own reference, as well as its own truth value. This furnishes the general semantic structure of reasoning based on meaning, structured by sense and reference and complemented by a semantic interpretation in truth-value. The availability of two distinct semantic elements, carried by different syntactic strings, allows all logical operations as firstly possible: negation, conjunction, disjunction, implication, identity. The availability of elements and the possibility to execute logical operations over them allows wellformed linguistic expressions to be analyzed both syntactically and semantically. This schema also allows for the elements of reason to be identified, analyzed, produced, and exchanged in a conversation-like

The Structure of Artificial Rationality

185

setting. For example, the singular term “John” does not form a meaningful expression and cannot be regarded as expressing something beyond itself. By itself, it does not provide us with the possibility to quantify over it and to predicate it: we have no quantifiers and no predicate terms available. So no introductions of logical operators can be performed on “John” unless we, before that, make available a second term, like “sings”. The availability of two terms allows all of the above that can give the rise of a syntactically well-formed expression “John sings” that expresses something beyond what the singular terms express: it expresses an order that holds between them. This order is encoded by Frege’s sense and fixes the references of the singular terms and the expression. This is highly non-trivial and represents the emergence of the semantic meaning of an expression, as opposed to the semantic meanings of the two singular terms, “John” and “sings”, the expression becomes meaningful semantically. The term assembly allows, for the first time, the functioning of the elements of reason. For example, we can affirm that “John sings” and by the act of affirmation or mere utterance we make the expression available for semantic interpretation with respect to the meaning of the expression. We can examine its meaning in accordance with our preferred theory of meaning and see that the singular terms denote a person, John, who acts, as denoted by “sings”. This is minimally the sense of the expression, conveyed to the conversant and which also defines the state of affairs, which would represent the reference of the expression. Singular terms have singular references, John refers to the person x such that x is a person, a human, male, physical, living, etc., who is connected in the depicted by the sense real life state of affairs to an act of singing, which refers to producing sounds, that have a melody (presumably), with the intention of expressing an emotion, etc. The connection between the reference of the name John and the reference of the act of singing in real life furnishes the state of affairs, which should provide the reference of the expression. Our initial expression “John sings” gives no information as to what other things John does or is: we have no information in the sense that he is a man or a dog, etc., but in a context we can infer those from the context and see that the utterer implicitly means that John is a man. Once the sense portion of the meaning of the expression

186

Chapter IX

is sufficiently interpreted, further steps can be added or modified later for the semantic interpretation is an ongoing process for the duration of the conversation. New and relevant expressions can relate to “John sings” and thus add to, take from and modify its sense and its reference. We can now, armed with our sense, enter the domain of referential interpretation, say the set of observational data that we have, and investigate it as to the reference state of affairs of the expression. We should first look for the existence of the referents of the singular terms, John and the singing. If we find them in the data, we should investigate if an order holds between them and if there is one if the order is the one described in the sense. Thus, we can find in the data both John and singing, but we can also happen to find Jane and, say, the only order that we can find, is an order that holds between the singing and Jane and not John. In this case, we would have found the referents of the singular terms, John and the singing, but not the presence of the sense prescribed order to hold between them. Our semantic interpretation of the expression would be that referentially John does not sing and consequently, the expression, while well formed semantically, does not express the true state of affairs that actually obtains. This would represent the semantic interpretation of the expression in truth-value, where we have approached and examined the expression with respect to its truth or falsity. Note that being meaningful does not at all lead automatically to truth, but allows for an interpretation in truth-value. To take another example, the expression “5 + 7 = 12”. The sense of the expression directs its meaning toward an interpretation, which defines the expression as formed within a concrete domain of description, mathematics. The domain comes with all its rules, elements and practices and therefore, an adequate interpretation of the expression should take place within it. The sense of the expression would be its examination with respect to the nature of the elements [5, +, 7, =, 12] and the referential interpretation would need to look for term and sense references within mathematics. What is also of interest here is that the sense of the expression defines the conditions for the reference examination: we should not look for the element “5” availability in the real life physical world, say, on the street, as we did when we looked for John. Instead, knowing that 5 is a (natural) number the adequate place to look for it are the sets of numbers. Those sets come with their rules of

The Structure of Artificial Rationality

187

existence onboard, established by the historical practice of mathematics and number theory in particular. Other practices are also relevant and inform the domain of numbers, like foundations of mathematics and mathematical epistemology. From them, among others, we know that numbers, even if being all the time related to the real life states of affairs, as in expressions like “John sang two songs”, do not by themselves inhabit the physical world but only the abstract, causally inert, outside physical space and time domain of abstract objects. Abstract mathematical objects, like numbers and functions, can be applied to non-abstract objects, via a non-trivial process of application;31 non-trivial because it combines two distinct domains with very different rules of meaning construction, meaning examination and most of all – reference examination. The main epistemological method in the empirically accessible domain, such as the physical world with (some, and far not all, as quantum mechanics revealed to use) its states of affairs, is empirical observation and measurement. This is the main epistemic tool of natural sciences 32 and common sense knowledge acquisition, the empirical method. Whereas in abstract domains such as the ones of pure, unapplied mathematics, logic and metalogic, we have very different conditions of truth and meaning, which are impenetrable for the empirical method 33. Thus, if we want to be able to interpret purely mathematical and logical expression, in meaning and in truth value, and especially, with respect to their delivering 31 Perhaps the best place to see a still very much up to date discussion of the applicability of pure mathematics is the classical paper by Wigner, E. P. (1960) “The unreasonable effectiveness of mathematics in the natural sciences. Richard Courant lecture in mathematical sciences delivered at New York University, May 11, 1959”. Communications on Pure and Applied Mathematics. 13: 1–14. 32 As I have argued elsewhere, not the only tool of scientific epistemology, see Grozdanoff, B. [2014] A Priori revisability in Science, Cambridge Scholars Publishing. Newcastle upon Tyne. 33 With the exception of theories like John Stuart Mill’s and Philip Kitcher’s, as well as Penelope Maddy’s. For their views see: The Collected Works of John Stuart Mill, edited by John M. Robson (Toronto: University of Toronto Press, London: Routledge and Kegan Paul, 1963–91); Philip Kitcher (1980) “Arithmetic for the millian” in Philosophical Studies 37 (3):215 - 236 (1980); Penelope Maddy (1992) Realism in Mathematics, Clarendon Press; James Robert Brown (2008) Philosophy of Mathematics: A Contemporary Introduction to the World of Proofs and Pictures, Routledge.

Chapter IX

188

knowledge, we need a different tool than the one we use in the empirical domain. We cannot observe numbers (perhaps only in a neo-Platonic epistemology of mathematics or in a neo-formalist kind of epistemology of mathematical knowledge as the one influentially proposed by Charles Parsons34) the way we observe birds and cars, but we nevertheless know that humanity has an enormous body of truths about them that are considered by science firm truths that bring about knowledge to those who are lucky enough to gain access to them, if not even firmer than the ones of physics. The reference of a singular term, as it is considered by itself and as it figures in a well-formed sentence, is not atomic. That is, it is not functionally and structurally identical to any other reference of a term used in the same way. This difference can be best captured as a difference in content but also as a difference in function and as a difference in structure. We should distinguish between distinct stages in the process of reference, which furnishes an event of a semantic nature. Thus, roughly and not exhaustively, we can distinguish between the following stages of meaning formation: 1. 2. 3. 4.

Formation of sense Formation of reference Execution of sense – via uttering, communicating or formalizing Execution of reference – act of intended reference, act of fixing reference (by conversant)

(4) obtains when the sense and the reference have been well-formulated and an interpretation of both has been executed. To use a loose analogy from quantum mechanics, we can approach the wff sense and reference as a quantum object that remains in a certain state while unobserved, that is, while not semantically interpreted. In QM, any observation of a quantum object collapses its wave function and thus changes the parameters of its physical state, and correspondingly, values that figure in its mathematical description. In our “quantum sentence” case, we can assign a structure of quantification that would have one set of values while the sense of the

34

Charles Parsons (2007) Mathematical Thought and its Objects, Cambridge University Press.

The Structure of Artificial Rationality

189

sentence is in an unobserved state, or in an uninterpreted state and another after observation. The same can be done for the reference of the sentence. Any interpretation of the sense changes the formal uninterpreted state. For example, the semantic load of the interpreter is anything but trivial: if the sense of the term Venus as Hesperus is the only one used by the formulator, and Venus as Phosphorus is the only one available at the interpretor, we will end up in a miscommunication situation, where same string of symbols (for Venus) is intended as Hesperus by the formulator and interpreted as Phosphorus by the interpretor. Thus, the identity of syntax would not be sufficient in order to convey the intended sense to a user different from the formulator and this would prevent the communication with a conversant. And we need not even agree that the main function of language is the communicative function.35 We should only agree that the communicative function of language is non-trivial for its emergence, existence and usage. Thus, we can distinguish between functioning structures on the layer of the sense of formulator and on the level of the sense as interpreted. The same distinct structures can be found immediately on the layers of the reference as brought about semantically by the sense of the formulator and the interpreted reference by the interpreter. In our simple case, we see that we can arrive at a structural version of the Frege’s puzzle on the dynamic level of communication: the interpreter fails to acquire the intended formulation due to distinctly different senses and references even given the same syntax of the wff of the formulator and the wff of the interpreter. Therefore, we should formulate a set of measures that would eliminate or diminish the probability of such miscommunications at the core of our AI rational engine. The first and minimal logical act of reasoning is the first logical operation over a logical operandum. This could be, on an intra-expression level, the attribution of a predicate to a subject (“John is tall”), the quantification over subjects (“there is an x”), or quantification over predicates (“for all x” where x is a predicate). But on the inter-expression level, the logical acts of reasoning emerge via the usage of logical operators between expressions

35 Halliday, M.A.K. (1973) Explorations in the Functions of Language. London: Edward Arnold.

190

Chapter IX

(“P & Q” where P and Q are expressions). Those become available only when we first have the expressions, syntactically well formed and semantically interpreted. Once we have a number of available expressions, like the text-actions in the DRL model, we can use the rules of introduction and discharge of logical operators to establish order onto them. Thus, if we have P, Q and R, we can introduce conjunctions to form “P&Q” or “P&R” or “P&R”, etc. We can further develop logical complexity by using the forms of logical laws (modus ponens, modus tollens, etc.) and attempt to find if one of their forms holds for our available expressions, given their syntactic well-formedness, and their semantic interpretation. If we manage this, we would have reached a logical conclusion of some form, where we can use selected expressions as premises of a logical argument in order to deduce a conclusion. The logical conclusion is, since the times of Aristotle, the best candidate for a reasoning activity of humans with naturally linguistic abilities. If the premises are not interpreted with respect to their truth-value but the conclusion follows from them with necessity, we have a valid argument. If we interpret the premises with respect to their truth-values, given the valid argument, we have arrived at an immensely valuable for human reasoning, rationality and knowledge sound argument, where the truth of the premises is preserved via the validity of the argument into a truth of the conclusion, allowing reasoning with truths and arriving at novel truths. This ability, following the tradition of western philosophy, I take as the highest form of human rationality; the ability to arrive at truths by reasoning with truths, which, if emulated by DRL, would provide us with a very strong reason to accept the DRL model as true human-like artificial intelligence.

The DRL Structure of the Reasoning Ability The reasoning ability (RA) represents human’s real life practice where we reason within natural language. Generally, this would be the ability to operate with syntactically well-formed linguistic expressions, to interpret them semantically, and on the basis of these to introduce and discharge wellformed atomic formulae and to concatenate them into molecular wffs. All this would allow the system to construct ordered chains of expressions following the logical forms found in human practice: the logical laws, such

The Structure of Artificial Rationality

191

as modus ponens and modus tollens and others. Most of all, the system would acquire the ability to reason syllogistically, to draw logically valid conclusions and to assess logical arguments as valid, invalid, sound or unsound. The observed text-states from SGS, ST, can now, after a full and successful semantic interpretation in meaning, be fed to the reasoning ability of our DRL agent. Each term, formalizable syntactically can be retrieved from SGS and presented to the reasoning engine for logical operations. Thus, if we have as a ST the expression “Hesperus is Phosphorus” we can send it to the reasoning engine, along with all relevant definitions for the terms and connected expressions and we can attempt to reason about it. For example, the agent already has the results of the semantic interpretations and knows that the referent r of both terms is actually one and the same, the planet Venus. But he also knows that the statement uses two distinct meanings, which even if they happen to refer to the same r are not identical. The graph structures for the 1st sense and the 2nd sense do not coincide in SGS. The agent can compile tentative sets of expressions, connect them according to logical rules and see whether a law can deliver a conclusion from some assembled lists.

Methods The RAISON model is a deep reinforcement learning model, which would develop the suggested abilities for syntax recognition, semantic interpretations, logical reasoning and rationality. These abilities are very distinct, but in order RAISON to function they need to be seamlessly integrated. In the DRL setting, the abilities will be modeled as possessed by the RL agent, which develops them and perfects them in the course of its interaction with an environment. For the AGI purposes the environment would possess a complex structure and would have two layers: the deep layer,36 which possesses the syntactic, the semantic and the reasoning structures, modeled by the semantic graph space (SGS), and the surface

36

Not to be confused with the “deep” notion from the deep neural networks (DNNs).

192

Chapter IX

layer, which is the portion of the environment the agents observes and with which it interacts. The agent would learn a strategy to find the most adequate text responseactions, modeled by a policy. The policy in DRL is best found through deep neural networks and especially so in cases, like the present one, where the space of environment states or the space of agents’ actions is immense and practically unknowable. The general deep neural networks (NG) of the RAISON model are functions ƒș(s), where the learnable parameters are denoted by ș and the observed by the agent state of the environment is denoted by s; ƒș(s) takes natural linguistic text data as input and outputs the probability of a text-response given state s: pa = Pr (a|s) and a scalar value v, which assesses the quality of the adequacy of the response. Each ability in the suggested architecture would have a dedicated DRL agent whose expertise would be integrated in NG. The syntactic ability would have a DRL agent ASY which, which interacts with the environment but whose DNN, NSY, learns to recognize syntactic formulae. The semantic ability would have a DRL agent ASE, which interacts with the environment but whose DNN, NSE, learns the human-like semantic interpretations via dedicated deep neural networks. The semantic interpretations to be learned by NSE are the above formulated interpretations in sense, reference and truth-value. Each semantic interpretation would have its own dedicated neural network, NSS, for the sense interpretation, NSR for the reference interpretation and NST for the truth-value interpretation. Once each semantic neural network learns its ability to perform adequate interpretations in sense, in reference and in truth-value, these outputs would be fed to the RL agent in order to choose an action, given the state of the environment. He would grow to learn a strategy that would endow it with the ability to “beat” the machine, that is, to deliver the most adequate semantic text-response to the observed state of the environment, or the “move” of the environment, in game terms. Thus, the reasoning ability would have a DRL agent AR, which interacts with the environment but whose DNN, NR, learns to reason like humans.

The Structure of Artificial Rationality

193

The reasoning ability, acquired by NR, would take as input syntactically recognized linguistic expressions that are already semantically interpreted (in sense, reference and truth-value) and would operate over them with the logical operations of rules of introduction and discharge of logical connectives in order to recognize logical forms. NR would output a number of logically adequate (meaningful, true and relevant) expressions, ranked by scalar values and these would provide the policy expertise that would be fed to the AR agent, in order to come up with its best reasoning “move”, effectively his reasoning response to the machine-player. Thus, the rational ability would have the same DRL agent ARA, which interacts with the environment, with the key difference that his DNN, NRA, has been trained on immense volumes of actual world data which training would render him immensely “educated”. NRA would take as input the training data and it would output the semantic graph space, SGS, that would receive upon training artificial education of real world knowledge. This process would be the artificial analog of an artificial education of the agents, that would draw on the trained SGS for real world facts. The distinction between a reasoning agent and a rational agent would be that the rational one is educated and knows many things, whereas the reasoning agent only needs a minimal amount of training, which would make it able to reason like humans. The rational agent would be a functional cognitive agent in a possession of human-like knowledge.

Self-Play, Training Datasets and Hardware The training of the syntactic ability would be executed by supervised learning since the rules of syntax in 2nd order predicate logic are explicit and as rigorous as they could be. The training of the semantic abilities of sense interpretation, reference interpretation and interpretation in truth-value are a function of the SGS space and thus they require some minimal structuring with actual semantic data. This structuring would “carve” graph-represented data for subjects, predicates, quantifiers and relations and would also be executed by supervised or semi-supervised learning. The reasoning ability would be learned by training on actual logical rules of introduction and discharge of logical connectives, as well as on syllogism forms and laws of

194

Chapter IX

human thought. Again, since they are explicit and rigorously defined the training would be performed by supervised learning. The last task of training would be actual world knowledge training, or artificial “education”. Due to the actuality of knowledge, the technique of training would again be supervised learning. The tabula rasa, approach, familiar from Aristotle’s epistemology and used to great success by DeepMind in their DRL agents, if at all available, would be present only within the agent-environment interaction, where the agent would learn in a semi-tabula rasa fashion the environment of the conversant, which would consist of the ad hoc naturally linguistic communication free of any previous load but the choice of the environment. Those can be performed artificially, emulating, say, a human Turing kind jury, by using quantum randomness37 to determine the choices for the topic and the directions of the conversation with the agent. The enormously successful technique of self-play, familiar by a growing number of impressive AI systems, like DeepMind’s AlphaZero and MuZero, shows a highly non-trivial potential in training AI models. For our purposes, the RAISON model would benefit enormously by a self-play, as the environment is driven by quantum random choices of topics, themes and text-actions that would provide the conversant side of the RL agentenvironment interaction. The RAISON model can be trained on available NLP datasets, like (as of 2022) UCI’s Spambase, Enron dataset, Recommender Systems dataset, Dictionaries for Movies and Finance, Sentiment 140, Multi-Domain Sentiment dataset, Legal Case Reports Dataset, but mostly on the Wiki QA Corpus and the Jeopardy datasets. The syntax training would be supervised and the syntax rules would be explicitly trained, as well as the reasoning rules. The main challenge is the semantic graph space model SGS, the main layer of RAISON, which would directly benefit from the NLP datasets training. 37

Most suitable would be the quantum random number generator or QRNG hardware, that provides objectively random numbers in a variety of hardware formfactors, including PCIe cards.

The Structure of Artificial Rationality

195

The semantic structures, that will emerge upon training the SGS space on available big volume NL datasets would establish a significant number of subject-predicate structures, predicate-predicate structures, the subjectrelations structures, predicate-relation structures as well as linguistic forms, found in real life human linguistic practice. Those structures are available in the datasets and would be embedded in the SGS space upon the training that would seek the functionality, devised for the semantic module of RAISON. The RAISON model is devised as an AGI candidate and, given the generality of its purpose would naturally require an immense computational resources in order to be trained, tested and improved as well as to acquire the artificial abilities. The RAISON model would need CPU or TPU computational resources in order to perform the discrete interactions between the agent and the environment. The most important computational resource here is the number of threads that would allow the parallelization of the computation. The accurate estimate of the number of CPU cores and threads needed at this stage is difficult to arrive at but a multi-core and multithread CPU supercomputer would perhaps be the only practical way to develop, preliminary assess the expected performance of RAISON and train its agent in the discrete tasks of environment observations and agent responses. The other type of required computational resource are GPU or graphic processor units that would handle the tensor multiplications required by the dedicated deep neural networks of RAISON. In order the policies learning not to become a computational bottleneck in the agentenvironment interaction speed, we would need a GPU supercomputer matching the computational power of the CPU (TPU) supercomputer. The GPU computational load is a function of the immense volumes of training data, syntactic, and reasoning but mostly semantic. The suggested AGI model has the main justification for its proviso structure in the rich and critically evolved tradition of analytic philosophy, which is reflected in the syntax, semantics, reasoning and relational abilities to be learned. Due to the immense development and training work, the model would be truly in a position to be assessed both as an AI potential and as prospects for improvement only when developed in code and after a certain preliminary but still significant training on large volumes of data.

196

Chapter IX

Bibliography Alayrac, J.B., Donahue, J. et al. (2002) “Tackling multiple tasks with a single visual language model” in https://arxiv.org/abs/2204.14198 Brown, J.B. (2008) Philosophy of Mathematics: A Contemporary Introduction to the World of Proofs and Pictures, Routledge. Halliday, M.A.K. (1973) Explorations in the Functions of Language. London: Edward Arnold. Etchemendy, J. (1999): The Concept of Logical Consequence. Stanford: CSLI Publications. van Fraassen, Bas C. (1971) Formal Semantics and Logic, The Macmillan Company, New York. Frege, Gottlob (1948) “Sense and Reference” in The Philosophical Review Vol. 57, No. 3 pp. 209-230 (22 pages), Duke University Press. Gettier, E. (1963). “Is Justified True Belief Knowledge?”. Analysis. 23 (6): pp. 121–123. Grozdanoff, B. (2014) A Priori revisability in Science, Cambridge Scholars Publishing. Newcastle upon Tyne. He, J. Jianshu Chen, Xiaodong He, Jianfeng Gao, Lihong Li, Li Deng and Mari Ostendorf (2016) Deep Reinforcement Learning with a Natural Language Action Space in: arXiv:1511.04636v5 [cs.AI] https://doi.org/10.48550/arXiv.1511.04636 , last accessed 17.06.2022. Hilbert, David (1984) “On the infinite” in Philosophy of Mathematics, Edited by Paul Benacerraf and Hilary Putnam, pp. 183 – 201, Cambridge University Press Kitcher, Ph. (1980) “Arithmetic for the millian” in Philosophical Studies 37 (3):215 – 236. Linnebo, Øystein (2008) “Compositionality and Frege’s Context Principle” in Philosophy. Lycan, W. G. (2006). “On the Gettier Problem Problem.” In Epistemology Futures, (ed.) S. Hetherington, Oxford University Press. Maddy, P. (1992) Realism in Mathematics, Clarendon Press. Marcus, G. F. (2001) The Algebraic Mind: Integrating Connectionism and Cognitive Science, Cambridge, MA: MIT Press.

The Structure of Artificial Rationality

197

Martinich, A. and Sosa, D. (eds.). 2001a: Analytic Philosophy: An Anthology, Blackwell Publishers, and Martinich, A. and Sosa, D. (eds.). 2001b: A Companion to Analytic Philosophy, Blackwell. Matsuoa, LeCun et Al. (2022) “Deep learning, reinforcement learning, and world models” in Neural Networks, Volume 152, August 2022, 267-275. Mill, John Stuart (1963-91) "The Collected Works of John Stuart Mill", edited by J. Robson (University of Toronto Press, London: Routledge and Kegan Paul, 1963–91) Olivier, S. and Olivier B. (Eds.) (2010) Markov Decision Processes in Artificial Intelligence, Wiley. Parsons, Charles (2007) Mathematical Thought and its Objects, Cambridge University Press. Puppe, F. (1993) “Characterization and History of Expert Systems” In: Systematic Introduction to Expert Systems. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-77971-8_1 Reed, S. et Al. (2022) “A Generalist Agent” in arXiv:2205.06175v2 [cs.AI]. Schrittwieser, J. et al. (2020) “Mastering Atari, Go, chess and shogi by planning with a learned model” in Nature 588, 604–609. Schrittwieser, Silver et al. (2020) «Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model» in https://arxiv.org/abs/1911.08265 Silver, D., Hubert, T. et al. (2017) “Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm” in https://arxiv.org/pdf/1712.01815 . Sosa, Ernest (2018) Epistemology (Princeton Foundations of Contemporary Philosophy, 14) Princeton University Press. Sparkes, M. (2022) “Is DeepMind's Gato AI really a human-level intelligence breakthrough?” in New Scientist, 19.05. Sutton, R. and Barto, A. (2018) Reinforcement Learning: An Introduction, Second Edition, MIT Press, Cambridge, MA. Thoppilan, R. De Freitas et al. (2022) “LaMDA: Language Models for Dialog Applications”, in https://arxiv.org/abs/2201.08239v3 Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean (2013) “Efficient Estimation of Word Representations in Vector Space”, in https://arxiv.org/pdf/1301.3781.pdf

198

Chapter IX

Wigner, E. P. (1960) “The unreasonable effectiveness of mathematics in the natural sciences”, Richard Courant lecture in mathematical sciences delivered at New York University, May 11, 1959”. Communications on Pure and Applied Mathematics. 13: 1–14.

CHAPTER X DISCRIMINATOR OF WELL-FORMED FORMULAE AS A COMPONENT OF ARTIFICIAL HUMAN RATIONALITY DIMITAR POPOV The contemporaneous progress of modern Artificial Intelligence (AI) methodologies has had an unprecedented impact on nowadays industrialized society. Such applications are heavily employed to predict, discriminate, rule-govern and generate relations upon learning knowledge state representation between entities in a given data set. Deep Learning (DL) has enabled such representations to be integrated with automated reasoning mechanisms, pursuing the goal of Artificial Human Cognition (AHC). Although deep learning architectures have had significant success in resolving well-defined tasks, where optimization criteria rely on mathematical approximation between data inputs and data outputs, such approximations remain a volatile candidate for achieving AHC, since rationality needs to be persistent outside the training domain, but ANNs fail to adapt greatly, to domain changes, domain crossings or just domain expansions.1 That is why, the underlying building blocks of DL, Artificial Neural Networks (ANN), although having tremendous power of discovering correlations between data and system states, are poor performers when it comes to transferring or applying bare abstract model representation to new tasks in order to achieve robustness. Even more, the generalization outside the training set seems at

1

Michael McCloskey and Neal J. Cohen, “Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem. Psychology of Learning and Motivation,” Vol. 24 (January: 1989): 109–165. Roger Ratcliff, “Connectionist Models of Recognition Memory: Constraints Imposed by Learning and Forgetting Functions,” Psychological Review, 97 No. 2 (April: 1990): 285–308.

200

Chapter X

best debatable and only when certain caveats are applied 2before training. Neural-symbolic computing has been an active area of research for many years seeking to bring together robust learning in artificial neural networks with human-like reasoning capabilities and self-explainability via symbolic computational frameworks. The central aspect of such symbolic frameworks is that intelligence results from the manipulation of abstract compositional representations whose elements stand for objects and relations.3 In this paper, I argue the foundation of a state-of-the art computational framework, which relies on logical formalization and operation for reasoning and neural networks for knowledge embedding. By knowledge embedding, we mean the elicitation of features of a particular real-world object or concept in a given domain, those features are then encoded inside the connections of the NN, representing real world justifiable true beliefs. The utilization of this encoding is transformed into a set of relational rules that collectively represent the captured knowledge. The end goal of achieving systems with better robustness, as it is believed to be one of the basic trends of Human Rationality (HR), constitutes what Gary Marcus defines as:

22

Yann LeCun, Yoshua Bengio and Geoffrey Hinton, “Deep Learning,” Nature, 521(7553) (May: 2015): 436-444. Irina Higgins Loïc Matthey, Acra Pal et al., “BetaVAE: Learning Basic Visual Concepts with a Constrained Variational Framework,” Proceedings of ICLR (2017), accessed May 3, 2022, https://www.semanticscholar.org/paper/beta-VAE%3A-Learning-Basic-VisualConcepts-with-a-HigginsMatthey/a90226c41b79f8b06007609f39f82757073641e2. Irina Higgins, Nicolas Sonnerat, Loïc Matthey et al., “Scan: Learning Abstract Hierarchical Compositional Visual Concepts,” Proceedings of ICLR (2018), accessed March 23, 2021, arXiv preprint arXiv:1707.03389. Marta Garnelo, Kai Arulkumaran and Murray Shanahan, “Towards Deep Symbolic Reinforcement Learning”, accessed May 25, 2022, arXiv preprint arXiv:1609.05518. Brendan M. Lake and Marco Baroni, “Generalization without Systematicity: On the Compositional Skills of Sequenceto-Sequence Recurrent Networks,” accessed June 14, 2022, arXiv preprint arXiv:1711.00350. Gary Marcus, “Deep Learning: A Critical Appraisal,” accessed June 29, 2022, arXiv preprint arXiv:1801.00631. Matthew Botvinick, David Barrett and Peter Battaglia, “Building Machines that Learn and Think for Themselves: Commentary on Lake et al.,” Behavioral and Brain Sciences (2017), accessed July 2, 2022, arXiv preprint arXiv:1711.08378. 3 Marta Garnelo and Murray Shanahan, “Reconciling Deep Learning with Symbolic Artificial Intelligence: Representing Objects and Relations,” Current Opinion in Behavioral Sciences, Vol. 29 (October: 2019): 17-23.

Discriminator of Well-Formed Formulae as a Component of Artificial Human Rationality

201

“...systems that can routinely acquire, represent and manipulate abstract knowledge, using that knowledge in the service of building, updating, and reasoning over complex, internal models of the external world”. 4

Contemporary Uncertainty Integration between learning and reasoning is one of the key challenges in artificial intelligence and Machine Learning (ML) today. Where in humans the ability to learn and reason, from and to the surrounding environment is categorized as a central preordain feature of the natural intelligence, engineering the same two interrelated concepts inside programmable hardware has spurred a plethoric creation of novelty approaches in Computer Science. Naturally, those approaches derive a substantial amount of their underlying operating principles from the approximation of chemical, biological, and psychological mechanisms occurring inside the human brain, moderating our individual behavior, whenever the need of response due to environmental interaction arises. Categorically, we group those responses as the result of two separate systems, automatic and conscious once, performing intuitive and conscious reasoning, not independently from each other or in hierarchical fashion, but operating in synergy, simultaneously.5 Intuitive reasoning is exerted predominantly upon innate mental activities, such as a preparedness to perceive the world around us, recognizing objects, orient attention. Conscious activities, such as when we do something that does not come automatically and requires some sort of conscious mental exertion, like rule. We could then surmise learning, as being the set of processes of creating lasting change in behavior that is the result of experience, and conscious reasoning, being the process of applying logical rules to produce valid arguments in formal or informal environments.6

4

Gary Marcus, “The Next Decade in AI: Four Steps towards Robust Artificial Intelligence,” accessed February 12, 2022, arXiv:2002.06177. 5 Daniel Kahneman, Thinking, Fast and Slow (New York: Farrar, Straus and Giroux, 2011). 66Jaakko Hintikka, “Philosophy of Logic”, Encyclopædia Britannica, accessed March 21, 2022, https://www.britannica.com/topic/philosophy-of-logic.

202

Chapter X

To emulate natural learning, the research field of machine learning is aiming to create computer algorithms that also improve automatically through experience. Applications range from data-mining programs that discover general rules in large data sets, to information filtering systems that automatically learn users’ interests.7 Deep Learning is a subset of Machine Learning, which relies on the creation of complex interwoven acyclic graphs. Any vertices of the graph are implemented by a computational unit named Artificial Neuron (Figure1), and any edge that propagates from and to any neuron is weighted. A particular method of stacking those neurons together in layers and establishing the number of interconnections to each neuron describes the network architecture. The first layer of the network is called the input layer and its last is called the output layer, while any other layer between the input and the output layer is named hidden layer. Data is fetched into the network through the input layer, for example, if we have an input layer of eight neurons, we could pass information in the form of eight floating point numbers or eight vectors (floating point arrays) for each neuron.

Figure 1. Artificial Neuron, image generated by TikZ open source code. Each neuron at given moment t, could only exist in an active or inactive state. Fetching data to the input layer will cause propagation through the network in the form of a signal, denoted as x and xn denotes the signal coming from the input neurons to the interior of the network. Propagating

7 Tom Mitchell and Hill McGraw, Machine Learning (New York: McGraw Hill, 1997).

Discriminator of Well-Formed Formulae as a Component of Artificial Human Rationality

203

forward, the signal is first scaled by the weights of the edges, wi,j , where i denotes the position of the neuron in the previous layer and j denotes the position of the neuron in the layer the signal is been propagated to, for example w3,5 is the weight between the third and the fifth neurons between two consecutive layers. The signal is then fed to the first hidden layer. Inside each neuron in the hidden layer, two operations take place, summations of all weighted signals from a previous layer and applying an activation function, if a provided threshold is exceeded, the neuron is activated and propagation through this neuron is enabled, or we are saying that the neuron fires. Then, at time tk to tk+1, the input of a neuron is calculated as:

Figure 2. Input of a neuron Where bi is called a bias and is an arbitrary value predefined for each neuron, it could also be omitted. The activation function could vary, depending on the problem at hand, in our case we are using the Softmax activation function:

Figure 3. Softmax activation function Here, the exponent of a given neuron is divided by the sum of all exponents of the rest of the neuron. The Softmax function returns the probability of a given example to be in a class. After each cycle, an optimization technique is applied to adjust the weights, if the output value of the Softmax functions in the output neurons differs from those of the pre-labeled training examples. This optimization is performed by calculating the Gradient Descent of each neuron weight in relation to the difference between the current output value and the desired value, for every single neuron in the entire network. The difference between the sum of the outputs of the last layer and the pre-labeled training examples is computed by a math function called a loss function, which could take various mathematical forms. That difference is then used to start an automatic differentiation algorithm called

Chapter X

204

backpropagation.8 For our particular work, we are using a neural network architecture called a Recurrent Neural Network (RNN). The special thing about this network is that it can process all inputs as a sequence and remembers its intermediate state. This is done by connecting the input for every step, back to the input layer of the function, combining it with the new data. At each new input at time t + 1, the new intermediate state is calculated by:

Figure 4. Recurrent network intermediate state Where, Ɣ

is the input at time step is the output at time step

Ɣ

is the intermediate state as vector at time

Ɣ

are weights associated with inputs in

Ɣ recurrent layer.

are weights associated with hidden

Ɣ units in recurrent layer. Ɣ

is the bias associated with the recurrent layer.

and the output of the network is:

Figure 5. Output of RNN

8 David Rumelhart, Geoffrey Hinton and Ronald Williams, “Learning Representations by Backpropagating Errors,” Nature 323 (October: 1986): 533-536.

Discriminator of Well-Formed Formulae as a Component of Artificial Human Rationality

205

Neuro-Symbolic Approach Neuro-Symbolic Artificial Intelligence can be defined as a subfield of Artificial Intelligence (AI) that combines neural and symbolic approaches. By neural, we mean approaches based on artificial neural networks – sometimes called connectionist or sub-symbolic approaches – and in particular this includes deep learning, which has provided very significant breakthrough results in the recent decade and is fueling the current general interest in AI. By symbolic we mean approaches that rely on the explicit representation of knowledge using formal languages – including formal logic – and the manipulation of language items (“ symbols”) by algorithms to achieve a goal. Mostly, neuro-symbolic AI utilizes formal logics, as studied in the Knowledge Representation and Reasoning subfield of AI, but the lines blur, and tasks such as general term rewriting or planning, that may not be framed explicitly in formal logic, bear significant similarities, and should reasonably be included. The neuro-symbolic approach has gained significant attention, primarily because the leading trend of trying to leverage statistical inference over massively parametrized network models cannot be backtracked. To understand sufficiently the prediction or even how the result was derived, the inference process itself needs to be demystified. Although that behavior, admittedly, cannot be considered a showstopper in commercial applications, where statistical dependencies are sufficient in narrowing down a group of patterns as best fitting solutions to a recommendation or classification problem, the obscurity of the pattern generator, cannot be abstracted to any intelligible or human-interpretable reference, so a due-process by carefully following a rational argumentative sequence is not possible. If we follow this train of thoughts, a certain philosophical suggestion could be furnished, or namely that Artificial Intelligence based exclusively on statistical models, takes the vague likeness of human hunch or gut feeling that cannot be substantially rationalized by the person experiencing it. The latter, however, possesses potentially destructive capability, when statistical inferences are applied in decision-making systems, where the rationale behind a certain decision must be traceable, in order to be justified, defended, or appealed. As a whole, the pursuit of automated human-like rationality, or the quality of being on or in accordance with reason and logic,

206

Chapter X

cannot be successfully achieved, without the successful adherence to the rules of logic. The key contribution of this article is to propose a foundational layout of the neuro-symbolic model, which has in its core logical inference engine. That engine will gain as an input, syntactically correct logical expressions derived from text or speech and by applying rules of inference in a given formal system, should conclude if the provided argument could be judged as a valid one. In order for the inputs to be syntactically correct, we need to grand filtering or discriminating capabilities of the engine. For example, a text-to-expression parser creates symbolic representation of the arguments out of provisioning context, following the rules of valid syntax the expression could be accepted and incorporated into this argument, or returned for reparsing. This article will use the rules of first-order logic to present a discriminator of well-formed formulas. That approach is to train a sequence-to-sequence model based on Recurrent Neural Networks (RNN), that will grant the capability of distinguishing between any correct formal expression and such that we could consider as noise. Refinement on the training approach, brings forward even better discriminator performance, as the RNN model yields good results while distinguishing between atomic and molecular wellformed formulas.

Neuro-Symbolic Computational Framework In this section, a blueprint layout of a state-of-the-art computational framework based on the neural-symbolic paradigm will be discussed. We argue that a pure connectionism system cannot account for a valid theory of cognition. In such a system, combinatorial and semantic structures of mental representation seem impossible to achieve. On the other end, purely symbolic theories cannot be regarded as more than an approximation of what happens in the brain, they cannot be accounted as a complete description of a cognitive phenomenon.9 Our view is that between these two philosophies the most stubble and yet the least unexplored path is the integration between symbolic and sub-symbolic. We will now go forward and describe the 9

Steven Pinker and Alan Prince, “On Language and Connectionism: Analysis of a Parallel Distributed Processing Model of Language Acquisition,” Cognition, 28(12) (March: 1988): 73–193.

Discriminator of Well-Formed Formulae as a Component of Artificial Human Rationality

207

existing taxonomical classification of such integrations that researchers engage while implementing such systems. Usually a construction of neurosymbolic architectures falls inside two strategic paths: unified and hybrid. Unified strategies are research paths that aim at creating neural structures, which have symbolic capabilities or translating symbolic computation inside neural networks alone. They can be additionally subdivided into neuronal symbol processing and connectionist symbol processing. The neuronal sub-approach stands for close emulation with the properties of the biological neuron, but it only applies general similarities to actual neuron. Connectionist symbol processing is a strategy that relies on a formal mathematical approximation of a neuron, that we have introduced in section II. Stacking such neurons is used to create complex architectures that perform symbolic computations. To create a model based on that design pattern, first a general conception about the desired function has to be introduced as possible mapping between data distribution points, that implementation in the form of architecture is explored. For example, if we want to develop an architecture that could perform logical conjunction operation (AND, ‫ )ר‬symbolic computation between two variables we need first to ground these variables to data points inside a given data distribution and then learn syntactic and semantic mapping transformation to it. Of course, what we hope to learn is the truth-value table of logical conjugation. In this specific example, we are taking advantage of the particular state, about the data distribution of the outputs, or precisely that the outputs of the mapping function are linearly separable, and the implementation could be achieved by applying only a single artificial neuron (perception). In the case of nonlinear separability of the outputs, for example, in learning how to compute logical expression with exclusive disjunction (XOR, Չ), and the respective symbolic notation (x Չ y) more complex architecture has to be created with minimum two stacked perceptrons and as final the symbolic computation that will be learned by the

model

could

be

the

following,

. We are hypothesizing the exact symbolic representation since accurate elicitation of symbolic expression is not possible, once mapping is learned inside the neural nets. In this case, we

208

Chapter X

could also have a symbolic representation of the form, . Both representations are logically equivalent and in both of the cases, two additional neurons must be included in the architecture, one for the operator and one for the operator. In the end, what matters is the learned mapping function and the internal representations of the networks. Hybrid strategies are strategies that rely on a synergistic combination of neural and symbolic models. The goal in mind while utilizing this strategy is that only by constructing a modular neuro-symbolic structure we could attain a full range of cognitive and computational potentials. Hybrid modules also subdivide themselves into translational or functional hybrids. Translational models are composites that we could put between models based on unified strategies and functional models, as well as in unified models in translational models, the sole computational process relies on neural networks, but that model does not confide itself on bottom-up approach from a single to multiple stacked neurons when it comes to its core implementation. The translational model, as the name suggests, translates the provided symbolic representations from a formal symbolic system or structure to a neural network computational core. These systems also, as in the provided example of a unified system, try to elicit after neural network computation symbolic representation back to its initial symbolic structure. Such systems are assigned to symbolic model discovery from physical systems.10 Functional hybrids in addition to neural networks have complete symbolic structures and processes, like rule interpreters, parsers, reason and inference engines, theorem provers. These systems rely on synergy between the different functional modules. The idea is that great model robustness could be achieved to a certain degree by adding compositionality to these models with levels of integration between the different functional models. This degree of integration is a quantitative benchmark, which reflects the progression from loosely to tight coupling between the modules, for example, as loosely coupled we could classify systems that only possess simple juxtaposition between the symbolic and the neural models. In strongly coupled, a continuous interaction between the neural and symbolic 10

Miles Cranmer, Alvaro Sanchez-Gonzalez, Peter Battaglia et al., “Discovering Symbolic Models from Deep Learning with Inductive Biases”, NeurlPS 2020, accessed July 3, 2022, arXiv:2006.11287.

Discriminator of Well-Formed Formulae as a Component of Artificial Human Rationality

209

submodules operates in internal structures, meaning that knowledge and data are shared during the runtime of the system and a change in the operational state of a certain neural module will trigger a unified response across all symbolic modules, provided that more than one neural module is present in the architecture, this unification will also constitute an adaptation in the rest of the neural modules. Now after the introduction of the different strategies, which are used in neuro-symbolic systems, we will discuss our proposed design as a particular instance of these strategies. Considering that any design pattern carries within its pros and cons and cons. Our choice in this matter is to follow the hybrid strategy, as a design paradigm. Considering that in the past 65 years since the dawn of the field of artificial intelligence, there was little progress in decoding the essence of human rationality and implementing common sense as a machine ability, providing a schematic what might catalyze or induce as final all missing ingredients to the creation of artificial human rationality, in a single paper would not be possible, what we will describe here is our view of what a system might need in order to at least being able to get modular abilities that would enable more elaborative future research to be based upon this one. We approach the problem by firstly defining what we think is the most pressing lack of abilities that could curb an artificial system from obtaining rationality levels next to that of a human. Deep learning has been predominantly a huge success when it comes to perception tasks, like object and pattern recognition in computer vision or natural language processing, but for systems that perform automated reasoning over a set of descriptive rules, such models have proven to be inadequate, for example, in closed environment, competitive board games or Atari, a learning mechanism has to be combined with symbolic computational mechanisms like Monte Carlo Tree Search (MCTS). For example, in 2016 the AlphaGo11 system was able to beat the current GO world champion by continuously inferring better pieces’ positions and ergo building better strategies, the model, although constricted to the geometry of board desk, was able to fill the gaps in previously unseen pieces’ configurations, based on the existing knowledge representation which was rooted in the time of past offline learning. The 11

David Silver, Aja Huang, Chris Maddison et al., “Mastering the Game of Go with Deep Neural Networks and Tree Search,” Nature 529 (January: 2016): 484-489.

210

Chapter X

learning procedure was a special case of supervised learning named Reinforcement Learning (RL). To beat the reigning champion, however, only RL was not enough, what was learned had to be integrated with a symbolic search technique to create the best reasoning patterns over the board. Referring back to section II and the definition of reasoning, this comes in a close approximation to what people do. Our schematic is a strategy for creating artificial agents, capable of reasoning over unseen environmental configurations, much like AlphaGo. Of course, however, we are interested in the general case, searching for an approach or architecture that will discover and apply strategies to solve a variety of problems. To be able to create those strategies, there are three crucial aspects that must be imbued as abilities into an agent, environment identification, problem semantic representation, and decision making. Our test studies focus on natural language processing, meaning that for now we are constraining the environment to knowledge elicitation and transformation from text. In a corpus of textual information, in order to make sense of the information interlocked, an agent needs to discover the semantics or the meaning behind each sentence, both as a single unit and as a text. A decision making, to a specified problem, like query-answer entailment or sentiment analysis. Our approach here is to convert a text to sets of logical expressions using a technique called logical chunking, with that technique what we are aiming at is a system for parsing and generating text using combinatory categorial grammar for syntax and hybrid logic dependency semantics for, well, the semantic representation. For example, the sentence Brian loves Jain would be transformed to love(B, J) or even L(b, j), where L, b and j are respectively the predicate and two constants bounded to real objects Bounding of variables, constants and definitions is a particular hard problem when it comes to natural language, our approach is to define them as relational connections or intersections between sets In this, particular example, two members of the intersection of the sets containing the names of all people, the members of all leaving peoples, the set of people loving each other etc.. Historically, such relational categorization is painfully slow process, provided that group of experts is responsible for defining, categorizing and connecting all the relations, such approach has already been tried as part of

Discriminator of Well-Formed Formulae as a Component of Artificial Human Rationality

211

the infamous CYC project,12 which results, although strikingly impressive, when it comes to completing tasks as automatic story generation for specific literary works it remains extremely limited when knowledge relations are not encoded, and has to be discovered on a flight.13 For automation of this part, an automatic semantic parser has to be employed. The architecture of this parser should rely on the already discovered connections. Meaning we could leverage the existing knowledge represented as graph- connection in Google Knowledge Graphs or Wikipedia. Furthermore, convolution of this graph’s spaces would discover deeper connections between the entities. This discovered knowledge in a graph formalism, must exist on demand and will be used as a semantic interpretation later on. The syntactical parser must also have unified functionality to take those relations and combine them with predefined syntax to create syntactically valid logical expressions, we believe that the best formal system to be used here by which these logical expressions should be constructed is the Second-Order Logic (SOL). The syntax must also be learned via supervised learning, however, the rich expressive power of SOL will require a more powerful technique through RL to be applied here since a large enough dataset with enough possible symbolic representations is not feasible. After that, the expressions, that then will be passed to two components, namely the Semantic and the Reasoning engines. The semantic engine will interpret the expression using, interpretation by sense and reference, and truth. Interpretation by truth is simply the logical value of the expressions, and by sense and reference we are referring here to the idea proposed by Gottlob Frege in Begriffsschrift.14 The Reasoning engine will have encoded into itself, rules of introduction of logical operators, rules of discharge and rules of logical inference (modus ponens, modus tollens…). Following this blueprint, an agent will be endowed with a powerful enough mechanism to enable common sense reasoning or to rationalize over presented ideas and topics from text. What we have described above is just the architectural approach: current 12

“CYC”, Encyclopedia Britannica, accessed June 6, 2022, https://www.britannica.com/topic/CYC. 13 Doug Lenat, “What AI Can Learn from Romeo & Juliet,” Cognitive World, Forbes, July 3, 2019. 14 Gottlob Frege, Begriffsschrift: Eine der arithmetischen nachgebildete formelsprache des reinen denkens (Halle a/S: L. Nebert, 1879).

212

Chapter X

implementations as modules of functional hybrids are still under investigation. However, as a part of creating valid logical expressions, the hybrid system must have an integrated module for discriminating, between valid and invalid ones. That discrimination or classification will be conducted by a unit that, we have developed, and it will be integrated in the RL pipeline.

Discriminator of a Well-formed Formula In this section, a technical implementation of the previously discussed idea of syntactical classifiers is implemented. The technical implementation is performed using the Python programming language and the popular machine learning framework PyTorch, which provides extensive capabilities for creating sequential models for natural language processing in Python. We chose a vanilla Recurrent Neural Network for the architecture of the model. When a standard RNN network is exposed to long sequences or phrases it tends to lose the information because it cannot store the long sequences and as the methodology is concerned it focuses only on the latest information available at the node. This problem is commonly referred to as vanishing gradients. In our case, we adopt RNNs as the implementation because of the nature of the training data set. Since our model will be trained on artificially synthesized expressions, we are converting them to relatively short strings. The aim of this model is to acquire the ability to classify between noise and syntactically correct formulas which are generated by the graph parser described in the previous section. The motivation behind developing such model is that formal systems constitute logical formalism as a reasoning tool. There are many different formal systems and their referential syntactical rules. We hope that this model could be used as a general study case when it comes to creating preprocessing pipelines for reasoning engines based as part of functional hybrids. We follow the syntactical rules for creating logical expressions for FirstOrder Logic to restrict the formalism. Such formalism, although having its limitations, for example, cannot define finiteness or countability, is considered powerful enough to represent the natural language statements sufficiently well. The allowed symbols are restricted over a signature, here

Discriminator of Well-Formed Formulae as a Component of Artificial Human Rationality

213

we are considering all letters from the English alphabet as well as the following operational symbols. Ɣ Identity operator and iff : Ɣ Implication operator : Ɣ Entailment (semantic and syntactic): Ɣ Binary operators for Boolean algebra:

In the last row, the last three symbols are making, respectively, the NOR, NAND and XOR Boolean operators; these particular symbol’s annotations are purely our own choice. We will refer to this list as the operational symbol list. We are also using the following symbols for existential (exists, does-not-exist) and universal quantifiers:

Every symbol described above is used to construct the following sets, when we create a syntactically valid expression: 1. Set of predicate symbols P. Contains all uppercase letters of the English language. 2. Set of constant and variables and functions symbols S Every lowercase letter of English, which is not a member of function symbol set F. Provided that we are just interested in form of an expression and do not ground the variables and constants, we will not create two separate sets of symbols assignment, since we are not interested in the semantics of expression.

the the the for the

Chapter X

214

3. Set of function symbols F Every lower-case letter of the English language, which is not a member of the set S. The rule that we should follow is that once a symbol is selected for a function notation it cannot be used for a variable or a constant 4. Set of function strings Fs. , where is It contains every string of the form an unbound variable or a constant from S. A function has an arity of a maximum of six symbols. These are artificial constraints that we are introducing, theoretically it must be adjusted for every particular formal system. 5. Set of terms T. Is the union

.

6. Set of atomic formulas strings A. It contains all strings of the form

, where

. The terms inside the brackets after the predicate symbol Q are called arguments. Atomic formulas are also restricted to a maximum of six terms as arguments to their predicate. A could also be noted as the union contains all the strings of the form all the strings of the form

, where and

contains

.

7. Set of molecular formulae strings M Contains every string m such that, (

), where

is a quantifiable

atomic formula such that and ‘o’ is any operational symbol from the operational symbol list.

Discriminator of Well-Formed Formulae as a Component of Artificial Human Rationality

215

Training Data The training data will be synthesized from a custom code that will first create the mentioned collection of sets . Second, the sets (A, M) will be written into two text files, one for the atomic formulas and one for the molecular. In these files, each member of the set will be written in a separate line as the string that we want to pass to the input layer of our RNN. One additional text file will also be added, any line of this file will be a string of jumbled random characters in ascii, this will be our noise. As a whole, we have three labels for classification Noise, Atomic, Molecular. The training set contains the following records described in the table below. Training set Label

Records

Noise Atomic Molecular

18 32617 300000

Table 1. Training set of the experiment In our experiment, the atomic set is used to generate the molecular set, this is a tricky part for the network to discover, since the information encoded into the molecular set is also encoded in the atomic set. The noise represents a particularly blunt difference between the form of a syntactically correct expression and random strings, that is why the network, should need very few of these examples to train itself.

Discriminator’s Architecture As we pointed out, the discriminator would be a many-to-one recurrent neural network. Our first layer will have a size of 100 input neurons, this is exactly to cover all printable characters, digits, punctuation and brackets in ascii encoding. The hidden layer is constructed out of 128 fully connected neurons, which are connected to the three output neurons. In practice, because of the specifics of PyTorch, what we end up is a combination of

216

Chapter X

two regular feedforward neural networks. One is connected to the output layer of three neurons for each output label that we want to classify and the second is reconnected again with the input coming from the training data, in order to combine the current state of the network with the new state. After combination, the new values are fed as inputs to both networks. To begin training, the training data must be converted from a string to a tensor. To put it simply, a tensor is just a multidimensional matrix. The notation corresponds to the dimensionality of the tensor, for example 3-d is a 3dimensional tensor. We are taking advantage of the positional index that each character has in Python “string.printable”; this command returns a string containing all the numbers, letters and special printable characters in Python “0..9A..Z..-./:;?@[..”. Each character has an indexed position in it, for example the character ‘?’ is in position 82 and it is at 81 characters away from the beginning of the string. We denote this index as r. Each line will be encoded to 3-d tensor where, with r, 2-d tensors inside and each twodimensional tensor will have one 1-d tensor with m = 100 elements. For example, let is the shape of the final three-dimensional tensor, if we encode the line as:

Figure 6. An example expression from the learning set. We will get a tensor with dimensions (‘’ consists of two printable characters ‘-’ and ‘>’), From here, the generated 3-d tensor will contain twelve 2-d tensor, each having one row of one hundred zeros, except for one 1.0, which corresponds to the same position indexed from the “string.printable”. If a character happens not to be contained in “string.printable”, the last element of the respectable 2-d tensor is filled with “1.0”. From here, each 3-d tensor is passed to the network, and the collected 2-d tensors are passed sequentially to the network. The output is a 1-d tensor with 1 element equal to the positional index of a list with the labeled outputs (Noise, Atomic, Molecular).

Discriminator of Well-Formed Formulae as a Component of Artificial Human Rationality

217

Training and Learning The learning is conducted on GPU NVIDIA 2060 RTX with 6 GB RAM. The required loss function for minimization in the test study is the negative log-likehood function:

Figure 7. Log-likehood loss function 15 The training is performed along the course of 100 000 cycles, at each cycle a random training sample is selected from any of (N, A), and is fed to the 100 input neurons, for each character in the line. The selected learning rate is . We are selecting such a miniscule learning parameter because we want the model to learn the subtle difference which members of A are used for in the creation of the members of M. From here, the cycle of learning is the following: Ɣ Ɣ Ɣ Ɣ Ɣ Ɣ

Create the input output tensor. Create initial hidden state Fed the input tensor corresponding to a single line from N, A, M Compare with the desired output Perform backpropagation Repeat

The result from the training can be seen in Figure 8. This is the so-called confusion matrix, which represents the average normalized prediction of the model for each 1000 cycles. We could see that the model has achieved great performance, during the training phase, as the yellow color in the diagonal corresponds to the normalized degree of prediction in this case 1.0, meaning extremely close to 100%.

15 Lester J. Miranda, “Understanding Softmax and the Negative Log-likelihood,” accessed July 3, 2022, https://ljvmiranda921.github.io/notebook/2017/08/13/softmax-and-the-negativelog-likelihood/.

Chapter X

218

Figure 8. Confusion matrix

Testing In order to test our model, we have generated a new logical expression using anew construction of the sets (S, F, Fs, T, A, M). The table below shows the record distribution of the test set and the test results. Test set Label

Records

Noise

3 274 584

Atomic

2 976 230

Molecular

3 000 000

Table 2. Test set with records from the categories.

Discriminator of Well-Formed Formulae as a Component of Artificial Human Rationality

Results from testing Random test records Correct predictions Incorrect predictions Success rate Elapsed time

219

3 000 000 2 972 853 27147 99.095% 2:29:39h

Table 3. Test Results

Conclusion This work has introduced a theoretical compositional framework for neuralsymbolic integration that utilizes functional hybrid as a leading design paradigm. The main contribution of this paper is the elaborated schema of compositional design for an artificial agent with human-like level of rationality. One module of this framework, a sub-symbolic discriminator for valid logical syntax, is constructed in this article. The results from training and test cycles have achieved over 99.13% accuracy. In future work, a better discriminator of valid logical arguments, semantic parser from natural language to logical expression reasoning and semantic engines will be built to complete the neuro-symbolic framework proposed in this article.

References Botvinick, Matthew, David Barrett, Peter Battaglia, Nando de Freitas, Darshan Kumaran, Joel Leibo, , Timothy Lillicrap et al.. “Building Machines to Learn and Think for Themselves: Commentary on Lake et al.” Behavioral and Brain Sciences (2017). arXiv preprint arXiv:1711.08378. Cranmer, Miles, Alvaro Sanchez-Gonzalez, Peter Battaglia, Riu Xu, Kyle Cranmer, David Spergel, and Shirley Ho. “Discovering Symbolic Models from Deep Learning with Inductive Biases”. NeurlPS 2020. Accessed July 3, 2022. arXiv:2006.11287. Encyclopedia Britannica. “CYC”. Accessed June 6, 2022. https://www.britannica.com/topic/CYC.

220

Chapter X

Frege, Gottlob. 1879. Begriffsschrift: Eine der arithmetischen nachgebildete formelsprache des reinen denkens. Halle a/S: L. Nebert. Garnelo, Marta, Kai Arulkumaran, and Murray Shanahan. “Towards Deep Symbolic Reinforcement Learning”. Accessed May 25, 2022. arXiv preprint arXiv:1609.05518. Garnelo, Marta, and Murray Shanahan. “Reconciling Deep Learning with Symbolic Artificial Intelligence: Representing Objects and Relations.” Current Opinion in Behavioral Sciences, Vol. 29 (2019): 17-23. https://doi.org/10.1016/j.cobeha.2018.12.010. Higgins, Irina, Loïc Matthey, Arka Pal, Christopher Burges, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. “BetaVAE: Learning Basic Visual Concepts with a Constrained Variational Framework.” Proceedings of ICLR (2017). Accessed May 3, 2022. https://www.semanticscholar.org/paper/beta-VAE%3A-LearningBasic-Visual-Concepts-with-a-HigginsMatthey/a90226c41b79f8b06007609f39f82757073641e2. Higgins, Irina, Nicolas Sonnerat, Loïc Matthey, and Arka Pal. “Scan: Learning Abstract Hierarchical Compositional Visual Concepts.” Proceedings of ICLR (2018). Accessed March 23, 2021. arXiv preprint arXiv:1707.03389. Hintikka, Jaakko. “Philosophy of Logic”. Encyclopædia Britannica. Accessed March 21, 2022. https://www.britannica.com/topic/philosophy-of-logic. Kahneman, Daniel. 2011. Thinking, Fast and Slow. New York: Farrar, Straus and Giroux. Lake, Brendan M., and Marco Baroni. “Generalization without Systematicity: On the Compositional Skills of Sequence-to-Sequence Recurrent Networks”. Accessed June 14, 2022. arXiv preprint arXiv:1711.00350. LeCun, Yann, Yoshua Bengio, and Geoffrey Hinton. “Deep Learning.” Nature, 521(7553) (2015): 436-444. Lenat, Doug. “What AI Can Learn from Romeo & Juliet.” Cognitive World, Forbes. July 3, 2019. https://www.forbes.com/sites/cognitiveworld/2019/07/03/what-ai-canlearn-from-romeo--juliet/.

Discriminator of Well-Formed Formulae as a Component of Artificial Human Rationality

221

Marcus, Gary. “The Next Decade in AI: Four Steps toward Robust Artificial Intelligence”. Accessed February 12, 2021. arXiv:2002.06177. Marcus, Gary. “Deep Learning: A Critical Appraisal”. Accessed June 29, 2022. arXiv preprint arXiv:1801.00631. McCloskey, Michael, and Neal J. Cohen. “Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem.” Psychology of Learning and Motivation, Vol. 24 (1989): 109–165. https://doi.org/10.1016/S0079- 7421(08)60536-8. Miranda, Lester J. “Understanding Softmax and the Negative Loglikelihood”. Accessed July 3, 2022. https://ljvmiranda921.github.io/notebook/2017/08/13/softmax-and-thenegative-log-likelihood/. Mitchell, Tom, and Hill McGraw. 1997. Machine Learning. New York: McGraw Hill. Pinker, Steven, and Alan Prince. “On Language and Connectionism: Analysis of a Parallel Distributed Processing Model of Language Acquisition.” Cognition, 28(1-2) (1988): 73–193. Ratcliff, Roger. “Connectionist Models of Recognition Memory: Constraints Imposed by Learning and Forgetting Functions.” Psychological Review, 97 No. 2 (1990): 285–308. https://doi.org/10.1037/0033-295x.97.2.285. Rumelhart, David, Geoffrey Hinton, and Ronald Williams. “Learning Representations by Backpropagating Errors.” Nature 323 (1986): 533536. https://doi.org/10.1038/323533a0. Silver, David, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, George van den Driessche, Julian Schrittwiesser et al. “Mastering the Game of Go with Deep Neural Networks and Tree Search.” Nature 529 (2016): 484-489. https://www.nature.com/articles/nature16961.